`--copies number` in `froctfs-cli object put` works incorrectly #538

New issue

Closed

opened 2023-07-24 09:27:02 +00:00 by EliChin · 1 comment

EliChin commented

2023-07-24 09:27:02 +00:00

Member

Expected Behavior

While using --copies-number and disabling policer, we expect the number of object copies to equal the argument set via --copies-number regardless of the placement policy.

Current Behavior

While using --copies-number and disabling policer, the number of object copies is taken from the placement policy.

Steps to Reproduce (for bugs)

Disable policer through creating a config file /etc/frostfs/storage/conf.d/99_config.yml:

policer:
  unsafe_disable: true

Reset the service:
systemctl reset frostfs-storage
Set the placement policy in /etc/frostfs/s3/conf.d/99_placement_policy.yml to a big number. Eg.:

...
placement_policy:
  copies_numbers: ''
  default: REP 4 SELECT 4 FROM *
...

Reset the service
systemctl reset frostfs-s3
Create a bucket
aws s3api create-bucket --bucket <bucket_name> --no-verify-ssl --endpoint-url https://<data0> --acl public-read-write
Check the cid of the bucket
curl --head https://<data0> -k
Put an object to the bucket
frostfs-cli object put -r localhost:8080 --cid <bucket_cid> --wallet <wallet_path> --file 3<file_path> --copies-number <num>
Check the number of copies put on the nodes
frostfs-cli object nodes -r localhost:8080 --cid <bucket_cid> --wallet <wallet_path> --oid <oid>
The number of actual copies doesn't match the number se tvia --copies-number:

                                                            Node ID| Should contain object| Actually contains object|
 0226d47a2028951958abd7a66e9e5a915cf076d1615ac5c4bfec940ec6b6ed042f|                  true|                     true|
 02ac7635bf1b88899b38cdb5194e262bfa94b39a9a88e567e85c9302494996fbca|                  true|                     true|
 03378ce140a5e6bd00108a78d74d0b027b955c9f4463bfdeff875d37e9a3721d21|                  true|                     true|
 03645ee78a88eeae47cb75f72273c6fd79668445de97e995827934862b95b86926|                  true|                     true|

Context

The only case it worked was when 1 was set in the placement policy file

frostfs:
  set_copies_number:
  - 1 
...

and the same number was set via --copies-number while putting an object.

Regression

Your Environment

SberCloud v1.3.0-65-echichindaeva

## Expected Behavior While using `--copies-number` and disabling policer, we expect the number of object copies to equal the argument set via `--copies-number` regardless of the placement policy. ## Current Behavior While using `--copies-number` and disabling policer, the number of object copies is taken from the placement policy. ## Steps to Reproduce (for bugs) 1. Disable policer through creating a config file `/etc/frostfs/storage/conf.d/99_config.yml`: ``` policer: unsafe_disable: true ``` 2. Reset the service: `systemctl reset frostfs-storage` 3. Set the placement policy in `/etc/frostfs/s3/conf.d/99_placement_policy.yml` to a big number. Eg.: ``` ... placement_policy: copies_numbers: '' default: REP 4 SELECT 4 FROM * ... ``` 4. Reset the service `systemctl reset frostfs-s3` 5. Create a bucket `aws s3api create-bucket --bucket <bucket_name> --no-verify-ssl --endpoint-url https://<data0> --acl public-read-write` 6. Check the cid of the bucket `curl --head https://<data0> -k` 7. Put an object to the bucket `frostfs-cli object put -r localhost:8080 --cid <bucket_cid> --wallet <wallet_path> --file 3<file_path> --copies-number <num>` 8. Check the number of copies put on the nodes `frostfs-cli object nodes -r localhost:8080 --cid <bucket_cid> --wallet <wallet_path> --oid <oid>` The number of actual copies doesn't match the number se tvia `--copies-number`: ``` Node ID| Should contain object| Actually contains object| 0226d47a2028951958abd7a66e9e5a915cf076d1615ac5c4bfec940ec6b6ed042f| true| true| 02ac7635bf1b88899b38cdb5194e262bfa94b39a9a88e567e85c9302494996fbca| true| true| 03378ce140a5e6bd00108a78d74d0b027b955c9f4463bfdeff875d37e9a3721d21| true| true| 03645ee78a88eeae47cb75f72273c6fd79668445de97e995827934862b95b86926| true| true| ``` ## Context The only case it worked was when 1 was set in the placement policy file ``` frostfs: set_copies_number: - 1 ... ``` and the same number was set via `--copies-number` while putting an object. ## Regression No ## Your Environment SberCloud v1.3.0-65-echichindaeva

EliChin added the

bug

triage

labels 2023-07-24 09:27:02 +00:00

aarifullin self-assigned this 2023-07-24 09:47:28 +00:00

aarifullin commented

2023-07-26 18:07:58 +00:00

Member

Alright. I tried to reproduce the problem described in the issue , with the cloud, and I got proper behavior when I restarted services on all hosts.
The problem is that policers that are still enabled and running on other hosts and they still replicate objects by given container policy.

Briefly: you should restart services on all hosts after the config is changed :)

Alright. I tried to reproduce the problem described in the issue , with the cloud, and I got proper behavior when I restarted services on all hosts. The problem is that policers that are still enabled and running on other hosts and they still replicate objects by given container policy. Briefly: you should restart services on all hosts after the config is changed :)