put-bucket-acl doesn't put just downloaded ACL.json #52

New issue

Closed

opened 2023-03-08 23:38:13 +00:00 by snegurochka · 2 comments

snegurochka commented

2023-03-08 23:38:13 +00:00

Member

Original issue: https://github.com/nspcc-dev/neofs-s3-gw/issues/681

Steps to reproduce:

Create bucket with public-read-write ACL
Download bucket ACL to json file

aws --no-verify-ssl s3api get-bucket-acl --bucket b-test-lock-20   --endpoint-url http://172.26.163.38:8084 > ACL.json

Try to upload the same json file (without any changes) following this command:

aws --no-verify-ssl s3api put-bucket-acl --bucket b-test-lock-20   --endpoint-url http://172.26.163.38:8084 --access-control-policy file://ACL.json

An error occurred (InternalError) when calling the PutBucketAcl operation (reached max retries: 2): could not translate ast to table: form records: public key from string: encoding/hex: invalid byte: U+004E 'N'

ACL.json:

anikeev@NB-1670:~/neofs$ cat ACL.json
{
    "Owner": {
        "DisplayName": "NS9iuCpxq8VzpKWLSwceguV5pz2YAYtG6M",
        "ID": "NS9iuCpxq8VzpKWLSwceguV5pz2YAYtG6M"
    },
    "Grants": [
        {
            "Grantee": {
                "ID": "02efc49d370eb40238b85d8469439b5dd70dac7b1567aaf251eef311a312098b13",
                "Type": "CanonicalUser"
            },
            "Permission": "FULL_CONTROL"
        },
        {
            "Grantee": {
                "ID": "02b6c1dc2f13c909918d05e1379f2d684c6fcf668986d199ede10053206acdc4a4",
                "Type": "CanonicalUser"
            },
            "Permission": "READ"
        },
        {
            "Grantee": {
                "Type": "Group",
                "URI": "http://acs.amazonaws.com/groups/global/AllUsers"
            },
            "Permission": "FULL_CONTROL"
        }
    ]
}

Logs:

Aug 25 08:23:34 az neofs-s3-gw[5874]: 2022-08-25T08:23:34.187Z        error        handler/util.go:25        could not update bucket acl        {"request_id": "40bf94c6-3c4d-43d9-b6d3-797fa1f7b94c", "method": "PutBucketACL", "bucket_name": "b-test-lock-20", "object_name": "", "error": "could not translate ast to table: form records: public key from string: encoding/hex: invalid byte: U+004E 'N'"}
Aug 25 08:23:34 az neofs-s3-gw[5874]: 2022-08-25T08:23:34.187Z        error        api/router.go:158        something went wrong        {"status": 500, "request_id": "40bf94c6-3c4d-43d9-b6d3-797fa1f7b94c", "method": "PutBucketACL", "description": "Internal Server Error"}
Aug 25 08:23:34 az neofs-s3-gw[5874]: 2022-08-25T08:23:34.565Z        error        handler/util.go:25        could not update bucket acl        {"request_id": "b787761c-7dd4-489a-8618-dfe99f3a7139", "method": "PutBucketACL", "bucket_name": "b-test-lock-20", "object_name": "", "error": "could not translate ast to table: form records: public key from string: encoding/hex: invalid byte: U+004E 'N'"}
Aug 25 08:23:34 az neofs-s3-gw[5874]: 2022-08-25T08:23:34.565Z        error        api/router.go:158        something went wrong        {"status": 500, "request_id": "b787761c-7dd4-489a-8618-dfe99f3a7139", "method": "PutBucketACL", "description": "Internal Server Error"}
Aug 25 08:23:36 az neofs-s3-gw[5874]: 2022-08-25T08:23:36.081Z        error        handler/util.go:25        could not update bucket acl        {"request_id": "f8ae9dd4-8b6c-4685-b83c-572caf448461", "method": "PutBucketACL", "bucket_name": "b-test-lock-20", "object_name": "", "error": "could not translate ast to table: form records: public key from string: encoding/hex: invalid byte: U+004E 'N'"}
Aug 25 08:23:36 az neofs-s3-gw[5874]: 2022-08-25T08:23:36.081Z        error        api/router.go:158        something went wrong        {"status": 500, "request_id": "f8ae9dd4-8b6c-4685-b83c-572caf448461", "method": "PutBucketACL", "description": "Internal Server Error"}

Product versions:

s3 gateway
Version: v0.23.0-36-g3b343d1-dirty
GoVersion: go1.18.4

NeoFS Storage node
Version: v0.31.0
GoVersion: go1.18.4

NeoGo
Version: 0.99.1
GoVersion: go1.18.4

s3 gateway config:

default_policy: REP 1 IN X CBF 1 SELECT 1 FROM * AS X
listen_address: 0.0.0.0:8084
logger:
  level: debug
max_clients_count: 600
max_clients_deadline: 60s
peers:
  '0':
    address: node1.neofs:8080
    priority: '1'
    weight: '1'
  '1':
    address: node2.neofs:8080
    priority: '2'
    weight: '0.25'
  '2':
    address: node3.neofs:8080
    priority: '2'
    weight: '0.25'
  '3':
    address: node4.neofs:8080
    priority: '2'
    weight: '0.25'
pool_error_threshold: 100
pprof:
  address: localhost:8085
  enabled: true
prometheus:
  address: localhost:8086
  enabled: true
resolve_order:
- nns
rpc_endpoint: http://node1.neofs:40332
tree:
  service: 172.26.163.38:8080
wallet:
  address: ''
  passphrase: ''
  path: /etc/neofs/s3/wallet.json

Original issue: https://github.com/nspcc-dev/neofs-s3-gw/issues/681 **Steps to reproduce:** 1. Create bucket with public-read-write ACL 2. Download bucket ACL to json file ``` aws --no-verify-ssl s3api get-bucket-acl --bucket b-test-lock-20 --endpoint-url http://172.26.163.38:8084 > ACL.json ``` 3. Try to upload the same json file (without any changes) following this command: ``` aws --no-verify-ssl s3api put-bucket-acl --bucket b-test-lock-20 --endpoint-url http://172.26.163.38:8084 --access-control-policy file://ACL.json An error occurred (InternalError) when calling the PutBucketAcl operation (reached max retries: 2): could not translate ast to table: form records: public key from string: encoding/hex: invalid byte: U+004E 'N' ``` ACL.json: ``` anikeev@NB-1670:~/neofs$ cat ACL.json { "Owner": { "DisplayName": "NS9iuCpxq8VzpKWLSwceguV5pz2YAYtG6M", "ID": "NS9iuCpxq8VzpKWLSwceguV5pz2YAYtG6M" }, "Grants": [ { "Grantee": { "ID": "02efc49d370eb40238b85d8469439b5dd70dac7b1567aaf251eef311a312098b13", "Type": "CanonicalUser" }, "Permission": "FULL_CONTROL" }, { "Grantee": { "ID": "02b6c1dc2f13c909918d05e1379f2d684c6fcf668986d199ede10053206acdc4a4", "Type": "CanonicalUser" }, "Permission": "READ" }, { "Grantee": { "Type": "Group", "URI": "http://acs.amazonaws.com/groups/global/AllUsers" }, "Permission": "FULL_CONTROL" } ] } ``` **Logs:** ``` Aug 25 08:23:34 az neofs-s3-gw[5874]: 2022-08-25T08:23:34.187Z error handler/util.go:25 could not update bucket acl {"request_id": "40bf94c6-3c4d-43d9-b6d3-797fa1f7b94c", "method": "PutBucketACL", "bucket_name": "b-test-lock-20", "object_name": "", "error": "could not translate ast to table: form records: public key from string: encoding/hex: invalid byte: U+004E 'N'"} Aug 25 08:23:34 az neofs-s3-gw[5874]: 2022-08-25T08:23:34.187Z error api/router.go:158 something went wrong {"status": 500, "request_id": "40bf94c6-3c4d-43d9-b6d3-797fa1f7b94c", "method": "PutBucketACL", "description": "Internal Server Error"} Aug 25 08:23:34 az neofs-s3-gw[5874]: 2022-08-25T08:23:34.565Z error handler/util.go:25 could not update bucket acl {"request_id": "b787761c-7dd4-489a-8618-dfe99f3a7139", "method": "PutBucketACL", "bucket_name": "b-test-lock-20", "object_name": "", "error": "could not translate ast to table: form records: public key from string: encoding/hex: invalid byte: U+004E 'N'"} Aug 25 08:23:34 az neofs-s3-gw[5874]: 2022-08-25T08:23:34.565Z error api/router.go:158 something went wrong {"status": 500, "request_id": "b787761c-7dd4-489a-8618-dfe99f3a7139", "method": "PutBucketACL", "description": "Internal Server Error"} Aug 25 08:23:36 az neofs-s3-gw[5874]: 2022-08-25T08:23:36.081Z error handler/util.go:25 could not update bucket acl {"request_id": "f8ae9dd4-8b6c-4685-b83c-572caf448461", "method": "PutBucketACL", "bucket_name": "b-test-lock-20", "object_name": "", "error": "could not translate ast to table: form records: public key from string: encoding/hex: invalid byte: U+004E 'N'"} Aug 25 08:23:36 az neofs-s3-gw[5874]: 2022-08-25T08:23:36.081Z error api/router.go:158 something went wrong {"status": 500, "request_id": "f8ae9dd4-8b6c-4685-b83c-572caf448461", "method": "PutBucketACL", "description": "Internal Server Error"} ``` **Product versions:** ``` s3 gateway Version: v0.23.0-36-g3b343d1-dirty GoVersion: go1.18.4 NeoFS Storage node Version: v0.31.0 GoVersion: go1.18.4 NeoGo Version: 0.99.1 GoVersion: go1.18.4 ``` **s3 gateway config:** ``` default_policy: REP 1 IN X CBF 1 SELECT 1 FROM * AS X listen_address: 0.0.0.0:8084 logger: level: debug max_clients_count: 600 max_clients_deadline: 60s peers: '0': address: node1.neofs:8080 priority: '1' weight: '1' '1': address: node2.neofs:8080 priority: '2' weight: '0.25' '2': address: node3.neofs:8080 priority: '2' weight: '0.25' '3': address: node4.neofs:8080 priority: '2' weight: '0.25' pool_error_threshold: 100 pprof: address: localhost:8085 enabled: true prometheus: address: localhost:8086 enabled: true resolve_order: - nns rpc_endpoint: http://node1.neofs:40332 tree: service: 172.26.163.38:8080 wallet: address: '' passphrase: '' path: /etc/neofs/s3/wallet.json ```