diff --git a/.gitignore b/.gitignore index 009aceac0..1e76ffef3 100644 --- a/.gitignore +++ b/.gitignore @@ -28,4 +28,3 @@ debian/files debian/*.log debian/*.substvars debian/frostfs-s3-gw/ - diff --git a/CHANGELOG.md b/CHANGELOG.md index d5bfd04a4..b89f88dc7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,7 +24,7 @@ This document outlines major changes between releases. - Return error on invalid LocationConstraint (TrueCloudLab#23) - Place billing metrics to separate url path (TrueCloudLab#26) - Add generated deb builder files to .gitignore, and fix typo (TrueCloudLab#28) -- Limit number of objects to delete at one time (TrueCloudLab#37) +- Limit number of objects to delete at one time (TrueCloudLab#37) - CompleteMultipartUpload handler now sends whitespace characters to keep alive client's connection (#60) - Support new system attributes (#64) @@ -91,7 +91,7 @@ If you configure application using `.yaml` file change: ### Changed - GitHub actions update (#710) - Makefile help (#725) -- Optimized object tags setting (#669) +- Optimized object tags setting (#669) - Improved logging (#728) - Unified unit test names (#617) - Improved docs (#732) @@ -156,7 +156,7 @@ If you configure application using environment variables change: * `S3_GW_LISTEN_DOMAINS_N` -> `S3_GW_LISTEN_DOMAINS` (use it as array variable) If you configure application using `.yaml` file change: -* `wallet` -> `wallet.path` +* `wallet` -> `wallet.path` * `address` -> `wallet.address` * `listen_domains.n` -> `listen_domains` (use it as array param) @@ -179,8 +179,8 @@ If you configure application using `.yaml` file change: - Rely on string sanitizing from zap (#498) ### Updating from v0.22.0 -1. To enable pprof use `pprof.enabled` instead of `pprof` in config. -To enable prometheus metrics use `prometheus.enabled` instead of `metrics` in config. +1. To enable pprof use `pprof.enabled` instead of `pprof` in config. +To enable prometheus metrics use `prometheus.enabled` instead of `metrics` in config. If you are using the command line flags you can skip this step. ## [0.22.0] - 2022-07-25 @@ -202,7 +202,7 @@ Tree service support - Cache type cast error logging (#465) - `docker/*` target in Makefile (#471) - Pre signed requests (#529) -- Tagging and ACL notifications (#361) +- Tagging and ACL notifications (#361) - AWSv4 signer package to improve compatibility with S3 clients (#528) - Extension mimetype detector (#289) - Default params documentation (#592) @@ -236,7 +236,7 @@ Tree service support - Obtainment of ETag value (#431) ### Changed -- Authmate doesn't parse session context anymore, now it accepts application defined +- Authmate doesn't parse session context anymore, now it accepts application defined flexible structure with container ID in human-readable format (#428) ## [0.20.0] - 2022-04-29 @@ -246,19 +246,19 @@ Tree service support - Support of basic notifications (#357, #358, #359) ### Changed -- Logger behavior: now it writes to stderr instead of stdout, app name and - version are always presented and fixed, all user options except of `level` are +- Logger behavior: now it writes to stderr instead of stdout, app name and + version are always presented and fixed, all user options except of `level` are dropped (#380) - Improved docs, added config examples (#396, #398) - Updated NeoFS SDK (#365, #409) ### Fixed - Added check of `SetEACL` tokens before processing of requests (#347) -- Authmate: returned lost session tokens when a parameter `--session-token` is +- Authmate: returned lost session tokens when a parameter `--session-token` is omitted (#387) - Error when a bucket hasn't a settings file (#389) - Response to a request to delete not existing object (#392) -- Replaced gate key in ACL Grantee by key of bearer token issuer (#395) +- Replaced gate key in ACL Grantee by key of bearer token issuer (#395) - Missing attach of bearer token to requests to put system object (#399) - Deletion of system object while CompleteMultipartUpload (#400) - Improved English in docs and comments (#405) @@ -285,7 +285,7 @@ Tree service support - Updated NeoFS SDK to v1.0.0-rc.3 (#297, #333, #346, #376) - Authmate: changed session token rules handling (#329, #336, #338, #352) - Changed status code for some failed requests (#308) -- GetBucketLocation returns policy name used at bucket creation (#301) +- GetBucketLocation returns policy name used at bucket creation (#301) ### Fixed - Waiting for bucket to be deleted (#366) @@ -305,7 +305,7 @@ Tree service support ## [0.18.0] - 2021-12-16 ### Added -- Support for MultipartUpload (#186, #187) +- Support for MultipartUpload (#186, #187) - CORS support (#217) - Authmate supports setting of tokens lifetime in a more convenient format (duration) (#258) - Generation of a random key for `--no-sign-request` (#276) @@ -334,10 +334,10 @@ With this release we introduce [ceph-based](https://github.com/ceph/s3-tests) S3 * POST uploading support (#190) * Delete marker support (#248) * Expiration for access box (#255) -* AWS CLI credential generating by authmate (#241) +* AWS CLI credential generating by authmate (#241) ### Changed -* Default placement policy is now configurable (#218) +* Default placement policy is now configurable (#218) * README is split into different files (#210) * Unified error handling (#89, #149, #184) * Authmate issue-secret response contains container id (#163) @@ -382,7 +382,7 @@ cryptography with NEP-6 wallet support. * Support of time-based conditional CopyObject and GetObject (#94) ### Changed - * Accesskey format: now `0` used as a delimiter between container ID and object + * Accesskey format: now `0` used as a delimiter between container ID and object ID instead of `_` (#164) * Accessbox is encoded in protobuf format (#48) * Authentication uses secp256r1 instead of ed25519 (#75) diff --git a/CREDITS.md b/CREDITS.md index b7896e2bf..6338de593 100644 --- a/CREDITS.md +++ b/CREDITS.md @@ -16,4 +16,4 @@ In chronological order: - Elizaveta Chichindaeva - Stanislav Bogatyrev - Anastasia Prasolova -- Leonard Liubich \ No newline at end of file +- Leonard Liubich diff --git a/README.md b/README.md index c76226c56..6dd789d0d 100644 --- a/README.md +++ b/README.md @@ -66,7 +66,7 @@ $ S3_GW_PEERS_0_ADDRESS=grpcs://192.168.130.72:8080 \ ## Domains -By default, s3-gw enable only `path-style access`. +By default, s3-gw enable only `path-style access`. To be able to use both: `virtual-hosted-style` and `path-style` access you must configure `listen_domains`: ```shell @@ -100,6 +100,6 @@ Also, you can configure domains using `.env` variables or `yaml` file. - [AWS S3 API compatibility](./docs/aws_s3_compat.md) - [AWS S3 Compatibility test results](./docs/s3_test_results.md) -## Credits +## Credits Please see [CREDITS](CREDITS.md) for details. diff --git a/api/handler/acl_test.go b/api/handler/acl_test.go index 8c25c9346..f3c877c52 100644 --- a/api/handler/acl_test.go +++ b/api/handler/acl_test.go @@ -1368,7 +1368,7 @@ func TestBucketPolicyUnmarshal(t *testing.T) { }, "Effect": "Allow", "Action": [ - "s3:GetObject", + "s3:GetObject", "s3:GetObjectVersion" ], "Resource": [ diff --git a/api/handler/multipart_upload.go b/api/handler/multipart_upload.go index 480f3764f..f0b36ab70 100644 --- a/api/handler/multipart_upload.go +++ b/api/handler/multipart_upload.go @@ -753,12 +753,3 @@ func periodicXMLWriter(w io.Writer, dur time.Duration) (stop func() bool) { return stop } - -// periodicWriterErrorSender returns handler function to send error. If header is -// alreay written by periodic XML writer, do not send HTTP and XML headers. -func (h *handler) periodicWriterErrorSender(headerWritten bool) func(http.ResponseWriter, string, *api.ReqInfo, error, ...zap.Field) { - if headerWritten { - return h.logAndSendErrorNoHeader - } - return h.logAndSendError -} diff --git a/cmd/s3-authmate/main.go b/cmd/s3-authmate/main.go index bfdf88196..a1629ad71 100644 --- a/cmd/s3-authmate/main.go +++ b/cmd/s3-authmate/main.go @@ -245,7 +245,7 @@ func issueSecret() *cli.Command { }, &cli.DurationFlag{ Name: "lifetime", - Usage: `Lifetime of tokens. For example 50h30m (note: max time unit is an hour so to set a day you should use 24h). + Usage: `Lifetime of tokens. For example 50h30m (note: max time unit is an hour so to set a day you should use 24h). It will be ceil rounded to the nearest amount of epoch.`, Required: false, Destination: &lifetimeFlag, @@ -394,7 +394,7 @@ Note to override credentials you must provide both access key and secret key.`, Flags: []cli.Flag{ &cli.DurationFlag{ Name: "lifetime", - Usage: `Lifetime of presigned URL. For example 50h30m (note: max time unit is an hour so to set a day you should use 24h). + Usage: `Lifetime of presigned URL. For example 50h30m (note: max time unit is an hour so to set a day you should use 24h). It will be ceil rounded to the nearest amount of epoch.`, Required: false, Destination: &lifetimeFlag, diff --git a/creds/accessbox/accessbox.proto b/creds/accessbox/accessbox.proto index 3d0a13cac..1cec16d18 100644 --- a/creds/accessbox/accessbox.proto +++ b/creds/accessbox/accessbox.proto @@ -27,4 +27,3 @@ message Tokens { bytes bearerToken = 2 [json_name = "bearerToken"]; repeated bytes sessionTokens = 3 [json_name = "sessionTokens"]; } - diff --git a/debian/copyright b/debian/copyright index 9f0180be1..e5754d95b 100644 --- a/debian/copyright +++ b/debian/copyright @@ -14,11 +14,11 @@ License: AGPL-3.0-only This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation; version 3. - + This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. - + You should have received a copy of the GNU General Public License - along with this program. If not, see . + along with this program. If not, see . diff --git a/debian/frostfs-s3-gw.postinst b/debian/frostfs-s3-gw.postinst old mode 100644 new mode 100755 index 82345f655..117b88273 --- a/debian/frostfs-s3-gw.postinst +++ b/debian/frostfs-s3-gw.postinst @@ -29,9 +29,9 @@ case "$1" in chmod -f 0640 /etc/frostfs/$USERNAME/config.yaml || true fi USERDIR=$(getent passwd "frostfs-$USERNAME" | cut -d: -f6) - if ! dpkg-statoverride --list frostfs-$USERDIR >/dev/null; then - chown -f frostfs-$USERNAME: $USERDIR - chown -f frostfs-$USERNAME: $USERDIR/rules.json + if ! dpkg-statoverride --list frostfs-"$USERDIR" >/dev/null; then + chown -f frostfs-$USERNAME: "$USERDIR" + chown -f frostfs-$USERNAME: "$USERDIR"/rules.json fi ;; diff --git a/debian/frostfs-s3-gw.postrm b/debian/frostfs-s3-gw.postrm old mode 100644 new mode 100755 diff --git a/debian/frostfs-s3-gw.preinst b/debian/frostfs-s3-gw.preinst old mode 100644 new mode 100755 diff --git a/debian/frostfs-s3-gw.prerm b/debian/frostfs-s3-gw.prerm old mode 100644 new mode 100755 diff --git a/debian/rules b/debian/rules index fbd95cc71..519ed2f7f 100755 --- a/debian/rules +++ b/debian/rules @@ -8,9 +8,7 @@ SERVICE = frostfs-s3-gw dh $@ override_dh_installsystemd: - dh_installsystemd --no-enable --no-start $(SERVICE).service + dh_installsystemd --no-enable --no-start $(SERVICE).service override_dh_installchangelogs: - dh_installchangelogs -k CHANGELOG.md - - + dh_installchangelogs -k CHANGELOG.md diff --git a/docs/authmate.md b/docs/authmate.md index e753b417a..4e33f7a44 100644 --- a/docs/authmate.md +++ b/docs/authmate.md @@ -35,8 +35,8 @@ To generate a wallet for a gateway, run the following command: $ ./neo-go wallet init -a -w wallet.json Enter the name of the account > AccountTestName -Enter passphrase > -Confirm passphrase > +Enter passphrase > +Confirm passphrase > { "version": "3.0", @@ -89,18 +89,18 @@ put them as an object into a container on the FrostFS network. **Required parameters:** * `--wallet` is a path to a wallet `.json` file. You can provide a passphrase to decrypt -a wallet via environment variable `AUTHMATE_WALLET_PASSPHRASE`, or you will be asked to enter a passphrase +a wallet via environment variable `AUTHMATE_WALLET_PASSPHRASE`, or you will be asked to enter a passphrase interactively. You can also specify an account address to use from a wallet using the `--address` parameter. * `--peer` is an address of a FrostFS peer to connect to * `--gate-public-key` is a public `secp256r1` 33-byte short key of a gate (use flags repeatedly for multiple gates). The tokens are encrypted by a set of gateway keys, so you need to pass them as well. -You can issue a secret using the parameters above only. The tool will +You can issue a secret using the parameters above only. The tool will 1. create a new container 1. without a friendly name - 2. with ACL `0x3c8c8cce` -- all operations are forbidden for `OTHERS` and `BEARER` user groups, except for `GET` - 3. with policy `REP 2 IN X CBF 3 SELECT 2 FROM * AS X` -2. put bearer and session tokens with default rules (details in [Bearer tokens](#Bearer tokens) and + 2. with ACL `0x3c8c8cce` -- all operations are forbidden for `OTHERS` and `BEARER` user groups, except for `GET` + 3. with policy `REP 2 IN X CBF 3 SELECT 2 FROM * AS X` +2. put bearer and session tokens with default rules (details in [Bearer tokens](#Bearer tokens) and [Session tokens](#Session tokens)) E.g.: @@ -109,9 +109,9 @@ $ frostfs-s3-authmate issue-secret --wallet wallet.json \ --peer 192.168.130.71:8080 \ --gate-public-key 0313b1ac3a8076e155a7e797b24f0b650cccad5941ea59d7cfd51a024a8b2a06bf\ --gate-public-key 0317585fa8274f7afdf1fc5f2a2e7bece549d5175c4e5182e37924f30229aef967 - - Enter password for wallet.json > - + + Enter password for wallet.json > + { "access_key_id": "5g933dyLEkXbbAspouhPPTiyLZRg4axBW1axSPD87eVT0AiXsH4AjYy1iTJ4C1WExzjBrSobJsQFWEyKLREe5sQYM", "secret_access_key": "438bbd8243060e1e1c9dd4821756914a6e872ce29bf203b68f81b140ac91231c", @@ -130,9 +130,9 @@ the secret. Format of `access_key_id`: `%cid0%oid`, where 0(zero) is a delimiter * `--container-friendly-name` -- name of a container with tokens, by default container will not have a friendly name * `--container-placement-policy` - placement policy of auth container to put the secret into. Default value is `REP 2 IN X CBF 3 SELECT 2 FROM * AS X` -* `--lifetime`-- lifetime of tokens. For example 50h30m (note: max time unit is an hour so to set a day you should use +* `--lifetime`-- lifetime of tokens. For example 50h30m (note: max time unit is an hour so to set a day you should use 24h). Default value is `720h` (30 days). It will be ceil rounded to the nearest amount of epoch -* `--aws-cli-credentials` - path to the aws cli credentials file, where authmate will write `access_key_id` and +* `--aws-cli-credentials` - path to the aws cli credentials file, where authmate will write `access_key_id` and `secret_access_key` to ### Bearer tokens @@ -161,7 +161,7 @@ where content of `bearer-rules.json`: } ``` -**Note:** such rules allow all operations for all users (the same behavior when records are empty). +**Note:** such rules allow all operations for all users (the same behavior when records are empty). To restrict access you MUST provide records with `DENY` action. That's why we recommend always place such records at the end of records (see default rules below) to prevent undesirable access violation. Since the rules are applied from top to bottom, they do not override what was previously allowed. @@ -192,7 +192,7 @@ If bearer rules are not set, a token will be auto-generated with a value: ### Session tokens -With a session token, there are 3 options: +With a session token, there are 3 options: 1. append `--session-tokens` parameter with your custom rules in json format (as a string or file path). E.g.: ```shell $ frostfs-s3-authmate issue-secret --wallet wallet.json \ @@ -224,8 +224,8 @@ If `containerID` is `null` or omitted, then session token rule will be applied to all containers. Otherwise, specify `containerID` value in human-redabale format (base58 encoded string). -> **_NB!_** To create buckets in FrostFS it's necessary to have session tokens with `PUT` and `SETEACL` permissions, that's why -the authmate creates a `SETEACL` session token automatically in case when a user specified the token rule with `PUT` and +> **_NB!_** To create buckets in FrostFS it's necessary to have session tokens with `PUT` and `SETEACL` permissions, that's why +the authmate creates a `SETEACL` session token automatically in case when a user specified the token rule with `PUT` and forgot about the rule with `SETEACL`. 2. append `--session-tokens` parameter with the value `none` -- no session token will be created @@ -234,7 +234,7 @@ in example above) ### Containers policy -Rules for mapping of `LocationConstraint` ([aws spec](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateBucket.html#API_CreateBucket_RequestBody)) +Rules for mapping of `LocationConstraint` ([aws spec](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateBucket.html#API_CreateBucket_RequestBody)) to `PlacementPolicy` can be set via parameter `--container-policy` (json-string and file path allowed): ```json @@ -248,7 +248,7 @@ can be set via parameter `--container-policy` (json-string and file path allowed ## Obtainment of a secret access key You can get a secret access key associated with an access key ID by obtaining a -secret stored on the FrostFS network. Here is an example of providing one password (for `wallet.json`) via env variable +secret stored on the FrostFS network. Here is an example of providing one password (for `wallet.json`) via env variable and the other (for `gate-wallet.json`) interactively: ```shell @@ -267,14 +267,14 @@ Enter password for gate-wallet.json > ## Generate presigned URL -You can generate [presigned url](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html) -using AWS credentials from `~/.aws/credentials` (you can specify profile using the `--profile` flag) +You can generate [presigned url](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html) +using AWS credentials from `~/.aws/credentials` (you can specify profile using the `--profile` flag) with the following command: ```shell $ frostfs-s3-authmate generate-presigned-url --endpoint http://localhost:8084 \ --method get --bucket presigned --object obj --lifetime 30s - + { "URL": "http://localhost:8084/presigned/obj?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=6UpmiuYspPLMWfyhEKYmZQSsTGkFLS5MhQVdsda3fhz908Hw9eo9urTmaJtfvHMHUpY8SWAptk61bns2Js8f1M5tZ%2F20220615%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20220615T084203Z&X-Amz-Expires=30&X-Amz-SignedHeaders=host&X-Amz-Signature=47f74d4b84566708a17dded05cce732690745f141235215104ad051e265e3c59" } @@ -297,7 +297,7 @@ $ frostfs-s3-authmate generate-presigned-url --endpoint http://localhost:8084 \ You can also can get the presigned URL (only for GET) using aws cli v2: ```shell -$ aws s3 --endpoint http://localhost:8084 presign s3://pregigned/obj +$ aws s3 --endpoint http://localhost:8084 presign s3://pregigned/obj http://localhost:8084/pregigned/obj?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=6UpmiuYspPLMWfyhEKYmZQSsTGkFLS5MhQVdsda3fhz908Hw9eo9urTmaJtfvHMHUpY8SWAptk61bns2Js8f1M5tZ%2F20220615%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20220615T072348Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=b82c13952534b1bba699a718f2d42d135c2833a1e64030d4ce0e198af46551d4 ``` diff --git a/docs/aws_cli.md b/docs/aws_cli.md index 270b1d55d..8614af60d 100644 --- a/docs/aws_cli.md +++ b/docs/aws_cli.md @@ -16,8 +16,8 @@ after you enter this command, the AWS CLI will prompt you for four pieces of inf ``` AWS Access Key ID [None]: 5g933dyLEkXbbAspouhPPTiyLZRg4axBW1axSPD87eVT0AiXsH4AjYy1iTJ4C1WExzjBrSobJsQFWEyKLREe5sQYM AWS Secret Access Key [None]: 438bbd8243060e1e1c9dd4821756914a6e872ce29bf203b68f81b140ac91231c -Default region name [None]: ru -Default output format [none]: json +Default region name [None]: ru +Default output format [none]: json ``` ## Basic usage @@ -26,11 +26,11 @@ Default output format [none]: json ### Bucket -#### Obtainment of a list of buckets +#### Obtainment of a list of buckets To view the list of the buckets in the FrostFS node, to which the gateway is connected, enter the following command: ``` -$ aws s3 ls +$ aws s3 ls ``` #### Creation of a bucket @@ -41,7 +41,7 @@ To create a bucket, run the following command: ``` $ aws s3api create-bucket --bucket %BUCKET_NAME --acl %ACL ``` -where `%ACL` can be represented by a hex encoded value or by keywords `public-read-write`, `private`, `public-read`. +where `%ACL` can be represented by a hex encoded value or by keywords `public-read-write`, `private`, `public-read`. If the parameter is not set, the default value is `private`. > **_NOTE:_** Bucket creation uses async-poll approach. `BucketAlreadyOwnedByYou` @@ -54,7 +54,7 @@ If the parameter is not set, the default value is `private`. > (previous success will result in the status mentioned above). It is worth clarifying > that the final success of the bucket creation is not guaranteed after a timeout error. -#### Deletion of a bucket +#### Deletion of a bucket To delete a bucket, execute the following command: ``` @@ -67,7 +67,7 @@ $ aws s3api delete-bucket --bucket %BUCKET_NAME To view the list of the objects in a bucket, run: ``` -$ aws s3api list-objects --bucket %BUCKET_NAME +$ aws s3api list-objects --bucket %BUCKET_NAME ``` #### Upload of a file @@ -83,7 +83,7 @@ where %OBJECT_KEY is the filepath of an object in FrostFS To upload a dir into a bucket in the FrostFS network, run the following command: ``` -$ aws s3 sync %DIRPATH s3://%BUCKET_NAME +$ aws s3 sync %DIRPATH s3://%BUCKET_NAME ``` #### Download of a file diff --git a/docs/aws_s3_compat.md b/docs/aws_s3_compat.md index 4bde43ae5..7d0c07f7b 100644 --- a/docs/aws_s3_compat.md +++ b/docs/aws_s3_compat.md @@ -31,7 +31,7 @@ Reference: ## ACL For now there are some limitations: -* [Bucket policy](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-policies.html) supports only one `Principal` per `Statement`. +* [Bucket policy](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-policies.html) supports only one `Principal` per `Statement`. Principal must be `"AWS": "*"` (to refer all users) or `"CanonicalUser": "0313b1ac3a8076e155a7e797b24f0b650cccad5941ea59d7cfd51a024a8b2a06bf"` (hex encoded public key of desired user). * Resource in bucket policy is an array. Each item MUST contain bucket name, CAN contain object name (wildcards are not supported): ```json @@ -160,7 +160,7 @@ See also `GetObject` and other method parameters. | 🔵 | GetBucketInventoryConfiguration | | | 🔵 | ListBucketInventoryConfigurations | | | 🔵 | PutBucketInventoryConfiguration | | - + ## Lifecycle | | Method | Comments | diff --git a/docs/configuration.md b/docs/configuration.md index 115382779..695109cdb 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -62,7 +62,7 @@ $ frostfs-s3-gw --listen_address 192.168.130.130:443 \ --tls.key_file=key.pem --tls.cert_file=cert.pem ``` -Using these flag you can configure only one address. To set multiple addresses use yaml config. +Using these flag you can configure only one address. To set multiple addresses use yaml config. ### RPC endpoint and resolving of bucket names @@ -129,7 +129,7 @@ enable pprof (value from `./config/dir/pprof.yaml`) and prometheus (value from ` ### Reload on SIGHUP -Some config values can be reloaded on SIGHUP signal. +Some config values can be reloaded on SIGHUP signal. Such parameters have special mark in tables below. You can send SIGHUP signal to app using the following command: @@ -207,7 +207,7 @@ pool_error_threshold: 100 max_clients_count: 100 max_clients_deadline: 30s -allowed_access_key_id_prefixes: +allowed_access_key_id_prefixes: - Ck9BHsgKcnwfCTUSFm6pxhoNS4cBqgN2NQ8zVgPjqZDX - 3stjWenX15YwYzczMr88gy3CQr4NYFBQ8P7keGzH5QFn ``` @@ -296,7 +296,7 @@ File for `region_mapping` must contain something like this: } ``` -**Note:** on SIGHUP reload policies will be updated only if both parameters are valid. +**Note:** on SIGHUP reload policies will be updated only if both parameters are valid. So if you change `default` to some valid value and set invalid path in `region_mapping` the `default` value won't be changed. ### `server` section @@ -469,7 +469,7 @@ prometheus: # `frostfs` section -Contains parameters of requests to FrostFS. +Contains parameters of requests to FrostFS. This value can be overridden with `X-Amz-Meta-Frostfs-Copies-Number` header for `PutObject`, `CopyObject`, `CreateMultipartUpload`. ```yaml diff --git a/docs/s3_test_results.md b/docs/s3_test_results.md index 1bc7eb6ec..532bfc365 100644 --- a/docs/s3_test_results.md +++ b/docs/s3_test_results.md @@ -77,72 +77,72 @@ Compatibility: 30/25/29 out of 33 Compatibility: 33/43/37 out of 64 -| | Test | s3-gw | minio | aws s3 | -|----|------------------------------------------------------------------------------------------------|-------------|-------|--------| -| 1 | s3tests_boto3.functional.test_s3.test_put_object_ifmatch_good | ok | ok | ERROR | -| 2 | s3tests_boto3.functional.test_s3.test_put_object_ifmatch_failed | FAIL | FAIL | FAIL | -| 3 | s3tests_boto3.functional.test_s3.test_put_object_ifmatch_overwrite_existed_good | ok | ok | ERROR | -| 4 | s3tests_boto3.functional.test_s3.test_put_object_ifmatch_nonexisted_failed | FAIL | FAIL | FAIL | -| 5 | s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_good | ok | ok | ERROR | -| 6 | s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_failed | FAIL | FAIL | FAIL | -| 7 | s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_nonexisted_good | ok | ok | ERROR | -| 8 | s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_overwrite_existed_failed | FAIL | FAIL | FAIL | -| 9 | s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_short | UNSUPPORTED | ok | ok | -| 10 | s3tests_boto3.functional.test_headers.test_object_create_bad_md5_bad | UNSUPPORTED | ok | ok | -| 11 | s3tests_boto3.functional.test_headers.test_object_create_bad_md5_empty | UNSUPPORTED | ok | ok | -| 12 | s3tests_boto3.functional.test_headers.test_object_create_bad_md5_none | ok | ok | ok | -| 13 | s3tests_boto3.functional.test_headers.test_object_create_bad_expect_mismatch | ERROR | ERROR | ok | -| 14 | s3tests_boto3.functional.test_headers.test_object_create_bad_expect_empty | ok | ok | ok | -| 15 | s3tests_boto3.functional.test_headers.test_object_create_bad_expect_none | ok | ok | ok | -| 16 | s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_empty | FAIL | FAIL | ok | -| 17 | s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_negative | ok | ok | ok | -| 18 | s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_none | FAIL | FAIL | FAIL | -| 19 | s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_mismatch_above | ERROR | ERROR | ERROR | -| 20 | s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_invalid | ok | ok | ok | -| 21 | s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_empty | ok | ok | ok | -| 22 | s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_none | ok | ok | ok | -| 23 | s3tests_boto3.functional.test_headers.test_object_create_bad_authorization_empty | FAIL | FAIL | FAIL | -| 24 | s3tests_boto3.functional.test_headers.test_object_create_date_and_amz_date | ERROR | ERROR | ERROR | -| 25 | s3tests_boto3.functional.test_headers.test_object_create_amz_date_and_no_date | ERROR | ERROR | ERROR | -| 26 | s3tests_boto3.functional.test_headers.test_object_create_bad_authorization_none | FAIL | FAIL | FAIL | -| 27 | s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_garbage_aws2 | UNSUPPORTED | ok | ok | -| 28 | s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_mismatch_below_aws2 | FAIL | FAIL | ok | -| 29 | s3tests_boto3.functional.test_headers.test_object_create_bad_authorization_incorrect_aws2 | FAIL | FAIL | FAIL | -| 30 | s3tests_boto3.functional.test_headers.test_object_create_bad_authorization_invalid_aws2 | FAIL | FAIL | FAIL | -| 31 | s3tests_boto3.functional.test_headers.test_object_create_bad_ua_empty_aws2 | ERROR | ok | ok | -| 32 | s3tests_boto3.functional.test_headers.test_object_create_bad_ua_none_aws2 | ERROR | ok | ok | -| 33 | s3tests_boto3.functional.test_headers.test_object_create_bad_date_invalid_aws2 | FAIL | FAIL | ok | -| 34 | s3tests_boto3.functional.test_headers.test_object_create_bad_date_empty_aws2 | FAIL | FAIL | ok | -| 35 | s3tests_boto3.functional.test_headers.test_object_create_bad_date_none_aws2 | FAIL | FAIL | FAIL | -| 36 | s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_today_aws2 | FAIL | ok | ok | -| 37 | s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_epoch_aws2 | FAIL | FAIL | ok | -| 38 | s3tests_boto3.functional.test_headers.test_object_create_bad_date_after_end_aws2 | FAIL | ok | ok | -| 39 | s3tests_boto3.functional.test_s3.test_object_anon_put | ok | ok | ok | -| 40 | s3tests_boto3.functional.test_s3.test_object_put_authenticated | ok | ok | ok | -| 41 | s3tests_boto3.functional.test_s3.test_object_raw_put_authenticated_expired | ok | FAIL | FAIL | -| 42 | s3tests_boto3.functional.test_s3.test_object_write_file | ok | ok | ok | -| 43 | s3tests_boto3.functional.test_s3.test_object_write_check_etag | FAIL | ok | ok | -| 44 | s3tests_boto3.functional.test_s3.test_object_write_cache_control | ok | ok | ok | -| 45 | s3tests_boto3.functional.test_s3.test_object_write_expires | ok | ok | ok | -| 46 | s3tests_boto3.functional.test_s3.test_object_write_read_update_read_delete | ok | ok | ok | -| 47 | s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_good | ok | ok | ok | -| 48 | s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_empty | ERROR | ok | ok | -| 49 | s3tests_boto3.functional.test_s3.test_object_set_get_metadata_overwrite_to_empty | ERROR | ok | ok | -| 50 | s3tests_boto3.functional.test_s3.test_object_set_get_non_utf8_metadata | ok | ok | FAIL | -| 51 | s3tests_boto3.functional.test_s3.test_object_set_get_metadata_empty_to_unreadable_prefix | ok | ok | FAIL | -| 52 | s3tests_boto3.functional.test_s3.test_object_set_get_metadata_empty_to_unreadable_suffix | ok | ok | FAIL | -| 53 | s3tests_boto3.functional.test_s3.test_object_set_get_metadata_empty_to_unreadable_infix | ok | ok | FAIL | -| 54 | s3tests_boto3.functional.test_s3.test_object_metadata_replaced_on_put | ok | ok | ok | -| 55 | s3tests_boto3.functional.test_s3.test_object_write_to_nonexist_bucket | ok | ok | ok | -| 56 | s3tests_boto3.functional.test_s3.test_atomic_write_1mb | ok | ok | ok | -| 57 | s3tests_boto3.functional.test_s3.test_atomic_write_4mb | ok | ok | ok | -| 58 | s3tests_boto3.functional.test_s3.test_atomic_write_8mb | ok | ok | ok | -| 59 | s3tests_boto3.functional.test_s3.test_atomic_dual_write_1mb | ok | ok | ERROR | -| 60 | s3tests_boto3.functional.test_s3.test_atomic_dual_write_4mb | ok | ok | ERROR | -| 61 | s3tests_boto3.functional.test_s3.test_atomic_dual_write_8mb | ok | ok | ERROR | -| 62 | s3tests_boto3.functional.test_s3.test_atomic_conditional_write_1mb | ok | ok | ERROR | -| 63 | s3tests_boto3.functional.test_s3.test_atomic_dual_conditional_write_1mb | FAIL | FAIL | FAIL | -| 64 | s3tests_boto3.functional.test_s3.test_atomic_write_bucket_gone | ok | ok | ok | +| | Test | s3-gw | minio | aws s3 | +|-----|------------------------------------------------------------------------------------------------|-------------|-------|--------| +| 1 | s3tests_boto3.functional.test_s3.test_put_object_ifmatch_good | ok | ok | ERROR | +| 2 | s3tests_boto3.functional.test_s3.test_put_object_ifmatch_failed | FAIL | FAIL | FAIL | +| 3 | s3tests_boto3.functional.test_s3.test_put_object_ifmatch_overwrite_existed_good | ok | ok | ERROR | +| 4 | s3tests_boto3.functional.test_s3.test_put_object_ifmatch_nonexisted_failed | FAIL | FAIL | FAIL | +| 5 | s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_good | ok | ok | ERROR | +| 6 | s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_failed | FAIL | FAIL | FAIL | +| 7 | s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_nonexisted_good | ok | ok | ERROR | +| 8 | s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_overwrite_existed_failed | FAIL | FAIL | FAIL | +| 9 | s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_short | UNSUPPORTED | ok | ok | +| 10 | s3tests_boto3.functional.test_headers.test_object_create_bad_md5_bad | UNSUPPORTED | ok | ok | +| 11 | s3tests_boto3.functional.test_headers.test_object_create_bad_md5_empty | UNSUPPORTED | ok | ok | +| 12 | s3tests_boto3.functional.test_headers.test_object_create_bad_md5_none | ok | ok | ok | +| 13 | s3tests_boto3.functional.test_headers.test_object_create_bad_expect_mismatch | ERROR | ERROR | ok | +| 14 | s3tests_boto3.functional.test_headers.test_object_create_bad_expect_empty | ok | ok | ok | +| 15 | s3tests_boto3.functional.test_headers.test_object_create_bad_expect_none | ok | ok | ok | +| 16 | s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_empty | FAIL | FAIL | ok | +| 17 | s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_negative | ok | ok | ok | +| 18 | s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_none | FAIL | FAIL | FAIL | +| 19 | s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_mismatch_above | ERROR | ERROR | ERROR | +| 20 | s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_invalid | ok | ok | ok | +| 21 | s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_empty | ok | ok | ok | +| 22 | s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_none | ok | ok | ok | +| 23 | s3tests_boto3.functional.test_headers.test_object_create_bad_authorization_empty | FAIL | FAIL | FAIL | +| 24 | s3tests_boto3.functional.test_headers.test_object_create_date_and_amz_date | ERROR | ERROR | ERROR | +| 25 | s3tests_boto3.functional.test_headers.test_object_create_amz_date_and_no_date | ERROR | ERROR | ERROR | +| 26 | s3tests_boto3.functional.test_headers.test_object_create_bad_authorization_none | FAIL | FAIL | FAIL | +| 27 | s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_garbage_aws2 | UNSUPPORTED | ok | ok | +| 28 | s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_mismatch_below_aws2 | FAIL | FAIL | ok | +| 29 | s3tests_boto3.functional.test_headers.test_object_create_bad_authorization_incorrect_aws2 | FAIL | FAIL | FAIL | +| 30 | s3tests_boto3.functional.test_headers.test_object_create_bad_authorization_invalid_aws2 | FAIL | FAIL | FAIL | +| 31 | s3tests_boto3.functional.test_headers.test_object_create_bad_ua_empty_aws2 | ERROR | ok | ok | +| 32 | s3tests_boto3.functional.test_headers.test_object_create_bad_ua_none_aws2 | ERROR | ok | ok | +| 33 | s3tests_boto3.functional.test_headers.test_object_create_bad_date_invalid_aws2 | FAIL | FAIL | ok | +| 34 | s3tests_boto3.functional.test_headers.test_object_create_bad_date_empty_aws2 | FAIL | FAIL | ok | +| 35 | s3tests_boto3.functional.test_headers.test_object_create_bad_date_none_aws2 | FAIL | FAIL | FAIL | +| 36 | s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_today_aws2 | FAIL | ok | ok | +| 37 | s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_epoch_aws2 | FAIL | FAIL | ok | +| 38 | s3tests_boto3.functional.test_headers.test_object_create_bad_date_after_end_aws2 | FAIL | ok | ok | +| 39 | s3tests_boto3.functional.test_s3.test_object_anon_put | ok | ok | ok | +| 40 | s3tests_boto3.functional.test_s3.test_object_put_authenticated | ok | ok | ok | +| 41 | s3tests_boto3.functional.test_s3.test_object_raw_put_authenticated_expired | ok | FAIL | FAIL | +| 42 | s3tests_boto3.functional.test_s3.test_object_write_file | ok | ok | ok | +| 43 | s3tests_boto3.functional.test_s3.test_object_write_check_etag | FAIL | ok | ok | +| 44 | s3tests_boto3.functional.test_s3.test_object_write_cache_control | ok | ok | ok | +| 45 | s3tests_boto3.functional.test_s3.test_object_write_expires | ok | ok | ok | +| 46 | s3tests_boto3.functional.test_s3.test_object_write_read_update_read_delete | ok | ok | ok | +| 47 | s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_good | ok | ok | ok | +| 48 | s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_empty | ERROR | ok | ok | +| 49 | s3tests_boto3.functional.test_s3.test_object_set_get_metadata_overwrite_to_empty | ERROR | ok | ok | +| 50 | s3tests_boto3.functional.test_s3.test_object_set_get_non_utf8_metadata | ok | ok | FAIL | +| 51 | s3tests_boto3.functional.test_s3.test_object_set_get_metadata_empty_to_unreadable_prefix | ok | ok | FAIL | +| 52 | s3tests_boto3.functional.test_s3.test_object_set_get_metadata_empty_to_unreadable_suffix | ok | ok | FAIL | +| 53 | s3tests_boto3.functional.test_s3.test_object_set_get_metadata_empty_to_unreadable_infix | ok | ok | FAIL | +| 54 | s3tests_boto3.functional.test_s3.test_object_metadata_replaced_on_put | ok | ok | ok | +| 55 | s3tests_boto3.functional.test_s3.test_object_write_to_nonexist_bucket | ok | ok | ok | +| 56 | s3tests_boto3.functional.test_s3.test_atomic_write_1mb | ok | ok | ok | +| 57 | s3tests_boto3.functional.test_s3.test_atomic_write_4mb | ok | ok | ok | +| 58 | s3tests_boto3.functional.test_s3.test_atomic_write_8mb | ok | ok | ok | +| 59 | s3tests_boto3.functional.test_s3.test_atomic_dual_write_1mb | ok | ok | ERROR | +| 60 | s3tests_boto3.functional.test_s3.test_atomic_dual_write_4mb | ok | ok | ERROR | +| 61 | s3tests_boto3.functional.test_s3.test_atomic_dual_write_8mb | ok | ok | ERROR | +| 62 | s3tests_boto3.functional.test_s3.test_atomic_conditional_write_1mb | ok | ok | ERROR | +| 63 | s3tests_boto3.functional.test_s3.test_atomic_dual_conditional_write_1mb | FAIL | FAIL | FAIL | +| 64 | s3tests_boto3.functional.test_s3.test_atomic_write_bucket_gone | ok | ok | ok | ## PostObject @@ -342,30 +342,30 @@ Compatibility: 4/5/29 out of 29 Compatibility: 19/15/19 out of 22 -| | Test | s3-gw | minio | aws s3 | -|----|----------------------------------------------------------------------------------|-------|-------|--------| -| 1 | s3tests_boto3.functional.test_s3.test_multipart_upload_empty | ok | FAIL | FAIL | -| 2 | s3tests_boto3.functional.test_s3.test_multipart_upload_small | ERROR | ERROR | ok | -| 3 | s3tests_boto3.functional.test_s3.test_multipart_copy_small | ok | ok | ok | -| 4 | s3tests_boto3.functional.test_s3.test_multipart_copy_invalid_range | ok | FAIL | FAIL | -| 5 | s3tests_boto3.functional.test_s3.test_multipart_copy_improper_range | ok | ok | ok | -| 6 | s3tests_boto3.functional.test_s3.test_multipart_copy_without_range | ok | ok | ok | -| 7 | s3tests_boto3.functional.test_s3.test_multipart_copy_special_names | ok | ok | ok | -| 8 | s3tests_boto3.functional.test_s3.test_multipart_upload | ERROR | ERROR | ok | -| 9 | s3tests_boto3.functional.test_s3.test_multipart_upload_resend_part | ok | ok | ok | -| 10 | s3tests_boto3.functional.test_s3.test_multipart_upload_multiple_sizes | ok | ok | ok | -| 11 | s3tests_boto3.functional.test_s3.test_multipart_copy_multiple_sizes | ok | ok | ok | -| 12 | s3tests_boto3.functional.test_s3.test_multipart_upload_size_too_small | ok | ok | ok | -| 13 | s3tests_boto3.functional.test_s3.test_multipart_upload_contents | ok | ok | ok | -| 14 | s3tests_boto3.functional.test_s3.test_multipart_upload_overwrite_existing_object | ok | ok | ok | -| 15 | s3tests_boto3.functional.test_s3.test_abort_multipart_upload | ok | ok | ok | -| 16 | s3tests_boto3.functional.test_s3.test_abort_multipart_upload_not_found | ok | ok | ok | -| 17 | s3tests_boto3.functional.test_s3.test_list_multipart_upload | ok | ERROR | ok | -| 18 | s3tests_boto3.functional.test_s3.test_multipart_upload_missing_part | ok | ok | ok | -| 19 | s3tests_boto3.functional.test_s3.test_multipart_upload_incorrect_etag | ok | ok | ok | -| 20 | s3tests_boto3.functional.test_s3.test_multipart_resend_first_finishes_last | ERROR | ERROR | ERROR | -| 21 | s3tests_boto3.functional.test_s3.test_atomic_multipart_upload_write | ok | ok | ok | -| 22 | s3tests_boto3.functional.test_s3.test_multipart_copy_versioned | ok | ERROR | ok | +| | Test | s3-gw | minio | aws s3 | +|-----|----------------------------------------------------------------------------------|-------|-------|--------| +| 1 | s3tests_boto3.functional.test_s3.test_multipart_upload_empty | ok | FAIL | FAIL | +| 2 | s3tests_boto3.functional.test_s3.test_multipart_upload_small | ERROR | ERROR | ok | +| 3 | s3tests_boto3.functional.test_s3.test_multipart_copy_small | ok | ok | ok | +| 4 | s3tests_boto3.functional.test_s3.test_multipart_copy_invalid_range | ok | FAIL | FAIL | +| 5 | s3tests_boto3.functional.test_s3.test_multipart_copy_improper_range | ok | ok | ok | +| 6 | s3tests_boto3.functional.test_s3.test_multipart_copy_without_range | ok | ok | ok | +| 7 | s3tests_boto3.functional.test_s3.test_multipart_copy_special_names | ok | ok | ok | +| 8 | s3tests_boto3.functional.test_s3.test_multipart_upload | ERROR | ERROR | ok | +| 9 | s3tests_boto3.functional.test_s3.test_multipart_upload_resend_part | ok | ok | ok | +| 10 | s3tests_boto3.functional.test_s3.test_multipart_upload_multiple_sizes | ok | ok | ok | +| 11 | s3tests_boto3.functional.test_s3.test_multipart_copy_multiple_sizes | ok | ok | ok | +| 12 | s3tests_boto3.functional.test_s3.test_multipart_upload_size_too_small | ok | ok | ok | +| 13 | s3tests_boto3.functional.test_s3.test_multipart_upload_contents | ok | ok | ok | +| 14 | s3tests_boto3.functional.test_s3.test_multipart_upload_overwrite_existing_object | ok | ok | ok | +| 15 | s3tests_boto3.functional.test_s3.test_abort_multipart_upload | ok | ok | ok | +| 16 | s3tests_boto3.functional.test_s3.test_abort_multipart_upload_not_found | ok | ok | ok | +| 17 | s3tests_boto3.functional.test_s3.test_list_multipart_upload | ok | ERROR | ok | +| 18 | s3tests_boto3.functional.test_s3.test_multipart_upload_missing_part | ok | ok | ok | +| 19 | s3tests_boto3.functional.test_s3.test_multipart_upload_incorrect_etag | ok | ok | ok | +| 20 | s3tests_boto3.functional.test_s3.test_multipart_resend_first_finishes_last | ERROR | ERROR | ERROR | +| 21 | s3tests_boto3.functional.test_s3.test_atomic_multipart_upload_write | ok | ok | ok | +| 22 | s3tests_boto3.functional.test_s3.test_multipart_copy_versioned | ok | ERROR | ok | Comments: in [PR](https://github.com/nspcc-dev/s3-tests/pull/5) @@ -373,118 +373,118 @@ Comments: in [PR](https://github.com/nspcc-dev/s3-tests/pull/5) Compatibility: 9/6/8 out of 11 -| | Test | s3-gw | minio | aws s3 | -|----|------------------------------------------------------------|-------|-------|--------| -| 1 | s3tests_boto3.functional.test_s3.test_set_bucket_tagging | FAIL | FAIL | FAIL | -| 2 | s3tests_boto3.functional.test_s3.test_get_obj_tagging | ok | ok | ok | -| 3 | s3tests_boto3.functional.test_s3.test_get_obj_head_tagging | ok | ok | ok | -| 4 | s3tests_boto3.functional.test_s3.test_put_max_tags | ok | FAIL | ok | -| 5 | s3tests_boto3.functional.test_s3.test_put_excess_tags | FAIL | FAIL | FAIL | -| 6 | s3tests_boto3.functional.test_s3.test_put_max_kvsize_tags | ok | ok | ok | -| 7 | s3tests_boto3.functional.test_s3.test_put_excess_key_tags | ok | ok | ok | -| 8 | s3tests_boto3.functional.test_s3.test_put_excess_val_tags | ok | ok | ok | -| 9 | s3tests_boto3.functional.test_s3.test_put_modify_tags | ok | FAIL | FAIL | -| 10 | s3tests_boto3.functional.test_s3.test_put_delete_tags | ok | ok | ok | -| 11 | s3tests_boto3.functional.test_s3.test_put_obj_with_tags | ok | FAIL | ok | +| | Test | s3-gw | minio | aws s3 | +|-----|------------------------------------------------------------|-------|-------|--------| +| 1 | s3tests_boto3.functional.test_s3.test_set_bucket_tagging | FAIL | FAIL | FAIL | +| 2 | s3tests_boto3.functional.test_s3.test_get_obj_tagging | ok | ok | ok | +| 3 | s3tests_boto3.functional.test_s3.test_get_obj_head_tagging | ok | ok | ok | +| 4 | s3tests_boto3.functional.test_s3.test_put_max_tags | ok | FAIL | ok | +| 5 | s3tests_boto3.functional.test_s3.test_put_excess_tags | FAIL | FAIL | FAIL | +| 6 | s3tests_boto3.functional.test_s3.test_put_max_kvsize_tags | ok | ok | ok | +| 7 | s3tests_boto3.functional.test_s3.test_put_excess_key_tags | ok | ok | ok | +| 8 | s3tests_boto3.functional.test_s3.test_put_excess_val_tags | ok | ok | ok | +| 9 | s3tests_boto3.functional.test_s3.test_put_modify_tags | ok | FAIL | FAIL | +| 10 | s3tests_boto3.functional.test_s3.test_put_delete_tags | ok | ok | ok | +| 11 | s3tests_boto3.functional.test_s3.test_put_obj_with_tags | ok | FAIL | ok | ## Versioning Compatibility: 23/19/24 out of 26 -| | Test | s3-gw | minio | aws s3 | -|----|---------------------------------------------------------------------------------------------|-------|-------|--------| -| 1 | s3tests_boto3.functional.test_s3.test_versioning_bucket_create_suspend | ok | ok | ok | -| 2 | s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove | ok | ok | ok | -| 3 | s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove_head | ok | ok | ok | -| 4 | s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_removal | ok | ok | ok | -| 5 | s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite | ok | ok | ok | -| 6 | s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite_suspended | ok | ok | ok | -| 7 | s3tests_boto3.functional.test_s3.test_versioning_obj_suspend_versions | ok | ok | ok | -| 8 | s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_all | ok | ok | ok | -| 9 | s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_special_names | ok | ok | ok | -| 10 | s3tests_boto3.functional.test_s3.test_versioning_obj_create_overwrite_multipart | ok | ok | ok | -| 11 | s3tests_boto3.functional.test_s3.test_versioning_obj_list_marker | ok | ok | ok | -| 12 | s3tests_boto3.functional.test_s3.test_versioning_copy_obj_version | ok | ok | ok | -| 13 | s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete | ok | ok | ok | -| 14 | s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker | ok | ok | ok | -| 15 | s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker_create | ok | ERROR | ok | -| 16 | s3tests_boto3.functional.test_s3.test_versioned_object_acl | FAIL | FAIL | FAIL | -| 17 | s3tests_boto3.functional.test_s3.test_versioned_object_acl_no_version_specified | FAIL | FAIL | FAIL | -| 18 | s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_concurrent_remove | ok | ok | ok | -| 19 | s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_and_remove | ok | ok | ok | -| 20 | s3tests_boto3.functional.test_s3.test_versioning_bucket_atomic_upload_return_version_id | ok | FAIL | ok | -| 21 | s3tests_boto3.functional.test_s3.test_versioning_bucket_multipart_upload_return_version_id | ok | FAIL | ok | -| 22 | s3tests_boto3.functional.test_s3.test_bucket_list_return_data_versioning | ERROR | ERROR | ok | -| 23 | s3tests_boto3.functional.test_s3.test_object_copy_versioned_bucket | ok | ok | ok | -| 24 | s3tests_boto3.functional.test_s3.test_object_copy_versioned_url_encoding | ok | ok | ok | -| 25 | s3tests_boto3.functional.test_s3.test_object_copy_versioning_multipart_upload | ok | ok | ok | -| 26 | s3tests_boto3.functional.test_s3.test_multipart_copy_versioned | ok | ERROR | ok | +| | Test | s3-gw | minio | aws s3 | +|-----|---------------------------------------------------------------------------------------------|-------|-------|--------| +| 1 | s3tests_boto3.functional.test_s3.test_versioning_bucket_create_suspend | ok | ok | ok | +| 2 | s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove | ok | ok | ok | +| 3 | s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove_head | ok | ok | ok | +| 4 | s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_removal | ok | ok | ok | +| 5 | s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite | ok | ok | ok | +| 6 | s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite_suspended | ok | ok | ok | +| 7 | s3tests_boto3.functional.test_s3.test_versioning_obj_suspend_versions | ok | ok | ok | +| 8 | s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_all | ok | ok | ok | +| 9 | s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_special_names | ok | ok | ok | +| 10 | s3tests_boto3.functional.test_s3.test_versioning_obj_create_overwrite_multipart | ok | ok | ok | +| 11 | s3tests_boto3.functional.test_s3.test_versioning_obj_list_marker | ok | ok | ok | +| 12 | s3tests_boto3.functional.test_s3.test_versioning_copy_obj_version | ok | ok | ok | +| 13 | s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete | ok | ok | ok | +| 14 | s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker | ok | ok | ok | +| 15 | s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker_create | ok | ERROR | ok | +| 16 | s3tests_boto3.functional.test_s3.test_versioned_object_acl | FAIL | FAIL | FAIL | +| 17 | s3tests_boto3.functional.test_s3.test_versioned_object_acl_no_version_specified | FAIL | FAIL | FAIL | +| 18 | s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_concurrent_remove | ok | ok | ok | +| 19 | s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_and_remove | ok | ok | ok | +| 20 | s3tests_boto3.functional.test_s3.test_versioning_bucket_atomic_upload_return_version_id | ok | FAIL | ok | +| 21 | s3tests_boto3.functional.test_s3.test_versioning_bucket_multipart_upload_return_version_id | ok | FAIL | ok | +| 22 | s3tests_boto3.functional.test_s3.test_bucket_list_return_data_versioning | ERROR | ERROR | ok | +| 23 | s3tests_boto3.functional.test_s3.test_object_copy_versioned_bucket | ok | ok | ok | +| 24 | s3tests_boto3.functional.test_s3.test_object_copy_versioned_url_encoding | ok | ok | ok | +| 25 | s3tests_boto3.functional.test_s3.test_object_copy_versioning_multipart_upload | ok | ok | ok | +| 26 | s3tests_boto3.functional.test_s3.test_multipart_copy_versioned | ok | ERROR | ok | ## Bucket Compatibility: 38/38/45 out of 59 -| | Test | s3-gw | minio | aws s3 | -|----|----------------------------------------------------------------------------------------------|-------|-------|--------| -| 1 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_authorization_invalid_aws2 | FAIL | FAIL | FAIL | -| 2 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_empty_aws2 | ERROR | ok | ok | -| 3 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_none_aws2 | ERROR | ok | ok | -| 4 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_invalid_aws2 | FAIL | FAIL | ok | -| 5 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_empty_aws2 | FAIL | FAIL | ok | -| 6 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_none_aws2 | FAIL | FAIL | FAIL | -| 7 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_today_aws2 | FAIL | ok | ok | -| 8 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_after_today_aws2 | FAIL | ok | ok | -| 9 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_epoch_aws2 | FAIL | FAIL | ok | -| 10 | s3tests_boto3.functional.test_headers.test_bucket_create_contentlength_none | ok | ok | ok | -| 11 | s3tests_boto3.functional.test_headers.test_bucket_put_bad_canned_acl | ok | FAIL | ok | -| 12 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_mismatch | ERROR | ERROR | ok | -| 13 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_empty | ok | ok | ok | -| 14 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_contentlength_empty | FAIL | FAIL | ok | -| 15 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_contentlength_negative | ok | ok | ok | -| 16 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_contentlength_none | ok | ok | ok | -| 17 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_authorization_empty | FAIL | FAIL | FAIL | -| 18 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_authorization_none | FAIL | FAIL | FAIL | -| 19 | s3tests_boto3.functional.test_s3.test_bucket_notexist | ok | ok | ok | -| 20 | s3tests_boto3.functional.test_s3.test_bucketv2_notexist | ok | ok | ok | -| 21 | s3tests_boto3.functional.test_s3.test_bucket_delete_notexist | ok | ok | ok | -| 22 | s3tests_boto3.functional.test_s3.test_bucket_delete_nonempty | ok | ok | ok | -| 23 | s3tests_boto3.functional.test_s3.test_bucket_concurrent_set_canned_acl | ok | FAIL | FAIL | -| 24 | s3tests_boto3.functional.test_s3.test_bucket_create_delete | ok | ok | ok | -| 25 | s3tests_boto3.functional.test_s3.test_bucket_head | ok | ok | ok | -| 26 | s3tests_boto3.functional.test_s3.test_bucket_head_notexist | ok | ok | ok | -| 27 | s3tests_boto3.functional.test_s3.test_bucket_head_extended | ERROR | ERROR | ERROR | -| 28 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_starts_nonalpha | ok | ok | ok | -| 29 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_empty | ERROR | ERROR | ERROR | -| 30 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_one | ok | ok | ok | -| 31 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_two | ok | ok | ok | -| 32 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_long | ERROR | ERROR | ERROR | -| 33 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_long_60 | ok | ok | ok | -| 34 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_long_61 | ok | ok | ok | -| 35 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_long_62 | ok | ok | ok | -| 36 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_long_63 | ok | ok | ok | -| 37 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_ip | ok | ok | FAIL | -| 38 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_punctuation | ERROR | ERROR | ERROR | -| 39 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_underscore | ok | ok | ok | -| 40 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_long | ok | ok | ok | -| 41 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_dash_at_end | ok | ok | ok | -| 42 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_dot_dot | ok | ok | ok | -| 43 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_dot_dash | ok | ok | ok | -| 44 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_dash_dot | ok | ok | ok | -| 45 | s3tests_boto3.functional.test_s3.test_bucket_create_exists | ERROR | ERROR | ok | -| 46 | s3tests_boto3.functional.test_s3.test_bucket_get_location | ok | FAIL | ERROR | -| 47 | s3tests_boto3.functional.test_s3.test_bucket_create_exists_nonowner | ok | FAIL | ok | -| 48 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_alpha | ok | ok | ok | -| 49 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_digit | ok | ok | ok | -| 50 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_period | ERROR | ok | ok | -| 51 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_hyphen | ok | ok | ok | -| 52 | s3tests_boto3.functional.test_s3.test_bucket_recreate_not_overriding | ERROR | ERROR | ok | -| 53 | s3tests_boto3.functional.test_s3.test_bucket_create_special_key_names | ok | ok | ok | -| 54 | s3tests_boto3.functional.test_s3.test_bucket_policy_set_condition_operator_end_with_IfExists | ERROR | ERROR | FAIL | -| 55 | s3tests_boto3.functional.test_s3.test_buckets_create_then_list | ok | ok | ok | -| 56 | s3tests_boto3.functional.test_s3.test_buckets_list_ctime | ok | ok | FAIL | -| 57 | s3tests_boto3.functional.test_s3.test_list_buckets_anonymous | ok | ERROR | ERROR | -| 58 | s3tests_boto3.functional.test_s3.test_list_buckets_invalid_auth | ok | ok | ok | -| 59 | s3tests_boto3.functional.test_s3.test_list_buckets_bad_auth | ok | ok | ok | +| | Test | s3-gw | minio | aws s3 | +|-----|----------------------------------------------------------------------------------------------|-------|-------|--------| +| 1 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_authorization_invalid_aws2 | FAIL | FAIL | FAIL | +| 2 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_empty_aws2 | ERROR | ok | ok | +| 3 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_none_aws2 | ERROR | ok | ok | +| 4 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_invalid_aws2 | FAIL | FAIL | ok | +| 5 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_empty_aws2 | FAIL | FAIL | ok | +| 6 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_none_aws2 | FAIL | FAIL | FAIL | +| 7 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_today_aws2 | FAIL | ok | ok | +| 8 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_after_today_aws2 | FAIL | ok | ok | +| 9 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_epoch_aws2 | FAIL | FAIL | ok | +| 10 | s3tests_boto3.functional.test_headers.test_bucket_create_contentlength_none | ok | ok | ok | +| 11 | s3tests_boto3.functional.test_headers.test_bucket_put_bad_canned_acl | ok | FAIL | ok | +| 12 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_mismatch | ERROR | ERROR | ok | +| 13 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_empty | ok | ok | ok | +| 14 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_contentlength_empty | FAIL | FAIL | ok | +| 15 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_contentlength_negative | ok | ok | ok | +| 16 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_contentlength_none | ok | ok | ok | +| 17 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_authorization_empty | FAIL | FAIL | FAIL | +| 18 | s3tests_boto3.functional.test_headers.test_bucket_create_bad_authorization_none | FAIL | FAIL | FAIL | +| 19 | s3tests_boto3.functional.test_s3.test_bucket_notexist | ok | ok | ok | +| 20 | s3tests_boto3.functional.test_s3.test_bucketv2_notexist | ok | ok | ok | +| 21 | s3tests_boto3.functional.test_s3.test_bucket_delete_notexist | ok | ok | ok | +| 22 | s3tests_boto3.functional.test_s3.test_bucket_delete_nonempty | ok | ok | ok | +| 23 | s3tests_boto3.functional.test_s3.test_bucket_concurrent_set_canned_acl | ok | FAIL | FAIL | +| 24 | s3tests_boto3.functional.test_s3.test_bucket_create_delete | ok | ok | ok | +| 25 | s3tests_boto3.functional.test_s3.test_bucket_head | ok | ok | ok | +| 26 | s3tests_boto3.functional.test_s3.test_bucket_head_notexist | ok | ok | ok | +| 27 | s3tests_boto3.functional.test_s3.test_bucket_head_extended | ERROR | ERROR | ERROR | +| 28 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_starts_nonalpha | ok | ok | ok | +| 29 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_empty | ERROR | ERROR | ERROR | +| 30 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_one | ok | ok | ok | +| 31 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_two | ok | ok | ok | +| 32 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_long | ERROR | ERROR | ERROR | +| 33 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_long_60 | ok | ok | ok | +| 34 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_long_61 | ok | ok | ok | +| 35 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_long_62 | ok | ok | ok | +| 36 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_long_63 | ok | ok | ok | +| 37 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_ip | ok | ok | FAIL | +| 38 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_punctuation | ERROR | ERROR | ERROR | +| 39 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_underscore | ok | ok | ok | +| 40 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_long | ok | ok | ok | +| 41 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_dash_at_end | ok | ok | ok | +| 42 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_dot_dot | ok | ok | ok | +| 43 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_dot_dash | ok | ok | ok | +| 44 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_dash_dot | ok | ok | ok | +| 45 | s3tests_boto3.functional.test_s3.test_bucket_create_exists | ERROR | ERROR | ok | +| 46 | s3tests_boto3.functional.test_s3.test_bucket_get_location | ok | FAIL | ERROR | +| 47 | s3tests_boto3.functional.test_s3.test_bucket_create_exists_nonowner | ok | FAIL | ok | +| 48 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_alpha | ok | ok | ok | +| 49 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_digit | ok | ok | ok | +| 50 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_period | ERROR | ok | ok | +| 51 | s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_hyphen | ok | ok | ok | +| 52 | s3tests_boto3.functional.test_s3.test_bucket_recreate_not_overriding | ERROR | ERROR | ok | +| 53 | s3tests_boto3.functional.test_s3.test_bucket_create_special_key_names | ok | ok | ok | +| 54 | s3tests_boto3.functional.test_s3.test_bucket_policy_set_condition_operator_end_with_IfExists | ERROR | ERROR | FAIL | +| 55 | s3tests_boto3.functional.test_s3.test_buckets_create_then_list | ok | ok | ok | +| 56 | s3tests_boto3.functional.test_s3.test_buckets_list_ctime | ok | ok | FAIL | +| 57 | s3tests_boto3.functional.test_s3.test_list_buckets_anonymous | ok | ERROR | ERROR | +| 58 | s3tests_boto3.functional.test_s3.test_list_buckets_invalid_auth | ok | ok | ok | +| 59 | s3tests_boto3.functional.test_s3.test_list_buckets_bad_auth | ok | ok | ok | ## Bucket ACL @@ -581,90 +581,90 @@ This group is not explicitly supported by s3-gw, but some other tests may pass. Compatibility: 0/10/18 out of 29 This group is not explicitly supported by s3-gw, but some tests may pass. -| | Test | s3-gw | minio | aws s3 | -|----|---------------------------------------------------------------------------------|-------|-------|--------| -| 1 | s3tests_boto3.functional.test_s3.test_lifecycle_set | ERROR | ok | ok | -| 2 | s3tests_boto3.functional.test_s3.test_lifecycle_get | ERROR | FAIL | ok | -| 3 | s3tests_boto3.functional.test_s3.test_lifecycle_get_no_id | ERROR | ERROR | ok | -| 4 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration | ERROR | FAIL | FAIL | -| 5 | s3tests_boto3.functional.test_s3.test_lifecyclev2_expiration | ERROR | FAIL | FAIL | -| 6 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_versioning_enabled | ERROR | ERROR | ok | -| 7 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_tags1 | ERROR | ERROR | ERROR | -| 8 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_tags2 | ERROR | ERROR | ERROR | -| 9 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_versioned_tags2 | ERROR | ERROR | ERROR | -| 10 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_noncur_tags1 | ERROR | ERROR | ERROR | -| 11 | s3tests_boto3.functional.test_s3.test_lifecycle_id_too_long | FAIL | FAIL | ok | -| 12 | s3tests_boto3.functional.test_s3.test_lifecycle_same_id | FAIL | FAIL | ok | -| 13 | s3tests_boto3.functional.test_s3.test_lifecycle_invalid_status | FAIL | FAIL | ok | -| 14 | s3tests_boto3.functional.test_s3.test_lifecycle_set_date | ERROR | ok | ok | -| 15 | s3tests_boto3.functional.test_s3.test_lifecycle_set_invalid_date | FAIL | ok | ok | -| 16 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_date | ERROR | FAIL | FAIL | -| 17 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_days0 | FAIL | FAIL | ok | -| 18 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_header_put | ERROR | ok | ok | -| 19 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_header_head | ERROR | ok | ok | -| 20 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_header_tags_head | ERROR | ok | FAIL | -| 21 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_header_and_tags_head | ERROR | ERROR | ok | -| 22 | s3tests_boto3.functional.test_s3.test_lifecycle_set_noncurrent | ERROR | ok | ok | -| 23 | s3tests_boto3.functional.test_s3.test_lifecycle_noncur_expiration | ERROR | ERROR | FAIL | -| 24 | s3tests_boto3.functional.test_s3.test_lifecycle_set_deletemarker | ERROR | ok | ok | -| 25 | s3tests_boto3.functional.test_s3.test_lifecycle_set_filter | ERROR | ok | ok | -| 26 | s3tests_boto3.functional.test_s3.test_lifecycle_set_empty_filter | ERROR | ok | ok | -| 27 | s3tests_boto3.functional.test_s3.test_lifecycle_deletemarker_expiration | ERROR | ERROR | FAIL | -| 28 | s3tests_boto3.functional.test_s3.test_lifecycle_set_multipart | ERROR | ERROR | ok | -| 29 | s3tests_boto3.functional.test_s3.test_lifecycle_multipart_expiration | ERROR | ERROR | FAIL | +| | Test | s3-gw | minio | aws s3 | +|-----|---------------------------------------------------------------------------------|-------|-------|--------| +| 1 | s3tests_boto3.functional.test_s3.test_lifecycle_set | ERROR | ok | ok | +| 2 | s3tests_boto3.functional.test_s3.test_lifecycle_get | ERROR | FAIL | ok | +| 3 | s3tests_boto3.functional.test_s3.test_lifecycle_get_no_id | ERROR | ERROR | ok | +| 4 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration | ERROR | FAIL | FAIL | +| 5 | s3tests_boto3.functional.test_s3.test_lifecyclev2_expiration | ERROR | FAIL | FAIL | +| 6 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_versioning_enabled | ERROR | ERROR | ok | +| 7 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_tags1 | ERROR | ERROR | ERROR | +| 8 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_tags2 | ERROR | ERROR | ERROR | +| 9 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_versioned_tags2 | ERROR | ERROR | ERROR | +| 10 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_noncur_tags1 | ERROR | ERROR | ERROR | +| 11 | s3tests_boto3.functional.test_s3.test_lifecycle_id_too_long | FAIL | FAIL | ok | +| 12 | s3tests_boto3.functional.test_s3.test_lifecycle_same_id | FAIL | FAIL | ok | +| 13 | s3tests_boto3.functional.test_s3.test_lifecycle_invalid_status | FAIL | FAIL | ok | +| 14 | s3tests_boto3.functional.test_s3.test_lifecycle_set_date | ERROR | ok | ok | +| 15 | s3tests_boto3.functional.test_s3.test_lifecycle_set_invalid_date | FAIL | ok | ok | +| 16 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_date | ERROR | FAIL | FAIL | +| 17 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_days0 | FAIL | FAIL | ok | +| 18 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_header_put | ERROR | ok | ok | +| 19 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_header_head | ERROR | ok | ok | +| 20 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_header_tags_head | ERROR | ok | FAIL | +| 21 | s3tests_boto3.functional.test_s3.test_lifecycle_expiration_header_and_tags_head | ERROR | ERROR | ok | +| 22 | s3tests_boto3.functional.test_s3.test_lifecycle_set_noncurrent | ERROR | ok | ok | +| 23 | s3tests_boto3.functional.test_s3.test_lifecycle_noncur_expiration | ERROR | ERROR | FAIL | +| 24 | s3tests_boto3.functional.test_s3.test_lifecycle_set_deletemarker | ERROR | ok | ok | +| 25 | s3tests_boto3.functional.test_s3.test_lifecycle_set_filter | ERROR | ok | ok | +| 26 | s3tests_boto3.functional.test_s3.test_lifecycle_set_empty_filter | ERROR | ok | ok | +| 27 | s3tests_boto3.functional.test_s3.test_lifecycle_deletemarker_expiration | ERROR | ERROR | FAIL | +| 28 | s3tests_boto3.functional.test_s3.test_lifecycle_set_multipart | ERROR | ERROR | ok | +| 29 | s3tests_boto3.functional.test_s3.test_lifecycle_multipart_expiration | ERROR | ERROR | FAIL | ## Policy and replication Compatibility: 0/7/20 out of 35 This group is not explicitly supported by s3-gw, but some tests may pass. -| | Test | s3-gw | minio | aws s3 | -|----|-------------------------------------------------------------------------------------|-------|-------|--------| -| 1 | s3tests_boto3.functional.test_s3.test_bucket_policy | ERROR | ok | ok | -| 2 | s3tests_boto3.functional.test_s3.test_bucketv2_policy | ERROR | ok | ok | -| 3 | s3tests_boto3.functional.test_s3.test_bucket_policy_acl | ERROR | ERROR | ok | -| 4 | s3tests_boto3.functional.test_s3.test_bucketv2_policy_acl | ERROR | ERROR | ok | -| 5 | s3tests_boto3.functional.test_s3.test_bucket_policy_different_tenant | ERROR | ERROR | ERROR | -| 6 | s3tests_boto3.functional.test_s3.test_bucketv2_policy_different_tenant | ERROR | ERROR | ERROR | -| 7 | s3tests_boto3.functional.test_s3.test_bucket_policy_another_bucket | ERROR | ok | ERROR | -| 8 | s3tests_boto3.functional.test_s3.test_bucketv2_policy_another_bucket | ERROR | ok | ERROR | -| 9 | s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_existing_tag | ERROR | ERROR | ok | -| 10 | s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_tagging_existing_tag | ERROR | ERROR | ok | -| 11 | s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_tagging_existing_tag | ERROR | ERROR | ok | -| 12 | s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source | ERROR | FAIL | ok | -| 13 | s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source_meta | ERROR | FAIL | ok | -| 14 | s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_acl | ERROR | ERROR | ok | -| 15 | s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_grant | ERROR | ERROR | ok | -| 16 | s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_enc | ERROR | FAIL | ERROR | -| 17 | s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_request_obj_tag | ERROR | ERROR | FAIL | -| 18 | s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_acl_existing_tag | ERROR | ERROR | ok | -| 19 | s3tests_boto3.functional.test_s3.test_user_policy | ERROR | ERROR | ERROR | -| 20 | s3tests_boto3.functional.test_s3.test_get_bucket_policy_status | ERROR | ok | ERROR | -| 21 | s3tests_boto3.functional.test_s3.test_get_public_acl_bucket_policy_status | ERROR | ERROR | ERROR | -| 22 | s3tests_boto3.functional.test_s3.test_get_authpublic_acl_bucket_policy_status | ERROR | ERROR | ERROR | -| 23 | s3tests_boto3.functional.test_s3.test_get_publicpolicy_acl_bucket_policy_status | ERROR | FAIL | ERROR | -| 24 | s3tests_boto3.functional.test_s3.test_get_nonpublicpolicy_acl_bucket_policy_status | ERROR | ok | ERROR | -| 25 | s3tests_boto3.functional.test_s3.test_get_nonpublicpolicy_deny_bucket_policy_status | ERROR | ERROR | ERROR | -| 26 | s3tests_boto3.functional.test_s3.test_get_default_public_block | ERROR | ERROR | ERROR | -| 27 | s3tests_boto3.functional.test_s3.test_put_public_block | ERROR | ERROR | ok | -| 28 | s3tests_boto3.functional.test_s3.test_block_public_put_bucket_acls | ERROR | ERROR | ok | -| 29 | s3tests_boto3.functional.test_s3.test_block_public_object_canned_acls | ERROR | ERROR | ok | -| 30 | s3tests_boto3.functional.test_s3.test_block_public_policy | ERROR | ERROR | ok | -| 31 | s3tests_boto3.functional.test_s3.test_ignore_public_acls | ERROR | ERROR | FAIL | -| 32 | s3tests_boto3.functional.test_s3.test_get_tags_acl_public | ERROR | FAIL | ok | -| 33 | s3tests_boto3.functional.test_s3.test_put_tags_acl_public | ERROR | FAIL | ok | -| 34 | s3tests_boto3.functional.test_s3.test_delete_tags_obj_public | ERROR | ok | ok | -| 35 | s3tests_boto3.functional.test_s3.test_multipart_upload_on_a_bucket_with_policy | ERROR | ERROR | ok | +| | Test | s3-gw | minio | aws s3 | +|-----|-------------------------------------------------------------------------------------|-------|-------|--------| +| 1 | s3tests_boto3.functional.test_s3.test_bucket_policy | ERROR | ok | ok | +| 2 | s3tests_boto3.functional.test_s3.test_bucketv2_policy | ERROR | ok | ok | +| 3 | s3tests_boto3.functional.test_s3.test_bucket_policy_acl | ERROR | ERROR | ok | +| 4 | s3tests_boto3.functional.test_s3.test_bucketv2_policy_acl | ERROR | ERROR | ok | +| 5 | s3tests_boto3.functional.test_s3.test_bucket_policy_different_tenant | ERROR | ERROR | ERROR | +| 6 | s3tests_boto3.functional.test_s3.test_bucketv2_policy_different_tenant | ERROR | ERROR | ERROR | +| 7 | s3tests_boto3.functional.test_s3.test_bucket_policy_another_bucket | ERROR | ok | ERROR | +| 8 | s3tests_boto3.functional.test_s3.test_bucketv2_policy_another_bucket | ERROR | ok | ERROR | +| 9 | s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_existing_tag | ERROR | ERROR | ok | +| 10 | s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_tagging_existing_tag | ERROR | ERROR | ok | +| 11 | s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_tagging_existing_tag | ERROR | ERROR | ok | +| 12 | s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source | ERROR | FAIL | ok | +| 13 | s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source_meta | ERROR | FAIL | ok | +| 14 | s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_acl | ERROR | ERROR | ok | +| 15 | s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_grant | ERROR | ERROR | ok | +| 16 | s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_enc | ERROR | FAIL | ERROR | +| 17 | s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_request_obj_tag | ERROR | ERROR | FAIL | +| 18 | s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_acl_existing_tag | ERROR | ERROR | ok | +| 19 | s3tests_boto3.functional.test_s3.test_user_policy | ERROR | ERROR | ERROR | +| 20 | s3tests_boto3.functional.test_s3.test_get_bucket_policy_status | ERROR | ok | ERROR | +| 21 | s3tests_boto3.functional.test_s3.test_get_public_acl_bucket_policy_status | ERROR | ERROR | ERROR | +| 22 | s3tests_boto3.functional.test_s3.test_get_authpublic_acl_bucket_policy_status | ERROR | ERROR | ERROR | +| 23 | s3tests_boto3.functional.test_s3.test_get_publicpolicy_acl_bucket_policy_status | ERROR | FAIL | ERROR | +| 24 | s3tests_boto3.functional.test_s3.test_get_nonpublicpolicy_acl_bucket_policy_status | ERROR | ok | ERROR | +| 25 | s3tests_boto3.functional.test_s3.test_get_nonpublicpolicy_deny_bucket_policy_status | ERROR | ERROR | ERROR | +| 26 | s3tests_boto3.functional.test_s3.test_get_default_public_block | ERROR | ERROR | ERROR | +| 27 | s3tests_boto3.functional.test_s3.test_put_public_block | ERROR | ERROR | ok | +| 28 | s3tests_boto3.functional.test_s3.test_block_public_put_bucket_acls | ERROR | ERROR | ok | +| 29 | s3tests_boto3.functional.test_s3.test_block_public_object_canned_acls | ERROR | ERROR | ok | +| 30 | s3tests_boto3.functional.test_s3.test_block_public_policy | ERROR | ERROR | ok | +| 31 | s3tests_boto3.functional.test_s3.test_ignore_public_acls | ERROR | ERROR | FAIL | +| 32 | s3tests_boto3.functional.test_s3.test_get_tags_acl_public | ERROR | FAIL | ok | +| 33 | s3tests_boto3.functional.test_s3.test_put_tags_acl_public | ERROR | FAIL | ok | +| 34 | s3tests_boto3.functional.test_s3.test_delete_tags_obj_public | ERROR | ok | ok | +| 35 | s3tests_boto3.functional.test_s3.test_multipart_upload_on_a_bucket_with_policy | ERROR | ERROR | ok | ## Others Compatibility: 2/2/3 out of 6 -| | Test | s3-gw | minio | aws s3 | -|---|-------------------------------------------------------------|-------|-------|--------| -| 1 | s3tests_boto3.functional.test_s3.test_100_continue | FAIL | ERROR | ok | -| 2 | s3tests_boto3.functional.test_s3.test_account_usage | ERROR | ERROR | ERROR | -| 3 | s3tests_boto3.functional.test_s3.test_head_bucket_usage | ERROR | ERROR | ERROR | -| 4 | s3tests_boto3.functional.test_s3.test_logging_toggle | ERROR | ERROR | ERROR | -| 5 | s3tests_boto3.functional.test_s3.test_multi_object_delete | ok | ok | ok | -| 6 | s3tests_boto3.functional.test_s3.test_multi_objectv2_delete | ok | ok | ok | +| | Test | s3-gw | minio | aws s3 | +|-----|-------------------------------------------------------------|-------|-------|--------| +| 1 | s3tests_boto3.functional.test_s3.test_100_continue | FAIL | ERROR | ok | +| 2 | s3tests_boto3.functional.test_s3.test_account_usage | ERROR | ERROR | ERROR | +| 3 | s3tests_boto3.functional.test_s3.test_head_bucket_usage | ERROR | ERROR | ERROR | +| 4 | s3tests_boto3.functional.test_s3.test_logging_toggle | ERROR | ERROR | ERROR | +| 5 | s3tests_boto3.functional.test_s3.test_multi_object_delete | ok | ok | ok | +| 6 | s3tests_boto3.functional.test_s3.test_multi_objectv2_delete | ok | ok | ok | diff --git a/docs/tree_service.md b/docs/tree_service.md index f47f8a452..d9b119c64 100644 --- a/docs/tree_service.md +++ b/docs/tree_service.md @@ -1,17 +1,17 @@ # Tree service -To get objects' metadata and system information, the S3 GW makes requests to the Tree service. -This is a service in FrostFS storage that keeps different information as a tree structure. +To get objects' metadata and system information, the S3 GW makes requests to the Tree service. +This is a service in FrostFS storage that keeps different information as a tree structure. Each node keeps one of the types of data as a set of **key-value pairs**: -* Bucket settings: lock configuration and versioning mode +* Bucket settings: lock configuration and versioning mode * Bucket tagging * Object tagging * Object metadata: OID, name, creation time, system metadata * Object locking settings * Active multipart upload info -Some data takes up a lot of memory, so we store it in FrostFS nodes as an object with payload. +Some data takes up a lot of memory, so we store it in FrostFS nodes as an object with payload. But we keep these objects' metadata in the Tree service too: * Notification configuration * CORS diff --git a/updateTestsResult.sh b/updateTestsResult.sh index 4e41c00fc..0ac713603 100755 --- a/updateTestsResult.sh +++ b/updateTestsResult.sh @@ -14,7 +14,7 @@ fi RESULT_FILE=docs/s3_test_results.md -get_adjusted_result () { +get_adjusted_result() { local OLD_RESULT=$1 local NEW_RESULT=$2 local OLD_RESULT_LEN=${#OLD_RESULT} @@ -23,13 +23,11 @@ get_adjusted_result () { printf "%s%*s" "$NEW_RESULT" $ADDITIONAL_SPACES '' } -while read -r line; -do +while read -r line; do RES_LINE=$(echo "$line" | sed -nE '/^s3tests_boto3/p') - if [ -n "$RES_LINE" ] - then - TEST=$(echo "$RES_LINE" | sed -e 's/[[:space:]]*\.\.\..*//') - RESULT=$(echo "$RES_LINE" | sed -e 's/^.*\.\.\.[[:space:]]*//') + if [ -n "$RES_LINE" ]; then + TEST=${RES_LINE%%[[:space:]]*} + RESULT=${RES_LINE##*[[:space:]]} # beautify trailing spaces OLD_RESULT_S3GW=$(sed -n "s/^.*${TEST}[[:space:]]*|[[:space:]]\(.*\)[[:space:]]|.*|.*|$/\1/p" "$RESULT_FILE" | head -1) @@ -49,4 +47,4 @@ do fi fi -done < "$INPUT_FILE" +done <"$INPUT_FILE"