TrueCloudLab
/
frostfs-s3-gw
16
2
Fork 14
Code Issues 19 Pull Requests 4 Actions Packages Releases 6 Activity

[#343] docs: Actualize s3 compatibility table
/ Vulncheck (pull_request) Failing after 9s Details
/ DCO (pull_request) Successful in 1m30s Details
/ Builds (1.20) (pull_request) Successful in 2m11s Details
/ Builds (1.21) (pull_request) Successful in 1m49s Details
/ Lint (pull_request) Successful in 4m5s Details
/ Tests (1.20) (pull_request) Successful in 2m26s Details
/ Tests (1.21) (pull_request) Successful in 2m20s Details 

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
pull/348/head
Denis Kirillov 2024-04-02 10:00:44 +03:00
parent 1f5f2bd3d5
commit fd310f5e9f
1 changed files with 155 additions and 170 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

325
docs/aws_s3_compat.md
View File

@ -1,73 +1,58 @@
# S3 API support # S3 API support
Reference: Reference:
* [AWS S3 API Reference](https://docs.aws.amazon.com/AmazonS3/latest/API/s3-api.pdf) * [AWS S3 API Reference](https://docs.aws.amazon.com/AmazonS3/latest/API/s3-api.pdf)
| | Legend | | | Legend |
|----|-------------------------------------------| |-----|-------------------------------------------|
| 🟢 | Supported | | 🟢 | Supported |
| 🟡 | Partially supported | | 🟡 | Partially supported |
| 🔵 | Not supported yet, but will be in future | | 🔵 | Not supported yet, but will be in future |
| 🔴 | Not applicable or will never be supported | | 🔴 | Not applicable or will never be supported |
## Object ## Object
| | Method | Comments | | | Method | Comments |
|----|------------------------|-----------------------------------------| |-----|------------------------|-----------------------------------------|
| 🟢 | CopyObject | Done on gateway side | | 🟢 | CopyObject | Done on gateway side |
| 🟢 | DeleteObject | | | 🟢 | DeleteObject | |
| 🟢 | DeleteObjects | aka DeleteMultipleObjects | | 🟢 | DeleteObjects | aka DeleteMultipleObjects |
| 🟢 | GetObject | | | 🟢 | GetObject | |
| 🔴 | GetObjectTorrent | We don't plan implementing BT gateway | | 🔴 | GetObjectTorrent | We don't plan implementing BT gateway |
| 🟢 | HeadObject | | | 🟢 | HeadObject | |
| 🟢 | ListParts | Parts loaded with MultipartUpload | | 🟢 | ListParts | Parts loaded with MultipartUpload |
| 🟢 | ListObjects | | | 🟢 | ListObjects | |
| 🟢 | ListObjectsV2 | | | 🟢 | ListObjectsV2 | |
| 🟢 | PutObject | Content-MD5 header deprecated | | 🟢 | PutObject | Content-MD5 header deprecated |
| 🔵 | SelectObjectContent | Need to have some Lambda to execute SQL | | 🔵 | SelectObjectContent | Need to have some Lambda to execute SQL |
| 🔵 | WriteGetObjectResponse | Waiting for Lambda to be developed | | 🔵 | WriteGetObjectResponse | Waiting for Lambda to be developed |
| 🟢 | GetObjectAttributes | | | 🟢 | GetObjectAttributes | |
## ACL ## ACL
For now there are some limitations: For now there are some limitations:
* [Bucket policy](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-policies.html) supports only one `Principal` per `Statement`.
Principal must be `"AWS": "*"` (to refer all users) or `"CanonicalUser": "0313b1ac3a8076e155a7e797b24f0b650cccad5941ea59d7cfd51a024a8b2a06bf"` (hex encoded public key of desired user).
* Resource in bucket policy is an array. Each item MUST contain bucket name, CAN contain object name (wildcards are not supported):
```json
{
"Statement": [
{
"Resource": [
"arn:aws:s3:::bucket",
"arn:aws:s3:::bucket/some/object"
]
}
]
}
```
* AWS conditions and wildcard are not supported in [resources](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-arn-format.html)
* Only `CanonicalUser` (with hex encoded public key) and `All Users Group` are supported in [ACL](https://docs.aws.amazon.com/AmazonS3/latest/userguide/acl-overview.html)
| | Method | Comments | | | Method | Comments |
|----|--------------|-----------------| |-----|--------------|-----------------------------------|
| 🟡 | GetObjectAcl | See Limitations | | 🟢 | GetObjectAcl | Objects can have only private acl |
| 🟡 | PutObjectAcl | See Limitations | | 🔴 | PutObjectAcl | Use PutBucketPolicy instead |
## Locking ## Locking
For now there are some limitations: For now there are some limitations:
* Retention period can't be shortened, only extended. * Retention period can't be shortened, only extended.
* You can't delete locks or object with unexpired lock. * You can't delete locks or object with unexpired lock.
| | Method | Comments | | | Method | Comments |
|-----|----------------------------|---------------------------| |-----|----------------------------|-------------------------------|
| 🟡 | GetObjectLegalHold | | | 🟡 | GetObjectLegalHold | |
| 🟢 | GetObjectLockConfiguration | GetBucketObjectLockConfig | | 🟢 | GetObjectLockConfiguration | aka GetBucketObjectLockConfig |
| 🟡 | GetObjectRetention | | | 🟡 | GetObjectRetention | |
| 🟡 | PutObjectLegalHold | | | 🟡 | PutObjectLegalHold | |
| 🟢 | PutObjectLockConfiguration | PutBucketObjectLockConfig | | 🟢 | PutObjectLockConfiguration | aka PutBucketObjectLockConfig |
| 🟡 | PutObjectRetention | | | 🟡 | PutObjectRetention | |
## Multipart ## Multipart
@ -75,185 +60,185 @@ CompleteMultipartUpload operations may take long time to complete. Gateway
sends whitespace characters to keep connection with the client alive. In this sends whitespace characters to keep connection with the client alive. In this
case, gateway is unable to set proper HTTP headers like `X-Amz-Version-Id`. case, gateway is unable to set proper HTTP headers like `X-Amz-Version-Id`.
| | Method | Comments | | | Method | Comments |
|----|-------------------------|----------| |-----|-------------------------|----------|
| 🟢 | AbortMultipartUpload | | | 🟢 | AbortMultipartUpload | |
| 🟢 | CompleteMultipartUpload | | | 🟢 | CompleteMultipartUpload | |
| 🟢 | CreateMultipartUpload | | | 🟢 | CreateMultipartUpload | |
| 🟢 | ListMultipartUploads | | | 🟢 | ListMultipartUploads | |
| 🟢 | ListParts | | | 🟢 | ListParts | |
| 🟢 | UploadPart | | | 🟢 | UploadPart | |
| 🟢 | UploadPartCopy | | | 🟢 | UploadPartCopy | |
## Tagging ## Tagging
| | Method | Comments | | | Method | Comments |
|----|---------------------|----------| |-----|---------------------|----------|
| 🟢 | DeleteObjectTagging | | | 🟢 | DeleteObjectTagging | |
| 🟢 | GetObjectTagging | | | 🟢 | GetObjectTagging | |
| 🟢 | PutObjectTagging | | | 🟢 | PutObjectTagging | |
## Versioning ## Versioning
See also `GetObject` and other method parameters. See also `GetObject` and other method parameters.
| | Method | Comments | | | Method | Comments |
|----|--------------------|--------------------------| |-----|--------------------|--------------------------|
| 🟢 | ListObjectVersions | ListBucketObjectVersions | | 🟢 | ListObjectVersions | ListBucketObjectVersions |
| 🔵 | RestoreObject | | | 🔵 | RestoreObject | |
## Bucket ## Bucket
| | Method | Comments | | | Method | Comments |
|----|----------------------|-----------| |-----|----------------------|-----------|
| 🟢 | CreateBucket | PutBucket | | 🟢 | CreateBucket | PutBucket |
| 🟢 | DeleteBucket | | | 🟢 | DeleteBucket | |
| 🟢 | GetBucketLocation | | | 🟢 | GetBucketLocation | |
| 🟢 | HeadBucket | | | 🟢 | HeadBucket | |
| 🟢 | ListBuckets | | | 🟢 | ListBuckets | |
| 🔵 | PutPublicAccessBlock | | | 🔵 | PutPublicAccessBlock | |
## Acceleration ## Acceleration
| | Method | Comments | | | Method | Comments |
|----|----------------------------------|---------------------| |-----|----------------------------------|---------------------|
| 🔴 | GetBucketAccelerateConfiguration | GetBucketAccelerate | | 🔴 | GetBucketAccelerateConfiguration | GetBucketAccelerate |
| 🔴 | PutBucketAccelerateConfiguration | | | 🔴 | PutBucketAccelerateConfiguration | |
## ACL ## ACL
| | Method | Comments | | | Method | Comments |
|----|--------------|---------------------| |-----|--------------|------------------------------|
| 🟡 | GetBucketAcl | See ACL limitations | | 🟡 | GetBucketAcl | Only canned acl is supported |
| 🟡 | PutBucketAcl | See ACL Limitations | | 🟡 | PutBucketAcl | Only canned acl is supported |
## Analytics ## Analytics
| | Method | Comments | | | Method | Comments |
|----|------------------------------------|----------| |-----|------------------------------------|----------|
| 🔵 | DeleteBucketAnalyticsConfiguration | | | 🔵 | DeleteBucketAnalyticsConfiguration | |
| 🔵 | GetBucketAnalyticsConfiguration | | | 🔵 | GetBucketAnalyticsConfiguration | |
| 🔵 | ListBucketAnalyticsConfigurations | | | 🔵 | ListBucketAnalyticsConfigurations | |
| 🔵 | PutBucketAnalyticsConfiguration | | | 🔵 | PutBucketAnalyticsConfiguration | |
## CORS ## CORS
| | Method | Comments | | | Method | Comments |
|----|------------------|----------| |-----|------------------|----------|
| 🟢 | DeleteBucketCors | | | 🟢 | DeleteBucketCors | |
| 🟢 | GetBucketCors | | | 🟢 | GetBucketCors | |
| 🟢 | PutBucketCors | | | 🟢 | PutBucketCors | |
## Encryption ## Encryption
| | Method | Comments | | | Method | Comments |
|----|------------------------|----------| |-----|------------------------|----------|
| 🔵 | DeleteBucketEncryption | | | 🔵 | DeleteBucketEncryption | |
| 🔵 | GetBucketEncryption | | | 🔵 | GetBucketEncryption | |
| 🔵 | PutBucketEncryption | | | 🔵 | PutBucketEncryption | |
## Inventory ## Inventory
| | Method | Comments | | | Method | Comments |
|----|------------------------------------|----------| |-----|------------------------------------|----------|
| 🔵 | DeleteBucketInventoryConfiguration | | | 🔵 | DeleteBucketInventoryConfiguration | |
| 🔵 | GetBucketInventoryConfiguration | | | 🔵 | GetBucketInventoryConfiguration | |
| 🔵 | ListBucketInventoryConfigurations | | | 🔵 | ListBucketInventoryConfigurations | |
| 🔵 | PutBucketInventoryConfiguration | | | 🔵 | PutBucketInventoryConfiguration | |
## Lifecycle ## Lifecycle
| | Method | Comments | | | Method | Comments |
|----|---------------------------------|----------| |-----|---------------------------------|----------|
| 🔵 | DeleteBucketLifecycle | | | 🔵 | DeleteBucketLifecycle | |
| 🔵 | GetBucketLifecycle | | | 🔵 | GetBucketLifecycle | |
| 🔵 | GetBucketLifecycleConfiguration | | | 🔵 | GetBucketLifecycleConfiguration | |
| 🔵 | PutBucketLifecycle | | | 🔵 | PutBucketLifecycle | |
| 🔵 | PutBucketLifecycleConfiguration | | | 🔵 | PutBucketLifecycleConfiguration | |
## Logging ## Logging
| | Method | Comments | | | Method | Comments |
|----|------------------|----------| |-----|------------------|----------|
| 🔵 | GetBucketLogging | | | 🔵 | GetBucketLogging | |
| 🔵 | PutBucketLogging | | | 🔵 | PutBucketLogging | |
## Metrics ## Metrics
| | Method | Comments | | | Method | Comments |
|----|----------------------------------|----------| |-----|----------------------------------|----------|
| 🔵 | DeleteBucketMetricsConfiguration | | | 🔵 | DeleteBucketMetricsConfiguration | |
| 🔵 | GetBucketMetricsConfiguration | | | 🔵 | GetBucketMetricsConfiguration | |
| 🔵 | ListBucketMetricsConfigurations | | | 🔵 | ListBucketMetricsConfigurations | |
| 🔵 | PutBucketMetricsConfiguration | | | 🔵 | PutBucketMetricsConfiguration | |
## Notifications ## Notifications
| | Method | Comments | | | Method | Comments |
|----|------------------------------------|---------------| |-----|------------------------------------|---------------|
| 🔵 | GetBucketNotification | | | 🔵 | GetBucketNotification | |
| 🔵 | GetBucketNotificationConfiguration | | | 🔵 | GetBucketNotificationConfiguration | |
| 🔵 | ListenBucketNotification | non-standard? | | 🔵 | ListenBucketNotification | non-standard? |
| 🔵 | PutBucketNotification | | | 🔵 | PutBucketNotification | |
| 🔵 | PutBucketNotificationConfiguration | | | 🔵 | PutBucketNotificationConfiguration | |
## Ownership controls ## Ownership controls
| | Method | Comments | | | Method | Comments |
|----|-------------------------------|----------| |-----|-------------------------------|----------|
| 🔵 | DeleteBucketOwnershipControls | | | 🔵 | DeleteBucketOwnershipControls | |
| 🔵 | GetBucketOwnershipControls | | | 🔵 | GetBucketOwnershipControls | |
| 🔵 | PutBucketOwnershipControls | | | 🔵 | PutBucketOwnershipControls | |
## Policy and replication ## Policy and replication
| | Method | Comments | | | Method | Comments |
|----|-------------------------|-----------------------------| |-----|-------------------------|------------------------------|
| 🔵 | DeleteBucketPolicy | | | 🟢 | DeleteBucketPolicy | |
| 🔵 | DeleteBucketReplication | | | 🔵 | DeleteBucketReplication | |
| 🔵 | DeletePublicAccessBlock | | | 🔵 | DeletePublicAccessBlock | |
| 🟡 | GetBucketPolicy | See ACL limitations | | 🟢 | GetBucketPolicy | |
| 🔵 | GetBucketPolicyStatus | | | 🔵 | GetBucketPolicyStatus | |
| 🔵 | GetBucketReplication | | | 🔵 | GetBucketReplication | |
| 🟢 | PostPolicyBucket | Upload file using POST form | | 🟢 | PostPolicyBucket | Upload file using POST form |
| 🟡 | PutBucketPolicy | See ACL limitations | | 🟡 | PutBucketPolicy | Conditions are not supported |
| 🔵 | PutBucketReplication | | | 🔵 | PutBucketReplication | |
## Request payment ## Request payment
| | Method | Comments | | | Method | Comments |
|----|-------------------------|----------| |-----|-------------------------|----------|
| 🔴 | GetBucketRequestPayment | | | 🔴 | GetBucketRequestPayment | |
| 🔴 | PutBucketRequestPayment | | | 🔴 | PutBucketRequestPayment | |
## Tagging ## Tagging
| | Method | Comments | | | Method | Comments |
|----|---------------------|----------| |-----|---------------------|----------|
| 🟢 | DeleteBucketTagging | | | 🟢 | DeleteBucketTagging | |
| 🟢 | GetBucketTagging | | | 🟢 | GetBucketTagging | |
| 🟢 | PutBucketTagging | | | 🟢 | PutBucketTagging | |
## Tiering ## Tiering
| | Method | Comments | | | Method | Comments |
|----|---------------------------------------------|----------| |-----|---------------------------------------------|----------|
| 🔵 | DeleteBucketIntelligentTieringConfiguration | | | 🔵 | DeleteBucketIntelligentTieringConfiguration | |
| 🔵 | GetBucketIntelligentTieringConfiguration | | | 🔵 | GetBucketIntelligentTieringConfiguration | |
| 🔵 | ListBucketIntelligentTieringConfigurations | | | 🔵 | ListBucketIntelligentTieringConfigurations | |
| 🔵 | PutBucketIntelligentTieringConfiguration | | | 🔵 | PutBucketIntelligentTieringConfiguration | |
## Versioning ## Versioning
| | Method | Comments | | | Method | Comments |
|----|---------------------|----------| |-----|---------------------|----------|
| 🟢 | GetBucketVersioning | | | 🟢 | GetBucketVersioning | |
| 🟢 | PutBucketVersioning | | | 🟢 | PutBucketVersioning | |
## Website ## Website
| | Method | Comments | | | Method | Comments |
|----|---------------------|----------| |-----|---------------------|----------|
| 🔵 | DeleteBucketWebsite | | | 🔵 | DeleteBucketWebsite | |
| 🔵 | GetBucketWebsite | | | 🔵 | GetBucketWebsite | |
| 🔵 | PutBucketWebsite | | | 🔵 | PutBucketWebsite | |