FrostFS S3 Protocol Gateway
Find a file
2021-08-24 12:44:23 +03:00
.github/workflows *: Add docker to workflows 2021-05-24 16:11:46 +03:00
api Merge pull request #222 from KirillovDenis/feature/221-unify_source_filenames 2021-08-24 12:44:23 +03:00
authmate Merge pull request #170 from KirillovDenis/feature/89-placement_policy 2021-07-26 13:43:38 +03:00
cmd [#221] Unify source file naming 2021-08-20 15:19:49 +03:00
creds [#89] Add placement policy 2021-07-16 16:01:35 +03:00
docs [#206] docs: Language check 2021-08-19 17:10:14 +03:00
internal [#104] Support NEP-6 for authmate 2021-06-25 12:16:24 +03:00
.dockerignore Initial commit based on https://github.com/minio/minio/releases/tag/RELEASE.2020-07-02T00-15-09Z 2020-07-03 15:03:06 +03:00
.gitignore gitignore: more ignores 2021-05-13 22:08:20 +03:00
.golangci.yml golangci: add configuration 2021-05-13 23:26:05 +03:00
CHANGELOG.md CHANGELOG: Add latest changes of v0.16.0 release 2021-07-16 15:49:49 +03:00
Dockerfile [#57] *: Fix docker builds 2021-05-24 15:07:08 +03:00
go.mod [#206] api: Add objects cache 2021-08-18 10:16:27 +03:00
go.sum [#206] api: Add objects cache 2021-08-18 10:16:27 +03:00
help.mk Refactoring Makefile 2021-02-08 12:45:18 +03:00
LICENSE Initial commit based on https://github.com/minio/minio/releases/tag/RELEASE.2020-07-02T00-15-09Z 2020-07-03 15:03:06 +03:00
Makefile [#48] creda: Add accessbox in protobuf format 2021-06-14 16:38:37 +03:00
README.md [#210] *: Split README into docs 2021-08-19 17:10:14 +03:00

NeoFS S3 Gateway

NeoFS S3 gateway provides API compatible with Amazon S3 cloud storage service.

Installation

go get -u github.com/nspcc-dev/neofs-s3-gw

Or you can call make to build it from the cloned repository (the binary will end up in bin/neofs-s3-gw with authmate helper in bin/neofs-authmate).

Notable make targets:

dep          Check and ensure dependencies
image        Build clean docker image
dirty-image  Build dirty docker image with host-built binaries
format       Run all code formatters
lint         Run linters
version      Show current version

Or you can also use a Docker image provided for released (and occasionally unreleased) versions of gateway (:latest points to the latest stable release).

Execution

Minimalistic S3 gateway setup needs:

  • NeoFS node(s) address (S3 gateway itself is not a NeoFS node) Passed via -p parameter or via S3_GW_PEERS_<N>_ADDRESS and S3_GW_PEERS_<N>_WEIGHT environment variables (gateway supports multiple NeoFS nodes with weighted load balancing).
  • a wallet used to fetch key and communicate with NeoFS nodes Passed via --wallet parameter or S3_GW_WALLET environment variable.

These two commands are functionally equivalent, they run the gate with one backend node, some keys and otherwise default settings:

$ neofs-s3-gw -p 192.168.130.72:8080 --wallet wallet.json

$ S3_GW_PEERS_0_ADDRESS=192.168.130.72:8080 \
  S3_GW_WALLET=wallet.json \
  neofs-s3-gw

It's also possible to specify uri scheme (grpc or grpcs) when using -p or environment variables:

$ neofs-s3-gw -p grpc://192.168.130.72:8080 --wallet wallet.json

$ S3_GW_PEERS_0_ADDRESS=grpcs://192.168.130.72:8080 \
  S3_GW_WALLET=wallet.json \
  neofs-s3-gw

Documentation

S3 API supported

Reference:

Limitations

ACL

For now there are some restrictions:

  • Bucket policy support only one Principal (type AWS) per Statement. To refer all users use "AWS": "*"
  • AWS conditions and wildcard are not supported in resources
  • Only CanonicalUser (with hex encoded public key) and All Users Group are supported in ACL

Object

Method Status
CopyObject Supported
DeleteObject Supported
DeleteObjects Supported, aka DeleteMultipleObjects
GetObject Supported
GetObjectTorrent Unsupported, won't be
HeadObject Supported
ListObjectParts Unsupported
ListObjects Supported
ListObjectsV2 Supported
PutObject Supported (Content-MD5 option is not supported)
SelectObjectContent Unsupported
WriteGetObjectResponse Unsupported

ACL

Method Status
GetObjectAcl Supported
PutObjectAcl Supported

Locking

Method Status
GetObjectLegalHold Unsupported
GetObjectLockConfiguration Unsupported, aka GetBucketObjectLockConfig
GetObjectRetention Unsupported
PutObjectLegalHold Unsupported
PutObjectLockConfiguration Unsupported, aka PutBucketObjectLockConfig
PutObjectRetention Unsupported

Multipart

Should be supported eventually.

Method Status
AbortMultipartUpload Unsupported
CompleteMultipartUpload Unsupported
CreateMultipartUpload Unsupported, aka InitiateMultipartUpload and NewMultipartUpload
ListMultipartUploads Unsupported
ListParts Unsupported
UploadPart Unsupported, aka PutObjectPart
UploadPartCopy Unsupported, aka CopyObjectPart

Tagging

Also passed in PutObject parameters. We can support adding via PutObject and getting via GetBucketTagging, but deleting and putting can't be supported normally.

Method Status
DeleteObjectTagging Unsupported
GetObjectTagging Unsupported
PutObjectTagging Unsupported

Versioning

See also GetObject and other method parameters.

Method Status
ListObjectVersions Supported (null-versioning), aka ListBucketObjectVersions
RestoreObject Unsupported

Bucket

Method Status
CreateBucket Supported, aka PutBucket
DeleteBucket Supported
GetBucketLocation Unsupported
HeadBucket Supported
ListBuckets Supported
PutPublicAccessBlock Unsupported

Acceleration

Method Status
GetBucketAccelerateConfiguration Unsupported, aka GetBucketAccelerate
PutBucketAccelerateConfiguration Unsupported

ACL

Method Status
GetBucketAcl Supported
PutBucketAcl Supported

Analytics

Method Status
DeleteBucketAnalyticsConfiguration Unsupported
GetBucketAnalyticsConfiguration Unsupported
ListBucketAnalyticsConfigurations Unsupported
PutBucketAnalyticsConfiguration Unsupported

Cors

Method Status
DeleteBucketCors Unsupported
GetBucketCors Unsupported
PutBucketCors Unsupported

Encryption

Method Status
DeleteBucketEncryption Unsupported
GetBucketEncryption Unsupported
PutBucketEncryption Unsupported

Inventory

Method Status
DeleteBucketInventoryConfiguration Unsupported
GetBucketInventoryConfiguration Unsupported
ListBucketInventoryConfigurations Unsupported
PutBucketInventoryConfiguration Unsupported

Lifecycle

Method Status
DeleteBucketLifecycle Unsupported
GetBucketLifecycle Unsupported
GetBucketLifecycleConfiguration Unsupported
PutBucketLifecycle Unsupported
PutBucketLifecycleConfiguration Unsupported

Logging

Method Status
GetBucketLogging Unsupported
PutBucketLogging Unsupported

Metrics

Method Status
DeleteBucketMetricsConfiguration Unsupported
GetBucketMetricsConfiguration Unsupported
ListBucketMetricsConfigurations Unsupported
PutBucketMetricsConfiguration Unsupported

Notifications

Method Status
GetBucketNotification Unsupported
GetBucketNotificationConfiguration Unsupported
ListenBucketNotification Unsupported, non-standard?
PutBucketNotification Unsupported
PutBucketNotificationConfiguration Unsupported

Ownership controls

Method Status
DeleteBucketOwnershipControls Unsupported
GetBucketOwnershipControls Unsupported
PutBucketOwnershipControls Unsupported

Policy and replication

Method Status
DeleteBucketPolicy Unsupported
DeleteBucketReplication Unsupported
DeletePublicAccessBlock Unsupported
GetBucketPolicy Supported
GetBucketPolicyStatus Unsupported
GetBucketReplication Unsupported
PostPolicyBucket Unsupported, non-standard?
PutBucketPolicy Supported
PutBucketReplication Unsupported

Request payment

Method Status
GetBucketRequestPayment Unsupported
PutBucketRequestPayment Unsupported

Tagging

Method Status
DeleteBucketTagging Unsupported
GetBucketTagging Unsupported
PutBucketTagging Unsupported

Tiering

Method Status
DeleteBucketIntelligentTieringConfiguration Unsupported
GetBucketIntelligentTieringConfiguration Unsupported
ListBucketIntelligentTieringConfigurations Unsupported
PutBucketIntelligentTieringConfiguration Unsupported

Versioning

Method Status
GetBucketVersioning Unsupported
PutBucketVersioning Unsupported

Website

Method Status
DeleteBucketWebsite Unsupported
GetBucketWebsite Unsupported
PutBucketWebsite Unsupported