FrostFS S3 Protocol Gateway
Find a file
Angira Kekteeva 5d52f056eb [#583] Fix error when policy file cannot be read
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-07 23:45:31 +04:00
.docker [#570] Fix docker image builds 2022-07-05 15:12:33 +03:00
.github/workflows [#543] Drop workflow dispatch 2022-06-20 18:33:08 +03:00
api [#553] Add more comments about eacl.RoleUnknown 2022-07-07 12:05:25 +03:00
authmate [#539] Add context to errors 2022-06-27 02:23:19 +04:00
cmd [#583] Fix error when policy file cannot be read 2022-07-07 23:45:31 +04:00
config [#396] Rename rpc-endpoint and resolve-order 2022-04-18 11:58:54 +03:00
creds [#539] Fix nonce size computation 2022-06-27 02:23:19 +04:00
docs [#562] Rename authmate directory 2022-07-04 11:08:50 +03:00
internal [#572] Upgrade NeoFS SDK Go to v1.0.0-rc.5 2022-07-06 17:36:27 +03:00
.dockerignore [#471] Add docker/* target in Makefile 2022-06-16 11:12:42 +03:00
.gitignore [#579] Remove cache dir at cleanup 2022-07-06 10:49:46 +03:00
.golangci.yml [#272] Replace golint by revive 2021-10-01 12:25:54 +03:00
CHANGELOG.md Release v0.21.1 2022-05-16 15:29:35 +03:00
CONTRIBUTING.md [#579] Add CONTRIBUTING guide 2022-07-06 10:49:46 +03:00
go.mod [#572] Upgrade NeoFS SDK Go to v1.0.0-rc.5 2022-07-06 17:36:27 +03:00
go.sum [#572] Upgrade NeoFS SDK Go to v1.0.0-rc.5 2022-07-06 17:36:27 +03:00
help.mk Refactoring Makefile 2021-02-08 12:45:18 +03:00
LICENSE [#264] Change NeoFS S3 Gateway license to AGPLv3 2021-09-20 10:38:28 +03:00
Makefile [#579] Remove cache dir at cleanup 2022-07-06 10:49:46 +03:00
README.md [#562] Rename authmate directory 2022-07-04 11:08:50 +03:00
updateTestsResult.sh [#454] Support repeating tests in resulting file 2022-06-07 17:37:05 +03:00
VERSION Release v0.21.1 2022-05-16 15:29:35 +03:00

NeoFS S3 Gateway

NeoFS S3 gateway provides API compatible with Amazon S3 cloud storage service.

Installation

go get -u github.com/nspcc-dev/neofs-s3-gw

Or you can call make to build it from the cloned repository (the binary will end up in bin/neofs-s3-gw with authmate helper in bin/neofs-s3-authmate). To build binaries in clean docker environment, call make docker/all.

Other notable make targets:

dep          Check and ensure dependencies
image        Build clean docker image
dirty-image  Build dirty docker image with host-built binaries
format       Run all code formatters
lint         Run linters
version      Show current version

Or you can also use a Docker image provided for released (and occasionally unreleased) versions of gateway (:latest points to the latest stable release).

Execution

Minimalistic S3 gateway setup needs:

  • NeoFS node(s) address (S3 gateway itself is not a NeoFS node) Passed via -p parameter or via S3_GW_PEERS_<N>_ADDRESS and S3_GW_PEERS_<N>_WEIGHT environment variables (gateway supports multiple NeoFS nodes with weighted load balancing).
  • a wallet used to fetch key and communicate with NeoFS nodes Passed via --wallet parameter or S3_GW_WALLET environment variable.

These two commands are functionally equivalent, they run the gate with one backend node, some keys and otherwise default settings:

$ neofs-s3-gw -p 192.168.130.72:8080 --wallet wallet.json

$ S3_GW_PEERS_0_ADDRESS=192.168.130.72:8080 \
  S3_GW_WALLET=wallet.json \
  neofs-s3-gw

It's also possible to specify uri scheme (grpc or grpcs) when using -p or environment variables:

$ neofs-s3-gw -p grpc://192.168.130.72:8080 --wallet wallet.json

$ S3_GW_PEERS_0_ADDRESS=grpcs://192.168.130.72:8080 \
  S3_GW_WALLET=wallet.json \
  neofs-s3-gw

AWS SDK Compatibility

To match signature of the request you must not include the following headers to SignedHeaders:

  • User-Agent
  • X-Amzn-Trace-Id

AWS SDK JAVA v1

Using aws-sdk-java you can get the following error:

Exception in thread "main" com.amazonaws.services.s3.model.AmazonS3Exception: 
The request signature we calculated does not match the signature you provided. Check your key and signing method.

To solve this problem you should configure client properly:

RequestHandler2 handler = new RequestHandler2() {
   @Override
   public void beforeAttempt(HandlerBeforeAttemptContext context) {
      context.getRequest().getHeaders().remove("User-Agent");
      super.beforeAttempt(context);
   }
};

AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
   .withRequestHandlers(handler)
   .enablePathStyleAccess()
   // ...
   .build()

Documentation