FrostFS S3 Protocol Gateway
Find a file
Vitaliy Potyarkin 642fc27d32
All checks were successful
/ DCO (pull_request) Successful in 9m31s
/ Vulncheck (pull_request) Successful in 9m51s
/ Builds (pull_request) Successful in 10m24s
/ OCI image (pull_request) Successful in 10m46s
/ Lint (pull_request) Successful in 11m7s
/ Tests (pull_request) Successful in 10m33s
[#587] Build OCI images for release tags
OCI images will be built often (on each PR) to make sure our recipe
doesn't get rusty.
Publishing to the registry will happen only for `v*` tags.

Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
2024-12-20 14:44:07 +00:00
.docker [#369] Enhanced http requests logging 2024-09-11 15:25:09 +03:00
.forgejo [#587] Build OCI images for release tags 2024-12-20 14:44:07 +00:00
api [#577] Update SDK to support new tree/pool version 2024-12-20 13:50:31 +03:00
authmate [#570] Remove frostfs-api-go dependency 2024-12-10 11:03:30 +03:00
cmd [#577] Update SDK to support new tree/pool version 2024-12-20 13:50:31 +03:00
config [#577] Update SDK to support new tree/pool version 2024-12-20 13:50:31 +03:00
creds [#509] Support fallback address when getting box 2024-10-23 15:01:31 +03:00
debian [#68] Fix pre-commit issues 2023-03-24 16:22:06 +03:00
docs [#577] Update SDK to support new tree/pool version 2024-12-20 13:50:31 +03:00
internal [#577] Update SDK to support new tree/pool version 2024-12-20 13:50:31 +03:00
metrics [#502] Add Dropped logs (by sampling) metric 2024-12-03 12:16:56 +00:00
pkg [#488] Renamed api/errors, layer/frostfs and layer/tree package names 2024-10-02 12:35:04 +03:00
.dockerignore [#542] Stop using obsolete .github directory 2024-11-06 15:31:16 +03:00
.gitignore [#133] Drop sync-tree 2023-06-09 09:34:36 +03:00
.gitlint [#65] Enable pre-commit 2023-03-24 07:28:04 +00:00
.golangci.yml [#339] lint: Ignore aws sdk dirs 2024-12-10 15:04:56 +03:00
.pre-commit-config.yaml [#195] Add log constants linter 2023-08-28 12:58:44 +03:00
CHANGELOG.md Release v0.32.0 2024-12-20 13:18:21 +00:00
CODEOWNERS [#573] Refine CODEOWNERS settings 2024-12-10 14:35:40 +00:00
CONTRIBUTING.md [#2] Update CONTRIBUTING 2023-06-07 15:35:57 +00:00
CREDITS.md [#68] Fix pre-commit issues 2023-03-24 16:22:06 +03:00
go.mod [#577] Update SDK to support new tree/pool version 2024-12-20 13:50:31 +03:00
go.sum [#577] Update SDK to support new tree/pool version 2024-12-20 13:50:31 +03:00
help.mk [#725] Fix help 2022-10-17 19:16:05 +03:00
LICENSE [#264] Change NeoFS S3 Gateway license to AGPLv3 2021-09-20 10:38:28 +03:00
Makefile [#590] Use selfhosted OCI registry instead of Docker Hub 2024-12-20 13:57:29 +00:00
README.md [#542] Stop using obsolete .github directory 2024-11-06 15:31:16 +03:00
SECURITY.md [#413] Add SECURITY.md 2024-09-03 11:45:05 +00:00
updateTestsResult.sh [#68] Fix pre-commit issues 2023-03-24 16:22:06 +03:00
VERSION Release v0.32.0 2024-12-20 13:18:21 +00:00

FrostFS logo

FrostFS is a decentralized distributed object storage integrated with the NEO Blockchain.


Report Release License

FrostFS S3 Gateway

FrostFS S3 gateway provides API compatible with Amazon S3 cloud storage service.

Installation

go get -u git.frostfs.info/TrueCloudLab/frostfs-s3-gw

Or you can call make to build it from the cloned repository (the binary will end up in bin/frostfs-s3-gw with authmate helper in bin/frostfs-s3-authmate). To build binaries in clean docker environment, call make docker/all.

Other notable make targets:

dep          Check and ensure dependencies
image        Build clean docker image
dirty-image  Build dirty docker image with host-built binaries
format       Run all code formatters
lint         Run linters
version      Show current version

Or you can also use a Docker image provided for released (and occasionally unreleased) versions of gateway (:latest points to the latest stable release).

Execution

Minimalistic S3 gateway setup needs:

  • FrostFS node(s) address (S3 gateway itself is not a FrostFS node) Passed via -p parameter or via S3_GW_PEERS_<N>_ADDRESS and S3_GW_PEERS_<N>_WEIGHT environment variables (gateway supports multiple FrostFS nodes with weighted load balancing).
  • a wallet used to fetch key and communicate with FrostFS nodes Passed via --wallet parameter or S3_GW_WALLET_PATH environment variable.

These two commands are functionally equivalent, they run the gate with one backend node, some keys and otherwise default settings:

$ frostfs-s3-gw -p 192.168.130.72:8080 --wallet wallet.json

$ S3_GW_PEERS_0_ADDRESS=192.168.130.72:8080 \
  S3_GW_WALLET=wallet.json \
  frostfs-s3-gw

It's also possible to specify uri scheme (grpc or grpcs) when using -p or environment variables:

$ frostfs-s3-gw -p grpc://192.168.130.72:8080 --wallet wallet.json

$ S3_GW_PEERS_0_ADDRESS=grpcs://192.168.130.72:8080 \
  S3_GW_WALLET=wallet.json \
  frostfs-s3-gw

Domains

By default, s3-gw enable only path-style access. To be able to use both: virtual-hosted-style and path-style access you must configure listen_domains:

$ frostfs-s3-gw -p 192.168.130.72:8080 --wallet wallet.json --listen_domains your.first.domain --listen_domains your.second.domain

So now you can use (e.g. HeadBucket. Make sure DNS is properly configured):

$ curl --head http://bucket-name.your.first.domain:8080
HTTP/1.1 200 OK
...

or

$ curl --head http://your.second.domain:8080/bucket-name
HTTP/1.1 200 OK
...

Also, you can configure domains using .env variables or yaml file.

Fuzzing

To run fuzzing tests use the following command:

$ make fuzz

This command will install dependencies for the fuzzing process and run existing fuzzing tests.

You can also use the following arguments:

FUZZ_TIMEOUT - time to run each fuzzing test (default 30) 
FUZZ_FUNCTIONS - fuzzing tests that will be started (default "all")
FUZZ_AUX - additional parameters for the fuzzer (for example, "-debug")
FUZZ_NGFUZZ_DIR - path to ngfuzz tool

Documentation

Credits

Please see CREDITS for details.