Denis Kirillov
5ee73fad6a
All checks were successful
/ DCO (pull_request) Successful in 4m40s
/ Vulncheck (pull_request) Successful in 5m43s
/ Lint (pull_request) Successful in 10m27s
/ Tests (1.20) (pull_request) Successful in 5m41s
/ Tests (1.21) (pull_request) Successful in 5m47s
/ Builds (1.20) (pull_request) Successful in 1m9s
/ Builds (1.21) (pull_request) Successful in 4m7s
Despite the spec https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html#API_ListObjectVersions_ResponseElements says that "When the number of responses exceeds the value of MaxKeys, NextVersionIdMarker specifies the first object version not returned that satisfies the search criteria. Use this value for the version-id-marker request parameter in a subsequent request." the actual behavior of AWS S3 is returning NextVersionIdMarker as the last returned object version Signed-off-by: Denis Kirillov <d.kirillov@yadro.com> |
||
---|---|---|
.docker | ||
.forgejo/workflows | ||
.github | ||
api | ||
authmate | ||
cmd | ||
config | ||
creds | ||
debian | ||
docs | ||
internal | ||
metrics | ||
pkg/service/tree | ||
.dockerignore | ||
.gitignore | ||
.gitlint | ||
.golangci.yml | ||
.pre-commit-config.yaml | ||
CHANGELOG.md | ||
CONTRIBUTING.md | ||
CREDITS.md | ||
go.mod | ||
go.sum | ||
help.mk | ||
LICENSE | ||
Makefile | ||
README.md | ||
updateTestsResult.sh | ||
VERSION |
FrostFS is a decentralized distributed object storage integrated with the NEO Blockchain.
FrostFS S3 Gateway
FrostFS S3 gateway provides API compatible with Amazon S3 cloud storage service.
Installation
go get -u git.frostfs.info/TrueCloudLab/frostfs-s3-gw
Or you can call make
to build it from the cloned repository (the binary will
end up in bin/frostfs-s3-gw
with authmate helper in bin/frostfs-s3-authmate
).
To build binaries in clean docker environment, call make docker/all
.
Other notable make targets:
dep Check and ensure dependencies
image Build clean docker image
dirty-image Build dirty docker image with host-built binaries
format Run all code formatters
lint Run linters
version Show current version
Or you can also use a Docker
image provided for released
(and occasionally unreleased) versions of gateway (:latest
points to the
latest stable release).
Execution
Minimalistic S3 gateway setup needs:
- FrostFS node(s) address (S3 gateway itself is not a FrostFS node)
Passed via
-p
parameter or viaS3_GW_PEERS_<N>_ADDRESS
andS3_GW_PEERS_<N>_WEIGHT
environment variables (gateway supports multiple FrostFS nodes with weighted load balancing). - a wallet used to fetch key and communicate with FrostFS nodes
Passed via
--wallet
parameter orS3_GW_WALLET_PATH
environment variable.
These two commands are functionally equivalent, they run the gate with one backend node, some keys and otherwise default settings:
$ frostfs-s3-gw -p 192.168.130.72:8080 --wallet wallet.json
$ S3_GW_PEERS_0_ADDRESS=192.168.130.72:8080 \
S3_GW_WALLET=wallet.json \
frostfs-s3-gw
It's also possible to specify uri scheme (grpc or grpcs) when using -p
or environment variables:
$ frostfs-s3-gw -p grpc://192.168.130.72:8080 --wallet wallet.json
$ S3_GW_PEERS_0_ADDRESS=grpcs://192.168.130.72:8080 \
S3_GW_WALLET=wallet.json \
frostfs-s3-gw
Domains
By default, s3-gw enable only path-style access
.
To be able to use both: virtual-hosted-style
and path-style
access you must configure listen_domains
:
$ frostfs-s3-gw -p 192.168.130.72:8080 --wallet wallet.json --listen_domains your.first.domain --listen_domains your.second.domain
So now you can use (e.g. HeadBucket
. Make sure DNS is properly configured):
$ curl --head http://bucket-name.your.first.domain:8080
HTTP/1.1 200 OK
...
or
$ curl --head http://your.second.domain:8080/bucket-name
HTTP/1.1 200 OK
...
Also, you can configure domains using .env
variables or yaml
file.
Documentation
- Configuration
- FrostFS S3 AuthMate
- FrostFS Tree service
- AWS CLI basic usage
- AWS S3 API compatibility
- AWS S3 Compatibility test results
Credits
Please see CREDITS for details.