Denis Kirillov
5ee73fad6a
Despite the spec https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectVersions.html#API_ListObjectVersions_ResponseElements says that "When the number of responses exceeds the value of MaxKeys, NextVersionIdMarker specifies the first object version not returned that satisfies the search criteria. Use this value for the version-id-marker request parameter in a subsequent request." the actual behavior of AWS S3 is returning NextVersionIdMarker as the last returned object version Signed-off-by: Denis Kirillov <d.kirillov@yadro.com> |
||
|---|---|---|
| .docker | ||
| .forgejo/workflows | ||
| .github | ||
| api | ||
| authmate | ||
| cmd | ||
| config | ||
| creds | ||
| debian | ||
| docs | ||
| internal | ||
| metrics | ||
| pkg/service/tree | ||
| .dockerignore | ||
| .gitignore | ||
| .gitlint | ||
| .golangci.yml | ||
| .pre-commit-config.yaml | ||
| CHANGELOG.md | ||
| CONTRIBUTING.md | ||
| CREDITS.md | ||
| go.mod | ||
| go.sum | ||
| help.mk | ||
| LICENSE | ||
| Makefile | ||
| README.md | ||
| updateTestsResult.sh | ||
| VERSION | ||
FrostFS is a decentralized distributed object storage integrated with the NEO Blockchain.
FrostFS S3 Gateway
FrostFS S3 gateway provides API compatible with Amazon S3 cloud storage service.
Installation
go get -u git.frostfs.info/TrueCloudLab/frostfs-s3-gw
Or you can call make to build it from the cloned repository (the binary will
end up in bin/frostfs-s3-gw with authmate helper in bin/frostfs-s3-authmate).
To build binaries in clean docker environment, call make docker/all.
Other notable make targets:
dep Check and ensure dependencies
image Build clean docker image
dirty-image Build dirty docker image with host-built binaries
format Run all code formatters
lint Run linters
version Show current version
Or you can also use a Docker
image provided for released
(and occasionally unreleased) versions of gateway (:latest points to the
latest stable release).
Execution
Minimalistic S3 gateway setup needs:
- FrostFS node(s) address (S3 gateway itself is not a FrostFS node)
Passed via
-pparameter or viaS3_GW_PEERS_<N>_ADDRESSandS3_GW_PEERS_<N>_WEIGHTenvironment variables (gateway supports multiple FrostFS nodes with weighted load balancing). - a wallet used to fetch key and communicate with FrostFS nodes
Passed via
--walletparameter orS3_GW_WALLET_PATHenvironment variable.
These two commands are functionally equivalent, they run the gate with one backend node, some keys and otherwise default settings:
$ frostfs-s3-gw -p 192.168.130.72:8080 --wallet wallet.json
$ S3_GW_PEERS_0_ADDRESS=192.168.130.72:8080 \
S3_GW_WALLET=wallet.json \
frostfs-s3-gw
It's also possible to specify uri scheme (grpc or grpcs) when using -p or environment variables:
$ frostfs-s3-gw -p grpc://192.168.130.72:8080 --wallet wallet.json
$ S3_GW_PEERS_0_ADDRESS=grpcs://192.168.130.72:8080 \
S3_GW_WALLET=wallet.json \
frostfs-s3-gw
Domains
By default, s3-gw enable only path-style access.
To be able to use both: virtual-hosted-style and path-style access you must configure listen_domains:
$ frostfs-s3-gw -p 192.168.130.72:8080 --wallet wallet.json --listen_domains your.first.domain --listen_domains your.second.domain
So now you can use (e.g. HeadBucket. Make sure DNS is properly configured):
$ curl --head http://bucket-name.your.first.domain:8080
HTTP/1.1 200 OK
...
or
$ curl --head http://your.second.domain:8080/bucket-name
HTTP/1.1 200 OK
...
Also, you can configure domains using .env variables or yaml file.
Documentation
- Configuration
- FrostFS S3 AuthMate
- FrostFS Tree service
- AWS CLI basic usage
- AWS S3 API compatibility
- AWS S3 Compatibility test results
Credits
Please see CREDITS for details.