Alex Vanin
fae03c2b50
When object exists in tree but missing in storage, we can't remove bucket. While storage node does not sync tree service and object service, the only way to delete such broken bucket is to ignore 'object not found' error, clear cache and do not include missing objects in the listing result. Signed-off-by: Alex Vanin <a.vanin@yadro.com> |
||
|---|---|---|
| .docker | ||
| .forgejo | ||
| api | ||
| authmate | ||
| cmd | ||
| config | ||
| creds | ||
| debian | ||
| docs | ||
| internal | ||
| metrics | ||
| pkg/service/tree | ||
| .dockerignore | ||
| .gitignore | ||
| .gitlint | ||
| .golangci.yml | ||
| .pre-commit-config.yaml | ||
| CHANGELOG.md | ||
| CONTRIBUTING.md | ||
| CREDITS.md | ||
| go.mod | ||
| go.sum | ||
| help.mk | ||
| LICENSE | ||
| Makefile | ||
| README.md | ||
| syncTree.sh | ||
| updateTestsResult.sh | ||
| VERSION | ||
FrostFS is a decentralized distributed object storage integrated with the NEO Blockchain.
FrostFS S3 Gateway
FrostFS S3 gateway provides API compatible with Amazon S3 cloud storage service.
Installation
go get -u git.frostfs.info/TrueCloudLab/frostfs-s3-gw
Or you can call make to build it from the cloned repository (the binary will
end up in bin/frostfs-s3-gw with authmate helper in bin/frostfs-s3-authmate).
To build binaries in clean docker environment, call make docker/all.
Other notable make targets:
dep Check and ensure dependencies
image Build clean docker image
dirty-image Build dirty docker image with host-built binaries
format Run all code formatters
lint Run linters
version Show current version
Or you can also use a Docker
image provided for released
(and occasionally unreleased) versions of gateway (:latest points to the
latest stable release).
Execution
Minimalistic S3 gateway setup needs:
- FrostFS node(s) address (S3 gateway itself is not a FrostFS node)
Passed via
-pparameter or viaS3_GW_PEERS_<N>_ADDRESSandS3_GW_PEERS_<N>_WEIGHTenvironment variables (gateway supports multiple FrostFS nodes with weighted load balancing). - a wallet used to fetch key and communicate with FrostFS nodes
Passed via
--walletparameter orS3_GW_WALLET_PATHenvironment variable.
These two commands are functionally equivalent, they run the gate with one backend node, some keys and otherwise default settings:
$ frostfs-s3-gw -p 192.168.130.72:8080 --wallet wallet.json
$ S3_GW_PEERS_0_ADDRESS=192.168.130.72:8080 \
S3_GW_WALLET=wallet.json \
frostfs-s3-gw
It's also possible to specify uri scheme (grpc or grpcs) when using -p or environment variables:
$ frostfs-s3-gw -p grpc://192.168.130.72:8080 --wallet wallet.json
$ S3_GW_PEERS_0_ADDRESS=grpcs://192.168.130.72:8080 \
S3_GW_WALLET=wallet.json \
frostfs-s3-gw
Domains
By default, s3-gw enable only path-style access.
To be able to use both: virtual-hosted-style and path-style access you must configure listen_domains:
$ frostfs-s3-gw -p 192.168.130.72:8080 --wallet wallet.json --listen_domains your.first.domain --listen_domains your.second.domain
So now you can use (e.g. HeadBucket. Make sure DNS is properly configured):
$ curl --head http://bucket-name.your.first.domain:8080
HTTP/1.1 200 OK
...
or
$ curl --head http://your.second.domain:8080/bucket-name
HTTP/1.1 200 OK
...
Also, you can configure domains using .env variables or yaml file.
Documentation
- Configuration
- FrostFS S3 AuthMate
- FrostFS Tree service
- AWS CLI basic usage
- AWS S3 API compatibility
- AWS S3 Compatibility test results
Credits
Please see CREDITS for details.