Denis Kirillov
056f168d77
Previously after tree split we can have duplicated parts (several objects and tree node referred to the same part number). Some of them couldn't be deleted after abort or compete action. Signed-off-by: Denis Kirillov <d.kirillov@yadro.com> |
||
|---|---|---|
| .docker | ||
| .forgejo/workflows | ||
| .github | ||
| api | ||
| authmate | ||
| cmd | ||
| config | ||
| creds | ||
| debian | ||
| docs | ||
| internal | ||
| metrics | ||
| pkg | ||
| .dockerignore | ||
| .gitignore | ||
| .gitlint | ||
| .golangci.yml | ||
| .pre-commit-config.yaml | ||
| CHANGELOG.md | ||
| CONTRIBUTING.md | ||
| CREDITS.md | ||
| go.mod | ||
| go.sum | ||
| help.mk | ||
| LICENSE | ||
| Makefile | ||
| README.md | ||
| SECURITY.md | ||
| updateTestsResult.sh | ||
| VERSION | ||
FrostFS is a decentralized distributed object storage integrated with the NEO Blockchain.
FrostFS S3 Gateway
FrostFS S3 gateway provides API compatible with Amazon S3 cloud storage service.
Installation
go get -u git.frostfs.info/TrueCloudLab/frostfs-s3-gw
Or you can call make to build it from the cloned repository (the binary will
end up in bin/frostfs-s3-gw with authmate helper in bin/frostfs-s3-authmate).
To build binaries in clean docker environment, call make docker/all.
Other notable make targets:
dep Check and ensure dependencies
image Build clean docker image
dirty-image Build dirty docker image with host-built binaries
format Run all code formatters
lint Run linters
version Show current version
Or you can also use a Docker
image provided for released
(and occasionally unreleased) versions of gateway (:latest points to the
latest stable release).
Execution
Minimalistic S3 gateway setup needs:
- FrostFS node(s) address (S3 gateway itself is not a FrostFS node)
Passed via
-pparameter or viaS3_GW_PEERS_<N>_ADDRESSandS3_GW_PEERS_<N>_WEIGHTenvironment variables (gateway supports multiple FrostFS nodes with weighted load balancing). - a wallet used to fetch key and communicate with FrostFS nodes
Passed via
--walletparameter orS3_GW_WALLET_PATHenvironment variable.
These two commands are functionally equivalent, they run the gate with one backend node, some keys and otherwise default settings:
$ frostfs-s3-gw -p 192.168.130.72:8080 --wallet wallet.json
$ S3_GW_PEERS_0_ADDRESS=192.168.130.72:8080 \
S3_GW_WALLET=wallet.json \
frostfs-s3-gw
It's also possible to specify uri scheme (grpc or grpcs) when using -p or environment variables:
$ frostfs-s3-gw -p grpc://192.168.130.72:8080 --wallet wallet.json
$ S3_GW_PEERS_0_ADDRESS=grpcs://192.168.130.72:8080 \
S3_GW_WALLET=wallet.json \
frostfs-s3-gw
Domains
By default, s3-gw enable only path-style access.
To be able to use both: virtual-hosted-style and path-style access you must configure listen_domains:
$ frostfs-s3-gw -p 192.168.130.72:8080 --wallet wallet.json --listen_domains your.first.domain --listen_domains your.second.domain
So now you can use (e.g. HeadBucket. Make sure DNS is properly configured):
$ curl --head http://bucket-name.your.first.domain:8080
HTTP/1.1 200 OK
...
or
$ curl --head http://your.second.domain:8080/bucket-name
HTTP/1.1 200 OK
...
Also, you can configure domains using .env variables or yaml file.
Fuzzing
To run fuzzing tests use the following command:
$ make fuzz
This command will install dependencies for the fuzzing process and run existing fuzzing tests.
You can also use the following arguments:
FUZZ_TIMEOUT - time to run each fuzzing test (default 30)
FUZZ_FUNCTIONS - fuzzing tests that will be started (default "all")
FUZZ_AUX - additional parameters for the fuzzer (for example, "-debug")
FUZZ_NGFUZZ_DIR - path to ngfuzz tool
Documentation
- Configuration
- FrostFS S3 AuthMate
- FrostFS Tree service
- AWS CLI basic usage
- AWS S3 API compatibility
- AWS S3 Compatibility test results
Credits
Please see CREDITS for details.