Roman Loginov
aaed083d82
We can always add interceptors to the grpc connection to the storage, since the actual use will be controlled by the configuration from the frostfs-observability library. Signed-off-by: Roman Loginov <r.loginov@yadro.com> |
||
|---|---|---|
| .docker | ||
| .forgejo/workflows | ||
| .github | ||
| api | ||
| authmate | ||
| cmd | ||
| config | ||
| creds | ||
| debian | ||
| docs | ||
| internal | ||
| metrics | ||
| pkg | ||
| .dockerignore | ||
| .gitignore | ||
| .gitlint | ||
| .golangci.yml | ||
| .pre-commit-config.yaml | ||
| CHANGELOG.md | ||
| CONTRIBUTING.md | ||
| CREDITS.md | ||
| go.mod | ||
| go.sum | ||
| help.mk | ||
| LICENSE | ||
| Makefile | ||
| README.md | ||
| SECURITY.md | ||
| updateTestsResult.sh | ||
| VERSION | ||
FrostFS is a decentralized distributed object storage integrated with the NEO Blockchain.
FrostFS S3 Gateway
FrostFS S3 gateway provides API compatible with Amazon S3 cloud storage service.
Installation
go get -u git.frostfs.info/TrueCloudLab/frostfs-s3-gw
Or you can call make to build it from the cloned repository (the binary will
end up in bin/frostfs-s3-gw with authmate helper in bin/frostfs-s3-authmate).
To build binaries in clean docker environment, call make docker/all.
Other notable make targets:
dep Check and ensure dependencies
image Build clean docker image
dirty-image Build dirty docker image with host-built binaries
format Run all code formatters
lint Run linters
version Show current version
Or you can also use a Docker
image provided for released
(and occasionally unreleased) versions of gateway (:latest points to the
latest stable release).
Execution
Minimalistic S3 gateway setup needs:
- FrostFS node(s) address (S3 gateway itself is not a FrostFS node)
Passed via
-pparameter or viaS3_GW_PEERS_<N>_ADDRESSandS3_GW_PEERS_<N>_WEIGHTenvironment variables (gateway supports multiple FrostFS nodes with weighted load balancing). - a wallet used to fetch key and communicate with FrostFS nodes
Passed via
--walletparameter orS3_GW_WALLET_PATHenvironment variable.
These two commands are functionally equivalent, they run the gate with one backend node, some keys and otherwise default settings:
$ frostfs-s3-gw -p 192.168.130.72:8080 --wallet wallet.json
$ S3_GW_PEERS_0_ADDRESS=192.168.130.72:8080 \
S3_GW_WALLET=wallet.json \
frostfs-s3-gw
It's also possible to specify uri scheme (grpc or grpcs) when using -p or environment variables:
$ frostfs-s3-gw -p grpc://192.168.130.72:8080 --wallet wallet.json
$ S3_GW_PEERS_0_ADDRESS=grpcs://192.168.130.72:8080 \
S3_GW_WALLET=wallet.json \
frostfs-s3-gw
Domains
By default, s3-gw enable only path-style access.
To be able to use both: virtual-hosted-style and path-style access you must configure listen_domains:
$ frostfs-s3-gw -p 192.168.130.72:8080 --wallet wallet.json --listen_domains your.first.domain --listen_domains your.second.domain
So now you can use (e.g. HeadBucket. Make sure DNS is properly configured):
$ curl --head http://bucket-name.your.first.domain:8080
HTTP/1.1 200 OK
...
or
$ curl --head http://your.second.domain:8080/bucket-name
HTTP/1.1 200 OK
...
Also, you can configure domains using .env variables or yaml file.
Fuzzing
To run fuzzing tests use the following command:
$ make fuzz
This command will install dependencies for the fuzzing process and run existing fuzzing tests.
You can also use the following arguments:
FUZZ_TIMEOUT - time to run each fuzzing test (default 30)
FUZZ_FUNCTIONS - fuzzing tests that will be started (default "all")
FUZZ_AUX - additional parameters for the fuzzer (for example, "-debug")
FUZZ_NGFUZZ_DIR - path to ngfuzz tool
Documentation
- Configuration
- FrostFS S3 AuthMate
- FrostFS Tree service
- AWS CLI basic usage
- AWS S3 API compatibility
- AWS S3 Compatibility test results
Credits
Please see CREDITS for details.