Aleksey Savchuk
59fc75de0b
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 1m38s
DCO action / DCO (pull_request) Successful in 1m53s
Vulncheck / Vulncheck (pull_request) Successful in 2m39s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m53s
Build / Build Components (pull_request) Successful in 3m15s
Tests and linters / Staticcheck (pull_request) Successful in 3m15s
Tests and linters / gopls check (pull_request) Successful in 3m21s
Tests and linters / Lint (pull_request) Successful in 4m29s
Tests and linters / Tests (pull_request) Successful in 7m18s
Tests and linters / Tests with -race (pull_request) Successful in 7m30s
Add a worker pool for `Inhume` operation and use it to handle objects in parallel. Since metabase `Inhume` uses `bbolt.Batch`, handling many objects one by one may be inefficient. ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/engine cpu: 12th Gen Intel(R) Core(TM) i5-1235U │ old.txt │ new.txt │ │ sec/op │ sec/op vs base │ InhumeMultipart/objects=1-12 11.45m ± 0% 11.43m ± 0% ~ (p=0.315 n=10) InhumeMultipart/objects=10-12 113.94m ± 1% 11.60m ± 0% -89.82% (p=0.000 n=10) InhumeMultipart/objects=100-12 1144.08m ± 1% 30.17m ± 12% -97.36% (p=0.000 n=10) InhumeMultipart/objects=1000-12 11367.5m ± 0% 265.7m ± 2% -97.66% (p=0.000 n=10) InhumeMultipart/objects=10000-12 113.505 ± 0% 2.436 ± 1% -97.85% (p=0.000 n=10) geomean 1.140 76.32m -93.31% ``` Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com> |
||
---|---|---|
.docker | ||
.forgejo | ||
cmd | ||
config | ||
dev | ||
docs | ||
internal | ||
misc | ||
pkg | ||
scripts | ||
.dockerignore | ||
.gitattributes | ||
.gitignore | ||
.golangci.yml | ||
.pre-commit-config.yaml | ||
CHANGELOG.md | ||
CONTRIBUTING.md | ||
CREDITS.md | ||
go.mod | ||
go.sum | ||
help.mk | ||
LICENSE | ||
Makefile | ||
README.md | ||
VERSION |
FrostFS is a decentralized distributed object storage integrated with the NEO Blockchain.
Overview
FrostFS Nodes are organized in a peer-to-peer network that takes care of storing and distributing user's data. Any Neo user may participate in the network and get paid for providing storage resources to other users or store their data in FrostFS and pay a competitive price for it.
Users can reliably store object data in the FrostFS network and have a transparent data placement process due to a decentralized architecture and flexible storage policies. Each node is responsible for executing the storage policies that the users select for geographical location, reliability level, number of nodes, type of disks, capacity, etc. Thus, FrostFS gives full control over data to users.
Deep Neo Blockchain integration allows FrostFS to be used by dApps directly from NeoVM on the Smart Contract code level. This way dApps are not limited to on-chain storage and can manipulate large amounts of data without paying a prohibitive price.
FrostFS has a native gRPC API and has protocol gateways for popular protocols such as AWS S3, HTTP, FUSE and sFTP allowing developers to integrate applications without rewriting their code.
Supported platforms
Now, we only support GNU/Linux on amd64 CPUs with AVX/AVX2 instructions. More
platforms will be officially supported after release 1.0
.
The latest version of frostfs-node works with frostfs-contract v0.19.2.
Building
To make all binaries you need Go 1.22+ and make
:
make all
The resulting binaries will appear in bin/
folder.
To make a specific binary use:
make bin/frostfs-<name>
See the list of all available commands in the cmd
folder.
Building with Docker
Building can also be performed in a container:
make docker/all # build all binaries
make docker/bin/frostfs-<name> # build a specific binary
Docker images
To make docker images suitable for use in frostfs-dev-env use:
make images
Debugging
VSCode
To run and debug single node cluster with VSCode:
- Clone and build frostfs-contract repository to the same directory level as
frostfs-node
. For example:
/
├── src
├── frostfs-node
└── frostfs-contract
See frostfs-contract
's README.md for build instructions.
-
Copy
launch.json
andtasks.json
fromdev/.vscode-example
directory to.vscode
directory. If you already have such files in.vscode
directory, then merge them manually. -
Go to Run and Debug (
Ctrl+Shift+D
) and startIR+Storage node
configuration. -
To create container and put object into it run (container and object IDs will be different):
./bin/frostfs-cli container create -r 127.0.0.1:8080 --wallet ./dev/wallet.json --policy "REP 1 IN X CBF 1 SELECT 1 FROM * AS X" --await
Enter password > <- press ENTER, the is no password for wallet
CID: CfPhEuHQ2PRvM4gfBQDC4dWZY3NccovyfcnEdiq2ixju
./bin/frostfs-cli object put -r 127.0.0.1:8080 --wallet ./dev/wallet.json --file README.md --cid CfPhEuHQ2PRvM4gfBQDC4dWZY3NccovyfcnEdiq2ixju
Enter password >
4300 / 4300 [===========================================================================================================================================================================================================] 100.00% 0s
[README.md] Object successfully stored
OID: 78sohnudVMnPsczXqsTUcvezosan2YDNVZwDE8Kq5YwU
CID: CfPhEuHQ2PRvM4gfBQDC4dWZY3NccovyfcnEdiq2ixju
./bin/frostfs-cli object get -r 127.0.0.1:8080 --wallet ./dev/wallet.json --cid CfPhEuHQ2PRvM4gfBQDC4dWZY3NccovyfcnEdiq2ixju --oid 78sohnudVMnPsczXqsTUcvezosan2YDNVZwDE8Kq5YwU
...
Contributing
Feel free to contribute to this project after reading the contributing guidelines.
Before starting to work on a certain topic, create a new issue first, describing the feature/topic you are going to implement.
Credits
FrostFS is maintained by True Cloud Lab with the help and contributions from community members.
Please see CREDITS for details.