FrostFS is a decentralized distributed object storage integrated with the NEO Blockchain.
Find a file
Evgenii Stratonikov 9db82319aa
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 4m56s
DCO action / DCO (pull_request) Successful in 5m1s
Build / Build Components (1.21) (pull_request) Successful in 6m26s
Build / Build Components (1.20) (pull_request) Successful in 6m41s
Tests and linters / Lint (pull_request) Successful in 8m14s
Tests and linters / Staticcheck (pull_request) Successful in 8m5s
Tests and linters / Tests (1.21) (pull_request) Successful in 10m13s
Tests and linters / Tests (1.20) (pull_request) Successful in 12m50s
Tests and linters / Tests with -race (pull_request) Successful in 13m44s
[#996] metabase: Speed up bucket creation
Most of the time it exits, e.g. when it is per-container and use on each
object PUT. Bbolt implementation first tries to create bucket and then
returns it if it exists. Create operation uses cursor and thus is not
very lightweight, we can avoid it.

```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                 │     old     │                new                 │
                 │   sec/op    │   sec/op     vs base               │
Put/parallel-8     174.4µ ± 3%   163.3µ ± 3%  -6.39% (p=0.000 n=10)
Put/sequential-8   263.3µ ± 2%   259.0µ ± 1%  -1.64% (p=0.000 n=10)
geomean            214.3µ        205.6µ       -4.05%

                 │     old      │                 new                 │
                 │     B/op     │     B/op      vs base               │
Put/parallel-8     275.3Ki ± 3%   281.1Ki ± 4%       ~ (p=0.063 n=10)
Put/sequential-8   413.0Ki ± 2%   426.6Ki ± 2%  +3.29% (p=0.003 n=10)
geomean            337.2Ki        346.3Ki       +2.70%

                 │     old     │                 new                 │
                 │  allocs/op  │  allocs/op   vs base                │
Put/parallel-8      678.0 ± 1%    524.5 ± 2%  -22.64% (p=0.000 n=10)
Put/sequential-8   1.329k ± 0%   1.183k ± 0%  -10.91% (p=0.000 n=10)
geomean             949.1         787.9       -16.98%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-20 15:43:21 +03:00
.docker Revert "[#972] go.mod: Bump go version to go1.21" 2024-02-19 15:36:01 +00:00
.forgejo/workflows Revert "[#972] go.mod: Bump go version to go1.21" 2024-02-19 15:36:01 +00:00
.github Add to Possible Solutions 2023-03-21 15:41:20 +03:00
.woodpecker [#171] Remove unit tests from pipeline 2023-03-24 10:50:11 +03:00
cmd Revert "[#972] Use min/max builtins" 2024-02-19 15:36:01 +00:00
config [#736] logger: Add journald support 2024-02-12 06:17:22 +00:00
debian [#552] Add status notification to systemd 2023-12-13 15:02:39 +03:00
dev [#917] dev: Extend launch.json example 2024-02-06 14:49:47 +03:00
docs [#947] docs: Extend evacuation docs 2024-02-09 11:33:15 +03:00
internal [#947] engine: Evacuate trees to remote nodes 2024-02-09 11:33:15 +03:00
misc [#1592] Remove debug builds 2022-07-14 10:39:51 +03:00
pkg [#996] metabase: Speed up bucket creation 2024-02-20 15:43:21 +03:00
scripts/export-metrics [#772] node: Apply gofumpt 2023-10-31 17:03:03 +03:00
.dockerignore [#144] Don't copy cache inside Docker environment 2023-03-17 10:36:50 +03:00
.gitattributes [#246] .gitattributes: Do not show diff for go.sum 2023-04-14 06:02:04 +00:00
.gitignore [#53] Fix dirty in version 2023-02-16 11:10:43 +03:00
.gitlint [#171] Syncrhonize pre-commit settings across FrostFS repos 2023-03-24 10:43:55 +03:00
.golangci.yml [#857] golangci: Add protogetter linter 2023-12-12 16:27:02 +03:00
.pre-commit-config.yaml [#837] .pre-commit: Update hook versions 2023-12-06 11:07:10 +00:00
CHANGELOG.md [#581] Bump required go version to go1.20 2023-08-09 10:16:48 +00:00
CONTRIBUTING.md [#511] docs: Remove GitHub mentions from CONTRIBUTING.md 2023-07-18 13:24:02 +00:00
CREDITS.md [#83] pre-commit: Add initial configuration 2023-03-13 07:07:29 +00:00
go.mod Revert "[#972] go.mod: Bump go version to go1.21" 2024-02-19 15:36:01 +00:00
go.sum Revert "[#972] Drop x/exp/slices dependency" 2024-02-19 15:36:01 +00:00
help.mk [#1578] Minor Makefile fixes 2022-07-06 10:00:19 +03:00
LICENSE Initial commit 2020-07-10 17:45:00 +03:00
Makefile Revert "[#972] go.mod: Bump go version to go1.21" 2024-02-19 15:36:01 +00:00
README.md Revert "[#972] go.mod: Bump go version to go1.21" 2024-02-19 15:36:01 +00:00
VERSION Release v0.36.0 2023-04-12 16:57:02 +03:00

FrostFS

FrostFS is a decentralized distributed object storage integrated with the NEO Blockchain.


Report GitHub release (latest SemVer) License

Overview

FrostFS Nodes are organized in a peer-to-peer network that takes care of storing and distributing user's data. Any Neo user may participate in the network and get paid for providing storage resources to other users or store their data in FrostFS and pay a competitive price for it.

Users can reliably store object data in the FrostFS network and have a transparent data placement process due to a decentralized architecture and flexible storage policies. Each node is responsible for executing the storage policies that the users select for geographical location, reliability level, number of nodes, type of disks, capacity, etc. Thus, FrostFS gives full control over data to users.

Deep Neo Blockchain integration allows FrostFS to be used by dApps directly from NeoVM on the Smart Contract code level. This way dApps are not limited to on-chain storage and can manipulate large amounts of data without paying a prohibitive price.

FrostFS has a native gRPC API and has protocol gateways for popular protocols such as AWS S3, HTTP, FUSE and sFTP allowing developers to integrate applications without rewriting their code.

Supported platforms

Now, we only support GNU/Linux on amd64 CPUs with AVX/AVX2 instructions. More platforms will be officially supported after release 1.0.

The latest version of frostfs-node works with frostfs-contract v0.16.0.

Building

To make all binaries you need Go 1.20+ and make:

make all

The resulting binaries will appear in bin/ folder.

To make a specific binary use:

make bin/frostfs-<name>

See the list of all available commands in the cmd folder.

Building with Docker

Building can also be performed in a container:

make docker/all                     # build all binaries
make docker/bin/frostfs-<name> # build a specific binary

Docker images

To make docker images suitable for use in frostfs-dev-env use:

make images

Debugging

VSCode

To run and debug single node cluster with VSCode:

  1. Clone and build frostfs-contract repository to the same directory level as frostfs-node. For example:
/
├── src
    ├── frostfs-node
    └── frostfs-contract

See frostfs-contract's README.md for build instructions.

  1. Copy launch.json and tasks.json from dev/.vscode-example directory to .vscode directory. If you already have such files in .vscode directory, then merge them manually.

  2. Go to Run and Debug (Ctrl+Shift+D) and start IR+Storage node configuration.

  3. To create container and put object into it run (container and object IDs will be different):

./bin/frostfs-cli container create -r 127.0.0.1:8080 --wallet ./dev/wallet.json --policy "REP 1 IN X CBF 1 SELECT 1 FROM * AS X" --basic-acl public-read-write --await
Enter password >  <- press ENTER, the is no password for wallet
CID: CfPhEuHQ2PRvM4gfBQDC4dWZY3NccovyfcnEdiq2ixju

./bin/frostfs-cli object put -r 127.0.0.1:8080 --wallet ./dev/wallet.json --file README.md --cid CfPhEuHQ2PRvM4gfBQDC4dWZY3NccovyfcnEdiq2ixju
Enter password >
 4300 / 4300 [===========================================================================================================================================================================================================] 100.00% 0s
[README.md] Object successfully stored
  OID: 78sohnudVMnPsczXqsTUcvezosan2YDNVZwDE8Kq5YwU
  CID: CfPhEuHQ2PRvM4gfBQDC4dWZY3NccovyfcnEdiq2ixju

./bin/frostfs-cli object get -r 127.0.0.1:8080 --wallet ./dev/wallet.json --cid CfPhEuHQ2PRvM4gfBQDC4dWZY3NccovyfcnEdiq2ixju --oid 78sohnudVMnPsczXqsTUcvezosan2YDNVZwDE8Kq5YwU
...

Contributing

Feel free to contribute to this project after reading the contributing guidelines.

Before starting to work on a certain topic, create a new issue first, describing the feature/topic you are going to implement.

Credits

FrostFS is maintained by True Cloud Lab with the help and contributions from community members.

Please see CREDITS for details.

License