Commit graph

1434 commits

Author SHA1 Message Date
4d88bdd4e5 [#421] Try using badger for the write-cache
All checks were successful
Build / Build Components (1.19) (pull_request) Successful in 2m15s
Tests and linters / Tests (1.19) (pull_request) Successful in 2m22s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m50s
Build / Build Components (1.20) (pull_request) Successful in 12m23s
Tests and linters / Tests with -race (pull_request) Successful in 5m47s
Vulncheck / Vulncheck (pull_request) Successful in 1m6s
Tests and linters / Lint (pull_request) Successful in 14m38s
Tests and linters / Staticcheck (pull_request) Successful in 6m14s
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-02 18:25:53 +03:00
ec8a631d31 [#542] Update test for SizeInBytesSafe function
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-01 13:38:00 +00:00
9ca63ac8c3 [#542] Fix bug in SizeInBytesSafe function
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-01 13:38:00 +00:00
32c77f3a23 [#537] node: Add runtime.memory_limit config parameter
Some checks failed
Build / Build Components (1.20) (pull_request) Successful in 4m6s
Build / Build Components (1.19) (pull_request) Successful in 4m18s
ci/woodpecker/pr/pre-commit Pipeline failed
Tests and linters / Lint (pull_request) Failing after 2m10s
Tests and linters / Tests with -race (pull_request) Has started running
Tests and linters / Tests (1.19) (pull_request) Successful in 3m58s
Tests and linters / Tests (1.20) (pull_request) Successful in 3m44s
Vulncheck / Vulncheck (pull_request) Failing after 3m51s
Tests and linters / Staticcheck (pull_request) Successful in 8m3s
ci/woodpecker/push/pre-commit Pipeline was successful
This parameter allows to set soft memory limit for Go GC.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-25 17:27:09 +03:00
fef172c5b0 [#6] services/util: Simplify response.Service
It has only 1 parameter, which is obligatory.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-21 18:29:57 +03:00
57e7fb5ccf [#521] cli: Add common aliases to policy playground commands
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-19 11:34:08 +00:00
24dffdac6f [#521] cli: Add netmap load command to policy playground
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-19 11:34:08 +00:00
a9d04ba86f [#244] Remove --local-dump from frostfs-adm config
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-18 13:51:25 +00:00
4680087711 [#527] Add support for select-filter expressions in policy playground
Some checks failed
ci/woodpecker/push/pre-commit Pipeline failed
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-18 13:15:58 +00:00
f0355a453e [#463] policer: Remove capacity rebalance logic
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Current implementation has some quirks. For example,
using only half of object.put.pool_size_remote threads
tells replicator that is node is 50% loaded,
but in reality we could be putting lot's of big objects.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-18 10:52:12 +00:00
b2487e8cc5 [#516] node: Do not bootstrap if node is online candidate
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-18 10:51:02 +00:00
a0c7045f29 [#512] cli: Refactor UX for bearer create command
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Build / Build Components (1.19) (pull_request) Successful in 3m56s
Tests and linters / Lint (pull_request) Successful in 3m9s
Tests and linters / Tests (1.19) (pull_request) Successful in 5m27s
Build / Build Components (1.20) (pull_request) Successful in 9m25s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m57s
Tests and linters / Tests with -race (pull_request) Successful in 5m13s
Tests and linters / Staticcheck (pull_request) Successful in 10m30s
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-17 11:45:27 +03:00
d8e37a827f [#497] config: Add examples and unit tests
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Add examples and unit tests for tree.authorized_keys section.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-17 08:23:28 +00:00
486287c2f7 [#524] cli: Add impersonate flag for bearer token creation
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-17 08:20:50 +00:00
c83e7c875f [#294] searchsvcv2: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
e8091101c7 [#294] searchsvc: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
ec9b738465 [#294] putsvcv2: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
800a685e84 [#294] putsvc: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
1420b8b9ea [#294] deletesvc: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
a476d8285a [#294] deletesvcv2: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
70a1081988 [#294] aclsvcv2: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
61541eaec2 [#294] aclsvc: Refactor checker constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
0754e6e654 [#390] cli: Fix bearer token reading for tree subcommands
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-07-11 14:11:30 +00:00
676a3efa79 [#390] cli: Make tree commands errors and messages more verbose
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-07-11 14:11:30 +00:00
c42db4e761 [#390] cli: Support more tree service API calls in CLI
* Support healthcheck API call
* Support move API call
* Support remove API call
* Support getSubtree API call
* Support getOpLog API call

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-07-11 14:11:30 +00:00
7b76527759 [#486] node: Add PutSingle wrappers
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-10 15:49:21 +03:00
040a623d39 [#476] cli: Fix object nodes command
Some checks failed
Build / Build Components (1.20) (pull_request) Successful in 3m56s
Build / Build Components (1.19) (pull_request) Successful in 4m18s
ci/woodpecker/pr/pre-commit Pipeline was successful
Tests and linters / Lint (pull_request) Failing after 7m27s
Tests and linters / Tests (1.19) (pull_request) Successful in 7m45s
Tests and linters / Tests with -race (pull_request) Successful in 7m40s
Tests and linters / Tests (1.20) (pull_request) Successful in 14m7s
Tests and linters / Staticcheck (pull_request) Successful in 13m37s
ci/woodpecker/push/pre-commit Pipeline was successful
Do not fail if client creation failed.
Use external addresses to create the client too.
Use public key as node ID.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-10 09:58:31 +03:00
e858479a74 [#498] policer: Explicitly Rewind() iterator after finish
Previously, we can continue to return `EndOfListing` infinitely.
Reflect iterator reuse via Rewind() method.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-07 14:52:33 +00:00
033eaf77e1 [#496] node: Fix linter importas
All checks were successful
Build / Build Components (1.20) (pull_request) Successful in 3m52s
Build / Build Components (1.19) (pull_request) Successful in 4m1s
ci/woodpecker/pr/pre-commit Pipeline was successful
Tests and linters / Tests with -race (pull_request) Successful in 5m36s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m55s
Tests and linters / Lint (pull_request) Successful in 14m40s
Tests and linters / Tests (1.19) (pull_request) Successful in 15m29s
ci/woodpecker/push/pre-commit Pipeline was successful
Standardize the alias of the
import frostfs-sdk-go/object as objectSDK.

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-07-06 15:36:41 +03:00
b520a3049e [#483] cli: Fix object put cmd with flag --prepare-locally
All checks were successful
Build / Build Components (1.20) (pull_request) Successful in 3m31s
Build / Build Components (1.19) (pull_request) Successful in 3m46s
ci/woodpecker/pr/pre-commit Pipeline was successful
Tests and linters / Tests with -race (pull_request) Successful in 5m2s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m28s
Tests and linters / Lint (pull_request) Successful in 9m28s
Tests and linters / Tests (1.19) (pull_request) Successful in 13m46s
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-07-04 18:58:26 +03:00
4c248d573e [#483] cli: Allow to split object on the client side
Some checks failed
Build / Build Components (1.20) (pull_request) Successful in 3m50s
Build / Build Components (1.19) (pull_request) Successful in 4m2s
ci/woodpecker/pr/pre-commit Pipeline was successful
Tests and linters / Lint (pull_request) Successful in 8m50s
Tests and linters / Tests with -race (pull_request) Successful in 5m16s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m10s
Tests and linters / Tests (1.19) (pull_request) Successful in 13m36s
ci/woodpecker/push/pre-commit Pipeline failed
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-07-03 18:31:46 +03:00
90e9a85acc [#483] Update dependencies
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-07-03 15:44:59 +03:00
f9730f090d [#92] Refactor policer and add some unit tests
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-03 07:05:31 +00:00
0c5b025788 [#470] grpc: Increase message limits
All checks were successful
Build / Build Components (1.19) (pull_request) Successful in 3m27s
ci/woodpecker/pr/pre-commit Pipeline was successful
Build / Build Components (1.20) (pull_request) Successful in 3m25s
Tests and linters / Lint (pull_request) Successful in 2m15s
Tests and linters / Tests (1.19) (pull_request) Successful in 2m51s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m54s
Tests and linters / Tests with -race (pull_request) Successful in 6m31s
ci/woodpecker/push/pre-commit Pipeline was successful
For send message limit set to 2GiB, but there are custom
GET/GET RANGE limiters.
For receive message limit set to 256 MiB, but actual chunk size
will be managed by client.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-30 16:32:15 +03:00
8a4e250dae [#468] *: replace outdated TODO crypto-related links
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
73a71a71b0 [#1] node: Use a proper validation of a substorage type
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
f4c71cea65 [#1] *: Replace outdated FIXME/TODO links
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
cac4ed93d6 [#428] engine: Add low_mem config parameter
Concurrent initialization in case of the metabase resync leads to
high memory consumption and potential OOM.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-26 13:29:39 +00:00
72fedff7ad [#426] cli: Add object nodes command
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-26 07:41:34 +00:00
ab489265b3 [#449] cli: Allow to exec tree get-by-path without bearer token
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-23 14:44:19 +03:00
71889234b7 [#449] tree: Allow reading requests signed by keys from allow list
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-23 14:44:19 +03:00
167a67f0b8 [#460] services/util: Remove HandleUnaryRequest
There is no need in a wrapper with many from-`interface{}` conversions.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-21 17:07:56 +03:00
b5d9f4a285 [#373] metrics: Add pilorama metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
e89fa110c7 [#373] metrics: Add metabase metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
af608da952 [#373] metrics: Add blobovnizca metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
56f320dd85 [#373] metrics: Add blobstor metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
16a142cd0c [#373] metrics: Add FSTree metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
059e9e88a2 [#373] metabase: Add metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
26b305f82b [#424] morph: Fix cache metrics
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Use separate morph cache metrics for node and IR

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 18:26:19 +03:00
c348ae35b0 [#424] metrics: Drop embedded metrics
It was not obvious where metrics are used.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 14:53:32 +03:00