Commit graph

238 commits

Author SHA1 Message Date
627b302745 [#709] node: Put in log info about listening endpoints
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-02 13:31:10 +00:00
aa9f8dce3d [#677] client: Refactor PrmAnnounceSpace/EndpointInfo/NetworkInfo usage
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 1m28s
DCO action / DCO (pull_request) Successful in 2m25s
Build / Build Components (1.21) (pull_request) Successful in 4m14s
Tests and linters / Staticcheck (pull_request) Successful in 4m24s
Tests and linters / Tests (1.21) (pull_request) Successful in 4m34s
Tests and linters / Lint (pull_request) Successful in 5m28s
Tests and linters / Tests (1.20) (pull_request) Successful in 7m46s
Tests and linters / Tests with -race (pull_request) Successful in 8m10s
Build / Build Components (1.20) (pull_request) Successful in 9m1s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-08 09:42:28 +03:00
8a81af5a3b [#653] Add context parameter to Open functions
All checks were successful
DCO action / DCO (pull_request) Successful in 1m38s
Build / Build Components (1.20) (pull_request) Successful in 4m22s
Build / Build Components (1.21) (pull_request) Successful in 4m25s
Vulncheck / Vulncheck (pull_request) Successful in 4m56s
Tests and linters / Lint (pull_request) Successful in 6m1s
Tests and linters / Tests (1.20) (pull_request) Successful in 7m43s
Tests and linters / Staticcheck (pull_request) Successful in 8m1s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m14s
Tests and linters / Tests with -race (pull_request) Successful in 8m32s
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-09-07 18:03:29 +03:00
ae81d6660a [#529] objectcore: Fix object content validation
There are old objects where the owner of the object
may not match the one who issued the token.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-29 10:33:06 +03:00
4ea0df77d0 [#574] policer: Check if the container was really removed
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-28 14:21:38 +00:00
554ff2c06b [#574] core: Extend Source interface with DeletionInfo method
* Introduce common method EverExisted
* Define DeletionInfo for struct that must implement Source
* Refactor tree srv

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-28 14:21:38 +00:00
23be3eb627 [#574] tree: Check if container is really removed
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 2m37s
Build / Build Components (1.21) (pull_request) Successful in 3m36s
Tests and linters / Staticcheck (pull_request) Successful in 3m36s
DCO action / DCO (pull_request) Successful in 4m16s
Tests and linters / Lint (pull_request) Successful in 5m34s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m36s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m46s
Tests and linters / Tests with -race (pull_request) Successful in 6m21s
Build / Build Components (1.20) (pull_request) Successful in 8m1s
* Use DeletionInfo method from morph client to check if
  the container has been really removed from neo-go

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-21 12:50:20 +03:00
354a92ea2c [#602] blobovnicza: Add leaf width implementation
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 1m14s
Build / Build Components (1.21) (pull_request) Successful in 3m36s
DCO action / DCO (pull_request) Successful in 4m4s
Tests and linters / Staticcheck (pull_request) Successful in 4m5s
Tests and linters / Tests (1.20) (pull_request) Successful in 4m49s
Tests and linters / Lint (pull_request) Successful in 5m18s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m44s
Tests and linters / Tests with -race (pull_request) Successful in 5m47s
Build / Build Components (1.20) (pull_request) Successful in 10m45s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-21 10:27:32 +03:00
d3904ec599 [#602] config: Add blobovnicza leaf width parameter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-21 10:27:32 +03:00
e604a3d749 [#607] *: Use zap.Stringer() where possible
Semantic patch:
```
@@
var f expression
var t expression
var a expression
@@
 f(
    ...,
-    zap.String(t, a.String()),
+    zap.Stringer(t, a),
    ...,
)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 16:50:02 +00:00
58c8722c81 [#585] fstree: Add optional file counter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-16 14:47:44 +03:00
21800e9fcc [#162] core: Move literals to constants
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-08-11 15:48:42 +03:00
c54fcb297d [#248] cli,node: Remove obsolete network parameters
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-10 16:38:43 +00:00
4ad0ebb32f [#565] Add metrics for current GRPC endpoint status
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-10 16:37:49 +00:00
5b7e4a51b7 [#481] Update frostfs-sdk-go and error pointer receivers
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-09 10:26:53 +00:00
ae322e9f73 [#576] Set SyncWrites for badger writecache by default
Some checks failed
Build / Build Components (1.19) (pull_request) Failing after 3s
Build / Build Components (1.20) (pull_request) Failing after 2s
Tests and linters / Tests (1.19) (pull_request) Failing after 3s
Tests and linters / Tests with -race (pull_request) Failing after 3s
Tests and linters / Lint (pull_request) Failing after 12s
Tests and linters / Staticcheck (pull_request) Failing after 3s
Vulncheck / Vulncheck (pull_request) Successful in 5m46s
Tests and linters / Tests (1.20) (pull_request) Successful in 11m37s
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-09 11:28:26 +03:00
d3a52ec73a [#516] node: Send bootstrap request if attributes were updated
Consider following situation:
1. Epoch tick.
2. Update node attributes in the config.
3. Restart node.

Because we already sent bootstrap query after (1) and we are still
online, new bootstrap query won't be sent. This leads to the node
attributes being updated after another epoch.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-08 07:40:26 +00:00
5bbfebba2d [#570] Fix writecache type constant copy-pasta bug
Some checks failed
Build / Build Components (1.19) (pull_request) Failing after 3s
Build / Build Components (1.20) (pull_request) Failing after 3s
Tests and linters / Lint (pull_request) Failing after 4s
Tests and linters / Tests (1.19) (pull_request) Failing after 3s
Tests and linters / Tests (1.20) (pull_request) Failing after 3s
Tests and linters / Tests with -race (pull_request) Failing after 3s
Tests and linters / Staticcheck (pull_request) Failing after 3s
Vulncheck / Vulncheck (pull_request) Failing after 3s
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-07 12:06:44 +03:00
1a0cb0f34a [#421] Try using badger for the write-cache
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-07 08:16:57 +00:00
ec8a631d31 [#542] Update test for SizeInBytesSafe function
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-01 13:38:00 +00:00
9ca63ac8c3 [#542] Fix bug in SizeInBytesSafe function
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-01 13:38:00 +00:00
32c77f3a23 [#537] node: Add runtime.memory_limit config parameter
Some checks failed
Build / Build Components (1.20) (pull_request) Successful in 4m6s
Build / Build Components (1.19) (pull_request) Successful in 4m18s
ci/woodpecker/pr/pre-commit Pipeline failed
Tests and linters / Lint (pull_request) Failing after 2m10s
Tests and linters / Tests with -race (pull_request) Has started running
Tests and linters / Tests (1.19) (pull_request) Successful in 3m58s
Tests and linters / Tests (1.20) (pull_request) Successful in 3m44s
Vulncheck / Vulncheck (pull_request) Failing after 3m51s
Tests and linters / Staticcheck (pull_request) Successful in 8m3s
ci/woodpecker/push/pre-commit Pipeline was successful
This parameter allows to set soft memory limit for Go GC.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-25 17:27:09 +03:00
fef172c5b0 [#6] services/util: Simplify response.Service
It has only 1 parameter, which is obligatory.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-21 18:29:57 +03:00
f0355a453e [#463] policer: Remove capacity rebalance logic
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Current implementation has some quirks. For example,
using only half of object.put.pool_size_remote threads
tells replicator that is node is 50% loaded,
but in reality we could be putting lot's of big objects.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-18 10:52:12 +00:00
b2487e8cc5 [#516] node: Do not bootstrap if node is online candidate
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-18 10:51:02 +00:00
d8e37a827f [#497] config: Add examples and unit tests
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Add examples and unit tests for tree.authorized_keys section.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-17 08:23:28 +00:00
c83e7c875f [#294] searchsvcv2: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
e8091101c7 [#294] searchsvc: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
ec9b738465 [#294] putsvcv2: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
800a685e84 [#294] putsvc: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
1420b8b9ea [#294] deletesvc: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
a476d8285a [#294] deletesvcv2: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
70a1081988 [#294] aclsvcv2: Refactor service constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
61541eaec2 [#294] aclsvc: Refactor checker constructor
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
7b76527759 [#486] node: Add PutSingle wrappers
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-10 15:49:21 +03:00
e858479a74 [#498] policer: Explicitly Rewind() iterator after finish
Previously, we can continue to return `EndOfListing` infinitely.
Reflect iterator reuse via Rewind() method.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-07 14:52:33 +00:00
f9730f090d [#92] Refactor policer and add some unit tests
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-03 07:05:31 +00:00
0c5b025788 [#470] grpc: Increase message limits
All checks were successful
Build / Build Components (1.19) (pull_request) Successful in 3m27s
ci/woodpecker/pr/pre-commit Pipeline was successful
Build / Build Components (1.20) (pull_request) Successful in 3m25s
Tests and linters / Lint (pull_request) Successful in 2m15s
Tests and linters / Tests (1.19) (pull_request) Successful in 2m51s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m54s
Tests and linters / Tests with -race (pull_request) Successful in 6m31s
ci/woodpecker/push/pre-commit Pipeline was successful
For send message limit set to 2GiB, but there are custom
GET/GET RANGE limiters.
For receive message limit set to 256 MiB, but actual chunk size
will be managed by client.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-30 16:32:15 +03:00
8a4e250dae [#468] *: replace outdated TODO crypto-related links
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
73a71a71b0 [#1] node: Use a proper validation of a substorage type
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
cac4ed93d6 [#428] engine: Add low_mem config parameter
Concurrent initialization in case of the metabase resync leads to
high memory consumption and potential OOM.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-26 13:29:39 +00:00
71889234b7 [#449] tree: Allow reading requests signed by keys from allow list
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-23 14:44:19 +03:00
167a67f0b8 [#460] services/util: Remove HandleUnaryRequest
There is no need in a wrapper with many from-`interface{}` conversions.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-21 17:07:56 +03:00
b5d9f4a285 [#373] metrics: Add pilorama metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
e89fa110c7 [#373] metrics: Add metabase metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
af608da952 [#373] metrics: Add blobovnizca metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
56f320dd85 [#373] metrics: Add blobstor metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
16a142cd0c [#373] metrics: Add FSTree metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
059e9e88a2 [#373] metabase: Add metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
26b305f82b [#424] morph: Fix cache metrics
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Use separate morph cache metrics for node and IR

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 18:26:19 +03:00