TrueCloudLab/frostfs-node
21
1
Fork 27
Code Issues 104 Pull requests 4 Projects Releases 2 Activity Actions
4322 commits 9 branches 96 tags 51 MiB
Commit graph

49 commits

Author SHA1 Message Date
Dmitrii Stepanov
e54dc3dc7c [#698] blobovnicza: Store counter values 
Blobovnicza initialization take a long time because of bucket
Stat() call. So now blobovnicza stores counters in META bucket.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov
b2769ca3de [#661] blobovniczatree: Make Rebuild failover safe 
Now move info stores in blobovnicza, so in case of failover
rebuild completes previous operation first.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov
422226da18 [#661] blobovniczatree: Add Rebuild implementation 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:32 +03:00
Dmitrii Stepanov
a531eaf8bc [#661] blobstor: Add Rebuild implementation 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:32 +03:00
Dmitrii Stepanov
484eb59893 [#661] blobovniczatree: Use .db extension for db files 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:32 +03:00
Dmitrii Stepanov
44552a849b [#661] shard: Add blobstor rebuilder 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:32 +03:00
Dmitrii Stepanov
26e4f7005c [#741] treesvc: Refactor tree sync 
Fix linter issues.
Add error logging.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-30 12:45:02 +00:00
Dmitrii Stepanov
b215817e14 [#741] treesvc: Remove unused height variables 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-30 12:45:02 +00:00
Dmitrii Stepanov
8d18fa159e [#667] writecache: Fix flush test
All checks were successful
DCO action / DCO (pull_request) Successful in 1m28s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m55s
Details
Tests and linters / Staticcheck (pull_request) Successful in 4m15s
Details
Build / Build Components (1.21) (pull_request) Successful in 4m40s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m54s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m32s
Details
Tests and linters / Lint (pull_request) Successful in 5m59s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m59s
Details
Tests and linters / Tests with -race (pull_request) Successful in 6m57s
Details 
Allow to disable background flusher for testing purposes.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-17 17:45:43 +03:00
aarifullin
4d5be5ccb5 [#811] ape: Update policy-engine module version and rebase
All checks were successful
DCO action / DCO (pull_request) Successful in 4m23s
Details
Vulncheck / Vulncheck (pull_request) Successful in 5m31s
Details
Build / Build Components (1.21) (pull_request) Successful in 7m33s
Details
Build / Build Components (1.20) (pull_request) Successful in 7m40s
Details
Tests and linters / Staticcheck (pull_request) Successful in 8m22s
Details
Tests and linters / Lint (pull_request) Successful in 9m23s
Details
Tests and linters / Tests with -race (pull_request) Successful in 11m20s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 11m32s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m41s
Details 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-11-16 11:31:37 +03:00
Dmitrii Stepanov
fd9128d051 [#800] node: eACL -> APE converter
All checks were successful
DCO action / DCO (pull_request) Successful in 3m43s
Details
Vulncheck / Vulncheck (pull_request) Successful in 6m9s
Details
Build / Build Components (1.21) (pull_request) Successful in 6m53s
Details
Build / Build Components (1.20) (pull_request) Successful in 7m18s
Details
Tests and linters / Staticcheck (pull_request) Successful in 7m36s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m1s
Details
Tests and linters / Lint (pull_request) Successful in 8m15s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m59s
Details
Tests and linters / Tests with -race (pull_request) Successful in 10m3s
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-15 11:55:55 +03:00
Dmitrii Stepanov
70ab1ebd54 [#763] metrics: Add container_objects_total metric
All checks were successful
DCO action / DCO (pull_request) Successful in 3m54s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m58s
Details
Build / Build Components (1.21) (pull_request) Successful in 5m16s
Details
Vulncheck / Vulncheck (pull_request) Successful in 9m54s
Details
Tests and linters / Lint (pull_request) Successful in 10m57s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 12m40s
Details
Tests and linters / Staticcheck (pull_request) Successful in 12m34s
Details
Tests and linters / Tests with -race (pull_request) Successful in 12m48s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 13m19s
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-08 12:30:57 +03:00
Anton Nikiforov
b0cf100427 [#49] node: React on SIGHUP only when node in READY state 
Add more info in logs when node is going to shut down,
but initialization process still in progress.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-10-24 15:55:29 +00:00
Anton Nikiforov
627b302745 [#709] node: Put in log info about listening endpoints 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-10-02 13:31:10 +00:00
Evgenii Stratonikov
c6af4a3ec8 [#679] engine: Do not increase error counter on meta mismatch
All checks were successful
DCO action / DCO (pull_request) Successful in 1m43s
Details
Vulncheck / Vulncheck (pull_request) Successful in 2m0s
Details
Tests and linters / Staticcheck (pull_request) Successful in 2m54s
Details
Build / Build Components (1.20) (pull_request) Successful in 2m49s
Details
Build / Build Components (1.21) (pull_request) Successful in 2m44s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 3m43s
Details
Tests and linters / Lint (pull_request) Successful in 3m58s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 23m28s
Details
Tests and linters / Tests with -race (pull_request) Successful in 29m0s
Details 
It was introduced in 69e1e6ca to help node determine faulty shards.
However, the situation is possible in a real-life scenario:
1. Object O is evacuated from shard A to B.
2. Shard A is unmounted because of lower-level errors.
3. We now have object in meta on A and in blobstor on B. Technically we
   have it in meta on shard B too, but we still got the error if B goes
   to a degraded mode.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-09-14 10:39:18 +03:00
Anton Nikiforov
88d50e4c77 [#656] policer: Add "bad" testcase 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-09-06 08:04:59 +00:00
Dmitrii Stepanov
c672f59ab8 [#536] blobovnicza: Drop cache 
Each blobovnicza instance is opened
while is in use.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-09-01 13:51:26 +03:00
Anton Nikiforov
4dff9555f1 [#568] writecache: Improve flushing scheme for badger 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-08-30 17:22:28 +00:00
Airat Arifullin
4ea0df77d0 [#574] policer: Check if the container was really removed 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-08-28 14:21:38 +00:00
Evgenii Stratonikov
f8ba60aa0c [#648] objsvc/delete: Handle errors in Go style 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-25 09:45:35 +00:00
Evgenii Stratonikov
d2084ece41 [#648] objsvc/delete: Remove redundant logs 
We never propagate delete requests to the container node, because
tombstone broadcast is done via PUT. No need to pollute logs.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-25 09:45:35 +00:00
Airat Arifullin
23be3eb627 [#574] tree: Check if container is really removed
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 2m37s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m36s
Details
Tests and linters / Staticcheck (pull_request) Successful in 3m36s
Details
DCO action / DCO (pull_request) Successful in 4m16s
Details
Tests and linters / Lint (pull_request) Successful in 5m34s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m36s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m46s
Details
Tests and linters / Tests with -race (pull_request) Successful in 6m21s
Details
Build / Build Components (1.20) (pull_request) Successful in 8m1s
Details 
* Use DeletionInfo method from morph client to check if
  the container has been really removed from neo-go

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-08-21 12:50:20 +03:00
Evgenii Stratonikov
62c2ad4b22 [#626] logs: Remove autogenerated comments
All checks were successful
DCO action / DCO (pull_request) Successful in 2m38s
Details
Vulncheck / Vulncheck (pull_request) Successful in 4m25s
Details
Build / Build Components (1.21) (pull_request) Successful in 5m25s
Details
Build / Build Components (1.20) (pull_request) Successful in 5m58s
Details
Tests and linters / Staticcheck (pull_request) Successful in 6m24s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m4s
Details
Tests and linters / Lint (pull_request) Successful in 8m27s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m19s
Details
Tests and linters / Tests with -race (pull_request) Successful in 8m44s
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-21 11:15:06 +03:00
Dmitrii Stepanov
eec97d177e [#585] writecache: Count items periodically 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-16 14:47:44 +03:00
Alexander Chuprov
21800e9fcc [#162] core: Move literals to constants 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-08-11 15:48:42 +03:00
Evgenii Stratonikov
32a9f51586 [#562] logs: Remove superfluous comment 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-10 16:38:43 +00:00
Alejandro Lopez
1a0cb0f34a [#421] Try using badger for the write-cache 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-07 08:16:57 +00:00
Dmitrii Stepanov
32c77f3a23 [#537] node: Add runtime.memory_limit config parameter
Some checks failed
Build / Build Components (1.20) (pull_request) Successful in 4m6s
Details
Build / Build Components (1.19) (pull_request) Successful in 4m18s
Details
ci/woodpecker/pr/pre-commit Pipeline failed
Details
Tests and linters / Lint (pull_request) Failing after 2m10s
Details
Tests and linters / Tests with -race (pull_request) Has started running
Details
Tests and linters / Tests (1.19) (pull_request) Successful in 3m58s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 3m44s
Details
Vulncheck / Vulncheck (pull_request) Failing after 3m51s
Details
Tests and linters / Staticcheck (pull_request) Successful in 8m3s
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
This parameter allows to set soft memory limit for Go GC.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-25 17:27:09 +03:00
Dmitrii Stepanov
f0355a453e [#463] policer: Remove capacity rebalance logic
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Current implementation has some quirks. For example,
using only half of object.put.pool_size_remote threads
tells replicator that is node is 50% loaded,
but in reality we could be putting lot's of big objects.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-18 10:52:12 +00:00
Dmitrii Stepanov
b2487e8cc5 [#516] node: Do not bootstrap if node is online candidate
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-18 10:51:02 +00:00
Dmitrii Stepanov
01a0c97760 [#453] engine: Set Disabled mode to deleted shard
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-20 12:04:07 +03:00
Dmitrii Stepanov
4f83ab0fb4 [#409] node: Do not sent initial bootstrap under maintenance
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-13 10:02:36 +00:00
Airat Arifullin
a3e30062df [#325] node: Introduce unsafe_disable param to disable policer
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-05-31 10:09:05 +00:00
Dmitrii Stepanov
2613351008 [#387] gc: Cancel GC is change mode requested
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-25 09:38:16 +03:00
Dmitrii Stepanov
e4889e06ba [#329] node: Make evacuate async 
Now it's possible to run evacuate shard in async.
Also only one evacuate process can be in progress.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-19 08:43:52 +00:00
Anton Nikiforov
6f47c75e43 [#125] ir: Set extra wallets on SIGHUP
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-05-16 12:47:43 +00:00
Dmitrii Stepanov
869fcbf591 [#332] gc: Fix expired complex object deletion
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-16 12:44:57 +00:00
Roman Khimov
a5f118a987 [#337] subscriber: Drop unused UnsubscribeForNotification 
It's not really needed, closing the connection works fine when exiting and
normally the app doesn't need to unsubscribe at all.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-12 09:19:38 +00:00
Dmitrii Stepanov
a181c9e434 [#332] gc: Add additional logging
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-10 17:34:40 +03:00
Anton Nikiforov
cedd07bbc8 [#304] morph: Iterate endpoints when create ws client in constructor
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-05-05 10:07:30 +03:00
Dmitrii Stepanov
eca5c210dd [#299] evacuate: Add context cancel checks
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-02 15:55:30 +03:00
Evgenii Stratonikov
b2bc1fccbc [#248] logs: Remove unused messages 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-28 17:00:43 +03:00
Dmitrii Stepanov
7b981bfe97 [#249] logs: Drop unused consts
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-17 09:46:02 +03:00
Pavel Karpy
b453bb754c [#247] logs: Drop reputation log messages
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/pre-commit Pipeline was successful
Details 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-14 14:44:14 +03:00
Alejandro Lopez
41eb3129ae [#139] Refactor blobovnicza exist test to not use chmod
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-04-14 12:16:14 +03:00
Evgenii Stratonikov
cffcc7745e [#240] logs: Factor out common service log messages
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-14 05:06:09 +00:00
Evgenii Stratonikov
0e31c12e63 [#240] logs: Move log messages to constants 
Drop duplicate entities.
Format entities.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-14 05:06:09 +00:00
Stanislav Bogatyrev
b7b5079934 Add Inner Ring code 2020-07-24 17:07:37 +03:00
alexvanin
dadfd90dcd Initial commit 
Initial public review release v0.10.0
 2020-07-10 17:45:00 +03:00