TrueCloudLab
/
frostfs-node
17
1
Fork 22
Code Issues 99 Pull Requests 7 Actions Projects Releases 1 Activity
4,391 Commits
9 Branches
38 Tags
52 MiB
Commit Graph

994 Commits (19937c67f8968e229479ea0029a1129a745c1a65)

Author SHA1 Message Date
Dmitrii Stepanov 63d3ed1ad8 [#904] tests: Close test engine after test
DCO action / DCO (pull_request) Successful in 2m7s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m37s Details
Build / Build Components (1.20) (pull_request) Successful in 4m0s Details
Build / Build Components (1.21) (pull_request) Successful in 3m56s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m25s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m22s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m43s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m47s Details
Tests and linters / Lint (pull_request) Successful in 7m5s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-17 19:04:39 +03:00
Dmitrii Stepanov 57171907e3 [#904] metabase: Return if object was actuall inserted 
This requires to count metrics properly.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-17 19:04:39 +03:00
Dmitrii Stepanov c1a80235db [#904] metabase: Log Inhume operation 
It will be very useful for troubleshooting.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-17 18:42:52 +03:00
Dmitrii Stepanov a2ab373a0a [#895] metabase: Do not delete GC mark for virtual objects 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Dmitrii Stepanov 7166e77c2b [#895] test: Add logger to test shard 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Dmitrii Stepanov 47dcfa20f3 [#895] test: Use t.Cleanup only for external resources 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Dmitrii Stepanov f1b2b8bffa [#895] test: Fix NewLogger arguments list 
`debug` is always true.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Dmitrii Stepanov 4b8b4da681 [#864] engine: Drop container count metric if container removed
DCO action / DCO (pull_request) Successful in 1m30s Details
Build / Build Components (1.21) (pull_request) Successful in 3m29s Details
Build / Build Components (1.20) (pull_request) Successful in 3m53s Details
Tests and linters / Lint (pull_request) Successful in 4m31s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m1s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m51s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m13s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m34s Details
Vulncheck / Vulncheck (pull_request) Successful in 58s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-10 10:45:32 +03:00
Dmitrii Stepanov d75e7e9a21 [#864] engine: Drop container size metric if container deleted 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-10 10:44:54 +03:00
Dmitrii Stepanov dfd62ca6b1 [#864] metabase: Refactor delete/inhume 
Available -> Logic, Raw -> Phy for delete/inhume results.
Use single counter instead of vectors.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-09 09:59:42 +03:00
Evgenii Stratonikov 225fe2d4d5 [#894] blobovniczatree: Speedup rebuild test 
Down from 3s to 300ms.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-12-29 16:28:54 +00:00
Dmitrii Stepanov 581887148a [#569] cli: Add `control shards writecache seal` command
DCO action / DCO (pull_request) Successful in 2m46s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m3s Details
Build / Build Components (1.21) (pull_request) Successful in 3m53s Details
Build / Build Components (1.20) (pull_request) Successful in 4m0s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m31s Details
Tests and linters / Lint (pull_request) Successful in 6m17s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m47s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m14s Details
Tests and linters / Tests with -race (pull_request) Successful in 13m12s Details 
It does the same as `control shards flush-writecache --seal`, but
has better name.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 16:05:37 +03:00
Dmitrii Stepanov 7a9db5bcdd [#569] writecache: Do not wait modeMtx if mode changes 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 16:05:37 +03:00
Dmitrii Stepanov 32c282ca10 [#569] writecache: Refactor flush 
Make single RUnlock call instead of two.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 16:05:37 +03:00
Dmitrii Stepanov 0cb0fc1735 [#569] writecache: Allow to seal writecache after flush 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 16:05:37 +03:00
Dmitrii Stepanov 8180a0664f [#887] node: Drop badger writecache implementation
Vulncheck / Vulncheck (pull_request) Successful in 6m1s Details
Build / Build Components (1.21) (pull_request) Successful in 7m37s Details
Build / Build Components (1.20) (pull_request) Successful in 7m52s Details
Tests and linters / Staticcheck (pull_request) Successful in 8m56s Details
Tests and linters / Lint (pull_request) Successful in 9m26s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 15m5s Details
Tests and linters / Tests with -race (pull_request) Successful in 15m7s Details
DCO action / DCO (pull_request) Successful in 1m1s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 4m1s Details 
Badger implementation isn't tested and works not well,
but requires human resources to maintain.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-22 13:00:54 +03:00
Anton Nikiforov d9cbb16bd3 [#866] Use TTL for blobovnicza tree cache 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-12-19 16:36:28 +00:00
Dmitrii Stepanov 7eb46404a1 [#863] blobovnicza: Fix counters
Vulncheck / Vulncheck (pull_request) Successful in 2m27s Details
DCO action / DCO (pull_request) Successful in 2m39s Details
Build / Build Components (1.21) (pull_request) Successful in 3m38s Details
Build / Build Components (1.20) (pull_request) Successful in 3m45s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m15s Details
Tests and linters / Lint (pull_request) Successful in 5m47s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m49s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 12m16s Details
Tests and linters / Tests with -race (pull_request) Successful in 12m31s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-13 13:34:29 +03:00
Dmitrii Stepanov 94ffe8bb45 [#857] golangci: Add testifylint linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 16:27:02 +03:00
Dmitrii Stepanov 3b7c0362a8 [#861] shard: Fix Delete object
DCO action / DCO (pull_request) Successful in 12m22s Details
Build / Build Components (1.20) (pull_request) Successful in 12m44s Details
Build / Build Components (1.21) (pull_request) Successful in 13m2s Details
Vulncheck / Vulncheck (pull_request) Successful in 13m3s Details
Tests and linters / Staticcheck (pull_request) Successful in 15m32s Details
Tests and linters / Lint (pull_request) Successful in 16m26s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 26m11s Details
Tests and linters / Tests with -race (pull_request) Successful in 28m29s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 3m28s Details 
It is possible that object doesn't exist in metabase.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 14:25:40 +03:00
Dmitrii Stepanov 681b2c5fd4 [#825] policer: Do not drop required linking objects 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 11:04:03 +00:00
Dmitrii Stepanov db49ad16cc [#826] blobovniczatree: Do not create DB's on init
DCO action / DCO (pull_request) Successful in 4m10s Details
Build / Build Components (1.21) (pull_request) Successful in 7m30s Details
Vulncheck / Vulncheck (pull_request) Successful in 7m5s Details
Tests and linters / Staticcheck (pull_request) Successful in 10m6s Details
Tests and linters / Lint (pull_request) Successful in 10m26s Details
Build / Build Components (1.20) (pull_request) Successful in 12m39s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 16m55s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 17m13s Details
Tests and linters / Tests with -race (pull_request) Successful in 17m14s Details 
Blobovniczas will be created on write requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov ad0697adc4 [#661] blobovnicza: Compute size with record size 
To get more accurate size of blobovnicza use record
size (lenght of key + lenght of data).

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov e54dc3dc7c [#698] blobovnicza: Store counter values 
Blobovnicza initialization take a long time because of bucket
Stat() call. So now blobovnicza stores counters in META bucket.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov 5e8c08da3e [#661] blobstore: Add address to error logs 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov 8911656b1a [#661] metrcis: Add rebuild percent metric 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov 2407e5f5ff [#661] blobovniczatree: Do not sort DB's and indicies 
Put stores object to next active DB, so there is no need to sort DBs.
In addition, it adds unnecessary DB openings.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov c6a739e746 [#661] blobovniczatree: Make Rebuild concurrent for objects 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov f1c7905263 [#661] blobovniczatree: Make Rebuild concurrent 
Different DBs can be rebuild concurrently.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov d4d905ecc6 [#661] metrics: Add blobovniczatree rebuild metrics 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov b2769ca3de [#661] blobovniczatree: Make Rebuild failover safe 
Now move info stores in blobovnicza, so in case of failover
rebuild completes previous operation first.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov da4fee2d0b [#698] blobovniczatree: Init blobovniczas concurrently 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov 422226da18 [#661] blobovniczatree: Add Rebuild implementation 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:32 +03:00
Dmitrii Stepanov a531eaf8bc [#661] blobstor: Add Rebuild implementation 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:32 +03:00
Dmitrii Stepanov c1667a11d2 [#661] blobovniczatree: Allow to change depth or width 
Now it is possible to change depth or with of blobovniczatree.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:32 +03:00
Dmitrii Stepanov 484eb59893 [#661] blobovniczatree: Use .db extension for db files 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:32 +03:00
Dmitrii Stepanov 44552a849b [#661] shard: Add blobstor rebuilder 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:32 +03:00
Dmitrii Stepanov a478050639 [#838] metabase: Resolve funlen linter
DCO action / DCO (pull_request) Successful in 1m59s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m0s Details
Build / Build Components (1.21) (pull_request) Successful in 3m40s Details
Tests and linters / Lint (pull_request) Successful in 4m57s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m38s Details
Build / Build Components (1.20) (pull_request) Successful in 5m17s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m1s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m39s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m46s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-06 15:44:21 +03:00
Dmitrii Stepanov d30ab5f29e [#838] metabase: Count user objects 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-06 15:44:21 +03:00
Dmitrii Stepanov f314da4af3 [#838] metabase: Add user object type counter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-06 15:44:21 +03:00
Dmitrii Stepanov 29550fe600 [#838] shard: Refactor `updateMetrics` method 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-06 15:44:21 +03:00
Alexander Chuprov b2c63e57ba [#651] engine/test: Speedup StorageEngine_Inhume 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-11-30 13:19:43 +00:00
Alexander Chuprov 445ebcc0e7 [#651] shard/test: Speedup Shard_Delete 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-11-30 13:19:43 +00:00
Alexander Chuprov 2302e5d342 [#651] shard/test: Refactor Shard_Delete 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-11-30 13:19:43 +00:00
Dmitrii Stepanov 07390ad4e3 [#715] node: Unify config parameter names
DCO action / DCO (pull_request) Successful in 2m54s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m11s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m55s Details
Build / Build Components (1.21) (pull_request) Successful in 3m51s Details
Build / Build Components (1.20) (pull_request) Successful in 4m6s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m10s Details
Tests and linters / Lint (pull_request) Successful in 5m24s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m56s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m46s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-22 17:13:50 +03:00
Dmitrii Stepanov 8d18fa159e [#667] writecache: Fix flush test
DCO action / DCO (pull_request) Successful in 1m28s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m55s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m15s Details
Build / Build Components (1.21) (pull_request) Successful in 4m40s Details
Build / Build Components (1.20) (pull_request) Successful in 4m54s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m32s Details
Tests and linters / Lint (pull_request) Successful in 5m59s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m59s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m57s Details 
Allow to disable background flusher for testing purposes.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-17 17:45:43 +03:00
Dmitrii Stepanov 76ff26039c [#96] node: Drop neo-go's `slices` package
DCO action / DCO (pull_request) Successful in 2m13s Details
Build / Build Components (1.20) (pull_request) Successful in 5m48s Details
Build / Build Components (1.21) (pull_request) Successful in 5m47s Details
Vulncheck / Vulncheck (pull_request) Successful in 5m47s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m33s Details
Tests and linters / Staticcheck (pull_request) Successful in 7m37s Details
Tests and linters / Lint (pull_request) Successful in 8m8s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 10m38s Details
Tests and linters / Tests with -race (pull_request) Successful in 11m22s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-17 13:24:04 +03:00
Airat Arifullin 47286ebf32 [#805] pilorama: Fix TreeDrop 
* If treeID is empty then deleting buckets for cursor may get
  invalidated. So, buckets should be gathered before deleting.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-11-17 10:21:35 +00:00
Dmitrii Stepanov 29fe8c41f3 [#655] storage: Drop `ErrorHandler`
DCO action / DCO (pull_request) Successful in 1m42s Details
Build / Build Components (1.21) (pull_request) Successful in 3m23s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m25s Details
Build / Build Components (1.20) (pull_request) Successful in 4m16s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m24s Details
Tests and linters / Lint (pull_request) Successful in 5m2s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m36s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m42s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m18s Details 
The only one usage was for logging.
Now logging performed by storage anyway.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-16 17:27:38 +03:00
Dmitrii Stepanov 137e987a4e [#655] storage: Drop `LazyHandler` 
LazyHandler is implemented and used incorrectly.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-16 17:27:38 +03:00