TrueCloudLab/frostfs-node
19
1
Fork 27
Code Issues 91 Pull requests 7 Projects Releases 1 Activity Actions
3715 commits 8 branches 81 tags 52 MiB
Commit graph

3715 commits

This branch
This branch
All branches
Author SHA1 Message Date
Evgenii Stratonikov
f152f5d553 Release v0.36.0
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
ci/woodpecker/tag/pre-commit Pipeline was successful
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-12 18:15:46 +03:00
Dmitrii Stepanov
2ea3be33c1 [#493] node: Update Go version in go.mod 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 13:31:02 +03:00
Dmitrii Stepanov
de36cb9a7d [#493] node: Fix minimum Go version in description 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 13:30:53 +03:00
Dmitrii Stepanov
14886ae7a4 [#493] node: Set valid Go version in Docker files 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 13:30:42 +03:00
Dmitrii Stepanov
1cae03c47c [#452] engine: Set Disabled mode to deleted shard
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-19 19:04:06 +03:00
Dmitrii Stepanov
0252875aec [#425] node: Up api-go version
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-08 12:43:29 +03:00
Dmitrii Stepanov
25bedab91a [#425] object: Do not store large slices in pool
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details 
Dynamically growing an unbounded buffers can cause a large
amount of memory to be pinned and never be freed.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-02 17:48:24 +03:00
Evgenii Stratonikov
405e17b2ec [#409] node: Do not sent initial bootstrap under maintenance
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-29 18:42:25 +03:00
Evgenii Stratonikov
a7dab2a22b [#409] node: Fetch last bootstrap info on startup 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-29 18:42:25 +03:00
Evgenii Stratonikov
8aa5907e1b [#409] node: Log maintenance state on startup 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-29 18:42:25 +03:00
Evgenii Stratonikov
5e2fcec60f [#396] treesvc: properly remember last height on shutdown
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Previously `newHeight` was updated in parallel, so that applying
operation at height H did not imply successful TreeApply() for H-1.
And because we have no context in TreeUpdateLastSyncHeight(), invalid
starting height could be written if the context was canceled.

In this commit we return the new height only if all operations were
successfully applied.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-29 10:25:26 +00:00
Evgenii Stratonikov
139ded93e1 [#396] pilorama: Disallow applying same operations 
1. In redo() we save the old state.
2. If we do redo() for the same operation twice, the old state will be
   overritten with the new one.
3. This in turn affects undo() and subsequent isAncestor() check.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-29 10:25:26 +00:00
Dmitrii Stepanov
2360cf263b [#392] shard: Create tombstone source when reload
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-25 15:52:59 +03:00
Dmitrii Stepanov
f866ec1399 [#392] gc: Use defer to mark handler done 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-25 15:52:49 +03:00
Dmitrii Stepanov
a506da97d6 [#384] shard: Add unit test
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Add test to check that oject not found error will be returned,
if object doesn't exist in blobstore.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-25 09:55:16 +03:00
Dmitrii Stepanov
1dd84eef77 [#384] shard: Cancel GC if change mode requested 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-25 09:55:16 +03:00
Anton Nikiforov
1501f11e4d [#351] cli: Support copies number parameter in object put
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-05-18 15:52:51 +00:00
Anton Nikiforov
4f55417914 [#351] Fix end of files 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-05-18 15:52:51 +00:00
Dmitrii Stepanov
9bda6e0b8b [#332] gc: Fix expired complex object deletion
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-18 12:51:34 +00:00
Evgenii Stratonikov
ceb9deb7f1 [#337] morph: Move subscription logic to subscriber
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-18 11:28:50 +03:00
Evgenii Stratonikov
4148590668 [#365] go.mod: Update api-go
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-18 09:51:07 +03:00
Alejandro Lopez
493cafc62a [#355] Increase tree svc client cache size to test hypotheses
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-05-17 15:17:35 +03:00
Evgenii Stratonikov
3711976dfc [#314] writecache: remove objects right after they are flushed
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-16 14:16:25 +03:00
Pavel Karpy
c3f5045842 [#314] wc: Do not lose small objects on disk errors 
Do return error if an object could not been stored on WC's disk.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-05-15 16:27:44 +03:00
Pavel Karpy
ab65063d6d [#314] wc: Simplify background workers naming 
Also, drop not used arg.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-05-15 16:27:42 +03:00
Pavel Karpy
c60029d3b0 [#323] node: Fix tree svc panic
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
If a connection has not been established earlier, it stores `nil` in LRU
cache. Cache eviction tries to close every connection (even a `nil` one) and
panics but not crash the app because we are using pools.
That ugly bug also leads to a deadlock where `Unlock` is not called via
`defer` func (and that is the way I found it).

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-05-04 20:04:30 +03:00
Pavel Karpy
0beb7ccf5c [#284] node: Use copy_number on server side
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-26 10:57:34 +03:00
Dmitrii Stepanov
0fe5e34fb0 [#231] node: Fix race condition in TTL cache
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Use key locker to lock by key.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-25 16:50:27 +03:00
Dmitrii Stepanov
bcf3f0f517 [#231] node: Invalidate container cache on PutSuccess event
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details 
For example: frostfs-cli creates container and makes polling
GetContainer requests. These requests go through container cache,
so not found error stores in container cache.
So container cache can contain not found error when PutSuccess event received.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-20 17:34:00 +03:00
Evgenii Stratonikov
79d59e4ed2 [#266] services/tree: Do not accept requests until initial sync is finished
Some checks failed
ci/woodpecker/pr/pre-commit Pipeline failed
Details
ci/woodpecker/push/pre-commit Pipeline failed
Details 
`Apply` is deliberately left out -- we don't want to miss anything new.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-18 13:30:45 +03:00
Evgenii Stratonikov
364b4ac572 [#266] services/tree: Batch operations on the service level 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-18 13:30:45 +03:00
Evgenii Stratonikov
f7679a8168 [#266] services/tree: Return operation log up to some height 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-18 13:30:45 +03:00
Evgenii Stratonikov
2dc2fe8780 [#266] pilorama: Allow to get current tree height 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-18 13:30:45 +03:00
Dmitrii Stepanov
21412ef24a [#263] node: Up api-go version
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Fix panic in tracing.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-18 11:45:42 +03:00
Evgenii Stratonikov
7d39fecc6a Release v0.36.0
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-12 16:57:02 +03:00
Evgenii Stratonikov
04727ce1d6 Revert "[#135] signature: Add tracing" 
This reverts commit 5778980252.
 2023-04-12 16:57:02 +03:00
Evgenii Stratonikov
08769f413f Revert "[#135] acl: Add tracing spans" 
This reverts commit b2ca730547.
 2023-04-12 16:54:13 +03:00
Evgenii Stratonikov
5d2affa5cd testutil: Fix linter warning 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-12 11:00:02 +03:00
Dmitrii Stepanov
5778980252 [#135] signature: Add tracing
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Add tracing to verify request and sign response.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Dmitrii Stepanov
b2ca730547 [#135] acl: Add tracing spans 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Dmitrii Stepanov
0920d848d0 [#135] get-object: Add tracing spans 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Dmitrii Stepanov
5af9f58469 [#135] tracing: Add tracing to node gRPC endpoints 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Dmitrii Stepanov
72565a91ef [#135] node: Update api-go version 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Evgenii Stratonikov
c4865783fc [#236] blobstor/test: Prefill storage in parallel in read benchmark
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
`blobovniczatree` takes a really long time to prefill, because each
batch takes at least 10ms, so for 10k iterations we have at least 100s of
prefill.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-11 20:52:11 +03:00
Evgenii Stratonikov
6ad5c38225 [#236] testutil: Use random object id in RandObjGenerator 
Before this commit it was like this:
```
BenchmarkSubstorageWritePerf/memstore-rand10-8            227425              4859 ns/op
BenchmarkSubstorageWritePerf/fstree_nosync-rand10-8     --- FAIL: BenchmarkSubstorageWritePerf/fstree_nosync-rand10-8
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
BenchmarkSubstorageWritePerf/fstree-rand10-8            --- FAIL: BenchmarkSubstorageWritePerf/fstree-rand10-8
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-11 20:52:04 +03:00
Evgenii Stratonikov
c85a0bc866 [#236] blobstor/test: Reduce test descriptions 
I tried to add 4 more tests and suddenly, it became harder to navigate in
code. Move directory creation in a common function.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-11 20:52:04 +03:00
Evgenii Stratonikov
6bf11f7cca [#230] CHANGELOG.md: Remove older entries
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-11 10:07:27 +00:00
Dmitrii Stepanov
2c07f831c7 [#223] node: Refactor cache usage
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Drop excess type args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-10 14:16:34 +03:00
Dmitrii Stepanov
93eba19a8e [#223] objectsvc: Refactor split-tree traverse 
Resolve funlen & gocognit linters for traverseSplitChain method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-10 14:16:34 +03:00
Dmitrii Stepanov
2ed9fd3f94 [#223] objectsvc: Refactor request parameters 
Resolve containedctx linter for commonPrm.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-10 14:16:34 +03:00