realloc/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Releases Packages Activity
5500 commits 1 branch 26 tags 15 MiB
Commit graph

422 commits

Author SHA1 Message Date
Dmitrii Stepanov
fd37cea443
[#1700] engine: Drop unused block execution methods 
`BlockExecution` and `ResumeExecution` were used only by unit test.
So drop them and simplify code.

Change-Id: Ib3de324617e8a27fc1f015542ac5e94df5c60a6e
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-04-14 14:35:15 +03:00
Dmitrii Stepanov
e06ecacf57
[#1705] engine: Use condition var for evacuation unit tests 
To know exactly when the evacuation was completed,
a conditional variable was added.

Closes #1705

Change-Id: I86f6d7d2ad2b9759905b6b5e9341008cb74f5dfd
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-04-11 09:20:46 +03:00
Alexander Chuprov
4f9d237042
[#1689] linter: Fix staticcheck warning: 'probably want to use time.Time.Equal instead' 
Change-Id: Idb119d3f4f167c9e42ed48633d301185589553ed
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-04-09 16:55:35 +03:00
Alexander Chuprov
6f7b6b65f3 [#1689] linter: Fix staticcheck warning: 'embedded field can be simplified' 
Change-Id: I8f454f7d09973cdea096495c3949b88cdd01102e
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-04-08 09:32:24 +00:00
Alexander Chuprov
b4b053cecd
[#1679] node: Fix 'gocognit' warning 
Change-Id: I6e2a278af51869c05c306c2910ba85130e39532e
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-04-07 14:10:25 +03:00
Vitaliy Potyarkin
5350632e01 [#1705] engine/test: Increase evacuation timeout 
This test was flaky in CI probably because of runner load fluctuations.
Let's increase the timeout and see if the flakiness goes away.

(close #1705)

Change-Id: I76f96e3d6f4adb3d5de0e27b8ee6b47685236277
Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
 2025-04-04 14:41:56 +00:00
Dmitrii Stepanov
27899598dc
[#1700] gc: Drop Event interface 
There is only one event: new epoch.

Change-Id: I982f3650f7bc753ff2782393625452f0f8cdcc35
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-04-01 14:23:02 +03:00
Dmitrii Stepanov
bc6cc9ae2a
[#1700] engine: Print stacks on test request limiter 
Change-Id: I4952769ca431d1049955823b41b99b0984b385fc
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-04-01 14:23:02 +03:00
Airat Arifullin
a11b2d27e4 [#1642] tree: Introduce Cursor type 
* Use `Cursor` as parameter for `TreeSortedByFilename`

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2025-03-20 10:12:49 +00:00
Evgenii Stratonikov
bcc84c85a0 [#1671] Replace interface{} with any 
gopatch:
```
@@
@@
-interface{}
+any
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2025-03-13 08:12:20 +00:00
Dmitrii Stepanov
2005fdda09
[#1667] shard: Drop shard pool 
After adding an ops limiter, shard's `put` pool is redundant.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-03-11 13:59:51 +03:00
Dmitrii Stepanov
3727d60331 [#1653] qos: Add metrics 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-03-11 10:57:47 +00:00
Ekaterina Lebedeva
9e31cb249f [#1635] control: Add method to search shards by object 
Added method `ListShardsForObject` to ControlService and to
StorageEngine. It returns information about shards storing
object on the node.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2025-03-07 14:32:01 +03:00
Dmitrii Stepanov
4685afb1dc
[#1636] engine: Validate limiter release in unit tests 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 17:25:30 +03:00
Dmitrii Stepanov
c2d855aedd
[#1636] qos: Return Resource Exhausted error 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 17:25:28 +03:00
Dmitrii Stepanov
ceff5e1f6a
[#1636] storage: Refactor shard rebuild 
Drop redundant interfaces.
Rename fields.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 17:25:27 +03:00
Dmitrii Stepanov
e0dc3c3d0c
[#1636] shard: Add limiter usage 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 17:25:26 +03:00
Dmitrii Stepanov
0991077cb3 [#1657] engine: Fix data race in evacuation tests 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-02-28 14:14:12 +00:00
Alexander Chuprov
9b113c3156
[#1613] morph: Add tracing for morph queries to neo-go 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2025-02-05 16:38:20 +03:00
Dmitrii Stepanov
e1a984e9d8
[#1620] logs: Drop redundant trace_id fields 
`trace_id` is taken from context.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-01-29 16:13:51 +03:00
Evgenii Stratonikov
c98357606b
[#1606] Use slices.Clone()/bytes.Clone() where possible 
gopatch:
```
@@
var from, to expression
@@
+import "bytes"
-to := make([]byte, len(from))
-copy(to, from)
+to := bytes.Clone(from)

@@
var from, to expression
@@
+import "bytes"
-to = make([]byte, len(from))
-copy(to, from)
+to = bytes.Clone(from)

@@
var from, to, typ expression
@@
+import "slices"
-to := make([]typ, len(from))
-copy(to, from)
+to := slices.Clone(from)

@@
var from, to, typ expression
@@
+import "slices"
-to = make([]typ, len(from))
-copy(to, from)
+to = slices.Clone(from)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2025-01-17 14:50:14 +03:00
Dmitrii Stepanov
eff95bd632
[#1598] engine: Drop unnecessary result structs 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-01-14 11:15:21 +03:00
Dmitrii Stepanov
fb928616cc
[#1598] golangci: Enable unparam linter 
To drop unnecessary parameters and return values.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-01-14 09:06:47 +03:00
Dmitrii Stepanov
4d5ae59a52
[#1598] golangci: Enable unconvert linters 
To drop unnecessary conversions.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-01-14 09:06:40 +03:00
Anton Nikiforov
6c51f48aab [#1596] metrics: Create public aliases for internal engine metrics 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2025-01-13 10:05:01 +00:00
Evgenii Stratonikov
bd0197eaa8 [#1568] storage: Remove "could not/can't/failed to" from error messages 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-12-18 15:52:26 +00:00
Aleksey Savchuk
f0c43c8d80
[#1502] Use zap.Error for logging errors 
Use `zap.Error` instead of `zap.String` for logging errors: change all expressions like
`zap.String("error", err.Error())` or `zap.String("err", err.Error())` to `zap.Error(err)`.
Leave similar expressions with other messages unchanged, for example,
`zap.String("last_error", lastErr.Error())` or `zap.String("reason", ctx.Err().Error())`.

This change was made by applying the following patch:
```diff
@@
var err expression
@@
-zap.String("error", err.Error())
+zap.Error(err)

@@
var err expression
@@
-zap.String("err", err.Error())
+zap.Error(err)
```

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-12-16 11:13:42 +03:00
Aleksey Savchuk
3821645085
[#1555] engine: Refactor (*StorageEngine).GetLocks 
Refactored after renaming the method to replace the confusing `locked`
variable with `locks`.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-12-11 15:06:38 +03:00
Aleksey Savchuk
72470d6b48
[#1555] local_object_storage: Rename method GetLocked -> GetLocks 
Renamed to better reflect the method's purpose of returning locks
for the specified object.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-12-11 15:06:37 +03:00
Dmitrii Stepanov
1f6cf57e30 [#1548] metabase: Check if EC parent is removed or expired 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-12-11 07:26:33 +00:00
Dmitrii Stepanov
386a12eea4 [#1548] engine: Rename parent -> ecParent 
Parent could mean split parent or EC parent. In this case it is EC parent only.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-12-11 07:26:33 +00:00
Dmitrii Stepanov
41da27dad5
[#1549] engine: Drop Async flag from evacuation parameters 
Now it is only async evacuation.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-12-10 17:00:00 +03:00
Aleksey Savchuk
281d65435e
[#1450] engine: Group object by shard before Inhume 
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/engine
cpu: 12th Gen Intel(R) Core(TM) i5-1235U
                                 │   old.txt    │              new.txt                │
                                 │    sec/op    │   sec/op     vs base                │
InhumeMultipart/objects=1-12        11.42m ± 1%   10.71m ± 0%   -6.27% (p=0.000 n=10)
InhumeMultipart/objects=10-12       113.5m ± 0%   100.9m ± 3%  -11.08% (p=0.000 n=10)
InhumeMultipart/objects=100-12     1135.4m ± 1%   681.3m ± 2%  -40.00% (p=0.000 n=10)
InhumeMultipart/objects=1000-12     11.358 ± 0%    1.089 ± 1%  -90.41% (p=0.000 n=10)
InhumeMultipart/objects=10000-12   113.251 ± 0%    1.645 ± 1%  -98.55% (p=0.000 n=10)
geomean                              1.136        265.5m       -76.63%
```

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-12-04 10:09:00 +03:00
Aleksey Savchuk
b348b20289
[#1450] engine: Add benchmark for Inhume operation 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-12-04 10:08:34 +03:00
Aleksey Savchuk
748edd1999
[#1450] engine: Return shard-level error if object is expired on inhume 
Since we have errors defined on the shard-level, it looks strage that we
check an error againt the shard-level error `ErrLockObjectRemoval`, but
then return the metabase-level error. Let's return the same shard-level
error instead.

Since we have errors defined on the shard-level

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-12-04 10:06:57 +03:00
Aleksey Savchuk
432042c534
[#1527] engine: Add tests for handling expired objects on inhume and lock 
Currently, it's allowed to inhume or lock an expired object.
Consider the following scenario:

1) An user inhumes or locks an object
2) The object expires
3) GC hasn't yet deleted the object
4) The node loses the associated tombstone or lock
5) Another node replicates tombstone or lock to the first node

In this case, the second node succeeds, which is the desired behavior.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-12-03 12:29:45 +03:00
Aleksey Savchuk
9cabca9dfe
[#1527] engine/test: Move default metabase options to separate function 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-12-02 16:13:37 +03:00
Aleksey Savchuk
60feed3b5f
[#1527] engine/test: Allow to specify current epoch in epochState 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-12-02 15:37:25 +03:00
Aleksey Savchuk
7fc6101bec
[#1491] engine/test: Rework engine test utils 
- Remove `testNewShard` and `setInitializedShards` because they
violated the default engine workflow. The correct workflow is:
first use `New()`, followed by `Open()`, and then `Init()`. As a
result, adding new logic to `(*StorageEngine).Init` caused several
tests to fail with a panic when attempting to access uninitialized
resources. Now, all engines created with the test utils must be
initialized manually. The new helper method `prepare` can be used
for that purpose.
- Additionally, `setInitializedShards` hardcoded the shard worker
pool size, which prevented it from being configured in tests and
benchmarks. This has been fixed as well.
- Ensure engine initialization is done wherever it was missing.
- Refactor `setShardsNumOpts`, `setShardsNumAdditionalOpts`, and
`setShardsNum`. Make them all depend on `setShardsNumOpts`.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-11-13 14:42:53 +03:00
Aleksey Savchuk
7ef36749d0
[#1491] engine/test: Move BenchmarkExists to exists_test.go 
Move `BenchmarkExists` from `engine_test.go` to `exists_test.go`
for better organization and clarity.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-11-13 14:09:29 +03:00
Aleksey Savchuk
c6066d6ee4
[#1491] engine/test: Use more suitable testing utils here and there 
Use `setShardsNum` instead of `setInitializedShards` wherever possible.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-11-13 14:09:29 +03:00
Dmitrii Stepanov
7429553266
[#1437] node: Fix contextcheck linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-11-13 10:36:10 +03:00
Dmitrii Stepanov
16598553d9
[#1437] shard: Fix contextcheck linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-11-13 10:36:09 +03:00
Dmitrii Stepanov
62b5181618
[#1437] blobovnicza: Fix contextcheck linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-11-13 10:36:08 +03:00
Dmitrii Stepanov
6db46257c0
[#1437] node: Use ctx for logging 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-11-13 10:36:07 +03:00
Dmitrii Stepanov
9bd05e94c8 [#1449] tree: Add ApplyBatch method 
Concurrent Apply can lead to child node applies before parent, so
undo/redo operations will perform. This leads to performance degradation
in case of tree with many sublevels.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-11-12 12:11:07 +00:00
Dmitrii Stepanov
8a57c78f5f
[#1484] engine: Fix engine metrics 
1. Add forgotten metrics for client requests
2. Include execIfNotBlocked into metrics

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-11-11 12:59:20 +03:00
Airat Arifullin
9b13a18aac [#1479] go.mod: Bump frostfs-sdk-go version 
* Update version within go.mod;
* Fix deprecated frostfs-api-go/v2 package and use frostfs-sdk-go/api
  instead.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-11-08 10:43:19 +03:00
Evgenii Stratonikov
7ac0852364 [#1459] .golangci.yml: Add intrange linter, fix issues 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-10-30 15:18:22 +00:00
Alexander Chuprov
d53732f663 [#1431] engine: Delete always false condition 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-10-15 08:53:59 +00:00