dstepanov-yadro/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Issues Pull requests Projects Releases Packages Wiki Activity
5185 commits 298 branches 26 tags 21 MiB
Commit graph

3416 commits

Author SHA1 Message Date
Dmitrii Stepanov
d0592df3c1
[#9999] writecache: Fix after revert 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2025-01-22 15:16:42 +03:00
Dmitrii Stepanov
6e4b75b5f9
Revert "[#1367] writecache: Drop bbolt DB" 
This reverts commit 5f6c7cbdb1.
 2025-01-22 11:41:08 +03:00
Dmitrii Stepanov
627c1f94d3
Revert "[#1367] fstree: Add size to file counter" 
This reverts commit b142b6f48e.
 2025-01-22 11:39:30 +03:00
Dmitrii Stepanov
af758ba25f
Revert "[#1367] fstree: Add IterateInfo method" 
This reverts commit 2dd3a6f7a8.
 2025-01-22 11:39:24 +03:00
Dmitrii Stepanov
087619f5c5
Revert "[#1367] writecache: Flush from FSTree concurrently" 
This reverts commit 8a6e3025a0.
 2025-01-22 11:39:15 +03:00
Dmitrii Stepanov
f44ba3b76b
Revert "[#1367] writecache: Add background flushing objects limiter" 
This reverts commit e39378b1c3.
 2025-01-22 11:38:21 +03:00
Dmitrii Stepanov
105278f1b3
Revert "[#1367] writecache: Drop BBolt related config variables" 
This reverts commit 25d2ae8aaf.
 2025-01-22 11:37:40 +03:00
Dmitrii Stepanov
5dcbb7d7aa
Revert "[#1367] writecache: Drop DB label from metrics" 
This reverts commit 3b236160a6.
 2025-01-22 11:36:52 +03:00
Dmitrii Stepanov
056a1e0bbd
Revert "[#1367] writecache: Move DB related code to upgrade.go" 
This reverts commit f345fe9a58.
 2025-01-22 11:36:42 +03:00
Dmitrii Stepanov
1fc95f6ed8
Revert "[#1367] fstree: Add size hint for Delete" 
This reverts commit b33559754d.
 2025-01-22 11:36:34 +03:00
Evgenii Stratonikov
74e11b3cf3
[#1586] objsvc: Allow to send search response in multiple messages 
Previously, `ln` was only set once, so search has really worked for
small number of objects.

Fix panic:
```
panic: runtime error: slice bounds out of range [:43690] with capacity 21238
goroutine 6859775 [running]:
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object.(*searchStreamMsgSizeCtrl).Send(0xc001eec8d0, 0xc005734000)
        git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/transport_splitter.go:173 +0x1f0
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/v2.(*streamWriter).WriteIDs(0xc000520320, {0xc00eb1a000, 0x4fd9c, 0x7fd6475a9a68?})
        git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/v2/streamer.go:28 +0x155
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search.(*uniqueIDWriter).WriteIDs(0xc001386420, {0xc00eb1a000?, 0xc0013ea9c0?, 0x113eef3?})
        git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/util.go:62 +0x202
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search.(*execCtx).writeIDList(0xc00011aa38?, {0xc00eb1a000?, 0xc001eec9f0?, 0xc0008f4380?})
        git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/exec.go:68 +0x91
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search.(*execCtx).executeLocal(0xc0008f4380, {0x176c538, 0xc001eec9f0})
        git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/local.go:18 +0x16b
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-12-28 09:23:54 +03:00
Evgenii Stratonikov
f527556735
Revert "[#1492] metabase: Ensure Unmarshal() is called on a cloned slice" 
This reverts commit 8ed7a676d5.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-12-26 16:25:13 +03:00
Evgenii Stratonikov
23b426ec4a
[#1584] go.mod: Update sdk-go 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-12-26 15:37:21 +03:00
Airat Arifullin
7e3f2fca47 [#1563] tree: Wrap only ChainRouterError erros with ObjectAccessDenied 
* Such wrapping helps to differentiate logical check errors and server internal
  errors.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-12-16 16:19:04 +03:00
Airat Arifullin
25ed595354 [#1563] object: Wrap only ChainRouterError erros with ObjectAccessDenied 
* Such wrapping helps to differentiate logical check errors and server internal
  errors.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-12-16 16:18:58 +03:00
Airat Arifullin
fb2f40aea1 [#1563] ape: Introduce ChainRouterError error type 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-12-16 16:18:51 +03:00
Dmitrii Stepanov
15b4288d80
[#1548] metabase: Check if EC parent is removed or expired 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-12-11 10:42:12 +03:00
Dmitrii Stepanov
6ea8f2b23c
[#1548] engine: Rename parent -> ecParent 
Parent could mean split parent or EC parent. In this case it is EC parent only.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-12-11 10:41:32 +03:00
Dmitrii Stepanov
47f12f8440
[#1548] policer: Do not replicate EC chunk if object already removed 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-12-11 10:41:00 +03:00
Evgenii Stratonikov
edc1824c23
[#1544] go.mod: Update sdk-go 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-12-06 12:50:39 +03:00
Airat Arifullin
cf48dfd55e [#1524] tree: Make check APE error get wrapped to api status 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-12-04 09:57:46 +03:00
Airat Arifullin
274ac61236 [#1524] ape: Make APE checker return error without status 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-12-04 09:57:41 +03:00
Anton Nikiforov
36fe470956 [#1530] node: Keep order for equal elements when sort priority metrics 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-12-03 11:45:38 +03:00
Anton Nikiforov
f12f04199e [#1516] traverser: Check for placement vector out of range 
Placement vector may contain fewer nodes count than it required by policy
due to the outage of the one of the node.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-11-21 14:18:55 +03:00
Evgenii Stratonikov
49a4e727fd [#1507] timer/test: Use const for constants 
Make it easy to see what the test is about.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-11-20 08:36:25 +00:00
Evgenii Stratonikov
2e974f734c [#1507] timer/test: Improve test coverage 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-11-20 08:36:25 +00:00
Evgenii Stratonikov
3042490340 [#1507] timer: Remove unused OnDelta() method 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-11-20 08:36:25 +00:00
Airat Arifullin
ffe9906266 [#1501] cli: Move APE-chain parser methods to pkg/util 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-11-20 07:58:32 +00:00
Airat Arifullin
e2cb0640f1 [#1501] util: Move eACL-to-APE converter to pkg/util 
* `ConvertEACLToAPE` is useful method which couldn't be imported
  out of frostfs-node so far as it has been in `internal`
* Since `ConvertEACLToAPE` and related structures and unit-tests
  are placed in `pkg/util`

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-11-20 07:58:32 +00:00
Evgenii Stratonikov
d82f0d1926
[#1496] node/control: Await until SetNetmapStatus() persists 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-11-15 16:36:07 +03:00
Evgenii Stratonikov
acd5babd86
[#1496] morph: Merge InvokeRes and WaitParams 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-11-15 16:36:07 +03:00
Evgenii Stratonikov
b65874d1c3
[#1496] morph: Return InvokeRes from all invoke*() methods 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-11-15 16:36:07 +03:00
Evgenii Stratonikov
69c63006da
[#1496] morph: Move tx waiter to morph package 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-11-15 16:36:07 +03:00
Evgenii Stratonikov
d77a218f7c [#1493] metabase: Merge Inhume() and DropGraves() for tombstones 
DropGraves() is only used to drop gravemarks after a tombstone
removal. Thus, it makes sense to do Inhume() and DropGraves() in one
transaction. It has less overhead and no unexpected problems in case
of sudden power failure.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-11-14 06:47:04 +00:00
Evgenii Stratonikov
44df67492f [#1493] metabase: Split inhumeTx() into 2 functions 
No functional changes.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-11-14 06:47:04 +00:00
Evgenii Stratonikov
1e6f132b4e [#1493] metabase: Pass InhumePrm by value 
Unify with the other code, no functional changes.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-11-14 06:47:04 +00:00
Evgenii Stratonikov
6dc0dc6691 [#1493] shard: Take mode mutex in HandleExpiredTombstones() 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-11-14 06:47:04 +00:00
Aleksey Savchuk
7fc6101bec
[#1491] engine/test: Rework engine test utils 
- Remove `testNewShard` and `setInitializedShards` because they
violated the default engine workflow. The correct workflow is:
first use `New()`, followed by `Open()`, and then `Init()`. As a
result, adding new logic to `(*StorageEngine).Init` caused several
tests to fail with a panic when attempting to access uninitialized
resources. Now, all engines created with the test utils must be
initialized manually. The new helper method `prepare` can be used
for that purpose.
- Additionally, `setInitializedShards` hardcoded the shard worker
pool size, which prevented it from being configured in tests and
benchmarks. This has been fixed as well.
- Ensure engine initialization is done wherever it was missing.
- Refactor `setShardsNumOpts`, `setShardsNumAdditionalOpts`, and
`setShardsNum`. Make them all depend on `setShardsNumOpts`.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-11-13 14:42:53 +03:00
Aleksey Savchuk
7ef36749d0
[#1491] engine/test: Move BenchmarkExists to exists_test.go 
Move `BenchmarkExists` from `engine_test.go` to `exists_test.go`
for better organization and clarity.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-11-13 14:09:29 +03:00
Aleksey Savchuk
c6066d6ee4
[#1491] engine/test: Use more suitable testing utils here and there 
Use `setShardsNum` instead of `setInitializedShards` wherever possible.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-11-13 14:09:29 +03:00
Dmitrii Stepanov
612b34d570
[#1437] logger: Add caller skip to log original caller position 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-11-13 10:36:12 +03:00
Dmitrii Stepanov
7429553266
[#1437] node: Fix contextcheck linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-11-13 10:36:10 +03:00
Dmitrii Stepanov
6921a89061
[#1437] ir: Fix contextcheck linters 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-11-13 10:36:10 +03:00
Dmitrii Stepanov
16598553d9
[#1437] shard: Fix contextcheck linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-11-13 10:36:09 +03:00
Dmitrii Stepanov
c139892117
[#1437] ir: Fix contextcheck linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-11-13 10:36:09 +03:00
Dmitrii Stepanov
62b5181618
[#1437] blobovnicza: Fix contextcheck linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-11-13 10:36:08 +03:00
Dmitrii Stepanov
6db46257c0
[#1437] node: Use ctx for logging 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-11-13 10:36:07 +03:00
Dmitrii Stepanov
c16dae8b4d
[#1437] logger: Use context to log trace id 
Signed-off-by: Dmitrii Stepanov
 2024-11-13 10:36:07 +03:00
Evgenii Stratonikov
fd004add00 [#1492] metabase: Fix import formatting 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-11-13 07:30:25 +00:00
Evgenii Stratonikov
8ed7a676d5 [#1492] metabase: Ensure Unmarshal() is called on a cloned slice 
The slice returned from bucket.Get() is only valid during the tx
lifetime. Cloning it is not necessary everywhere, but better safe than
sorry.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-11-13 07:30:25 +00:00