Commit graph

3400 commits

Author SHA1 Message Date
15b4288d80
[#1548] metabase: Check if EC parent is removed or expired
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 2m32s
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m24s
Tests and linters / Run gofumpt (pull_request) Successful in 3m27s
DCO action / DCO (pull_request) Successful in 3m45s
Tests and linters / gopls check (pull_request) Successful in 4m3s
Build / Build Components (pull_request) Successful in 4m23s
Tests and linters / Staticcheck (pull_request) Successful in 4m52s
Tests and linters / Tests (pull_request) Successful in 5m29s
Tests and linters / Lint (pull_request) Successful in 5m44s
Tests and linters / Tests with -race (pull_request) Successful in 6m40s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-12-11 10:42:12 +03:00
6ea8f2b23c
[#1548] engine: Rename parent -> ecParent
Parent could mean split parent or EC parent. In this case it is EC parent only.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-12-11 10:41:32 +03:00
47f12f8440
[#1548] policer: Do not replicate EC chunk if object already removed
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-12-11 10:41:00 +03:00
edc1824c23
[#1544] go.mod: Update sdk-go
All checks were successful
DCO action / DCO (pull_request) Successful in 1m48s
Tests and linters / Run gofumpt (pull_request) Successful in 1m48s
Tests and linters / Staticcheck (pull_request) Successful in 3m8s
Vulncheck / Vulncheck (pull_request) Successful in 3m9s
Build / Build Components (pull_request) Successful in 3m45s
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m5s
Tests and linters / Lint (pull_request) Successful in 4m28s
Tests and linters / Tests (pull_request) Successful in 4m32s
Tests and linters / gopls check (pull_request) Successful in 4m44s
Tests and linters / Tests with -race (pull_request) Successful in 5m17s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-06 12:50:39 +03:00
cf48dfd55e [#1524] tree: Make check APE error get wrapped to api status
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 2m11s
DCO action / DCO (pull_request) Successful in 2m26s
Vulncheck / Vulncheck (pull_request) Successful in 2m37s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m55s
Build / Build Components (pull_request) Successful in 3m21s
Tests and linters / gopls check (pull_request) Successful in 3m25s
Tests and linters / Staticcheck (pull_request) Successful in 3m55s
Tests and linters / Lint (pull_request) Successful in 4m36s
Tests and linters / Tests (pull_request) Successful in 5m1s
Tests and linters / Tests with -race (pull_request) Successful in 6m4s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-12-04 09:57:46 +03:00
274ac61236 [#1524] ape: Make APE checker return error without status
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-12-04 09:57:41 +03:00
36fe470956 [#1530] node: Keep order for equal elements when sort priority metrics
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-12-03 11:45:38 +03:00
f12f04199e [#1516] traverser: Check for placement vector out of range
All checks were successful
DCO action / DCO (pull_request) Successful in 2m41s
Vulncheck / Vulncheck (pull_request) Successful in 3m13s
Tests and linters / Staticcheck (pull_request) Successful in 4m1s
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m16s
Tests and linters / Tests with -race (pull_request) Successful in 4m27s
Tests and linters / Run gofumpt (pull_request) Successful in 4m37s
Build / Build Components (pull_request) Successful in 5m0s
Tests and linters / gopls check (pull_request) Successful in 4m56s
Tests and linters / Lint (pull_request) Successful in 5m2s
Tests and linters / Tests (pull_request) Successful in 6m27s
Tests and linters / Run gofumpt (push) Successful in 2m13s
Vulncheck / Vulncheck (push) Successful in 2m17s
Build / Build Components (push) Successful in 2m45s
Tests and linters / Staticcheck (push) Successful in 2m44s
Pre-commit hooks / Pre-commit (push) Successful in 3m14s
Tests and linters / gopls check (push) Successful in 4m4s
Tests and linters / Tests (push) Successful in 4m11s
Tests and linters / Tests with -race (push) Successful in 4m10s
Tests and linters / Lint (push) Successful in 4m17s
Placement vector may contain fewer nodes count than it required by policy
due to the outage of the one of the node.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-11-21 14:18:55 +03:00
49a4e727fd [#1507] timer/test: Use const for constants
Make it easy to see what the test is about.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-20 08:36:25 +00:00
2e974f734c [#1507] timer/test: Improve test coverage
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-20 08:36:25 +00:00
3042490340 [#1507] timer: Remove unused OnDelta() method
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-20 08:36:25 +00:00
ffe9906266 [#1501] cli: Move APE-chain parser methods to pkg/util
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-20 07:58:32 +00:00
e2cb0640f1 [#1501] util: Move eACL-to-APE converter to pkg/util
* `ConvertEACLToAPE` is useful method which couldn't be imported
  out of frostfs-node so far as it has been in `internal`
* Since `ConvertEACLToAPE` and related structures and unit-tests
  are placed in `pkg/util`

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-20 07:58:32 +00:00
d82f0d1926
[#1496] node/control: Await until SetNetmapStatus() persists
All checks were successful
DCO action / DCO (pull_request) Successful in 1m1s
Vulncheck / Vulncheck (pull_request) Successful in 2m15s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m39s
Build / Build Components (pull_request) Successful in 3m0s
Tests and linters / Run gofumpt (pull_request) Successful in 2m53s
Tests and linters / Staticcheck (pull_request) Successful in 2m59s
Tests and linters / Lint (pull_request) Successful in 3m41s
Tests and linters / gopls check (pull_request) Successful in 3m58s
Tests and linters / Tests (pull_request) Successful in 4m23s
Tests and linters / Tests with -race (pull_request) Successful in 4m44s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-15 16:36:07 +03:00
acd5babd86
[#1496] morph: Merge InvokeRes and WaitParams
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-15 16:36:07 +03:00
b65874d1c3
[#1496] morph: Return InvokeRes from all invoke*() methods
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-15 16:36:07 +03:00
69c63006da
[#1496] morph: Move tx waiter to morph package
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-15 16:36:07 +03:00
d77a218f7c [#1493] metabase: Merge Inhume() and DropGraves() for tombstones
DropGraves() is only used to drop gravemarks after a tombstone
removal. Thus, it makes sense to do Inhume() and DropGraves() in one
transaction. It has less overhead and no unexpected problems in case
of sudden power failure.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-14 06:47:04 +00:00
44df67492f [#1493] metabase: Split inhumeTx() into 2 functions
No functional changes.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-14 06:47:04 +00:00
1e6f132b4e [#1493] metabase: Pass InhumePrm by value
Unify with the other code, no functional changes.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-14 06:47:04 +00:00
6dc0dc6691 [#1493] shard: Take mode mutex in HandleExpiredTombstones()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-14 06:47:04 +00:00
7fc6101bec
[#1491] engine/test: Rework engine test utils
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 2m24s
DCO action / DCO (pull_request) Successful in 2m37s
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m46s
Vulncheck / Vulncheck (pull_request) Successful in 3m46s
Build / Build Components (pull_request) Successful in 4m4s
Tests and linters / Staticcheck (pull_request) Successful in 4m1s
Tests and linters / gopls check (pull_request) Successful in 4m14s
Tests and linters / Lint (pull_request) Successful in 4m49s
Tests and linters / Tests with -race (pull_request) Successful in 5m25s
Tests and linters / Tests (pull_request) Successful in 5m46s
- Remove `testNewShard` and `setInitializedShards` because they
violated the default engine workflow. The correct workflow is:
first use `New()`, followed by `Open()`, and then `Init()`. As a
result, adding new logic to `(*StorageEngine).Init` caused several
tests to fail with a panic when attempting to access uninitialized
resources. Now, all engines created with the test utils must be
initialized manually. The new helper method `prepare` can be used
for that purpose.
- Additionally, `setInitializedShards` hardcoded the shard worker
pool size, which prevented it from being configured in tests and
benchmarks. This has been fixed as well.
- Ensure engine initialization is done wherever it was missing.
- Refactor `setShardsNumOpts`, `setShardsNumAdditionalOpts`, and
`setShardsNum`. Make them all depend on `setShardsNumOpts`.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-11-13 14:42:53 +03:00
7ef36749d0
[#1491] engine/test: Move BenchmarkExists to exists_test.go
Move `BenchmarkExists` from `engine_test.go` to `exists_test.go`
for better organization and clarity.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-11-13 14:09:29 +03:00
c6066d6ee4
[#1491] engine/test: Use more suitable testing utils here and there
Use `setShardsNum` instead of `setInitializedShards` wherever possible.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-11-13 14:09:29 +03:00
612b34d570
[#1437] logger: Add caller skip to log original caller position
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 1m36s
DCO action / DCO (pull_request) Successful in 1m50s
Vulncheck / Vulncheck (pull_request) Successful in 2m19s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m34s
Build / Build Components (pull_request) Successful in 2m49s
Tests and linters / gopls check (pull_request) Successful in 3m6s
Tests and linters / Staticcheck (pull_request) Successful in 3m15s
Tests and linters / Lint (pull_request) Successful in 3m56s
Tests and linters / Tests (pull_request) Successful in 4m44s
Tests and linters / Tests with -race (pull_request) Successful in 5m36s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:12 +03:00
7429553266
[#1437] node: Fix contextcheck linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:10 +03:00
6921a89061
[#1437] ir: Fix contextcheck linters
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:10 +03:00
16598553d9
[#1437] shard: Fix contextcheck linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:09 +03:00
c139892117
[#1437] ir: Fix contextcheck linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:09 +03:00
62b5181618
[#1437] blobovnicza: Fix contextcheck linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:08 +03:00
6db46257c0
[#1437] node: Use ctx for logging
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:07 +03:00
c16dae8b4d
[#1437] logger: Use context to log trace id
Signed-off-by: Dmitrii Stepanov
2024-11-13 10:36:07 +03:00
fd004add00 [#1492] metabase: Fix import formatting
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-13 07:30:25 +00:00
8ed7a676d5 [#1492] metabase: Ensure Unmarshal() is called on a cloned slice
The slice returned from bucket.Get() is only valid during the tx
lifetime. Cloning it is not necessary everywhere, but better safe than
sorry.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-13 07:30:25 +00:00
b451de94c8 [#1492] metabase: Fix typo in objData
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-13 07:30:25 +00:00
46fef276b4 [#1449] tree: Log tree sync with Info level
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-12 12:11:07 +00:00
9bd05e94c8 [#1449] tree: Add ApplyBatch method
Concurrent Apply can lead to child node applies before parent, so
undo/redo operations will perform. This leads to performance degradation
in case of tree with many sublevels.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-12 12:11:07 +00:00
3324c26fd8 [#1483] morph: Remove container.GetEACL()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-12 12:10:51 +00:00
a692298533 [#1483] node: Remove eACL cache
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-12 12:10:51 +00:00
8a57c78f5f
[#1484] engine: Fix engine metrics
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 2m38s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m51s
DCO action / DCO (pull_request) Successful in 2m53s
Vulncheck / Vulncheck (pull_request) Successful in 3m44s
Tests and linters / gopls check (pull_request) Successful in 3m53s
Build / Build Components (pull_request) Successful in 4m15s
Tests and linters / Staticcheck (pull_request) Successful in 4m56s
Tests and linters / Tests with -race (pull_request) Successful in 5m5s
Tests and linters / Lint (pull_request) Successful in 5m47s
Tests and linters / Tests (pull_request) Successful in 5m51s
1. Add forgotten metrics for client requests
2. Include execIfNotBlocked into metrics

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-11 12:59:20 +03:00
c82c753e9f [#1480] objsvc: Remove useless stream wrappers
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-08 12:01:14 +00:00
f666898e5d [#1480] objsvc: Remove EACL checks
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-08 12:01:14 +00:00
b1a31281e4 [#1480] ape: Remove SoftAPECheck flag
Previous release was EACL-compatible.
Starting from now all EACL should've been migrated to APE chains.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-08 12:01:14 +00:00
764450d04a [#1479] tree: Regenerate service protobufs
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 1m27s
DCO action / DCO (pull_request) Successful in 1m41s
Vulncheck / Vulncheck (pull_request) Successful in 2m18s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m34s
Build / Build Components (pull_request) Successful in 2m44s
Tests and linters / Staticcheck (pull_request) Successful in 3m2s
Tests and linters / gopls check (pull_request) Successful in 3m6s
Tests and linters / Lint (pull_request) Successful in 3m56s
Tests and linters / Tests (pull_request) Successful in 4m44s
Tests and linters / Tests with -race (pull_request) Successful in 6m31s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-08 10:43:44 +03:00
755cae3f19 [#1479] control: Regenerate protobufs for service
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-08 10:43:44 +03:00
9b13a18aac [#1479] go.mod: Bump frostfs-sdk-go version
* Update version within go.mod;
* Fix deprecated frostfs-api-go/v2 package and use frostfs-sdk-go/api
  instead.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-08 10:43:19 +03:00
ef64930fef
[#1477] ape: Fix EC chunk test
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 1m33s
DCO action / DCO (pull_request) Successful in 1m46s
Vulncheck / Vulncheck (pull_request) Successful in 2m16s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m35s
Build / Build Components (pull_request) Successful in 2m50s
Tests and linters / Staticcheck (pull_request) Successful in 3m1s
Tests and linters / gopls check (pull_request) Successful in 3m6s
Tests and linters / Lint (pull_request) Successful in 3m53s
Tests and linters / Tests (pull_request) Successful in 4m43s
Tests and linters / Tests with -race (pull_request) Successful in 6m18s
Initially, this test was a check that only the container node can
assemble an EC object. But the implementation of this test was wrong.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-07 16:00:20 +03:00
c8fb154151
[#1475] Remove container estimation code
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 1m38s
Vulncheck / Vulncheck (pull_request) Successful in 5m20s
Build / Build Components (pull_request) Successful in 6m9s
Tests and linters / gopls check (pull_request) Successful in 6m1s
Tests and linters / Lint (pull_request) Successful in 6m46s
Tests and linters / Tests with -race (pull_request) Successful in 9m35s
Tests and linters / Tests (pull_request) Successful in 2m6s
Tests and linters / Staticcheck (pull_request) Successful in 2m27s
DCO action / DCO (pull_request) Successful in 1m1s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m45s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-06 15:32:33 +03:00
7edec9193c [#1451] placement: Return copy of slice from container nodes cache
Nodes from cache could be changed by traverser, if no objectID specified.
So it is required to return copy of cache's slice.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-06 08:18:10 +00:00
3cf6ea745d [#1451] ec: Check all parts are saved
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-06 08:18:10 +00:00