Commit graph

5008 commits

Author SHA1 Message Date
6ccac0a476
[#1403] metabase: Refactor inGraveyardWithKey function
This function was very obfuscated. I hope the newer version is more
clear, but IMHO it keeps being bad because:
- Its name is confusing because it checks both the graveyard and the
  garbage.
- It has no interface. We use that function in several metabase methods,
  it just returns some 'magic' uint8 numbers and has no doc comment, I
  mean it's ridiculous.
- It checks out for 'the node being in incorrect state' for some reason
  but that result isn't used further. I kept a comment about that but it
  has no logic for me.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-10-07 16:17:20 +03:00
3f4df881c9
[#1403] engine: Add tests for object deletion on read-only shard
Currently, when an object on some shard is inhumed, the engine places
a tombstone on the same shard. If the target shard is read-only, the
engine fails.

In that case, we want the engine to correctly place a tombstone on
a different shard, ensuring that:
- An object becomes unavailable if a tombstone was placed on a different
  shard. See `TestObjectUnavailableIfTombstoneOnDifferentShard` test.
- GC deletes an object if a tombstone was placed on a different shard.
  See `TestObjectDeletedIfTombstoneOnDifferentShard` test.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-10-07 16:16:24 +03:00
a5de74a249 [#1418] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-07 12:55:28 +00:00
fc032838c0 [#1215] blobstor/test: Cover iteration behaviour
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-07 12:44:27 +00:00
2f710d8f94 [#1414] metabase: Check parameter for CountAliveObjectsInBucket
All checks were successful
DCO action / DCO (pull_request) Successful in 1m33s
Tests and linters / Run gofumpt (pull_request) Successful in 1m28s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m10s
Vulncheck / Vulncheck (pull_request) Successful in 2m6s
Build / Build Components (pull_request) Successful in 2m32s
Tests and linters / gopls check (pull_request) Successful in 2m44s
Tests and linters / Staticcheck (pull_request) Successful in 3m6s
Tests and linters / Lint (pull_request) Successful in 3m35s
Tests and linters / Tests (pull_request) Successful in 4m21s
Tests and linters / Tests with -race (pull_request) Successful in 5m44s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-10-04 17:11:15 +03:00
4dc9a1b300 [#1413] engine: Remove error counting methods from Shard
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 2m4s
DCO action / DCO (pull_request) Successful in 2m22s
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m10s
Vulncheck / Vulncheck (pull_request) Successful in 4m5s
Build / Build Components (pull_request) Successful in 4m31s
Tests and linters / Staticcheck (pull_request) Successful in 4m21s
Tests and linters / gopls check (pull_request) Successful in 4m43s
Tests and linters / Lint (pull_request) Successful in 4m58s
Tests and linters / Tests (pull_request) Successful in 6m36s
Tests and linters / Tests with -race (pull_request) Successful in 7m41s
All error counting and hangling logic is present on the engine level.
Currently, we pass engine metrics with shard ID metric to shard, then
export 3 methods to manipulate these metrics.
In this commits all methods are removed and error counter is tracked on
the engine level exlusively.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-04 15:10:17 +03:00
963faa615a [#1413] engine: Cleanup shard error reporting
- `reportShardErrorBackground()` no longer differs from
  `reportShardError()`, reflect this in its name;
- reuse common pieces of code to make it simpler.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-04 15:10:17 +03:00
9a87acb87a [#1410] engine: Provide the default implementation to MetricsRegister
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-03 08:23:06 +00:00
9206ce5cd2 [#1410] shard: Provide the default implementation for MetricsWriter
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-03 08:23:06 +00:00
6c46044c9c [#1410] shard: Move MetricsWriter interface to a separate file
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-03 08:23:06 +00:00
01e3944b31 [#1408] metabase: Fix tests
All checks were successful
DCO action / DCO (pull_request) Successful in 1m3s
Tests and linters / Run gofumpt (pull_request) Successful in 1m21s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m27s
Vulncheck / Vulncheck (pull_request) Successful in 2m20s
Build / Build Components (pull_request) Successful in 2m41s
Tests and linters / gopls check (pull_request) Successful in 2m57s
Tests and linters / Staticcheck (pull_request) Successful in 3m5s
Tests and linters / Lint (pull_request) Successful in 3m37s
Tests and linters / Tests (pull_request) Successful in 4m31s
Tests and linters / Tests with -race (pull_request) Successful in 6m22s
No need to specify container ID for objects created with `testutil.GenerateObjectWithCID`.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-02 14:41:40 +03:00
434048e8d9 [#1408] metabase: Fix EC search with slow and fast filters
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-02 14:36:59 +03:00
f83f7feb8c [#1391] adm: Properly check whether transfers were made
All checks were successful
DCO action / DCO (pull_request) Successful in 35s
Tests and linters / Run gofumpt (pull_request) Successful in 1m16s
Vulncheck / Vulncheck (pull_request) Successful in 1m25s
Build / Build Components (pull_request) Successful in 2m17s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m26s
Tests and linters / gopls check (pull_request) Successful in 2m45s
Tests and linters / Staticcheck (pull_request) Successful in 2m56s
Tests and linters / Lint (pull_request) Successful in 3m32s
Tests and linters / Tests (pull_request) Successful in 4m17s
Tests and linters / Tests with -race (pull_request) Successful in 5m59s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-02 11:41:33 +03:00
62028cd7ee [#1409] adm: Uncommonize DeltaFlag
It is used only in `force-new-epoch`, it is not _common_ between
multiple commands.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-02 11:41:27 +03:00
f45e75e3eb [#1409] adm: Do not bind DeltaFlag to viper
We bind flag that could be specified in config.
This is not a config flag, just a command option.
Also fix TestInitialize failures:
```
                Error:          Received unexpected error:
                                number of epochs cannot be less than 1
                Test:           TestInitialize/16_nodes/force-new-epoch
```
Refs #1372 (945b7c740b)

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-02 11:41:22 +03:00
57c31e9802 [#1306] node: Allow tombstone_lifetime config to be loaded on the fly
All checks were successful
DCO action / DCO (pull_request) Successful in 37s
Tests and linters / Run gofumpt (pull_request) Successful in 31s
Vulncheck / Vulncheck (pull_request) Successful in 2m0s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m13s
Build / Build Components (pull_request) Successful in 2m25s
Tests and linters / Staticcheck (pull_request) Successful in 2m22s
Tests and linters / gopls check (pull_request) Successful in 3m4s
Tests and linters / Lint (pull_request) Successful in 3m18s
Tests and linters / Tests (pull_request) Successful in 3m42s
Tests and linters / Tests with -race (pull_request) Successful in 4m18s
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-10-02 10:22:00 +03:00
9c5ddc4dfe [#1407] tree: Set ContainerOwner in parameter for CheckAPE
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-10-02 07:21:02 +00:00
54eb005822 [#1404] go.mod: Update api-go
Fix #1398
Fix #1399

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-02 06:19:38 +00:00
a13219808a [#1375] node: Configure of the container cache size
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-10-01 14:05:52 +00:00
7f8a1dcf8e [#1400] adm: Support flag alphabet-wallets for commands proxy-add/remove-account
All checks were successful
DCO action / DCO (pull_request) Successful in 1m20s
Tests and linters / Run gofumpt (pull_request) Successful in 1m31s
Vulncheck / Vulncheck (pull_request) Successful in 1m58s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m10s
Build / Build Components (pull_request) Successful in 2m26s
Tests and linters / gopls check (pull_request) Successful in 2m39s
Tests and linters / Staticcheck (pull_request) Successful in 3m9s
Tests and linters / Lint (pull_request) Successful in 3m30s
Tests and linters / Tests (pull_request) Successful in 4m5s
Tests and linters / Tests with -race (pull_request) Successful in 5m43s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-09-30 14:15:13 +03:00
d0ed29b3c7 [#1350] node: Add ability to evacuate objects from REP 1 only
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 50s
DCO action / DCO (pull_request) Successful in 1m41s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m53s
Vulncheck / Vulncheck (pull_request) Successful in 2m10s
Build / Build Components (pull_request) Successful in 2m43s
Tests and linters / gopls check (pull_request) Successful in 3m0s
Tests and linters / Staticcheck (pull_request) Successful in 3m5s
Tests and linters / Lint (pull_request) Successful in 3m52s
Tests and linters / Tests (pull_request) Successful in 4m40s
Tests and linters / Tests with -race (pull_request) Successful in 6m17s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-09-27 15:41:17 +03:00
5f22ba6f38 [#1397] object: Correctly set namespace before APE check
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-27 11:43:29 +00:00
a5e1aa22c9 [#1394] putSvc: Fix relay
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 1m44s
DCO action / DCO (pull_request) Successful in 1m58s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m37s
Vulncheck / Vulncheck (pull_request) Successful in 3m11s
Tests and linters / Staticcheck (pull_request) Successful in 3m26s
Tests and linters / Lint (pull_request) Successful in 3m33s
Build / Build Components (pull_request) Successful in 3m42s
Tests and linters / gopls check (pull_request) Successful in 4m8s
Tests and linters / Tests (pull_request) Successful in 5m46s
Tests and linters / Tests with -race (pull_request) Successful in 7m31s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 17:28:03 +03:00
772b471aab [#1388] lens: Add nolint annotations
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
4fbfffd44c [#1388] putSvc: Drop unused
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
29e4cf7ba1 [#1388] ir: Annotate cmode as nolint
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
a2ab6d4942 [#1388] node: Drop unused
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
bdd57c8b6b [#1388] sessionSvc: Add nolint annotations
Used as map key.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
d1d6e3471c [#1388] signSvc: Drop unused
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
401c398704 [#1388] metabase: Drop unused
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
004ff9e9bf [#1388] blobstor: Drop unused
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
63a567a1de [#1388] engine: Drop unused
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
580cd55180 [#1388] getSvc: Drop unused
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
e319bf403e [#1388] apeSvc: Drop unused and make annotations
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
aedb55f913 [#1388] governance: Drop unused
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
b69e07da7a [#1388] metrics: Mark nolint:unused metrics
Although these fields could be deleted, I annotated them so that all the
metrics used would be defined in one place.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
2bd560e528 [#1388] cli: Drop unused flag/parameter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
95597d3437 [#1388] golangci: Make unused linter stricker
Add aditional checks. The most important false positive - structs used as
map keys.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
fd18aa363b [#1385] metabase: Optimize isTomb check
All checks were successful
DCO action / DCO (pull_request) Successful in 52s
Tests and linters / Run gofumpt (pull_request) Successful in 46s
Vulncheck / Vulncheck (pull_request) Successful in 1m44s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m59s
Build / Build Components (pull_request) Successful in 2m9s
Tests and linters / Staticcheck (pull_request) Successful in 2m15s
Tests and linters / gopls check (pull_request) Successful in 3m3s
Tests and linters / Lint (pull_request) Successful in 3m20s
Tests and linters / Tests (pull_request) Successful in 3m30s
Tests and linters / Tests with -race (pull_request) Successful in 5m50s
As tombstone and target must have the same containerID, do not iterate
other containers.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-24 17:09:06 +03:00
76268e3ea2 [#1385] metabase: Validate that tombstone and target have the same container ID
Target container ID is taken from tombstone: cmd/frostfs-node/object.go:507
Also object of type `TOMBSTONE` contains objectID, so tombstone and
tombstoned object must have the same containerID.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-24 17:09:06 +03:00
8434f3dbfc [#1385] metabase: Use Batch for delete-related operations
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-24 17:09:06 +03:00
34e6a309c6 [#1356] engine: Evacuate object from shards concurrently
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 1m34s
DCO action / DCO (pull_request) Successful in 1m45s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m43s
Vulncheck / Vulncheck (pull_request) Successful in 2m35s
Tests and linters / Staticcheck (pull_request) Successful in 2m44s
Build / Build Components (pull_request) Successful in 3m19s
Tests and linters / gopls check (pull_request) Successful in 3m26s
Tests and linters / Lint (pull_request) Successful in 3m37s
Tests and linters / Tests with -race (pull_request) Successful in 4m44s
Tests and linters / Tests (pull_request) Successful in 5m13s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-09-24 12:50:19 +03:00
bdf386366c [#1297] dev: Bump neo-go version
All checks were successful
DCO action / DCO (pull_request) Successful in 54s
Tests and linters / Run gofumpt (pull_request) Successful in 1m16s
Vulncheck / Vulncheck (pull_request) Successful in 1m28s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m14s
Build / Build Components (pull_request) Successful in 2m23s
Tests and linters / Staticcheck (pull_request) Successful in 2m35s
Tests and linters / gopls check (pull_request) Successful in 2m44s
Tests and linters / Lint (pull_request) Successful in 3m21s
Tests and linters / Tests (pull_request) Successful in 4m17s
Tests and linters / Tests with -race (pull_request) Successful in 5m50s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-24 12:05:09 +03:00
839dead226 [#1297] getSvc: Return AccessDenied instead of ObjectNotFound
Do not replace the access denied error if it was received earlier.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-24 12:05:09 +03:00
3bb65ba820 [#1392] object: Fix target initialization within put streamer
* Remove `relay` field from put streamer as it's no longer used;
* Fix initialization of `Relay` object writer parameter.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-24 09:04:13 +00:00
d4493a6d08 [#1390] getSvc: Fix Head EC1.1
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 2m16s
DCO action / DCO (pull_request) Successful in 2m32s
Tests and linters / Staticcheck (pull_request) Successful in 3m37s
Vulncheck / Vulncheck (pull_request) Successful in 3m44s
Build / Build Components (pull_request) Successful in 3m58s
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m56s
Tests and linters / gopls check (pull_request) Successful in 4m26s
Tests and linters / Lint (pull_request) Successful in 5m8s
Tests and linters / Tests (pull_request) Successful in 5m48s
Tests and linters / Tests with -race (pull_request) Successful in 6m22s
If local EC chunk found, but remote node is off, then `HEAD --raw` request
returns object not found.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-23 15:12:06 +03:00
0b87be804a [#1381] engine: Fix tests
All checks were successful
DCO action / DCO (pull_request) Successful in 1m35s
Tests and linters / Run gofumpt (pull_request) Successful in 1m37s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m40s
Vulncheck / Vulncheck (pull_request) Successful in 2m35s
Tests and linters / Staticcheck (pull_request) Successful in 2m52s
Build / Build Components (pull_request) Successful in 3m33s
Tests and linters / gopls check (pull_request) Successful in 3m35s
Tests and linters / Lint (pull_request) Successful in 4m44s
Tests and linters / Tests (pull_request) Successful in 5m14s
Tests and linters / Tests with -race (pull_request) Successful in 6m1s
Drop not required `Eventually` calls.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-23 11:10:29 +03:00
f71418b73c [#1386] frostfs-adm: Add info to error messages
These error messages bubble up to human users - adding more context helps
to find the cause of the issue faster.

Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
2024-09-23 07:47:03 +00:00
c34b8acedd [#1312] Drop handling of system attributes with NeoFS prefix
All checks were successful
DCO action / DCO (pull_request) Successful in 1m2s
Tests and linters / Run gofumpt (pull_request) Successful in 1m14s
Vulncheck / Vulncheck (pull_request) Successful in 1m46s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m16s
Build / Build Components (pull_request) Successful in 2m29s
Tests and linters / Staticcheck (pull_request) Successful in 2m42s
Tests and linters / gopls check (pull_request) Successful in 3m5s
Tests and linters / Lint (pull_request) Successful in 3m32s
Tests and linters / Tests (pull_request) Successful in 4m14s
Tests and linters / Tests with -race (pull_request) Successful in 5m8s
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-20 11:37:18 +00:00
c290d079fd [#1312] go.mod: Update sdk-go
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-20 11:37:18 +00:00