Commit Graph

4611 Commits (1005bf4f56d98cf09b285ee56ba1d447c508ffd8)

Author SHA1 Message Date
Dmitrii Stepanov 1005bf4f56 [#1024] shard: Add refill metabase benchmark
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:21:10 +03:00
Dmitrii Stepanov 76398c06b0 [#1080] metabase: Add StorageID metric
DCO action / DCO (pull_request) Successful in 1m19s Details
Build / Build Components (1.21) (pull_request) Successful in 4m6s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m35s Details
Build / Build Components (1.20) (pull_request) Successful in 7m32s Details
Tests and linters / gopls check (pull_request) Successful in 7m43s Details
Tests and linters / Lint (pull_request) Successful in 9m51s Details
Tests and linters / Staticcheck (pull_request) Successful in 9m53s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m48s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m11s Details
Tests and linters / Tests with -race (pull_request) Successful in 14m4s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:00:08 +03:00
Dmitrii Stepanov 7b1adfed3e [#1080] metabase: Open bucket for container counter once
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:00:08 +03:00
Dmitrii Stepanov e74bdaa5d5 [#1080] ape: Use value for APE request
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-09 18:42:03 +03:00
Dmitrii Stepanov 338d8cbebd [#1080] ape: Do not read object headers before Head/Get
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-09 15:27:40 +03:00
Dmitrii Stepanov 5b8200de88 [#984] blobovnicza: Do not fail rebuild on big objects
If blobovnicza contains objects larger than object size parameter
value, then rebuild fails with an error, because there is no such
bucket in database. This commit forces to create bucket on rebuild.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-09 11:51:18 +00:00
Anton Nikiforov 2b88361849 [#1062] object: Fix buffer allocation for `PayloadRange`
DCO action / DCO (pull_request) Successful in 5m34s Details
Vulncheck / Vulncheck (pull_request) Successful in 5m14s Details
Build / Build Components (1.20) (pull_request) Successful in 8m56s Details
Build / Build Components (1.21) (pull_request) Successful in 8m56s Details
Tests and linters / gopls check (pull_request) Successful in 9m3s Details
Tests and linters / Staticcheck (pull_request) Successful in 9m39s Details
Tests and linters / Lint (pull_request) Successful in 10m9s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 13m43s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 14m4s Details
Tests and linters / Tests with -race (pull_request) Successful in 14m40s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-09 11:59:07 +03:00
Dmitrii Stepanov f5b67c6735 [#1064] policer: Disable EC processing
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-09 07:08:53 +00:00
Dmitrii Stepanov bdf4990904 [#1064] cli: Add EC header output to object head
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-09 07:08:53 +00:00
Dmitrii Stepanov 8668cbf147 [#1064] dev: Add `IR + 4 storage nodes` configuration
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-09 07:08:53 +00:00
Dmitrii Stepanov 1c5e0f90aa [#1064] putsvc: Add EC put
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-09 07:08:53 +00:00
Dmitrii Stepanov 39da643354 [#1064] putsvc: Refactor distributed target
Extract object builder.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-09 07:08:53 +00:00
Alexander Chuprov 92569b9bbf [#1065] cli: Add support EC parameters
DCO action / DCO (pull_request) Successful in 20m19s Details
Vulncheck / Vulncheck (pull_request) Successful in 23m37s Details
Build / Build Components (1.20) (pull_request) Successful in 25m19s Details
Build / Build Components (1.21) (pull_request) Successful in 25m42s Details
Tests and linters / gopls check (pull_request) Successful in 28m36s Details
Tests and linters / Staticcheck (pull_request) Successful in 30m21s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m41s Details
Tests and linters / Lint (pull_request) Successful in 8m48s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 4m29s Details
Tests and linters / Tests with -race (pull_request) Successful in 4m46s Details
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-04-08 12:27:51 +03:00
Alexander Chuprov 17f7adb640 [#1065] adm: Add support EC parameters
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-04-08 12:27:30 +03:00
Alexander Chuprov 0290f86579 [#1065] adm: refactor dump-config
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-04-08 12:06:23 +03:00
Anton Nikiforov ffb1a6f81a [#1072] Fix issue from `govulncheck`
Vulncheck / Vulncheck (pull_request) Successful in 1m21s Details
DCO action / DCO (pull_request) Successful in 2m6s Details
Build / Build Components (1.21) (pull_request) Successful in 4m17s Details
Build / Build Components (1.20) (pull_request) Successful in 4m52s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m37s Details
Tests and linters / Lint (pull_request) Successful in 7m13s Details
Tests and linters / gopls check (pull_request) Successful in 7m2s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m8s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 10m22s Details
Tests and linters / Tests with -race (pull_request) Successful in 10m20s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-05 18:20:07 +03:00
Anton Nikiforov 9aa533e59a [#1072] node, ir: Add new config option `kludge_compatibility_mode`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-04 11:05:25 +03:00
Anton Nikiforov d614f04a0a [#1072] Fix gofumpt issues
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-03 22:21:14 +03:00
Airat Arifullin 531542ce60 [#1063] cli: Validate container creation for EC policy
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-03 10:33:51 +00:00
Airat Arifullin 4738508ce2 [#1063] go.mod: Update SDK version
* Update frostfs-sdk and frostfs-api-go versions.
* Refactor depreacted method ReplicaNumberByIndex.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-03 10:33:51 +00:00
Evgenii Stratonikov ff4c23f59a [#1070] services/tree: Fix fast listing depth processing
For unsorted `GetSubTree()` we return a single node for depth=1.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-02 14:41:31 +00:00
Evgenii Stratonikov 17af91619a [#1070] pilorama: Fix cycling behaviour for sorted listing
In case there are no items left, return empty slice.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-02 14:41:31 +00:00
Airat Arifullin 4080b99310 [#1061] node: Set TTL for morph rule cache from morph config
DCO action / DCO (pull_request) Successful in 1m43s Details
Build / Build Components (1.21) (pull_request) Successful in 4m3s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m1s Details
Build / Build Components (1.20) (pull_request) Successful in 5m28s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m16s Details
Tests and linters / Lint (pull_request) Successful in 7m0s Details
Tests and linters / gopls check (pull_request) Successful in 6m47s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m55s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m43s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m6s Details
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-02 12:40:56 +03:00
Ekaterina Lebedeva d5194ab2a6 [#949] metabase: fix shard.UpdateID()
Vulncheck / Vulncheck (pull_request) Successful in 1m20s Details
DCO action / DCO (pull_request) Successful in 1m59s Details
Build / Build Components (1.21) (pull_request) Successful in 3m25s Details
Build / Build Components (1.20) (pull_request) Successful in 4m46s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m5s Details
Tests and linters / gopls check (pull_request) Successful in 6m17s Details
Tests and linters / Lint (pull_request) Successful in 7m7s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m38s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m51s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m56s Details
metabase.Open() now reports metabase mode metric. shard.UpdateID()
needs to read shard ID from metabase => needs to open metabase.
It caused reporting 'shard undefined' metrics. To avoid reporting
wrong metrics metabase.GetShardID() was added which also opens
metabase and does not report metrics.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-04-01 17:27:34 +03:00
Ekaterina Lebedeva 81a0346a96 [#949] metabase: fix metabase mode metric
It used to always show CLOSED regardless of actual mode.
Now metric represents actual metabase mode of operations.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-04-01 17:27:34 +03:00
Evgenii Stratonikov e12fcc041d [#1059] services/tree: Fast sorted listing
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-01 12:37:34 +00:00
Evgenii Stratonikov f23e38c285 Revert "[#446] engine: Move to read-only on blobstor errors"
DCO action / DCO (pull_request) Successful in 2m14s Details
Build / Build Components (1.20) (pull_request) Successful in 4m7s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m30s Details
Build / Build Components (1.21) (pull_request) Successful in 4m15s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m41s Details
Tests and linters / Lint (pull_request) Successful in 6m6s Details
Tests and linters / gopls check (pull_request) Successful in 6m42s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m47s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m21s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m20s Details
This reverts commit 69df0d21c2.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-01 12:48:30 +03:00
Dmitrii Stepanov 942d83611b [#874] engine: Revert Check object existance concurrently
This reverts commit f526f49995.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-01 08:42:34 +00:00
Evgenii Stratonikov fd8cdb9671 [#1057] netmap: Do not iterate over external addresses in Node
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-29 20:46:24 +00:00
Ekaterina Lebedeva 0990a9b0bd [#1055] blobstor: fix mode metric
It used to always show CLOSED after setting shard mode
to read-only regardless of actual mode.
Now metric represents actual blobstor mode of operations.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-03-29 20:44:47 +00:00
Ekaterina Lebedeva 8690db697c [#1056] Makefile: fix target locode-download
DCO action / DCO (pull_request) Successful in 6m2s Details
Build / Build Components (1.20) (pull_request) Successful in 9m35s Details
Build / Build Components (1.21) (pull_request) Successful in 10m43s Details
Vulncheck / Vulncheck (pull_request) Successful in 14m6s Details
Tests and linters / gopls check (pull_request) Successful in 7m19s Details
Tests and linters / Staticcheck (pull_request) Successful in 7m57s Details
Tests and linters / Lint (pull_request) Successful in 11m0s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 11m40s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m58s Details
Tests and linters / Tests with -race (pull_request) Successful in 12m5s Details
wget fails if .cache dir does not exist

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-03-28 14:50:27 +03:00
Evgenii Stratonikov c7a12ca3d8 [#1054] network: Optimize IsTLSEnabled()
DCO action / DCO (pull_request) Successful in 5m33s Details
Vulncheck / Vulncheck (pull_request) Successful in 5m54s Details
Build / Build Components (1.21) (pull_request) Successful in 10m3s Details
Tests and linters / gopls check (pull_request) Successful in 11m49s Details
Build / Build Components (1.20) (pull_request) Successful in 12m47s Details
Tests and linters / Staticcheck (pull_request) Successful in 13m21s Details
Tests and linters / Lint (pull_request) Successful in 14m21s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 16m39s Details
Tests and linters / Tests with -race (pull_request) Successful in 16m46s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 16m59s Details
No big deal, but it is called multiple times in sorting routine, this
easily results in 20 allocations per group traversal.

```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/network
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                    │     old     │                 new                 │
                    │   sec/op    │   sec/op     vs base                │
AddressTLSEnabled-8   184.6n ± 1%   103.3n ± 6%  -44.04% (p=0.000 n=10)

                    │    old     │                new                │
                    │    B/op    │   B/op    vs base                 │
AddressTLSEnabled-8   704.0 ± 0%   0.0 ± 0%  -100.00% (p=0.000 n=10)

                    │    old     │                 new                 │
                    │ allocs/op  │ allocs/op   vs base                 │
AddressTLSEnabled-8   1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-21 18:48:35 +03:00
Dmitrii Stepanov c09c701613 [#1048] metabase: Fix drop buckets during resync
Vulncheck / Vulncheck (pull_request) Successful in 1m35s Details
DCO action / DCO (pull_request) Successful in 2m12s Details
Build / Build Components (1.21) (pull_request) Successful in 3m57s Details
Build / Build Components (1.20) (pull_request) Successful in 4m45s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m32s Details
Tests and linters / gopls check (pull_request) Successful in 5m33s Details
Tests and linters / Lint (pull_request) Successful in 6m18s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m8s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m14s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m36s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-19 14:28:31 +03:00
Dmitrii Stepanov 5d58b44bc8 [#1044] node: Drop unused methods from APE implementation
DCO action / DCO (pull_request) Successful in 1m23s Details
Build / Build Components (1.21) (pull_request) Successful in 3m26s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m31s Details
Build / Build Components (1.20) (pull_request) Successful in 4m25s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m40s Details
Tests and linters / gopls check (pull_request) Successful in 5m46s Details
Tests and linters / Lint (pull_request) Successful in 6m17s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m30s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m34s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 9m0s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-18 18:44:10 +03:00
Dmitrii Stepanov 6bf77cabd4 [#1044] ape: Add morph chain cache
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-18 18:44:10 +03:00
Airat Arifullin 6959e617c4 [#1047] object: Set container owner ID property to ape request
* Introduce ContainerOwner field in RequestContext.
* Set ContainerOwner in aclv2 middleware.
* Set PropertyKeyContainerOwnerID for object ape request.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-18 15:39:50 +00:00
Anton Nikiforov 7278201753 [#1030] adm: Add command `morph ape list-targets`
Vulncheck / Vulncheck (pull_request) Successful in 4m16s Details
DCO action / DCO (pull_request) Successful in 4m14s Details
Build / Build Components (1.20) (pull_request) Successful in 5m58s Details
Build / Build Components (1.21) (pull_request) Successful in 5m56s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m35s Details
Tests and linters / Lint (pull_request) Successful in 7m2s Details
Tests and linters / gopls check (pull_request) Successful in 6m55s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m5s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 9m39s Details
Tests and linters / Tests with -race (pull_request) Successful in 11m35s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-15 13:48:43 +03:00
Evgenii Stratonikov bd216b79cb Release v0.38.0
DCO action / DCO (pull_request) Successful in 9m41s Details
Vulncheck / Vulncheck (pull_request) Successful in 11m6s Details
Build / Build Components (1.21) (pull_request) Successful in 16m3s Details
Build / Build Components (1.20) (pull_request) Successful in 17m4s Details
Tests and linters / Staticcheck (pull_request) Successful in 18m48s Details
Tests and linters / gopls check (pull_request) Successful in 18m43s Details
Tests and linters / Tests with -race (pull_request) Failing after 25m33s Details
Tests and linters / Lint (pull_request) Successful in 25m49s Details
Tests and linters / Tests (1.21) (pull_request) Failing after 25m38s Details
Tests and linters / Tests (1.20) (pull_request) Failing after 25m43s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-15 10:16:01 +03:00
Airat Arifullin d7be70e93f [#1040] object: Wrap CheckAPE errors to status errors
* All methods should wrap CheckAPE error, if it occurs, to
  status error.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-14 07:34:03 +00:00
Anton Nikiforov fb9219af39 [#976] Fix trailing whitespace and end of file
DCO action / DCO (pull_request) Successful in 1m24s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m43s Details
Build / Build Components (1.21) (pull_request) Successful in 6m3s Details
Build / Build Components (1.20) (pull_request) Successful in 6m16s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m47s Details
Tests and linters / Lint (pull_request) Successful in 7m32s Details
Tests and linters / gopls check (pull_request) Successful in 9m12s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m36s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m27s Details
Tests and linters / Tests with -race (pull_request) Successful in 3m31s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-13 15:33:26 +03:00
Anton Nikiforov bf70d77844 [#976] adm: Allow to remove all chains by target
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-13 15:33:26 +03:00
Anton Nikiforov 11fde3cde4 [#976] cli: Allow to remove all chains by target
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-13 15:33:26 +03:00
Anton Nikiforov 5ee5f1df42 [#976] control: Introduce new method `RemoveChainLocalOverridesByTarget`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-13 15:33:19 +03:00
Anton Nikiforov 2d595ec15f [#976] ape: Update dependency
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-13 14:54:19 +03:00
Anton Nikiforov 7ed07d2dfd [#976] morph: Implement missing methods for `SwitchRPCGuardedActor`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-13 14:54:19 +03:00
Dmitrii Stepanov 0a600521ad [#1043] dev: Add empty pass config
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-13 10:29:45 +00:00
Dmitrii Stepanov 17f5463389 [#1043] cli: Add reset evacuation status command
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-13 10:29:45 +00:00
Dmitrii Stepanov 31e2396a5f [#1043] control: Add ResetEvacuationStatus implementation
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-13 10:29:45 +00:00
Dmitrii Stepanov 926cdeb072 [#1043] services: Regenerate proto
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-13 10:29:45 +00:00
Airat Arifullin d1d53d2bb6 [#963] node: Add logging for waitNotaryDeposit
DCO action / DCO (pull_request) Successful in 2m19s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m13s Details
Build / Build Components (1.21) (pull_request) Successful in 4m13s Details
Build / Build Components (1.20) (pull_request) Successful in 4m18s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m57s Details
Tests and linters / Lint (pull_request) Successful in 6m31s Details
Tests and linters / gopls check (pull_request) Successful in 7m11s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m42s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 10m30s Details
Tests and linters / Tests with -race (pull_request) Successful in 10m33s Details
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-13 11:59:51 +03:00