dstepanov-yadro
/
frostfs-node
forked from TrueCloudLab/frostfs-node
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,614 Commits
210 Branches
26 Tags
28 MiB
Commit Graph

4614 Commits (feat/resync_speedup)

Author SHA1 Message Date
Dmitrii Stepanov 1b17258c04 [#1029] metabase: Add refill metrics 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-10 13:05:44 +03:00
Dmitrii Stepanov e3d9dd6ee8 [#1024] blobovnicza: Copy data on iterate 
DB value is only valid while the tx is alive.
But handler may to run something in other goroutine.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-10 10:21:11 +03:00
Dmitrii Stepanov 57466594fb [#1024] shard: Resync metabase concurrently 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-10 10:21:10 +03:00
Dmitrii Stepanov 1005bf4f56 [#1024] shard: Add refill metabase benchmark 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-10 10:21:10 +03:00
Dmitrii Stepanov 76398c06b0 [#1080] metabase: Add StorageID metric 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-10 10:00:08 +03:00
Dmitrii Stepanov 7b1adfed3e [#1080] metabase: Open bucket for container counter once 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-10 10:00:08 +03:00
Dmitrii Stepanov e74bdaa5d5 [#1080] ape: Use value for APE request 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-09 18:42:03 +03:00
Dmitrii Stepanov 338d8cbebd [#1080] ape: Do not read object headers before Head/Get 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-09 15:27:40 +03:00
Dmitrii Stepanov 5b8200de88 [#984] blobovnicza: Do not fail rebuild on big objects 
If blobovnicza contains objects larger than object size parameter
value, then rebuild fails with an error, because there is no such
bucket in database. This commit forces to create bucket on rebuild.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-09 11:51:18 +00:00
Anton Nikiforov 2b88361849 [#1062] object: Fix buffer allocation for `PayloadRange` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-09 11:59:07 +03:00
Dmitrii Stepanov f5b67c6735 [#1064] policer: Disable EC processing 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-09 07:08:53 +00:00
Dmitrii Stepanov bdf4990904 [#1064] cli: Add EC header output to object head 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-09 07:08:53 +00:00
Dmitrii Stepanov 8668cbf147 [#1064] dev: Add `IR + 4 storage nodes` configuration 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-09 07:08:53 +00:00
Dmitrii Stepanov 1c5e0f90aa [#1064] putsvc: Add EC put 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-09 07:08:53 +00:00
Dmitrii Stepanov 39da643354 [#1064] putsvc: Refactor distributed target 
Extract object builder.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-09 07:08:53 +00:00
Alexander Chuprov 92569b9bbf [#1065] cli: Add support EC parameters 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-04-08 12:27:51 +03:00
Alexander Chuprov 17f7adb640 [#1065] adm: Add support EC parameters 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-04-08 12:27:30 +03:00
Alexander Chuprov 0290f86579 [#1065] adm: refactor dump-config 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-04-08 12:06:23 +03:00
Anton Nikiforov ffb1a6f81a [#1072] Fix issue from `govulncheck` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-05 18:20:07 +03:00
Anton Nikiforov 9aa533e59a [#1072] node, ir: Add new config option `kludge_compatibility_mode` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-04 11:05:25 +03:00
Anton Nikiforov d614f04a0a [#1072] Fix gofumpt issues 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-03 22:21:14 +03:00
Airat Arifullin 531542ce60 [#1063] cli: Validate container creation for EC policy 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-04-03 10:33:51 +00:00
Airat Arifullin 4738508ce2 [#1063] go.mod: Update SDK version 
* Update frostfs-sdk and frostfs-api-go versions.
* Refactor depreacted method ReplicaNumberByIndex.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-04-03 10:33:51 +00:00
Evgenii Stratonikov ff4c23f59a [#1070] services/tree: Fix fast listing depth processing 
For unsorted `GetSubTree()` we return a single node for depth=1.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-02 14:41:31 +00:00
Evgenii Stratonikov 17af91619a [#1070] pilorama: Fix cycling behaviour for sorted listing 
In case there are no items left, return empty slice.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-02 14:41:31 +00:00
Airat Arifullin 4080b99310 [#1061] node: Set TTL for morph rule cache from morph config 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-04-02 12:40:56 +03:00
Ekaterina Lebedeva d5194ab2a6 [#949] metabase: fix shard.UpdateID() 
metabase.Open() now reports metabase mode metric. shard.UpdateID()
needs to read shard ID from metabase => needs to open metabase.
It caused reporting 'shard undefined' metrics. To avoid reporting
wrong metrics metabase.GetShardID() was added which also opens
metabase and does not report metrics.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-04-01 17:27:34 +03:00
Ekaterina Lebedeva 81a0346a96 [#949] metabase: fix metabase mode metric 
It used to always show CLOSED regardless of actual mode.
Now metric represents actual metabase mode of operations.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-04-01 17:27:34 +03:00
Evgenii Stratonikov e12fcc041d [#1059] services/tree: Fast sorted listing 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-01 12:37:34 +00:00
Evgenii Stratonikov f23e38c285 Revert "[#446] engine: Move to read-only on blobstor errors" 
This reverts commit 69df0d21c2.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-01 12:48:30 +03:00
Dmitrii Stepanov 942d83611b [#874] engine: Revert Check object existance concurrently 
This reverts commit f526f49995.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-01 08:42:34 +00:00
Evgenii Stratonikov fd8cdb9671 [#1057] netmap: Do not iterate over external addresses in Node 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-03-29 20:46:24 +00:00
Ekaterina Lebedeva 0990a9b0bd [#1055] blobstor: fix mode metric 
It used to always show CLOSED after setting shard mode
to read-only regardless of actual mode.
Now metric represents actual blobstor mode of operations.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-03-29 20:44:47 +00:00
Ekaterina Lebedeva 8690db697c [#1056] Makefile: fix target locode-download 
wget fails if .cache dir does not exist

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-03-28 14:50:27 +03:00
Evgenii Stratonikov c7a12ca3d8 [#1054] network: Optimize IsTLSEnabled() 
No big deal, but it is called multiple times in sorting routine, this
easily results in 20 allocations per group traversal.

```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/network
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                    │     old     │                 new                 │
                    │   sec/op    │   sec/op     vs base                │
AddressTLSEnabled-8   184.6n ± 1%   103.3n ± 6%  -44.04% (p=0.000 n=10)

                    │    old     │                new                │
                    │    B/op    │   B/op    vs base                 │
AddressTLSEnabled-8   704.0 ± 0%   0.0 ± 0%  -100.00% (p=0.000 n=10)

                    │    old     │                 new                 │
                    │ allocs/op  │ allocs/op   vs base                 │
AddressTLSEnabled-8   1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=10)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-03-21 18:48:35 +03:00
Dmitrii Stepanov c09c701613 [#1048] metabase: Fix drop buckets during resync 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-19 14:28:31 +03:00
Dmitrii Stepanov 5d58b44bc8 [#1044] node: Drop unused methods from APE implementation 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-18 18:44:10 +03:00
Dmitrii Stepanov 6bf77cabd4 [#1044] ape: Add morph chain cache 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-18 18:44:10 +03:00
Airat Arifullin 6959e617c4 [#1047] object: Set container owner ID property to ape request 
* Introduce ContainerOwner field in RequestContext.
* Set ContainerOwner in aclv2 middleware.
* Set PropertyKeyContainerOwnerID for object ape request.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-03-18 15:39:50 +00:00
Anton Nikiforov 7278201753 [#1030] adm: Add command `morph ape list-targets` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-03-15 13:48:43 +03:00
Evgenii Stratonikov bd216b79cb Release v0.38.0 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-03-15 10:16:01 +03:00
Airat Arifullin d7be70e93f [#1040] object: Wrap CheckAPE errors to status errors 
* All methods should wrap CheckAPE error, if it occurs, to
  status error.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-03-14 07:34:03 +00:00
Anton Nikiforov fb9219af39 [#976] Fix trailing whitespace and end of file 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-03-13 15:33:26 +03:00
Anton Nikiforov bf70d77844 [#976] adm: Allow to remove all chains by target 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-03-13 15:33:26 +03:00
Anton Nikiforov 11fde3cde4 [#976] cli: Allow to remove all chains by target 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-03-13 15:33:26 +03:00
Anton Nikiforov 5ee5f1df42 [#976] control: Introduce new method `RemoveChainLocalOverridesByTarget` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-03-13 15:33:19 +03:00
Anton Nikiforov 2d595ec15f [#976] ape: Update dependency 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-03-13 14:54:19 +03:00
Anton Nikiforov 7ed07d2dfd [#976] morph: Implement missing methods for `SwitchRPCGuardedActor` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-03-13 14:54:19 +03:00
Dmitrii Stepanov 0a600521ad [#1043] dev: Add empty pass config 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-13 10:29:45 +00:00
Dmitrii Stepanov 17f5463389 [#1043] cli: Add reset evacuation status command 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-13 10:29:45 +00:00