3025 commits

Author	SHA1	Message	Date
Dmitrii Stepanov	b66ea2c713	[#1029] metabase: Add refill metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 19:47:46 +03:00
Dmitrii Stepanov	2677d94054	[#1024] blobovnicza: Copy data on iterate ... DB value is only valid while the tx is alive. But handler may to run something in other goroutine. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 19:06:38 +03:00
Dmitrii Stepanov	5b7a7d7436	[#1024] shard: Resync metabase concurrently ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 19:06:38 +03:00
Dmitrii Stepanov	c5f9b470da	[#1024] shard: Add refill metabase benchmark ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 19:06:38 +03:00
Dmitrii Stepanov	5b8200de88	[#984] blobovnicza: Do not fail rebuild on big objects ... If blobovnicza contains objects larger than object size parameter value, then rebuild fails with an error, because there is no such bucket in database. This commit forces to create bucket on rebuild. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 11:51:18 +00:00
Anton Nikiforov	2b88361849	[#1062] object: Fix buffer allocation for PayloadRange ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-09 11:59:07 +03:00
Dmitrii Stepanov	f5b67c6735	[#1064] policer: Disable EC processing ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 07:08:53 +00:00
Dmitrii Stepanov	1c5e0f90aa	[#1064] putsvc: Add EC put ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 07:08:53 +00:00
Dmitrii Stepanov	39da643354	[#1064] putsvc: Refactor distributed target ... Extract object builder. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 07:08:53 +00:00
Alexander Chuprov	17f7adb640	[#1065] adm: Add support EC parameters ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-04-08 12:27:30 +03:00
Anton Nikiforov	9aa533e59a	[#1072] node, ir: Add new config option kludge_compatibility_mode ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-04 11:05:25 +03:00
Anton Nikiforov	d614f04a0a	[#1072] Fix gofumpt issues ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-03 22:21:14 +03:00
aarifullin	4738508ce2	[#1063] go.mod: Update SDK version ... * Update frostfs-sdk and frostfs-api-go versions. * Refactor depreacted method ReplicaNumberByIndex. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-04-03 10:33:51 +00:00
Evgenii Stratonikov	ff4c23f59a	[#1070] services/tree: Fix fast listing depth processing ... For unsorted `GetSubTree()` we return a single node for depth=1. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-02 14:41:31 +00:00
Evgenii Stratonikov	17af91619a	[#1070] pilorama: Fix cycling behaviour for sorted listing ... In case there are no items left, return empty slice. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-02 14:41:31 +00:00
Ekaterina Lebedeva	d5194ab2a6	[#949] metabase: fix shard.UpdateID() ... metabase.Open() now reports metabase mode metric. shard.UpdateID() needs to read shard ID from metabase => needs to open metabase. It caused reporting 'shard undefined' metrics. To avoid reporting wrong metrics metabase.GetShardID() was added which also opens metabase and does not report metrics. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-04-01 17:27:34 +03:00
Ekaterina Lebedeva	81a0346a96	[#949] metabase: fix metabase mode metric ... It used to always show CLOSED regardless of actual mode. Now metric represents actual metabase mode of operations. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-04-01 17:27:34 +03:00
Evgenii Stratonikov	e12fcc041d	[#1059] services/tree: Fast sorted listing ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-01 12:37:34 +00:00
Evgenii Stratonikov	f23e38c285	Revert "[#446] engine: Move to read-only on blobstor errors" ... This reverts commit 69df0d21c2. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-01 12:48:30 +03:00
Dmitrii Stepanov	942d83611b	[#874] engine: Revert Check object existance concurrently ... This reverts commit f526f49995. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-01 08:42:34 +00:00
Evgenii Stratonikov	fd8cdb9671	[#1057] netmap: Do not iterate over external addresses in Node ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-03-29 20:46:24 +00:00
Ekaterina Lebedeva	0990a9b0bd	[#1055] blobstor: fix mode metric ... It used to always show CLOSED after setting shard mode to read-only regardless of actual mode. Now metric represents actual blobstor mode of operations. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-03-29 20:44:47 +00:00
Evgenii Stratonikov	c7a12ca3d8	[#1054] network: Optimize IsTLSEnabled() ... No big deal, but it is called multiple times in sorting routine, this easily results in 20 allocations per group traversal. ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/network cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ new │ │ sec/op │ sec/op vs base │ AddressTLSEnabled-8 184.6n ± 1% 103.3n ± 6% -44.04% (p=0.000 n=10) │ old │ new │ │ B/op │ B/op vs base │ AddressTLSEnabled-8 704.0 ± 0% 0.0 ± 0% -100.00% (p=0.000 n=10) │ old │ new │ │ allocs/op │ allocs/op vs base │ AddressTLSEnabled-8 1.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-03-21 18:48:35 +03:00
Dmitrii Stepanov	c09c701613	[#1048] metabase: Fix drop buckets during resync ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-19 14:28:31 +03:00
aarifullin	6959e617c4	[#1047] object: Set container owner ID property to ape request ... * Introduce ContainerOwner field in RequestContext. * Set ContainerOwner in aclv2 middleware. * Set PropertyKeyContainerOwnerID for object ape request. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-03-18 15:39:50 +00:00
aarifullin	d7be70e93f	[#1040] object: Wrap CheckAPE errors to status errors ... * All methods should wrap CheckAPE error, if it occurs, to status error. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-03-14 07:34:03 +00:00
Anton Nikiforov	5ee5f1df42	[#976] control: Introduce new method RemoveChainLocalOverridesByTarget ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-13 15:33:19 +03:00
Anton Nikiforov	2d595ec15f	[#976] ape: Update dependency ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-13 14:54:19 +03:00
Anton Nikiforov	7ed07d2dfd	[#976] morph: Implement missing methods for SwitchRPCGuardedActor ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-13 14:54:19 +03:00
Dmitrii Stepanov	17f5463389	[#1043] cli: Add reset evacuation status command ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-13 10:29:45 +00:00
Dmitrii Stepanov	31e2396a5f	[#1043] control: Add ResetEvacuationStatus implementation ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-13 10:29:45 +00:00
Dmitrii Stepanov	926cdeb072	[#1043] services: Regenerate proto ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-13 10:29:45 +00:00
Airat Arifullin	5c252c9193	[#1039] object: Skip APE check for certain request roles ... * Skip APE check if a role is Container. * Skip APE check if a role is IR and methods are get-like. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-03-12 16:15:20 +03:00
Anton Nikiforov	3195142d67	[#959] writecache: Avoid manipulation with cache in DEGRADED mode ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-11 18:35:41 +00:00
Dmitrii Stepanov	d433b49265	[#973] node: Resolve perfsprint linter ... `fmt.Errorf can be replaced with errors.New` and `fmt.Sprintf can be replaced with string addition` Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-11 17:55:50 +03:00
Dmitrii Stepanov	66a26b7775	[#973] node: Resolve revive: unused-parameter linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-11 17:11:49 +03:00
Dmitrii Stepanov	0882840bf5	[#634] shard: Add writecache inhume tests ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-06 13:12:34 +03:00
Anton Nikiforov	ae5bb87e70	Revert "[#866] Use TTL for blobovnicza tree cache" ... This reverts commit d9cbb16bd3. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-01 19:29:33 +03:00
Dmitrii Stepanov	d6534fd755	[#1016] frostfs-node: Fix gopls issues ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-01 12:13:43 +03:00
Airat Arifullin	93bf9acbc2	[#898] control: Remove removed flag from RemoveChainLocalOverrideResponse ... * Remove removed flag in service.proto for RemoveChainLocalOverrideResponse. * Regenerate control API. * Return error only if RemoveOverride returns non-NotFound code. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-28 19:07:07 +00:00
Airat Arifullin	75a1a95c2c	[#986] tree: Skip ACL checks if basicACL mask is unset ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-28 19:05:57 +00:00
Airat Arifullin	b1d171c261	[#986] container: Interpret APE NoRuleFound as request deny ... * If APE check returns NoRuleFound, then it is taken for request deny. * Add more unit-test for ape container middleware. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-28 19:05:57 +00:00
Airat Arifullin	7cc368e188	[#986] object: Introduce soft ape checks ... * Soft APE check means that APE should allow request even it gets status NoRuleFound for a request. Otherwise, it is interpreted as Deny. * Soft APE check is performed if basic ACL mask is not set. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-28 19:05:57 +00:00
Dmitrii Stepanov	918613546f	[#1008] metabase: Do not update storageID on put ... There may be a race condition between put an object and flushing the writecache: 1. Put object to the writecache 2. Writecache flushes object to the blobstore and sets blobstore's storageID 3. Put object to the metabase, set writecache's storageID Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-28 11:01:50 +03:00
Dmitrii Stepanov	2ad433dbcb	[#1005] engine: Drop shards weights ... Unused. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-26 17:25:05 +03:00
Evgenii Stratonikov	4b13b85173	[#1000] morph: Fix batch size in TraverseIterator() ... Initial prefetch size can be arbitrary an restricted only by VM/RPC limits. For TraverseIterator() there is an explicit check on the server-side, though. Introduced in df055fead5. Refs #931. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-23 06:24:40 +00:00
Evgenii Stratonikov	e18f0f5178	[#999] morph: Use Global scope for proxy contract ... Proxy contract can now be used as an owner of NNS domains, thus we need it not only to pay for the transaction but also to check domain ownership. CalledByEntry is not enough, because we may register NNS domains owned by proxy indirectly from the container contract. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-21 14:28:42 +03:00
Dmitrii Stepanov	7470c383dd	[#997] metabase: Drop toMoveIt bucket ... It is not used. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-21 10:06:05 +03:00
Evgenii Stratonikov	adf7ebab5b	[#996] metabase: Speed up bucket creation ... Most of the time it exits, e.g. when it is per-container and use on each object PUT. Bbolt implementation first tries to create bucket and then returns it if it exists. Create operation uses cursor and thus is not very lightweight, we can avoid it. ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ new │ │ sec/op │ sec/op vs base │ Put/parallel-8 174.4µ ± 3% 163.3µ ± 3% -6.39% (p=0.000 n=10) Put/sequential-8 263.3µ ± 2% 259.0µ ± 1% -1.64% (p=0.000 n=10) geomean 214.3µ 205.6µ -4.05% │ old │ new │ │ B/op │ B/op vs base │ Put/parallel-8 275.3Ki ± 3% 281.1Ki ± 4% ~ (p=0.063 n=10) Put/sequential-8 413.0Ki ± 2% 426.6Ki ± 2% +3.29% (p=0.003 n=10) geomean 337.2Ki 346.3Ki +2.70% │ old │ new │ │ allocs/op │ allocs/op vs base │ Put/parallel-8 678.0 ± 1% 524.5 ± 2% -22.64% (p=0.000 n=10) Put/sequential-8 1.329k ± 0% 1.183k ± 0% -10.91% (p=0.000 n=10) geomean 949.1 787.9 -16.98% ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-20 15:42:58 +00:00
Evgenii Stratonikov	9f68305c2e	Revert "[#972] pilorama: Remove removeDuplicatesInPlace()" ... This reverts commit 45fd4e4ff1. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 15:36:01 +00:00