3347 commits

Author	SHA1	Message	Date
Anton Nikiforov	7ed07d2dfd	[#976] morph: Implement missing methods for SwitchRPCGuardedActor ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-13 14:54:19 +03:00
Dmitrii Stepanov	17f5463389	[#1043] cli: Add reset evacuation status command ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-13 10:29:45 +00:00
Dmitrii Stepanov	31e2396a5f	[#1043] control: Add ResetEvacuationStatus implementation ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-13 10:29:45 +00:00
Dmitrii Stepanov	926cdeb072	[#1043] services: Regenerate proto ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-13 10:29:45 +00:00
Airat Arifullin	5c252c9193	[#1039] object: Skip APE check for certain request roles ... * Skip APE check if a role is Container. * Skip APE check if a role is IR and methods are get-like. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-03-12 16:15:20 +03:00
Anton Nikiforov	3195142d67	[#959] writecache: Avoid manipulation with cache in DEGRADED mode ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-11 18:35:41 +00:00
Dmitrii Stepanov	d433b49265	[#973] node: Resolve perfsprint linter ... `fmt.Errorf can be replaced with errors.New` and `fmt.Sprintf can be replaced with string addition` Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-11 17:55:50 +03:00
Dmitrii Stepanov	66a26b7775	[#973] node: Resolve revive: unused-parameter linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-11 17:11:49 +03:00
Dmitrii Stepanov	0882840bf5	[#634] shard: Add writecache inhume tests ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-06 13:12:34 +03:00
Anton Nikiforov	ae5bb87e70	Revert "[#866] Use TTL for blobovnicza tree cache" ... This reverts commit d9cbb16bd3. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-01 19:29:33 +03:00
Dmitrii Stepanov	d6534fd755	[#1016] frostfs-node: Fix gopls issues ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-01 12:13:43 +03:00
Airat Arifullin	93bf9acbc2	[#898] control: Remove removed flag from RemoveChainLocalOverrideResponse ... * Remove removed flag in service.proto for RemoveChainLocalOverrideResponse. * Regenerate control API. * Return error only if RemoveOverride returns non-NotFound code. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-28 19:07:07 +00:00
Airat Arifullin	75a1a95c2c	[#986] tree: Skip ACL checks if basicACL mask is unset ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-28 19:05:57 +00:00
Airat Arifullin	b1d171c261	[#986] container: Interpret APE NoRuleFound as request deny ... * If APE check returns NoRuleFound, then it is taken for request deny. * Add more unit-test for ape container middleware. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-28 19:05:57 +00:00
Airat Arifullin	7cc368e188	[#986] object: Introduce soft ape checks ... * Soft APE check means that APE should allow request even it gets status NoRuleFound for a request. Otherwise, it is interpreted as Deny. * Soft APE check is performed if basic ACL mask is not set. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-28 19:05:57 +00:00
Dmitrii Stepanov	918613546f	[#1008] metabase: Do not update storageID on put ... There may be a race condition between put an object and flushing the writecache: 1. Put object to the writecache 2. Writecache flushes object to the blobstore and sets blobstore's storageID 3. Put object to the metabase, set writecache's storageID Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-28 11:01:50 +03:00
Dmitrii Stepanov	2ad433dbcb	[#1005] engine: Drop shards weights ... Unused. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-26 17:25:05 +03:00
Evgenii Stratonikov	4b13b85173	[#1000] morph: Fix batch size in TraverseIterator() ... Initial prefetch size can be arbitrary an restricted only by VM/RPC limits. For TraverseIterator() there is an explicit check on the server-side, though. Introduced in df055fead5. Refs #931. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-23 06:24:40 +00:00
Evgenii Stratonikov	e18f0f5178	[#999] morph: Use Global scope for proxy contract ... Proxy contract can now be used as an owner of NNS domains, thus we need it not only to pay for the transaction but also to check domain ownership. CalledByEntry is not enough, because we may register NNS domains owned by proxy indirectly from the container contract. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-21 14:28:42 +03:00
Dmitrii Stepanov	7470c383dd	[#997] metabase: Drop toMoveIt bucket ... It is not used. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-21 10:06:05 +03:00
Evgenii Stratonikov	adf7ebab5b	[#996] metabase: Speed up bucket creation ... Most of the time it exits, e.g. when it is per-container and use on each object PUT. Bbolt implementation first tries to create bucket and then returns it if it exists. Create operation uses cursor and thus is not very lightweight, we can avoid it. ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ new │ │ sec/op │ sec/op vs base │ Put/parallel-8 174.4µ ± 3% 163.3µ ± 3% -6.39% (p=0.000 n=10) Put/sequential-8 263.3µ ± 2% 259.0µ ± 1% -1.64% (p=0.000 n=10) geomean 214.3µ 205.6µ -4.05% │ old │ new │ │ B/op │ B/op vs base │ Put/parallel-8 275.3Ki ± 3% 281.1Ki ± 4% ~ (p=0.063 n=10) Put/sequential-8 413.0Ki ± 2% 426.6Ki ± 2% +3.29% (p=0.003 n=10) geomean 337.2Ki 346.3Ki +2.70% │ old │ new │ │ allocs/op │ allocs/op vs base │ Put/parallel-8 678.0 ± 1% 524.5 ± 2% -22.64% (p=0.000 n=10) Put/sequential-8 1.329k ± 0% 1.183k ± 0% -10.91% (p=0.000 n=10) geomean 949.1 787.9 -16.98% ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-20 15:42:58 +00:00
Evgenii Stratonikov	9f68305c2e	Revert "[#972] pilorama: Remove removeDuplicatesInPlace()" ... This reverts commit 45fd4e4ff1. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 15:36:01 +00:00
Evgenii Stratonikov	dad56d2e98	Revert "[#972] Use min/max builtins" ... This reverts commit 89784b2e0a. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 15:36:01 +00:00
Evgenii Stratonikov	7627d08914	Revert "[#972] Use require.ElementsMatch() where possible" ... This reverts commit 6d9707ff1f. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 15:36:01 +00:00
Evgenii Stratonikov	3359349acb	Revert "[#972] Use slices.Sort* when useful" ... This reverts commit b871d7a5e8. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 15:36:01 +00:00
Evgenii Stratonikov	946f2ec2bf	Revert "[#972] Drop x/exp/slices dependency" ... This reverts commit f3e50772fd. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 15:36:01 +00:00
Dmitrii Stepanov	13d5cd3e21	[#991] logger: Fix journald logger ... Allow to change logger level. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-19 16:18:13 +03:00
Evgenii Stratonikov	f3e50772fd	[#972] Drop x/exp/slices dependency ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 13:13:09 +00:00
Evgenii Stratonikov	b871d7a5e8	[#972] Use slices.Sort* when useful ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 13:13:09 +00:00
Evgenii Stratonikov	6d9707ff1f	[#972] Use require.ElementsMatch() where possible ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 13:13:09 +00:00
Evgenii Stratonikov	89784b2e0a	[#972] Use min/max builtins ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 13:13:09 +00:00
Evgenii Stratonikov	45fd4e4ff1	[#972] pilorama: Remove removeDuplicatesInPlace() ... Also, check that slices.CompareFunc() indeed passes all the tests before removal. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 13:13:09 +00:00
Evgenii Stratonikov	15fc5bac26	[#972] keyer: Use UncompressedBytes() for marshaling ... elliptic.Marshal() becomes deprecated in go1.21 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 13:13:09 +00:00
Dmitrii Stepanov	2680192ba0	[#988] objectSvc: Fix SetMarshalData for PutSingle ... After api-go update it is required to pass marshal data to `SetMarshalData`. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-15 17:21:08 +03:00
Anton Nikiforov	05b5f5ca85	[#959] writecache: Fix panic on Get when it is not initialized ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-14 16:10:33 +03:00
Anton Nikiforov	2429508ac5	[#959] shard: Skip rebuild in DEGRADED mode ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-14 15:39:28 +03:00
Anton Nikiforov	0bd030507e	[#948] metrics: Set actual value for shard_id after restart ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-13 09:43:21 +03:00
Anton Nikiforov	6a5769d1da	[#948] Fix gofumpt issue ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-13 09:40:46 +03:00
Dmitrii Stepanov	962e5a9c19	[#736] logger: Add journald support ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-12 06:17:22 +00:00
Evgenii Stratonikov	b36a453238	[#970] fstree: Add build tag to enable generic version on linux ... Unless tested, generic version can start gaining bugs. With a separate build tag we can have the best of both worlds: 1. Use optimized implementation for linux by default. 2. Run tests or benchmarks for both. Note that they are not actually run automatically now, but this is at leas possible. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-09 16:12:11 +00:00
Evgenii Stratonikov	abd502215f	[#970] fstree: Move file locking to the generic writer ... It is not a part of FSTree itself, but rather a way to solve concurrent counter update on non-linux implementations. New linux implementations is pretty simple: link fails when the file exists, unlink fails when the file doesn't exist. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-09 16:12:11 +00:00
Evgenii Stratonikov	fb74524ac7	[#970] fstree: Move delete implementation to a separate file ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-09 16:12:11 +00:00
Evgenii Stratonikov	7f692409cf	[#970] fstree: Handle unsupported O_TMPFILE ... Metabase test relied on this behaviour, so fix the test too. Cherry-picking was hard and did too many conflicts, here is an original PR: https://github.com/nspcc-dev/neofs-node/pull/2624 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-09 16:12:11 +00:00
Roman Khimov	fc31b9c947	[#970] fstree: Add linux-specific file writer using O_TMPFILE ... O_TMPFILE is implemented for all modern FSes and it's much easier and safer to use. If application crashes in the middle of writing this file would be gone and won't leave any garbage. Notice that this implementation makes a different choice wrt EEXIST handling, generic one always overwrites, while this one keeps the old data. There is no real performance difference. SSD (slow&old), XFS, Core i7-8565U: Sync ``` name old time/op new time/op delta Put/size=1024,thread=1/fstree-8 1.74ms ± 3% 0.06ms ± 7% -96.31% (p=0.000 n=10+10) Put/size=1024,thread=20/fstree-8 10.0ms ±41% 1.1ms ±18% -88.95% (p=0.000 n=9+10) Put/size=1024,thread=100/fstree-8 32.3ms ±60% 6.5ms ±14% -79.97% (p=0.000 n=10+10) Put/size=1048576,thread=1/fstree-8 17.8ms ±90% 3.4ms ±70% -81.08% (p=0.000 n=10+10) Put/size=1048576,thread=20/fstree-8 103ms ±174% 112ms ±158% ~ (p=0.971 n=10+10) Put/size=1048576,thread=100/fstree-8 949ms ±78% 583ms ±132% ~ (p=0.089 n=10+10) name old alloc/op new alloc/op delta Put/size=1024,thread=1/fstree-8 3.17kB ± 1% 1.96kB ± 0% -38.09% (p=0.000 n=10+10) Put/size=1024,thread=20/fstree-8 59.6kB ± 1% 39.2kB ± 1% -34.30% (p=0.000 n=8+10) Put/size=1024,thread=100/fstree-8 299kB ± 0% 198kB ± 0% -33.90% (p=0.000 n=7+9) Put/size=1048576,thread=1/fstree-8 3.38kB ± 1% 2.36kB ± 1% -30.22% (p=0.000 n=10+10) Put/size=1048576,thread=20/fstree-8 65.7kB ± 4% 47.7kB ± 6% -27.27% (p=0.000 n=10+10) Put/size=1048576,thread=100/fstree-8 351kB ± 8% 245kB ± 8% -30.22% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Put/size=1024,thread=1/fstree-8 30.3 ± 2% 21.0 ± 0% -30.69% (p=0.000 n=10+10) Put/size=1024,thread=20/fstree-8 554 ± 1% 413 ± 0% -25.35% (p=0.000 n=8+10) Put/size=1024,thread=100/fstree-8 2.77k ± 0% 2.07k ± 0% -25.27% (p=0.000 n=7+10) Put/size=1048576,thread=1/fstree-8 32.0 ± 0% 25.0 ± 0% -21.88% (p=0.000 n=9+8) Put/size=1048576,thread=20/fstree-8 609 ± 5% 494 ± 6% -18.93% (p=0.000 n=10+10) Put/size=1048576,thread=100/fstree-8 3.25k ± 9% 2.50k ± 8% -23.21% (p=0.000 n=10+10) ``` No sync ``` name old time/op new time/op delta Put/size=1024,thread=1/fstree-8 71.3µs ±10% 59.8µs ±10% -16.21% (p=0.000 n=10+10) Put/size=1024,thread=20/fstree-8 1.43ms ± 6% 1.22ms ±13% -14.53% (p=0.000 n=10+10) Put/size=1024,thread=100/fstree-8 8.12ms ± 3% 6.36ms ± 2% -21.67% (p=0.000 n=8+9) Put/size=1048576,thread=1/fstree-8 1.88ms ±70% 1.61ms ±78% ~ (p=0.393 n=10+10) Put/size=1048576,thread=20/fstree-8 32.7ms ±28% 34.2ms ±112% ~ (p=0.968 n=9+10) Put/size=1048576,thread=100/fstree-8 262ms ±56% 226ms ±34% ~ (p=0.447 n=10+9) name old alloc/op new alloc/op delta Put/size=1024,thread=1/fstree-8 2.89kB ± 0% 1.96kB ± 0% -32.28% (p=0.000 n=10+10) Put/size=1024,thread=20/fstree-8 58.2kB ± 0% 39.5kB ± 0% -32.09% (p=0.000 n=8+8) Put/size=1024,thread=100/fstree-8 291kB ± 0% 198kB ± 0% -32.19% (p=0.000 n=9+9) Put/size=1048576,thread=1/fstree-8 3.05kB ± 1% 2.13kB ± 1% -30.16% (p=0.000 n=10+9) Put/size=1048576,thread=20/fstree-8 62.6kB ± 0% 44.3kB ± 0% -29.23% (p=0.000 n=9+9) Put/size=1048576,thread=100/fstree-8 302kB ± 0% 210kB ± 1% -30.39% (p=0.000 n=9+9) name old allocs/op new allocs/op delta Put/size=1024,thread=1/fstree-8 27.0 ± 0% 21.0 ± 0% -22.22% (p=0.000 n=10+10) Put/size=1024,thread=20/fstree-8 539 ± 0% 415 ± 0% -22.98% (p=0.000 n=10+10) Put/size=1024,thread=100/fstree-8 2.69k ± 0% 2.07k ± 0% -23.09% (p=0.000 n=9+9) Put/size=1048576,thread=1/fstree-8 28.0 ± 0% 22.3 ± 3% -20.36% (p=0.000 n=8+10) Put/size=1048576,thread=20/fstree-8 577 ± 0% 458 ± 0% -20.72% (p=0.000 n=9+9) Put/size=1048576,thread=100/fstree-8 2.76k ± 0% 2.15k ± 0% -22.05% (p=0.000 n=9+8) ``` HDD (LVM), ext4, Ryzen 5 1600: Sync ``` │ fs.sync-generic │ fs.sync-linux │ │ sec/op │ sec/op vs base │ Put/size=1024,thread=1/fstree-12 34.70m ± 19% 33.59m ± 16% ~ (p=0.529 n=10) Put/size=1024,thread=20/fstree-12 188.8m ± 8% 189.2m ± 16% ~ (p=0.739 n=10) Put/size=1024,thread=100/fstree-12 264.8m ± 22% 273.6m ± 28% ~ (p=0.353 n=10) Put/size=1048576,thread=1/fstree-12 54.90m ± 14% 47.08m ± 18% ~ (p=0.063 n=10) Put/size=1048576,thread=20/fstree-12 244.1m ± 14% 220.4m ± 22% ~ (p=0.579 n=10) Put/size=1048576,thread=100/fstree-12 847.2m ± 5% 893.6m ± 3% +5.48% (p=0.000 n=10) geomean 164.3m 158.9m -3.29% │ fs.sync-generic │ fs.sync-linux │ │ B/op │ B/op vs base │ Put/size=1024,thread=1/fstree-12 3.375Ki ± 1% 2.471Ki ± 1% -26.80% (p=0.000 n=10) Put/size=1024,thread=20/fstree-12 66.62Ki ± 6% 49.21Ki ± 6% -26.15% (p=0.000 n=10) Put/size=1024,thread=100/fstree-12 319.2Ki ± 1% 230.9Ki ± 2% -27.64% (p=0.000 n=10) Put/size=1048576,thread=1/fstree-12 3.457Ki ± 1% 2.559Ki ± 1% -25.97% (p=0.000 n=10) Put/size=1048576,thread=20/fstree-12 66.91Ki ± 1% 49.16Ki ± 1% -26.52% (p=0.000 n=10) Put/size=1048576,thread=100/fstree-12 338.8Ki ± 2% 252.3Ki ± 3% -25.54% (p=0.000 n=10) geomean 42.17Ki 31.02Ki -26.44% │ fs.sync-generic │ fs.sync-linux │ │ allocs/op │ allocs/op vs base │ Put/size=1024,thread=1/fstree-12 33.00 ± 0% 27.00 ± 0% -18.18% (p=0.000 n=10) Put/size=1024,thread=20/fstree-12 639.5 ± 1% 519.0 ± 2% -18.84% (p=0.000 n=10) Put/size=1024,thread=100/fstree-12 3.059k ± 1% 2.478k ± 2% -18.99% (p=0.000 n=10) Put/size=1048576,thread=1/fstree-12 33.50 ± 1% 28.00 ± 4% -16.42% (p=0.000 n=10) Put/size=1048576,thread=20/fstree-12 638.5 ± 1% 520.0 ± 1% -18.56% (p=0.000 n=10) Put/size=1048576,thread=100/fstree-12 3.209k ± 2% 2.655k ± 2% -17.28% (p=0.000 n=10) geomean 405.3 332.1 -18.05% ``` No sync ``` │ fs.nosync-generic │ fs.nosync-linux │ │ sec/op │ sec/op vs base │ Put/size=1024,thread=1/fstree-12 148.2µ ± 20% 136.6µ ± 19% -7.89% (p=0.029 n=10) Put/size=1024,thread=20/fstree-12 1.140m ± 26% 1.364m ± 16% ~ (p=0.143 n=10) Put/size=1024,thread=100/fstree-12 11.93m ± 68% 26.89m ± 62% ~ (p=0.123 n=10) Put/size=1048576,thread=1/fstree-12 1.302m ± 3% 1.287m ± 5% ~ (p=0.481 n=10) Put/size=1048576,thread=20/fstree-12 77.52m ± 8% 74.07m ± 7% ~ (p=0.278 n=10+9) Put/size=1048576,thread=100/fstree-12 226.1m ± ∞ ¹ geomean 5.986m 3.434m +18.60% ² ¹ need >= 6 samples for confidence interval at level 0.95 ² benchmark set differs from baseline; geomeans may not be comparable │ fs.nosync-generic │ fs.nosync-linux │ │ B/op │ B/op vs base │ Put/size=1024,thread=1/fstree-12 2.879Ki ± 0% 1.972Ki ± 0% -31.51% (p=0.000 n=10) Put/size=1024,thread=20/fstree-12 55.94Ki ± 1% 37.90Ki ± 1% -32.25% (p=0.000 n=10) Put/size=1024,thread=100/fstree-12 272.6Ki ± 0% 182.1Ki ± 9% -33.21% (p=0.000 n=10) Put/size=1048576,thread=1/fstree-12 3.158Ki ± 0% 2.259Ki ± 0% -28.46% (p=0.000 n=10) Put/size=1048576,thread=20/fstree-12 58.87Ki ± 0% 41.03Ki ± 0% -30.30% (p=0.000 n=10+9) Put/size=1048576,thread=100/fstree-12 299.8Ki ± ∞ ¹ geomean 36.71Ki 16.60Ki -31.17% ² ¹ need >= 6 samples for confidence interval at level 0.95 ² benchmark set differs from baseline; geomeans may not be comparable │ fs.nosync-generic │ fs.nosync-linux │ │ allocs/op │ allocs/op vs base │ Put/size=1024,thread=1/fstree-12 28.00 ± 0% 22.00 ± 0% -21.43% (p=0.000 n=10) Put/size=1024,thread=20/fstree-12 530.0 ± 0% 407.5 ± 1% -23.11% (p=0.000 n=10) Put/size=1024,thread=100/fstree-12 2.567k ± 0% 1.956k ± 9% -23.77% (p=0.000 n=10) Put/size=1048576,thread=1/fstree-12 30.00 ± 0% 24.00 ± 0% -20.00% (p=0.000 n=10) Put/size=1048576,thread=20/fstree-12 553.5 ± 0% 434.0 ± 0% -21.59% (n=10+9) Put/size=1048576,thread=100/fstree-12 2.803k ± ∞ ¹ geomean 347.9 178.8 -21.99% ² ¹ need >= 6 samples for confidence interval at level 0.95 ² benchmark set differs from baseline; geomeans may not be comparable ``` Signed-off-by: Roman Khimov <roman@nspcc.ru> Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-09 16:12:11 +00:00
Evgenii Stratonikov	ff488b53a1	[#970] fstree: Move write functions to a separate file ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-09 16:12:11 +00:00
Evgenii Stratonikov	9a622a750d	[#970] fstree: Move temporary path handling in a separate function ... Allow to easier test different implementations. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-09 16:12:11 +00:00
Anton Nikiforov	d19ade23c8	[#959] node: Set mode to shard's components when open it ... Avoid opening database for `metabase` and `cache` in `Degraded` mode. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-09 14:04:01 +00:00
Dmitrii Stepanov	db67c21d55	[#947] engine: Evacuate trees to remote nodes ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-09 11:33:15 +03:00
Dmitrii Stepanov	728150d1d2	[#947] engine: Evacuate trees to local shards ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-09 11:33:15 +03:00
Dmitrii Stepanov	15d853ea22	[#947] controlSvc: Return tree evacuation stat ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-09 11:20:39 +03:00
Dmitrii Stepanov	b3f3505ada	[#947] cli: Allow to specify evacuation scope ... It may be required to evacuate only objects or only tree or all, so now it spossible to specify. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-09 11:20:38 +03:00
Dmitrii Stepanov	a6eb66bf9c	[#947] evacuate: Refactor evacuate parameters ... Drop methods to make it easier to extend. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-09 11:20:38 +03:00
Dmitrii Stepanov	8e2a0611f4	[#947] tree: Add method to list all trees ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-09 11:20:38 +03:00
Evgenii Stratonikov	edbe06e07e	[#956] policer/test: Reuse testPool helper ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-08 15:49:43 +00:00
Evgenii Stratonikov	cbfeb72466	[#956] policer: Remove WithMaxCapacity option ... We already provide the pool and this argument is used only for preallocation. No functional changes. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-08 15:49:43 +00:00
Dmitrii Stepanov	cfc5ce7853	[#964] metabase: Drop GC marks if object not found ... GC inhumes expired locks and tombstones on all the shards. So it could be GC mark without object. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-08 07:54:39 +00:00
Evgenii Stratonikov	c3fa902780	[#969] policer: Restrict the number of remembered errors ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-08 10:10:41 +03:00
Evgenii Stratonikov	6010dfdf3d	[#969] policer: Make error skip thread-safe ... Introduces in afd2ba9a66. Refs #914 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-08 10:10:41 +03:00
Evgenii Stratonikov	a6c9a337cd	[#965] morph: Get rid of container.List invocations ... ContainersOf() is better in almost every aspect, besides creating a session when the containers number is between 1024 and 2048 (prefetch script does limited unwrapping). Making List() private helps to ensure it is no longer used and can be safely removed in future. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-07 08:56:27 +00:00
Dmitrii Stepanov	9ba48c582d	[#917] engine: Allow to detach shards ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-06 14:49:47 +03:00
Dmitrii Stepanov	4358d3c423	[#917] controlSvc: Add DetachShards handler ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-06 14:47:52 +03:00
Ekaterina Lebedeva	afd2ba9a66	[#110] Add check for repeated error log in policer ... processObject() returns 3 types of errors: container not found errors, could not get container error and placement vector building error. Every error will occur for all objects in container simultaneously, so we can log each error once and safely ignore the rest. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-02-06 00:56:41 +03:00
Evgenii Stratonikov	602ee11123	[#934] containersvc: Marhal public key in short format for APE ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-02 17:51:38 +00:00
Airat Arifullin	c1a5b831b6	[#955] chainbase: Fix rule chain unmarshalling ... * Use correct way DecodeBytes instead unmarshalling by json. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-02 20:28:04 +03:00
Anton Nikiforov	9916598dfb	[#922] control: Extend api with ListOverrideDefinedTargets ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-02 12:09:51 +00:00
Anton Nikiforov	95e15f499f	[#922] Update files generated by protoc ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-02 12:09:51 +00:00
Anton Nikiforov	2cb04379a4	[#922] go.mod: Update APE ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-02 12:09:51 +00:00
Airat Arifullin	a5446bc17d	[#952] object: Pass namespace within context in ACL service ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-02 14:48:11 +03:00
Anton Nikiforov	d0eadf7ea2	[#799] engine: Skip put when object removed from shard ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-01 17:49:22 +00:00
Anton Nikiforov	6534252c22	[#799] policer: Refactor method processNodes ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-01 17:49:22 +00:00
Airat Arifullin	5be2af881a	[#934] container: Make container APE middleware read namespaces ... * Those methods that can access already existing containers and thus can get container properties should read namespace from Zone property. If Zone is not set, take a namespace for root. * Otherwise, define namespaces by owner ID via frostfs-id contract. * Improve unit-tests, consider more cases. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-01 17:38:24 +00:00
Airat Arifullin	4352bd0e8e	[#934] ape: Transform empty namespace within chainbase ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-01 17:38:24 +00:00
Anton Nikiforov	483a67b170	[#937] ape: Validate chain resource name ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-01-31 11:34:35 +03:00
Anton Nikiforov	e3573de6db	[#930] gc: Stop internal activity by context ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-01-31 08:30:34 +00:00
Anton Nikiforov	c441296592	[#930] policer: Release task pool when context cancelled ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-01-31 08:30:34 +00:00
Dmitrii Stepanov	675eec91f3	[#938] shard: Update only changed counters ... If metric value hasn't changed, but we update metric, then non existed metric will apear with zero value. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-30 12:37:48 +03:00
Dmitrii Stepanov	c681354afd	[#938] engine: Fix container count removal ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-30 12:37:48 +03:00
Evgenii Stratonikov	df055fead5	[#931] morph: Provide batch size for container listing explicitly ... Besides VM stack item limit we also have restrictions on the size of JSON for a stackitem, which is 128k. This limit is much harder to calculate, because JSON representation includes type and the encoding is different for different items. Thus is makes no sense to invent our own default, so use the one provided by neo-go. But for container listing we know exactly what we process, so use big enough value, which is tested. Introduced in be8607a1f6. Refs #902 Refs https://github.com/nspcc-dev/neo-go/blob/v0.105.0/pkg/vm/stackitem/json.go#L353 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-01-29 14:00:11 +00:00
Evgenii Stratonikov	6e2cc32768	[#681] objsvc: Validate session token owner for local sessions ... Previously, the check was in place only when session token was missing. Format validator checks are applied only to fully-prepared object, so this lead to the following situation: 1. Object is put locally with malformed token, because there are no checks. 2. Object cannot be replicated, because the token is malformed. This is now fixed and token check is done before any payload receival. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-01-26 08:52:29 +00:00
Anton Nikiforov	b6fc3321c5	[#876] Fix linters ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-01-25 20:26:13 +03:00
Airat Arifullin	1fe7736d92	[#925] morph: Introduce switch rpc guarded rpc actor ... * Introduce switch rpc guarded rpc actor in morph client to prevent using invalidated rpc actor when RPC switch happens. * Initialize NewContractStorage with SwitchRPCGuardedActor. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-25 13:24:13 +00:00
Dmitrii Stepanov	5ed330e436	[#927] metabase: Delete GC marks ... `key` is changed inside `db.get`, so encode address again after get. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-24 18:51:16 +03:00
Dmitrii Stepanov	931a5e9aaf	[#918] engine: Move shard to degraded mode if metabase open failed ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-23 11:16:40 +03:00
Airat Arifullin	f2f3294fc3	[#919] ape: Improve error messages in ape service ... * Wrap all APE middleware errors in apeErr that makes errors more explicit with status AccessDenied. * Use denyingRuleErr for denying status from chain router. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-23 08:11:24 +00:00
Dmitrii Stepanov	f526f49995	[#874] engine: Check object existance concurrently ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-23 09:28:29 +03:00
Dmitrii Stepanov	f5160b27fc	[#920] tests: Fix data races ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-19 14:06:05 +03:00
Airat Arifullin	be33070550	[#910] control: Take empty namespace for 'root' ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-19 06:22:10 +00:00
Dmitrii Stepanov	63d3ed1ad8	[#904] tests: Close test engine after test ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-17 19:04:39 +03:00
Dmitrii Stepanov	57171907e3	[#904] metabase: Return if object was actuall inserted ... This requires to count metrics properly. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-17 19:04:39 +03:00
Dmitrii Stepanov	c1a80235db	[#904] metabase: Log Inhume operation ... It will be very useful for troubleshooting. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-17 18:42:52 +03:00
Airat Arifullin	96b020626f	[#915] ape: Fix method name in getStreamBasicChecker ... * Replace incorrect MethodGetContainer by MethodGetObject constant. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-16 23:52:37 +03:00
Airat Arifullin	c8baf76fae	[#872] object: Introduce APE middlewar for object service ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-12 18:41:35 +03:00
Dmitrii Stepanov	52ffa9f164	[#891] getSvc: Refactor Get service V2 creation ... Use arguments for mandatory fields. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-12 13:35:38 +03:00
Dmitrii Stepanov	394f086fe2	[#891] getSvc: Fix get range hash implementation ... Get range can perform GET request, so this request must be done from container node to not to get access denied error. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-12 13:35:38 +03:00
Evgenii Stratonikov	be8607a1f6	[#902] morph: Avoid creating session in TestInvokeIterator ... When the number of items to iterate over is less than 2048, there is no need to create a session and consume resources. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-01-12 10:25:56 +00:00
Dmitrii Stepanov	a2ab373a0a	[#895] metabase: Do not delete GC mark for virtual objects ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Dmitrii Stepanov	7166e77c2b	[#895] test: Add logger to test shard ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Dmitrii Stepanov	47dcfa20f3	[#895] test: Use t.Cleanup only for external resources ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Dmitrii Stepanov	836818fb75	[#895] test: Use zaptest logger ... Previous implementation mixes logs from different tests, now logs are separeted by test. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Dmitrii Stepanov	f1b2b8bffa	[#895] test: Fix NewLogger arguments list ... `debug` is always true. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Airat Arifullin	a8e52ef7aa	[#898] control: Fix codes for returning APE errors ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-11 12:31:31 +00:00
Airat Arifullin	5d982976fd	[#898] ape: Fix bug with creating type bucket in chainbase ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-11 12:31:31 +00:00
Denis Kirillov	c19396d203	[#885] control: Make chain id bytes in grpc ... Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>	2024-01-11 07:24:22 +00:00
Dmitrii Stepanov	5c0a736a25	[#899] containerSvc: Fix invalid session token type ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-10 18:37:54 +03:00
Alexander Chuprov	79bebe4a68	[#884] cli: Fix error message for undefined endpoint ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-01-10 10:02:16 +00:00
Dmitrii Stepanov	4b8b4da681	[#864] engine: Drop container count metric if container removed ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-10 10:45:32 +03:00
Dmitrii Stepanov	d75e7e9a21	[#864] engine: Drop container size metric if container deleted ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-10 10:44:54 +03:00
Dmitrii Stepanov	dfd62ca6b1	[#864] metabase: Refactor delete/inhume ... Available -> Logic, Raw -> Phy for delete/inhume results. Use single counter instead of vectors. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-09 09:59:42 +03:00
Evgenii Stratonikov	225fe2d4d5	[#894] blobovniczatree: Speedup rebuild test ... Down from 3s to 300ms. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-12-29 16:28:54 +00:00
Evgenii Stratonikov	530249e3bd	[#893] go.mod: Update neo-go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-12-29 15:03:47 +00:00
Dmitrii Stepanov	581887148a	[#569] cli: Add control shards writecache seal command ... It does the same as `control shards flush-writecache --seal`, but has better name. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 16:05:37 +03:00
Dmitrii Stepanov	7a9db5bcdd	[#569] writecache: Do not wait modeMtx if mode changes ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 16:05:37 +03:00
Dmitrii Stepanov	32c282ca10	[#569] writecache: Refactor flush ... Make single RUnlock call instead of two. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 16:05:37 +03:00
Dmitrii Stepanov	0cb0fc1735	[#569] writecache: Allow to seal writecache after flush ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 16:05:37 +03:00
Dmitrii Stepanov	b118734909	[#890] getsvc: Log node PK ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 14:24:13 +03:00
Dmitrii Stepanov	764f70634d	[#881] containerSvc: Add APE validation ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-27 11:05:34 +03:00
Dmitrii Stepanov	8180a0664f	[#887] node: Drop badger writecache implementation ... Badger implementation isn't tested and works not well, but requires human resources to maintain. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-22 13:00:54 +03:00
Anton Nikiforov	eab981bf1a	[#876] Fix linter error ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-12-21 18:21:37 +03:00
Anton Nikiforov	d9cbb16bd3	[#866] Use TTL for blobovnicza tree cache ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-12-19 16:36:28 +00:00
Airat Arifullin	7d7cf05575	[#851] ape: Initialize and use policy contract interface ... * Replace inmemory policy contract mock by initialized policy contract interface. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-12-18 11:06:35 +00:00
Airat Arifullin	bdd43f6211	[#869] object: Pass just CID to chain router ... * Do not convert CID from request to native-schema resource format - this step is unneccessary for APE. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-12-14 11:01:20 +00:00
Ekaterina Lebedeva	2d4c0a0f4a	[#552] Add systemd notifications to ir service ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2023-12-13 17:51:41 +03:00
Ekaterina Lebedeva	eca7ac9f0d	[#552] Add sdnotify package ... To avoid using third-party dependencies. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2023-12-13 17:49:26 +03:00
Ekaterina Lebedeva	05f8f49289	[#552] gofumpt changes ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2023-12-13 15:02:25 +03:00
Dmitrii Stepanov	7eb46404a1	[#863] blobovnicza: Fix counters ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-13 13:34:29 +03:00
Dmitrii Stepanov	11add38e87	[#857] golangci: Add protogetter linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 16:27:02 +03:00
Dmitrii Stepanov	94ffe8bb45	[#857] golangci: Add testifylint linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 16:27:02 +03:00
Dmitrii Stepanov	5d7833c89b	[#857] golangci: Add perfsprint linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 16:27:02 +03:00
Dmitrii Stepanov	3b7c0362a8	[#861] shard: Fix Delete object ... It is possible that object doesn't exist in metabase. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 14:25:40 +03:00
Dmitrii Stepanov	681b2c5fd4	[#825] policer: Do not drop required linking objects ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 11:04:03 +00:00
Dmitrii Stepanov	a3ef7b58b4	[#755] innerring: Check container owner namespace ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 12:36:34 +03:00
Dmitrii Stepanov	1cd2bfe51a	[#755] morph: Drop FrostFSID contract usage ... Unused. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 12:36:34 +03:00
Anton Nikiforov	70e0c1e082	[#841] ir: Execute netmap.addPeerIR only for state online ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-12-11 13:14:41 +00:00
aarifullin	0f45e3d344	[#804] ape: Implement boltdb storage for local overrides ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-12-07 19:08:41 +03:00
aarifullin	e361e017f3	[#842] control: Pass target instead resource name ... * Update policy-engine package version in go.mod, go.sum. * Refactor CheckIfRequestPermitted: pass container target instead container ID. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-12-07 14:21:55 +00:00
aarifullin	39060382a1	[#842] control: Recieve target in gRPC methods for APE managing ... * Introduce Target type and pass it to all gRPC methods for APE chain managing instead CID. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-12-07 14:21:55 +00:00
Dmitrii Stepanov	db49ad16cc	[#826] blobovniczatree: Do not create DB's on init ... Blobovniczas will be created on write requests. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	ad0697adc4	[#661] blobovnicza: Compute size with record size ... To get more accurate size of blobovnicza use record size (lenght of key + lenght of data). Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	e54dc3dc7c	[#698] blobovnicza: Store counter values ... Blobovnicza initialization take a long time because of bucket Stat() call. So now blobovnicza stores counters in META bucket. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	5e8c08da3e	[#661] blobstore: Add address to error logs ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	8911656b1a	[#661] metrcis: Add rebuild percent metric ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	2407e5f5ff	[#661] blobovniczatree: Do not sort DB's and indicies ... Put stores object to next active DB, so there is no need to sort DBs. In addition, it adds unnecessary DB openings. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	c6a739e746	[#661] blobovniczatree: Make Rebuild concurrent for objects ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	f1c7905263	[#661] blobovniczatree: Make Rebuild concurrent ... Different DBs can be rebuild concurrently. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	d4d905ecc6	[#661] metrics: Add blobovniczatree rebuild metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	b2769ca3de	[#661] blobovniczatree: Make Rebuild failover safe ... Now move info stores in blobovnicza, so in case of failover rebuild completes previous operation first. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	da4fee2d0b	[#698] blobovniczatree: Init blobovniczas concurrently ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	422226da18	[#661] blobovniczatree: Add Rebuild implementation ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:32 +03:00
Dmitrii Stepanov	a531eaf8bc	[#661] blobstor: Add Rebuild implementation ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:32 +03:00
Dmitrii Stepanov	c1667a11d2	[#661] blobovniczatree: Allow to change depth or width ... Now it is possible to change depth or with of blobovniczatree. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:32 +03:00
Dmitrii Stepanov	484eb59893	[#661] blobovniczatree: Use .db extension for db files ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:32 +03:00
Dmitrii Stepanov	44552a849b	[#661] shard: Add blobstor rebuilder ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:32 +03:00
Dmitrii Stepanov	a478050639	[#838] metabase: Resolve funlen linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-06 15:44:21 +03:00
Dmitrii Stepanov	d30ab5f29e	[#838] metabase: Count user objects ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-06 15:44:21 +03:00
Dmitrii Stepanov	f314da4af3	[#838] metabase: Add user object type counter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-06 15:44:21 +03:00
Dmitrii Stepanov	29550fe600	[#838] shard: Refactor updateMetrics method ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-06 15:44:21 +03:00
Alexander Chuprov	b2c63e57ba	[#651] engine/test: Speedup StorageEngine_Inhume ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-11-30 13:19:43 +00:00
Alexander Chuprov	445ebcc0e7	[#651] shard/test: Speedup Shard_Delete ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-11-30 13:19:43 +00:00
Alexander Chuprov	2302e5d342	[#651] shard/test: Refactor Shard_Delete ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-11-30 13:19:43 +00:00
Dmitrii Stepanov	26e4f7005c	[#741] treesvc: Refactor tree sync ... Fix linter issues. Add error logging. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-30 12:45:02 +00:00
Dmitrii Stepanov	b21be1abdd	[#741] treesvc: Do not update sync height if some node is unavailable ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-30 12:45:02 +00:00
Dmitrii Stepanov	b215817e14	[#741] treesvc: Remove unused height variables ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-30 12:45:02 +00:00
Dmitrii Stepanov	c516c7c5f4	[#821] node: Pass user.ID by value ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-23 10:21:07 +03:00
Dmitrii Stepanov	07390ad4e3	[#715] node: Unify config parameter names ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-22 17:13:50 +03:00
Dmitrii Stepanov	8d18fa159e	[#667] writecache: Fix flush test ... Allow to disable background flusher for testing purposes. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-17 17:45:43 +03:00
aarifullin	02454df14a	[#809] client: Refactor PrmInit, PrmDial usage ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-17 13:37:03 +00:00
Dmitrii Stepanov	76ff26039c	[#96] node: Drop neo-go's slices package ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-17 13:24:04 +03:00
aarifullin	47286ebf32	[#805] pilorama: Fix TreeDrop ... * If treeID is empty then deleting buckets for cursor may get invalidated. So, buckets should be gathered before deleting. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-17 10:21:35 +00:00
Dmitrii Stepanov	29fe8c41f3	[#655] storage: Drop ErrorHandler ... The only one usage was for logging. Now logging performed by storage anyway. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-16 17:27:38 +03:00
Dmitrii Stepanov	137e987a4e	[#655] storage: Drop LazyHandler ... LazyHandler is implemented and used incorrectly. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-16 17:27:38 +03:00
aarifullin	4d5be5ccb5	[#811] ape: Update policy-engine module version and rebase ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-16 11:31:37 +03:00
Evgenii Stratonikov	a952a406a2	[#506] container: Use uint64 for epoch type ... It is `uint64` in netmap source interfaces and other code. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-11-14 19:00:11 +00:00
Evgenii Stratonikov	f04806ccd3	[#506] container: Use user.ID in DeletionInfo response ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-11-14 19:00:11 +00:00
Dmitrii Stepanov	8088063195	[#787] netmap: Refactor NewEpoch method ... Split for user and control methods. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-13 17:22:31 +03:00
Dmitrii Stepanov	c8a62ffedd	[#787] morph: Calculate VUB and nonce when hash is nil ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-13 17:13:03 +03:00
Dmitrii Stepanov	2393d13e4d	[#787] morph: Return VUB for IR service calls ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-13 17:13:03 +03:00
Dmitrii Stepanov	518f3baf41	[#787] morph: Return VUB from Invoke method ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-13 17:13:03 +03:00
Dmitrii Stepanov	bdfa523487	[#787] proto: Add VUB field for IR service ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-13 17:13:03 +03:00
Dmitrii Stepanov	0f75e48138	[#796] policer: Fix tombstone objects replication ... Tombstone objects must be replicated to all container nodes. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-09 13:39:33 +03:00
Dmitrii Stepanov	7cdae4f660	[#792] proto: Regenerate with fixed version ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-09 10:09:13 +00:00
Dmitrii Stepanov	9133b4389e	[#788] objectsvc: Fix formatting (gofumpt) ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-09 10:27:32 +03:00
Dmitrii Stepanov	1b22801eed	[#788] engine: Fix flaky tests ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-09 10:25:46 +03:00
aarifullin	3534d6d05b	[#794] objectsvc: Return accidentally removed acl checks for Head ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-08 17:13:58 +03:00
Airat Arifullin	66848d3288	[#770] cli: Add methods to work with APE rules via control svc ... * Add methods to frostfs-cli * Implement rpc in control service Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-08 13:34:03 +00:00
Airat Arifullin	8e11ef46b8	[#770] object: Introduce ape chain checker for object svc ... * Introduce Request type converted from RequestInfo type to implement policy-engine's Request interface * Implement basic ape checker to check if a request is permitted to be performed * Make put handlers use APE checker instead EACL Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-08 13:34:03 +00:00
Airat Arifullin	5ec73fe8a0	[#770] node: Introduce ape chain source ... * Provide methods to access rule chains with access policy engine (APE) chain source * Initialize apeChainSource within object service initialization * Share apeChainSource with control service * Implement dummy apeChainSource instance based on in-memory implementation Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-08 13:34:03 +00:00
Airat Arifullin	3a2c319b87	[#770] control: Generate gRPC methods to manipulate APE chains ... * Define new types and gRPC methods to manipulate APE chains in control service. * Stub gRPC handlers for the generated methods. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-08 13:34:03 +00:00
Dmitrii Stepanov	70ab1ebd54	[#763] metrics: Add container_objects_total metric ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-08 12:30:57 +03:00
Dmitrii Stepanov	9c98fa6152	[#763] metabase: Add container objects counter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-08 12:30:57 +03:00
Dmitrii Stepanov	226e84d782	[#684] node: Add skipped objects count to evacuation result ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-07 12:17:11 +00:00
Dmitrii Stepanov	523fb3ca51	[#684] proto: Add skipped count to evacuation status response ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-07 12:17:11 +00:00
aarifullin	74c91eeef5	[#777] client: Refactor PrmContainerList, PrmObjectSearch usage ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-06 06:50:11 +00:00
Dmitrii Stepanov	20d6132f31	[#531] signSvc: Add SetMarshaledData method call ... To reduce memory allocations add `SetMarshaledData` method call to return already marshalled data in next `StableMarshal` calls. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-02 17:34:33 +03:00
Anton Nikiforov	a26483fc30	[#749] morph: Fix panic when closing morph client ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-11-01 10:48:10 +00:00
Dmitrii Stepanov	c80b46fad3	[#754] blobstor: Estimate compressability ... Now it is possible to enable compressability estimation. If data is likely uncompressable, it should reduce CPU time and memory. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-01 11:24:32 +03:00
Dmitrii Stepanov	05b508f79a	[#772] proto: Fix file ending ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-31 17:03:04 +03:00
Dmitrii Stepanov	79088baa06	[#772] node: Apply gofumpt ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-31 17:03:03 +03:00
Evgenii Stratonikov	00aa6d9749	[#633] shard/test: Fix TestCounters() ... Introduced in 362f24953a, forgotten to be changed because test generator didn't provide payload size. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-31 12:53:28 +00:00
Evgenii Stratonikov	b8f79f4227	[#633] shard/test: Fix race conditions in TestCounters() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-31 12:53:28 +00:00
Evgenii Stratonikov	261d281154	[#762] go.mod: Update sdk-go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-31 11:22:29 +00:00
Dmitrii Stepanov	869518be0a	[#728] writecache: Fix Badger writecache race. ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-30 18:36:41 +03:00
Dmitrii Stepanov	d4b6ebe7e7	[#725] writecache: Fix metric values ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-27 12:22:29 +03:00
Anton Nikiforov	121f5c4dd8	[#757] ir: Do not exclude node in maintenance mode from netmap ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-10-26 10:50:32 +03:00
Evgenii Stratonikov	9f7c2d8810	[#752] innerring: Simplify keyPosition() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-25 16:06:44 +03:00
Evgenii Stratonikov	cddc58ace2	[#752] innerring: Optimize keyPosition() ... ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/innerring cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ new │ │ sec/op │ sec/op vs base │ KeyPosition-8 2771.50n ± 10% 40.32n ± 4% -98.55% (p=0.000 n=10) │ old │ new │ │ B/op │ B/op vs base │ KeyPosition-8 1.531Ki ± 0% 0.000Ki ± 0% -100.00% (p=0.000 n=10) │ old │ new │ │ allocs/op │ allocs/op vs base │ KeyPosition-8 21.00 ± 0% 0.00 ± 0% -100.00% (p=0.000 n=10) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-25 16:06:44 +03:00
Evgenii Stratonikov	0a9830564f	[#752] morph: Adopt neo-go RPC statuses ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-25 16:06:44 +03:00
Evgenii Stratonikov	6950312967	[#752] morph: Drop loop copy kludges ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-25 16:06:44 +03:00
Evgenii Stratonikov	fe1acf9e9a	[#750] morph: Remove deprecated channel use ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-25 07:57:05 +00:00
Anton Nikiforov	559ad58ab1	[#642] writecache: Remove usage of close channel in bbolt ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-10-24 15:57:50 +00:00
Anton Nikiforov	c0b86f2d93	[#642] writecache: Remove usage of close channel in badger ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-10-24 15:57:50 +00:00
Anton Nikiforov	b0cf100427	[#49] node: React on SIGHUP only when node in READY state ... Add more info in logs when node is going to shut down, but initialization process still in progress. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-10-24 15:55:29 +00:00
aarifullin	58b6224dd8	[#747] client: Refactor PrmObjectPutInit usage ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-10-20 11:55:40 +00:00
aarifullin	12b7cf2533	[#747] client: Refactor PrmObjectPutSingle usage ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-10-20 11:55:40 +00:00
Dmitrii Stepanov	dc4d27201b	[#733] morph: Fix delete container signature check ... Committed invalid condition, it was just for debug. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-19 18:07:37 +03:00
Dmitrii Stepanov	189dbb01be	[#733] frostfs-cli: Add control ir remove-container ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-19 16:22:18 +03:00
Dmitrii Stepanov	f2437f7ae9	[#734] shard: Fix Delete method ... Due to the flushing data from the writecache to the storage and simultaneous deletion, a partial deletion situation is possible. So as a solution, deletion is allowed only when the object is in storage, because object will be deleted from writecache by flush goroutine. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-16 17:00:18 +03:00
Dmitrii Stepanov	f26233b47a	[#734] metabase: Include UpdateStorageID in metrics and traces ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-16 17:00:18 +03:00
Dmitrii Stepanov	7e0c5a55de	[#734] writecache: Fix flush ... Now UpdateStorageID doesn't return error in case of logical error. If object is in graveyard or GC market, it is still required to update storage ID. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-16 17:00:17 +03:00
Anton Nikiforov	3a997d1207	[#680] metrics: Initialize log metrics together with services ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-10-11 17:08:03 +03:00
Anton Nikiforov	994f48f8bb	[#680] metrics: Export log and morph with script ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-10-11 17:08:03 +03:00
Evgenii Stratonikov	aca11d7474	[#735] policer: Allow to provide metrics from the outside ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-11 15:14:13 +03:00
Ekaterina Lebedeva	5e229dc248	[#701] metrics: add metric to evaluate policer performance ... Add processed objects counter in policerMetrics, add policer field to NodeMetrics Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2023-10-09 19:02:08 +00:00
Evgenii Stratonikov	4caa934eea	[#729] containersvc: Remove load announcement ... IR code was removed in 8879c6ea. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-09 19:01:13 +00:00
Dmitrii Stepanov	d07afd803c	[#726] writecache: Fix small object flush for Badger ... Do not marshal object twice. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-06 11:32:50 +03:00
Dmitrii Stepanov	997ac7cd8d	[#726] writecache: Fix small object flush for BBolt ... Do not marshal object twice. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-06 11:32:44 +03:00
Alexander Chuprov	c1e4130020	[#146] node: Add trace_id to logs ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-09-27 11:05:27 +03:00
a.arifullin	3889e829e6	[#667] writecache: Add logs for report error func in tests ... Signed-off-by: Airat Arifullin a.arifullin@yadro.com	2023-09-14 17:02:54 +00:00
Evgenii Stratonikov	c6af4a3ec8	[#679] engine: Do not increase error counter on meta mismatch ... It was introduced in 69e1e6ca to help node determine faulty shards. However, the situation is possible in a real-life scenario: 1. Object O is evacuated from shard A to B. 2. Shard A is unmounted because of lower-level errors. 3. We now have object in meta on A and in blobstor on B. Technically we have it in meta on shard B too, but we still got the error if B goes to a degraded mode. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-14 10:39:18 +03:00
Ekaterina Lebedeva	8a81af5a3b	[#653] Add context parameter to Open functions ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2023-09-07 18:03:29 +03:00
Evgenii Stratonikov	a716db99db	[#668] shard/test: Do not alter rootPath option ... Supposedly, this was added to allow creating 2 different shards without subtest. Now we use t.TempDir() everywhere, so this should not be a problem. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	36759f8434	[#668] shard/test: Properly check event processing ... See https://git.frostfs.info/TrueCloudLab/frostfs-node/actions/runs/1594/jobs/2 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	39879fa868	[#668] shard/test: Add dontRelease options ... Most of the time we would like to close shard with minor exceptions. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	c661ba1312	[#668] shard/test: Use sane defaults in the test constructor ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	268adb79cb	[#668] shard/test: Simplify shard construction ... newCustomShard() has many parameters but only the first is obligatory. `enableWriteCache` is left as-is, because it directly affects the functionality. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	429f941cda	[#668] shard/test: Release shard in t.Cleanup() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	382eb8a485	[#668] shard/test: Disable GC where it is not needed ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	42696016de	[#668] shard: Close stopChannel in GC ... It is done once, but now we could read it from multiple places. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	bdecfbc1be	[#668] shard/test: Move tests to the main package ... Semantic patch (also, duplicate definitions are removed): ``` @@ var e identifier @@ -import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/shard" -shard.e +e ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	aa23c6a83a	[#668] shard/test: Remove subtest from TestCounters ... Otherwise, individual tests cannot be run. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Evgenii Stratonikov	da8f384324	[#668] shard/test: Fix typo in existence ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-07 07:39:39 +00:00
Anton Nikiforov	aeeb8193d2	[#676] node: Fix header source creation when checking eacl ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-09-06 17:06:54 +03:00
Anton Nikiforov	88d50e4c77	[#656] policer: Add "bad" testcase ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-09-06 08:04:59 +00:00
Evgenii Stratonikov	a54b4472de	[#674] network: Close connections on address updates ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-09-04 15:34:24 +03:00
Dmitrii Stepanov	7456c8556a	[#536] blobovnicza: Add blobovniczatree DB cache ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-09-01 13:53:11 +03:00
Dmitrii Stepanov	c672f59ab8	[#536] blobovnicza: Drop cache ... Each blobovnicza instance is opened while is in use. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-09-01 13:51:26 +03:00
Evgenii Stratonikov	b9b86d2ec8	[#666] shard/test: Fix data race in metrics tests ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-31 08:39:42 +00:00
Anton Nikiforov	4dff9555f1	[#568] writecache: Improve flushing scheme for badger ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-08-30 17:22:28 +00:00
Airat Arifullin	806cc13d9f	[#658] client: Refactor PrmObjectGet/Head/Range usage ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-08-30 17:13:23 +00:00
Alejandro Lopez	1daef2ceeb	[#660] writecache: Fix remaining addr2key uses ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-30 17:12:33 +00:00
Anton Nikiforov	fe5aa06a75	[#665] node: Bind length of copies number to number of replicas ... Allow to use one digit in copies number array for backward compatibility. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-08-30 17:11:55 +00:00
Dmitrii Stepanov	55b82e744b	[#529] objectcore: Use common sender classifier ... Use common sender classifier for ACL service and format validator. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-29 10:33:06 +03:00
Dmitrii Stepanov	ae81d6660a	[#529] objectcore: Fix object content validation ... There are old objects where the owner of the object may not match the one who issued the token. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-29 10:33:06 +03:00
Dmitrii Stepanov	ab2614ec2d	[#528] objectcore: Validate token issuer ... Add token issuer against object owner validation. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-29 10:10:10 +03:00
Airat Arifullin	4ea0df77d0	[#574] policer: Check if the container was really removed ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-08-28 14:21:38 +00:00
Airat Arifullin	554ff2c06b	[#574] core: Extend Source interface with DeletionInfo method ... * Introduce common method EverExisted * Define DeletionInfo for struct that must implement Source * Refactor tree srv Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-08-28 14:21:38 +00:00
Evgenii Stratonikov	9072772a09	[#649] shard/test: Increase GC remover interval ... This was set in #348 to speed up tests. It seems 100ms doesn't increase overall test time, but it reduces the amount of logs by 100x factor. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-25 10:10:25 +00:00
Evgenii Stratonikov	c4db8e7690	[#637] shard/test: Fix data race ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-25 10:10:25 +00:00
Evgenii Stratonikov	f8ba60aa0c	[#648] objsvc/delete: Handle errors in Go style ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-25 09:45:35 +00:00
Evgenii Stratonikov	d2084ece41	[#648] objsvc/delete: Remove redundant logs ... We never propagate delete requests to the container node, because tombstone broadcast is done via PUT. No need to pollute logs. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-25 09:45:35 +00:00
Evgenii Stratonikov	40b556fc19	[#647] objsvc/search: Improve testing coverage ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-25 10:40:01 +03:00
Evgenii Stratonikov	4db2cbc927	[#647] objsvc/search: Wrap in uniqueIDWriter during parameter setting ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-25 10:40:01 +03:00
Evgenii Stratonikov	966ad22abf	[#647] objsvc/search: Simplify error handling ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-25 10:40:01 +03:00
Evgenii Stratonikov	56f841b022	[#647] objsvc/search: Remove TraverserGenerator wrapper ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-25 10:40:01 +03:00
Evgenii Stratonikov	ba58144de1	[#647] objsvc/search: Remove netmap.Source wrapper ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-25 10:40:01 +03:00
Evgenii Stratonikov	c9e3c9956e	[#643] objsvc/put: Unify extraBroadcastEnabled usage ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-24 11:03:17 +03:00
Evgenii Stratonikov	facd3b2c4b	[#643] objsvc/put: Unify placement iterators ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-24 11:03:17 +03:00
Evgenii Stratonikov	3fcf56f2fb	[#643] objsvc/put: Copy config to distributedTarget ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-24 11:03:17 +03:00
Evgenii Stratonikov	96e690883f	[#638] Unify test loggers ... In some places we have debug=false, in others debug=true. Let's be consistent. Semantic patch: ``` @@ @@ -test.NewLogger(..., false) +test.NewLogger(..., true) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-23 11:21:05 +00:00
Evgenii Stratonikov	322c1dc273	[#638] Use test.NewLogger() in tests ... Semantic patch (restricted to **/*_test.go): ``` @@ @@ +import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger/test" -import "go.uber.org/zap" -import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger" -&logger.Logger{Logger: zap.L()} +test.NewLogger(t, false) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-23 11:21:05 +00:00
Evgenii Stratonikov	02b03d9c4f	[#638] logger: Remove sampling from test loggers ... Losing logs is always a bad idea, especially when we debug tests. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-23 11:21:05 +00:00
Evgenii Stratonikov	82cc453be9	[#xx] shard: Fix data race in metrics tests ... Protect test metric store fields with a mutex. Probably, not every field should be protected, but better safe than sorry. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-23 10:26:12 +00:00
Alejandro Lopez	345a1a69a2	[#635] Use internal key type when deleting from badger wc ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-22 10:53:19 +03:00
Airat Arifullin	23be3eb627	[#574] tree: Check if container is really removed ... * Use DeletionInfo method from morph client to check if the container has been really removed from neo-go Signed-off-by: Airat Arifullin a.arifullin@yadro.com	2023-08-21 12:50:20 +03:00
Airat Arifullin	42fb6fb372	[#574] morph: Add DeletionInfo method for morph client ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-08-21 12:49:06 +03:00
Dmitrii Stepanov	354a92ea2c	[#602] blobovnicza: Add leaf width implementation ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-21 10:27:32 +03:00
Alexander Chuprov	4d9a6c07fb	[#618] core: Replace fmt.Sprintf with strconv.FormatUint ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-08-21 07:14:50 +00:00
Dmitrii Stepanov	f2811f8585	[#602] metrics: Add blobovnicza items counter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-18 13:01:27 +03:00
Dmitrii Stepanov	c4e1d8eb07	[#602] node: Fix blobovnicza typos ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-18 11:14:10 +03:00
Dmitrii Stepanov	10e63537b2	[#602] metrics: Rename blobovnicza size metric ... `Size` is not size, but open db size. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-17 19:17:35 +00:00
Dmitrii Stepanov	809e97626b	[#602] blobovnicza: Fix size counter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-17 19:17:35 +00:00
Dmitrii Stepanov	2e49d7ea7e	[#602] blobovnicza: Init before using ... Fix blobovnicza size: after restart size metric resets. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-17 19:17:35 +00:00
Evgenii Stratonikov	f7042c5a6f	[#609] Replace zaptest.NewLogger() with zap.L() ... Semantic patch: ``` @@ @@ -import "go.uber.org/zap/zaptest" +import "go.uber.org/zap" -zaptest.NewLogger(t) +zap.L() ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-17 16:51:10 +00:00
Evgenii Stratonikov	127c676786	[#607] *: Use keys.PublicKeys.Copy() where possible ... Semantic patch: ``` @@ var dst identifier var src identifier var keys identifier @@ import keys "github.com/nspcc-dev/neo-go/pkg/crypto/keys" -dst := make(keys.PublicKeys, len(src)) -copy(dst, src) +dst := src.Copy() ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-17 16:50:02 +00:00
Evgenii Stratonikov	e604a3d749	[#607] *: Use zap.Stringer() where possible ... Semantic patch: ``` @@ var f expression var t expression var a expression @@ f( ..., - zap.String(t, a.String()), + zap.Stringer(t, a), ..., ) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-17 16:50:02 +00:00
Evgenii Stratonikov	a8de37c8a2	[#607] *: Remove redundant if on error returns ... Semantic patch: ``` @@ @@ -if err != nil { return err } -return nil +return err ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-17 16:50:02 +00:00
Evgenii Stratonikov	5a51b78946	[#620] object: Send status response for server-side streams ... Previously status responses were wrapped in the gRPC error and thus couldn't be correctly handled on client. Introduced in c2617baf63, thanks @ale64bit for having found. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-17 12:52:38 +00:00
Anton Nikiforov	6407bb5bd1	[#619] node: Fix object put when copies numbers contains only zeros ... In this case object should placement according to replicas. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-08-17 11:31:34 +03:00
Evgenii Stratonikov	5335e7089e	[#615] pilorama: Speedup TestForest_ApplyRandom() ... Some of our pilorama tests fail on CI. The reasons are not obvious, but one possible improvement is using `WithNoSync` option for these. It should have much effect, because we are writing on the tmpfs, but doesn't hurt anyway. If I replace `t.TempDir()` with a local directory, test execution time goes down from 5s (sync) to 0.4s (nosync), which is the same time as with `t.TempDir()`. Maybe we have some strange CI configuration. ``` panic: test timed out after 10m0s running tests: TestForest_ApplyRandom (8m22s) TestForest_ApplyRandom/bbolt (8m21s) ... goroutine 170 [syscall]: syscall.Syscall(0xc000100000?, 0xc00047b758?, 0x6aff9a?, 0xc00041c1b0?) /opt/hostedtoolcache/go/1.20.7/x64/src/syscall/syscall_linux.go:69 +0x27 syscall.Fdatasync(0x9e35c0?) /opt/hostedtoolcache/go/1.20.7/x64/src/syscall/zsyscall_linux_amd64.go:418 +0x2a go.etcd.io/bbolt.fdatasync(0xc000189000?) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-16 19:09:08 +00:00
Dmitrii Stepanov	2efe9cc1be	[#585] writecache: Fix DB counter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-16 14:47:44 +03:00
Dmitrii Stepanov	58c8722c81	[#585] fstree: Add optional file counter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-16 14:47:44 +03:00
Dmitrii Stepanov	baad49990c	[#585] fstree: Return logical error if object deleted ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-16 14:47:44 +03:00
Dmitrii Stepanov	0c52186572	[#585] fstree: Remove unused method ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-16 14:47:44 +03:00
Dmitrii Stepanov	eec97d177e	[#585] writecache: Count items periodically ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-16 14:47:44 +03:00
Evgenii Stratonikov	d15199c5d8	[#596] engine: Consider context errors as logical ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-16 10:39:41 +03:00
Alejandro Lopez	88b6755c5e	[#598] Fix use-after-close bug in badger writecache ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-15 07:04:06 +00:00
Alejandro Lopez	ae8be495c8	[#xx] Avoid manual management of files in tests ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-14 14:01:39 +03:00
Alejandro Lopez	376f03a445	[#598] Hold mode mutex when setting mode ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-14 07:12:27 +00:00
Alexander Chuprov	21800e9fcc	[#162] core: Move literals to constants ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-08-11 15:48:42 +03:00
Evgenii Stratonikov	b44a8dd46c	[#597] *: Fix linter warnings ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-11 09:55:51 +03:00
Evgenii Stratonikov	e084c47bd6	[#248] innerring: Remove audit from tests ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-10 16:38:43 +00:00
Evgenii Stratonikov	779da6ec35	[#248] morph: Remove audit contract name ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-10 16:38:43 +00:00
Evgenii Stratonikov	55ce4dc075	[#248] morph: Remove obsolete network parameters ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-10 16:38:43 +00:00
Anton Nikiforov	4ad0ebb32f	[#565] Add metrics for current GRPC endpoint status ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-08-10 16:37:49 +00:00
Evgenii Stratonikov	c3e23a1448	[#578] gendoc: Allow to override flags ... The command is used in multiple places across the whole FrostFS ecosystem. While we want to have uniform interfaces everywhere, sometimes we can't: already defined global flags can be harder to change because of our obligations to the users. Cobra framework doesn't allow conflicting flags (we can have global ones), so allow to override them. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-10 11:10:11 +00:00
Evgenii Stratonikov	6bcba27757	[#578] gendoc: Remove flag shorthands ... Make it harder to encounter conflicts in already existing commands. Because the command is executed once, I don't think the usability is worse. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-10 11:10:11 +00:00
Evgenii Stratonikov	dca31e8888	[#578] gendoc: Add date to man pages ... ``` HISTORY 8-Aug-2023 Auto generated by spf13/cobra ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-10 11:10:11 +00:00
Evgenii Stratonikov	b02a1a34c1	[#578] gendoc: Allow to customize man pages ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-10 11:10:11 +00:00
Evgenii Stratonikov	082370583f	[#578] gendoc: Replace NSPCC with FrostFS ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-10 11:10:11 +00:00
Alejandro Lopez	8f994163ee	[#586] Fix writecache benchmarks and refactor hacky NeedsCompression ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-10 08:05:18 +00:00
Alejandro Lopez	023b90342c	[#584] Disable compression in badger writecache ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-10 10:14:06 +03:00
Evgenii Stratonikov	d641cba2fc	[#587] Do not use math/rand.Read ... Fix staticcheck warnings after go1.20 update. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-09 16:02:44 +03:00
Alejandro Lopez	5b7e4a51b7	[#481] Update frostfs-sdk-go and error pointer receivers ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-09 10:26:53 +00:00
Alejandro Lopez	ae322e9f73	[#576] Set SyncWrites for badger writecache by default ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-09 11:28:26 +03:00
Anton Nikiforov	8d589314b5	[#560] node: Fix Put in multi REP with intersecting sets of nodes ... Once the node was processed it skipped, at the step of forming result in case when all nodes skipped, because processed for previous REP, service mark the whole request as incomplete. Example of policies which are unblocked: - REP 1 REP 1 CBF 1 - REP 4 IN X REP 4 IN Y CBF 4 SELECT 2 FROM FX AS X SELECT 2 FROM FY AS Y FILTER Country EQ Russia OR Country EQ Sweden OR Country EQ Finland AS FY FILTER Price GE 0 AS FX Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-08-08 10:22:53 +00:00
Alejandro Lopez	7da4306e38	[#575] writecache: Fix log level for badger writecache ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-08 08:16:36 +00:00
Dmitrii Stepanov	0e697266c3	[#563] writecache: Fix metrics and bolt delete ... Estimate cache size after delete objects to update metric. Update counters on small object deletion. Do not count bbolt DB file as FSTree object. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-07 12:53:28 +00:00
Alejandro Lopez	1a0cb0f34a	[#421] Try using badger for the write-cache ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-07 08:16:57 +00:00
Alejandro Lopez	65c72f3e0b	[#559] Remove manual path handling in fstree tests ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-03 10:03:41 +03:00
Evgenii Stratonikov	1e8b4b8a17	[#557] services: Regenerate stable marshalers ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-02 13:37:40 +00:00
Airat Arifullin	b3695411d9	[#553] eacl: Fix bug with casting to ObjectAccessDenied error ... Signed-off-by: Airat Arifullin a.arifullin@yadro.com	2023-08-02 07:22:48 +00:00
Evgenii Stratonikov	35dc64bd7b	[#547] metabase: Fix datarace in tests ... Quite an old one bf9e938a3b. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-31 15:35:15 +00:00
Evgenii Stratonikov	05ac9e3637	[#547] objectsvc: Work with traversal struct from a single thread ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-31 15:35:15 +00:00
Evgenii Stratonikov	7b0fdf0202	[#533] services: Assume API supports status codes ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-31 15:35:15 +00:00
Evgenii Stratonikov	ec8b4fdc48	[#541] writecache/test: Close writecache on exit ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-29 10:57:25 +00:00
Evgenii Stratonikov	ad5f527bd3	[#541] writecache/test: Remove initWC() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-29 10:57:25 +00:00
Dmitrii Stepanov	ea32913430	[#543] putsvc: Fix PutSingle implementation ... Add Lock and Delete handlers to local PutSingle. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-28 12:09:41 +00:00
Dmitrii Stepanov	99bb488ebd	[#539] getsvc: Write payload direct to out stream ... To reduce memory allocations. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-27 17:02:08 +03:00
Dmitrii Stepanov	286242cad0	[#539] getsvc: Use buffer to assemble object ... To reduce memory consumption. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-27 17:02:08 +03:00
Evgenii Stratonikov	5ff82ff04f	[#6] services/object: Simplify local/remote targets ... We do not use the return result from Close() and we always execute both methods in succession. It makes sense to unite them. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-21 18:39:12 +03:00
Evgenii Stratonikov	448b48287c	[#6] services/util: Do not panic in sign function ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-21 18:39:12 +03:00
Evgenii Stratonikov	c2617baf63	[#6] services/util: Remove remaining stream wrappers ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-21 18:39:09 +03:00
Evgenii Stratonikov	372160d048	[#6] services/util: Remove SignService.HandleUnaryRequest ... There is no need in a wrapper with many from-`interface{}` conversions. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-21 18:36:08 +03:00
Evgenii Stratonikov	fef172c5b0	[#6] services/util: Simplify response.Service ... It has only 1 parameter, which is obligatory. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-21 18:29:57 +03:00
Dmitrii Stepanov	eed594431f	[#335] treesvc: Add GetSubTree ordering unit test ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-20 10:14:10 +03:00
Dmitrii Stepanov	af82c2865e	[#335] treesvc: Fix inmemory unit tests and nil meta items ... Bolt forest saves empty slice of items. Now inmemory forest does it the same way. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-20 10:14:10 +03:00
Dmitrii Stepanov	b4e72a2dfd	[#335] treesvc: Sort nodes by Filename in GetSubTree ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-20 10:14:10 +03:00
Dmitrii Stepanov	f0355a453e	[#463] policer: Remove capacity rebalance logic ... Current implementation has some quirks. For example, using only half of object.put.pool_size_remote threads tells replicator that is node is 50% loaded, but in reality we could be putting lot's of big objects. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-18 10:52:12 +00:00
Dmitrii Stepanov	8b78db74bc	[#463] replicator: Add tracing span ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-18 10:52:12 +00:00
Dmitrii Stepanov	8966dd8e35	[#463] putsvc: Use PutSingle RPC for remote target ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-18 10:52:12 +00:00
Alejandro Lopez	3b66f98f27	[#519] Use Address.Equals in policer tests ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-07-17 10:45:07 +00:00
Dmitrii Stepanov	3e8de14e7d	[#382] evacuate: Fix unit tests ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-17 10:24:05 +00:00
Dmitrii Stepanov	397131b0ea	[#520] objectcore: Refactor format validator ... Remove redundant FIXME. Move error to consts. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-17 07:07:21 +00:00
Alejandro Lopez	11027945d8	[#479] writecache: Fix writecache fstree flush premature ctx cancel ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-07-14 10:25:52 +03:00
Evgenii Stratonikov	8a9fc2c372	[#510] treesvc: Rename tableFromBearer to useBearer ... With impersonation, the old name is no longer descriptive. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-12 10:23:21 +00:00
Evgenii Stratonikov	b8bcfac531	[#510] treesvc: Fix panic in bearer token processing ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-12 10:23:21 +00:00
Dmitrii Stepanov	24eb988897	[#294] deletesvc: Drop cast ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-12 07:42:10 +00:00
Dmitrii Stepanov	c83e7c875f	[#294] searchsvcv2: Refactor service constructor ... Pass required deps as args. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-12 07:42:10 +00:00
Dmitrii Stepanov	e8091101c7	[#294] searchsvc: Refactor service constructor ... Pass required deps as args. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-12 07:42:10 +00:00
Dmitrii Stepanov	ec9b738465	[#294] putsvcv2: Refactor service constructor ... Pass required deps as args. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-12 07:42:10 +00:00
Dmitrii Stepanov	800a685e84	[#294] putsvc: Refactor service constructor ... Pass required deps as args. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-12 07:42:10 +00:00
Dmitrii Stepanov	1420b8b9ea	[#294] deletesvc: Refactor service constructor ... Pass required deps as args. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-12 07:42:10 +00:00
Dmitrii Stepanov	a476d8285a	[#294] deletesvcv2: Refactor service constructor ... Pass required deps as args. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-12 07:42:10 +00:00
Dmitrii Stepanov	70a1081988	[#294] aclsvcv2: Refactor service constructor ... Pass required deps as args. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-12 07:42:10 +00:00
Dmitrii Stepanov	18d8898b00	[#294] aclsvc: Refactor service constructor ... Pass required deps as args. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-12 07:42:10 +00:00
Dmitrii Stepanov	61541eaec2	[#294] aclsvc: Refactor checker constructor ... Pass required deps as args. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-12 07:42:10 +00:00
Evgenii Stratonikov	7da284f3e8	[#509] go.mod: Update sdk-go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-11 17:38:00 +03:00
Evgenii Stratonikov	80481c015c	[#509] putsvc: Omit AccessIdentifiers from iteratePlacement() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-11 17:38:00 +03:00
Dmitrii Stepanov	a65e26878b	[#486] put service: Fix error typo ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-10 15:49:21 +03:00
Dmitrii Stepanov	fcbf90d31b	[#486] node: Add PutSingle implemetation ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-10 15:49:21 +03:00
Dmitrii Stepanov	7b76527759	[#486] node: Add PutSingle wrappers ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-10 15:49:21 +03:00
Evgenii Stratonikov	140d970a95	[#498] policer: Fix objectSDK import ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-07 18:52:40 +03:00
Evgenii Stratonikov	2310a5c7ba	[#498] policer: Allow to set sleep duration between iterations ... Speed up tests on CI. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-07 14:52:33 +00:00
Evgenii Stratonikov	e858479a74	[#498] policer: Explicitly Rewind() iterator after finish ... Previously, we can continue to return `EndOfListing` infinitely. Reflect iterator reuse via Rewind() method. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-07 14:52:33 +00:00
Alexander Chuprov	033eaf77e1	[#496] node: Fix linter importas ... Standardize the alias of the import frostfs-sdk-go/object as objectSDK. Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-07-06 15:36:41 +03:00
Evgenii Stratonikov	6eefe9747e	treesvc: Do not provide credentials unless TLS is used ... 4f413fe86e was perfect, except it did the opposite of what we needed. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-06 08:04:28 +00:00
Alex Vanin	f354b8a270	[#473] Add tests for rem values of traverser ... Signed-off-by: Alex Vanin <a.vanin@yadro.com>	2023-07-05 20:13:23 +03:00
Alex Vanin	c6df6c84ae	[#473] Do not modify traverse plan when [0] copies number vector is provided ... Signed-off-by: Alex Vanin <a.vanin@yadro.com>	2023-07-05 19:54:43 +03:00
Anton Nikiforov	90e9a85acc	[#483] Update dependencies ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-07-03 15:44:59 +03:00
Dmitrii Stepanov	8d16d95376	[#484] morph: Add expired tx logging ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-03 08:03:25 +00:00
Alejandro Lopez	26acf5689e	[#92] Ensure policer objects cannot be worked on concurrently ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-07-03 07:05:31 +00:00
Alejandro Lopez	3223402c90	[#92] Embed policer's objectsInWork mutex ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-07-03 07:05:31 +00:00
Alejandro Lopez	f9730f090d	[#92] Refactor policer and add some unit tests ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-07-03 07:05:31 +00:00
Airat Arifullin	f437ab8f15	[#197] object: Make Delete method return correct status ... Signed-off-by: Airat Arifullin a.arifullin@yadro.com	2023-06-30 12:58:56 +00:00
Anton Nikiforov	d01c064674	[#455] Fix non-informative message when parse node attributes ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-06-28 15:02:29 +00:00
Evgenii Stratonikov	d0ab552a90	[#478] *: Fix funlen linter warnings ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-28 15:01:49 +00:00
Evgenii Stratonikov	8a4e250dae	[#468] *: replace outdated TODO crypto-related links ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-28 12:13:20 +00:00
Evgenii Stratonikov	4f413fe86e	[#1] treesvc: Properly check for secure transport ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-28 12:13:20 +00:00
Evgenii Stratonikov	cab51c8cbe	[#1] metabase: Rename blindlyProcess() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-28 12:13:20 +00:00
Evgenii Stratonikov	f4c71cea65	[#1] *: Replace outdated FIXME/TODO links ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-28 12:13:20 +00:00
Evgenii Stratonikov	0c866f62c5	[#473] placement: Fix backwards compatibility for copies_number ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-27 11:13:16 +03:00
Dmitrii Stepanov	43d263c3d5	[#428] linter: Fix unkeyed assignment ... Thanks to gopls. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-26 13:29:39 +00:00
Dmitrii Stepanov	cac4ed93d6	[#428] engine: Add low_mem config parameter ... Concurrent initialization in case of the metabase resync leads to high memory consumption and potential OOM. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-26 13:29:39 +00:00
Evgenii Stratonikov	4bf345225c	[#447] pilorama: Use named constant for the key size ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-26 07:42:05 +00:00
Evgenii Stratonikov	b4ce0b0412	[#447] pilorama: Do not undo log for create ops ... ``` goos: linux goarch: amd64 cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ new │ │ sec/op │ sec/op vs base │ Create-8 36.48µ ± 11% 30.34µ ± 14% -16.84% (p=0.000 n=10) │ old │ new │ │ B/op │ B/op vs base │ Create-8 43.01Ki ± 4% 37.78Ki ± 5% -12.15% (p=0.000 n=10) │ old │ new │ │ allocs/op │ allocs/op vs base │ Create-8 166.0 ± 3% 146.0 ± 3% -12.05% (p=0.000 n=10) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-26 07:42:05 +00:00
Evgenii Stratonikov	dd3874eff1	[#447] pilorama: Add benchmark for create ops ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-26 07:42:05 +00:00
Anton Nikiforov	71889234b7	[#449] tree: Allow reading requests signed by keys from allow list ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-06-23 14:44:19 +03:00
Evgenii Stratonikov	40eae22109	[#460] services/util: Remove CreateRequestStreamer ... There is no need in a wrapper with many from-`interface{}` conversions. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-21 17:08:04 +03:00
Evgenii Stratonikov	a64dc9ad70	[#460] services/util: Remove HandleServerStreamRequest ... There is no need in a wrapper with many from-`interface{}` conversions. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-21 17:08:01 +03:00
Evgenii Stratonikov	167a67f0b8	[#460] services/util: Remove HandleUnaryRequest ... There is no need in a wrapper with many from-`interface{}` conversions. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-21 17:07:56 +03:00
Evgenii Stratonikov	785d81a68a	[#460] services/object: Reduce distibutedTarget memory footprint ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-21 17:07:50 +03:00
Dmitrii Stepanov	4d48377cec	[#459] blobovniczatree: Fix get error ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-21 16:44:53 +03:00
Dmitrii Stepanov	03aa210145	[#373] metrics: Move labels to consts ... To unify label naming all lable keys and other consts are moved to one file. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-21 15:13:26 +03:00
Dmitrii Stepanov	b5d9f4a285	[#373] metrics: Add pilorama metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-21 15:13:26 +03:00
Dmitrii Stepanov	e89fa110c7	[#373] metrics: Add metabase metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-21 15:13:26 +03:00
Dmitrii Stepanov	af608da952	[#373] metrics: Add blobovnizca metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-21 15:13:26 +03:00
Dmitrii Stepanov	56f320dd85	[#373] metrics: Add blobstor metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-21 15:13:26 +03:00
Dmitrii Stepanov	16a142cd0c	[#373] metrics: Add FSTree metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-21 15:13:26 +03:00
Dmitrii Stepanov	d8ecc69d00	[#373] local storage: Pass parent ID ... This is required to add shard ID as metric label. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-21 15:13:26 +03:00
Dmitrii Stepanov	d5aaec1107	[#373] pilorama: Add metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-21 15:13:26 +03:00
Dmitrii Stepanov	059e9e88a2	[#373] metabase: Add metrics ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-21 15:13:26 +03:00