Anton Nikiforov
ae5bb87e70
Revert "[ #866 ] Use TTL for blobovnicza tree cache"
...
DCO action / DCO (pull_request) Successful in 2m7s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m7s
Details
Tests and linters / Staticcheck (pull_request) Successful in 5m12s
Details
Tests and linters / Lint (pull_request) Successful in 6m23s
Details
Tests and linters / Tests with -race (pull_request) Successful in 7m11s
Details
Tests and linters / gopls check (pull_request) Successful in 7m27s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m5s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m22s
Details
Build / Build Components (1.21) (pull_request) Successful in 1m41s
Details
Build / Build Components (1.20) (pull_request) Successful in 1m56s
Details
This reverts commit d9cbb16bd3
.
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-01 19:29:33 +03:00
Dmitrii Stepanov
d6534fd755
[ #1016 ] frostfs-node: Fix gopls issues
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-01 12:13:43 +03:00
Airat Arifullin
93bf9acbc2
[ #898 ] control: Remove removed flag from RemoveChainLocalOverrideResponse
...
* Remove removed flag in service.proto for RemoveChainLocalOverrideResponse.
* Regenerate control API.
* Return error only if RemoveOverride returns non-NotFound code.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:07:07 +00:00
Airat Arifullin
75a1a95c2c
[ #986 ] tree: Skip ACL checks if basicACL mask is unset
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:05:57 +00:00
Airat Arifullin
b1d171c261
[ #986 ] container: Interpret APE NoRuleFound as request deny
...
* If APE check returns NoRuleFound, then it is taken for request deny.
* Add more unit-test for ape container middleware.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:05:57 +00:00
Airat Arifullin
7cc368e188
[ #986 ] object: Introduce soft ape checks
...
* Soft APE check means that APE should allow request even
it gets status NoRuleFound for a request. Otherwise,
it is interpreted as Deny.
* Soft APE check is performed if basic ACL mask is not set.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:05:57 +00:00
Dmitrii Stepanov
918613546f
[ #1008 ] metabase: Do not update storageID on put
...
DCO action / DCO (pull_request) Successful in 4m56s
Details
Build / Build Components (1.21) (pull_request) Successful in 7m52s
Details
Build / Build Components (1.20) (pull_request) Successful in 8m11s
Details
Vulncheck / Vulncheck (pull_request) Successful in 8m9s
Details
Tests and linters / Staticcheck (pull_request) Successful in 9m58s
Details
Tests and linters / Lint (pull_request) Successful in 10m42s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m14s
Details
Tests and linters / Tests with -race (pull_request) Successful in 12m17s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 12m34s
Details
There may be a race condition between put an object and
flushing the writecache:
1. Put object to the writecache
2. Writecache flushes object to the blobstore and sets blobstore's
storageID
3. Put object to the metabase, set writecache's storageID
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-28 11:01:50 +03:00
Dmitrii Stepanov
2ad433dbcb
[ #1005 ] engine: Drop shards weights
...
DCO action / DCO (pull_request) Successful in 4m7s
Details
Vulncheck / Vulncheck (pull_request) Successful in 4m53s
Details
Build / Build Components (1.21) (pull_request) Successful in 5m46s
Details
Build / Build Components (1.20) (pull_request) Successful in 6m21s
Details
Tests and linters / Staticcheck (pull_request) Successful in 7m45s
Details
Tests and linters / Lint (pull_request) Successful in 8m44s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m1s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 15m42s
Details
Tests and linters / Tests with -race (pull_request) Successful in 16m10s
Details
Unused.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-26 17:25:05 +03:00
Evgenii Stratonikov
4b13b85173
[ #1000 ] morph: Fix batch size in TraverseIterator()
...
Initial prefetch size can be arbitrary an restricted only by VM/RPC
limits. For TraverseIterator() there is an explicit check on the
server-side, though.
Introduced in df055fead5
.
Refs #931 .
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-23 06:24:40 +00:00
Evgenii Stratonikov
e18f0f5178
[ #999 ] morph: Use Global scope for proxy contract
...
DCO action / DCO (pull_request) Successful in 2m28s
Details
Vulncheck / Vulncheck (pull_request) Successful in 2m52s
Details
Build / Build Components (1.21) (pull_request) Successful in 4m5s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m12s
Details
Tests and linters / Lint (pull_request) Successful in 6m10s
Details
Tests and linters / Staticcheck (pull_request) Successful in 5m59s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m24s
Details
Tests and linters / Tests with -race (pull_request) Successful in 8m38s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m58s
Details
Proxy contract can now be used as an owner of NNS domains, thus we need
it not only to pay for the transaction but also to check domain
ownership. CalledByEntry is not enough, because we may register NNS
domains owned by proxy indirectly from the container contract.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-21 14:28:42 +03:00
Dmitrii Stepanov
7470c383dd
[ #997 ] metabase: Drop toMoveIt bucket
...
DCO action / DCO (pull_request) Successful in 2m9s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m3s
Details
Build / Build Components (1.20) (pull_request) Successful in 3m56s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m54s
Details
Tests and linters / Staticcheck (pull_request) Successful in 5m26s
Details
Tests and linters / Lint (pull_request) Successful in 6m19s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m6s
Details
Tests and linters / Tests with -race (pull_request) Successful in 8m20s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m39s
Details
It is not used.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-21 10:06:05 +03:00
Evgenii Stratonikov
adf7ebab5b
[ #996 ] metabase: Speed up bucket creation
...
Most of the time it exits, e.g. when it is per-container and use on each
object PUT. Bbolt implementation first tries to create bucket and then
returns it if it exists. Create operation uses cursor and thus is not
very lightweight, we can avoid it.
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ old │ new │
│ sec/op │ sec/op vs base │
Put/parallel-8 174.4µ ± 3% 163.3µ ± 3% -6.39% (p=0.000 n=10)
Put/sequential-8 263.3µ ± 2% 259.0µ ± 1% -1.64% (p=0.000 n=10)
geomean 214.3µ 205.6µ -4.05%
│ old │ new │
│ B/op │ B/op vs base │
Put/parallel-8 275.3Ki ± 3% 281.1Ki ± 4% ~ (p=0.063 n=10)
Put/sequential-8 413.0Ki ± 2% 426.6Ki ± 2% +3.29% (p=0.003 n=10)
geomean 337.2Ki 346.3Ki +2.70%
│ old │ new │
│ allocs/op │ allocs/op vs base │
Put/parallel-8 678.0 ± 1% 524.5 ± 2% -22.64% (p=0.000 n=10)
Put/sequential-8 1.329k ± 0% 1.183k ± 0% -10.91% (p=0.000 n=10)
geomean 949.1 787.9 -16.98%
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-20 15:42:58 +00:00
Evgenii Stratonikov
9f68305c2e
Revert "[ #972 ] pilorama: Remove removeDuplicatesInPlace()"
...
This reverts commit 45fd4e4ff1
.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov
dad56d2e98
Revert "[ #972 ] Use min/max builtins"
...
This reverts commit 89784b2e0a
.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov
7627d08914
Revert "[ #972 ] Use require.ElementsMatch() where possible"
...
This reverts commit 6d9707ff1f
.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov
3359349acb
Revert "[ #972 ] Use slices.Sort* when useful"
...
This reverts commit b871d7a5e8
.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov
946f2ec2bf
Revert "[ #972 ] Drop x/exp/slices dependency"
...
This reverts commit f3e50772fd
.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Dmitrii Stepanov
13d5cd3e21
[ #991 ] logger: Fix journald logger
...
DCO action / DCO (pull_request) Successful in 1m7s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m32s
Details
Build / Build Components (1.22) (pull_request) Successful in 3m52s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m58s
Details
Tests and linters / Lint (pull_request) Successful in 5m13s
Details
Tests and linters / Tests (1.22) (pull_request) Successful in 7m22s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m34s
Details
Tests and linters / Tests with -race (pull_request) Successful in 7m32s
Details
Tests and linters / Staticcheck (pull_request) Successful in 2m22s
Details
Allow to change logger level.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-19 16:18:13 +03:00
Evgenii Stratonikov
f3e50772fd
[ #972 ] Drop x/exp/slices dependency
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Evgenii Stratonikov
b871d7a5e8
[ #972 ] Use slices.Sort* when useful
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Evgenii Stratonikov
6d9707ff1f
[ #972 ] Use require.ElementsMatch() where possible
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Evgenii Stratonikov
89784b2e0a
[ #972 ] Use min/max builtins
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Evgenii Stratonikov
45fd4e4ff1
[ #972 ] pilorama: Remove removeDuplicatesInPlace()
...
Also, check that slices.CompareFunc() indeed passes all the tests before
removal.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Evgenii Stratonikov
15fc5bac26
[ #972 ] keyer: Use UncompressedBytes() for marshaling
...
elliptic.Marshal() becomes deprecated in go1.21
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Dmitrii Stepanov
2680192ba0
[ #988 ] objectSvc: Fix `SetMarshalData` for PutSingle
...
DCO action / DCO (pull_request) Successful in 5m10s
Details
Vulncheck / Vulncheck (pull_request) Successful in 5m24s
Details
Build / Build Components (1.21) (pull_request) Successful in 7m40s
Details
Build / Build Components (1.20) (pull_request) Successful in 7m50s
Details
Tests and linters / Staticcheck (pull_request) Successful in 7m52s
Details
Tests and linters / Lint (pull_request) Successful in 9m8s
Details
Tests and linters / Tests with -race (pull_request) Successful in 10m54s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m12s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 11m7s
Details
After api-go update it is required to pass marshal data
to `SetMarshalData`.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 17:21:08 +03:00
Anton Nikiforov
05b5f5ca85
[ #959 ] writecache: Fix panic on `Get` when it is not initialized
...
Vulncheck / Vulncheck (pull_request) Successful in 2m41s
Details
DCO action / DCO (pull_request) Successful in 2m43s
Details
Tests and linters / Staticcheck (pull_request) Successful in 3m55s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m45s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m19s
Details
Tests and linters / Lint (pull_request) Successful in 5m30s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m57s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m22s
Details
Tests and linters / Tests with -race (pull_request) Successful in 8m39s
Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-14 16:10:33 +03:00
Anton Nikiforov
2429508ac5
[ #959 ] shard: Skip rebuild in `DEGRADED` mode
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-14 15:39:28 +03:00
Anton Nikiforov
0bd030507e
[ #948 ] metrics: Set actual value for `shard_id` after restart
...
DCO action / DCO (pull_request) Successful in 2m26s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m56s
Details
Tests and linters / Staticcheck (pull_request) Successful in 4m18s
Details
Build / Build Components (1.21) (pull_request) Successful in 4m11s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m19s
Details
Tests and linters / Lint (pull_request) Successful in 6m3s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m7s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m26s
Details
Tests and linters / Tests with -race (pull_request) Successful in 9m20s
Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:43:21 +03:00
Anton Nikiforov
6a5769d1da
[ #948 ] Fix `gofumpt` issue
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:40:46 +03:00
Dmitrii Stepanov
962e5a9c19
[ #736 ] logger: Add journald support
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-12 06:17:22 +00:00
Evgenii Stratonikov
b36a453238
[ #970 ] fstree: Add build tag to enable generic version on linux
...
Unless tested, generic version can start gaining bugs. With a separate
build tag we can have the best of both worlds:
1. Use optimized implementation for linux by default.
2. Run tests or benchmarks for both. Note that they are not actually
run automatically now, but this is at leas possible.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
Evgenii Stratonikov
abd502215f
[ #970 ] fstree: Move file locking to the generic writer
...
It is not a part of FSTree itself, but rather a way to solve concurrent
counter update on non-linux implementations. New linux implementations
is pretty simple: link fails when the file exists, unlink fails when the
file doesn't exist.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
Evgenii Stratonikov
fb74524ac7
[ #970 ] fstree: Move delete implementation to a separate file
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
Evgenii Stratonikov
7f692409cf
[ #970 ] fstree: Handle unsupported O_TMPFILE
...
Metabase test relied on this behaviour, so fix the test too.
Cherry-picking was hard and did too many conflicts,
here is an original PR:
https://github.com/nspcc-dev/neofs-node/pull/2624
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
Roman Khimov
fc31b9c947
[ #970 ] fstree: Add linux-specific file writer using O_TMPFILE
...
O_TMPFILE is implemented for all modern FSes and it's much easier and safer to
use. If application crashes in the middle of writing this file would be gone
and won't leave any garbage.
Notice that this implementation makes a different choice wrt EEXIST handling,
generic one always overwrites, while this one keeps the old data.
There is no real performance difference.
SSD (slow&old), XFS, Core i7-8565U:
Sync
```
name old time/op new time/op delta
Put/size=1024,thread=1/fstree-8 1.74ms ± 3% 0.06ms ± 7% -96.31% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 10.0ms ±41% 1.1ms ±18% -88.95% (p=0.000 n=9+10)
Put/size=1024,thread=100/fstree-8 32.3ms ±60% 6.5ms ±14% -79.97% (p=0.000 n=10+10)
Put/size=1048576,thread=1/fstree-8 17.8ms ±90% 3.4ms ±70% -81.08% (p=0.000 n=10+10)
Put/size=1048576,thread=20/fstree-8 103ms ±174% 112ms ±158% ~ (p=0.971 n=10+10)
Put/size=1048576,thread=100/fstree-8 949ms ±78% 583ms ±132% ~ (p=0.089 n=10+10)
name old alloc/op new alloc/op delta
Put/size=1024,thread=1/fstree-8 3.17kB ± 1% 1.96kB ± 0% -38.09% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 59.6kB ± 1% 39.2kB ± 1% -34.30% (p=0.000 n=8+10)
Put/size=1024,thread=100/fstree-8 299kB ± 0% 198kB ± 0% -33.90% (p=0.000 n=7+9)
Put/size=1048576,thread=1/fstree-8 3.38kB ± 1% 2.36kB ± 1% -30.22% (p=0.000 n=10+10)
Put/size=1048576,thread=20/fstree-8 65.7kB ± 4% 47.7kB ± 6% -27.27% (p=0.000 n=10+10)
Put/size=1048576,thread=100/fstree-8 351kB ± 8% 245kB ± 8% -30.22% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Put/size=1024,thread=1/fstree-8 30.3 ± 2% 21.0 ± 0% -30.69% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 554 ± 1% 413 ± 0% -25.35% (p=0.000 n=8+10)
Put/size=1024,thread=100/fstree-8 2.77k ± 0% 2.07k ± 0% -25.27% (p=0.000 n=7+10)
Put/size=1048576,thread=1/fstree-8 32.0 ± 0% 25.0 ± 0% -21.88% (p=0.000 n=9+8)
Put/size=1048576,thread=20/fstree-8 609 ± 5% 494 ± 6% -18.93% (p=0.000 n=10+10)
Put/size=1048576,thread=100/fstree-8 3.25k ± 9% 2.50k ± 8% -23.21% (p=0.000 n=10+10)
```
No sync
```
name old time/op new time/op delta
Put/size=1024,thread=1/fstree-8 71.3µs ±10% 59.8µs ±10% -16.21% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 1.43ms ± 6% 1.22ms ±13% -14.53% (p=0.000 n=10+10)
Put/size=1024,thread=100/fstree-8 8.12ms ± 3% 6.36ms ± 2% -21.67% (p=0.000 n=8+9)
Put/size=1048576,thread=1/fstree-8 1.88ms ±70% 1.61ms ±78% ~ (p=0.393 n=10+10)
Put/size=1048576,thread=20/fstree-8 32.7ms ±28% 34.2ms ±112% ~ (p=0.968 n=9+10)
Put/size=1048576,thread=100/fstree-8 262ms ±56% 226ms ±34% ~ (p=0.447 n=10+9)
name old alloc/op new alloc/op delta
Put/size=1024,thread=1/fstree-8 2.89kB ± 0% 1.96kB ± 0% -32.28% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 58.2kB ± 0% 39.5kB ± 0% -32.09% (p=0.000 n=8+8)
Put/size=1024,thread=100/fstree-8 291kB ± 0% 198kB ± 0% -32.19% (p=0.000 n=9+9)
Put/size=1048576,thread=1/fstree-8 3.05kB ± 1% 2.13kB ± 1% -30.16% (p=0.000 n=10+9)
Put/size=1048576,thread=20/fstree-8 62.6kB ± 0% 44.3kB ± 0% -29.23% (p=0.000 n=9+9)
Put/size=1048576,thread=100/fstree-8 302kB ± 0% 210kB ± 1% -30.39% (p=0.000 n=9+9)
name old allocs/op new allocs/op delta
Put/size=1024,thread=1/fstree-8 27.0 ± 0% 21.0 ± 0% -22.22% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 539 ± 0% 415 ± 0% -22.98% (p=0.000 n=10+10)
Put/size=1024,thread=100/fstree-8 2.69k ± 0% 2.07k ± 0% -23.09% (p=0.000 n=9+9)
Put/size=1048576,thread=1/fstree-8 28.0 ± 0% 22.3 ± 3% -20.36% (p=0.000 n=8+10)
Put/size=1048576,thread=20/fstree-8 577 ± 0% 458 ± 0% -20.72% (p=0.000 n=9+9)
Put/size=1048576,thread=100/fstree-8 2.76k ± 0% 2.15k ± 0% -22.05% (p=0.000 n=9+8)
```
HDD (LVM), ext4, Ryzen 5 1600:
Sync
```
│ fs.sync-generic │ fs.sync-linux │
│ sec/op │ sec/op vs base │
Put/size=1024,thread=1/fstree-12 34.70m ± 19% 33.59m ± 16% ~ (p=0.529 n=10)
Put/size=1024,thread=20/fstree-12 188.8m ± 8% 189.2m ± 16% ~ (p=0.739 n=10)
Put/size=1024,thread=100/fstree-12 264.8m ± 22% 273.6m ± 28% ~ (p=0.353 n=10)
Put/size=1048576,thread=1/fstree-12 54.90m ± 14% 47.08m ± 18% ~ (p=0.063 n=10)
Put/size=1048576,thread=20/fstree-12 244.1m ± 14% 220.4m ± 22% ~ (p=0.579 n=10)
Put/size=1048576,thread=100/fstree-12 847.2m ± 5% 893.6m ± 3% +5.48% (p=0.000 n=10)
geomean 164.3m 158.9m -3.29%
│ fs.sync-generic │ fs.sync-linux │
│ B/op │ B/op vs base │
Put/size=1024,thread=1/fstree-12 3.375Ki ± 1% 2.471Ki ± 1% -26.80% (p=0.000 n=10)
Put/size=1024,thread=20/fstree-12 66.62Ki ± 6% 49.21Ki ± 6% -26.15% (p=0.000 n=10)
Put/size=1024,thread=100/fstree-12 319.2Ki ± 1% 230.9Ki ± 2% -27.64% (p=0.000 n=10)
Put/size=1048576,thread=1/fstree-12 3.457Ki ± 1% 2.559Ki ± 1% -25.97% (p=0.000 n=10)
Put/size=1048576,thread=20/fstree-12 66.91Ki ± 1% 49.16Ki ± 1% -26.52% (p=0.000 n=10)
Put/size=1048576,thread=100/fstree-12 338.8Ki ± 2% 252.3Ki ± 3% -25.54% (p=0.000 n=10)
geomean 42.17Ki 31.02Ki -26.44%
│ fs.sync-generic │ fs.sync-linux │
│ allocs/op │ allocs/op vs base │
Put/size=1024,thread=1/fstree-12 33.00 ± 0% 27.00 ± 0% -18.18% (p=0.000 n=10)
Put/size=1024,thread=20/fstree-12 639.5 ± 1% 519.0 ± 2% -18.84% (p=0.000 n=10)
Put/size=1024,thread=100/fstree-12 3.059k ± 1% 2.478k ± 2% -18.99% (p=0.000 n=10)
Put/size=1048576,thread=1/fstree-12 33.50 ± 1% 28.00 ± 4% -16.42% (p=0.000 n=10)
Put/size=1048576,thread=20/fstree-12 638.5 ± 1% 520.0 ± 1% -18.56% (p=0.000 n=10)
Put/size=1048576,thread=100/fstree-12 3.209k ± 2% 2.655k ± 2% -17.28% (p=0.000 n=10)
geomean 405.3 332.1 -18.05%
```
No sync
```
│ fs.nosync-generic │ fs.nosync-linux │
│ sec/op │ sec/op vs base │
Put/size=1024,thread=1/fstree-12 148.2µ ± 20% 136.6µ ± 19% -7.89% (p=0.029 n=10)
Put/size=1024,thread=20/fstree-12 1.140m ± 26% 1.364m ± 16% ~ (p=0.143 n=10)
Put/size=1024,thread=100/fstree-12 11.93m ± 68% 26.89m ± 62% ~ (p=0.123 n=10)
Put/size=1048576,thread=1/fstree-12 1.302m ± 3% 1.287m ± 5% ~ (p=0.481 n=10)
Put/size=1048576,thread=20/fstree-12 77.52m ± 8% 74.07m ± 7% ~ (p=0.278 n=10+9)
Put/size=1048576,thread=100/fstree-12 226.1m ± ∞ ¹
geomean 5.986m 3.434m +18.60% ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable
│ fs.nosync-generic │ fs.nosync-linux │
│ B/op │ B/op vs base │
Put/size=1024,thread=1/fstree-12 2.879Ki ± 0% 1.972Ki ± 0% -31.51% (p=0.000 n=10)
Put/size=1024,thread=20/fstree-12 55.94Ki ± 1% 37.90Ki ± 1% -32.25% (p=0.000 n=10)
Put/size=1024,thread=100/fstree-12 272.6Ki ± 0% 182.1Ki ± 9% -33.21% (p=0.000 n=10)
Put/size=1048576,thread=1/fstree-12 3.158Ki ± 0% 2.259Ki ± 0% -28.46% (p=0.000 n=10)
Put/size=1048576,thread=20/fstree-12 58.87Ki ± 0% 41.03Ki ± 0% -30.30% (p=0.000 n=10+9)
Put/size=1048576,thread=100/fstree-12 299.8Ki ± ∞ ¹
geomean 36.71Ki 16.60Ki -31.17% ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable
│ fs.nosync-generic │ fs.nosync-linux │
│ allocs/op │ allocs/op vs base │
Put/size=1024,thread=1/fstree-12 28.00 ± 0% 22.00 ± 0% -21.43% (p=0.000 n=10)
Put/size=1024,thread=20/fstree-12 530.0 ± 0% 407.5 ± 1% -23.11% (p=0.000 n=10)
Put/size=1024,thread=100/fstree-12 2.567k ± 0% 1.956k ± 9% -23.77% (p=0.000 n=10)
Put/size=1048576,thread=1/fstree-12 30.00 ± 0% 24.00 ± 0% -20.00% (p=0.000 n=10)
Put/size=1048576,thread=20/fstree-12 553.5 ± 0% 434.0 ± 0% -21.59% (n=10+9)
Put/size=1048576,thread=100/fstree-12 2.803k ± ∞ ¹
geomean 347.9 178.8 -21.99% ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable
```
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
Evgenii Stratonikov
ff488b53a1
[ #970 ] fstree: Move write functions to a separate file
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
Evgenii Stratonikov
9a622a750d
[ #970 ] fstree: Move temporary path handling in a separate function
...
Allow to easier test different implementations.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
Anton Nikiforov
d19ade23c8
[ #959 ] node: Set mode to shard's components when open it
...
Avoid opening database for `metabase` and `cache` in `Degraded` mode.
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-09 14:04:01 +00:00
Dmitrii Stepanov
db67c21d55
[ #947 ] engine: Evacuate trees to remote nodes
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:33:15 +03:00
Dmitrii Stepanov
728150d1d2
[ #947 ] engine: Evacuate trees to local shards
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:33:15 +03:00
Dmitrii Stepanov
15d853ea22
[ #947 ] controlSvc: Return tree evacuation stat
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:39 +03:00
Dmitrii Stepanov
b3f3505ada
[ #947 ] cli: Allow to specify evacuation scope
...
It may be required to evacuate only objects or only tree or all, so
now it spossible to specify.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00
Dmitrii Stepanov
a6eb66bf9c
[ #947 ] evacuate: Refactor evacuate parameters
...
Drop methods to make it easier to extend.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00
Dmitrii Stepanov
8e2a0611f4
[ #947 ] tree: Add method to list all trees
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00
Evgenii Stratonikov
edbe06e07e
[ #956 ] policer/test: Reuse testPool helper
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 15:49:43 +00:00
Evgenii Stratonikov
cbfeb72466
[ #956 ] policer: Remove WithMaxCapacity option
...
We already provide the pool and this argument is used only for
preallocation. No functional changes.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 15:49:43 +00:00
Dmitrii Stepanov
cfc5ce7853
[ #964 ] metabase: Drop GC marks if object not found
...
GC inhumes expired locks and tombstones on all the shards.
So it could be GC mark without object.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-08 07:54:39 +00:00
Evgenii Stratonikov
c3fa902780
[ #969 ] policer: Restrict the number of remembered errors
...
DCO action / DCO (pull_request) Successful in 4m23s
Details
Vulncheck / Vulncheck (pull_request) Successful in 4m57s
Details
Build / Build Components (1.21) (pull_request) Successful in 6m3s
Details
Build / Build Components (1.20) (pull_request) Successful in 9m2s
Details
Tests and linters / Staticcheck (pull_request) Successful in 10m4s
Details
Tests and linters / Lint (pull_request) Successful in 10m51s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 10m44s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 10m56s
Details
Tests and linters / Tests with -race (pull_request) Successful in 10m57s
Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 10:10:41 +03:00
Evgenii Stratonikov
6010dfdf3d
[ #969 ] policer: Make error skip thread-safe
...
Introduces in afd2ba9a66
.
Refs #914
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 10:10:41 +03:00
Evgenii Stratonikov
a6c9a337cd
[ #965 ] morph: Get rid of container.List invocations
...
ContainersOf() is better in almost every aspect, besides creating a
session when the containers number is between 1024 and 2048 (prefetch
script does limited unwrapping). Making List() private helps to ensure
it is no longer used and can be safely removed in future.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-07 08:56:27 +00:00