Dmitrii Stepanov
1eb47ab2ce
[ #1080 ] metabase: Add StorageID metric
...
DCO action / DCO (pull_request) Successful in 4m22s
Details
Vulncheck / Vulncheck (pull_request) Successful in 6m22s
Details
Build / Build Components (1.20) (pull_request) Successful in 7m55s
Details
Build / Build Components (1.21) (pull_request) Successful in 7m50s
Details
Tests and linters / gopls check (pull_request) Successful in 10m10s
Details
Tests and linters / Staticcheck (pull_request) Successful in 10m18s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m11s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 12m31s
Details
Tests and linters / Tests with -race (pull_request) Successful in 13m45s
Details
Tests and linters / Lint (pull_request) Successful in 12m42s
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:00:58 +03:00
Dmitrii Stepanov
954881f1ef
[ #1080 ] metabase: Open bucket for container counter once
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:00:58 +03:00
Evgenii Stratonikov
c1d90f018b
[ #1074 ] pilorama: Allow empty filenames in SortedByFilename()
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-04 10:27:45 +00:00
Evgenii Stratonikov
064e18b277
[ #1074 ] pilorama: Remove debug print in tests
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-04 10:27:45 +00:00
Anton Nikiforov
748da78dc7
[ #1072 ] Fix gofumpt issues
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-04 11:24:27 +03:00
Evgenii Stratonikov
17af91619a
[ #1070 ] pilorama: Fix cycling behaviour for sorted listing
...
In case there are no items left, return empty slice.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-02 14:41:31 +00:00
Ekaterina Lebedeva
d5194ab2a6
[ #949 ] metabase: fix shard.UpdateID()
...
Vulncheck / Vulncheck (pull_request) Successful in 1m20s
Details
DCO action / DCO (pull_request) Successful in 1m59s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m25s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m46s
Details
Tests and linters / Staticcheck (pull_request) Successful in 6m5s
Details
Tests and linters / gopls check (pull_request) Successful in 6m17s
Details
Tests and linters / Lint (pull_request) Successful in 7m7s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m38s
Details
Tests and linters / Tests with -race (pull_request) Successful in 8m51s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m56s
Details
metabase.Open() now reports metabase mode metric. shard.UpdateID()
needs to read shard ID from metabase => needs to open metabase.
It caused reporting 'shard undefined' metrics. To avoid reporting
wrong metrics metabase.GetShardID() was added which also opens
metabase and does not report metrics.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-04-01 17:27:34 +03:00
Ekaterina Lebedeva
81a0346a96
[ #949 ] metabase: fix metabase mode metric
...
It used to always show CLOSED regardless of actual mode.
Now metric represents actual metabase mode of operations.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-04-01 17:27:34 +03:00
Evgenii Stratonikov
e12fcc041d
[ #1059 ] services/tree: Fast sorted listing
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-01 12:37:34 +00:00
Evgenii Stratonikov
f23e38c285
Revert "[ #446 ] engine: Move to read-only on blobstor errors"
...
DCO action / DCO (pull_request) Successful in 2m14s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m7s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m30s
Details
Build / Build Components (1.21) (pull_request) Successful in 4m15s
Details
Tests and linters / Staticcheck (pull_request) Successful in 5m41s
Details
Tests and linters / Lint (pull_request) Successful in 6m6s
Details
Tests and linters / gopls check (pull_request) Successful in 6m42s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m47s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m21s
Details
Tests and linters / Tests with -race (pull_request) Successful in 8m20s
Details
This reverts commit 69df0d21c2
.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-01 12:48:30 +03:00
Dmitrii Stepanov
942d83611b
[ #874 ] engine: Revert Check object existance concurrently
...
This reverts commit f526f49995
.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-01 08:42:34 +00:00
Ekaterina Lebedeva
0990a9b0bd
[ #1055 ] blobstor: fix mode metric
...
It used to always show CLOSED after setting shard mode
to read-only regardless of actual mode.
Now metric represents actual blobstor mode of operations.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-03-29 20:44:47 +00:00
Dmitrii Stepanov
c09c701613
[ #1048 ] metabase: Fix drop buckets during resync
...
Vulncheck / Vulncheck (pull_request) Successful in 1m35s
Details
DCO action / DCO (pull_request) Successful in 2m12s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m57s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m45s
Details
Tests and linters / Staticcheck (pull_request) Successful in 5m32s
Details
Tests and linters / gopls check (pull_request) Successful in 5m33s
Details
Tests and linters / Lint (pull_request) Successful in 6m18s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m8s
Details
Tests and linters / Tests with -race (pull_request) Successful in 8m14s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m36s
Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-19 14:28:31 +03:00
Dmitrii Stepanov
31e2396a5f
[ #1043 ] control: Add ResetEvacuationStatus implementation
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-13 10:29:45 +00:00
Anton Nikiforov
3195142d67
[ #959 ] writecache: Avoid manipulation with cache in `DEGRADED` mode
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-11 18:35:41 +00:00
Dmitrii Stepanov
d433b49265
[ #973 ] node: Resolve perfsprint linter
...
DCO action / DCO (pull_request) Successful in 2m40s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m41s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m27s
Details
Build / Build Components (1.21) (pull_request) Successful in 5m6s
Details
Tests and linters / Staticcheck (pull_request) Successful in 6m16s
Details
Tests and linters / gopls check (pull_request) Successful in 6m23s
Details
Tests and linters / Lint (pull_request) Successful in 6m48s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m4s
Details
Tests and linters / Tests with -race (pull_request) Successful in 9m9s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 9m23s
Details
`fmt.Errorf can be replaced with errors.New` and `fmt.Sprintf can be replaced with string addition`
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-11 17:55:50 +03:00
Dmitrii Stepanov
66a26b7775
[ #973 ] node: Resolve revive: unused-parameter linter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-11 17:11:49 +03:00
Dmitrii Stepanov
0882840bf5
[ #634 ] shard: Add writecache inhume tests
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-06 13:12:34 +03:00
Anton Nikiforov
ae5bb87e70
Revert "[ #866 ] Use TTL for blobovnicza tree cache"
...
DCO action / DCO (pull_request) Successful in 2m7s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m7s
Details
Tests and linters / Staticcheck (pull_request) Successful in 5m12s
Details
Tests and linters / Lint (pull_request) Successful in 6m23s
Details
Tests and linters / Tests with -race (pull_request) Successful in 7m11s
Details
Tests and linters / gopls check (pull_request) Successful in 7m27s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m5s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m22s
Details
Build / Build Components (1.21) (pull_request) Successful in 1m41s
Details
Build / Build Components (1.20) (pull_request) Successful in 1m56s
Details
This reverts commit d9cbb16bd3
.
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-01 19:29:33 +03:00
Dmitrii Stepanov
d6534fd755
[ #1016 ] frostfs-node: Fix gopls issues
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-01 12:13:43 +03:00
Dmitrii Stepanov
918613546f
[ #1008 ] metabase: Do not update storageID on put
...
DCO action / DCO (pull_request) Successful in 4m56s
Details
Build / Build Components (1.21) (pull_request) Successful in 7m52s
Details
Build / Build Components (1.20) (pull_request) Successful in 8m11s
Details
Vulncheck / Vulncheck (pull_request) Successful in 8m9s
Details
Tests and linters / Staticcheck (pull_request) Successful in 9m58s
Details
Tests and linters / Lint (pull_request) Successful in 10m42s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m14s
Details
Tests and linters / Tests with -race (pull_request) Successful in 12m17s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 12m34s
Details
There may be a race condition between put an object and
flushing the writecache:
1. Put object to the writecache
2. Writecache flushes object to the blobstore and sets blobstore's
storageID
3. Put object to the metabase, set writecache's storageID
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-28 11:01:50 +03:00
Dmitrii Stepanov
2ad433dbcb
[ #1005 ] engine: Drop shards weights
...
DCO action / DCO (pull_request) Successful in 4m7s
Details
Vulncheck / Vulncheck (pull_request) Successful in 4m53s
Details
Build / Build Components (1.21) (pull_request) Successful in 5m46s
Details
Build / Build Components (1.20) (pull_request) Successful in 6m21s
Details
Tests and linters / Staticcheck (pull_request) Successful in 7m45s
Details
Tests and linters / Lint (pull_request) Successful in 8m44s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m1s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 15m42s
Details
Tests and linters / Tests with -race (pull_request) Successful in 16m10s
Details
Unused.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-26 17:25:05 +03:00
Dmitrii Stepanov
7470c383dd
[ #997 ] metabase: Drop toMoveIt bucket
...
DCO action / DCO (pull_request) Successful in 2m9s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m3s
Details
Build / Build Components (1.20) (pull_request) Successful in 3m56s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m54s
Details
Tests and linters / Staticcheck (pull_request) Successful in 5m26s
Details
Tests and linters / Lint (pull_request) Successful in 6m19s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m6s
Details
Tests and linters / Tests with -race (pull_request) Successful in 8m20s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m39s
Details
It is not used.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-21 10:06:05 +03:00
Evgenii Stratonikov
adf7ebab5b
[ #996 ] metabase: Speed up bucket creation
...
Most of the time it exits, e.g. when it is per-container and use on each
object PUT. Bbolt implementation first tries to create bucket and then
returns it if it exists. Create operation uses cursor and thus is not
very lightweight, we can avoid it.
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ old │ new │
│ sec/op │ sec/op vs base │
Put/parallel-8 174.4µ ± 3% 163.3µ ± 3% -6.39% (p=0.000 n=10)
Put/sequential-8 263.3µ ± 2% 259.0µ ± 1% -1.64% (p=0.000 n=10)
geomean 214.3µ 205.6µ -4.05%
│ old │ new │
│ B/op │ B/op vs base │
Put/parallel-8 275.3Ki ± 3% 281.1Ki ± 4% ~ (p=0.063 n=10)
Put/sequential-8 413.0Ki ± 2% 426.6Ki ± 2% +3.29% (p=0.003 n=10)
geomean 337.2Ki 346.3Ki +2.70%
│ old │ new │
│ allocs/op │ allocs/op vs base │
Put/parallel-8 678.0 ± 1% 524.5 ± 2% -22.64% (p=0.000 n=10)
Put/sequential-8 1.329k ± 0% 1.183k ± 0% -10.91% (p=0.000 n=10)
geomean 949.1 787.9 -16.98%
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-20 15:42:58 +00:00
Evgenii Stratonikov
9f68305c2e
Revert "[ #972 ] pilorama: Remove removeDuplicatesInPlace()"
...
This reverts commit 45fd4e4ff1
.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov
dad56d2e98
Revert "[ #972 ] Use min/max builtins"
...
This reverts commit 89784b2e0a
.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov
7627d08914
Revert "[ #972 ] Use require.ElementsMatch() where possible"
...
This reverts commit 6d9707ff1f
.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov
946f2ec2bf
Revert "[ #972 ] Drop x/exp/slices dependency"
...
This reverts commit f3e50772fd
.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov
f3e50772fd
[ #972 ] Drop x/exp/slices dependency
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Evgenii Stratonikov
6d9707ff1f
[ #972 ] Use require.ElementsMatch() where possible
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Evgenii Stratonikov
89784b2e0a
[ #972 ] Use min/max builtins
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Evgenii Stratonikov
45fd4e4ff1
[ #972 ] pilorama: Remove removeDuplicatesInPlace()
...
Also, check that slices.CompareFunc() indeed passes all the tests before
removal.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Anton Nikiforov
05b5f5ca85
[ #959 ] writecache: Fix panic on `Get` when it is not initialized
...
Vulncheck / Vulncheck (pull_request) Successful in 2m41s
Details
DCO action / DCO (pull_request) Successful in 2m43s
Details
Tests and linters / Staticcheck (pull_request) Successful in 3m55s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m45s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m19s
Details
Tests and linters / Lint (pull_request) Successful in 5m30s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m57s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m22s
Details
Tests and linters / Tests with -race (pull_request) Successful in 8m39s
Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-14 16:10:33 +03:00
Anton Nikiforov
2429508ac5
[ #959 ] shard: Skip rebuild in `DEGRADED` mode
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-14 15:39:28 +03:00
Anton Nikiforov
0bd030507e
[ #948 ] metrics: Set actual value for `shard_id` after restart
...
DCO action / DCO (pull_request) Successful in 2m26s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m56s
Details
Tests and linters / Staticcheck (pull_request) Successful in 4m18s
Details
Build / Build Components (1.21) (pull_request) Successful in 4m11s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m19s
Details
Tests and linters / Lint (pull_request) Successful in 6m3s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m7s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m26s
Details
Tests and linters / Tests with -race (pull_request) Successful in 9m20s
Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:43:21 +03:00
Anton Nikiforov
6a5769d1da
[ #948 ] Fix `gofumpt` issue
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:40:46 +03:00
Evgenii Stratonikov
b36a453238
[ #970 ] fstree: Add build tag to enable generic version on linux
...
Unless tested, generic version can start gaining bugs. With a separate
build tag we can have the best of both worlds:
1. Use optimized implementation for linux by default.
2. Run tests or benchmarks for both. Note that they are not actually
run automatically now, but this is at leas possible.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
Evgenii Stratonikov
abd502215f
[ #970 ] fstree: Move file locking to the generic writer
...
It is not a part of FSTree itself, but rather a way to solve concurrent
counter update on non-linux implementations. New linux implementations
is pretty simple: link fails when the file exists, unlink fails when the
file doesn't exist.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
Evgenii Stratonikov
fb74524ac7
[ #970 ] fstree: Move delete implementation to a separate file
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
Evgenii Stratonikov
7f692409cf
[ #970 ] fstree: Handle unsupported O_TMPFILE
...
Metabase test relied on this behaviour, so fix the test too.
Cherry-picking was hard and did too many conflicts,
here is an original PR:
https://github.com/nspcc-dev/neofs-node/pull/2624
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
Roman Khimov
fc31b9c947
[ #970 ] fstree: Add linux-specific file writer using O_TMPFILE
...
O_TMPFILE is implemented for all modern FSes and it's much easier and safer to
use. If application crashes in the middle of writing this file would be gone
and won't leave any garbage.
Notice that this implementation makes a different choice wrt EEXIST handling,
generic one always overwrites, while this one keeps the old data.
There is no real performance difference.
SSD (slow&old), XFS, Core i7-8565U:
Sync
```
name old time/op new time/op delta
Put/size=1024,thread=1/fstree-8 1.74ms ± 3% 0.06ms ± 7% -96.31% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 10.0ms ±41% 1.1ms ±18% -88.95% (p=0.000 n=9+10)
Put/size=1024,thread=100/fstree-8 32.3ms ±60% 6.5ms ±14% -79.97% (p=0.000 n=10+10)
Put/size=1048576,thread=1/fstree-8 17.8ms ±90% 3.4ms ±70% -81.08% (p=0.000 n=10+10)
Put/size=1048576,thread=20/fstree-8 103ms ±174% 112ms ±158% ~ (p=0.971 n=10+10)
Put/size=1048576,thread=100/fstree-8 949ms ±78% 583ms ±132% ~ (p=0.089 n=10+10)
name old alloc/op new alloc/op delta
Put/size=1024,thread=1/fstree-8 3.17kB ± 1% 1.96kB ± 0% -38.09% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 59.6kB ± 1% 39.2kB ± 1% -34.30% (p=0.000 n=8+10)
Put/size=1024,thread=100/fstree-8 299kB ± 0% 198kB ± 0% -33.90% (p=0.000 n=7+9)
Put/size=1048576,thread=1/fstree-8 3.38kB ± 1% 2.36kB ± 1% -30.22% (p=0.000 n=10+10)
Put/size=1048576,thread=20/fstree-8 65.7kB ± 4% 47.7kB ± 6% -27.27% (p=0.000 n=10+10)
Put/size=1048576,thread=100/fstree-8 351kB ± 8% 245kB ± 8% -30.22% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Put/size=1024,thread=1/fstree-8 30.3 ± 2% 21.0 ± 0% -30.69% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 554 ± 1% 413 ± 0% -25.35% (p=0.000 n=8+10)
Put/size=1024,thread=100/fstree-8 2.77k ± 0% 2.07k ± 0% -25.27% (p=0.000 n=7+10)
Put/size=1048576,thread=1/fstree-8 32.0 ± 0% 25.0 ± 0% -21.88% (p=0.000 n=9+8)
Put/size=1048576,thread=20/fstree-8 609 ± 5% 494 ± 6% -18.93% (p=0.000 n=10+10)
Put/size=1048576,thread=100/fstree-8 3.25k ± 9% 2.50k ± 8% -23.21% (p=0.000 n=10+10)
```
No sync
```
name old time/op new time/op delta
Put/size=1024,thread=1/fstree-8 71.3µs ±10% 59.8µs ±10% -16.21% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 1.43ms ± 6% 1.22ms ±13% -14.53% (p=0.000 n=10+10)
Put/size=1024,thread=100/fstree-8 8.12ms ± 3% 6.36ms ± 2% -21.67% (p=0.000 n=8+9)
Put/size=1048576,thread=1/fstree-8 1.88ms ±70% 1.61ms ±78% ~ (p=0.393 n=10+10)
Put/size=1048576,thread=20/fstree-8 32.7ms ±28% 34.2ms ±112% ~ (p=0.968 n=9+10)
Put/size=1048576,thread=100/fstree-8 262ms ±56% 226ms ±34% ~ (p=0.447 n=10+9)
name old alloc/op new alloc/op delta
Put/size=1024,thread=1/fstree-8 2.89kB ± 0% 1.96kB ± 0% -32.28% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 58.2kB ± 0% 39.5kB ± 0% -32.09% (p=0.000 n=8+8)
Put/size=1024,thread=100/fstree-8 291kB ± 0% 198kB ± 0% -32.19% (p=0.000 n=9+9)
Put/size=1048576,thread=1/fstree-8 3.05kB ± 1% 2.13kB ± 1% -30.16% (p=0.000 n=10+9)
Put/size=1048576,thread=20/fstree-8 62.6kB ± 0% 44.3kB ± 0% -29.23% (p=0.000 n=9+9)
Put/size=1048576,thread=100/fstree-8 302kB ± 0% 210kB ± 1% -30.39% (p=0.000 n=9+9)
name old allocs/op new allocs/op delta
Put/size=1024,thread=1/fstree-8 27.0 ± 0% 21.0 ± 0% -22.22% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 539 ± 0% 415 ± 0% -22.98% (p=0.000 n=10+10)
Put/size=1024,thread=100/fstree-8 2.69k ± 0% 2.07k ± 0% -23.09% (p=0.000 n=9+9)
Put/size=1048576,thread=1/fstree-8 28.0 ± 0% 22.3 ± 3% -20.36% (p=0.000 n=8+10)
Put/size=1048576,thread=20/fstree-8 577 ± 0% 458 ± 0% -20.72% (p=0.000 n=9+9)
Put/size=1048576,thread=100/fstree-8 2.76k ± 0% 2.15k ± 0% -22.05% (p=0.000 n=9+8)
```
HDD (LVM), ext4, Ryzen 5 1600:
Sync
```
│ fs.sync-generic │ fs.sync-linux │
│ sec/op │ sec/op vs base │
Put/size=1024,thread=1/fstree-12 34.70m ± 19% 33.59m ± 16% ~ (p=0.529 n=10)
Put/size=1024,thread=20/fstree-12 188.8m ± 8% 189.2m ± 16% ~ (p=0.739 n=10)
Put/size=1024,thread=100/fstree-12 264.8m ± 22% 273.6m ± 28% ~ (p=0.353 n=10)
Put/size=1048576,thread=1/fstree-12 54.90m ± 14% 47.08m ± 18% ~ (p=0.063 n=10)
Put/size=1048576,thread=20/fstree-12 244.1m ± 14% 220.4m ± 22% ~ (p=0.579 n=10)
Put/size=1048576,thread=100/fstree-12 847.2m ± 5% 893.6m ± 3% +5.48% (p=0.000 n=10)
geomean 164.3m 158.9m -3.29%
│ fs.sync-generic │ fs.sync-linux │
│ B/op │ B/op vs base │
Put/size=1024,thread=1/fstree-12 3.375Ki ± 1% 2.471Ki ± 1% -26.80% (p=0.000 n=10)
Put/size=1024,thread=20/fstree-12 66.62Ki ± 6% 49.21Ki ± 6% -26.15% (p=0.000 n=10)
Put/size=1024,thread=100/fstree-12 319.2Ki ± 1% 230.9Ki ± 2% -27.64% (p=0.000 n=10)
Put/size=1048576,thread=1/fstree-12 3.457Ki ± 1% 2.559Ki ± 1% -25.97% (p=0.000 n=10)
Put/size=1048576,thread=20/fstree-12 66.91Ki ± 1% 49.16Ki ± 1% -26.52% (p=0.000 n=10)
Put/size=1048576,thread=100/fstree-12 338.8Ki ± 2% 252.3Ki ± 3% -25.54% (p=0.000 n=10)
geomean 42.17Ki 31.02Ki -26.44%
│ fs.sync-generic │ fs.sync-linux │
│ allocs/op │ allocs/op vs base │
Put/size=1024,thread=1/fstree-12 33.00 ± 0% 27.00 ± 0% -18.18% (p=0.000 n=10)
Put/size=1024,thread=20/fstree-12 639.5 ± 1% 519.0 ± 2% -18.84% (p=0.000 n=10)
Put/size=1024,thread=100/fstree-12 3.059k ± 1% 2.478k ± 2% -18.99% (p=0.000 n=10)
Put/size=1048576,thread=1/fstree-12 33.50 ± 1% 28.00 ± 4% -16.42% (p=0.000 n=10)
Put/size=1048576,thread=20/fstree-12 638.5 ± 1% 520.0 ± 1% -18.56% (p=0.000 n=10)
Put/size=1048576,thread=100/fstree-12 3.209k ± 2% 2.655k ± 2% -17.28% (p=0.000 n=10)
geomean 405.3 332.1 -18.05%
```
No sync
```
│ fs.nosync-generic │ fs.nosync-linux │
│ sec/op │ sec/op vs base │
Put/size=1024,thread=1/fstree-12 148.2µ ± 20% 136.6µ ± 19% -7.89% (p=0.029 n=10)
Put/size=1024,thread=20/fstree-12 1.140m ± 26% 1.364m ± 16% ~ (p=0.143 n=10)
Put/size=1024,thread=100/fstree-12 11.93m ± 68% 26.89m ± 62% ~ (p=0.123 n=10)
Put/size=1048576,thread=1/fstree-12 1.302m ± 3% 1.287m ± 5% ~ (p=0.481 n=10)
Put/size=1048576,thread=20/fstree-12 77.52m ± 8% 74.07m ± 7% ~ (p=0.278 n=10+9)
Put/size=1048576,thread=100/fstree-12 226.1m ± ∞ ¹
geomean 5.986m 3.434m +18.60% ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable
│ fs.nosync-generic │ fs.nosync-linux │
│ B/op │ B/op vs base │
Put/size=1024,thread=1/fstree-12 2.879Ki ± 0% 1.972Ki ± 0% -31.51% (p=0.000 n=10)
Put/size=1024,thread=20/fstree-12 55.94Ki ± 1% 37.90Ki ± 1% -32.25% (p=0.000 n=10)
Put/size=1024,thread=100/fstree-12 272.6Ki ± 0% 182.1Ki ± 9% -33.21% (p=0.000 n=10)
Put/size=1048576,thread=1/fstree-12 3.158Ki ± 0% 2.259Ki ± 0% -28.46% (p=0.000 n=10)
Put/size=1048576,thread=20/fstree-12 58.87Ki ± 0% 41.03Ki ± 0% -30.30% (p=0.000 n=10+9)
Put/size=1048576,thread=100/fstree-12 299.8Ki ± ∞ ¹
geomean 36.71Ki 16.60Ki -31.17% ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable
│ fs.nosync-generic │ fs.nosync-linux │
│ allocs/op │ allocs/op vs base │
Put/size=1024,thread=1/fstree-12 28.00 ± 0% 22.00 ± 0% -21.43% (p=0.000 n=10)
Put/size=1024,thread=20/fstree-12 530.0 ± 0% 407.5 ± 1% -23.11% (p=0.000 n=10)
Put/size=1024,thread=100/fstree-12 2.567k ± 0% 1.956k ± 9% -23.77% (p=0.000 n=10)
Put/size=1048576,thread=1/fstree-12 30.00 ± 0% 24.00 ± 0% -20.00% (p=0.000 n=10)
Put/size=1048576,thread=20/fstree-12 553.5 ± 0% 434.0 ± 0% -21.59% (n=10+9)
Put/size=1048576,thread=100/fstree-12 2.803k ± ∞ ¹
geomean 347.9 178.8 -21.99% ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable
```
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
Evgenii Stratonikov
ff488b53a1
[ #970 ] fstree: Move write functions to a separate file
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
Evgenii Stratonikov
9a622a750d
[ #970 ] fstree: Move temporary path handling in a separate function
...
Allow to easier test different implementations.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
Anton Nikiforov
d19ade23c8
[ #959 ] node: Set mode to shard's components when open it
...
Avoid opening database for `metabase` and `cache` in `Degraded` mode.
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-09 14:04:01 +00:00
Dmitrii Stepanov
db67c21d55
[ #947 ] engine: Evacuate trees to remote nodes
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:33:15 +03:00
Dmitrii Stepanov
728150d1d2
[ #947 ] engine: Evacuate trees to local shards
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:33:15 +03:00
Dmitrii Stepanov
15d853ea22
[ #947 ] controlSvc: Return tree evacuation stat
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:39 +03:00
Dmitrii Stepanov
b3f3505ada
[ #947 ] cli: Allow to specify evacuation scope
...
It may be required to evacuate only objects or only tree or all, so
now it spossible to specify.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00
Dmitrii Stepanov
a6eb66bf9c
[ #947 ] evacuate: Refactor evacuate parameters
...
Drop methods to make it easier to extend.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00
Dmitrii Stepanov
8e2a0611f4
[ #947 ] tree: Add method to list all trees
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00