ff4c23f59a
[ #1070 ] services/tree: Fix fast listing depth processing
...
For unsorted `GetSubTree()` we return a single node for depth=1.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-02 14:41:31 +00:00
17af91619a
[ #1070 ] pilorama: Fix cycling behaviour for sorted listing
...
In case there are no items left, return empty slice.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-02 14:41:31 +00:00
d5194ab2a6
[ #949 ] metabase: fix shard.UpdateID()
...
metabase.Open() now reports metabase mode metric. shard.UpdateID()
needs to read shard ID from metabase => needs to open metabase.
It caused reporting 'shard undefined' metrics. To avoid reporting
wrong metrics metabase.GetShardID() was added which also opens
metabase and does not report metrics.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-04-01 17:27:34 +03:00
81a0346a96
[ #949 ] metabase: fix metabase mode metric
...
It used to always show CLOSED regardless of actual mode.
Now metric represents actual metabase mode of operations.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-04-01 17:27:34 +03:00
e12fcc041d
[ #1059 ] services/tree: Fast sorted listing
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-01 12:37:34 +00:00
f23e38c285
Revert "[ #446 ] engine: Move to read-only on blobstor errors"
...
This reverts commit 69df0d21c2
2024-04-01 12:48:30 +03:00
942d83611b
[ #874 ] engine: Revert Check object existance concurrently
...
This reverts commit f526f49995
2024-04-01 08:42:34 +00:00
fd8cdb9671
[ #1057 ] netmap: Do not iterate over external addresses in Node
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-29 20:46:24 +00:00
0990a9b0bd
[ #1055 ] blobstor: fix mode metric
...
It used to always show CLOSED after setting shard mode
to read-only regardless of actual mode.
Now metric represents actual blobstor mode of operations.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-03-29 20:44:47 +00:00
c7a12ca3d8
[ #1054 ] network: Optimize IsTLSEnabled()
...
No big deal, but it is called multiple times in sorting routine, this
easily results in 20 allocations per group traversal.
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/network
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ old │ new │
│ sec/op │ sec/op vs base │
AddressTLSEnabled-8 184.6n ± 1% 103.3n ± 6% -44.04% (p=0.000 n=10)
│ old │ new │
│ B/op │ B/op vs base │
AddressTLSEnabled-8 704.0 ± 0% 0.0 ± 0% -100.00% (p=0.000 n=10)
│ old │ new │
│ allocs/op │ allocs/op vs base │
AddressTLSEnabled-8 1.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-21 18:48:35 +03:00
c09c701613
[ #1048 ] metabase: Fix drop buckets during resync
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-19 14:28:31 +03:00
6959e617c4
[ #1047 ] object: Set container owner ID property to ape request
...
* Introduce ContainerOwner field in RequestContext.
* Set ContainerOwner in aclv2 middleware.
* Set PropertyKeyContainerOwnerID for object ape request.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-18 15:39:50 +00:00
d7be70e93f
[ #1040 ] object: Wrap CheckAPE errors to status errors
...
* All methods should wrap CheckAPE error, if it occurs, to
status error.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-14 07:34:03 +00:00
5ee5f1df42
[ #976 ] control: Introduce new method RemoveChainLocalOverridesByTarget
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-13 15:33:19 +03:00
2d595ec15f
[ #976 ] ape: Update dependency
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-13 14:54:19 +03:00
7ed07d2dfd
[ #976 ] morph: Implement missing methods for SwitchRPCGuardedActor
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-13 14:54:19 +03:00
17f5463389
[ #1043 ] cli: Add reset evacuation status command
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-13 10:29:45 +00:00
31e2396a5f
[ #1043 ] control: Add ResetEvacuationStatus implementation
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-13 10:29:45 +00:00
926cdeb072
[ #1043 ] services: Regenerate proto
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-13 10:29:45 +00:00
5c252c9193
[ #1039 ] object: Skip APE check for certain request roles
...
* Skip APE check if a role is Container.
* Skip APE check if a role is IR and methods are get-like.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-12 16:15:20 +03:00
3195142d67
[ #959 ] writecache: Avoid manipulation with cache in DEGRADED mode
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-11 18:35:41 +00:00
d433b49265
[ #973 ] node: Resolve perfsprint linter
...
`fmt.Errorf can be replaced with errors.New` and `fmt.Sprintf can be replaced with string addition`
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-11 17:55:50 +03:00
66a26b7775
[ #973 ] node: Resolve revive: unused-parameter linter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-11 17:11:49 +03:00
0882840bf5
[ #634 ] shard: Add writecache inhume tests
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-06 13:12:34 +03:00
ae5bb87e70
Revert "[ #866 ] Use TTL for blobovnicza tree cache"
...
This reverts commit d9cbb16bd3
2024-03-01 19:29:33 +03:00
d6534fd755
[ #1016 ] frostfs-node: Fix gopls issues
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-01 12:13:43 +03:00
93bf9acbc2
[ #898 ] control: Remove removed flag from RemoveChainLocalOverrideResponse
...
* Remove removed flag in service.proto for RemoveChainLocalOverrideResponse.
* Regenerate control API.
* Return error only if RemoveOverride returns non-NotFound code.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:07:07 +00:00
75a1a95c2c
[ #986 ] tree: Skip ACL checks if basicACL mask is unset
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:05:57 +00:00
b1d171c261
[ #986 ] container: Interpret APE NoRuleFound as request deny
...
* If APE check returns NoRuleFound, then it is taken for request deny.
* Add more unit-test for ape container middleware.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:05:57 +00:00
7cc368e188
[ #986 ] object: Introduce soft ape checks
...
* Soft APE check means that APE should allow request even
it gets status NoRuleFound for a request. Otherwise,
it is interpreted as Deny.
* Soft APE check is performed if basic ACL mask is not set.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:05:57 +00:00
918613546f
[ #1008 ] metabase: Do not update storageID on put
...
There may be a race condition between put an object and
flushing the writecache:
1. Put object to the writecache
2. Writecache flushes object to the blobstore and sets blobstore's
storageID
3. Put object to the metabase, set writecache's storageID
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-28 11:01:50 +03:00
2ad433dbcb
[ #1005 ] engine: Drop shards weights
...
Unused.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-26 17:25:05 +03:00
4b13b85173
[ #1000 ] morph: Fix batch size in TraverseIterator()
...
Initial prefetch size can be arbitrary an restricted only by VM/RPC
limits. For TraverseIterator() there is an explicit check on the
server-side, though.
Introduced in df055fead5#931 .
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-23 06:24:40 +00:00
e18f0f5178
[ #999 ] morph: Use Global scope for proxy contract
...
Proxy contract can now be used as an owner of NNS domains, thus we need
it not only to pay for the transaction but also to check domain
ownership. CalledByEntry is not enough, because we may register NNS
domains owned by proxy indirectly from the container contract.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-21 14:28:42 +03:00
7470c383dd
[ #997 ] metabase: Drop toMoveIt bucket
...
It is not used.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-21 10:06:05 +03:00
adf7ebab5b
[ #996 ] metabase: Speed up bucket creation
...
Most of the time it exits, e.g. when it is per-container and use on each
object PUT. Bbolt implementation first tries to create bucket and then
returns it if it exists. Create operation uses cursor and thus is not
very lightweight, we can avoid it.
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ old │ new │
│ sec/op │ sec/op vs base │
Put/parallel-8 174.4µ ± 3% 163.3µ ± 3% -6.39% (p=0.000 n=10)
Put/sequential-8 263.3µ ± 2% 259.0µ ± 1% -1.64% (p=0.000 n=10)
geomean 214.3µ 205.6µ -4.05%
│ old │ new │
│ B/op │ B/op vs base │
Put/parallel-8 275.3Ki ± 3% 281.1Ki ± 4% ~ (p=0.063 n=10)
Put/sequential-8 413.0Ki ± 2% 426.6Ki ± 2% +3.29% (p=0.003 n=10)
geomean 337.2Ki 346.3Ki +2.70%
│ old │ new │
│ allocs/op │ allocs/op vs base │
Put/parallel-8 678.0 ± 1% 524.5 ± 2% -22.64% (p=0.000 n=10)
Put/sequential-8 1.329k ± 0% 1.183k ± 0% -10.91% (p=0.000 n=10)
geomean 949.1 787.9 -16.98%
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-20 15:42:58 +00:00
9f68305c2e
Revert "[ #972 ] pilorama: Remove removeDuplicatesInPlace()"
...
This reverts commit 45fd4e4ff1
2024-02-19 15:36:01 +00:00
dad56d2e98
Revert "[ #972 ] Use min/max builtins"
...
This reverts commit 89784b2e0a
2024-02-19 15:36:01 +00:00
7627d08914
Revert "[ #972 ] Use require.ElementsMatch() where possible"
...
This reverts commit 6d9707ff1f
2024-02-19 15:36:01 +00:00
3359349acb
Revert "[ #972 ] Use slices.Sort* when useful"
...
This reverts commit b871d7a5e8
2024-02-19 15:36:01 +00:00
946f2ec2bf
Revert "[ #972 ] Drop x/exp/slices dependency"
...
This reverts commit f3e50772fd
2024-02-19 15:36:01 +00:00
13d5cd3e21
[ #991 ] logger: Fix journald logger
...
Allow to change logger level.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-19 16:18:13 +03:00
f3e50772fd
[ #972 ] Drop x/exp/slices dependency
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
b871d7a5e8
[ #972 ] Use slices.Sort* when useful
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
6d9707ff1f
[ #972 ] Use require.ElementsMatch() where possible
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
89784b2e0a
[ #972 ] Use min/max builtins
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
45fd4e4ff1
[ #972 ] pilorama: Remove removeDuplicatesInPlace()
...
Also, check that slices.CompareFunc() indeed passes all the tests before
removal.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
15fc5bac26
[ #972 ] keyer: Use UncompressedBytes() for marshaling
...
elliptic.Marshal() becomes deprecated in go1.21
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
2680192ba0
[ #988 ] objectSvc: Fix SetMarshalData for PutSingle
...
After api-go update it is required to pass marshal data
to `SetMarshalData`.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 17:21:08 +03:00
05b5f5ca85
[ #959 ] writecache: Fix panic on Get when it is not initialized
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-14 16:10:33 +03:00
2429508ac5
[ #959 ] shard: Skip rebuild in DEGRADED mode
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-14 15:39:28 +03:00
0bd030507e
[ #948 ] metrics: Set actual value for shard_id after restart
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:43:21 +03:00
6a5769d1da
[ #948 ] Fix gofumpt issue
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-13 09:40:46 +03:00
962e5a9c19
[ #736 ] logger: Add journald support
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-12 06:17:22 +00:00
b36a453238
[ #970 ] fstree: Add build tag to enable generic version on linux
...
Unless tested, generic version can start gaining bugs. With a separate
build tag we can have the best of both worlds:
1. Use optimized implementation for linux by default.
2. Run tests or benchmarks for both. Note that they are not actually
run automatically now, but this is at leas possible.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
abd502215f
[ #970 ] fstree: Move file locking to the generic writer
...
It is not a part of FSTree itself, but rather a way to solve concurrent
counter update on non-linux implementations. New linux implementations
is pretty simple: link fails when the file exists, unlink fails when the
file doesn't exist.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
fb74524ac7
[ #970 ] fstree: Move delete implementation to a separate file
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
7f692409cf
[ #970 ] fstree: Handle unsupported O_TMPFILE
...
Metabase test relied on this behaviour, so fix the test too.
Cherry-picking was hard and did too many conflicts,
here is an original PR:
https://github.com/nspcc-dev/neofs-node/pull/2624
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
Roman Khimov
fc31b9c947
[ #970 ] fstree: Add linux-specific file writer using O_TMPFILE
...
O_TMPFILE is implemented for all modern FSes and it's much easier and safer to
use. If application crashes in the middle of writing this file would be gone
and won't leave any garbage.
Notice that this implementation makes a different choice wrt EEXIST handling,
generic one always overwrites, while this one keeps the old data.
There is no real performance difference.
SSD (slow&old), XFS, Core i7-8565U:
Sync
```
name old time/op new time/op delta
Put/size=1024,thread=1/fstree-8 1.74ms ± 3% 0.06ms ± 7% -96.31% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 10.0ms ±41% 1.1ms ±18% -88.95% (p=0.000 n=9+10)
Put/size=1024,thread=100/fstree-8 32.3ms ±60% 6.5ms ±14% -79.97% (p=0.000 n=10+10)
Put/size=1048576,thread=1/fstree-8 17.8ms ±90% 3.4ms ±70% -81.08% (p=0.000 n=10+10)
Put/size=1048576,thread=20/fstree-8 103ms ±174% 112ms ±158% ~ (p=0.971 n=10+10)
Put/size=1048576,thread=100/fstree-8 949ms ±78% 583ms ±132% ~ (p=0.089 n=10+10)
name old alloc/op new alloc/op delta
Put/size=1024,thread=1/fstree-8 3.17kB ± 1% 1.96kB ± 0% -38.09% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 59.6kB ± 1% 39.2kB ± 1% -34.30% (p=0.000 n=8+10)
Put/size=1024,thread=100/fstree-8 299kB ± 0% 198kB ± 0% -33.90% (p=0.000 n=7+9)
Put/size=1048576,thread=1/fstree-8 3.38kB ± 1% 2.36kB ± 1% -30.22% (p=0.000 n=10+10)
Put/size=1048576,thread=20/fstree-8 65.7kB ± 4% 47.7kB ± 6% -27.27% (p=0.000 n=10+10)
Put/size=1048576,thread=100/fstree-8 351kB ± 8% 245kB ± 8% -30.22% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Put/size=1024,thread=1/fstree-8 30.3 ± 2% 21.0 ± 0% -30.69% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 554 ± 1% 413 ± 0% -25.35% (p=0.000 n=8+10)
Put/size=1024,thread=100/fstree-8 2.77k ± 0% 2.07k ± 0% -25.27% (p=0.000 n=7+10)
Put/size=1048576,thread=1/fstree-8 32.0 ± 0% 25.0 ± 0% -21.88% (p=0.000 n=9+8)
Put/size=1048576,thread=20/fstree-8 609 ± 5% 494 ± 6% -18.93% (p=0.000 n=10+10)
Put/size=1048576,thread=100/fstree-8 3.25k ± 9% 2.50k ± 8% -23.21% (p=0.000 n=10+10)
```
No sync
```
name old time/op new time/op delta
Put/size=1024,thread=1/fstree-8 71.3µs ±10% 59.8µs ±10% -16.21% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 1.43ms ± 6% 1.22ms ±13% -14.53% (p=0.000 n=10+10)
Put/size=1024,thread=100/fstree-8 8.12ms ± 3% 6.36ms ± 2% -21.67% (p=0.000 n=8+9)
Put/size=1048576,thread=1/fstree-8 1.88ms ±70% 1.61ms ±78% ~ (p=0.393 n=10+10)
Put/size=1048576,thread=20/fstree-8 32.7ms ±28% 34.2ms ±112% ~ (p=0.968 n=9+10)
Put/size=1048576,thread=100/fstree-8 262ms ±56% 226ms ±34% ~ (p=0.447 n=10+9)
name old alloc/op new alloc/op delta
Put/size=1024,thread=1/fstree-8 2.89kB ± 0% 1.96kB ± 0% -32.28% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 58.2kB ± 0% 39.5kB ± 0% -32.09% (p=0.000 n=8+8)
Put/size=1024,thread=100/fstree-8 291kB ± 0% 198kB ± 0% -32.19% (p=0.000 n=9+9)
Put/size=1048576,thread=1/fstree-8 3.05kB ± 1% 2.13kB ± 1% -30.16% (p=0.000 n=10+9)
Put/size=1048576,thread=20/fstree-8 62.6kB ± 0% 44.3kB ± 0% -29.23% (p=0.000 n=9+9)
Put/size=1048576,thread=100/fstree-8 302kB ± 0% 210kB ± 1% -30.39% (p=0.000 n=9+9)
name old allocs/op new allocs/op delta
Put/size=1024,thread=1/fstree-8 27.0 ± 0% 21.0 ± 0% -22.22% (p=0.000 n=10+10)
Put/size=1024,thread=20/fstree-8 539 ± 0% 415 ± 0% -22.98% (p=0.000 n=10+10)
Put/size=1024,thread=100/fstree-8 2.69k ± 0% 2.07k ± 0% -23.09% (p=0.000 n=9+9)
Put/size=1048576,thread=1/fstree-8 28.0 ± 0% 22.3 ± 3% -20.36% (p=0.000 n=8+10)
Put/size=1048576,thread=20/fstree-8 577 ± 0% 458 ± 0% -20.72% (p=0.000 n=9+9)
Put/size=1048576,thread=100/fstree-8 2.76k ± 0% 2.15k ± 0% -22.05% (p=0.000 n=9+8)
```
HDD (LVM), ext4, Ryzen 5 1600:
Sync
```
│ fs.sync-generic │ fs.sync-linux │
│ sec/op │ sec/op vs base │
Put/size=1024,thread=1/fstree-12 34.70m ± 19% 33.59m ± 16% ~ (p=0.529 n=10)
Put/size=1024,thread=20/fstree-12 188.8m ± 8% 189.2m ± 16% ~ (p=0.739 n=10)
Put/size=1024,thread=100/fstree-12 264.8m ± 22% 273.6m ± 28% ~ (p=0.353 n=10)
Put/size=1048576,thread=1/fstree-12 54.90m ± 14% 47.08m ± 18% ~ (p=0.063 n=10)
Put/size=1048576,thread=20/fstree-12 244.1m ± 14% 220.4m ± 22% ~ (p=0.579 n=10)
Put/size=1048576,thread=100/fstree-12 847.2m ± 5% 893.6m ± 3% +5.48% (p=0.000 n=10)
geomean 164.3m 158.9m -3.29%
│ fs.sync-generic │ fs.sync-linux │
│ B/op │ B/op vs base │
Put/size=1024,thread=1/fstree-12 3.375Ki ± 1% 2.471Ki ± 1% -26.80% (p=0.000 n=10)
Put/size=1024,thread=20/fstree-12 66.62Ki ± 6% 49.21Ki ± 6% -26.15% (p=0.000 n=10)
Put/size=1024,thread=100/fstree-12 319.2Ki ± 1% 230.9Ki ± 2% -27.64% (p=0.000 n=10)
Put/size=1048576,thread=1/fstree-12 3.457Ki ± 1% 2.559Ki ± 1% -25.97% (p=0.000 n=10)
Put/size=1048576,thread=20/fstree-12 66.91Ki ± 1% 49.16Ki ± 1% -26.52% (p=0.000 n=10)
Put/size=1048576,thread=100/fstree-12 338.8Ki ± 2% 252.3Ki ± 3% -25.54% (p=0.000 n=10)
geomean 42.17Ki 31.02Ki -26.44%
│ fs.sync-generic │ fs.sync-linux │
│ allocs/op │ allocs/op vs base │
Put/size=1024,thread=1/fstree-12 33.00 ± 0% 27.00 ± 0% -18.18% (p=0.000 n=10)
Put/size=1024,thread=20/fstree-12 639.5 ± 1% 519.0 ± 2% -18.84% (p=0.000 n=10)
Put/size=1024,thread=100/fstree-12 3.059k ± 1% 2.478k ± 2% -18.99% (p=0.000 n=10)
Put/size=1048576,thread=1/fstree-12 33.50 ± 1% 28.00 ± 4% -16.42% (p=0.000 n=10)
Put/size=1048576,thread=20/fstree-12 638.5 ± 1% 520.0 ± 1% -18.56% (p=0.000 n=10)
Put/size=1048576,thread=100/fstree-12 3.209k ± 2% 2.655k ± 2% -17.28% (p=0.000 n=10)
geomean 405.3 332.1 -18.05%
```
No sync
```
│ fs.nosync-generic │ fs.nosync-linux │
│ sec/op │ sec/op vs base │
Put/size=1024,thread=1/fstree-12 148.2µ ± 20% 136.6µ ± 19% -7.89% (p=0.029 n=10)
Put/size=1024,thread=20/fstree-12 1.140m ± 26% 1.364m ± 16% ~ (p=0.143 n=10)
Put/size=1024,thread=100/fstree-12 11.93m ± 68% 26.89m ± 62% ~ (p=0.123 n=10)
Put/size=1048576,thread=1/fstree-12 1.302m ± 3% 1.287m ± 5% ~ (p=0.481 n=10)
Put/size=1048576,thread=20/fstree-12 77.52m ± 8% 74.07m ± 7% ~ (p=0.278 n=10+9)
Put/size=1048576,thread=100/fstree-12 226.1m ± ∞ ¹
geomean 5.986m 3.434m +18.60% ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable
│ fs.nosync-generic │ fs.nosync-linux │
│ B/op │ B/op vs base │
Put/size=1024,thread=1/fstree-12 2.879Ki ± 0% 1.972Ki ± 0% -31.51% (p=0.000 n=10)
Put/size=1024,thread=20/fstree-12 55.94Ki ± 1% 37.90Ki ± 1% -32.25% (p=0.000 n=10)
Put/size=1024,thread=100/fstree-12 272.6Ki ± 0% 182.1Ki ± 9% -33.21% (p=0.000 n=10)
Put/size=1048576,thread=1/fstree-12 3.158Ki ± 0% 2.259Ki ± 0% -28.46% (p=0.000 n=10)
Put/size=1048576,thread=20/fstree-12 58.87Ki ± 0% 41.03Ki ± 0% -30.30% (p=0.000 n=10+9)
Put/size=1048576,thread=100/fstree-12 299.8Ki ± ∞ ¹
geomean 36.71Ki 16.60Ki -31.17% ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable
│ fs.nosync-generic │ fs.nosync-linux │
│ allocs/op │ allocs/op vs base │
Put/size=1024,thread=1/fstree-12 28.00 ± 0% 22.00 ± 0% -21.43% (p=0.000 n=10)
Put/size=1024,thread=20/fstree-12 530.0 ± 0% 407.5 ± 1% -23.11% (p=0.000 n=10)
Put/size=1024,thread=100/fstree-12 2.567k ± 0% 1.956k ± 9% -23.77% (p=0.000 n=10)
Put/size=1048576,thread=1/fstree-12 30.00 ± 0% 24.00 ± 0% -20.00% (p=0.000 n=10)
Put/size=1048576,thread=20/fstree-12 553.5 ± 0% 434.0 ± 0% -21.59% (n=10+9)
Put/size=1048576,thread=100/fstree-12 2.803k ± ∞ ¹
geomean 347.9 178.8 -21.99% ²
¹ need >= 6 samples for confidence interval at level 0.95
² benchmark set differs from baseline; geomeans may not be comparable
```
Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
ff488b53a1
[ #970 ] fstree: Move write functions to a separate file
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
9a622a750d
[ #970 ] fstree: Move temporary path handling in a separate function
...
Allow to easier test different implementations.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
d19ade23c8
[ #959 ] node: Set mode to shard's components when open it
...
Avoid opening database for `metabase` and `cache` in `Degraded` mode.
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-09 14:04:01 +00:00
db67c21d55
[ #947 ] engine: Evacuate trees to remote nodes
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:33:15 +03:00
728150d1d2
[ #947 ] engine: Evacuate trees to local shards
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:33:15 +03:00
15d853ea22
[ #947 ] controlSvc: Return tree evacuation stat
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:39 +03:00
b3f3505ada
[ #947 ] cli: Allow to specify evacuation scope
...
It may be required to evacuate only objects or only tree or all, so
now it spossible to specify.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00
a6eb66bf9c
[ #947 ] evacuate: Refactor evacuate parameters
...
Drop methods to make it easier to extend.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00
8e2a0611f4
[ #947 ] tree: Add method to list all trees
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00
edbe06e07e
[ #956 ] policer/test: Reuse testPool helper
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 15:49:43 +00:00
cbfeb72466
[ #956 ] policer: Remove WithMaxCapacity option
...
We already provide the pool and this argument is used only for
preallocation. No functional changes.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 15:49:43 +00:00
cfc5ce7853
[ #964 ] metabase: Drop GC marks if object not found
...
GC inhumes expired locks and tombstones on all the shards.
So it could be GC mark without object.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-08 07:54:39 +00:00
c3fa902780
[ #969 ] policer: Restrict the number of remembered errors
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 10:10:41 +03:00
6010dfdf3d
[ #969 ] policer: Make error skip thread-safe
...
Introduces in afd2ba9a66#914
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 10:10:41 +03:00
a6c9a337cd
[ #965 ] morph: Get rid of container.List invocations
...
ContainersOf() is better in almost every aspect, besides creating a
session when the containers number is between 1024 and 2048 (prefetch
script does limited unwrapping). Making List() private helps to ensure
it is no longer used and can be safely removed in future.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-07 08:56:27 +00:00
9ba48c582d
[ #917 ] engine: Allow to detach shards
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-06 14:49:47 +03:00
4358d3c423
[ #917 ] controlSvc: Add DetachShards handler
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-06 14:47:52 +03:00
afd2ba9a66
[ #110 ] Add check for repeated error log in policer
...
processObject() returns 3 types of errors: container not found errors,
could not get container error and placement vector building error. Every
error will occur for all objects in container simultaneously, so we can
log each error once and safely ignore the rest.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-02-06 00:56:41 +03:00
602ee11123
[ #934 ] containersvc: Marhal public key in short format for APE
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-02 17:51:38 +00:00
c1a5b831b6
[ #955 ] chainbase: Fix rule chain unmarshalling
...
* Use correct way DecodeBytes instead unmarshalling by json.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-02 20:28:04 +03:00
9916598dfb
[ #922 ] control: Extend api with ListOverrideDefinedTargets
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
95e15f499f
[ #922 ] Update files generated by protoc
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
2cb04379a4
[ #922 ] go.mod: Update APE
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
a5446bc17d
[ #952 ] object: Pass namespace within context in ACL service
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-02 14:48:11 +03:00
d0eadf7ea2
[ #799 ] engine: Skip put when object removed from shard
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-01 17:49:22 +00:00
6534252c22
[ #799 ] policer: Refactor method processNodes
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-01 17:49:22 +00:00
5be2af881a
[ #934 ] container: Make container APE middleware read namespaces
...
* Those methods that can access already existing containers and thus
can get container properties should read namespace from Zone
property. If Zone is not set, take a namespace for root.
* Otherwise, define namespaces by owner ID via frostfs-id contract.
* Improve unit-tests, consider more cases.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-01 17:38:24 +00:00
4352bd0e8e
[ #934 ] ape: Transform empty namespace within chainbase
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-01 17:38:24 +00:00
483a67b170
[ #937 ] ape: Validate chain resource name
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-31 11:34:35 +03:00
e3573de6db
[ #930 ] gc: Stop internal activity by context
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-31 08:30:34 +00:00
c441296592
[ #930 ] policer: Release task pool when context cancelled
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-31 08:30:34 +00:00
675eec91f3
[ #938 ] shard: Update only changed counters
...
If metric value hasn't changed, but we update metric, then
non existed metric will apear with zero value.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-30 12:37:48 +03:00
c681354afd
[ #938 ] engine: Fix container count removal
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-30 12:37:48 +03:00
df055fead5
[ #931 ] morph: Provide batch size for container listing explicitly
...
Besides VM stack item limit we also have restrictions on the size of
JSON for a stackitem, which is 128k. This limit is much harder to
calculate, because JSON representation includes type and the encoding is
different for different items. Thus is makes no sense to invent our own
default, so use the one provided by neo-go. But for container listing we
know exactly what we process, so use big enough value, which is tested.
Introduced in be8607a1f6#902
Refs https://github.com/nspcc-dev/neo-go/blob/v0.105.0/pkg/vm/stackitem/json.go#L353
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-29 14:00:11 +00:00
6e2cc32768
[ #681 ] objsvc: Validate session token owner for local sessions
...
Previously, the check was in place only when session token was missing.
Format validator checks are applied only to fully-prepared object, so
this lead to the following situation:
1. Object is put locally with malformed token, because there are no
checks.
2. Object cannot be replicated, because the token is malformed.
This is now fixed and token check is done before any payload receival.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-26 08:52:29 +00:00
b6fc3321c5
[ #876 ] Fix linters
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-25 20:26:13 +03:00
1fe7736d92
[ #925 ] morph: Introduce switch rpc guarded rpc actor
...
* Introduce switch rpc guarded rpc actor in morph client
to prevent using invalidated rpc actor when RPC switch
happens.
* Initialize NewContractStorage with SwitchRPCGuardedActor.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-25 13:24:13 +00:00
5ed330e436
[ #927 ] metabase: Delete GC marks
...
`key` is changed inside `db.get`, so encode address again after get.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-24 18:51:16 +03:00
931a5e9aaf
[ #918 ] engine: Move shard to degraded mode if metabase open failed
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-23 11:16:40 +03:00
f2f3294fc3
[ #919 ] ape: Improve error messages in ape service
...
* Wrap all APE middleware errors in apeErr that
makes errors more explicit with status AccessDenied.
* Use denyingRuleErr for denying status from chain router.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-23 08:11:24 +00:00
f526f49995
[ #874 ] engine: Check object existance concurrently
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-23 09:28:29 +03:00
f5160b27fc
[ #920 ] tests: Fix data races
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-19 14:06:05 +03:00
be33070550
[ #910 ] control: Take empty namespace for 'root'
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-19 06:22:10 +00:00
63d3ed1ad8
[ #904 ] tests: Close test engine after test
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-17 19:04:39 +03:00
57171907e3
[ #904 ] metabase: Return if object was actuall inserted
...
This requires to count metrics properly.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-17 19:04:39 +03:00
c1a80235db
[ #904 ] metabase: Log Inhume operation
...
It will be very useful for troubleshooting.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-17 18:42:52 +03:00
96b020626f
[ #915 ] ape: Fix method name in getStreamBasicChecker
...
* Replace incorrect MethodGetContainer by MethodGetObject constant.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-16 23:52:37 +03:00
c8baf76fae
[ #872 ] object: Introduce APE middlewar for object service
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-12 18:41:35 +03:00
52ffa9f164
[ #891 ] getSvc: Refactor Get service V2 creation
...
Use arguments for mandatory fields.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-12 13:35:38 +03:00
394f086fe2
[ #891 ] getSvc: Fix get range hash implementation
...
Get range can perform GET request, so this request must be done
from container node to not to get access denied error.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-12 13:35:38 +03:00
be8607a1f6
[ #902 ] morph: Avoid creating session in TestInvokeIterator
...
When the number of items to iterate over is less than 2048, there is no
need to create a session and consume resources.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-12 10:25:56 +00:00
a2ab373a0a
[ #895 ] metabase: Do not delete GC mark for virtual objects
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-11 12:32:09 +00:00
7166e77c2b
[ #895 ] test: Add logger to test shard
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-11 12:32:09 +00:00
47dcfa20f3
[ #895 ] test: Use t.Cleanup only for external resources
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-11 12:32:09 +00:00
836818fb75
[ #895 ] test: Use zaptest logger
...
Previous implementation mixes logs from different tests, now logs
are separeted by test.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-11 12:32:09 +00:00
f1b2b8bffa
[ #895 ] test: Fix NewLogger arguments list
...
`debug` is always true.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-11 12:32:09 +00:00
a8e52ef7aa
[ #898 ] control: Fix codes for returning APE errors
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-11 12:31:31 +00:00
5d982976fd
[ #898 ] ape: Fix bug with creating type bucket in chainbase
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-11 12:31:31 +00:00
c19396d203
[ #885 ] control: Make chain id bytes in grpc
...
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-01-11 07:24:22 +00:00
5c0a736a25
[ #899 ] containerSvc: Fix invalid session token type
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-10 18:37:54 +03:00
79bebe4a68
[ #884 ] cli: Fix error message for undefined endpoint
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-01-10 10:02:16 +00:00
4b8b4da681
[ #864 ] engine: Drop container count metric if container removed
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-10 10:45:32 +03:00
d75e7e9a21
[ #864 ] engine: Drop container size metric if container deleted
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-10 10:44:54 +03:00
dfd62ca6b1
[ #864 ] metabase: Refactor delete/inhume
...
Available -> Logic, Raw -> Phy for delete/inhume results.
Use single counter instead of vectors.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-09 09:59:42 +03:00
225fe2d4d5
[ #894 ] blobovniczatree: Speedup rebuild test
...
Down from 3s to 300ms.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-12-29 16:28:54 +00:00
530249e3bd
[ #893 ] go.mod: Update neo-go
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-12-29 15:03:47 +00:00
581887148a
[ #569 ] cli: Add control shards writecache seal command
...
It does the same as `control shards flush-writecache --seal`, but
has better name.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 16:05:37 +03:00
7a9db5bcdd
[ #569 ] writecache: Do not wait modeMtx if mode changes
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 16:05:37 +03:00
32c282ca10
[ #569 ] writecache: Refactor flush
...
Make single RUnlock call instead of two.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 16:05:37 +03:00
0cb0fc1735
[ #569 ] writecache: Allow to seal writecache after flush
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 16:05:37 +03:00
b118734909
[ #890 ] getsvc: Log node PK
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 14:24:13 +03:00
764f70634d
[ #881 ] containerSvc: Add APE validation
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-27 11:05:34 +03:00
8180a0664f
[ #887 ] node: Drop badger writecache implementation
...
Badger implementation isn't tested and works not well,
but requires human resources to maintain.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-22 13:00:54 +03:00
eab981bf1a
[ #876 ] Fix linter error
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-12-21 18:21:37 +03:00
d9cbb16bd3
[ #866 ] Use TTL for blobovnicza tree cache
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-12-19 16:36:28 +00:00
7d7cf05575
[ #851 ] ape: Initialize and use policy contract interface
...
* Replace inmemory policy contract mock by initialized
policy contract interface.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-18 11:06:35 +00:00
bdd43f6211
[ #869 ] object: Pass just CID to chain router
...
* Do not convert CID from request to native-schema resource
format - this step is unneccessary for APE.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-14 11:01:20 +00:00
2d4c0a0f4a
[ #552 ] Add systemd notifications to ir service
...
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-12-13 17:51:41 +03:00
eca7ac9f0d
[ #552 ] Add sdnotify package
...
To avoid using third-party dependencies.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-12-13 17:49:26 +03:00
05f8f49289
[ #552 ] gofumpt changes
...
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-12-13 15:02:25 +03:00
7eb46404a1
[ #863 ] blobovnicza: Fix counters
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-13 13:34:29 +03:00
11add38e87
[ #857 ] golangci: Add protogetter linter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-12 16:27:02 +03:00
94ffe8bb45
[ #857 ] golangci: Add testifylint linter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-12 16:27:02 +03:00
5d7833c89b
[ #857 ] golangci: Add perfsprint linter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-12 16:27:02 +03:00
3b7c0362a8
[ #861 ] shard: Fix Delete object
...
It is possible that object doesn't exist in metabase.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-12 14:25:40 +03:00
681b2c5fd4
[ #825 ] policer: Do not drop required linking objects
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-12 11:04:03 +00:00
a3ef7b58b4
[ #755 ] innerring: Check container owner namespace
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-12 12:36:34 +03:00
1cd2bfe51a
[ #755 ] morph: Drop FrostFSID contract usage
...
Unused.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-12 12:36:34 +03:00
70e0c1e082
[ #841 ] ir: Execute netmap.addPeerIR only for state online
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-12-11 13:14:41 +00:00
0f45e3d344
[ #804 ] ape: Implement boltdb storage for local overrides
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 19:08:41 +03:00
e361e017f3
[ #842 ] control: Pass target instead resource name
...
* Update policy-engine package version in go.mod, go.sum.
* Refactor CheckIfRequestPermitted: pass container target
instead container ID.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 14:21:55 +00:00
39060382a1
[ #842 ] control: Recieve target in gRPC methods for APE managing
...
* Introduce Target type and pass it to all gRPC methods
for APE chain managing instead CID.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 14:21:55 +00:00
db49ad16cc
[ #826 ] blobovniczatree: Do not create DB's on init
...
Blobovniczas will be created on write requests.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
ad0697adc4
[ #661 ] blobovnicza: Compute size with record size
...
To get more accurate size of blobovnicza use record
size (lenght of key + lenght of data).
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
e54dc3dc7c
[ #698 ] blobovnicza: Store counter values
...
Blobovnicza initialization take a long time because of bucket
Stat() call. So now blobovnicza stores counters in META bucket.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
5e8c08da3e
[ #661 ] blobstore: Add address to error logs
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
8911656b1a
[ #661 ] metrcis: Add rebuild percent metric
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
2407e5f5ff
[ #661 ] blobovniczatree: Do not sort DB's and indicies
...
Put stores object to next active DB, so there is no need to sort DBs.
In addition, it adds unnecessary DB openings.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
c6a739e746
[ #661 ] blobovniczatree: Make Rebuild concurrent for objects
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
f1c7905263
[ #661 ] blobovniczatree: Make Rebuild concurrent
...
Different DBs can be rebuild concurrently.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
d4d905ecc6
[ #661 ] metrics: Add blobovniczatree rebuild metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
b2769ca3de
[ #661 ] blobovniczatree: Make Rebuild failover safe
...
Now move info stores in blobovnicza, so in case of failover
rebuild completes previous operation first.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
da4fee2d0b
[ #698 ] blobovniczatree: Init blobovniczas concurrently
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
422226da18
[ #661 ] blobovniczatree: Add Rebuild implementation
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:32 +03:00
a531eaf8bc
[ #661 ] blobstor: Add Rebuild implementation
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:32 +03:00
c1667a11d2
[ #661 ] blobovniczatree: Allow to change depth or width
...
Now it is possible to change depth or with of blobovniczatree.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:32 +03:00
484eb59893
[ #661 ] blobovniczatree: Use .db extension for db files
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:32 +03:00
44552a849b
[ #661 ] shard: Add blobstor rebuilder
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:32 +03:00
a478050639
[ #838 ] metabase: Resolve funlen linter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-06 15:44:21 +03:00
d30ab5f29e
[ #838 ] metabase: Count user objects
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-06 15:44:21 +03:00
f314da4af3
[ #838 ] metabase: Add user object type counter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-06 15:44:21 +03:00
29550fe600
[ #838 ] shard: Refactor updateMetrics method
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-06 15:44:21 +03:00
b2c63e57ba
[ #651 ] engine/test: Speedup StorageEngine_Inhume
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-30 13:19:43 +00:00
445ebcc0e7
[ #651 ] shard/test: Speedup Shard_Delete
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-30 13:19:43 +00:00
2302e5d342
[ #651 ] shard/test: Refactor Shard_Delete
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-30 13:19:43 +00:00
26e4f7005c
[ #741 ] treesvc: Refactor tree sync
...
Fix linter issues.
Add error logging.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-30 12:45:02 +00:00
b21be1abdd
[ #741 ] treesvc: Do not update sync height if some node is unavailable
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-30 12:45:02 +00:00
b215817e14
[ #741 ] treesvc: Remove unused height variables
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-30 12:45:02 +00:00
c516c7c5f4
[ #821 ] node: Pass user.ID by value
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-23 10:21:07 +03:00
07390ad4e3
[ #715 ] node: Unify config parameter names
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 17:13:50 +03:00
8d18fa159e
[ #667 ] writecache: Fix flush test
...
Allow to disable background flusher for testing purposes.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-17 17:45:43 +03:00
02454df14a
[ #809 ] client: Refactor PrmInit, PrmDial usage
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-17 13:37:03 +00:00
76ff26039c
[ #96 ] node: Drop neo-go's slices package
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-17 13:24:04 +03:00
47286ebf32
[ #805 ] pilorama: Fix TreeDrop
...
* If treeID is empty then deleting buckets for cursor may get
invalidated. So, buckets should be gathered before deleting.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-17 10:21:35 +00:00
29fe8c41f3
[ #655 ] storage: Drop ErrorHandler
...
The only one usage was for logging.
Now logging performed by storage anyway.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-16 17:27:38 +03:00
137e987a4e
[ #655 ] storage: Drop LazyHandler
...
LazyHandler is implemented and used incorrectly.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-16 17:27:38 +03:00
4d5be5ccb5
[ #811 ] ape: Update policy-engine module version and rebase
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-16 11:31:37 +03:00
a952a406a2
[ #506 ] container: Use uint64 for epoch type
...
It is `uint64` in netmap source interfaces and other code.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-14 19:00:11 +00:00
f04806ccd3
[ #506 ] container: Use user.ID in DeletionInfo response
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-14 19:00:11 +00:00
8088063195
[ #787 ] netmap: Refactor NewEpoch method
...
Split for user and control methods.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:22:31 +03:00
c8a62ffedd
[ #787 ] morph: Calculate VUB and nonce when hash is nil
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:13:03 +03:00
2393d13e4d
[ #787 ] morph: Return VUB for IR service calls
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:13:03 +03:00
518f3baf41
[ #787 ] morph: Return VUB from Invoke method
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:13:03 +03:00
bdfa523487
[ #787 ] proto: Add VUB field for IR service
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:13:03 +03:00
0f75e48138
[ #796 ] policer: Fix tombstone objects replication
...
Tombstone objects must be replicated to all container nodes.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 13:39:33 +03:00
7cdae4f660
[ #792 ] proto: Regenerate with fixed version
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 10:09:13 +00:00
9133b4389e
[ #788 ] objectsvc: Fix formatting (gofumpt)
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 10:27:32 +03:00
1b22801eed
[ #788 ] engine: Fix flaky tests
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 10:25:46 +03:00
3534d6d05b
[ #794 ] objectsvc: Return accidentally removed acl checks for Head
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 17:13:58 +03:00
66848d3288
[ #770 ] cli: Add methods to work with APE rules via control svc
...
* Add methods to frostfs-cli
* Implement rpc in control service
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
8e11ef46b8
[ #770 ] object: Introduce ape chain checker for object svc
...
* Introduce Request type converted from RequestInfo type
to implement policy-engine's Request interface
* Implement basic ape checker to check if a request is
permitted to be performed
* Make put handlers use APE checker instead EACL
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
5ec73fe8a0
[ #770 ] node: Introduce ape chain source
...
* Provide methods to access rule chains with access
policy engine (APE) chain source
* Initialize apeChainSource within object service
initialization
* Share apeChainSource with control service
* Implement dummy apeChainSource instance based on
in-memory implementation
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
3a2c319b87
[ #770 ] control: Generate gRPC methods to manipulate APE chains
...
* Define new types and gRPC methods to manipulate APE chains
in control service.
* Stub gRPC handlers for the generated methods.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
70ab1ebd54
[ #763 ] metrics: Add container_objects_total metric
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-08 12:30:57 +03:00
9c98fa6152
[ #763 ] metabase: Add container objects counter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-08 12:30:57 +03:00
226e84d782
[ #684 ] node: Add skipped objects count to evacuation result
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-07 12:17:11 +00:00
523fb3ca51
[ #684 ] proto: Add skipped count to evacuation status response
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-07 12:17:11 +00:00
74c91eeef5
[ #777 ] client: Refactor PrmContainerList, PrmObjectSearch usage
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-06 06:50:11 +00:00
20d6132f31
[ #531 ] signSvc: Add SetMarshaledData method call
...
To reduce memory allocations add `SetMarshaledData` method call
to return already marshalled data in next `StableMarshal` calls.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-02 17:34:33 +03:00
a26483fc30
[ #749 ] morph: Fix panic when closing morph client
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-11-01 10:48:10 +00:00
c80b46fad3
[ #754 ] blobstor: Estimate compressability
...
Now it is possible to enable compressability estimation.
If data is likely uncompressable, it should reduce CPU time and memory.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-01 11:24:32 +03:00
05b508f79a
[ #772 ] proto: Fix file ending
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-31 17:03:04 +03:00
79088baa06
[ #772 ] node: Apply gofumpt
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-31 17:03:03 +03:00
00aa6d9749
[ #633 ] shard/test: Fix TestCounters()
...
Introduced in 362f24953a
2023-10-31 12:53:28 +00:00
b8f79f4227
[ #633 ] shard/test: Fix race conditions in TestCounters()
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-31 12:53:28 +00:00
261d281154
[ #762 ] go.mod: Update sdk-go
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-31 11:22:29 +00:00
869518be0a
[ #728 ] writecache: Fix Badger writecache race.
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-30 18:36:41 +03:00
d4b6ebe7e7
[ #725 ] writecache: Fix metric values
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-27 12:22:29 +03:00
121f5c4dd8
[ #757 ] ir: Do not exclude node in maintenance mode from netmap
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-26 10:50:32 +03:00
9f7c2d8810
[ #752 ] innerring: Simplify keyPosition()
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-25 16:06:44 +03:00
cddc58ace2
[ #752 ] innerring: Optimize keyPosition()
...
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/innerring
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
│ old │ new │
│ sec/op │ sec/op vs base │
KeyPosition-8 2771.50n ± 10% 40.32n ± 4% -98.55% (p=0.000 n=10)
│ old │ new │
│ B/op │ B/op vs base │
KeyPosition-8 1.531Ki ± 0% 0.000Ki ± 0% -100.00% (p=0.000 n=10)
│ old │ new │
│ allocs/op │ allocs/op vs base │
KeyPosition-8 21.00 ± 0% 0.00 ± 0% -100.00% (p=0.000 n=10)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-25 16:06:44 +03:00
0a9830564f
[ #752 ] morph: Adopt neo-go RPC statuses
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-25 16:06:44 +03:00
6950312967
[ #752 ] morph: Drop loop copy kludges
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-25 16:06:44 +03:00
fe1acf9e9a
[ #750 ] morph: Remove deprecated channel use
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-25 07:57:05 +00:00
559ad58ab1
[ #642 ] writecache: Remove usage of close channel in bbolt
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-24 15:57:50 +00:00
c0b86f2d93
[ #642 ] writecache: Remove usage of close channel in badger
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-24 15:57:50 +00:00
b0cf100427
[ #49 ] node: React on SIGHUP only when node in READY state
...
Add more info in logs when node is going to shut down,
but initialization process still in progress.
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-24 15:55:29 +00:00
58b6224dd8
[ #747 ] client: Refactor PrmObjectPutInit usage
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-20 11:55:40 +00:00
12b7cf2533
[ #747 ] client: Refactor PrmObjectPutSingle usage
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-20 11:55:40 +00:00
dc4d27201b
[ #733 ] morph: Fix delete container signature check
...
Committed invalid condition, it was just for debug.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-19 18:07:37 +03:00
189dbb01be
[ #733 ] frostfs-cli: Add control ir remove-container
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-19 16:22:18 +03:00
f2437f7ae9
[ #734 ] shard: Fix Delete method
...
Due to the flushing data from the writecache to the storage
and simultaneous deletion, a partial deletion situation is possible.
So as a solution, deletion is allowed only when the object is in storage,
because object will be deleted from writecache by flush goroutine.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-16 17:00:18 +03:00
f26233b47a
[ #734 ] metabase: Include UpdateStorageID in metrics and traces
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-16 17:00:18 +03:00
7e0c5a55de
[ #734 ] writecache: Fix flush
...
Now UpdateStorageID doesn't return error in case of logical error.
If object is in graveyard or GC market, it is still required to
update storage ID.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-16 17:00:17 +03:00
3a997d1207
[ #680 ] metrics: Initialize log metrics together with services
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-11 17:08:03 +03:00
994f48f8bb
[ #680 ] metrics: Export log and morph with script
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-11 17:08:03 +03:00
aca11d7474
[ #735 ] policer: Allow to provide metrics from the outside
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-11 15:14:13 +03:00
5e229dc248
[ #701 ] metrics: add metric to evaluate policer performance
...
Add processed objects counter in policerMetrics,
add policer field to NodeMetrics
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-10-09 19:02:08 +00:00
4caa934eea
[ #729 ] containersvc: Remove load announcement
...
IR code was removed in 8879c6ea
2023-10-09 19:01:13 +00:00
d07afd803c
[ #726 ] writecache: Fix small object flush for Badger
...
Do not marshal object twice.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-06 11:32:50 +03:00
997ac7cd8d
[ #726 ] writecache: Fix small object flush for BBolt
...
Do not marshal object twice.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-06 11:32:44 +03:00
c1e4130020
[ #146 ] node: Add trace_id to logs
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-09-27 11:05:27 +03:00
3889e829e6
[ #667 ] writecache: Add logs for report error func in tests
...
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-09-14 17:02:54 +00:00
c6af4a3ec8
[ #679 ] engine: Do not increase error counter on meta mismatch
...
It was introduced in 69e1e6ca
2023-09-14 10:39:18 +03:00
8a81af5a3b
[ #653 ] Add context parameter to Open functions
...
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-09-07 18:03:29 +03:00
a716db99db
[ #668 ] shard/test: Do not alter rootPath option
...
Supposedly, this was added to allow creating 2 different shards without
subtest. Now we use t.TempDir() everywhere, so this should not be a
problem.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
36759f8434
[ #668 ] shard/test: Properly check event processing
...
See https://git.frostfs.info/TrueCloudLab/frostfs-node/actions/runs/1594/jobs/2
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
39879fa868
[ #668 ] shard/test: Add dontRelease options
...
Most of the time we would like to close shard with minor exceptions.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
c661ba1312
[ #668 ] shard/test: Use sane defaults in the test constructor
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
268adb79cb
[ #668 ] shard/test: Simplify shard construction
...
newCustomShard() has many parameters but only the first is obligatory.
`enableWriteCache` is left as-is, because it directly affects the
functionality.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
429f941cda
[ #668 ] shard/test: Release shard in t.Cleanup()
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
