3fe0a84364
[ #1450 ] node/config: Allow to configure engine's InhumePoolSize
...
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-11-19 23:10:34 +03:00
9706a03b89
[ #1450 ] engine: Inhume objects in parallel with a worker pool
...
Add a worker pool for `Inhume` operation and use it to handle objects
in parallel. Since metabase `Inhume` uses `bbolt.Batch`, handling many
objects one by one may be inefficient.
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/engine
cpu: 12th Gen Intel(R) Core(TM) i5-1235U
│ old.txt │ new1.txt │
│ sec/op │ sec/op vs base │
InhumeMultipart/objects=1-12 11.42m ± 1% 11.42m ± 1% ~ (p=0.739 n=10)
InhumeMultipart/objects=10-12 113.51m ± 0% 11.62m ± 1% -89.76% (p=0.000 n=10)
InhumeMultipart/objects=100-12 1135.41m ± 1% 28.30m ± 1% -97.51% (p=0.000 n=10)
InhumeMultipart/objects=1000-12 11357.8m ± 0% 259.8m ± 1% -97.71% (p=0.000 n=10)
InhumeMultipart/objects=10000-12 113.251 ± 0% 2.277 ± 1% -97.99% (p=0.000 n=10)
geomean 1.136 74.03m -93.48%
```
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-11-19 23:10:27 +03:00
11f7170f1f
[ #1450 ] engine: Add benchmark for Inhume operation
...
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-11-18 16:13:09 +03:00
d82f0d1926
[ #1496 ] node/control: Await until SetNetmapStatus() persists
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-15 16:36:07 +03:00
acd5babd86
[ #1496 ] morph: Merge InvokeRes and WaitParams
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-15 16:36:07 +03:00
b65874d1c3
[ #1496 ] morph: Return InvokeRes from all invoke*() methods
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-15 16:36:07 +03:00
69c63006da
[ #1496 ] morph: Move tx waiter to morph package
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-15 16:36:07 +03:00
d77a218f7c
[ #1493 ] metabase: Merge Inhume() and DropGraves() for tombstones
...
DropGraves() is only used to drop gravemarks after a tombstone
removal. Thus, it makes sense to do Inhume() and DropGraves() in one
transaction. It has less overhead and no unexpected problems in case
of sudden power failure.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-14 06:47:04 +00:00
44df67492f
[ #1493 ] metabase: Split inhumeTx() into 2 functions
...
No functional changes.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-14 06:47:04 +00:00
1e6f132b4e
[ #1493 ] metabase: Pass InhumePrm by value
...
Unify with the other code, no functional changes.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-14 06:47:04 +00:00
6dc0dc6691
[ #1493 ] shard: Take mode mutex in HandleExpiredTombstones()
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-14 06:47:04 +00:00
f7cb6b4d87
[ #1482 ] Makefile: Update golangci-lint
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-11-13 15:01:41 +00:00
7fc6101bec
[ #1491 ] engine/test: Rework engine test utils
...
- Remove `testNewShard` and `setInitializedShards` because they
violated the default engine workflow. The correct workflow is:
first use `New()`, followed by `Open()`, and then `Init()`. As a
result, adding new logic to `(*StorageEngine).Init` caused several
tests to fail with a panic when attempting to access uninitialized
resources. Now, all engines created with the test utils must be
initialized manually. The new helper method `prepare` can be used
for that purpose.
- Additionally, `setInitializedShards` hardcoded the shard worker
pool size, which prevented it from being configured in tests and
benchmarks. This has been fixed as well.
- Ensure engine initialization is done wherever it was missing.
- Refactor `setShardsNumOpts`, `setShardsNumAdditionalOpts`, and
`setShardsNum`. Make them all depend on `setShardsNumOpts`.
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-11-13 14:42:53 +03:00
7ef36749d0
[ #1491 ] engine/test: Move BenchmarkExists to exists_test.go
...
Move `BenchmarkExists` from `engine_test.go` to `exists_test.go`
for better organization and clarity.
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-11-13 14:09:29 +03:00
c6066d6ee4
[ #1491 ] engine/test: Use more suitable testing utils here and there
...
Use `setShardsNum` instead of `setInitializedShards` wherever possible.
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-11-13 14:09:29 +03:00
612b34d570
[ #1437 ] logger: Add caller skip to log original caller position
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:12 +03:00
7429553266
[ #1437 ] node: Fix contextcheck linter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:10 +03:00
6921a89061
[ #1437 ] ir: Fix contextcheck linters
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:10 +03:00
16598553d9
[ #1437 ] shard: Fix contextcheck linter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:09 +03:00
c139892117
[ #1437 ] ir: Fix contextcheck linter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:09 +03:00
62b5181618
[ #1437 ] blobovnicza: Fix contextcheck linter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:08 +03:00
6db46257c0
[ #1437 ] node: Use ctx for logging
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:07 +03:00
c16dae8b4d
[ #1437 ] logger: Use context to log trace id
...
Signed-off-by: Dmitrii Stepanov
2024-11-13 10:36:07 +03:00
fd004add00
[ #1492 ] metabase: Fix import formatting
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-13 07:30:25 +00:00
8ed7a676d5
[ #1492 ] metabase: Ensure Unmarshal() is called on a cloned slice
...
The slice returned from bucket.Get() is only valid during the tx
lifetime. Cloning it is not necessary everywhere, but better safe than
sorry.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-13 07:30:25 +00:00
b451de94c8
[ #1492 ] metabase: Fix typo in objData
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-13 07:30:25 +00:00
f1556e3c42
[ #1488 ] Makefile: Drop all containers created on env-up
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-12 17:24:44 +03:00
e122ff6013
[ #1488 ] dev: Add Jaeger image and enable tracing on debug
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-12 17:24:44 +03:00
e2658c7519
[ #1488 ] tracing: KV attributes for spans from config
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-12 17:24:43 +03:00
c00f4bab18
[ #1488 ] go.mod: Bump observability version
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-12 17:24:43 +03:00
46fef276b4
[ #1449 ] tree: Log tree sync with Info level
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-12 12:11:07 +00:00
9bd05e94c8
[ #1449 ] tree: Add ApplyBatch method
...
Concurrent Apply can lead to child node applies before parent, so
undo/redo operations will perform. This leads to performance degradation
in case of tree with many sublevels.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-12 12:11:07 +00:00
16830033f8
[ #1483 ] cli: Remove --basic-acl flag
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-12 12:10:51 +00:00
1cf51a8079
[ #1483 ] cli/docs: Remove set-eacl mention
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-12 12:10:51 +00:00
3324c26fd8
[ #1483 ] morph: Remove container.GetEACL()
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-12 12:10:51 +00:00
a692298533
[ #1483 ] node: Remove eACL cache
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-12 12:10:51 +00:00
be2753de00
[ #1490 ] docs: Update description for object.get.priority
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-11-12 13:35:13 +03:00
b543569c3f
[ #1486 ] node: Introduce dual service support
...
* Register GRPC services for both neo.fs.v2 and frost.fs namespaces
* Use this temporary solution until all nodes are updated
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-12 08:43:47 +00:00
80f8a8fd3a
[ #1396 ] cli/playground: Refactor
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-11-12 11:09:27 +03:00
2f3bc6eb84
[ #1396 ] cli/playground: Improve terminal control key handling
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-11-12 11:07:48 +03:00
8a57c78f5f
[ #1484 ] engine: Fix engine metrics
...
1. Add forgotten metrics for client requests
2. Include execIfNotBlocked into metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-11 12:59:20 +03:00
ad01fb958a
[ #1474 ] Stop using obsolete .github directory
...
This commit is a part of multi-repo cleanup effort:
TrueCloudLab/frostfs-infra#136
Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
2024-11-11 06:44:13 +00:00
d336f2d487
[ #1393 ] adm: Make NewLocalActor receive accout name
...
* Some RPC-clients for contracts require different wallet account types.
Since, `Policy` contract gets `consensus` accounts while `NNS` gets
`committee` accounts.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-08 13:57:51 +00:00
c82c753e9f
[ #1480 ] objsvc: Remove useless stream wrappers
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-08 12:01:14 +00:00
f666898e5d
[ #1480 ] objsvc: Remove EACL checks
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-08 12:01:14 +00:00
b1a31281e4
[ #1480 ] ape: Remove SoftAPECheck flag
...
Previous release was EACL-compatible.
Starting from now all EACL should've been migrated to APE chains.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-08 12:01:14 +00:00
764450d04a
[ #1479 ] tree: Regenerate service protobufs
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-08 10:43:44 +03:00
755cae3f19
[ #1479 ] control: Regenerate protobufs for service
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-08 10:43:44 +03:00
9b13a18aac
[ #1479 ] go.mod: Bump frostfs-sdk-go version
...
* Update version within go.mod;
* Fix deprecated frostfs-api-go/v2 package and use frostfs-sdk-go/api
instead.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-08 10:43:19 +03:00
ef64930fef
[ #1477 ] ape: Fix EC chunk test
...
Initially, this test was a check that only the container node can
assemble an EC object. But the implementation of this test was wrong.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-07 16:00:20 +03:00
