7ffc82a066
[ #1452 ] container: Add ListStream method
...
* Added new method for listing containers to container service.
It opens stream and sends containers in batches.
* Added TransportSplitter wrapper around ExecutionService to
split container ID list read from contract in parts that are
smaller than grpc max message size. Batch size can be changed
in node configuration file (as in example config file).
* Changed `container list` implementaion in cli: now ListStream
is called by default. Old List is called only if ListStream
is not implemented.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-12-13 16:08:48 +03:00
3821645085
[ #1555 ] engine: Refactor (*StorageEngine).GetLocks
...
Refactored after renaming the method to replace the confusing `locked`
variable with `locks`.
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-12-11 15:06:38 +03:00
72470d6b48
[ #1555 ] local_object_storage: Rename method GetLocked -> GetLocks
...
Renamed to better reflect the method's purpose of returning locks
for the specified object.
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-12-11 15:06:37 +03:00
e9837bbcf9
[ #1554 ] morph/event: Remove unused AlphabetUpdate event
...
Refs TrueCloudLab/frostfs-contract#138 .
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-11 12:01:17 +00:00
b1614a284d
[ #1546 ] morph/event: Export NotificationHandlerInfo fields
...
Hiding them achieves nothing, as the struct has no methods and is not
used concurrently.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-11 07:39:49 +00:00
d0ce835fbf
[ #1546 ] morph/event: Merge notification parser and handlers
...
They are decoupled, but it is an error to have a handler without a
corresponding parser. Register them together on the code level and get
rid of unreachable code.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-11 07:39:49 +00:00
dfa51048a8
[ #1546 ] morph/event: Remove "is started" checks from event handler registrar
...
This codepath hides possible bugs in code.
All initialization function should run before init stage.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-11 07:39:49 +00:00
670305a721
[ #1546 ] morph/event: Remove nil checks from event handler registrar
...
This codepath hides possible bugs in code.
We would rather panic then silently fail.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-11 07:39:49 +00:00
1f6cf57e30
[ #1548 ] metabase: Check if EC parent is removed or expired
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-12-11 07:26:33 +00:00
386a12eea4
[ #1548 ] engine: Rename parent -> ecParent
...
Parent could mean split parent or EC parent. In this case it is EC parent only.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-12-11 07:26:33 +00:00
15139d80c9
[ #1548 ] policer: Do not replicate EC chunk if object already removed
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-12-11 07:26:33 +00:00
41da27dad5
[ #1549 ] engine: Drop Async flag from evacuation parameters
...
Now it is only async evacuation.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-12-10 17:00:00 +03:00
ac0511d214
[ #1549 ] controlSvc: Drop deprecated EvacuateShard rpc
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-12-10 16:59:52 +03:00
7e542906ef
[ #1539 ] go.mod: Bump frostfs-sdk-go version
...
* Also fix placement unit-test in object manager
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-12-06 15:29:37 +03:00
d1bc4351c3
[ #1545 ] morph/event: Simplify frostfs contract event parsing
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-06 14:00:23 +03:00
1c12f23b84
[ #1541 ] morph/event: Simplify netmap contract event parsing
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-06 10:26:39 +00:00
a353d45742
[ #1541 ] morph/event: Simplify container contract event parsing
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-06 10:26:39 +00:00
d5d5ce2074
[ #1541 ] morph/event: Simplify balance contract event parsing
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-06 10:26:39 +00:00
7df3520d48
[ #1540 ] getSvc: Drop redundant returns
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-12-05 12:39:49 +00:00
5fe78e51d1
[ #1540 ] getSvc: Do not log context canceled errors during EC assemble
...
Those errors are fired when it is enough chunks retrieved and error group
cancels other requests.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-12-05 12:39:49 +00:00
84b4051b4d
[ #1538 ] morph/container: Make opts struct similar to that of other contracts
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-04 15:30:58 +03:00
6a51086030
[ #1538 ] morph/client: Remove TryNotary() option from side-chain contracts
...
The notary is always enabled and this option does always work.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-04 15:30:58 +03:00
2d5d4093be
[ #1537 ] morph: Use (user.ID).ScriptHash() where possible
...
Pick up changes from TrueCloudLab/frostfs-sdk-go#198 .
gopatch:
```
@@
var user expression
@@
-address.StringToUint160(user.EncodeToString())
+user.ScriptHash()
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-04 13:25:44 +03:00
e3487d5af5
[ #1535 ] morph: Unify test invoke error messages
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-04 09:50:20 +00:00
e37dcdf88b
[ #1535 ] morph/netmap: Unify error messages for config retrieval
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-04 09:50:20 +00:00
6c679d1535
[ #1535 ] morph: Unify client creation error messages
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-04 09:50:20 +00:00
281d65435e
[ #1450 ] engine: Group object by shard before Inhume
...
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/engine
cpu: 12th Gen Intel(R) Core(TM) i5-1235U
│ old.txt │ new.txt │
│ sec/op │ sec/op vs base │
InhumeMultipart/objects=1-12 11.42m ± 1% 10.71m ± 0% -6.27% (p=0.000 n=10)
InhumeMultipart/objects=10-12 113.5m ± 0% 100.9m ± 3% -11.08% (p=0.000 n=10)
InhumeMultipart/objects=100-12 1135.4m ± 1% 681.3m ± 2% -40.00% (p=0.000 n=10)
InhumeMultipart/objects=1000-12 11.358 ± 0% 1.089 ± 1% -90.41% (p=0.000 n=10)
InhumeMultipart/objects=10000-12 113.251 ± 0% 1.645 ± 1% -98.55% (p=0.000 n=10)
geomean 1.136 265.5m -76.63%
```
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-12-04 10:09:00 +03:00
b348b20289
[ #1450 ] engine: Add benchmark for Inhume operation
...
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-12-04 10:08:34 +03:00
748edd1999
[ #1450 ] engine: Return shard-level error if object is expired on inhume
...
Since we have errors defined on the shard-level, it looks strage that we
check an error againt the shard-level error `ErrLockObjectRemoval`, but
then return the metabase-level error. Let's return the same shard-level
error instead.
Since we have errors defined on the shard-level
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-12-04 10:06:57 +03:00
432042c534
[ #1527 ] engine: Add tests for handling expired objects on inhume and lock
...
Currently, it's allowed to inhume or lock an expired object.
Consider the following scenario:
1) An user inhumes or locks an object
2) The object expires
3) GC hasn't yet deleted the object
4) The node loses the associated tombstone or lock
5) Another node replicates tombstone or lock to the first node
In this case, the second node succeeds, which is the desired behavior.
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-12-03 12:29:45 +03:00
9cabca9dfe
[ #1527 ] engine/test: Move default metabase options to separate function
...
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-12-02 16:13:37 +03:00
60feed3b5f
[ #1527 ] engine/test: Allow to specify current epoch in epochState
...
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-12-02 15:37:25 +03:00
edfa3f4825
[ #1528 ] node: Keep order for equal elements when sort priority metrics
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-12-02 13:58:19 +03:00
e0ac3a583f
[ #1523 ] metabase: Remove (*DB).IterateCoveredByTombstones
...
Remove this method because it isn't used anywhere since 7799f8e4c
2024-11-29 10:49:24 +00:00
00c608c05e
[ #1524 ] tree: Make check APE error get wrapped to api status
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-29 10:48:16 +00:00
bba1892fa1
[ #1524 ] ape: Make APE checker return error without status
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-29 10:48:16 +00:00
01acec708f
[ #1525 ] pilorama: Use AppendUint* helpers from stdlib
...
gopatch:
```
@@
var slice, e expression
@@
+import "encoding/binary"
-append(slice, byte(e), byte(e >> 8))
+binary.LittleEndian.AppendUint16(slice, e)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-28 09:40:20 +03:00
f12f04199e
[ #1516 ] traverser: Check for placement vector out of range
...
Placement vector may contain fewer nodes count than it required by policy
due to the outage of the one of the node.
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-11-21 14:18:55 +03:00
49a4e727fd
[ #1507 ] timer/test: Use const for constants
...
Make it easy to see what the test is about.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-20 08:36:25 +00:00
2e974f734c
[ #1507 ] timer/test: Improve test coverage
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-20 08:36:25 +00:00
3042490340
[ #1507 ] timer: Remove unused OnDelta() method
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-20 08:36:25 +00:00
ffe9906266
[ #1501 ] cli: Move APE-chain parser methods to pkg/util
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-20 07:58:32 +00:00
e2cb0640f1
[ #1501 ] util: Move eACL-to-APE converter to pkg/util
...
* `ConvertEACLToAPE` is useful method which couldn't be imported
out of frostfs-node so far as it has been in `internal`
* Since `ConvertEACLToAPE` and related structures and unit-tests
are placed in `pkg/util`
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-20 07:58:32 +00:00
d82f0d1926
[ #1496 ] node/control: Await until SetNetmapStatus() persists
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-15 16:36:07 +03:00
acd5babd86
[ #1496 ] morph: Merge InvokeRes and WaitParams
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-15 16:36:07 +03:00
b65874d1c3
[ #1496 ] morph: Return InvokeRes from all invoke*() methods
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-15 16:36:07 +03:00
69c63006da
[ #1496 ] morph: Move tx waiter to morph package
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-15 16:36:07 +03:00
d77a218f7c
[ #1493 ] metabase: Merge Inhume() and DropGraves() for tombstones
...
DropGraves() is only used to drop gravemarks after a tombstone
removal. Thus, it makes sense to do Inhume() and DropGraves() in one
transaction. It has less overhead and no unexpected problems in case
of sudden power failure.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-14 06:47:04 +00:00
44df67492f
[ #1493 ] metabase: Split inhumeTx() into 2 functions
...
No functional changes.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-14 06:47:04 +00:00
1e6f132b4e
[ #1493 ] metabase: Pass InhumePrm by value
...
Unify with the other code, no functional changes.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-14 06:47:04 +00:00
