f0a67f948d
[ #1431 ] pilorama: Cache attributes in the index
...
Currently to find a node by path we iterate over all the children on
each level. This is far from optimal and scales badly with the number of
nodes on a single level. Thus we introduce "indexed attributes" for
which an additional information is stored and which can be use in
`*ByPath` operations. Currently this set only includes `FileName`
attribute but this may change in future.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
f5d35571d0
[ #1431 ] engine: Add benchmark for `Select` vs `TreeGetByPath`
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
f4bc1c601a
[ #1329 ] services/tree: Synchronize from the last stored height
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
835170e452
[ #1329 ] pilorama: Allow to benchmark all tree backends
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
536857ea5a
[ #1329 ] services/tree: Implement `GetOpLog` RPC
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
886baf3136
[ #1426 ] services/tree: Do not replicate to a local node
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
7703dd5d7f
[ #1419 ] pilorama: Create new nodes in path if needed
...
Consider a node `{FileName: "dir", Attribute: "xxx"}`. In case we add
a new node by path `["dir", "file.txt"]`, create a new intermediate node
with a single attribute.
`GetByPath` now also considers only nodes with a single attribute while building a path.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
ad48918a97
[ #1406 ] pilorama: Return parent from `TreeGetMeta`
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
b30f14978d
[ #1404 ] services/tree: Fix nodeId in GetSubTree
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
45321b3f6d
[ #1328 ] services/tree: Fix grpc import path
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
16e3421825
[ #1328 ] services/tree: Implement access control
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
94b9e13431
[ #1328 ] services/tree: Fix proto field numbers
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
36e24f7f78
[ #1342 ] neofs-node: Use the default endpoint for tree service
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
8844d9b2db
[ #1344 ] pilorama: Document errors for Get* methods
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
d8ad68d613
[ #1344 ] engine: Log errors in Tree* operations
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
910db42748
[ #1344 ] pilorama: Use `require.ErrorIs` in tests
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
aea855e8f3
[ #1326 ] services/tree: Implement GetSubTree RPC
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
62154da17c
[ #1324 ] services/tree: Implement Object Tree Service
...
Object Tree Service allows changing trees assotiated with
the container in runtime.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
46f4ce2773
[ #1324 ] engine: Implement Forest interface for storage engine
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
8cf71b7f1c
[ #1324 ] local_object_storage: Implement tree service backend
...
In this commit we implement algorithm for CRDT trees from
https://martin.klepmann.com/papers/move-op.pdf
Each tree is identified by the ID of a container it belongs to
and the tree name itself. Essentially, it is a sequence of operations
which should be applied in chronological order to get a usual tree
representation.
There are 2 backends for now: bbolt database and in-memory.
In-memory backend is here for debugging and will eventually act
as a memory-cache for the on-disk database.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
198beae703
[ #1613 ] neofs-cli: Print lock object ID
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-19 17:15:15 +03:00
5c8d725447
[ #1610 ] neofs-adm: Fix generated storage config
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-19 16:19:41 +03:00
11c9df6f00
[ #1611 ] shard: Print shard ID in component logs
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-19 15:57:53 +03:00
9f7a22e2aa
[ #1561 ] object: Return `OUT_OF_RANGE` status
...
Replace `ErrRangeOutOfBounds` error from `pkg/core/object` package with
`ObjectOutOfRange` from `apistatus` package. That error is returned by
storage node's server as NeoFS API statuses.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-19 13:29:45 +03:00
ccba07fe19
[ #1608 ] neofs-adm: Register candidates in a single transaction
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-19 13:28:17 +03:00
59d08b8dae
[ #1608 ] neofs-adm: Print info about already performed steps
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-19 13:28:17 +03:00
5e1636e50c
[ #1604 ] docs: Describe storage node configuration
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-19 11:12:52 +03:00
eeef83a74c
[ #1604 ] config: Change yaml example a bit
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-19 11:12:52 +03:00
e5c0338f8c
[ #1602 ] neofs-ir: Fix defaults for pprof and prometheus
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-18 19:41:22 +03:00
3a0b06d5da
[ #1602 ] config: Rename `metrics` and `profiler` sections
...
Depracate old names and remove them in the next release.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-18 19:41:22 +03:00
ac46d1a11f
[ #1602 ] config: Enable `metrics` and `profiler` services with a flag
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-18 19:41:22 +03:00
12dc5c3395
[ #1563 ] config: Replace `127.0.0.1` with `localhost`
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-18 19:41:22 +03:00
4a8dfd0ce4
[ #1606 ] Remove deprecated functions
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-18 12:41:06 +03:00
6cc180391e
[ #1606 ] go.mod: Update neo-go
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-18 12:41:06 +03:00
cb0bb7207c
[ #1461 ] shard: Add a separate `ErrLockObjectRemoval`
...
Do not return `meta.ErrLockObjectRemoval` from shard's methods, add shard's
own error for that.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
558cc1193a
[ #1461 ] engine: Clarify force removal
...
Document force removal behaviour in all the Storage engine parts.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
b1e54dec9f
[ #1461 ] cli: Add context to `GetCurrentEpoch` helper
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
51afcc1182
[ #1461 ] engine, policer: Force remove objects w/o container
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
18ec5d7c8e
[ #1461 ] meta: Return error on lock object removal
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
b5c56d459a
[ #1461 ] engine: Add force lock removal tests
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
63c00e785d
[ #1461 ] shard: Fix option naming
...
`WitDeletedLockCallback` => `WithDeletedLockCallback`.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
5122be34e7
[ #1461 ] shard: Add lock tests
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
eaf96bccf7
[ #1461 ] meta: Add lock tests
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
c6cf8e5c0b
[ #1461 ] engine: Fix LOCK test
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
9c5ef3bab8
[ #1461 ] node: Allow force LOCK removal
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
25826bd96d
[ #1461 ] node: Do not accept lock object w/o expiration
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
de31bb27b6
[ #1461 ] cli: Require LOCK object to be expired
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
595fc60d72
[ #1461 ] cli: Reorganize common functions
...
Make `ParseEpoch` and `GetCurrentEpoch` funcs public and move the first one
to the `common` package.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
fed9e6679d
[ #1461 ] node: Unlock locked object on its lock removal
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
72c044e2eb
[ #1599 ] engine: Parallelize shard initialization
...
Shard is intended to be used as a separate failure domain,
which usually resides on a separate disk. Thus, sequential
initialization is bound by IO and this change speeds up thing a bit.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-15 13:44:54 +03:00
Evgenii Stratonikov
Denis Kirillov
Pavel Karpy