898689ec14
[ #1731 ] services/replicator: Unify Task interface with other parameters
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-24 13:47:48 +03:00
4e043a801c
[ #1731 ] services/control: Replicate object over network in EvacuateShard RPC
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-24 13:47:48 +03:00
091d7d30f6
[ #1731 ] services/control: Allow to evacuate data from shard
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-19 11:33:52 +03:00
b064fb24d8
[ #1616 ] engine: Do not use batches in delete
...
Use a simple loop at the callsite. This way we remove as much as we can.
Also, `Delete` metrics is more meaningful now.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-15 10:28:48 +03:00
cda8f9df2e
[ #1786 ] services/control: Remove `WithDeletedObjectHandler` option
...
Use storage engine directly instead. It is already provided in the
options.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-15 10:28:48 +03:00
ae655b74f0
*: Exec `make fmts`
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-09-14 17:52:49 +04:00
4083a62679
[ #1628 ] tree: Pre-allocate errors in ACL checks
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-09-13 10:33:50 +03:00
5adf089c1d
[ #1628 ] tree: Log unacceptable bearer attachment
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-09-13 10:33:50 +03:00
2ffcd02ac3
[ #1628 ] tree: Document ACL checks in tree service
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-09-13 10:33:50 +03:00
8d0906c6ab
[ #1628 ] tree: Skip eACL filters for tree requests
...
Do not call `CalculateAction` for the eACL checks since it requires object
headers that are meaningless in the tree context.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-09-13 10:33:50 +03:00
876e014b5d
[ #1628 ] tree: Make ACL checks the same way as for object requests
...
1. Do not require a request to be signed by the container owner if a
bearer token is missing
2. Do not check the system role since public requests are not expected to
be signed by IR or a container node (unlike the object requests)
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-09-13 10:33:50 +03:00
4f18893d9b
[ #1628 ] node: Move common `EACLSource` interface to `core` pkg
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-09-13 10:33:50 +03:00
4afb928ab6
[ #1674 ] *: Expire entities after the expiration epoch
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-09-12 19:56:59 +04:00
9da5d784cb
[ #1630 ] neofs-node: Remove trees on container removal event
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-12 09:54:15 +03:00
0140ac354b
[ #1759 ] services/tree: Make logs more descriptive
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-07 12:57:33 +04:00
74c861342e
[ #1753 ] services/tree: Add parent ID to the `GetNodeByPath` response
...
Currently, you need to use `GetSubTree` to get parent, which seems an
overkill.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-07 10:52:37 +03:00
8a47f9ba12
[ #1753 ] services/tree: Remove MaxGetSubTreeCount constant
...
It was used only in `GetOpLog` and ignored on the server side.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-07 10:52:37 +03:00
9840936a4f
[ #1753 ] services/tree: Do not restrict depth in `GetSubTree`
...
Previously, the depth was restricted because with BFS the amount of
nodes we have in memory blows up exponentially. With DFS is is linear,
so we can process trees of arbitrary depth.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-07 10:52:37 +03:00
180e5e938f
[ #1735 ] go.mod: Update API and SDK
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-08-30 12:27:46 +03:00
0720d96c9d
[ #1687 ] go.mod: Update neofs-sdk-go
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-22 18:59:57 +03:00
30341f2192
[ #1687 ] *: Perform `go fmt` using go `v1.19`
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-22 18:59:57 +03:00
c2918fce3a
[ #1645 ] node: Support `EACL_NOT_FOUND` status
...
Remove internal `ErrEACLNotFound` error.
Also, update `neofs-api-go` and `neofs-sdk-go` libraries.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-01 20:45:36 +03:00
4558f30575
[ #1644 ] services/tree: Do not use deprecated gRPC options
...
Get rid of `grpc.WithInsecure`.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-07-30 18:39:08 +03:00
10c855efef
[ #1624 ] go.mod: Update dependencies
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-25 16:56:02 +03:00
589a54805d
[ #1618 ] node: Use OID/CID from the request in eACL checks
...
Also, try to fetch object header info from the local storage to find as much
object info as possible for the requests which do not assume returning
object header as a response.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-25 09:41:11 +03:00
7e717e80ef
[ #1623 ] services/tree: Fix error message
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-22 15:13:03 +03:00
4944490ffb
[ #1559 ] local_object_storage: Move shard to the `DegradedReadOnly` mode
...
`Degraded` mode can be set by the administrator if needed.
Modifying operations in this mode can lead node into an inconsistent state
because metabase checks such as lock checking are not performed.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
339864b720
[ #1559 ] local_object_storage: Move `shard.Mode` to a separate package
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
b549cc314c
[ #1607 ] services/tree: allow to customize some parameters
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
f80e52fbea
[ #1334 ] services/tree: Document *.proto files
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
bfdd68dcb3
[ #1333 ] services/control: Allow to synchronize local trees
...
Do not check that a node indeed belongs to the container, because the
synchronization will fail in this case anyway.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
5e843a73f9
[ #1333 ] services/control: Return pilorama info in `ListShards` RPC
...
Do not return backend type from the service for now, because memory
backend is expected to vanish.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
5ffbeb76e6
[ #1456 ] services/tree: Wait some time before reconnecting after failure
...
In case node is down or failing for some reason, we can expect `Dial` to
fail. In case we actively try to replicate and `Dial` always takes 2
seconds, replication-related channels quickly become full. That affects
latency of all other write operations.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
39f47f61c6
[ #1445 ] services/tree: Cache the list of container nodes
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
a2edfec0c3
[ #1446 ] services/tree: Use `grpc.WithInsecure` only for nodes without TLS
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
33d8fb187a
[ #1427 ] services/tree: Parallelize replicator
...
Before this commit the replication channel was quickly filled under
heavy load. This lead to the continuously increasing latency for all
write operations. Now it looks better.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
b04f712773
[ #1446 ] services/tree: Cache connections to the container nodes
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
4437cd7113
[ #1442 ] pilorama: Generate timestamp based on node position in the container
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
3caa982283
[ #1442 ] services/tree: Fix log message for failed `Apply`
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
01d2e06a9b
[ #1401 ] services/tree: Marshal public key once
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
1f5a650b05
[ #1401 ] services/tree: Retransmit queries to container nodes
...
Also fix a bug with replicator using the multiaddress instead of
<host>:<port> format expected by gRPC library.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
f4bc1c601a
[ #1329 ] services/tree: Synchronize from the last stored height
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
536857ea5a
[ #1329 ] services/tree: Implement `GetOpLog` RPC
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
886baf3136
[ #1426 ] services/tree: Do not replicate to a local node
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
ad48918a97
[ #1406 ] pilorama: Return parent from `TreeGetMeta`
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
b30f14978d
[ #1404 ] services/tree: Fix nodeId in GetSubTree
...
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
45321b3f6d
[ #1328 ] services/tree: Fix grpc import path
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
16e3421825
[ #1328 ] services/tree: Implement access control
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
94b9e13431
[ #1328 ] services/tree: Fix proto field numbers
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
aea855e8f3
[ #1326 ] services/tree: Implement GetSubTree RPC
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
Leonard Lyubich
Pavel Karpy
Evgenii Stratonikov
Denis Kirillov