Commit graph

1515 commits

Author SHA1 Message Date
Evgenii Stratonikov
692790a899 [#1115] services/object: reuse errors from core service
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-11 12:58:59 +03:00
Evgenii Stratonikov
4caa330ddc [#1115] morph/client: reuse errors from container contract
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-11 12:58:59 +03:00
Evgenii Stratonikov
f9d3111825 [#1115] blobovnicza: remove GetRange method
It is unused and incorrect as described in FIXME.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-11 12:58:59 +03:00
Evgenii Stratonikov
c1530dec5e [#1115] shard/gc: provide parsed addresses in HandleExpiredTombstones
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-11 12:58:59 +03:00
Evgenii Stratonikov
0695ec4125 [#1133] services/session: remove expired tokens every epoch
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-08 14:58:32 +03:00
Pavel Karpy
1667ec9e6d [#1131] *: Adopt SDK changes
`object.Address` has been moved to `object/address`
`object.ID` has been moved to `object/id`

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-02-08 09:45:38 +03:00
Evgenii Stratonikov
1c821d6c36 [#625] morph/client: remove unused interface
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-08 09:43:54 +03:00
Evgenii Stratonikov
c34cfa1f35 [#625] client/container: remove intermediate wrapper
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-08 09:43:54 +03:00
Evgenii Stratonikov
819d80a7a9 [#625] client/audit: remove intermediate wrapper
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-08 09:43:54 +03:00
Evgenii Stratonikov
767ee5c0cd [#625] client/reputation: remove intermediate wrapper
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-08 09:43:54 +03:00
Evgenii Stratonikov
97d18bc515 [#625] client/netmap: remove intermediate wrapper
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-08 09:43:54 +03:00
Evgenii Stratonikov
8c5c3ac9e8 [#625] client/neofsid: remove intermediate wrapper
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-08 09:43:54 +03:00
Evgenii Stratonikov
8474abb911 [#625] client/neofs: remove intermediate wrapper
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-08 09:43:54 +03:00
Evgenii Stratonikov
6f50fefbea [#625] client/balance: remove intermediate wrapper
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-08 09:43:54 +03:00
Evgenii Stratonikov
ed50cf6207 [#1118] services/control: return error counter in ListShards
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-03 15:14:27 +03:00
Evgenii Stratonikov
7fb15fa1d0 [#1118] services/control: allow to reset error counter in SetShardMode
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-03 15:14:27 +03:00
Evgenii Stratonikov
6ad2624552 [#1118] engine: allow to set error threshold
There are certain errors which are not expected during usual node
operation and which tell us that something is wrong with the shard.
To prevent possible data corruption, move shard in read-only mode after
amount of errors exceeded some threshold. By default no actions are performed.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-03 15:14:27 +03:00
Leonard Lyubich
4f3323f084 [#1101] *: Adopt interface changes of API client from SDK library
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-02-03 14:00:14 +03:00
Evgenii Stratonikov
674f520da7 [#1132] *: Use path/filepath package when working with files
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-03 10:12:07 +03:00
Evgenii Stratonikov
0decb95591 [#625] morph/client: make all constructors return no error
In some places we panic, in some we return error, in some (audit) just return a client.
However in all of the places static client is created immediately before
the sugared-client creation.

This commit makes all constructors to just return a client for the sake
of code simplification and unification.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-01 15:47:54 +03:00
Evgenii Stratonikov
3c5b62d839 [#625] morph/client: make method names constant
We don't use custom names and the only place where custom method option
is used it provides the default name and can be omitted.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-01 15:47:54 +03:00
Evgenii Stratonikov
476528361e [#1023] *: fix linter errors
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-31 13:01:58 +03:00
Evgenii Stratonikov
eb99b65134 [#1023] morph/event: remove notaryRequest from subnet.Delete
Subnet removal is done by owner.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-31 13:01:58 +03:00
Evgenii Stratonikov
f96bb8d3b8 [#1023] innerring/subnet: fix subnet removal event processing
Caught by `unused` linter.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-31 13:01:58 +03:00
Evgenii Stratonikov
a82ac0edcd [#1023] services/object: remove TTL from Get parameters
It was unused an should be provided as a `CallOption` instead.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-31 13:01:58 +03:00
Evgenii Stratonikov
5a03a14940 [#1094] metabase: move Select filters sanity check outside of transaction
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-31 12:36:40 +03:00
Evgenii Stratonikov
651adf46c6 [#1094] blobovnicza: calculate size in Init properly
If pre-existing blobovnicza is initialized, it's size should be updated
even if all buckets are in place.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-31 12:36:40 +03:00
Evgenii Stratonikov
418b553920 [#1094] shard: remove TODO for shard mode description
It was added back in 2fb379b7 when we had many shard modes. Now we have
only two and comments for constants are rather descriptive.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-31 12:36:40 +03:00
Evgenii Stratonikov
ed7c732676 [#1094] shard: unify collection of expired objects
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-31 12:36:40 +03:00
Evgenii Stratonikov
fbb95cff14 [#1094] services/object: remove some FIXME comments
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-31 12:36:40 +03:00
Evgenii Stratonikov
1bdbd6ed22 [#1094] eacl/v2: preallocate slice in headersFromObject
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-31 12:36:40 +03:00
Evgenii Stratonikov
18cfd8b042 [#1086] services/control: implement RestoreShard RPC
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-31 12:36:14 +03:00
Evgenii Stratonikov
2ec4a3c897 [#1086] neofs-cli: add command for shard dump
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-31 12:36:14 +03:00
Evgenii Stratonikov
0e60b1d6c9 [#1086] services/control: implement DumpShard RPC
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-31 12:36:14 +03:00
Evgenii Stratonikov
aa53418119 [#1086] services/control: fix comments
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-31 12:36:14 +03:00
Leonard Lyubich
20b3ff84b3 [#1110] ir/container: Fix check of SetEACL operation
In previous implementation IR incorrectly verified `SetEACL` event of
`Container` contract. The incorrect behavior could be reproduced in two
ways:
  1. Create container using session, and perform `SetEACL` operation
  with a key that is different from the session one.
  2. Create container using session, and perform `SetEACL` w/o a
  session, but sign it using session key from the `Put` operation.

The problem was in the `checkSetEACL` validation method of IR container
processor. It always used session token used for container creation
during session ownership check.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-01-27 16:37:24 +03:00
Evgenii Stratonikov
adb80bebb2 [#1085] shard: rename Evacuate to Dump
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00
Evgenii Stratonikov
d06425c852 [#1085] shard: allow to ignore errors in Evacuate
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00
Evgenii Stratonikov
36eebb5932 [#1085] writecache: allow to ignore errors during iteration
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00
Evgenii Stratonikov
cd75638ce3 [#1085] blobstor: allow to ignore errors during iteration
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00
Evgenii Stratonikov
c954f0e71b [#1085] fstree: allow to ignore errors during iteration
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00
Evgenii Stratonikov
e53ad2f468 [#1085] blobovnicza: allow to ignore errors during iteration
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00
Evgenii Stratonikov
9c60ab893c [#1085] shard: allow to ignore errors during restore
We could also ignore errors during evacuate, but this requires
unmarshaling objects first which slowers the process considerably.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00
Evgenii Stratonikov
0ef3d5ab03 [#1085] writecache: persist in-memory objects before shutdown
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00
Evgenii Stratonikov
3215210d0c [#1085] shard: Close components in reverse order
Write-cache should be closed before blobstor and metabase.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00
Evgenii Stratonikov
b432ec3a03 [#1085] shard/test: add object integrity check for Evacuate
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00
Evgenii Stratonikov
c0f6c988f0 [#1085] shard: dump data from write-cache
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00
Evgenii Stratonikov
ad01aaf8bf [#1085] writecache: add read-only mode
In read-only mode modifying operations are immediately returned with
error and all background operations are suspended.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00
Evgenii Stratonikov
9f963e001b [#1085] writecache: set metaOnly flag for receiving from priority channel
This can lead to objects being flushed twice.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00
Evgenii Stratonikov
7eedb23eb7 [#1085] shard: allow to dump/restore objects
Dump contains magic and a list of objects prefixed by object size in bytes.
We can't use proto-marshaled list because this requires having all dump
in memory. Using TAR induces 512 byte overhead for each object which can
be a problem in some cases.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00