Commit Graph

1695 Commits (dda56f13191c23e2d69b29d98c53c9bd5845ad11)

Author SHA1 Message Date
Pavel Karpy 57c5fccc8c [#1428] node/acl: Make OID optional
Not all the NeoFS requests must contain OID in their bodies (or must NOT
contain them at all). Do not pass object address in helper functions, pass
CID and OID separately instead.
Also, fixed NPE in the ACL service: updated SDK library brought errors
when working with `Put` and `Search` requests without OID fields.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-05-25 12:11:03 +03:00
Pavel Karpy d69eb2aaf3 [#1428] node/acl: Make container ID as required param
Change pointer to value in request information since requests could not
exist without container ID.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-05-25 12:11:03 +03:00
Evgenii Stratonikov c57114def3 [#1422] morph/client: Fetch group key properly
NNS contract `resolve` method returns an array.
`neofs-adm` is already up to date.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-23 19:24:46 +03:00
Leonard Lyubich 86552cf3ae engine: make `corruptSubDir` to ignore `blobovnicza` dir
There is a dir for Blobovnicza tree in BlobStor, so we need to ignore
it.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-23 15:33:14 +03:00
Leonard Lyubich bb25ecbd15 [#1400] owner: Upgrade SDK package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-23 15:33:14 +03:00
Leonard Lyubich f8ac4632f8 [#1335] policer: Prevent potential object loss
In previous implementation `Policer` considered local object copy as
redundant on processing single placement vector.

Make `Policer` to call redundant copy callback after full placement
processing. Also fix 404 error parsing.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-23 15:24:23 +03:00
Pavel Karpy 9f62d25b50 [#1224] ir: Delete nodes that do not have subnets
Also, update their subnet entrance on subnet removal events.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-05-23 08:16:24 +03:00
Pavel Karpy b3e1e90c01 [#1224] ir: Listen to subnet removal events
IR must listen to subnet removal notifications always (regardless of notary
mode).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-05-23 08:16:24 +03:00
Pavel Karpy 71f18ba9ec [#1224] morph: Call Alphabet methods flexibly
Call `UpdateStateIR` and `AddPeerIR` method instead of `UpdateState` and
`AddPeer` if calling client is configured as Alphabet in notary enabled
environment.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-05-23 08:16:24 +03:00
Pavel Karpy a129fc98da [#1224] morph: Add configuration getters
Includes `WithNotary` and `IsAlpha` methods to the `StaticClient`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-05-23 08:16:24 +03:00
Pavel Karpy f0c9a68ad3 [#1224] morph: Delete wrong comment
Current implementation does not use MurmurHash3.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-05-23 08:16:24 +03:00
Evgenii Stratonikov df48ddcb3d Revert "[#1141] morph/client: Allow to use more integer types as arguments"
This reverts commit 9349f422fd.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-20 16:45:15 +03:00
Evgenii Stratonikov 4761857fb3 [#1396] cmd: Allow to autogenerate documentation for all commands
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-20 13:35:22 +03:00
Pavel Karpy 4be5dce848 [#1405] ir: Fix settlement log message
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-05-20 12:40:12 +03:00
Evgenii Stratonikov 3f2889a4e9 [#1392] eacl/v2: Use `EncodeToString` where possible
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-20 12:04:10 +03:00
Evgenii Stratonikov 63cdc16088 [#1386] eacl/v2: Do not allocate intermediate slices for XHeaders
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-20 12:04:10 +03:00
Evgenii Stratonikov 915f87959e [#1386] eacl/v2: Allocate []eaclSDK.Header slice immediately
There is no point in having intermediate `[]sessionSDK.XHeader`.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-20 12:04:10 +03:00
Evgenii Stratonikov b24589b62d [#1386] eacl/v2: Use raw structs where possible
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-20 12:04:10 +03:00
Evgenii Stratonikov 3f4475f97b [#1386] services/object: Fail eACL check if the request is invalid
Parse all headers beforehand and reject invalid requests.
Another approach would be to remember the error and check
it after `CalculateAction`, which is a bit faster.
The rule of thumb here is "first validate, then use".

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-20 12:04:10 +03:00
Pavel Karpy f99a0498da [#1403] util: Add details to errors
Make error messages more descriptive when parsing LOCODE from string errors
appear.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-05-20 10:53:27 +03:00
Leonard Lyubich aeb9884218 [#1389] crypto: Upgrade SDK package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-19 10:12:14 +03:00
Leonard Lyubich 5e50ddd7f5 [#1389] storagegroup: Upgrade SDK package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-19 10:12:14 +03:00
Evgenii Stratonikov 0f376a5d83 [#1388] services/control: Extend tests for `ShardInfo` marshaling
It is nice to have different paths for different components and also
check that the information returned is different for different shards.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-17 13:48:07 +03:00
Leonard Lyubich f15e6e888f [#1377] oid, cid: Upgrade SDK package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-16 15:33:22 +03:00
Evgenii Stratonikov f65898a354 [#1141] morph/client: Use boolean type directly
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-16 15:09:35 +03:00
Evgenii Stratonikov 230a5cd037 [#1141] morph/client: Remove unneeded argument conversions
Client supports `util.Uint160` directly, no need to convert it to bytes.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-16 15:09:35 +03:00
Evgenii Stratonikov 1e8391d216 [#1141] morph/client: Allow to use more integer types as arguments
Also, use `*big.Int` as integer value, see nspcc-dev/neo-go#2413.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-16 15:09:35 +03:00
Leonard Lyubich 3a188bb2e5 [#1371] bearer: Upgrade SDK package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-12 15:16:17 +03:00
Leonard Lyubich ae92074272 [#1369] checksum: Upgrade SDK package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-12 09:48:31 +03:00
Leonard Lyubich 088df0e2a9 [#1369] version: Upgrade SDK package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-12 09:48:31 +03:00
Leonard Lyubich 7a57d7b076 [#1369] audit: Upgrade SDK package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-12 09:48:31 +03:00
Elizaveta Chichindaeva cc7a723d77 [#1320] English Check
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
2022-05-11 10:40:02 +03:00
Leonard Lyubich cd545f0160 [#1351] cli: Fix connection scheme loss during endpoint parsing
In previous implementation NeoFS CLI app used `network.Address.HostAddr`
as a server URI, which caused scheme loss since host address doesn't
contain it.

Rename `HostAddr` to `URIAddr` and make it to return URI address with
`grpcs` scheme if TLS is enabled. Make `TLSEnabled` unexported since it
was used to provide default `tls.Config` only (it is used by default in
SDK).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-05 10:43:27 +03:00
Evgenii Stratonikov a4769d8624 *: use `require.ErrorIs` where possible
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-29 16:47:57 +03:00
Evgenii Stratonikov 3c39e6df11 [#1319] blobstor: Compress big objects in a streaming fashion
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-29 16:46:40 +03:00
Evgenii Stratonikov 1219ff89d4 [#1319] services/object_manager: Fix error message
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-29 16:46:40 +03:00
Evgenii Stratonikov 057d53459b [#1319] services/object: Remove `Write` method from `local` and `remote` targets
They work with prepared objects only.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-29 16:46:40 +03:00
Evgenii Stratonikov 4ea03c01b5 [#1319] services/object: Store payload directly in `distributedTarget`
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-04-29 16:46:40 +03:00
Pavel Karpy 9da7df4d42 [#1318] metabase: Init metabase in test
`Init()` creates buckets. It prevents NPE in tests.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy 486ec8a2e5 [#1318] metabase: Reuse comparing func from `require`
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy 1caeb0b648 [#1318] metabase: Add missing error check in tests
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy 6f2723b47e [#1318] node: Fix linter
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy 2aec5736e0 [#1318] node/gc: Fix tombstones handling
Handle only expired tombstones.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy 24ca8ca5c7 [#1318] object: Change default ttl
This is need to reuse object service without passing any API v2 structs.
Also, fix comment.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy 7799f8e4cf [#1318] engine: Change tombstone clear process
- Delete objects physically on tombstone's arrival;
- Store information about tombstones in the Graveyard;
- Clear Graveyard every epoch based on the information about TS in the
network.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy e4cfeec449 [#1318] service: Add tombstone checker service
The service fetches tombstones from the network via object service, every
request is handled in the following order:
1. checks local LRU cache;
2. checks local storage engine;
3. tries to find object in the placement nodes.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy 2583f608e8 [#1318] metabase: Add `DropGraves` method
Method clean up Graveyard and should be called by the routine that handles
tombstone expiration.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy daab30c391 [#1318] metabase: Add iteration offset
Add offset element to the iterations over deleted objects (both the
Graveyard and the Garbage buckets).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy fe8076e60a [#1318] metabase: Add immediate object deletion
Mark objects with GC immediately after a Tombstone is received.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Pavel Karpy 8107c8d1a9 [#1318] metabase: Separate buckets with TS and GC marks
It allows storing information about object in both ways at the same time:
1. Metabase should know if an object is covered by a tombstone (that is
not expired yet);
2. It should be possible to physically delete objects covered by a
tombstone immediately (mark with GC) but keep tombstone knowledge.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00