Commit graph

4593 commits

Author SHA1 Message Date
Alex Vanin
afb83c610c [#273] service/audit: Add cache of head responses in PoR
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
9212864f42 [#258] services/audit: Implement PoP check
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
a5320408a5 [#271] service/audit: Implement PoR
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
5f65ec0265 [#271] audit/report: Add PoR related setters
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
85773c419b [#271] service/object: Fill all fields of new short header structure
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
af7ae0ec4c [#271] acl: Add permission to get objects for inner ring
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
782bcadd92 [#271] innerring: Implement GetSG and Head of audit communicator
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
dd5e67ff0c [#271] service/audit: Add netmap structure to audit task
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
64b022a5ed [#271] placement: Add FlattenNodes function
This function converts 2-dimension array of container nodes
into single dimension array. Useful when we need to iterate
over whole container.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
58033a3adc [#271] placement: Add BuildObjectPlacement function
This function allows to shuffle prepared container nodes
according to HRW weights from network map.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
a8c26378b1 [#271] pkg/network: Add multiaddr to ipaddr converter
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
2ebcbe70fb [#255] ir/audit: Make task manager to return number of skipped tasks
Add numeric return from TaskManager.Reset method that shows the number of
canceled tasks. This values will be used for assessment of the progress of
the audit.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
03e3afb0e8 [#255] services/audit: Define interface of container communicator
Define interface of the container communicator which methods are going to be
used in audit checks. Make innerring Server to implement this interface.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
ecb5227c27 [#255] cmd/ir: Set default values of audit configuration
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
754cd8b579 [#255] ir: Replace Server.WriteReport method to another source file
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
54523d2949 [#255] ir: Replace audit client creation to invoke package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
76d4e53ea0 [#255] services/audit: Skip all tasks from previous epoch in audit processor
Implement Reset method on audit task manager that cleans task queue.
Extended TaskManager interface with Reset method on IR side. Call Reset
method in audit processor before new audit start.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
33804e024d [#255] services/audit: Complete audit report after the last check
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
4dc09b19f3 [#255] ir: Make audit processor to push tasks to audit task manager
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
580c9c974a [#255] services/audit: Implement task manager
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
0f0be2377b [#255] services/audit: Implement audit executor without checks
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
babfbc18f2 [#255] services/audit: Implement Task structure
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
076f201807 [#255] services/audit: Define Report structure and Reporter interface
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
919f4364f1 [#269] morph/audit: Implement wrapper over Audit contract client
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
07da9d31f2 [#269] morph/client: Implement Audit contract client
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
8dd7c689f2 [#265] innerring: Select storage groups to audit
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
22cffbf529 [#265] innerring: Add client cache
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
3d3d058b05 [#265] innerring: Select containers to audit
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
87e1252065 [#265] innerring: Generate new audit event
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
8c4e033db3 [#80] core/object: Add SG-object format validation
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
f7ca4a8dce [#80] cmd/cli: Implement storagegroup commands
Implement PUT/GET/LIST/DELETE commands of storage group module.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
43ee94a572 [#80] storageroup: Implement function to collect structure from container
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
44466f4a29 [#80] storagegroup: Implement search query generator
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
6245e58d1f [#80] services/object: Implement iterators over split-tree leaves
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
bb85ac24eb [#254] innerring: Increase extra fee for container registration
Some rough estimation until there will be p2p signature collection
inside the chain.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
feec559e47 [#254] Update neo-go to preview4 compatible version
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
c539657567 Update changelog for v0.13.2
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 15:34:20 +03:00
Leonard Lyubich
0302dc7490 Pull neofs-api-go v1.21.2
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 15:34:20 +03:00
Leonard Lyubich
51b85b0a73 [#278] cli: Support request X-Headers
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 15:03:02 +03:00
Leonard Lyubich
e53bf574b5 [#279] container: Use new methods to work with container format
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 13:38:20 +03:00
Leonard Lyubich
9680dfbdea Update changelog for v0.13.1
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:58:38 +03:00
Leonard Lyubich
b56cfa4318 Pull neofs-api-go release v1.21.1
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:58:38 +03:00
Leonard Lyubich
bbe8f34ffd [#253] object/eacl: Process only CID/OID object headers in RANGEHASH/DELETE
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:31:20 +03:00
Leonard Lyubich
9e27263df3 [#253] object/put: Fix loss of session token
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:31:20 +03:00
Leonard Lyubich
8e661d5c37 [#264] object/get: Prevent receiving payloads through GETRANGE when GET
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
Leonard Lyubich
66f9532857 [#264] object/get: Check parent address in linking/last child
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
Leonard Lyubich
ec21040542 [#264] transformer: Finalize parent header once
In previous implementation parent object header finalized twice in size
limiter + formatter. On the one hand, this added redundant action, on the
other hand, it could provoke a difference in the headers of the linking and
the last part. Change formatter to finalize parent header if it does not
container the signature. Change size limiter to reuse parent header after
last child finalization in linking child.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
Leonard Lyubich
edef9463d7 [#253] metabase: Add prefix to Graveyard and ToMoveIt bucket names
In previous implementation DB.Containers method could return an error about
invalid container ID string format. This could happen if some of top-level
buckets had name w/o "_" substring.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-17 12:03:18 +03:00
Leonard Lyubich
df3d30d1ec Update changelog for v0.13.0
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:48:18 +03:00
Leonard Lyubich
49131f1bc7 [#247] object/eacl: Use address from session token in request validation
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00