Commit Graph

198 Commits (b62a2a0f546019e727493dbacc798cb9a177ba9d)

Author SHA1 Message Date
Leonard Lyubich 718a2fad26 [#425] services: Define service interfaces that was removed from API lib
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-17 15:00:37 +03:00
Alex Vanin 892b4f4f12 [#426] service/object: Fix put duration
At object.Put operation node does not transfer any
data. It is done in the stream inside, so we should
register duration from `object.Put` invocation until
`CloseAndRecv` on the stream.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-17 10:58:00 +03:00
Alex Vanin b8e10571c6 [#426] Put prometheus behind pkg/metrics
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-17 10:58:00 +03:00
Alex Vanin bc6e7a823b [#426] service/object: Add object size metrics
We can't rely on object size in the header because it
might be omitted on initial put or it can be 0xFF.. on
streaming data.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-17 10:58:00 +03:00
Alex Vanin 0b93e8a029 [#426] service/object: Add request duration metrics
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-17 10:58:00 +03:00
Alex Vanin f8ab7a9c60 [#426] service/object: Add object service metrics collector
Collect number of request per method in object service.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-03-17 10:58:00 +03:00
Evgenii Stratonikov 6679d59e89 [#422] pkg/services: Provide client options on cache creation
Because options are not used when client is already in cache
providing them to shared cache is misleading at best.
In the worst case `dial_timeout` is set randomly (because of race
condition) which can lead to one service having `dial_timeout` of
another. Thus we set default client creation options when cache is
created.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-03-15 14:07:53 +03:00
Evgenii Stratonikov cc7287d6f7 [#422] pkg/services: Cache clients by address only
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-03-15 14:07:53 +03:00
Leonard Lyubich ed808c3f1b [#419] eacl: Replace eACL storage implementation to app-side
Replace `eacl.Storage` interface implementation from eACL lib to neofs-node
app package. Remove `eacl.WithMorphClient` option in order to abstract eACL
validator from eACL storage implementation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-03-04 16:33:29 +03:00
Leonard Lyubich b8d1144839 [#378] object/delete: Set expiration epoch of the created tombstones
Make object delete service to use network information to calculate and set
expiration of the created tombstone.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-19 11:56:32 +03:00
Leonard Lyubich 38727c2930 [#368] object: Reject expired objects
The lifetime of an object can be limited by specifying a correspondin
well-known attribute. Node should refuse to save expired objects.

Checking objects in FormatValidator is extended with an expiration attribute
parsing step.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-15 19:04:33 +03:00
Leonard Lyubich f85e5f0238 [#202] object/eacl: Fix algorithm for checking eACL table
Add SignWithRFC6979 option to signature verification function since eACL
table are signed by users with this option.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-03 15:14:59 +03:00
Leonard Lyubich ad348afcd6 [#202] object/eacl: Verify signature of eACL table
Since the contract started returning the table signature, it became
necessary to check its correctness.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-02-03 15:14:59 +03:00
Alex Vanin e252f93dbf [#350] object/acl: Make best effort to classify request
Classifier looks at list of inner ring nodes and container
nodes from current and previous epoch to classify request.
Sometimes these checks might return error.

Consider there is a request from unknown key and container's
placement policy valid for current epoch and invalid for past
epoch. Classifier tries to find if key belongs to container
node from current epoch -- it is not. Then it tries to find if
key belongs to container node from past epoch and it throws
error, because placement policy is invalid for past epoch.

This is a legit case and classifier should ignore such errors
to provide best effort in matching. The only error classifier
should return is an error when request does not contain
public key to classify it.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-27 20:18:33 +03:00
Leonard Lyubich f311be3ab6 [#234] Fix ineffassign linter remark
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00
Leonard Lyubich 1e170c3812 [#234] services/object: Support netmap epoch and lookup dead in read ops
Support processing of NetmapEpoch and NetmapLookupDepth X-headers when
processing object read operations. Placement for operations
Get/Head/GetRange/GetRangeHash/Search is built for the epoch specified in
NetmapEpoch X-header (by default latest). Also the specified operations are
processed until success is achieved for network maps from the past up to
NetmapLookupDepth value. Behavior for default values (zero or missing) left
unchanged.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-12 19:19:11 +03:00
Alex Vanin e88c1f750d [#302] Fix golint else/return linter errors
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:47:02 +03:00
Alex Vanin 74a68af5c2 [#302] Fix golint underscore linter errors
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:47:02 +03:00
Alex Vanin 04c0c1b8f5 [#302] Fix errcheck linter errors
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:47:02 +03:00
Alex Vanin 5e231c515a [#302] Fix whitespace linter errors
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:47:02 +03:00
Leonard Lyubich 3a7f6701d8 [#290] object/rangehash: Apply salt to data before hashing
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-11 18:40:59 +03:00
Leonard Lyubich 0e2c761481 [#290] object/rangehash: Add salt to RangeHashPrm structure
Add binary salt field to RangeHashPrm struct. Implement field setter. Set
salt from the request in v2 service.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-11 18:40:59 +03:00
Leonard Lyubich 9a86fff7e0 [#291] Remove some unused code from repository
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Leonard Lyubich a51211eda7 [#291] Remove unused code from pkg/services/object/head
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Leonard Lyubich 3260e9263e [#291] Remove unused pkg/services/object/rangehash packages
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Leonard Lyubich d299d94049 [#291] Remove unused pkg/services/object/search/query packages
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Leonard Lyubich 1dd1762163 [#285] object/put: Fix the loss of X-headers when forwarding objects
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-29 12:22:05 +03:00
Leonard Lyubich 2897e83fb2 [#285] object/eacl: Validate X-headers from the requests, not the responses
In previous implementation of eACL service v2 the response X-headers were
validated at the stage of re-checking eACL. This provoked a mismatch of
records in the eACL table with requests. Fix this behavior by checking the
headers from the request, not the response.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-29 12:22:05 +03:00
Leonard Lyubich c69f867af1 [#285] services/object: Add X-Headers to client call options
Forward request X-headers to client calls during internal processing of
Object operations on the node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-29 12:22:05 +03:00
Alex Vanin 85773c419b [#271] service/object: Fill all fields of new short header structure
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin af7ae0ec4c [#271] acl: Add permission to get objects for inner ring
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich 6245e58d1f [#80] services/object: Implement iterators over split-tree leaves
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich bbe8f34ffd [#253] object/eacl: Process only CID/OID object headers in RANGEHASH/DELETE
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:31:20 +03:00
Leonard Lyubich 9e27263df3 [#253] object/put: Fix loss of session token
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:31:20 +03:00
Leonard Lyubich 8e661d5c37 [#264] object/get: Prevent receiving payloads through GETRANGE when GET
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
Leonard Lyubich 66f9532857 [#264] object/get: Check parent address in linking/last child
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
Leonard Lyubich 49131f1bc7 [#247] object/eacl: Use address from session token in request validation
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00
Leonard Lyubich dba88c79b4 [#247] object/range: Process only object_id and container_id headers
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00
Leonard Lyubich 8654458b19 [#247] object/eacl: Use object ID from session token context
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00
Leonard Lyubich 168dcbdccd [#247] object/eacl: Process object address from request
In previous implementation eACL validator didn't take into account container
and object ID fields of request bodies.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-15 13:21:13 +03:00
Leonard Lyubich fb50362dcc [#243] services/object: Share common parameters across services
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin a01262d8bd [#243] service/object: Fix object chain assembly for tombstone body
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich 9265e31e65 [#243] services/object: Fix lost tokens when generating requests
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich fe3906c295 [#243] object/delete: Implement new service processing
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich 510e9ff2ec [#243] core/object: Replace test content of tombstone with an API structure
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich 627012e4fe [#241] object/search: Set container ID parameter of storage engine call
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich 611a29f682 [#241] object/search: Refactor service processing
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich 3bfb18386b object/get: Replace code with function call
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00
Alex Vanin 9fb7190358 [#242] service/object: Support ContainerID field in local search
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-11 17:19:37 +03:00
Leonard Lyubich 0e1f05ff45 [#239] object/head: Implement new service processing
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-11 17:19:37 +03:00