Leonard Lyubich
6c9dab481c
[ #311 ] control: Cover StableMarshal methods with unit tests
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
5f22e7fac7
Add newline at end of Control service proto file
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
36d8e2d3b7
[ #311 ] control: Implement JSON marshaler on Netmap message
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
dde24bc9e5
[ #311 ] control: Rewrite methods for binary encoding and signatures
...
Rewrite marshalers and sizers using proto library from API repo.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
e9d4b2d25b
[ #311 ] Recompile protobuf files
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
3e882d963f
[ #311 ] services/control: Implement NetmapSnapshot method
...
Implement methods required for signatures. Receive network map from netmap
storage, convert it to Control service message and return in response.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
28777911fb
[ #311 ] services/control: Define NetmapSnapshot rpc
...
Add NetmapSnapshot rpc to ControlService protobuf definition. Recompile
proto files. Add required method to server structure.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Leonard Lyubich
88023f3655
[ #311 ] services/control: Define Netmap structure
...
Define NodeInfo protobuf type. Define Netmap protobuf type.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-15 16:11:50 +03:00
Alex Vanin
62de9f327e
[ #319 ] Do not return payload formatting error on regular objects
...
Regular objects by definition have valid payload, so there should
be no errors.
Related to #302
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-15 16:01:29 +03:00
Leonard Lyubich
f311be3ab6
[ #234 ] Fix ineffassign linter remark
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00
Leonard Lyubich
0a0ee89665
[ #306 ] control: Rename WithAllowedKeys function to WithAuthorizedKeys
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00
Leonard Lyubich
abd9952e46
[ #306 ] Rename Private service to Control service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00
Leonard Lyubich
44a0fb5a69
[ #306 ] private: Define and use HealthChecker interface to get the status
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00
Leonard Lyubich
c1b8a4815f
[ #306 ] private: Implement server of gRPC private node service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00
Leonard Lyubich
e75ddb0549
[ #306 ] private: Implement setters on generated proto messages
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00
Leonard Lyubich
85ec633938
[ #306 ] Define and compile proto files for private node service
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-14 12:13:49 +03:00
Alex Vanin
842fb9248c
[ #304 ] cmd/neofs-node: Tidy default configuration and log output
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 19:20:27 +03:00
Leonard Lyubich
1e170c3812
[ #234 ] services/object: Support netmap epoch and lookup dead in read ops
...
Support processing of NetmapEpoch and NetmapLookupDepth X-headers when
processing object read operations. Placement for operations
Get/Head/GetRange/GetRangeHash/Search is built for the epoch specified in
NetmapEpoch X-header (by default latest). Also the specified operations are
processed until success is achieved for network maps from the past up to
NetmapLookupDepth value. Behavior for default values (zero or missing) left
unchanged.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-12 19:19:11 +03:00
Leonard Lyubich
2f4d90025f
[ #234 ] core/netmap: Extend Source interface
...
Add GetNetMapByEpoch method. Add Epoch method.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-12 19:19:11 +03:00
Alex Vanin
1d56e60589
[ #302 ] Remove unused FSBucket component
...
FSBucket became obsolete when storage object engine has
been implemented.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:47:02 +03:00
Alex Vanin
b62b19cb5a
[ #302 ] Fix goimports linter errors
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:47:02 +03:00
Alex Vanin
e88c1f750d
[ #302 ] Fix golint else/return linter errors
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:47:02 +03:00
Alex Vanin
74a68af5c2
[ #302 ] Fix golint underscore linter errors
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:47:02 +03:00
Alex Vanin
7e30b0b9c8
[ #302 ] Fix exhaustive linter errors
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:47:02 +03:00
Alex Vanin
04c0c1b8f5
[ #302 ] Fix errcheck linter errors
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:47:02 +03:00
Alex Vanin
5e231c515a
[ #302 ] Fix whitespace linter errors
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-12 18:47:02 +03:00
Leonard Lyubich
9ea75e51b2
[ #303 ] morph/netmap: Implement getting network map snapshot by epoch
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-12 15:48:01 +03:00
Leonard Lyubich
d58e28afee
[ #298 ] morph/netmap: Use netmap.NodeState argument in UpdatePeerState
...
Replace custom NodeState enum with the one from netmap lib. Remove no longer
used NodeState enum.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-11 18:42:09 +03:00
Leonard Lyubich
159351fd55
[ #298 ] morph/netmap: Fix invocation of UpdateState contract method
...
Fix incorrect method name. Fix the order of arguments (should be {state,
key}).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-11 18:42:09 +03:00
Leonard Lyubich
d26716e792
[ #298 ] ir: Fix parser of UpdateState notification event of Netmap contract
...
The order of event stack items is {state, key}.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-11 18:42:09 +03:00
Leonard Lyubich
3a7f6701d8
[ #290 ] object/rangehash: Apply salt to data before hashing
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-11 18:40:59 +03:00
Leonard Lyubich
0e2c761481
[ #290 ] object/rangehash: Add salt to RangeHashPrm structure
...
Add binary salt field to RangeHashPrm struct. Implement field setter. Set
salt from the request in v2 service.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-11 18:40:59 +03:00
Leonard Lyubich
ba03f46316
[ #290 ] util: Implement salting writer
...
Implement wrapper over io.Writer that applies binary salt to written data.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-01-11 18:40:59 +03:00
Alex Vanin
3bbd0bc95c
[ #300 ] innerring: Fix status value in update peer state invocation
...
We can't use enum values from SDK library directly, they can be
different from API specification. Therefore we need to convert them
into protocol level format.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-01-11 16:40:25 +03:00
Leonard Lyubich
9a86fff7e0
[ #291 ] Remove some unused code from repository
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Leonard Lyubich
a51211eda7
[ #291 ] Remove unused code from pkg/services/object/head
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Leonard Lyubich
3260e9263e
[ #291 ] Remove unused pkg/services/object/rangehash packages
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Leonard Lyubich
d299d94049
[ #291 ] Remove unused pkg/services/object/search/query packages
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Leonard Lyubich
0ce124a6e3
[ #291 ] Remove empty pkg/network/peers dir
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Leonard Lyubich
786da5313e
[ #291 ] Remove unused pkg/network/muxer package
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Leonard Lyubich
293af7b78e
[ #291 ] Remove empty pkg/network/bootstrap dir
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-30 10:27:53 +03:00
Leonard Lyubich
1dd1762163
[ #285 ] object/put: Fix the loss of X-headers when forwarding objects
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-29 12:22:05 +03:00
Leonard Lyubich
2897e83fb2
[ #285 ] object/eacl: Validate X-headers from the requests, not the responses
...
In previous implementation of eACL service v2 the response X-headers were
validated at the stage of re-checking eACL. This provoked a mismatch of
records in the eACL table with requests. Fix this behavior by checking the
headers from the request, not the response.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-29 12:22:05 +03:00
Leonard Lyubich
c69f867af1
[ #285 ] services/object: Add X-Headers to client call options
...
Forward request X-headers to client calls during internal processing of
Object operations on the node.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-29 12:22:05 +03:00
Alex Vanin
fe2b1fdc4b
[ #283 ] morph/client: Implement all calls for audit contract
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
7174abcc7c
[ #281 ] service/audit: Fix typos
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
e5108cf135
[ #281 ] service/audit: Run each SG check in separate routing at PoR
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
77cf97c236
[ #281 ] innerring: Create audit context once for one audit round
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
0d7832f5e9
[ #281 ] service/audit: Add separate pool for SG checks in PoR
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
8e72abaab7
[ #281 ] service/audit: Make report structure threadsafe
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
44d9f95137
[ #276 ] service/audit: Count Head requests and retries in PoR
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
9273a89f61
[ #276 ] service/audit: Use random nodes for object.Head in PoR
...
Since PoR audit check uses object.Head with large TTL values to
make real proof of retrievability, we may use random remote
nodes.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
10b548275a
[ #276 ] innerring: Use Head with TTL in PoR
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
f9e81383ae
[ #259 ] innerring: Use blocking pools for audit
...
Audit task manager should not discard tasks if all workers are
busy, therefore pools should not be non-blocking.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
3ec342b2c3
[ #259 ] services/audit: Implement PDP check
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
310a4c3e4d
[ #275 ] innerring: Add storage group search timeout
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
6836975272
[ #275 ] innerring: Use crypto rand shuffle in audit
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
f593f0b2d6
[ #274 ] innerring: Implement GetRangeHash in audit communicator
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
6977adec66
[ #273 ] service/audit: Ignore coverage of very small objects
...
PDP audit check is not quite working with very small objects, so
we try to build coverage with bigger objects.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
afb83c610c
[ #273 ] service/audit: Add cache of head responses in PoR
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
9212864f42
[ #258 ] services/audit: Implement PoP check
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
a5320408a5
[ #271 ] service/audit: Implement PoR
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
5f65ec0265
[ #271 ] audit/report: Add PoR related setters
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
85773c419b
[ #271 ] service/object: Fill all fields of new short header structure
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
af7ae0ec4c
[ #271 ] acl: Add permission to get objects for inner ring
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
782bcadd92
[ #271 ] innerring: Implement GetSG
and Head
of audit communicator
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
dd5e67ff0c
[ #271 ] service/audit: Add netmap structure to audit task
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
64b022a5ed
[ #271 ] placement: Add FlattenNodes function
...
This function converts 2-dimension array of container nodes
into single dimension array. Useful when we need to iterate
over whole container.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
58033a3adc
[ #271 ] placement: Add BuildObjectPlacement function
...
This function allows to shuffle prepared container nodes
according to HRW weights from network map.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
a8c26378b1
[ #271 ] pkg/network: Add multiaddr to ipaddr converter
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
2ebcbe70fb
[ #255 ] ir/audit: Make task manager to return number of skipped tasks
...
Add numeric return from TaskManager.Reset method that shows the number of
canceled tasks. This values will be used for assessment of the progress of
the audit.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
03e3afb0e8
[ #255 ] services/audit: Define interface of container communicator
...
Define interface of the container communicator which methods are going to be
used in audit checks. Make innerring Server to implement this interface.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
754cd8b579
[ #255 ] ir: Replace Server.WriteReport method to another source file
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
54523d2949
[ #255 ] ir: Replace audit client creation to invoke package
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
76d4e53ea0
[ #255 ] services/audit: Skip all tasks from previous epoch in audit processor
...
Implement Reset method on audit task manager that cleans task queue.
Extended TaskManager interface with Reset method on IR side. Call Reset
method in audit processor before new audit start.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
33804e024d
[ #255 ] services/audit: Complete audit report after the last check
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
4dc09b19f3
[ #255 ] ir: Make audit processor to push tasks to audit task manager
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
580c9c974a
[ #255 ] services/audit: Implement task manager
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
0f0be2377b
[ #255 ] services/audit: Implement audit executor without checks
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
babfbc18f2
[ #255 ] services/audit: Implement Task structure
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
076f201807
[ #255 ] services/audit: Define Report structure and Reporter interface
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
919f4364f1
[ #269 ] morph/audit: Implement wrapper over Audit contract client
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
07da9d31f2
[ #269 ] morph/client: Implement Audit contract client
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
8dd7c689f2
[ #265 ] innerring: Select storage groups to audit
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
22cffbf529
[ #265 ] innerring: Add client cache
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
3d3d058b05
[ #265 ] innerring: Select containers to audit
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
87e1252065
[ #265 ] innerring: Generate new audit event
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
8c4e033db3
[ #80 ] core/object: Add SG-object format validation
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
43ee94a572
[ #80 ] storageroup: Implement function to collect structure from container
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
44466f4a29
[ #80 ] storagegroup: Implement search query generator
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
6245e58d1f
[ #80 ] services/object: Implement iterators over split-tree leaves
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
bb85ac24eb
[ #254 ] innerring: Increase extra fee for container registration
...
Some rough estimation until there will be p2p signature collection
inside the chain.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Alex Vanin
feec559e47
[ #254 ] Update neo-go to preview4 compatible version
...
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2020-12-25 16:49:27 +03:00
Leonard Lyubich
e53bf574b5
[ #279 ] container: Use new methods to work with container format
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-24 13:38:20 +03:00
Leonard Lyubich
bbe8f34ffd
[ #253 ] object/eacl: Process only CID/OID object headers in RANGEHASH/DELETE
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:31:20 +03:00
Leonard Lyubich
9e27263df3
[ #253 ] object/put: Fix loss of session token
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 18:31:20 +03:00
Leonard Lyubich
8e661d5c37
[ #264 ] object/get: Prevent receiving payloads through GETRANGE when GET
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
Leonard Lyubich
66f9532857
[ #264 ] object/get: Check parent address in linking/last child
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
Leonard Lyubich
ec21040542
[ #264 ] transformer: Finalize parent header once
...
In previous implementation parent object header finalized twice in size
limiter + formatter. On the one hand, this added redundant action, on the
other hand, it could provoke a difference in the headers of the linking and
the last part. Change formatter to finalize parent header if it does not
container the signature. Change size limiter to reuse parent header after
last child finalization in linking child.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-18 16:42:17 +03:00
Leonard Lyubich
edef9463d7
[ #253 ] metabase: Add prefix to Graveyard and ToMoveIt bucket names
...
In previous implementation DB.Containers method could return an error about
invalid container ID string format. This could happen if some of top-level
buckets had name w/o "_" substring.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2020-12-17 12:03:18 +03:00