Dump contains magic and a list of objects prefixed by object size in bytes.
We can't use proto-marshaled list because this requires having all dump
in memory. Using TAR induces 512 byte overhead for each object which can
be a problem in some cases.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
- Update `neofs-sdk-go`:
v0.0.0-20211230072947-1fe37df88f80 => v0.0.0-20220113123743-7f3162110659
- Add client interface that duplicates SDK's client behaviour and new
`MultiAddressClient` interface that has method that iterates over wrapped
clients.
- Also start using simple client mode that does not require parsing statuses
outside the SDK library.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
For some data compression makes little sense, as it is already compressed.
This commit allows to leave such data unchanged based on `Content-Type`
attribute. Currently exact, prefix and suffix matching are supported.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Reverse payload overtake triggers direct payload overtake that
sets status and error. We should not override that.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
It is much more convenient to skip source creation.
Also fix some bugs:
1. `cryptoSource.Int63()` now returns number in [0, 1<<63) as required
by `rand.Source` interface.
2. Replace `cryptoSource.Uint63()` with `cryptoSource.Uint64` to allow
generate uint64 numbers directly (see rand.Source64 docs).
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Indent sections corresponding to different shells for faster navigation
and remove `#` at the line beginning.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
If mainchain is disabled in IR config then the node should read inner
ring list via role management contract.
Use `NeoFSAlphabetList` method of morph client as IR lister if
`withoutMainNet` flag is set in IR app.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation `Search` method of transport splitter skipped
responses with empty ID list.
Replace while-loop with do-while one in `TransportSplitter.Search`
method implementation in order to send responses with empty result too.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Provide shard mode information via `DumpInfo()`. Delete atomic field from
Shard structure since it duplicates new field.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Shard's mode was not used in the Node, so added only two modes whose roles
are clear. More modes will be added in the future.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Without sanity check, container service provides successful response,
even though such request will never be approved by Alphabet nodes.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
When application is being terminated, replicator routine
might be on the object picking phase. Storage is terminated
asynchronously, thus `Select()` may return corresponding
error. If we don't process `context.Done()` in this case,
then application freezes on shutdown.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Make `flushBigObjects` routine to mark objects which are written to
`BlobStor`. This prevents already flushed objects from being written on
the next iterator tick.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
For fullness estimation of `Blobovnicza` we use number of object stored
in each size bucket. In previous implementation we multiplied the number
by the difference in bucket boundaries. This expression rather
estimated the minimum volume (and for the smallest bucket, the maximum)
of objects in the bucket.
Multiply number of objects by mean bucket size.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
