The lifetime of an object can be limited by specifying a correspondin
well-known attribute. Node should refuse to save expired objects.
Checking objects in FormatValidator is extended with an expiration attribute
parsing step.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
In previous implementation StorageEngine.Inhume operation forced Shard
.Inhume call on all internal shards. There is a need to inhume object in a
single shard. To achieve this, Inhume operation is performed in next steps:
1. iterate over sorted shards, check object presence through Exists call;
2. if object exists at any shard in step 1 => inhume it and return on
success;
3. if no shards contain the object => iterate over sorted shards again and
try to inhume the object at first possible shard;
4. if all Inhume calls are failed => return an error.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
All node info attribute transformations can't guarantee
the order of attributes. However it should be consistent
otherwise smart-contract won't be able to collect signatures
and approve transaction.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Node info validator may change node attributes, e.g. update
it with human-readable location attributes based on LOCODE.
So inner ring node should provide new node info binary to
smart contract.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Scanning subdivision csv-table entries one-by-one takes significant time and
system resources. To speed up random access to table records, on the first
call, the table is pumped into memory (map). On subsequent calls, I/O
operations are not performed.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Scanning csv-table entries one-by-one takes significant time and system
resources. To speed up random access to table records, on the first call,
the table is pumped into memory (map). On subsequent calls, I/O operations
are not performed.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Inner ring Server does not modify LOCODE database (Put method), thus it is
better to open it in RO mode.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Add ReadOnly function that returns Option that enables read-only mode in DB.
RO mode can be used by processes that won't modify the DB in order to not
acquire write flock.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Set timeout option of BoltDB Open operation to 3s in order to prevent
indefinite waiting for file lock.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement DB interface required by LOCODE validator on new wrapper over the
LOCODE Bolt DB (Record on new wrapper over LOCODE Bolt DB entries).
Construct LOCODE validator and pass it to Netmap processor's constructor as
NodeValidator parameter.
Thus, candidates for a network map must set LOCODE attribute for which there
is an entry in the NeoFS location database.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Keep list of functions which are called first when the server starts (method
Server.Start). If any of the starters returns an error, the server will not
start. Such starters will mainly be used for resources that need to be
initialized after a successful server construction, but before its main work
(e.g. local files).
Keep list of functions which are called when the server stops (Server.Stop
method). Such closers will mainly be used for resources that need to be
released after server shutdown (e.g. initialized by starters).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Define a structure for dealing with the geographic location of nodes.
Implement VerifyAndUpdate (with the same purpose as NodeValidator interface)
that checks LOCODE attribute and fills other attributes of the location.
Technically the entity is a wrapper over the NeoFS location database: it
maps the node LOCODE to the database record from which the new attributes
are generated.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Define NodeValidator interface of the entity that checks and finalizes
NodeInfo structure. Add NodeValidator to Netmap processor. Pass NodeInfo
structures of network map candidates to NodeValidator in order to verify it
and prepare to final state required by network.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Define NeoFS location database based on BoltDB. Implement methods to save
the record by key (Put) and to read the record by key (Get).
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Define database of Earth's polygons in GeoJSON format. Implement resolving
of geo point to continent in which it is located through point-in-polygon
calculation.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement airport database based on csv OpenFlights table. Implement
UN/LOCODE entry matcher. Implement country namespace.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Define structure of keys and records of the location database. Define the
interfaces of all components necessary for the formation of the database.
Implement the function of filling the database.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Define the data types needed to work with LOCODE's in NeoFS (country code,
location code, coordinates). Implement string parsers for new types.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Check if `new(big.Int)` will be efficient later and replace
all `big.NewInt()` in code or leave it as it is.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
TxTable used twice in context to transfer assets to
and from banking account. There is no need to store
instance of table persistently.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Some transfers from container owners into bank account may
fail due to lack of assets, so we have to deal with remaining
amount of assets in banking account.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Add SignWithRFC6979 option to signature verification function since eACL
table are signed by users with this option.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Since the contract started returning the table signature, it became
necessary to check its correctness.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Basic settlement context is a main structure that
implement logic of basic settlement phases: collecting
assets from container owners and then distributing them
to storage nodes.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
Implement route Builder interface on wrapper over the container placement
builder, Component implies exactly one transfer to each of the most weight
nodes of the container (according to some weighing algorithm).
Implementation is planned for use when transferring local estimates of
storage nodes.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement a component for transmitting the value of the used container space
along a route defined in the system. Implement WriterProvider interface on
it. By implementation, it is the link between the route planner and the
point-to-point transmitter, and abstracts from the implementation of both of
them. In the future, this implementation will be used as a transmitter of
local estimates of storage nodes among themselves.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement API methods of the Container contracts client corresponding
to the calls of the contract for storing the used space of the container
("putContainerSize", "getContainerSize", "listContainerSizes"). Extend
the wrapper over the client with methods abstracted from sidechen calls.
In particular, the method of storing the value will be used to record
the estimates of the used space of containers calculated by the network
participants.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement functions to wrap Writer or Iterator. The resulting wrapper
provides WriterProvider or IteratorProvider interface respectively.
Such a wrapper can be used as a single storage instance provider
regardless of context.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement a component that stores the values of the used space of
containers. The storage allows you to write several values for a fixed
container and epoch number, and read the averaged estimates of all
accumulated values. All values are stored in memory. This component is
planned to be used as an accumulator of opinions from various network
participants about the fullness of the container.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Implement a component that connects the value stores of the used space of
containers. Implement the Start/Stop operations on it, which will later
become the application handlers of the corresponding events from the
sidechain. The main task of the controller is to temporarily synchronize the
stages of calculating the global estimate of the used space in the
container. The details of the score calculation (the way of collecting /
transmitting local scores, the final score formula and writing to the
contract) are encapsulated in the dependency components, the controller is
abstracted from them.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
