Commit graph

1098 commits

Author SHA1 Message Date
Leonard Lyubich
56d4410913 [#638] Update to Go 1.16
Changes:

  * replace `iotuil` elements with the ones from `os` package;
  * replace `os.Filemode` with `fs.FileMode`;
  * use `signal.NotifyContext` instead of `NewGracefulContext` (removed).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-29 13:44:59 +03:00
Leonard Lyubich
8eadf18e01 [#607] services/control: Fix incorrect allocation in NetmapSnapshot
Fix make with length + subsequent append.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
f8ee6b1164 [#607] services/netmap: Support backward compatibility
Make Netmap service to return single network address on `LocalNodeInfo` RPC
in order to older clients.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
6f861b6489 [#607] network: Support URI address strings
Make Address.FromString method to parse URI addresses and enable TLS for TLS
schemes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
43eff09944 [#607] *: Do not use deprecated elements of code
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
cbe20a2bac [#607] network: Add helper function to iterate over all addresses
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
119031c8c7 [#607] network: Make AddressGroup.WriteToNodeInfo method a function
Method implementation doesn't use any private logic.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
359bbe319a [#607] network: Remove no longer needed GroupFromAddress function
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
e3eade619d [#607] network: Remove no longer used Address.AddTLS method
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
b723c92ef8 [#607] network: Make Address.Equal method unexported
The method is used only in the same package.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
61cc70d899 [#607] network: Remove no longer used Address.WriteToNodeInfo method
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
149367cab9 [#607] network: Sort addresses in AddressGroup.FromIterator method
Implement `sort.Interface` interface on `AddressGroup` and perform sorting
in `AddressGroup.FromIterator` method. Addresses with enabled TLS are "less"
in terms of slice position.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
1e52e86bbc [#607] node/control: Make group address in NodeInfo message
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
cede2b4ed7 [#607] reputation,container: Support address groups in ServerInfo
There is a need to support multiple server endpoints for reputation and
container transmission.

Replace `ServerInfo.Address` getter with `ServerInfo.IterateAddresses`
iterator.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
d6bb697726 [#607] container/announce: Use public keys as keys to server cache
In future server info will contain multiple endpoints whose string
representation will be worse suited to the cache key.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
44a2c81f8e [#607] reputation/router: Use public keys as keys to server cache
In future server info will contain multiple endpoints whose string
representation will be worse suited to the cache key.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
f7b42a1525 [#607] ir: Work with AddressGroup instead of single Address
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
9033cf5aa1 [#607] placement: Work with AddressGroup
Make `PlacementBuilder` implementations to work `network.AddressGroup` type.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
b3dd9a3254 [#607] placement: Make traverser to return list of address groups
Make placement `Traverser.Next` method to return ``[]network.AddressGroup`
in order to support multiple addresses of the storeage nodes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
8ac3c62518 [#607] object/head: Make client constructor to work with group address
Make Object Head service to work with `AddressGroup` instead of `Address`
in order to support multiple addresses of the storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
d0e48c949b [#607] object/search: Make client constructor to work with group address
Make Object Search service to work with `AddressGroup` instead of `Address`
in order to support multiple addresses of the storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
ad14df07f6 [#607] object/get: Make client constructor to work with group address
Make Object Get service to work with `AddressGroup` instead of `Address` in
order to support multiple addresses of the storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
8972f84672 [#607] object/put: Make client constructor to work with group address
Make Object Put service to work with `AddressGroup` instead of `Address` in
order to support multiple addresses of the storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
6e5d7f84af [#607] network: Generalize LocalAddressSource to address group
Make `LocalAddressSource.LocalAddress` method to return `AddressGroup`. Make
`IsLocalAddress` function to accept parameter of type `AddressGroup`. Adopt
the application code with temporary `GroupFromAddress` helper.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
e11f50ec8e [#607] network: Make ClientCache to accept AddressGroup
Change type of the `ClientCache.Get` method's parameter to `AddressGroup`.
Use `GroupFromAddress` to call the method from the wrappers in order to no
change their interface.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
5db7c5c2a8 [#607] network: Implement function to stringify AddressGroup
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
c82615667d [#607] network: Return group-address client from ClientCache
Add group-address `Client` implementation. Return instances of this
implementation from `ClientCache.Get` method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
3805b0f638 [#607] client: Overload Client interface
There is a need to generalize single-address client to group-address client.
To do this, we can re-implement `Client` interface from NeoFS API Go library
and still use it in the application code. There is a problem with method
`Raw` which must return single-address raw client. So as not to make changes
to API library we need to overload Client interface in order to support
`Raw` method in group-address client implementation.

Define `Client` interface in new `pkg/core/client` package. Completely
inherit API `Client` interface. Add `RawForAddress` method to build raw
client for the single node address. Adopt the application code that used Raw
method to work with new `Client`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
5e4208648a [#607] network: Implement AddressGroup type
Define `network.AddressGroup` type which represents group of network
addresses. This type is going to be used to support group-address of the
storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Alex Vanin
5df55bcfaf [#648] morph/client: Fix method name in NewEpoch
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-25 16:29:00 +03:00
Pavel Karpy
48827f42d3 [#643] pkg: Sync method names and commentaries to them
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-24 16:10:44 +03:00
Leonard Lyubich
25391111ad [#633] object: Disallow empty attribute values
Values of object attributes must not be empty according to NeoFS
specification.

Make `FormatValidator.Validate` method to return an error if at least one
attribute has empty value.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-24 11:12:23 +03:00
Leonard Lyubich
01dd17e30a [#633] object: Prevent duplicate attributes by key
Keys of object attributes must be unique according to NeoFS specification.

Make `FormatValidator.Validate` method to return an error if at least one
attribute is duplicated.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-24 11:12:23 +03:00
Pavel Karpy
6efeee5ce0 [#641] ir/container: Add unique attributes check
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-24 11:11:13 +03:00
Pavel Karpy
8ddea8a5ee [#622] pkg/innerring: Change NodeValidator interface description
Make changes of the `NodeInfo` by implementations
of the `NodeValidator` interface optional.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-22 12:12:45 +03:00
Pavel Karpy
7b3897253c [#622] pkg/innerring: Add multiaddress validation
Start using multiaddress validation in netmap
processor.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-22 12:12:45 +03:00
Pavel Karpy
1cd0352bab [#622] pkg/innerring: Add composite validator
Add `CompositeValidator` that wraps
`netmap.NodeValidator`s and implements
`NodeValidator` interface itself.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-22 12:12:45 +03:00
Pavel Karpy
53b7e05b65 [#622] pkg/innerring: Add multiaddress validator
Add multiaddress validator that calls
`network.VerifyAddress`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-22 12:12:45 +03:00
Pavel Karpy
ea5c74e761 [#622] pkg/network: Add multiaddress validation
Validation checks:
1. if address can be parsed by network package;
2. if address contains correct amount of protocols;
3. if address's protocols are in correct order.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-22 12:12:45 +03:00
Evgenii Stratonikov
16e9e726ff [#496] morph/client: add wrapper for neofs contract
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-21 15:16:42 +03:00
Evgenii Stratonikov
458fc4f5ae [#496] morph/client: provide notary options on client creation
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-21 15:16:42 +03:00
Evgenii Stratonikov
7cf0093012 [#496] pkg/innerring: remove unused processor parameters
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-21 15:16:42 +03:00
Evgenii Stratonikov
8a2b7f4501 [#496] pkg/innerring: provide wrappers to processors
The only thing we need hashes for is to process notifications.
Balance contract if left for now, as it has some initialization.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-21 15:16:42 +03:00
ZhangTao1596
9b87e6267d [#568] shard/writecache: Maintain memory size and db size
Signed-off-by: ZhangTao1596 <zhangtao@ngd.neo.org>
2021-06-21 14:26:30 +03:00
Leonard Lyubich
adbbad0beb [#607] network: Do not work with Address pointers
`network.Address` structure in most cases created once and used read-only.

Replace `AddressFromString` function with `Address.FromString` method with
the same purpose and implementation. Make all libraries to work with value.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
5de074f24c [#607] network: Implement WriteToNodeInfo method on Address
Implement `Address.WriteToNodeInfo` method which sets address of `NodeInfo`
structure. Use it in storage node application.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
e5504c7130 [#607] network: Do not use Address.String for address comparison
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
dd67e2b690 [#607] network: Remove Address methods used only for testing
Remove `Encapsulate`, `Decapsulate` and `IPAddrString` methods of `Address`
since they are used in unit tests only.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
47fe8911a3 [#607] network: Rename Address.HostAddrString method to HostAddr
Return tyype is clear from the method's signature and docs, there is no
point in reflecting it in the name.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
35f81729e4 [#607] network: Do not return error from Address.HostAddrString method
Panic if internal `manet.DialArgs` call returns error since this is
unexpected according to `AddressFromString` implementation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00