Commit graph

1424 commits

Author SHA1 Message Date
Leonard Lyubich
f14987c06d [#645] core: Define interface of the tool for working with netmap keys
Announced keys of storage nodes are required for many application components
to function.

Define a unified interface for the utility for working with public keys of
nodes. Add a method to check if the key has been advertised by the local
node in the application context.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-07 09:53:18 +03:00
Leonard Lyubich
22926e8f28 [#645] placement/traverser: Extend Node type with public key
There is a need to process public keys of the placement result.

Implement `Node.PublicKey` method which returns storage node's key announced
in netmap.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-07 09:53:18 +03:00
Leonard Lyubich
fe90456dcc [#645] placement: Overload result of Traverser.Next method
In previous implementation `placement.Traverser.Next` method returned slice
of `network.AddressGroup` elements. There is a need to process keys of
storage nodes besides network addresses for intra-container communication.

Wrap `network.AddressGroup` in a new type `placement.Node` that summarizes
the storage node information required for communication. Return slice of
`Node` instances from `Traverser.Next` method. Fix compilation breaks in
dependent packages.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-07 09:53:18 +03:00
Leonard Lyubich
ba234699cd [#794] ir/neofs: Print tx IDs in LE on Withdraw and Deposit events
IDs come from NeoFS contract in big endian, but it is customary to write in
the node logs in little endian.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-06 18:11:38 +03:00
Leonard Lyubich
3f293cb55e [#762] storage/shard: Try to read headers from write-cache
Shard should try to read object headers from write-cache if it is enabled.

Extend `writecache.Cache` interface with `Head` method. Call the method in
`Shard.Head` if `Shard.hasWriteCache` returns true.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-02 11:33:17 +03:00
Leonard Lyubich
aa2151fbeb [#762] write-cache: Implement Head operation
Write cache should be able to execute HEAD operations according to spec.

Add simple implementation of `Head` method through the `Get` one. Leave
notes for future optimization.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-02 11:33:17 +03:00
Leonard Lyubich
81722c373b [#746] morph/client: Don't cache GAS contract address on NeoFS-side
Caching is performed inside `GetNativeContractHash` method of neo-go client,
so the additional cache level is redundant.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-02 11:04:38 +03:00
Leonard Lyubich
18c49c2cb9 [#746] morph/client: Break loop in iterateClients on NeoFS error
There is no need to continue iterating over Neo RPC endpoints in case of
some address-independent error (e.g. NeoFS logic error).

Unwrap and immediately return `neofsError` errors from loop in
`iterateClients`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-02 11:04:38 +03:00
Leonard Lyubich
a2b81f13b0 [#746] morph/client: Simplify loop control in iterateClients
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-02 11:04:38 +03:00
Leonard Lyubich
3b7e884e74 [#746] morph/client: Distinguish between neo-go errors and NeoFS logic
Implement `error` interface on new `neofsError` type which is a wrapper over
NeoFS-specific error. Wrap all `Client` errors except neo-go client API ones
into `neofsError`. Wrapped errors are going to be used for multi-endpoint
loop control.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-02 11:04:38 +03:00
Leonard Lyubich
85bd2a1cdf [#746] morph: Add error return of MagicNumber method
Since morph `Client` works in multi-client mode, there is an error case when
we can not get network magic when all endpoints are unavailable.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-02 11:04:38 +03:00
Leonard Lyubich
ad7ad12a0c [#746] morph: Implement and use multi-client
There is a need to work with a set of Neo RPC nodes in order not to depend
on the failure of some nodes while others are active.

Support "multi-client" mode of morph `Client` entity. If instance is not
"multi-client", it works as before. Constructor `New` creates multi-client,
and each method performs iterating over the fixed set of endpoints until
success. Opened client connections are cached (without eviction for now).

Storage (as earlier) and IR (from now) nodes can be configured with multiple
Neo endpoints. As above, `New` creates multi-client instance, so we don't
need initialization changes on app-side.

`Wait` and `GetDesignateHash` methods of `Client` return an error from now
to detect connection errors. `NotaryEnabled` method is removed as unused.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-02 11:04:38 +03:00
Alex Vanin
f9218bf84f [#787] util/attributes: Restore escape characters after caching the result
As soon as resulting list and cache operate with the attribute pointer,
we can add attribute structure immediately and set restored key and
value of the attribute later.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-01 14:29:26 +03:00
Alex Vanin
f09b7e48af [#787] util/attributes: Fix order of require.Equal arguments
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-01 14:29:26 +03:00
Alex Vanin
2d7e9f0c40 [#787] util/attributes: Return consistent list of attributes
Attributes are linked to each other through parents, so they can
be returned in any order. However, it will be better to return
the list in consistent order to reduce entropy.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-01 14:29:26 +03:00
Alex Vanin
d8a04726ad [#787] util/attributes: Reuse attribute struct to update parents list
Parser should reuse existing attributes from the cache to update list
of the parent links in it. Parent links should be unique.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-09-01 14:29:26 +03:00
Leonard Lyubich
e738699fcc [#676] services/container: Cache the results of read operations
In previous implementation Container service handlers didn't cache the
results of `Get` / `GetEACL` / `List` operations. As a consequence of this,
high load on the service caused neo-go client's connection errors. To avoid
this there is a need to use cache. Object service already uses `Get` and
`GetEACL` caches.

Implement cache of `List` results. Share already implemented cache of Object
service with the Container one. Provide new instance of read-only container
storage (defined as an interface)to morph executor's constructor on which
container service is based. Write operations remained unchanged.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-08-30 18:01:26 +03:00
Pavel Karpy
c54f524df9 [#773] writecache: Delete unused dbSize param
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-08-27 15:05:53 +03:00
Pavel Karpy
4ff98a7e2b [#760] metabase: Support COMMON_PREFIX matchtype
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-08-27 15:03:15 +03:00
Alex Vanin
d6439369ab [#768] object/acl: Support all eACL filter fields from the spec
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-08-26 19:01:03 +03:00
Alex Vanin
cadd94f08f [#766] Fix stylecheck import linter error
Remove redundant imports

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-08-26 17:57:15 +03:00
Alex Vanin
53f031e98c [#766] Fix misspell linter error
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-08-26 17:57:15 +03:00
Alex Vanin
30738549ef [#766] Fix makezero linter error
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-08-26 17:57:15 +03:00
Leonard Lyubich
abfcc7498c [#715] services/policer: Select pseudo-random list of objects to check
In previous implementation of Policer's job queue the same list of objects for
processing was selected at each iteration. This was caused by consistent
return of `engine.List` function.

Use `rand.Shuffle` function to compose pseudo-random list of all objects in
order to approximately evenly distribute objects to work.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-08-25 14:40:12 +03:00
Pavel Karpy
85a4682c0c [#759] pkg/innerring: Log notary status on startup
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-08-23 15:26:10 +03:00
Alex Vanin
300d8879a6 [#751] innerring: Use CalledByEntry scope in main chain invocations
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-08-06 17:29:27 +03:00
Alex Vanin
43dfccd9b3 [#751] morph/client: Add custom signer scope support
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-08-06 17:29:27 +03:00
ZhangTao1596
f121a73049 [#737] network/cache: handle request canceled in multiclient
Signed-off-by: ZhangTao1596 <zhangtao@ngd.neo.org>
2021-08-05 14:47:41 +03:00
Pavel Karpy
3d981320c6 [#745] gc: Stop GC work on Shard's Close
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-08-05 14:44:05 +03:00
Alex Vanin
6527f9157c [#744] util/attributes: Support escaped symbols
To encode attributes with semicolon or slash, use
backslash as escaped character.

Example:
  User-Agent:NeoFS\/0.23
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-08-04 16:48:54 +03:00
Pavel Karpy
7a52bb31f1 [#740] innerring/notary: Fix reading notary config
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-08-03 21:00:57 +03:00
Pavel Karpy
e8665f6cef [#730] node/morph: Use disable_cache config param
Do not init caches for eACL, containers and netmap
if `disable_cache` config options is `true`, use
direct RPC calls instead.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-08-03 09:40:10 +03:00
Pavel Karpy
c423aa432a [#730] node/config/morph: Add disable_cache
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-08-03 09:40:10 +03:00
Pavel Karpy
896c749b92 [#720] pkg/innerring: Check Notary availability automatically
Do not read `without_notary` config value from env.
Make morph client constructor return client without
notary support. Enabling notary support should be done
with public `EnableNotarySupport` method separately.

Notary availability is deducted with client. Further,
if notary is presented on chain its support is
enabled at the corresponding client.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-08-03 09:30:11 +03:00
Pavel Karpy
28aa0f521e [#735] pkg/network: Delete tls multiaddr implementation
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-08-02 14:28:58 +03:00
Pavel Karpy
7a10d902be [#727] Use util.MkdirAllX instead of os.MkdirAll
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-30 16:42:22 +03:00
Pavel Karpy
51eeae3640 [#727] util: Add MkdirAllX func
Add `MkdirAllX` function that wraps `os.MkdirAll`
and adds `x` permission for user and group.
This is used to make all created dirs openable
for user and group regardless of the read config
permissions.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-30 16:42:22 +03:00
ZhangTao1596
01806db612 [#723] container/announcement/storage: Map announcements by epoch and container id
Signed-off-by: ZhangTao1596 <zhangtao@ngd.neo.org>
2021-07-30 09:58:30 +03:00
Pavel Karpy
c41ec2e2e9 [#718] ir: Do not require processing hash in main notary disabled mode
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-23 10:59:45 +03:00
Alex Vanin
bba470570f [#708] morph/client: Update last epoch block method name
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-22 20:40:09 +03:00
Alex Vanin
3561cf5873 [#712] moprh/client: Log VUB in notary invocations
For easier debug when notary transaction didn't execute.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-22 15:46:48 +03:00
Alex Vanin
62efa3f098 [#708] innerring: Synchronize initial epoch tick
When Inner Ring node starts, it should sync nearest epoch tick
event based on the block of the latest epoch. Otherwise epoch
ticking can be stopped, because ballots or notary transactions
are valid for limited period of time.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-22 15:20:34 +03:00
Alex Vanin
67b3682348 [#708] morph/timer: Add single tick timer
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-22 15:20:34 +03:00
Alex Vanin
7146afcd28 [#708] morph/client: Add epoch block getter from netmap
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-22 15:20:34 +03:00
Alex Vanin
7e43af3de3 [#708] morph/client: Add BlockCount method
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-22 15:20:34 +03:00
Pavel Karpy
dfbde0004e [#705] innerring/governance: Delete useless sort
Do not sort slice that is already sorted.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-22 15:12:06 +03:00
Pavel Karpy
c8d58d56b9 [#705] innerring/governance: Hide fetching ir keys behind interface
Add to governance processor IRFetcher interface.
Implementation must deduct if notary enabled or not.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-22 15:12:06 +03:00
Pavel Karpy
73ba35b379 [#705] pkg/innerring: Fix disable notary mode
Get innerring list and committee in innerring
indexer from interfaces, not from structs.
Passing IR fetcher to indexer constructor depends
on having Notary contract in sidechain(use NeoFS
role or netmap contract).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-22 15:12:06 +03:00
Pavel Karpy
8e66c67a74 [#705] pkg/innerring: Add IR keys fetchers
Add `IrFetcherWithNotary` and `IrFetcherWithoutNotary`
that can fetch IR keys with and without usage of
notary contract.
Both can be hidden behind
`InnerRingKeys() (keys.PublicKeys, error)` interface.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-22 15:12:06 +03:00
Pavel Karpy
8ea5744326 [#705] pkg/morph/netmap: Fix updateInnerRing method
Add initialization of `updateInnerRing` method name
in config on startup.
Rename var `setInnerRing` => `updateInnerRing`.
Rename method `SetInnerRing` => `UpdateInnerRing`

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-22 15:12:06 +03:00
Pavel Karpy
84361beada [#705] pkg/morph/netmap/wrapper: Add GetInnerRingList method
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-22 15:12:06 +03:00
Pavel Karpy
b3c81116b9 [#705] pkg/morph/netmap: Add innerRingList method
Add `innerRingList` method to `netmap` client
wrapper. Method parses current IR nodes list.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-22 15:12:06 +03:00
Pavel Karpy
00b487e3c3 [#705] morph/notary: Add ProbeNotary method to Client
`ProbeNotary` method checks if native `Notary`
contract is presented on chain.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-22 15:12:06 +03:00
Pavel Karpy
46a66a21ec [#705] pkg/util: Fix commentary error
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-22 15:12:06 +03:00
Pavel Karpy
795c49d16b [#706] ir: Do not require proxy hash in notary disabled mode
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-21 15:32:17 +03:00
Alex Vanin
96da7ceb4f [#697] governance: Make best effort traversing main chain list of keys
We should go through every key in main chain list to merget lists
as fast as possible. Previously we drop main chain traversing as
soon as we have no more new keys to add. Instead we should try
to go for old keys in the list and add it as more as we can.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-15 10:49:21 +03:00
Alex Vanin
aed76f6b5a [#694] pkg/network: Use single function to get or create client instance
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-13 15:17:48 +03:00
Alex Vanin
38afb82926 [#694] pkg/network: Use client cache in raw client getter
Every raw client getter invocation produced new connection, that lead
to connection leak.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-13 15:17:48 +03:00
Alex Vanin
8112f73023 [#689] innerring: Remove debug output
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-13 13:11:29 +03:00
Pavel Karpy
c20eb15bd5 [#688] pkg/ir/governance: Add unsorted test case
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-12 18:46:09 +03:00
Pavel Karpy
bb3f16d487 [#688] pkg/ir/governance: Add public key sorting
Add sorting of passed public keys in `newAlphabetList`
function before Alphabet list merge to make returned
results more expected.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-12 18:46:09 +03:00
Pavel Karpy
361bff216a [#688] pkg/ir/governance: Check err type in tests
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-12 18:46:09 +03:00
Pavel Karpy
9da777ac8c [#681] ir: Do not require MainNet attributes
If `WITHOUT_MAINNET` environmental variable is
`true`:
- Do not read `NeoFS` and `processing`
script-hashes from envs;
- Do not init Governance processor;
- Do not init NeoFS processor.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-12 15:35:35 +03:00
Alex Vanin
24d1725cc8 [#666] innerring: Make GlagoliticLetter enum public for re-use
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-08 14:37:38 +03:00
Pavel Karpy
0bcc505682 [#668] pkg/audit: Skip audit for containers without SG
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-07-06 21:19:18 +03:00
Leonard Lyubich
7c1432aa57 [#660] services/container: Don't check container format in Put
Storage node should only reflect container Put requests to sidechain w/o any
validation. All verification steps are performed by IR.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-07-05 11:05:44 +03:00
Leonard Lyubich
54d6f3de55 [#660] core/container: Use version.IsValid in CheckFormat
Container can have version later than the current lib version, and it should
not be considered as invalid. But it should still have adequate version.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-07-05 11:05:44 +03:00
Leonard Lyubich
8192933313 [#660] services/netmap: Don't call IsSupportedVersion in LocalNodeInfo
Clients can have version later than the server. The server should not deny
these clients.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-07-05 11:05:44 +03:00
Leonard Lyubich
69826ebd90 [#660] core: Implement function to check protocol version adequacy
Create `version` package and implement `IsValid` function which checks if
version is not earlier than the genesis one.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-07-05 11:05:44 +03:00
Alex Vanin
c90f054f35 [#658] morph/neofs: Add TryNotary() option
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-02 09:53:34 +03:00
Alex Vanin
c0093b2b79 [#658] morph/neofsid: Add TryNotary() option
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-02 09:53:34 +03:00
Alex Vanin
92451c08af [#658] innerring: Do not use notary in audit client
Audit client is used to send audit results and they should be
signed by inner ring node itself on order to be saved in
smart contract.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-07-02 09:53:34 +03:00
Pavel Karpy
a2b622d62e [#647] pkg/morph/netmap/wrapper: GetCandidates method
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-29 22:29:23 +03:00
Pavel Karpy
d664a00e94 [#647] pkg/morph/netmap: Add netmapCandidates method
Add `netmapCandidates` method to `netmap` client
wrapper. Method parses node storages candidates
for the next epoch.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-29 22:29:23 +03:00
Pavel Karpy
403f836968 [#647] pkg/morph: Add constant var to err message
Use defined constant variable instead of
hardcoded number in error message.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-29 22:29:23 +03:00
Leonard Lyubich
dab33e0f9b [#652] Update Neo Go to v0.95.3
Add `keys.NEP2ScryptParams()` to `keys.NEP2Decrypt` call arguments.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-29 13:44:59 +03:00
Leonard Lyubich
56d4410913 [#638] Update to Go 1.16
Changes:

  * replace `iotuil` elements with the ones from `os` package;
  * replace `os.Filemode` with `fs.FileMode`;
  * use `signal.NotifyContext` instead of `NewGracefulContext` (removed).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-29 13:44:59 +03:00
Leonard Lyubich
8eadf18e01 [#607] services/control: Fix incorrect allocation in NetmapSnapshot
Fix make with length + subsequent append.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
f8ee6b1164 [#607] services/netmap: Support backward compatibility
Make Netmap service to return single network address on `LocalNodeInfo` RPC
in order to older clients.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
6f861b6489 [#607] network: Support URI address strings
Make Address.FromString method to parse URI addresses and enable TLS for TLS
schemes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
43eff09944 [#607] *: Do not use deprecated elements of code
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
cbe20a2bac [#607] network: Add helper function to iterate over all addresses
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
119031c8c7 [#607] network: Make AddressGroup.WriteToNodeInfo method a function
Method implementation doesn't use any private logic.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
359bbe319a [#607] network: Remove no longer needed GroupFromAddress function
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
e3eade619d [#607] network: Remove no longer used Address.AddTLS method
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
b723c92ef8 [#607] network: Make Address.Equal method unexported
The method is used only in the same package.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
61cc70d899 [#607] network: Remove no longer used Address.WriteToNodeInfo method
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
149367cab9 [#607] network: Sort addresses in AddressGroup.FromIterator method
Implement `sort.Interface` interface on `AddressGroup` and perform sorting
in `AddressGroup.FromIterator` method. Addresses with enabled TLS are "less"
in terms of slice position.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
1e52e86bbc [#607] node/control: Make group address in NodeInfo message
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
cede2b4ed7 [#607] reputation,container: Support address groups in ServerInfo
There is a need to support multiple server endpoints for reputation and
container transmission.

Replace `ServerInfo.Address` getter with `ServerInfo.IterateAddresses`
iterator.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
d6bb697726 [#607] container/announce: Use public keys as keys to server cache
In future server info will contain multiple endpoints whose string
representation will be worse suited to the cache key.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
44a2c81f8e [#607] reputation/router: Use public keys as keys to server cache
In future server info will contain multiple endpoints whose string
representation will be worse suited to the cache key.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
f7b42a1525 [#607] ir: Work with AddressGroup instead of single Address
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
9033cf5aa1 [#607] placement: Work with AddressGroup
Make `PlacementBuilder` implementations to work `network.AddressGroup` type.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
b3dd9a3254 [#607] placement: Make traverser to return list of address groups
Make placement `Traverser.Next` method to return ``[]network.AddressGroup`
in order to support multiple addresses of the storeage nodes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
8ac3c62518 [#607] object/head: Make client constructor to work with group address
Make Object Head service to work with `AddressGroup` instead of `Address`
in order to support multiple addresses of the storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
d0e48c949b [#607] object/search: Make client constructor to work with group address
Make Object Search service to work with `AddressGroup` instead of `Address`
in order to support multiple addresses of the storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
ad14df07f6 [#607] object/get: Make client constructor to work with group address
Make Object Get service to work with `AddressGroup` instead of `Address` in
order to support multiple addresses of the storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
8972f84672 [#607] object/put: Make client constructor to work with group address
Make Object Put service to work with `AddressGroup` instead of `Address` in
order to support multiple addresses of the storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
6e5d7f84af [#607] network: Generalize LocalAddressSource to address group
Make `LocalAddressSource.LocalAddress` method to return `AddressGroup`. Make
`IsLocalAddress` function to accept parameter of type `AddressGroup`. Adopt
the application code with temporary `GroupFromAddress` helper.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
e11f50ec8e [#607] network: Make ClientCache to accept AddressGroup
Change type of the `ClientCache.Get` method's parameter to `AddressGroup`.
Use `GroupFromAddress` to call the method from the wrappers in order to no
change their interface.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
5db7c5c2a8 [#607] network: Implement function to stringify AddressGroup
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
c82615667d [#607] network: Return group-address client from ClientCache
Add group-address `Client` implementation. Return instances of this
implementation from `ClientCache.Get` method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
3805b0f638 [#607] client: Overload Client interface
There is a need to generalize single-address client to group-address client.
To do this, we can re-implement `Client` interface from NeoFS API Go library
and still use it in the application code. There is a problem with method
`Raw` which must return single-address raw client. So as not to make changes
to API library we need to overload Client interface in order to support
`Raw` method in group-address client implementation.

Define `Client` interface in new `pkg/core/client` package. Completely
inherit API `Client` interface. Add `RawForAddress` method to build raw
client for the single node address. Adopt the application code that used Raw
method to work with new `Client`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Leonard Lyubich
5e4208648a [#607] network: Implement AddressGroup type
Define `network.AddressGroup` type which represents group of network
addresses. This type is going to be used to support group-address of the
storage node.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-28 15:52:50 +03:00
Alex Vanin
5df55bcfaf [#648] morph/client: Fix method name in NewEpoch
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-25 16:29:00 +03:00
Pavel Karpy
48827f42d3 [#643] pkg: Sync method names and commentaries to them
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-24 16:10:44 +03:00
Leonard Lyubich
25391111ad [#633] object: Disallow empty attribute values
Values of object attributes must not be empty according to NeoFS
specification.

Make `FormatValidator.Validate` method to return an error if at least one
attribute has empty value.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-24 11:12:23 +03:00
Leonard Lyubich
01dd17e30a [#633] object: Prevent duplicate attributes by key
Keys of object attributes must be unique according to NeoFS specification.

Make `FormatValidator.Validate` method to return an error if at least one
attribute is duplicated.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-24 11:12:23 +03:00
Pavel Karpy
6efeee5ce0 [#641] ir/container: Add unique attributes check
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-24 11:11:13 +03:00
Pavel Karpy
8ddea8a5ee [#622] pkg/innerring: Change NodeValidator interface description
Make changes of the `NodeInfo` by implementations
of the `NodeValidator` interface optional.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-22 12:12:45 +03:00
Pavel Karpy
7b3897253c [#622] pkg/innerring: Add multiaddress validation
Start using multiaddress validation in netmap
processor.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-22 12:12:45 +03:00
Pavel Karpy
1cd0352bab [#622] pkg/innerring: Add composite validator
Add `CompositeValidator` that wraps
`netmap.NodeValidator`s and implements
`NodeValidator` interface itself.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-22 12:12:45 +03:00
Pavel Karpy
53b7e05b65 [#622] pkg/innerring: Add multiaddress validator
Add multiaddress validator that calls
`network.VerifyAddress`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-22 12:12:45 +03:00
Pavel Karpy
ea5c74e761 [#622] pkg/network: Add multiaddress validation
Validation checks:
1. if address can be parsed by network package;
2. if address contains correct amount of protocols;
3. if address's protocols are in correct order.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-22 12:12:45 +03:00
Evgenii Stratonikov
16e9e726ff [#496] morph/client: add wrapper for neofs contract
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-21 15:16:42 +03:00
Evgenii Stratonikov
458fc4f5ae [#496] morph/client: provide notary options on client creation
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-21 15:16:42 +03:00
Evgenii Stratonikov
7cf0093012 [#496] pkg/innerring: remove unused processor parameters
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-21 15:16:42 +03:00
Evgenii Stratonikov
8a2b7f4501 [#496] pkg/innerring: provide wrappers to processors
The only thing we need hashes for is to process notifications.
Balance contract if left for now, as it has some initialization.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-21 15:16:42 +03:00
ZhangTao1596
9b87e6267d [#568] shard/writecache: Maintain memory size and db size
Signed-off-by: ZhangTao1596 <zhangtao@ngd.neo.org>
2021-06-21 14:26:30 +03:00
Leonard Lyubich
adbbad0beb [#607] network: Do not work with Address pointers
`network.Address` structure in most cases created once and used read-only.

Replace `AddressFromString` function with `Address.FromString` method with
the same purpose and implementation. Make all libraries to work with value.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
5de074f24c [#607] network: Implement WriteToNodeInfo method on Address
Implement `Address.WriteToNodeInfo` method which sets address of `NodeInfo`
structure. Use it in storage node application.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
e5504c7130 [#607] network: Do not use Address.String for address comparison
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
dd67e2b690 [#607] network: Remove Address methods used only for testing
Remove `Encapsulate`, `Decapsulate` and `IPAddrString` methods of `Address`
since they are used in unit tests only.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
47fe8911a3 [#607] network: Rename Address.HostAddrString method to HostAddr
Return tyype is clear from the method's signature and docs, there is no
point in reflecting it in the name.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
35f81729e4 [#607] network: Do not return error from Address.HostAddrString method
Panic if internal `manet.DialArgs` call returns error since this is
unexpected according to `AddressFromString` implementation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
4c2d8d5ac9 [#607] network: Reflect all supported formats in AddressFromString docs
Comment that `AddressFromString` function supports host addresses.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
9e241ab91e [#607] network: Remove unused HostAddrFromMultiaddr function.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
95ccbbc2f9 [#607] network: Accept value instead of pointer in IsLocalAddress
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
fdd123ad98 [#607] network: Prevent potential panic in Address.Equal method
Make `Address.Equal` method to accept value instead of pointer in order to
prevent NPE.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
6de0af0650 [#607] network: Use Equal method in IsLocalAddress function
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Leonard Lyubich
b2b33aeac9 [#607] network: Don't export L4 protocol constant
Const is used for internal package needs only.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-18 18:09:50 +03:00
Pavel Karpy
68c7f6ce8a [#613] pkg/innerring/reputation: Add FIXME to ToV2 usage
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-17 18:30:06 +03:00
Pavel Karpy
ed9a5e44b6 [#613] pkg/innerring: Add sanity check of GlobalTrust
Add sanity checks of GlobalTrust value:
check if "got manager" is real manager for
peer with building managers for peer.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-17 18:30:06 +03:00
Pavel Karpy
6b176e8769 [#613] pkg/reputation: Move manager building to pkg
Move `managers` package to `pkg` since
it can be reused in other packages.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-17 18:30:06 +03:00
Leonard Lyubich
cc81723b01 [#615] blobovnicza: Simplify byte size stringer
Make `stringifyByteSize` to return number of bytes as string.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-16 12:14:04 +03:00
Alex Vanin
08b5f29e16 [#609] morph/client: Make halt state error private
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-15 16:52:51 +03:00
Alex Vanin
0ec3faefc9 [#609] morph/client: Check return state in invoke
Test invocations are used in `Invoke` method to calculate
consumed gas. We can check return code and return error
if panic happened in contract.

This is already done the same way in `TestInvoke` method.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-15 16:52:51 +03:00
Leonard Lyubich
455fd952dd [#414] ir: Serve ControlService
Serve `ControlService` instance on configured endpoint (do not serve if not
specified). Read allowed keys from config.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-15 15:21:52 +03:00
Leonard Lyubich
4001ba2967 [#414] ir: Switch health status on application state transitions
Set health status to

  * `UNDEFINED` during Server construction;
  * `STARTING` on `Server.Start` call;
  * `READY` after successful `Server.Start` call;
  * `SHUTTING_DOWN` on `Server.Stop` call.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-15 15:21:52 +03:00
Leonard Lyubich
567421a9b5 [#414] ir: Implement HealthChecker on Server
`HealthChecker` interface is required to construct `ControlServiceServer`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-15 15:21:52 +03:00
Leonard Lyubich
dcfe9a6504 [#414] ir/control: Implement service server
Implement `ControlServiceServer` on `Server` type. The `Server` requires all
requests to be signed with keys from the so-called whitelist. To obtain
health status, it uses the abstraction in the form of `HealthChecker`
interface.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-15 15:21:52 +03:00
Leonard Lyubich
93803b1a90 [#414] ir: Define Control service
Define `ControlService` for IR similar to the one from storage node. Add
`HealthStatus` RPC which returns health status of the IR application.
Implement getters, setters and methods to sign/verify the messages.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-15 15:21:52 +03:00
Evgenii Stratonikov
9142c778c7 [#562] pkg: remove nspcc-dev/neofs-crypto dependency
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-15 14:49:59 +03:00
Evgenii Stratonikov
5cab0026c3 [#562] pkg/morph: remove neofs-crypto uses
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-15 14:49:59 +03:00
Evgenii Stratonikov
1553967328 [#562] cmd/neofs-ir: use NEP-6 wallet for keys
Also remove neofs-crypto uses from `pkg/innerring`.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-15 14:49:59 +03:00
Alex Vanin
e50abeab0c [#598] innerring/config: Override global config only in debug build
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-09 16:20:37 +03:00
Leonard Lyubich
68e85e4b33 [#412] ir/locode: Disallow explicit setting of LOCODE-derived attributes
Make `Validator.VerifyAndUpdate` method to return an error if at least one
of LOCODE-derived attributes is set explicitly. Thus, IR will not confirm
the candidates for the network map who independently set these attributes.

Cover `Validator.VerifyAndUpdate` method with unit tests.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-09 15:08:12 +03:00
Leonard Lyubich
75d6702d2e [#412] ir/locode: Fix docs of the VerifyAndUpdate method
`Validator.VerifyAndUpdate` method does not add location code attribute.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-09 15:08:12 +03:00
Leonard Lyubich
096acb2a44 [#580] v2/object/put: Check payload size of the relayed objects
NeoFS network dictates the limitation of the maximum size of the "finished"
objects. The payload size of an object is recorded in its header. When
writing finished objects, the nodes must check:

  * satisfying the constraint;
  * matching the value in the header to the number of payload bytes.

Provide value returned by `MaxSizeSource` component to `validatingTarget`.
Check max size value during the stream of the "finished" objects. Check
header value during the streaming and on-close. Check payload size in v2
relay scenario.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-09 12:08:37 +03:00
Leonard Lyubich
2f38fef31a [#589] ir/container: Verify session token lifetime
Session tokens have limited lifetime in NeoFS. Container processor should
verify lifetime of the incoming tokens.

Define `NetworkState` interface with `Epoch` method to get number of the
current epoch. Use Netmap contract client's wrapper as `NetworkState` of
Container `Processor`. Check values of token lifetime, and deny if:

  * NBF value is gt the current epoch;
  * IAT is gt the current epoch;
  * EXP is le the current epoch.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-09 10:17:49 +03:00
Alex Vanin
e1ac2fe2e5 [#588] Fix go fmt linter errors
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-06-03 18:59:07 +03:00
Pavel Karpy
752efc6f8c [#584] pkg/innerring: Delete using deprecated methods
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-03 13:59:59 +03:00
Pavel Karpy
057ebf9d51 [#584] pkg/innerring: Stop using deprecated methods
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-06-03 13:59:59 +03:00
Leonard Lyubich
ed80f704d0 [#556] innerring/neofs: Process Bind/Unbind events
Make IR processor of NeoFS contract to handle `Bind`/`Unbind` notification
events. The processor verifies the format of wallet script hash and public
keys, and call NeoFS ID client wrapper in order to approve adding/removing
keys from NeoFS account.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-02 10:50:44 +03:00
Leonard Lyubich
48d934ecf8 [#556] morph/neofsid: Construct static client with enabled notary
Pass `TryNotary()` option to constructor of the static client of the NeoFS
ID contract in `NewFromMorph`. This will allow to use client wrapper for key
management in IR application.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-02 10:50:44 +03:00
Leonard Lyubich
f76083484b [#556] morph/container: Do not parse public key in Put event parser
Morph event structures defined in `pkg/morph/event` should only carry
notification values without any additional interpretation. All logical work
should be concentrated on app-side. Change `Bind.User` / `Unbind.User` to
return byte slice. Change `Bind.Keys` / `Unbind.Keys` to return `[][]byte`.
`ParseBind` / `ParseUnbind` don't decode data from now.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-02 10:50:44 +03:00
Leonard Lyubich
55c83454b6 [#556] morph/neofsid: Implement key management on client wrapper
Implement method `ClientWrapper.ManageKeys` method which provides the
interface to add/remove keys to/from NeoFS account.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-02 10:50:44 +03:00
Leonard Lyubich
4713e6b2b8 [#556] morph/neofsid: Add Client methods to add/remove keys
Implement `AddKeys` / `RemoveKeys` methods to call `addKey` / `removeKey`
methods of NeoFS ID contract.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-02 10:50:44 +03:00
Leonard Lyubich
8c2d42368a [#556] morph/neofs: Implement wrapper over contract client
Implement wrapper over NeoFS contact's client which allows you to
conveniently interact with the contract. Implement `ManageKeys` method
for binding or unbinding public keys to the NeoFS account.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-02 10:50:44 +03:00
Leonard Lyubich
9dc741d43e [#556] morph/neofs: Implement contract client
Implement NeoFS contact's client which is responsible for collecting call
arguments and parsing stack items. Initially key binding and unbinding are
supported.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-06-02 10:50:44 +03:00
Evgenii Stratonikov
b8a7c11e57 [#501] object/put: reduce TTL of the relayed request
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-02 10:31:14 +03:00
Evgenii Stratonikov
3468491224 [#501] object/put: relay requests for signed objects
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-02 10:31:14 +03:00
Evgenii Stratonikov
a422f42ca9 [#501] object/put: refactor distributed target
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-06-02 10:31:14 +03:00
Leonard Lyubich
d34de558f0 [#570] *: Use generator of test owner IDs from API Go lib
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 20:45:15 +03:00
Leonard Lyubich
3e1463cc76 [#570] *: Use generator of test container IDs from API Go lib
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 20:45:15 +03:00
Leonard Lyubich
3dd10b6795 [#570] *: Remove usage of deprecated elements from API Go library
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 20:45:15 +03:00
Leonard Lyubich
70a7354e9d [#570] *: Use new Equal method of owner.ID type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 20:45:15 +03:00
Leonard Lyubich
49a42b1d3e [#567] innerring: Close cached clients on shutdown
Register `CloseAll` method to be called on IR application shutdown.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 13:11:37 +03:00
Leonard Lyubich
bf92e895c0 [#567] network/clients: Implement method to close the cached clients
Update API Go library with introduce `Client.Conn` method. Implement
`ClientCache.CloseAll` method which reads and closes connections of all
cached clients.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 13:11:37 +03:00
Leonard Lyubich
9359f28161 [#525] ir/container: Compare owner IDs via Equal method
In recent change of API Go library `owner.ID.Equal` signature was
implemented.

Replace the comparison of string representations with `Equal` method call
and remove related TODOs.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 11:53:06 +03:00
Leonard Lyubich
937171911a [#525] ir/container: Simplify the approach to verify token signature
In recent change of API Go library `Token.Verify` signature was implemented.

Replace previous version-casting approach with new method call in token
signature check stage.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 11:53:06 +03:00
Leonard Lyubich
23c5277014 [#525] ir/container: Check session verb and container ID
Token of the container session should be written out with container context.
The context should have the verb corresponding to the operation. If an
operation is performed on a fixed container, the session should be
propagated to it or to all user containers

Implement all described checks in validation of `Put` / `Delete` / `SetEACL`
events.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 11:53:06 +03:00
Leonard Lyubich
b73c0c67a2 [#525] ir/container: Fix checks without session token
In previous implementation verification of `SetEACL` events failed on events
without session token. It was caused by redundant tries to verify `nil`
session token.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-31 11:53:06 +03:00
Leonard Lyubich
ead4513feb [#525] ir/container: Verify operations with session token
Session token can be presented `Put`, `Delete` and `SetEACL` notification
events. IR should consider this case as issuing a power of attorney to a
third party. Thus, checking the eligibility for an operation should be
complicated:

 - token owner should be the owner of the related container;
 - the intent must be signed with a session key;
 - the power of attorney must be signed by the owner of the container.

Omitted checks (TBD):

 - session token should have container session context;
 - the verb of the context should correspond to the operation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 17:07:20 +03:00
Pavel Karpy
f89c8bf239 [#549] clientCache: Add TLS to client
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-27 17:07:04 +03:00
Pavel Karpy
f267fbc56a [#549] network/Address: Add TLS
There is no TLS protocol support in
`go-multiaddr` library, but there is
public function that can register any
protocol that can be implemented outside
the library. Also `TLSEnabled` function
for parsing TLS protocol from
`network.Address` was added.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-27 17:07:04 +03:00
Pavel Karpy
33bef46f31 [#549] network/cache: Change Get signature
Make network cache's `Get` method accept
`network.Address` argument instead of
string.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-27 17:07:04 +03:00
Leonard Lyubich
5a3a27ba9f [#525] ir/container: Remove no longer needed FIXME
In latest design of `Container` contract key binding is removed.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
fb4e328171 [#525] morph/container: Remove binding key argument from Put call
`bindKey` parameter of `Put` method of `Container` contract was removed in
latest version.

Do not pass bind key of type `[]byte` to `Put` invocation. Remove no longer
needed field from `PutArgs`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
e1fbc28851 [#525] ir/container: Remove resolved FIXME about notary call
From `017fb6abed9455c7c99631adcb0bb04d42741f87` used container client is
constructed with enabled notary mode.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
a1bfcdc74a [#525] v2/container: Write session token from header to removal witness
If container is removed via session, then session token should be included
in removal witness.

Write session token from request meta header to `container.RemovalWitness`
structure which is passed to `wrapper.Delete` function.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
40c796bd7e [#525] ir/container: Write session token on delete approval
Approved removed container ID should be stored in sidechain along with
related session token.

Forward session token from `Delete` event to `Wrapper.Delete` method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
631d7b0e07 [#525] morph/container: Accept container session token in Delete
`Delete` method of latest `Container` contract accepts binary session token
as an argument.

Provide `DeleteArgs.SetSessionToken` method. Accept session token as a
`[]byte` in `Wrapper.Put` method and attach it to `PutArgs`. Marshal session
token from `RemovalWitness` in `wrapper.Delete` function and pass it to the
method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
166e5440ab [#525] event/container: Parse session token from Delete notification
The 3rd item of `Delete` container notification event is a byte array of
serialized session token.  Parse session token in `ParseDelete` function.
Provide `Delete.SessionToken` method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
c12ea210da Fix typo in error details of ParseSetEACL function
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
9659bdc14d [#525] event/container: Fix docs of SessionToken methods
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
a1e1350db0 [#525] core/container: Extend removal witness with session token
NeoFS containers can be removed within a trusted session. There is a need to
take this into account during removal inspection.

Define `SessionToken` / `SetSessionToken` methods on `RemovalWitness` struct
in order to embed `session.Token` to it.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
9a0964efa4 [#525] morph/container: Accept RemovalWitness in Delete function
Make `wrapper.Delete` function to accept `container.RemovalWitness` struct
instead of its separated elements. `Signature` type is replaced by binary
signature since public key is unused.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
d691a20d52 [#525] core/container: Define removal witness
Define `RemovalWitness` structure which groups the information required to
prove and verify the removal of a container. This type is going to be used
in container Delete-related methods.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
edfdc4274a [#525] morph/container: Pass session token to Delete call
Pass session token (byte array) argument to `Delete` method call of
`Container` contract.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
cfa209d74e [#525] morph/container: Attach parsed session token to table in GetEACL
Unmarshal session token from `EACLValues` and write it to resulting
`eacl.Table` structure in `Wrapper.GetEACL` method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
7ca6f601ef [#525] morph/container: Parse session token, key and signature in EACL
`EACL` method of `Container` contract returns binary session token, key and
signature along with eACL table.

Provide `Signature`, `PublicKey` and `SessionToken` getters from
`EACLValues` structure. Parse and set all values in `Client.EACL` methods.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
4ef369732a [#525] ir/container: Write session token on eACL table approval
Approved eACL table should be stored in sidechain along with related session
token.

Forward session token from `SetEACL` event to `Wrapper.PutEACL` method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
db7312274c [#525] event/container: Parse session token from SetEACL notification
The 4th item of `SetEACL` container notification event is a byte array of
serialized session token.

Parse session token in `ParseSetEACL` function. Provide
`SetEACL.SessionToken` method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
b09f212990 [#525] v2/container: Write session token from header to eACL table
If eACL table is set via session, then session token should be written to
it.

Write session token from request meta header to `eacl.Table` structure which
is passed to `wrapper.PutEACL` function.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
5c2b8de87d [#525] v2/container: Make context with token reusable
Rename `PutContext` to `ContextWithToken` and implement its constructor as a
separate function in order to reuse it in other RPCs.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
959610080a [#525] morph/container: Accept container session token in PutEACL
`SetEACL` method of latest `Container` contract accepts binary session token
as an argument.

Provide `SetEACLArgs.SetSessionToken` method. Accept session token as a
`[]byte` in `Wrapper.PutEACL` method and attach it to `SetEACLArgs`. Marshal
session token from container in `wrapper.PutEACL` function and pass it to
the method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
c4a3adc8b0 [#525] morph/container: Attach all parsed items to container in Get
Unmarshal session token from `GetValues` and write it to resulting
`Container` structure in `Wrapper.Get` method. Write key-signature pair from
`GetValues` to resulting `Container` structure in `Wrapper.Get` method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
b9b369dd5b [#525] morph/container: Parse session token, key and signature in Get
`Get` method of `Container` contract returns binary session token, key and
signature along with container.

Provide `Signature`, `PublicKey` and `SessionToken` getters from `GetValues`
structure. Parse and set all values in `Client.Get` methods.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
461fcfcf19 [#525] ir/container: Write session token on container approval
Approved container should be stored in sidechain along with related session
token.

Forward session token from `Put` event to `Wrapper.Put` method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
0f91b78df1 [#525] event/container: Parse binary session token from Put notification
The 4th item of `Put` container notification event is a byte array of
serialized session token.

Parse session token in `ParsePut` function. Provide `Put.SessionToken`
method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
98cc685a9b [#525] v2/container: Write session token from header to container
If container is created via session, then session token should be written to
it.

Write session token from request meta header to `Container` structure which
is passed to `wrapper.Put` function.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
db67a117f0 [#525] morph/container: Accept container session token in Put
`Put` method of latest `Container` contract accepts binary session token as
an argument.

Provide `PutArgs.SetSessionToken` method. Accept session token as a `[]byte`
in `Wrapper.Put` method and attach it to `PutArgs`. Marshal session token
from container in `wrapper.Put` function and pass it to the method.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
1deb3f3d01 [#525] morph/container: Do not accept signature in PutEACL function
In previous implementation wrapper over the Container contract's client
accepted the signature of the eACL table in addition to itself. After recent
changes in API Go lib table carries its signature. Thus, it is redundant
to pass the eACL table signature separately.

Make `wrapper.PutEACL` method to accept `eacl.Table` only.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
32828d2b90 [#525] morph/container: Do not accept signature in Put function
In previous implementation wrapper over the Container contract's client
accepted the signature of the container in addition to itself. After recent
changes in API Go lib container carries its signature. Thus, it is redundant
to pass the container signature separately.

Make `wrapper.Put` method to accept `Container` only.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
df197dc38b [#525] morph/container: Do not return signature from GetEACL method
In previous implementation wrapper over the Container contract's client
returned the signature of the eACL table in addition to itself. After recent
changes in API Go lib table carries its signature. Thus, it is redundant to
return the table signature separately.

Make `Wrapper.GetEACL` method to return only `eacl.Table` with error.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
615813d6de [#525] v2/container: Return session token in GetExtendedACL
In recent API changes `GetExtendedACLResponseBody` carries session token.
In recent API Go lib changes `eacl.Table` structure carries related session
token.

Write session token of eACL table from sidechain to GetExtendedACL response
body in node's `ContainerService` server.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Leonard Lyubich
a7f71bf167 [#525] v2/container: Return session token and signature in Get
In recent API changes `GetResponseBody` carries session token and signature.
In recent API Go lib changes `Container` structure carries its session
token and signature.

Write session token and signature of container from sidechain to Get
response body in node's `ContainerService` server.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-27 12:03:49 +03:00
Alex Vanin
975ab0dff7 [#561] acl: Add tests for request meta header traverser
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-27 11:21:14 +03:00
Alex Vanin
a025e6250e [#561] acl: Fetch session token from original request meta header
As it explained in previous commit, session token also should
be presented in original meta header but can be omitted in higher
layers.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-27 11:21:14 +03:00
Alex Vanin
d368afffe5 [#561] acl: Fetch bearer token from original request meta header
Request meta headers are organized in a layers, where
upper layers re-sign down layers. Bearer token should be
a part of original meta header and it can be omitted in
upper layers. Therefore we need to traverse over linked list
of meta header to the original meta header to get bearer token.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-27 11:21:14 +03:00
Leonard Lyubich
8448207854 [#505] morph/container: Change parsing of eACL contract method's result
`eACL` method of `Container` contract returns structure with 4 fields (the
4th is a recently added binary session token).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 18:58:25 +03:00
Leonard Lyubich
70f800eb76 [#505] event/container: Change number of items in SetEACL event
Change expected event items to 4 in order to support new binary session
token item. Parsing of the session token will be implemented later.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 18:58:25 +03:00
Leonard Lyubich
62281b91f4 [#505] morph/container: Rename PutEACLBinary to PutEACl in wrapper
There is no need to concretize the type of arguments in the method name.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 18:58:25 +03:00
Leonard Lyubich
0282994f3e [#505] morph/container: Fix setting of SetEACL arguments in wrapper
Extended ACL table and its signature were mixed up.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 18:58:25 +03:00
Leonard Lyubich
6d9cc0dc60 [#505] morph/container: Add SetEACL method arguments
Pass session token (byte array) argument to `SetEACL` method call of
`Container` contract.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 18:58:25 +03:00
Leonard Lyubich
cf40e19985 [#505] morph/container: Change parsing of Get contract method's result
`Get` method of `Container` contract returns structure with 4 fields
(container is the 1st one).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 18:58:25 +03:00
Leonard Lyubich
017fb6abed [#505] ir/container: Use client wrapper with enabled notary calls
Construct wrapper over the Container contract client with `TryNotary` option
since it is required to perform invocations of notary contract.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
8d201f920e [#496] morph/wrappers: Deprecate all notary-dedicated methods
All client wrappers should use underlying static client with enabled notary
work mode in order to produce invocations of notary contract.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
9f122f279a [#496] morph/container: Construct client wrapper in notary mode
Some of the client wrapper's methods should produce notary contract's
invocations. In previous implementation all wrappers provided separate
methods to do it. Since notary and non-notary invocation scenarios have very
different goals, it makes sense to separate the scenarios of using the
client wrapper  at the stage of its creation.

Define `Option` constructor for container client wrapper. Add `TryNotary`
option which enables tries of the notary invocations on underlying static
client. Mark all notary-dedicated methods as deprecated.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
b794aeab63 [#496] morph/client: Construct StaticClient in two work modes
There are two scenarios of invocation of contract methods:
  1. do not invoke notary contract;
  2. try to invoke notary contract if it is enabled in Client.

Taking this into account, `StaticClient` can work in one of the two described
modes. Based on this, it makes sense at the stage of creating `StaticClient`
to fix the call mode, and the further abstract from it.

Define `StaticClientOption` setters of `StaticClient` optional parameters.
Add `TryNotary` constructor of option which enables notary tries. Call
`NotaryInvoke` on underlying `Client` if the option is provided, otherwise
call `Invoke`. Mark `NotaryInvoke` method of `StaticClient` as deprecated.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
97bd85f9e1 [#505] morph/container: Add Put method arguments
Pass session token (byte array) and bind key flag (boolean) arguments to
`Put` method call of `Container` contract.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
fad477df2a [#505] morph/client: Support boolean invocation argument
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
f2778361c8 [#505] event/container: Change number of items in Put event
Change expected event items to 4 in order to support new binary session
token item. Parsing of the session token will be implemented later.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
ac2d347884 [#505] ir/container: Check key-to-owner mapping in key ownership check
Owner identifier can be calculated from public key. If it matches, no
additional verification of key ownership is required.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
369c12b702 [#505] morph/container: Verify signature of deleting container ID
Get all owner keys and verify container ID signature until first success. If
none of the keys match, then prohibit deletion. Thus, the delete operation
is only allowed to the owner of the container. With this approach, a
separate check for key ownership is not required.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
83c27f6e8a [#505] morph/container: Change get container API
Make `Get` method of the wrapper over Container contract's client to
accept binary container ID. Create `Get` function similar to the previous
`Get` variation. Use this function in Container service server in the place
where `Get` method was used.

Additionally implement `AsContainerSource` function which allows
to simply compose container Source interface from the wrapper.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
6310535b3c [#505] ir/container: Check key ownership in Put container handler
Check if new container was signed by its owner, and otherwise prohibit
operation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
5287c194e5 [#505] ir/container: Replace key ownership check into a separate method
Method of key ownership verification is going to be reused by the handlers
of the other events.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
6239d5c0c7 [#505] ir/container: Verify signature in check of Put container event
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
ff715c8037 [#505] ir/container: Change the way to approve container delete event
Call `Delete` method on the wrapper over the Container contract's client
directly from `Processor.approveDeleteContainer`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
e3b4c9eda0 [#505] morph/container: Change delete container API
Make `Delete` method of the wrapper over Container contract's client to accept
two binary parameters: container ID and signature. Create `Delete` function
similar to the previous `Delete` variation, but accepting `Signature`
structure instead of binary signature. Use this function in Container
service server in the place where `Delete` method was used.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
24ad60e1c8 [#505] ir/container: Change the way to approve container put event
Call `Put` method on the wrapper over the Container contract's client
directly from `Processor.approvePutContainer`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
3a5849fadb [#505] morph/container: Do not return ID from Wrapper.Put method
`Put` method of the wrapper over the Container contract's client does not
modify passed binary container, so it makes no sense to calculate the
identifier.

`Put` method returns the error only from now. Function `Put` calculates
identifier itself since it is still required by function signature.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
565ad51b42 [#505] morph/container: Change put container API
Make `Put` method of the wrapper over Container contract's client to accept
three binary parameters: container, key and signature. Create `Put` function
similar to the previous `Put` variation, but accepting `Signature`
structure instead of binary key and signature. Use this function in
Container service server in the place where `Put` method was used.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
9259ae640e [#505] ir/container: Slightly refactor Put and Delete handlers
Split up `processContainerPut` and `processContainerDelete` methods of
container `Processor` into two sub-methods: checking the event and its
assertion.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
a3ac294902 [#505] morph/container: Do not parse public key in Put event parser
Morph event structures defined in `pkg/morph/event`  should only carry
notification values without any additional interpretation. All logical work
should be concentrated on app-side.

Change data type of `Put.PublicKey` return to byte slice. `ParsePut` doesn't
unmarshal public key from now.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
a306eb9ce7 [#505] ir: Process set eACL notifications from Container contract
Add `setEACL` notification event parser (handler) to the return of the
`ListenerParsers` (`ListenerHandlers`) method. Read address of NeoFS ID
contract from `contracts.neofsid` config. Implement `NewNeoFSIDClient`
constructor in `invoke` package and use it in IR application.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
4949f4b064 [#505] ir/container: Refactor ListenerParsers and ListenerHandlers
Pre-allocate slices for a known number of elements. Use single `ParserInfo`
/ `HandlerInfo` variable in order to set Container contracts's address once
and change only values that differ between events.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
372cba1fca [#505] ir/container: Check key ownership during set eACL handling
Use NeoFS ID contract client to check if public key from notification event
is tied to the owner of the container for which the eACL is being changed.
Approve changes coming from the owner of the container only.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
b0271aa478 [#505] ir/container: Verify signature of binary eACL tables
Add signature check to `checkSetEACL` method of the `setEACL` notification
handler in Container processor.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
87d83174d9 [#505] ir/container: Implement simplified handling of SetEACL event
Implement `handleSetEACL` method similar to other handling methods in
Container processor. To begin with, the validation logic is skipped, and all
tables will be sent to the contract. In the future, the necessary checks
will be implemented. Listening for events in the IR node will also be added.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
8c632f6966 [#505] morph/events: Define eACL table change notification
Define `SetEACL` structure of eACL table change notification from Container
contract. Implement function which parses `SetEACL` event structure from
stack item list.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
395fd187ac [#505] morph/container: Add key argument to client wrapper's SetEACL
In recent changes argument list of set eACL call of Container contract
client was extended with binary public key. In the future there will be a
need to pass the table in binary format.

Replace `PutEACL` method with `PutEACLBinary` one which accepts three binary
parameters: eACL table, key and signature. Create `PutEACL` function similar
to the removed method, but accepting `Signature` structure instead of just
a signature. Use this function in Container service server in the place
where `PutEACL` was used.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Leonard Lyubich
02079a4f89 [#505] morph/container: Add public key argument to set eACL operation
Add `SetEACLArgs.SetPublicKey` method which sets binary public key argument
of the "set eACL" contract call. Attach key to the `Invoke` call.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-25 16:35:52 +03:00
Evgenii Stratonikov
b21a6ccede [#496] Remove unused type aliases and errors
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-25 13:41:39 +03:00
Evgenii Stratonikov
2b2b2c2c45 [#496] Use single contract wrapper constructor
There is no need in a separate `New()` or `WrapClient()`

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-25 13:41:39 +03:00
Evgenii Stratonikov
b52751e992 [#496] innerring/invoke: move wrapper structs to separate packages
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-25 13:41:39 +03:00
Evgenii Stratonikov
ca0e3211be [#496] innerring/invoke: remove function wrappers
Use morph.Client directly.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-25 13:41:39 +03:00
Evgenii Stratonikov
b5cda8cd41 [#496] morph/client: fallback to simple invoke in NotaryInvoke
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-25 13:41:39 +03:00
Evgenii Stratonikov
2e31cd34e6 [#502] innerring: synchronize validators on mainnet alphabet update
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-25 11:10:01 +03:00
Angira Kekteeva
7eab752923 [#116] *: Replace pkg policy by neofs-sdk policy
Processing transfer of policy pkg to neofs-sdk repository.

Replace local dependency in neofs-cli.

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-05-24 09:49:15 +03:00
Evgenii Stratonikov
213bbcbf2b [#541] blobstor/fstree: fix a bug in Iterate()
Be able to recover address from the path. Also add tests.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-20 19:24:42 +03:00
Leonard Lyubich
1b51dcf8a4 [#546] engine/inhume: Fix incorrect Inhume behavior for root objects
If object to be inhumed is root we need to continue first traverse over the
shards. In case when several children are stored in different shards,
inhuming object in a single shard leads to appearance of inhumed object in
subsequent selections. Also, any object can be already inhumed, and this
case is equivalent to successful inhume.

Do not fail on `object.ErrAlreadyRemoved` error. Continue first iterating
over shards if we detected root object (`SplitInfoError`).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-20 17:39:38 +03:00
Leonard Lyubich
c5dae76c7d [#546] engine/inhume: Write unit tests
Write unit tests of `StorageEngine.Inhume` which assert that inhumed objects
don't appear in `Select` result.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-20 17:39:38 +03:00
Leonard Lyubich
d1d846cf4d [#546] engine/inhume: Fix incorrect reaction on successful traverse
In previous implementation storage engine false-negatively reacted to a
successful Inhume operation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-20 17:39:38 +03:00
Leonard Lyubich
b587b23e79 [#543] object/search: Fix potential NPE on request forwarder
Request forwarding callback should be called only if set since it is an
optional parameter.

Call `forwarder` function only if it is non-nil.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-20 11:33:24 +03:00
Leonard Lyubich
b2002ffc6d [#543] object/get: Fix potential NPE on request forwarder
Request forwarding callback should be called only if set since it is an
optional parameter. In GetRangeHash forwarder is never set.

Call `forwarder` function only if it is non-nil. Remove no longer needed
`hashOnly` option.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-20 11:33:24 +03:00
Alex Vanin
89b147ebe7 [#521] Recompile protobuf files after transition from pkg/errors
Autogenerated files were accidentally modified. This commit returns
them back to the original.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-19 19:09:35 +03:00
Alex Vanin
b5256ccf4c [#521] Fix issues with transition from pkg/errors pkg
Wrap functions at `pkg/errors` return nil if error argument
was nil. fmt.Errorf always returns error so we need to add
missing error checks to the code.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-19 19:09:35 +03:00
Evgenii Stratonikov
71b87155ef [#521] *: use stdlib errors package
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-19 16:42:54 +03:00
Leonard Lyubich
eb26f92678 [#505] neofsid: Implement wrapper over contract client
Implement wrapper over NeoFS ID contact's client which allows to which
allows you to conveniently interact with the contract. Implement
`AccountKeys` method for getting a list of keys by account ID.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-19 13:55:10 +03:00
Leonard Lyubich
68b469a79d [#505] neofsid: Implement contract client
Implement NeoFS ID contact's client which is responsible for collecting call
arguments and parsing stack items. Initially only key listing method is
supported.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-19 13:55:10 +03:00
Leonard Lyubich
72e2bc8fb6 [#532] object/hash: Fix NPE during request forwarding
In current implementation `Object.GetRangeHash` RPC handler forwards range
requests for payload data. Missing request forwarder of the original request
caused NPE during execution.

Do not call request forwarder if payload range hash is requested.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-18 18:05:18 +03:00
Evgenii Stratonikov
9ea8d11ec3 [#523] go.mod: update go-multiraddr library
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-05-18 15:50:47 +03:00
Alex Vanin
8d8d9eccbd [#522] network: Fix issue with empty endpoint in multiaddr
Before fix `:8080` host address was parsed as `/dns4/tcp/8080`
multiaddress. However such multiaddress is not correct. In this
case `dns4` section should be omitted, but it breaks `manet.DialArgs`.

To solve this issue we explicitly set 0.0.0.0 address.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-18 12:07:00 +03:00
Alex Vanin
16f13bc0a5 [#522] Use HostAddrString as RPC endpoint instead of IPAddrString
To enable TLS support we can't operate with IP addresses directly.
Certificates are issued with host names so it is required to
pass them into RPC client. DNS resolving should be done by transport
layer and not be a part of node. Therefore `IPAddrString` usage is
removed from code.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-18 12:07:00 +03:00
Alex Vanin
2456873473 [#522] network: Add host address stringer
network.Address is a wrapper around multiaddress. Multiaddress
can be resolved to ip netaddr with `IPAddrString` function:
  /dns4/localhost/tcp/8080 => 127.0.0.1:8080
With `HostAddrString` multi address will be transformed to
host address:
  /dns4/localhost/tcp8080 => localhost:8080

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-18 12:07:00 +03:00
Pavel Karpy
9fbc6f5efe [#527] reputation: Fix EigenTrust algorithm
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-18 10:29:25 +03:00
Alex Vanin
e1e4a61ba7 [#520] Update neo-go to pre N3 testnet RC2 compatible version
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-17 10:20:08 +03:00
Leonard Lyubich
80ef4492c1 [#493] node: Get rid of outdated object GC worker
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-13 19:29:10 +03:00
Leonard Lyubich
8d17dab86e [#493] Refactor serving of prometheus and pprof services
Rename `util/profiler` package to `httputil` and refactor it:

  * simplify utility HTTP server;

  * make more generic server's parameters in order to remove `viper.Viper`
    dependency;

  * use single constructor for creating the pprof and prometheus servers;

  * replace `enabled` config value with empty-check of the network address.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-12 10:46:17 +03:00
Leonard Lyubich
db2e43387b [#493] logger: Simplify Logger's parameters
Replace `viper.Viper` parameter with `Prm` structure. Currently only logging
level can be parameterized through string setter.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Leonard Lyubich
2a970b0125 [#493] logger: Do not set global fields in constructor
Remove setting of `app_name` and `app_version` fields in `NewLogger`
constructor. Set these fields in Node and IR application in already
constructed log and remove them from viper .

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Leonard Lyubich
d17526f8ac [#493] logger: Abolish non-usable options
Change logger's encoding to `console`, time encoding to `ISO8601TimeEncoder`
and leave all other options as they are in `zap.NewProductionConfig`.

Remove default values of no longer existing options in node/ir config.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-05-11 12:35:12 +03:00
Alex Vanin
d49bd4b94a Update neofs-api-go to v1.26.0
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-10 09:45:11 +03:00
Pavel Karpy
7319ca5a00 [#504] morph/client: Add more global config value getters
Including:
- typo fix for `StringFromStackItem` error msg
- EigenTrust alpha getter.
- renaming local var in reading uin64 values
from global cfg function

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-06 13:52:43 +03:00
Alex Vanin
dddbf0368c [#486] innerring: Add option to disable only main chain notary support
For N3 Testnet RC2 release inner ring app supports three modes:
- notary enabled in all chains (default),
- notary disabled in all chains,
- notary enabled only in side chain.

All notary related functions are moved to notary.go

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
94a1947482 Fix typo
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
5a167f3991 [#486] innerring: Adopt disabled notary work flow
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
f2562e8c47 [#486] innerring: Use fee provider and notary disabled flag in processors
Processors that use `invoke` package to make chain invocation should provide
fee config and client with enabled or disabled notary support. If notary
support is disabled, then functions from `invoke` package will perform
ordinary method invocation with extra fee.

Processors that use `morph/client` wrappers should check `notaryDisabled`
flag to call corresponding wrapper function.

Netmap processor omits some actions during validator syncronization
if notary is disabled.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
91a1896b8b [#486] innerring: Use fee provider interface in invoke package
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
1f3bb33db8 [#486] innerring: Add fee configuration
When notary disabled, inner ring should be able to
configure extra fee for vote collections inside the
contracts. Previously these values were hardcoded,
however we might want to change them depending on
a environment.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
c4f4381b13 [#486] morph/client: Separate container estimation functions
Inner ring should be able to invoke contract methods both notary
and non notary way.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
7c559aaeaa [#486] morph/client: Add notary enabled check function
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
fd16b5701f [#486] morph/client: Add more global config getters
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
bd65e41257 [#486] innerring: Add notary support in main chain client
With `mainnet.notary_deposit=false` inner ring will ignore
notary deposit \ awaiting routines in the application start,
so it can run on the environments without notary support.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
dd1ace12f7 [#486] morph/client: Add option to setup custom alphabet source in notary
To enable notary support in main chain, notary subsystem should not get
alphabet keys from (main chain) committee. This key fetcher is now
separated and may be overwritten by `WithAlphabetSource` option.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Alex Vanin
c84fe1360e [#486] morph/client: Remove unused fields in notary
With neo-go v0.94.1 verification fee can be calculated
precisely and alphabet keys are fetched from committee
instead of network map contract.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-05-05 12:45:16 +03:00
Pavel Karpy
20b7295087 [#488] reputation/router: Add commentaries and TODO
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
d1db54acf8 [#488] reputation: Change Writer interface
Includes:
- Delete first `ctx` argument in `Write` method.
- Move intermediate Initial trust struct and method
to `calculator` file.
- Change Alpha to 0.1.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
eb74a9cafc [#488] reputation: Add commentaries
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
2c8c9f69c8 [#488] storage/calculator: Make alpha dynamic
Delete reading `alpha` from env var. Cover
retrieving `alpha` behind interface in
intermediate calculator. Add TODO to decide
if it is necessary to receive that param from
global config or not.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
ea781664cf [#488] reputation/eigentrust/calculator: Implement calc wrapper
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
d3c1fc7dda [#488] reputation/local/storage: Add trusting peer
Fix iteration logic after adding trusting peer
in `Trust` struct.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
e69917b27a [#488] reputation/common: Move ServerInfo to common pkg
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
49d477f466 [#488] reputation/managers: Implement route builders
Add implementation of Builder interface for
intermediate trusts. Move all code associated
with managers to `common` directory in `cmd`
and `pkg/services/reputation`

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
e2a1b0e0ee [#488] reputation/eigentrust/storages: Fix args
Change anonymous func arg for `Iterate`
methods of Storages to `PeerTrustsHandler`
type for implementing corresponding
interface.
Implement missing `Iterate` method for
daughter Storage.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00
Pavel Karpy
f6783f4f81 [#488] cmd/reputation: Add DaughterStorage
Add `DaughterStorage` init in main pkg
and start write all received daughters'
trusts to it.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-05-04 17:53:02 +03:00