Commit graph

634 commits

Author SHA1 Message Date
Evgenii Stratonikov
088c894f44 [#733] neofs-adm: read contracts on start
Fail early and reduce disk operations when reading from archive.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-01 15:17:18 +03:00
Evgenii Stratonikov
0b6350d463 [#733] neofs-adm: fetch contracts release from Github
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-01 15:17:18 +03:00
Evgenii Stratonikov
c1e1b65ad9 [#733] neofs-adm: allow to read contract from tar archive
Assume contracts path is tar.gz archive if it is not a directory.
In theory we could support raw tar but it has little value as our
distribution format is tar.gz and this is the primary use-case for this
feature (download and use).

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-11-01 15:17:18 +03:00
Evgenii Stratonikov
49c9dbfba8 [#877] config/engine: use default section for shards
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-28 15:04:14 +03:00
Evgenii Stratonikov
cb9bf00ceb [#877] neofs-node/config: allow to provide default values
In case we have multiple sections with similar structure (e.g. shards)
having defaults in a single place is easier to work with.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-28 15:04:14 +03:00
Evgenii Stratonikov
118c3b3cfe [#940] neofs-adm: print contract version in dump-hashes
```
nns         (unknown):  7061fbd31562664b58f422c3dee4acfd70dba8af
alphabet 0  (v0.12.1):  2392438eb31100857c0f161c66791872b249aa13
alphabet 1  (v0.12.1):  83ef4226d5d6519ca9c99a5de13b1b5ca223a6ad
alphabet 2  (v0.12.1):  6250927beaa9aa5a00171379dcb7187b0c91d17d
alphabet 3  (v0.12.1):  1d6a2519ba41a139b2ced1bfd5013938271a7578
alphabet 4  (v0.12.1):  b65fc7a3c31cf57a90d7eb1c0e9909e4ca69133c
alphabet 5  (v0.12.1):  f95b6ff8cd3b027c9911c18115518ad8c5d2f591
alphabet 6  (v0.12.1):  5b17c579bf56884fd68af152432b3b5aee7aee76
audit       (v0.12.1):  85fe181f4aa3cbdc94023d97c69001ece0730398
balance     (v0.12.1):  dc1ec98d9d0c5f9dfade16144defe08cffc5ca55
container   (v0.12.1):  1b6e68d299b570e1cb7e86eadfdc06aa2e8e0cc5
neofsid     (v0.12.1):  0a64ce753653cc97c0467e1334d9d3678ca8c682
netmap      (v0.12.1):  7c5bdb23e36cc7cce95bf42f3ab9e452c2501df1
reputation  (v0.12.1):  7ad824fd1eeb1565be2cee3889214b9aa605d2fc
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-28 09:35:52 +03:00
Alex Vanin
fbed86da2c [#943] service/object: Remove KeyStore from CommonPrm
There is no point to pass key storage in parameters because
it can be defined on the service level of application.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-27 20:25:25 +03:00
Alex Vanin
2fbdcbdee1 [#943] service/object: Check session token expiration
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-27 20:25:25 +03:00
Alex Vanin
0f598289f6 [#942] neofs-cli: Return endpoint flag to control netmap-snapshot command
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-27 19:37:19 +03:00
Evgenii Stratonikov
41a9261069 [#935] config/example: quote spaces in node.env
```
> source config/example/node.env
config/example/node.env: строка 14: /dns4/s02.neofs.devenv/tcp/8081: Нет такого файла или каталога
config/example/node.env: строка 16: MSK: команда не найдена
config/example/node.env: строка 35: 028f42cfcb74499d7b15b35d9bff260a1c8d27de4f446a627406a382d8961486d6: команда не найдена
config/example/node.env: строка 48: https://rpc2.morph.fs.neo.org:40341: Нет такого файла или каталога
config/example/node.env: строка 49: wss://rpc2.morph.fs.neo.org:40341/ws: Нет такого файла или каталога
config/example/node.env: строка 53: https://rpc2.n3.nspcc.ru:30341: Нет такого файла или каталога
```

New implementation is still basic, but we don't need to parse any env
file, just have a correct example.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-25 11:10:01 +03:00
Pavel Karpy
208d150500 [#937] node: Init node state before gRPC server start
Fix panic when node processes RPC requests
before state initialization is finished.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-22 15:05:00 +03:00
Alex Vanin
eaecc438f8 [#932] neofs-cli: Support ContainerAliasFee global config parameter
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-19 19:15:56 +03:00
Evgenii Stratonikov
751147793f [#927] neofs-adm: provide better error message if endpoint is missing
Note that we cannot mark `--rpc-endpoint` flag as required because
it can be taken from config.

Before:
```
Error: can't create N3 client: failed to get network magic: Post "": unsupported protocol scheme ""
```

Now:
```
Error: can't create N3 client: missing endpoint
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-19 17:47:29 +03:00
Evgenii Stratonikov
f1202a5738 [#927] neofs-adm: fix dump-config command
```
Error: can't to initialize context: can't read alphabet wallets dir: open : no such file or directory
```
It needs neither contracts nor wallets, just N3 client.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-19 17:47:29 +03:00
Evgenii Stratonikov
c93c5e726c [#927] neofs-adm: add fee flags to morph init
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-19 17:47:29 +03:00
Evgenii Stratonikov
26e11a732d [#927] neofs-adm: provide container alias fee on deploy
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-19 17:47:29 +03:00
Pavel Karpy
0866c1fb90 [#920] cli: Rename RPC flag for control service
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-18 12:19:02 +03:00
Pavel Karpy
f0252e00c1 [#920] cli: Do not add rpc flags to util
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-18 12:19:02 +03:00
Evgenii Stratonikov
1a1435be3d [#791] neofs-lens: inspect objects by address
```
> neofs-lens list --path ./blob/blobovnicza/1/1/0
6ay4GfhR9RgN28d5ufg63toPetkYHGcpcW7G3b7QWSek/9ibXu6v4uTwLEcME5vyHev6Zi8LpxqiWTe1dahKpAbb6
H3VBttoLQoknzMDgnVNyLZ8EpkDnQjnaxDr9fnAWeEHA/Hw1titdGh7BrTe2yLotiYbVh9FQaRRNhoNzXTyetpFgt

> neofs-lens inspect --path ./blob/blobovnicza/1/1/0 \
    --address 6ay4GfhR9RgN28d5ufg63toPetkYHGcpcW7G3b7QWSek/9ibXu6v4uTwLEcME5vyHev6Zi8LpxqiWTe1dahKpAbb6 \
    --header --out payload
Version: v2.1
Type: REGULAR
CID: 6ay4GfhR9RgN28d5ufg63toPetkYHGcpcW7G3b7QWSek
ID: 9ibXu6v4uTwLEcME5vyHev6Zi8LpxqiWTe1dahKpAbb6
Owner: 2dokPzmmcLnnR21jQB3qPppTQRgwMNMKEWD
CreatedAt: 0
PayloadSize: 32
Attributes:
  foo: bar

> hexdump -C payload
00000000  ff 6c d4 71 c4 83 f1 5f  b9 0b ad b3 7c 58 21 b6  |.l.q..._....|X!.|
00000010  d9 55 26 a4 1a 95 04 68  0b 4e 7c 8b 76 3a 1b 1d  |.U&....h.N|.v:..|
00000020
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-18 11:55:02 +03:00
Leonard Lyubich
e5e5395830 [#833] cli/netmap: Beautify output of MillisecondsPerBlock parameter
Convert milliseconds to `time.Duration` value and use its stringer in
`netinfo` command.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-10-15 13:56:19 +03:00
Leonard Lyubich
8f9b1fe090 [#833] cli: Write MillisecondsPerBlock and NeoFS network config
Print MillisecondsPerBlock and NeoFS network parameters in `netinfo` command
of `netmap` section.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-10-15 13:56:19 +03:00
Leonard Lyubich
283ccc04b4 [#833] services/netmap: Support new fields of netmap.NetworkInfo
Make the implementation of network info source (Netmap V2 service
dependency) to read MillisecondsPerBlock sidechain parameter and NeoFS
network parameters depending on the client version.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-10-15 13:56:19 +03:00
Pavel Karpy
fae2c21165 [#873] innerring/config: Delete useless defaults
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-15 12:22:25 +03:00
Pavel Karpy
2f2258733b [#873] node: Delete useless notary deposit Amount, Duration envs
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-15 12:22:25 +03:00
Pavel Karpy
8374c3d2f5 [#873] node: Delete useless notary timer
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-15 12:22:25 +03:00
Pavel Karpy
2a3a6cc0ba [#873] node: Use dynamic notary deposit duration
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-15 12:22:25 +03:00
Pavel Karpy
5e816dc01a [#873] node: Make deposits depend on epoch and balances
SN tries to keep 1:3 proportion of GAS and
notary balances respectively. If that proportion
has been messed(means that notary balance is
lower than required) it sends half of its
GAS balance to the notary service.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-15 12:22:25 +03:00
Leonard Lyubich
40a4a7faa2 [#674] object/put: Use pseudo worker pool for local operations
After storage engine started to limit number of PUT operations there is no
need to limited worker pool in Object Put service.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-10-14 10:20:39 +03:00
Leonard Lyubich
2126235f0e [#674] node: Configure size of per-shard worker pools
Add `shard_pool_size` config to `storage` section. Set app default to 20.
Pass the value to `WithShardPoolSize` option.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-10-14 10:20:39 +03:00
Evgenii Stratonikov
f836e7c1dc [#888] neofs-adm: use constant for update methods
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-12 15:50:22 +03:00
Evgenii Stratonikov
31b3c71457 [#888] neofs-adm: allow to work with multiple NNS versions
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-12 15:50:22 +03:00
Evgenii Stratonikov
6571f9214f [#888] neofs-adm: update container contract deploy parameters
It supports NNS domain and zone now.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-12 15:50:22 +03:00
Evgenii Stratonikov
8ddd0aab55 [#888] neofs-adm: set contract hashes on read
Non-alphabet contract are always deployed from committee
so it makes sense to calculate hashes in one place.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-12 15:50:22 +03:00
Evgenii Stratonikov
6bcd4811e2 [#888] neofs-adm: update NNS wrappers
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-12 15:50:22 +03:00
Evgenii Stratonikov
bca41f87af [#888] neofs-adm: deploy NNS contract first
Container contract uses actual NNS interface. This also aleviates some
pain related to update as neofs-adm code itself uses current NNS
version.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-12 15:50:22 +03:00
Evgenii Stratonikov
1177f0ca78 [#888] neofs-adm: deploy contracts with Global scope
Container contract now calls NNS contract to register TLD for nice
names. This must be witnessed by a committee.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-12 15:50:22 +03:00
Evgenii Stratonikov
ec27a96b8c [#888] neofs-adm: dump NNS contract hash
This can be useful for debugging.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-12 15:50:22 +03:00
Alex Vanin
e500bd13a9 [#906] neofs-adm/docs: Fix notes from Fyrchik
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-11 18:33:45 +03:00
Elizaveta Chichindaeva
6c155f62c4 [#906] neofs-adm/docs: Fix language
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
2021-10-11 18:33:45 +03:00
Alex Vanin
d50e8f2e72 [#906] neofs-adm/docs: Add step-by-step deploy guide
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-11 18:33:45 +03:00
Alex Vanin
3c47cff717 [#906] neofs-adm: Add README file
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-11 18:33:45 +03:00
Alex Vanin
0126f18531 [#886] cli: Fix RPC flags for control service
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-08 17:18:18 +03:00
Evgenii Stratonikov
cc377b34d2 [#868] blobstor: initialize (de-)compressors in Init
Do not log in options constructors. Also failure to
initialize compression module (possibly due to invalid options) is
certainly an error deserving proper treatment.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-08 16:01:45 +03:00
Evgenii Stratonikov
00f14d4dcd [#857] config: use size suffixes where possible
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-08 11:26:24 +03:00
Evgenii Stratonikov
a0abf10c8e [#857] config: allow to parse sizes with suffix
The code is taken from viper as `cast` package that we use
doesn't have needed converter.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-10-08 11:26:24 +03:00
Pavel Karpy
4615ff1392 [#886] cli: Fix API flags
Bind API cobra flags(`ttl' and xhdr')
to the Viper.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-10-07 18:54:34 +03:00
Alex Vanin
e41e74b5fc [#872] services/container: Ignore passed routes in load router
Higher level solution for original issue with lost announcements
from single node container.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-05 19:29:27 +03:00
Alex Vanin
f08636c518 [#872] cmd/neofs-node: Don't lose local container size estimations
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-10-05 19:29:27 +03:00
Leonard Lyubich
4d7915a366 [#791] neofs-lens: Implement object listing
Implement `list` command for object listing. Support listing of the objects
from write-cache and blobovnicza.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-10-05 10:51:10 +03:00
Leonard Lyubich
522cbab47c [#791] cmd: Implement application observing storage engine
Implement skeleton of `neofs-lens` app which is going to be used for working
with storage engine's data.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-10-05 10:51:10 +03:00