dstepanov-yadro/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Issues Pull requests Projects Releases Packages Wiki Activity
4384 commits 260 branches 26 tags 20 MiB
Commit graph

1568 commits

Author SHA1 Message Date
Dmitrii Stepanov
92f8810970 [#161] adm: Refactor commands initialization 
Resolve funlen linter for init function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-27 12:42:37 +00:00
Dmitrii Stepanov
b0fefcb21f [#161] adm: Refactor deposit-notary command 
Resolve funlen linter for depositNotary function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-27 12:42:37 +00:00
Dmitrii Stepanov
d62b11f5e7 [#161] adm: Refactor init context creation 
Resolve funlen linter for newInitializeContext function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-27 12:42:37 +00:00
Dmitrii Stepanov
ecbc211016 [#161] adm: Refactor update-contracts command 
Resolve funlen linter for updateContracts method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-27 12:42:37 +00:00
Dmitrii Stepanov
5f7d70c59c [#161] adm: Refactor deploy command 
Resolve funlen linter for deployContractCmd function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-27 12:42:37 +00:00
Dmitrii Stepanov
9534ade716 [#161] adm: Refactor restore-containers command 
Resolve funlen linter for restoreContainers function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-27 12:42:37 +00:00
Dmitrii Stepanov
f2e880465e [#165] ir: Refactor config default init 
Resolve funlen linter for defaultConfiguration function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-27 07:43:56 +00:00
Dmitrii Stepanov
04b3d9d068 [#165] cli: Drop redundant nolint comment 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-27 07:43:56 +00:00
Dmitrii Stepanov
1d5f2dd681 [#165] cli: Refactor collectObjectRelatives 
Resolve funlen linter

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-27 07:43:56 +00:00
Dmitrii Stepanov
c2cf708e0e [#165] cli: Refactor put command 
Resolve funlen linter for putObject method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-27 07:43:56 +00:00
Dmitrii Stepanov
c78e9cc857 [#165] cli: Refactor get command 
Resolve funlen linter for getObject method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-27 07:43:56 +00:00
Pavel Karpy
3c7ed21f74 [#141] Add big object's parts to a lock object's body 
That will prevent part/link object from being removed by both an external
`DELETE` call and the object expiration procedure.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-27 07:28:32 +00:00
Denis Kirillov
342e571d89 [#159] Add handle __SYSTEM__ sys attributes 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-03-22 17:35:20 +03:00
Evgenii Stratonikov
484ac502ca [#156] frostfs-adm: Make dumpBalances() pass linter checks 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-22 07:14:18 +00:00
Dmitrii Stepanov
5059dcc19d [#145] shard-gc: Delete expired objects after locks 
GC deletes expired locks and objects sequentially. Expired locks and
objects are now being deleted concurrently in batches. Added a config
parameter that controls the number of concurrent workers and batch size.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 11:31:08 +03:00
Dmitrii Stepanov
481a1ca6f3 [#148] linter: Add gocognit linter 
Code with high cognitive complexity is hard intuitively to understand

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:54:41 +03:00
Dmitrii Stepanov
97c36ed3ec [#148] linter: Add funlen linter 
Long functions are hard to understand and source of errors

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:54:41 +03:00
Dmitrii Stepanov
cc8ff015b4 [#148] linter: Add containedctx linter 
Context has to be passed as an argument: https://pkg.go.dev/context

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:52:39 +03:00
Artem Tataurov
db3ccd2762 [#128] innerring: Add GAS pouring mechanism for a configurable list of wallets 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-03-20 12:50:05 +00:00
Pavel Karpy
10c419adf0 [#67] node: Fix infinite recursion in SE's wrapper 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-17 18:47:59 +03:00
Pavel Karpy
64bde68fb9 [#67] node: Accept expired locked objects 
Allow replication of any (expired too) locked object. Information about
object locking is considered to be presented on the _container nodes_.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-16 16:22:19 +03:00
Anton Nikiforov
b4582239bf [#130] adm: Fix adding of pub key for group.frostfs at init step 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-03-13 15:05:53 +03:00
Evgenii Stratonikov
861e9ab59a [#83] pre-commit: Add initial configuration 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-13 07:07:29 +00:00
Leonard Lyubich
24a540caa8 [#132] cli/util: Fix basic ACL rendering 
In previous implementation pretty-printer of basic ACL in NeoFS CLI had
mistakes:
 * F-bit was set to `Extendable()` property instead of its inversion
 * B-bits were set to `acl.RoleInnerRing` rights

Make `PrettyPrintTableBACL` to correctly render mentioned bits.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2023-03-10 14:55:45 +03:00
Anton Nikiforov
9929dcf50b [#126] adm: Exclude group.frostfs key from output of the dump-hashes 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-03-09 18:29:36 +03:00
Anton Nikiforov
7486c02bbc [#88] adm: Fix method nnsResolveKey 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-03-09 15:41:28 +03:00
Alex Vanin
20de74a505 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 16:38:26 +03:00
Anton Nikiforov
79ba34714a [#79] cli: Fix panic when setting domain for container 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-03-01 11:11:11 +03:00
Anton Nikiforov
e9f3c24229 [#65] Use strings.Cut instead of strings.Split* where possible 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-28 13:39:14 +03:00
Anton Nikiforov
88e3868f47 [#37] cli: Add nns-name and nns-zone for container create 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-28 13:37:23 +03:00
Alejandro Lopez
73bb590cb1 [#64] node: Use pool_size_local and separate pool for local puts 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-02-22 13:43:19 +03:00
Dmitrii Stepanov
7eaf159a8b [#63] adm: Fix contract wallet creation 
Create contract wallet only by init and update-config command.

Close #63

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-02-22 10:08:37 +03:00
Alejandro Lopez
cb5468abb8 [#66] node: Replace interface{} with any 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-02-21 16:47:07 +03:00
Denis Kirillov
633c5a35de [#44] adm: Support multiple configs 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-02-21 10:00:28 +03:00
Denis Kirillov
5f06232d34 [#44] cli: Support multiple configs 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-02-21 10:00:28 +03:00
Denis Kirillov
bed5a36235 [#44] ir: Support multiple configs 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-02-21 10:00:28 +03:00
Denis Kirillov
87e69b9349 [#44] node: Support multiple configs 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2023-02-21 10:00:28 +03:00
Anton Nikiforov
22f3c7d080 [#1868] Reload config for pprof and metrics on SIGHUP 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-20 13:53:27 +03:00
Evgenii Stratonikov
2b755ddb12 [#2260] node: Use a separate client cache for PUT service 
Currently, under a mixed load one failed PUT can lead to closing
connection for all concurrent GETs. For PUT it does no harm: we have
many other nodes to choose from. For GET we are limited by `REP N`
factor, so in case of failover we can close the connection with the only
node posessing an object, which leads to failing the whole operation.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-20 13:53:27 +03:00
Evgenii Stratonikov
351fdd9fa2 [#2246] node: Allow to configure tombsone lifetime 
Currently, DELETE service sets tombstone expiration epoch to
`current epoch + 5`. This works less than ideal in private networks
where an epoch can be e.g. 10 minutes. In this case, after a node is
unavailable for more than 1 hour, already deleted objects have a chance
to reappear.

After this commit tombstone lifetime can be configured.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-20 13:53:27 +03:00
Evgenii Stratonikov
2272c55c4d [#2238] engine: Add test for component initialization failures 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-20 13:53:27 +03:00
Evgenii Stratonikov
59748b7ae8 [#2238] neofs-node: Gracefully handle shard initialization errors 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-20 13:53:27 +03:00
Evgenii Stratonikov
236c4af615 [#2224] adm: Use native neo-go sessions in dump-hashes 
If we had lots of domains in one zone, `dump-hashes` for all others
can miss some domains, because we need to restrict ourselves with _some_
number.
In this commit we use neo-go sessions by default, with a proper
failback to in-script iterator unwrapping.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-20 13:53:27 +03:00
Anton Nikiforov
c43b2dbac9 [#1465] Add log entry for morph components shutdown action 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-17 12:13:00 +03:00
Anton Nikiforov
85cf1f47ac [#1465] node: Prevent process from killing by systemd when shutting down 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-17 12:13:00 +03:00
Evgenii Stratonikov
3e5bc394b5 [#48] adm: Add initialize test for 1 node 
Single node is used in dev-env, worth testing.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-16 09:07:18 +03:00
Evgenii Stratonikov
1d3669232e [#48] adm: Allow using nonzero magic with local client 
neo-go actor API uses `getVersion` call which returned incorrect magic.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-16 09:07:18 +03:00
Stanislav Bogatyrev
f825cfac78 [#1] Fix project name in comments here and there 
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
 2023-02-06 17:41:14 +03:00
Stanislav Bogatyrev
c6645ef775 [#1] Documentation rebranding 
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
 2023-02-06 17:41:14 +03:00
Stanislav Bogatyrev
1858f11146 [#1] Fix viper env prefix in cli tools 
Changing env prefix and corresponding example config files.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
 2023-02-06 17:41:14 +03:00
Pavel Karpy
73bc1b0b68 [#38] node: Fix linter warnings 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-02-06 17:27:54 +03:00
Anton Nikiforov
515c60bdf4 [#1889] adm: Add command morph netmap-candidates 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-06 17:26:34 +03:00
Anton Nikiforov
ee24815748 [#1889] Move flag --config in cmd/frostfs-adm/internal/commonflags/flags.go 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-06 17:26:34 +03:00
Anton Nikiforov
2b09564355 [#1889] Move netmap.go and exit.go from cli to cmd/internal/common 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-06 17:26:34 +03:00
Anton Nikiforov
5a9d6a09d8 [#8] cli: Set flag mode required for control shards set-mode 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-06 16:45:03 +03:00
Alex Vanin
5d64a354cb [#1] Fix naming in FrostFS Adm help output 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-01-31 11:24:50 +03:00
Alex Vanin
d7e9e2ef9e [#1] Fix naming in FrostFS Lens help output 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-01-31 11:24:50 +03:00
Alex Vanin
d31d8c5335 [#1] Fix naming in FrostFS CLI help output 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-01-31 11:24:50 +03:00
Alex Vanin
406ff1360f [#1] Fix version output for all compiled binaries 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-01-31 11:24:50 +03:00
Evgenii Stratonikov
072a7d61ab [#2176] neofs-node: Do not cache full container list 
We rarely need to list all containers: as one example
we need it for tree service synchronization once per epoch.
Given that cache TTL has the order of block time it makes no sense
to cache the list of all containers.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov
8078af3424 [#2176] neofs-node: Do not invalidate old cache items 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov
e5a40b90b6 [#2176] neofs-node: Peek during cache invalidation 
`Get` needs write mutex and makes our item move to top.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov
93eb72ef44 [#2176] neofs-node: Remove unused field from morphContainerWriter 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Anton Nikiforov
8ee590794f [#1962] cli: common.PrintVerbose prints via cobra.Command.Printf 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Anton Nikiforov
cff4184cd3 [#2128] Add doc for extended headers 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov
d4d2a8c865 [#11] Rename NeoFS to FrostFS in docs 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-12 08:55:47 +03:00
Evgenii Stratonikov
9cb4b4cc17 [#11] Rename neofsid contract to frostfsid 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-12 08:55:47 +03:00
Evgenii Stratonikov
023396e6a4 [#11] adm: Remove NeoFS mentions 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-12 08:55:47 +03:00
Evgenii Stratonikov
19d180b510 [#11] Rename neofs contract to frostfs 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-12 08:55:47 +03:00
Evgenii Stratonikov
fdb0affc31 [#5] frost-node: Used generic cache 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-31 23:04:06 +03:00
Evgenii Stratonikov
a2fe912d1a [#2164] neofs-node: Use a separate client cache for replicator 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov
741482c26f [#2164] neofs-node: Allow to set reconnect interval 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov
d3054e577a [#2165] services/tree: Allow to set custom synchronization interval 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Anton Nikiforov
b7a99c757a [#2097] cli: Clarify help for --expire-at parameter for commands object lock/put and bearer create 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov
e406036629 [#2166] go.mod: Update dependencies 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Anton Nikiforov
33d279a3f2 [#2152] cli: Do not search for LOCK objects when delete container when session provided 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-12-30 11:07:35 +03:00
Pavel Karpy
306609030a [#2159] node: Add tree replication timeout configuration 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-30 11:07:35 +03:00
Pavel Karpy
a30310b2ca [#2145] cli: Do not print "rpc error:" twice 
That error part is already included in GRPC's library.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-30 11:07:35 +03:00
Leonard Lyubich
a68ff67ed8 [#2106] cli: Verify container owner in container delete command 
In NeoFS containers can be removed on behalf of its owner only. To
improve user experience, there is a need to add ownership check to the
removal command of the NeoFS CLI.

Check container ownership in `container delete` command `Run` function.
The check can be skipped by `--force` option.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-12-30 11:07:35 +03:00
Pavel Karpy
d6196c3971 [#2095] cli: Do not panic on object range 
Also, includes range parsing error messages enhancement.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-12-30 11:07:35 +03:00
Pavel Karpy
923f84722a Move to frostfs-node 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-28 15:04:29 +03:00
Alex Vanin
b0eeb0dfcf [#1486] node: Use endless notary deposit for side chain 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-12-02 11:43:33 +03:00
Anton Nikiforov
afabd6be91 [#2038] neofs-cli: Check the sufficiency of the number of nodes in the selector for replicas 
Perform this check on container creation.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-12-01 11:32:27 +03:00
Pavel Karpy
50d28b72c3 [#2075] node, ir: Log notary state with info 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-30 19:26:33 +03:00
Pavel Karpy
0f41c09207 [#2075] node: Do not make notary requests on shutdown 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-30 19:26:33 +03:00
Pavel Karpy
c01d4ecb50 [#2080] morph: Close morph clients 
Could be related to "websocket users limit reached" on the `neo-go` server
side when an SN/IR is rebooting repeatedly.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-30 19:26:33 +03:00
Pavel Karpy
761e82fecd [#2079] cli: Do not panic in object hash 
Sign RPC requests with the provided key.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-30 16:58:52 +03:00
Alex Vanin
8d15c14be6 [#2078] adm: Pack parameters for setPrice invocation 
Contract arguments have to be packed.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-11-30 16:58:52 +03:00
Anton Nikiforov
8a77b4638a [#2012] Add commands neofs-cli acl basic/extended print to show ACL table in human readable format 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-11-30 16:45:37 +03:00
Evgenii Stratonikov
59db66cdb6 [#2091] neofs-adm: Sign blocks properly in tests 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-30 16:45:13 +03:00
Evgenii Stratonikov
6882887bdd [#2091] neofs-adm: Do not query hashes via network 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-30 16:45:13 +03:00
Evgenii Stratonikov
cfefebd5b3 [#2091] neofs-adm: Do not use deprecated methods 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-30 16:45:13 +03:00
Anton Nikiforov
51e886dd67 [#2090] neofs-cli: Remove --header from object get 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-11-30 11:46:02 +03:00
Anton Nikiforov
79130f781e [#2089] neofs-cli: Remove -g option from neofs-cli control ... and neofs-cli container create commands 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-11-30 09:14:50 +03:00
Pavel Karpy
ca3596dd05 [#2104] cli: OID signature in output 
OID signature should always be present in an object; it does not relate
to the object split.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-29 08:52:09 +03:00
Vladimir Domnich
71853348b9 [#2063] morph/client: Support new hash format in morph nns client 
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
 2022-11-19 11:01:04 +03:00
Anton Nikiforov
9a20498f34 [#1940] Removing all trees by container ID if tree ID is empty in pilorama.Forest.TreeDrop 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
426fe97990 [#2026] neofs-adm: Make contract update idempotent 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
7ef0303e13 [#2003] neofs-node: Allow to configure replicator pool size 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
597ed18269 [#2048] neofs-node: Use a separate client cache for client operations 
Background workers can prevent user operations to complete because of
locking in cache.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-19 11:01:04 +03:00
Pavel Karpy
c85bea15ef [#1978] cli: Add children to the static session on DELETE 
If an external session is provided and is not opened by CLI itself, add
children objects to it too. It fixes "not found" errors when removing a big
object with a predefined session (from a file).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00
Pavel Karpy
f48b1de54b [#2029] cli: Fix panic caused by flag redefinition 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00
Pavel Karpy
09a59fef56 [#2029] cli: Allow attaching static session to object hash 
All the other object commands already have it.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00
Pavel Karpy
37f813604f [#2000] cli: Provide a bearer token to spawned HEAD by DELETE 
If a `neofs-cli object delete` operation is performing using a bearer token,
add it to the new `HEAD` requests that collects children OIDs.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00
Pavel Karpy
aab398f4f5 [#1972] node: Do not save objects if node not in a container 
Do not use node's local storage if it is clear that an object will be
removed anyway as a redundant. It requires moving the changing local storage
logic from the validation step to the local target implementation.
It allows performing any relations checks (e.g. object locking) only if a
node is considered as a valid container member and is expected to store
(stored previously) all the helper objects (e.g. `LOCK`, `TOMBSTONE`, etc).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00
Pavel Karpy
a77392e9ce [#1972] cli: Fix lifetime flag in the lock command 
That part of the code was refactored incorrectly.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00
Anton Nikiforov
bb52857b2b [#1338] neofs-cli: Add support to store/restore/delete binary objects 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-11-01 15:30:00 +03:00
Pavel Karpy
1f82c583e3 [#1971] cli: Unify CID and OID flags provision 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-01 15:07:19 +03:00
Pavel Karpy
7daa57d4d2 [#1991] cli: Refine container placement description 
Not to confuse a user by mixing a replication vector number with its copy
number.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-01 15:00:00 +03:00
Anton Nikiforov
51e3810285 [#1689] Add new command morph list-containers in neofs-adm 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-11-01 09:43:02 +03:00
Evgenii Stratonikov
98a152256b [#1992] writecache: Allow to open in NOSYNC mode 
Applicable only to FSTree as we cannot handle corrupted databases
properly yet.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-01 09:42:26 +03:00
Evgenii Stratonikov
b6930f2219 [#1992] neofs-node: Allow to open fstree in NOSYNC mode 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-01 09:42:26 +03:00
Leonard Lyubich
98034005f1 [#1900] node: Fix loosing the "maintenance" status 
In previous implementation node lost maintenance status after successful
switching to it. For example, after some period of time node sent
bootstrap requests with the "online" state instead of "maintenance".

Make `startMaintenance` method to set maintenance status in the
`networkState`.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 18:30:59 +03:00
Leonard Lyubich
db92e96e40 [#1900] node: Refactor bootstrap methods 
In previous implementation bootstrapping state was chosen according to
bool flag which was not convenient.

Create separate method to boostrap with "online" and the current state.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 18:30:59 +03:00
Leonard Lyubich
98ac525272 [#1978] cli/object: Gather all related object in delete session 
Object removal session should reflect all objects related to the
removing one.

Make `OpenSessionViaClient` to gather the split members of the original
object in order to spread the session to them.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 13:49:41 +03:00
Evgenii Stratonikov
280e56f4bb [#1893] neofs-node: Do not fail unless all gRPC endpoints are unavailable 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 12:48:38 +03:00
Leonard Lyubich
3c6daa2995 [#1883] cli/shards: Exclude degraded-read-write mode from help 
The mode is not expected to be used by the user. It is supported only
for developers.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-26 08:24:40 +03:00
Leonard Lyubich
c4a2a283ae [#1883] cli/shards: Make supporting new modes easier 
Add new shard modes as a map entry to automatically parse them in
`set-mode` command. The change also automatically adds new modes to help
message.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-26 08:24:40 +03:00
Leonard Lyubich
7f54462e60 [#1920] cli: Print used RPC timeout in verbose exec 
Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-26 08:20:36 +03:00
Pavel Karpy
9a0824766c [#1713] node: Add Neo RPC switch interval to the config 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-25 17:38:48 +03:00
Pavel Karpy
0631b38da1 [#1713] ir: Add Neo RPC switch interval to the config 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-25 17:38:48 +03:00
Pavel Karpy
d09ec2e3fe [#1615] node/config: Remove unused default constants 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-25 17:38:48 +03:00
Evgenii Stratonikov
4fe1aad30f [#1966] neofs-adm: Fix signature creation for multiactor scenario 
`SignTx` from neo-go appended verification script only if `Scripts`
array was not completely filled.

```
can't persist transactions: transaction b9cd02fb81721c7316c908687301eeaf8ef384e381476632c24a77bdf9ffacd5 failed to verify: witness #1: unknown verification contract
```

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-25 11:22:00 +03:00
Evgenii Stratonikov
2dcf8d51be [#1966] neofs-adm: Set VerifyBlocks: true and fix NetworkFee calculation 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-25 11:22:00 +03:00
Evgenii Stratonikov
6c9423cfb0 [#1959] neofs-adm: Add tests for more commands 
Also, fix `restore-containers`, but it is hard to test it now.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
7c3ac7150d [#1959] neofs-adm: Remove newTempDir function 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
2404a267b9 [#1959] neofs-adm: Move testdata generation to a separate function 
Later we could make it a separate tests an run neofs-adm tests on CI.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
6c21e2cc28 [#1959] neofs-adm: Fix set-config for >4 nodes 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
c87c3d3721 [#1959] neofs-adm: Refactor tests 
Make it easier to test both common deployment scenarios.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
02bea52f40 [#1959] neofs-adm: Allow to test all commands with a local dump 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
8648b102db [#1959] neofs-adm: Dump all blocks in file 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
23622ed283 [#1959] neofs-adm: Check transaction validity in tests 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
942155be6e [#1959] neofs-adm: Fix double multisig 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Evgenii Stratonikov
2484c1d35e [#1959] neofs-adm: Verify blocks and transactions in tests 
Also update to the actual contracts version.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 21:43:23 +03:00
Leonard Lyubich
a4573d5026 [#1933] cli: Document sessions 
Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-24 17:45:22 +03:00
Leonard Lyubich
7b418c36b4 [#1930] services/session: Log calling Create RPC 
There is a need to check if session is opened during system
testing/debug.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-24 17:45:22 +03:00
Leonard Lyubich
8bba490c30 [#1933] cli: Support binary sessions 
There is a need to support NeoFS-binary sessions along with JSON ones in
NeoFS CLI.

Provide generic `common.ReadBinaryOrJSON` functions which tries to
decode NeoFS-binary structure and falls back to JSON format. Use this
function in all places with token reading.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-24 17:45:22 +03:00
Evgenii Stratonikov
04b67f3ba5 [#1949] neofs-adm: Fix epoch tick for >4 node committee 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 13:36:08 +03:00
Leonard Lyubich
60e9de8d63 [#1916] control: Check maintenance allowance on Control server 
In previous implementation turning to maintenance mode using NeoFS CLI
required NeoFS API endpoint. This was not convenient from the user
perspective. It's worth to move networks settings' check to the server
side.

Add `force_maintenance` field to `SetNetmapStatusRequest.Body` message
of Control API. Add `force` flag to `neofs-cli control set-status`
command which sets corresponding field in the requests body if status is
`maintenance`. Force flag is ignored for any other status.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-24 09:20:24 +04:00
Leonard Lyubich
c50603494b [#1916] cli/control: Pre-check maintenance mode in set-status 
Node's maintenance state can be set only if the network allows it.

Make `neofs-cli control set-status` to read current network settings and
check is maintenance state is allowed. Fail the execution if the mode is
not allowed.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-24 09:20:24 +04:00
Pavel Karpy
e1be0180f6 [#1329] tree: Sync trees when a node first time appears in a container 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-21 18:47:56 +03:00
Leonard Lyubich
217b030d20 [#1946] cli/internal: Fix test by wallet non-closing 
Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-21 15:21:59 +03:00
Leonard Lyubich
810087d8b9 [#1946] Bump neo-go module to v0.99.4 
Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-21 15:21:59 +03:00
Evgenii Stratonikov
de055f27c7 [#1942] neofs-adm: Fix multisig checks in tests 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-21 10:16:51 +03:00
Pavel Karpy
aa37078570 [#1902] node: Sync all the trees on bootstrap 
Do not limit synchronization by "system" and "version" tree IDs.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-20 16:17:57 +03:00
Pavel Karpy
1766ca2039 [#1902] tree: Allow synchronize all the container trees 
Add `SynchronizeAllTrees` method of the Tree service. It allows fetching
tree IDs and sync all of them. Share common logic b/w the new method and
the `SynchronizeTree`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-20 16:17:57 +03:00
Pavel Karpy
1805c6606d [#1332] cli: Add tree list 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-20 16:17:57 +03:00
Pavel Karpy
3aa9938b8f [#1902] Update protoc to v3.21.7 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-20 16:17:57 +03:00
Evgenii Stratonikov
0d14ef69f0 [#1922] neofs-node: Allow to go online after maintenance 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-19 10:14:56 +03:00
Evgenii Stratonikov
1980ed968a [#1926] neofs-node: Use writecache.max_object_size from the config 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-18 16:44:28 +03:00
Evgenii Stratonikov
1cb892c579 [#1910] .golangci.yml: Add misspell linker 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-18 15:08:26 +03:00
Evgenii Stratonikov
d772e35aba [#1910] .golangci.yml: Add godot linker 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-18 15:08:26 +03:00
Evgenii Stratonikov
0a1530afa0 [#1917] neofs-cli: Support timeout in control commands 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-18 10:58:10 +03:00
Pavel Karpy
c16a31ef3e [#1329] node: Set undefined netmap status by default 
It allows general usage of `controlNetmapStatus` getter: does not matter of
handler (`NewEpoch` handler of initial setter).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-17 20:53:34 +03:00
Pavel Karpy
f4a3fa2977 [#1329] tree: Sync tree on startup 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-17 20:53:34 +03:00
Leonard Lyubich
d96139b201 [#1900] node: Do not override maintenance state on bootstrap 
In previous implementation node always sent bootstrap requests with
"online" network state. This provoked switching to online of the nodes
under maintenance.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-17 18:59:22 +03:00
Evgenii Stratonikov
8145e79d7f [#1903] neofs-adm: Expand update parameters in MakeCall 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-17 13:37:48 +03:00
Leonard Lyubich
1406d096a2 [#1680] service/object: Fail all operations in maintenance mode 
Storage node should not provide NeoFS Object API service when it is
under maintenance.

Declare `Common` service that unifies behavior of all object operations.
The implementation pre-checks if node is under maintenance and returns
`apistatus.NodeUnderMaintenance` if so. Use `Common` service as a first
logical processor in object service pipeline.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-14 13:54:32 +04:00
Evgenii Stratonikov
2050a623ac [#1896] neofs-node: Set MAINTENANCE status on update 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-14 11:46:19 +03:00
Pavel Karpy
beb1998ed1 [#1332] cli: Print missing nodes message verbosely 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-13 20:01:48 +03:00
Pavel Karpy
6e4f7180fe [#1332] cli: Implement tree add-by-path command 
It is `TreeService.AddNodeByPath` method implementation.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-13 20:01:48 +03:00
Pavel Karpy
3d8349d7f8 [#1332] cli: Implement tree get-by-path command 
It is `TreeService.GetNodeByPath` method implementation.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-13 20:01:48 +03:00
Pavel Karpy
2c02e66939 [#1332] cli: Implement tree add command 
It is `TreeService.Add` method implementation.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-13 20:01:48 +03:00
Evgenii Stratonikov
80d3c7f9d6 [#1892] neofs-adm: Support MaintenanceModeAllowed network setting 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-13 17:45:47 +03:00
Evgenii Stratonikov
f2793060c5 [#1892] neofs-cli: Display maintenance status in netmap nodeinfo 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-13 17:45:47 +03:00
Anton Nikiforov
8714fc42b5 [#1765] Use hex format to print storage node ID 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-10-13 12:55:21 +03:00
Pavel Karpy
b6806ea6b9 [#1770] node: Support logger config rereading 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-12 18:11:05 +03:00
Pavel Karpy
8c75cb1dad [#1770] node: Validate logger config section 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-12 18:11:05 +03:00
Pavel Karpy
d7c7022bbd [#1770] logger: Support runtime level reloading 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-12 18:11:05 +03:00
Pavel Karpy
f037022a7a [#1770] logger: Refactor Logger component 
Make it store its internal `zap.Logger`'s level. Also, make all the
components to accept internal `logger.Logger` instead of `zap.Logger`; it
will simplify future refactor.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-12 18:11:05 +03:00
Leonard Lyubich
9a006ac14f [#1865] cli/object: Do not open remote sessions in reading ops 
It does not make sense to open remote sessions with the storage node in
`get`, `head`, `search`, `range` and `hash` sub-commands of `neofs-cli
object` command.

Do not use NeoFS API `SessionService` in mentioned commands. Decode
object session from JSON file specified `--session` flag. Perform some
sanity checks instantly on CLI side.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-12 13:22:19 +03:00
Evgenii Stratonikov
c0199dee93 [#1867] services/control: Interpret empty list of IDs as all shards 
In neofs-cli the flag is still required, but `all` can be used to
process all shards.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-12 11:20:48 +03:00
Evgenii Stratonikov
b632260995 [#1867] neofs-cli: Support multiple shard IDs in control subcommands 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-12 11:20:48 +03:00
Evgenii Stratonikov
19c0a74e94 [#1867] services/control: Allow to provide multiple shard IDs to some commands 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-12 11:20:48 +03:00
Anton Nikiforov
74d2f2c8d3 [#1854] cli: Unify help messages 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-10-12 09:53:11 +03:00
Pavel Karpy
9e2edfedc6 [#1833] node: Update local node info on nil updates 
If the contract returns a netmap that does not contain the node, update
local `NodeInfo`. It fixes `neofs-cli netmap nodeinfo` command that printed
"state: online" previously.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-12 09:52:51 +03:00
Evgenii Stratonikov
2d43892fc9 [#1840] neofs-node: Use blobstor paths to identify shard 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Evgenii Stratonikov
4b005d3178 [#1840] blobstor: Return info about all components 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Evgenii Stratonikov
9b241e4a17 [#1840] neofs-node: Allow to use mode: disabled in config 
Currently, when removing shard special care must be taken with respect
to shard numbering. `mode: disabled` allows to leave shard configuration
in place while also ignoring it during initialization. This makes
disk replacement much more convenient.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Pavel Karpy
1360273fec [#1856] node: Fix config rereading 
Config was reread from the file only once in two SIGHUPs.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-10 10:35:52 +03:00
Anton Nikiforov
5e493b7f1c [#1704] Add command container nodes to output list of nodes for container, grouped by replica (#1704) 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-10-07 14:58:01 +03:00
Evgenii Stratonikov
90bfe0bad9 [#1826] neofs-cli: Add --timeout flag 
Allow to specify it everywhere `--rpc-endpoint` flag is present.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-07 13:29:15 +03:00
Evgenii Stratonikov
ca8dc872b2 [#1846] neofs-node: Make morph.cache_ttl equal to block time by default 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-07 10:34:55 +03:00
Leonard Lyubich
e54b52ec03 [#1420] object/acl: Fix correlation of object session to request 
In previous implementation of `neofs-node` app object session was not
checked for substitution of the object related to it. Also, for access
checks, the session object was substituted instead of the one from the
request. This, on the one hand, made it possible to inherit the session
from the parent object for authorization for certain actions. On the
other hand, it covered the mentioned object substitution, which is a
critical vulnerability.

Next changes are applied to processing of all Object service requests:
 - check if object session relates to the requested object
 - use requested object in access checks.

Disclosed problem of object context inheritance will be solved within

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-07 10:34:38 +03:00
Pavel Karpy
01ddb3f8e6 [#1770] node: Refactor application configuration 
Split all the fields in `cfg` structure on:
1. `applicationConfiguration`;
2. `internals`; // shared entities for an application work, such as
`context.Context`
3. `shared`; // holder for the shared entities b/w;
4. `cfgXXX`; // configuration for internal services.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-05 14:25:33 +04:00
Leonard Lyubich
713aea06fa [#1681] node: Block only Object service ops under maintenance 
In previous implementation node blocked any operation of local object
storage in maintenance mode. There is a need to perform some storage
operations like data evacuation or restoration.

Do not call block storage engine in maintenance mode. Make all Object
service operations to return `apistatus.NodeUnderMaintenance` error from
each local op.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-05 11:41:49 +03:00
Leonard Lyubich
a7668618c9 [#1680] cli/netmap: Support MAINTENANCE mode 
Make `netmap snapshot` command to print `MAINTENANCE` state of the nodes
with `IsMaintenance()` flag set.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-10-05 11:41:49 +03:00
Leonard Lyubich
eb1fba5182 [#1680] morph/netmap: Adopt to recent contract changes 
After recent Netmap contract changes all read methods which return
network map (either candidates or snapshots) encode node descriptors
into same structure.

Decode `netmap.Node` contract-side structure from the call results.
Replace node state with the value from the `netmap.Node.State` field.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-05 11:41:49 +03:00
Leonard Lyubich
d6c01199c8 [#1680] node: Return MAINTENANCE status in corresponding mode 
Make storage node to return `NODE_UNDER_MAINTENANCE` status
error on each local object operation if the node is in `MAINTENANCE`
mode.

Pass `apistatus.NodeUnderMaintenance` to `StorageEngine.BlockExecution`
during `ControlService.SetNetmapStatus` RPC processing.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-10-05 11:41:49 +03:00
Leonard Lyubich
9fcc80ea38 [#1680] node/control: Send request to switch to MAINTENANCE state 
After recent changes `MAINTENANCE` state is reflected in the Sidechain.
Storage node should switch its state to "maintenance" during serving the
`ControlService.SetNetmapStatus` RPC with correspoding status in the
request.

Call `UpdatePeerState` operation of Netmap contract's client in
`control.NodeState` provider on Storage node app side. The op is
executed if `BlockExecution` on local object storage is succeeded.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-10-05 11:41:49 +03:00
Leonard Lyubich
876e132d22 [#1680] cli/netmap: Print MaintenanceModeAllowed config 
After recent changes network configuration provided by NeoFS storage
nodes contains `MaintenanceModeAllowed` flag. There is
a need to support this value in NeoFS CLI application.

Print `MaintenanceModeAllowed` flag in `netmap netinfo` command.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-10-05 11:41:49 +03:00
Leonard Lyubich
7a3a827d35 [#1680] node/netmap: Support MaintenanceModeAllowed config in RPC 
After recent changes network configuration stored in the Netmap contract
of the NeoFS Sidechain contains `MaintenanceModeAllowed` flag. There is
a need to support this value in Storage node application.

Make `NetmapService.NetworkInfo` RPC server of the storage node to set
`MaintenanceModeAllowed` flag according to corresponding value in the
Sidechain.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-10-05 11:41:49 +03:00
Leonard Lyubich
d2d4191868 [#1680] morph/netmap: Support HomomorphicHashingDisabled config 
`NetworkConfiguration` represents NeoFS network configuration stored in
the Sidechain. In previous implementation the configuration missed flag
of disabled homomorphic hashing.

Add `NetworkConfiguration.HomomorphicHashingDisabled` boolean field.
Decode the field in `Client.ReadNetworkConfiguration` method. Print this
value in `netmap netinfo` command of NeoFS CLI.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-10-05 11:41:49 +03:00
Leonard Lyubich
4c94faac67 [#1797] node/control: Verify states in SetNetmapStatus RPC server 
In previous implementation storage node interpreted all status values
sent in `SetNetmapStatus` RPC as `OFFLINE` except `ONLINE` and
`MAINTENANCE`. This could lead to incorrect processing of new values,
and also didn't allow detection of problems with sending garbage values.

Make implementation of `NodeState` interface used by Control API server
to deny requests with statuses other than protocol-declared enum.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-10-05 11:41:49 +03:00
Anton Nikiforov
8bf82d738b [#1704] cli: Add force option to the command container create 
Validate policy before container creation

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-10-05 09:14:13 +03:00
Evgenii Stratonikov
236414df49 [#1817] network: Allow to use network addresses from the iterator 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-04 12:12:18 +03:00
Pavel Karpy
1e35c12cc1 [#1770] node: Validate config before apply it 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
ab3ef7110e [#1770] node: Reread config files on SIGHUP 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
6fc3268ebf [#1770] node: Support configuration reread on SIGHUP 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
fbd5bc8c38 [#1770] engine: Support configuration reload 
Currently, it only supports changing the compound of the shards.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
91b56ad3e8 [#1770] node: Read storage config in a separate struct 
It will allow rereading config values and will simplify distinguishing them
from the custom values in the `cfg` structure.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
5c69e19016 [#1770] fstree: Depth parameter int -> uint64 
Negative values have no sense. On the other hand it differs from the
blobovnicza's configuration and prevents unification.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
4aa4694152 [#1770] node: Do not init local storage in reading config routine 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
2d7166f8d0 [#1770] shard: Move NewEpoch event routing on SE level 
It will allow dynamic shard management. Closing a shard does not allow
removing event handlers.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
9374823950 [#1770] node: Do not create meta dir on config read 
It is created in `Open` anyway.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Alex Vanin
76cfcc242c [#1820] neofs-adm: Add wallet-address flag in refill command 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-10-03 10:11:38 +03:00
Evgenii Stratonikov
1edc048870 [#1697] services/object: Return proper error if session token is missing 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-30 16:13:01 +03:00
Vladimir Domnich
5284ac53f9 [#1821] neofs-cli: Fix description of cli commands 
1. nodeinfo - updated description of node.
2. eacl - fixed misprint.

Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
 2022-09-28 16:09:24 +03:00
Evgenii Stratonikov
1c62f1b2c4 [#1806] neofs-cli: Add control flush-cache command 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-28 09:28:01 +03:00
Evgenii Stratonikov
36ab1a2472 [#1731] neofs-cli: Make --endpoint flag optional 
Allow to provide it using config file.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-28 09:28:01 +03:00
Leonard Lyubich
93742d37b7 [#1793] node/netmap: Change interface of the latest network map reader 
Replace `ProcessCurrentNetMap` method of `NodeState` interface with
`ReadCurrentNetMap` one with two changes:
 * Replace network map type from NeoFS SDK package with the
   protocol-generated message. This replaces all the business logic to
   the application layer.
 * Support error return. This allows to cover problem node states.

Return an error from `NodeState.ReadCurrentNetMap` method implemeted
through `atomic.Value` if `Store` method has not been called yet.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-27 09:03:17 +03:00
Leonard Lyubich
485a5418d2 [#1793] node: Serve NetmapService.NetmapSnapshot RPC 
There is no more need to serve the same request on Control API.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-27 09:03:17 +03:00
Evgenii Stratonikov
59de20fbba [#1764] neofs-node: Allow to check configuration 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-27 08:57:20 +03:00
Evgenii Stratonikov
0fb5c51ac9 [#1764] neofs-node: Validate config before usage 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-27 08:57:20 +03:00
Evgenii Stratonikov
6f45cc81fc [#1764] neofs-node: Use constants for storage types 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-27 08:57:20 +03:00
Evgenii Stratonikov
9113793688 [#1764] neofs-node: Allow to return error from IterateShards 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-27 08:57:20 +03:00
Evgenii Stratonikov
4e043a801c [#1731] services/control: Replicate object over network in EvacuateShard RPC 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-24 13:47:48 +03:00
Anton Nikiforov
c7f85994e5 [nspcc-dev#1692] cli: Remove --generate-key option in neofs-cli container delete 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-09-20 07:19:46 +04:00
Anton Nikiforov
bb02913c39 [nspcc-dev#1128] cli: Remove WIF and NEP2 support in --wallet argument 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2022-09-19 14:21:37 +03:00
Evgenii Stratonikov
8fc88487db [#1731] neofs-cli: Add control shards evacuate command 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-19 11:33:52 +03:00
Evgenii Stratonikov
4208f7c0cf [#1741] neofs-adm: Use ValidUntilBlock to wait for tx persist 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-15 14:27:16 +03:00
Evgenii Stratonikov
cda8f9df2e [#1786] services/control: Remove WithDeletedObjectHandler option 
Use storage engine directly instead. It is already provided in the
options.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-15 10:28:48 +03:00
Leonard Lyubich
ae655b74f0 *: Exec make fmts 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-14 17:52:49 +04:00
Leonard Lyubich
d25b7e177b [#1189] node/config: Fix PersistentSessionsConfig.Path method's docs 
Method never returns `PersistentStatePathDefault` value, and this is
correct.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-13 21:29:04 +04:00
Pavel Karpy
876e014b5d [#1628] tree: Make ACL checks the same way as for object requests 
1. Do not require a request to be signed by the container owner if a
bearer token is missing
2. Do not check the system role since public requests are not expected to
be signed by IR or a container node (unlike the object requests)

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 10:33:50 +03:00
Pavel Karpy
4f18893d9b [#1628] node: Move common EACLSource interface to core pkg 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 10:33:50 +03:00
Evgenii Stratonikov
d25ec52459 [#1778] neofs-node: Panic if config type is invalid 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-12 17:06:00 +03:00
Evgenii Stratonikov
60b2930417 [#1778] neofs-node: Fix blobovnicza config type 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-12 17:06:00 +03:00
Evgenii Stratonikov
9da5d784cb [#1630] neofs-node: Remove trees on container removal event 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-12 09:54:15 +03:00
Leonard Lyubich
699b534416 [#1763] node/netmap: Write log message about parsed NewEpoch event 
There is a need to have the ability to track NeoFS timeline on storage
nodes. Epochs tick on notifications receipt, so the most obvious way to
know about received epochs is logging the events.

Wrap `morphEvent.ParseNewEpoch` event parser into function which writes
log message about new epoch number.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-12 09:53:05 +04:00
Pavel Karpy
8e6e89aca3 [#1711] amd: Cache committee actor in the init context 
Also simplify the code using `invoker.Invoker`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-08 10:09:44 +03:00
Pavel Karpy
60aa53651b [#1711] adm: Append witnesses only if they are missing 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-08 10:09:44 +03:00
Pavel Karpy
1e57565f6c [#1711] adm: Implement local client's methods 
Implement methods that `actor.Actor` requires:
1. `InvokeFunction` -- wrapper over `InvokeScript`
2. `GetVersion` returns default struct
3. `CalculateNetworkFee` copied and simplified from Neo-go server side.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-08 10:09:44 +03:00
Pavel Karpy
4bd8608b37 [#1711] *: Simplify code using neo-go actors 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-08 10:09:44 +03:00
Evgenii Stratonikov
4354359aed [#1746] network: Set timeout for streaming operations 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-07 16:00:51 +04:00
Alex Vanin
c4c2840b52 [#1632] neofs-adm: Update autogenerated storage node config 
Replace `diable_cache` with `cache_ttl` value.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-09-07 10:34:46 +03:00
Evgenii Stratonikov
e4dcc4d6a9 [#1749] neofs-adm: Register contract hashes in NNS in 2 formats 
NNS proposal describes string N3 address format, however we must also
have hex-string for backwards compatibility.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-07 09:58:05 +03:00
Evgenii Stratonikov
fe0c6db67d [#1749] neofs-adm: Delete NNS records before updating 
In case we already have the record and it is invalid, we should
overwrite it instead of having several conflicting records.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-07 09:58:05 +03:00
Pavel Karpy
fa18100489 [#1714] lens: Add open*COMPONENT* funcs 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-06 18:09:18 +04:00
Pavel Karpy
2132b78aba [#1714] lens: Add meta subcommand 
Includes:
1. `inspect`
2. `list-garbage`
3. `list-graveyard`

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-06 18:09:18 +04:00
Pavel Karpy
adff08ad02 [#1714] lens: Separate commands 
Separate `inspect` and `list` on `write-cache` and `blobovnicza`
subcommands.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-06 18:09:18 +04:00
Evgenii Stratonikov
fef4f6d155 [#1748] neofs-adm: Allow to dump hashes from a custom zone 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-02 21:31:32 +03:00
Evgenii Stratonikov
d4ac5bdb97 [#1748] neofs-adm: Allow to parse hashes in multiple formats 
NEO NNS proposal uses addresses. We should eventually use the same,
but must stay compatible now.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-02 21:31:32 +03:00
Evgenii Stratonikov
4ccacb89e8 [#1748] neofs-adm: Set custom contract hashes as N3 address 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-02 21:31:32 +03:00
Evgenii Stratonikov
11e6f03aa2 [#1748] neofs-adm: Refactor NNS price handling 
Make `nnsRegisterDomainScript` simpler.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-02 21:31:32 +03:00
Evgenii Stratonikov
9290a7e1fe [#1748] neofs-adm: Remove addRecord from nnsRegisterDomainScript 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-02 21:31:32 +03:00
Evgenii Stratonikov
eb5e1121c5 [#1748] neofs-adm: Move dump-hashes command to a separate file 
I have just lost 3 minutes trying to find it.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-02 21:31:32 +03:00
Leonard Lyubich
33143b18a9 [#1652] cli/container: Invert pre-check flag of set-eacl command 
Flag `--pre-check` of `set-eacl` command found to be in demand in most
cases. based on this, it makes sense to add its action to the default
behavior.

Pre-check container extensibility by default. Rename flag to
`--no-precheck` and invert its action.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 16:22:16 +04:00
Leonard Lyubich
46b815c863 [#1652] cli/container: Move eaclPathFrom flag var into the struct 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 16:22:16 +04:00
Leonard Lyubich
491a908af1 [#1652] cli/container: Pre-check ACL extensibility in set-eacl 
Container ACL in NeoFS can be extended only for container in which the
corresponding option is enabled. In previous implementation command
`set-eacl` could hang up on modifying eACL of the non-existent or
non-extendable container. To improve UX, there is a need to pre-check
the availability of `SETEACL` operation.

Add boolean `precheck` flag to `set-eacl` cmd which reads the container
before the actual transaction formation. If flag is set, command fails
on non-extendable container ACL.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 16:22:16 +04:00
Evgenii Stratonikov
177e8e01b1 [#1745] neofs-node: Remove memcache_capacity from the configuration 
It is unused since ddaed283e9 .

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-02 11:59:53 +03:00
Pavel Karpy
62b293b7ab [#1711] adm: Add error context 
To make all the calls in the command to be in the same form.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-02 10:41:18 +03:00
Pavel Karpy
a99474c40e [#1711] adm: Drop deprecated neo-go calls 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-02 10:41:18 +03:00
Leonard Lyubich
ae6ebccd9c [#1632] node/container: Don't add new container to missed cache record 
If container listing cache on node's side is missing (for particular
owner), then updating it as a reaction to successful container creation
leads to potentially invalid cache value for a period of time equivalent
to cache TTL.

Immediately return from `ttlContainerLister.update` method if owner's
container list isn't cached.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 11:29:55 +04:00
Leonard Lyubich
fa3124fc33 [#1632] node: Configure TTL of Sidechain caches 
From now cache TTL can be parameterized in the `neofs-node` app using
`cache_ttl` config key. `disable_cache` value is no longer supported.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 11:29:55 +04:00
Leonard Lyubich
5ce8315cd8 [#1632] node/container: Update LIST cache on successful writes 
In previous implementation storage node responded with the outdated
container list after successful creation/removal up until cache
invalidation due to TTL. In order to decrease the probability of
outdated responses node should update its cache on event receipts.

Implement `ttlContainerLister.update` method which actualizes cached
list of the owner's containers. Make node to call `update` method
on `PutSuccess`/`DeleteSuccess` notifications from the `Container`
contract.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 11:29:55 +04:00
Leonard Lyubich
d15a7d8d3d [#1632] node/container: Update GET cache on successful removal 
In previous implementation storage node responded with the removed
container up until cache invalidation due to TTL. In order to avoid
false-positive responses node should update its cache on `DeleteSuccess`
events.

Make node to call `handleRemoval` method of the container cache which
leads to subsequent `apistatus.ErrContainerNotFound` errors.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 11:29:55 +04:00
Leonard Lyubich
a529149e6f [#1632] node: Cache Sidechain errors along with the values 
In previous implementation failed requests to the Sidechain weren't
cached. It makes sense to cache errors along with the values in order to
decrease potential load spikes onto Sidechain nodes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 11:29:55 +04:00
Leonard Lyubich
7c1babb7d6 [#1632] node: Subscribe on the successful container creations/removals 
There is a need to sync container-related caching mechanism with the
actual Sidechain changes. To do this, node should be able to listen
incoming notifications about container ops.

Define `PutSuccess` / `DeleteSuccess` notification event's parsers.
Subscribe to these events in node app. As initial implementation node
will log event receipts. Later handling is going to be practically
complicated.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-09-02 11:29:55 +04:00
Evgenii Stratonikov
df88afeef3 [#1738] neofs-adm: Clean tx hash list after awaiting 
Do not query application log for the same transaction more than once.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 16:53:00 +03:00
Evgenii Stratonikov
180e5e938f [#1735] go.mod: Update API and SDK 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 12:27:46 +03:00
Evgenii Stratonikov
61f0d85834 [#1727] neofs-adm: Allow to register custom TLD for contracts 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-25 12:31:00 +03:00
Evgenii Stratonikov
ddbe129cb2 [#1729] neofs-adm: Change default domain expiration time 
It is a noop for current contract versions, but will matter in the next
release.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-25 12:31:00 +03:00
Alex Vanin
e68c80df0a [#1728] Don't describe default value twice 
Cobra will mention default value automatically.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-08-24 16:44:44 +03:00
Alex Vanin
31b6af5667 [#1728] Update neofs-adm morph deploy command description 
Add more details about expected file names in contract dir.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2022-08-24 16:44:44 +03:00
Pavel Karpy
b54f34d7df [#1683] adm: Do not deploy custom contract with a group 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-24 13:52:55 +03:00
Pavel Karpy
bbd6b9780f [#1683] adm: Minimize registration price 
Register NNS domain in one TX:
1. Set minimal (`1`) registration price;
2. Register domain;
3. Return registration price back.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-24 13:52:55 +03:00
Evgenii Stratonikov
d57c57010f [#1726] neofs-cli: Truncate file before writing the object 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-23 18:07:12 +03:00
Evgenii Stratonikov
40f1cf36e1 [#1719] neofs-cli: Save object with proper permissions 
`curl` uses 0644, for example.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-23 18:07:12 +03:00
Evgenii Stratonikov
e4370c4129 [#1702] neofs-adm: Set HighPriority attribute for committee transactions 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-23 11:59:23 +03:00
Pavel Karpy
0720d96c9d [#1687] go.mod: Update neofs-sdk-go 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-22 18:59:57 +03:00
Pavel Karpy
30341f2192 [#1687] *: Perform go fmt using go v1.19 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-22 18:59:57 +03:00
Pavel Karpy
37ab26bfa9 [#1296] cli: "ID" -> "OID" 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-22 14:29:50 +04:00
Evgenii Stratonikov
26b4a258e0 [#1523] neofs-node: Refactor configuration 
1. Move compression parameters to the `shard` section.
2. Allow to use multiple sub-storage components in the blobstor.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Pavel Karpy
5139dc9864 [#1706] cli: Do not duplicate payload on SG put 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-19 09:49:47 +04:00
Evgenii Stratonikov
f8b106ac85 [#1684] *: Fix linter warnings 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-15 10:57:31 +03:00
Leonard Lyubich
67af4c89a8 [#1649] cli: Refactor --owner flag definition in container list cmd 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-08-15 10:18:51 +03:00
Leonard Lyubich
7a26b2a57a [#1649] cli: Add option to print attributes in list 
Define `--with-attr` flag of `container list` which makes the command to
request and print user attributes for each found element.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-08-15 10:18:51 +03:00
Leonard Lyubich
2740bf7ee4 [#1649] cli: Add option to print attributes in list-objects 
Define `--with-attr` flag of `container list-objects` which makes the
command to request and print user attributes for each object from the
container.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-08-15 10:18:51 +03:00
a.y.volkov
81684b6f04 [#1665] neofs-lens: Use stdout as default output 
Signed-off-by: a.y.volkov <a.y.volkov@yadro.com>
 2022-08-09 13:23:32 +03:00
Pavel Karpy
9aba0ba512 [#1634] meta: Add epoch state 
It allows performing expiration checks on the stored objects.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-04 16:31:49 +03:00
Pavel Karpy
a97dee008c [#1648] morph: Change endpoint priority order 
The lowest value means the highest priority.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-04 16:11:24 +03:00
Pavel Karpy
8f44335925 [#1651] cli: Fix CID setting in eACL creation 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-04 17:04:42 +04:00
Pavel Karpy
0a60524a9c [#1654] ir, node: Drop deprecated profiler and metrics config sections 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-04 15:38:35 +03:00
Pavel Karpy
713cfa5610 [#1655] cli: Do not force specifying session lifetime 
We have the default value which is also printed in the help messages but any
call that does not specify that flag leads to an error.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-04 15:37:50 +03:00
Evgenii Stratonikov
7ecf54c6df [#1631] neofs-adm: Set group for updated alphabet contracts 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-29 12:09:48 +03:00
Evgenii Stratonikov
d3eac7626d [#1631] neofs-adm: Set group for a proper contract 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-29 12:09:48 +03:00
Evgenii Stratonikov
5d02613c33 [#1633] neofs-adm: Allow to set custom NNS zone for deployed contracts 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
6146df4998 [#1633] neofs-adm: Allow to provide arguments to deployed contracts 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
dd30703a6b [#1633] neofs-adm: Allow to update arbitrary contracts 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
fc06b6e89a [#1633] neofs-adm: Allow to deploy arbitrary contracts 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
38558a3238 [#1633] neofs-adm: Start chain only on happy path 
If the error is returned, it will not be closed properly.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
642d7328ab [#1633] neofs-adm: Fix dump generation in init 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-29 12:02:40 +03:00
Evgenii Stratonikov
07465849a4 [#1637] go.mod: Update neo-go to v0.99.1 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-07-28 20:11:45 +03:00
Pavel Karpy
8162b27264 [#1627] node: Add tree service to the config framework 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-27 12:15:56 +03:00
Evgenii Stratonikov
10c855efef [#1624] go.mod: Update dependencies 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-25 16:56:02 +03:00
Evgenii Stratonikov
6049022f7e [#1612] neofs-cli: Remove RunE functions from commands 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-22 16:18:09 +03:00
Evgenii Stratonikov
a52e7c2c99 [#1612] neofs-cli: Unify expiration flags 
Use `expire-at` everywhere expiration epoch is expected.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-22 16:18:09 +03:00
Evgenii Stratonikov
50e28f22f9 [#1559] shard: Change Degraded mode string representation 
It is a flag, but is a `degraded-read-write` mode for a user.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
4944490ffb [#1559] local_object_storage: Move shard to the DegradedReadOnly mode 
`Degraded` mode can be set by the administrator if needed.
Modifying operations in this mode can lead node into an inconsistent state
because metabase checks such as lock checking are not performed.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
1e786233bf [#1559] local_object_storage: Provide readOnly flag to Open 
We should be able to reopen storage in readonly in runtime.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
339864b720 [#1559] local_object_storage: Move shard.Mode to a separate package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
7410827db8 [#1609] config: Allow to prioritize N3 endpoints 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 16:08:42 +03:00
Evgenii Stratonikov
aed83d1660 [#1609] config: Remove mainchain section 
It is deprecated and unused, there is no need to have it in the
documentation.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 16:08:42 +03:00
Pavel Karpy
a4ed91f9b5 [#1614] node: Update health metric on its change 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-21 16:07:47 +03:00
Pavel Karpy
581a9901c9 [#1614] metrics: Add health metrics 
Also, rename metrics structure since it collects not only storage metrics
now.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-21 16:07:47 +03:00
Evgenii Stratonikov
b549cc314c [#1607] services/tree: allow to customize some parameters 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
2334e94fdb [#1607] neofs-node: Initialize storage before other services 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
a4adb79db7 [#1607] pilorama: Enable tree service explicitly 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
26041f18bf [#1505] pilorama: Allow to customize database parameters 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
3904b0e017 [#1333] neofs-cli: Add control synchronize-tree command 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
bfdd68dcb3 [#1333] services/control: Allow to synchronize local trees 
Do not check that a node indeed belongs to the container, because the
synchronization will fail in this case anyway.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
0dc1a4e336 [#1333] neofs-node: Initialize tree service before the control one 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
5e843a73f9 [#1333] services/control: Return pilorama info in ListShards RPC 
Do not return backend type from the service for now, because memory
backend is expected to vanish.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
36e24f7f78 [#1342] neofs-node: Use the default endpoint for tree service 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
62154da17c [#1324] services/tree: Implement Object Tree Service 
Object Tree Service allows changing trees assotiated with
the container in runtime.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
198beae703 [#1613] neofs-cli: Print lock object ID 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-19 17:15:15 +03:00
Evgenii Stratonikov
5c8d725447 [#1610] neofs-adm: Fix generated storage config 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-19 16:19:41 +03:00
Evgenii Stratonikov
ccba07fe19 [#1608] neofs-adm: Register candidates in a single transaction 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-19 13:28:17 +03:00
Evgenii Stratonikov
59d08b8dae [#1608] neofs-adm: Print info about already performed steps 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-19 13:28:17 +03:00
Evgenii Stratonikov
e5c0338f8c [#1602] neofs-ir: Fix defaults for pprof and prometheus 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-18 19:41:22 +03:00
Evgenii Stratonikov
3a0b06d5da [#1602] config: Rename metrics and profiler sections 
Depracate old names and remove them in the next release.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-18 19:41:22 +03:00
Evgenii Stratonikov
ac46d1a11f [#1602] config: Enable metrics and profiler services with a flag 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-18 19:41:22 +03:00
Evgenii Stratonikov
12dc5c3395 [#1563] config: Replace 127.0.0.1 with localhost 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-18 19:41:22 +03:00
Evgenii Stratonikov
4a8dfd0ce4 [#1606] Remove deprecated functions 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-18 12:41:06 +03:00
Evgenii Stratonikov
6cc180391e [#1606] go.mod: Update neo-go 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-18 12:41:06 +03:00
Pavel Karpy
b1e54dec9f [#1461] cli: Add context to GetCurrentEpoch helper 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-18 11:42:25 +03:00
Pavel Karpy
9c5ef3bab8 [#1461] node: Allow force LOCK removal 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-18 11:42:25 +03:00
Pavel Karpy
de31bb27b6 [#1461] cli: Require LOCK object to be expired 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-18 11:42:25 +03:00
Pavel Karpy
595fc60d72 [#1461] cli: Reorganize common functions 
Make `ParseEpoch` and `GetCurrentEpoch` funcs public and move the first one
to the `common` package.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-18 11:42:25 +03:00
Evgenii Stratonikov
6eb5260562 [#1500] neofs-cli: Check if container contains LOCK objects before removal 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-15 13:25:56 +03:00
Evgenii Stratonikov
c4b86cf1f1 [#1592] Remove debug builds 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-14 10:39:51 +03:00
Evgenii Stratonikov
b8508585a5 [#1592] cmd: Ignore error when fetching a defined flag 
There is an error only if the flag is not defined, such errors should be
caught during debugging.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-14 10:39:51 +03:00
Evgenii Stratonikov
f7d5045876 [#1592] neofs-lens: Add --version flag 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-14 10:39:51 +03:00
Evgenii Stratonikov
5569ff82ef [#1587] Do not print build time in version 
This makes our build more reproducible.
Also print `Component` and `GoVersion`.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-14 10:39:51 +03:00
Pavel Karpy
b3272e7cf1 [#1367] adm: Support homomorphic hashing config 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-13 17:10:06 +03:00
Pavel Karpy
9bd1951ca4 [#1365] adm: Add homomorphic hash disabling option 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-13 17:10:06 +03:00
Pavel Karpy
ae86d2907c [#1365] cli: Calculate homomorphic hash flexibly 
Do not use homomorphic hash in storage group for containers that have
`homomorphic_hashing_disabled` set to `true`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-13 17:10:06 +03:00
Pavel Karpy
68e583f143 [#1365] cli: Sync container with network config 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-13 17:10:06 +03:00
Pavel Karpy
49ba1ef1b6 [#1365] cli: Add SyncContainerSettings func to internal client 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-13 17:10:06 +03:00
Evgenii Stratonikov
1119014e53 [#1579] config: Move storage.default to shard.default 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-13 12:31:42 +03:00
Evgenii Stratonikov
f4a3fc5f11 [#1594] neofs-adm: Support better completion for set-policy 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-13 11:00:35 +03:00
Pavel Karpy
466b9b4273 [#1593] adm: Support changing network config 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-12 21:03:43 +03:00
Pavel Karpy
96a24b5721 [#1490] cli: Require SG members to be unique 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-12 17:35:46 +03:00
Leonard Lyubich
9a11a75b77 [#1570] Upgrade NeoFS SDK Go with changed reputation API 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-07-06 18:21:24 +03:00
Pavel Karpy
12d41918e6 [#1402] cli: Support storage group expiration 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-06 09:56:26 +03:00
Pavel Karpy
cfca88fc5c [#1402] cli: object <-> SG conversion via SDK 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-06 09:56:26 +03:00
Pavel Karpy
bc5882fc89 [#1402] cli: Add cid flag constant 
Also, move common flag constants in `root` file of the `storagegroup`
package.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-07-06 09:56:26 +03:00
Leonard Lyubich
c165d1a9b5 [#1556] Upgrade NeoFS SDK Go with changed container API 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-07-05 11:26:06 +03:00
Evgenii Stratonikov
ad81b6c90a [#1555] neofs-adm: Remove trailing dots from help messages 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-06-28 13:46:02 +03:00