Compare commits

...

317 commits

Author SHA1 Message Date
9aeea0b974 [#153] ci: Minor pipeline fixes
- We can skip full pre-commit run
- On a very slow agent golangci run may take up to 10 minutes

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-03-21 09:33:45 +00:00
Pavel Karpy
9a4f40626c [#128] IR: Do not try to emit GAS to nobody
Fix sending GAS to an empty extra wallets receivers list. Also, send GAS to
extra wallets even if netmap is empty.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-03-21 08:59:46 +00:00
7a31988a36 [#145] docs: Add expired object collector params
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 11:31:08 +03:00
5059dcc19d [#145] shard-gc: Delete expired objects after locks
GC deletes expired locks and objects sequentially. Expired locks and
objects are now being deleted concurrently in batches. Added a config
parameter that controls the number of concurrent workers and batch size.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 11:31:08 +03:00
6c4a1699ef [#145] shard-gc: Expired locked unit test
Added unit test that verifies that GC deletes expired
locked objects in one epoch.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 11:31:08 +03:00
Pavel Karpy
9cd8f7cea0 [#152] IR: Process empty basic incomes
If network is not configured for basic income earnings, do not distribute
GAS by the Alphabet nodes.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-03-21 06:59:33 +00:00
44b86bac5a [#148] linter: Add contextcheck linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 09:54:41 +03:00
481a1ca6f3 [#148] linter: Add gocognit linter
Code with high cognitive complexity is hard intuitively to understand

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 09:54:41 +03:00
97c36ed3ec [#148] linter: Add funlen linter
Long functions are hard to understand and source of errors

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 09:54:41 +03:00
cc8ff015b4 [#148] linter: Add containedctx linter
Context has to be passed as an argument: https://pkg.go.dev/context

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 09:52:39 +03:00
2dc86058c3 [#148] memstore: Drop space line
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 09:52:39 +03:00
573d920821 [#149] Use custom image and kludges for node
Until #139 is fixed, we can't use root inside Docker container running
CI, but Woodpecker CI can't run non-root containers until they fix
https://github.com/woodpecker-ci/woodpecker/issues/1077, hence we use
temporary kludges with custom image and manual permissions in pipelines.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-03-20 18:37:37 +03:00
d64fb887ff [#149] Reorder pre-commit hooks
Minor changes to see what fails first

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-03-20 18:37:34 +03:00
392be818e5 [#149] Fix gitlint regex to match our policy
In our policy we mark commits not having a PR/Issue yet with a `[#XX]`
reference to be replaced after PR creation.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-03-20 18:37:27 +03:00
db3ccd2762 [#128] innerring: Add GAS pouring mechanism for a configurable list of wallets
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-03-20 12:50:05 +00:00
Pavel Karpy
abd21f8099 [#136] Revert "[#2260] services/object: Do not assemble object with TTL=1"
This reverts commit 2567f8020e. It assumes
that assembling logic could break some failover scenarios if request
forwarding is done. However, it also breaks requesting big objects via a
non-container node with TTL=2. Failover has been rechecked without that
commit and no problems were found. Any (if found) other bugs related to
the forwarding and object assembling must be solved more carefully.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-03-20 10:52:18 +03:00
Pavel Karpy
10c419adf0 [#67] node: Fix infinite recursion in SE's wrapper
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-03-17 18:47:59 +03:00
b70caa216b [#144] Don't copy cache inside Docker environment
May make docker builds not so clean.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-03-17 10:36:50 +03:00
Pavel Karpy
64bde68fb9 [#67] node: Accept expired locked objects
Allow replication of any (expired too) locked object. Information about
object locking is considered to be presented on the _container nodes_.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-03-16 16:22:19 +03:00
Pavel Karpy
f006f3b342 [#67] node: Make engine's IsLocked public
It will allow reusing that method in expiration checks.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-03-16 16:20:45 +03:00
22be532cbd object/put: Persist session token till the end of a session
Previously a token could've expired in the middle of an object.PUT
stream, leading to upload being interrupted. This is bad, because user
doesn't always now what is the right values for the session token
lifetime. More than that, setting it to a very high value will
eventually blow up the session token database.

In this commit we read the session token once and reuse it for the whole
stream duration.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-16 06:45:50 +00:00
724debfdcd [#81] node: Add basic read/write benchmarks for substorages
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-03-15 16:37:04 +00:00
b1c165a93b [#83] Fix shellcheck pre-commit action
Original shellcheck action requires Docker to run and it's not always
available, especially inside Docker containers. Replacing it with python
wrapper to simplify usage with Docker-based CI systems like Drone and
WoodpeckerCI.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-03-15 12:24:41 +00:00
ac0a278a05 [#85] get-service: Drop unused assemble flag
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-15 09:19:45 +03:00
b8e93d4c08 [#85] get-service: Use assembler to assemble LOB
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-15 09:19:45 +03:00
07de839f18 [#85] get-service: Fix corrupted chain logic
Should return an error in case of a broken LOB reference chain.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-15 09:19:45 +03:00
2886b1581b [#85] get-service: Add unit tests
Add unit tests to cover all assemble statements

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-15 09:19:45 +03:00
8b9e40a848 [#85] get-service: Add assembler
Extract assemble logic to assembler

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-15 09:19:45 +03:00
b4582239bf [#130] adm: Fix adding of pub key for group.frostfs at init step
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-03-13 15:05:53 +03:00
4e244686cf [#83] Makefile fixes for pre-commit
Add make targets to simplify pre-commit setup for individual developers.

Signed-off-by: Stanislav Bogatyrev <realloc@realloc.spb.ru>
2023-03-13 11:39:18 +00:00
6cd806f998 [#82] services/tree: Save last synchronized height in a persistent storage
Remember the last synchronized height and use it after service restart.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-13 11:25:44 +00:00
3e6fd4c611 [#82] pilorama: Allow to store last sync height
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-13 11:25:44 +00:00
5ae4446280 [#50] ir: Add Health status
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-03-13 11:24:42 +00:00
5890cd4d7d [#50] ir: Fix config property name for prometheus
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-03-13 11:24:42 +00:00
365adb4ebd [#133] .github: Restore logo.svg
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-13 10:39:12 +03:00
bce5827f64 [#83] pre-commit: Add shellcheck hook
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-13 07:07:30 +00:00
05471d3827 [#83] util/autocomplete: Fix deprecated warning
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-13 07:07:30 +00:00
8226d49376 [#83] pre-commit: Add gitlint hook
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-13 07:07:30 +00:00
0893689c6a [#83] pre-commit: Add golangci-lint hook
Skip deprecated warning for now, adopting new neo-go API will be done in
another task.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-13 07:07:29 +00:00
a4931ea4c7 [#83] .github: Remove CODEOWNERS and actions
Issue templates are still supported by Gitea:
https://docs.gitea.io/en-us/issue-pull-request-templates/ .

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-13 07:07:29 +00:00
861e9ab59a [#83] pre-commit: Add initial configuration
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-13 07:07:29 +00:00
Leonard Lyubich
24a540caa8 [#132] cli/util: Fix basic ACL rendering
In previous implementation pretty-printer of basic ACL in NeoFS CLI had
mistakes:
 * F-bit was set to `Extendable()` property instead of its inversion
 * B-bits were set to `acl.RoleInnerRing` rights

Make `PrettyPrintTableBACL` to correctly render mentioned bits.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
2023-03-10 14:55:45 +03:00
6226c3ba86 [#129] policer: Use safer defaults
If `processNodes` exits earlier for some reason, `needLocalCopy` could
be false.
See https://github.com/nspcc-dev/neofs-node/issues/2267

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-10 10:59:15 +00:00
f2250a316f [#129] tree: Do not remove tree if the netmap is empty
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-10 10:59:15 +00:00
9929dcf50b [#126] adm: Exclude group.frostfs key from output of the dump-hashes
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-03-09 18:29:36 +03:00
7486c02bbc [#88] adm: Fix method nnsResolveKey
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-03-09 15:41:28 +03:00
Pavel Karpy
f1f3c80dbf [#32] node: Init write-cache asynchronously
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-03-09 11:07:33 +00:00
Pavel Karpy
381e363a8b [#32] node: Always close general components after testing
It will prevent test fails with `-race` flag on components that have
background processes and make some actions on test framework.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-03-09 11:07:33 +00:00
20de74a505 Rename package name
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 16:38:26 +03:00
79ba34714a [#79] cli: Fix panic when setting domain for container
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-03-01 11:11:11 +03:00
6d4f48f37a [TrueCloudLab#78] .github: Fix CODEOWNERS
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-28 16:44:50 +03:00
e9f3c24229 [#65] Use strings.Cut instead of strings.Split* where possible
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-28 13:39:14 +03:00
88e3868f47 [#37] cli: Add nns-name and nns-zone for container create
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-28 13:37:23 +03:00
6925fb4c59 [TrueCloudLab/hrw#2] node: Use typed HRW methods
Update HRW lib and use typed HRW methods to sort shards and nodes

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-02-28 13:36:25 +03:00
c3a7039801 [TrueCloudLab/hrw#2] node: Optimize shard hash
Compute shard hash only once

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-02-28 13:36:25 +03:00
a1ab25b33e [#72] .github: Fix CODEOWNERS
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-22 16:29:58 +03:00
73bb590cb1 [#64] node: Use pool_size_local and separate pool for local puts
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-02-22 13:43:19 +03:00
7eaf159a8b [#63] adm: Fix contract wallet creation
Create contract wallet only by init and update-config command.

Close #63

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-02-22 10:08:37 +03:00
cb5468abb8 [#66] node: Replace interface{} with any
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-02-21 16:47:07 +03:00
3d873237d5 [#44] Update Changelog
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-21 10:00:28 +03:00
633c5a35de [#44] adm: Support multiple configs
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-21 10:00:28 +03:00
5f06232d34 [#44] cli: Support multiple configs
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-21 10:00:28 +03:00
bed5a36235 [#44] ir: Support multiple configs
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-21 10:00:28 +03:00
87e69b9349 [#44] node: Support multiple configs
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-21 10:00:28 +03:00
Pavel Karpy
337049b2ce [#56] node: Allow reading expired locked object
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-02-21 09:56:57 +03:00
Pavel Karpy
3beef10f89 [#61] node: Do not fetch missing objects
If an object is missing in a `meta`, shard should not look for it in
a `blobstor`.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-02-20 14:47:38 +03:00
5303736acd [#62] Fix CHANGELOG.md
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-02-20 14:25:21 +03:00
22f3c7d080 [#1868] Reload config for pprof and metrics on SIGHUP
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-20 13:53:27 +03:00
2b755ddb12 [#2260] node: Use a separate client cache for PUT service
Currently, under a mixed load one failed PUT can lead to closing
connection for all concurrent GETs. For PUT it does no harm: we have
many other nodes to choose from. For GET we are limited by `REP N`
factor, so in case of failover we can close the connection with the only
node posessing an object, which leads to failing the whole operation.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
0b61a3c961 [#2260] network/cache: Ignore clients only on Dial errors
The problem is that accidental timeout errors can make us to ignore
other nodes for some time. The primary purpose of the whole ignore
mechanism is not to degrade in case of failover. For this case,
closing connection and limiting the amount of dials is enough.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
bf1e59bb83 [#2260] network/cache: Ignore context cancelled errors
Timeouts on client side should node affect inter-node communication.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
2567f8020e [#2260] services/object: Do not assemble object with TTL=1
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
d1d123d180 [#2234] writecache: Fix possible panic in initFlushMarks
In case we have many small objects in the write-cache, `indices` should
not be reused between iterations.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
315141dc2c [#2252] fstree: Allow concurrent writes
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
Pavel Karpy
b422ac9f94 [#2164] node: Fix multi-client error reporting
Missing `ReportError` method did not allow casing multi-client interface to
`errorReporter` interface and dropping broken connections.
`replicationClient` embeds that interface, and it is widely used across
node's code. Embedded interface does not allow casting its parent structure
to `errorReporter` and breaks multi client error reporting logic.
Multi-client scheme is extremely hard to maintain, it makes unpredictable
casts and does not allow tracking code flow, so it will be refactored in the
future anyway.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-02-20 13:53:27 +03:00
Pavel Karpy
95ee905861 [#2244] node: Fix subscriptions lock
Subscribing without async listening could lead to a dead-lock in the
`neo-go` client.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-02-20 13:53:27 +03:00
Pavel Karpy
07ec51ea60 [#2244] node: Add object address to WC's operations
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-02-20 13:53:27 +03:00
Pavel Karpy
dbbbef9ddb [#2244] node: Update expired storage ID by WC
Previously, node could get an "infinite" small object: it could be expired
and thus could not be flushed (update its storage ID) to metabase => could
not be marked as flushed => node never removes such object and repeat all
the cycle one more time. If object exists and is not marked with GC (meta
returns `ErrObjectIsExpired`, not `ObjectNotFound` and not
`ObjectAlreadyRemoved`), its ID is safe to update _in the same_ bbolt
transaction.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-02-20 13:53:27 +03:00
351fdd9fa2 [#2246] node: Allow to configure tombsone lifetime
Currently, DELETE service sets tombstone expiration epoch to
`current epoch + 5`. This works less than ideal in private networks
where an epoch can be e.g. 10 minutes. In this case, after a node is
unavailable for more than 1 hour, already deleted objects have a chance
to reappear.

After this commit tombstone lifetime can be configured.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
6fd88a036f [#2241] metrics: Fix request count metrics names
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
2272c55c4d [#2238] engine: Add test for component initialization failures
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
5cb2c5ae62 [#2238] engine: Add test for component initialization failures
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
59748b7ae8 [#2238] neofs-node: Gracefully handle shard initialization errors
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
427fe276f2 [#2238] shard: Try closing all components
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
c53903ccd0 [#2238] engine: Make Open and Init similar
1. Both could initialize shards in parallel.
2. Both should close shards after an error.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
e0309e398c [#2239] writecache: Fix possible deadlock
LRU `Peek`/`Contains` take LRU mutex _inside_ of a `View` transaction.
`View` transaction itself takes `mmapLock` [1], which is lifted after tx
finishes (in `tx.Commit()` -> `tx.close()` -> `tx.db.removeTx`)

When we evict items from LRU cache mutex order is different:
first we take LRU mutex and then execute `Batch` which _does_ take
`mmapLock` in case we need to remap. Thus the deadlock.

[1] 8f4a7e1f92/db.go (L708)

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
58367e4df6 [#2232] pilorama: Merge in-queue batches
To achieve high performance we must choose proper values for both
batch size and delay. For user operations we want to set low delay.
However it would prevent tree synchronization operations to form big
enough batches. For these operations, batching gives the most benefit
not only in terms of on-CPU execution cost, but also by speeding up
transaction persist (`fsync`).
In this commit we try merging batches that are already
_triggered_, but not yet _started to execute_. This way we can still
query batches for execution after the provided delay while also allowing
multiple formed batches to execute faster.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
236c4af615 [#2224] adm: Use native neo-go sessions in dump-hashes
If we had lots of domains in one zone, `dump-hashes` for all others
can miss some domains, because we need to restrict ourselves with _some_
number.
In this commit we use neo-go sessions by default, with a proper
failback to in-script iterator unwrapping.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
Pavel Karpy
40822adb51 [#2213] node: Do not return object expired object
"Object is expired" means that object is presented in `meta` but it is not
`ObjectNotFound` error. Previous implementation made `shard` search for an
object without `meta` which was an error.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-02-20 13:53:27 +03:00
Roman Khimov
ad93d4db7c CHANGELOG: add more fancy glyphs
How could you forget adding it?

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-02-20 13:53:27 +03:00
Roman Khimov
a6f071d66f CHANGELOG: fix whitespacing errors
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2023-02-20 13:53:27 +03:00
9afe86ba3e [#2212] morph: Fix subscription restoration
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
c43b2dbac9 [#1465] Add log entry for morph components shutdown action
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-17 12:13:00 +03:00
85cf1f47ac [#1465] node: Prevent process from killing by systemd when shutting down
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-17 12:13:00 +03:00
362f24953a [#47] shard: Switch container size metric from physical to logical capacity
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-02-17 12:03:42 +03:00
Pavel Karpy
901d62567d [#57] node: Broadcast link objects
It boosts object assembling by an _average_ container node.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-02-17 11:58:27 +03:00
Aleksey Pastukhov
269a4e9b50 [#53] Fix dirty in version
Signed-off-by: Aleksey Pastukhov <a.pastukhov@yadro.com>
2023-02-16 11:10:43 +03:00
3e5bc394b5 [#48] adm: Add initialize test for 1 node
Single node is used in dev-env, worth testing.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-16 09:07:18 +03:00
1d3669232e [#48] adm: Allow using nonzero magic with local client
neo-go actor API uses `getVersion` call which returned incorrect magic.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-16 09:07:18 +03:00
204cd3a11c [#31] fstree: Optimize treePath
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-10 12:49:31 +03:00
dee4498c1e [#31] fstree: Do not check for a file existence twice
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-10 12:49:31 +03:00
abbecf49d6 [#31] fstree: Speedup string-to-address conversion
```
name                  old time/op    new time/op    delta
_addressFromString-8    1.25µs ±30%    1.02µs ± 6%  -18.49%  (p=0.000 n=9+9)

name                  old alloc/op   new alloc/op   delta
_addressFromString-8      352B ± 0%      256B ± 0%  -27.27%  (p=0.000 n=9+10)

name                  old allocs/op  new allocs/op  delta
_addressFromString-8      6.00 ± 0%      4.00 ± 0%  -33.33%  (p=0.000
n=10+10)
```

Also, assure compiler that `s` doesn't escape:
Before this commit:
```
./fstree.go:74:24: leaking param: s
./fstree.go:90:6: moved to heap: addr
```

After this commit:
```
./fstree.go:74:24: s does not escape
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-10 12:49:31 +03:00
ab21d90cfb [#1794] shard: Add increasing case for the payload size metric
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-02-09 13:30:23 +03:00
5ffa826897 [#1] Update Changelog
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-02-06 17:41:14 +03:00
cb016d53a6 [#1] Fix comments and error messages
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-02-06 17:41:14 +03:00
c761a95eef [#1] Fix project name in control service
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-02-06 17:41:14 +03:00
f825cfac78 [#1] Fix project name in comments here and there
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-02-06 17:41:14 +03:00
c6645ef775 [#1] Documentation rebranding
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-02-06 17:41:14 +03:00
488eece25f [#1] Fix Debian package references to old upstream
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-02-06 17:41:14 +03:00
1cedd446bb [#1] Fix sample configs
- Update sample configs to match unit tests
- Remove Docker container for N3 testnet
  Will return with updates when FrostFS enters N3

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-02-06 17:41:14 +03:00
1858f11146 [#1] Fix viper env prefix in cli tools
Changing env prefix and corresponding example config files.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-02-06 17:41:14 +03:00
Pavel Karpy
73bc1b0b68 [#38] node: Fix linter warnings
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-02-06 17:27:54 +03:00
515c60bdf4 [#1889] adm: Add command morph netmap-candidates
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-06 17:26:34 +03:00
ee24815748 [#1889] Move flag --config in cmd/frostfs-adm/internal/commonflags/flags.go
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-06 17:26:34 +03:00
2b09564355 [#1889] Move netmap.go and exit.go from cli to cmd/internal/common
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-06 17:26:34 +03:00
5a9d6a09d8 [#8] cli: Set flag mode required for control shards set-mode
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-06 16:45:03 +03:00
2540598779 [#36] Packaging: allow any version
Also updated changelog

Signed-off-by: Dmitriy Zabolotskiy <d.zabolotskiy@yadro.com>
2023-01-31 16:33:32 +03:00
5d64a354cb [#1] Fix naming in FrostFS Adm help output
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-01-31 11:24:50 +03:00
d7e9e2ef9e [#1] Fix naming in FrostFS Lens help output
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-01-31 11:24:50 +03:00
d31d8c5335 [#1] Fix naming in FrostFS CLI help output
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-01-31 11:24:50 +03:00
406ff1360f [#1] Fix version output for all compiled binaries
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-01-31 11:24:50 +03:00
Pavel Karpy
89a0266f5e [#1794] metrics: Track physical object capacity per shard
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-01-26 20:06:28 +03:00
Evgenii Stratonikov
9513f163aa [#2116] metrics: Track physical object capacity in the container
Currently we track based on `PayloadSize`, because it is already stored
in the metabase and it is easier to calculate without slowing down the
whole system.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-01-26 20:06:28 +03:00
d65a95a2c6 [#28] pilorama: Remove LogMove struct
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
46c62be7e8 [#28] Fix linter issues
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
072a7d61ab [#2176] neofs-node: Do not cache full container list
We rarely need to list all containers: as one example
we need it for tree service synchronization once per epoch.
Given that cache TTL has the order of block time it makes no sense
to cache the list of all containers.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
8078af3424 [#2176] neofs-node: Do not invalidate old cache items
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
e5a40b90b6 [#2176] neofs-node: Peek during cache invalidation
`Get` needs write mutex and makes our item move to top.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
93eb72ef44 [#2176] neofs-node: Remove unused field from morphContainerWriter
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
8ee590794f [#1962] cli: common.PrintVerbose prints via cobra.Command.Printf
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
cff4184cd3 [#2128] Add doc for extended headers
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
c72576e72f [#2208] engine: Log time-consuming shard operations
Currently the only way to tell whether `evacuate/set-mode` is finished
is to set a very big timeout and _hope_ that the operation will finish.
In this commit we add INFO logs for such operations which should
simplify the life of an administrator.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
87f0e3ea25 [#2208] fstree: Rename file after write
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
792319a044 [#2208] fstree: Remove file if there was an error during write
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
67c97c6804 [#2210] services/tree: Drop messages not in queue
Currently, under high load clients are blocked on channel send
and the number of goroutines can increase indefinitely.
In this commit we drop replication messages if send/recv queue is full
and rely on a background synchronization.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
25d5995cef [#2210] pilorama: Allocate bucket name outside of batches
1. Reduce allocations inside transactions.
2. Do not encode container ID to string: it allocates a lot and takes more
space.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
165a600624 [#2210] pilorama: Reduce the amount of keys per node
Under high load we are limited by the _amount_ of keys we need to update
in a single transaction. In this commit we try storing all state
with a single key.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
Pavel Karpy
64a5294b27 [#2200] shard: Do not fetch big objects from blobovniczas
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
Pavel Karpy
91757329ae [#2200] shard: Fix blobstor obj fetching
In the previous implementation any non-nil error that preceded object
fetching from blobstor led to iterating over every storage (in other words,
no storage ID information was taken into account). Now storage ID is
skipped only if metabase (storage ID source) returns any error.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
Pavel Karpy
cf1a91a758 [#2206] blobovnicza: Use Latin letters in the code
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
Pavel Karpy
c33ad3c474 [#2164] node: Use reconnect_interval from config
Not always the default one.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
0d8366f475 [#2207] object/acl: Return status error for expired session token
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
6451f019d2 [#2203] shard: Do not panic in Close after unsuccessful Init
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
Evgenii Stratonikov
6efa93be0a [#1621] services/tree: Return Apply result asyncronously
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
Evgenii Stratonikov
ac81c70c09 [#1621] pilorama: Batch related operations
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
9009612a82 [#2198] blobovniczatree: Properly handle concurrent active blobovnicza update
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
cedbd380f2 [#2197] pilorama: Close database in degraded mode
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
Pavel Karpy
1d21b1e3e8 [#1978] node: Do not drop clients on split errors
After the reconnection interval feature there was an bug related to the big
objects collecting: split error is returned from a client directly, not
via API status and was considered as a connection error.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
554b85411f [#2190] services/object: Log service error with INFO level
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
b0ad1b9ed2 [#2193] pilorama: Use do in TreeMove
It should be similar to a `TreeAddByPath`. `applyOperation` is used for
`Apply` when the operation can be inserted in the middle of a log.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
ba393e3e91 [#2188] engine: Fix panic during setting shard mode
Under load changing shard mode can lead to it being removed from the
list during some other PUT.
```
Dec 28 07:01:26 az neofs-node[364505]: panic: runtime error: invalid memory address or nil pointer dereference
Dec 28 07:01:26 az neofs-node[364505]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0xc9fbb1]
Dec 28 07:01:26 az neofs-node[364505]: goroutine 11791912 [running]:
Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine.(*StorageEngine).putToShard(0xc000435490, {0xc0003f7a28?, 0xc0001192c0?}, 0x2, {0x0, 0x>
Dec 28 07:01:26 az neofs-node[364505]:         github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine/put.go:91 +0x1b1
Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine.(*StorageEngine).put.func1(0xc000435490?, {0xc0003f7a28?, 0xc0001192c0?})
Dec 28 07:01:26 az neofs-node[364505]:         github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine/put.go:71 +0x19c
Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine.(*StorageEngine).iterateOverSortedShards(0x1?, {{0x62, 0x23, 0xfe, 0x60, 0x67, 0xd5, 0x>
Dec 28 07:01:26 az neofs-node[364505]:         github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine/shards.go:225 +0xc8
Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine.(*StorageEngine).put(0xc000435490, {0x1?})
Dec 28 07:01:26 az neofs-node[364505]:         github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine/put.go:66 +0x2a9
Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine.(*StorageEngine).Put.func1()
Dec 28 07:01:26 az neofs-node[364505]:         github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine/put.go:43 +0x2a
Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine.(*StorageEngine).execIfNotBlocked(0x8?, 0x38?)
Dec 28 07:01:26 az neofs-node[364505]:         github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine/control.go:147 +0xcf
Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine.(*StorageEngine).Put(0xc4df775a80?, {0x0?})
Dec 28 07:01:26 az neofs-node[364505]:         github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine/put.go:42 +0x65
Dec 28 07:01:26 az neofs-node[364505]: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine.Put(0xc06d928b80?, 0xc06b1b8dc8?)
Dec 28 07:01:26 az neofs-node[364505]:         github.com/nspcc-dev/neofs-node/pkg/local_object_storage/engine/put.go:158 +0x19
Dec 28 07:01:26 az neofs-node[364505]: main.engineWithoutNotifications.Put({0x20301b?}, 0x20301b?)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
118783c4cf [#26] .github: Fix CODEOWNERS
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:23 +03:00
3d57f4c961 [#2179] test: Fix test TestEvacuateNetwork/multiple_shards,_evacuate_many
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-01-24 13:37:49 +03:00
d4d2a8c865 [#11] Rename NeoFS to FrostFS in docs
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-12 08:55:47 +03:00
3d1d2ee7b1 [#11] Regenerate proto files
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-12 08:55:47 +03:00
054bc4a727 [#11] Rename contract-related NeoFS occurences
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-12 08:55:47 +03:00
9cb4b4cc17 [#11] Rename neofsid contract to frostfsid
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-12 08:55:47 +03:00
023396e6a4 [#11] adm: Remove NeoFS mentions
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-12 08:55:47 +03:00
19d180b510 [#11] Rename neofs contract to frostfs
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-12 08:55:47 +03:00
e355442532 [#10] .docker: Update go version to 1.18
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-01-10 12:27:25 +03:00
cc71e6902a [#7] Rebranding leftovers
- logo update
- docker image build fixes
- README and other texts updates

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-01-09 11:41:51 +03:00
fdb0affc31 [#5] frost-node: Used generic cache
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-31 23:04:06 +03:00
9936b112b8 [#5] blobstor: Use generic LRU cache
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-31 23:04:06 +03:00
f0be0befc5 [#5] services/object_manager: Use generic LRU cache
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-31 23:04:06 +03:00
1b3374ac7f [#5] services/tree: User generic LRU cache
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-31 23:04:06 +03:00
d0a0432a51 [#5] innerring: Use generic LRU cache
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-31 23:04:06 +03:00
4155c1bdff [#5] writecache: Use generic LRU cache
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-31 23:04:06 +03:00
8f61cc1dcc [#5] policer: Use generic LRU client
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-31 23:04:06 +03:00
55b403e0ee [#5] morph/client: Use generic LRU cache
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-31 23:04:06 +03:00
d8fb9c85eb [#5] go.mod: Bump supported go version to 1.18
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-31 23:04:06 +03:00
6f8dd816fb [#2189] go.mod: Update nats-io
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
26560b6b8d [#2184] go.mod: Update dependencies
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
0272218eb9 [#2184] compression: Properly calculate upper bound
If the data is not compressible allocating `len(data)` will lead to a
slice reallocation. For a compressible data the results for small size
are flaky and we allocate a bit more. However, it feels right to use a
provided function if we need to pick any size at all.

```
name                                                           old time/op    new time/op    delta
Compression/size=128/zeroed_slice-8                              2.23µs ±12%    2.06µs ± 6%   -7.35%  (p=0.009 n=10+10)
Compression/size=128/not_so_random_slice_(block_=_123)-8         19.0µs ±10%    15.8µs ±16%  -17.09%  (p=0.000 n=9+10)
Compression/size=128/random_slice-8                              17.6µs ±15%    16.1µs ±16%     ~     (p=0.075 n=10+10)
Compression/size=1024/zeroed_slice-8                             3.05µs ±11%    2.84µs ±10%     ~     (p=0.089 n=10+10)
Compression/size=1024/not_so_random_slice_(block_=_123)-8        18.1µs ± 6%    18.2µs ±12%     ~     (p=0.971 n=10+10)
Compression/size=1024/random_slice-8                             48.6µs ± 6%    45.6µs ± 5%   -6.07%  (p=0.006 n=10+9)
Compression/size=32768/zeroed_slice-8                            26.8µs ± 3%    28.7µs ± 8%   +7.23%  (p=0.001 n=10+10)
Compression/size=32768/not_so_random_slice_(block_=_123)-8       44.3µs ± 8%    43.7µs ±13%     ~     (p=0.762 n=8+10)
Compression/size=32768/random_slice-8                            97.3µs ±32%    68.9µs ±15%  -29.13%  (p=0.000 n=10+10)
Compression/size=33554432/zeroed_slice-8                         29.8ms ± 9%    30.3ms ±17%     ~     (p=1.000 n=9+9)
Compression/size=33554432/not_so_random_slice_(block_=_123)-8    33.1ms ±14%    30.3ms ±11%   -8.61%  (p=0.043 n=10+10)
Compression/size=33554432/random_slice-8                         41.7ms ± 3%    30.1ms ± 8%  -27.72%  (p=0.000 n=9+10)

name                                                           old alloc/op   new alloc/op   delta
Compression/size=128/zeroed_slice-8                                128B ± 0%      144B ± 0%  +12.50%  (p=0.000 n=10+10)
Compression/size=128/not_so_random_slice_(block_=_123)-8           384B ± 0%      144B ± 0%  -62.50%  (p=0.000 n=10+10)
Compression/size=128/random_slice-8                                384B ± 0%      144B ± 0%  -62.50%  (p=0.000 n=10+10)
Compression/size=1024/zeroed_slice-8                             1.02kB ± 0%    1.15kB ± 0%  +12.50%  (p=0.000 n=10+10)
Compression/size=1024/not_so_random_slice_(block_=_123)-8        1.02kB ± 0%    1.15kB ± 0%  +12.50%  (p=0.000 n=10+10)
Compression/size=1024/random_slice-8                             2.56kB ± 0%    1.15kB ± 0%  -55.00%  (p=0.000 n=10+10)
Compression/size=32768/zeroed_slice-8                            32.8kB ± 0%    41.0kB ± 0%  +25.00%  (p=0.000 n=10+10)
Compression/size=32768/not_so_random_slice_(block_=_123)-8       32.8kB ± 0%    41.0kB ± 0%  +25.00%  (p=0.000 n=10+10)
Compression/size=32768/random_slice-8                            81.9kB ± 0%    41.0kB ± 0%  -50.00%  (p=0.000 n=10+10)
Compression/size=33554432/zeroed_slice-8                         33.6MB ± 0%    33.6MB ± 0%   +0.02%  (p=0.000 n=9+9)
Compression/size=33554432/not_so_random_slice_(block_=_123)-8    33.6MB ± 0%    33.6MB ± 0%   +0.02%  (p=0.000 n=8+10)
Compression/size=33554432/random_slice-8                         75.5MB ± 0%    33.6MB ± 0%  -55.55%  (p=0.000 n=10+10)

name                                                           old allocs/op  new allocs/op  delta
Compression/size=128/zeroed_slice-8                                1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=128/not_so_random_slice_(block_=_123)-8           2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
Compression/size=128/random_slice-8                                2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
Compression/size=1024/zeroed_slice-8                               1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=1024/not_so_random_slice_(block_=_123)-8          1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=1024/random_slice-8                               2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
Compression/size=32768/zeroed_slice-8                              1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=32768/not_so_random_slice_(block_=_123)-8         1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=32768/random_slice-8                              2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
Compression/size=33554432/zeroed_slice-8                           1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=33554432/not_so_random_slice_(block_=_123)-8      1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=33554432/random_slice-8                           2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
874be99076 [#2184] go.mod: Update neo-go to v0.100.1
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
71c64ae253 Release v0.35.0
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2022-12-30 11:07:35 +03:00
Mikhail Petrov
cfdb53a788 [#614] Add logo for dark theme
Signed-off-by: Mikhail Petrov <mike@nspcc.ru>
2022-12-30 11:07:35 +03:00
a2fe912d1a [#2164] neofs-node: Use a separate client cache for replicator
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
741482c26f [#2164] neofs-node: Allow to set reconnect interval
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
6f5edac730 [#2164] network/cache: Do not reconnect to failed clients immediately
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
f3caf6acfe [#2164] network/cache: Separate mutex for addr
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
0ace28e43d [#2175] blobovniczatree: Close all non-active blobovniczas
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
c1cf418956 [#2175] blobovniczatree: Make function parameters more descriptive
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
b4e90cdf51 [#2165] pilorama: Optimize TreeApply when used for synchronization
Because synchronization _most likely_ will have apply already existing
operations, it is much faster to check their presence in a read
transaction. However, always doing this will degrade the perfomance
for normal `Apply`. And, let's be honest, it is already not good.
Thus we add a separate parameter which specifies whether this logic is
enabled.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
f9fcd85363 [#2165] services/tree: Remember starting height for the synchronization
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
06137dbf8e [#2165] services/tree: Do not export synchronizeAllTrees
It is used only privately.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
c299b98afe [#2165] services/tree: Parallelize synchronization
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
de9957e076 [#2165] services/tree: Always synchronize all containers
In case of split-brain we must synchronize everything.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
d3054e577a [#2165] services/tree: Allow to set custom synchronization interval
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
Pavel Karpy
6a4e5e6f0a [#2144] node: Try node's private key if dynamic token fetching failed
`GETRANGEHASH` request spawns `GETRANGE` requests if an object could not be
found locally. If the original request contains session, it can be static
and, therefore, fetching session key can not be performed successfully.
As the best effort a node could request object's range with its own key.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-30 11:07:35 +03:00
Pavel Karpy
86a4fba571 [#2144] node: Clarify KeyStorage.GetKey method
Actualize the doc, fix API status error return.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-30 11:07:35 +03:00
b7a99c757a [#2097] cli: Clarify help for --expire-at parameter for commands object lock/put and bearer create
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2022-12-30 11:07:35 +03:00
e406036629 [#2166] go.mod: Update dependencies
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
Pavel Karpy
21717262ec [#2016] shard: Check meta first on Get
`meta` should prevent returning removed objects (`GCMark` and `TS` relations
are `meta` abstractions).

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-30 11:07:35 +03:00
Pavel Karpy
74ec71446f [#2167] shard: Do not use write-cache by default in Head
Both `meta` and `write-cache` are expected to have a fast underlying disk,
so it does not seem like an optimisation. Moreover, `write-cache`'s `Head`
is a `Get` with payload cutting, it _must_ use more memory for no reason
(`meta` was created for such requests). Also, `write-cache` does not allow
performing any "meta" relations checks (such as locking, tombstoning).

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-30 11:07:35 +03:00
Pavel Karpy
1608fd1c07 [#2167] write-cache: Add "write-cache" to its logs
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-30 11:07:35 +03:00
Pavel Karpy
eea2892109 [#1956] node: Lock shard's mode on its methods switch
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-30 11:07:35 +03:00
33d279a3f2 [#2152] cli: Do not search for LOCK objects when delete container when session provided
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2022-12-30 11:07:35 +03:00
Evgenii Stratonikov
04b5ec759b [#2139] object/put: Use sync.Pool for temporary payloads
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
9e0decd12d [#2162] services/tree: Close connection after the syncronization
There was a goroutine leak here.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
Evgenii Stratonikov
e1c3bdbfa6 [#1621] pilorama: Remove Timestamp field from nodeInfo
It is already present in `Meta`.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-12-30 11:07:35 +03:00
Evgenii Stratonikov
1044adbe94 [#1621] pilorama: Improve memory allocation
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-12-30 11:07:35 +03:00
Evgenii Stratonikov
2539d466a6 [#1621] pilorama: Seek after cursor invalidation
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-12-30 11:07:35 +03:00
Evgenii Stratonikov
e9ba8931f8 [#1621] pilorama: Simplify bucket creation
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-12-30 11:07:35 +03:00
Evgenii Stratonikov
fe7ddfdc6a [#1621] pilorama: Compare memory forests properly
Node children are not sorted and could occur in any order.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-12-30 11:07:35 +03:00
Pavel Karpy
306609030a [#2159] node: Add tree replication timeout configuration
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-30 11:07:35 +03:00
edb1428248 [#2022] Add metric readonly to get shards mode
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2022-12-30 11:07:35 +03:00
e5c304536b [#2161] pilorama: Do not apply already existing operations
Speeds up synchronization a bit.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
3bb5a320d7 [#2154] services/tree: Do not log an error when synchronizing container of 1 node
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
387d1e2977 [#2127] services/tree: Randomize node order for synchronization
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
Pavel Karpy
21c58c92a9 [#2145] meta: Do allow force inhuming a locked object
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-30 11:07:35 +03:00
Pavel Karpy
a30310b2ca [#2145] cli: Do not print "rpc error:" twice
That error part is already included in GRPC's library.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-30 11:07:35 +03:00
b207dc424f [#2158] policer: Reduce default cache size
We use cache to avoid policing the same object multiple times in a short
time span (< 30 seconds). If we have 200_000 objects in a blobstor, it is a bit useless
-- if it takes 1 second to process an object and we have `replicator.pool_size: 20`
in config, the next iteration will happen in 10_000 second which is much
larger than 30 second. However we still consume a lot of memory, so it
makes sense to use saner default.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
0244f2e5ce [#2156] core/netmap: Change node addresses iteration order
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
Leonard Lyubich
a68ff67ed8 [#2106] cli: Verify container owner in container delete command
In NeoFS containers can be removed on behalf of its owner only. To
improve user experience, there is a need to add ownership check to the
removal command of the NeoFS CLI.

Check container ownership in `container delete` command `Run` function.
The check can be skipped by `--force` option.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
2022-12-30 11:07:35 +03:00
Pavel Karpy
73ef5b18c7 [#2092] go.mod: Update SDK
Includes a fix that adds error messages' text, it affects incomplete object
put error message.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-30 11:07:35 +03:00
Pavel Karpy
b413094704 [#2095] node: Fix collecting child objects
Stop child objects collection if the last returned object (the most "left"
object in the collected chain) starts exactly from the `GETRANGE`'s `from`
value.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-12-30 11:07:35 +03:00
Pavel Karpy
350eecfa13 [#2095] node: Do not allow GETRANGE requests with zero length
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-12-30 11:07:35 +03:00
Pavel Karpy
d6196c3971 [#2095] cli: Do not panic on object range
Also, includes range parsing error messages enhancement.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-12-30 11:07:35 +03:00
0b78af467e [#2140] engine: Fix error handling in TreeMove
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
Pavel Karpy
923f84722a Move to frostfs-node
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-28 15:04:29 +03:00
Evgenii Stratonikov
42554a9298 [#2068] writecache: Remove deleted objects from the writecache
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-12-02 11:52:05 +03:00
Evgenii Stratonikov
4a49ea0855 [#2068] writecache: Allow to open FSTree in read-only mode
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-12-02 11:52:05 +03:00
Evgenii Stratonikov
857d2dc3f5 [#2068] writecache: Optimize initial flush existence checking
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-12-02 11:52:05 +03:00
Evgenii Stratonikov
63f604e948 [#2068] blobstor: Allow to provide storage ID in Exists
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-12-02 11:52:05 +03:00
Evgenii Stratonikov
6ad2b5d5b8 [#2068] blobovnicza: Add Exists method
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-12-02 11:52:05 +03:00
9eccf3bbf5 [#1486] Update changelog
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2022-12-02 11:43:33 +03:00
b0eeb0dfcf [#1486] node: Use endless notary deposit for side chain
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2022-12-02 11:43:33 +03:00
ba74244d1b [#1486] ir: Use endless notary deposit for side chain
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2022-12-02 11:43:33 +03:00
b6a40241f8 [#1486] morph: Add method for endless notary deposit
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2022-12-02 11:43:33 +03:00
Pavel Karpy
d54022eacc [#2047] node: Do not send chunk twice on request forwarding
That could happen if a node forwards request to a node that closed the
connection during the original object stream.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-12-02 11:27:48 +03:00
Evgenii Stratonikov
bd25db5d4a [#1984] metrics: Use separate metrics for success/failed requests
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-12-01 14:18:10 +03:00
Evgenii Stratonikov
e21c472dc7 [#1984] services/object: Increase put_req_count after the request is processed
As it is specified in metrics description.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-12-01 14:18:10 +03:00
afabd6be91 [#2038] neofs-cli: Check the sufficiency of the number of nodes in the selector for replicas
Perform this check on container creation.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2022-12-01 11:32:27 +03:00
Pavel Karpy
50d28b72c3 [#2075] node, ir: Log notary state with info
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-30 19:26:33 +03:00
Pavel Karpy
0f41c09207 [#2075] node: Do not make notary requests on shutdown
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-30 19:26:33 +03:00
Pavel Karpy
b8c30b88f6 [#2075] morph: Do not do notary deposit with zero balance
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-30 19:26:33 +03:00
Pavel Karpy
c01d4ecb50 [#2080] morph: Close morph clients
Could be related to "websocket users limit reached" on the `neo-go` server
side when an SN/IR is rebooting repeatedly.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-30 19:26:33 +03:00
Pavel Karpy
3d0768a1d3 [#2061] node: Unify meta.Get benchmarks
Make them get exactly one (different) object per a bench iteration.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-30 18:29:14 +03:00
Pavel Karpy
bc905f169d [#2061] meta: Add parallel bench for Get
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-30 18:29:14 +03:00
Pavel Karpy
761e82fecd [#2079] cli: Do not panic in object hash
Sign RPC requests with the provided key.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-30 16:58:52 +03:00
Pavel Karpy
960a2d0629 [#2081] ir: Set default key in IR's SDK clients
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-30 16:58:52 +03:00
8d15c14be6 [#2078] adm: Pack parameters for setPrice invocation
Contract arguments have to be packed.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2022-11-30 16:58:52 +03:00
Pavel Karpy
51963abce7 [#1972] node: Fix errors comments in the Put service
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-30 16:58:52 +03:00
Evgenii Stratonikov
7335a52f29 [#1732] pilorama: Improve logical error handling
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-30 16:58:52 +03:00
Evgenii Stratonikov
ae7b473768 [#2064] blobovniczatree: Remove index too big log
There is no need to log about a situation which is expected.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-30 16:53:18 +03:00
8a77b4638a [#2012] Add commands neofs-cli acl basic/extended print to show ACL table in human readable format
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2022-11-30 16:45:37 +03:00
Evgenii Stratonikov
59db66cdb6 [#2091] neofs-adm: Sign blocks properly in tests
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-30 16:45:13 +03:00
Evgenii Stratonikov
6882887bdd [#2091] neofs-adm: Do not query hashes via network
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-30 16:45:13 +03:00
Evgenii Stratonikov
c1ea6fd854 [#2091] morph/client: Use notary.FakeContractAccount wrapper
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-30 16:45:13 +03:00
Evgenii Stratonikov
b93be8869b [#2091] morph/client: Simplify code
1. Replace `mn` function with a `sigCount`.
2. Use `notary.FakeMultisigAccount` for account creation.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-30 16:45:13 +03:00
Evgenii Stratonikov
cfefebd5b3 [#2091] neofs-adm: Do not use deprecated methods
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-30 16:45:13 +03:00
Evgenii Stratonikov
6af52c46d8 [#2091] morph/client: Do not use deprecated methods
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-30 16:45:13 +03:00
51e886dd67 [#2090] neofs-cli: Remove --header from object get
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2022-11-30 11:46:02 +03:00
79130f781e [#2089] neofs-cli: Remove -g option from neofs-cli control ... and neofs-cli container create commands
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2022-11-30 09:14:50 +03:00
Evgenii Stratonikov
dd76ceadf1 [#2060] go.mod: Retract v1.* versions
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-29 09:29:42 +03:00
Pavel Karpy
ca3596dd05 [#2104] cli: OID signature in output
OID signature should always be present in an object; it does not relate
to the object split.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-29 08:52:09 +03:00
Evgenii Stratonikov
816c74d185 [#2075] morph/client: Ignore error if a transaction already exists
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
71853348b9 [#2063] morph/client: Support new hash format in morph nns client
Signed-off-by: Vladimir Domnich <v.domnich@yadro.com>
2022-11-19 11:01:04 +03:00
Pavel Karpy
ed4351aab0 [#2074] write-cache: Do not flush same object twice
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
dd225906a0 [#2074] write-cache: Remove unused variables
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
9f7ac6bf9f [#2069] innerring: Do not panic in Head
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
98e6dc5ce8 [#409] debian: Refactor storage service paths
Separate User data and Service data:
 - /var/lib/neofs/storage for service persistence
 - /srv/neofs for user data

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
660c38d07e [#2062] services/policer: Use a proper key for object cache
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
b1025bdb42 [#2057] meta: Fail write operations in R/O mode
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
fdeea1dfac [#2057] meta: Fix concurrent mode changes
Includes:
1. mode change read lock operation in every exported method that r/w the
underlying database;
2. returning `ErrDegradedMode` logical error if any exported method is
called in degraded (without a metabase) mode.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
3d6defd3e8 [#2057] meta: Do not lock the whole meta on GET
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
fa231b8c56 [#2057] blobstor: Block operations on a mode change
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
1779664644 [#2058] services/policer: Fix panic in shardPolicyWorker
```
2022/11/15 08:40:56 worker exits from a panic: runtime error: index out of range [0] with length 0
2022/11/15 08:40:56 worker exits from panic: goroutine 1188 [running]:
github.com/panjf2000/ants/v2.(*goWorker).run.func1.1()
	github.com/panjf2000/ants/v2@v2.4.0/worker.go:58 +0x10c
panic({0x1042b60, 0xc0015ae018})
	runtime/panic.go:1038 +0x215
github.com/nspcc-dev/neofs-node/pkg/services/policer.(*Policer).shardPolicyWorker.func1()
	github.com/nspcc-dev/neofs-node/pkg/services/policer/process.go:65 +0x366
github.com/panjf2000/ants/v2.(*goWorker).run.func1()
	github.com/panjf2000/ants/v2@v2.4.0/worker.go:68 +0x97
created by github.com/panjf2000/ants/v2.(*goWorker).run
	github.com/panjf2000/ants/v2@v2.4.0/worker.go:48 +0x68
```

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
b673d9e472 [#2053] engine: Do not switch mode because of logical errors
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
9a20498f34 [#1940] Removing all trees by container ID if tree ID is empty in pilorama.Forest.TreeDrop
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
426fe97990 [#2026] neofs-adm: Make contract update idempotent
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
634792077e [#1502] node: Store lock object on every container node
Includes extending listing methods in the Storage Engine with object types.
It allows tuning replication/policer algorithms: container nodes do
not remove `LOCK` objects as redundant and try to fulfill `LOCK` placement
on the ohter container nodes.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
fe09cd9c70 [#1502] core: Add AddressWithType
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
3b61cb4f49 [#1502] engine: Check all shards for LOCK'ing before inhuming
It allows keeping all the locked objects safe after metabase
resynchronization. Currently, all `LOCK` objects are broadcast to all nodes
in a container, it guarantees `LOCK` object presence in a regular situation.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
34e8d2ba56 [#1502] shard: Add IsLocked method
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
9a039ba582 [#1502] meta: Add IsLocked method
It gets an object and returns its locking status.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
7ef0303e13 [#2003] neofs-node: Allow to configure replicator pool size
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
659011c143 [#2048] network/cache: Optimize ClientCache
1. Remove a layer of indirection for mutex, `ClientCache` is already
   used by pointer.
2. Fix duplication of a `AllowExternal` field.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
597ed18269 [#2048] neofs-node: Use a separate client cache for client operations
Background workers can prevent user operations to complete because of
locking in cache.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
d5a14041e0 [#2040] node: Do not attach tokens in the assembly process
A container node is expected to have full "get" access to assemble the
object.
A non-container node is expected to forward any request to a container node.
Any token is expected to be issued for an original request sender not for a
node so any new request is invalid by design with that token.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
fd61bdadcb [#2040] node: Attach original meta to the spawned requests
Do not lose meta information of the original requests: cache session and
bearer tokens of the original request b/w a new generated ones. Middle
request wrappers should not contain any meta information, since it is
useless (e.g. ACL service checks only the original tokens).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
d65604ad30 [#1985] blobstor: Allow to report multiple errors to caller
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
f2d7e65e39 [#2035] engine: Allow moving to degraded from background workers
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
c85bea15ef [#1978] cli: Add children to the static session on DELETE
If an external session is provided and is not opened by CLI itself, add
children objects to it too. It fixes "not found" errors when removing a big
object with a predefined session (from a file).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
f48b1de54b [#2029] cli: Fix panic caused by flag redefinition
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
09a59fef56 [#2029] cli: Allow attaching static session to object hash
All the other object commands already have it.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
481b48b942 [#2028] node: Check session token's NBF and IAT
ACL service did not check "Not Valid Before" and "Issued At" claims.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
aadd2ad050 [#2028] node: Do not wrap malformed request errors
After presenting request statuses on the API level, all the errors are
unwrapped before sending to the caller side. It led to a losing invalid
request's context.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
2522d924b9 [#2037] services/object: Fix concurrent map writes in traverser
```
fatal error: concurrent map writes

goroutine 4337 [running]:
github.com/nspcc-dev/neofs-node/pkg/services/object/put.(*traversal).submitProcessed(...)
        github.com/nspcc-dev/neofs-node/pkg/services/object/put/distributed.go:78
github.com/nspcc-dev/neofs-node/pkg/services/object/put.(*distributedTarget).iteratePlacement.func1()
        github.com/nspcc-dev/neofs-node/pkg/services/object/put/distributed.go:198 +0x265
github.com/panjf2000/ants/v2.(*goWorker).run.func1()
        github.com/panjf2000/ants/v2@v2.4.0/worker.go:68 +0x97
created by github.com/panjf2000/ants/v2.(*goWorker).run
        github.com/panjf2000/ants/v2@v2.4.0/worker.go:48 +0x65
```

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
37f813604f [#2000] cli: Provide a bearer token to spawned HEAD by DELETE
If a `neofs-cli object delete` operation is performing using a bearer token,
add it to the new `HEAD` requests that collects children OIDs.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
b0e94b6a6b [#1906] writecache: Do not require read-only mode in Flush
It was needed before we started to flush during transition to
`degraded` mode. Now it is confusing.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
aa478f1def [#2024] services/object: Unify status errors
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
3875fef542 [#2024] services/object: Cover corner cases for children OutOfRange
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
b9fb0d6050 [#1972] node: Fix object format unit tests
Includes:
1. Unused func removal;
2. Err check of the `Sign` method.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
aab398f4f5 [#1972] node: Do not save objects if node not in a container
Do not use node's local storage if it is clear that an object will be
removed anyway as a redundant. It requires moving the changing local storage
logic from the validation step to the local target implementation.
It allows performing any relations checks (e.g. object locking) only if a
node is considered as a valid container member and is expected to store
(stored previously) all the helper objects (e.g. `LOCK`, `TOMBSTONE`, etc).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
a77392e9ce [#1972] cli: Fix lifetime flag in the lock command
That part of the code was refactored incorrectly.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
2849e465f9 [#1699] meta: Do not return SplitInfoError on Delete
It is not an error: removing virtual object is expected and should be just
skipped. Getting a virtual object with `raw` flag is considered as an
impossible action, all the virtual objects removals will be handled via
their children's removals implicitly.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
a3e7365cbd [#1732] pilorama: Fill parent mark correctly
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
134f2ba02e [#1732] pilorama: Fix backwards log insertion
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
2ef38cfbc4 [#1996] engine: Ignore pilorama.ErrTreeNotFound for write operations
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
a455ec18c3 [#2007] services/object: Allocate memory on-demand in GET_RANGE
For big objects we want to get OutOfRange error before all the memory is
allocated.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
ff5526038d [#2007] services/object: Fix comment
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
d8d3588e1b [#1996] engine: Always select proper shard for a tree
Currently there is a possibility for modifying operations to fail
because of I/O errors and a new tree to be created on another shard.
This commit adds existence check for modifying operations.
Read operations remain as they are, not to slow things.
`TreeDrop` is an exception, because this is a tree removal and trying
multiple shards is not an unwanted behaviour.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-03 15:29:23 +03:00
Pavel Karpy
08efe6eb11 [#2010] github: Run CI on support branches
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-02 14:29:17 +03:00
Evgenii Stratonikov
777fd32d4f [#1818] writecache: Increase error counter on background errors
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-02 14:24:02 +03:00
Evgenii Stratonikov
bffb0f894c [#1818] writecache: Update storage ID during flush
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-02 14:24:02 +03:00
Evgenii Stratonikov
5cf75404dc [#1818] metabase: Add UpdateStorageID operation
By default writecache puts the whole object to update storage ID.
This logic comes from the times when we needed to put objects
in the metabase by the writecache itself. Now this is done by the
blobstor at unmarshaling objects during flush only to update storage ID
is an overkill.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-02 14:24:02 +03:00
Evgenii Stratonikov
b64b14eb54 [#1818] writecache: Reuse FSTree flushing code between flushes
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-02 14:24:02 +03:00
Evgenii Stratonikov
a56927e3d4 [#1818] writecache: Remove unused variable
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-02 14:24:02 +03:00
bb52857b2b [#1338] neofs-cli: Add support to store/restore/delete binary objects
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2022-11-01 15:30:00 +03:00
Pavel Karpy
1f82c583e3 [#1971] cli: Unify CID and OID flags provision
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-01 15:07:19 +03:00
Pavel Karpy
7daa57d4d2 [#1991] cli: Refine container placement description
Not to confuse a user by mixing a replication vector number with its copy
number.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-01 15:00:00 +03:00
51e3810285 [#1689] Add new command morph list-containers in neofs-adm
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2022-11-01 09:43:02 +03:00
Evgenii Stratonikov
98a152256b [#1992] writecache: Allow to open in NOSYNC mode
Applicable only to FSTree as we cannot handle corrupted databases
properly yet.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-01 09:42:26 +03:00
Evgenii Stratonikov
b6930f2219 [#1992] neofs-node: Allow to open fstree in NOSYNC mode
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-01 09:42:26 +03:00
Evgenii Stratonikov
148da5fdbb [#1994] docs: Update storage node configuration
Reflect the reality after a not so recent refactoring.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-01 09:42:26 +03:00
Evgenii Stratonikov
f564430b90 [#1992] fstree: Allow working in SYNC mode
Make O_SYNC the default and allow to opt-out explicitly.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-01 09:42:26 +03:00
1039 changed files with 14616 additions and 8982 deletions

View file

@ -1,19 +1,19 @@
FROM golang:1.17 as builder FROM golang:1.18 as builder
ARG BUILD=now ARG BUILD=now
ARG VERSION=dev ARG VERSION=dev
ARG REPO=repository ARG REPO=repository
WORKDIR /src WORKDIR /src
COPY . /src COPY . /src
RUN make bin/neofs-adm RUN make bin/frostfs-adm
# Executable image # Executable image
FROM alpine AS neofs-adm FROM alpine AS frostfs-adm
RUN apk add --no-cache bash RUN apk add --no-cache bash
WORKDIR / WORKDIR /
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=builder /src/bin/neofs-adm /bin/neofs-adm COPY --from=builder /src/bin/frostfs-adm /bin/frostfs-adm
CMD ["neofs-adm"] CMD ["frostfs-adm"]

25
.docker/Dockerfile.ci Normal file
View file

@ -0,0 +1,25 @@
FROM golang:1.19
WORKDIR /tmp
# Install apt packages
RUN apt-get update && apt-get install --no-install-recommends -y \
pip \
&& apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false \
&& rm -rf /var/lib/apt/lists/*
# Dash → Bash
RUN echo "dash dash/sh boolean false" | debconf-set-selections
RUN DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
RUN useradd -u 1234 -d /home/ci -m ci
USER ci
ENV PATH="$PATH:/home/ci/.local/bin"
COPY .pre-commit-config.yaml .
RUN pip install "pre-commit==3.1.1" \
&& git init . \
&& pre-commit install-hooks \
&& rm -rf /tmp/*

View file

@ -1,19 +1,19 @@
FROM golang:1.17 as builder FROM golang:1.18 as builder
ARG BUILD=now ARG BUILD=now
ARG VERSION=dev ARG VERSION=dev
ARG REPO=repository ARG REPO=repository
WORKDIR /src WORKDIR /src
COPY . /src COPY . /src
RUN make bin/neofs-cli RUN make bin/frostfs-cli
# Executable image # Executable image
FROM alpine AS neofs-cli FROM alpine AS frostfs-cli
RUN apk add --no-cache bash RUN apk add --no-cache bash
WORKDIR / WORKDIR /
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=builder /src/bin/neofs-cli /bin/neofs-cli COPY --from=builder /src/bin/frostfs-cli /bin/frostfs-cli
CMD ["neofs-cli"] CMD ["frostfs-cli"]

View file

@ -3,6 +3,6 @@ RUN apk add --no-cache bash ca-certificates
WORKDIR / WORKDIR /
COPY bin/neofs-adm /bin/neofs-adm COPY bin/frostfs-adm /bin/frostfs-adm
CMD ["neofs-adm"] CMD ["frostfs-adm"]

View file

@ -3,6 +3,6 @@ RUN apk add --no-cache bash ca-certificates
WORKDIR / WORKDIR /
COPY bin/neofs-cli /bin/neofs-cli COPY bin/frostfs-cli /bin/frostfs-cli
CMD ["neofs-cli"] CMD ["frostfs-cli"]

View file

@ -3,6 +3,6 @@ RUN apk add --no-cache bash ca-certificates
WORKDIR / WORKDIR /
COPY bin/neofs-ir /bin/neofs-ir COPY bin/frostfs-ir /bin/frostfs-ir
CMD ["neofs-ir"] CMD ["frostfs-ir"]

View file

@ -3,6 +3,6 @@ RUN apk add --no-cache bash ca-certificates
WORKDIR / WORKDIR /
COPY bin/neofs-node /bin/neofs-node COPY bin/frostfs-node /bin/frostfs-node
CMD ["neofs-node"] CMD ["frostfs-node"]

View file

@ -1,18 +1,18 @@
FROM golang:1.17 as builder FROM golang:1.18 as builder
ARG BUILD=now ARG BUILD=now
ARG VERSION=dev ARG VERSION=dev
ARG REPO=repository ARG REPO=repository
WORKDIR /src WORKDIR /src
COPY . /src COPY . /src
RUN make bin/neofs-ir RUN make bin/frostfs-ir
# Executable image # Executable image
FROM alpine AS neofs-ir FROM alpine AS frostfs-ir
RUN apk add --no-cache bash RUN apk add --no-cache bash
WORKDIR / WORKDIR /
COPY --from=builder /src/bin/neofs-ir /bin/neofs-ir COPY --from=builder /src/bin/frostfs-ir /bin/frostfs-ir
CMD ["neofs-ir"] CMD ["frostfs-ir"]

View file

@ -1,18 +1,18 @@
FROM golang:1.17 as builder FROM golang:1.18 as builder
ARG BUILD=now ARG BUILD=now
ARG VERSION=dev ARG VERSION=dev
ARG REPO=repository ARG REPO=repository
WORKDIR /src WORKDIR /src
COPY . /src COPY . /src
RUN make bin/neofs-node RUN make bin/frostfs-node
# Executable image # Executable image
FROM alpine AS neofs-node FROM alpine AS frostfs-node
RUN apk add --no-cache bash RUN apk add --no-cache bash
WORKDIR / WORKDIR /
COPY --from=builder /src/bin/neofs-node /bin/neofs-node COPY --from=builder /src/bin/frostfs-node /bin/frostfs-node
CMD ["neofs-node"] CMD ["frostfs-node"]

View file

@ -1,19 +1,19 @@
FROM golang:1.17 as builder FROM golang:1.18 as builder
ARG BUILD=now ARG BUILD=now
ARG VERSION=dev ARG VERSION=dev
ARG REPO=repository ARG REPO=repository
WORKDIR /src WORKDIR /src
COPY . /src COPY . /src
RUN make bin/neofs-node RUN make bin/frostfs-node
# Executable image # Executable image
FROM alpine AS neofs-node FROM alpine AS frostfs-node
RUN apk add --no-cache bash RUN apk add --no-cache bash
WORKDIR / WORKDIR /
COPY --from=builder /src/bin/neofs-node /bin/neofs-node COPY --from=builder /src/bin/frostfs-node /bin/frostfs-node
COPY --from=builder /src/config/testnet/config.yml /config.yml COPY --from=builder /src/config/testnet/config.yml /config.yml
CMD ["neofs-node", "--config", "/config.yml"] CMD ["frostfs-node", "--config", "/config.yml"]

View file

@ -5,4 +5,5 @@ docker-compose.yml
Dockerfile Dockerfile
temp temp
.dockerignore .dockerignore
docker docker
.cache

1
.github/CODEOWNERS vendored
View file

@ -1 +0,0 @@
* @alexvanin @carpawell @fyrchik @cthulhu-rider

View file

@ -43,7 +43,7 @@ assignees: ''
* Operating System and version (`uname -a`): * Operating System and version (`uname -a`):
## Don't forget to add labels! ## Don't forget to add labels!
- component label (`neofs-adm`, `neofs-storage`, ...) - component label (`frostfs-adm`, `frostfs-storage`, ...)
- `goodfirstissue`, `helpwanted` if needed - `goodfirstissue`, `helpwanted` if needed
- does this issue belong to an epic? - does this issue belong to an epic?
- priority (`P0`-`P4`) if already triaged - priority (`P0`-`P4`) if already triaged

197
.github/logo.svg vendored
View file

@ -1,129 +1,70 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml version="1.0" encoding="utf-8"?>
<svg <!-- Generator: Adobe Illustrator 25.0.1, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
xmlns:dc="http://purl.org/dc/elements/1.1/" <svg version="1.1" id="Слой_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
xmlns:cc="http://creativecommons.org/ns#" viewBox="0 0 184.2 51.8" style="enable-background:new 0 0 184.2 51.8;" xml:space="preserve">
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" <style type="text/css">
xmlns:svg="http://www.w3.org/2000/svg" .st0{display:none;}
xmlns="http://www.w3.org/2000/svg" .st1{display:inline;}
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" .st2{fill:#01E397;}
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" .st3{display:inline;fill:#010032;}
sodipodi:docname="logo_fs.svg" .st4{display:inline;fill:#00E599;}
inkscape:version="1.0 (4035a4fb49, 2020-05-01)" .st5{display:inline;fill:#00AF92;}
id="svg57" .st6{fill:#00C3E5;}
version="1.1" </style>
viewBox="0 0 105 25" <g id="Layer_2">
height="25mm" <g id="Layer_1-2" class="st0">
width="105mm"> <g class="st1">
<defs <path class="st2" d="M146.6,18.3v7.2h10.9V29h-10.9v10.7h-4V14.8h18v3.5H146.6z"/>
id="defs51"> <path class="st2" d="M180,15.7c1.7,0.9,3,2.2,4,3.8l-3,2.7c-0.6-1.3-1.5-2.4-2.6-3.3c-1.3-0.7-2.8-1-4.3-1
<clipPath c-1.4-0.1-2.8,0.3-4,1.1c-0.9,0.5-1.5,1.5-1.4,2.6c0,1,0.5,1.9,1.4,2.4c1.5,0.8,3.2,1.3,4.9,1.5c1.9,0.3,3.7,0.8,5.4,1.6
clipPathUnits="userSpaceOnUse" c1.2,0.5,2.2,1.3,2.9,2.3c0.6,1,1,2.2,0.9,3.4c0,1.4-0.5,2.7-1.3,3.8c-0.9,1.2-2.1,2.1-3.5,2.6c-1.7,0.6-3.4,0.9-5.2,0.8
id="clipPath434"> c-5,0-8.6-1.6-10.7-5l2.9-2.8c0.7,1.4,1.8,2.5,3.1,3.3c1.5,0.7,3.1,1.1,4.7,1c1.5,0.1,2.9-0.2,4.2-0.9c0.9-0.5,1.5-1.5,1.5-2.6
<path c0-0.9-0.5-1.8-1.3-2.2c-1.5-0.7-3.1-1.2-4.8-1.5c-1.9-0.3-3.7-0.8-5.5-1.5c-1.2-0.5-2.2-1.4-3-2.4c-0.6-1-1-2.2-0.9-3.4
d="M 0,0 H 1366 V 768 H 0 Z" c0-1.4,0.4-2.7,1.2-3.8c0.8-1.2,2-2.2,3.3-2.8c1.6-0.7,3.4-1.1,5.2-1C176.1,14.3,178.2,14.8,180,15.7z"/>
id="path432" /> </g>
</clipPath> <path class="st3" d="M73.3,16.3c1.9,1.9,2.9,4.5,2.7,7.1v15.9h-4V24.8c0-2.6-0.5-4.5-1.6-5.7c-1.2-1.2-2.8-1.8-4.5-1.7
</defs> c-1.3,0-2.5,0.3-3.7,0.8c-1.2,0.7-2.2,1.7-2.9,2.9c-0.8,1.5-1.1,3.2-1.1,4.9v13.3h-4V15.1l3.6,1.5v1.7c0.8-1.5,2.1-2.6,3.6-3.3
<sodipodi:namedview c1.5-0.8,3.2-1.2,4.9-1.1C68.9,13.8,71.3,14.7,73.3,16.3z"/>
inkscape:window-maximized="0" <path class="st3" d="M104.4,28.3H85.6c0.1,2.2,1,4.3,2.5,5.9c1.5,1.4,3.5,2.2,5.6,2.1c1.6,0.1,3.2-0.2,4.6-0.9
inkscape:window-y="0" c1.1-0.6,2-1.6,2.5-2.8l3.3,1.8c-0.9,1.7-2.3,3.1-4,4c-2,1-4.2,1.5-6.4,1.4c-3.7,0-6.7-1.1-8.8-3.4s-3.2-5.5-3.2-9.6s1-7.2,3-9.5
inkscape:window-x="130" s5-3.4,8.7-3.4c2.1-0.1,4.2,0.5,6.1,1.5c1.6,1,3,2.5,3.8,4.2c0.9,1.8,1.3,3.9,1.3,5.9C104.6,26.4,104.6,27.4,104.4,28.3z
inkscape:window-height="1040" M88.1,19.3c-1.4,1.5-2.2,3.4-2.4,5.5h15.1c-0.2-2-1-3.9-2.3-5.5c-1.4-1.3-3.2-2-5.1-1.9C91.5,17.3,89.6,18,88.1,19.3z"/>
inkscape:window-width="1274" <path class="st3" d="M131,17.3c2.2,2.3,3.2,5.5,3.2,9.5s-1,7.3-3.2,9.6s-5.1,3.4-8.8,3.4s-6.7-1.1-8.9-3.4s-3.2-5.5-3.2-9.6
height="50mm" s1.1-7.2,3.2-9.5s5.1-3.4,8.9-3.4S128.9,15,131,17.3z M116.2,19.9c-1.5,2-2.2,4.4-2.1,6.9c-0.2,2.5,0.6,5,2.1,7
units="mm" c1.5,1.7,3.7,2.7,6,2.6c2.3,0.1,4.4-0.9,5.9-2.6c1.5-2,2.3-4.5,2.1-7c0.1-2.5-0.6-4.9-2.1-6.9c-1.5-1.7-3.6-2.7-5.9-2.6
showgrid="false" C119.9,17.2,117.7,18.2,116.2,19.9z"/>
inkscape:document-rotation="0" <polygon class="st4" points="0,9.1 0,43.7 22.5,51.8 22.5,16.9 46.8,7.9 24.8,0 "/>
inkscape:current-layer="layer1" <polygon class="st5" points="24.3,17.9 24.3,36.8 46.8,44.9 46.8,9.6 "/>
inkscape:document-units="mm" </g>
inkscape:cy="344.49897" <g>
inkscape:cx="468.64708" <g>
inkscape:zoom="0.7" <path class="st6" d="M41.6,17.5H28.2v6.9h10.4v3.3H28.2v10.2h-3.9V14.2h17.2V17.5z"/>
inkscape:pageshadow="2" <path class="st6" d="M45.8,37.9v-18h3.3l0.4,3.2c0.5-1.2,1.2-2.1,2.1-2.7c0.9-0.6,2.1-0.9,3.5-0.9c0.4,0,0.7,0,1.1,0.1
inkscape:pageopacity="0.0" c0.4,0.1,0.7,0.2,0.9,0.3l-0.5,3.4c-0.3-0.1-0.6-0.2-0.9-0.2C55.4,23,54.9,23,54.4,23c-0.7,0-1.5,0.2-2.2,0.6
borderopacity="1.0" c-0.7,0.4-1.3,1-1.8,1.8s-0.7,1.8-0.7,3v9.5H45.8z"/>
bordercolor="#666666" <path class="st6" d="M68.6,19.6c1.8,0,3.3,0.4,4.6,1.1c1.3,0.7,2.4,1.8,3.1,3.2s1.1,3.1,1.1,5c0,1.9-0.4,3.6-1.1,5
pagecolor="#ffffff" c-0.8,1.4-1.8,2.5-3.1,3.2c-1.3,0.7-2.9,1.1-4.6,1.1s-3.3-0.4-4.6-1.1c-1.3-0.7-2.4-1.8-3.2-3.2c-0.8-1.4-1.2-3.1-1.2-5
id="base" /> c0-1.9,0.4-3.6,1.2-5s1.8-2.5,3.2-3.2C65.3,19.9,66.8,19.6,68.6,19.6z M68.6,22.6c-1.1,0-2,0.2-2.8,0.7c-0.8,0.5-1.3,1.2-1.7,2.1
<metadata s-0.6,2.1-0.6,3.5c0,1.3,0.2,2.5,0.6,3.4s1,1.7,1.7,2.2s1.7,0.7,2.8,0.7c1.1,0,2-0.2,2.7-0.7c0.7-0.5,1.3-1.2,1.7-2.2
id="metadata54"> s0.6-2.1,0.6-3.4c0-1.4-0.2-2.5-0.6-3.5s-1-1.6-1.7-2.1C70.6,22.8,69.6,22.6,68.6,22.6z"/>
<rdf:RDF> <path class="st6" d="M89.2,38.3c-1.8,0-3.4-0.3-4.9-1c-1.5-0.7-2.7-1.7-3.5-3l2.7-2.3c0.5,1,1.3,1.8,2.3,2.4
<cc:Work c1,0.6,2.2,0.9,3.6,0.9c1.1,0,2-0.2,2.6-0.6c0.6-0.4,1-0.9,1-1.6c0-0.5-0.2-0.9-0.5-1.2s-0.9-0.6-1.7-0.8l-3.8-0.8
rdf:about=""> c-1.9-0.4-3.3-1-4.1-1.9c-0.8-0.9-1.2-1.9-1.2-3.3c0-1,0.3-1.9,0.9-2.7c0.6-0.8,1.4-1.5,2.5-2s2.5-0.8,4-0.8c1.8,0,3.3,0.3,4.6,1
<dc:format>image/svg+xml</dc:format> c1.3,0.6,2.2,1.5,2.9,2.7l-2.7,2.2c-0.5-1-1.1-1.7-2-2.1c-0.9-0.5-1.8-0.7-2.8-0.7c-0.8,0-1.4,0.1-2,0.3c-0.6,0.2-1,0.5-1.3,0.8
<dc:type c-0.3,0.3-0.4,0.7-0.4,1.2c0,0.5,0.2,0.9,0.5,1.3s1,0.6,1.9,0.8l4.1,0.9c1.7,0.3,2.9,0.9,3.7,1.7c0.7,0.8,1.1,1.8,1.1,2.9
rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> c0,1.2-0.3,2.2-0.9,3c-0.6,0.9-1.5,1.6-2.6,2C92.1,38.1,90.7,38.3,89.2,38.3z"/>
<dc:title></dc:title> <path class="st6" d="M112.8,19.9v3H99.3v-3H112.8z M106.6,14.6v17.9c0,0.9,0.2,1.5,0.7,1.9c0.5,0.4,1.1,0.6,1.9,0.6
</cc:Work> c0.6,0,1.2-0.1,1.7-0.3c0.5-0.2,0.9-0.5,1.3-0.8l0.9,2.8c-0.6,0.5-1.2,0.9-2,1.1c-0.8,0.3-1.7,0.4-2.7,0.4c-1,0-2-0.2-2.8-0.5
</rdf:RDF> s-1.5-0.9-2-1.6c-0.5-0.8-0.7-1.7-0.8-3V15.7L106.6,14.6z"/>
</metadata> <path d="M137.9,17.5h-13.3v6.9h10.4v3.3h-10.4v10.2h-3.9V14.2h17.2V17.5z"/>
<g <path d="M150.9,13.8c2.1,0,4,0.4,5.5,1.2c1.6,0.8,2.9,2,4,3.5l-2.6,2.5c-0.9-1.4-1.9-2.4-3.1-3c-1.1-0.6-2.5-0.9-4-0.9
id="layer1" c-1.2,0-2.1,0.2-2.8,0.5c-0.7,0.3-1.3,0.7-1.6,1.2c-0.3,0.5-0.5,1.1-0.5,1.7c0,0.7,0.3,1.4,0.8,1.9c0.5,0.6,1.5,1,2.9,1.3
inkscape:groupmode="layer" l4.8,1.1c2.3,0.5,3.9,1.3,4.9,2.3c1,1,1.4,2.3,1.4,3.9c0,1.5-0.4,2.7-1.2,3.8c-0.8,1.1-1.9,1.9-3.3,2.5s-3.1,0.9-5,0.9
inkscape:label="Layer 1"> c-1.7,0-3.2-0.2-4.5-0.6c-1.3-0.4-2.5-1-3.5-1.8c-1-0.7-1.8-1.6-2.5-2.6l2.7-2.7c0.5,0.8,1.1,1.6,1.9,2.2
<g c0.8,0.7,1.7,1.2,2.7,1.5c1,0.4,2.2,0.5,3.4,0.5c1.1,0,2.1-0.1,2.9-0.4c0.8-0.3,1.4-0.7,1.8-1.2c0.4-0.5,0.6-1.1,0.6-1.9
id="g424" c0-0.7-0.2-1.3-0.7-1.8c-0.5-0.5-1.3-0.9-2.6-1.2l-5.2-1.2c-1.4-0.3-2.6-0.8-3.6-1.3c-0.9-0.6-1.6-1.3-2.1-2.1s-0.7-1.8-0.7-2.8
transform="matrix(0.35277777,0,0,-0.35277777,63.946468,10.194047)"> c0-1.3,0.4-2.6,1.1-3.7c0.7-1.1,1.8-2,3.2-2.6C147.3,14.1,148.9,13.8,150.9,13.8z"/>
<path </g>
d="m 0,0 v -8.093 h 12.287 v -3.94 H 0 V -24.067 H -4.534 V 3.898 H 15.677 V 0 Z" </g>
style="fill:#00e396;fill-opacity:1;fill-rule:nonzero;stroke:none" </g>
id="path426" />
</g>
<g
transform="matrix(0.35277777,0,0,-0.35277777,-315.43002,107.34005)"
id="g428">
<g
id="g430"
clip-path="url(#clipPath434)">
<g
id="g436"
transform="translate(1112.874,278.2981)">
<path
d="M 0,0 C 1.822,-0.932 3.354,-2.359 4.597,-4.28 L 1.165,-7.373 c -0.791,1.695 -1.779,2.924 -2.966,3.686 -1.186,0.763 -2.768,1.145 -4.745,1.145 -1.949,0 -3.461,-0.389 -4.534,-1.166 -1.074,-0.777 -1.61,-1.772 -1.61,-2.987 0,-1.13 0.523,-2.027 1.568,-2.69 1.045,-0.664 2.909,-1.236 5.593,-1.716 2.514,-0.452 4.512,-1.024 5.995,-1.716 1.483,-0.693 2.564,-1.554 3.242,-2.585 0.677,-1.031 1.016,-2.309 1.016,-3.834 0,-1.639 -0.466,-3.079 -1.398,-4.322 -0.932,-1.243 -2.239,-2.197 -3.919,-2.86 -1.681,-0.664 -3.623,-0.996 -5.826,-0.996 -5.678,0 -9.689,1.892 -12.033,5.678 l 3.178,3.178 c 0.903,-1.695 2.068,-2.939 3.495,-3.729 1.426,-0.791 3.199,-1.186 5.318,-1.186 2.005,0 3.58,0.345 4.724,1.038 1.144,0.692 1.716,1.674 1.716,2.945 0,1.017 -0.516,1.835 -1.547,2.457 -1.031,0.621 -2.832,1.172 -5.402,1.653 -2.571,0.479 -4.618,1.073 -6.143,1.779 -1.526,0.706 -2.635,1.582 -3.326,2.627 -0.693,1.045 -1.039,2.316 -1.039,3.813 0,1.582 0.438,3.023 1.314,4.322 0.875,1.299 2.14,2.33 3.792,3.093 1.653,0.763 3.58,1.144 5.783,1.144 C -4.018,1.398 -1.822,0.932 0,0"
style="fill:#00e396;fill-opacity:1;fill-rule:nonzero;stroke:none"
id="path438" />
</g>
<g
id="g440"
transform="translate(993.0239,277.5454)">
<path
d="m 0,0 c 2.054,-1.831 3.083,-4.465 3.083,-7.902 v -17.935 h -4.484 v 16.366 c 0,2.914 -0.626,5.024 -1.877,6.332 -1.253,1.308 -2.924,1.962 -5.016,1.962 -1.495,0 -2.896,-0.327 -4.204,-0.981 -1.308,-0.654 -2.381,-1.719 -3.222,-3.194 -0.841,-1.477 -1.261,-3.335 -1.261,-5.576 v -14.909 h -4.484 V 1.328 l 4.086,-1.674 0.118,-1.84 c 0.933,1.681 2.222,2.923 3.867,3.727 1.643,0.803 3.493,1.205 5.548,1.205 C -4.671,2.746 -2.055,1.83 0,0"
style="fill:#000033;fill-opacity:1;fill-rule:nonzero;stroke:none"
id="path442" />
</g>
<g
id="g444"
transform="translate(1027.9968,264.0386)">
<path
d="m 0,0 h -21.128 c 0.261,-2.84 1.205,-5.044 2.83,-6.613 1.625,-1.57 3.727,-2.355 6.305,-2.355 2.054,0 3.763,0.356 5.128,1.065 1.363,0.71 2.288,1.738 2.774,3.083 l 3.755,-1.961 c -1.121,-1.981 -2.616,-3.495 -4.484,-4.54 -1.868,-1.046 -4.259,-1.569 -7.173,-1.569 -4.223,0 -7.538,1.289 -9.948,3.867 -2.41,2.578 -3.615,6.146 -3.615,10.704 0,4.558 1.149,8.127 3.447,10.705 2.298,2.578 5.557,3.867 9.779,3.867 2.615,0 4.876,-0.58 6.782,-1.738 1.905,-1.158 3.343,-2.728 4.315,-4.707 C -0.262,7.827 0.224,5.605 0.224,3.139 0.224,2.092 0.149,1.046 0,0 m -18.298,10.144 c -1.513,-1.457 -2.438,-3.512 -2.775,-6.165 h 16.982 c -0.3,2.615 -1.159,4.661 -2.578,6.137 -1.42,1.476 -3.307,2.214 -5.661,2.214 -2.466,0 -4.455,-0.728 -5.968,-2.186"
style="fill:#000033;fill-opacity:1;fill-rule:nonzero;stroke:none"
id="path446" />
</g>
<g
id="g448"
transform="translate(1057.8818,276.4246)">
<path
d="m 0,0 c 2.41,-2.578 3.615,-6.147 3.615,-10.705 0,-4.558 -1.205,-8.126 -3.615,-10.704 -2.41,-2.578 -5.726,-3.867 -9.948,-3.867 -4.222,0 -7.537,1.289 -9.947,3.867 -2.41,2.578 -3.615,6.146 -3.615,10.704 0,4.558 1.205,8.127 3.615,10.705 2.41,2.578 5.725,3.867 9.947,3.867 C -5.726,3.867 -2.41,2.578 0,0 m -16.617,-2.858 c -1.607,-1.906 -2.41,-4.522 -2.41,-7.847 0,-3.326 0.803,-5.94 2.41,-7.846 1.607,-1.905 3.83,-2.858 6.669,-2.858 2.839,0 5.063,0.953 6.67,2.858 1.606,1.906 2.41,4.52 2.41,7.846 0,3.325 -0.804,5.941 -2.41,7.847 C -4.885,-0.953 -7.109,0 -9.948,0 c -2.839,0 -5.062,-0.953 -6.669,-2.858"
style="fill:#000033;fill-opacity:1;fill-rule:nonzero;stroke:none"
id="path450" />
</g>
</g>
</g>
<g
id="g452"
transform="matrix(0.35277777,0,0,-0.35277777,5.8329581,6.5590171)">
<path
d="m 0,0 0.001,-38.946 25.286,-9.076 V -8.753 L 52.626,1.321 27.815,10.207 Z"
style="fill:#00e599;fill-opacity:1;fill-rule:nonzero;stroke:none"
id="path454" />
</g>
<g
id="g456"
transform="matrix(0.35277777,0,0,-0.35277777,15.479008,10.041927)">
<path
d="M 0,0 V -21.306 L 25.293,-30.364 25.282,9.347 Z"
style="fill:#00b091;fill-opacity:1;fill-rule:nonzero;stroke:none"
id="path458" />
</g>
</g>
</svg> </svg>

Before

Width:  |  Height:  |  Size: 6.5 KiB

After

Width:  |  Height:  |  Size: 5.5 KiB

View file

@ -1,28 +0,0 @@
name: CHANGELOG check
on:
pull_request:
branches:
- master
jobs:
build:
runs-on: ubuntu-latest
name: Check for updates
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Get changed CHANGELOG
id: changelog-diff
uses: tj-actions/changed-files@v29
with:
files: CHANGELOG.md
- name: Fail if changelog not updated
if: steps.changelog-diff.outputs.any_changed == 'false'
uses: actions/github-script@v3
with:
script: |
core.setFailed('CHANGELOG.md has not been updated')

View file

@ -1,36 +0,0 @@
name: Configuration check
on:
pull_request:
branches:
- master
jobs:
build:
runs-on: ubuntu-latest
name: config-check
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Get changed config-related files
id: config-diff
uses: tj-actions/changed-files@v29
with:
files: |
config/**
cmd/neofs-node/config/**
- name: Get changed doc files
id: docs-diff
uses: tj-actions/changed-files@v29
with:
files: docs/**
- name: Fail if config files are changed but the documentation is not updated
if: steps.config-diff.outputs.any_changed == 'true' && steps.docs-diff.outputs.any_changed == 'false'
uses: actions/github-script@v3
with:
script: |
core.setFailed('Documentation has not been updated')

View file

@ -1,21 +0,0 @@
name: DCO check
on:
pull_request:
branches:
- master
jobs:
commits_check_job:
runs-on: ubuntu-latest
name: Commits Check
steps:
- name: Get PR Commits
id: 'get-pr-commits'
uses: tim-actions/get-pr-commits@master
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: DCO Check
uses: tim-actions/dco@master
with:
commits: ${{ steps.get-pr-commits.outputs.commits }}

View file

@ -1,58 +0,0 @@
name: neofs-node tests
on:
push:
branches:
- master
paths-ignore:
- '*.md'
pull_request:
branches:
- master
paths-ignore:
- '*.md'
jobs:
test:
runs-on: ubuntu-20.04
strategy:
matrix:
go: [ '1.17.x', '1.18.x', '1.19.x' ]
steps:
- name: Setup go
uses: actions/setup-go@v3
with:
go-version: ${{ matrix.go }}
- name: Check out code
uses: actions/checkout@v3
- name: Cache go mod
uses: actions/cache@v3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ matrix.go }}-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ matrix.go }}-
- name: Run go test
run: go test -coverprofile=coverage.txt -covermode=atomic ./...
- name: Codecov
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
run: bash <(curl -s https://codecov.io/bash)
lint:
runs-on: ubuntu-20.04
steps:
- uses: actions/setup-go@v3
with:
go-version: 1.19
- uses: actions/checkout@v3
- name: golangci-lint
uses: golangci/golangci-lint-action@v3
with:
version: v1.50.0
args: --timeout=5m
only-new-issues: true

20
.gitignore vendored
View file

@ -30,4 +30,22 @@ testfile
.neofs-cli.yml .neofs-cli.yml
# debhelpers # debhelpers
**/.debhelper debian/*debhelper*
# logfiles
debian/*.log
# .substvars
debian/*.substvars
# .bash-completion
debian/*.bash-completion
# Install folders and files
debian/frostfs-cli/
debian/frostfs-ir/
debian/files
debian/frostfs-storage/
debian/changelog
man/
debs/

11
.gitlint Normal file
View file

@ -0,0 +1,11 @@
[general]
fail-without-commits=true
regex-style-search=true
contrib=CC1
[title-match-regex]
regex=^\[\#[0-9X]+\]\s
[ignore-by-title]
regex=^Release(.*)
ignore=title-match-regex

View file

@ -4,7 +4,7 @@
# options for analysis running # options for analysis running
run: run:
# timeout for analysis, e.g. 30s, 5m, default is 1m # timeout for analysis, e.g. 30s, 5m, default is 1m
timeout: 5m timeout: 10m
# include test files or not, default is true # include test files or not, default is true
tests: false tests: false
@ -24,6 +24,13 @@ linters-settings:
govet: govet:
# report about shadowed variables # report about shadowed variables
check-shadowing: false check-shadowing: false
staticcheck:
checks: ["all", "-SA1019"] # TODO Enable SA1019 after deprecated warning are fixed.
funlen:
lines: 80 # default 60
statements: 60 # default 40
gocognit:
min-complexity: 40 # default 30
linters: linters:
enable: enable:
@ -51,6 +58,9 @@ linters:
- predeclared - predeclared
- reassign - reassign
- whitespace - whitespace
- containedctx
- funlen
- gocognit
- contextcheck
disable-all: true disable-all: true
fast: false fast: false

35
.pre-commit-config.yaml Normal file
View file

@ -0,0 +1,35 @@
ci:
autofix_prs: false
repos:
- repo: https://github.com/jorisroovers/gitlint
rev: v0.19.1
hooks:
- id: gitlint
stages: [commit-msg]
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: check-added-large-files
- id: check-case-conflict
- id: check-executables-have-shebangs
- id: check-shebang-scripts-are-executable
- id: check-merge-conflict
- id: check-json
- id: check-xml
- id: check-yaml
- id: trailing-whitespace
args: [--markdown-linebreak-ext=md]
- id: end-of-file-fixer
exclude: ".key$"
- repo: https://github.com/shellcheck-py/shellcheck-py
rev: v0.9.0.2
hooks:
- id: shellcheck
- repo: https://github.com/golangci/golangci-lint
rev: v1.51.2
hooks:
- id: golangci-lint

View file

@ -0,0 +1,17 @@
pipeline:
# Kludge for non-root containers under WoodPecker
fix-ownership:
image: alpine:latest
commands: chown -R 1234:1234 .
pre-commit:
image: git.frostfs.info/truecloudlab/frostfs-ci:v0.36
commands:
- export HOME="$(getent passwd $(id -u) | cut '-d:' -f6)"
- pre-commit run
unit:
image: git.frostfs.info/truecloudlab/frostfs-ci:v0.36
commands:
- export HOME="$(getent passwd $(id -u) | cut '-d:' -f6)"
- make test

View file

@ -1,14 +1,170 @@
# Changelog # Changelog
Changelog for NeoFS Node Changelog for FrostFS Node
## [Unreleased] ## [Unreleased]
### Added ### Added
- Add GAS pouring mechanism for a configurable list of wallets (#128)
- Separate batching for replicated operations over the same container in pilorama (#1621)
- Doc for extended headers (#2128)
- New `frostfs_node_object_container_size` metric for tracking size of reqular objects in a container (#2116)
- New `frostfs_node_object_payload_size` metric for tracking size of reqular objects on a single shard (#1794)
- Add command `frostfs-adm morph netmap-candidates` (#1889)
- `object.delete.tombstone_lifetime` config parameter to set tombstone lifetime in the DELETE service (#2246)
- Reload config for pprof and metrics on SIGHUP in `neofs-node` (#1868)
- Multiple configs support (#44)
- Parameters `nns-name` and `nns-zone` for command `frostfs-cli container create` (#37)
- Tree service now saves the last synchronization height which persists across restarts (#82)
### Changed ### Changed
- Change `frostfs_node_engine_container_size` to counting sizes of logical objects
- `common.PrintVerbose` prints via `cobra.Command.Printf` (#1962)
- Env prefix in configuration changed to `FROSTFS_*` (#43)
- Link object is broadcast throughout the whole container now (#57)
- Pilorama now can merge multiple batches into one (#2231)
- Storage engine now can start even when some shard components are unavailable (#2238)
- `neofs-cli` buffer for object put increased from 4 KiB to 3 MiB (#2243)
- Expired locked object is available for reading (#56)
- Initialize write-cache asynchronously (#32)
### Fixed ### Fixed
- Increase payload size metric on shards' `put` operation (#1794)
- Big object removal with non-local parts (#1978)
- Disable pilorama when moving to degraded mode (#2197)
- Fetching blobovnicza objects that not found in write-cache (#2206)
- Do not search for the small objects in FSTree (#2206)
- Correct status error for expired session token (#2207)
- Set flag `mode` required for `frostfs-cli control shards set-mode` (#8)
- Fix `dirty` suffix in debian package version (#53)
- Prevent node process from killing by systemd when shutting down (#1465)
- Restore subscriptions correctly on morph client switch (#2212)
- Expired objects could be returned if not marked with GC yet (#2213)
- `neofs-adm morph dump-hashes` now properly iterates over custom domain (#2224)
- Possible deadlock in write-cache (#2239)
- Fix `*_req_count` and `*_req_count_success` metric values (#2241)
- Storage ID update by write-cache (#2244)
- `neo-go` client deadlock on subscription restoration (#2244)
- Possible panic during write-cache initialization (#2234)
- Do not fetch an object if `meta` is missing it (#61)
- Create contract wallet only by `init` and `update-config` command (#63)
- Actually use `object.put.pool_size_local` and independent pool for local puts (#64).
- Pretty printer of basic ACL in the NeoFS CLI (#2259)
- Adding of public key for nns group `group.frostfs` at init step (#130)
### Removed ### Removed
### Updated ### Updated
- `neo-go` to `v0.100.1`
- `github.com/klauspost/compress` to `v1.15.13`
- `github.com/multiformats/go-multiaddr` to `v0.8.0`
- `golang.org/x/term` to `v0.3.0`
- `google.golang.org/grpc` to `v1.51.0`
- `github.com/nats-io/nats.go` to `v1.22.1`
- `github.com/TrueCloudLab/hrw` to `v.1.1.1`
- Minimum go version to v1.18
### Updating from v0.35.0
You need to change configuration environment variables to `FROSTFS_*` if you use any.
New config field `object.delete.tombstone_lifetime` allows to set tombstone lifetime
more appropriate for a specific deployment.
## [0.35.0] - 2022-12-28 - Sindo (신도, 信島)
### Added
- `morph list-containers` in `neofs-adm` (#1689)
- `--binary` flag in `neofs-cli object put/get/delete` commands (#1338)
- `session` flag support to `neofs-cli object hash` (#2029)
- Shard can now change mode when encountering background disk errors (#2035)
- Background workers and object service now use separate client caches (#2048)
- `replicator.pool_size` config field to tune replicator pool size (#2049)
- Fix NNS hash parsing in morph client (#2063)
- `neofs-cli neofs-cli acl basic/extended print` commands (#2012)
- `neofs_node_object_*_req_count_success` prometheus metrics for tracking successfully executed requests (#1984)
- Metric 'readonly' to get shards mode (#2022)
- Tree service replication timeout (#2159)
- `apiclient.reconnect_timeout` setting allowing to ignore failed clients for some time (#2164)
### Changed
- `object lock` command reads CID and OID the same way other commands do (#1971)
- `LOCK` object are stored on every container node (#1502)
- `neofs-cli container get-eacl` print ACL table in json format only with arg `--json' (#2012)
- Side chain notary deposits use max uint32 as till parameter (#1486)
- Allow object removal without linking object (#2100)
- `neofs-cli container delete` command pre-checks container ownership (#2106)
- Policer cache size is now 1024 (#2158)
- Tree service now synchronizes with container nodes in a random order (#2127)
- Pilorama no longer tries to apply already applied operations (#2161)
- Use `sync.Pool` in Object.PUT service (#2139)
- Shard uses metabase for `HEAD` requests by default, not write-cache (#2167)
- Clarify help for `--expire-at` parameter for commands `object lock/put` and `bearer create` (#2097)
- Node spawns `GETRANGE` requests signed with the node's key if session key was not found for `RANGEHASH` (#2144)
- Full list of container is no longer cached (#2176)
### Fixed
- Open FSTree in sync mode by default (#1992)
- `neofs-cli container nodes`'s output (#1991)
- Increase error counter for write-cache flush errors (#1818)
- Correctly select the shard for applying tree service operations (#1996)
- Do not panic and return correct errors for bad inputs in `GET_RANGE` (#2007, #2024)
- Physical child object removal by GC (#1699)
- Broadcasting helper objects (#1972)
- `neofs-cli lock object`'s `lifetime` flag handling (#1972)
- Do not move write-cache in read-only mode for flushing (#1906)
- Child object collection on CLI side with a bearer token (#2000)
- Fix concurrent map writes in `Object.Put` service (#2037)
- Malformed request errors' reasons in the responses (#2028)
- Session token's IAT and NBF checks in ACL service (#2028)
- Losing meta information on request forwarding (#2040)
- Assembly process triggered by a request with a bearer token (#2040)
- Losing locking context after metabase resync (#1502)
- Removing all trees by container ID if tree ID is empty in `pilorama.Forest.TreeDrop` (#1940)
- Concurrent mode changes in the metabase and blobstor (#2057)
- Panic in IR when performing HEAD requests (#2069)
- Write-cache flush duplication (#2074)
- Ignore error if a transaction already exists in a morph client (#2075)
- ObjectID signature output in the CLI (#2104)
- Pack arguments of `setPrice` invocation during contract update (#2078)
- `neofs-cli object hash` panic (#2079)
- Closing `neo-go` WS clients on shutdown and switch processes (#2080)
- Making notary deposits with a zero GAS balance (#2080)
- Notary requests on shutdown (#2075)
- `neofs-cli container create ` check the sufficiency of the number of nodes in the selector for replicas (#2038)
- Data duplication during request forwarding (#2047)
- Tree service panic on `TreeMove` operation (#2140)
- Panic in `GETRANGE` with zero length (#2095)
- Spawning useless `GETRANGE` with zero length for a big object (#2101)
- Incomplete object put errors do contain the deepest error's message (#2092)
- Prioritize internal addresses for clients (#2156)
- Force object removal via control service (#2145)
- Synchronizing a tree now longer reports an error for a single-node container (#2154)
- Prevent leaking goroutines in the tree service (#2162)
- Do not search for LOCK objects when delete container when session provided (#2152)
- Race conditions on shard's mode switch (#1956)
- Returning expired/removed objects from write-cache (#2016)
### Removed
- `-g` option from `neofs-cli control ...` and `neofs-cli container create` commands (#2089)
- `--header` from `neofs-cli object get` (#2090)
### Updated
- `neo-go` to `v0.100.0`
- `spf13/cobra` to `v1.6.1`
- `spf13/viper` to `v1.8.0`
- `google.golang.org/grpc` to `v1.50.1`
### Updating from v0.34.0 ### Updating from v0.34.0
Pass CID and OID parameters via the `--cid` and `--oid` flags, not as the command arguments.
Replicator pool size can now be fine-tuned with `replicator.pool_size` config field.
The default value is taken from `object.put.pool_size_remote` as in earlier versions.
Added `neofs_node_object_*_req_count_success` metrics for tracking successfully executed requests.
`neofs-cli container delete` command now requires given account or session issuer
to match the container owner. Use `--force` (`-f`) flag to bypass this requirement.
Tree service network replication can now be fine-tuned with `tree.replication_timeout` config field.
## [0.34.0] - 2022-10-31 - Marado (마라도, 馬羅島) ## [0.34.0] - 2022-10-31 - Marado (마라도, 馬羅島)
@ -84,7 +240,7 @@ command.
- Policer marks nodes under maintenance as OK without requests (#1680) - Policer marks nodes under maintenance as OK without requests (#1680)
- Unify help messages in CLI (#1854) - Unify help messages in CLI (#1854)
- `evacuate`, `set-mode` and `flush-cache` control subcommands now accept a list of shard ids (#1867) - `evacuate`, `set-mode` and `flush-cache` control subcommands now accept a list of shard ids (#1867)
- Reading `object` commands of NeoFS CLI don't open remote sessions (#1865) - Reading `object` commands of NeoFS CLI don't open remote sessions (#1865)
- Use hex format to print storage node ID (#1765) - Use hex format to print storage node ID (#1765)
### Fixed ### Fixed
@ -112,7 +268,7 @@ command.
- `neofs-contract` to `v0.16.0` - `neofs-contract` to `v0.16.0`
- `neofs-api-go` to `v2.14.0` - `neofs-api-go` to `v2.14.0`
### Updating from v0.32.0 ### Updating from v0.32.0
Replace using the `control netmap-snapshot` command with `netmap snapshot` one in NeoFS CLI. Replace using the `control netmap-snapshot` command with `netmap snapshot` one in NeoFS CLI.
Node can now specify additional addresses in `ExternalAddr` attribute. To allow a node to dial Node can now specify additional addresses in `ExternalAddr` attribute. To allow a node to dial
other nodes external address, use `apiclient.allow_external` config setting. other nodes external address, use `apiclient.allow_external` config setting.
@ -122,7 +278,7 @@ Pass `maintenance` state to `neofs-cli control set-status` to enter maintenance
If network allows maintenance state (*), it will be reflected in the network map. If network allows maintenance state (*), it will be reflected in the network map.
Storage nodes under maintenance are not excluded from the network map, but don't Storage nodes under maintenance are not excluded from the network map, but don't
serve object operations. (*) can be fetched from network configuration via serve object operations. (*) can be fetched from network configuration via
`neofs-cli netmap netinfo` command. `neofs-cli netmap netinfo` command.
To allow maintenance mode during neofs-adm deployments, set To allow maintenance mode during neofs-adm deployments, set
`network.maintenance_mode_allowed` parameter in config. `network.maintenance_mode_allowed` parameter in config.
@ -417,15 +573,15 @@ Clean up all metabases and re-sync them using `resync_metabase` config flag.
- Reduced amount of slices with pointers (#1239) - Reduced amount of slices with pointers (#1239)
### Updating from v0.28.0-rc.2 ### Updating from v0.28.0-rc.2
Remove `NEOFS_IR_MAINNET_ENDPOINT_NOTIFICATION`, Remove `NEOFS_IR_MAINNET_ENDPOINT_NOTIFICATION`,
`NEOFS_IR_MORPH_ENDPOINT_NOTIFICATION`, and `NEOFS_MORPH_NOTIFICATION_ENDPOINT` `NEOFS_IR_MORPH_ENDPOINT_NOTIFICATION`, and `NEOFS_MORPH_NOTIFICATION_ENDPOINT`
from Inner Ring and Storage configurations. from Inner Ring and Storage configurations.
Specify _WebSocket_ endpoints in `NEOFS_IR_MAINNET_ENDPOINT_CLIENT`, Specify _WebSocket_ endpoints in `NEOFS_IR_MAINNET_ENDPOINT_CLIENT`,
`NEOFS_IR_MORPH_ENDPOINT_CLIENT`, and `NEOFS_MORPH_RPC_ENDPOINT` at Inner Ring `NEOFS_IR_MORPH_ENDPOINT_CLIENT`, and `NEOFS_MORPH_RPC_ENDPOINT` at Inner Ring
and Storage configurations. and Storage configurations.
Specify path to persistent session token db in Storage configuration with Specify path to persistent session token db in Storage configuration with
`NEOFS_NODE_PERSISTENT_SESSIONS_PATH`. `NEOFS_NODE_PERSISTENT_SESSIONS_PATH`.
## [0.28.0-rc.2] - 2022-03-24 ## [0.28.0-rc.2] - 2022-03-24
@ -441,7 +597,7 @@ Specify path to persistent session token db in Storage configuration with
## [0.28.0-rc.1] - 2022-03-18 ## [0.28.0-rc.1] - 2022-03-18
Native RFC-6979 signatures of messages and tokens, LOCK object types, Native RFC-6979 signatures of messages and tokens, LOCK object types,
experimental notifications over NATS with NeoFS API v2.12 support experimental notifications over NATS with NeoFS API v2.12 support
### Fixed ### Fixed
@ -477,8 +633,8 @@ experimental notifications over NATS with NeoFS API v2.12 support
- Deprecated structures from SDK v1.0.0 rc (#1181) - Deprecated structures from SDK v1.0.0 rc (#1181)
### Updating from neofs-node v0.27.5 ### Updating from neofs-node v0.27.5
Set shard error threshold for read-only mode switch with Set shard error threshold for read-only mode switch with
`NEOFS_STORAGE_SHARD_RO_ERROR_THRESHOLD` (default: 0, deactivated). `NEOFS_STORAGE_SHARD_RO_ERROR_THRESHOLD` (default: 0, deactivated).
Set NATS configuration for notifications in `NEOFS_NODE_NOTIFICATION` section. Set NATS configuration for notifications in `NEOFS_NODE_NOTIFICATION` section.
See example config for more details. See example config for more details.
@ -544,7 +700,7 @@ See example config for more details.
Use `--wallet` key in CLI to provide WIF or binary key file instead of `--wif` Use `--wallet` key in CLI to provide WIF or binary key file instead of `--wif`
and `--binary-key`. and `--binary-key`.
Replace `NEOFS_STORAGE_SHARD_N_USE_WRITE_CACHE` with Replace `NEOFS_STORAGE_SHARD_N_USE_WRITE_CACHE` with
`NEOFS_STORAGE_SHARD_N_WRITECACHE_ENABLED` in Storage node config. `NEOFS_STORAGE_SHARD_N_WRITECACHE_ENABLED` in Storage node config.
Specify `password: xxx` in config file for NeoFS CLI to avoid password input. Specify `password: xxx` in config file for NeoFS CLI to avoid password input.
@ -621,7 +777,7 @@ NeoFS API v2.11.0 support with response status codes and storage subnetworks.
- CLI now opens LOCODE database in read-only mode for listing command (#958) - CLI now opens LOCODE database in read-only mode for listing command (#958)
- Tombstone owner now is always set (#842) - Tombstone owner now is always set (#842)
- Node in relay mode does not require shard config anymore (#969) - Node in relay mode does not require shard config anymore (#969)
- Alphabet nodes now ignore notary notifications with non-HALT main tx (#976) - Alphabet nodes now ignore notary notifications with non-HALT main tx (#976)
- neofs-adm now prints version of NNS contract (#1014) - neofs-adm now prints version of NNS contract (#1014)
- Possible NPE in blobovnicza (#1007) - Possible NPE in blobovnicza (#1007)
- More precise calculation of blobovnicza size (#915) - More precise calculation of blobovnicza size (#915)
@ -638,13 +794,13 @@ NeoFS API v2.11.0 support with response status codes and storage subnetworks.
- Alphabet nodes resign `AddPeer` request if it updates Storage node info (#938) - Alphabet nodes resign `AddPeer` request if it updates Storage node info (#938)
- All applications now use client from neofs-sdk-go library (#966) - All applications now use client from neofs-sdk-go library (#966)
- Some shard configuration records were renamed, see upgrading section (#859) - Some shard configuration records were renamed, see upgrading section (#859)
- `Nonce` and `VUB` values of notary transactions generated from notification - `Nonce` and `VUB` values of notary transactions generated from notification
hash (#844) hash (#844)
- Non alphabet notary invocations now have 4 witnesses (#975) - Non alphabet notary invocations now have 4 witnesses (#975)
- Object replication is now async and continuous (#965) - Object replication is now async and continuous (#965)
- NeoFS ADM updated for the neofs-contract v0.13.0 deploy (#984) - NeoFS ADM updated for the neofs-contract v0.13.0 deploy (#984)
- Minimal TLS version is set to v1.2 (#878) - Minimal TLS version is set to v1.2 (#878)
- Alphabet nodes now invoke `netmap.Register` to add node to the network map - Alphabet nodes now invoke `netmap.Register` to add node to the network map
candidates in notary enabled environment (#1008) candidates in notary enabled environment (#1008)
### Upgrading from v0.26.1 ### Upgrading from v0.26.1
@ -674,7 +830,7 @@ with `NEOFS_IR_FEE_NAMED_CONTAINER_REGISTER`.
### Fixed ### Fixed
- Storage Node handles requests before its initialization is finished (#934) - Storage Node handles requests before its initialization is finished (#934)
- Release worker pools gracefully (#901) - Release worker pools gracefully (#901)
- Metabase ignored containers of storage group and tombstone objects - Metabase ignored containers of storage group and tombstone objects
in listing (#945) in listing (#945)
- CLI missed endpoint flag in `control netmap-snapshot` command (#942) - CLI missed endpoint flag in `control netmap-snapshot` command (#942)
- Write cache object persisting (#866) - Write cache object persisting (#866)
@ -688,16 +844,16 @@ with `NEOFS_IR_FEE_NAMED_CONTAINER_REGISTER`.
### Changed ### Changed
- Use FSTree counter in write cache (#821) - Use FSTree counter in write cache (#821)
- Calculate notary deposit `till` parameter depending on available - Calculate notary deposit `till` parameter depending on available
deposit (#910) deposit (#910)
- Storage node returns session token error if attached token's private key - Storage node returns session token error if attached token's private key
is not available (#943) is not available (#943)
- Refactor of NeoFS API client in inner ring (#946) - Refactor of NeoFS API client in inner ring (#946)
- LOCODE generator tries to find the closest continent if there are - LOCODE generator tries to find the closest continent if there are
no exact match (#955) no exact match (#955)
### Upgrading from v0.26.0 ### Upgrading from v0.26.0
You can specify default section in storage engine configuration. You can specify default section in storage engine configuration.
See [example](./config/example/node.yaml) for more details. See [example](./config/example/node.yaml) for more details.
## [0.26.0] - 2021-10-19 - Udo (우도, 牛島) ## [0.26.0] - 2021-10-19 - Udo (우도, 牛島)
@ -707,7 +863,7 @@ NeoFS API v2.10 support
### Fixed ### Fixed
- Check remote node public key in every response message (#645) - Check remote node public key in every response message (#645)
- Do not lose local container size estimations (#872) - Do not lose local container size estimations (#872)
- Compressed and uncompressed objects are always available for reading - Compressed and uncompressed objects are always available for reading
regardless of compression configuration (#868) regardless of compression configuration (#868)
- Use request session token in ACL check of object.Put (#881) - Use request session token in ACL check of object.Put (#881)
- Parse URI in neofs-cli properly (#883) - Parse URI in neofs-cli properly (#883)
@ -761,7 +917,7 @@ instead.
### Added ### Added
- Support of multiple Neo RPC endpoints in Inner Ring node (#792) - Support of multiple Neo RPC endpoints in Inner Ring node (#792)
`mainchain` section of storage node config is left unused by the application. `mainchain` section of storage node config is left unused by the application.
## [0.25.0] - 2021-09-27 - Mungapdo (문갑도, 文甲島) ## [0.25.0] - 2021-09-27 - Mungapdo (문갑도, 文甲島)
@ -769,7 +925,7 @@ instead.
- Work of a storage node with one Neo RPC endpoint instead of a list (#746) - Work of a storage node with one Neo RPC endpoint instead of a list (#746)
- Lack of support for HEAD operation on the object write cache (#762) - Lack of support for HEAD operation on the object write cache (#762)
- Storage node attribute parsing is stable now (#787) - Storage node attribute parsing is stable now (#787)
- Inner Ring node now logs transaction hashes of Deposit and Withdraw events - Inner Ring node now logs transaction hashes of Deposit and Withdraw events
in LittleEndian encoding (#794) in LittleEndian encoding (#794)
- Storage node uses public keys of the remote nodes in placement traverser - Storage node uses public keys of the remote nodes in placement traverser
checks (#645) checks (#645)
@ -777,7 +933,7 @@ instead.
(#816) (#816)
- neofs-adm supports update and deploy of neofs-contract v0.11.0 (#834, #836) - neofs-adm supports update and deploy of neofs-contract v0.11.0 (#834, #836)
- Possible NPE in public key conversion (#848) - Possible NPE in public key conversion (#848)
- Object assembly routine do not forward existing request instead of creating - Object assembly routine do not forward existing request instead of creating
new one (#839) new one (#839)
- Shard now returns only physical stored objects for replication (#840) - Shard now returns only physical stored objects for replication (#840)
@ -786,7 +942,7 @@ instead.
- Smart contract address auto negotiation with NNS contract (#736) - Smart contract address auto negotiation with NNS contract (#736)
- Detailed logs for all data writing operations in storage engine (#790) - Detailed logs for all data writing operations in storage engine (#790)
- Docker build and release targets in Makefile (#785) - Docker build and release targets in Makefile (#785)
- Metabase restore option in the shard config (#789) - Metabase restore option in the shard config (#789)
- Write cache used size limit in bytes (#776) - Write cache used size limit in bytes (#776)
### Changed ### Changed
@ -821,7 +977,7 @@ Added `NEOFS_STORAGE_SHARD_<N>_WRITECACHE_SIZE_LIMIT` where `<N>` is shard ID.
This is the size limit for the all write cache storages combined in bytes. Default This is the size limit for the all write cache storages combined in bytes. Default
size limit is 1 GiB. size limit is 1 GiB.
Added `NEOFS_STORAGE_SHARD_<N>_REFILL_METABASE` bool flag where `<N>` is shard Added `NEOFS_STORAGE_SHARD_<N>_REFILL_METABASE` bool flag where `<N>` is shard
ID. This flag purges metabase instance at the application start and reinitialize ID. This flag purges metabase instance at the application start and reinitialize
it with available objects from the blobstor. it with available objects from the blobstor.
@ -830,12 +986,12 @@ Object service pool size now split into `NEOFS_OBJECT_PUT_POOL_SIZE_REMOTE` and
## [0.24.1] - 2021-09-07 ## [0.24.1] - 2021-09-07
### Fixed ### Fixed
- Storage and Inner Ring will not start until Neo RPC node will have the height - Storage and Inner Ring will not start until Neo RPC node will have the height
of the latest processed block by the nodes (#795) of the latest processed block by the nodes (#795)
### Upgrading from v0.24.0 ### Upgrading from v0.24.0
Specify path to the local state DB in Inner Ring node config with Specify path to the local state DB in Inner Ring node config with
`NEOFS_IR_NODE_PERSISTENT_STATE_PATH`. Specify path to the local state DB in `NEOFS_IR_NODE_PERSISTENT_STATE_PATH`. Specify path to the local state DB in
Storage node config with `NEOFS_NODE_PERSISTENT_STATE_PATH`. Storage node config with `NEOFS_NODE_PERSISTENT_STATE_PATH`.
@ -854,7 +1010,7 @@ Storage node config with `NEOFS_NODE_PERSISTENT_STATE_PATH`.
- Contract update support in `neofs-adm` utility (#748) - Contract update support in `neofs-adm` utility (#748)
- Container transferring support in `neofs-adm` utility (#755) - Container transferring support in `neofs-adm` utility (#755)
- Storage Node's balance refilling support in `neofs-adm` utility (#758) - Storage Node's balance refilling support in `neofs-adm` utility (#758)
- Support `COMMON_PREFIX` filter for object attributes in storage engine and `neofs-cli` (#760) - Support `COMMON_PREFIX` filter for object attributes in storage engine and `neofs-cli` (#760)
- Node's and IR's notary status debug message on startup (#758) - Node's and IR's notary status debug message on startup (#758)
- Go `1.17` unit tests in CI (#766) - Go `1.17` unit tests in CI (#766)
- Supporting all eACL filter fields from the specification (#768) - Supporting all eACL filter fields from the specification (#768)
@ -916,7 +1072,7 @@ Improved stability for notary disabled environment.
- Storage Node configuration example contains usable parameters (#699) - Storage Node configuration example contains usable parameters (#699)
### Fixed ### Fixed
- Do not use side chain RoleManagement contract as source of Inner Ring list - Do not use side chain RoleManagement contract as source of Inner Ring list
when notary disabled in side chain (#672) when notary disabled in side chain (#672)
- Alphabet list transition is even more effective (#697) - Alphabet list transition is even more effective (#697)
- Inner Ring node does not require proxy and processing contracts if notary - Inner Ring node does not require proxy and processing contracts if notary
@ -983,9 +1139,9 @@ Storage nodes with a group of network endpoints.
- Control service with healthcheck RPC in IR and CLI support ([#414](https://github.com/nspcc-dev/neofs-node/issues/414)). - Control service with healthcheck RPC in IR and CLI support ([#414](https://github.com/nspcc-dev/neofs-node/issues/414)).
### Fixed ### Fixed
- Approval of objects with with duplicate attribute keys or empty values ([#633](https://github.com/nspcc-dev/neofs-node/issues/633)). - Approval of objects with with duplicate attribute keys or empty values ([#633](https://github.com/nspcc-dev/neofs-node/issues/633)).
- Approval of containers with with duplicate attribute keys or empty values ([#634](https://github.com/nspcc-dev/neofs-node/issues/634)). - Approval of containers with with duplicate attribute keys or empty values ([#634](https://github.com/nspcc-dev/neofs-node/issues/634)).
- Default path for CLI config ([#626](https://github.com/nspcc-dev/neofs-node/issues/626)). - Default path for CLI config ([#626](https://github.com/nspcc-dev/neofs-node/issues/626)).
### Changed ### Changed
- `version` command replaced with `--version` flag in CLI ([#571](https://github.com/nspcc-dev/neofs-node/issues/571)). - `version` command replaced with `--version` flag in CLI ([#571](https://github.com/nspcc-dev/neofs-node/issues/571)).
@ -1013,7 +1169,7 @@ Storage nodes with a group of network endpoints.
- grpc: [v1.38.0](https://github.com/grpc/grpc-go/releases/tag/v1.38.0). - grpc: [v1.38.0](https://github.com/grpc/grpc-go/releases/tag/v1.38.0).
- cast: [v1.3.1](https://github.com/spf13/cast/releases/tag/v1.3.1). - cast: [v1.3.1](https://github.com/spf13/cast/releases/tag/v1.3.1).
- cobra: [1.1.3](https://github.com/spf13/cobra/releases/tag/v1.1.3). - cobra: [1.1.3](https://github.com/spf13/cobra/releases/tag/v1.1.3).
- viper: [v1.8.1](https://github.com/spf13/viper/releases/tag/v1.8.1). - viper: [v1.8.1](https://github.com/spf13/viper/releases/tag/v1.8.1).
## [0.21.1] - 2021-06-10 ## [0.21.1] - 2021-06-10
@ -1033,7 +1189,7 @@ Session token support in container service, refactored config in storage node,
TLS support on gRPC servers. TLS support on gRPC servers.
### Fixed ### Fixed
- ACL service traverses over all RequestMetaHeader chain to find - ACL service traverses over all RequestMetaHeader chain to find
bearer and session tokens (#548). bearer and session tokens (#548).
- Object service correctly resends complete objects without attached - Object service correctly resends complete objects without attached
session token (#501). session token (#501).
@ -1041,7 +1197,7 @@ TLS support on gRPC servers.
- Client cache now gracefully closes all available connections (#567). - Client cache now gracefully closes all available connections (#567).
### Added ### Added
- Session token support in container service for `container.Put`, - Session token support in container service for `container.Put`,
`container.Delete` and `container.SetEACL` operations. `container.Delete` and `container.SetEACL` operations.
- Session token support in container and sign command of NeoFS CLI. - Session token support in container and sign command of NeoFS CLI.
- TLS encryption support of gRPC service in storage node. - TLS encryption support of gRPC service in storage node.
@ -1051,8 +1207,8 @@ TLS support on gRPC servers.
update earlier. update earlier.
- Inner ring processes extended ACL changes. - Inner ring processes extended ACL changes.
- Inner ring makes signature checks of containers and extended ACLs. - Inner ring makes signature checks of containers and extended ACLs.
- Refactored config of storage node. - Refactored config of storage node.
- Static clients from `morph/client` do not process notary invocations - Static clients from `morph/client` do not process notary invocations
explicitly anymore. Now notary support specified at static client creation. explicitly anymore. Now notary support specified at static client creation.
- Updated neo-go to v0.95.1 release. - Updated neo-go to v0.95.1 release.
- Updated neofs-api-go to v1.27.0 release. - Updated neofs-api-go to v1.27.0 release.
@ -1063,7 +1219,7 @@ TLS support on gRPC servers.
## [0.20.0] - 2021-05-21 - Dolsando (돌산도, 突山島) ## [0.20.0] - 2021-05-21 - Dolsando (돌산도, 突山島)
NeoFS is N3 RC2 compatible. NeoFS is N3 RC2 compatible.
### Fixed ### Fixed
- Calculations in EigenTrust algorithm (#527). - Calculations in EigenTrust algorithm (#527).
@ -1076,7 +1232,7 @@ NeoFS is N3 RC2 compatible.
- Client for NeoFSID contract. - Client for NeoFSID contract.
### Changed ### Changed
- Reorganized and removed plenty of application configuration records - Reorganized and removed plenty of application configuration records
(#510, #511, #512, #514). (#510, #511, #512, #514).
- Nodes do not resolve remote addresses manually. - Nodes do not resolve remote addresses manually.
- Presets for basic ACL in CLI are `private` ,`public-read` and - Presets for basic ACL in CLI are `private` ,`public-read` and
@ -1094,11 +1250,11 @@ NeoFS is N3 RC2 compatible.
Storage nodes exchange, calculate, aggregate and store reputation information Storage nodes exchange, calculate, aggregate and store reputation information
in reputation contract. Inner ring nodes support workflows with and without in reputation contract. Inner ring nodes support workflows with and without
notary subsystem in chains. notary subsystem in chains.
### Fixed ### Fixed
- Build with go1.16. - Build with go1.16.
- Notary deposits last more blocks. - Notary deposits last more blocks.
- TX hashes now prints in little endian in logs. - TX hashes now prints in little endian in logs.
- Metabase deletes graves regardless of the presence of objects. - Metabase deletes graves regardless of the presence of objects.
- SplitInfo error created from all shards instead of first matched shard. - SplitInfo error created from all shards instead of first matched shard.
@ -1106,7 +1262,7 @@ notary subsystem in chains.
- Storage node does not send rebootstrap messages after it went offline. - Storage node does not send rebootstrap messages after it went offline.
### Added ### Added
- Reputation subsystem that includes reputation collection, exchange, - Reputation subsystem that includes reputation collection, exchange,
calculation and storage components. calculation and storage components.
- Notary and non notary workflows in inner ring. - Notary and non notary workflows in inner ring.
- Audit fee transfer for inner ring nodes that performed audit. - Audit fee transfer for inner ring nodes that performed audit.
@ -1116,7 +1272,7 @@ calculation and storage components.
### Changed ### Changed
- Metabase puts data in batches. - Metabase puts data in batches.
- Network related new epoch handlers in storage node executed asynchronously. - Network related new epoch handlers in storage node executed asynchronously.
- Storage node gets epoch duration from global config. - Storage node gets epoch duration from global config.
- Storage node resign and resend Search, Range, Head, Get requests of object - Storage node resign and resend Search, Range, Head, Get requests of object
service without modification. service without modification.
@ -1133,7 +1289,7 @@ alphabet keys are synchronized with main chain.
### Fixed ### Fixed
- Metabase does not store object payloads anymore. - Metabase does not store object payloads anymore.
- TTLNetCache now always evict data after a timeout. - TTLNetCache now always evict data after a timeout.
- NeoFS CLI keyer could misinterpret hex value as base58. - NeoFS CLI keyer could misinterpret hex value as base58.
### Added ### Added
- Local trust controller in storage node. - Local trust controller in storage node.
@ -1145,7 +1301,7 @@ alphabet keys are synchronized with main chain.
## [0.17.0] - 2021-03-22 - Jebudo (제부도, 濟扶島) ## [0.17.0] - 2021-03-22 - Jebudo (제부도, 濟扶島)
Notary contract support, updated neofs-api-go with raw client, some performance Notary contract support, updated neofs-api-go with raw client, some performance
tweaks with extra caches and enhanced metrics. tweaks with extra caches and enhanced metrics.
### Added ### Added
@ -1164,7 +1320,7 @@ tweaks with extra caches and enhanced metrics.
Garbage collector is now running inside storage engine. It is accessed Garbage collector is now running inside storage engine. It is accessed
via Control API, from `policer` component and through object expiration via Control API, from `policer` component and through object expiration
scrubbers. scrubbers.
Inner ring configuration now supports single chain mode with any number of Inner ring configuration now supports single chain mode with any number of
alphabet contracts. alphabet contracts.
@ -1195,39 +1351,39 @@ Storage node now supports NetworkInfo method in netmap service.
## [0.15.0] - 2021-02-12 - Seonyudo (선유도, 仙遊島) ## [0.15.0] - 2021-02-12 - Seonyudo (선유도, 仙遊島)
NeoFS nodes are now preview5-compatible. NeoFS nodes are now preview5-compatible.
IR nodes are now engaged in the distribution of funds to the storage nodes: IR nodes are now engaged in the distribution of funds to the storage nodes:
for the passed audit and for the amount of stored information. All timers for the passed audit and for the amount of stored information. All timers
of the IR nodes related to the generation and processing of global system of the IR nodes related to the generation and processing of global system
events are decoupled from astronomical time, and are measured in the number events are decoupled from astronomical time, and are measured in the number
of blockchain blocks. of blockchain blocks.
For the geographic positioning of storage nodes, a global NeoFS location For the geographic positioning of storage nodes, a global NeoFS location
database is now used, the key in which is a UN/LOCODE, and the base itself database is now used, the key in which is a UN/LOCODE, and the base itself
is generated on the basis of the UN/LOCODE and OpenFlights databases. is generated on the basis of the UN/LOCODE and OpenFlights databases.
### Added ### Added
- Timers with time in blocks of the chain. - Timers with time in blocks of the chain.
- Subscriptions to new blocks in blockchain event `Listener`. - Subscriptions to new blocks in blockchain event `Listener`.
- Tracking the volume of stored information by containers in the - Tracking the volume of stored information by containers in the
storage engine and an external interface for obtaining this data. storage engine and an external interface for obtaining this data.
- `TransferX` operation in sidechain client. - `TransferX` operation in sidechain client.
- Calculators of audit and basic settlements. - Calculators of audit and basic settlements.
- Distribution of funds to storage nodes for audit and for the amount - Distribution of funds to storage nodes for audit and for the amount
of stored information (settlement processors of IR). of stored information (settlement processors of IR).
- NeoFS API `Container.AnnounceUsedSpace` RPC service. - NeoFS API `Container.AnnounceUsedSpace` RPC service.
- Exchange of information about container volumes between storage nodes - Exchange of information about container volumes between storage nodes
controlled by IR through sidechain notifications. controlled by IR through sidechain notifications.
- Support of new search matchers (`STRING_NOT_EQUAL`, `NOT_PRESENT`). - Support of new search matchers (`STRING_NOT_EQUAL`, `NOT_PRESENT`).
- Functional for the formation of NeoFS location database. - Functional for the formation of NeoFS location database.
- CLI commands for generating and reading the location database. - CLI commands for generating and reading the location database.
- Checking the locode attribute and generating geographic attributes - Checking the locode attribute and generating geographic attributes
for candidates for a network map on IR side. for candidates for a network map on IR side.
- Verification of the eACL signature when checking Object ACL rules. - Verification of the eACL signature when checking Object ACL rules.
### Fixed ### Fixed
- Overwriting the local configuration of node attributes when updating - Overwriting the local configuration of node attributes when updating
the network map. the network map.
- Ignoring the X-headers CLI `storagegroup` commands. - Ignoring the X-headers CLI `storagegroup` commands.
- Inability to attach bearer token in CLI `storagegroup` commands. - Inability to attach bearer token in CLI `storagegroup` commands.
@ -1245,7 +1401,7 @@ is generated on the basis of the UN/LOCODE and OpenFlights databases.
### Fixed ### Fixed
- Upload of objects bigger than single gRPC message. - Upload of objects bigger than single gRPC message.
- Inconsistent placement issues (#347, #349). - Inconsistent placement issues (#347, #349).
- Bug when ACL request classifier failed to classify `RoleOthers` in - Bug when ACL request classifier failed to classify `RoleOthers` in
first epoch. first epoch.
### Added ### Added
@ -1259,13 +1415,13 @@ is generated on the basis of the UN/LOCODE and OpenFlights databases.
Testnet4 related bugfixes. Testnet4 related bugfixes.
### Fixed ### Fixed
- Default values for blobovnicza object size limit and blobstor small object - Default values for blobovnicza object size limit and blobstor small object
size are not zero. size are not zero.
- Various storage engine log messages. - Various storage engine log messages.
- Bug when inner ring node ignored bootstrap messages from restarted storage - Bug when inner ring node ignored bootstrap messages from restarted storage
nodes. nodes.
### Added ### Added
- Timeout for reading boltDB files at storage node initialization. - Timeout for reading boltDB files at storage node initialization.
@ -1419,8 +1575,8 @@ NeoFS-API v2.0 support and updated brand-new storage node application.
## [0.10.0] - 2020-07-10 ## [0.10.0] - 2020-07-10
First public review release. First public review release.
[Unreleased]: https://github.com/nspcc-dev/neofs-node/compare/v0.35.0...master
[Unreleased]: https://github.com/nspcc-dev/neofs-node/compare/v0.34.0...master [0.35.0]: https://github.com/nspcc-dev/neofs-node/compare/v0.34.0...v0.35.0
[0.34.0]: https://github.com/nspcc-dev/neofs-node/compare/v0.33.0...v0.34.0 [0.34.0]: https://github.com/nspcc-dev/neofs-node/compare/v0.33.0...v0.34.0
[0.33.0]: https://github.com/nspcc-dev/neofs-node/compare/v0.32.0...v0.33.0 [0.33.0]: https://github.com/nspcc-dev/neofs-node/compare/v0.32.0...v0.33.0
[0.32.0]: https://github.com/nspcc-dev/neofs-node/compare/v0.31.0...v0.32.0 [0.32.0]: https://github.com/nspcc-dev/neofs-node/compare/v0.31.0...v0.32.0

View file

@ -3,8 +3,8 @@
First, thank you for contributing! We love and encourage pull requests from First, thank you for contributing! We love and encourage pull requests from
everyone. Please follow the guidelines: everyone. Please follow the guidelines:
- Check the open [issues](https://github.com/nspcc-dev/neofs-node/issues) and - Check the open [issues](https://github.com/TrueCloudLab/frostfs-node/issues) and
[pull requests](https://github.com/nspcc-dev/neofs-node/pulls) for existing [pull requests](https://github.com/TrueCloudLab/frostfs-node/pulls) for existing
discussions. discussions.
- Open an issue first, to discuss a new feature or enhancement. - Open an issue first, to discuss a new feature or enhancement.
@ -23,23 +23,23 @@ everyone. Please follow the guidelines:
## Development Workflow ## Development Workflow
Start by forking the `neofs-node` repository, make changes in a branch and then Start by forking the `frostfs-node` repository, make changes in a branch and then
send a pull request. We encourage pull requests to discuss code changes. Here send a pull request. We encourage pull requests to discuss code changes. Here
are the steps in details: are the steps in details:
### Set up your GitHub Repository ### Set up your GitHub Repository
Fork [NeoFS node upstream](https://github.com/nspcc-dev/neofs-node/fork) source Fork [FrostFS node upstream](https://github.com/TrueCloudLab/frostfs-node/fork) source
repository to your own personal repository. Copy the URL of your fork (you will repository to your own personal repository. Copy the URL of your fork (you will
need it for the `git clone` command below). need it for the `git clone` command below).
```sh ```sh
$ git clone https://github.com/nspcc-dev/neofs-node $ git clone https://github.com/TrueCloudLab/frostfs-node
``` ```
### Set up git remote as ``upstream`` ### Set up git remote as ``upstream``
```sh ```sh
$ cd neofs-node $ cd frostfs-node
$ git remote add upstream https://github.com/nspcc-dev/neofs-node $ git remote add upstream https://github.com/TrueCloudLab/frostfs-node
$ git fetch upstream $ git fetch upstream
$ git merge upstream/master $ git merge upstream/master
... ...
@ -79,7 +79,7 @@ Description
``` ```
``` ```
$ git commit -am '[#123] Add some feature' $ git commit -sam '[#123] Add some feature'
``` ```
### Push to the branch ### Push to the branch
@ -106,7 +106,8 @@ contributors".
To sign your work, just add a line like this at the end of your commit message: To sign your work, just add a line like this at the end of your commit message:
``` ```
Signed-off-by: Samii Sakisaka <samii@nspcc.ru> Signed-off-by: Samii Sakisaka <samii@ivunojikan.co.jp>
``` ```
This can easily be done with the `--signoff` option to `git commit`. This can easily be done with the `--signoff` option to `git commit`.

View file

@ -1,5 +1,7 @@
# Credits # Credits
FrostFS continues the development of NeoFS.
Initial NeoFS research and development (2018-2020) was done by Initial NeoFS research and development (2018-2020) was done by
[NeoSPCC](https://nspcc.ru) team. [NeoSPCC](https://nspcc.ru) team.
@ -8,7 +10,7 @@ In alphabetical order:
- Alexey Vanin - Alexey Vanin
- Anastasia Prasolova - Anastasia Prasolova
- Anatoly Bogatyrev - Anatoly Bogatyrev
- Evgeny Kulikov - Evgeny Kulikov
- Evgeny Stratonikov - Evgeny Stratonikov
- Leonard Liubich - Leonard Liubich
- Sergei Liubich - Sergei Liubich

35
Makefile Normal file → Executable file
View file

@ -4,10 +4,10 @@ SHELL = bash
REPO ?= $(shell go list -m) REPO ?= $(shell go list -m)
VERSION ?= $(shell git describe --tags --dirty --match "v*" --always --abbrev=8 2>/dev/null || cat VERSION 2>/dev/null || echo "develop") VERSION ?= $(shell git describe --tags --dirty --match "v*" --always --abbrev=8 2>/dev/null || cat VERSION 2>/dev/null || echo "develop")
HUB_IMAGE ?= nspccdev/neofs HUB_IMAGE ?= truecloudlab/frostfs
HUB_TAG ?= "$(shell echo ${VERSION} | sed 's/^v//')" HUB_TAG ?= "$(shell echo ${VERSION} | sed 's/^v//')"
GO_VERSION ?= 1.17 GO_VERSION ?= 1.19
LINT_VERSION ?= 1.50.0 LINT_VERSION ?= 1.50.0
ARCH = amd64 ARCH = amd64
@ -16,7 +16,7 @@ RELEASE = release
DIRS = $(BIN) $(RELEASE) DIRS = $(BIN) $(RELEASE)
# List of binaries to build. # List of binaries to build.
CMDS = $(notdir $(basename $(wildcard cmd/*))) CMDS = $(notdir $(basename $(wildcard cmd/frostfs-*)))
BINS = $(addprefix $(BIN)/, $(CMDS)) BINS = $(addprefix $(BIN)/, $(CMDS))
# .deb package versioning # .deb package versioning
@ -26,10 +26,10 @@ PKG_VERSION ?= $(shell echo $(VERSION) | sed "s/^v//" | \
sed "s/-/~/")-${OS_RELEASE} sed "s/-/~/")-${OS_RELEASE}
.PHONY: help all images dep clean fmts fmt imports test lint docker/lint .PHONY: help all images dep clean fmts fmt imports test lint docker/lint
prepare-release debpackage prepare-release debpackage pre-commit unpre-commit
# To build a specific binary, use it's name prefix with bin/ as a target # To build a specific binary, use it's name prefix with bin/ as a target
# For example `make bin/neofs-node` will build only storage node binary # For example `make bin/frostfs-node` will build only storage node binary
# Just `make` will build all possible binaries # Just `make` will build all possible binaries
all: $(DIRS) $(BINS) all: $(DIRS) $(BINS)
@ -50,7 +50,7 @@ $(DIRS):
# Prepare binaries and archives for release # Prepare binaries and archives for release
.ONESHELL: .ONESHELL:
prepare-release: docker/all prepare-release: docker/all
@for file in `ls -1 $(BIN)/neofs-*`; do @for file in `ls -1 $(BIN)/frostfs-*`; do
cp $$file $(RELEASE)/`basename $$file`-$(ARCH) cp $$file $(RELEASE)/`basename $$file`-$(ARCH)
strip $(RELEASE)/`basename $$file`-$(ARCH) strip $(RELEASE)/`basename $$file`-$(ARCH)
tar -czf $(RELEASE)/`basename $$file`-$(ARCH).tar.gz $(RELEASE)/`basename $$file`-$(ARCH) tar -czf $(RELEASE)/`basename $$file`-$(ARCH).tar.gz $(RELEASE)/`basename $$file`-$(ARCH)
@ -67,26 +67,26 @@ dep:
# Regenerate proto files: # Regenerate proto files:
protoc: protoc:
@GOPRIVATE=github.com/nspcc-dev go mod vendor @GOPRIVATE=github.com/TrueCloudLab go mod vendor
# Install specific version for protobuf lib # Install specific version for protobuf lib
@go list -f '{{.Path}}/...@{{.Version}}' -m github.com/golang/protobuf | xargs go install -v @go list -f '{{.Path}}/...@{{.Version}}' -m github.com/golang/protobuf | xargs go install -v
@GOBIN=$(abspath $(BIN)) go install -mod=mod -v github.com/nspcc-dev/neofs-api-go/v2/util/protogen @GOBIN=$(abspath $(BIN)) go install -mod=mod -v git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/util/protogen
# Protoc generate # Protoc generate
@for f in `find . -type f -name '*.proto' -not -path './vendor/*'`; do \ @for f in `find . -type f -name '*.proto' -not -path './vendor/*'`; do \
echo "⇒ Processing $$f "; \ echo "⇒ Processing $$f "; \
protoc \ protoc \
--proto_path=.:./vendor:/usr/local/include \ --proto_path=.:./vendor:/usr/local/include \
--plugin=protoc-gen-go-neofs=$(BIN)/protogen \ --plugin=protoc-gen-go-frostfs=$(BIN)/protogen \
--go-neofs_out=. --go-neofs_opt=paths=source_relative \ --go-frostfs_out=. --go-frostfs_opt=paths=source_relative \
--go_out=. --go_opt=paths=source_relative \ --go_out=. --go_opt=paths=source_relative \
--go-grpc_opt=require_unimplemented_servers=false \ --go-grpc_opt=require_unimplemented_servers=false \
--go-grpc_out=. --go-grpc_opt=paths=source_relative $$f; \ --go-grpc_out=. --go-grpc_opt=paths=source_relative $$f; \
done done
rm -rf vendor rm -rf vendor
# Build NeoFS component's docker image # Build FrostFS component's docker image
image-%: image-%:
@echo "⇒ Build NeoFS $* docker image " @echo "⇒ Build FrostFS $* docker image "
@docker build \ @docker build \
--build-arg REPO=$(REPO) \ --build-arg REPO=$(REPO) \
--build-arg VERSION=$(VERSION) \ --build-arg VERSION=$(VERSION) \
@ -140,10 +140,19 @@ docker/lint:
--env HOME=/src \ --env HOME=/src \
golangci/golangci-lint:v$(LINT_VERSION) bash -c 'cd /src/ && make lint' golangci/golangci-lint:v$(LINT_VERSION) bash -c 'cd /src/ && make lint'
# Activate pre-commit hooks
pre-commit:
pre-commit install -t pre-commit -t commit-msg
# Deactivate pre-commit hooks
unpre-commit:
pre-commit uninstall -t pre-commit -t commit-msg
# Print version # Print version
version: version:
@echo $(VERSION) @echo $(VERSION)
# Delete built artifacts
clean: clean:
rm -rf vendor rm -rf vendor
rm -rf .cache rm -rf .cache
@ -152,7 +161,7 @@ clean:
# Package for Debian # Package for Debian
debpackage: debpackage:
dch --package neofs-node \ dch -b --package frostfs-node \
--controlmaint \ --controlmaint \
--newversion $(PKG_VERSION) \ --newversion $(PKG_VERSION) \
--distribution $(OS_RELEASE) \ --distribution $(OS_RELEASE) \

View file

@ -1,39 +1,40 @@
<p align="center"> <p align="center">
<img src="./.github/logo.svg" width="500px" alt="NeoFS"> <img src="./.github/logo.svg" width="500px" alt="FrostFS">
</p> </p>
<p align="center"> <p align="center">
<a href="https://fs.neo.org">NeoFS</a> is a decentralized distributed object storage integrated with the <a href="https://neo.org">NEO Blockchain</a>. <a href="https://frostfs.info">FrostFS</a> is a decentralized distributed object storage integrated with the <a href="https://neo.org">NEO Blockchain</a>.
</p> </p>
--- ---
[![Report](https://goreportcard.com/badge/github.com/nspcc-dev/neofs-node)](https://goreportcard.com/report/github.com/nspcc-dev/neofs-node) [![Report](https://goreportcard.com/badge/github.com/TrueCloudLab/frostfs-node)](https://goreportcard.com/report/github.com/TrueCloudLab/frostfs-node)
![GitHub release (latest SemVer)](https://img.shields.io/github/v/release/nspcc-dev/neofs-node?sort=semver) ![GitHub release (latest SemVer)](https://img.shields.io/github/v/release/TrueCloudLab/frostfs-node?sort=semver)
![License](https://img.shields.io/github/license/nspcc-dev/neofs-node.svg?style=popout) ![License](https://img.shields.io/github/license/TrueCloudLab/frostfs-node.svg?style=popout)
# Overview # Overview
NeoFS Nodes are organized in a peer-to-peer network that takes care of storing FrostFS Nodes are organized in a peer-to-peer network that takes care of storing
and distributing user's data. Any Neo user may participate in the network and and distributing user's data. Any Neo user may participate in the network and
get paid for providing storage resources to other users or store their data in get paid for providing storage resources to other users or store their data in
NeoFS and pay a competitive price for it. FrostFS and pay a competitive price for it.
Users can reliably store object data in the NeoFS network and have a transparent Users can reliably store object data in the FrostFS network and have a transparent
data placement process due to a decentralized architecture and flexible storage data placement process due to a decentralized architecture and flexible storage
policies. Each node is responsible for executing the storage policies that the policies. Each node is responsible for executing the storage policies that the
users select for geographical location, reliability level, number of nodes, type users select for geographical location, reliability level, number of nodes, type
of disks, capacity, etc. Thus, NeoFS gives full control over data to users. of disks, capacity, etc. Thus, FrostFS gives full control over data to users.
Deep [Neo Blockchain](https://neo.org) integration allows NeoFS to be used by Deep [Neo Blockchain](https://neo.org) integration allows FrostFS to be used by
dApps directly from dApps directly from
[NeoVM](https://docs.neo.org/docs/en-us/basic/technology/neovm.html) on the [NeoVM](https://docs.neo.org/docs/en-us/basic/technology/neovm.html) on the
[Smart Contract](https://docs.neo.org/docs/en-us/intro/glossary.html) [Smart Contract](https://docs.neo.org/docs/en-us/intro/glossary.html)
code level. This way dApps are not limited to on-chain storage and can code level. This way dApps are not limited to on-chain storage and can
manipulate large amounts of data without paying a prohibitive price. manipulate large amounts of data without paying a prohibitive price.
NeoFS has a native [gRPC API](https://github.com/nspcc-dev/neofs-api) and has FrostFS has a native [gRPC API](https://git.frostfs.info/TrueCloudLab/frostfs-api) and has
protocol gateways for popular protocols such as [AWS protocol gateways for popular protocols such as [AWS
S3](https://github.com/nspcc-dev/neofs-s3-gw), S3](https://github.com/TrueCloudLab/frostfs-s3-gw),
[HTTP](https://github.com/nspcc-dev/neofs-http-gw), [HTTP](https://github.com/TrueCloudLab/frostfs-http-gw),
[FUSE](https://wikipedia.org/wiki/Filesystem_in_Userspace) and [FUSE](https://wikipedia.org/wiki/Filesystem_in_Userspace) and
[sFTP](https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol) allowing [sFTP](https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol) allowing
developers to integrate applications without rewriting their code. developers to integrate applications without rewriting their code.
@ -43,12 +44,12 @@ developers to integrate applications without rewriting their code.
Now, we only support GNU/Linux on amd64 CPUs with AVX/AVX2 instructions. More Now, we only support GNU/Linux on amd64 CPUs with AVX/AVX2 instructions. More
platforms will be officially supported after release `1.0`. platforms will be officially supported after release `1.0`.
The latest version of neofs-node works with neofs-contract The latest version of frostfs-node works with frostfs-contract
[v0.16.0](https://github.com/nspcc-dev/neofs-contract/releases/tag/v0.16.0). [v0.16.0](https://github.com/TrueCloudLab/frostfs-contract/releases/tag/v0.16.0).
# Building # Building
To make all binaries you need Go 1.17+ and `make`: To make all binaries you need Go 1.18+ and `make`:
``` ```
make all make all
``` ```
@ -56,7 +57,7 @@ The resulting binaries will appear in `bin/` folder.
To make a specific binary use: To make a specific binary use:
``` ```
make bin/neofs-<name> make bin/frostfs-<name>
``` ```
See the list of all available commands in the `cmd` folder. See the list of all available commands in the `cmd` folder.
@ -65,12 +66,12 @@ See the list of all available commands in the `cmd` folder.
Building can also be performed in a container: Building can also be performed in a container:
``` ```
make docker/all # build all binaries make docker/all # build all binaries
make docker/bin/neofs-<name> # build a specific binary make docker/bin/frostfs-<name> # build a specific binary
``` ```
## Docker images ## Docker images
To make docker images suitable for use in [neofs-dev-env](https://github.com/nspcc-dev/neofs-dev-env/) use: To make docker images suitable for use in [frostfs-dev-env](https://github.com/TrueCloudLab/frostfs-dev-env/) use:
``` ```
make images make images
``` ```
@ -85,7 +86,7 @@ the feature/topic you are going to implement.
# Credits # Credits
NeoFS is maintained by [NeoSPCC](https://nspcc.ru) with the help and FrostFS is maintained by [True Cloud Lab](https://github.com/TrueCloudLab/) with the help and
contributions from community members. contributions from community members.
Please see [CREDITS](CREDITS.md) for details. Please see [CREDITS](CREDITS.md) for details.

View file

@ -1 +1 @@
v0.34.0 v0.35.0

View file

@ -1,25 +1,24 @@
# NeoFS Admin Tool # FrostFS Admin Tool
## Overview ## Overview
Admin tool provides an easier way to deploy and maintain private installation Admin tool provides an easier way to deploy and maintain private installation
of NeoFS. Private installation provides a set of N3 consensus nodes, NeoFS of FrostFS. Private installation provides a set of N3 consensus nodes, FrostFS
Alphabet, and Storage nodes. Admin tool generates consensus keys, initializes Alphabet, and Storage nodes. Admin tool generates consensus keys, initializes
the sidechain, and provides functions to update the network and register new the sidechain, and provides functions to update the network and register new
Storage nodes. Storage nodes.
## Build ## Build
To build binary locally, use `make bin/neofs-adm` command. To build binary locally, use `make bin/frostfs-adm` command.
For clean build inside a docker container, use `make docker/bin/neofs-adm`. For clean build inside a docker container, use `make docker/bin/frostfs-adm`.
Build docker image with `make image-adm`. Build docker image with `make image-adm`.
At NeoFS private install deployment, neofs-adm requires compiled NeoFS At FrostFS private install deployment, frostfs-adm requires compiled FrostFS
contracts. Find them in the latest release of contracts. Find them in the latest release of
[neofs-contract repository](https://github.com/nspcc-dev/neofs-contract/releases). [frostfs-contract repository](https://git.frostfs.info/TrueCloudLab/frostfs-contract/releases).
## Commands ## Commands
@ -27,7 +26,7 @@ contracts. Find them in the latest release of
Config section provides `init` command that creates a configuration file for Config section provides `init` command that creates a configuration file for
private installation deployment and updates. Config file is optional, all private installation deployment and updates. Config file is optional, all
parameters can be passed by arguments or read from standard input (wallet parameters can be passed by arguments or read from standard input (wallet
passwords). passwords).
Config example: Config example:
@ -35,8 +34,8 @@ Config example:
rpc-endpoint: https://address:port # sidechain RPC node endpoint rpc-endpoint: https://address:port # sidechain RPC node endpoint
alphabet-wallets: /path # path to consensus node / alphabet wallets storage alphabet-wallets: /path # path to consensus node / alphabet wallets storage
network: network:
max_object_size: 67108864 # max size of a single NeoFS object, bytes max_object_size: 67108864 # max size of a single FrostFS object, bytes
epoch_duration: 240 # duration of a NeoFS epoch in blocks, consider block generation frequency in the sidechain epoch_duration: 240 # duration of a FrostFS epoch in blocks, consider block generation frequency in the sidechain
basic_income_rate: 0 # basic income rate, for private consider 0 basic_income_rate: 0 # basic income rate, for private consider 0
fee: fee:
audit: 0 # network audit fee, for private installation consider 0 audit: 0 # network audit fee, for private installation consider 0
@ -58,24 +57,24 @@ credentials: # passwords for consensus node / alphabet wallets
#### Network deployment #### Network deployment
- `generate-alphabet` generates a set of wallets for consensus and - `generate-alphabet` generates a set of wallets for consensus and
Alphabet nodes. Alphabet nodes.
- `init` initializes the sidechain by deploying smart contracts and - `init` initializes the sidechain by deploying smart contracts and
setting provided NeoFS network configuration. setting provided FrostFS network configuration.
- `generate-storage-wallet` generates a wallet for the Storage node that - `generate-storage-wallet` generates a wallet for the Storage node that
is ready for deployment. It also transfers a bit of sidechain GAS, so this is ready for deployment. It also transfers a bit of sidechain GAS, so this
wallet can be used for NeoFS bootstrap. wallet can be used for FrostFS bootstrap.
#### Network maintenance #### Network maintenance
- `set-config` add/update configuration values in the Netmap contract. - `set-config` add/update configuration values in the Netmap contract.
- `force-new-epoch` increments NeoFS epoch number and executes new epoch - `force-new-epoch` increments FrostFS epoch number and executes new epoch
handlers in NeoFS nodes. handlers in FrostFS nodes.
- `refill-gas` transfers sidechain GAS to the specified wallet. - `refill-gas` transfers sidechain GAS to the specified wallet.
- `update-contracts` updates contracts to a new version. - `update-contracts` updates contracts to a new version.
@ -87,14 +86,16 @@ info. These commands **do not migrate actual objects**.
- `dump-containers` saves all containers and metadata registered in the container - `dump-containers` saves all containers and metadata registered in the container
contract to a file. contract to a file.
- `restore-containers` restores previously saved containers by their repeated registration in - `restore-containers` restores previously saved containers by their repeated registration in
the container contract. the container contract.
- `list-containers` output all containers ids.
#### Network info #### Network info
- `dump-config` prints NeoFS network configuration. - `dump-config` prints FrostFS network configuration.
- `dump-hashes` prints NeoFS contract addresses stored in NNS. - `dump-hashes` prints FrostFS contract addresses stored in NNS.
## Private network deployment ## Private network deployment

View file

@ -1,33 +1,33 @@
# Step-by-step private NeoFS deployment # Step-by-step private FrostFS deployment
This is a short guide on how to deploy a private NeoFS storage network on bare This is a short guide on how to deploy a private FrostFS storage network on bare
metal without docker images. This guide does not cover details on how to start metal without docker images. This guide does not cover details on how to start
consensus, Alphabet, or Storage nodes. This guide covers only `neofs-adm` consensus, Alphabet, or Storage nodes. This guide covers only `frostfs-adm`
related configuration details. related configuration details.
## Prerequisites ## Prerequisites
To follow this guide you need: To follow this guide you need:
- latest released version of [neo-go](https://github.com/nspcc-dev/neo-go/releases) (v0.97.2 at the moment), - latest released version of [neo-go](https://github.com/nspcc-dev/neo-go/releases) (v0.97.2 at the moment),
- latest released version of [neofs-adm](https://github.com/nspcc-dev/neofs-node/releases) utility (v0.25.1 at the moment), - latest released version of [frostfs-adm](https://github.com/TrueCloudLab/frostfs-node/releases) utility (v0.25.1 at the moment),
- latest released version of compiled [neofs-contract](https://github.com/nspcc-dev/neofs-contract/releases) (v0.11.0 at the moment). - latest released version of compiled [frostfs-contract](https://github.com/TrueCloudLab/frostfs-contract/releases) (v0.11.0 at the moment).
## Step 1: Prepare network configuration ## Step 1: Prepare network configuration
To start a network, you need a set of consensus nodes, the same number of To start a network, you need a set of consensus nodes, the same number of
Alphabet nodes and any number of Storage nodes. While the number of Storage Alphabet nodes and any number of Storage nodes. While the number of Storage
nodes can be scaled almost infinitely, the number of consensus and Alphabet nodes can be scaled almost infinitely, the number of consensus and Alphabet
nodes can't be changed so easily right now. Consider this before going any further. nodes can't be changed so easily right now. Consider this before going any further.
It is easier to use`neofs-adm` with a predefined configuration. First, create It is easier to use`frostfs-adm` with a predefined configuration. First, create
a network configuration file. In this example, there is going to be only one a network configuration file. In this example, there is going to be only one
consensus / Alphabet node in the network. consensus / Alphabet node in the network.
``` ```
$ neofs-adm config init --path foo.network.yml $ frostfs-adm config init --path foo.network.yml
Initial config file saved to foo.network.yml Initial config file saved to foo.network.yml
$ cat foo.network.yml $ cat foo.network.yml
rpc-endpoint: https://neo.rpc.node:30333 rpc-endpoint: https://neo.rpc.node:30333
alphabet-wallets: /home/user/deploy/alphabet-wallets alphabet-wallets: /home/user/deploy/alphabet-wallets
network: network:
@ -43,21 +43,21 @@ credentials:
az: hunter2 az: hunter2
``` ```
For private installation, it is recommended to set all **fees** and **basic For private installation, it is recommended to set all **fees** and **basic
income rate** to 0. income rate** to 0.
As for **epoch duration**, consider consensus node block generation frequency. As for **epoch duration**, consider consensus node block generation frequency.
With default 15 seconds per block, 240 blocks are going to be a 1-hour epoch. With default 15 seconds per block, 240 blocks are going to be a 1-hour epoch.
For **max object size**, 67108864 (64 MiB) or 134217728 (128 MiB) should provide For **max object size**, 67108864 (64 MiB) or 134217728 (128 MiB) should provide
good chunk distribution in most cases. good chunk distribution in most cases.
With this config, generate wallets (private keys) of consensus nodes. The same With this config, generate wallets (private keys) of consensus nodes. The same
wallets will be used for Alphabet nodes. Make sure, that dir for alphabet wallets will be used for Alphabet nodes. Make sure, that dir for alphabet
wallets already exists. wallets already exists.
``` ```
$ neofs-adm -c foo.network.yml morph generate-alphabet --size 1 $ frostfs-adm -c foo.network.yml morph generate-alphabet --size 1
size: 1 size: 1
alphabet-wallets: /home/user/deploy/alphabet-wallets alphabet-wallets: /home/user/deploy/alphabet-wallets
wallet[0]: hunter2 wallet[0]: hunter2
@ -69,14 +69,14 @@ storage.
## Step 2: Launch consensus nodes ## Step 2: Launch consensus nodes
Configure blockchain nodes with the generated wallets from the previous step. Configure blockchain nodes with the generated wallets from the previous step.
Config examples can be found in Config examples can be found in
[neo-go repository](https://github.com/nspcc-dev/neo-go/tree/master/config). [neo-go repository](https://github.com/nspcc-dev/neo-go/tree/master/config).
Gather public keys from **all** generated wallets. We are interested in the first Gather public keys from **all** generated wallets. We are interested in the first
`simple signature contract` public key. `simple signature contract` public key.
``` ```
$ neo-go wallet dump-keys -w alphabet-wallets/az.json $ neo-go wallet dump-keys -w alphabet-wallets/az.json
NitdS4k4f1Hh5mbLJhAswBK3WC2gQgPN1o (simple signature contract): NitdS4k4f1Hh5mbLJhAswBK3WC2gQgPN1o (simple signature contract):
02c1cc85f9c856dbe2d02017349bcb7b4e5defa78b8056a09b3240ba2a8c078869 02c1cc85f9c856dbe2d02017349bcb7b4e5defa78b8056a09b3240ba2a8c078869
@ -87,10 +87,10 @@ NiMKabp3ddi3xShmLAXhTfbnuWb4cSJT6E (1 out of 1 multisig contract):
02c1cc85f9c856dbe2d02017349bcb7b4e5defa78b8056a09b3240ba2a8c078869 02c1cc85f9c856dbe2d02017349bcb7b4e5defa78b8056a09b3240ba2a8c078869
``` ```
Put the list of public keys into `ProtocolConfiguration.StandbyCommittee` Put the list of public keys into `ProtocolConfiguration.StandbyCommittee`
section. Specify the wallet path and the password in `ApplicationConfiguration.P2PNotary` section. Specify the wallet path and the password in `ApplicationConfiguration.P2PNotary`
and `ApplicationConfiguration.UnlockWallet` sections. If config includes and `ApplicationConfiguration.UnlockWallet` sections. If config includes
`ProtocolConfiguration.NativeActivations` section, add notary `ProtocolConfiguration.NativeActivations` section, add notary
contract `Notary: [0]`. contract `Notary: [0]`.
```yaml ```yaml
@ -118,19 +118,19 @@ and possible overload issues.
## Step 3: Initialize sidechain ## Step 3: Initialize sidechain
Use archive with compiled NeoFS contracts to initialize the sidechain. Use archive with compiled FrostFS contracts to initialize the sidechain.
``` ```
$ tar -xzvf neofs-contract-v0.11.0.tar.gz $ tar -xzvf frostfs-contract-v0.11.0.tar.gz
$ ./neofs-adm -c foo.network.yml morph init --contracts ./neofs-contract-v0.11.0 $ ./frostfs-adm -c foo.network.yml morph init --contracts ./frostfs-contract-v0.11.0
Stage 1: transfer GAS to alphabet nodes. Stage 1: transfer GAS to alphabet nodes.
Waiting for transactions to persist... Waiting for transactions to persist...
Stage 2: set notary and alphabet nodes in designate contract. Stage 2: set notary and alphabet nodes in designate contract.
Waiting for transactions to persist... Waiting for transactions to persist...
Stage 3: deploy NNS contract. Stage 3: deploy NNS contract.
Waiting for transactions to persist... Waiting for transactions to persist...
Stage 4: deploy NeoFS contracts. Stage 4: deploy FrostFS contracts.
Waiting for transactions to persist... Waiting for transactions to persist...
Stage 4.1: Transfer GAS to proxy contract. Stage 4.1: Transfer GAS to proxy contract.
Waiting for transactions to persist... Waiting for transactions to persist...
@ -140,21 +140,21 @@ Stage 6: transfer NEO to alphabet contracts.
Waiting for transactions to persist... Waiting for transactions to persist...
Stage 7: set addresses in NNS. Stage 7: set addresses in NNS.
Waiting for transactions to persist... Waiting for transactions to persist...
NNS: Set alphabet0.neofs -> f692dfb4d43a15b464eb51a7041160fb29c44b6a NNS: Set alphabet0.frostfs -> f692dfb4d43a15b464eb51a7041160fb29c44b6a
NNS: Set audit.neofs -> 7df847b993affb3852074345a7c2bd622171ee0d NNS: Set audit.frostfs -> 7df847b993affb3852074345a7c2bd622171ee0d
NNS: Set balance.neofs -> 103519b3067a66307080a66570c0491ee8f68879 NNS: Set balance.frostfs -> 103519b3067a66307080a66570c0491ee8f68879
NNS: Set container.neofs -> cae60bdd689d185901e495352d0247752ce50846 NNS: Set container.frostfs -> cae60bdd689d185901e495352d0247752ce50846
NNS: Set neofsid.neofs -> c421fb60a3895865a8f24d197d6a80ef686041d2 NNS: Set frostfsid.frostfs -> c421fb60a3895865a8f24d197d6a80ef686041d2
NNS: Set netmap.neofs -> 894eb854632f50fb124412ce7951ebc00763525e NNS: Set netmap.frostfs -> 894eb854632f50fb124412ce7951ebc00763525e
NNS: Set proxy.neofs -> ac6e6fe4b373d0ca0ca4969d1e58fa0988724e7d NNS: Set proxy.frostfs -> ac6e6fe4b373d0ca0ca4969d1e58fa0988724e7d
NNS: Set reputation.neofs -> 6eda57c9d93d990573646762d1fea327ce41191f NNS: Set reputation.frostfs -> 6eda57c9d93d990573646762d1fea327ce41191f
Waiting for transactions to persist... Waiting for transactions to persist...
``` ```
## Step 4: Launch Alphabet nodes ## Step 4: Launch Alphabet nodes
Configure Alphabet nodes with the wallets generated in step 1. For Configure Alphabet nodes with the wallets generated in step 1. For
`morph.validators` use a list of public keys from `morph.validators` use a list of public keys from
`ProtocolConfiguration.StandbyCommittee`. `ProtocolConfiguration.StandbyCommittee`.
```yaml ```yaml
@ -177,11 +177,11 @@ contracts:
Generate a new wallet for a Storage node. Generate a new wallet for a Storage node.
``` ```
$ neofs-adm -c foo.network.yml morph generate-storage-wallet --storage-wallet ./sn01.json --initial-gas 10.0 $ frostfs-adm -c foo.network.yml morph generate-storage-wallet --storage-wallet ./sn01.json --initial-gas 10.0
New password > New password >
Waiting for transactions to persist... Waiting for transactions to persist...
$ neo-go wallet dump-keys -w sn01.json $ neo-go wallet dump-keys -w sn01.json
Ngr7p8Z9S22XDH6VkUG9oXobv8zZRAWwwv (simple signature contract): Ngr7p8Z9S22XDH6VkUG9oXobv8zZRAWwwv (simple signature contract):
0355eccb72cd46f09a3e5237eaa0f4949cceb5ecfa5a225bd3bb9fd021c4d75b85 0355eccb72cd46f09a3e5237eaa0f4949cceb5ecfa5a225bd3bb9fd021c4d75b85
``` ```
@ -196,16 +196,16 @@ node:
password: "foobar" password: "foobar"
``` ```
The storage node will be included in the network map in the next NeoFS epoch. To The storage node will be included in the network map in the next FrostFS epoch. To
speed up this process, you can increment epoch counter immediately. speed up this process, you can increment epoch counter immediately.
``` ```
$ neofs-adm -c foo.network.yml morph force-new-epoch $ frostfs-adm -c foo.network.yml morph force-new-epoch
Current epoch: 8, increase to 9. Current epoch: 8, increase to 9.
Waiting for transactions to persist... Waiting for transactions to persist...
``` ```
--- ---
After that, NeoFS Storage is ready to work. You can access it directly or After that, FrostFS Storage is ready to work. You can access it directly or
with protocol gates. with protocol gates.

View file

@ -1,20 +1,20 @@
# NeoFS subnetwork creation # FrostFS subnetwork creation
This is a short guide on how to create NeoFS subnetworks. This guide This is a short guide on how to create FrostFS subnetworks. This guide
considers that the sidechain and the inner ring (alphabet nodes) have already been considers that the sidechain and the inner ring (alphabet nodes) have already been
deployed and the sidechain contains a deployed `subnet` contract. deployed and the sidechain contains a deployed `subnet` contract.
## Prerequisites ## Prerequisites
To follow this guide, you need: To follow this guide, you need:
- neo-go sidechain RPC endpoint; - neo-go sidechain RPC endpoint;
- latest released version of [neofs-adm](https://github.com/nspcc-dev/neofs-node/releases); - latest released version of [frostfs-adm](https://github.com/TrueCloudLab/frostfs-node/releases);
- wallet with NeoFS account. - wallet with FrostFS account.
## Creation ## Creation
```shell ```shell
$ neofs-adm morph subnet create \ $ frostfs-adm morph subnet create \
-r <side_chain_RPC_endpoint> \ -r <side_chain_RPC_endpoint> \
-w </path/to/owner/wallet> \ -w </path/to/owner/wallet> \
--notary --notary
@ -31,7 +31,7 @@ of the just created subnetwork.
You can check if your subnetwork was created successfully: You can check if your subnetwork was created successfully:
```shell ```shell
$ neofs-adm morph subnet get \ $ frostfs-adm morph subnet get \
-r <side_chain_RPC_endpoint> \ -r <side_chain_RPC_endpoint> \
--subnet <subnet_ID> --subnet <subnet_ID>
Owner: NUc734PMJXiqa2J9jRtvskU3kCdyyuSN8Q Owner: NUc734PMJXiqa2J9jRtvskU3kCdyyuSN8Q

View file

@ -1,18 +1,18 @@
# Managing Subnetworks # Managing Subnetworks
This is a short guide on how to manage NeoFS subnetworks. This guide This is a short guide on how to manage FrostFS subnetworks. This guide
considers that the sidechain and the inner ring (alphabet nodes) have already been considers that the sidechain and the inner ring (alphabet nodes) have already been
deployed, and the sidechain contains a deployed `subnet` contract. deployed, and the sidechain contains a deployed `subnet` contract.
## Prerequisites ## Prerequisites
- neo-go sidechain RPC endpoint; - neo-go sidechain RPC endpoint;
- latest released version of [neofs-adm](https://github.com/nspcc-dev/neofs-node/releases); - latest released version of [frostfs-adm](https://github.com/TrueCloudLab/frostfs-node/releases);
- [created](subnetwork-creation.md) subnetwork; - [created](subnetwork-creation.md) subnetwork;
- wallet with the account that owns the subnetwork; - wallet with the account that owns the subnetwork;
- public key of the Storage Node; - public key of the Storage Node;
- public keys of the node and client administrators; - public keys of the node and client administrators;
- owner IDs of the NeoFS users. - owner IDs of the FrostFS users.
## Add node administrator ## Add node administrator
@ -21,7 +21,7 @@ the whitelist of the nodes which can be included to a subnetwork. Only the subne
owner is allowed to add and remove node administrators from the subnetwork. owner is allowed to add and remove node administrators from the subnetwork.
```shell ```shell
$ neofs-adm morph subnet admin add \ $ frostfs-adm morph subnet admin add \
-r <side_chain_RPC_endpoint> \ -r <side_chain_RPC_endpoint> \
-w </path/to/owner/wallet> \ -w </path/to/owner/wallet> \
--admin <HEX_admin_public_key> \ --admin <HEX_admin_public_key> \
@ -37,7 +37,7 @@ the list of the allowed nodes. Node is not required to be bootstrapped at the
moment of its inclusion. moment of its inclusion.
```shell ```shell
$ neofs-adm morph subnet node add \ $ frostfs-adm morph subnet node add \
-r <side_chain_RPC_endpoint> \ -r <side_chain_RPC_endpoint> \
-w </path/to/node_admin/wallet> \ -w </path/to/node_admin/wallet> \
--node <HEX_node_public_key> \ --node <HEX_node_public_key> \
@ -55,7 +55,7 @@ subnetwork. Only the subnet owner is allowed to add and remove client
administrators from the subnetwork. administrators from the subnetwork.
```shell ```shell
$ neofs-adm morph subnet admin add \ $ frostfs-adm morph subnet admin add \
-r <side_chain_RPC_endpoint> \ -r <side_chain_RPC_endpoint> \
-w </path/to/owner/wallet> \ -w </path/to/owner/wallet> \
--admin <HEX_admin_public_key> \ --admin <HEX_admin_public_key> \
@ -72,7 +72,7 @@ positive integer number.
## Add client ## Add client
```shell ```shell
$ neofs-adm morph subnet client add \ $ frostfs-adm morph subnet client add \
-r <side_chain_RPC_endpoint> \ -r <side_chain_RPC_endpoint> \
-w </path/to/client_admin/wallet> \ -w </path/to/client_admin/wallet> \
--client <client_ownerID> \ --client <client_ownerID> \
@ -88,25 +88,25 @@ has been added by the subnet owner).
# Bootstrapping Storage Node # Bootstrapping Storage Node
After a subnetwork [is created](subnetwork-creation.md) and a node is included into it, the After a subnetwork [is created](subnetwork-creation.md) and a node is included into it, the
node could be bootstrapped and service subnetwork containers. node could be bootstrapped and service subnetwork containers.
For bootstrapping, you need to specify the ID of the subnetwork in the node's For bootstrapping, you need to specify the ID of the subnetwork in the node's
configuration: configuration:
```yaml ```yaml
... ...
node: node:
... ...
subnet: subnet:
entries: # list of IDs of subnets to enter in a text format of NeoFS API protocol (overrides corresponding attributes) entries: # list of IDs of subnets to enter in a text format of FrostFS API protocol (overrides corresponding attributes)
- <subnetwork_ID> - <subnetwork_ID>
... ...
... ...
``` ```
**NOTE:** specifying subnetwork that is denied for the node is not an error: **NOTE:** specifying subnetwork that is denied for the node is not an error:
that configuration value would be ignored. You do not need to specify zero that configuration value would be ignored. You do not need to specify zero
(with 0 ID) subnetwork: its inclusion is implicit. On the contrary, to exclude (with 0 ID) subnetwork: its inclusion is implicit. On the contrary, to exclude
a node from the default zero subnetwork, you need to specify it explicitly: a node from the default zero subnetwork, you need to specify it explicitly:
@ -122,16 +122,16 @@ node:
# Creating container in non-zero subnetwork # Creating container in non-zero subnetwork
Creating containers without using `--subnet` flag is equivalent to Creating containers without using `--subnet` flag is equivalent to
creating container in the zero subnetwork. creating container in the zero subnetwork.
To create a container in a private network, your wallet must be added to To create a container in a private network, your wallet must be added to
the client whitelist by the client admins or the subnet owners: the client whitelist by the client admins or the subnet owners:
```shell ```shell
$ neofs-cli container create \ $ frostfs-cli container create \
--policy 'REP 1' \ --policy 'REP 1' \
-w </path/to/wallet> \ -w </path/to/wallet> \
-r s01.neofs.devenv:8080 \ -r s01.frostfs.devenv:8080 \
--subnet <subnet_ID> --subnet <subnet_ID>
``` ```

View file

@ -0,0 +1,14 @@
package commonflags
const (
ConfigFlag = "config"
ConfigFlagShorthand = "c"
ConfigFlagUsage = "Config file"
ConfigDirFlag = "config-dir"
ConfigDirFlagUsage = "Config directory"
Verbose = "verbose"
VerboseShorthand = "v"
VerboseUsage = "Verbose output"
)

View file

@ -7,8 +7,8 @@ import (
"path/filepath" "path/filepath"
"text/template" "text/template"
"git.frostfs.info/TrueCloudLab/frostfs-node/pkg/innerring"
"github.com/nspcc-dev/neo-go/cli/input" "github.com/nspcc-dev/neo-go/cli/input"
"github.com/nspcc-dev/neofs-node/pkg/innerring"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
) )
@ -41,7 +41,7 @@ network:
container: {{ .ContainerFee}} container: {{ .ContainerFee}}
container_alias: {{ .ContainerAliasFee }} container_alias: {{ .ContainerAliasFee }}
withdraw: {{ .WithdrawFee}} withdraw: {{ .WithdrawFee}}
# if credentials section is omitted, then neofs-adm will require manual password input # if credentials section is omitted, then frostfs-adm will require manual password input
credentials: credentials:
contract: password # wallet for contract group signature{{ range.Glagolitics}} contract: password # wallet for contract group signature{{ range.Glagolitics}}
{{.}}: password{{end}} {{.}}: password{{end}}
@ -99,7 +99,7 @@ func defaultConfigPath() (string, error) {
return "", fmt.Errorf("getting home dir path: %w", err) return "", fmt.Errorf("getting home dir path: %w", err)
} }
return filepath.Join(home, ".neofs", "adm", "config.yml"), nil return filepath.Join(home, ".frostfs", "adm", "config.yml"), nil
} }
// generateConfigExample builds .yml representation of the config file. It is // generateConfigExample builds .yml representation of the config file. It is

View file

@ -5,7 +5,7 @@ import (
"path/filepath" "path/filepath"
"testing" "testing"
"github.com/nspcc-dev/neofs-node/pkg/innerring" "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/innerring"
"github.com/spf13/viper" "github.com/spf13/viper"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
@ -13,7 +13,7 @@ import (
func TestGenerateConfigExample(t *testing.T) { func TestGenerateConfigExample(t *testing.T) {
const ( const (
n = 10 n = 10
appDir = "/home/example/.neofs" appDir = "/home/example/.frostfs"
) )
configText, err := generateConfigExample(appDir, n) configText, err := generateConfigExample(appDir, n)

View file

@ -10,14 +10,14 @@ var (
// RootCmd is a root command of config section. // RootCmd is a root command of config section.
RootCmd = &cobra.Command{ RootCmd = &cobra.Command{
Use: "config", Use: "config",
Short: "Section for neofs-adm config related commands", Short: "Section for frostfs-adm config related commands",
} }
initCmd = &cobra.Command{ initCmd = &cobra.Command{
Use: "init", Use: "init",
Short: "Initialize basic neofs-adm configuration file", Short: "Initialize basic frostfs-adm configuration file",
Example: `neofs-adm config init Example: `frostfs-adm config init
neofs-adm config init --path .config/neofs-adm.yml`, frostfs-adm config init --path .config/frostfs-adm.yml`,
RunE: initConfig, RunE: initConfig,
} }
) )
@ -25,5 +25,5 @@ neofs-adm config init --path .config/neofs-adm.yml`,
func init() { func init() {
RootCmd.AddCommand(initCmd) RootCmd.AddCommand(initCmd)
initCmd.Flags().String(configPathFlag, "", "Path to config (default ~/.neofs/adm/config.yml)") initCmd.Flags().String(configPathFlag, "", "Path to config (default ~/.frostfs/adm/config.yml)")
} }

View file

@ -6,22 +6,23 @@ import (
"fmt" "fmt"
"math/big" "math/big"
"github.com/nspcc-dev/neo-go/pkg/core/native/nativenames" "git.frostfs.info/TrueCloudLab/frostfs-contract/nns"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap"
"github.com/nspcc-dev/neo-go/pkg/core/native/noderoles" "github.com/nspcc-dev/neo-go/pkg/core/native/noderoles"
"github.com/nspcc-dev/neo-go/pkg/core/state" "github.com/nspcc-dev/neo-go/pkg/core/state"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/nspcc-dev/neo-go/pkg/encoding/address" "github.com/nspcc-dev/neo-go/pkg/encoding/address"
"github.com/nspcc-dev/neo-go/pkg/encoding/fixedn" "github.com/nspcc-dev/neo-go/pkg/encoding/fixedn"
"github.com/nspcc-dev/neo-go/pkg/io" "github.com/nspcc-dev/neo-go/pkg/io"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/gas"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/invoker" "github.com/nspcc-dev/neo-go/pkg/rpcclient/invoker"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/rolemgmt"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/unwrap" "github.com/nspcc-dev/neo-go/pkg/rpcclient/unwrap"
"github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag" "github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag"
"github.com/nspcc-dev/neo-go/pkg/util" "github.com/nspcc-dev/neo-go/pkg/util"
"github.com/nspcc-dev/neo-go/pkg/vm/emit" "github.com/nspcc-dev/neo-go/pkg/vm/emit"
"github.com/nspcc-dev/neo-go/pkg/vm/stackitem" "github.com/nspcc-dev/neo-go/pkg/vm/stackitem"
"github.com/nspcc-dev/neo-go/pkg/vm/vmstate" "github.com/nspcc-dev/neo-go/pkg/vm/vmstate"
"github.com/nspcc-dev/neofs-contract/nns"
"github.com/nspcc-dev/neofs-sdk-go/netmap"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
) )
@ -43,6 +44,7 @@ const (
notaryEnabled = true notaryEnabled = true
) )
// nolint: funlen, gocognit
func dumpBalances(cmd *cobra.Command, _ []string) error { func dumpBalances(cmd *cobra.Command, _ []string) error {
var ( var (
dumpStorage, _ = cmd.Flags().GetBool(dumpBalancesStorageFlag) dumpStorage, _ = cmd.Flags().GetBool(dumpBalancesStorageFlag)
@ -59,39 +61,24 @@ func dumpBalances(cmd *cobra.Command, _ []string) error {
inv := invoker.New(c, nil) inv := invoker.New(c, nil)
ns, err := getNativeHashes(c)
if err != nil {
return fmt.Errorf("can't fetch the list of native contracts: %w", err)
}
gasHash, ok := ns[nativenames.Gas]
if !ok {
return fmt.Errorf("can't find the %s native contract hash", nativenames.Gas)
}
desigHash, ok := ns[nativenames.Designation]
if !ok {
return fmt.Errorf("can't find the %s native contract hash", nativenames.Designation)
}
if !notaryEnabled || dumpStorage || dumpAlphabet || dumpProxy { if !notaryEnabled || dumpStorage || dumpAlphabet || dumpProxy {
nnsCs, err = c.GetContractStateByID(1) nnsCs, err = c.GetContractStateByID(1)
if err != nil { if err != nil {
return fmt.Errorf("can't get NNS contract info: %w", err) return fmt.Errorf("can't get NNS contract info: %w", err)
} }
nmHash, err = nnsResolveHash(inv, nnsCs.Hash, netmapContract+".neofs") nmHash, err = nnsResolveHash(inv, nnsCs.Hash, netmapContract+".frostfs")
if err != nil { if err != nil {
return fmt.Errorf("can't get netmap contract hash: %w", err) return fmt.Errorf("can't get netmap contract hash: %w", err)
} }
} }
irList, err := fetchIRNodes(c, nmHash, desigHash) irList, err := fetchIRNodes(c, nmHash, rolemgmt.Hash)
if err != nil { if err != nil {
return err return err
} }
if err := fetchBalances(inv, gasHash, irList); err != nil { if err := fetchBalances(inv, gas.Hash, irList); err != nil {
return err return err
} }
printBalances(cmd, "Inner ring nodes balances:", irList) printBalances(cmd, "Inner ring nodes balances:", irList)
@ -123,20 +110,20 @@ func dumpBalances(cmd *cobra.Command, _ []string) error {
snList[i].scriptHash = pub.GetScriptHash() snList[i].scriptHash = pub.GetScriptHash()
} }
if err := fetchBalances(inv, gasHash, snList); err != nil { if err := fetchBalances(inv, gas.Hash, snList); err != nil {
return err return err
} }
printBalances(cmd, "\nStorage node balances:", snList) printBalances(cmd, "\nStorage node balances:", snList)
} }
if dumpProxy { if dumpProxy {
h, err := nnsResolveHash(inv, nnsCs.Hash, proxyContract+".neofs") h, err := nnsResolveHash(inv, nnsCs.Hash, proxyContract+".frostfs")
if err != nil { if err != nil {
return fmt.Errorf("can't get hash of the proxy contract: %w", err) return fmt.Errorf("can't get hash of the proxy contract: %w", err)
} }
proxyList := []accBalancePair{{scriptHash: h}} proxyList := []accBalancePair{{scriptHash: h}}
if err := fetchBalances(inv, gasHash, proxyList); err != nil { if err := fetchBalances(inv, gas.Hash, proxyList); err != nil {
return err return err
} }
printBalances(cmd, "\nProxy contract balance:", proxyList) printBalances(cmd, "\nProxy contract balance:", proxyList)
@ -168,7 +155,7 @@ func dumpBalances(cmd *cobra.Command, _ []string) error {
alphaList[i].scriptHash = h alphaList[i].scriptHash = h
} }
if err := fetchBalances(inv, gasHash, alphaList); err != nil { if err := fetchBalances(inv, gas.Hash, alphaList); err != nil {
return err return err
} }
printBalances(cmd, "\nAlphabet contracts balances:", alphaList) printBalances(cmd, "\nAlphabet contracts balances:", alphaList)

View file

@ -35,7 +35,7 @@ func dumpNetworkConfig(cmd *cobra.Command, _ []string) error {
return fmt.Errorf("can't get NNS contract info: %w", err) return fmt.Errorf("can't get NNS contract info: %w", err)
} }
nmHash, err := nnsResolveHash(inv, cs.Hash, netmapContract+".neofs") nmHash, err := nnsResolveHash(inv, cs.Hash, netmapContract+".frostfs")
if err != nil { if err != nil {
return fmt.Errorf("can't get netmap contract hash: %w", err) return fmt.Errorf("can't get netmap contract hash: %w", err)
} }
@ -109,7 +109,7 @@ func setConfigCmd(cmd *cobra.Command, args []string) error {
return fmt.Errorf("can't get NNS contract info: %w", err) return fmt.Errorf("can't get NNS contract info: %w", err)
} }
nmHash, err := nnsResolveHash(wCtx.ReadOnlyInvoker, cs.Hash, netmapContract+".neofs") nmHash, err := nnsResolveHash(wCtx.ReadOnlyInvoker, cs.Hash, netmapContract+".frostfs")
if err != nil { if err != nil {
return fmt.Errorf("can't get netmap contract hash: %w", err) return fmt.Errorf("can't get netmap contract hash: %w", err)
} }
@ -140,14 +140,14 @@ func setConfigCmd(cmd *cobra.Command, args []string) error {
return wCtx.awaitTx() return wCtx.awaitTx()
} }
func parseConfigPair(kvStr string, force bool) (key string, val interface{}, err error) { func parseConfigPair(kvStr string, force bool) (key string, val any, err error) {
kv := strings.SplitN(kvStr, "=", 2) k, v, found := strings.Cut(kvStr, "=")
if len(kv) != 2 { if !found {
return "", nil, fmt.Errorf("invalid parameter format: must be 'key=val', got: %s", kvStr) return "", nil, fmt.Errorf("invalid parameter format: must be 'key=val', got: %s", kvStr)
} }
key = kv[0] key = k
valRaw := kv[1] valRaw := v
switch key { switch key {
case netmapAuditFeeKey, netmapBasicIncomeRateKey, case netmapAuditFeeKey, netmapBasicIncomeRateKey,
@ -162,7 +162,7 @@ func parseConfigPair(kvStr string, force bool) (key string, val interface{}, err
case netmapEigenTrustAlphaKey: case netmapEigenTrustAlphaKey:
// just check that it could // just check that it could
// be parsed correctly // be parsed correctly
_, err = strconv.ParseFloat(kv[1], 64) _, err = strconv.ParseFloat(v, 64)
if err != nil { if err != nil {
err = fmt.Errorf("could not parse %s's value '%s' as float: %w", key, valRaw, err) err = fmt.Errorf("could not parse %s's value '%s' as float: %w", key, valRaw, err)
} }

View file

@ -7,6 +7,7 @@ import (
"os" "os"
"sort" "sort"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"github.com/nspcc-dev/neo-go/pkg/crypto/hash" "github.com/nspcc-dev/neo-go/pkg/crypto/hash"
"github.com/nspcc-dev/neo-go/pkg/io" "github.com/nspcc-dev/neo-go/pkg/io"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/invoker" "github.com/nspcc-dev/neo-go/pkg/rpcclient/invoker"
@ -15,13 +16,43 @@ import (
"github.com/nspcc-dev/neo-go/pkg/util" "github.com/nspcc-dev/neo-go/pkg/util"
"github.com/nspcc-dev/neo-go/pkg/vm/emit" "github.com/nspcc-dev/neo-go/pkg/vm/emit"
"github.com/nspcc-dev/neo-go/pkg/vm/stackitem" "github.com/nspcc-dev/neo-go/pkg/vm/stackitem"
cid "github.com/nspcc-dev/neofs-sdk-go/container/id"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
) )
var errInvalidContainerResponse = errors.New("invalid response from container contract") var errInvalidContainerResponse = errors.New("invalid response from container contract")
func getContainerContractHash(cmd *cobra.Command, inv *invoker.Invoker, c Client) (util.Uint160, error) {
s, err := cmd.Flags().GetString(containerContractFlag)
var ch util.Uint160
if err == nil {
ch, err = util.Uint160DecodeStringLE(s)
}
if err != nil {
nnsCs, err := c.GetContractStateByID(1)
if err != nil {
return util.Uint160{}, fmt.Errorf("can't get NNS contract state: %w", err)
}
ch, err = nnsResolveHash(inv, nnsCs.Hash, containerContract+".frostfs")
if err != nil {
return util.Uint160{}, err
}
}
return ch, nil
}
func getContainersList(inv *invoker.Invoker, ch util.Uint160) ([][]byte, error) {
res, err := inv.Call(ch, "list", "")
if err != nil {
return nil, fmt.Errorf("%w: %v", errInvalidContainerResponse, err)
}
itm, err := unwrap.Item(res, err)
if _, ok := itm.(stackitem.Null); !ok {
return unwrap.ArrayOfBytes(res, err)
}
return nil, nil
}
func dumpContainers(cmd *cobra.Command, _ []string) error { func dumpContainers(cmd *cobra.Command, _ []string) error {
filename, err := cmd.Flags().GetString(containerDumpFlag) filename, err := cmd.Flags().GetString(containerDumpFlag)
if err != nil { if err != nil {
@ -35,24 +66,12 @@ func dumpContainers(cmd *cobra.Command, _ []string) error {
inv := invoker.New(c, nil) inv := invoker.New(c, nil)
nnsCs, err := c.GetContractStateByID(1) ch, err := getContainerContractHash(cmd, inv, c)
if err != nil { if err != nil {
return fmt.Errorf("can't get NNS contract state: %w", err) return fmt.Errorf("unable to get contaract hash: %w", err)
} }
var ch util.Uint160 cids, err := getContainersList(inv, ch)
s, err := cmd.Flags().GetString(containerContractFlag)
if err == nil {
ch, err = util.Uint160DecodeStringLE(s)
}
if err != nil {
ch, err = nnsResolveHash(inv, nnsCs.Hash, containerContract+".neofs")
if err != nil {
return err
}
}
cids, err := unwrap.ArrayOfBytes(inv.Call(ch, "list", ""))
if err != nil { if err != nil {
return fmt.Errorf("%w: %v", errInvalidContainerResponse, err) return fmt.Errorf("%w: %v", errInvalidContainerResponse, err)
} }
@ -104,6 +123,36 @@ func dumpContainers(cmd *cobra.Command, _ []string) error {
return os.WriteFile(filename, out, 0o660) return os.WriteFile(filename, out, 0o660)
} }
func listContainers(cmd *cobra.Command, _ []string) error {
c, err := getN3Client(viper.GetViper())
if err != nil {
return fmt.Errorf("can't create N3 client: %w", err)
}
inv := invoker.New(c, nil)
ch, err := getContainerContractHash(cmd, inv, c)
if err != nil {
return fmt.Errorf("unable to get contaract hash: %w", err)
}
cids, err := getContainersList(inv, ch)
if err != nil {
return fmt.Errorf("%w: %v", errInvalidContainerResponse, err)
}
for _, id := range cids {
var idCnr cid.ID
err = idCnr.Decode(id)
if err != nil {
return fmt.Errorf("unable to decode container id: %w", err)
}
cmd.Println(idCnr)
}
return nil
}
// nolint: funlen
func restoreContainers(cmd *cobra.Command, _ []string) error { func restoreContainers(cmd *cobra.Command, _ []string) error {
filename, err := cmd.Flags().GetString(containerDumpFlag) filename, err := cmd.Flags().GetString(containerDumpFlag)
if err != nil { if err != nil {
@ -121,7 +170,7 @@ func restoreContainers(cmd *cobra.Command, _ []string) error {
return fmt.Errorf("can't get NNS contract state: %w", err) return fmt.Errorf("can't get NNS contract state: %w", err)
} }
ch, err := nnsResolveHash(wCtx.ReadOnlyInvoker, nnsCs.Hash, containerContract+".neofs") ch, err := nnsResolveHash(wCtx.ReadOnlyInvoker, nnsCs.Hash, containerContract+".frostfs")
if err != nil { if err != nil {
return fmt.Errorf("can't fetch container contract hash: %w", err) return fmt.Errorf("can't fetch container contract hash: %w", err)
} }

View file

@ -6,16 +6,16 @@ import (
"os" "os"
"strings" "strings"
"git.frostfs.info/TrueCloudLab/frostfs-contract/nns"
"github.com/nspcc-dev/neo-go/cli/cmdargs" "github.com/nspcc-dev/neo-go/cli/cmdargs"
"github.com/nspcc-dev/neo-go/pkg/core/native/nativenames"
"github.com/nspcc-dev/neo-go/pkg/core/state" "github.com/nspcc-dev/neo-go/pkg/core/state"
"github.com/nspcc-dev/neo-go/pkg/encoding/address" "github.com/nspcc-dev/neo-go/pkg/encoding/address"
"github.com/nspcc-dev/neo-go/pkg/io" "github.com/nspcc-dev/neo-go/pkg/io"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/management"
"github.com/nspcc-dev/neo-go/pkg/services/rpcsrv/params" "github.com/nspcc-dev/neo-go/pkg/services/rpcsrv/params"
"github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag" "github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag"
"github.com/nspcc-dev/neo-go/pkg/vm/emit" "github.com/nspcc-dev/neo-go/pkg/vm/emit"
"github.com/nspcc-dev/neo-go/pkg/vm/opcode" "github.com/nspcc-dev/neo-go/pkg/vm/opcode"
"github.com/nspcc-dev/neofs-contract/nns"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
) )
@ -35,7 +35,7 @@ Optionally, arguments can be provided to be passed to a contract's _deploy funct
The syntax is the same as for 'neo-go contract testinvokefunction' command. The syntax is the same as for 'neo-go contract testinvokefunction' command.
Compiled contract file name must contain '_contract.nef' suffix. Compiled contract file name must contain '_contract.nef' suffix.
Contract's manifest file name must be 'config.json'. Contract's manifest file name must be 'config.json'.
NNS name is taken by stripping '_contract.nef' from the NEF file (similar to neofs contracts).`, NNS name is taken by stripping '_contract.nef' from the NEF file (similar to frostfs contracts).`,
PreRun: func(cmd *cobra.Command, _ []string) { PreRun: func(cmd *cobra.Command, _ []string) {
_ = viper.BindPFlag(alphabetWalletsFlag, cmd.Flags().Lookup(alphabetWalletsFlag)) _ = viper.BindPFlag(alphabetWalletsFlag, cmd.Flags().Lookup(alphabetWalletsFlag))
_ = viper.BindPFlag(endpointFlag, cmd.Flags().Lookup(endpointFlag)) _ = viper.BindPFlag(endpointFlag, cmd.Flags().Lookup(endpointFlag))
@ -54,9 +54,10 @@ func init() {
_ = deployCmd.MarkFlagFilename(contractPathFlag) _ = deployCmd.MarkFlagFilename(contractPathFlag)
ff.Bool(updateFlag, false, "Update an existing contract") ff.Bool(updateFlag, false, "Update an existing contract")
ff.String(customZoneFlag, "neofs", "Custom zone for NNS") ff.String(customZoneFlag, "frostfs", "Custom zone for NNS")
} }
// nolint: funlen
func deployContractCmd(cmd *cobra.Command, args []string) error { func deployContractCmd(cmd *cobra.Command, args []string) error {
v := viper.GetViper() v := viper.GetViper()
c, err := newInitializeContext(cmd, v) c, err := newInitializeContext(cmd, v)
@ -81,7 +82,7 @@ func deployContractCmd(cmd *cobra.Command, args []string) error {
return fmt.Errorf("can't fetch NNS contract state: %w", err) return fmt.Errorf("can't fetch NNS contract state: %w", err)
} }
callHash := c.nativeHash(nativenames.Management) callHash := management.Hash
method := deployMethodName method := deployMethodName
zone, _ := cmd.Flags().GetString(customZoneFlag) zone, _ := cmd.Flags().GetString(customZoneFlag)
domain := ctrName + "." + zone domain := ctrName + "." + zone

View file

@ -14,7 +14,7 @@ import (
func downloadContractsFromGithub(cmd *cobra.Command) (io.ReadCloser, error) { func downloadContractsFromGithub(cmd *cobra.Command) (io.ReadCloser, error) {
gcl := github.NewClient(nil) gcl := github.NewClient(nil)
release, _, err := gcl.Repositories.GetLatestRelease(context.Background(), "nspcc-dev", "neofs-contract") release, _, err := gcl.Repositories.GetLatestRelease(context.Background(), "nspcc-dev", "frostfs-contract")
if err != nil { if err != nil {
return nil, fmt.Errorf("can't fetch release info: %w", err) return nil, fmt.Errorf("can't fetch release info: %w", err)
} }
@ -23,7 +23,7 @@ func downloadContractsFromGithub(cmd *cobra.Command) (io.ReadCloser, error) {
var url string var url string
for _, a := range release.Assets { for _, a := range release.Assets {
if strings.HasPrefix(a.GetName(), "neofs-contract") { if strings.HasPrefix(a.GetName(), "frostfs-contract") {
url = a.GetBrowserDownloadURL() url = a.GetBrowserDownloadURL()
break break
} }

View file

@ -2,10 +2,13 @@ package morph
import ( import (
"bytes" "bytes"
"errors"
"fmt" "fmt"
"strings" "strings"
"text/tabwriter" "text/tabwriter"
"git.frostfs.info/TrueCloudLab/frostfs-contract/nns"
morphClient "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/morph/client"
"github.com/nspcc-dev/neo-go/pkg/io" "github.com/nspcc-dev/neo-go/pkg/io"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/invoker" "github.com/nspcc-dev/neo-go/pkg/rpcclient/invoker"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/unwrap" "github.com/nspcc-dev/neo-go/pkg/rpcclient/unwrap"
@ -15,7 +18,6 @@ import (
"github.com/nspcc-dev/neo-go/pkg/vm/opcode" "github.com/nspcc-dev/neo-go/pkg/vm/opcode"
"github.com/nspcc-dev/neo-go/pkg/vm/stackitem" "github.com/nspcc-dev/neo-go/pkg/vm/stackitem"
"github.com/nspcc-dev/neo-go/pkg/vm/vmstate" "github.com/nspcc-dev/neo-go/pkg/vm/vmstate"
"github.com/nspcc-dev/neofs-contract/nns"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
) )
@ -83,7 +85,7 @@ func dumpContractHashes(cmd *cobra.Command, _ []string) error {
for _, ctrName := range contractList { for _, ctrName := range contractList {
bw.Reset() bw.Reset()
emit.AppCall(bw.BinWriter, cs.Hash, "resolve", callflag.ReadOnly, emit.AppCall(bw.BinWriter, cs.Hash, "resolve", callflag.ReadOnly,
ctrName+".neofs", int64(nns.TXT)) ctrName+".frostfs", int64(nns.TXT))
res, err := c.InvokeScript(bw.Bytes(), nil) res, err := c.InvokeScript(bw.Bytes(), nil)
if err != nil { if err != nil {
@ -107,31 +109,30 @@ func dumpContractHashes(cmd *cobra.Command, _ []string) error {
func dumpCustomZoneHashes(cmd *cobra.Command, nnsHash util.Uint160, zone string, c Client) error { func dumpCustomZoneHashes(cmd *cobra.Command, nnsHash util.Uint160, zone string, c Client) error {
const nnsMaxTokens = 100 const nnsMaxTokens = 100
inv := invoker.New(c, nil)
arr, err := unwrap.Array(inv.CallAndExpandIterator(nnsHash, "tokens", nnsMaxTokens)) inv := invoker.New(c, nil)
if err != nil {
return fmt.Errorf("can't get a list of NNS domains: %w", err)
}
if !strings.HasPrefix(zone, ".") { if !strings.HasPrefix(zone, ".") {
zone = "." + zone zone = "." + zone
} }
var infos []contractDumpInfo var infos []contractDumpInfo
for i := range arr { processItem := func(item stackitem.Item) {
bs, err := arr[i].TryBytes() bs, err := item.TryBytes()
if err != nil { if err != nil {
continue cmd.PrintErrf("Invalid NNS record: %v\n", err)
return
} }
if !bytes.HasSuffix(bs, []byte(zone)) { if !bytes.HasSuffix(bs, []byte(zone)) || bytes.HasPrefix(bs, []byte(morphClient.NNSGroupKeyName)) {
continue // Related https://github.com/nspcc-dev/neofs-contract/issues/316.
return
} }
h, err := nnsResolveHash(inv, nnsHash, string(bs)) h, err := nnsResolveHash(inv, nnsHash, string(bs))
if err != nil { if err != nil {
continue cmd.PrintErrf("Could not resolve name %s: %v\n", string(bs), err)
return
} }
infos = append(infos, contractDumpInfo{ infos = append(infos, contractDumpInfo{
@ -140,6 +141,39 @@ func dumpCustomZoneHashes(cmd *cobra.Command, nnsHash util.Uint160, zone string,
}) })
} }
sessionID, iter, err := unwrap.SessionIterator(inv.Call(nnsHash, "tokens"))
if err != nil {
if errors.Is(err, unwrap.ErrNoSessionID) {
items, err := unwrap.Array(inv.CallAndExpandIterator(nnsHash, "tokens", nnsMaxTokens))
if err != nil {
return fmt.Errorf("can't get a list of NNS domains: %w", err)
}
if len(items) == nnsMaxTokens {
cmd.PrintErrln("Provided RPC endpoint doesn't support sessions, some hashes might be lost.")
}
for i := range items {
processItem(items[i])
}
} else {
return err
}
} else {
defer func() {
_ = inv.TerminateSession(sessionID)
}()
items, err := inv.TraverseIterator(sessionID, &iter, nnsMaxTokens)
for err == nil && len(items) != 0 {
for i := range items {
processItem(items[i])
}
items, err = inv.TraverseIterator(sessionID, &iter, nnsMaxTokens)
}
if err != nil {
return fmt.Errorf("error during NNS domains iteration: %w", err)
}
}
fillContractVersion(cmd, c, infos) fillContractVersion(cmd, c, infos)
printContractInfo(cmd, infos) printContractInfo(cmd, infos)

View file

@ -24,7 +24,7 @@ func forceNewEpochCmd(cmd *cobra.Command, args []string) error {
return fmt.Errorf("can't get NNS contract info: %w", err) return fmt.Errorf("can't get NNS contract info: %w", err)
} }
nmHash, err := nnsResolveHash(wCtx.ReadOnlyInvoker, cs.Hash, netmapContract+".neofs") nmHash, err := nnsResolveHash(wCtx.ReadOnlyInvoker, cs.Hash, netmapContract+".frostfs")
if err != nil { if err != nil {
return fmt.Errorf("can't get netmap contract hash: %w", err) return fmt.Errorf("can't get netmap contract hash: %w", err)
} }

View file

@ -6,19 +6,19 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"github.com/nspcc-dev/neo-go/pkg/core/native/nativenames" "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/modules/config"
"git.frostfs.info/TrueCloudLab/frostfs-node/pkg/innerring"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/nspcc-dev/neo-go/pkg/encoding/address" "github.com/nspcc-dev/neo-go/pkg/encoding/address"
"github.com/nspcc-dev/neo-go/pkg/encoding/fixedn" "github.com/nspcc-dev/neo-go/pkg/encoding/fixedn"
"github.com/nspcc-dev/neo-go/pkg/io" "github.com/nspcc-dev/neo-go/pkg/io"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/gas"
"github.com/nspcc-dev/neo-go/pkg/smartcontract" "github.com/nspcc-dev/neo-go/pkg/smartcontract"
"github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag" "github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag"
"github.com/nspcc-dev/neo-go/pkg/util" "github.com/nspcc-dev/neo-go/pkg/util"
"github.com/nspcc-dev/neo-go/pkg/vm/emit" "github.com/nspcc-dev/neo-go/pkg/vm/emit"
"github.com/nspcc-dev/neo-go/pkg/vm/opcode" "github.com/nspcc-dev/neo-go/pkg/vm/opcode"
"github.com/nspcc-dev/neo-go/pkg/wallet" "github.com/nspcc-dev/neo-go/pkg/wallet"
"github.com/nspcc-dev/neofs-node/cmd/neofs-adm/internal/modules/config"
"github.com/nspcc-dev/neofs-node/pkg/innerring"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
) )
@ -198,10 +198,8 @@ func refillGas(cmd *cobra.Command, gasFlag string, createWallet bool) (err error
return err return err
} }
gasHash := wCtx.nativeHash(nativenames.Gas)
bw := io.NewBufBinWriter() bw := io.NewBufBinWriter()
emit.AppCall(bw.BinWriter, gasHash, "transfer", callflag.All, emit.AppCall(bw.BinWriter, gas.Hash, "transfer", callflag.All,
wCtx.CommitteeAcc.Contract.ScriptHash(), gasReceiver, int64(gasAmount), nil) wCtx.CommitteeAcc.Contract.ScriptHash(), gasReceiver, int64(gasAmount), nil)
emit.Opcodes(bw.BinWriter, opcode.ASSERT) emit.Opcodes(bw.BinWriter, opcode.ASSERT)
if bw.Err != nil { if bw.Err != nil {

View file

@ -9,11 +9,11 @@ import (
"strconv" "strconv"
"testing" "testing"
"git.frostfs.info/TrueCloudLab/frostfs-node/pkg/innerring"
"github.com/nspcc-dev/neo-go/cli/input" "github.com/nspcc-dev/neo-go/cli/input"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/nspcc-dev/neo-go/pkg/smartcontract" "github.com/nspcc-dev/neo-go/pkg/smartcontract"
"github.com/nspcc-dev/neo-go/pkg/wallet" "github.com/nspcc-dev/neo-go/pkg/wallet"
"github.com/nspcc-dev/neofs-node/pkg/innerring"
"github.com/spf13/viper" "github.com/spf13/viper"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"golang.org/x/term" "golang.org/x/term"

View file

@ -6,11 +6,11 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/modules/config"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/nspcc-dev/neo-go/pkg/smartcontract/manifest" "github.com/nspcc-dev/neo-go/pkg/smartcontract/manifest"
"github.com/nspcc-dev/neo-go/pkg/util" "github.com/nspcc-dev/neo-go/pkg/util"
"github.com/nspcc-dev/neo-go/pkg/wallet" "github.com/nspcc-dev/neo-go/pkg/wallet"
"github.com/nspcc-dev/neofs-node/cmd/neofs-adm/internal/modules/config"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
) )

View file

@ -7,6 +7,9 @@ import (
"path/filepath" "path/filepath"
"time" "time"
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/modules/config"
"git.frostfs.info/TrueCloudLab/frostfs-node/pkg/innerring"
morphClient "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/morph/client"
"github.com/nspcc-dev/neo-go/pkg/core/native/nativenames" "github.com/nspcc-dev/neo-go/pkg/core/native/nativenames"
"github.com/nspcc-dev/neo-go/pkg/core/state" "github.com/nspcc-dev/neo-go/pkg/core/state"
"github.com/nspcc-dev/neo-go/pkg/core/transaction" "github.com/nspcc-dev/neo-go/pkg/core/transaction"
@ -16,9 +19,6 @@ import (
"github.com/nspcc-dev/neo-go/pkg/util" "github.com/nspcc-dev/neo-go/pkg/util"
"github.com/nspcc-dev/neo-go/pkg/vm/vmstate" "github.com/nspcc-dev/neo-go/pkg/vm/vmstate"
"github.com/nspcc-dev/neo-go/pkg/wallet" "github.com/nspcc-dev/neo-go/pkg/wallet"
"github.com/nspcc-dev/neofs-node/cmd/neofs-adm/internal/modules/config"
"github.com/nspcc-dev/neofs-node/pkg/innerring"
morphClient "github.com/nspcc-dev/neofs-node/pkg/morph/client"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
) )
@ -43,7 +43,6 @@ type initializeContext struct {
Contracts map[string]*contractState Contracts map[string]*contractState
Command *cobra.Command Command *cobra.Command
ContractPath string ContractPath string
Natives map[string]util.Uint160
} }
func initializeSideChainCmd(cmd *cobra.Command, args []string) error { func initializeSideChainCmd(cmd *cobra.Command, args []string) error {
@ -109,6 +108,7 @@ func (c *initializeContext) close() {
} }
} }
// nolint: funlen
func newInitializeContext(cmd *cobra.Command, v *viper.Viper) (*initializeContext, error) { func newInitializeContext(cmd *cobra.Command, v *viper.Viper) (*initializeContext, error) {
walletDir := config.ResolveHomePath(viper.GetString(alphabetWalletsFlag)) walletDir := config.ResolveHomePath(viper.GetString(alphabetWalletsFlag))
wallets, err := openAlphabetWallets(v, walletDir) wallets, err := openAlphabetWallets(v, walletDir)
@ -116,8 +116,10 @@ func newInitializeContext(cmd *cobra.Command, v *viper.Viper) (*initializeContex
return nil, err return nil, err
} }
needContracts := cmd.Name() == "update-contracts" || cmd.Name() == "init"
var w *wallet.Wallet var w *wallet.Wallet
if cmd.Name() != "deploy" { if needContracts {
w, err = openContractWallet(v, cmd, walletDir) w, err = openContractWallet(v, cmd, walletDir)
if err != nil { if err != nil {
return nil, err return nil, err
@ -158,7 +160,6 @@ func newInitializeContext(cmd *cobra.Command, v *viper.Viper) (*initializeContex
} }
} }
needContracts := cmd.Name() == "update-contracts" || cmd.Name() == "init"
if needContracts { if needContracts {
ctrPath, err = cmd.Flags().GetString(contractsInitFlag) ctrPath, err = cmd.Flags().GetString(contractsInitFlag)
if err != nil { if err != nil {
@ -166,8 +167,7 @@ func newInitializeContext(cmd *cobra.Command, v *viper.Viper) (*initializeContex
} }
} }
nativeHashes, err := getNativeHashes(c) if err := checkNotaryEnabled(c); err != nil {
if err != nil {
return nil, err return nil, err
} }
@ -195,7 +195,6 @@ func newInitializeContext(cmd *cobra.Command, v *viper.Viper) (*initializeContex
Command: cmd, Command: cmd,
Contracts: make(map[string]*contractState), Contracts: make(map[string]*contractState),
ContractPath: ctrPath, ContractPath: ctrPath,
Natives: nativeHashes,
} }
if needContracts { if needContracts {
@ -208,10 +207,6 @@ func newInitializeContext(cmd *cobra.Command, v *viper.Viper) (*initializeContex
return initCtx, nil return initCtx, nil
} }
func (c *initializeContext) nativeHash(name string) util.Uint160 {
return c.Natives[name]
}
func openAlphabetWallets(v *viper.Viper, walletDir string) ([]*wallet.Wallet, error) { func openAlphabetWallets(v *viper.Viper, walletDir string) ([]*wallet.Wallet, error) {
walletFiles, err := os.ReadDir(walletDir) walletFiles, err := os.ReadDir(walletDir)
if err != nil { if err != nil {
@ -450,10 +445,10 @@ func getWalletAccount(w *wallet.Wallet, typ string) (*wallet.Account, error) {
return nil, fmt.Errorf("account for '%s' not found", typ) return nil, fmt.Errorf("account for '%s' not found", typ)
} }
func getNativeHashes(c Client) (map[string]util.Uint160, error) { func checkNotaryEnabled(c Client) error {
ns, err := c.GetNativeContracts() ns, err := c.GetNativeContracts()
if err != nil { if err != nil {
return nil, fmt.Errorf("can't get native contract hashes: %w", err) return fmt.Errorf("can't get native contract hashes: %w", err)
} }
notaryEnabled := false notaryEnabled := false
@ -465,7 +460,7 @@ func getNativeHashes(c Client) (map[string]util.Uint160, error) {
nativeHashes[ns[i].Manifest.Name] = ns[i].Hash nativeHashes[ns[i].Manifest.Name] = ns[i].Hash
} }
if !notaryEnabled { if !notaryEnabled {
return nil, errors.New("notary contract must be enabled") return errors.New("notary contract must be enabled")
} }
return nativeHashes, nil return nil
} }

View file

@ -12,13 +12,17 @@ import (
"path/filepath" "path/filepath"
"strings" "strings"
"github.com/nspcc-dev/neo-go/pkg/core/native/nativenames" "git.frostfs.info/TrueCloudLab/frostfs-contract/common"
"git.frostfs.info/TrueCloudLab/frostfs-contract/nns"
"git.frostfs.info/TrueCloudLab/frostfs-node/pkg/innerring"
morphClient "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/morph/client"
"github.com/nspcc-dev/neo-go/pkg/core/state" "github.com/nspcc-dev/neo-go/pkg/core/state"
"github.com/nspcc-dev/neo-go/pkg/core/transaction" "github.com/nspcc-dev/neo-go/pkg/core/transaction"
"github.com/nspcc-dev/neo-go/pkg/encoding/address" "github.com/nspcc-dev/neo-go/pkg/encoding/address"
io2 "github.com/nspcc-dev/neo-go/pkg/io" io2 "github.com/nspcc-dev/neo-go/pkg/io"
"github.com/nspcc-dev/neo-go/pkg/rpcclient" "github.com/nspcc-dev/neo-go/pkg/rpcclient"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/actor" "github.com/nspcc-dev/neo-go/pkg/rpcclient/actor"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/management"
"github.com/nspcc-dev/neo-go/pkg/smartcontract" "github.com/nspcc-dev/neo-go/pkg/smartcontract"
"github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag" "github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag"
"github.com/nspcc-dev/neo-go/pkg/smartcontract/manifest" "github.com/nspcc-dev/neo-go/pkg/smartcontract/manifest"
@ -27,21 +31,18 @@ import (
"github.com/nspcc-dev/neo-go/pkg/vm/emit" "github.com/nspcc-dev/neo-go/pkg/vm/emit"
"github.com/nspcc-dev/neo-go/pkg/vm/opcode" "github.com/nspcc-dev/neo-go/pkg/vm/opcode"
"github.com/nspcc-dev/neo-go/pkg/vm/vmstate" "github.com/nspcc-dev/neo-go/pkg/vm/vmstate"
"github.com/nspcc-dev/neofs-contract/nns"
"github.com/nspcc-dev/neofs-node/pkg/innerring"
morphClient "github.com/nspcc-dev/neofs-node/pkg/morph/client"
"github.com/spf13/viper" "github.com/spf13/viper"
) )
const ( const (
nnsContract = "nns" nnsContract = "nns"
neofsContract = "neofs" // not deployed in side-chain. frostfsContract = "frostfs" // not deployed in side-chain.
processingContract = "processing" // not deployed in side-chain. processingContract = "processing" // not deployed in side-chain.
alphabetContract = "alphabet" alphabetContract = "alphabet"
auditContract = "audit" auditContract = "audit"
balanceContract = "balance" balanceContract = "balance"
containerContract = "container" containerContract = "container"
neofsIDContract = "neofsid" frostfsIDContract = "frostfsid"
netmapContract = "netmap" netmapContract = "netmap"
proxyContract = "proxy" proxyContract = "proxy"
reputationContract = "reputation" reputationContract = "reputation"
@ -71,7 +72,7 @@ var (
auditContract, auditContract,
balanceContract, balanceContract,
containerContract, containerContract,
neofsIDContract, frostfsIDContract,
netmapContract, netmapContract,
proxyContract, proxyContract,
reputationContract, reputationContract,
@ -79,7 +80,7 @@ var (
} }
fullContractList = append([]string{ fullContractList = append([]string{
neofsContract, frostfsContract,
processingContract, processingContract,
nnsContract, nnsContract,
alphabetContract, alphabetContract,
@ -106,7 +107,11 @@ func (c *initializeContext) deployNNS(method string) error {
nnsCs, err := c.nnsContractState() nnsCs, err := c.nnsContractState()
if err == nil { if err == nil {
if nnsCs.NEF.Checksum == cs.NEF.Checksum { if nnsCs.NEF.Checksum == cs.NEF.Checksum {
c.Command.Println("NNS contract is already deployed.") if method == deployMethodName {
c.Command.Println("NNS contract is already deployed.")
} else {
c.Command.Println("NNS contract is already updated.")
}
return nil return nil
} }
h = nnsCs.Hash h = nnsCs.Hash
@ -123,7 +128,7 @@ func (c *initializeContext) deployNNS(method string) error {
Scopes: transaction.CalledByEntry, Scopes: transaction.CalledByEntry,
} }
invokeHash := c.nativeHash(nativenames.Management) invokeHash := management.Hash
if method == updateMethodName { if method == updateMethodName {
invokeHash = nnsCs.Hash invokeHash = nnsCs.Hash
} }
@ -151,8 +156,8 @@ func (c *initializeContext) deployNNS(method string) error {
return c.awaitTx() return c.awaitTx()
} }
// nolint: funlen
func (c *initializeContext) updateContracts() error { func (c *initializeContext) updateContracts() error {
mgmtHash := c.nativeHash(nativenames.Management)
alphaCs := c.getContract(alphabetContract) alphaCs := c.getContract(alphabetContract)
nnsCs, err := c.nnsContractState() nnsCs, err := c.nnsContractState()
@ -163,7 +168,7 @@ func (c *initializeContext) updateContracts() error {
w := io2.NewBufBinWriter() w := io2.NewBufBinWriter()
var keysParam []interface{} var keysParam []any
// Update script size for a single-node committee is close to the maximum allowed size of 65535. // Update script size for a single-node committee is close to the maximum allowed size of 65535.
// Because of this we want to reuse alphabet contract NEF and manifest for different updates. // Because of this we want to reuse alphabet contract NEF and manifest for different updates.
@ -206,7 +211,10 @@ func (c *initializeContext) updateContracts() error {
} }
if err := c.sendCommitteeTx(w.Bytes(), false); err != nil { if err := c.sendCommitteeTx(w.Bytes(), false); err != nil {
return err if !strings.Contains(err.Error(), common.ErrAlreadyUpdated) {
return err
}
c.Command.Println("Alphabet contracts are already updated.")
} }
w.Reset() w.Reset()
@ -220,7 +228,7 @@ func (c *initializeContext) updateContracts() error {
cs := c.getContract(ctrName) cs := c.getContract(ctrName)
method := updateMethodName method := updateMethodName
ctrHash, err := nnsResolveHash(c.ReadOnlyInvoker, nnsHash, ctrName+".neofs") ctrHash, err := nnsResolveHash(c.ReadOnlyInvoker, nnsHash, ctrName+".frostfs")
if err != nil { if err != nil {
if errors.Is(err, errMissingNNSRecord) { if errors.Is(err, errMissingNNSRecord) {
// if contract not found we deploy it instead of update // if contract not found we deploy it instead of update
@ -235,7 +243,7 @@ func (c *initializeContext) updateContracts() error {
return fmt.Errorf("can't sign manifest group: %v", err) return fmt.Errorf("can't sign manifest group: %v", err)
} }
invokeHash := mgmtHash invokeHash := management.Hash
if method == updateMethodName { if method == updateMethodName {
invokeHash = ctrHash invokeHash = ctrHash
} }
@ -243,14 +251,18 @@ func (c *initializeContext) updateContracts() error {
params := getContractDeployParameters(cs, c.getContractDeployData(ctrName, keysParam)) params := getContractDeployParameters(cs, c.getContractDeployData(ctrName, keysParam))
res, err := c.CommitteeAct.MakeCall(invokeHash, method, params...) res, err := c.CommitteeAct.MakeCall(invokeHash, method, params...)
if err != nil { if err != nil {
return fmt.Errorf("deploy contract: %w", err) if method != updateMethodName || !strings.Contains(err.Error(), common.ErrAlreadyUpdated) {
return fmt.Errorf("deploy contract: %w", err)
}
c.Command.Printf("%s contract is already updated.\n", ctrName)
continue
} }
w.WriteBytes(res.Script) w.WriteBytes(res.Script)
if method == deployMethodName { if method == deployMethodName {
// same actions are done in initializeContext.setNNS, can be unified // same actions are done in initializeContext.setNNS, can be unified
domain := ctrName + ".neofs" domain := ctrName + ".frostfs"
script, ok, err := c.nnsRegisterDomainScript(nnsHash, cs.Hash, domain) script, ok, err := c.nnsRegisterDomainScript(nnsHash, cs.Hash, domain)
if err != nil { if err != nil {
return err return err
@ -275,6 +287,8 @@ func (c *initializeContext) updateContracts() error {
c.Command.Printf("NNS: Set %s -> %s\n", morphClient.NNSGroupKeyName, hex.EncodeToString(groupKey.Bytes())) c.Command.Printf("NNS: Set %s -> %s\n", morphClient.NNSGroupKeyName, hex.EncodeToString(groupKey.Bytes()))
emit.Opcodes(w.BinWriter, opcode.LDSFLD0) emit.Opcodes(w.BinWriter, opcode.LDSFLD0)
emit.Int(w.BinWriter, 1)
emit.Opcodes(w.BinWriter, opcode.PACK)
emit.AppCallNoArgs(w.BinWriter, nnsHash, "setPrice", callflag.All) emit.AppCallNoArgs(w.BinWriter, nnsHash, "setPrice", callflag.All)
if err := c.sendCommitteeTx(w.Bytes(), false); err != nil { if err := c.sendCommitteeTx(w.Bytes(), false); err != nil {
@ -284,10 +298,9 @@ func (c *initializeContext) updateContracts() error {
} }
func (c *initializeContext) deployContracts() error { func (c *initializeContext) deployContracts() error {
mgmtHash := c.nativeHash(nativenames.Management)
alphaCs := c.getContract(alphabetContract) alphaCs := c.getContract(alphabetContract)
var keysParam []interface{} var keysParam []any
baseGroups := alphaCs.Manifest.Groups baseGroups := alphaCs.Manifest.Groups
@ -313,7 +326,7 @@ func (c *initializeContext) deployContracts() error {
return fmt.Errorf("could not create actor: %w", err) return fmt.Errorf("could not create actor: %w", err)
} }
txHash, vub, err := act.SendCall(mgmtHash, deployMethodName, params...) txHash, vub, err := act.SendCall(management.Hash, deployMethodName, params...)
if err != nil { if err != nil {
return fmt.Errorf("can't deploy alphabet #%d contract: %w", i, err) return fmt.Errorf("can't deploy alphabet #%d contract: %w", i, err)
} }
@ -336,7 +349,7 @@ func (c *initializeContext) deployContracts() error {
} }
params := getContractDeployParameters(cs, c.getContractDeployData(ctrName, keysParam)) params := getContractDeployParameters(cs, c.getContractDeployData(ctrName, keysParam))
res, err := c.CommitteeAct.MakeCall(mgmtHash, deployMethodName, params...) res, err := c.CommitteeAct.MakeCall(management.Hash, deployMethodName, params...)
if err != nil { if err != nil {
return fmt.Errorf("can't deploy %s contract: %w", ctrName, err) return fmt.Errorf("can't deploy %s contract: %w", ctrName, err)
} }
@ -498,22 +511,22 @@ func readContractsFromArchive(file io.Reader, names []string) (map[string]*contr
return m, nil return m, nil
} }
func getContractDeployParameters(cs *contractState, deployData []interface{}) []interface{} { func getContractDeployParameters(cs *contractState, deployData []any) []any {
return []interface{}{cs.RawNEF, cs.RawManifest, deployData} return []any{cs.RawNEF, cs.RawManifest, deployData}
} }
func (c *initializeContext) getContractDeployData(ctrName string, keysParam []interface{}) []interface{} { func (c *initializeContext) getContractDeployData(ctrName string, keysParam []any) []any {
items := make([]interface{}, 1, 6) items := make([]any, 1, 6)
items[0] = false // notaryDisabled is false items[0] = false // notaryDisabled is false
switch ctrName { switch ctrName {
case neofsContract: case frostfsContract:
items = append(items, items = append(items,
c.Contracts[processingContract].Hash, c.Contracts[processingContract].Hash,
keysParam, keysParam,
smartcontract.Parameter{}) smartcontract.Parameter{})
case processingContract: case processingContract:
items = append(items, c.Contracts[neofsContract].Hash) items = append(items, c.Contracts[frostfsContract].Hash)
return items[1:] // no notary info return items[1:] // no notary info
case auditContract: case auditContract:
items = append(items, c.Contracts[netmapContract].Hash) items = append(items, c.Contracts[netmapContract].Hash)
@ -531,15 +544,15 @@ func (c *initializeContext) getContractDeployData(ctrName string, keysParam []in
items = append(items, items = append(items,
c.Contracts[netmapContract].Hash, c.Contracts[netmapContract].Hash,
c.Contracts[balanceContract].Hash, c.Contracts[balanceContract].Hash,
c.Contracts[neofsIDContract].Hash, c.Contracts[frostfsIDContract].Hash,
nnsCs.Hash, nnsCs.Hash,
"container") "container")
case neofsIDContract: case frostfsIDContract:
items = append(items, items = append(items,
c.Contracts[netmapContract].Hash, c.Contracts[netmapContract].Hash,
c.Contracts[containerContract].Hash) c.Contracts[containerContract].Hash)
case netmapContract: case netmapContract:
configParam := []interface{}{ configParam := []any{
netmapEpochKey, viper.GetInt64(epochDurationInitFlag), netmapEpochKey, viper.GetInt64(epochDurationInitFlag),
netmapMaxObjectSizeKey, viper.GetInt64(maxObjectSizeInitFlag), netmapMaxObjectSizeKey, viper.GetInt64(maxObjectSizeInitFlag),
netmapAuditFeeKey, viper.GetInt64(auditFeeInitFlag), netmapAuditFeeKey, viper.GetInt64(auditFeeInitFlag),
@ -568,8 +581,8 @@ func (c *initializeContext) getContractDeployData(ctrName string, keysParam []in
return items return items
} }
func (c *initializeContext) getAlphabetDeployItems(i, n int) []interface{} { func (c *initializeContext) getAlphabetDeployItems(i, n int) []any {
items := make([]interface{}, 6) items := make([]any, 6)
items[0] = false items[0] = false
items[1] = c.Contracts[netmapContract].Hash items[1] = c.Contracts[netmapContract].Hash
items[2] = c.Contracts[proxyContract].Hash items[2] = c.Contracts[proxyContract].Hash

View file

@ -7,6 +7,8 @@ import (
"strconv" "strconv"
"time" "time"
"git.frostfs.info/TrueCloudLab/frostfs-contract/nns"
morphClient "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/morph/client"
"github.com/nspcc-dev/neo-go/pkg/core/state" "github.com/nspcc-dev/neo-go/pkg/core/state"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/nspcc-dev/neo-go/pkg/encoding/address" "github.com/nspcc-dev/neo-go/pkg/encoding/address"
@ -20,8 +22,6 @@ import (
"github.com/nspcc-dev/neo-go/pkg/vm/opcode" "github.com/nspcc-dev/neo-go/pkg/vm/opcode"
"github.com/nspcc-dev/neo-go/pkg/vm/stackitem" "github.com/nspcc-dev/neo-go/pkg/vm/stackitem"
"github.com/nspcc-dev/neo-go/pkg/vm/vmstate" "github.com/nspcc-dev/neo-go/pkg/vm/vmstate"
"github.com/nspcc-dev/neofs-contract/nns"
morphClient "github.com/nspcc-dev/neofs-node/pkg/morph/client"
) )
const defaultExpirationTime = 10 * 365 * 24 * time.Hour / time.Second const defaultExpirationTime = 10 * 365 * 24 * time.Hour / time.Second
@ -32,13 +32,13 @@ func (c *initializeContext) setNNS() error {
return err return err
} }
ok, err := c.nnsRootRegistered(nnsCs.Hash, "neofs") ok, err := c.nnsRootRegistered(nnsCs.Hash, "frostfs")
if err != nil { if err != nil {
return err return err
} else if !ok { } else if !ok {
bw := io.NewBufBinWriter() bw := io.NewBufBinWriter()
emit.AppCall(bw.BinWriter, nnsCs.Hash, "register", callflag.All, emit.AppCall(bw.BinWriter, nnsCs.Hash, "register", callflag.All,
"neofs", c.CommitteeAcc.Contract.ScriptHash(), "frostfs", c.CommitteeAcc.Contract.ScriptHash(),
"ops@nspcc.ru", int64(3600), int64(600), int64(defaultExpirationTime), int64(3600)) "ops@nspcc.ru", int64(3600), int64(600), int64(defaultExpirationTime), int64(3600))
emit.Opcodes(bw.BinWriter, opcode.ASSERT) emit.Opcodes(bw.BinWriter, opcode.ASSERT)
if err := c.sendCommitteeTx(bw.Bytes(), true); err != nil { if err := c.sendCommitteeTx(bw.Bytes(), true); err != nil {
@ -63,7 +63,7 @@ func (c *initializeContext) setNNS() error {
for _, ctrName := range contractList { for _, ctrName := range contractList {
cs := c.getContract(ctrName) cs := c.getContract(ctrName)
domain := ctrName + ".neofs" domain := ctrName + ".frostfs"
if err := c.nnsRegisterDomain(nnsCs.Hash, cs.Hash, domain); err != nil { if err := c.nnsRegisterDomain(nnsCs.Hash, cs.Hash, domain); err != nil {
return err return err
} }
@ -82,13 +82,13 @@ func (c *initializeContext) setNNS() error {
func (c *initializeContext) updateNNSGroup(nnsHash util.Uint160, pub *keys.PublicKey) error { func (c *initializeContext) updateNNSGroup(nnsHash util.Uint160, pub *keys.PublicKey) error {
bw := io.NewBufBinWriter() bw := io.NewBufBinWriter()
needUpdate, needRegister, err := c.emitUpdateNNSGroupScript(bw, nnsHash, pub) keyAlreadyAdded, domainRegCodeEmitted, err := c.emitUpdateNNSGroupScript(bw, nnsHash, pub)
if !needUpdate || err != nil { if keyAlreadyAdded || err != nil {
return err return err
} }
script := bw.Bytes() script := bw.Bytes()
if needRegister { if domainRegCodeEmitted {
w := io.NewBufBinWriter() w := io.NewBufBinWriter()
emit.Instruction(w.BinWriter, opcode.INITSSLOT, []byte{1}) emit.Instruction(w.BinWriter, opcode.INITSSLOT, []byte{1})
wrapRegisterScriptWithPrice(w, nnsHash, script) wrapRegisterScriptWithPrice(w, nnsHash, script)
@ -125,15 +125,15 @@ func (c *initializeContext) emitUpdateNNSGroupScript(bw *io.BufBinWriter, nnsHas
emit.Opcodes(bw.BinWriter, opcode.ASSERT) emit.Opcodes(bw.BinWriter, opcode.ASSERT)
} }
emit.AppCall(bw.BinWriter, nnsHash, "deleteRecords", callflag.All, "group.neofs", int64(nns.TXT)) emit.AppCall(bw.BinWriter, nnsHash, "deleteRecords", callflag.All, "group.frostfs", int64(nns.TXT))
emit.AppCall(bw.BinWriter, nnsHash, "addRecord", callflag.All, emit.AppCall(bw.BinWriter, nnsHash, "addRecord", callflag.All,
"group.neofs", int64(nns.TXT), hex.EncodeToString(pub.Bytes())) "group.frostfs", int64(nns.TXT), hex.EncodeToString(pub.Bytes()))
return false, isAvail, nil return false, isAvail, nil
} }
func getAlphabetNNSDomain(i int) string { func getAlphabetNNSDomain(i int) string {
return alphabetContract + strconv.FormatUint(uint64(i), 10) + ".neofs" return alphabetContract + strconv.FormatUint(uint64(i), 10) + ".frostfs"
} }
// wrapRegisterScriptWithPrice wraps a given script with `getPrice`/`setPrice` calls for NNS. // wrapRegisterScriptWithPrice wraps a given script with `getPrice`/`setPrice` calls for NNS.
@ -228,20 +228,27 @@ func nnsResolve(inv *invoker.Invoker, nnsHash util.Uint160, domain string) (stac
} }
func nnsResolveKey(inv *invoker.Invoker, nnsHash util.Uint160, domain string) (*keys.PublicKey, error) { func nnsResolveKey(inv *invoker.Invoker, nnsHash util.Uint160, domain string) (*keys.PublicKey, error) {
item, err := nnsResolve(inv, nnsHash, domain) res, err := nnsResolve(inv, nnsHash, domain)
if err != nil { if err != nil {
return nil, err return nil, err
} }
v, ok := item.Value().(stackitem.Null) if _, ok := res.Value().(stackitem.Null); ok {
if ok {
return nil, errors.New("NNS record is missing") return nil, errors.New("NNS record is missing")
} }
bs, err := v.TryBytes() arr, ok := res.Value().([]stackitem.Item)
if err != nil { if !ok {
return nil, errors.New("malformed response") return nil, errors.New("API of the NNS contract method `resolve` has changed")
} }
for i := range arr {
var bs []byte
bs, err = arr[i].TryBytes()
if err != nil {
continue
}
return keys.NewPublicKeyFromString(string(bs)) return keys.NewPublicKeyFromString(string(bs))
}
return nil, errors.New("no valid keys are found")
} }
// parseNNSResolveResult parses the result of resolving NNS record. // parseNNSResolveResult parses the result of resolving NNS record.
@ -261,6 +268,8 @@ func parseNNSResolveResult(res stackitem.Item) (util.Uint160, error) {
continue continue
} }
// We support several formats for hash encoding, this logic should be maintained in sync
// with nnsResolve from pkg/morph/client/nns.go
h, err := util.Uint160DecodeStringLE(string(bs)) h, err := util.Uint160DecodeStringLE(string(bs))
if err == nil { if err == nil {
return h, nil return h, nil
@ -279,7 +288,7 @@ func nnsIsAvailable(c Client, nnsHash util.Uint160, name string) (bool, error) {
case *rpcclient.Client: case *rpcclient.Client:
return ct.NNSIsAvailable(nnsHash, name) return ct.NNSIsAvailable(nnsHash, name)
default: default:
b, err := unwrap.Bool(invokeFunction(c, nnsHash, "isAvailable", []interface{}{name}, nil)) b, err := unwrap.Bool(invokeFunction(c, nnsHash, "isAvailable", []any{name}, nil))
if err != nil { if err != nil {
return false, fmt.Errorf("`isAvailable`: invalid response: %w", err) return false, fmt.Errorf("`isAvailable`: invalid response: %w", err)
} }

View file

@ -5,11 +5,11 @@ import (
"fmt" "fmt"
"github.com/nspcc-dev/neo-go/pkg/core/native" "github.com/nspcc-dev/neo-go/pkg/core/native"
"github.com/nspcc-dev/neo-go/pkg/core/native/nativenames"
"github.com/nspcc-dev/neo-go/pkg/core/state" "github.com/nspcc-dev/neo-go/pkg/core/state"
"github.com/nspcc-dev/neo-go/pkg/core/transaction" "github.com/nspcc-dev/neo-go/pkg/core/transaction"
"github.com/nspcc-dev/neo-go/pkg/io" "github.com/nspcc-dev/neo-go/pkg/io"
"github.com/nspcc-dev/neo-go/pkg/rpcclient" "github.com/nspcc-dev/neo-go/pkg/rpcclient"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/neo"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/unwrap" "github.com/nspcc-dev/neo-go/pkg/rpcclient/unwrap"
"github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag" "github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag"
"github.com/nspcc-dev/neo-go/pkg/util" "github.com/nspcc-dev/neo-go/pkg/util"
@ -21,7 +21,7 @@ import (
const initialAlphabetNEOAmount = native.NEOTotalSupply const initialAlphabetNEOAmount = native.NEOTotalSupply
func (c *initializeContext) registerCandidates() error { func (c *initializeContext) registerCandidates() error {
neoHash := c.nativeHash(nativenames.Neo) neoHash := neo.Hash
cc, err := unwrap.Array(c.ReadOnlyInvoker.Call(neoHash, "getCandidates")) cc, err := unwrap.Array(c.ReadOnlyInvoker.Call(neoHash, "getCandidates"))
if err != nil { if err != nil {
@ -83,7 +83,7 @@ func (c *initializeContext) registerCandidates() error {
} }
func (c *initializeContext) transferNEOToAlphabetContracts() error { func (c *initializeContext) transferNEOToAlphabetContracts() error {
neoHash := c.nativeHash(nativenames.Neo) neoHash := neo.Hash
ok, err := c.transferNEOFinished(neoHash) ok, err := c.transferNEOFinished(neoHash)
if ok || err != nil { if ok || err != nil {
@ -120,7 +120,7 @@ func (c *initializeContext) getCandidateRegisterPrice() (int64, error) {
case *rpcclient.Client: case *rpcclient.Client:
return ct.GetCandidateRegisterPrice() return ct.GetCandidateRegisterPrice()
default: default:
neoHash := c.nativeHash(nativenames.Neo) neoHash := neo.Hash
res, err := invokeFunction(c.Client, neoHash, "getRegisterPrice", nil, nil) res, err := invokeFunction(c.Client, neoHash, "getRegisterPrice", nil, nil)
if err != nil { if err != nil {
return 0, err return 0, err

View file

@ -1,9 +1,9 @@
package morph package morph
import ( import (
"github.com/nspcc-dev/neo-go/pkg/core/native/nativenames"
"github.com/nspcc-dev/neo-go/pkg/core/native/noderoles" "github.com/nspcc-dev/neo-go/pkg/core/native/noderoles"
"github.com/nspcc-dev/neo-go/pkg/io" "github.com/nspcc-dev/neo-go/pkg/io"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/rolemgmt"
"github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag" "github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag"
"github.com/nspcc-dev/neo-go/pkg/vm/emit" "github.com/nspcc-dev/neo-go/pkg/vm/emit"
) )
@ -16,17 +16,15 @@ func (c *initializeContext) setNotaryAndAlphabetNodes() error {
return err return err
} }
designateHash := c.nativeHash(nativenames.Designation) var pubs []any
var pubs []interface{}
for _, acc := range c.Accounts { for _, acc := range c.Accounts {
pubs = append(pubs, acc.PrivateKey().PublicKey().Bytes()) pubs = append(pubs, acc.PrivateKey().PublicKey().Bytes())
} }
w := io.NewBufBinWriter() w := io.NewBufBinWriter()
emit.AppCall(w.BinWriter, designateHash, "designateAsRole", emit.AppCall(w.BinWriter, rolemgmt.Hash, "designateAsRole",
callflag.States|callflag.AllowNotify, int64(noderoles.P2PNotary), pubs) callflag.States|callflag.AllowNotify, int64(noderoles.P2PNotary), pubs)
emit.AppCall(w.BinWriter, designateHash, "designateAsRole", emit.AppCall(w.BinWriter, rolemgmt.Hash, "designateAsRole",
callflag.States|callflag.AllowNotify, int64(noderoles.NeoFSAlphabet), pubs) callflag.States|callflag.AllowNotify, int64(noderoles.NeoFSAlphabet), pubs)
if err := c.sendCommitteeTx(w.Bytes(), false); err != nil { if err := c.sendCommitteeTx(w.Bytes(), false); err != nil {
@ -42,7 +40,6 @@ func (c *initializeContext) setRolesFinished() (bool, error) {
return false, err return false, err
} }
h := c.nativeHash(nativenames.Designation) pubs, err := getDesignatedByRole(c.ReadOnlyInvoker, rolemgmt.Hash, noderoles.NeoFSAlphabet, height)
pubs, err := getDesignatedByRole(c.ReadOnlyInvoker, h, noderoles.NeoFSAlphabet, height)
return len(pubs) == len(c.Wallets), err return len(pubs) == len(c.Wallets), err
} }

View file

@ -7,26 +7,29 @@ import (
"strconv" "strconv"
"testing" "testing"
"git.frostfs.info/TrueCloudLab/frostfs-node/pkg/innerring"
"github.com/nspcc-dev/neo-go/pkg/config" "github.com/nspcc-dev/neo-go/pkg/config"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/nspcc-dev/neo-go/pkg/vm" "github.com/nspcc-dev/neo-go/pkg/vm"
"github.com/nspcc-dev/neo-go/pkg/wallet" "github.com/nspcc-dev/neo-go/pkg/wallet"
"github.com/nspcc-dev/neofs-node/pkg/innerring"
"github.com/spf13/viper" "github.com/spf13/viper"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"gopkg.in/yaml.v3" "gopkg.in/yaml.v3"
) )
const ( const (
contractsPath = "../../../../../../neofs-contract/neofs-contract-v0.16.0.tar.gz" contractsPath = "../../../../../../frostfs-contract/frostfs-contract-v0.16.0.tar.gz"
protoFileName = "proto.yml" protoFileName = "proto.yml"
) )
func TestInitialize(t *testing.T) { func TestInitialize(t *testing.T) {
// This test needs neofs-contract tarball, so it is skipped by default. // This test needs frostfs-contract tarball, so it is skipped by default.
// It is here for performing local testing after the changes. // It is here for performing local testing after the changes.
t.Skip() t.Skip()
t.Run("1 nodes", func(t *testing.T) {
testInitialize(t, 1)
})
t.Run("4 nodes", func(t *testing.T) { t.Run("4 nodes", func(t *testing.T) {
testInitialize(t, 4) testInitialize(t, 4)
}) })
@ -96,6 +99,7 @@ func generateTestData(t *testing.T, dir string, size int) {
} }
cfg := config.Config{} cfg := config.Config{}
cfg.ProtocolConfiguration.Magic = 12345
cfg.ProtocolConfiguration.ValidatorsCount = size cfg.ProtocolConfiguration.ValidatorsCount = size
cfg.ProtocolConfiguration.SecondsPerBlock = 1 cfg.ProtocolConfiguration.SecondsPerBlock = 1
cfg.ProtocolConfiguration.StandbyCommittee = pubs // sorted by glagolic letters cfg.ProtocolConfiguration.StandbyCommittee = pubs // sorted by glagolic letters

View file

@ -4,10 +4,11 @@ import (
"fmt" "fmt"
"github.com/nspcc-dev/neo-go/pkg/core/native" "github.com/nspcc-dev/neo-go/pkg/core/native"
"github.com/nspcc-dev/neo-go/pkg/core/native/nativenames"
"github.com/nspcc-dev/neo-go/pkg/core/transaction" "github.com/nspcc-dev/neo-go/pkg/core/transaction"
"github.com/nspcc-dev/neo-go/pkg/io" "github.com/nspcc-dev/neo-go/pkg/io"
"github.com/nspcc-dev/neo-go/pkg/rpcclient" "github.com/nspcc-dev/neo-go/pkg/rpcclient"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/gas"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/neo"
"github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag" "github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag"
scContext "github.com/nspcc-dev/neo-go/pkg/smartcontract/context" scContext "github.com/nspcc-dev/neo-go/pkg/smartcontract/context"
"github.com/nspcc-dev/neo-go/pkg/vm/emit" "github.com/nspcc-dev/neo-go/pkg/vm/emit"
@ -32,15 +33,12 @@ func (c *initializeContext) transferFunds() error {
return err return err
} }
gasHash := c.nativeHash(nativenames.Gas)
neoHash := c.nativeHash(nativenames.Neo)
var transfers []rpcclient.TransferTarget var transfers []rpcclient.TransferTarget
for _, acc := range c.Accounts { for _, acc := range c.Accounts {
to := acc.Contract.ScriptHash() to := acc.Contract.ScriptHash()
transfers = append(transfers, transfers = append(transfers,
rpcclient.TransferTarget{ rpcclient.TransferTarget{
Token: gasHash, Token: gas.Hash,
Address: to, Address: to,
Amount: initialAlphabetGASAmount, Amount: initialAlphabetGASAmount,
}, },
@ -50,12 +48,12 @@ func (c *initializeContext) transferFunds() error {
// It is convenient to have all funds at the committee account. // It is convenient to have all funds at the committee account.
transfers = append(transfers, transfers = append(transfers,
rpcclient.TransferTarget{ rpcclient.TransferTarget{
Token: gasHash, Token: gas.Hash,
Address: c.CommitteeAcc.Contract.ScriptHash(), Address: c.CommitteeAcc.Contract.ScriptHash(),
Amount: (gasInitialTotalSupply - initialAlphabetGASAmount*int64(len(c.Wallets))) / 2, Amount: (gasInitialTotalSupply - initialAlphabetGASAmount*int64(len(c.Wallets))) / 2,
}, },
rpcclient.TransferTarget{ rpcclient.TransferTarget{
Token: neoHash, Token: neo.Hash,
Address: c.CommitteeAcc.Contract.ScriptHash(), Address: c.CommitteeAcc.Contract.ScriptHash(),
Amount: native.NEOTotalSupply, Amount: native.NEOTotalSupply,
}, },
@ -80,10 +78,9 @@ func (c *initializeContext) transferFunds() error {
} }
func (c *initializeContext) transferFundsFinished() (bool, error) { func (c *initializeContext) transferFundsFinished() (bool, error) {
gasHash := c.nativeHash(nativenames.Gas)
acc := c.Accounts[0] acc := c.Accounts[0]
res, err := c.Client.NEP17BalanceOf(gasHash, acc.Contract.ScriptHash()) res, err := c.Client.NEP17BalanceOf(gas.Hash, acc.Contract.ScriptHash())
return res > initialAlphabetGASAmount/2, err return res > initialAlphabetGASAmount/2, err
} }
@ -147,16 +144,15 @@ func (c *initializeContext) multiSign(tx *transaction.Transaction, accType strin
} }
func (c *initializeContext) transferGASToProxy() error { func (c *initializeContext) transferGASToProxy() error {
gasHash := c.nativeHash(nativenames.Gas)
proxyCs := c.getContract(proxyContract) proxyCs := c.getContract(proxyContract)
bal, err := c.Client.NEP17BalanceOf(gasHash, proxyCs.Hash) bal, err := c.Client.NEP17BalanceOf(gas.Hash, proxyCs.Hash)
if err != nil || bal > 0 { if err != nil || bal > 0 {
return err return err
} }
tx, err := createNEP17MultiTransferTx(c.Client, c.CommitteeAcc, 0, []rpcclient.TransferTarget{{ tx, err := createNEP17MultiTransferTx(c.Client, c.CommitteeAcc, 0, []rpcclient.TransferTarget{{
Token: gasHash, Token: gas.Hash,
Address: proxyCs.Hash, Address: proxyCs.Hash,
Amount: initialProxyGASAmount, Amount: initialProxyGASAmount,
}}, nil) }}, nil)

View file

@ -18,7 +18,7 @@ func StringifySubnetClientGroupID(id *SubnetClientGroupID) string {
return string(text) return string(text)
} }
// MarshalText encodes SubnetClientGroupID into text format according to NeoFS API V2 protocol: // MarshalText encodes SubnetClientGroupID into text format according to FrostFS API V2 protocol:
// value in base-10 integer string format. // value in base-10 integer string format.
// //
// It implements encoding.TextMarshaler. // It implements encoding.TextMarshaler.
@ -28,7 +28,7 @@ func (x *SubnetClientGroupID) MarshalText() ([]byte, error) {
return []byte(strconv.FormatUint(uint64(num), 10)), nil return []byte(strconv.FormatUint(uint64(num), 10)), nil
} }
// UnmarshalText decodes the SubnetID from the text according to NeoFS API V2 protocol: // UnmarshalText decodes the SubnetID from the text according to FrostFS API V2 protocol:
// should be base-10 integer string format with bitsize = 32. // should be base-10 integer string format with bitsize = 32.
// //
// Returns strconv.ErrRange if integer overflows uint32. // Returns strconv.ErrRange if integer overflows uint32.
@ -47,13 +47,13 @@ func (x *SubnetClientGroupID) UnmarshalText(txt []byte) error {
return nil return nil
} }
// Marshal encodes the SubnetClientGroupID into a binary format of NeoFS API V2 protocol // Marshal encodes the SubnetClientGroupID into a binary format of FrostFS API V2 protocol
// (Protocol Buffers with direct field order). // (Protocol Buffers with direct field order).
func (x *SubnetClientGroupID) Marshal() ([]byte, error) { func (x *SubnetClientGroupID) Marshal() ([]byte, error) {
return proto.Marshal(x) return proto.Marshal(x)
} }
// Unmarshal decodes the SubnetClientGroupID from NeoFS API V2 binary format (see Marshal). Must not be called on nil. // Unmarshal decodes the SubnetClientGroupID from FrostFS API V2 binary format (see Marshal). Must not be called on nil.
func (x *SubnetClientGroupID) Unmarshal(data []byte) error { func (x *SubnetClientGroupID) Unmarshal(data []byte) error {
return proto.Unmarshal(data, x) return proto.Unmarshal(data, x)
} }

Binary file not shown.

View file

@ -2,9 +2,9 @@ syntax = "proto3";
package neo.fs.v2.refs; package neo.fs.v2.refs;
option go_package = "github.com/nspcc-dev/neofs-node/cmd/neofs-adm/internal/modules/morph/internal"; option go_package = "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/modules/morph/internal";
// Client group identifier in the NeoFS subnet. // Client group identifier in the FrostFS subnet.
// //
// String representation of a value is base-10 integer. // String representation of a value is base-10 integer.
// //

View file

@ -1,12 +1,12 @@
package morph package morph
import ( import (
"bytes"
"crypto/elliptic" "crypto/elliptic"
"errors" "errors"
"fmt" "fmt"
"os" "os"
"sort" "sort"
"time"
"github.com/google/uuid" "github.com/google/uuid"
"github.com/nspcc-dev/neo-go/pkg/config" "github.com/nspcc-dev/neo-go/pkg/config"
@ -21,6 +21,7 @@ import (
"github.com/nspcc-dev/neo-go/pkg/core/transaction" "github.com/nspcc-dev/neo-go/pkg/core/transaction"
"github.com/nspcc-dev/neo-go/pkg/crypto/hash" "github.com/nspcc-dev/neo-go/pkg/crypto/hash"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/nspcc-dev/neo-go/pkg/encoding/address"
"github.com/nspcc-dev/neo-go/pkg/encoding/fixedn" "github.com/nspcc-dev/neo-go/pkg/encoding/fixedn"
"github.com/nspcc-dev/neo-go/pkg/io" "github.com/nspcc-dev/neo-go/pkg/io"
"github.com/nspcc-dev/neo-go/pkg/neorpc/result" "github.com/nspcc-dev/neo-go/pkg/neorpc/result"
@ -56,7 +57,7 @@ func newLocalClient(cmd *cobra.Command, v *viper.Viper, wallets []*wallet.Wallet
return nil, err return nil, err
} }
bc, err := core.NewBlockchain(storage.NewMemoryStore(), cfg.ProtocolConfiguration, zap.NewNop()) bc, err := core.NewBlockchain(storage.NewMemoryStore(), cfg.Blockchain(), zap.NewNop())
if err != nil { if err != nil {
return nil, err return nil, err
} }
@ -81,9 +82,7 @@ func newLocalClient(cmd *cobra.Command, v *viper.Viper, wallets []*wallet.Wallet
return indexMap[string(pi)] < indexMap[string(pj)] return indexMap[string(pi)] < indexMap[string(pj)]
}) })
sort.Slice(accounts[:cfg.ProtocolConfiguration.ValidatorsCount], func(i, j int) bool { sort.Slice(accounts[:cfg.ProtocolConfiguration.ValidatorsCount], func(i, j int) bool {
pi := accounts[i].PrivateKey().PublicKey().Bytes() return accounts[i].PublicKey().Cmp(accounts[j].PublicKey()) == -1
pj := accounts[j].PrivateKey().PublicKey().Bytes()
return bytes.Compare(pi, pj) == -1
}) })
go bc.Run() go bc.Run()
@ -191,7 +190,7 @@ func (l *localClient) GetCommittee() (keys.PublicKeys, error) {
func (l *localClient) InvokeFunction(h util.Uint160, method string, sPrm []smartcontract.Parameter, ss []transaction.Signer) (*result.Invoke, error) { func (l *localClient) InvokeFunction(h util.Uint160, method string, sPrm []smartcontract.Parameter, ss []transaction.Signer) (*result.Invoke, error) {
var err error var err error
pp := make([]interface{}, len(sPrm)) pp := make([]any, len(sPrm))
for i, p := range sPrm { for i, p := range sPrm {
pp[i], err = smartcontract.ExpandParameterToEmitable(p) pp[i], err = smartcontract.ExpandParameterToEmitable(p)
if err != nil { if err != nil {
@ -229,7 +228,24 @@ func (l *localClient) TraverseIterator(_, _ uuid.UUID, _ int) ([]stackitem.Item,
// GetVersion return default version. // GetVersion return default version.
func (l *localClient) GetVersion() (*result.Version, error) { func (l *localClient) GetVersion() (*result.Version, error) {
return &result.Version{}, nil c := l.bc.GetConfig()
return &result.Version{
Protocol: result.Protocol{
AddressVersion: address.NEO3Prefix,
Network: c.Magic,
MillisecondsPerBlock: int(c.TimePerBlock / time.Millisecond),
MaxTraceableBlocks: c.MaxTraceableBlocks,
MaxValidUntilBlockIncrement: c.MaxValidUntilBlockIncrement,
MaxTransactionsPerBlock: c.MaxTransactionsPerBlock,
MemoryPoolMaxTransactions: c.MemPoolSize,
ValidatorsCount: byte(c.ValidatorsCount),
InitialGasDistribution: c.InitialGASSupply,
CommitteeHistory: c.CommitteeHistory,
P2PSigExtensions: c.P2PSigExtensions,
StateRootInHeader: c.StateRootInHeader,
ValidatorsHistory: c.ValidatorsHistory,
},
}, nil
} }
func (l *localClient) InvokeContractVerify(contract util.Uint160, params []smartcontract.Parameter, signers []transaction.Signer, witnesses ...transaction.Witness) (*result.Invoke, error) { func (l *localClient) InvokeContractVerify(contract util.Uint160, params []smartcontract.Parameter, signers []transaction.Signer, witnesses ...transaction.Witness) (*result.Invoke, error) {
@ -330,7 +346,7 @@ func getSigners(sender *wallet.Account, cosigners []rpcclient.SignerAccount) ([]
} }
func (l *localClient) NEP17BalanceOf(h util.Uint160, acc util.Uint160) (int64, error) { func (l *localClient) NEP17BalanceOf(h util.Uint160, acc util.Uint160) (int64, error) {
res, err := invokeFunction(l, h, "balanceOf", []interface{}{acc}, nil) res, err := invokeFunction(l, h, "balanceOf", []any{acc}, nil)
if err != nil { if err != nil {
return 0, err return 0, err
} }
@ -435,7 +451,7 @@ func (l *localClient) putTransactions() error {
return l.bc.AddBlock(b) return l.bc.AddBlock(b)
} }
func invokeFunction(c Client, h util.Uint160, method string, parameters []interface{}, signers []transaction.Signer) (*result.Invoke, error) { func invokeFunction(c Client, h util.Uint160, method string, parameters []any, signers []transaction.Signer) (*result.Invoke, error) {
w := io.NewBufBinWriter() w := io.NewBufBinWriter()
emit.Array(w.BinWriter, parameters...) emit.Array(w.BinWriter, parameters...)
emit.AppCallNoArgs(w.BinWriter, h, method, callflag.All) emit.AppCallNoArgs(w.BinWriter, h, method, callflag.All)

View file

@ -0,0 +1,29 @@
package morph
import (
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/commonflags"
commonCmd "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/internal/common"
"git.frostfs.info/TrueCloudLab/frostfs-node/pkg/morph/client/netmap"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/invoker"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
func listNetmapCandidatesNodes(cmd *cobra.Command, _ []string) {
c, err := getN3Client(viper.GetViper())
commonCmd.ExitOnErr(cmd, "can't create N3 client: %w", err)
inv := invoker.New(c, nil)
cs, err := c.GetContractStateByID(1)
commonCmd.ExitOnErr(cmd, "can't get NNS contract info: %w", err)
nmHash, err := nnsResolveHash(inv, cs.Hash, netmapContract+".frostfs")
commonCmd.ExitOnErr(cmd, "can't get netmap contract hash: %w", err)
res, err := inv.Call(nmHash, "netmapCandidates")
commonCmd.ExitOnErr(cmd, "can't fetch list of network config keys from the netmap contract", err)
nm, err := netmap.DecodeNetMap(res.Stack)
commonCmd.ExitOnErr(cmd, "unable to decode netmap: %w", err)
commonCmd.PrettyPrintNetMap(cmd, *nm, !viper.GetBool(commonflags.Verbose))
}

View file

@ -6,12 +6,13 @@ import (
"strconv" "strconv"
"github.com/nspcc-dev/neo-go/cli/input" "github.com/nspcc-dev/neo-go/cli/input"
"github.com/nspcc-dev/neo-go/pkg/core/native/nativenames"
"github.com/nspcc-dev/neo-go/pkg/core/transaction" "github.com/nspcc-dev/neo-go/pkg/core/transaction"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/nspcc-dev/neo-go/pkg/encoding/address" "github.com/nspcc-dev/neo-go/pkg/encoding/address"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/actor" "github.com/nspcc-dev/neo-go/pkg/rpcclient/actor"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/gas"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/nep17" "github.com/nspcc-dev/neo-go/pkg/rpcclient/nep17"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/notary"
"github.com/nspcc-dev/neo-go/pkg/wallet" "github.com/nspcc-dev/neo-go/pkg/wallet"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
@ -21,6 +22,7 @@ import (
// https://github.com/nspcc-dev/neo-go/blob/master/pkg/core/native/notary.go#L48 // https://github.com/nspcc-dev/neo-go/blob/master/pkg/core/native/notary.go#L48
const defaultNotaryDepositLifetime = 5760 const defaultNotaryDepositLifetime = 5760
// nolint: funlen
func depositNotary(cmd *cobra.Command, _ []string) error { func depositNotary(cmd *cobra.Command, _ []string) error {
p, err := cmd.Flags().GetString(storageWalletFlag) p, err := cmd.Flags().GetString(storageWalletFlag)
if err != nil { if err != nil {
@ -84,18 +86,8 @@ func depositNotary(cmd *cobra.Command, _ []string) error {
return err return err
} }
nhs, err := getNativeHashes(c) if err := checkNotaryEnabled(c); err != nil {
if err != nil { return err
return fmt.Errorf("can't get native contract hashes: %w", err)
}
gasHash, ok := nhs[nativenames.Gas]
if !ok {
return fmt.Errorf("can't retrieve %s contract hash", nativenames.Gas)
}
notaryHash, ok := nhs[nativenames.Notary]
if !ok {
return fmt.Errorf("can't retrieve %s contract hash", nativenames.Notary)
} }
height, err := c.GetBlockCount() height, err := c.GetBlockCount()
@ -114,13 +106,13 @@ func depositNotary(cmd *cobra.Command, _ []string) error {
return fmt.Errorf("could not create actor: %w", err) return fmt.Errorf("could not create actor: %w", err)
} }
gas := nep17.New(act, gasHash) gasActor := nep17.New(act, gas.Hash)
txHash, vub, err := gas.Transfer( txHash, vub, err := gasActor.Transfer(
accHash, accHash,
notaryHash, notary.Hash,
big.NewInt(int64(gasAmount)), big.NewInt(int64(gasAmount)),
[]interface{}{nil, int64(height) + till}, []any{nil, int64(height) + till},
) )
if err != nil { if err != nil {
return fmt.Errorf("could not send tx: %w", err) return fmt.Errorf("could not send tx: %w", err)

View file

@ -5,8 +5,8 @@ import (
"strconv" "strconv"
"strings" "strings"
"github.com/nspcc-dev/neo-go/pkg/core/native/nativenames"
"github.com/nspcc-dev/neo-go/pkg/io" "github.com/nspcc-dev/neo-go/pkg/io"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/policy"
"github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag" "github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag"
"github.com/nspcc-dev/neo-go/pkg/vm/emit" "github.com/nspcc-dev/neo-go/pkg/vm/emit"
"github.com/spf13/cobra" "github.com/spf13/cobra"
@ -25,27 +25,25 @@ func setPolicyCmd(cmd *cobra.Command, args []string) error {
return fmt.Errorf("can't to initialize context: %w", err) return fmt.Errorf("can't to initialize context: %w", err)
} }
policyHash := wCtx.nativeHash(nativenames.Policy)
bw := io.NewBufBinWriter() bw := io.NewBufBinWriter()
for i := range args { for i := range args {
kv := strings.SplitN(args[i], "=", 2) k, v, found := strings.Cut(args[i], "=")
if len(kv) != 2 { if !found {
return fmt.Errorf("invalid parameter format, must be Parameter=Value") return fmt.Errorf("invalid parameter format, must be Parameter=Value")
} }
switch kv[0] { switch k {
case execFeeParam, storagePriceParam, setFeeParam: case execFeeParam, storagePriceParam, setFeeParam:
default: default:
return fmt.Errorf("parameter must be one of %s, %s and %s", execFeeParam, storagePriceParam, setFeeParam) return fmt.Errorf("parameter must be one of %s, %s and %s", execFeeParam, storagePriceParam, setFeeParam)
} }
value, err := strconv.ParseUint(kv[1], 10, 32) value, err := strconv.ParseUint(v, 10, 32)
if err != nil { if err != nil {
return fmt.Errorf("can't parse parameter value '%s': %w", args[1], err) return fmt.Errorf("can't parse parameter value '%s': %w", args[1], err)
} }
emit.AppCall(bw.BinWriter, policyHash, "set"+kv[0], callflag.All, int64(value)) emit.AppCall(bw.BinWriter, policy.Hash, "set"+k, callflag.All, int64(value))
} }
if err := wCtx.sendCommitteeTx(bw.Bytes(), false); err != nil { if err := wCtx.sendCommitteeTx(bw.Bytes(), false); err != nil {

View file

@ -4,11 +4,11 @@ import (
"errors" "errors"
"fmt" "fmt"
netmapcontract "git.frostfs.info/TrueCloudLab/frostfs-contract/netmap"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/nspcc-dev/neo-go/pkg/io" "github.com/nspcc-dev/neo-go/pkg/io"
"github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag" "github.com/nspcc-dev/neo-go/pkg/smartcontract/callflag"
"github.com/nspcc-dev/neo-go/pkg/vm/emit" "github.com/nspcc-dev/neo-go/pkg/vm/emit"
netmapcontract "github.com/nspcc-dev/neofs-contract/netmap"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
) )
@ -38,7 +38,7 @@ func removeNodesCmd(cmd *cobra.Command, args []string) error {
return fmt.Errorf("can't get NNS contract info: %w", err) return fmt.Errorf("can't get NNS contract info: %w", err)
} }
nmHash, err := nnsResolveHash(wCtx.ReadOnlyInvoker, cs.Hash, netmapContract+".neofs") nmHash, err := nnsResolveHash(wCtx.ReadOnlyInvoker, cs.Hash, netmapContract+".frostfs")
if err != nil { if err != nil {
return fmt.Errorf("can't get netmap contract hash: %w", err) return fmt.Errorf("can't get netmap contract hash: %w", err)
} }

View file

@ -108,7 +108,7 @@ var (
forceNewEpoch = &cobra.Command{ forceNewEpoch = &cobra.Command{
Use: "force-new-epoch", Use: "force-new-epoch",
Short: "Create new NeoFS epoch event in the side chain", Short: "Create new FrostFS epoch event in the side chain",
PreRun: func(cmd *cobra.Command, _ []string) { PreRun: func(cmd *cobra.Command, _ []string) {
_ = viper.BindPFlag(alphabetWalletsFlag, cmd.Flags().Lookup(alphabetWalletsFlag)) _ = viper.BindPFlag(alphabetWalletsFlag, cmd.Flags().Lookup(alphabetWalletsFlag))
_ = viper.BindPFlag(endpointFlag, cmd.Flags().Lookup(endpointFlag)) _ = viper.BindPFlag(endpointFlag, cmd.Flags().Lookup(endpointFlag))
@ -130,7 +130,7 @@ var (
setConfig = &cobra.Command{ setConfig = &cobra.Command{
Use: "set-config key1=val1 [key2=val2 ...]", Use: "set-config key1=val1 [key2=val2 ...]",
DisableFlagsInUseLine: true, DisableFlagsInUseLine: true,
Short: "Add/update global config value in the NeoFS network", Short: "Add/update global config value in the FrostFS network",
PreRun: func(cmd *cobra.Command, _ []string) { PreRun: func(cmd *cobra.Command, _ []string) {
_ = viper.BindPFlag(alphabetWalletsFlag, cmd.Flags().Lookup(alphabetWalletsFlag)) _ = viper.BindPFlag(alphabetWalletsFlag, cmd.Flags().Lookup(alphabetWalletsFlag))
_ = viper.BindPFlag(endpointFlag, cmd.Flags().Lookup(endpointFlag)) _ = viper.BindPFlag(endpointFlag, cmd.Flags().Lookup(endpointFlag))
@ -164,7 +164,7 @@ var (
dumpNetworkConfigCmd = &cobra.Command{ dumpNetworkConfigCmd = &cobra.Command{
Use: "dump-config", Use: "dump-config",
Short: "Dump NeoFS network config", Short: "Dump FrostFS network config",
PreRun: func(cmd *cobra.Command, _ []string) { PreRun: func(cmd *cobra.Command, _ []string) {
_ = viper.BindPFlag(endpointFlag, cmd.Flags().Lookup(endpointFlag)) _ = viper.BindPFlag(endpointFlag, cmd.Flags().Lookup(endpointFlag))
}, },
@ -182,7 +182,7 @@ var (
updateContractsCmd = &cobra.Command{ updateContractsCmd = &cobra.Command{
Use: "update-contracts", Use: "update-contracts",
Short: "Update NeoFS contracts", Short: "Update FrostFS contracts",
PreRun: func(cmd *cobra.Command, _ []string) { PreRun: func(cmd *cobra.Command, _ []string) {
_ = viper.BindPFlag(alphabetWalletsFlag, cmd.Flags().Lookup(alphabetWalletsFlag)) _ = viper.BindPFlag(alphabetWalletsFlag, cmd.Flags().Lookup(alphabetWalletsFlag))
_ = viper.BindPFlag(endpointFlag, cmd.Flags().Lookup(endpointFlag)) _ = viper.BindPFlag(endpointFlag, cmd.Flags().Lookup(endpointFlag))
@ -192,7 +192,7 @@ var (
dumpContainersCmd = &cobra.Command{ dumpContainersCmd = &cobra.Command{
Use: "dump-containers", Use: "dump-containers",
Short: "Dump NeoFS containers to file", Short: "Dump FrostFS containers to file",
PreRun: func(cmd *cobra.Command, _ []string) { PreRun: func(cmd *cobra.Command, _ []string) {
_ = viper.BindPFlag(endpointFlag, cmd.Flags().Lookup(endpointFlag)) _ = viper.BindPFlag(endpointFlag, cmd.Flags().Lookup(endpointFlag))
}, },
@ -201,7 +201,7 @@ var (
restoreContainersCmd = &cobra.Command{ restoreContainersCmd = &cobra.Command{
Use: "restore-containers", Use: "restore-containers",
Short: "Restore NeoFS containers from file", Short: "Restore FrostFS containers from file",
PreRun: func(cmd *cobra.Command, _ []string) { PreRun: func(cmd *cobra.Command, _ []string) {
_ = viper.BindPFlag(alphabetWalletsFlag, cmd.Flags().Lookup(alphabetWalletsFlag)) _ = viper.BindPFlag(alphabetWalletsFlag, cmd.Flags().Lookup(alphabetWalletsFlag))
_ = viper.BindPFlag(endpointFlag, cmd.Flags().Lookup(endpointFlag)) _ = viper.BindPFlag(endpointFlag, cmd.Flags().Lookup(endpointFlag))
@ -209,6 +209,15 @@ var (
RunE: restoreContainers, RunE: restoreContainers,
} }
listContainersCmd = &cobra.Command{
Use: "list-containers",
Short: "List FrostFS containers",
PreRun: func(cmd *cobra.Command, _ []string) {
_ = viper.BindPFlag(endpointFlag, cmd.Flags().Lookup(endpointFlag))
},
RunE: listContainers,
}
depositNotaryCmd = &cobra.Command{ depositNotaryCmd = &cobra.Command{
Use: "deposit-notary", Use: "deposit-notary",
Short: "Deposit GAS for notary service", Short: "Deposit GAS for notary service",
@ -217,8 +226,19 @@ var (
}, },
RunE: depositNotary, RunE: depositNotary,
} }
netmapCandidatesCmd = &cobra.Command{
Use: "netmap-candidates",
Short: "List netmap candidates nodes",
PreRun: func(cmd *cobra.Command, _ []string) {
_ = viper.BindPFlag(endpointFlag, cmd.Flags().Lookup(endpointFlag))
_ = viper.BindPFlag(alphabetWalletsFlag, cmd.Flags().Lookup(alphabetWalletsFlag))
},
Run: listNetmapCandidatesNodes,
}
) )
// nolint: funlen
func init() { func init() {
RootCmd.AddCommand(generateAlphabetCmd) RootCmd.AddCommand(generateAlphabetCmd)
generateAlphabetCmd.Flags().String(alphabetWalletsFlag, "", "Path to alphabet wallets dir") generateAlphabetCmd.Flags().String(alphabetWalletsFlag, "", "Path to alphabet wallets dir")
@ -227,8 +247,8 @@ func init() {
RootCmd.AddCommand(initCmd) RootCmd.AddCommand(initCmd)
initCmd.Flags().String(alphabetWalletsFlag, "", "Path to alphabet wallets dir") initCmd.Flags().String(alphabetWalletsFlag, "", "Path to alphabet wallets dir")
initCmd.Flags().StringP(endpointFlag, "r", "", "N3 RPC node endpoint") initCmd.Flags().StringP(endpointFlag, "r", "", "N3 RPC node endpoint")
initCmd.Flags().String(contractsInitFlag, "", "Path to archive with compiled NeoFS contracts (default fetched from latest github release)") initCmd.Flags().String(contractsInitFlag, "", "Path to archive with compiled FrostFS contracts (default fetched from latest github release)")
initCmd.Flags().Uint(epochDurationCLIFlag, 240, "Amount of side chain blocks in one NeoFS epoch") initCmd.Flags().Uint(epochDurationCLIFlag, 240, "Amount of side chain blocks in one FrostFS epoch")
initCmd.Flags().Uint(maxObjectSizeCLIFlag, 67108864, "Max single object size in bytes") initCmd.Flags().Uint(maxObjectSizeCLIFlag, 67108864, "Max single object size in bytes")
initCmd.Flags().Bool(homomorphicHashDisabledCLIFlag, false, "Disable object homomorphic hashing") initCmd.Flags().Bool(homomorphicHashDisabledCLIFlag, false, "Disable object homomorphic hashing")
// Defaults are taken from neo-preodolenie. // Defaults are taken from neo-preodolenie.
@ -280,7 +300,7 @@ func init() {
RootCmd.AddCommand(updateContractsCmd) RootCmd.AddCommand(updateContractsCmd)
updateContractsCmd.Flags().String(alphabetWalletsFlag, "", "Path to alphabet wallets dir") updateContractsCmd.Flags().String(alphabetWalletsFlag, "", "Path to alphabet wallets dir")
updateContractsCmd.Flags().StringP(endpointFlag, "r", "", "N3 RPC node endpoint") updateContractsCmd.Flags().StringP(endpointFlag, "r", "", "N3 RPC node endpoint")
updateContractsCmd.Flags().String(contractsInitFlag, "", "Path to archive with compiled NeoFS contracts (default fetched from latest github release)") updateContractsCmd.Flags().String(contractsInitFlag, "", "Path to archive with compiled FrostFS contracts (default fetched from latest github release)")
RootCmd.AddCommand(dumpContainersCmd) RootCmd.AddCommand(dumpContainersCmd)
dumpContainersCmd.Flags().StringP(endpointFlag, "r", "", "N3 RPC node endpoint") dumpContainersCmd.Flags().StringP(endpointFlag, "r", "", "N3 RPC node endpoint")
@ -294,6 +314,10 @@ func init() {
restoreContainersCmd.Flags().String(containerDumpFlag, "", "File to restore containers from") restoreContainersCmd.Flags().String(containerDumpFlag, "", "File to restore containers from")
restoreContainersCmd.Flags().StringSlice(containerIDsFlag, nil, "Containers to restore") restoreContainersCmd.Flags().StringSlice(containerIDsFlag, nil, "Containers to restore")
RootCmd.AddCommand(listContainersCmd)
listContainersCmd.Flags().StringP(endpointFlag, "r", "", "N3 RPC node endpoint")
listContainersCmd.Flags().String(containerContractFlag, "", "Container contract hash (for networks without NNS)")
RootCmd.AddCommand(refillGasCmd) RootCmd.AddCommand(refillGasCmd)
refillGasCmd.Flags().String(alphabetWalletsFlag, "", "Path to alphabet wallets dir") refillGasCmd.Flags().String(alphabetWalletsFlag, "", "Path to alphabet wallets dir")
refillGasCmd.Flags().StringP(endpointFlag, "r", "", "N3 RPC node endpoint") refillGasCmd.Flags().StringP(endpointFlag, "r", "", "N3 RPC node endpoint")
@ -310,4 +334,7 @@ func init() {
depositNotaryCmd.Flags().String(walletAccountFlag, "", "Wallet account address") depositNotaryCmd.Flags().String(walletAccountFlag, "", "Wallet account address")
depositNotaryCmd.Flags().String(refillGasAmountFlag, "", "Amount of GAS to deposit") depositNotaryCmd.Flags().String(refillGasAmountFlag, "", "Amount of GAS to deposit")
depositNotaryCmd.Flags().String(notaryDepositTillFlag, "", "Notary deposit duration in blocks") depositNotaryCmd.Flags().String(notaryDepositTillFlag, "", "Notary deposit duration in blocks")
RootCmd.AddCommand(netmapCandidatesCmd)
netmapCandidatesCmd.Flags().StringP(endpointFlag, "r", "", "N3 RPC node endpoint")
} }

View file

@ -5,6 +5,12 @@ import (
"errors" "errors"
"fmt" "fmt"
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/modules/morph/internal"
"git.frostfs.info/TrueCloudLab/frostfs-node/pkg/morph/client"
"git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/rand"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/subnet"
subnetid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/subnet/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
"github.com/nspcc-dev/neo-go/cli/flags" "github.com/nspcc-dev/neo-go/cli/flags"
"github.com/nspcc-dev/neo-go/cli/input" "github.com/nspcc-dev/neo-go/cli/input"
"github.com/nspcc-dev/neo-go/pkg/core/native/nativenames" "github.com/nspcc-dev/neo-go/pkg/core/native/nativenames"
@ -20,12 +26,6 @@ import (
"github.com/nspcc-dev/neo-go/pkg/vm/opcode" "github.com/nspcc-dev/neo-go/pkg/vm/opcode"
"github.com/nspcc-dev/neo-go/pkg/vm/stackitem" "github.com/nspcc-dev/neo-go/pkg/vm/stackitem"
"github.com/nspcc-dev/neo-go/pkg/wallet" "github.com/nspcc-dev/neo-go/pkg/wallet"
"github.com/nspcc-dev/neofs-node/cmd/neofs-adm/internal/modules/morph/internal"
"github.com/nspcc-dev/neofs-node/pkg/morph/client"
"github.com/nspcc-dev/neofs-node/pkg/util/rand"
"github.com/nspcc-dev/neofs-sdk-go/subnet"
subnetid "github.com/nspcc-dev/neofs-sdk-go/subnet/id"
"github.com/nspcc-dev/neofs-sdk-go/user"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
) )
@ -39,7 +39,7 @@ func viperBindFlags(cmd *cobra.Command, flags ...string) {
// subnet command section. // subnet command section.
var cmdSubnet = &cobra.Command{ var cmdSubnet = &cobra.Command{
Use: "subnet", Use: "subnet",
Short: "NeoFS subnet management", Short: "FrostFS subnet management",
PreRun: func(cmd *cobra.Command, _ []string) { PreRun: func(cmd *cobra.Command, _ []string) {
viperBindFlags(cmd, viperBindFlags(cmd,
endpointFlag, endpointFlag,
@ -112,7 +112,7 @@ func readSubnetKey(key *keys.PrivateKey) error {
// create subnet command. // create subnet command.
var cmdSubnetCreate = &cobra.Command{ var cmdSubnetCreate = &cobra.Command{
Use: "create", Use: "create",
Short: "Create NeoFS subnet", Short: "Create FrostFS subnet",
PreRun: func(cmd *cobra.Command, _ []string) { PreRun: func(cmd *cobra.Command, _ []string) {
viperBindFlags(cmd, viperBindFlags(cmd,
flagSubnetWallet, flagSubnetWallet,
@ -177,7 +177,7 @@ var errZeroSubnet = errors.New("zero subnet")
// remove subnet command. // remove subnet command.
var cmdSubnetRemove = &cobra.Command{ var cmdSubnetRemove = &cobra.Command{
Use: "remove", Use: "remove",
Short: "Remove NeoFS subnet", Short: "Remove FrostFS subnet",
PreRun: func(cmd *cobra.Command, _ []string) { PreRun: func(cmd *cobra.Command, _ []string) {
viperBindFlags(cmd, viperBindFlags(cmd,
flagSubnetWallet, flagSubnetWallet,
@ -226,7 +226,7 @@ const (
// get subnet command. // get subnet command.
var cmdSubnetGet = &cobra.Command{ var cmdSubnetGet = &cobra.Command{
Use: "get", Use: "get",
Short: "Read information about the NeoFS subnet", Short: "Read information about the FrostFS subnet",
PreRun: func(cmd *cobra.Command, _ []string) { PreRun: func(cmd *cobra.Command, _ []string) {
viperBindFlags(cmd, viperBindFlags(cmd,
flagSubnetGetID, flagSubnetGetID,
@ -290,7 +290,7 @@ const (
// command to manage subnet admins. // command to manage subnet admins.
var cmdSubnetAdmin = &cobra.Command{ var cmdSubnetAdmin = &cobra.Command{
Use: "admin", Use: "admin",
Short: "Manage administrators of the NeoFS subnet", Short: "Manage administrators of the FrostFS subnet",
PreRun: func(cmd *cobra.Command, args []string) { PreRun: func(cmd *cobra.Command, args []string) {
viperBindFlags(cmd, viperBindFlags(cmd,
flagSubnetWallet, flagSubnetWallet,
@ -307,6 +307,8 @@ const (
) )
// common executor cmdSubnetAdminAdd and cmdSubnetAdminRemove commands. // common executor cmdSubnetAdminAdd and cmdSubnetAdminRemove commands.
//
// nolint: funlen
func manageSubnetAdmins(cmd *cobra.Command, rm bool) error { func manageSubnetAdmins(cmd *cobra.Command, rm bool) error {
// read private key // read private key
var key keys.PrivateKey var key keys.PrivateKey
@ -340,7 +342,7 @@ func manageSubnetAdmins(cmd *cobra.Command, rm bool) error {
} }
// prepare call parameters // prepare call parameters
prm := make([]interface{}, 0, 3) prm := make([]any, 0, 3)
prm = append(prm, id.Marshal()) prm = append(prm, id.Marshal())
var method string var method string
@ -397,7 +399,7 @@ func manageSubnetAdmins(cmd *cobra.Command, rm bool) error {
// command to add subnet admin. // command to add subnet admin.
var cmdSubnetAdminAdd = &cobra.Command{ var cmdSubnetAdminAdd = &cobra.Command{
Use: "add", Use: "add",
Short: "Add admin to the NeoFS subnet", Short: "Add admin to the FrostFS subnet",
PreRun: func(cmd *cobra.Command, _ []string) { PreRun: func(cmd *cobra.Command, _ []string) {
viperBindFlags(cmd, viperBindFlags(cmd,
flagSubnetAdminAddGroup, flagSubnetAdminAddGroup,
@ -412,7 +414,7 @@ var cmdSubnetAdminAdd = &cobra.Command{
// command to remove subnet admin. // command to remove subnet admin.
var cmdSubnetAdminRemove = &cobra.Command{ var cmdSubnetAdminRemove = &cobra.Command{
Use: "remove", Use: "remove",
Short: "Remove admin of the NeoFS subnet", Short: "Remove admin of the FrostFS subnet",
PreRun: func(cmd *cobra.Command, _ []string) { PreRun: func(cmd *cobra.Command, _ []string) {
viperBindFlags(cmd, viperBindFlags(cmd,
flagSubnetAdminClient, flagSubnetAdminClient,
@ -433,7 +435,7 @@ const (
// command to manage subnet clients. // command to manage subnet clients.
var cmdSubnetClient = &cobra.Command{ var cmdSubnetClient = &cobra.Command{
Use: "client", Use: "client",
Short: "Manage clients of the NeoFS subnet", Short: "Manage clients of the FrostFS subnet",
PreRun: func(cmd *cobra.Command, _ []string) { PreRun: func(cmd *cobra.Command, _ []string) {
viperBindFlags(cmd, viperBindFlags(cmd,
flagSubnetWallet, flagSubnetWallet,
@ -516,7 +518,7 @@ func manageSubnetClients(cmd *cobra.Command, rm bool) error {
// command to add subnet client. // command to add subnet client.
var cmdSubnetClientAdd = &cobra.Command{ var cmdSubnetClientAdd = &cobra.Command{
Use: "add", Use: "add",
Short: "Add client to the NeoFS subnet", Short: "Add client to the FrostFS subnet",
RunE: func(cmd *cobra.Command, _ []string) error { RunE: func(cmd *cobra.Command, _ []string) error {
return manageSubnetClients(cmd, false) return manageSubnetClients(cmd, false)
}, },
@ -525,7 +527,7 @@ var cmdSubnetClientAdd = &cobra.Command{
// command to remove subnet client. // command to remove subnet client.
var cmdSubnetClientRemove = &cobra.Command{ var cmdSubnetClientRemove = &cobra.Command{
Use: "remove", Use: "remove",
Short: "Remove client of the NeoFS subnet", Short: "Remove client of the FrostFS subnet",
RunE: func(cmd *cobra.Command, _ []string) error { RunE: func(cmd *cobra.Command, _ []string) error {
return manageSubnetClients(cmd, true) return manageSubnetClients(cmd, true)
}, },
@ -598,7 +600,7 @@ func manageSubnetNodes(cmd *cobra.Command, rm bool) error {
// command to manage subnet nodes. // command to manage subnet nodes.
var cmdSubnetNode = &cobra.Command{ var cmdSubnetNode = &cobra.Command{
Use: "node", Use: "node",
Short: "Manage nodes of the NeoFS subnet", Short: "Manage nodes of the FrostFS subnet",
PreRun: func(cmd *cobra.Command, _ []string) { PreRun: func(cmd *cobra.Command, _ []string) {
viperBindFlags(cmd, viperBindFlags(cmd,
flagSubnetWallet, flagSubnetWallet,
@ -611,7 +613,7 @@ var cmdSubnetNode = &cobra.Command{
// command to add subnet node. // command to add subnet node.
var cmdSubnetNodeAdd = &cobra.Command{ var cmdSubnetNodeAdd = &cobra.Command{
Use: "add", Use: "add",
Short: "Add node to the NeoFS subnet", Short: "Add node to the FrostFS subnet",
RunE: func(cmd *cobra.Command, _ []string) error { RunE: func(cmd *cobra.Command, _ []string) error {
return manageSubnetNodes(cmd, false) return manageSubnetNodes(cmd, false)
}, },
@ -620,7 +622,7 @@ var cmdSubnetNodeAdd = &cobra.Command{
// command to remove subnet node. // command to remove subnet node.
var cmdSubnetNodeRemove = &cobra.Command{ var cmdSubnetNodeRemove = &cobra.Command{
Use: "remove", Use: "remove",
Short: "Remove node from the NeoFS subnet", Short: "Remove node from the FrostFS subnet",
RunE: func(cmd *cobra.Command, _ []string) error { RunE: func(cmd *cobra.Command, _ []string) error {
return manageSubnetNodes(cmd, true) return manageSubnetNodes(cmd, true)
}, },
@ -651,6 +653,8 @@ func addCommandInheritPreRun(par *cobra.Command, subs ...*cobra.Command) {
} }
// registers flags and binds sub-commands for subnet commands. // registers flags and binds sub-commands for subnet commands.
//
// nolint: funlen
func init() { func init() {
cmdSubnetCreate.Flags().StringP(flagSubnetWallet, "w", "", "Path to file with wallet") cmdSubnetCreate.Flags().StringP(flagSubnetWallet, "w", "", "Path to file with wallet")
_ = cmdSubnetCreate.MarkFlagRequired(flagSubnetWallet) _ = cmdSubnetCreate.MarkFlagRequired(flagSubnetWallet)
@ -693,7 +697,7 @@ func init() {
_ = cmdSubnetClient.MarkFlagRequired(flagSubnetClientSubnet) _ = cmdSubnetClient.MarkFlagRequired(flagSubnetClientSubnet)
clientFlags.String(flagSubnetClientGroup, "", "ID of the client group to work with") clientFlags.String(flagSubnetClientGroup, "", "ID of the client group to work with")
_ = cmdSubnetClient.MarkFlagRequired(flagSubnetClientGroup) _ = cmdSubnetClient.MarkFlagRequired(flagSubnetClientGroup)
clientFlags.String(flagSubnetClientID, "", "Client's user ID in NeoFS system in text format") clientFlags.String(flagSubnetClientID, "", "Client's user ID in FrostFS system in text format")
_ = cmdSubnetClient.MarkFlagRequired(flagSubnetClientID) _ = cmdSubnetClient.MarkFlagRequired(flagSubnetClientID)
clientFlags.StringP(flagSubnetWallet, "w", "", "Path to file with wallet") clientFlags.StringP(flagSubnetWallet, "w", "", "Path to file with wallet")
_ = cmdSubnetClient.MarkFlagRequired(flagSubnetWallet) _ = cmdSubnetClient.MarkFlagRequired(flagSubnetWallet)
@ -742,7 +746,7 @@ func init() {
) )
} }
func testInvokeMethod(key keys.PrivateKey, method string, args ...interface{}) ([]stackitem.Item, error) { func testInvokeMethod(key keys.PrivateKey, method string, args ...any) ([]stackitem.Item, error) {
c, err := getN3Client(viper.GetViper()) c, err := getN3Client(viper.GetViper())
if err != nil { if err != nil {
return nil, fmt.Errorf("morph client creation: %w", err) return nil, fmt.Errorf("morph client creation: %w", err)
@ -762,7 +766,7 @@ func testInvokeMethod(key keys.PrivateKey, method string, args ...interface{}) (
inv := invoker.New(c, cosigner) inv := invoker.New(c, cosigner)
subnetHash, err := nnsResolveHash(inv, nnsCs.Hash, subnetContract+".neofs") subnetHash, err := nnsResolveHash(inv, nnsCs.Hash, subnetContract+".frostfs")
if err != nil { if err != nil {
return nil, fmt.Errorf("subnet hash resolving: %w", err) return nil, fmt.Errorf("subnet hash resolving: %w", err)
} }
@ -780,7 +784,7 @@ func testInvokeMethod(key keys.PrivateKey, method string, args ...interface{}) (
return res.Stack, nil return res.Stack, nil
} }
func invokeMethod(key keys.PrivateKey, tryNotary bool, method string, args ...interface{}) error { func invokeMethod(key keys.PrivateKey, tryNotary bool, method string, args ...any) error {
c, err := getN3Client(viper.GetViper()) c, err := getN3Client(viper.GetViper())
if err != nil { if err != nil {
return fmt.Errorf("morph client creation: %w", err) return fmt.Errorf("morph client creation: %w", err)
@ -821,7 +825,7 @@ func invokeMethod(key keys.PrivateKey, tryNotary bool, method string, args ...in
return nil return nil
} }
func invokeNonNotary(c Client, key keys.PrivateKey, method string, args ...interface{}) error { func invokeNonNotary(c Client, key keys.PrivateKey, method string, args ...any) error {
nnsCs, err := c.GetContractStateByID(1) nnsCs, err := c.GetContractStateByID(1)
if err != nil { if err != nil {
return fmt.Errorf("NNS contract resolving: %w", err) return fmt.Errorf("NNS contract resolving: %w", err)
@ -845,7 +849,7 @@ func invokeNonNotary(c Client, key keys.PrivateKey, method string, args ...inter
inv := invoker.New(c, cosigner) inv := invoker.New(c, cosigner)
subnetHash, err := nnsResolveHash(inv, nnsCs.Hash, subnetContract+".neofs") subnetHash, err := nnsResolveHash(inv, nnsCs.Hash, subnetContract+".frostfs")
if err != nil { if err != nil {
return fmt.Errorf("subnet hash resolving: %w", err) return fmt.Errorf("subnet hash resolving: %w", err)
} }
@ -868,7 +872,8 @@ func invokeNonNotary(c Client, key keys.PrivateKey, method string, args ...inter
return nil return nil
} }
func invokeNotary(c Client, key keys.PrivateKey, method string, notaryHash util.Uint160, args ...interface{}) error { // nolint: funlen
func invokeNotary(c Client, key keys.PrivateKey, method string, notaryHash util.Uint160, args ...any) error {
nnsCs, err := c.GetContractStateByID(1) nnsCs, err := c.GetContractStateByID(1)
if err != nil { if err != nil {
return fmt.Errorf("NNS contract resolving: %w", err) return fmt.Errorf("NNS contract resolving: %w", err)
@ -891,7 +896,7 @@ func invokeNotary(c Client, key keys.PrivateKey, method string, notaryHash util.
inv := invoker.New(c, cosigners) inv := invoker.New(c, cosigners)
subnetHash, err := nnsResolveHash(inv, nnsCs.Hash, subnetContract+".neofs") subnetHash, err := nnsResolveHash(inv, nnsCs.Hash, subnetContract+".frostfs")
if err != nil { if err != nil {
return fmt.Errorf("subnet hash resolving: %w", err) return fmt.Errorf("subnet hash resolving: %w", err)
} }
@ -968,7 +973,7 @@ func invokeNotary(c Client, key keys.PrivateKey, method string, notaryHash util.
func notaryCosigners(c Client, notaryHash util.Uint160, nnsCs *state.Contract, func notaryCosigners(c Client, notaryHash util.Uint160, nnsCs *state.Contract,
key keys.PrivateKey, alphabetAccount util.Uint160) ([]transaction.Signer, error) { key keys.PrivateKey, alphabetAccount util.Uint160) ([]transaction.Signer, error) {
proxyHash, err := nnsResolveHash(invoker.New(c, nil), nnsCs.Hash, proxyContract+".neofs") proxyHash, err := nnsResolveHash(invoker.New(c, nil), nnsCs.Hash, proxyContract+".frostfs")
if err != nil { if err != nil {
return nil, fmt.Errorf("proxy hash resolving: %w", err) return nil, fmt.Errorf("proxy hash resolving: %w", err)
} }

View file

@ -0,0 +1,85 @@
package modules
import (
"os"
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/commonflags"
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/modules/config"
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/modules/morph"
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/modules/storagecfg"
"git.frostfs.info/TrueCloudLab/frostfs-node/misc"
"git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/autocomplete"
utilConfig "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/config"
"git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/gendoc"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
var (
rootCmd = &cobra.Command{
Use: "frostfs-adm",
Short: "FrostFS Administrative Tool",
Long: `FrostFS Administrative Tool provides functions to setup and
manage FrostFS network deployment.`,
RunE: entryPoint,
SilenceUsage: true,
}
)
func init() {
cobra.OnInitialize(func() { initConfig(rootCmd) })
// we need to init viper config to bind viper and cobra configurations for
// rpc endpoint, alphabet wallet dir, key credentials, etc.
// use stdout as default output for cmd.Print()
rootCmd.SetOut(os.Stdout)
rootCmd.PersistentFlags().StringP(commonflags.ConfigFlag, commonflags.ConfigFlagShorthand, "", commonflags.ConfigFlagUsage)
rootCmd.PersistentFlags().String(commonflags.ConfigDirFlag, "", commonflags.ConfigDirFlagUsage)
rootCmd.PersistentFlags().BoolP(commonflags.Verbose, commonflags.VerboseShorthand, false, commonflags.VerboseUsage)
_ = viper.BindPFlag(commonflags.Verbose, rootCmd.PersistentFlags().Lookup(commonflags.Verbose))
rootCmd.Flags().Bool("version", false, "Application version")
rootCmd.AddCommand(config.RootCmd)
rootCmd.AddCommand(morph.RootCmd)
rootCmd.AddCommand(storagecfg.RootCmd)
rootCmd.AddCommand(autocomplete.Command("frostfs-adm"))
rootCmd.AddCommand(gendoc.Command(rootCmd))
}
func Execute() error {
return rootCmd.Execute()
}
func entryPoint(cmd *cobra.Command, args []string) error {
printVersion, _ := cmd.Flags().GetBool("version")
if printVersion {
cmd.Print(misc.BuildInfo("FrostFS Adm"))
return nil
}
return cmd.Usage()
}
func initConfig(cmd *cobra.Command) {
configFile, err := cmd.Flags().GetString(commonflags.ConfigFlag)
if err != nil {
return
}
if configFile != "" {
viper.SetConfigType("yml")
viper.SetConfigFile(configFile)
_ = viper.ReadInConfig() // if config file is set but unavailable, ignore it
}
configDir, err := cmd.Flags().GetString(commonflags.ConfigDirFlag)
if err != nil {
return
}
if configDir != "" {
_ = utilConfig.ReadConfigDir(viper.GetViper(), configDir) // if config files cannot be read, ignore it
}
}

View file

@ -14,7 +14,7 @@ node:
relay: {{ .Relay }} # start Storage node in relay mode without bootstrapping into the Network map relay: {{ .Relay }} # start Storage node in relay mode without bootstrapping into the Network map
subnet: subnet:
exit_zero: false # toggle entrance to zero subnet (overrides corresponding attribute and occurrence in entries) exit_zero: false # toggle entrance to zero subnet (overrides corresponding attribute and occurrence in entries)
entries: [] # list of IDs of subnets to enter in a text format of NeoFS API protocol (overrides corresponding attributes) entries: [] # list of IDs of subnets to enter in a text format of FrostFS API protocol (overrides corresponding attributes)
grpc: grpc:
num: 1 # total number of listener endpoints num: 1 # total number of listener endpoints

View file

@ -16,20 +16,20 @@ import (
"text/template" "text/template"
"time" "time"
netutil "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/network"
"github.com/chzyer/readline" "github.com/chzyer/readline"
"github.com/nspcc-dev/neo-go/cli/flags" "github.com/nspcc-dev/neo-go/cli/flags"
"github.com/nspcc-dev/neo-go/cli/input" "github.com/nspcc-dev/neo-go/cli/input"
"github.com/nspcc-dev/neo-go/pkg/core/native/nativenames"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/nspcc-dev/neo-go/pkg/encoding/address" "github.com/nspcc-dev/neo-go/pkg/encoding/address"
"github.com/nspcc-dev/neo-go/pkg/encoding/fixedn" "github.com/nspcc-dev/neo-go/pkg/encoding/fixedn"
"github.com/nspcc-dev/neo-go/pkg/rpcclient" "github.com/nspcc-dev/neo-go/pkg/rpcclient"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/actor" "github.com/nspcc-dev/neo-go/pkg/rpcclient/actor"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/gas"
"github.com/nspcc-dev/neo-go/pkg/rpcclient/nep17" "github.com/nspcc-dev/neo-go/pkg/rpcclient/nep17"
"github.com/nspcc-dev/neo-go/pkg/smartcontract/trigger" "github.com/nspcc-dev/neo-go/pkg/smartcontract/trigger"
"github.com/nspcc-dev/neo-go/pkg/util" "github.com/nspcc-dev/neo-go/pkg/util"
"github.com/nspcc-dev/neo-go/pkg/wallet" "github.com/nspcc-dev/neo-go/pkg/wallet"
netutil "github.com/nspcc-dev/neofs-node/pkg/network"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
@ -79,6 +79,7 @@ type config struct {
MetabasePath string MetabasePath string
} }
// nolint: funlen
func storageConfig(cmd *cobra.Command, args []string) { func storageConfig(cmd *cobra.Command, args []string) {
var outPath string var outPath string
if len(args) != 0 { if len(args) != 0 {
@ -90,7 +91,7 @@ func storageConfig(cmd *cobra.Command, args []string) {
} }
} }
historyPath := filepath.Join(os.TempDir(), "neofs-adm.history") historyPath := filepath.Join(os.TempDir(), "frostfs-adm.history")
readline.SetHistoryPath(historyPath) readline.SetHistoryPath(historyPath)
var c config var c config
@ -153,7 +154,7 @@ func storageConfig(cmd *cobra.Command, args []string) {
validator := netutil.Address{} validator := netutil.Address{}
err := validator.FromString(c.AnnouncedAddress) err := validator.FromString(c.AnnouncedAddress)
if err != nil { if err != nil {
cmd.Println("Incorrect address format. See https://github.com/nspcc-dev/neofs-node/blob/master/pkg/network/address.go for details.") cmd.Println("Incorrect address format. See https://git.frostfs.info/TrueCloudLab/frostfs-node/src/branch/master/pkg/network/address.go for details.")
continue continue
} }
uriAddr, err := url.Parse(validator.URIAddr()) uriAddr, err := url.Parse(validator.URIAddr())
@ -209,7 +210,7 @@ func storageConfig(cmd *cobra.Command, args []string) {
out := applyTemplate(c) out := applyTemplate(c)
fatalOnErr(os.WriteFile(outPath, out, 0644)) fatalOnErr(os.WriteFile(outPath, out, 0644))
cmd.Println("Node is ready for work! Run `neofs-node -config " + outPath + "`") cmd.Println("Node is ready for work! Run `frostfs-node -config " + outPath + "`")
} }
func getWalletAccount(w *wallet.Wallet, prompt string) string { func getWalletAccount(w *wallet.Wallet, prompt string) string {
@ -349,17 +350,12 @@ func depositGas(cmd *cobra.Command, acc *wallet.Account, network string) {
mainClient := initClient(n3config[network].RPC) mainClient := initClient(n3config[network].RPC)
neofsHash, _ := util.Uint160DecodeStringLE(n3config[network].NeoFSContract) neofsHash, _ := util.Uint160DecodeStringLE(n3config[network].NeoFSContract)
gasHash, err := mainClient.GetNativeContractHash(nativenames.Gas)
if err != nil {
fatalOnErr(fmt.Errorf("gas contract hash: %w", err))
}
mainActor, err := actor.NewSimple(mainClient, acc) mainActor, err := actor.NewSimple(mainClient, acc)
if err != nil { if err != nil {
fatalOnErr(fmt.Errorf("creating actor over main chain client: %w", err)) fatalOnErr(fmt.Errorf("creating actor over main chain client: %w", err))
} }
mainGas := nep17.New(mainActor, gasHash) mainGas := nep17.New(mainActor, gas.Hash)
txHash, _, err := mainGas.Transfer(accSH, neofsHash, amount, nil) txHash, _, err := mainGas.Transfer(accSH, neofsHash, amount, nil)
if err != nil { if err != nil {

View file

@ -3,7 +3,7 @@ package main
import ( import (
"os" "os"
"github.com/nspcc-dev/neofs-node/cmd/neofs-adm/internal/modules" "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/modules"
) )
func main() { func main() {

View file

@ -1,8 +1,8 @@
# How NeoFS CLI uses session mechanism of the NeoFS # How FrostFS CLI uses session mechanism of the FrostFS
## Overview ## Overview
NeoFS sessions implement a mechanism for issuing a power of attorney by one FrostFS sessions implement a mechanism for issuing a power of attorney by one
party to another. A trusted party can provide a so-called session token as party to another. A trusted party can provide a so-called session token as
proof of the right to act on behalf of another member of the network. The proof of the right to act on behalf of another member of the network. The
client of operations carried out with such a token will be the user who opened client of operations carried out with such a token will be the user who opened
@ -15,7 +15,7 @@ attached session token is treated as performed by the original client.
## Types ## Types
NeoFS CLI supports two ways to execute operation within a session depending on FrostFS CLI supports two ways to execute operation within a session depending on
whether the user of the command application is an original user (1) or a trusted whether the user of the command application is an original user (1) or a trusted
one (2). one (2).
@ -30,7 +30,7 @@ to the trusted server. At the moment, the approach is applicable only to
creating objects. creating objects.
```shell ```shell
$ neofs-cli session create --rpc-endpoint <server_ip> --out ./blank_token $ frostfs-cli session create --rpc-endpoint <server_ip> --out ./blank_token
``` ```
After this example command remote node holds session private key while its After this example command remote node holds session private key while its
public part is written into the session token encoded into the output file. public part is written into the session token encoded into the output file.
@ -49,7 +49,7 @@ original client, and the CLI uses it only for reading. Ready token MUST have:
To sign the session token, exec: To sign the session token, exec:
```shell ```shell
$ neofs-cli --wallet <client_wallet> util sign session-token --from ./blank_token --to ./token $ frostfs-cli --wallet <client_wallet> util sign session-token --from ./blank_token --to ./token
``` ```
Once the token is signed, it MUST NOT be modified. Once the token is signed, it MUST NOT be modified.

View file

@ -0,0 +1,34 @@
# Extended headers
## Overview
Extended headers are used for request/response. They may contain any
user-defined headers to be interpreted on application level. Key name must be a
unique valid UTF-8 string. Value can't be empty. Requests or Responses with
duplicated header names or headers with empty values are considered invalid.
## Existing headers
There are some "well-known" headers starting with `__FROSTFS__` prefix that
affect system behaviour. For backward compatibility, the same set of
"well-known" headers may also use `__NEOFS__` prefix:
* `__FROSTFS__NETMAP_EPOCH` - netmap epoch to use for object placement calculation. The `value` is string
encoded `uint64` in decimal presentation. If set to '0' or omitted, the
current epoch only will be used.
* `__FROSTFS__NETMAP_LOOKUP_DEPTH` - if object can't be found using current epoch's netmap, this header limits
how many past epochs the node can look up through. Depth is applied to a current epoch or the value
of `__FROSTFS__NETMAP_EPOCH` attribute. The `value` is string encoded `uint64` in decimal presentation.
If set to '0' or not set, only the current epoch is used.
## `frostfs-cli` commands with `--xhdr`
List of commands with support of extended headers:
* `container list-objects`
* `object delete/get/hash/head/lock/put/range/search`
* `storagegroup delete/get/list/put`
Example:
```shell
$ frostfs-cli object put -r s01.frostfs.devenv:8080 -w wallet.json --cid CID --file FILE --xhdr "__FROSTFS__NETMAP_EPOCH=777"
```

View file

@ -7,15 +7,15 @@ import (
"fmt" "fmt"
"io" "io"
"github.com/nspcc-dev/neofs-sdk-go/accounting" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/accounting"
"github.com/nspcc-dev/neofs-sdk-go/client" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client"
containerSDK "github.com/nspcc-dev/neofs-sdk-go/container" containerSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
cid "github.com/nspcc-dev/neofs-sdk-go/container/id" cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"github.com/nspcc-dev/neofs-sdk-go/eacl" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl"
"github.com/nspcc-dev/neofs-sdk-go/netmap" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap"
"github.com/nspcc-dev/neofs-sdk-go/object" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "github.com/nspcc-dev/neofs-sdk-go/object/id" oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"github.com/nspcc-dev/neofs-sdk-go/version" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/version"
) )
// BalanceOfPrm groups parameters of BalanceOf operation. // BalanceOfPrm groups parameters of BalanceOf operation.
@ -34,7 +34,7 @@ func (x BalanceOfRes) Balance() accounting.Decimal {
return x.cliRes.Amount() return x.cliRes.Amount()
} }
// BalanceOf requests the current balance of a NeoFS user. // BalanceOf requests the current balance of a FrostFS user.
// //
// Returns any error which prevented the operation from completing correctly in error return. // Returns any error which prevented the operation from completing correctly in error return.
func BalanceOf(prm BalanceOfPrm) (res BalanceOfRes, err error) { func BalanceOf(prm BalanceOfPrm) (res BalanceOfRes, err error) {
@ -59,7 +59,7 @@ func (x ListContainersRes) IDList() []cid.ID {
return x.cliRes.Containers() return x.cliRes.Containers()
} }
// ListContainers requests a list of NeoFS user's containers. // ListContainers requests a list of FrostFS user's containers.
// //
// Returns any error which prevented the operation from completing correctly in error return. // Returns any error which prevented the operation from completing correctly in error return.
func ListContainers(prm ListContainersPrm) (res ListContainersRes, err error) { func ListContainers(prm ListContainersPrm) (res ListContainersRes, err error) {
@ -84,7 +84,7 @@ func (x PutContainerRes) ID() cid.ID {
return x.cnr return x.cnr
} }
// PutContainer sends a request to save the container in NeoFS. // PutContainer sends a request to save the container in FrostFS.
// //
// Operation is asynchronous and not guaranteed even in the absence of errors. // Operation is asynchronous and not guaranteed even in the absence of errors.
// The required time is also not predictable. // The required time is also not predictable.
@ -122,7 +122,7 @@ func (x GetContainerRes) Container() containerSDK.Container {
return x.cliRes.Container() return x.cliRes.Container()
} }
// GetContainer reads a container from NeoFS by ID. // GetContainer reads a container from FrostFS by ID.
// //
// Returns any error which prevented the operation from completing correctly in error return. // Returns any error which prevented the operation from completing correctly in error return.
func GetContainer(prm GetContainerPrm) (res GetContainerRes, err error) { func GetContainer(prm GetContainerPrm) (res GetContainerRes, err error) {
@ -140,7 +140,7 @@ func IsACLExtendable(c *client.Client, cnr cid.ID) (bool, error) {
res, err := GetContainer(prm) res, err := GetContainer(prm)
if err != nil { if err != nil {
return false, fmt.Errorf("get container from the NeoFS: %w", err) return false, fmt.Errorf("get container from the FrostFS: %w", err)
} }
return res.Container().BasicACL().Extendable(), nil return res.Container().BasicACL().Extendable(), nil
@ -155,7 +155,7 @@ type DeleteContainerPrm struct {
// DeleteContainerRes groups the resulting values of DeleteContainer operation. // DeleteContainerRes groups the resulting values of DeleteContainer operation.
type DeleteContainerRes struct{} type DeleteContainerRes struct{}
// DeleteContainer sends a request to remove a container from NeoFS by ID. // DeleteContainer sends a request to remove a container from FrostFS by ID.
// //
// Operation is asynchronous and not guaranteed even in the absence of errors. // Operation is asynchronous and not guaranteed even in the absence of errors.
// The required time is also not predictable. // The required time is also not predictable.
@ -185,7 +185,7 @@ func (x EACLRes) EACL() eacl.Table {
return x.cliRes.Table() return x.cliRes.Table()
} }
// EACL reads eACL table from NeoFS by container ID. // EACL reads eACL table from FrostFS by container ID.
// //
// Returns any error which prevented the operation from completing correctly in error return. // Returns any error which prevented the operation from completing correctly in error return.
func EACL(prm EACLPrm) (res EACLRes, err error) { func EACL(prm EACLPrm) (res EACLRes, err error) {
@ -203,7 +203,7 @@ type SetEACLPrm struct {
// SetEACLRes groups the resulting values of SetEACL operation. // SetEACLRes groups the resulting values of SetEACL operation.
type SetEACLRes struct{} type SetEACLRes struct{}
// SetEACL requests to save an eACL table in NeoFS. // SetEACL requests to save an eACL table in FrostFS.
// //
// Operation is asynchronous and no guaranteed even in the absence of errors. // Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable. // The required time is also not predictable.
@ -228,12 +228,12 @@ type NetworkInfoRes struct {
cliRes *client.ResNetworkInfo cliRes *client.ResNetworkInfo
} }
// NetworkInfo returns structured information about the NeoFS network. // NetworkInfo returns structured information about the FrostFS network.
func (x NetworkInfoRes) NetworkInfo() netmap.NetworkInfo { func (x NetworkInfoRes) NetworkInfo() netmap.NetworkInfo {
return x.cliRes.Info() return x.cliRes.Info()
} }
// NetworkInfo reads information about the NeoFS network. // NetworkInfo reads information about the FrostFS network.
// //
// Returns any error which prevented the operation from completing correctly in error return. // Returns any error which prevented the operation from completing correctly in error return.
func NetworkInfo(prm NetworkInfoPrm) (res NetworkInfoRes, err error) { func NetworkInfo(prm NetworkInfoPrm) (res NetworkInfoRes, err error) {
@ -258,12 +258,12 @@ func (x NodeInfoRes) NodeInfo() netmap.NodeInfo {
return x.cliRes.NodeInfo() return x.cliRes.NodeInfo()
} }
// LatestVersion returns the latest NeoFS API version in use. // LatestVersion returns the latest FrostFS API version in use.
func (x NodeInfoRes) LatestVersion() version.Version { func (x NodeInfoRes) LatestVersion() version.Version {
return x.cliRes.LatestVersion() return x.cliRes.LatestVersion()
} }
// NodeInfo requests information about the remote server from NeoFS netmap. // NodeInfo requests information about the remote server from FrostFS netmap.
// //
// Returns any error which prevented the operation from completing correctly in error return. // Returns any error which prevented the operation from completing correctly in error return.
func NodeInfo(prm NodeInfoPrm) (res NodeInfoRes, err error) { func NodeInfo(prm NodeInfoPrm) (res NodeInfoRes, err error) {
@ -282,7 +282,7 @@ type NetMapSnapshotRes struct {
cliRes *client.ResNetMapSnapshot cliRes *client.ResNetMapSnapshot
} }
// NetMap returns current local snapshot of the NeoFS network map. // NetMap returns current local snapshot of the FrostFS network map.
func (x NetMapSnapshotRes) NetMap() netmap.NetMap { func (x NetMapSnapshotRes) NetMap() netmap.NetMap {
return x.cliRes.NetMap() return x.cliRes.NetMap()
} }
@ -362,7 +362,7 @@ func (x PutObjectRes) ID() oid.ID {
return x.id return x.id
} }
// PutObject saves the object in NeoFS network. // PutObject saves the object in FrostFS network.
// //
// Returns any error which prevented the operation from completing correctly in error return. // Returns any error which prevented the operation from completing correctly in error return.
func PutObject(prm PutObjectPrm) (*PutObjectRes, error) { func PutObject(prm PutObjectPrm) (*PutObjectRes, error) {
@ -404,8 +404,7 @@ func PutObject(prm PutObjectPrm) (*PutObjectRes, error) {
} }
if prm.rdr != nil { if prm.rdr != nil {
// TODO: (neofs-node#1198) explore better values or configure it const defaultBufferSizePut = 3 << 20 // Maximum chunk size is 3 MiB in the SDK.
const defaultBufferSizePut = 4096
if sz == 0 || sz > defaultBufferSizePut { if sz == 0 || sz > defaultBufferSizePut {
sz = defaultBufferSizePut sz = defaultBufferSizePut
@ -460,7 +459,7 @@ func (x DeleteObjectRes) Tombstone() oid.ID {
return x.tomb return x.tomb
} }
// DeleteObject marks an object to be removed from NeoFS through tombstone placement. // DeleteObject marks an object to be removed from FrostFS through tombstone placement.
// //
// Returns any error which prevented the operation from completing correctly in error return. // Returns any error which prevented the operation from completing correctly in error return.
func DeleteObject(prm DeleteObjectPrm) (*DeleteObjectRes, error) { func DeleteObject(prm DeleteObjectPrm) (*DeleteObjectRes, error) {
@ -576,7 +575,7 @@ type HeadObjectPrm struct {
mainOnly bool mainOnly bool
} }
// SetMainOnlyFlag sets flag to get only main fields of an object header in terms of NeoFS API. // SetMainOnlyFlag sets flag to get only main fields of an object header in terms of FrostFS API.
func (x *HeadObjectPrm) SetMainOnlyFlag(v bool) { func (x *HeadObjectPrm) SetMainOnlyFlag(v bool) {
x.mainOnly = v x.mainOnly = v
} }
@ -812,7 +811,7 @@ func (x *PayloadRangePrm) SetRange(rng *object.Range) {
// PayloadRangeRes groups the resulting values of PayloadRange operation. // PayloadRangeRes groups the resulting values of PayloadRange operation.
type PayloadRangeRes struct{} type PayloadRangeRes struct{}
// PayloadRange reads object payload range from NeoFS and writes it to the specified writer. // PayloadRange reads object payload range from FrostFS and writes it to the specified writer.
// //
// Interrupts on any writer error. // Interrupts on any writer error.
// //
@ -872,7 +871,7 @@ func (s *SyncContainerPrm) SetContainer(c *containerSDK.Container) {
// operation. // operation.
type SyncContainerRes struct{} type SyncContainerRes struct{}
// SyncContainerSettings reads global network config from NeoFS and // SyncContainerSettings reads global network config from FrostFS and
// syncs container settings with it. // syncs container settings with it.
// //
// Interrupts on any writer error. // Interrupts on any writer error.

View file

@ -0,0 +1,15 @@
// Package internal provides functionality for FrostFS CLI application
// communication with FrostFS network.
//
// The base client for accessing remote nodes via FrostFS API is a FrostFS SDK
// Go API client. However, although it encapsulates a useful piece of business
// logic (e.g. the signature mechanism), the FrostFS CLI application does not
// fully use the client's flexible interface.
//
// In this regard, this package provides functions over base API client
// necessary for the application. This allows you to concentrate the entire
// spectrum of the client's use in one place (this will be convenient both when
// updating the base client and for evaluating the UX of SDK library). So it is
// expected that all application packages will be limited to this package for
// the development of functionality requiring FrostFS API communication.
package internal

View file

@ -3,11 +3,11 @@ package internal
import ( import (
"io" "io"
"github.com/nspcc-dev/neofs-sdk-go/bearer" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
"github.com/nspcc-dev/neofs-sdk-go/client" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client"
cid "github.com/nspcc-dev/neofs-sdk-go/container/id" cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
oid "github.com/nspcc-dev/neofs-sdk-go/object/id" oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"github.com/nspcc-dev/neofs-sdk-go/session" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
) )
// here are small structures with public setters to share between parameter structures // here are small structures with public setters to share between parameter structures
@ -16,7 +16,7 @@ type commonPrm struct {
cli *client.Client cli *client.Client
} }
// SetClient sets the base client for NeoFS API communication. // SetClient sets the base client for FrostFS API communication.
func (x *commonPrm) SetClient(cli *client.Client) { func (x *commonPrm) SetClient(cli *client.Client) {
x.cli = cli x.cli = cli
} }

View file

@ -8,38 +8,39 @@ import (
"errors" "errors"
"fmt" "fmt"
"github.com/nspcc-dev/neofs-node/cmd/neofs-cli/internal/common" "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/internal/common"
"github.com/nspcc-dev/neofs-node/cmd/neofs-cli/internal/commonflags" "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/internal/commonflags"
"github.com/nspcc-dev/neofs-node/pkg/network" commonCmd "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/internal/common"
"github.com/nspcc-dev/neofs-sdk-go/client" "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/network"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
) )
var errInvalidEndpoint = errors.New("provided RPC endpoint is incorrect") var errInvalidEndpoint = errors.New("provided RPC endpoint is incorrect")
// GetSDKClientByFlag returns default neofs-sdk-go client using the specified flag for the address. // GetSDKClientByFlag returns default frostfs-sdk-go client using the specified flag for the address.
// On error, outputs to stderr of cmd and exits with non-zero code. // On error, outputs to stderr of cmd and exits with non-zero code.
func GetSDKClientByFlag(cmd *cobra.Command, key *ecdsa.PrivateKey, endpointFlag string) *client.Client { func GetSDKClientByFlag(cmd *cobra.Command, key *ecdsa.PrivateKey, endpointFlag string) *client.Client {
cli, err := getSDKClientByFlag(key, endpointFlag) cli, err := getSDKClientByFlag(cmd, key, endpointFlag)
if err != nil { if err != nil {
common.ExitOnErr(cmd, "can't create API client: %w", err) commonCmd.ExitOnErr(cmd, "can't create API client: %w", err)
} }
return cli return cli
} }
func getSDKClientByFlag(key *ecdsa.PrivateKey, endpointFlag string) (*client.Client, error) { func getSDKClientByFlag(cmd *cobra.Command, key *ecdsa.PrivateKey, endpointFlag string) (*client.Client, error) {
var addr network.Address var addr network.Address
err := addr.FromString(viper.GetString(endpointFlag)) err := addr.FromString(viper.GetString(endpointFlag))
if err != nil { if err != nil {
return nil, fmt.Errorf("%v: %w", errInvalidEndpoint, err) return nil, fmt.Errorf("%v: %w", errInvalidEndpoint, err)
} }
return GetSDKClient(key, addr) return GetSDKClient(cmd, key, addr)
} }
// GetSDKClient returns default neofs-sdk-go client. // GetSDKClient returns default frostfs-sdk-go client.
func GetSDKClient(key *ecdsa.PrivateKey, addr network.Address) (*client.Client, error) { func GetSDKClient(cmd *cobra.Command, key *ecdsa.PrivateKey, addr network.Address) (*client.Client, error) {
var ( var (
c client.Client c client.Client
prmInit client.PrmInit prmInit client.PrmInit
@ -47,7 +48,7 @@ func GetSDKClient(key *ecdsa.PrivateKey, addr network.Address) (*client.Client,
) )
prmInit.SetDefaultPrivateKey(*key) prmInit.SetDefaultPrivateKey(*key)
prmInit.ResolveNeoFSFailures() prmInit.ResolveFrostFSFailures()
prmDial.SetServerURI(addr.URIAddr()) prmDial.SetServerURI(addr.URIAddr())
if timeout := viper.GetDuration(commonflags.Timeout); timeout > 0 { if timeout := viper.GetDuration(commonflags.Timeout); timeout > 0 {
// In CLI we can only set a timeout for the whole operation. // In CLI we can only set a timeout for the whole operation.
@ -56,7 +57,7 @@ func GetSDKClient(key *ecdsa.PrivateKey, addr network.Address) (*client.Client,
prmDial.SetTimeout(timeout) prmDial.SetTimeout(timeout)
prmDial.SetStreamTimeout(timeout) prmDial.SetStreamTimeout(timeout)
common.PrintVerbose("Set request timeout to %s.", timeout) common.PrintVerbose(cmd, "Set request timeout to %s.", timeout)
} }
c.Init(prmInit) c.Init(prmInit)
@ -69,7 +70,7 @@ func GetSDKClient(key *ecdsa.PrivateKey, addr network.Address) (*client.Client,
} }
// GetCurrentEpoch returns current epoch. // GetCurrentEpoch returns current epoch.
func GetCurrentEpoch(ctx context.Context, endpoint string) (uint64, error) { func GetCurrentEpoch(ctx context.Context, cmd *cobra.Command, endpoint string) (uint64, error) {
var addr network.Address var addr network.Address
if err := addr.FromString(endpoint); err != nil { if err := addr.FromString(endpoint); err != nil {
@ -81,7 +82,7 @@ func GetCurrentEpoch(ctx context.Context, endpoint string) (uint64, error) {
return 0, fmt.Errorf("can't generate key to sign query: %w", err) return 0, fmt.Errorf("can't generate key to sign query: %w", err)
} }
c, err := GetSDKClient(key, addr) c, err := GetSDKClient(cmd, key, addr)
if err != nil { if err != nil {
return 0, err return 0, err
} }

View file

@ -4,9 +4,10 @@ import (
"errors" "errors"
"os" "os"
"github.com/nspcc-dev/neofs-node/pkg/core/version" commonCmd "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/internal/common"
"github.com/nspcc-dev/neofs-sdk-go/eacl" "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/core/version"
versionSDK "github.com/nspcc-dev/neofs-sdk-go/version" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl"
versionSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/version"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
@ -16,29 +17,29 @@ var errUnsupportedEACLFormat = errors.New("unsupported eACL format")
func ReadEACL(cmd *cobra.Command, eaclPath string) *eacl.Table { func ReadEACL(cmd *cobra.Command, eaclPath string) *eacl.Table {
_, err := os.Stat(eaclPath) // check if `eaclPath` is an existing file _, err := os.Stat(eaclPath) // check if `eaclPath` is an existing file
if err != nil { if err != nil {
ExitOnErr(cmd, "", errors.New("incorrect path to file with EACL")) commonCmd.ExitOnErr(cmd, "", errors.New("incorrect path to file with EACL"))
} }
PrintVerbose("Reading EACL from file: %s", eaclPath) PrintVerbose(cmd, "Reading EACL from file: %s", eaclPath)
data, err := os.ReadFile(eaclPath) data, err := os.ReadFile(eaclPath)
ExitOnErr(cmd, "can't read file with EACL: %w", err) commonCmd.ExitOnErr(cmd, "can't read file with EACL: %w", err)
table := eacl.NewTable() table := eacl.NewTable()
if err = table.UnmarshalJSON(data); err == nil { if err = table.UnmarshalJSON(data); err == nil {
validateAndFixEACLVersion(table) validateAndFixEACLVersion(table)
PrintVerbose("Parsed JSON encoded EACL table") PrintVerbose(cmd, "Parsed JSON encoded EACL table")
return table return table
} }
if err = table.Unmarshal(data); err == nil { if err = table.Unmarshal(data); err == nil {
validateAndFixEACLVersion(table) validateAndFixEACLVersion(table)
PrintVerbose("Parsed binary encoded EACL table") PrintVerbose(cmd, "Parsed binary encoded EACL table")
return table return table
} }
ExitOnErr(cmd, "", errUnsupportedEACLFormat) commonCmd.ExitOnErr(cmd, "", errUnsupportedEACLFormat)
return nil return nil
} }

View file

@ -11,12 +11,12 @@ import (
func PrettyPrintJSON(cmd *cobra.Command, m json.Marshaler, entity string) { func PrettyPrintJSON(cmd *cobra.Command, m json.Marshaler, entity string) {
data, err := m.MarshalJSON() data, err := m.MarshalJSON()
if err != nil { if err != nil {
PrintVerbose("Can't convert %s to json: %w", entity, err) PrintVerbose(cmd, "Can't convert %s to json: %w", entity, err)
return return
} }
buf := new(bytes.Buffer) buf := new(bytes.Buffer)
if err := json.Indent(buf, data, "", " "); err != nil { if err := json.Indent(buf, data, "", " "); err != nil {
PrintVerbose("Can't pretty print json: %w", err) PrintVerbose(cmd, "Can't pretty print json: %w", err)
return return
} }
cmd.Println(buf) cmd.Println(buf)

View file

@ -6,31 +6,32 @@ import (
"fmt" "fmt"
"os" "os"
"github.com/nspcc-dev/neofs-sdk-go/bearer" commonCmd "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/internal/common"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
// ReadBearerToken reads bearer token from the path provided in a specified flag. // ReadBearerToken reads bearer token from the path provided in a specified flag.
func ReadBearerToken(cmd *cobra.Command, flagname string) *bearer.Token { func ReadBearerToken(cmd *cobra.Command, flagname string) *bearer.Token {
path, err := cmd.Flags().GetString(flagname) path, err := cmd.Flags().GetString(flagname)
ExitOnErr(cmd, "", err) commonCmd.ExitOnErr(cmd, "", err)
if len(path) == 0 { if len(path) == 0 {
return nil return nil
} }
PrintVerbose("Reading bearer token from file [%s]...", path) PrintVerbose(cmd, "Reading bearer token from file [%s]...", path)
var tok bearer.Token var tok bearer.Token
err = ReadBinaryOrJSON(&tok, path) err = ReadBinaryOrJSON(cmd, &tok, path)
ExitOnErr(cmd, "invalid bearer token: %v", err) commonCmd.ExitOnErr(cmd, "invalid bearer token: %v", err)
return &tok return &tok
} }
// BinaryOrJSON is an interface of entities which provide json.Unmarshaler // BinaryOrJSON is an interface of entities which provide json.Unmarshaler
// and NeoFS binary decoder. // and FrostFS binary decoder.
type BinaryOrJSON interface { type BinaryOrJSON interface {
Unmarshal([]byte) error Unmarshal([]byte) error
json.Unmarshaler json.Unmarshaler
@ -38,8 +39,8 @@ type BinaryOrJSON interface {
// ReadBinaryOrJSON reads file data using provided path and decodes // ReadBinaryOrJSON reads file data using provided path and decodes
// BinaryOrJSON from the data. // BinaryOrJSON from the data.
func ReadBinaryOrJSON(dst BinaryOrJSON, fPath string) error { func ReadBinaryOrJSON(cmd *cobra.Command, dst BinaryOrJSON, fPath string) error {
PrintVerbose("Reading file [%s]...", fPath) PrintVerbose(cmd, "Reading file [%s]...", fPath)
// try to read session token from file // try to read session token from file
data, err := os.ReadFile(fPath) data, err := os.ReadFile(fPath)
@ -47,17 +48,17 @@ func ReadBinaryOrJSON(dst BinaryOrJSON, fPath string) error {
return fmt.Errorf("read file <%s>: %w", fPath, err) return fmt.Errorf("read file <%s>: %w", fPath, err)
} }
PrintVerbose("Trying to decode binary...") PrintVerbose(cmd, "Trying to decode binary...")
err = dst.Unmarshal(data) err = dst.Unmarshal(data)
if err != nil { if err != nil {
PrintVerbose("Failed to decode binary: %v", err) PrintVerbose(cmd, "Failed to decode binary: %v", err)
PrintVerbose("Trying to decode JSON...") PrintVerbose(cmd, "Trying to decode JSON...")
err = dst.UnmarshalJSON(data) err = dst.UnmarshalJSON(data)
if err != nil { if err != nil {
PrintVerbose("Failed to decode JSON: %v", err) PrintVerbose(cmd, "Failed to decode JSON: %v", err)
return errors.New("invalid format") return errors.New("invalid format")
} }
} }

View file

@ -2,20 +2,19 @@ package common
import ( import (
"encoding/hex" "encoding/hex"
"fmt"
"strconv" "strconv"
"time" "time"
"github.com/nspcc-dev/neofs-node/cmd/neofs-cli/internal/commonflags" "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/internal/commonflags"
"github.com/nspcc-dev/neofs-sdk-go/checksum" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/checksum"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
) )
// PrintVerbose prints to the stdout if the commonflags.Verbose flag is on. // PrintVerbose prints to the stdout if the commonflags.Verbose flag is on.
func PrintVerbose(format string, a ...interface{}) { func PrintVerbose(cmd *cobra.Command, format string, a ...any) {
if viper.GetBool(commonflags.Verbose) { if viper.GetBool(commonflags.Verbose) {
fmt.Printf(format+"\n", a...) cmd.Printf(format+"\n", a...)
} }
} }

View file

@ -41,6 +41,12 @@ const (
ForceFlag = "force" ForceFlag = "force"
ForceFlagShorthand = "f" ForceFlagShorthand = "f"
CIDFlag = "cid"
CIDFlagUsage = "Container ID."
OIDFlag = "oid"
OIDFlagUsage = "Object ID."
) )
// Init adds common flags to the command: // Init adds common flags to the command:

View file

@ -8,8 +8,8 @@ import (
const SessionToken = "session" const SessionToken = "session"
// InitSession registers SessionToken flag representing filepath to the token // InitSession registers SessionToken flag representing file path to the token of
// of the session with the given name. Supports NeoFS-binary and JSON files. // the session with the given name. Supports FrostFS-binary and JSON files.
func InitSession(cmd *cobra.Command, name string) { func InitSession(cmd *cobra.Command, name string) {
cmd.Flags().String( cmd.Flags().String(
SessionToken, SessionToken,

View file

@ -7,15 +7,22 @@ import (
"path/filepath" "path/filepath"
"testing" "testing"
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/internal/commonflags"
"github.com/nspcc-dev/neo-go/cli/input" "github.com/nspcc-dev/neo-go/cli/input"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/nspcc-dev/neo-go/pkg/wallet" "github.com/nspcc-dev/neo-go/pkg/wallet"
"github.com/nspcc-dev/neofs-node/cmd/neofs-cli/internal/commonflags" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"golang.org/x/term" "golang.org/x/term"
) )
var testCmd = &cobra.Command{
Use: "test",
Short: "test",
Run: func(cmd *cobra.Command, args []string) {},
}
func Test_getOrGenerate(t *testing.T) { func Test_getOrGenerate(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
@ -96,7 +103,7 @@ func Test_getOrGenerate(t *testing.T) {
t.Run("generate", func(t *testing.T) { t.Run("generate", func(t *testing.T) {
viper.Set(commonflags.GenerateKey, true) viper.Set(commonflags.GenerateKey, true)
actual, err := getOrGenerate() actual, err := getOrGenerate(testCmd)
require.NoError(t, err) require.NoError(t, err)
require.NotNil(t, actual) require.NotNil(t, actual)
for _, p := range []*keys.PrivateKey{nep2Key, rawKey, wifKey, acc1.PrivateKey(), acc2.PrivateKey()} { for _, p := range []*keys.PrivateKey{nep2Key, rawKey, wifKey, acc1.PrivateKey(), acc2.PrivateKey()} {
@ -107,13 +114,13 @@ func Test_getOrGenerate(t *testing.T) {
func checkKeyError(t *testing.T, desc string, err error) { func checkKeyError(t *testing.T, desc string, err error) {
viper.Set(commonflags.WalletPath, desc) viper.Set(commonflags.WalletPath, desc)
_, actualErr := getOrGenerate() _, actualErr := getOrGenerate(testCmd)
require.ErrorIs(t, actualErr, err) require.ErrorIs(t, actualErr, err)
} }
func checkKey(t *testing.T, desc string, expected *keys.PrivateKey) { func checkKey(t *testing.T, desc string, expected *keys.PrivateKey) {
viper.Set(commonflags.WalletPath, desc) viper.Set(commonflags.WalletPath, desc)
actual, err := getOrGenerate() actual, err := getOrGenerate(testCmd)
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, &expected.PrivateKey, actual) require.Equal(t, &expected.PrivateKey, actual)
} }

View file

@ -6,10 +6,10 @@ import (
"fmt" "fmt"
"os" "os"
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/internal/commonflags"
commonCmd "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/internal/common"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/nspcc-dev/neo-go/pkg/wallet" "github.com/nspcc-dev/neo-go/pkg/wallet"
"github.com/nspcc-dev/neofs-node/cmd/neofs-cli/internal/common"
"github.com/nspcc-dev/neofs-node/cmd/neofs-cli/internal/commonflags"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
) )
@ -20,12 +20,12 @@ var errCantGenerateKey = errors.New("can't generate new private key")
// Ideally we want to touch file-system on the last step. // Ideally we want to touch file-system on the last step.
// This function assumes that all flags were bind to viper in a `PersistentPreRun`. // This function assumes that all flags were bind to viper in a `PersistentPreRun`.
func Get(cmd *cobra.Command) *ecdsa.PrivateKey { func Get(cmd *cobra.Command) *ecdsa.PrivateKey {
pk, err := get() pk, err := get(cmd)
common.ExitOnErr(cmd, "can't fetch private key: %w", err) commonCmd.ExitOnErr(cmd, "can't fetch private key: %w", err)
return pk return pk
} }
func get() (*ecdsa.PrivateKey, error) { func get(cmd *cobra.Command) (*ecdsa.PrivateKey, error) {
keyDesc := viper.GetString(commonflags.WalletPath) keyDesc := viper.GetString(commonflags.WalletPath)
data, err := os.ReadFile(keyDesc) data, err := os.ReadFile(keyDesc)
if err != nil { if err != nil {
@ -36,7 +36,7 @@ func get() (*ecdsa.PrivateKey, error) {
if err != nil { if err != nil {
w, err := wallet.NewWalletFromFile(keyDesc) w, err := wallet.NewWalletFromFile(keyDesc)
if err == nil { if err == nil {
return FromWallet(w, viper.GetString(commonflags.Account)) return FromWallet(cmd, w, viper.GetString(commonflags.Account))
} }
return nil, fmt.Errorf("%w: %v", ErrInvalidKey, err) return nil, fmt.Errorf("%w: %v", ErrInvalidKey, err)
} }
@ -45,12 +45,12 @@ func get() (*ecdsa.PrivateKey, error) {
// GetOrGenerate is similar to get but generates a new key if commonflags.GenerateKey is set. // GetOrGenerate is similar to get but generates a new key if commonflags.GenerateKey is set.
func GetOrGenerate(cmd *cobra.Command) *ecdsa.PrivateKey { func GetOrGenerate(cmd *cobra.Command) *ecdsa.PrivateKey {
pk, err := getOrGenerate() pk, err := getOrGenerate(cmd)
common.ExitOnErr(cmd, "can't fetch private key: %w", err) commonCmd.ExitOnErr(cmd, "can't fetch private key: %w", err)
return pk return pk
} }
func getOrGenerate() (*ecdsa.PrivateKey, error) { func getOrGenerate(cmd *cobra.Command) (*ecdsa.PrivateKey, error) {
if viper.GetBool(commonflags.GenerateKey) { if viper.GetBool(commonflags.GenerateKey) {
priv, err := keys.NewPrivateKey() priv, err := keys.NewPrivateKey()
if err != nil { if err != nil {
@ -58,5 +58,5 @@ func getOrGenerate() (*ecdsa.PrivateKey, error) {
} }
return &priv.PrivateKey, nil return &priv.PrivateKey, nil
} }
return get() return get(cmd)
} }

View file

@ -3,13 +3,14 @@ package key
import ( import (
"crypto/ecdsa" "crypto/ecdsa"
"errors" "errors"
"fmt"
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/internal/common"
"github.com/nspcc-dev/neo-go/cli/flags" "github.com/nspcc-dev/neo-go/cli/flags"
"github.com/nspcc-dev/neo-go/cli/input" "github.com/nspcc-dev/neo-go/cli/input"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/nspcc-dev/neo-go/pkg/util" "github.com/nspcc-dev/neo-go/pkg/util"
"github.com/nspcc-dev/neo-go/pkg/wallet" "github.com/nspcc-dev/neo-go/pkg/wallet"
"github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
) )
@ -22,37 +23,37 @@ var (
) )
// FromWallet returns private key of the wallet account. // FromWallet returns private key of the wallet account.
func FromWallet(w *wallet.Wallet, addrStr string) (*ecdsa.PrivateKey, error) { func FromWallet(cmd *cobra.Command, w *wallet.Wallet, addrStr string) (*ecdsa.PrivateKey, error) {
var ( var (
addr util.Uint160 addr util.Uint160
err error err error
) )
if addrStr == "" { if addrStr == "" {
printVerbose("Using default wallet address") common.PrintVerbose(cmd, "Using default wallet address")
addr = w.GetChangeAddress() addr = w.GetChangeAddress()
} else { } else {
addr, err = flags.ParseAddress(addrStr) addr, err = flags.ParseAddress(addrStr)
if err != nil { if err != nil {
printVerbose("Can't parse address: %s", addrStr) common.PrintVerbose(cmd, "Can't parse address: %s", addrStr)
return nil, ErrInvalidAddress return nil, ErrInvalidAddress
} }
} }
acc := w.GetAccount(addr) acc := w.GetAccount(addr)
if acc == nil { if acc == nil {
printVerbose("Can't find wallet account for %s", addrStr) common.PrintVerbose(cmd, "Can't find wallet account for %s", addrStr)
return nil, ErrInvalidAddress return nil, ErrInvalidAddress
} }
pass, err := getPassword() pass, err := getPassword()
if err != nil { if err != nil {
printVerbose("Can't read password: %v", err) common.PrintVerbose(cmd, "Can't read password: %v", err)
return nil, ErrInvalidPassword return nil, ErrInvalidPassword
} }
if err := acc.Decrypt(pass, keys.NEP2ScryptParams()); err != nil { if err := acc.Decrypt(pass, keys.NEP2ScryptParams()); err != nil {
printVerbose("Can't decrypt account: %v", err) common.PrintVerbose(cmd, "Can't decrypt account: %v", err)
return nil, ErrInvalidPassword return nil, ErrInvalidPassword
} }
@ -67,9 +68,3 @@ func getPassword() (string, error) {
return input.ReadPassword("Enter password > ") return input.ReadPassword("Enter password > ")
} }
func printVerbose(format string, a ...interface{}) {
if viper.GetBool("verbose") {
fmt.Printf(format+"\n", a...)
}
}

7
cmd/frostfs-cli/main.go Normal file
View file

@ -0,0 +1,7 @@
package main
import cmd "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/modules"
func main() {
cmd.Execute()
}

View file

@ -3,14 +3,14 @@ package accounting
import ( import (
"math/big" "math/big"
internalclient "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/internal/client"
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/internal/commonflags"
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/internal/key"
commonCmd "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/internal/common"
"git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/precision"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/accounting"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
"github.com/nspcc-dev/neo-go/pkg/encoding/fixedn" "github.com/nspcc-dev/neo-go/pkg/encoding/fixedn"
internalclient "github.com/nspcc-dev/neofs-node/cmd/neofs-cli/internal/client"
"github.com/nspcc-dev/neofs-node/cmd/neofs-cli/internal/common"
"github.com/nspcc-dev/neofs-node/cmd/neofs-cli/internal/commonflags"
"github.com/nspcc-dev/neofs-node/cmd/neofs-cli/internal/key"
"github.com/nspcc-dev/neofs-node/pkg/util/precision"
"github.com/nspcc-dev/neofs-sdk-go/accounting"
"github.com/nspcc-dev/neofs-sdk-go/user"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
) )
@ -21,8 +21,8 @@ const (
var accountingBalanceCmd = &cobra.Command{ var accountingBalanceCmd = &cobra.Command{
Use: "balance", Use: "balance",
Short: "Get internal balance of NeoFS account", Short: "Get internal balance of FrostFS account",
Long: `Get internal balance of NeoFS account`, Long: `Get internal balance of FrostFS account`,
Run: func(cmd *cobra.Command, args []string) { Run: func(cmd *cobra.Command, args []string) {
var idUser user.ID var idUser user.ID
@ -32,7 +32,7 @@ var accountingBalanceCmd = &cobra.Command{
if balanceOwner == "" { if balanceOwner == "" {
user.IDFromKey(&idUser, pk.PublicKey) user.IDFromKey(&idUser, pk.PublicKey)
} else { } else {
common.ExitOnErr(cmd, "can't decode owner ID wallet address: %w", idUser.DecodeString(balanceOwner)) commonCmd.ExitOnErr(cmd, "can't decode owner ID wallet address: %w", idUser.DecodeString(balanceOwner))
} }
cli := internalclient.GetSDKClientByFlag(cmd, pk, commonflags.RPC) cli := internalclient.GetSDKClientByFlag(cmd, pk, commonflags.RPC)
@ -42,7 +42,7 @@ var accountingBalanceCmd = &cobra.Command{
prm.SetAccount(idUser) prm.SetAccount(idUser)
res, err := internalclient.BalanceOf(prm) res, err := internalclient.BalanceOf(prm)
common.ExitOnErr(cmd, "rpc error: %w", err) commonCmd.ExitOnErr(cmd, "rpc error: %w", err)
// print to stdout // print to stdout
prettyPrintDecimal(cmd, res.Balance()) prettyPrintDecimal(cmd, res.Balance())

View file

@ -1,7 +1,7 @@
package accounting package accounting
import ( import (
"github.com/nspcc-dev/neofs-node/cmd/neofs-cli/internal/commonflags" "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/internal/commonflags"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/spf13/viper" "github.com/spf13/viper"
) )

View file

@ -0,0 +1,28 @@
package basic
import (
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/modules/util"
commonCmd "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/internal/common"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/acl"
"github.com/spf13/cobra"
)
var printACLCmd = &cobra.Command{
Use: "print",
Short: "Pretty print basic ACL from the HEX representation",
Example: `frostfs-cli acl basic print 0x1C8C8CCC`,
Long: `Pretty print basic ACL from the HEX representation.
Few roles have exclusive default access to set of operation, even if particular bit deny it.
Container have access to the operations of the data replication mechanism:
Get, Head, Put, Search, Hash.
InnerRing members are allowed to data audit ops only:
Get, Head, Hash, Search.`,
Run: printACL,
Args: cobra.ExactArgs(1),
}
func printACL(cmd *cobra.Command, args []string) {
var bacl acl.Basic
commonCmd.ExitOnErr(cmd, "unable to parse basic acl: %w", bacl.DecodeString(args[0]))
util.PrettyPrintTableBACL(cmd, &bacl)
}

View file

@ -0,0 +1,14 @@
package basic
import (
"github.com/spf13/cobra"
)
var Cmd = &cobra.Command{
Use: "basic",
Short: "Operations with Basic Access Control Lists",
}
func init() {
Cmd.AddCommand(printACLCmd)
}

View file

@ -0,0 +1,127 @@
package extended
import (
"bytes"
"encoding/json"
"os"
"strings"
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/internal/commonflags"
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/modules/util"
commonCmd "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/internal/common"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl"
"github.com/spf13/cobra"
)
var createCmd = &cobra.Command{
Use: "create",
Short: "Create extended ACL from the text representation",
Long: `Create extended ACL from the text representation.
Rule consist of these blocks: <action> <operation> [<filter1> ...] [<target1> ...]
Action is 'allow' or 'deny'.
Operation is an object service verb: 'get', 'head', 'put', 'search', 'delete', 'getrange', or 'getrangehash'.
Filter consists of <typ>:<key><match><value>
Typ is 'obj' for object applied filter or 'req' for request applied filter.
Key is a valid unicode string corresponding to object or request header key.
Well-known system object headers start with '$Object:' prefix.
User defined headers start without prefix.
Read more about filter keys at git.frostfs.info.com/TrueCloudLab/frostfs-api/src/branch/master/proto-docs/acl.md#message-eaclrecordfilter
Match is '=' for matching and '!=' for non-matching filter.
Value is a valid unicode string corresponding to object or request header value.
Target is
'user' for container owner,
'system' for Storage nodes in container and Inner Ring nodes,
'others' for all other request senders,
'pubkey:<key1>,<key2>,...' for exact request sender, where <key> is a hex-encoded 33-byte public key.
When both '--rule' and '--file' arguments are used, '--rule' records will be placed higher in resulting extended ACL table.
`,
Example: `frostfs-cli acl extended create --cid EutHBsdT1YCzHxjCfQHnLPL1vFrkSyLSio4vkphfnEk -f rules.txt --out table.json
frostfs-cli acl extended create --cid EutHBsdT1YCzHxjCfQHnLPL1vFrkSyLSio4vkphfnEk -r 'allow get obj:Key=Value others' -r 'deny put others'`,
Run: createEACL,
}
func init() {
createCmd.Flags().StringArrayP("rule", "r", nil, "Extended ACL table record to apply")
createCmd.Flags().StringP("file", "f", "", "Read list of extended ACL table records from text file")
createCmd.Flags().StringP("out", "o", "", "Save JSON formatted extended ACL table in file")
createCmd.Flags().StringP(commonflags.CIDFlag, "", "", commonflags.CIDFlagUsage)
_ = cobra.MarkFlagFilename(createCmd.Flags(), "file")
_ = cobra.MarkFlagFilename(createCmd.Flags(), "out")
}
func createEACL(cmd *cobra.Command, _ []string) {
rules, _ := cmd.Flags().GetStringArray("rule")
fileArg, _ := cmd.Flags().GetString("file")
outArg, _ := cmd.Flags().GetString("out")
cidArg, _ := cmd.Flags().GetString(commonflags.CIDFlag)
var containerID cid.ID
if cidArg != "" {
if err := containerID.DecodeString(cidArg); err != nil {
cmd.PrintErrf("invalid container ID: %v\n", err)
os.Exit(1)
}
}
rulesFile, err := getRulesFromFile(fileArg)
if err != nil {
cmd.PrintErrf("can't read rules from file: %v\n", err)
os.Exit(1)
}
rules = append(rules, rulesFile...)
if len(rules) == 0 {
cmd.PrintErrln("no extended ACL rules has been provided")
os.Exit(1)
}
tb := eacl.NewTable()
commonCmd.ExitOnErr(cmd, "unable to parse provided rules: %w", util.ParseEACLRules(tb, rules))
tb.SetCID(containerID)
data, err := tb.MarshalJSON()
if err != nil {
cmd.PrintErrln(err)
os.Exit(1)
}
buf := new(bytes.Buffer)
err = json.Indent(buf, data, "", " ")
if err != nil {
cmd.PrintErrln(err)
os.Exit(1)
}
if len(outArg) == 0 {
cmd.Println(buf)
return
}
err = os.WriteFile(outArg, buf.Bytes(), 0644)
if err != nil {
cmd.PrintErrln(err)
os.Exit(1)
}
}
func getRulesFromFile(filename string) ([]string, error) {
if len(filename) == 0 {
return nil, nil
}
data, err := os.ReadFile(filename)
if err != nil {
return nil, err
}
return strings.Split(strings.TrimSpace(string(data)), "\n"), nil
}

View file

@ -1,10 +1,10 @@
package extended package extended
import ( import (
"strings"
"testing" "testing"
"github.com/nspcc-dev/neofs-sdk-go/eacl" "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/modules/util"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
@ -63,8 +63,7 @@ func TestParseTable(t *testing.T) {
for _, test := range tests { for _, test := range tests {
t.Run(test.name, func(t *testing.T) { t.Run(test.name, func(t *testing.T) {
ss := strings.Split(test.rule, " ") err := util.ParseEACLRule(eaclTable, test.rule)
err := parseTable(eaclTable, ss)
ok := len(test.jsonRecord) > 0 ok := len(test.jsonRecord) > 0
require.Equal(t, ok, err == nil, err) require.Equal(t, ok, err == nil, err)
if ok { if ok {

View file

@ -0,0 +1,38 @@
package extended
import (
"os"
"strings"
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/modules/util"
commonCmd "git.frostfs.info/TrueCloudLab/frostfs-node/cmd/internal/common"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl"
"github.com/spf13/cobra"
)
var printEACLCmd = &cobra.Command{
Use: "print",
Short: "Pretty print extended ACL from the file(in text or json format) or for given container.",
Run: printEACL,
}
func init() {
flags := printEACLCmd.Flags()
flags.StringP("file", "f", "",
"Read list of extended ACL table records from text or json file")
_ = printEACLCmd.MarkFlagRequired("file")
}
func printEACL(cmd *cobra.Command, _ []string) {
file, _ := cmd.Flags().GetString("file")
eaclTable := new(eacl.Table)
data, err := os.ReadFile(file)
commonCmd.ExitOnErr(cmd, "can't read file with EACL: %w", err)
if strings.HasSuffix(file, ".json") {
commonCmd.ExitOnErr(cmd, "unable to parse json: %w", eaclTable.UnmarshalJSON(data))
} else {
rules := strings.Split(strings.TrimSpace(string(data)), "\n")
commonCmd.ExitOnErr(cmd, "can't parse file with EACL: %w", util.ParseEACLRules(eaclTable, rules))
}
util.PrettyPrintTableEACL(cmd, eaclTable)
}

View file

@ -1,6 +1,8 @@
package extended package extended
import "github.com/spf13/cobra" import (
"github.com/spf13/cobra"
)
var Cmd = &cobra.Command{ var Cmd = &cobra.Command{
Use: "extended", Use: "extended",
@ -9,4 +11,5 @@ var Cmd = &cobra.Command{
func init() { func init() {
Cmd.AddCommand(createCmd) Cmd.AddCommand(createCmd)
Cmd.AddCommand(printEACLCmd)
} }

View file

@ -0,0 +1,17 @@
package acl
import (
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/modules/acl/basic"
"git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-cli/modules/acl/extended"
"github.com/spf13/cobra"
)
var Cmd = &cobra.Command{
Use: "acl",
Short: "Operations with Access Control Lists",
}
func init() {
Cmd.AddCommand(extended.Cmd)
Cmd.AddCommand(basic.Cmd)
}

Some files were not shown because too many files have changed in this diff Show more