Commit graph

53 commits

Author SHA1 Message Date
Ekaterina Pavlova
616c805ac5 cli: refactor index file search in upload-bin command
Align index file searching logic with block search.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-22 17:45:35 +03:00
Ekaterina Pavlova
5dab154582 cli: split searchBatchSize usage in upload-bin command
Split searchBatchSize into constant for searching and uploading.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-22 17:45:35 +03:00
Ekaterina Pavlova
9fd375d56d cli: fix nil error of fetchLatestMissingBlockIndex in upload-bin
Close #3653

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-22 17:45:35 +03:00
Ekaterina Pavlova
5f6284de05 cli: fix skip-blocks-uploading mode of upload-bin command
If uploadBlocks succeeded uploadIndexFiles should work based on the
currentBlockHeight.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-15 17:28:24 +03:00
Ekaterina Pavlova
58ed448f8d cli: add debug mode to upload-bin command
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-15 17:18:44 +03:00
Ekaterina Pavlova
375b095f78 cli: add details to the logs in upload-bin
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-15 14:34:36 +03:00
Ekaterina Pavlova
380d112599 cli: fix stopping of uploadIndexFiles in upload-bin
The expected count of index files should be counted based on the already
uploaded object number.

Close #3669

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-15 14:34:36 +03:00
Ekaterina Pavlova
6e863e9a06 cli: fix fetchLatestMissingBlockIndex in upload-bin command
Close #3667

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-15 14:34:10 +03:00
Ekaterina Pavlova
29e2d712f4 cli: add flag max-retries for upload-bin
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-06 11:18:41 +03:00
Ekaterina Pavlova
18ed3bb3e8 cli: fix missing errors on closure of writer in upload-bin
Ref #3658

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-06 11:16:46 +03:00
Anna Shaleva
a85f3ce83f cli: rename uploader.go to upload_bin.go
Follow the style of other files in the `util` package and existing
`dump_bin.go`.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2024-11-05 12:43:20 +03:00
Anna Shaleva
22cb7feb1d cli: fix format
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2024-11-05 11:25:08 +03:00
Anna Shaleva
82b508e8f3 cli: extend logs of index file construction for upload-bin
These logs are needed to track the progress of uploaders.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2024-11-05 11:24:57 +03:00
Ekaterina Pavlova
c955c1e4ea cli: use GET instead of HEAD in upload bin
Will be reverted when nspcc-dev/neofs-node#2988 is fixed. Currently HEAD
request can return incorrect attributes, which can lead to incorrect
index files.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-02 12:04:02 +03:00
Ekaterina Pavlova
35d12779d6 cli: adjust index file creation in upload-bin
In case of incomplete search result it will try to find each missed oid
and process it. In case of duplicates the first found will be in index
file.

Close #3647

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-01 19:51:01 +03:00
Ekaterina Pavlova
365bbe08ed cli: refactor errors and comments in upload-bin
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-11-01 14:58:48 +03:00
Ekaterina Pavlova
6a93f70728 cli: add flag for skipping block uploading in upload-bin
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-10-31 18:39:40 +03:00
Ekaterina Pavlova
7b3eeb9061 cli: fix usage of upload-bin
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-10-24 12:19:12 +03:00
Ekaterina Pavlova
36e855609d cli: add workers and searchers flags to upload-bin command
Number of workers to fetch, upload and search blocks concurrently is now
configurable.

Close #3641

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-10-24 12:19:10 +03:00
Ekaterina Pavlova
a5e9ab6979 cli: verify index file construction in upload-bin command
Verify that there are no empty OIDs in the constructed index file, as
empty payloads can result in improperly set attributes, leading to empty
OIDs being added to the index file.

Close #3628

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-10-23 15:30:12 +03:00
Ekaterina Pavlova
42c8e40eaa cli: add retry to all requests to NeoFS in upload-bin
Add retry to `NetworkInfo` and `uploadObj`.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-10-23 15:30:12 +03:00
Ekaterina Pavlova
8b43c33e44 cli: extend object attribute parsing error in upload-bin
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-10-23 15:30:12 +03:00
Ekaterina Pavlova
5b793bcf1b cli: fix process termination in upload-bin command
Add a return statement to properly handle errors, ensuring the process
terminates as expected.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-10-23 15:30:12 +03:00
Ekaterina Pavlova
e83b3e4839 cli: fix error handling in upload-bin command
Fix shared global error reuse.

Close #3634

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-10-23 15:30:12 +03:00
Ekaterina Pavlova
c47d4e6c5b BlockFetcher/BlockUploader: fix and add additional attributes
Capitalize block objects and index file objects attributes.
Add network magic attribute to container.

Close #3631

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-10-23 13:52:06 +03:00
Ekaterina Pavlova
e00d7fec7f cli: fix empty block uploading in util upload-bin
In case of uploading retry already drained bin buffer is returned in
`bw.Bytes`, which leads to empty block uploading.

Close #3630

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-10-21 22:44:06 +03:00
Ekaterina Pavlova
41873e35c6 cli: increase NeoFS pool deadlines for upload-bin command
`defaultDialTimeout` and `defaultStreamTimeout` increased because of
`code = DeadlineExceeded desc = context deadline exceeded`

Close #3620

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-10-18 19:59:27 +03:00
Ekaterina Pavlova
87bbff9831 cli: adjust error of upload-bin command
Miscalculation leads to negative values in errors:
```
2024-10-17 11:26:56.790	failed to fetch the latest missing block index
 from container: search of index files failed for batch with indexes
 from -260000 to -250000: error during object IDs iteration: context
 deadline exceeded
```

Close #3621

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-10-18 14:26:17 +03:00
Ekaterina Pavlova
6199240598 cli: add upload-bin
This command is used for keeping container with blocks for
blockfetcher updated.

Close #3578

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-10-17 11:56:07 +03:00
Roman Khimov
1c1d77c9b8 *: make use of min/max where appropriate
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2024-08-23 22:11:29 +03:00
Ekaterina Pavlova
fc2798f9d1 cli: fix canceltx usage text
We do not have `--account` flag.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-07-12 12:16:24 +03:00
Ekaterina Pavlova
39559b90e2 cli: add Required field for flags
Close #2861

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-07-12 11:58:12 +03:00
Ekaterina Pavlova
acde7bd0de cli: upgrade urfave lib to v2
Close #3097

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-07-12 11:58:12 +03:00
Ekaterina Pavlova
2f5c26f14b cli: use capital letter for Usage
To unified cli help only capital letter is used in Usage field.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-07-12 11:58:12 +03:00
Anna Shaleva
69bcd4b7d9 cli: unify commands description
Always put blank line in the end of the command description.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2024-04-10 19:21:40 +03:00
Anna Shaleva
b6a9c64c55 cli: ensure all outcomes are handled in TestAwaitUtilCancelTx
Close #3365.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2024-04-03 16:40:11 +03:00
Anna Shaleva
e0e7fdf810 cli: unify canceltx command output
Always return "Target transaction accepted" error if the target transation was
either accepted by the moment of command run or during the command
handling. Exit with code 1 in both cases. Adjust TestAwaitUtilCancelTx
correspondingly, allow both cases in test. Simplify the test along the
way, remove useless code.

Close #3365.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2024-04-02 15:23:25 +03:00
Ekaterina Pavlova
d80e14dcbb cli: fix TestAwaitUtilCancelTx failing
Occasionally the block is being accepted right after main transaction
submission. Added two branches into this TestAwaitUtilCancelTx. One
branch handles the case of original transaction acceptance, the
other branch handles the conflicting transaction acceptance.

Close #3365

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2024-03-22 16:22:11 +03:00
Ekaterina Pavlova
0ffa24932b cli: add await flag for operations with transactions
New --await flag is an option to synchronize on transaction execution
for CLI commands.

Closes #3244

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2023-12-29 15:14:16 +03:00
Ekaterina Pavlova
4f5e3f363a cli: fix canceltx ValidUntilBlock parameter of conflicting transaction
If main transaction is known, then conflicting transaction shouldn't be
valid longer than the main one.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2023-12-28 15:30:15 +03:00
Ekaterina Pavlova
8bdbac3c5e cli: move actor-related functions handling to options package
Move actor handling into CLI package.
GetRPCWithActor returns RPC with actor instances.

Close #2682.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2023-12-07 10:35:37 +03:00
Ekaterina Pavlova
d7cab3b82c cli: move wallet-related flags handling to options package
Move GetAccFromContext, GetUnlockedAccount, ReadWalletConfig handling
into options package to reuse this code from all CLI handlers.
getDecryptedAccount is replaced by GetUnlockedAccount.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2023-12-06 10:49:50 +03:00
Ekaterina Pavlova
fc77754098 cli: cancel transaction command
CLI side method for canceling not yet accepted transaction. It's
alternative to unsupported `canceltransaction` RPC method.

Close #3151.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
2023-11-27 18:10:54 +04:00
Anna Shaleva
e06051e14c cli: use fmt.Append* where possibble
Close #2629.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-10-12 18:07:54 +03:00
Anna Shaleva
541da7e83b cli: add util ops command
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
2023-04-12 17:12:18 +03:00
Anna Shaleva
0b717b0c22 vm: move vm CLI to cli/vm package 2022-10-07 15:56:34 +03:00
Roman Khimov
1ac60ada19 cli: move tests to subpackages
Refs. #2379, but not completely solves it, one package seriously outweights
others:

?       github.com/nspcc-dev/neo-go/cli [no test files]
ok      github.com/nspcc-dev/neo-go/cli/app     0.036s  coverage: 100.0% of statements
ok      github.com/nspcc-dev/neo-go/cli/cmdargs 0.011s  coverage: 60.8% of statements
ok      github.com/nspcc-dev/neo-go/cli/flags   0.009s  coverage: 97.7% of statements
?       github.com/nspcc-dev/neo-go/cli/input   [no test files]
ok      github.com/nspcc-dev/neo-go/cli/options 0.033s  coverage: 50.0% of statements
?       github.com/nspcc-dev/neo-go/cli/paramcontext    [no test files]
ok      github.com/nspcc-dev/neo-go/cli/query   2.155s  coverage: 45.3% of statements
ok      github.com/nspcc-dev/neo-go/cli/server  1.373s  coverage: 67.8% of statements
ok      github.com/nspcc-dev/neo-go/cli/smartcontract   8.819s  coverage: 94.3% of statements
ok      github.com/nspcc-dev/neo-go/cli/util    0.006s  coverage: 10.9% of statements
?       github.com/nspcc-dev/neo-go/cli/vm      [no test files]
ok      github.com/nspcc-dev/neo-go/cli/wallet  72.103s coverage: 88.2% of statements

Still a nice thing to have.
2022-10-06 09:21:26 +03:00
Roman Khimov
411ebdf51e cli: add complete support for offline signing, fix #2662
See documentation update for an example. Some code is made generic as well,
GetCompleteTransaction can now be used directly on ParameterContext.
2022-09-01 15:30:34 +03:00
Roman Khimov
1518019be8 cli: add excessive arguments checks
Some commands don't accept arguments, but users try giving them and don't
notice a mistake. It's a bit more user-friendly to tell the user that there is
something wrong with the way he tries to use the command.
2022-08-05 15:50:12 +03:00
Evgeniy Stratonikov
bf2ca35453 cli/wallet: allow to testinvoke transaction before signing
Signed-off-by: Evgeniy Stratonikov <evgeniy@nspcc.ru>
2021-09-23 11:48:25 +03:00