TrueCloudLab/neo-go
8
0
Fork 0
mirror of https://github.com/nspcc-dev/neo-go.git synced 2024-11-22 09:29:38 +00:00
Code Issues Projects Releases Packages Wiki Activity
7954 commits 41 branches 136 tags 43 MiB
Commit graph

7954 commits

This branch
This branch
All branches
Author SHA1 Message Date
Ekaterina Pavlova
29e2d712f4 cli: add flag max-retries for upload-bin 
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-11-06 11:18:41 +03:00
Ekaterina Pavlova
18ed3bb3e8 cli: fix missing errors on closure of writer in upload-bin 
Ref #3658

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-11-06 11:16:46 +03:00
Anna Shaleva
1a540d5883
Merge pull request #3663 from nspcc-dev/io-improve-bytes-doc 
io: specify Bytes() behavior a bit better for BufBinWriter
 2024-11-06 11:15:52 +03:00
Anna Shaleva
8c4d9432d7
Merge pull request #3662 from nspcc-dev/improve-uploadbin-log 
cli: extend logs of index file construction for `upload-bin`
 2024-11-06 11:13:44 +03:00
Anna Shaleva
b6516586d5
Merge pull request #3661 from alexey-savchuk/fix-panic-on-sighup 
services/rpcsrv: Return a new server by pointer
 2024-11-06 11:04:44 +03:00
Roman Khimov
23f9c5a43b io: specify Bytes() behavior a bit better for BufBinWriter 
It's not immediately obvious.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
 2024-11-05 16:58:33 +03:00
Alexey Savchuk
df9247c00b services/rpcsrv: Return a new server by pointer 
Before, a new server was returned by value which could cause
a panic `unlock of unlocked mutex` on SIGHUP handling. It's
because the new server overwrites a locked mutex of the already
existing server.

oct‚ 22 13:51:15 node1 neo-go[1183338]: fatal error: sync: Unlock of unlocked RWMutex
oct‚ 22 13:51:15 node1 neo-go[1183338]: goroutine 538 [running]:
oct‚ 22 13:51:15 node1 neo-go[1183338]: sync.fatal({0xf83d64?, 0xc001085880?})
oct‚ 22 13:51:15 node1 neo-go[1183338]:         runtime/panic.go:1007 +0x18
oct‚ 22 13:51:15 node1 neo-go[1183338]: sync.(*RWMutex).Unlock(0xc00019a4c8)
oct‚ 22 13:51:15 node1 neo-go[1183338]:         sync/rwmutex.go:208 +0x45
oct‚ 22 13:51:15 node1 neo-go[1183338]: github.com/nspcc-dev/neo-go/pkg/services/rpcsrv.(*Server).dropSubscriber(0xc00019a2c8, 0xc000a77740)
oct‚ 22 13:51:15 node1 neo-go[1183338]:         github.com/nspcc-dev/neo-go/pkg/services/rpcsrv/server.go:825 +0xce
oct‚ 22 13:51:15 node1 neo-go[1183338]: github.com/nspcc-dev/neo-go/pkg/services/rpcsrv.(*Server).handleWsReads(0xc00019a2c8, 0xc0034478c0, 0xc000af5f80, 0xc000a7
7740)
oct‚ 22 13:51:15 node1 neo-go[1183338]:         github.com/nspcc-dev/neo-go/pkg/services/rpcsrv/server.go:810 +0x266
oct‚ 22 13:51:15 node1 neo-go[1183338]: github.com/nspcc-dev/neo-go/pkg/services/rpcsrv.(*Server).handleHTTPRequest(0xc00019a2c8, {0x11c3900, 0xc003437dc0}, 0xc00
31945a0)
oct‚ 22 13:51:15 node1 neo-go[1183338]:         github.com/nspcc-dev/neo-go/pkg/services/rpcsrv/server.go:582 +0x54a
oct‚ 22 13:51:15 node1 neo-go[1183338]: net/http.HandlerFunc.ServeHTTP(0x471779?, {0x11c3900?, 0xc003437dc0?}, 0xc000943b68?)
oct‚ 22 13:51:15 node1 neo-go[1183338]:         net/http/server.go:2171 +0x29
oct‚ 22 13:51:15 node1 neo-go[1183338]: net/http.serverHandler.ServeHTTP({0xc000a77680?}, {0x11c3900?, 0xc003437dc0?}, 0x6?)
oct‚ 22 13:51:15 node1 neo-go[1183338]:         net/http/server.go:3142 +0x8e
oct‚ 22 13:51:15 node1 neo-go[1183338]: net/http.(*conn).serve(0xc0032030e0, {0x11c5220, 0xc000a76960})
oct‚ 22 13:51:15 node1 neo-go[1183338]:         net/http/server.go:2044 +0x5e8
oct‚ 22 13:51:15 node1 neo-go[1183338]: created by net/http.(*Server).Serve in goroutine 534
oct‚ 22 13:51:15 node1 neo-go[1183338]:         net/http/server.go:3290 +0x4b4

Signed-off-by: Alexey Savchuk <alexey.a.savchuk@yandex.com>
 2024-11-05 15:05:25 +03:00
Anna Shaleva
cee296eb92
Merge pull request #3659 from nspcc-dev/make-balance-work-without-wallet 
cli: make nep1X balance commands work without a wallet, fix #3275
 2024-11-05 13:24:20 +03:00
Roman Khimov
ddaf9c01ab cli: make nep1X balance commands work without a wallet, fix #3275 
Checking the balance should be easy without any wallets involved, the data
is available anyway.

Tests are extended a bit as well.

Adding the command to "query nep1X" is not so trivial, so let's have it this
way for now.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
 2024-11-05 13:12:13 +03:00
Anna Shaleva
a85f3ce83f cli: rename uploader.go to upload_bin.go 
Follow the style of other files in the `util` package and existing
`dump_bin.go`.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
 2024-11-05 12:43:20 +03:00
Anna Shaleva
22cb7feb1d cli: fix format 
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
 2024-11-05 11:25:08 +03:00
Anna Shaleva
82b508e8f3 cli: extend logs of index file construction for upload-bin 
These logs are needed to track the progress of uploaders.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
 2024-11-05 11:24:57 +03:00
Roman Khimov
95098d4b25
Merge pull request #3657 from nspcc-dev/drop-macos-12 
workflows: remove macos-12 support
 2024-11-02 13:30:21 +03:00
Anna Shaleva
5cffa90774 workflows: remove macos-12 support 
Support for macos-12 will be deprecated 10/7/24 and the image will
be fully unsupported by 12/3/24. Ref.
https://github.com/actions/runner-images/issues/10721.

Get rid of the build warning:
```
A brownout will take place on November 4, 14:00 UTC - November 5, 00:00
UTC to raise awareness of the upcoming macOS-12 environment removal.
```

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
 2024-11-02 13:08:12 +03:00
Anna Shaleva
21d6887969
Merge pull request #3650 from nspcc-dev/index-files 
cli: add retries if search results are not expected length in `upload-bin`
 2024-11-02 12:28:26 +03:00
Ekaterina Pavlova
c955c1e4ea cli: use GET instead of HEAD in upload bin 
Will be reverted when nspcc-dev/neofs-node#2988 is fixed. Currently HEAD
request can return incorrect attributes, which can lead to incorrect
index files.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-11-02 12:04:02 +03:00
Ekaterina Pavlova
35d12779d6 cli: adjust index file creation in upload-bin 
In case of incomplete search result it will try to find each missed oid
and process it. In case of duplicates the first found will be in index
file.

Close #3647

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-11-01 19:51:01 +03:00
Ekaterina Pavlova
365bbe08ed cli: refactor errors and comments in upload-bin 
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-11-01 14:58:48 +03:00
Ekaterina Pavlova
6a93f70728 cli: add flag for skipping block uploading in upload-bin 
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-31 18:39:40 +03:00
Anna Shaleva
a4633ce2c7
Merge pull request #3649 from nspcc-dev/vm-modpow 
vm: fix MODPOW operation
 2024-10-31 17:50:40 +03:00
Ekaterina Pavlova
cccbe843be vm: fix MODPOW operation 
Close #3612

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-31 17:49:02 +03:00
Anna Shaleva
e8b8c1a4b1
Merge pull request #3651 from nspcc-dev/tojson-test 
stackitem: extend ToJSON test
 2024-10-30 20:01:45 +03:00
Anna Shaleva
a0d2f95e42 stackitem: extend ToJSON test 
Inspired by https://github.com/neo-project/neo/pull/3558, although this
PR fixes different implementation of stackitem serializatior that is
used outside of VM.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
 2024-10-29 12:07:04 +03:00
Anna Shaleva
b8a65d3c37
Merge pull request #3643 from nspcc-dev/uploader-workers 
cli: add flag workers to `upload-bin` command
 2024-10-24 12:26:51 +03:00
Ekaterina Pavlova
7b3eeb9061 cli: fix usage of upload-bin 
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-24 12:19:12 +03:00
Ekaterina Pavlova
36e855609d cli: add workers and searchers flags to upload-bin command 
Number of workers to fetch, upload and search blocks concurrently is now
configurable.

Close #3641

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-24 12:19:10 +03:00
Anna Shaleva
4b10f23aec
Merge pull request #3638 from nspcc-dev/uploader-retry-errors 
cli: fix `upload-bin` error handling
 2024-10-23 16:00:38 +03:00
Ekaterina Pavlova
a5e9ab6979 cli: verify index file construction in upload-bin command 
Verify that there are no empty OIDs in the constructed index file, as
empty payloads can result in improperly set attributes, leading to empty
OIDs being added to the index file.

Close #3628

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-23 15:30:12 +03:00
Ekaterina Pavlova
42c8e40eaa cli: add retry to all requests to NeoFS in upload-bin 
Add retry to `NetworkInfo` and `uploadObj`.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-23 15:30:12 +03:00
Ekaterina Pavlova
8b43c33e44 cli: extend object attribute parsing error in upload-bin 
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-23 15:30:12 +03:00
Ekaterina Pavlova
5b793bcf1b cli: fix process termination in upload-bin command 
Add a return statement to properly handle errors, ensuring the process
terminates as expected.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-23 15:30:12 +03:00
Ekaterina Pavlova
e83b3e4839 cli: fix error handling in upload-bin command 
Fix shared global error reuse.

Close #3634

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-23 15:30:12 +03:00
Anna Shaleva
0b79901b7f
Merge pull request #3637 from nspcc-dev/attributes 
BlockFetcher/BlockUploader: fix and add additional attributes
 2024-10-23 15:07:09 +03:00
Ekaterina Pavlova
c47d4e6c5b BlockFetcher/BlockUploader: fix and add additional attributes 
Capitalize block objects and index file objects attributes.
Add network magic attribute to container.

Close #3631

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-23 13:52:06 +03:00
Anna Shaleva
e82d9a179c
Merge pull request #3636 from nspcc-dev/blockfetcher-errors 
blockfetcher: add more details to errors
 2024-10-22 16:29:28 +03:00
Ekaterina Pavlova
8a154d9189 blockfetcher: add more details to errors 
Close #3629

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-22 14:11:27 +03:00
Anna Shaleva
a0def2e5d2
Merge pull request #3633 from nspcc-dev/fix-empty-obj-uploading 
cli: fix empty block uploading in `util upload-bin`
 2024-10-22 10:12:17 +03:00
Ekaterina Pavlova
e00d7fec7f cli: fix empty block uploading in util upload-bin 
In case of uploading retry already drained bin buffer is returned in
`bw.Bytes`, which leads to empty block uploading.

Close #3630

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-21 22:44:06 +03:00
Roman Khimov
29bb3ff1cf
Merge pull request #3627 from nspcc-dev/fix-ntf 
core: prove contract notifications count is not restricted
 2024-10-18 21:16:56 +03:00
Anna Shaleva
c962edcf47
Merge pull request #3625 from nspcc-dev/uploader-timeouts 
cli: increase NeoFS pool deadlines for upload-bin command
 2024-10-18 20:02:24 +03:00
Ekaterina Pavlova
41873e35c6 cli: increase NeoFS pool deadlines for upload-bin command 
`defaultDialTimeout` and `defaultStreamTimeout` increased because of
`code = DeadlineExceeded desc = context deadline exceeded`

Close #3620

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-18 19:59:27 +03:00
Anna Shaleva
7b09812069 core: refactor TestEngineLimits 
Replace repetative hand-written code with generated one.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
 2024-10-18 19:10:47 +03:00
Anna Shaleva
b5b89881b7 core: prove contract notifications count is not restricted 
A part of #3490.

Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
 2024-10-18 19:10:34 +03:00
Anna Shaleva
8e1fdd5d70
Merge pull request #3626 from nspcc-dev/uploader-errors 
cli: adjust error of `upload-bin` command
 2024-10-18 14:28:02 +03:00
Ekaterina Pavlova
87bbff9831 cli: adjust error of upload-bin command 
Miscalculation leads to negative values in errors:
```
2024-10-17 11:26:56.790	failed to fetch the latest missing block index
 from container: search of index files failed for batch with indexes
 from -260000 to -250000: error during object IDs iteration: context
 deadline exceeded
```

Close #3621

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-18 14:26:17 +03:00
Anna Shaleva
14ea5a8d32
Merge pull request #3582 from nspcc-dev/block-fetcher-commands 
NeoFS block storage: add uploading commands
 2024-10-17 12:10:06 +03:00
Ekaterina Pavlova
6199240598 cli: add upload-bin 
This command is used for keeping container with blocks for
blockfetcher updated.

Close #3578

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-17 11:56:07 +03:00
Ekaterina Pavlova
59fab5d654 blockfetcher: adjust index file numeration from 0 
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-16 19:23:56 +03:00
Ekaterina Pavlova
852dcb0f64 config, docs: adjust NeoFSBlockFetcher index file and time attributes 
Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-16 19:23:56 +03:00
Ekaterina Pavlova
2435484dc4 neofs: add ObjectSearchInitter 
With ObjectSearchInitter ObjectSearch can can be done by both NeoFS SDK
client and pool.

Signed-off-by: Ekaterina Pavlova <ekt@morphbits.io>
 2024-10-16 19:23:56 +03:00