bc5882fc89
[ #1402 ] cli: Add `cid` flag constant
...
Also, move common flag constants in `root` file of the `storagegroup`
package.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-06 09:56:26 +03:00
6f2363cf31
[ #1438 ] README.md: add build instructions
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-05 14:47:12 +03:00
c165d1a9b5
[ #1556 ] Upgrade NeoFS SDK Go with changed container API
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-07-05 11:26:06 +03:00
f699e82ea7
[ #1560 ] morph/client: Remove `customFees` type
...
It is private, is a simple map and there is no complex logic to be wrapped in methods.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-04 09:29:01 +03:00
0ccea802e9
[ #1560 ] morph/client: Perform RPC switch and restore in one step
...
Otherwise we could switch infinitely if subscription restore has failed.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-04 09:29:01 +03:00
6358f4d746
[ #1053 ] config: Remove notification endpoints
...
Missed this in #1170 .
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-30 10:51:57 +03:00
ee27c66952
[ #1555 ] util/gendoc: Remove trailing dot from the help message
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-28 13:46:02 +03:00
ad81b6c90a
[ #1555 ] neofs-adm: Remove trailing dots from help messages
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-28 13:46:02 +03:00
b41658db04
[ #1555 ] neofs-adm: Allow to remove nodes from the netmap
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-28 13:46:02 +03:00
32fd3692bf
[ #1555 ] neofs-adm: Fix cached group signer
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-28 13:46:02 +03:00
5c2ef79080
[ #1547 ] .github: Assign issues to project when labeled
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-27 10:11:43 +03:00
305dd7598f
[ #1533 ] acl: Upgrade NeoFS SDK Go with refactored basic ACL
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-25 13:57:21 +03:00
b13dca8052
[ #1545 ] .github: Auto assign new issues to the project
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-24 20:27:45 +03:00
451343c751
[ #1539 ] neofs-adm: Retrieve storage wallet passwords from config
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-24 13:40:21 +03:00
fc7940a5ea
[ #1544 ] cli: Init bearer flags in `list-objects` cmd
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-24 13:40:07 +03:00
bb52372108
[ #1544 ] go.mod: Update SDK
...
The updated version supports reading network configuration uint64 values
that consist of less than 8 bytes.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-24 13:40:07 +03:00
9b3e1bd27b
[ #1307 ] neofs-adm: Add tests for `morph init` command
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-24 10:33:34 +03:00
189507dc89
[ #1541 ] neofs-cli: allow to print `shards list` output in JSON
...
Eventually more parameters will be supported (#1390 ) and after blobstor
configuration refactoring the output will certainly change. Implement
the simplest approach now.
Marshaling the result directly results in too ugly names and they cannot
be easily customized. Marshaling the results via `jsonpb` is better but
is not that flexible in terms of what we want to output.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-24 10:33:22 +03:00
9816d59ec0
[ #1323 ] neofs-cli: Reuse JSON flag for multiple commands
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-24 10:33:22 +03:00
e812d78672
[ #1537 ] node: Remove "trust" from non-reputation logs
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-23 10:06:36 +03:00
b67974a8d3
[#xxx] Upgrade NeoFS SDK Go with changed container sessions
...
After recent changes in NeoFS SDK Go library session tokens aren't
embedded into `container.Container` and `eacl.Table` structures.
Group value, session token and signature in a structure for container
and eACL.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-22 16:38:57 +03:00
633b4e7d2d
[ #1483 ] metabase: Add VERSION.md
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-21 17:48:28 +03:00
6f243a2a76
[ #1483 ] metabase: Store version
...
The main problem is to distinguish the case of initial initialization
and update from version 0. We can't do this at `Open`, because of
`resync_metabase` flag. Thus, the following approach was taken:
1. During `Open` check whether the metabase was initialized.
2. Check for the version in `Init` or write the new one if the metabase
is new.
3. Update version in `Reset`.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-21 17:48:28 +03:00
7df50297cd
[ #1520 ] shard: Ignore errors on metabase refill
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-21 17:25:28 +03:00
78ea450c25
[ #1502 ] shard: Process locks on metabase refill
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-21 11:07:26 +03:00
972ca83e23
[ #1524 ] writecache: Add some bolt parameters to the configuration
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-20 17:04:35 +03:00
07e06249d5
[ #1524 ] metabase: Add some bolt parameters to the configuration
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-20 17:04:35 +03:00
d6043abc24
[ #1524 ] neofs-node: Reuse boltdb config for different components
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-20 17:04:35 +03:00
0408acc40e
services/control: Autogenerate stable marshalers
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-20 16:52:58 +03:00
d2d1694113
[ #1513 ] cli/storagegroup: Fix linter remarks
...
`staticcheck` linter scolds on potential nil-dereference since it
doesn't know `common.ExitOnErr` definitely exists. Explicit `return`
fixes linter's remark and prevents potential bugs.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-17 15:53:18 +03:00
13e74fce8a
[ #1513 ] ir/settlement: Do not allocate intermediate slice of nodes
...
After recent changes `buildContainer` method returns two-dimensional
slice of `NodeInfo` so there is no need to flatten it to build slice of
`common.NodeInfo`.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-17 15:53:18 +03:00
c408a6a0db
[ #1513 ] morph/netmap: Use constant states in unit tests
...
Use values of the node state enumeration from Netmap contract instead of
numeric literals.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-17 15:53:18 +03:00
eb7b8bf7c4
[ #1513 ] node: Encode node's public key into binary format once
...
Avoid per-iteration key encoding due to the immutability of the key.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-17 15:53:18 +03:00
4574ba646d
[ #1513 ] cli/storagegroup: Fix parsing container and SG flags
...
Read `id` flag with the storage group ID. Prevent NPE-panic if flag is
missing.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-17 15:53:18 +03:00
820acebb7d
[ #1513 ] morph/netmap: Return result by value from net config reader
...
Make `ReadNetworkConfiguration` method to return `NetworkConfiguration`
by value in order to follow storage engine improvements and prevent heap
escaping.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-17 15:53:18 +03:00
808ba87e82
[ #1513 ] morph/netmap: Use node state constants from `Netmap` contract
...
`Netmap` contract exports enumeration of the node states.
Replace using literals and constants from NeoFS API Go V2 with the
values provided by contract.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-17 15:53:18 +03:00
21d2f8f861
[ #1513 ] Upgrade NeoFS SDK Go with changed `netmap` package
...
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-17 15:53:18 +03:00
24b4c1ecf4
[ #1484 ] neofs-cli: Print error on a separate line from a progress bar
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-16 16:38:33 +03:00
e42b7f6a65
[ #1484 ] neofs-cli: Print progress bar after the header has been sent
...
Make it similar to `object get`.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-16 16:38:33 +03:00
b68778ea90
[ #1380 ] neofs-cli: move `container` command to a separate package
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-16 09:27:07 +03:00
fd48b96082
[ #1384 ] neofs-cli: move `storagegroup` command to a separate package
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-16 09:27:07 +03:00
a219e3a667
[ #1383 ] neofs-cli: move `object` command to a separate package
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-16 09:27:07 +03:00
12bc5607f7
[ #1382 ] neofs-cli: move `netmap` command to a separate package
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-16 09:27:07 +03:00
81f925d5a0
[ #1516 ] metabase: Optimize `ListWithCursor` for long listings
...
Cache buckets outside of the main loop and allocate memory for the
resulting offset only once.
```
name old time/op new time/op delta
ListWithCursor/1_item-8 6.45µs ±14% 5.79µs ±11% -10.24% (p=0.002 n=10+10)
ListWithCursor/10_items-8 20.9µs ±17% 17.3µs ± 9% -17.27% (p=0.000 n=10+10)
ListWithCursor/100_items-8 153µs ±12% 131µs ± 9% -14.63% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
ListWithCursor/1_item-8 2.31kB ± 0% 1.91kB ± 0% -17.46% (p=0.000 n=10+10)
ListWithCursor/10_items-8 6.94kB ± 0% 5.50kB ± 0% -20.78% (p=0.000 n=8+8)
ListWithCursor/100_items-8 53.3kB ± 0% 41.5kB ± 0% -22.18% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
ListWithCursor/1_item-8 40.0 ± 0% 34.0 ± 0% -15.00% (p=0.000 n=10+10)
ListWithCursor/10_items-8 121 ± 0% 100 ± 0% -17.36% (p=0.000 n=10+10)
ListWithCursor/100_items-8 930 ± 0% 758 ± 0% -18.49% (p=0.000 n=10+10)
```
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-15 20:49:41 +03:00
a93373fe71
[ #1516 ] metabase: Cache graveyard buckets in `ListWithCursor`
...
```
name old time/op new time/op delta
ListWithCursor/1_item-8 6.40µs ±13% 6.45µs ±14% ~ (p=0.739 n=10+10)
ListWithCursor/10_items-8 30.9µs ±21% 20.9µs ±17% -32.49% (p=0.000 n=10+10)
ListWithCursor/100_items-8 274µs ±27% 153µs ±12% -44.09% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
ListWithCursor/1_item-8 2.26kB ± 0% 2.31kB ± 0% +2.46% (p=0.000 n=10+10)
ListWithCursor/10_items-8 10.8kB ± 0% 6.9kB ± 0% -36.07% (p=0.000 n=8+8)
ListWithCursor/100_items-8 96.8kB ± 0% 53.3kB ± 0% -44.98% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
ListWithCursor/1_item-8 39.0 ± 0% 40.0 ± 0% +2.56% (p=0.000 n=10+10)
ListWithCursor/10_items-8 192 ± 0% 121 ± 0% -36.98% (p=0.000 n=10+10)
ListWithCursor/100_items-8 1.72k ± 0% 0.93k ± 0% -45.93% (p=0.000 n=10+10)
```
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
name old time/op new time/op delta
ListWithCursor/1_item-8 5.23µs ±19% 5.26µs ±15% ~ (p=0.853 n=10+10)
ListWithCursor/10_items-8 27.2µs ±15% 18.0µs ±19% -33.80% (p=0.000 n=10+10)
ListWithCursor/100_items-8 250µs ±13% 139µs ±15% -44.27% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
ListWithCursor/1_item-8 1.99kB ± 0% 2.04kB ± 0% +2.82% (p=0.000 n=8+8)
ListWithCursor/10_items-8 10.3kB ± 0% 6.4kB ± 0% -37.83% (p=0.000 n=8+10)
ListWithCursor/100_items-8 93.9kB ± 0% 50.4kB ± 0% -46.37% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
ListWithCursor/1_item-8 35.0 ± 0% 36.0 ± 0% +2.86% (p=0.000 n=10+10)
ListWithCursor/10_items-8 184 ± 0% 113 ± 0% -38.59% (p=0.000 n=10+10)
ListWithCursor/100_items-8 1.67k ± 0% 0.88k ± 0% -47.29% (p=0.000 n=10+10)
```
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-15 20:49:41 +03:00
af4db8a73b
[ #1516 ] metabase: Cache address key and do not decode address twice
...
```
name old time/op new time/op delta
ListWithCursor/1_item-8 10.6µs ± 1% 6.4µs ±13% -39.62% (p=0.000 n=7+10)
ListWithCursor/10_items-8 75.3µs ± 2% 30.9µs ±21% -58.97% (p=0.000 n=10+10)
ListWithCursor/100_items-8 726µs ± 2% 274µs ±27% -62.28% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
ListWithCursor/1_item-8 3.19kB ± 0% 2.26kB ± 0% -29.21% (p=0.000 n=10+10)
ListWithCursor/10_items-8 20.7kB ± 0% 10.8kB ± 0% -47.68% (p=0.000 n=10+8)
ListWithCursor/100_items-8 196kB ± 0% 97kB ± 0% -50.65% (p=0.000 n=7+10)
name old allocs/op new allocs/op delta
ListWithCursor/1_item-8 55.0 ± 0% 39.0 ± 0% -29.09% (p=0.000 n=10+10)
ListWithCursor/10_items-8 346 ± 0% 192 ± 0% -44.51% (p=0.000 n=10+10)
ListWithCursor/100_items-8 3.25k ± 0% 1.72k ± 0% -47.13% (p=0.000 n=9+10)
```
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-15 20:49:41 +03:00
504f45e9ee
[ #1516 ] metabase: Add benchmark for `ListWithCursor`
...
```
name time/op
ListWithCursor/1_item-8 10.6µs ± 1%
ListWithCursor/10_items-8 75.3µs ± 2%
ListWithCursor/100_items-8 726µs ± 2%
name alloc/op
ListWithCursor/1_item-8 3.19kB ± 0%
ListWithCursor/10_items-8 20.7kB ± 0%
ListWithCursor/100_items-8 196kB ± 0%
name allocs/op
ListWithCursor/1_item-8 55.0 ± 0%
ListWithCursor/10_items-8 346 ± 0%
ListWithCursor/100_items-8 3.25k ± 0%
```
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-15 20:49:41 +03:00
36f4929e52
[ #1507 ] node: Do not handle object concurrently by the policer
...
Cache object that are being processed. That prevents concurrent
object handling when there is a few number of objects and object handling
takes more time that the policer needs for starting that object handling one
more time.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-15 20:43:32 +03:00
256165045b
[ #1508 ] node: Do not replicate object twice
...
If placement contains two vectors with intersecting nodes it was possible to
send the object to the nodes twice.
Also optimizes requests: do not ask about storing the object twice from the
same node.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-15 20:33:04 +03:00
df8a3807fe
[ #1508 ] node: Remove unused replicator code
...
The node does not support asynchronous object replication anymore, so it
does not need to have replicator worker, channel and `AddTask` function.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-15 20:33:04 +03:00
Pavel Karpy
Evgenii Stratonikov
Leonard Lyubich