Compare commits

...

69 commits

Author SHA1 Message Date
d342c0bc16 [#268] client: Make PayloadPatch correctly receive empty patch payload
* Make the method `PatchPayload` send a patch with empty
  payload patch if range's length is non-zero and if it's the
  first call.
* Empty payload patches just cut original object payload. So, these
  patches are also valid.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-12 15:12:52 +03:00
f0c599d06d [#268] client: Fix sequential PayloadPatch calls
* The flag 'firstPayloadPatch' keeps its state after first
  `PatchPayload` that make other calls incorrectly set patch
  ranges.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-12 15:09:01 +03:00
7d84d104fb [#260] *: Fix linter warnings
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:55:32 +00:00
812126a8ff [#260] .golangci.yml: Add protogetter linter
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:55:32 +00:00
d86223ed56 [#260] Makefile: Add pre-commit targets
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:55:32 +00:00
76a0cfdadb [#217] netmap: Return node netmap state directly
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:51:51 +00:00
46ee543899 [#265] go.mod: Use range over int
Since Go 1.22 a `for` statement with a `range` clause is able
to iterate through integer values from zero to an upper limit.

gopatch script:
@@
var i, e expression
@@
-for i := 0; i <= e - 1; i++ {
+for i := range e {
    ...
}

@@
var i, e expression
@@
-for i := 0; i <= e; i++ {
+for i := range e + 1 {
    ...
}

@@
var i, e expression
@@
-for i := 0; i < e; i++ {
+for i := range e {
    ...
}

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-09-04 12:37:46 +03:00
8f751d9dd0
[#263] go.mod: Update api-go
Remove `client.ContainerEACL` and related references. This change
was initiated by the removal of `ContainerService.GetExtendedACL`
from the API.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-03 12:36:28 +03:00
3c00f4eeac [#203] Add pool docs
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
2024-08-30 08:24:31 +03:00
f0b9493ce3 [#261] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-27 11:33:09 +03:00
28f140bf06 [#254] pool: Add parameter gracefulCloseOnSwitchTimeout
Add new param for waiting a little until current in-flight requests will be finished

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-22 08:02:51 +00:00
9115d3f281 [#1316] lint: Fix warnings
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-08-21 17:59:10 +03:00
cf225be0df [#1316] go.mod: Bump go version to 1.22
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-08-21 16:42:58 +03:00
338d1ef254 [#258] pool/tree: Add node address to error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-20 14:19:58 +03:00
6dd7be11d1 [#256] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-20 10:20:28 +03:00
203bba65a0 [#253] pool: Don't count regular FrostFS errors
Previously we count all frostfs errors like:
ObjectNotFound, EACLNotFound
because frostfs status is unconditionally resolved into built-in go errors
but handleError method handled built-in errors like internal network ones.
Since after resolving frostfs errors status is also returned we start check this first

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-14 16:48:51 +03:00
98aabc45a7 [#252] patcher: Fix applying patch from the same offset
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-13 18:58:21 +03:00
908c96a94d [#251] pool: Fix handlerError panic for objectPatch
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-13 15:28:39 +03:00
2077b35736 [#231] netmap: Add LIKE operation for filter
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-12 09:49:12 +03:00
92c7596157 [#231] go.mod: Update version for frostfs-api-go/v2
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-12 09:49:04 +03:00
a15b1264f5 [#231] policy: Bump jre version to 17
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-12 09:44:30 +03:00
5d58519253 [#249] pool: Introduce objectPatch method
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-08 18:26:40 +03:00
93171b3319 [#249] session: Support patch verb
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-08 18:26:40 +03:00
3ba7446157 [#249] client: Introduce ObjectPatch
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-08 18:26:35 +03:00
335aa18dc6 [#249] go.mod: Bump frostfs-api-go/v2 version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-06 16:57:42 +03:00
361739e860 [#247] object: Introduce patcher package
* Introduce `patcher` package that contains such interfaces to be
  implemented:
  - `PatchApplier` - the main patching engine that merges the stream
    of patches and the stream of original object payload divided by
    ranges. The merged streams result is output to `ChunkedObjectWriter`;
  - `RangeProvider` - provides the original object payload by ranges;
  - `HeaderProvider` - provides the original object header.
* Introduce `patcher` that implements `PatchApplier`;
* Cover all possible cases with unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-05 12:32:14 +00:00
6dd500def9 [#247] object: Introduce Patch type
* Make ToV2, FromV2 converters for Patch and PayloadPatch;
* Add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-05 12:32:14 +00:00
e83d6b7c6a [#244] pool/tree: Collect request duration statistic
After each request for tree pool statistic accumulated values are reset to zero.

Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-08-02 13:01:14 +03:00
9da46f566f [#243] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-26 14:13:49 +03:00
fa89999d91 [#242] pool: Log error that caused healthy status change
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-22 15:12:27 +03:00
7e94a6adf2 [#237] pool: Return creation epoch from object put
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-22 06:15:23 +00:00
ce8270568d [#239] go.mod: Update frostfs-contract
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-07-18 17:17:40 +03:00
7c06cdff2d [#239] pool/tree: Update tree service client
Update tree service to fix split tree problem.
Tree intermediate nodes can be duplicated so we must handle this.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-07-17 14:32:04 +03:00
e18b916231 [#238] go.mod: Update frostfs-contract to v0.19.3
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-07-16 11:36:21 +03:00
c4ff8a6cda [#236] go.mod: Update neo-go to v0.106.2
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 15:43:11 +03:00
fc7c524fcb [#236] *: Replace slice.Copy() with bytes.Clone()
The former is deprecated.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 15:43:07 +03:00
e977b8a94c [#236] netmap: Remove unused field from meanIQRAgg
It was there since the inception [1], but we never got to use it.

[1] 5931284e07

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:07 +03:00
6729f54c4e [#236] netmap: Reuse slice for weights in ContainerNodes()
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │    alloc     │               weights               │
                                                              │    sec/op    │    sec/op     vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.677µ ±  6%   8.384µ ± 10%       ~ (p=0.247 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.946µ ± 14%   7.998µ ±  6%       ~ (p=0.481 n=10)
geomean                                                         8.303µ         8.189µ        -1.38%

                                                              │    alloc     │               weights               │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   7.734Ki ± 0%   7.617Ki ± 0%  -1.52% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.156Ki ± 0%   7.039Ki ± 0%  -1.64% (p=0.000 n=10)
geomean                                                         7.440Ki        7.322Ki       -1.58%

                                                              │   alloc    │              weights               │
                                                              │ allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   92.00 ± 0%   77.00 ± 0%  -16.30% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   92.00 ± 0%   77.00 ± 0%  -16.30% (p=0.000 n=10)
geomean                                                         92.00        77.00       -16.30%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:07 +03:00
159a50fcf0 [#236] netmap: Reduce allocations in getSelection()
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │     new      │                alloc                 │
                                                              │    sec/op    │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                   9.227µ ± 13%   8.677µ ±  6%        ~ (p=0.165 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   9.189µ ±  7%   7.946µ ± 14%  -13.53% (p=0.001 n=10)
geomean                                                         9.208µ         8.303µ         -9.82%

                                                              │     new      │                alloc                │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.320Ki ± 0%   7.734Ki ± 0%  -7.04% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.742Ki ± 0%   7.156Ki ± 0%  -7.57% (p=0.000 n=10)
geomean                                                         8.026Ki        7.440Ki       -7.31%

                                                              │     new     │               alloc                │
                                                              │  allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   122.00 ± 0%   92.00 ± 0%  -24.59% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   122.00 ± 0%   92.00 ± 0%  -24.59% (p=0.000 n=10)
geomean                                                          122.0        92.00       -24.59%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:07 +03:00
a69f00903c [#236] netmap: Replace sort.Slice() with slices.Sort()
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │      old      │                 new                  │
                                                              │    sec/op     │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                   10.395µ ± 14%   9.227µ ± 13%  -11.24% (p=0.015 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   10.110µ ± 16%   9.189µ ±  7%        ~ (p=0.105 n=10)
geomean                                                          10.25µ         9.208µ        -10.18%

                                                              │     old      │                 new                 │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.695Ki ± 0%   8.320Ki ± 0%  -4.31% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   8.117Ki ± 0%   7.742Ki ± 0%  -4.62% (p=0.000 n=10)
geomean                                                         8.401Ki        8.026Ki       -4.47%

                                                              │    old     │                new                 │
                                                              │ allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   138.0 ± 0%   122.0 ± 0%  -11.59% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   138.0 ± 0%   122.0 ± 0%  -11.59% (p=0.000 n=10)
geomean                                                         138.0        122.0       -11.59%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:07 +03:00
9d89f08c7b [#236] go.mod: Bump min go version to go1.21
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:06 +03:00
51cefd4908 [#232] netmap: Allow empty values for unknown parameters in network config
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-07-09 12:19:25 +03:00
560cbbd1f1 [#234] pool: Update token expiration check in cache
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-05 12:36:17 +03:00
27e965007d [#233] pool: Introduce ape-manager methods
* Remove GetEACL, SetEACL methods as they are deprecated;
* Fix mock;
* Introduce add/remove/list methods to request ape-manager from
  the client.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-04 01:29:35 +03:00
1a5886e776 [#228] client: Move isClientErrMaintenance from node
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-06-17 17:07:30 +03:00
ebd8fcd168 [#224] object: Introduce parent attributes in EC-header
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 16:20:48 +03:00
717a7d00ef [#225] bearer: Introduce APEOverride field for the token
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 12:14:42 +00:00
dd23c6fd2b [#225] apemanager: Move apemanager to ape package
* Update go.mod;
* Fix packages;
* Fix client's `apemanager` methods.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 12:14:42 +00:00
6a52487edd [#226] pool/tree: Fix handling access denied error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-05-30 14:59:35 +03:00
c5c6272029 [#221] pool: Make sampler safe for concurrent using
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-05-20 14:14:14 +03:00
3de256d05e [#223] object: Introduce new fields for ECHeader
* Introduce `parentSplitID`, `parentSplitParentID` fields
  for `ECHeader`;
* Fix ECHeader's constructor;
* Fix `Split` and `Reconstruct`;
* Add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-17 15:16:28 +03:00
09b79d13f3 [#223] object: Introduce ec_parent search filter
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-17 15:03:40 +03:00
d4e6f4e125 [#223] go.mod: Update frosts-api-go/v2 version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-17 15:02:44 +03:00
b2ad1f3b3e [#215] client: Introduce apemanager rpc interface
* Introduce `APEManagerAddChain`, `APEManagerRemoveChain`, `APEManagerListChains`.
* Introduce reqeuest/response types for these handlers (Prm*, Res*).
* Inroduce status type for apemanager `APEManagerAccessDenied`; add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-14 10:59:52 +03:00
32a975a20d [#215] apemanager: Introduce apemanager types
* Introduce `Chain`, `ChainTarget` and `TargetType`.
* Implement api-v2 converters for the introduced types.
* Add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-14 10:59:49 +03:00
eaf36706a2 [#215] go.mod: Update frostfs-api-go/v2 package version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-08 11:18:18 +03:00
02c936f397 [#216] netmap: Add policy decode fuzz test
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-07 11:10:37 +00:00
99e02858af [#220] netmap: Fix setters for Replica.DataCount/ParityCount
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-07 09:34:14 +03:00
12ddefe078 [#218] object: Implement Range\RangeHash requests for EC object
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-02 11:01:21 +03:00
20ab57bf7e [#214] object: Implement Get\Head requests for EC object
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 11:07:26 +03:00
3790142b10 [#212] pool: Control sub tree nodes order
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-04-02 17:15:49 +03:00
ec0cb2169f [#211] object: Fix setIDWithSignature
* Calculate and set checksum before ID is calculated.
* Add header verification for parts in unit-test.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-29 13:48:04 +03:00
425d48f68b [#211] netmap: Introduce ReplicaDescriptor method
* Make ReplicaNumberByIndex deprecated.
* Introduce ReplicaDescriptor method that access i-th replica directly.
* Introduce new getters for ReplicaDescriptor.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-29 13:48:04 +03:00
6d0da3f861 [#211] go.mod: Update frostfs-api-go version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-29 13:48:04 +03:00
1af9b6d18b [#155] sdk-go: Add buffer support for payloadSizeLimiter
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-03-25 10:47:04 +03:00
bd2d350b09 [#205] object: Initial EC implementation
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
e9be3e6d94 [#205] netmap: Add well-known EC parameters to network config
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
70e9e40c7f [#205] netmap: Add EC statement to placement policy
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
d33b54d280 [#205] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
120 changed files with 8050 additions and 2265 deletions

View file

@ -13,7 +13,7 @@ jobs:
- name: Setup Go - name: Setup Go
uses: actions/setup-go@v3 uses: actions/setup-go@v3
with: with:
go-version: '1.21' go-version: '1.23'
- name: Run commit format checker - name: Run commit format checker
uses: https://git.frostfs.info/TrueCloudLab/dco-go@v3 uses: https://git.frostfs.info/TrueCloudLab/dco-go@v3

View file

@ -11,7 +11,7 @@ jobs:
- name: Set up Go - name: Set up Go
uses: actions/setup-go@v3 uses: actions/setup-go@v3
with: with:
go-version: '1.21' go-version: '1.23'
cache: true cache: true
- name: Install linters - name: Install linters
@ -25,7 +25,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
strategy: strategy:
matrix: matrix:
go_versions: [ '1.20', '1.21' ] go_versions: [ '1.22', '1.23' ]
fail-fast: false fail-fast: false
steps: steps:
- uses: actions/checkout@v3 - uses: actions/checkout@v3

View file

@ -12,7 +12,8 @@ run:
# output configuration options # output configuration options
output: output:
# colored-line-number|line-number|json|tab|checkstyle|code-climate, default is "colored-line-number" # colored-line-number|line-number|json|tab|checkstyle|code-climate, default is "colored-line-number"
format: tab formats:
- format: tab
# all available settings of specific linters # all available settings of specific linters
linters-settings: linters-settings:
@ -62,5 +63,6 @@ linters:
- funlen - funlen
- gocognit - gocognit
- contextcheck - contextcheck
- protogetter
disable-all: true disable-all: true
fast: false fast: false

View file

@ -1,4 +1,4 @@
FROM golang:1.21 FROM golang:1.22
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install make openjdk-11-jre -y RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install make openjdk-17-jre -y
WORKDIR /work WORKDIR /work

View file

@ -2,8 +2,8 @@
ANTLR_VERSION="4.13.0" ANTLR_VERSION="4.13.0"
TMP_DIR := .cache TMP_DIR := .cache
LINT_VERSION ?= 1.56.2 LINT_VERSION ?= 1.60.1
TRUECLOUDLAB_LINT_VERSION ?= 0.0.2 TRUECLOUDLAB_LINT_VERSION ?= 0.0.6
OUTPUT_LINT_DIR ?= $(shell pwd)/bin OUTPUT_LINT_DIR ?= $(shell pwd)/bin
LINT_DIR = $(OUTPUT_LINT_DIR)/golangci-lint-$(LINT_VERSION)-v$(TRUECLOUDLAB_LINT_VERSION) LINT_DIR = $(OUTPUT_LINT_DIR)/golangci-lint-$(LINT_VERSION)-v$(TRUECLOUDLAB_LINT_VERSION)
@ -77,3 +77,15 @@ help:
@echo ' Targets:' @echo ' Targets:'
@echo '' @echo ''
@awk '/^#/{ comment = substr($$0,3) } comment && /^[a-zA-Z][a-zA-Z0-9_-]+ ?:/{ print " ", $$1, comment }' $(MAKEFILE_LIST) | column -t -s ':' | grep -v 'IGNORE' | sort -u @awk '/^#/{ comment = substr($$0,3) } comment && /^[a-zA-Z][a-zA-Z0-9_-]+ ?:/{ print " ", $$1, comment }' $(MAKEFILE_LIST) | column -t -s ':' | grep -v 'IGNORE' | sort -u
# Activate pre-commit hooks
pre-commit:
pre-commit install --hook-type pre-commit
# Deactivate pre-commit hooks
unpre-commit:
pre-commit uninstall --hook-type pre-commit
# Run pre-commit hooks
pre-commit-run:
@pre-commit run --all-files --hook-stage manual

52
ape/chain.go Normal file
View file

@ -0,0 +1,52 @@
package ape
import (
"errors"
"fmt"
apeV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/ape"
)
var (
ErrInvalidChainRepresentation = errors.New("invalid chain representation")
)
// ChainID is Chain's identifier.
type ChainID []byte
// Chain is an SDK representation for v2's Chain.
//
// Note that Chain (as well as v2's Chain) and all related entities
// are NOT operated by Access-Policy-Engine (APE). The client is responsible
// to convert these types to policy-engine entities.
type Chain struct {
// Raw is the encoded chain kind.
// It assumes that Raw's bytes are the result of encoding provided by
// policy-engine package.
Raw []byte
}
// ToV2 converts Chain to v2.
func (c *Chain) ToV2() *apeV2.Chain {
v2ct := new(apeV2.Chain)
if c.Raw != nil {
v2Raw := new(apeV2.ChainRaw)
v2Raw.SetRaw(c.Raw)
v2ct.SetKind(v2Raw)
}
return v2ct
}
// ReadFromV2 fills Chain from v2.
func (c *Chain) ReadFromV2(v2ct *apeV2.Chain) error {
switch v := v2ct.GetKind().(type) {
default:
return fmt.Errorf("unsupported chain kind: %T", v)
case *apeV2.ChainRaw:
raw := v.GetRaw()
c.Raw = raw
}
return nil
}

53
ape/chain_target.go Normal file
View file

@ -0,0 +1,53 @@
package ape
import (
apeV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/ape"
)
// TargetType is an SDK representation for v2's TargetType.
type TargetType apeV2.TargetType
const (
TargetTypeUndefined TargetType = iota
TargetTypeNamespace
TargetTypeContainer
TargetTypeUser
TargetTypeGroup
)
// ToV2 converts TargetType to v2.
func (targetType TargetType) ToV2() apeV2.TargetType {
return apeV2.TargetType(targetType)
}
// FromV2 reads TargetType to v2.
func (targetType *TargetType) FromV2(v2targetType apeV2.TargetType) {
*targetType = TargetType(v2targetType)
}
// ChainTarget is an SDK representation for v2's ChainTarget.
//
// Note that ChainTarget (as well as v2's ChainTarget) and all related entities
// are NOT operated by Access-Policy-Engine (APE). The client is responsible
// to convert these types to policy-engine entities.
type ChainTarget struct {
TargetType TargetType
Name string
}
// ToV2 converts ChainTarget to v2.
func (ct *ChainTarget) ToV2() *apeV2.ChainTarget {
v2ct := new(apeV2.ChainTarget)
v2ct.SetTargetType(ct.TargetType.ToV2())
v2ct.SetName(ct.Name)
return v2ct
}
// FromV2 reads ChainTarget frpm v2.
func (ct *ChainTarget) FromV2(v2ct *apeV2.ChainTarget) {
ct.TargetType.FromV2(v2ct.GetTargetType())
ct.Name = v2ct.GetName()
}

65
ape/chain_target_test.go Normal file
View file

@ -0,0 +1,65 @@
package ape_test
import (
"testing"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
"github.com/stretchr/testify/require"
apeV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/ape"
)
var (
m = map[ape.TargetType]apeV2.TargetType{
ape.TargetTypeUndefined: apeV2.TargetTypeUndefined,
ape.TargetTypeNamespace: apeV2.TargetTypeNamespace,
ape.TargetTypeContainer: apeV2.TargetTypeContainer,
ape.TargetTypeUser: apeV2.TargetTypeUser,
ape.TargetTypeGroup: apeV2.TargetTypeGroup,
}
)
func TestTargetType(t *testing.T) {
for typesdk, typev2 := range m {
t.Run("from sdk to v2 "+typev2.String(), func(t *testing.T) {
v2 := typesdk.ToV2()
require.Equal(t, v2, typev2)
})
t.Run("from v2 to sdk "+typev2.String(), func(t *testing.T) {
var typ ape.TargetType
typ.FromV2(typev2)
require.Equal(t, typesdk, typ)
})
}
}
func TestChainTarget(t *testing.T) {
var (
typ = ape.TargetTypeNamespace
name = "namespaceXXYYZZ"
)
t.Run("from sdk to v2", func(t *testing.T) {
ct := ape.ChainTarget{
TargetType: typ,
Name: name,
}
v2 := ct.ToV2()
require.Equal(t, m[typ], v2.GetTargetType())
require.Equal(t, name, v2.GetName())
})
t.Run("from v2 to sdk", func(t *testing.T) {
v2 := &apeV2.ChainTarget{}
v2.SetTargetType(m[typ])
v2.SetName(name)
var ct ape.ChainTarget
ct.FromV2(v2)
require.Equal(t, typ, ct.TargetType)
require.Equal(t, name, ct.Name)
})
}

43
ape/chain_test.go Normal file
View file

@ -0,0 +1,43 @@
package ape_test
import (
"testing"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
"github.com/stretchr/testify/require"
apeV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/ape"
)
const (
encoded = `{"ID":"","Rules":[{"Status":"Allow","Actions":{"Inverted":false,"Names":["GetObject"]},"Resources":{"Inverted":false,"Names":["native:object/*"]},"Any":false,"Condition":[{"Op":"StringEquals","Object":"Resource","Key":"Department","Value":"HR"}]}],"MatchType":"DenyPriority"}`
)
func TestChainData(t *testing.T) {
t.Run("raw chain", func(t *testing.T) {
var c ape.Chain
b := []byte(encoded)
c.Raw = b
v2, ok := c.ToV2().GetKind().(*apeV2.ChainRaw)
require.True(t, ok)
require.Equal(t, b, v2.Raw)
})
}
func TestChainMessageV2(t *testing.T) {
b := []byte(encoded)
v2Raw := &apeV2.ChainRaw{}
v2Raw.SetRaw(b)
v2 := &apeV2.Chain{}
v2.SetKind(v2Raw)
var c ape.Chain
c.ReadFromV2(v2)
require.NotNil(t, c.Raw)
require.Equal(t, b, c.Raw)
}

View file

@ -6,7 +6,9 @@ import (
"fmt" "fmt"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
apeV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/ape"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
apeSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id" cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto" frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa" frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
@ -33,9 +35,85 @@ type Token struct {
sigSet bool sigSet bool
sig refs.Signature sig refs.Signature
apeOverrideSet bool
apeOverride APEOverride
impersonate bool impersonate bool
} }
// APEOverride is the list of APE chains defined for a target.
// These chains are meant to serve as overrides to the already defined (or even undefined)
// APE chains for the target (see contract `Policy`).
//
// The server-side processing of the bearer token with set APE overrides must verify if a client is permitted
// to override chains for the target, preventing unauthorized access through the APE mechanism.
type APEOverride struct {
// Target for which chains are applied.
Target apeSDK.ChainTarget
// The list of APE chains.
Chains []apeSDK.Chain
}
// Marshal marshals APEOverride into a protobuf binary form.
func (c *APEOverride) Marshal() ([]byte, error) {
return c.ToV2().StableMarshal(nil), nil
}
// Unmarshal unmarshals protobuf binary representation of APEOverride.
func (c *APEOverride) Unmarshal(data []byte) error {
overrideV2 := new(acl.APEOverride)
if err := overrideV2.Unmarshal(data); err != nil {
return err
}
return c.FromV2(overrideV2)
}
// MarshalJSON encodes APEOverride to protobuf JSON format.
func (c *APEOverride) MarshalJSON() ([]byte, error) {
return c.ToV2().MarshalJSON()
}
// UnmarshalJSON decodes APEOverride from protobuf JSON format.
func (c *APEOverride) UnmarshalJSON(data []byte) error {
overrideV2 := new(acl.APEOverride)
if err := overrideV2.UnmarshalJSON(data); err != nil {
return err
}
return c.FromV2(overrideV2)
}
func (c *APEOverride) FromV2(tokenAPEChains *acl.APEOverride) error {
c.Target.FromV2(tokenAPEChains.GetTarget())
if chains := tokenAPEChains.GetChains(); len(chains) > 0 {
c.Chains = make([]apeSDK.Chain, len(chains))
for i := range chains {
if err := c.Chains[i].ReadFromV2(chains[i]); err != nil {
return fmt.Errorf("invalid APE chain: %w", err)
}
}
}
return nil
}
func (c *APEOverride) ToV2() *acl.APEOverride {
if c == nil {
return nil
}
apeOverride := new(acl.APEOverride)
apeOverride.SetTarget(c.Target.ToV2())
chains := make([]*apeV2.Chain, len(c.Chains))
for i := range c.Chains {
chains[i] = c.Chains[i].ToV2()
}
apeOverride.SetChains(chains)
return apeOverride
}
// reads Token from the acl.BearerToken message. If checkFieldPresence is set, // reads Token from the acl.BearerToken message. If checkFieldPresence is set,
// returns an error on absence of any protocol-required field. // returns an error on absence of any protocol-required field.
func (b *Token) readFromV2(m acl.BearerToken, checkFieldPresence bool) error { func (b *Token) readFromV2(m acl.BearerToken, checkFieldPresence bool) error {
@ -48,10 +126,11 @@ func (b *Token) readFromV2(m acl.BearerToken, checkFieldPresence bool) error {
b.impersonate = body.GetImpersonate() b.impersonate = body.GetImpersonate()
apeOverrides := body.GetAPEOverride()
eaclTable := body.GetEACL() eaclTable := body.GetEACL()
if b.eaclTableSet = eaclTable != nil; b.eaclTableSet { if b.eaclTableSet = eaclTable != nil; b.eaclTableSet {
b.eaclTable = *eacl.NewTableFromV2(eaclTable) b.eaclTable = *eacl.NewTableFromV2(eaclTable)
} else if checkFieldPresence && !b.impersonate { } else if checkFieldPresence && !b.impersonate && apeOverrides == nil {
return errors.New("missing eACL table") return errors.New("missing eACL table")
} }
@ -72,6 +151,14 @@ func (b *Token) readFromV2(m acl.BearerToken, checkFieldPresence bool) error {
return errors.New("missing token lifetime") return errors.New("missing token lifetime")
} }
if b.apeOverrideSet = apeOverrides != nil; b.apeOverrideSet {
if err = b.apeOverride.FromV2(apeOverrides); err != nil {
return err
}
} else if checkFieldPresence && !b.impersonate && !b.eaclTableSet {
return errors.New("missing APE override")
}
sig := m.GetSignature() sig := m.GetSignature()
if b.sigSet = sig != nil; sig != nil { if b.sigSet = sig != nil; sig != nil {
b.sig = *sig b.sig = *sig
@ -90,7 +177,7 @@ func (b *Token) ReadFromV2(m acl.BearerToken) error {
} }
func (b Token) fillBody() *acl.BearerTokenBody { func (b Token) fillBody() *acl.BearerTokenBody {
if !b.eaclTableSet && !b.targetUserSet && !b.lifetimeSet && !b.impersonate { if !b.eaclTableSet && !b.targetUserSet && !b.lifetimeSet && !b.impersonate && !b.apeOverrideSet {
return nil return nil
} }
@ -116,6 +203,10 @@ func (b Token) fillBody() *acl.BearerTokenBody {
body.SetLifetime(&lifetime) body.SetLifetime(&lifetime)
} }
if b.apeOverrideSet {
body.SetAPEOverride(b.apeOverride.ToV2())
}
body.SetImpersonate(b.impersonate) body.SetImpersonate(b.impersonate)
return &body return &body
@ -214,6 +305,25 @@ func (b Token) EACLTable() eacl.Table {
return eacl.Table{} return eacl.Table{}
} }
// SetAPEOverride sets APE override to the bearer token.
//
// See also: APEOverride.
func (b *Token) SetAPEOverride(v APEOverride) {
b.apeOverride = v
b.apeOverrideSet = true
}
// APEOverride returns APE override set by SetAPEOverride.
//
// Zero Token has zero APEOverride.
func (b *Token) APEOverride() APEOverride {
if b.apeOverrideSet {
return b.apeOverride
}
return APEOverride{}
}
// SetImpersonate mark token as impersonate to consider token signer as request owner. // SetImpersonate mark token as impersonate to consider token signer as request owner.
// If this field is true extended EACLTable in token body isn't processed. // If this field is true extended EACLTable in token body isn't processed.
func (b *Token) SetImpersonate(v bool) { func (b *Token) SetImpersonate(v bool) {

View file

@ -3,6 +3,7 @@ package bearer_test
import ( import (
"bytes" "bytes"
"math/rand" "math/rand"
"reflect"
"testing" "testing"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
@ -81,6 +82,58 @@ func TestToken_SetEACLTable(t *testing.T) {
require.True(t, isEqualEACLTables(eaclTable, val.EACLTable())) require.True(t, isEqualEACLTables(eaclTable, val.EACLTable()))
} }
func TestToken_SetAPEOverrides(t *testing.T) {
var val bearer.Token
var m acl.BearerToken
filled := bearertest.Token()
val.WriteToV2(&m)
require.Zero(t, m.GetBody())
val2 := filled
require.NoError(t, val2.Unmarshal(val.Marshal()))
require.Zero(t, val2.APEOverride())
val2 = filled
jd, err := val.MarshalJSON()
require.NoError(t, err)
require.NoError(t, val2.UnmarshalJSON(jd))
require.Zero(t, val2.APEOverride())
// set value
tApe := bearertest.APEOverride()
val.SetAPEOverride(tApe)
require.Equal(t, tApe, val.APEOverride())
val.WriteToV2(&m)
require.NotNil(t, m.GetBody().GetAPEOverride())
require.True(t, tokenAPEOverridesEqual(tApe.ToV2(), m.GetBody().GetAPEOverride()))
val2 = filled
require.NoError(t, val2.Unmarshal(val.Marshal()))
apeOverride := val2.APEOverride()
require.True(t, tokenAPEOverridesEqual(tApe.ToV2(), apeOverride.ToV2()))
val2 = filled
jd, err = val.MarshalJSON()
require.NoError(t, err)
require.NoError(t, val2.UnmarshalJSON(jd))
apeOverride = val.APEOverride()
require.True(t, tokenAPEOverridesEqual(tApe.ToV2(), apeOverride.ToV2()))
}
func tokenAPEOverridesEqual(lhs, rhs *acl.APEOverride) bool {
return reflect.DeepEqual(lhs, rhs)
}
func TestToken_ForUser(t *testing.T) { func TestToken_ForUser(t *testing.T) {
var val bearer.Token var val bearer.Token
var m acl.BearerToken var m acl.BearerToken

View file

@ -1,6 +1,7 @@
package bearertest package bearertest
import ( import (
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
eacltest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl/test" eacltest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl/test"
usertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user/test" usertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user/test"
@ -15,6 +16,17 @@ func Token() (t bearer.Token) {
t.SetIat(1) t.SetIat(1)
t.ForUser(usertest.ID()) t.ForUser(usertest.ID())
t.SetEACLTable(*eacltest.Table()) t.SetEACLTable(*eacltest.Table())
t.SetAPEOverride(APEOverride())
return t return t
} }
func APEOverride() bearer.APEOverride {
return bearer.APEOverride{
Target: ape.ChainTarget{
TargetType: ape.TargetTypeContainer,
Name: "F8JsMnChywiPvbDvpxMbjTjx5KhWHHp6gCDt8BhzL9kF",
},
Chains: []ape.Chain{{Raw: []byte("{}")}},
}
}

View file

@ -0,0 +1,79 @@
package client
import (
"context"
"fmt"
apemanagerV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/apemanager"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
sessionV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apeSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
)
// PrmAPEManagerAddChain groups parameters of APEManagerAddChain operation.
type PrmAPEManagerAddChain struct {
XHeaders []string
ChainTarget apeSDK.ChainTarget
Chain apeSDK.Chain
}
func (prm *PrmAPEManagerAddChain) buildRequest(c *Client) (*apemanagerV2.AddChainRequest, error) {
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
req := new(apemanagerV2.AddChainRequest)
reqBody := new(apemanagerV2.AddChainRequestBody)
reqBody.SetTarget(prm.ChainTarget.ToV2())
reqBody.SetChain(prm.Chain.ToV2())
req.SetBody(reqBody)
var meta sessionV2.RequestMetaHeader
writeXHeadersToMeta(prm.XHeaders, &meta)
c.prepareRequest(req, &meta)
return req, nil
}
type ResAPEManagerAddChain struct {
statusRes
// ChainID of set Chain. If Chain does not contain chainID before request, then
// ChainID is generated.
ChainID apeSDK.ChainID
}
// APEManagerAddChain sets Chain for ChainTarget.
func (c *Client) APEManagerAddChain(ctx context.Context, prm PrmAPEManagerAddChain) (*ResAPEManagerAddChain, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.AddChain(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResAPEManagerAddChain
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
res.ChainID = resp.GetBody().GetChainID()
return &res, nil
}

View file

@ -0,0 +1,80 @@
package client
import (
"context"
"fmt"
apemanagerV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/apemanager"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
sessionV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apeSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
)
// PrmAPEManagerListChains groups parameters of APEManagerListChains operation.
type PrmAPEManagerListChains struct {
XHeaders []string
ChainTarget apeSDK.ChainTarget
}
func (prm *PrmAPEManagerListChains) buildRequest(c *Client) (*apemanagerV2.ListChainsRequest, error) {
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
req := new(apemanagerV2.ListChainsRequest)
reqBody := new(apemanagerV2.ListChainsRequestBody)
reqBody.SetTarget(prm.ChainTarget.ToV2())
req.SetBody(reqBody)
var meta sessionV2.RequestMetaHeader
writeXHeadersToMeta(prm.XHeaders, &meta)
c.prepareRequest(req, &meta)
return req, nil
}
type ResAPEManagerListChains struct {
statusRes
Chains []apeSDK.Chain
}
// APEManagerListChains lists Chains for ChainTarget.
func (c *Client) APEManagerListChains(ctx context.Context, prm PrmAPEManagerListChains) (*ResAPEManagerListChains, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.ListChains(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResAPEManagerListChains
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return nil, err
}
for _, ch := range resp.GetBody().GetChains() {
var chSDK apeSDK.Chain
if err := chSDK.ReadFromV2(ch); err != nil {
return nil, err
}
res.Chains = append(res.Chains, chSDK)
}
return &res, nil
}

View file

@ -0,0 +1,73 @@
package client
import (
"context"
"fmt"
apemanagerV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/apemanager"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
sessionV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apeSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
)
// PrmAPEManagerRemoveChain groups parameters of APEManagerRemoveChain operation.
type PrmAPEManagerRemoveChain struct {
XHeaders []string
ChainTarget apeSDK.ChainTarget
ChainID apeSDK.ChainID
}
func (prm *PrmAPEManagerRemoveChain) buildRequest(c *Client) (*apemanagerV2.RemoveChainRequest, error) {
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
req := new(apemanagerV2.RemoveChainRequest)
reqBody := new(apemanagerV2.RemoveChainRequestBody)
reqBody.SetTarget(prm.ChainTarget.ToV2())
reqBody.SetChainID(prm.ChainID)
req.SetBody(reqBody)
var meta sessionV2.RequestMetaHeader
writeXHeadersToMeta(prm.XHeaders, &meta)
c.prepareRequest(req, &meta)
return req, nil
}
type ResAPEManagerRemoveChain struct {
statusRes
}
// APEManagerRemoveChain removes Chain with ChainID defined for ChainTarget.
func (c *Client) APEManagerRemoveChain(ctx context.Context, prm PrmAPEManagerRemoveChain) (*ResAPEManagerRemoveChain, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.RemoveChain(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResAPEManagerRemoveChain
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
return &res, nil
}

View file

@ -46,15 +46,13 @@ func writeXHeadersToMeta(xHeaders []string, h *v2session.RequestMetaHeader) {
// error messages. // error messages.
var ( var (
errorMissingContainer = errors.New("missing container") errorMissingContainer = errors.New("missing container")
errorMissingObject = errors.New("missing object") errorMissingObject = errors.New("missing object")
errorAccountNotSet = errors.New("account not set") errorAccountNotSet = errors.New("account not set")
errorServerAddrUnset = errors.New("server address is unset or empty") errorServerAddrUnset = errors.New("server address is unset or empty")
errorEACLTableNotSet = errors.New("eACL table not set") errorZeroRangeLength = errors.New("zero range length")
errorMissingAnnouncements = errors.New("missing announcements") errorMissingRanges = errors.New("missing ranges")
errorZeroRangeLength = errors.New("zero range length") errorInvalidXHeaders = errors.New("xheaders must be presented only as key-value pairs")
errorMissingRanges = errors.New("missing ranges")
errorInvalidXHeaders = errors.New("xheaders must be presented only as key-value pairs")
) )
type request interface { type request interface {

View file

@ -1,123 +0,0 @@
package client
import (
"context"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmContainerEACL groups parameters of ContainerEACL operation.
type PrmContainerEACL struct {
// FrostFS request X-Headers.
XHeaders []string
ContainerID *cid.ID
Session *session.Container
}
// SetContainer sets identifier of the FrostFS container to read the eACL table.
// Required parameter.
//
// Deprecated: Use PrmContainerEACL.ContainerID instead.
func (x *PrmContainerEACL) SetContainer(id cid.ID) {
x.ContainerID = &id
}
func (x *PrmContainerEACL) buildRequest(c *Client) (*v2container.GetExtendedACLRequest, error) {
if x.ContainerID == nil {
return nil, errorMissingContainer
}
if len(x.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
var cidV2 refs.ContainerID
x.ContainerID.WriteToV2(&cidV2)
reqBody := new(v2container.GetExtendedACLRequestBody)
reqBody.SetContainerID(&cidV2)
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(x.XHeaders, &meta)
if x.Session != nil {
var tokv2 v2session.Token
x.Session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
var req v2container.GetExtendedACLRequest
req.SetBody(reqBody)
c.prepareRequest(&req, &meta)
return &req, nil
}
// ResContainerEACL groups resulting values of ContainerEACL operation.
type ResContainerEACL struct {
statusRes
table eacl.Table
}
// Table returns eACL table of the requested container.
func (x ResContainerEACL) Table() eacl.Table {
return x.table
}
// ContainerEACL reads eACL table of the FrostFS container.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Returns an error if parameters are set incorrectly (see PrmContainerEACL docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs);
// - *apistatus.ContainerNotFound;
// - *apistatus.EACLNotFound.
func (c *Client) ContainerEACL(ctx context.Context, prm PrmContainerEACL) (*ResContainerEACL, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.GetEACL(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResContainerEACL
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
eACL := resp.GetBody().GetEACL()
if eACL == nil {
return &res, newErrMissingResponseField("eACL")
}
res.table = *eacl.NewTableFromV2(eACL)
return &res, nil
}

View file

@ -1,136 +0,0 @@
package client
import (
"context"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmContainerSetEACL groups parameters of ContainerSetEACL operation.
type PrmContainerSetEACL struct {
// FrostFS request X-Headers.
XHeaders []string
Table *eacl.Table
Session *session.Container
}
// SetTable sets eACL table structure to be set for the container.
// Required parameter.
//
// Deprecated: Use PrmContainerSetEACL.Table instead.
func (x *PrmContainerSetEACL) SetTable(table eacl.Table) {
x.Table = &table
}
// WithinSession specifies session within which extended ACL of the container
// should be saved.
//
// Creator of the session acquires the authorship of the request. This affects
// the execution of an operation (e.g. access control).
//
// Session is optional, if set the following requirements apply:
// - if particular container is specified (ApplyOnlyTo), it MUST equal the container
// for which extended ACL is going to be set
// - session operation MUST be session.VerbContainerSetEACL (ForVerb)
// - token MUST be signed using private key of the owner of the container to be saved
//
// Deprecated: Use PrmContainerSetEACL.Session instead.
func (x *PrmContainerSetEACL) WithinSession(s session.Container) {
x.Session = &s
}
func (x *PrmContainerSetEACL) buildRequest(c *Client) (*v2container.SetExtendedACLRequest, error) {
if x.Table == nil {
return nil, errorEACLTableNotSet
}
if len(x.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
eaclV2 := x.Table.ToV2()
var sig frostfscrypto.Signature
err := sig.Calculate(frostfsecdsa.SignerRFC6979(c.prm.Key), eaclV2.StableMarshal(nil))
if err != nil {
return nil, fmt.Errorf("calculate signature: %w", err)
}
var sigv2 refs.Signature
sig.WriteToV2(&sigv2)
reqBody := new(v2container.SetExtendedACLRequestBody)
reqBody.SetEACL(eaclV2)
reqBody.SetSignature(&sigv2)
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(x.XHeaders, &meta)
if x.Session != nil {
var tokv2 v2session.Token
x.Session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
var req v2container.SetExtendedACLRequest
req.SetBody(reqBody)
c.prepareRequest(&req, &meta)
return &req, nil
}
// ResContainerSetEACL groups resulting values of ContainerSetEACL operation.
type ResContainerSetEACL struct {
statusRes
}
// ContainerSetEACL sends request to update eACL table of the FrostFS container.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see EACL).
//
// Returns an error if parameters are set incorrectly (see PrmContainerSetEACL docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerSetEACL(ctx context.Context, prm PrmContainerSetEACL) (*ResContainerSetEACL, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.SetEACL(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResContainerSetEACL
res.st, err = c.processResponse(resp)
return &res, err
}

View file

@ -1,92 +0,0 @@
package client
import (
"context"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
)
// PrmAnnounceSpace groups parameters of ContainerAnnounceUsedSpace operation.
type PrmAnnounceSpace struct {
XHeaders []string
Announcements []container.SizeEstimation
}
// SetValues sets values describing volume of space that is used for the container objects.
// Required parameter. Must not be empty.
//
// Must not be mutated before the end of the operation.
//
// Deprecated: Use PrmAnnounceSpace.Announcements instead.
func (x *PrmAnnounceSpace) SetValues(vs []container.SizeEstimation) {
x.Announcements = vs
}
func (x *PrmAnnounceSpace) buildRequest(c *Client) (*v2container.AnnounceUsedSpaceRequest, error) {
if len(x.Announcements) == 0 {
return nil, errorMissingAnnouncements
}
v2announce := make([]v2container.UsedSpaceAnnouncement, len(x.Announcements))
for i := range x.Announcements {
x.Announcements[i].WriteToV2(&v2announce[i])
}
reqBody := new(v2container.AnnounceUsedSpaceRequestBody)
reqBody.SetAnnouncements(v2announce)
var req v2container.AnnounceUsedSpaceRequest
req.SetBody(reqBody)
c.prepareRequest(&req, new(v2session.RequestMetaHeader))
return &req, nil
}
// ResAnnounceSpace groups resulting values of ContainerAnnounceUsedSpace operation.
type ResAnnounceSpace struct {
statusRes
}
// ContainerAnnounceUsedSpace sends request to announce volume of the space used for the container objects.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// At this moment success can not be checked.
//
// Returns an error if parameters are set incorrectly (see PrmAnnounceSpace docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerAnnounceUsedSpace(ctx context.Context, prm PrmAnnounceSpace) (*ResAnnounceSpace, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.AnnounceUsedSpace(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResAnnounceSpace
res.st, err = c.processResponse(resp)
return &res, err
}

View file

@ -61,6 +61,18 @@ func IsErrSessionNotFound(err error) bool {
return wrapsErrType[*apistatus.SessionTokenNotFound](err) return wrapsErrType[*apistatus.SessionTokenNotFound](err)
} }
// IsErrAPEManagerAccessDenied checks if err corresponds to FrostFS status return
// corresponding to apemanager access deny. Supports wrapped errors.
func IsErrAPEManagerAccessDenied(err error) bool {
return wrapsErrType[*apistatus.APEManagerAccessDenied](err)
}
// IsErrNodeUnderMaintenance checks if err corresponds to FrostFS status return
// corresponding to nodes being under maintenance. Supports wrapped errors.
func IsErrNodeUnderMaintenance(err error) bool {
return wrapsErrType[*apistatus.NodeUnderMaintenance](err)
}
// returns error describing missing field with the given name. // returns error describing missing field with the given name.
func newErrMissingResponseField(name string) error { func newErrMissingResponseField(name string) error {
return fmt.Errorf("missing %s field in the response", name) return fmt.Errorf("missing %s field in the response", name)

View file

@ -38,6 +38,10 @@ func TestErrors(t *testing.T) {
check: client.IsErrSessionNotFound, check: client.IsErrSessionNotFound,
err: new(apistatus.SessionTokenNotFound), err: new(apistatus.SessionTokenNotFound),
}, },
{
check: client.IsErrNodeUnderMaintenance,
err: new(apistatus.NodeUnderMaintenance),
},
} }
for i := range errs { for i := range errs {

View file

@ -150,6 +150,9 @@ func (x *ObjectReader) ReadHeader(dst *object.Object) bool {
case *v2object.SplitInfo: case *v2object.SplitInfo:
x.err = object.NewSplitInfoError(object.NewSplitInfoFromV2(v)) x.err = object.NewSplitInfoError(object.NewSplitInfoFromV2(v))
return false return false
case *v2object.ECInfo:
x.err = object.NewECInfoError(object.NewECInfoFromV2(v))
return false
case *v2object.GetObjectPartInit: case *v2object.GetObjectPartInit:
partInit = v partInit = v
} }
@ -258,6 +261,7 @@ func (x *ObjectReader) close(ignoreEOF bool) (*ResObjectGet, error) {
// Return errors: // Return errors:
// //
// *object.SplitInfoError (returned on virtual objects with PrmObjectGet.MakeRaw). // *object.SplitInfoError (returned on virtual objects with PrmObjectGet.MakeRaw).
// *object.ECInfoError (returned on erasure-coded objects with PrmObjectGet.MakeRaw).
// //
// Return statuses: // Return statuses:
// - global (see Client docs); // - global (see Client docs);
@ -454,6 +458,7 @@ func (prm *PrmObjectHead) buildRequest(c *Client) (*v2object.HeadRequest, error)
// Return errors: // Return errors:
// //
// *object.SplitInfoError (returned on virtual objects with PrmObjectHead.MakeRaw). // *object.SplitInfoError (returned on virtual objects with PrmObjectHead.MakeRaw).
// *object.ECInfoError (returned on erasure-coded objects with PrmObjectHead.MakeRaw).
// //
// Return statuses: // Return statuses:
// - global (see Client docs); // - global (see Client docs);
@ -502,6 +507,8 @@ func (c *Client) ObjectHead(ctx context.Context, prm PrmObjectHead) (*ResObjectH
return nil, fmt.Errorf("unexpected header type %T", v) return nil, fmt.Errorf("unexpected header type %T", v)
case *v2object.SplitInfo: case *v2object.SplitInfo:
return nil, object.NewSplitInfoError(object.NewSplitInfoFromV2(v)) return nil, object.NewSplitInfoError(object.NewSplitInfoFromV2(v))
case *v2object.ECInfo:
return nil, object.NewECInfoError(object.NewECInfoFromV2(v))
case *v2object.HeaderWithSignature: case *v2object.HeaderWithSignature:
res.hdr = v res.hdr = v
} }
@ -665,6 +672,9 @@ func (x *ObjectRangeReader) readChunk(buf []byte) (int, bool) {
case *v2object.SplitInfo: case *v2object.SplitInfo:
x.err = object.NewSplitInfoError(object.NewSplitInfoFromV2(v)) x.err = object.NewSplitInfoError(object.NewSplitInfoFromV2(v))
return read, false return read, false
case *v2object.ECInfo:
x.err = object.NewECInfoError(object.NewECInfoFromV2(v))
return read, false
case *v2object.GetRangePartChunk: case *v2object.GetRangePartChunk:
partChunk = v partChunk = v
} }
@ -725,6 +735,7 @@ func (x *ObjectRangeReader) close(ignoreEOF bool) (*ResObjectRange, error) {
// Return errors: // Return errors:
// //
// *object.SplitInfoError (returned on virtual objects with PrmObjectRange.MakeRaw). // *object.SplitInfoError (returned on virtual objects with PrmObjectRange.MakeRaw).
// *object.ECInfoError (returned on erasure-coded objects with PrmObjectRange.MakeRaw).
// //
// Return statuses: // Return statuses:
// - global (see Client docs); // - global (see Client docs);

270
client/object_patch.go Normal file
View file

@ -0,0 +1,270 @@
package client
import (
"context"
"crypto/ecdsa"
"errors"
"fmt"
"io"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// ObjectPatcher is designed to patch an object.
//
// Must be initialized using Client.ObjectPatchInit, any other
// usage is unsafe.
type ObjectPatcher interface {
// PatchAttributes patches attributes. Attributes can be patched no more than once,
// otherwise, the server returns an error.
//
// Result means success. Failure reason can be received via Close.
PatchAttributes(ctx context.Context, newAttrs []object.Attribute, replace bool) bool
// PatchPayload patches the object's payload.
//
// PatchPayload receives `payloadReader` and thus the payload of the patch is read and sent by chunks of
// `MaxChunkLength` length.
//
// Result means success. Failure reason can be received via Close.
PatchPayload(ctx context.Context, rng *object.Range, payloadReader io.Reader) bool
// Close ends patching the object and returns the result of the operation
// along with the final results. Must be called after using the ObjectPatcher.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as Go built-in error.
// If Client is tuned to resolve FrostFS API statuses, then FrostFS failures
// codes are returned as error.
//
// Return statuses:
// - global (see Client docs);
// - *apistatus.ContainerNotFound;
// - *apistatus.ContainerAccessDenied;
// - *apistatus.ObjectAccessDenied;
// - *apistatus.ObjectAlreadyRemoved;
// - *apistatus.ObjectLocked;
// - *apistatus.ObjectOutOfRange;
// - *apistatus.SessionTokenNotFound;
// - *apistatus.SessionTokenExpired.
Close(_ context.Context) (*ResObjectPatch, error)
}
// ResObjectPatch groups resulting values of ObjectPatch operation.
type ResObjectPatch struct {
statusRes
obj oid.ID
}
// ObjectID returns an object ID of the patched object.
func (r ResObjectPatch) ObjectID() oid.ID {
return r.obj
}
// PrmObjectPatch groups parameters of ObjectPatch operation.
type PrmObjectPatch struct {
XHeaders []string
Address oid.Address
BearerToken *bearer.Token
Session *session.Object
Key *ecdsa.PrivateKey
MaxChunkLength int
}
// ObjectPatchInit initializes object patcher.
func (c *Client) ObjectPatchInit(ctx context.Context, prm PrmObjectPatch) (ObjectPatcher, error) {
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
var objectPatcher objectPatcher
stream, err := rpcapi.Patch(&c.c, &objectPatcher.respV2, client.WithContext(ctx))
if err != nil {
return nil, fmt.Errorf("open stream: %w", err)
}
objectPatcher.addr = prm.Address
objectPatcher.key = &c.prm.Key
if prm.Key != nil {
objectPatcher.key = prm.Key
}
objectPatcher.client = c
objectPatcher.stream = stream
if prm.MaxChunkLength > 0 {
objectPatcher.maxChunkLen = prm.MaxChunkLength
} else {
objectPatcher.maxChunkLen = defaultGRPCPayloadChunkLen
}
objectPatcher.req.SetBody(&v2object.PatchRequestBody{})
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
c.prepareRequest(&objectPatcher.req, meta)
return &objectPatcher, nil
}
type objectPatcher struct {
client *Client
stream interface {
Write(*v2object.PatchRequest) error
Close() error
}
key *ecdsa.PrivateKey
res ResObjectPatch
err error
addr oid.Address
req v2object.PatchRequest
respV2 v2object.PatchResponse
maxChunkLen int
}
func (x *objectPatcher) PatchAttributes(_ context.Context, newAttrs []object.Attribute, replace bool) bool {
return x.patch(&object.Patch{
Address: x.addr,
NewAttributes: newAttrs,
ReplaceAttributes: replace,
})
}
func (x *objectPatcher) PatchPayload(_ context.Context, rng *object.Range, payloadReader io.Reader) bool {
offset := rng.GetOffset()
buf := make([]byte, x.maxChunkLen)
for patchIter := 0; ; patchIter++ {
n, err := payloadReader.Read(buf)
if err != nil && err != io.EOF {
x.err = fmt.Errorf("read payload: %w", err)
return false
}
if n == 0 {
if patchIter == 0 {
if rng.GetLength() == 0 {
x.err = errors.New("zero-length empty payload patch can't be applied")
return false
}
if !x.patch(&object.Patch{
Address: x.addr,
PayloadPatch: &object.PayloadPatch{
Range: rng,
Chunk: []byte{},
},
}) {
return false
}
}
break
}
rngPart := object.NewRange()
if patchIter == 0 {
rngPart.SetOffset(offset)
rngPart.SetLength(rng.GetLength())
} else {
rngPart.SetOffset(offset + rng.GetLength())
}
if !x.patch(&object.Patch{
Address: x.addr,
PayloadPatch: &object.PayloadPatch{
Range: rngPart,
Chunk: buf[:n],
},
}) {
return false
}
if err == io.EOF {
break
}
}
return true
}
func (x *objectPatcher) patch(patch *object.Patch) bool {
x.req.SetBody(patch.ToV2())
x.req.SetVerificationHeader(nil)
x.err = signature.SignServiceMessage(x.key, &x.req)
if x.err != nil {
x.err = fmt.Errorf("sign message: %w", x.err)
return false
}
x.err = x.stream.Write(&x.req)
return x.err == nil
}
func (x *objectPatcher) Close(_ context.Context) (*ResObjectPatch, error) {
// Ignore io.EOF error, because it is expected error for client-side
// stream termination by the server. E.g. when stream contains invalid
// message. Server returns an error in response message (in status).
if x.err != nil && !errors.Is(x.err, io.EOF) {
return nil, x.err
}
if x.err = x.stream.Close(); x.err != nil {
return nil, x.err
}
x.res.st, x.err = x.client.processResponse(&x.respV2)
if x.err != nil {
return nil, x.err
}
if !apistatus.IsSuccessful(x.res.st) {
return &x.res, nil
}
const fieldID = "ID"
idV2 := x.respV2.Body.ObjectID
if idV2 == nil {
return nil, newErrMissingResponseField(fieldID)
}
x.err = x.res.obj.ReadFromV2(*idV2)
if x.err != nil {
x.err = newErrInvalidResponseField(fieldID, x.err)
}
return &x.res, nil
}

295
client/object_patch_test.go Normal file
View file

@ -0,0 +1,295 @@
package client
import (
"bytes"
"context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"testing"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id/test"
"github.com/stretchr/testify/require"
)
type mockPatchStream struct {
streamedPayloadPatches []*object.PayloadPatch
}
func (m *mockPatchStream) Write(r *v2object.PatchRequest) error {
pp := new(object.PayloadPatch)
pp.FromV2(r.GetBody().GetPatch())
if r.GetBody().GetPatch() != nil {
bodyChunk := r.GetBody().GetPatch().Chunk
pp.Chunk = make([]byte, len(bodyChunk))
copy(pp.Chunk, bodyChunk)
}
m.streamedPayloadPatches = append(m.streamedPayloadPatches, pp)
return nil
}
func (m *mockPatchStream) Close() error {
return nil
}
func TestObjectPatcher(t *testing.T) {
type part struct {
offset int
length int
chunk string
}
for _, test := range []struct {
name string
patchPayload string
rng *object.Range
maxChunkLen int
expectParts []part
}{
{
name: "no split payload patch",
patchPayload: "011111",
rng: newRange(0, 6),
maxChunkLen: defaultGRPCPayloadChunkLen,
expectParts: []part{
{
offset: 0,
length: 6,
chunk: "011111",
},
},
},
{
name: "splitted payload patch",
patchPayload: "012345",
rng: newRange(0, 6),
maxChunkLen: 2,
expectParts: []part{
{
offset: 0,
length: 6,
chunk: "01",
},
{
offset: 6,
length: 0,
chunk: "23",
},
{
offset: 6,
length: 0,
chunk: "45",
},
},
},
{
name: "splitted payload patch with zero-length subpatches",
patchPayload: "0123456789!@",
rng: newRange(0, 4),
maxChunkLen: 2,
expectParts: []part{
{
offset: 0,
length: 4,
chunk: "01",
},
{
offset: 4,
length: 0,
chunk: "23",
},
{
offset: 4,
length: 0,
chunk: "45",
},
{
offset: 4,
length: 0,
chunk: "67",
},
{
offset: 4,
length: 0,
chunk: "89",
},
{
offset: 4,
length: 0,
chunk: "!@",
},
},
},
{
name: "splitted payload patch with zero-length subpatches only",
patchPayload: "0123456789!@",
rng: newRange(0, 0),
maxChunkLen: 2,
expectParts: []part{
{
offset: 0,
length: 0,
chunk: "01",
},
{
offset: 0,
length: 0,
chunk: "23",
},
{
offset: 0,
length: 0,
chunk: "45",
},
{
offset: 0,
length: 0,
chunk: "67",
},
{
offset: 0,
length: 0,
chunk: "89",
},
{
offset: 0,
length: 0,
chunk: "!@",
},
},
},
} {
t.Run(test.name, func(t *testing.T) {
m := &mockPatchStream{}
pk, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
patcher := objectPatcher{
client: &Client{},
stream: m,
addr: oidtest.Address(),
key: pk,
maxChunkLen: test.maxChunkLen,
}
success := patcher.PatchAttributes(context.Background(), nil, false)
require.True(t, success)
success = patcher.PatchPayload(context.Background(), test.rng, bytes.NewReader([]byte(test.patchPayload)))
require.True(t, success)
require.Len(t, m.streamedPayloadPatches, len(test.expectParts)+1)
// m.streamedPayloadPatches[0] is attribute patch, so skip it
for i, part := range test.expectParts {
requireRangeChunk(t, m.streamedPayloadPatches[i+1], part.offset, part.length, part.chunk)
}
})
}
}
func TestRepeatPayloadPatch(t *testing.T) {
t.Run("no payload patch partioning", func(t *testing.T) {
m := &mockPatchStream{}
pk, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
const maxChunkLen = 20
patcher := objectPatcher{
client: &Client{},
stream: m,
addr: oidtest.Address(),
key: pk,
maxChunkLen: maxChunkLen,
}
for _, pp := range []struct {
patchPayload string
rng *object.Range
}{
{
patchPayload: "xxxxxxxxxx",
rng: newRange(1, 6),
},
{
patchPayload: "yyyyyyyyyy",
rng: newRange(5, 9),
},
{
patchPayload: "zzzzzzzzzz",
rng: newRange(10, 0),
},
} {
success := patcher.PatchPayload(context.Background(), pp.rng, bytes.NewReader([]byte(pp.patchPayload)))
require.True(t, success)
}
requireRangeChunk(t, m.streamedPayloadPatches[0], 1, 6, "xxxxxxxxxx")
requireRangeChunk(t, m.streamedPayloadPatches[1], 5, 9, "yyyyyyyyyy")
requireRangeChunk(t, m.streamedPayloadPatches[2], 10, 0, "zzzzzzzzzz")
})
t.Run("payload patch partioning", func(t *testing.T) {
m := &mockPatchStream{}
pk, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
const maxChunkLen = 5
patcher := objectPatcher{
client: &Client{},
stream: m,
addr: oidtest.Address(),
key: pk,
maxChunkLen: maxChunkLen,
}
for _, pp := range []struct {
patchPayload string
rng *object.Range
}{
{
patchPayload: "xxxxxxxxxx",
rng: newRange(1, 6),
},
{
patchPayload: "yyyyyyyyyy",
rng: newRange(5, 9),
},
{
patchPayload: "zzzzzzzzzz",
rng: newRange(10, 0),
},
} {
success := patcher.PatchPayload(context.Background(), pp.rng, bytes.NewReader([]byte(pp.patchPayload)))
require.True(t, success)
}
requireRangeChunk(t, m.streamedPayloadPatches[0], 1, 6, "xxxxx")
requireRangeChunk(t, m.streamedPayloadPatches[1], 7, 0, "xxxxx")
requireRangeChunk(t, m.streamedPayloadPatches[2], 5, 9, "yyyyy")
requireRangeChunk(t, m.streamedPayloadPatches[3], 14, 0, "yyyyy")
requireRangeChunk(t, m.streamedPayloadPatches[4], 10, 0, "zzzzz")
requireRangeChunk(t, m.streamedPayloadPatches[5], 10, 0, "zzzzz")
})
}
func requireRangeChunk(t *testing.T, pp *object.PayloadPatch, offset, length int, chunk string) {
require.NotNil(t, pp)
require.Equal(t, uint64(offset), pp.Range.GetOffset())
require.Equal(t, uint64(length), pp.Range.GetLength())
require.Equal(t, []byte(chunk), pp.Chunk)
}
func newRange(offest, length uint64) *object.Range {
rng := &object.Range{}
rng.SetOffset(offest)
rng.SetLength(length)
return rng
}

View file

@ -4,6 +4,7 @@ import (
"context" "context"
"crypto/ecdsa" "crypto/ecdsa"
buffPool "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/util/pool"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id" oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
@ -36,6 +37,8 @@ type PrmObjectPutInit struct {
WithoutHomomorphHash bool WithoutHomomorphHash bool
Key *ecdsa.PrivateKey Key *ecdsa.PrivateKey
Pool *buffPool.BufferPool
} }
// SetCopiesNumber sets number of object copies that is enough to consider put successful. // SetCopiesNumber sets number of object copies that is enough to consider put successful.
@ -69,7 +72,8 @@ func (x *PrmObjectPutInit) SetGRPCPayloadChunkLen(v int) {
type ResObjectPut struct { type ResObjectPut struct {
statusRes statusRes
obj oid.ID obj oid.ID
epoch uint64
} }
// StoredObjectID returns identifier of the saved object. // StoredObjectID returns identifier of the saved object.
@ -77,6 +81,11 @@ func (x ResObjectPut) StoredObjectID() oid.ID {
return x.obj return x.obj
} }
// StoredEpoch returns creation epoch of the saved object.
func (x ResObjectPut) StoredEpoch() uint64 {
return x.epoch
}
// ObjectWriter is designed to write one object or // ObjectWriter is designed to write one object or
// multiple parts of one object to FrostFS system. // multiple parts of one object to FrostFS system.
// //

View file

@ -175,6 +175,7 @@ func (x *objectWriterRaw) Close(_ context.Context) (*ResObjectPut, error) {
if x.err != nil { if x.err != nil {
x.err = newErrInvalidResponseField(fieldID, x.err) x.err = newErrInvalidResponseField(fieldID, x.err)
} }
x.res.epoch = x.respV2.GetMetaHeader().GetEpoch()
return &x.res, nil return &x.res, nil
} }

View file

@ -93,6 +93,13 @@ func (prm *PrmObjectPutSingle) SetObject(o *v2object.Object) {
// ResObjectPutSingle groups resulting values of PutSingle operation. // ResObjectPutSingle groups resulting values of PutSingle operation.
type ResObjectPutSingle struct { type ResObjectPutSingle struct {
statusRes statusRes
epoch uint64
}
// Epoch returns creation epoch of the saved object.
func (r *ResObjectPutSingle) Epoch() uint64 {
return r.epoch
} }
func (prm *PrmObjectPutSingle) buildRequest(c *Client) (*v2object.PutSingleRequest, error) { func (prm *PrmObjectPutSingle) buildRequest(c *Client) (*v2object.PutSingleRequest, error) {
@ -162,6 +169,7 @@ func (c *Client) ObjectPutSingle(ctx context.Context, prm PrmObjectPutSingle) (*
if err != nil { if err != nil {
return nil, err return nil, err
} }
res.epoch = resp.GetMetaHeader().GetEpoch()
return &res, nil return &res, nil
} }

View file

@ -3,6 +3,7 @@ package client
import ( import (
"context" "context"
buffPool "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/util/pool"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status" apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer"
@ -26,6 +27,7 @@ func (c *Client) objectPutInitTransformer(prm PrmObjectPutInit) (*objectWriterTr
MaxSize: prm.MaxSize, MaxSize: prm.MaxSize,
WithoutHomomorphicHash: prm.WithoutHomomorphHash, WithoutHomomorphicHash: prm.WithoutHomomorphHash,
NetworkState: prm.EpochSource, NetworkState: prm.EpochSource,
Pool: prm.Pool,
}) })
return &w, nil return &w, nil
} }
@ -56,8 +58,11 @@ func (x *objectWriterTransformer) Close(ctx context.Context) (*ResObjectPut, err
return nil, err return nil, err
} }
if ai != nil && ai.ParentID != nil { if ai != nil {
x.it.res.obj = *ai.ParentID x.it.res.epoch = ai.Epoch
if ai.ParentID != nil {
x.it.res.obj = *ai.ParentID
}
} }
return x.it.res, nil return x.it.res, nil
} }
@ -114,10 +119,12 @@ func (it *internalTarget) tryPutSingle(ctx context.Context, o *object.Object) (b
} }
if err == nil { if err == nil {
it.returnBuffPool(o.Payload())
id, _ := o.ID() id, _ := o.ID()
it.res = &ResObjectPut{ it.res = &ResObjectPut{
statusRes: res.statusRes, statusRes: res.statusRes,
obj: id, obj: id,
epoch: res.epoch,
} }
if it.client.prm.DisableFrostFSErrorResolution && !apistatus.IsSuccessful(it.res.st) { if it.client.prm.DisableFrostFSErrorResolution && !apistatus.IsSuccessful(it.res.st) {
return true, apistatus.ErrFromStatus(it.res.st) return true, apistatus.ErrFromStatus(it.res.st)
@ -126,3 +133,12 @@ func (it *internalTarget) tryPutSingle(ctx context.Context, o *object.Object) (b
} }
return true, err return true, err
} }
func (it *internalTarget) returnBuffPool(playback []byte) {
if it.prm.Pool == nil {
return
}
var buffer buffPool.Buffer
buffer.Data = playback
it.prm.Pool.Put(&buffer)
}

View file

@ -0,0 +1,53 @@
package apistatus
import (
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/apemanager"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
)
// APEManagerAccessDenied describes status of the failure because of the access control violation.
// Instances provide Status and StatusV2 interfaces.
type APEManagerAccessDenied struct {
v2 status.Status
}
const defaultAPEManagerAccessDeniedMsg = "apemanager access denied"
func (x *APEManagerAccessDenied) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultAPEManagerAccessDeniedMsg
}
return errMessageStatusV2(
globalizeCodeV2(apemanager.StatusAPEManagerAccessDenied, apemanager.GlobalizeFail),
msg,
)
}
func (x *APEManagerAccessDenied) fromStatusV2(st *status.Status) {
x.v2 = *st
}
// ToStatusV2 converts APEManagerAccessDenied to v2's Status.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// - code: APE_MANAGER_ACCESS_DENIED;
// - string message: "apemanager access denied";
// - details: empty.
func (x APEManagerAccessDenied) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(apemanager.StatusAPEManagerAccessDenied, apemanager.GlobalizeFail))
x.v2.SetMessage(defaultAPEManagerAccessDeniedMsg)
return &x.v2
}
// WriteReason writes human-readable access rejection reason.
func (x *APEManagerAccessDenied) WriteReason(reason string) {
apemanager.WriteAccessDeniedDesc(&x.v2, reason)
}
// Reason returns human-readable access rejection reason returned by the server.
// Returns empty value is reason is not presented.
func (x APEManagerAccessDenied) Reason() string {
return apemanager.ReadAccessDeniedDesc(x.v2)
}

View file

@ -3,6 +3,7 @@ package apistatus
import ( import (
"fmt" "fmt"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/apemanager"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
@ -92,6 +93,12 @@ func FromStatusV2(st *status.Status) Status {
case session.StatusTokenExpired: case session.StatusTokenExpired:
decoder = new(SessionTokenExpired) decoder = new(SessionTokenExpired)
} }
case apemanager.LocalizeFailStatus(&code):
//nolint:exhaustive
switch code {
case apemanager.StatusAPEManagerAccessDenied:
decoder = new(APEManagerAccessDenied)
}
} }
if decoder == nil { if decoder == nil {

View file

@ -125,6 +125,12 @@ func TestToStatusV2(t *testing.T) {
}), }),
codeV2: 4097, codeV2: 4097,
}, },
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.APEManagerAccessDenied)
}),
codeV2: 5120,
},
{ {
status: (statusConstructor)(func() apistatus.Status { status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.NodeUnderMaintenance) return new(apistatus.NodeUnderMaintenance)
@ -278,6 +284,12 @@ func TestFromStatusV2(t *testing.T) {
}), }),
codeV2: 4097, codeV2: 4097,
}, },
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.APEManagerAccessDenied)
}),
codeV2: 5120,
},
{ {
status: (statusConstructor)(func() apistatus.Status { status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.NodeUnderMaintenance) return new(apistatus.NodeUnderMaintenance)

View file

@ -308,7 +308,7 @@ func (x *Container) SetAttribute(key, value string) {
attrs := x.v2.GetAttributes() attrs := x.v2.GetAttributes()
ln := len(attrs) ln := len(attrs)
for i := 0; i < ln; i++ { for i := range ln {
if attrs[i].GetKey() == key { if attrs[i].GetKey() == key {
attrs[i].SetValue(value) attrs[i].SetValue(value)
return return

View file

@ -1,104 +0,0 @@
package container
import (
"errors"
"fmt"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
)
// SizeEstimation groups information about estimation of the size of the data
// stored in the FrostFS container.
//
// SizeEstimation is mutually compatible with git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container.UsedSpaceAnnouncement
// message. See ReadFromV2 / WriteToV2 methods.
type SizeEstimation struct {
m container.UsedSpaceAnnouncement
}
// ReadFromV2 reads SizeEstimation from the container.UsedSpaceAnnouncement message.
// Checks if the message conforms to FrostFS API V2 protocol.
//
// See also WriteToV2.
func (x *SizeEstimation) ReadFromV2(m container.UsedSpaceAnnouncement) error {
cnrV2 := m.GetContainerID()
if cnrV2 == nil {
return errors.New("missing container")
}
var cnr cid.ID
err := cnr.ReadFromV2(*cnrV2)
if err != nil {
return fmt.Errorf("invalid container: %w", err)
}
x.m = m
return nil
}
// WriteToV2 writes SizeEstimation into the container.UsedSpaceAnnouncement message.
// The message MUST NOT be nil.
//
// See also ReadFromV2.
func (x SizeEstimation) WriteToV2(m *container.UsedSpaceAnnouncement) {
*m = x.m
}
// SetEpoch sets epoch when estimation of the container data size was calculated.
//
// See also Epoch.
func (x *SizeEstimation) SetEpoch(epoch uint64) {
x.m.SetEpoch(epoch)
}
// Epoch return epoch set using SetEpoch.
//
// Zero SizeEstimation represents estimation in zero epoch.
func (x SizeEstimation) Epoch() uint64 {
return x.m.GetEpoch()
}
// SetContainer specifies the container for which the amount of data is estimated.
// Required by the FrostFS API protocol.
//
// See also Container.
func (x *SizeEstimation) SetContainer(cnr cid.ID) {
var cidV2 refs.ContainerID
cnr.WriteToV2(&cidV2)
x.m.SetContainerID(&cidV2)
}
// Container returns container set using SetContainer.
//
// Zero SizeEstimation is not bound to any container (returns zero) which is
// incorrect according to FrostFS API protocol.
func (x SizeEstimation) Container() (res cid.ID) {
m := x.m.GetContainerID()
if m != nil {
err := res.ReadFromV2(*m)
if err != nil {
panic(fmt.Errorf("unexpected error from cid.ID.ReadFromV2: %w", err))
}
}
return
}
// SetValue sets estimated amount of data (in bytes) in the specified container.
//
// See also Value.
func (x *SizeEstimation) SetValue(value uint64) {
x.m.SetUsedSpace(value)
}
// Value returns data size estimation set using SetValue.
//
// Zero SizeEstimation has zero value.
func (x SizeEstimation) Value() uint64 {
return x.m.GetUsedSpace()
}

View file

@ -1,94 +0,0 @@
package container_test
import (
"crypto/sha256"
"testing"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
cidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id/test"
"github.com/stretchr/testify/require"
)
func TestSizeEstimation_Epoch(t *testing.T) {
var val container.SizeEstimation
require.Zero(t, val.Epoch())
const epoch = 123
val.SetEpoch(epoch)
require.EqualValues(t, epoch, val.Epoch())
var msg v2container.UsedSpaceAnnouncement
val.WriteToV2(&msg)
require.EqualValues(t, epoch, msg.GetEpoch())
}
func TestSizeEstimation_Container(t *testing.T) {
var val container.SizeEstimation
require.Zero(t, val.Container())
cnr := cidtest.ID()
val.SetContainer(cnr)
require.True(t, val.Container().Equals(cnr))
var msg v2container.UsedSpaceAnnouncement
val.WriteToV2(&msg)
var msgCnr refs.ContainerID
cnr.WriteToV2(&msgCnr)
require.Equal(t, &msgCnr, msg.GetContainerID())
}
func TestSizeEstimation_Value(t *testing.T) {
var val container.SizeEstimation
require.Zero(t, val.Value())
const value = 876
val.SetValue(value)
require.EqualValues(t, value, val.Value())
var msg v2container.UsedSpaceAnnouncement
val.WriteToV2(&msg)
require.EqualValues(t, value, msg.GetUsedSpace())
}
func TestSizeEstimation_ReadFromV2(t *testing.T) {
const epoch = 654
const value = 903
var cnrMsg refs.ContainerID
var msg v2container.UsedSpaceAnnouncement
var val container.SizeEstimation
require.Error(t, val.ReadFromV2(msg))
msg.SetContainerID(&cnrMsg)
require.Error(t, val.ReadFromV2(msg))
cnrMsg.SetValue(make([]byte, sha256.Size))
var cnr cid.ID
require.NoError(t, cnr.ReadFromV2(cnrMsg))
msg.SetEpoch(epoch)
msg.SetUsedSpace(value)
require.NoError(t, val.ReadFromV2(msg))
require.EqualValues(t, epoch, val.Epoch())
require.EqualValues(t, value, val.Value())
require.EqualValues(t, cnr, val.Container())
}

View file

@ -5,7 +5,6 @@ import (
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/acl" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/acl"
cidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id/test"
netmaptest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap/test" netmaptest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap/test"
usertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user/test" usertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user/test"
) )
@ -23,15 +22,6 @@ func Container() (x container.Container) {
return x return x
} }
// SizeEstimation returns random container.SizeEstimation.
func SizeEstimation() (x container.SizeEstimation) {
x.SetContainer(cidtest.ID())
x.SetEpoch(rand.Uint64())
x.SetValue(rand.Uint64())
return x
}
// BasicACL returns random acl.Basic. // BasicACL returns random acl.Basic.
func BasicACL() (x acl.Basic) { func BasicACL() (x acl.Basic) {
x.FromBits(rand.Uint32()) x.FromBits(rand.Uint32())

BIN
doc/image/pool.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 194 KiB

View file

@ -428,6 +428,13 @@ Comparison operators (all binary):
- `LE`: less or equal - `LE`: less or equal
- `LT`: less than - `LT`: less than
Pattern operator:
- `LIKE`: specifies pattern for an attribute. Uses as a wildcard symbol `*`.
- `... ATTR LIKE "VAL"` - the behaviour is equal to `EQ`
- `... ATTR LIKE "VAL*"` - matches all which starts with `VAL`
- `... ATTR LIKE "*VAL"` - matches all which ends with `VAL`
- `... ATTR LIKE "*VAL*"` - matches all which contains `VAL`
Logical operators: Logical operators:
- `NOT`: negation (unary) - `NOT`: negation (unary)
- `AND`: conjunction (binary) - `AND`: conjunction (binary)

27
doc/pool.md Normal file
View file

@ -0,0 +1,27 @@
# Node connection pool
* Distributes requests between fixed number of nodes
* Wraps
The distribution between nodes in connection pool is based on priority and weight parameters from
NodeParam struct. The distribution model is presented below. On this scheme nodes with the same
priority have the same color.
![](./image/pool.png "Pool connections distribution model")
## Priority
Pool component forwards request to the nodes with the highest priority (the lower the value -
the higher the priority). The `node 1` from the image's scenario (I) is healthy
and has the highest priority (1), that's why the pool forwards requests from it. There are no other
nodes with priority 1, so `node 1` receives all requests. In the second scenario (II) `node 1`
becomes unhealthy. In that case pool tries to connect nodes with next in priority nodes e.g.
`Node 4` and `node 2`. If all of them become unhealthy too, the pool sends requests to nodes with
priority 3 in scenario (III) and so on.
## Weights
If there are several nodes with the same priority, then requests are distributed randomly between
these nodes based on their weights. To do that the proportion of weights is calculated.
For example, for `node 2` and `node 4` with weights 2 and 8 the distribution would be 20 and 80 percent
respectively.

View file

@ -286,13 +286,13 @@ func equalRecords(r1, r2 Record) bool {
return false return false
} }
for i := 0; i < len(fs1); i++ { for i := range len(fs1) {
if !equalFilters(fs1[i], fs2[i]) { if !equalFilters(fs1[i], fs2[i]) {
return false return false
} }
} }
for i := 0; i < len(ts1); i++ { for i := range len(ts1) {
if !equalTargets(ts1[i], ts2[i]) { if !equalTargets(ts1[i], ts2[i]) {
return false return false
} }

View file

@ -212,7 +212,7 @@ func EqualTables(t1, t2 Table) bool {
return false return false
} }
for i := 0; i < len(rs1); i++ { for i := range len(rs1) {
if !equalRecords(rs1[i], rs2[i]) { if !equalRecords(rs1[i], rs2[i]) {
return false return false
} }

View file

@ -51,7 +51,7 @@ func SetTargetECDSAKeys(t *Target, pubs ...*ecdsa.PublicKey) {
binKeys = make([][]byte, 0, ln) binKeys = make([][]byte, 0, ln)
} }
for i := 0; i < ln; i++ { for i := range ln {
binKeys = append(binKeys, (*keys.PublicKey)(pubs[i]).Bytes()) binKeys = append(binKeys, (*keys.PublicKey)(pubs[i]).Bytes())
} }
@ -67,7 +67,7 @@ func TargetECDSAKeys(t *Target) []*ecdsa.PublicKey {
pubs := make([]*ecdsa.PublicKey, ln) pubs := make([]*ecdsa.PublicKey, ln)
for i := 0; i < ln; i++ { for i := range ln {
p := new(keys.PublicKey) p := new(keys.PublicKey)
if p.DecodeBytes(binKeys[i]) == nil { if p.DecodeBytes(binKeys[i]) == nil {
pubs[i] = (*ecdsa.PublicKey)(p) pubs[i] = (*ecdsa.PublicKey)(p)
@ -169,7 +169,7 @@ func equalTargets(t1, t2 Target) bool {
return false return false
} }
for i := 0; i < len(keys1); i++ { for i := range len(keys1) {
if !bytes.Equal(keys1[i], keys2[i]) { if !bytes.Equal(keys1[i], keys2[i]) {
return false return false
} }

View file

@ -19,7 +19,7 @@ func baseBenchmarkTableBinaryComparison(b *testing.B, factor int) {
b.StopTimer() b.StopTimer()
b.ResetTimer() b.ResetTimer()
b.StartTimer() b.StartTimer()
for i := 0; i < b.N; i++ { for range b.N {
got, _ := t.Marshal() got, _ := t.Marshal()
if !bytes.Equal(exp, got) { if !bytes.Equal(exp, got) {
b.Fail() b.Fail()
@ -38,7 +38,7 @@ func baseBenchmarkTableEqualsComparison(b *testing.B, factor int) {
b.StopTimer() b.StopTimer()
b.ResetTimer() b.ResetTimer()
b.StartTimer() b.StartTimer()
for i := 0; i < b.N; i++ { for range b.N {
if !eacl.EqualTables(*t, *t2) { if !eacl.EqualTables(*t, *t2) {
b.Fail() b.Fail()
} }
@ -76,7 +76,7 @@ func TargetN(n int) *eacl.Target {
x.SetRole(eacl.RoleSystem) x.SetRole(eacl.RoleSystem)
keys := make([][]byte, n) keys := make([][]byte, n)
for i := 0; i < n; i++ { for i := range n {
keys[i] = make([]byte, 32) keys[i] = make([]byte, 32)
rand.Read(keys[i]) rand.Read(keys[i])
} }
@ -94,7 +94,7 @@ func RecordN(n int) *eacl.Record {
x.SetOperation(eacl.OperationRangeHash) x.SetOperation(eacl.OperationRangeHash)
x.SetTargets(*TargetN(n)) x.SetTargets(*TargetN(n))
for i := 0; i < n; i++ { for range n {
x.AddFilter(eacl.HeaderFromObject, eacl.MatchStringEqual, "", cidtest.ID().EncodeToString()) x.AddFilter(eacl.HeaderFromObject, eacl.MatchStringEqual, "", cidtest.ID().EncodeToString())
} }
@ -106,7 +106,7 @@ func TableN(n int) *eacl.Table {
x.SetCID(cidtest.ID()) x.SetCID(cidtest.ID())
for i := 0; i < n; i++ { for range n {
x.AddRecord(RecordN(n)) x.AddRecord(RecordN(n))
} }

55
go.mod
View file

@ -1,46 +1,49 @@
module git.frostfs.info/TrueCloudLab/frostfs-sdk-go module git.frostfs.info/TrueCloudLab/frostfs-sdk-go
go 1.20 go 1.22
require ( require (
git.frostfs.info/TrueCloudLab/frostfs-api-go/v2 v2.16.1-0.20240306101814-c1c7b344b9c0 git.frostfs.info/TrueCloudLab/frostfs-api-go/v2 v2.16.1-0.20240902111049-c11f50efeccb
git.frostfs.info/TrueCloudLab/frostfs-contract v0.0.0-20230307110621-19a8ef2d02fb git.frostfs.info/TrueCloudLab/frostfs-contract v0.19.3-0.20240621131249-49e5270f673e
git.frostfs.info/TrueCloudLab/frostfs-crypto v0.6.0
git.frostfs.info/TrueCloudLab/hrw v1.2.1 git.frostfs.info/TrueCloudLab/hrw v1.2.1
git.frostfs.info/TrueCloudLab/tzhash v1.8.0 git.frostfs.info/TrueCloudLab/tzhash v1.8.0
github.com/antlr4-go/antlr/v4 v4.13.0 github.com/antlr4-go/antlr/v4 v4.13.0
github.com/google/uuid v1.3.0 github.com/google/uuid v1.6.0
github.com/hashicorp/golang-lru/v2 v2.0.2 github.com/hashicorp/golang-lru/v2 v2.0.7
github.com/klauspost/reedsolomon v1.12.1
github.com/mr-tron/base58 v1.2.0 github.com/mr-tron/base58 v1.2.0
github.com/nspcc-dev/neo-go v0.101.2-0.20230601131642-a0117042e8fc github.com/nspcc-dev/neo-go v0.106.2
github.com/stretchr/testify v1.8.3 github.com/stretchr/testify v1.9.0
go.uber.org/zap v1.24.0 go.uber.org/zap v1.27.0
google.golang.org/grpc v1.55.0 google.golang.org/grpc v1.63.2
google.golang.org/protobuf v1.33.0 google.golang.org/protobuf v1.33.0
gopkg.in/yaml.v3 v3.0.1 gopkg.in/yaml.v3 v3.0.1
) )
require ( require (
git.frostfs.info/TrueCloudLab/frostfs-crypto v0.6.0 // indirect
git.frostfs.info/TrueCloudLab/rfc6979 v0.4.0 // indirect git.frostfs.info/TrueCloudLab/rfc6979 v0.4.0 // indirect
github.com/benbjohnson/clock v1.1.0 // indirect github.com/VictoriaMetrics/easyproto v0.1.4 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect github.com/davecgh/go-spew v1.1.1 // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0 // indirect github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0 // indirect
github.com/golang/protobuf v1.5.3 // indirect github.com/golang/snappy v0.0.1 // indirect
github.com/gorilla/websocket v1.5.0 // indirect github.com/gorilla/websocket v1.5.1 // indirect
github.com/hashicorp/golang-lru v0.6.0 // indirect github.com/josharian/intern v1.0.0 // indirect
github.com/nspcc-dev/go-ordered-json v0.0.0-20220111165707-25110be27d22 // indirect github.com/klauspost/cpuid/v2 v2.2.6 // indirect
github.com/nspcc-dev/neo-go/pkg/interop v0.0.0-20230615193820-9185820289ce // indirect github.com/mailru/easyjson v0.7.7 // indirect
github.com/nspcc-dev/rfc6979 v0.2.0 // indirect github.com/nspcc-dev/go-ordered-json v0.0.0-20240301084351-0246b013f8b2 // indirect
github.com/nspcc-dev/neo-go/pkg/interop v0.0.0-20240521091047-78685785716d // indirect
github.com/nspcc-dev/rfc6979 v0.2.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/syndtr/goleveldb v1.0.1-0.20210305035536-64b5b1c73954 // indirect
github.com/twmb/murmur3 v1.1.8 // indirect github.com/twmb/murmur3 v1.1.8 // indirect
go.uber.org/atomic v1.10.0 // indirect go.etcd.io/bbolt v1.3.9 // indirect
go.uber.org/goleak v1.2.1 // indirect
go.uber.org/multierr v1.11.0 // indirect go.uber.org/multierr v1.11.0 // indirect
golang.org/x/crypto v0.9.0 // indirect golang.org/x/crypto v0.21.0 // indirect
golang.org/x/exp v0.0.0-20230515195305-f3d0a9c9a5cc // indirect golang.org/x/exp v0.0.0-20240222234643-814bf88cf225 // indirect
golang.org/x/net v0.10.0 // indirect golang.org/x/net v0.23.0 // indirect
golang.org/x/sync v0.2.0 // indirect golang.org/x/sync v0.6.0 // indirect
golang.org/x/sys v0.8.0 // indirect golang.org/x/sys v0.18.0 // indirect
golang.org/x/text v0.9.0 // indirect golang.org/x/text v0.14.0 // indirect
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20240227224415-6ceb2ff114de // indirect
) )

735
go.sum
View file

@ -1,40 +1,7 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= git.frostfs.info/TrueCloudLab/frostfs-api-go/v2 v2.16.1-0.20240902111049-c11f50efeccb h1:p9ByDsw+H6p6LyYSx8LKFtAG/oPKQpDVMNfjPqdevTw=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= git.frostfs.info/TrueCloudLab/frostfs-api-go/v2 v2.16.1-0.20240902111049-c11f50efeccb/go.mod h1:BDnEpkKMykCS8u1nLzR6SgNzCv6885RWlo5TnravQuI=
cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU= git.frostfs.info/TrueCloudLab/frostfs-contract v0.19.3-0.20240621131249-49e5270f673e h1:kcBqZBiFIUBATUqEuvVigtkJJWQ2Gug/eYXn967o3M4=
cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU= git.frostfs.info/TrueCloudLab/frostfs-contract v0.19.3-0.20240621131249-49e5270f673e/go.mod h1:F/fe1OoIDKr5Bz99q4sriuHDuf3aZefZy9ZsCqEtgxc=
cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=
cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=
cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=
cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To=
cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4=
cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M=
cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc=
cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk=
cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=
cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=
cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=
cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=
cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=
cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc=
cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=
cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=
cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=
cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=
cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU=
cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=
cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=
cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=
cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=
cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
git.frostfs.info/TrueCloudLab/frostfs-api-go/v2 v2.16.1-0.20240306101814-c1c7b344b9c0 h1:4iyAj9k7W29YpyzUTwMuMBbL3G3M96kMbX62OwGNGfE=
git.frostfs.info/TrueCloudLab/frostfs-api-go/v2 v2.16.1-0.20240306101814-c1c7b344b9c0/go.mod h1:OBDSr+DqV1z4VDouoX3YMleNc4DPBVBWTG3WDT2PK1o=
git.frostfs.info/TrueCloudLab/frostfs-contract v0.0.0-20230307110621-19a8ef2d02fb h1:S/TrbOOu9qEXZRZ9/Ddw7crnxbBUQLo68PSzQWYrc9M=
git.frostfs.info/TrueCloudLab/frostfs-contract v0.0.0-20230307110621-19a8ef2d02fb/go.mod h1:nkR5gaGeez3Zv2SE7aceP0YwxG2FzIB5cGKpQO2vV2o=
git.frostfs.info/TrueCloudLab/frostfs-crypto v0.6.0 h1:FxqFDhQYYgpe41qsIHVOcdzSVCB8JNSfPG7Uk4r2oSk= git.frostfs.info/TrueCloudLab/frostfs-crypto v0.6.0 h1:FxqFDhQYYgpe41qsIHVOcdzSVCB8JNSfPG7Uk4r2oSk=
git.frostfs.info/TrueCloudLab/frostfs-crypto v0.6.0/go.mod h1:RUIKZATQLJ+TaYQa60X2fTDwfuhMfm8Ar60bQ5fr+vU= git.frostfs.info/TrueCloudLab/frostfs-crypto v0.6.0/go.mod h1:RUIKZATQLJ+TaYQa60X2fTDwfuhMfm8Ar60bQ5fr+vU=
git.frostfs.info/TrueCloudLab/hrw v1.2.1 h1:ccBRK21rFvY5R1WotI6LNoPlizk7qSvdfD8lNIRudVc= git.frostfs.info/TrueCloudLab/hrw v1.2.1 h1:ccBRK21rFvY5R1WotI6LNoPlizk7qSvdfD8lNIRudVc=
@ -43,691 +10,183 @@ git.frostfs.info/TrueCloudLab/rfc6979 v0.4.0 h1:M2KR3iBj7WpY3hP10IevfIB9MURr4O9m
git.frostfs.info/TrueCloudLab/rfc6979 v0.4.0/go.mod h1:okpbKfVYf/BpejtfFTfhZqFP+sZ8rsHrP8Rr/jYPNRc= git.frostfs.info/TrueCloudLab/rfc6979 v0.4.0/go.mod h1:okpbKfVYf/BpejtfFTfhZqFP+sZ8rsHrP8Rr/jYPNRc=
git.frostfs.info/TrueCloudLab/tzhash v1.8.0 h1:UFMnUIk0Zh17m8rjGHJMqku2hCgaXDqjqZzS4gsb4UA= git.frostfs.info/TrueCloudLab/tzhash v1.8.0 h1:UFMnUIk0Zh17m8rjGHJMqku2hCgaXDqjqZzS4gsb4UA=
git.frostfs.info/TrueCloudLab/tzhash v1.8.0/go.mod h1:dhY+oy274hV8wGvGL4MwwMpdL3GYvaX1a8GQZQHvlF8= git.frostfs.info/TrueCloudLab/tzhash v1.8.0/go.mod h1:dhY+oy274hV8wGvGL4MwwMpdL3GYvaX1a8GQZQHvlF8=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= github.com/VictoriaMetrics/easyproto v0.1.4 h1:r8cNvo8o6sR4QShBXQd1bKw/VVLSQma/V2KhTBPf+Sc=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo= github.com/VictoriaMetrics/easyproto v0.1.4/go.mod h1:QlGlzaJnDfFd8Lk6Ci/fuLxfTo3/GThPs2KH23mv710=
github.com/CityOfZion/neo-go v0.62.1-pre.0.20191114145240-e740fbe708f8/go.mod h1:MJCkWUBhi9pn/CrYO1Q3P687y2KeahrOPS9BD9LDGb0=
github.com/CityOfZion/neo-go v0.70.1-pre.0.20191209120015-fccb0085941e/go.mod h1:0enZl0az8xA6PVkwzEOwPWVJGqlt/GO4hA4kmQ5Xzig=
github.com/CityOfZion/neo-go v0.70.1-pre.0.20191212173117-32ac01130d4c/go.mod h1:JtlHfeqLywZLswKIKFnAp+yzezY4Dji9qlfQKB2OD/I=
github.com/CityOfZion/neo-go v0.71.1-pre.0.20200129171427-f773ec69fb84/go.mod h1:FLI526IrRWHmcsO+mHsCbj64pJZhwQFTLJZu+A4PGOA=
github.com/Workiva/go-datastructures v1.0.50/go.mod h1:Z+F2Rca0qCsVYDS8z7bAGm8f3UkzuWYS/oBZz5a7VVA=
github.com/abiosoft/ishell v2.0.0+incompatible/go.mod h1:HQR9AqF2R3P4XXpMpI0NAzgHf/aS6+zVXRj14cVk9qg=
github.com/abiosoft/ishell/v2 v2.0.2/go.mod h1:E4oTCXfo6QjoCart0QYa5m9w4S+deXs/P/9jA77A9Bs=
github.com/abiosoft/readline v0.0.0-20180607040430-155bce2042db/go.mod h1:rB3B4rKii8V21ydCbIzH5hZiCQE7f5E9SzUb/ZZx530=
github.com/aead/siphash v1.0.1/go.mod h1:Nywa3cDsYNNK3gaciGTWPwHt0wlpNV15vwmswBAUSII=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho=
github.com/alicebob/gopher-json v0.0.0-20180125190556-5a6b3ba71ee6/go.mod h1:SGnFV6hVsYE877CKEZ6tDNTjaSXYUk6QqoIK6PrAtcc=
github.com/alicebob/miniredis v2.5.0+incompatible/go.mod h1:8HZjEj4yU0dwhYHky+DxYx+6BMjkBbe5ONFIF1MXffk=
github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
github.com/antlr/antlr4/runtime/Go/antlr v0.0.0-20210521073959-f0d4d129b7f1/go.mod h1:F7bn7fEU90QkQ3tnmaTx3LTKLEDqnwWODIYppRQ5hnY=
github.com/antlr4-go/antlr/v4 v4.13.0 h1:lxCg3LAv+EUK6t1i0y1V6/SLeUi0eKEKdhQAlS8TVTI= github.com/antlr4-go/antlr/v4 v4.13.0 h1:lxCg3LAv+EUK6t1i0y1V6/SLeUi0eKEKdhQAlS8TVTI=
github.com/antlr4-go/antlr/v4 v4.13.0/go.mod h1:pfChB/xh/Unjila75QW7+VU4TSnWnnk9UTnmpPaOR2g= github.com/antlr4-go/antlr/v4 v4.13.0/go.mod h1:pfChB/xh/Unjila75QW7+VU4TSnWnnk9UTnmpPaOR2g=
github.com/benbjohnson/clock v1.1.0 h1:Q92kusRqC1XV2MjkWETPvjJVqKetz1OzxZB7mHJLju8= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/btcsuite/btcd v0.20.1-beta/go.mod h1:wVuoA8VJLEcwgqHBwHmzLRazpKxTv13Px/pDuV7OomQ= github.com/bits-and-blooms/bitset v1.8.0 h1:FD+XqgOZDUxxZ8hzoBFuV9+cGWY9CslN6d5MS5JVb4c=
github.com/btcsuite/btcd v0.22.0-beta/go.mod h1:9n5ntfhhHQBIhUvlhDvD3Qg6fRUj4jkN0VB8L8svzOA= github.com/bits-and-blooms/bitset v1.8.0/go.mod h1:7hO7Gc7Pp1vODcmWvKMRA9BNmbv6a/7QIWpPxHddWR8=
github.com/btcsuite/btclog v0.0.0-20170628155309-84c8d2346e9f/go.mod h1:TdznJufoqS23FtqVCzL0ZqgP5MqXbb4fg/WgDys70nA= github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
github.com/btcsuite/btcutil v0.0.0-20190425235716-9e5f4b9a998d/go.mod h1:+5NJ2+qvTyV9exUAL/rxXi3DcLg2Ts+ymUAY5y4NvMg= github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/btcsuite/btcutil v1.0.3-0.20201208143702-a53e38424cce/go.mod h1:0DVlHczLPewLcPGEIeUEzfOJhqGPQ0mJJRDBtD307+o= github.com/consensys/bavard v0.1.13 h1:oLhMLOFGTLdlda/kma4VOJazblc7IM5y5QPd2A/YjhQ=
github.com/btcsuite/go-socks v0.0.0-20170105172521-4720035b7bfd/go.mod h1:HHNXQzUsZCxOoE+CPiyCTO6x34Zs86zZUiwtpXoGdtg= github.com/consensys/bavard v0.1.13/go.mod h1:9ItSMtA/dXMAiL7BG6bqW2m3NdSEObYWoH223nGHukI=
github.com/btcsuite/goleveldb v0.0.0-20160330041536-7834afc9e8cd/go.mod h1:F+uVaaLLH7j4eDXPRvw78tMflu7Ie2bzYOH4Y8rRKBY= github.com/consensys/gnark-crypto v0.12.2-0.20231013160410-1f65e75b6dfb h1:f0BMgIjhZy4lSRHCXFbQst85f5agZAjtDMixQqBWNpc=
github.com/btcsuite/goleveldb v1.0.0/go.mod h1:QiK9vBlgftBg6rWQIj6wFzbPfRjiykIEhBH4obrXJ/I= github.com/consensys/gnark-crypto v0.12.2-0.20231013160410-1f65e75b6dfb/go.mod h1:v2Gy7L/4ZRosZ7Ivs+9SfUDr0f5UlG+EM5t7MPHiLuY=
github.com/btcsuite/snappy-go v0.0.0-20151229074030-0bdef8d06723/go.mod h1:8woku9dyThutzjeg+3xrA5iCpBRH8XEEg3lh6TiUghc= github.com/cpuguy83/go-md2man/v2 v2.0.2 h1:p1EgwI/C7NhT0JmVkwCD2ZBK8j4aeHQX2pMHHBfMQ6w=
github.com/btcsuite/snappy-go v1.0.0/go.mod h1:8woku9dyThutzjeg+3xrA5iCpBRH8XEEg3lh6TiUghc= github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/btcsuite/websocket v0.0.0-20150119174127-31079b680792/go.mod h1:ghJtEyQwv5/p4Mg4C0fgbePVuGr935/5ddU9Z3TmDRY=
github.com/btcsuite/winsvc v1.0.0/go.mod h1:jsenWakMcC0zFBFurPLEAyrnc/teJEM1O46fmI40EZs=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/cespare/xxhash/v2 v2.1.0/go.mod h1:dgIUBU3pDso/gPgZ1osOZ0iQf77oPR28Tjxl5dIMyVM=
github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cespare/xxhash/v2 v2.1.2/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/cncf/xds/go v0.0.0-20210805033703-aa0b78936158/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v0.0.0-20171005155431-ecdeabc65495/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0 h1:8UrgZ3GkP4i/CLijOJx79Yu+etlyjdBU4sfcs2WYQMs= github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0 h1:8UrgZ3GkP4i/CLijOJx79Yu+etlyjdBU4sfcs2WYQMs=
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0/go.mod h1:v57UDF4pDQJcEfFUCRop3lJL149eHGSe9Jvczhzjo/0= github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0/go.mod h1:v57UDF4pDQJcEfFUCRop3lJL149eHGSe9Jvczhzjo/0=
github.com/decred/dcrd/lru v1.0.0/go.mod h1:mxKOwFd7lFjN2GZYsiz/ecgqR6kkYAl+0pz0tEMk218=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=
github.com/envoyproxy/go-control-plane v0.9.10-0.20210907150352-cf90f659a021/go.mod h1:AFq3mo9L8Lqqiid3OhADV3RfLJnjiw63cSpi+fDTRC0=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/etcd-io/bbolt v1.3.3/go.mod h1:ZF2nL25h33cCyBtcyWeZ2/I3HQOfTP+0PIEvHjkjCrw=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/fatih/color v1.12.0/go.mod h1:ELkj/draVOlAH/xkhN6mQ50Qd0MPOk5AAr3maGEBuJM=
github.com/flynn-archive/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:rZfgFAXFS/z/lEd6LJmf9HVZ1LkgYiHx5pHhV5DR16M=
github.com/frankban/quicktest v1.14.0/go.mod h1:NeW+ay9A/U67EYXNFA1nPE8e/tnQv/09mUdL/ijj8og=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4=
github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ= github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY=
github.com/go-kit/log v0.2.0/go.mod h1:NwTd00d/i8cPZ3xOwwiv2PO5MOcx78fFErGNcVmBjv0=
github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A=
github.com/go-logfmt/logfmt v0.5.1/go.mod h1:WYhtIu8zTZfxdn5+rREduYbwxfcBr/Vr6KEVveWlfTs=
github.com/go-redis/redis v6.10.2+incompatible/go.mod h1:NAIEuMOZ/fxfXJIrKDQDz8wamY7mA7PouImQ2Jvg6kA=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/go-yaml/yaml v2.1.0+incompatible/go.mod h1:w2MrLa16VYP0jy6N7M5kHaCkaLENm+P+Tv+MfurjSw0=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=
github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=
github.com/golang/mock v1.5.0/go.mod h1:CWnOUgYIOo4TcNZ0wHX3YZCqsaM1I1Jvs6v3mP3KVu8=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg=
github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/snappy v0.0.0-20170215233205-553a64147049/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/golang/snappy v0.0.1 h1:Qgr9rKW7uDUkrbSmQeiDsGa8SjGyCOGtuasMWwvp2P4= github.com/golang/snappy v0.0.1 h1:Qgr9rKW7uDUkrbSmQeiDsGa8SjGyCOGtuasMWwvp2P4=
github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/gomodule/redigo v2.0.0+incompatible/go.mod h1:B4C85qUVwatsJoIUNIfCRsp7qO0iAmpGFZ4EELWSbC4=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/gorilla/websocket v1.5.1 h1:gmztn0JnHVt9JZquRuzLw3g4wouNVzKL15iLr/zn/QY=
github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/gorilla/websocket v1.5.1/go.mod h1:x3kM2JMyaluk02fnUJpQuwD2dCS5NDG2ZHL0uE0tcaY=
github.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/holiman/uint256 v1.2.4 h1:jUc4Nk8fm9jZabQuqr2JzednajVmBpC+oiTiXZJEApU=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= github.com/holiman/uint256 v1.2.4/go.mod h1:EOMSn4q6Nyt9P6efbI3bueV4e1b3dGlUCXeiRV4ng7E=
github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.2.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/gorilla/websocket v1.5.0 h1:PPwGk2jz7EePpoHN/+ClbZu8SPxiqlu12wZP/3sWmnc=
github.com/gorilla/websocket v1.5.0/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
github.com/hashicorp/golang-lru v0.6.0 h1:uL2shRDx7RTrOrTCUZEGP/wJUFiUI8QT6E7z5o8jga4=
github.com/hashicorp/golang-lru v0.6.0/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
github.com/hashicorp/golang-lru/v2 v2.0.2 h1:Dwmkdr5Nc/oBiXgJS3CDHNhJtIHkuZ3DZF5twqnfBdU=
github.com/hashicorp/golang-lru/v2 v2.0.2/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/holiman/uint256 v1.2.0/go.mod h1:y4ga/t+u+Xwd7CpDgZESaRcWy0I7XMlTMA25ApIH5Jw=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
github.com/jessevdk/go-flags v0.0.0-20141203071132-1679536dcc89/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI= github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI= github.com/klauspost/cpuid/v2 v2.2.6 h1:ndNyv040zDGIDh8thGkXYjnFtiN02M1PVVF+JE/48xc=
github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4= github.com/klauspost/cpuid/v2 v2.2.6/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws=
github.com/jrick/logrotate v1.0.0/go.mod h1:LNinyqDIJnpAur+b8yyulnQw/wDuN1+BYKlTRt3OuAQ= github.com/klauspost/reedsolomon v1.12.1 h1:NhWgum1efX1x58daOBGCFWcxtEhOhXKKl1HAPQUp03Q=
github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= github.com/klauspost/reedsolomon v1.12.1/go.mod h1:nEi5Kjb6QqtbofI6s+cbG/j1da11c96IBYBSnVGtuBs=
github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/kkdai/bstream v0.0.0-20161212061736-f391b8402d23/go.mod h1:J+Gs4SYgM6CZQHDETBtE9HaSEkGmuNXF86RwHhHUvq4=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
github.com/mattn/go-colorable v0.1.8/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc= github.com/mmcloughlin/addchain v0.4.0 h1:SobOdjm2xLj1KkXN5/n0xTIWyZA2+s99UCY1iPfkHRY=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s= github.com/mmcloughlin/addchain v0.4.0/go.mod h1:A86O+tHqZLMNO4w6ZZ4FlVQEadcoqkyU72HC5wJ4RlU=
github.com/mattn/go-isatty v0.0.9/go.mod h1:YNRxwqDuOph6SZLI9vUUz6OYw3QyUt7WiY2yME+cCiQ=
github.com/mattn/go-isatty v0.0.10/go.mod h1:qgIWMr58cqv1PHHyhnkY9lrL7etaEgOFcMEpPG5Rm84=
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/mr-tron/base58 v1.1.2/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=
github.com/mr-tron/base58 v1.2.0 h1:T/HDJBh4ZCPbU39/+c3rRvE0uKBQlU27+QI8LJ4t64o= github.com/mr-tron/base58 v1.2.0 h1:T/HDJBh4ZCPbU39/+c3rRvE0uKBQlU27+QI8LJ4t64o=
github.com/mr-tron/base58 v1.2.0/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc= github.com/mr-tron/base58 v1.2.0/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/nspcc-dev/go-ordered-json v0.0.0-20240301084351-0246b013f8b2 h1:mD9hU3v+zJcnHAVmHnZKt3I++tvn30gBj2rP2PocZMk=
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/nspcc-dev/go-ordered-json v0.0.0-20240301084351-0246b013f8b2/go.mod h1:U5VfmPNM88P4RORFb6KSUVBdJBDhlqggJZYGXGPxOcc=
github.com/nspcc-dev/dbft v0.0.0-20191205084618-dacb1a30c254/go.mod h1:w1Ln2aT+dBlPhLnuZhBV+DfPEdS2CHWWLp5JTScY3bw= github.com/nspcc-dev/neo-go v0.106.2 h1:KXSJ2J5Oacc7LrX3r4jvnC8ihKqHs5NB21q4f2S3r9o=
github.com/nspcc-dev/dbft v0.0.0-20191209120240-0d6b7568d9ae/go.mod h1:3FjXOoHmA51EGfb5GS/HOv7VdmngNRTssSeQ729dvGY= github.com/nspcc-dev/neo-go v0.106.2/go.mod h1:Ojwfx3/lv0VTeEHMpQ17g0wTnXcCSoFQVq5GEeCZmGo=
github.com/nspcc-dev/dbft v0.0.0-20200117124306-478e5cfbf03a/go.mod h1:/YFK+XOxxg0Bfm6P92lY5eDSLYfp06XOdL8KAVgXjVk= github.com/nspcc-dev/neo-go/pkg/interop v0.0.0-20240521091047-78685785716d h1:Vcb7YkZuUSSIC+WF/xV3UDfHbAxZgyT2zGleJP3Ig5k=
github.com/nspcc-dev/dbft v0.0.0-20200219114139-199d286ed6c1/go.mod h1:O0qtn62prQSqizzoagHmuuKoz8QMkU3SzBoKdEvm3aQ= github.com/nspcc-dev/neo-go/pkg/interop v0.0.0-20240521091047-78685785716d/go.mod h1:/vrbWSHc7YS1KSYhVOyyeucXW/e+1DkVBOgnBEXUCeY=
github.com/nspcc-dev/dbft v0.0.0-20210721160347-1b03241391ac/go.mod h1:U8MSnEShH+o5hexfWJdze6uMFJteP0ko7J2frO7Yu1Y= github.com/nspcc-dev/rfc6979 v0.2.1 h1:8wWxkamHWFmO790GsewSoKUSJjVnL1fmdRpokU/RgRM=
github.com/nspcc-dev/dbft v0.0.0-20220902113116-58a5e763e647/go.mod h1:g9xisXmX9NP9MjioaTe862n9SlZTrP+6PVUWLBYOr98= github.com/nspcc-dev/rfc6979 v0.2.1/go.mod h1:Tk7h5kyUWkhjyO3zUgFFhy1v2vQv3BvQEntakdtqrWc=
github.com/nspcc-dev/go-ordered-json v0.0.0-20210915112629-e1b6cce73d02/go.mod h1:79bEUDEviBHJMFV6Iq6in57FEOCMcRhfQnfaf0ETA5U= github.com/nxadm/tail v1.4.4 h1:DQuhQpB1tVlglWS2hLQ5OV6B5r8aGxSrPc5Qo6uTN78=
github.com/nspcc-dev/go-ordered-json v0.0.0-20220111165707-25110be27d22 h1:n4ZaFCKt1pQJd7PXoMJabZWK9ejjbLOVrkl/lOUmshg=
github.com/nspcc-dev/go-ordered-json v0.0.0-20220111165707-25110be27d22/go.mod h1:79bEUDEviBHJMFV6Iq6in57FEOCMcRhfQnfaf0ETA5U=
github.com/nspcc-dev/hrw v1.0.9/go.mod h1:l/W2vx83vMQo6aStyx2AuZrJ+07lGv2JQGlVkPG06MU=
github.com/nspcc-dev/neo-go v0.73.1-pre.0.20200303142215-f5a1b928ce09/go.mod h1:pPYwPZ2ks+uMnlRLUyXOpLieaDQSEaf4NM3zHVbRjmg=
github.com/nspcc-dev/neo-go v0.98.0/go.mod h1:E3cc1x6RXSXrJb2nDWXTXjnXk3rIqVN8YdFyWv+FrqM=
github.com/nspcc-dev/neo-go v0.99.4/go.mod h1:mKTolfRUfKjFso5HPvGSQtUZc70n0VKBMs16eGuC5gA=
github.com/nspcc-dev/neo-go v0.101.2-0.20230601131642-a0117042e8fc h1:fySIWvUQsitK5e5qYIHnTDCXuPpwzz89SEUEIyY11sg=
github.com/nspcc-dev/neo-go v0.101.2-0.20230601131642-a0117042e8fc/go.mod h1:s9QhjMC784MWqTURovMbyYduIJc86mnCruxcMiAebpc=
github.com/nspcc-dev/neo-go/pkg/interop v0.0.0-20220927123257-24c107e3a262/go.mod h1:23bBw0v6pBYcrWs8CBEEDIEDJNbcFoIh8pGGcf2Vv8s=
github.com/nspcc-dev/neo-go/pkg/interop v0.0.0-20230615193820-9185820289ce h1:vLGuUNDkmQrWMa4rr4vTd1u8ULqejWxVmNz1L7ocTEI=
github.com/nspcc-dev/neo-go/pkg/interop v0.0.0-20230615193820-9185820289ce/go.mod h1:ZUuXOkdtHZgaC13za/zMgXfQFncZ0jLzfQTe+OsDOtg=
github.com/nspcc-dev/neofs-api-go/v2 v2.11.0-pre.0.20211201134523-3604d96f3fe1/go.mod h1:oS8dycEh8PPf2Jjp6+8dlwWyEv2Dy77h/XhhcdxYEFs=
github.com/nspcc-dev/neofs-api-go/v2 v2.11.1/go.mod h1:oS8dycEh8PPf2Jjp6+8dlwWyEv2Dy77h/XhhcdxYEFs=
github.com/nspcc-dev/neofs-crypto v0.2.0/go.mod h1:F/96fUzPM3wR+UGsPi3faVNmFlA9KAEAUQR7dMxZmNA=
github.com/nspcc-dev/neofs-crypto v0.2.3/go.mod h1:8w16GEJbH6791ktVqHN9YRNH3s9BEEKYxGhlFnp0cDw=
github.com/nspcc-dev/neofs-crypto v0.3.0/go.mod h1:8w16GEJbH6791ktVqHN9YRNH3s9BEEKYxGhlFnp0cDw=
github.com/nspcc-dev/neofs-crypto v0.4.0/go.mod h1:6XJ8kbXgOfevbI2WMruOtI+qUJXNwSGM/E9eClXxPHs=
github.com/nspcc-dev/neofs-sdk-go v0.0.0-20211201182451-a5b61c4f6477/go.mod h1:dfMtQWmBHYpl9Dez23TGtIUKiFvCIxUZq/CkSIhEpz4=
github.com/nspcc-dev/neofs-sdk-go v0.0.0-20220113123743-7f3162110659/go.mod h1:/jay1lr3w7NQd/VDBkEhkJmDmyPNsu4W+QV2obsUV40=
github.com/nspcc-dev/rfc6979 v0.1.0/go.mod h1:exhIh1PdpDC5vQmyEsGvc4YDM/lyQp/452QxGq/UEso=
github.com/nspcc-dev/rfc6979 v0.2.0 h1:3e1WNxrN60/6N0DW7+UYisLeZJyfqZTNOjeV/toYvOE=
github.com/nspcc-dev/rfc6979 v0.2.0/go.mod h1:exhIh1PdpDC5vQmyEsGvc4YDM/lyQp/452QxGq/UEso=
github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A= github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.10.3/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk= github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk=
github.com/onsi/ginkgo v1.14.0 h1:2mOpI4JVVPBN+WQRa0WKH2eXR+Ey+uK4n7Zj0aYpIQA=
github.com/onsi/ginkgo v1.14.0/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9klQyY= github.com/onsi/ginkgo v1.14.0/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9klQyY=
github.com/onsi/gomega v1.4.1/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA=
github.com/onsi/gomega v1.4.2/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY= github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY=
github.com/onsi/gomega v1.10.1 h1:o0+MgICZLuZ7xjH7Vx6zS/zcu93/BEp1VwkIW1mEXCE=
github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=
github.com/pierrec/lz4 v2.6.1+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= github.com/prometheus/client_golang v1.19.0 h1:ygXvpU1AoN1MhdzckN+PyD9QJOSD4x7kmXYlnfbA6JU=
github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= github.com/prometheus/client_golang v1.19.0/go.mod h1:ZRM9uEAypZakd+q/x7+gmsvXdURP+DABIEIjnmDdp+k=
github.com/prometheus/client_golang v1.2.1/go.mod h1:XMU6Z2MjaRKVu/dC1qupJI9SiNkDYzz3xecMgSW/F+U= github.com/prometheus/client_model v0.5.0 h1:VQw1hfvPvk3Uv6Qf29VrPF32JB6rtbgI6cYPYQjL0Qw=
github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M= github.com/prometheus/client_model v0.5.0/go.mod h1:dTiFglRmd66nLR9Pv9f0mZi7B7fk5Pm3gvsjB5tr+kI=
github.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0= github.com/prometheus/common v0.48.0 h1:QO8U2CdOzSn1BBsmXJXduaaW+dY/5QLjfB8svtSzKKE=
github.com/prometheus/client_golang v1.12.1/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY= github.com/prometheus/common v0.48.0/go.mod h1:0/KsvlIEfPQCQ5I2iNSAWKPZziNCvRs5EC6ILDTlAPc=
github.com/prometheus/client_golang v1.13.0/go.mod h1:vTeo+zgvILHsnnj/39Ou/1fPN5nJFOEMgftOUOmlvYQ= github.com/prometheus/procfs v0.12.0 h1:jluTpSng7V9hY0O2R9DzzJHYb2xULk9VTR1V1R/k6Bo=
github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= github.com/prometheus/procfs v0.12.0/go.mod h1:pcuDEFsWDnvcgNzo4EEweacyhjeA9Zk3cnaOZAZEfOo=
github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
github.com/prometheus/common v0.7.0/go.mod h1:DjGbpBbp5NYNiECxcL/VnbXCCaQpKd3tt26CguLLsqA=
github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo=
github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc=
github.com/prometheus/common v0.32.1/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls=
github.com/prometheus/common v0.37.0/go.mod h1:phzohg0JFMnBEFGxTDbfu3QyL5GI8gTQJFhYO5B3mfA=
github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
github.com/prometheus/procfs v0.0.5/go.mod h1:4A/X28fw3Fc593LaREMrKMqOKvUAntwMDaekg4FpcdQ=
github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU=
github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=
github.com/prometheus/procfs v0.7.3/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=
github.com/prometheus/procfs v0.8.0/go.mod h1:z7EfXMXOkbkqb9IINtpCn86r/to3BnA0uaxHdg830/4=
github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/go-internal v1.6.1 h1:/FiVV8dS/e+YqF2JvO3yXRFbBLTIuSDkuC7aBOAvL+k=
github.com/rogpeppe/go-internal v1.6.1/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTEfhy4qGm1nDQc=
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88=
github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.3 h1:RP3t2pwF7cMEbC1dqtB6poj3niw/9gnV4Cjg5oW5gtY= github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.8.3/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/syndtr/goleveldb v0.0.0-20180307113352-169b1b37be73/go.mod h1:Z4AUp2Km+PwemOoO/VB5AOx9XSsIItzFjoJlOSiYmn0=
github.com/syndtr/goleveldb v1.0.1-0.20210305035536-64b5b1c73954 h1:xQdMZ1WLrgkkvOZ/LDQxjVxMLdby7osSh4ZEVa5sIjs= github.com/syndtr/goleveldb v1.0.1-0.20210305035536-64b5b1c73954 h1:xQdMZ1WLrgkkvOZ/LDQxjVxMLdby7osSh4ZEVa5sIjs=
github.com/syndtr/goleveldb v1.0.1-0.20210305035536-64b5b1c73954/go.mod h1:u2MKkTVTVJWe5D1rCvame8WqhBd88EuIwODJZ1VHCPM= github.com/syndtr/goleveldb v1.0.1-0.20210305035536-64b5b1c73954/go.mod h1:u2MKkTVTVJWe5D1rCvame8WqhBd88EuIwODJZ1VHCPM=
github.com/twmb/murmur3 v1.1.5/go.mod h1:Qq/R7NUyOfr65zD+6Q5IHKsJLwP7exErjN6lyyq3OSQ=
github.com/twmb/murmur3 v1.1.8 h1:8Yt9taO/WN3l08xErzjeschgZU2QSrwm1kclYq+0aRg= github.com/twmb/murmur3 v1.1.8 h1:8Yt9taO/WN3l08xErzjeschgZU2QSrwm1kclYq+0aRg=
github.com/twmb/murmur3 v1.1.8/go.mod h1:Qq/R7NUyOfr65zD+6Q5IHKsJLwP7exErjN6lyyq3OSQ= github.com/twmb/murmur3 v1.1.8/go.mod h1:Qq/R7NUyOfr65zD+6Q5IHKsJLwP7exErjN6lyyq3OSQ=
github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA= github.com/urfave/cli v1.22.5 h1:lNq9sAHXK2qfdI8W+GRItjCEkI+2oR4d+MEHy1CKXoU=
github.com/urfave/cli v1.22.5/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0= github.com/urfave/cli v1.22.5/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
github.com/virtuald/go-ordered-json v0.0.0-20170621173500-b18e6e673d74 h1:JwtAtbp7r/7QSyGz8mKUbYJBg2+6Cd7OjM8o/GNOcVo= go.etcd.io/bbolt v1.3.9 h1:8x7aARPEXiXbHmtUwAIv7eV2fQFHrLLavdiJ3uzJXoI=
github.com/virtuald/go-ordered-json v0.0.0-20170621173500-b18e6e673d74/go.mod h1:RmMWU37GKR2s6pgrIEB4ixgpVCt/cf7dnJv3fuH1J1c= go.etcd.io/bbolt v1.3.9/go.mod h1:zaO32+Ti0PK1ivdPtgMESzuzL2VPoIG1PCQNvOdo/dE=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.4.1/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
github.com/yuin/gopher-lua v0.0.0-20190514113301-1cd887cd7036/go.mod h1:gqRgreBUhTSL0GeU64rtZ3Uq3wtjOa/TB2YfrtkCbVQ=
github.com/yuin/gopher-lua v0.0.0-20191128022950-c6266f4fe8d7/go.mod h1:gqRgreBUhTSL0GeU64rtZ3Uq3wtjOa/TB2YfrtkCbVQ=
go.etcd.io/bbolt v1.3.3/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
go.etcd.io/bbolt v1.3.6 h1:/ecaJf0sk1l4l6V4awd65v2C3ILy7MSj+s/x1ADCIMU=
go.etcd.io/bbolt v1.3.6/go.mod h1:qXsaaIqmgQH0T+OPdb99Bf+PKfBBQVAdyD6TY9G8XM4=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI=
go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
go.uber.org/atomic v1.10.0 h1:9qC72Qh0+3MqyJbAn8YU5xVq1frD8bn3JtD2oXtafVQ=
go.uber.org/atomic v1.10.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
go.uber.org/goleak v1.1.10/go.mod h1:8a7PlsEVH3e/a/GLqe5IIrQx6GzcnRmZEufDUTk4A7A=
go.uber.org/goleak v1.2.1 h1:NBol2c7O1ZokfZ0LEU9K6Whx/KnwvepVetCUhtKja4A=
go.uber.org/goleak v1.2.1/go.mod h1:qlT2yGI9QafXHhZZLxlSuNsMw3FFLxBr+tBRlmO1xH4=
go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0=
go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q= go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
go.uber.org/zap v1.18.1/go.mod h1:xg/QME4nWcxGxrpdeYfq7UvYrLh66cuVKdrbD1XF/NI= go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
go.uber.org/zap v1.24.0 h1:FiJd5l1UOLj0wCgbSE0rwwXHzEdAZS6hiiSnxJN/D60=
go.uber.org/zap v1.24.0/go.mod h1:2kMP+WWQ8aoFoedH3T2sq6iJ2yDWpHbP0f6MQbS9Gkg=
golang.org/x/crypto v0.0.0-20170930174604-9419663f5a44/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200115085410-6d4e4cb37c7d/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200510223506-06a226fb4e37/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210711020723-a769d52b0f97/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= golang.org/x/crypto v0.21.0 h1:X31++rzVUdKhX5sWmSOFZxx8UW/ldWx55cbf08iNAMA=
golang.org/x/crypto v0.9.0 h1:LF6fAI+IutBocDJ2OT0Q1g8plpYljMZ4+lty+dsqw3g= golang.org/x/crypto v0.21.0/go.mod h1:0BP7YvVV9gBbVKyeTG0Gyn+gZm94bibOW5BjDEYAOMs=
golang.org/x/crypto v0.9.0/go.mod h1:yrmDGqONDYtNj3tH8X9dzUun2m2lzPa9ngI6/RUPGR0= golang.org/x/exp v0.0.0-20240222234643-814bf88cf225 h1:LfspQV/FYTatPTr/3HzIcmiUFH7PGP+OQ6mgDYo3yuQ=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20240222234643-814bf88cf225/go.mod h1:CxmFvTBINI24O/j8iY7H1xHzx2i4OsyguNBmN/uPtqc=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/mod v0.16.0 h1:QX4fJ0Rr5cPQCF7O9lh9Se4pmwfwskqZfq5moyldzic=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= golang.org/x/mod v0.16.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=
golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=
golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
golang.org/x/exp v0.0.0-20230515195305-f3d0a9c9a5cc h1:mCRnTeVUjcrhlRmO0VK8a6k6Rrf6TF9htwo2pJVSjIU=
golang.org/x/exp v0.0.0-20230515195305-f3d0a9c9a5cc/go.mod h1:V1LtkGg67GoY2N1AnLN78QLrzxkLyJw7RJb1gzOOz9w=
golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=
golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.5.1/go.mod h1:5OXOZSfqPIIbmVBIIKWRFfZjPR0E5r58TLhUjH0a2Ro=
golang.org/x/net v0.0.0-20180719180050-a680a1efc54d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200813134508-3edf25e44fcc/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20200813134508-3edf25e44fcc/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.23.0 h1:7EYJ93RZ9vYSZAIb2x3lnuvqO5zneoD6IvWjuhfxjTs=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.23.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/net v0.10.0 h1:X2//UzNDwYmtCLn7To6G58Wr6f5ahEAQgKNzv9Y951M=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b/go.mod h1:DAh4E804XQdzx2j+YRIaUnCqCV2RuMz24cGBJ5QYIrc=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.6.0 h1:5BMeUDZ7vkXGfEr1x9B4bRcTH4lpkTkpdh0T/J+qjbQ=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220601150217-0de741cfad7f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.2.0 h1:PUR+T4wwASmuSTYdKjYHI5TD22Wy5ogLU5qZCOLxBrI=
golang.org/x/sync v0.2.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190204203706-41f3e6584952/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191008105621-543471e840be/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191010194322-b09406accb47/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200519105757-fe76b779f299/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200519105757-fe76b779f299/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200814200057-3d37ad5750ed/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200814200057-3d37ad5750ed/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200923182605-d9f96fdee20d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211019181941-9d821ace8654/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0 h1:EBmGv8NaZBZTWvrbjNoL6HVt+IVy3QDQpJs7VRIw3tU= golang.org/x/sys v0.18.0 h1:DBdB3niSjOA/O0blCZBqDefyWNYveAYMNF1Wum0DYQ4=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.18.0 h1:FcHjZXDMxI8mM3nwhX9HlKop4C0YQvCVCdwYl2wOtE8=
golang.org/x/term v0.0.0-20210429154555-c04ba851c2a4/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.18.0/go.mod h1:ILwASektA3OnRv7amZ1xhE/KTR+u50pbXfZ03+6Nx58=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.9.0 h1:2sjJmO8cDvYveuX97RDLsxlyUxLl+GHoLxBiRdHllBE=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180318012157-96caea41033d/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.19.0 h1:tfGCXNR1OsFG+sVdLAitlpjAvD/I6dHDKnYrpEZUHkw=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= golang.org/x/tools v0.19.0/go.mod h1:qoJWxmGSIBmAeriMx19ogtrEPrGtDbPK634QFIcLAhc=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191108193012-7d206e10da11/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=
golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20201022035929-9cf592e881e9/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.8/go.mod h1:nABZi5QlRsZVlzPpHl034qft6wpY4eDcsTt5AaioBiU=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE= google.golang.org/genproto/googleapis/rpc v0.0.0-20240227224415-6ceb2ff114de h1:cZGRis4/ot9uVm639a+rHCUaG0JJHEsdyzSQTMX+suY=
google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M= google.golang.org/genproto/googleapis/rpc v0.0.0-20240227224415-6ceb2ff114de/go.mod h1:H4O17MA/PE9BsGx3w+a+W2VOLLD1Qf7oJneAoU6WktY=
google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= google.golang.org/grpc v1.63.2 h1:MUeiw1B2maTVZthpU5xvASfTh3LDbxHd6IJ6QQVU+xM=
google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= google.golang.org/grpc v1.63.2/go.mod h1:WAX/8DgncnokcFUldAxq7GeB5DXHDbMF+lLvDomNkRA=
google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM=
google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=
google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=
google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA=
google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200513103714-09dca8ec2884/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=
google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 h1:KpwkzHKEF7B9Zxg18WzOa7djJ+Ha5DzthMyZYQfEn2A=
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1/go.mod h1:nKE/iIaLqn2bQwXBg8f1g2Ylh6r5MN5CmZvuzZCgsCU=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=
google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=
google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.33.1/go.mod h1:fr5YgcSWrqhRRxogOsw7RzIpsmvOZ6IcH4kBYTpR3n0=
google.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
google.golang.org/grpc v1.41.0/go.mod h1:U3l9uK9J0sini8mHphKoXyaqDA/8VyGnDee1zzIUK6k=
google.golang.org/grpc v1.55.0 h1:3Oj82/tFSCeUrRTg/5E/7d/W5A1tj6Ky1ABAuZuv5ag=
google.golang.org/grpc v1.55.0/go.mod h1:iYEXKGkEBhg1PjZQvoYEVPTDkHo1/bjTnfwTeGONTY8=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE= google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI= google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=
google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos= google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
gopkg.in/abiosoft/ishell.v2 v2.0.0/go.mod h1:sFp+cGtH6o4s1FtpVPTMcHq2yue+c4DGOVohJCPUzwY=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo= gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw= gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.3.0 h1:clyUAQHOM3G0M3f5vQj7LuJrETvjVot3Z5el9nffUtU=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= rsc.io/tmplfunc v0.0.3 h1:53XFQh69AfOa8Tw0Jm7t+GV7KZhOi6jzsCzTtKbMvzU=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= rsc.io/tmplfunc v0.0.3/go.mod h1:AG3sTPzElb1Io3Yg4voV9AGZJuleGAwaVRxL9M49PhA=
honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=

View file

@ -1,8 +1,6 @@
package netmap package netmap
import ( import "slices"
"sort"
)
type ( type (
// aggregator can calculate some value across all netmap // aggregator can calculate some value across all netmap
@ -28,7 +26,6 @@ type (
} }
meanIQRAgg struct { meanIQRAgg struct {
k float64
arr []float64 arr []float64
} }
@ -74,16 +71,10 @@ func newMinAgg() aggregator {
return new(minAgg) return new(minAgg)
} }
// newMeanIQRAgg returns an aggregator which
// computes mean value of values from IQR interval.
func newMeanIQRAgg() aggregator {
return new(meanIQRAgg)
}
// newReverseMinNorm returns a normalizer which // newReverseMinNorm returns a normalizer which
// normalize values in range of 0.0 to 1.0 to a minimum value. // normalize values in range of 0.0 to 1.0 to a minimum value.
func newReverseMinNorm(min float64) normalizer { func newReverseMinNorm(minV float64) normalizer {
return &reverseMinNorm{min: min} return &reverseMinNorm{min: minV}
} }
// newSigmoidNorm returns a normalizer which // newSigmoidNorm returns a normalizer which
@ -118,6 +109,10 @@ func (a *minAgg) Compute() float64 {
return a.min return a.min
} }
func (a *meanIQRAgg) clear() {
a.arr = a.arr[:0]
}
func (a *meanIQRAgg) Add(n float64) { func (a *meanIQRAgg) Add(n float64) {
a.arr = append(a.arr, n) a.arr = append(a.arr, n)
} }
@ -128,25 +123,24 @@ func (a *meanIQRAgg) Compute() float64 {
return 0 return 0
} }
sort.Slice(a.arr, func(i, j int) bool { return a.arr[i] < a.arr[j] }) slices.Sort(a.arr)
var min, max float64 var minV, maxV float64
const minLn = 4 const minLn = 4
if l < minLn { if l < minLn {
min, max = a.arr[0], a.arr[l-1] minV, maxV = a.arr[0], a.arr[l-1]
} else { } else {
start, end := l/minLn, l*3/minLn-1 start, end := l/minLn, l*3/minLn-1
iqr := a.k * (a.arr[end] - a.arr[start]) minV, maxV = a.arr[start], a.arr[end]
min, max = a.arr[start]-iqr, a.arr[end]+iqr
} }
count := 0 count := 0
sum := float64(0) sum := float64(0)
for _, e := range a.arr { for _, e := range a.arr {
if e >= min && e <= max { if e >= minV && e <= maxV {
sum += e sum += e
count++ count++
} }

View file

@ -47,7 +47,7 @@ func BenchmarkNetmap_ContainerNodes(b *testing.B) {
b.ResetTimer() b.ResetTimer()
b.ReportAllocs() b.ReportAllocs()
for i := 0; i < b.N; i++ { for range b.N {
_, err := nm.ContainerNodes(p, pivot) _, err := nm.ContainerNodes(p, pivot)
if err != nil { if err != nil {
b.Fatal(err) b.Fatal(err)

View file

@ -94,14 +94,14 @@ func (c *context) addUsedNodes(ns ...NodeInfo) {
func defaultWeightFunc(ns nodes) weightFunc { func defaultWeightFunc(ns nodes) weightFunc {
mean := newMeanAgg() mean := newMeanAgg()
min := newMinAgg() minV := newMinAgg()
for i := range ns { for i := range ns {
mean.Add(float64(ns[i].capacity())) mean.Add(float64(ns[i].capacity()))
min.Add(float64(ns[i].Price())) minV.Add(float64(ns[i].Price()))
} }
return newWeightFunc( return newWeightFunc(
newSigmoidNorm(mean.Compute()), newSigmoidNorm(mean.Compute()),
newReverseMinNorm(min.Compute())) newReverseMinNorm(minV.Compute()))
} }

View file

@ -3,6 +3,7 @@ package netmap
import ( import (
"fmt" "fmt"
"strconv" "strconv"
"strings"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
) )
@ -11,6 +12,8 @@ import (
// which points to the whole netmap. // which points to the whole netmap.
const mainFilterName = "*" const mainFilterName = "*"
const likeWildcard = "*"
// processFilters processes filters and returns error is any of them is invalid. // processFilters processes filters and returns error is any of them is invalid.
func (c *context) processFilters(p PlacementPolicy) error { func (c *context) processFilters(p PlacementPolicy) error {
for i := range p.filters { for i := range p.filters {
@ -53,7 +56,7 @@ func (c *context) processFilter(f netmap.Filter, top bool) error {
} }
switch op { switch op {
case netmap.EQ, netmap.NE: case netmap.EQ, netmap.NE, netmap.LIKE:
case netmap.GT, netmap.GE, netmap.LT, netmap.LE: case netmap.GT, netmap.GE, netmap.LT, netmap.LE:
val := f.GetValue() val := f.GetValue()
n, err := strconv.ParseUint(val, 10, 64) n, err := strconv.ParseUint(val, 10, 64)
@ -110,6 +113,19 @@ func (c *context) matchKeyValue(f *netmap.Filter, b NodeInfo) bool {
switch op := f.GetOp(); op { switch op := f.GetOp(); op {
case netmap.EQ: case netmap.EQ:
return b.Attribute(f.GetKey()) == f.GetValue() return b.Attribute(f.GetKey()) == f.GetValue()
case netmap.LIKE:
str, prefix := strings.CutPrefix(f.GetValue(), likeWildcard)
str, suffix := strings.CutSuffix(str, likeWildcard)
if prefix && suffix {
return strings.Contains(b.Attribute(f.GetKey()), str)
}
if prefix && !suffix {
return strings.HasSuffix(b.Attribute(f.GetKey()), str)
}
if !prefix && suffix {
return strings.HasPrefix(b.Attribute(f.GetKey()), str)
}
return b.Attribute(f.GetKey()) == f.GetValue()
case netmap.NE: case netmap.NE:
return b.Attribute(f.GetKey()) != f.GetValue() return b.Attribute(f.GetKey()) != f.GetValue()
default: default:

View file

@ -113,9 +113,10 @@ func (n nodes) Hash() uint64 {
return 0 return 0
} }
// weights returns slice of nodes weights W. func (n nodes) appendWeightsTo(wf weightFunc, w []float64) []float64 {
func (n nodes) weights(wf weightFunc) []float64 { if cap(w) < len(n) {
w := make([]float64, 0, len(n)) w = make([]float64, 0, len(n))
}
for i := range n { for i := range n {
w = append(w, wf(n[i])) w = append(w, wf(n[i]))
} }
@ -149,10 +150,13 @@ func (m NetMap) PlacementVectors(vectors [][]NodeInfo, pivot []byte) ([][]NodeIn
wf := defaultWeightFunc(m.nodes) wf := defaultWeightFunc(m.nodes)
result := make([][]NodeInfo, len(vectors)) result := make([][]NodeInfo, len(vectors))
var ws []float64
for i := range vectors { for i := range vectors {
result[i] = make([]NodeInfo, len(vectors[i])) result[i] = make([]NodeInfo, len(vectors[i]))
copy(result[i], vectors[i]) copy(result[i], vectors[i])
hrw.SortHasherSliceByWeightValue(result[i], nodes(result[i]).weights(wf), h) ws = nodes(result[i]).appendWeightsTo(wf, ws[:0])
hrw.SortHasherSliceByWeightValue(result[i], ws, h)
} }
return result, nil return result, nil
@ -209,6 +213,25 @@ func (m NetMap) SelectFilterNodes(expr *SelectFilterExpr) ([][]NodeInfo, error)
return ret, nil return ret, nil
} }
func countNodes(r netmap.Replica) uint32 {
if r.GetCount() != 0 {
return r.GetCount()
}
return r.GetECDataCount() + r.GetECParityCount()
}
func (p PlacementPolicy) isUnique() bool {
if p.unique {
return true
}
for _, r := range p.replicas {
if r.GetECDataCount() != 0 || r.GetECParityCount() != 0 {
return true
}
}
return false
}
// ContainerNodes returns two-dimensional list of nodes as a result of applying // ContainerNodes returns two-dimensional list of nodes as a result of applying
// given PlacementPolicy to the NetMap. Each line of the list corresponds to a // given PlacementPolicy to the NetMap. Each line of the list corresponds to a
// replica descriptor. Line order corresponds to order of ReplicaDescriptor list // replica descriptor. Line order corresponds to order of ReplicaDescriptor list
@ -230,6 +253,7 @@ func (m NetMap) ContainerNodes(p PlacementPolicy, pivot []byte) ([][]NodeInfo, e
return nil, err return nil, err
} }
unique := p.isUnique()
result := make([][]NodeInfo, len(p.replicas)) result := make([][]NodeInfo, len(p.replicas))
// Note that the cached selectors are not used when the policy contains the UNIQUE flag. // Note that the cached selectors are not used when the policy contains the UNIQUE flag.
@ -240,7 +264,7 @@ func (m NetMap) ContainerNodes(p PlacementPolicy, pivot []byte) ([][]NodeInfo, e
sName := p.replicas[i].GetSelector() sName := p.replicas[i].GetSelector()
if sName == "" && !(len(p.replicas) == 1 && len(p.selectors) == 1) { if sName == "" && !(len(p.replicas) == 1 && len(p.selectors) == 1) {
var s netmap.Selector var s netmap.Selector
s.SetCount(p.replicas[i].GetCount()) s.SetCount(countNodes(p.replicas[i]))
s.SetFilter(mainFilterName) s.SetFilter(mainFilterName)
nodes, err := c.getSelection(s) nodes, err := c.getSelection(s)
@ -250,14 +274,14 @@ func (m NetMap) ContainerNodes(p PlacementPolicy, pivot []byte) ([][]NodeInfo, e
result[i] = append(result[i], flattenNodes(nodes)...) result[i] = append(result[i], flattenNodes(nodes)...)
if p.unique { if unique {
c.addUsedNodes(result[i]...) c.addUsedNodes(result[i]...)
} }
continue continue
} }
if p.unique { if unique {
if c.processedSelectors[sName] == nil { if c.processedSelectors[sName] == nil {
return nil, fmt.Errorf("selector not found: '%s'", sName) return nil, fmt.Errorf("selector not found: '%s'", sName)
} }

View file

@ -49,11 +49,6 @@ func (x *NetworkInfo) readFromV2(m netmap.NetworkInfo, checkFieldPresence bool)
mNames[name] = struct{}{} mNames[name] = struct{}{}
switch name { switch name {
default:
if len(prm.GetValue()) == 0 {
err = fmt.Errorf("empty attribute value %s", name)
return true
}
case case
configAuditFee, configAuditFee,
configStoragePrice, configStoragePrice,
@ -62,6 +57,8 @@ func (x *NetworkInfo) readFromV2(m netmap.NetworkInfo, checkFieldPresence bool)
configEpochDuration, configEpochDuration,
configIRCandidateFee, configIRCandidateFee,
configMaxObjSize, configMaxObjSize,
configMaxECDataCount,
configMaxECParityCount,
configWithdrawalFee: configWithdrawalFee:
_, err = decodeConfigValueUint64(prm.GetValue()) _, err = decodeConfigValueUint64(prm.GetValue())
case configHomomorphicHashingDisabled, case configHomomorphicHashingDisabled,
@ -234,6 +231,8 @@ func (x *NetworkInfo) IterateRawNetworkParameters(f func(name string, value []by
configEpochDuration, configEpochDuration,
configIRCandidateFee, configIRCandidateFee,
configMaxObjSize, configMaxObjSize,
configMaxECDataCount,
configMaxECParityCount,
configWithdrawalFee, configWithdrawalFee,
configHomomorphicHashingDisabled, configHomomorphicHashingDisabled,
configMaintenanceModeAllowed: configMaintenanceModeAllowed:
@ -432,6 +431,34 @@ func (x NetworkInfo) MaxObjectSize() uint64 {
return x.configUint64(configMaxObjSize) return x.configUint64(configMaxObjSize)
} }
const configMaxECDataCount = "MaxECDataCount"
// SetMaxECDataCount sets maximum number of data shards for erasure codes.
//
// Zero means no restrictions.
func (x *NetworkInfo) SetMaxECDataCount(dataCount uint64) {
x.setConfigUint64(configMaxECDataCount, dataCount)
}
// MaxECDataCount returns maximum number of data shards for erasure codes.
func (x NetworkInfo) MaxECDataCount() uint64 {
return x.configUint64(configMaxECDataCount)
}
const configMaxECParityCount = "MaxECParityCount"
// SetMaxECParityCount sets maximum number of parity shards for erasure codes.
//
// Zero means no restrictions.
func (x *NetworkInfo) SetMaxECParityCount(parityCount uint64) {
x.setConfigUint64(configMaxECParityCount, parityCount)
}
// MaxECParityCount returns maximum number of parity shards for erasure codes.
func (x NetworkInfo) MaxECParityCount() uint64 {
return x.configUint64(configMaxECParityCount)
}
const configWithdrawalFee = "WithdrawFee" const configWithdrawalFee = "WithdrawFee"
// SetWithdrawalFee sets fee for withdrawals from the FrostFS accounts that // SetWithdrawalFee sets fee for withdrawals from the FrostFS accounts that

View file

@ -173,6 +173,32 @@ func TestNetworkInfo_MaxObjectSize(t *testing.T) {
) )
} }
func TestNetworkInfo_MaxECDataCount(t *testing.T) {
testConfigValue(t,
func(x NetworkInfo) any { return x.MaxECDataCount() },
func(info *NetworkInfo, val any) { info.SetMaxECDataCount(val.(uint64)) },
uint64(1), uint64(2),
"MaxECDataCount", func(val any) []byte {
data := make([]byte, 8)
binary.LittleEndian.PutUint64(data, val.(uint64))
return data
},
)
}
func TestNetworkInfo_MaxECParityCount(t *testing.T) {
testConfigValue(t,
func(x NetworkInfo) any { return x.MaxECParityCount() },
func(info *NetworkInfo, val any) { info.SetMaxECParityCount(val.(uint64)) },
uint64(1), uint64(2),
"MaxECParityCount", func(val any) []byte {
data := make([]byte, 8)
binary.LittleEndian.PutUint64(data, val.(uint64))
return data
},
)
}
func TestNetworkInfo_WithdrawalFee(t *testing.T) { func TestNetworkInfo_WithdrawalFee(t *testing.T) {
testConfigValue(t, testConfigValue(t,
func(x NetworkInfo) any { return x.WithdrawalFee() }, func(x NetworkInfo) any { return x.WithdrawalFee() },

View file

@ -3,7 +3,7 @@ package netmap
import ( import (
"errors" "errors"
"fmt" "fmt"
"sort" "slices"
"strconv" "strconv"
"strings" "strings"
@ -227,14 +227,6 @@ func (x NodeInfo) Hash() uint64 {
return hrw.Hash(x.m.GetPublicKey()) return hrw.Hash(x.m.GetPublicKey())
} }
// less declares "less than" comparison between two NodeInfo instances:
// x1 is less than x2 if it has less Hash().
//
// Method is needed for internal placement needs.
func less(x1, x2 NodeInfo) bool {
return x1.Hash() < x2.Hash()
}
func (x *NodeInfo) setNumericAttribute(key string, num uint64) { func (x *NodeInfo) setNumericAttribute(key string, num uint64) {
x.SetAttribute(key, strconv.FormatUint(num, 10)) x.SetAttribute(key, strconv.FormatUint(num, 10))
} }
@ -455,15 +447,11 @@ func (x *NodeInfo) SortAttributes() {
return return
} }
sort.Slice(as, func(i, j int) bool { slices.SortFunc(as, func(ai, aj netmap.Attribute) int {
switch strings.Compare(as[i].GetKey(), as[j].GetKey()) { if r := strings.Compare(ai.GetKey(), aj.GetKey()); r != 0 {
case -1: return r
return true
case 1:
return false
default:
return as[i].GetValue() < as[j].GetValue()
} }
return strings.Compare(ai.GetValue(), aj.GetValue())
}) })
x.m.SetAttributes(as) x.m.SetAttributes(as)
@ -472,6 +460,10 @@ func (x *NodeInfo) SortAttributes() {
// SetOffline sets the state of the node to "offline". When a node updates // SetOffline sets the state of the node to "offline". When a node updates
// information about itself in the network map, this action is interpreted as // information about itself in the network map, this action is interpreted as
// an intention to leave the network. // an intention to leave the network.
//
// See also IsOffline.
//
// Deprecated: use SetStatus instead.
func (x *NodeInfo) SetOffline() { func (x *NodeInfo) SetOffline() {
x.m.SetState(netmap.Offline) x.m.SetState(netmap.Offline)
} }
@ -482,6 +474,8 @@ func (x *NodeInfo) SetOffline() {
// mean online). // mean online).
// //
// See also SetOffline. // See also SetOffline.
//
// Deprecated: use Status instead.
func (x NodeInfo) IsOffline() bool { func (x NodeInfo) IsOffline() bool {
return x.m.GetState() == netmap.Offline return x.m.GetState() == netmap.Offline
} }
@ -491,6 +485,8 @@ func (x NodeInfo) IsOffline() bool {
// action is interpreted as an intention to enter the network. // action is interpreted as an intention to enter the network.
// //
// See also IsOnline. // See also IsOnline.
//
// Deprecated: use SetStatus instead.
func (x *NodeInfo) SetOnline() { func (x *NodeInfo) SetOnline() {
x.m.SetState(netmap.Online) x.m.SetState(netmap.Online)
} }
@ -501,6 +497,8 @@ func (x *NodeInfo) SetOnline() {
// mean offline). // mean offline).
// //
// See also SetOnline. // See also SetOnline.
//
// Deprecated: use Status instead.
func (x NodeInfo) IsOnline() bool { func (x NodeInfo) IsOnline() bool {
return x.m.GetState() == netmap.Online return x.m.GetState() == netmap.Online
} }
@ -510,6 +508,8 @@ func (x NodeInfo) IsOnline() bool {
// state declares temporal unavailability for a node. // state declares temporal unavailability for a node.
// //
// See also IsMaintenance. // See also IsMaintenance.
//
// Deprecated: use SetStatus instead.
func (x *NodeInfo) SetMaintenance() { func (x *NodeInfo) SetMaintenance() {
x.m.SetState(netmap.Maintenance) x.m.SetState(netmap.Maintenance)
} }
@ -519,6 +519,63 @@ func (x *NodeInfo) SetMaintenance() {
// Zero NodeInfo has undefined state. // Zero NodeInfo has undefined state.
// //
// See also SetMaintenance. // See also SetMaintenance.
//
// Deprecated: use Status instead.
func (x NodeInfo) IsMaintenance() bool { func (x NodeInfo) IsMaintenance() bool {
return x.m.GetState() == netmap.Maintenance return x.m.GetState() == netmap.Maintenance
} }
type NodeState netmap.NodeState
const (
UnspecifiedState = NodeState(netmap.UnspecifiedState)
Online = NodeState(netmap.Online)
Offline = NodeState(netmap.Offline)
Maintenance = NodeState(netmap.Maintenance)
)
// ToV2 converts NodeState to v2.
func (ns NodeState) ToV2() netmap.NodeState {
return netmap.NodeState(ns)
}
// FromV2 reads NodeState to v2.
func (ns *NodeState) FromV2(state netmap.NodeState) {
*ns = NodeState(state)
}
// Status returns the current state of the node in the network map.
//
// Zero NodeInfo has an undefined state, neither online nor offline.
func (x NodeInfo) Status() NodeState {
return NodeState(x.m.GetState())
}
// SetState updates the state of the node in the network map.
//
// The state determines the node's current status within the network:
// - "online": Indicates the node intends to enter the network.
// - "offline": Indicates the node intends to leave the network.
// - "maintenance": Indicates the node is temporarily unavailable.
//
// See also Status.
func (x *NodeInfo) SetStatus(state NodeState) {
x.m.SetState(netmap.NodeState(state))
}
// String implements fmt.Stringer.
//
// String is designed to be human-readable, and its format MAY differ between
// SDK versions.
func (ns NodeState) String() string {
return netmap.NodeState(ns).String()
}
// IsOnline checks if the current state is "online".
func (ns NodeState) IsOnline() bool { return ns == Online }
// IsOffline checks if the current state is "offline".
func (ns NodeState) IsOffline() bool { return ns == Offline }
// IsMaintenance checks if the current state is "maintenance".
func (ns NodeState) IsMaintenance() bool { return ns == Maintenance }

View file

@ -3,6 +3,7 @@ package netmap
import ( import (
"testing" "testing"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
@ -23,27 +24,75 @@ func TestNodeInfo_SetAttribute(t *testing.T) {
require.Equal(t, val, n.Attribute(key)) require.Equal(t, val, n.Attribute(key))
} }
func TestNodeState(t *testing.T) {
m := map[NodeState]netmap.NodeState{
UnspecifiedState: netmap.UnspecifiedState,
Online: netmap.Online,
Offline: netmap.Offline,
Maintenance: netmap.Maintenance,
}
t.Run("from sdk to v2", func(t *testing.T) {
for stateSDK, stateV2 := range m {
require.Equal(t, stateV2, stateSDK.ToV2())
}
})
t.Run("from v2 to sdk", func(t *testing.T) {
for stateSDK, stateV2 := range m {
var state NodeState
state.FromV2(stateV2)
require.Equal(t, stateSDK, state)
}
})
}
func TestNodeInfo_Status(t *testing.T) { func TestNodeInfo_Status(t *testing.T) {
var n NodeInfo t.Run("deprecated getters/setters", func(t *testing.T) {
var n NodeInfo
require.False(t, n.IsOnline()) require.False(t, n.IsOnline())
require.False(t, n.IsOffline()) require.False(t, n.IsOffline())
require.False(t, n.IsMaintenance()) require.False(t, n.IsMaintenance())
n.SetOnline() n.SetOnline()
require.True(t, n.IsOnline()) require.True(t, n.IsOnline())
require.False(t, n.IsOffline()) require.False(t, n.IsOffline())
require.False(t, n.IsMaintenance()) require.False(t, n.IsMaintenance())
n.SetOffline() n.SetOffline()
require.True(t, n.IsOffline()) require.True(t, n.IsOffline())
require.False(t, n.IsOnline()) require.False(t, n.IsOnline())
require.False(t, n.IsMaintenance()) require.False(t, n.IsMaintenance())
n.SetMaintenance() n.SetMaintenance()
require.True(t, n.IsMaintenance()) require.True(t, n.IsMaintenance())
require.False(t, n.IsOnline()) require.False(t, n.IsOnline())
require.False(t, n.IsOffline()) require.False(t, n.IsOffline())
})
t.Run("brand new getters/setters", func(t *testing.T) {
var n NodeInfo
require.False(t, n.Status().IsOnline())
require.False(t, n.Status().IsOffline())
require.False(t, n.Status().IsMaintenance())
n.SetStatus(Online)
require.True(t, n.Status().IsOnline())
require.False(t, n.Status().IsOffline())
require.False(t, n.Status().IsMaintenance())
n.SetStatus(Offline)
require.False(t, n.Status().IsOnline())
require.True(t, n.Status().IsOffline())
require.False(t, n.Status().IsMaintenance())
n.SetStatus(Maintenance)
require.False(t, n.Status().IsOnline())
require.False(t, n.Status().IsOffline())
require.True(t, n.Status().IsMaintenance())
})
} }
func TestNodeInfo_ExternalAddr(t *testing.T) { func TestNodeInfo_ExternalAddr(t *testing.T) {

View file

@ -4,10 +4,14 @@ options {
tokenVocab = QueryLexer; tokenVocab = QueryLexer;
} }
policy: UNIQUE? repStmt+ cbfStmt? selectStmt* filterStmt* EOF; policy: UNIQUE? (repStmt | ecStmt)+ cbfStmt? selectStmt* filterStmt* EOF;
selectFilterExpr: cbfStmt? selectStmt? filterStmt* EOF; selectFilterExpr: cbfStmt? selectStmt? filterStmt* EOF;
ecStmt:
EC Data = NUMBER1 DOT Parity = NUMBER1 // erasure code configuration
(IN Selector = ident)?; // optional selector name
repStmt: repStmt:
REP Count = NUMBER1 // number of object replicas REP Count = NUMBER1 // number of object replicas
(IN Selector = ident)?; // optional selector name (IN Selector = ident)?; // optional selector name

Binary file not shown.

Binary file not shown.

View file

@ -3,10 +3,11 @@ lexer grammar QueryLexer;
NOT_OP : 'NOT'; NOT_OP : 'NOT';
AND_OP : 'AND'; AND_OP : 'AND';
OR_OP : 'OR'; OR_OP : 'OR';
SIMPLE_OP : 'EQ' | 'NE' | 'GE' | 'GT' | 'LT' | 'LE'; SIMPLE_OP : 'EQ' | 'NE' | 'GE' | 'GT' | 'LT' | 'LE' | 'LIKE' ;
UNIQUE : 'UNIQUE'; UNIQUE : 'UNIQUE';
REP : 'REP'; REP : 'REP';
EC : 'EC';
IN : 'IN'; IN : 'IN';
AS : 'AS'; AS : 'AS';
CBF : 'CBF'; CBF : 'CBF';
@ -14,6 +15,7 @@ SELECT : 'SELECT';
FROM : 'FROM'; FROM : 'FROM';
FILTER : 'FILTER'; FILTER : 'FILTER';
WILDCARD : '*'; WILDCARD : '*';
DOT : '.';
CLAUSE_SAME : 'SAME'; CLAUSE_SAME : 'SAME';
CLAUSE_DISTINCT : 'DISTINCT'; CLAUSE_DISTINCT : 'DISTINCT';

Binary file not shown.

Binary file not shown.

View file

@ -1,4 +1,4 @@
// Code generated from Query.g4 by ANTLR 4.13.0. DO NOT EDIT. // Code generated from /repo/frostfs/sdk-go/netmap/parser/Query.g4 by ANTLR 4.13.0. DO NOT EDIT.
package parser // Query package parser // Query
@ -16,6 +16,10 @@ func (v *BaseQueryVisitor) VisitSelectFilterExpr(ctx *SelectFilterExprContext) i
return v.VisitChildren(ctx) return v.VisitChildren(ctx)
} }
func (v *BaseQueryVisitor) VisitEcStmt(ctx *EcStmtContext) interface{} {
return v.VisitChildren(ctx)
}
func (v *BaseQueryVisitor) VisitRepStmt(ctx *RepStmtContext) interface{} { func (v *BaseQueryVisitor) VisitRepStmt(ctx *RepStmtContext) interface{} {
return v.VisitChildren(ctx) return v.VisitChildren(ctx)
} }

View file

@ -1,4 +1,4 @@
// Code generated from QueryLexer.g4 by ANTLR 4.13.0. DO NOT EDIT. // Code generated from /repo/frostfs/sdk-go/netmap/parser/QueryLexer.g4 by ANTLR 4.13.0. DO NOT EDIT.
package parser package parser
@ -43,119 +43,125 @@ func querylexerLexerInit() {
"DEFAULT_MODE", "DEFAULT_MODE",
} }
staticData.LiteralNames = []string{ staticData.LiteralNames = []string{
"", "'NOT'", "'AND'", "'OR'", "", "'UNIQUE'", "'REP'", "'IN'", "'AS'", "", "'NOT'", "'AND'", "'OR'", "", "'UNIQUE'", "'REP'", "'EC'", "'IN'",
"'CBF'", "'SELECT'", "'FROM'", "'FILTER'", "'*'", "'SAME'", "'DISTINCT'", "'AS'", "'CBF'", "'SELECT'", "'FROM'", "'FILTER'", "'*'", "'.'", "'SAME'",
"'('", "')'", "'@'", "", "", "'0'", "'DISTINCT'", "'('", "')'", "'@'", "", "", "'0'",
} }
staticData.SymbolicNames = []string{ staticData.SymbolicNames = []string{
"", "NOT_OP", "AND_OP", "OR_OP", "SIMPLE_OP", "UNIQUE", "REP", "IN", "", "NOT_OP", "AND_OP", "OR_OP", "SIMPLE_OP", "UNIQUE", "REP", "EC",
"AS", "CBF", "SELECT", "FROM", "FILTER", "WILDCARD", "CLAUSE_SAME", "IN", "AS", "CBF", "SELECT", "FROM", "FILTER", "WILDCARD", "DOT", "CLAUSE_SAME",
"CLAUSE_DISTINCT", "L_PAREN", "R_PAREN", "AT", "IDENT", "NUMBER1", "ZERO", "CLAUSE_DISTINCT", "L_PAREN", "R_PAREN", "AT", "IDENT", "NUMBER1", "ZERO",
"STRING", "WS", "STRING", "WS",
} }
staticData.RuleNames = []string{ staticData.RuleNames = []string{
"NOT_OP", "AND_OP", "OR_OP", "SIMPLE_OP", "UNIQUE", "REP", "IN", "AS", "NOT_OP", "AND_OP", "OR_OP", "SIMPLE_OP", "UNIQUE", "REP", "EC", "IN",
"CBF", "SELECT", "FROM", "FILTER", "WILDCARD", "CLAUSE_SAME", "CLAUSE_DISTINCT", "AS", "CBF", "SELECT", "FROM", "FILTER", "WILDCARD", "DOT", "CLAUSE_SAME",
"L_PAREN", "R_PAREN", "AT", "IDENT", "Digit", "Nondigit", "NUMBER1", "CLAUSE_DISTINCT", "L_PAREN", "R_PAREN", "AT", "IDENT", "Digit", "Nondigit",
"ZERO", "STRING", "ESC", "UNICODE", "HEX", "SAFECODEPOINTSINGLE", "SAFECODEPOINTDOUBLE", "NUMBER1", "ZERO", "STRING", "ESC", "UNICODE", "HEX", "SAFECODEPOINTSINGLE",
"WS", "SAFECODEPOINTDOUBLE", "WS",
} }
staticData.PredictionContextCache = antlr.NewPredictionContextCache() staticData.PredictionContextCache = antlr.NewPredictionContextCache()
staticData.serializedATN = []int32{ staticData.serializedATN = []int32{
4, 0, 23, 213, 6, -1, 2, 0, 7, 0, 2, 1, 7, 1, 2, 2, 7, 2, 2, 3, 7, 3, 2, 4, 0, 25, 226, 6, -1, 2, 0, 7, 0, 2, 1, 7, 1, 2, 2, 7, 2, 2, 3, 7, 3, 2,
4, 7, 4, 2, 5, 7, 5, 2, 6, 7, 6, 2, 7, 7, 7, 2, 8, 7, 8, 2, 9, 7, 9, 2, 4, 7, 4, 2, 5, 7, 5, 2, 6, 7, 6, 2, 7, 7, 7, 2, 8, 7, 8, 2, 9, 7, 9, 2,
10, 7, 10, 2, 11, 7, 11, 2, 12, 7, 12, 2, 13, 7, 13, 2, 14, 7, 14, 2, 15, 10, 7, 10, 2, 11, 7, 11, 2, 12, 7, 12, 2, 13, 7, 13, 2, 14, 7, 14, 2, 15,
7, 15, 2, 16, 7, 16, 2, 17, 7, 17, 2, 18, 7, 18, 2, 19, 7, 19, 2, 20, 7, 7, 15, 2, 16, 7, 16, 2, 17, 7, 17, 2, 18, 7, 18, 2, 19, 7, 19, 2, 20, 7,
20, 2, 21, 7, 21, 2, 22, 7, 22, 2, 23, 7, 23, 2, 24, 7, 24, 2, 25, 7, 25, 20, 2, 21, 7, 21, 2, 22, 7, 22, 2, 23, 7, 23, 2, 24, 7, 24, 2, 25, 7, 25,
2, 26, 7, 26, 2, 27, 7, 27, 2, 28, 7, 28, 2, 29, 7, 29, 1, 0, 1, 0, 1, 2, 26, 7, 26, 2, 27, 7, 27, 2, 28, 7, 28, 2, 29, 7, 29, 2, 30, 7, 30, 2,
0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2, 1, 2, 1, 3, 1, 3, 1, 3, 1, 31, 7, 31, 1, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2,
3, 1, 3, 1, 3, 1, 3, 1, 3, 1, 3, 1, 3, 1, 3, 1, 3, 3, 3, 85, 8, 3, 1, 4, 1, 2, 1, 3, 1, 3, 1, 3, 1, 3, 1, 3, 1, 3, 1, 3, 1, 3, 1, 3, 1, 3, 1, 3,
1, 4, 1, 4, 1, 4, 1, 4, 1, 4, 1, 4, 1, 5, 1, 5, 1, 5, 1, 5, 1, 6, 1, 6, 1, 3, 1, 3, 1, 3, 1, 3, 1, 3, 3, 3, 93, 8, 3, 1, 4, 1, 4, 1, 4, 1, 4, 1,
1, 6, 1, 7, 1, 7, 1, 7, 1, 8, 1, 8, 1, 8, 1, 8, 1, 9, 1, 9, 1, 9, 1, 9, 4, 1, 4, 1, 4, 1, 5, 1, 5, 1, 5, 1, 5, 1, 6, 1, 6, 1, 6, 1, 7, 1, 7, 1,
1, 9, 1, 9, 1, 9, 1, 10, 1, 10, 1, 10, 1, 10, 1, 10, 1, 11, 1, 11, 1, 11, 7, 1, 8, 1, 8, 1, 8, 1, 9, 1, 9, 1, 9, 1, 9, 1, 10, 1, 10, 1, 10, 1, 10,
1, 11, 1, 11, 1, 11, 1, 11, 1, 12, 1, 12, 1, 13, 1, 13, 1, 13, 1, 13, 1, 1, 10, 1, 10, 1, 10, 1, 11, 1, 11, 1, 11, 1, 11, 1, 11, 1, 12, 1, 12, 1,
13, 1, 14, 1, 14, 1, 14, 1, 14, 1, 14, 1, 14, 1, 14, 1, 14, 1, 14, 1, 15, 12, 1, 12, 1, 12, 1, 12, 1, 12, 1, 13, 1, 13, 1, 14, 1, 14, 1, 15, 1, 15,
1, 15, 1, 16, 1, 16, 1, 17, 1, 17, 1, 18, 1, 18, 1, 18, 5, 18, 152, 8, 1, 15, 1, 15, 1, 15, 1, 16, 1, 16, 1, 16, 1, 16, 1, 16, 1, 16, 1, 16, 1,
18, 10, 18, 12, 18, 155, 9, 18, 1, 19, 1, 19, 1, 20, 1, 20, 1, 21, 1, 21, 16, 1, 16, 1, 17, 1, 17, 1, 18, 1, 18, 1, 19, 1, 19, 1, 20, 1, 20, 1, 20,
5, 21, 163, 8, 21, 10, 21, 12, 21, 166, 9, 21, 1, 22, 1, 22, 1, 23, 1, 5, 20, 165, 8, 20, 10, 20, 12, 20, 168, 9, 20, 1, 21, 1, 21, 1, 22, 1,
23, 1, 23, 5, 23, 173, 8, 23, 10, 23, 12, 23, 176, 9, 23, 1, 23, 1, 23, 22, 1, 23, 1, 23, 5, 23, 176, 8, 23, 10, 23, 12, 23, 179, 9, 23, 1, 24,
1, 23, 1, 23, 5, 23, 182, 8, 23, 10, 23, 12, 23, 185, 9, 23, 1, 23, 3, 1, 24, 1, 25, 1, 25, 1, 25, 5, 25, 186, 8, 25, 10, 25, 12, 25, 189, 9,
23, 188, 8, 23, 1, 24, 1, 24, 1, 24, 3, 24, 193, 8, 24, 1, 25, 1, 25, 1, 25, 1, 25, 1, 25, 1, 25, 1, 25, 5, 25, 195, 8, 25, 10, 25, 12, 25, 198,
25, 1, 25, 1, 25, 1, 25, 1, 26, 1, 26, 1, 27, 1, 27, 1, 28, 1, 28, 1, 29, 9, 25, 1, 25, 3, 25, 201, 8, 25, 1, 26, 1, 26, 1, 26, 3, 26, 206, 8, 26,
4, 29, 208, 8, 29, 11, 29, 12, 29, 209, 1, 29, 1, 29, 0, 0, 30, 1, 1, 3, 1, 27, 1, 27, 1, 27, 1, 27, 1, 27, 1, 27, 1, 28, 1, 28, 1, 29, 1, 29, 1,
2, 5, 3, 7, 4, 9, 5, 11, 6, 13, 7, 15, 8, 17, 9, 19, 10, 21, 11, 23, 12, 30, 1, 30, 1, 31, 4, 31, 221, 8, 31, 11, 31, 12, 31, 222, 1, 31, 1, 31,
25, 13, 27, 14, 29, 15, 31, 16, 33, 17, 35, 18, 37, 19, 39, 0, 41, 0, 43, 0, 0, 32, 1, 1, 3, 2, 5, 3, 7, 4, 9, 5, 11, 6, 13, 7, 15, 8, 17, 9, 19,
20, 45, 21, 47, 22, 49, 0, 51, 0, 53, 0, 55, 0, 57, 0, 59, 23, 1, 0, 8, 10, 21, 11, 23, 12, 25, 13, 27, 14, 29, 15, 31, 16, 33, 17, 35, 18, 37,
1, 0, 48, 57, 3, 0, 65, 90, 95, 95, 97, 122, 1, 0, 49, 57, 9, 0, 34, 34, 19, 39, 20, 41, 21, 43, 0, 45, 0, 47, 22, 49, 23, 51, 24, 53, 0, 55, 0,
39, 39, 47, 47, 92, 92, 98, 98, 102, 102, 110, 110, 114, 114, 116, 116, 57, 0, 59, 0, 61, 0, 63, 25, 1, 0, 8, 1, 0, 48, 57, 3, 0, 65, 90, 95, 95,
3, 0, 48, 57, 65, 70, 97, 102, 3, 0, 0, 31, 39, 39, 92, 92, 3, 0, 0, 31, 97, 122, 1, 0, 49, 57, 9, 0, 34, 34, 39, 39, 47, 47, 92, 92, 98, 98, 102,
34, 34, 92, 92, 3, 0, 9, 10, 13, 13, 32, 32, 220, 0, 1, 1, 0, 0, 0, 0, 102, 110, 110, 114, 114, 116, 116, 3, 0, 48, 57, 65, 70, 97, 102, 3, 0,
3, 1, 0, 0, 0, 0, 5, 1, 0, 0, 0, 0, 7, 1, 0, 0, 0, 0, 9, 1, 0, 0, 0, 0, 0, 31, 39, 39, 92, 92, 3, 0, 0, 31, 34, 34, 92, 92, 3, 0, 9, 10, 13, 13,
11, 1, 0, 0, 0, 0, 13, 1, 0, 0, 0, 0, 15, 1, 0, 0, 0, 0, 17, 1, 0, 0, 0, 32, 32, 234, 0, 1, 1, 0, 0, 0, 0, 3, 1, 0, 0, 0, 0, 5, 1, 0, 0, 0, 0, 7,
0, 19, 1, 0, 0, 0, 0, 21, 1, 0, 0, 0, 0, 23, 1, 0, 0, 0, 0, 25, 1, 0, 0, 1, 0, 0, 0, 0, 9, 1, 0, 0, 0, 0, 11, 1, 0, 0, 0, 0, 13, 1, 0, 0, 0, 0,
0, 0, 27, 1, 0, 0, 0, 0, 29, 1, 0, 0, 0, 0, 31, 1, 0, 0, 0, 0, 33, 1, 0, 15, 1, 0, 0, 0, 0, 17, 1, 0, 0, 0, 0, 19, 1, 0, 0, 0, 0, 21, 1, 0, 0, 0,
0, 0, 0, 35, 1, 0, 0, 0, 0, 37, 1, 0, 0, 0, 0, 43, 1, 0, 0, 0, 0, 45, 1, 0, 23, 1, 0, 0, 0, 0, 25, 1, 0, 0, 0, 0, 27, 1, 0, 0, 0, 0, 29, 1, 0, 0,
0, 0, 0, 0, 47, 1, 0, 0, 0, 0, 59, 1, 0, 0, 0, 1, 61, 1, 0, 0, 0, 3, 65, 0, 0, 31, 1, 0, 0, 0, 0, 33, 1, 0, 0, 0, 0, 35, 1, 0, 0, 0, 0, 37, 1, 0,
1, 0, 0, 0, 5, 69, 1, 0, 0, 0, 7, 84, 1, 0, 0, 0, 9, 86, 1, 0, 0, 0, 11, 0, 0, 0, 39, 1, 0, 0, 0, 0, 41, 1, 0, 0, 0, 0, 47, 1, 0, 0, 0, 0, 49, 1,
93, 1, 0, 0, 0, 13, 97, 1, 0, 0, 0, 15, 100, 1, 0, 0, 0, 17, 103, 1, 0, 0, 0, 0, 0, 51, 1, 0, 0, 0, 0, 63, 1, 0, 0, 0, 1, 65, 1, 0, 0, 0, 3, 69,
0, 0, 19, 107, 1, 0, 0, 0, 21, 114, 1, 0, 0, 0, 23, 119, 1, 0, 0, 0, 25, 1, 0, 0, 0, 5, 73, 1, 0, 0, 0, 7, 92, 1, 0, 0, 0, 9, 94, 1, 0, 0, 0, 11,
126, 1, 0, 0, 0, 27, 128, 1, 0, 0, 0, 29, 133, 1, 0, 0, 0, 31, 142, 1, 101, 1, 0, 0, 0, 13, 105, 1, 0, 0, 0, 15, 108, 1, 0, 0, 0, 17, 111, 1,
0, 0, 0, 33, 144, 1, 0, 0, 0, 35, 146, 1, 0, 0, 0, 37, 148, 1, 0, 0, 0, 0, 0, 0, 19, 114, 1, 0, 0, 0, 21, 118, 1, 0, 0, 0, 23, 125, 1, 0, 0, 0,
39, 156, 1, 0, 0, 0, 41, 158, 1, 0, 0, 0, 43, 160, 1, 0, 0, 0, 45, 167, 25, 130, 1, 0, 0, 0, 27, 137, 1, 0, 0, 0, 29, 139, 1, 0, 0, 0, 31, 141,
1, 0, 0, 0, 47, 187, 1, 0, 0, 0, 49, 189, 1, 0, 0, 0, 51, 194, 1, 0, 0, 1, 0, 0, 0, 33, 146, 1, 0, 0, 0, 35, 155, 1, 0, 0, 0, 37, 157, 1, 0, 0,
0, 53, 200, 1, 0, 0, 0, 55, 202, 1, 0, 0, 0, 57, 204, 1, 0, 0, 0, 59, 207, 0, 39, 159, 1, 0, 0, 0, 41, 161, 1, 0, 0, 0, 43, 169, 1, 0, 0, 0, 45, 171,
1, 0, 0, 0, 61, 62, 5, 78, 0, 0, 62, 63, 5, 79, 0, 0, 63, 64, 5, 84, 0, 1, 0, 0, 0, 47, 173, 1, 0, 0, 0, 49, 180, 1, 0, 0, 0, 51, 200, 1, 0, 0,
0, 64, 2, 1, 0, 0, 0, 65, 66, 5, 65, 0, 0, 66, 67, 5, 78, 0, 0, 67, 68, 0, 53, 202, 1, 0, 0, 0, 55, 207, 1, 0, 0, 0, 57, 213, 1, 0, 0, 0, 59, 215,
5, 68, 0, 0, 68, 4, 1, 0, 0, 0, 69, 70, 5, 79, 0, 0, 70, 71, 5, 82, 0, 1, 0, 0, 0, 61, 217, 1, 0, 0, 0, 63, 220, 1, 0, 0, 0, 65, 66, 5, 78, 0,
0, 71, 6, 1, 0, 0, 0, 72, 73, 5, 69, 0, 0, 73, 85, 5, 81, 0, 0, 74, 75, 0, 66, 67, 5, 79, 0, 0, 67, 68, 5, 84, 0, 0, 68, 2, 1, 0, 0, 0, 69, 70,
5, 78, 0, 0, 75, 85, 5, 69, 0, 0, 76, 77, 5, 71, 0, 0, 77, 85, 5, 69, 0, 5, 65, 0, 0, 70, 71, 5, 78, 0, 0, 71, 72, 5, 68, 0, 0, 72, 4, 1, 0, 0,
0, 78, 79, 5, 71, 0, 0, 79, 85, 5, 84, 0, 0, 80, 81, 5, 76, 0, 0, 81, 85, 0, 73, 74, 5, 79, 0, 0, 74, 75, 5, 82, 0, 0, 75, 6, 1, 0, 0, 0, 76, 77,
5, 84, 0, 0, 82, 83, 5, 76, 0, 0, 83, 85, 5, 69, 0, 0, 84, 72, 1, 0, 0, 5, 69, 0, 0, 77, 93, 5, 81, 0, 0, 78, 79, 5, 78, 0, 0, 79, 93, 5, 69, 0,
0, 84, 74, 1, 0, 0, 0, 84, 76, 1, 0, 0, 0, 84, 78, 1, 0, 0, 0, 84, 80, 0, 80, 81, 5, 71, 0, 0, 81, 93, 5, 69, 0, 0, 82, 83, 5, 71, 0, 0, 83, 93,
1, 0, 0, 0, 84, 82, 1, 0, 0, 0, 85, 8, 1, 0, 0, 0, 86, 87, 5, 85, 0, 0, 5, 84, 0, 0, 84, 85, 5, 76, 0, 0, 85, 93, 5, 84, 0, 0, 86, 87, 5, 76, 0,
87, 88, 5, 78, 0, 0, 88, 89, 5, 73, 0, 0, 89, 90, 5, 81, 0, 0, 90, 91, 0, 87, 93, 5, 69, 0, 0, 88, 89, 5, 76, 0, 0, 89, 90, 5, 73, 0, 0, 90, 91,
5, 85, 0, 0, 91, 92, 5, 69, 0, 0, 92, 10, 1, 0, 0, 0, 93, 94, 5, 82, 0, 5, 75, 0, 0, 91, 93, 5, 69, 0, 0, 92, 76, 1, 0, 0, 0, 92, 78, 1, 0, 0,
0, 94, 95, 5, 69, 0, 0, 95, 96, 5, 80, 0, 0, 96, 12, 1, 0, 0, 0, 97, 98, 0, 92, 80, 1, 0, 0, 0, 92, 82, 1, 0, 0, 0, 92, 84, 1, 0, 0, 0, 92, 86,
5, 73, 0, 0, 98, 99, 5, 78, 0, 0, 99, 14, 1, 0, 0, 0, 100, 101, 5, 65, 1, 0, 0, 0, 92, 88, 1, 0, 0, 0, 93, 8, 1, 0, 0, 0, 94, 95, 5, 85, 0, 0,
0, 0, 101, 102, 5, 83, 0, 0, 102, 16, 1, 0, 0, 0, 103, 104, 5, 67, 0, 0, 95, 96, 5, 78, 0, 0, 96, 97, 5, 73, 0, 0, 97, 98, 5, 81, 0, 0, 98, 99,
104, 105, 5, 66, 0, 0, 105, 106, 5, 70, 0, 0, 106, 18, 1, 0, 0, 0, 107, 5, 85, 0, 0, 99, 100, 5, 69, 0, 0, 100, 10, 1, 0, 0, 0, 101, 102, 5, 82,
108, 5, 83, 0, 0, 108, 109, 5, 69, 0, 0, 109, 110, 5, 76, 0, 0, 110, 111, 0, 0, 102, 103, 5, 69, 0, 0, 103, 104, 5, 80, 0, 0, 104, 12, 1, 0, 0, 0,
5, 69, 0, 0, 111, 112, 5, 67, 0, 0, 112, 113, 5, 84, 0, 0, 113, 20, 1, 105, 106, 5, 69, 0, 0, 106, 107, 5, 67, 0, 0, 107, 14, 1, 0, 0, 0, 108,
0, 0, 0, 114, 115, 5, 70, 0, 0, 115, 116, 5, 82, 0, 0, 116, 117, 5, 79, 109, 5, 73, 0, 0, 109, 110, 5, 78, 0, 0, 110, 16, 1, 0, 0, 0, 111, 112,
0, 0, 117, 118, 5, 77, 0, 0, 118, 22, 1, 0, 0, 0, 119, 120, 5, 70, 0, 0, 5, 65, 0, 0, 112, 113, 5, 83, 0, 0, 113, 18, 1, 0, 0, 0, 114, 115, 5, 67,
120, 121, 5, 73, 0, 0, 121, 122, 5, 76, 0, 0, 122, 123, 5, 84, 0, 0, 123, 0, 0, 115, 116, 5, 66, 0, 0, 116, 117, 5, 70, 0, 0, 117, 20, 1, 0, 0, 0,
124, 5, 69, 0, 0, 124, 125, 5, 82, 0, 0, 125, 24, 1, 0, 0, 0, 126, 127, 118, 119, 5, 83, 0, 0, 119, 120, 5, 69, 0, 0, 120, 121, 5, 76, 0, 0, 121,
5, 42, 0, 0, 127, 26, 1, 0, 0, 0, 128, 129, 5, 83, 0, 0, 129, 130, 5, 65, 122, 5, 69, 0, 0, 122, 123, 5, 67, 0, 0, 123, 124, 5, 84, 0, 0, 124, 22,
0, 0, 130, 131, 5, 77, 0, 0, 131, 132, 5, 69, 0, 0, 132, 28, 1, 0, 0, 0, 1, 0, 0, 0, 125, 126, 5, 70, 0, 0, 126, 127, 5, 82, 0, 0, 127, 128, 5,
133, 134, 5, 68, 0, 0, 134, 135, 5, 73, 0, 0, 135, 136, 5, 83, 0, 0, 136, 79, 0, 0, 128, 129, 5, 77, 0, 0, 129, 24, 1, 0, 0, 0, 130, 131, 5, 70,
137, 5, 84, 0, 0, 137, 138, 5, 73, 0, 0, 138, 139, 5, 78, 0, 0, 139, 140, 0, 0, 131, 132, 5, 73, 0, 0, 132, 133, 5, 76, 0, 0, 133, 134, 5, 84, 0,
5, 67, 0, 0, 140, 141, 5, 84, 0, 0, 141, 30, 1, 0, 0, 0, 142, 143, 5, 40, 0, 134, 135, 5, 69, 0, 0, 135, 136, 5, 82, 0, 0, 136, 26, 1, 0, 0, 0, 137,
0, 0, 143, 32, 1, 0, 0, 0, 144, 145, 5, 41, 0, 0, 145, 34, 1, 0, 0, 0, 138, 5, 42, 0, 0, 138, 28, 1, 0, 0, 0, 139, 140, 5, 46, 0, 0, 140, 30,
146, 147, 5, 64, 0, 0, 147, 36, 1, 0, 0, 0, 148, 153, 3, 41, 20, 0, 149, 1, 0, 0, 0, 141, 142, 5, 83, 0, 0, 142, 143, 5, 65, 0, 0, 143, 144, 5,
152, 3, 39, 19, 0, 150, 152, 3, 41, 20, 0, 151, 149, 1, 0, 0, 0, 151, 150, 77, 0, 0, 144, 145, 5, 69, 0, 0, 145, 32, 1, 0, 0, 0, 146, 147, 5, 68,
1, 0, 0, 0, 152, 155, 1, 0, 0, 0, 153, 151, 1, 0, 0, 0, 153, 154, 1, 0, 0, 0, 147, 148, 5, 73, 0, 0, 148, 149, 5, 83, 0, 0, 149, 150, 5, 84, 0,
0, 0, 154, 38, 1, 0, 0, 0, 155, 153, 1, 0, 0, 0, 156, 157, 7, 0, 0, 0, 0, 150, 151, 5, 73, 0, 0, 151, 152, 5, 78, 0, 0, 152, 153, 5, 67, 0, 0,
157, 40, 1, 0, 0, 0, 158, 159, 7, 1, 0, 0, 159, 42, 1, 0, 0, 0, 160, 164, 153, 154, 5, 84, 0, 0, 154, 34, 1, 0, 0, 0, 155, 156, 5, 40, 0, 0, 156,
7, 2, 0, 0, 161, 163, 3, 39, 19, 0, 162, 161, 1, 0, 0, 0, 163, 166, 1, 36, 1, 0, 0, 0, 157, 158, 5, 41, 0, 0, 158, 38, 1, 0, 0, 0, 159, 160, 5,
0, 0, 0, 164, 162, 1, 0, 0, 0, 164, 165, 1, 0, 0, 0, 165, 44, 1, 0, 0, 64, 0, 0, 160, 40, 1, 0, 0, 0, 161, 166, 3, 45, 22, 0, 162, 165, 3, 43,
0, 166, 164, 1, 0, 0, 0, 167, 168, 5, 48, 0, 0, 168, 46, 1, 0, 0, 0, 169, 21, 0, 163, 165, 3, 45, 22, 0, 164, 162, 1, 0, 0, 0, 164, 163, 1, 0, 0,
174, 5, 34, 0, 0, 170, 173, 3, 49, 24, 0, 171, 173, 3, 57, 28, 0, 172, 0, 165, 168, 1, 0, 0, 0, 166, 164, 1, 0, 0, 0, 166, 167, 1, 0, 0, 0, 167,
170, 1, 0, 0, 0, 172, 171, 1, 0, 0, 0, 173, 176, 1, 0, 0, 0, 174, 172, 42, 1, 0, 0, 0, 168, 166, 1, 0, 0, 0, 169, 170, 7, 0, 0, 0, 170, 44, 1,
1, 0, 0, 0, 174, 175, 1, 0, 0, 0, 175, 177, 1, 0, 0, 0, 176, 174, 1, 0, 0, 0, 0, 171, 172, 7, 1, 0, 0, 172, 46, 1, 0, 0, 0, 173, 177, 7, 2, 0,
0, 0, 177, 188, 5, 34, 0, 0, 178, 183, 5, 39, 0, 0, 179, 182, 3, 49, 24, 0, 174, 176, 3, 43, 21, 0, 175, 174, 1, 0, 0, 0, 176, 179, 1, 0, 0, 0,
0, 180, 182, 3, 55, 27, 0, 181, 179, 1, 0, 0, 0, 181, 180, 1, 0, 0, 0, 177, 175, 1, 0, 0, 0, 177, 178, 1, 0, 0, 0, 178, 48, 1, 0, 0, 0, 179, 177,
182, 185, 1, 0, 0, 0, 183, 181, 1, 0, 0, 0, 183, 184, 1, 0, 0, 0, 184, 1, 0, 0, 0, 180, 181, 5, 48, 0, 0, 181, 50, 1, 0, 0, 0, 182, 187, 5, 34,
186, 1, 0, 0, 0, 185, 183, 1, 0, 0, 0, 186, 188, 5, 39, 0, 0, 187, 169, 0, 0, 183, 186, 3, 53, 26, 0, 184, 186, 3, 61, 30, 0, 185, 183, 1, 0, 0,
1, 0, 0, 0, 187, 178, 1, 0, 0, 0, 188, 48, 1, 0, 0, 0, 189, 192, 5, 92, 0, 185, 184, 1, 0, 0, 0, 186, 189, 1, 0, 0, 0, 187, 185, 1, 0, 0, 0, 187,
0, 0, 190, 193, 7, 3, 0, 0, 191, 193, 3, 51, 25, 0, 192, 190, 1, 0, 0, 188, 1, 0, 0, 0, 188, 190, 1, 0, 0, 0, 189, 187, 1, 0, 0, 0, 190, 201,
0, 192, 191, 1, 0, 0, 0, 193, 50, 1, 0, 0, 0, 194, 195, 5, 117, 0, 0, 195, 5, 34, 0, 0, 191, 196, 5, 39, 0, 0, 192, 195, 3, 53, 26, 0, 193, 195, 3,
196, 3, 53, 26, 0, 196, 197, 3, 53, 26, 0, 197, 198, 3, 53, 26, 0, 198, 59, 29, 0, 194, 192, 1, 0, 0, 0, 194, 193, 1, 0, 0, 0, 195, 198, 1, 0,
199, 3, 53, 26, 0, 199, 52, 1, 0, 0, 0, 200, 201, 7, 4, 0, 0, 201, 54, 0, 0, 196, 194, 1, 0, 0, 0, 196, 197, 1, 0, 0, 0, 197, 199, 1, 0, 0, 0,
1, 0, 0, 0, 202, 203, 8, 5, 0, 0, 203, 56, 1, 0, 0, 0, 204, 205, 8, 6, 198, 196, 1, 0, 0, 0, 199, 201, 5, 39, 0, 0, 200, 182, 1, 0, 0, 0, 200,
0, 0, 205, 58, 1, 0, 0, 0, 206, 208, 7, 7, 0, 0, 207, 206, 1, 0, 0, 0, 191, 1, 0, 0, 0, 201, 52, 1, 0, 0, 0, 202, 205, 5, 92, 0, 0, 203, 206,
208, 209, 1, 0, 0, 0, 209, 207, 1, 0, 0, 0, 209, 210, 1, 0, 0, 0, 210, 7, 3, 0, 0, 204, 206, 3, 55, 27, 0, 205, 203, 1, 0, 0, 0, 205, 204, 1,
211, 1, 0, 0, 0, 211, 212, 6, 29, 0, 0, 212, 60, 1, 0, 0, 0, 12, 0, 84, 0, 0, 0, 206, 54, 1, 0, 0, 0, 207, 208, 5, 117, 0, 0, 208, 209, 3, 57,
151, 153, 164, 172, 174, 181, 183, 187, 192, 209, 1, 6, 0, 0, 28, 0, 209, 210, 3, 57, 28, 0, 210, 211, 3, 57, 28, 0, 211, 212, 3, 57,
28, 0, 212, 56, 1, 0, 0, 0, 213, 214, 7, 4, 0, 0, 214, 58, 1, 0, 0, 0,
215, 216, 8, 5, 0, 0, 216, 60, 1, 0, 0, 0, 217, 218, 8, 6, 0, 0, 218, 62,
1, 0, 0, 0, 219, 221, 7, 7, 0, 0, 220, 219, 1, 0, 0, 0, 221, 222, 1, 0,
0, 0, 222, 220, 1, 0, 0, 0, 222, 223, 1, 0, 0, 0, 223, 224, 1, 0, 0, 0,
224, 225, 6, 31, 0, 0, 225, 64, 1, 0, 0, 0, 12, 0, 92, 164, 166, 177, 185,
187, 194, 196, 200, 205, 222, 1, 6, 0, 0,
} }
deserializer := antlr.NewATNDeserializer(nil) deserializer := antlr.NewATNDeserializer(nil)
staticData.atn = deserializer.Deserialize(staticData.serializedATN) staticData.atn = deserializer.Deserialize(staticData.serializedATN)
@ -202,21 +208,23 @@ const (
QueryLexerSIMPLE_OP = 4 QueryLexerSIMPLE_OP = 4
QueryLexerUNIQUE = 5 QueryLexerUNIQUE = 5
QueryLexerREP = 6 QueryLexerREP = 6
QueryLexerIN = 7 QueryLexerEC = 7
QueryLexerAS = 8 QueryLexerIN = 8
QueryLexerCBF = 9 QueryLexerAS = 9
QueryLexerSELECT = 10 QueryLexerCBF = 10
QueryLexerFROM = 11 QueryLexerSELECT = 11
QueryLexerFILTER = 12 QueryLexerFROM = 12
QueryLexerWILDCARD = 13 QueryLexerFILTER = 13
QueryLexerCLAUSE_SAME = 14 QueryLexerWILDCARD = 14
QueryLexerCLAUSE_DISTINCT = 15 QueryLexerDOT = 15
QueryLexerL_PAREN = 16 QueryLexerCLAUSE_SAME = 16
QueryLexerR_PAREN = 17 QueryLexerCLAUSE_DISTINCT = 17
QueryLexerAT = 18 QueryLexerL_PAREN = 18
QueryLexerIDENT = 19 QueryLexerR_PAREN = 19
QueryLexerNUMBER1 = 20 QueryLexerAT = 20
QueryLexerZERO = 21 QueryLexerIDENT = 21
QueryLexerSTRING = 22 QueryLexerNUMBER1 = 22
QueryLexerWS = 23 QueryLexerZERO = 23
QueryLexerSTRING = 24
QueryLexerWS = 25
) )

File diff suppressed because it is too large Load diff

View file

@ -1,4 +1,4 @@
// Code generated from Query.g4 by ANTLR 4.13.0. DO NOT EDIT. // Code generated from /repo/frostfs/sdk-go/netmap/parser/Query.g4 by ANTLR 4.13.0. DO NOT EDIT.
package parser // Query package parser // Query
@ -14,6 +14,9 @@ type QueryVisitor interface {
// Visit a parse tree produced by Query#selectFilterExpr. // Visit a parse tree produced by Query#selectFilterExpr.
VisitSelectFilterExpr(ctx *SelectFilterExprContext) interface{} VisitSelectFilterExpr(ctx *SelectFilterExprContext) interface{}
// Visit a parse tree produced by Query#ecStmt.
VisitEcStmt(ctx *EcStmtContext) interface{}
// Visit a parse tree produced by Query#repStmt. // Visit a parse tree produced by Query#repStmt.
VisitRepStmt(ctx *RepStmtContext) interface{} VisitRepStmt(ctx *RepStmtContext) interface{}

View file

@ -34,8 +34,17 @@ type PlacementPolicy struct {
func (p *PlacementPolicy) readFromV2(m netmap.PlacementPolicy, checkFieldPresence bool) error { func (p *PlacementPolicy) readFromV2(m netmap.PlacementPolicy, checkFieldPresence bool) error {
p.replicas = m.GetReplicas() p.replicas = m.GetReplicas()
if checkFieldPresence && len(p.replicas) == 0 { if checkFieldPresence {
return errors.New("missing replicas") if len(p.replicas) == 0 {
return errors.New("missing replicas")
}
if len(p.replicas) != 1 {
for i := range p.replicas {
if p.replicas[i].GetECDataCount() != 0 || p.replicas[i].GetECParityCount() != 0 {
return errors.New("erasure code group must be used exclusively")
}
}
}
} }
p.backupFactor = m.GetContainerBackupFactor() p.backupFactor = m.GetContainerBackupFactor()
@ -130,6 +139,14 @@ func (r *ReplicaDescriptor) SetNumberOfObjects(c uint32) {
r.m.SetCount(c) r.m.SetCount(c)
} }
func (r *ReplicaDescriptor) SetECDataCount(v uint32) {
r.m.SetECDataCount(v)
}
func (r *ReplicaDescriptor) SetECParityCount(v uint32) {
r.m.SetECParityCount(v)
}
// NumberOfObjects returns number set using SetNumberOfObjects. // NumberOfObjects returns number set using SetNumberOfObjects.
// //
// Zero ReplicaDescriptor has zero number of objects. // Zero ReplicaDescriptor has zero number of objects.
@ -137,6 +154,19 @@ func (r ReplicaDescriptor) NumberOfObjects() uint32 {
return r.m.GetCount() return r.m.GetCount()
} }
func (r ReplicaDescriptor) GetECDataCount() uint32 {
return r.m.GetECDataCount()
}
func (r ReplicaDescriptor) GetECParityCount() uint32 {
return r.m.GetECParityCount()
}
// TotalECPartCount returns total sum of ECDataCount and ECParityCount.
func (r ReplicaDescriptor) TotalECPartCount() uint32 {
return r.m.GetECDataCount() + r.m.GetECParityCount()
}
// SetSelectorName sets name of the related Selector. // SetSelectorName sets name of the related Selector.
// //
// Zero ReplicaDescriptor references to the root bucket's selector: it contains // Zero ReplicaDescriptor references to the root bucket's selector: it contains
@ -170,10 +200,19 @@ func (p PlacementPolicy) NumberOfReplicas() int {
// descriptor. Index MUST be in range [0; NumberOfReplicas()). // descriptor. Index MUST be in range [0; NumberOfReplicas()).
// //
// Zero PlacementPolicy has no replicas. // Zero PlacementPolicy has no replicas.
//
// Deprecated: Use PlacementPolicy.ReplicaDescriptor(int).NumberOfObjects() instead.
func (p PlacementPolicy) ReplicaNumberByIndex(i int) uint32 { func (p PlacementPolicy) ReplicaNumberByIndex(i int) uint32 {
return p.replicas[i].GetCount() return p.replicas[i].GetCount()
} }
// ReplicaDescriptor returns i-th replica descriptor. Index MUST be in range [0; NumberOfReplicas()).
func (p PlacementPolicy) ReplicaDescriptor(i int) ReplicaDescriptor {
return ReplicaDescriptor{
m: p.replicas[i],
}
}
// SetContainerBackupFactor sets container backup factor: it controls how deep // SetContainerBackupFactor sets container backup factor: it controls how deep
// FrostFS will search for nodes alternatives to include into container's nodes subset. // FrostFS will search for nodes alternatives to include into container's nodes subset.
// //
@ -285,6 +324,13 @@ func (x *Filter) setAttribute(key string, op netmap.Operation, val string) {
x.m.SetValue(val) x.m.SetValue(val)
} }
// Like applies the rule to accept only nodes with attribute like value.
//
// Method SHOULD NOT be called along with other similar methods.
func (x *Filter) Like(key, value string) {
x.setAttribute(key, netmap.LIKE, value)
}
// Equal applies the rule to accept only nodes with the same attribute value. // Equal applies the rule to accept only nodes with the same attribute value.
// //
// Method SHOULD NOT be called along with other similar methods. // Method SHOULD NOT be called along with other similar methods.
@ -393,10 +439,14 @@ func (p PlacementPolicy) WriteStringTo(w io.StringWriter) (err error) {
c := p.replicas[i].GetCount() c := p.replicas[i].GetCount()
s := p.replicas[i].GetSelector() s := p.replicas[i].GetSelector()
if s != "" { if c != 0 {
_, err = w.WriteString(fmt.Sprintf("%sREP %d IN %s", delim, c, s))
} else {
_, err = w.WriteString(fmt.Sprintf("%sREP %d", delim, c)) _, err = w.WriteString(fmt.Sprintf("%sREP %d", delim, c))
} else {
ecx, ecy := p.replicas[i].GetECDataCount(), p.replicas[i].GetECParityCount()
_, err = w.WriteString(fmt.Sprintf("%sEC %d.%d", delim, ecx, ecy))
}
if s != "" {
_, err = w.WriteString(fmt.Sprintf(" IN %s", s))
} }
if err != nil { if err != nil {
@ -630,6 +680,8 @@ var (
"make sure to pair REP and SELECT clauses: \"REP .. IN X\" + \"SELECT ... AS X\"") "make sure to pair REP and SELECT clauses: \"REP .. IN X\" + \"SELECT ... AS X\"")
// errRedundantSelector is returned for errors found by filters policy validator. // errRedundantSelector is returned for errors found by filters policy validator.
errRedundantFilter = errors.New("policy: found redundant filter") errRedundantFilter = errors.New("policy: found redundant filter")
// errECFewSelectors is returned when EC keyword is used without UNIQUE keyword.
errECFewSelectors = errors.New("policy: too few nodes to select")
) )
type policyVisitor struct { type policyVisitor struct {
@ -657,11 +709,18 @@ func (p *policyVisitor) VisitPolicy(ctx *parser.PolicyContext) any {
pl.unique = ctx.UNIQUE() != nil pl.unique = ctx.UNIQUE() != nil
repStmts := ctx.AllRepStmt() stmts := ctx.GetChildren()
pl.replicas = make([]netmap.Replica, 0, len(repStmts)) for _, r := range stmts {
var res *netmap.Replica
for _, r := range repStmts { var ok bool
res, ok := r.Accept(p).(*netmap.Replica) switch r := r.(type) {
case parser.IRepStmtContext:
res, ok = r.Accept(p).(*netmap.Replica)
case parser.IEcStmtContext:
res, ok = r.Accept(p).(*netmap.Replica)
default:
continue
}
if !ok { if !ok {
return nil return nil
} }
@ -758,6 +817,28 @@ func (p *policyVisitor) VisitRepStmt(ctx *parser.RepStmtContext) any {
return rs return rs
} }
// VisitRepStmt implements parser.QueryVisitor interface.
func (p *policyVisitor) VisitEcStmt(ctx *parser.EcStmtContext) any {
dataCount, err := strconv.ParseUint(ctx.GetData().GetText(), 10, 32)
if err != nil {
return p.reportError(errInvalidNumber)
}
parityCount, err := strconv.ParseUint(ctx.GetParity().GetText(), 10, 32)
if err != nil {
return p.reportError(errInvalidNumber)
}
rs := new(netmap.Replica)
rs.SetECDataCount(uint32(dataCount))
rs.SetECParityCount(uint32(parityCount))
if sel := ctx.GetSelector(); sel != nil {
rs.SetSelector(sel.GetText())
}
return rs
}
// VisitSelectStmt implements parser.QueryVisitor interface. // VisitSelectStmt implements parser.QueryVisitor interface.
func (p *policyVisitor) VisitSelectStmt(ctx *parser.SelectStmtContext) any { func (p *policyVisitor) VisitSelectStmt(ctx *parser.SelectStmtContext) any {
res, err := strconv.ParseUint(ctx.GetCount().GetText(), 10, 32) res, err := strconv.ParseUint(ctx.GetCount().GetText(), 10, 32)
@ -910,6 +991,14 @@ func validatePolicy(p PlacementPolicy) error {
if seenSelectors[selName] == nil { if seenSelectors[selName] == nil {
return fmt.Errorf("%w: '%s'", errUnknownSelector, selName) return fmt.Errorf("%w: '%s'", errUnknownSelector, selName)
} }
dataCount := p.replicas[i].GetECDataCount()
parityCount := p.replicas[i].GetECParityCount()
if dataCount != 0 || parityCount != 0 {
if c := seenSelectors[selName].GetCount(); c < dataCount+parityCount {
return fmt.Errorf("%w: %d < %d + %d", errECFewSelectors, c, dataCount, parityCount)
}
}
} }
} }

View file

@ -7,48 +7,50 @@ import (
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
func TestDecodeString(t *testing.T) { var validPlacementPolicy = []string{
testCases := []string{ `REP 2
`REP 2
CBF 2 CBF 2
SELECT 2 FROM *`, SELECT 2 FROM *`,
`REP 1 IN X `REP 1 IN X
CBF 1 CBF 1
SELECT 2 IN SAME Location FROM * AS X`, SELECT 2 IN SAME Location FROM * AS X`,
`REP 1 IN X `REP 1 IN X
REP 2 IN Y REP 2 IN Y
CBF 1 CBF 1
SELECT 2 FROM * AS X SELECT 2 FROM * AS X
SELECT 3 FROM * AS Y`, SELECT 3 FROM * AS Y`,
`REP 1 IN X `REP 1 IN X
SELECT 2 IN City FROM Good AS X SELECT 2 IN City FROM Good AS X
FILTER Country EQ RU AS FromRU FILTER Country EQ RU AS FromRU
FILTER Country EQ EN AS FromEN FILTER Country EQ EN AS FromEN
FILTER @FromRU AND @FromEN AND Rating GT 7 AS Good`, FILTER @FromRU AND @FromEN AND Rating GT 7 AS Good`,
`REP 7 IN SPB `REP 7 IN SPB
SELECT 1 IN City FROM SPBSSD AS SPB SELECT 1 IN City FROM SPBSSD AS SPB
FILTER City EQ SPB AND SSD EQ true OR City EQ SPB AND Rating GE 5 AS SPBSSD`, FILTER City EQ SPB AND SSD EQ true OR City EQ SPB AND Rating GE 5 AS SPBSSD`,
`REP 7 IN SPB `REP 7 IN SPB
SELECT 1 IN City FROM SPBSSD AS SPB SELECT 1 IN City FROM SPBSSD AS SPB
FILTER NOT (NOT (City EQ SPB) AND SSD EQ true OR City EQ SPB AND Rating GE 5) AS SPBSSD`, FILTER NOT (NOT (City EQ SPB) AND SSD EQ true OR City EQ SPB AND Rating GE 5) AS SPBSSD`,
`REP 1 IN FNODE `REP 1 IN FNODE
CBF 1 CBF 1
SELECT 1 FROM F AS FNODE SELECT 1 FROM F AS FNODE
FILTER Node EQ '10.78.8.11' AS F`, FILTER Node EQ '10.78.8.11' AS F`,
`UNIQUE `UNIQUE
REP 1 REP 1
REP 1`, REP 1`,
} `EC 1.2 IN X
SELECT 3 IN City FROM * AS X`,
}
func TestDecodeString(t *testing.T) {
var p PlacementPolicy var p PlacementPolicy
for _, testCase := range testCases { for _, testCase := range validPlacementPolicy {
require.NoError(t, p.DecodeString(testCase), "unable parse %s", testCase) require.NoError(t, p.DecodeString(testCase), "unable parse %s", testCase)
var b strings.Builder var b strings.Builder
require.NoError(t, p.WriteStringTo(&b)) require.NoError(t, p.WriteStringTo(&b))

14
netmap/policy_fuzz.go Normal file
View file

@ -0,0 +1,14 @@
//go:build gofuzz
// +build gofuzz
package netmap
func DoFuzzPlacementPolicyDecode(data []byte) int {
p := string(data)
if p == "" {
return 0
}
var pp PlacementPolicy
_ = pp.DecodeString(p)
return 1
}

View file

@ -0,0 +1,17 @@
//go:build gofuzz
// +build gofuzz
package netmap
import (
"testing"
)
func FuzzDecodeString(f *testing.F) {
for _, pp := range validPlacementPolicy {
f.Add([]byte(pp))
}
f.Fuzz(func(t *testing.T, data []byte) {
DoFuzzPlacementPolicyDecode(data)
})
}

View file

@ -122,6 +122,11 @@ func TestDecodeSelectFilterExpr(t *testing.T) {
FILTER Color EQ 'Red' AS RedNode FILTER Color EQ 'Red' AS RedNode
FILTER @RedNode AND Shape EQ 'Cirle' AS RedCircleNode FILTER @RedNode AND Shape EQ 'Cirle' AS RedCircleNode
`, `,
`
CBF 1
SELECT 1 FROM R
FILTER Color LIKE 'R' AS R
`,
} { } {
_, err := DecodeSelectFilterString(s) _, err := DecodeSelectFilterString(s)
require.NoError(t, err) require.NoError(t, err)

View file

@ -1,8 +1,9 @@
package netmap package netmap
import ( import (
"cmp"
"fmt" "fmt"
"sort" "slices"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
"git.frostfs.info/TrueCloudLab/hrw" "git.frostfs.info/TrueCloudLab/hrw"
@ -70,12 +71,12 @@ func (c *context) getSelection(s netmap.Selector) ([]nodes, error) {
// we also need to have deterministic input to HRW sorting routine. // we also need to have deterministic input to HRW sorting routine.
if len(c.hrwSeed) == 0 { if len(c.hrwSeed) == 0 {
if s.GetAttribute() == "" { if s.GetAttribute() == "" {
sort.Slice(buckets, func(i, j int) bool { slices.SortFunc(buckets, func(b1, b2 nodeAttrPair) int {
return less(buckets[i].nodes[0], buckets[j].nodes[0]) return cmp.Compare(b1.nodes[0].Hash(), b2.nodes[0].Hash())
}) })
} else { } else {
sort.Slice(buckets, func(i, j int) bool { slices.SortFunc(buckets, func(b1, b2 nodeAttrPair) int {
return buckets[i].attr < buckets[j].attr return cmp.Compare(b1.attr, b2.attr)
}) })
} }
} }
@ -103,8 +104,10 @@ func (c *context) getSelection(s netmap.Selector) ([]nodes, error) {
if len(c.hrwSeed) != 0 { if len(c.hrwSeed) != 0 {
weights := make([]float64, len(res)) weights := make([]float64, len(res))
a := new(meanIQRAgg)
for i := range res { for i := range res {
weights[i] = calcBucketWeight(res[i], newMeanIQRAgg(), c.weightFunc) a.clear()
weights[i] = calcBucketWeight(res[i], a, c.weightFunc)
} }
hrw.SortHasherSliceByWeightValue(res, weights, c.hrwSeedHash) hrw.SortHasherSliceByWeightValue(res, weights, c.hrwSeedHash)
@ -168,8 +171,10 @@ func (c *context) getSelectionBase(s netmap.Selector) []nodeAttrPair {
} }
if len(c.hrwSeed) != 0 { if len(c.hrwSeed) != 0 {
var ws []float64
for i := range result { for i := range result {
hrw.SortHasherSliceByWeightValue(result[i].nodes, result[i].nodes.weights(c.weightFunc), c.hrwSeedHash) ws = result[i].nodes.appendWeightsTo(c.weightFunc, ws[:0])
hrw.SortHasherSliceByWeightValue(result[i].nodes, ws, c.hrwSeedHash)
} }
} }

View file

@ -1,12 +1,14 @@
package netmap package netmap
import ( import (
"cmp"
"crypto/rand" "crypto/rand"
"encoding/binary" "encoding/binary"
"fmt" "fmt"
mrand "math/rand" mrand "math/rand"
"sort" "slices"
"strconv" "strconv"
"strings"
"testing" "testing"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
@ -36,7 +38,7 @@ func BenchmarkHRWSort(b *testing.B) {
b.Run("sort by index, no weight", func(b *testing.B) { b.Run("sort by index, no weight", func(b *testing.B) {
realNodes := make([]nodes, netmapSize) realNodes := make([]nodes, netmapSize)
b.ResetTimer() b.ResetTimer()
for i := 0; i < b.N; i++ { for range b.N {
b.StopTimer() b.StopTimer()
copy(realNodes, vectors) copy(realNodes, vectors)
b.StartTimer() b.StartTimer()
@ -47,7 +49,7 @@ func BenchmarkHRWSort(b *testing.B) {
b.Run("sort by value, no weight", func(b *testing.B) { b.Run("sort by value, no weight", func(b *testing.B) {
realNodes := make([]nodes, netmapSize) realNodes := make([]nodes, netmapSize)
b.ResetTimer() b.ResetTimer()
for i := 0; i < b.N; i++ { for range b.N {
b.StopTimer() b.StopTimer()
copy(realNodes, vectors) copy(realNodes, vectors)
b.StartTimer() b.StartTimer()
@ -58,7 +60,7 @@ func BenchmarkHRWSort(b *testing.B) {
b.Run("only sort by index", func(b *testing.B) { b.Run("only sort by index", func(b *testing.B) {
realNodes := make([]nodes, netmapSize) realNodes := make([]nodes, netmapSize)
b.ResetTimer() b.ResetTimer()
for i := 0; i < b.N; i++ { for range b.N {
b.StopTimer() b.StopTimer()
copy(realNodes, vectors) copy(realNodes, vectors)
b.StartTimer() b.StartTimer()
@ -69,7 +71,7 @@ func BenchmarkHRWSort(b *testing.B) {
b.Run("sort by value", func(b *testing.B) { b.Run("sort by value", func(b *testing.B) {
realNodes := make([]nodes, netmapSize) realNodes := make([]nodes, netmapSize)
b.ResetTimer() b.ResetTimer()
for i := 0; i < b.N; i++ { for range b.N {
b.StopTimer() b.StopTimer()
copy(realNodes, vectors) copy(realNodes, vectors)
b.StartTimer() b.StartTimer()
@ -80,13 +82,13 @@ func BenchmarkHRWSort(b *testing.B) {
b.Run("sort by ID, then by index (deterministic)", func(b *testing.B) { b.Run("sort by ID, then by index (deterministic)", func(b *testing.B) {
realNodes := make([]nodes, netmapSize) realNodes := make([]nodes, netmapSize)
b.ResetTimer() b.ResetTimer()
for i := 0; i < b.N; i++ { for range b.N {
b.StopTimer() b.StopTimer()
copy(realNodes, vectors) copy(realNodes, vectors)
b.StartTimer() b.StartTimer()
sort.Slice(vectors, func(i, j int) bool { slices.SortFunc(vectors, func(vi, vj nodes) int {
return less(vectors[i][0], vectors[j][0]) return cmp.Compare(vi[0].Hash(), vj[0].Hash())
}) })
hrw.SortSliceByWeightIndex(realNodes, weights, pivot) hrw.SortSliceByWeightIndex(realNodes, weights, pivot)
} }
@ -132,7 +134,7 @@ func BenchmarkPolicyHRWType(b *testing.B) {
nm.SetNodes(nodes) nm.SetNodes(nodes)
b.ResetTimer() b.ResetTimer()
for i := 0; i < b.N; i++ { for range b.N {
_, err := nm.ContainerNodes(p, []byte{1}) _, err := nm.ContainerNodes(p, []byte{1})
if err != nil { if err != nil {
b.Fatal() b.Fatal()
@ -193,7 +195,7 @@ func TestPlacementPolicy_DeterministicOrder(t *testing.T) {
} }
a, b := getIndices(t) a, b := getIndices(t)
for i := 0; i < 10; i++ { for range 10 {
x, y := getIndices(t) x, y := getIndices(t)
require.Equal(t, a, x) require.Equal(t, a, x)
require.Equal(t, b, y) require.Equal(t, b, y)
@ -250,6 +252,92 @@ func TestPlacementPolicy_ProcessSelectors(t *testing.T) {
} }
} }
func TestPlacementPolicy_Like(t *testing.T) {
nodes := []NodeInfo{
nodeInfoFromAttributes("Country", "Russia"),
nodeInfoFromAttributes("Country", "Germany"),
nodeInfoFromAttributes("Country", "Belarus"),
}
var nm NetMap
nm.SetNodes(nodes)
t.Run("LIKE all", func(t *testing.T) {
ssNamed := []Selector{newSelector("X", "Country", 4, "FromRU", (*Selector).SelectDistinct)}
fsNamed := []Filter{newFilter("FromRU", "Country", "*", netmap.LIKE)}
rsNamed := []ReplicaDescriptor{newReplica(4, "X")}
pNamed := newPlacementPolicy(0, rsNamed, ssNamed, fsNamed)
n, err := nm.ContainerNodes(pNamed, []byte{1})
require.NoError(t, err)
require.Equal(t, 3, len(n[0]))
for _, n := range n[0] {
require.True(t, strings.Contains("GermanyRussiaBelarus", n.Attribute("Country")))
}
})
t.Run("LIKE no wildcard", func(t *testing.T) {
ssNamed := []Selector{newSelector("X", "Country", 1, "FromRU", (*Selector).SelectDistinct)}
fsNamed := []Filter{newFilter("FromRU", "Country", "Russia", netmap.LIKE)}
rsNamed := []ReplicaDescriptor{newReplica(1, "X")}
pNamed := newPlacementPolicy(0, rsNamed, ssNamed, fsNamed)
n, err := nm.ContainerNodes(pNamed, []byte{1})
require.NoError(t, err)
require.Equal(t, 1, len(n[0]))
for _, n := range n[0] {
require.True(t, n.Attribute("Country") == "Russia")
}
})
t.Run("LIKE prefix", func(t *testing.T) {
ssNamed := []Selector{newSelector("X", "Country", 1, "FromRU", (*Selector).SelectDistinct)}
fsNamed := []Filter{newFilter("FromRU", "Country", "Ge*", netmap.LIKE)}
rsNamed := []ReplicaDescriptor{newReplica(1, "X")}
pNamed := newPlacementPolicy(0, rsNamed, ssNamed, fsNamed)
n, err := nm.ContainerNodes(pNamed, []byte{1})
require.NoError(t, err)
require.Equal(t, 1, len(n[0]))
for _, n := range n[0] {
require.True(t, n.Attribute("Country") == "Germany")
}
})
t.Run("LIKE suffix", func(t *testing.T) {
ssNamed := []Selector{newSelector("X", "Country", 1, "FromRU", (*Selector).SelectDistinct)}
fsNamed := []Filter{newFilter("FromRU", "Country", "*sia", netmap.LIKE)}
rsNamed := []ReplicaDescriptor{newReplica(1, "X")}
pNamed := newPlacementPolicy(0, rsNamed, ssNamed, fsNamed)
n, err := nm.ContainerNodes(pNamed, []byte{1})
require.NoError(t, err)
require.Equal(t, 1, len(n[0]))
for _, n := range n[0] {
require.True(t, n.Attribute("Country") == "Russia")
}
})
t.Run("LIKE middle", func(t *testing.T) {
ssNamed := []Selector{newSelector("X", "Country", 2, "FromRU", (*Selector).SelectDistinct)}
fsNamed := []Filter{newFilter("FromRU", "Country", "*us*", netmap.LIKE)}
rsNamed := []ReplicaDescriptor{newReplica(2, "X")}
pNamed := newPlacementPolicy(0, rsNamed, ssNamed, fsNamed)
n, err := nm.ContainerNodes(pNamed, []byte{1})
require.NoError(t, err)
require.Equal(t, 2, len(n[0]))
for _, n := range n[0] {
require.True(t, strings.Contains("RussiaBelarus", n.Attribute("Country")))
}
})
}
func TestPlacementPolicy_Unique(t *testing.T) { func TestPlacementPolicy_Unique(t *testing.T) {
p := newPlacementPolicy(2, p := newPlacementPolicy(2,
[]ReplicaDescriptor{ []ReplicaDescriptor{
@ -264,7 +352,7 @@ func TestPlacementPolicy_Unique(t *testing.T) {
var nodes []NodeInfo var nodes []NodeInfo
for i, city := range []string{"Moscow", "Berlin", "Shenzhen"} { for i, city := range []string{"Moscow", "Berlin", "Shenzhen"} {
for j := 0; j < 3; j++ { for j := range 3 {
node := nodeInfoFromAttributes("City", city) node := nodeInfoFromAttributes("City", city)
node.SetPublicKey(binary.BigEndian.AppendUint16(nil, uint16(i*4+j))) node.SetPublicKey(binary.BigEndian.AppendUint16(nil, uint16(i*4+j)))
nodes = append(nodes, node) nodes = append(nodes, node)
@ -278,7 +366,7 @@ func TestPlacementPolicy_Unique(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
for i, vi := range v { for i, vi := range v {
for _, ni := range vi { for _, ni := range vi {
for j := 0; j < i; j++ { for j := range i {
for _, nj := range v[j] { for _, nj := range v[j] {
require.NotEqual(t, ni.hash, nj.hash) require.NotEqual(t, ni.hash, nj.hash)
} }
@ -367,7 +455,7 @@ func TestPlacementPolicy_MultiREP(t *testing.T) {
for _, additional := range []int{0, 1, 2} { for _, additional := range []int{0, 1, 2} {
t.Run(fmt.Sprintf("unique=%t, additional=%d", unique, additional), func(t *testing.T) { t.Run(fmt.Sprintf("unique=%t, additional=%d", unique, additional), func(t *testing.T) {
rs := []ReplicaDescriptor{newReplica(1, "SameRU")} rs := []ReplicaDescriptor{newReplica(1, "SameRU")}
for i := 0; i < additional; i++ { for range additional {
rs = append(rs, newReplica(1, "")) rs = append(rs, newReplica(1, ""))
} }

View file

@ -130,7 +130,7 @@ func BenchmarkPlacementPolicyInteropability(b *testing.B) {
b.Run(name, func(b *testing.B) { b.Run(name, func(b *testing.B) {
b.ReportAllocs() b.ReportAllocs()
b.ResetTimer() b.ResetTimer()
for i := 0; i < b.N; i++ { for range b.N {
b.StartTimer() b.StartTimer()
v, err := nm.ContainerNodes(tt.Policy, tt.Pivot) v, err := nm.ContainerNodes(tt.Policy, tt.Pivot)
b.StopTimer() b.StopTimer()
@ -173,7 +173,7 @@ func BenchmarkManySelects(b *testing.B) {
b.ResetTimer() b.ResetTimer()
b.ReportAllocs() b.ReportAllocs()
for i := 0; i < b.N; i++ { for range b.N {
_, err = nm.ContainerNodes(tt.Policy, tt.Pivot) _, err = nm.ContainerNodes(tt.Policy, tt.Pivot)
if err != nil { if err != nil {
b.FailNow() b.FailNow()

View file

@ -44,15 +44,16 @@ func convertNFilters(temp []tempFilter) []netmap.Filter {
} }
var stringToOperationMap = map[string]netmap.Operation{ var stringToOperationMap = map[string]netmap.Operation{
"EQ": netmap.EQ, "EQ": netmap.EQ,
"NE": netmap.NE, "NE": netmap.NE,
"GT": netmap.GT, "GT": netmap.GT,
"GE": netmap.GE, "GE": netmap.GE,
"LT": netmap.LT, "LT": netmap.LT,
"LE": netmap.LE, "LE": netmap.LE,
"OR": netmap.OR, "OR": netmap.OR,
"AND": netmap.AND, "AND": netmap.AND,
"NOT": netmap.NOT, "NOT": netmap.NOT,
"LIKE": netmap.LIKE,
} }
func convertStringToOperation(opStr string) netmap.Operation { func convertStringToOperation(opStr string) netmap.Operation {

69
object/ecinfo.go Normal file
View file

@ -0,0 +1,69 @@
package object
import (
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
)
type ECChunk object.ECChunk
func (c *ECChunk) SetID(id oid.ID) {
objV2 := new(refs.ObjectID)
id.WriteToV2(objV2)
c.ID = *objV2
}
// ToV2 converts ECChunk to v2 ECChunk message.
//
// Nil ECChunk converts to nil.
func (c *ECChunk) ToV2() *object.ECChunk {
return (*object.ECChunk)(c)
}
func NewECChunkFromV2(v2 *object.ECChunk) *ECChunk {
return (*ECChunk)(v2)
}
type ECInfo object.ECInfo
// NewECInfoFromV2 wraps v2 ECInfo message to ECInfo.
//
// Nil object.ECInfo converts to nil.
func NewECInfoFromV2(v2 *object.ECInfo) *ECInfo {
return (*ECInfo)(v2)
}
// NewECInfo creates and initializes blank ECInfo.
func NewECInfo() *ECInfo {
return NewECInfoFromV2(new(object.ECInfo))
}
// ToV2 converts ECInfo to v2 ECInfo message.
//
// Nil ECInfo converts to nil.
func (s *ECInfo) ToV2() *object.ECInfo {
return (*object.ECInfo)(s)
}
func (s *ECInfo) Marshal() ([]byte, error) {
return (*object.ECInfo)(s).StableMarshal(nil), nil
}
func (s *ECInfo) Unmarshal(data []byte) error {
return (*object.ECInfo)(s).Unmarshal(data)
}
// MarshalJSON implements json.Marshaler.
func (s *ECInfo) MarshalJSON() ([]byte, error) {
return (*object.ECInfo)(s).MarshalJSON()
}
// UnmarshalJSON implements json.Unmarshaler.
func (s *ECInfo) UnmarshalJSON(data []byte) error {
return (*object.ECInfo)(s).UnmarshalJSON(data)
}
func (s *ECInfo) AddChunk(chunk ECChunk) {
s.Chunks = append(s.Chunks, *chunk.ToV2())
}

193
object/erasure_code.go Normal file
View file

@ -0,0 +1,193 @@
package object
import (
"errors"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
refs "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
)
// ECHeader represents erasure coding header.
type ECHeader struct {
parent oid.ID
parentSplitID *SplitID
parentSplitParentID *oid.ID
parentAttributes []Attribute
index uint32
total uint32
header []byte
headerLength uint32
}
type ECParentInfo struct {
// EC-parent's ID.
ID oid.ID
// EC-parent's split ID if the parent is a part of Split itself.
SplitID *SplitID
// EC-parent's parent split ID if the parent is a part of Split itself.
SplitParentID *oid.ID
// EC-parent's attributes.
Attributes []Attribute
}
// NewECHeader constructs new erasure coding header.
func NewECHeader(ecParentInfo ECParentInfo, index, total uint32, header []byte, headerLength uint32) *ECHeader {
return &ECHeader{
parent: ecParentInfo.ID,
parentSplitID: ecParentInfo.SplitID,
parentSplitParentID: ecParentInfo.SplitParentID,
parentAttributes: ecParentInfo.Attributes,
index: index,
total: total,
header: header,
headerLength: headerLength,
}
}
// WriteToV2 converts SDK structure to v2-api one.
func (e *ECHeader) WriteToV2(h *object.ECHeader) {
var parent refs.ObjectID
e.parent.WriteToV2(&parent)
h.ParentSplitID = e.parentSplitID.ToV2()
if e.parentSplitParentID != nil {
parentSplitParentID := new(refs.ObjectID)
e.parentSplitParentID.WriteToV2(parentSplitParentID)
h.ParentSplitParentID = parentSplitParentID
}
h.Parent = &parent
attrs := make([]object.Attribute, len(e.parentAttributes))
for i := range e.parentAttributes {
attrs[i] = *e.parentAttributes[i].ToV2()
}
h.ParentAttributes = attrs
h.Index = e.index
h.Total = e.total
h.Header = e.header
h.HeaderLength = e.headerLength
}
// ReadFromV2 converts v2-api structure to SDK one.
func (e *ECHeader) ReadFromV2(h *object.ECHeader) error {
if h == nil {
return nil
}
if h.Parent == nil {
return errors.New("empty parent")
}
attrs := make([]Attribute, len(h.ParentAttributes))
for i := range h.ParentAttributes {
attrs[i] = *NewAttributeFromV2(&h.ParentAttributes[i])
}
e.parentAttributes = attrs
_ = e.parent.ReadFromV2(*h.Parent)
e.parentSplitID = NewSplitIDFromV2(h.ParentSplitID)
if h.ParentSplitParentID != nil {
if e.parentSplitParentID == nil {
e.parentSplitParentID = new(oid.ID)
}
_ = e.parentSplitParentID.ReadFromV2(*h.ParentSplitParentID)
}
e.index = h.Index
e.total = h.Total
e.header = h.Header
e.headerLength = h.HeaderLength
return nil
}
func (o *Object) ECHeader() *ECHeader {
ec := (*object.Object)(o).GetHeader().GetEC()
if ec == nil {
return nil
}
h := new(ECHeader)
_ = h.ReadFromV2(ec)
return h
}
func (o *Object) SetECHeader(ec *ECHeader) {
o.setHeaderField(func(h *object.Header) {
if ec == nil {
h.SetEC(nil)
return
}
v2 := new(object.ECHeader)
ec.WriteToV2(v2)
h.SetEC(v2)
})
}
func (e *ECHeader) Parent() oid.ID {
return e.parent
}
func (e *ECHeader) SetParent(id oid.ID) {
e.parent = id
}
func (e *ECHeader) ParentSplitID() *SplitID {
return e.parentSplitID
}
func (e *ECHeader) SetParentSplitID(parentSplitID *SplitID) {
e.parentSplitID = parentSplitID
}
func (e *ECHeader) ParentSplitParentID() *oid.ID {
return e.parentSplitParentID
}
func (e *ECHeader) SetParentSplitParentID(parentSplitParentID *oid.ID) {
e.parentSplitParentID = parentSplitParentID
}
func (e *ECHeader) ParentAttributes() []Attribute {
return e.parentAttributes
}
func (e *ECHeader) SetParentAttributes(attrs []Attribute) {
e.parentAttributes = attrs
}
func (e *ECHeader) Index() uint32 {
return e.index
}
func (e *ECHeader) SetIndex(i uint32) {
e.index = i
}
func (e *ECHeader) Total() uint32 {
return e.total
}
func (e *ECHeader) SetTotal(i uint32) {
e.total = i
}
func (e *ECHeader) Header() []byte {
return e.header
}
func (e *ECHeader) SetHeader(header []byte) {
e.header = header
}
func (e *ECHeader) HeaderLength() uint32 {
return e.headerLength
}
func (e *ECHeader) SetHeaderLength(l uint32) {
e.headerLength = l
}

View file

@ -0,0 +1,87 @@
package erasurecode
import (
"errors"
objectSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
"github.com/klauspost/reedsolomon"
)
var (
// ErrMalformedSlice is returned when a slice of EC chunks is inconsistent.
ErrMalformedSlice = errors.New("inconsistent EC headers")
// ErrInvShardNum is returned from NewConstructor when the number of shards is invalid.
ErrInvShardNum = reedsolomon.ErrInvShardNum
// ErrMaxShardNum is returned from NewConstructor when the number of shards is too big.
ErrMaxShardNum = reedsolomon.ErrMaxShardNum
)
// MaxShardCount is the maximum number of shards.
const MaxShardCount = 256
// Constructor is a wrapper around encoder allowing to reconstruct objects.
// It's methods are not thread-safe.
type Constructor struct {
enc reedsolomon.Encoder
headerLength uint32
payloadShards [][]byte
headerShards [][]byte
}
// NewConstructor returns new constructor instance.
func NewConstructor(dataCount int, parityCount int) (*Constructor, error) {
// The library supports up to 65536 shards with some restrictions.
// This can easily result in OOM or panic, thus SDK declares it's own restriction.
if dataCount+parityCount > MaxShardCount {
return nil, ErrMaxShardNum
}
enc, err := reedsolomon.New(dataCount, parityCount)
if err != nil {
return nil, err
}
return &Constructor{enc: enc}, nil
}
// clear clears internal state of the constructor, so it can be reused.
func (c *Constructor) clear() {
c.headerLength = 0
c.payloadShards = nil
c.headerShards = nil
}
func (c *Constructor) fillHeader(parts []*objectSDK.Object) error {
shards := make([][]byte, len(parts))
headerLength := 0
for i := range parts {
if parts[i] == nil {
continue
}
var err error
headerLength, err = validatePart(parts, i, headerLength)
if err != nil {
return err
}
shards[i] = parts[i].GetECHeader().Header()
}
c.headerLength = uint32(headerLength)
c.headerShards = shards
return nil
}
// fillPayload fills the payload shards.
// Currently there is no case when it can be called without reconstructing header,
// thus fillHeader() must be called before and this function performs no validation.
func (c *Constructor) fillPayload(parts []*objectSDK.Object) {
shards := make([][]byte, len(parts))
for i := range parts {
if parts[i] == nil {
continue
}
shards[i] = parts[i].Payload()
}
c.payloadShards = shards
}

View file

@ -0,0 +1,31 @@
package erasurecode_test
import (
"testing"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/erasurecode"
"github.com/stretchr/testify/require"
)
func TestErasureConstruct(t *testing.T) {
t.Run("negative, no panic", func(t *testing.T) {
_, err := erasurecode.NewConstructor(-1, 2)
require.ErrorIs(t, err, erasurecode.ErrInvShardNum)
})
t.Run("negative, no panic", func(t *testing.T) {
_, err := erasurecode.NewConstructor(2, -1)
require.ErrorIs(t, err, erasurecode.ErrInvShardNum)
})
t.Run("zero parity", func(t *testing.T) {
_, err := erasurecode.NewConstructor(1, 0)
require.NoError(t, err)
})
t.Run("max shard num", func(t *testing.T) {
_, err := erasurecode.NewConstructor(erasurecode.MaxShardCount, 0)
require.NoError(t, err)
})
t.Run("max+1 shard num", func(t *testing.T) {
_, err := erasurecode.NewConstructor(erasurecode.MaxShardCount+1, 0)
require.ErrorIs(t, err, erasurecode.ErrMaxShardNum)
})
}

View file

@ -0,0 +1,146 @@
package erasurecode
import (
"bytes"
"crypto/ecdsa"
"fmt"
objectSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
"github.com/klauspost/reedsolomon"
)
// Reconstruct returns full object reconstructed from parts.
// All non-nil objects in parts must have EC header with the same `total` field equal to len(parts).
// The slice must contain at least one non nil object.
// Index of the objects in parts must be equal to it's index field in the EC header.
// The parts slice isn't changed and can be used concurrently for reading.
func (c *Constructor) Reconstruct(parts []*objectSDK.Object) (*objectSDK.Object, error) {
res, err := c.ReconstructHeader(parts)
if err != nil {
return nil, err
}
c.fillPayload(parts)
payload, err := reconstructExact(c.enc, int(res.PayloadSize()), c.payloadShards)
if err != nil {
return nil, fmt.Errorf("%w: %w", ErrMalformedSlice, err)
}
res.SetPayload(payload)
return res, nil
}
// ReconstructHeader returns object header reconstructed from parts.
// All non-nil objects in parts must have EC header with the same `total` field equal to len(parts).
// The slice must contain at least one non nil object.
// Index of the objects in parts must be equal to it's index field in the EC header.
// The parts slice isn't changed and can be used concurrently for reading.
func (c *Constructor) ReconstructHeader(parts []*objectSDK.Object) (*objectSDK.Object, error) {
c.clear()
if err := c.fillHeader(parts); err != nil {
return nil, err
}
obj, err := c.reconstructHeader()
if err != nil {
return nil, fmt.Errorf("%w: %w", ErrMalformedSlice, err)
}
return obj, nil
}
// ReconstructParts reconstructs specific EC parts without reconstructing full object.
// All non-nil objects in parts must have EC header with the same `total` field equal to len(parts).
// The slice must contain at least one non nil object.
// Index of the objects in parts must be equal to it's index field in the EC header.
// Those parts for which corresponding element in required is true must be nil and will be overwritten.
// Because partial reconstruction only makes sense for full objects, all parts must have non-empty payload.
// If key is not nil, all reconstructed parts are signed with this key.
func (c *Constructor) ReconstructParts(parts []*objectSDK.Object, required []bool, key *ecdsa.PrivateKey) error {
if len(required) != len(parts) {
return fmt.Errorf("len(parts) != len(required): %d != %d", len(parts), len(required))
}
c.clear()
if err := c.fillHeader(parts); err != nil {
return err
}
c.fillPayload(parts)
if err := c.enc.ReconstructSome(c.payloadShards, required); err != nil {
return fmt.Errorf("%w: %w", ErrMalformedSlice, err)
}
if err := c.enc.ReconstructSome(c.headerShards, required); err != nil {
return fmt.Errorf("%w: %w", ErrMalformedSlice, err)
}
nonNilPart := 0
for i := range parts {
if parts[i] != nil {
nonNilPart = i
break
}
}
ec := parts[nonNilPart].GetECHeader()
ecParentInfo := objectSDK.ECParentInfo{
ID: ec.Parent(),
SplitID: ec.ParentSplitID(),
SplitParentID: ec.ParentSplitParentID(),
Attributes: ec.ParentAttributes(),
}
total := ec.Total()
for i := range required {
if parts[i] != nil || !required[i] {
continue
}
part := objectSDK.New()
copyRequiredFields(part, parts[nonNilPart])
part.SetPayload(c.payloadShards[i])
part.SetPayloadSize(uint64(len(c.payloadShards[i])))
part.SetECHeader(objectSDK.NewECHeader(ecParentInfo, uint32(i), total, c.headerShards[i], c.headerLength))
if err := setIDWithSignature(part, key); err != nil {
return err
}
parts[i] = part
}
return nil
}
func (c *Constructor) reconstructHeader() (*objectSDK.Object, error) {
data, err := reconstructExact(c.enc, int(c.headerLength), c.headerShards)
if err != nil {
return nil, err
}
var obj objectSDK.Object
return &obj, obj.Unmarshal(data)
}
func reconstructExact(enc reedsolomon.Encoder, size int, shards [][]byte) ([]byte, error) {
if err := enc.ReconstructData(shards); err != nil {
return nil, err
}
// Technically, this error will be returned from enc.Join().
// However, allocating based on unvalidated user data is an easy attack vector.
// Preallocating seems to have enough benefits to justify a slight increase in code complexity.
maxSize := 0
for i := range shards {
maxSize += len(shards[i])
}
if size > maxSize {
return nil, reedsolomon.ErrShortData
}
buf := bytes.NewBuffer(make([]byte, 0, size))
if err := enc.Join(buf, shards, size); err != nil {
return nil, err
}
return buf.Bytes(), nil
}

View file

@ -0,0 +1,284 @@
package erasurecode_test
import (
"context"
"crypto/rand"
"math"
"testing"
cidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id/test"
objectSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/erasurecode"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/version"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/stretchr/testify/require"
)
func TestErasureCodeReconstruct(t *testing.T) {
const payloadSize = 99
const dataCount = 3
const parityCount = 2
// We would also like to test padding behaviour,
// so ensure padding is done.
require.NotZero(t, payloadSize%(dataCount+parityCount))
pk, err := keys.NewPrivateKey()
require.NoError(t, err)
original := newObject(t, payloadSize, pk)
c, err := erasurecode.NewConstructor(dataCount, parityCount)
require.NoError(t, err)
parts, err := c.Split(original, &pk.PrivateKey)
require.NoError(t, err)
t.Run("reconstruct header", func(t *testing.T) {
original := original.CutPayload()
parts := cloneSlice(parts)
for i := range parts {
parts[i] = parts[i].CutPayload()
}
t.Run("from data", func(t *testing.T) {
parts := cloneSlice(parts)
for i := dataCount; i < dataCount+parityCount; i++ {
parts[i] = nil
}
reconstructed, err := c.ReconstructHeader(parts)
require.NoError(t, err)
verifyReconstruction(t, original, reconstructed)
})
t.Run("from parity", func(t *testing.T) {
parts := cloneSlice(parts)
for i := range parityCount {
parts[i] = nil
}
reconstructed, err := c.ReconstructHeader(parts)
require.NoError(t, err)
verifyReconstruction(t, original, reconstructed)
t.Run("not enough shards", func(t *testing.T) {
parts[parityCount] = nil
_, err := c.ReconstructHeader(parts)
require.ErrorIs(t, err, erasurecode.ErrMalformedSlice)
})
})
t.Run("only nil parts", func(t *testing.T) {
parts := make([]*objectSDK.Object, len(parts))
_, err := c.ReconstructHeader(parts)
require.ErrorIs(t, err, erasurecode.ErrMalformedSlice)
})
t.Run("missing EC header", func(t *testing.T) {
parts := cloneSlice(parts)
parts[0] = deepCopy(t, parts[0])
parts[0].SetECHeader(nil)
_, err := c.ReconstructHeader(parts)
require.ErrorIs(t, err, erasurecode.ErrMalformedSlice)
})
t.Run("invalid index", func(t *testing.T) {
parts := cloneSlice(parts)
parts[0] = deepCopy(t, parts[0])
ec := parts[0].GetECHeader()
ec.SetIndex(1)
parts[0].SetECHeader(ec)
_, err := c.ReconstructHeader(parts)
require.ErrorIs(t, err, erasurecode.ErrMalformedSlice)
})
t.Run("invalid total", func(t *testing.T) {
parts := cloneSlice(parts)
parts[0] = deepCopy(t, parts[0])
ec := parts[0].GetECHeader()
ec.SetTotal(uint32(len(parts) + 1))
parts[0].SetECHeader(ec)
_, err := c.ReconstructHeader(parts)
require.ErrorIs(t, err, erasurecode.ErrMalformedSlice)
})
t.Run("inconsistent header length", func(t *testing.T) {
parts := cloneSlice(parts)
parts[0] = deepCopy(t, parts[0])
ec := parts[0].GetECHeader()
ec.SetHeaderLength(ec.HeaderLength() - 1)
parts[0].SetECHeader(ec)
_, err := c.ReconstructHeader(parts)
require.ErrorIs(t, err, erasurecode.ErrMalformedSlice)
})
t.Run("invalid header length", func(t *testing.T) {
parts := cloneSlice(parts)
for i := range parts {
parts[i] = deepCopy(t, parts[i])
ec := parts[0].GetECHeader()
ec.SetHeaderLength(math.MaxUint32)
parts[0].SetECHeader(ec)
}
_, err := c.ReconstructHeader(parts)
require.ErrorIs(t, err, erasurecode.ErrMalformedSlice)
})
})
t.Run("reconstruct data", func(t *testing.T) {
t.Run("from data", func(t *testing.T) {
parts := cloneSlice(parts)
for i := dataCount; i < dataCount+parityCount; i++ {
parts[i] = nil
}
reconstructed, err := c.Reconstruct(parts)
require.NoError(t, err)
verifyReconstruction(t, original, reconstructed)
})
t.Run("from parity", func(t *testing.T) {
parts := cloneSlice(parts)
for i := range parityCount {
parts[i] = nil
}
reconstructed, err := c.Reconstruct(parts)
require.NoError(t, err)
verifyReconstruction(t, original, reconstructed)
t.Run("not enough shards", func(t *testing.T) {
parts[parityCount] = nil
_, err := c.Reconstruct(parts)
require.ErrorIs(t, err, erasurecode.ErrMalformedSlice)
})
})
})
t.Run("reconstruct parts", func(t *testing.T) {
// We would like to also test that ReconstructParts doesn't perform
// excessive work, so ensure this test makes sense.
require.GreaterOrEqual(t, parityCount, 2)
t.Run("from data", func(t *testing.T) {
oldParts := parts
parts := cloneSlice(parts)
for i := dataCount; i < dataCount+parityCount; i++ {
parts[i] = nil
}
required := make([]bool, len(parts))
required[dataCount] = true
require.NoError(t, c.ReconstructParts(parts, required, nil))
old := deepCopy(t, oldParts[dataCount])
old.SetSignature(nil)
require.Equal(t, old, parts[dataCount])
for i := dataCount + 1; i < dataCount+parityCount; i++ {
require.Nil(t, parts[i])
}
})
t.Run("from parity", func(t *testing.T) {
oldParts := parts
parts := cloneSlice(parts)
for i := range parityCount {
parts[i] = nil
}
required := make([]bool, len(parts))
required[0] = true
require.NoError(t, c.ReconstructParts(parts, required, nil))
old := deepCopy(t, oldParts[0])
old.SetSignature(nil)
require.Equal(t, old, parts[0])
for i := 1; i < parityCount; i++ {
require.Nil(t, parts[i])
}
})
})
}
func newObject(t *testing.T, size uint64, pk *keys.PrivateKey) *objectSDK.Object {
// Use transformer to form object to avoid potential bugs with yet another helper object creation in tests.
tt := &testTarget{}
p := transformer.NewPayloadSizeLimiter(transformer.Params{
Key: &pk.PrivateKey,
NextTargetInit: func() transformer.ObjectWriter { return tt },
NetworkState: dummyEpochSource(123),
MaxSize: size + 1,
WithoutHomomorphicHash: true,
})
cnr := cidtest.ID()
ver := version.Current()
hdr := objectSDK.New()
hdr.SetContainerID(cnr)
hdr.SetType(objectSDK.TypeRegular)
hdr.SetVersion(&ver)
var owner user.ID
user.IDFromKey(&owner, pk.PrivateKey.PublicKey)
hdr.SetOwnerID(owner)
var attr objectSDK.Attribute
attr.SetKey("somekey")
attr.SetValue("somevalue")
hdr.SetAttributes(attr)
expectedPayload := make([]byte, size)
_, _ = rand.Read(expectedPayload)
writeObject(t, context.Background(), p, hdr, expectedPayload)
require.Len(t, tt.objects, 1)
return tt.objects[0]
}
func writeObject(t *testing.T, ctx context.Context, target transformer.ChunkedObjectWriter, header *objectSDK.Object, payload []byte) *transformer.AccessIdentifiers {
require.NoError(t, target.WriteHeader(ctx, header))
_, err := target.Write(ctx, payload)
require.NoError(t, err)
ids, err := target.Close(ctx)
require.NoError(t, err)
return ids
}
func verifyReconstruction(t *testing.T, original, reconstructed *objectSDK.Object) {
require.True(t, reconstructed.VerifyIDSignature())
reconstructed.ToV2().SetMarshalData(nil)
original.ToV2().SetMarshalData(nil)
require.Equal(t, original, reconstructed)
}
func deepCopy(t *testing.T, obj *objectSDK.Object) *objectSDK.Object {
data, err := obj.Marshal()
require.NoError(t, err)
res := objectSDK.New()
require.NoError(t, res.Unmarshal(data))
return res
}
func cloneSlice[T any](src []T) []T {
dst := make([]T, len(src))
copy(dst, src)
return dst
}
type dummyEpochSource uint64
func (s dummyEpochSource) CurrentEpoch() uint64 {
return uint64(s)
}
type testTarget struct {
objects []*objectSDK.Object
}
func (tt *testTarget) WriteObject(_ context.Context, o *objectSDK.Object) error {
tt.objects = append(tt.objects, o)
return nil // AccessIdentifiers should not be used.
}

View file

@ -0,0 +1,85 @@
package erasurecode
import (
"crypto/ecdsa"
objectSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
)
// Split splits fully formed object into multiple chunks.
func (c *Constructor) Split(obj *objectSDK.Object, key *ecdsa.PrivateKey) ([]*objectSDK.Object, error) {
c.clear()
header, err := obj.CutPayload().Marshal()
if err != nil {
return nil, err
}
headerShards, err := c.encodeRaw(header)
if err != nil {
return nil, err
}
payloadShards, err := c.encodeRaw(obj.Payload())
if err != nil {
return nil, err
}
parts := make([]*objectSDK.Object, len(payloadShards))
parent, _ := obj.ID()
for i := range parts {
chunk := objectSDK.New()
copyRequiredFields(chunk, obj)
chunk.SetPayload(payloadShards[i])
chunk.SetPayloadSize(uint64(len(payloadShards[i])))
var parentSplitParentID *oid.ID
if obj.HasParent() {
splitParentID, ok := obj.Parent().ID()
if ok {
parentSplitParentID = &splitParentID
}
}
ecParentInfo := objectSDK.ECParentInfo{
ID: parent,
SplitID: obj.SplitID(),
SplitParentID: parentSplitParentID,
Attributes: obj.Attributes(),
}
ec := objectSDK.NewECHeader(ecParentInfo, uint32(i), uint32(len(payloadShards)), headerShards[i], uint32(len(header)))
chunk.SetECHeader(ec)
if err := setIDWithSignature(chunk, key); err != nil {
return nil, err
}
parts[i] = chunk
}
return parts, nil
}
func setIDWithSignature(obj *objectSDK.Object, key *ecdsa.PrivateKey) error {
objectSDK.CalculateAndSetPayloadChecksum(obj)
if err := objectSDK.CalculateAndSetID(obj); err != nil {
return err
}
if key == nil {
return nil
}
return objectSDK.CalculateAndSetSignature(*key, obj)
}
func (c *Constructor) encodeRaw(data []byte) ([][]byte, error) {
shards, err := c.enc.Split(data)
if err != nil {
return nil, err
}
if err := c.enc.Encode(shards); err != nil {
return nil, err
}
return shards, nil
}

View file

@ -0,0 +1,138 @@
package erasurecode_test
import (
"testing"
objectSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/erasurecode"
oidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id/test"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/stretchr/testify/require"
)
// The library can behave differently for big shard counts.
// This test checks we support the maximum number of chunks we promise.
func TestSplitMaxShardCount(t *testing.T) {
pk, err := keys.NewPrivateKey()
require.NoError(t, err)
original := newObject(t, 1024, pk)
t.Run("only data", func(t *testing.T) {
c, err := erasurecode.NewConstructor(erasurecode.MaxShardCount, 0)
require.NoError(t, err)
parts, err := c.Split(original, &pk.PrivateKey)
require.NoError(t, err)
require.Len(t, parts, erasurecode.MaxShardCount)
for _, part := range parts {
require.NoError(t, objectSDK.CheckHeaderVerificationFields(part))
}
})
t.Run("data + parity", func(t *testing.T) {
c, err := erasurecode.NewConstructor(1, erasurecode.MaxShardCount-1)
require.NoError(t, err)
parts, err := c.Split(original, &pk.PrivateKey)
require.NoError(t, err)
require.Len(t, parts, erasurecode.MaxShardCount)
for _, part := range parts {
require.NoError(t, objectSDK.CheckHeaderVerificationFields(part))
}
})
t.Run("ec parents are children of last Split part", func(t *testing.T) {
c, err := erasurecode.NewConstructor(1, erasurecode.MaxShardCount-1)
require.NoError(t, err)
splitted1 := newObject(t, 1024, pk)
splitted2 := newObject(t, 1024, pk)
splittedId1, _ := splitted1.ID()
splittedId2, _ := splitted2.ID()
splitID := objectSDK.NewSplitID()
parent := objectSDK.New()
parent.SetID(oidtest.ID())
parent.SetChildren(splittedId1, splittedId2)
parent.SetSplitID(splitID)
splitted1.SetSplitID(splitID)
splitted1.SetParent(parent)
splitted2.SetSplitID(splitID)
splitted2.SetParent(parent)
parts1, err := c.Split(splitted1, &pk.PrivateKey)
require.NoError(t, err)
require.Len(t, parts1, erasurecode.MaxShardCount)
for _, part := range parts1 {
require.NoError(t, objectSDK.CheckHeaderVerificationFields(part))
require.NotNil(t, part.ECHeader().ParentSplitID())
require.Equal(t, *splitID, *part.ECHeader().ParentSplitID())
require.NotNil(t, part.ECHeader().ParentSplitParentID())
}
parts2, err := c.Split(splitted2, &pk.PrivateKey)
require.NoError(t, err)
require.Len(t, parts1, erasurecode.MaxShardCount)
for _, part := range parts2 {
require.NoError(t, objectSDK.CheckHeaderVerificationFields(part))
require.NotNil(t, part.ECHeader().ParentSplitID())
require.Equal(t, *splitID, *part.ECHeader().ParentSplitID())
require.NotNil(t, part.ECHeader().ParentSplitParentID())
}
})
t.Run("ec parents are children of non-last Split part", func(t *testing.T) {
c, err := erasurecode.NewConstructor(1, erasurecode.MaxShardCount-1)
require.NoError(t, err)
splitted1 := newObject(t, 1024, pk)
splitted2 := newObject(t, 1024, pk)
splittedId1, _ := splitted1.ID()
splittedId2, _ := splitted2.ID()
splitID := objectSDK.NewSplitID()
parent := objectSDK.New()
// Such parent has got no ID.
parent.SetChildren(splittedId1, splittedId2)
parent.SetSplitID(splitID)
splitted1.SetSplitID(splitID)
splitted1.SetParent(parent)
splitted2.SetSplitID(splitID)
splitted2.SetParent(parent)
parts1, err := c.Split(splitted1, &pk.PrivateKey)
require.NoError(t, err)
require.Len(t, parts1, erasurecode.MaxShardCount)
for _, part := range parts1 {
require.NoError(t, objectSDK.CheckHeaderVerificationFields(part))
require.NotNil(t, part.ECHeader().ParentSplitID())
require.Equal(t, *splitID, *part.ECHeader().ParentSplitID())
require.Nil(t, part.ECHeader().ParentSplitParentID())
}
parts2, err := c.Split(splitted2, &pk.PrivateKey)
require.NoError(t, err)
require.Len(t, parts1, erasurecode.MaxShardCount)
for _, part := range parts2 {
require.NoError(t, objectSDK.CheckHeaderVerificationFields(part))
require.NotNil(t, part.ECHeader().ParentSplitID())
require.Equal(t, *splitID, *part.ECHeader().ParentSplitID())
require.Nil(t, part.ECHeader().ParentSplitParentID())
}
})
}

View file

@ -0,0 +1,44 @@
package erasurecode
import (
"context"
"crypto/ecdsa"
objectSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer"
)
// Target accepts regular objects and splits them into erasure-coded chunks.
type Target struct {
c *Constructor
key *ecdsa.PrivateKey
next transformer.ObjectWriter
}
// ObjectWriter is an interface of the object writer that writes prepared object.
type ObjectWriter interface {
WriteObject(context.Context, *objectSDK.Object) error
}
// NewTarget returns new target instance.
func NewTarget(c *Constructor, key *ecdsa.PrivateKey, next ObjectWriter) *Target {
return &Target{
c: c,
key: key,
next: next,
}
}
// WriteObject implements the transformer.ObjectWriter interface.
func (t *Target) WriteObject(ctx context.Context, obj *objectSDK.Object) error {
parts, err := t.c.Split(obj, t.key)
if err != nil {
return err
}
for i := range parts {
if err := t.next.WriteObject(ctx, parts[i]); err != nil {
return err
}
}
return nil
}

View file

@ -0,0 +1,104 @@
package erasurecode
import (
"fmt"
objectSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
)
// Verify verifies that parts are well formed.
// All parts are expected to be non-nil.
// The number of parts must be equal to `total` field of the EC header
// and parts must be sorted by index.
func (c *Constructor) Verify(parts []*objectSDK.Object) error {
c.clear()
var headerLength int
for i := range parts {
if parts[i] == nil {
return ErrMalformedSlice
}
var err error
headerLength, err = validatePart(parts, i, headerLength)
if err != nil {
return err
}
}
p0 := parts[0]
for i := 1; i < len(parts); i++ {
// This part must be kept in sync with copyRequiredFields().
pi := parts[i]
if p0.OwnerID().Equals(pi.OwnerID()) {
return fmt.Errorf("%w: owner id mismatch: %s != %s", ErrMalformedSlice, p0.OwnerID(), pi.OwnerID())
}
if p0.Version() == nil && pi.Version() != nil || !p0.Version().Equal(*pi.Version()) {
return fmt.Errorf("%w: version mismatch: %s != %s", ErrMalformedSlice, p0.Version(), pi.Version())
}
cnr0, _ := p0.ContainerID()
cnri, _ := pi.ContainerID()
if !cnr0.Equals(cnri) {
return fmt.Errorf("%w: container id mismatch: %s != %s", ErrMalformedSlice, cnr0, cnri)
}
}
if err := c.fillHeader(parts); err != nil {
return err
}
c.fillPayload(parts)
ok, err := c.enc.Verify(c.headerShards)
if err != nil {
return err
}
if !ok {
return ErrMalformedSlice
}
ok, err = c.enc.Verify(c.payloadShards)
if err != nil {
return err
}
if !ok {
return ErrMalformedSlice
}
return nil
}
// copyRequiredFields sets all fields in dst which are copied from src and shared among all chunks.
// src can be either another chunk of full object.
// dst must be a chunk.
func copyRequiredFields(dst *objectSDK.Object, src *objectSDK.Object) {
dst.SetVersion(src.Version())
dst.SetOwnerID(src.OwnerID())
dst.SetCreationEpoch(src.CreationEpoch())
dst.SetSessionToken(src.SessionToken())
cnr, _ := src.ContainerID()
dst.SetContainerID(cnr)
}
// validatePart makes i-th part is consistent with the rest.
// If headerLength is not zero it is asserted to be equal in the ec header.
// Otherwise, new headerLength is returned.
func validatePart(parts []*objectSDK.Object, i int, headerLength int) (int, error) {
ec := parts[i].GetECHeader()
if ec == nil {
return headerLength, fmt.Errorf("%w: missing EC header", ErrMalformedSlice)
}
if ec.Index() != uint32(i) {
return headerLength, fmt.Errorf("%w: index=%d, ec.index=%d", ErrMalformedSlice, i, ec.Index())
}
if ec.Total() != uint32(len(parts)) {
return headerLength, fmt.Errorf("%w: len(parts)=%d, total=%d", ErrMalformedSlice, len(parts), ec.Total())
}
if headerLength == 0 {
return int(ec.HeaderLength()), nil
}
if ec.HeaderLength() != uint32(headerLength) {
return headerLength, fmt.Errorf("%w: header length mismatch %d != %d", ErrMalformedSlice, headerLength, ec.HeaderLength())
}
return headerLength, nil
}

View file

@ -17,3 +17,21 @@ func (s *SplitInfoError) SplitInfo() *SplitInfo {
func NewSplitInfoError(v *SplitInfo) *SplitInfoError { func NewSplitInfoError(v *SplitInfo) *SplitInfoError {
return &SplitInfoError{si: v} return &SplitInfoError{si: v}
} }
type ECInfoError struct {
ei *ECInfo
}
const ecInfoErrorMsg = "object not found, ec info has been provided"
func (e *ECInfoError) Error() string {
return ecInfoErrorMsg
}
func (e *ECInfoError) ECInfo() *ECInfo {
return e.ei
}
func NewECInfoError(v *ECInfo) *ECInfoError {
return &ECInfoError{ei: v}
}

View file

@ -1,9 +1,12 @@
package object_test package object_test
import ( import (
"crypto/rand"
"errors" "errors"
"testing" "testing"
objectV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
@ -31,3 +34,43 @@ func generateSplitInfo() *object.SplitInfo {
return si return si
} }
func TestNewECInfoError(t *testing.T) {
var (
ei = generateECInfo()
err error = object.NewECInfoError(ei)
expectedErr *object.ECInfoError
)
require.True(t, errors.As(err, &expectedErr))
eiErr, ok := err.(*object.ECInfoError)
require.True(t, ok)
require.Equal(t, ei, eiErr.ECInfo())
}
func generateECInfo() *object.ECInfo {
ei := object.NewECInfo()
ei.Chunks = append(ei.Chunks, objectV2.ECChunk{
ID: generateV2ID(),
Index: 0,
Total: 2,
})
ei.Chunks = append(ei.Chunks, objectV2.ECChunk{
ID: generateV2ID(),
Index: 1,
Total: 2,
})
return ei
}
func generateV2ID() refs.ObjectID {
var buf [32]byte
_, _ = rand.Read(buf[:])
var id refs.ObjectID
id.SetValue(buf[:])
return id
}

View file

@ -61,7 +61,7 @@ func TestID_Equal(t *testing.T) {
func TestID_Parse(t *testing.T) { func TestID_Parse(t *testing.T) {
t.Run("should parse successful", func(t *testing.T) { t.Run("should parse successful", func(t *testing.T) {
for i := 0; i < 10; i++ { for i := range 10 {
t.Run(strconv.Itoa(i), func(t *testing.T) { t.Run(strconv.Itoa(i), func(t *testing.T) {
cs := randSHA256Checksum(t) cs := randSHA256Checksum(t)
str := base58.Encode(cs[:]) str := base58.Encode(cs[:])
@ -78,7 +78,7 @@ func TestID_Parse(t *testing.T) {
}) })
t.Run("should failure on parse", func(t *testing.T) { t.Run("should failure on parse", func(t *testing.T) {
for i := 0; i < 10; i++ { for i := range 10 {
j := i j := i
t.Run(strconv.Itoa(j), func(t *testing.T) { t.Run(strconv.Itoa(j), func(t *testing.T) {
cs := []byte{1, 2, 3, 4, 5, byte(j)} cs := []byte{1, 2, 3, 4, 5, byte(j)}
@ -98,7 +98,7 @@ func TestID_String(t *testing.T) {
}) })
t.Run("should be equal", func(t *testing.T) { t.Run("should be equal", func(t *testing.T) {
for i := 0; i < 10; i++ { for i := range 10 {
t.Run(strconv.Itoa(i), func(t *testing.T) { t.Run(strconv.Itoa(i), func(t *testing.T) {
cs := randSHA256Checksum(t) cs := randSHA256Checksum(t)
str := base58.Encode(cs[:]) str := base58.Encode(cs[:])

View file

@ -366,6 +366,14 @@ func (o *Object) Children() []oid.ID {
return res return res
} }
func (o *Object) GetECHeader() *ECHeader {
v2 := (*object.Object)(o).GetHeader().GetEC()
var ec ECHeader
_ = ec.ReadFromV2(v2) // Errors is checked on unmarshal.
return &ec
}
// SetChildren sets list of the identifiers of the child objects. // SetChildren sets list of the identifiers of the child objects.
func (o *Object) SetChildren(v ...oid.ID) { func (o *Object) SetChildren(v ...oid.ID) {
var ( var (

103
object/patch.go Normal file
View file

@ -0,0 +1,103 @@
package object
import (
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
)
// Patch is a patch that's applied for an object.
type Patch struct {
// The address of the object for which the patch is being applied.
Address oid.Address
// The list of new attributes to set in the object's header.
NewAttributes []Attribute
// If ReplaceAttributes flag is true, then the header's attributes are reset and
// filled with NewAttributes. Otherwise, the attributes are just merged.
ReplaceAttributes bool
// Payload patch. If this field is not set, then it assumed such Patch patches only
// header (see NewAttributes, ReplaceAttributes).
PayloadPatch *PayloadPatch
}
func (p *Patch) ToV2() *v2object.PatchRequestBody {
if p == nil {
return nil
}
v2 := new(v2object.PatchRequestBody)
addrV2 := new(refs.Address)
p.Address.WriteToV2(addrV2)
v2.SetAddress(addrV2)
attrs := make([]v2object.Attribute, len(p.NewAttributes))
for i := range p.NewAttributes {
attrs[i] = *p.NewAttributes[i].ToV2()
}
v2.SetNewAttributes(attrs)
v2.SetReplaceAttributes(p.ReplaceAttributes)
v2.SetPatch(p.PayloadPatch.ToV2())
return v2
}
func (p *Patch) FromV2(patch *v2object.PatchRequestBody) {
if patch == nil {
return
}
if addr := patch.GetAddress(); addr != nil {
_ = p.Address.ReadFromV2(*addr)
}
newAttrs := patch.GetNewAttributes()
p.NewAttributes = make([]Attribute, len(newAttrs))
for i := range newAttrs {
p.NewAttributes[i] = *NewAttributeFromV2(&newAttrs[i])
}
p.ReplaceAttributes = patch.GetReplaceAttributes()
if v2patch := patch.GetPatch(); v2patch != nil {
p.PayloadPatch = new(PayloadPatch)
p.PayloadPatch.FromV2(v2patch)
}
}
// Patch is a patch that's applied for an object's payload.
type PayloadPatch struct {
// Range of the patch application.
Range *Range
// Chunk is the payload that replaces (or is appended to) the original object payload.
Chunk []byte
}
func (p *PayloadPatch) ToV2() *v2object.PatchRequestBodyPatch {
if p == nil {
return nil
}
v2 := new(v2object.PatchRequestBodyPatch)
v2.Chunk = p.Chunk
v2.Range = p.Range.ToV2()
return v2
}
func (p *PayloadPatch) FromV2(patch *v2object.PatchRequestBodyPatch) {
if patch == nil {
return
}
p.Chunk = patch.Chunk
if patch.Range != nil {
p.Range = NewRangeFromV2(patch.Range)
}
}

161
object/patch_test.go Normal file
View file

@ -0,0 +1,161 @@
package object
import (
"testing"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
oidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id/test"
"github.com/stretchr/testify/require"
)
func TestPatch(t *testing.T) {
t.Run("to v2", func(t *testing.T) {
t.Run("only attributes", func(t *testing.T) {
var p Patch
var attr1, attr2 Attribute
attr1.SetKey("key1")
attr1.SetValue("val1")
attr2.SetKey("key2")
attr2.SetValue("val2")
p.Address = oidtest.Address()
p.NewAttributes = []Attribute{attr1, attr2}
p.ReplaceAttributes = true
v2patch := p.ToV2()
addr := new(oid.Address)
require.NotNil(t, v2patch.GetAddress())
addr.ReadFromV2(*v2patch.GetAddress())
require.True(t, p.Address.Equals(*addr))
require.Equal(t, p.ReplaceAttributes, v2patch.GetReplaceAttributes())
require.Nil(t, v2patch.GetPatch())
require.Len(t, v2patch.GetNewAttributes(), 2)
require.Equal(t, attr1.Key(), v2patch.GetNewAttributes()[0].GetKey())
require.Equal(t, attr2.Key(), v2patch.GetNewAttributes()[1].GetKey())
require.Equal(t, attr1.Value(), v2patch.GetNewAttributes()[0].GetValue())
require.Equal(t, attr2.Value(), v2patch.GetNewAttributes()[1].GetValue())
})
t.Run("with payload patch", func(t *testing.T) {
var p Patch
var attr1, attr2 Attribute
attr1.SetKey("key1")
attr1.SetValue("val1")
attr2.SetKey("key2")
attr2.SetValue("val2")
p.Address = oidtest.Address()
p.NewAttributes = []Attribute{attr1, attr2}
p.ReplaceAttributes = true
rng := &Range{}
rng.SetOffset(100)
rng.SetLength(10)
p.PayloadPatch = &PayloadPatch{
Range: rng,
Chunk: []byte("payload_patch_chunk"),
}
v2patch := p.ToV2()
addr := new(oid.Address)
require.NotNil(t, v2patch.GetAddress())
addr.ReadFromV2(*v2patch.GetAddress())
require.True(t, p.Address.Equals(*addr))
require.Equal(t, p.ReplaceAttributes, v2patch.GetReplaceAttributes())
require.Len(t, v2patch.GetNewAttributes(), 2)
require.Equal(t, attr1.Key(), v2patch.GetNewAttributes()[0].GetKey())
require.Equal(t, attr2.Key(), v2patch.GetNewAttributes()[1].GetKey())
require.Equal(t, attr1.Value(), v2patch.GetNewAttributes()[0].GetValue())
require.Equal(t, attr2.Value(), v2patch.GetNewAttributes()[1].GetValue())
require.NotNil(t, v2patch.GetPatch())
require.NotNil(t, v2patch.GetPatch().Range)
require.Equal(t, uint64(100), v2patch.GetPatch().Range.GetOffset())
require.Equal(t, uint64(10), v2patch.GetPatch().Range.GetLength())
require.Equal(t, []byte("payload_patch_chunk"), v2patch.GetPatch().Chunk)
})
})
t.Run("from v2", func(t *testing.T) {
t.Run("only attributes", func(t *testing.T) {
v2patch := new(v2object.PatchRequestBody)
address := oidtest.Address()
v2addr := new(refs.Address)
address.WriteToV2(v2addr)
v2patch.SetAddress(v2addr)
var attr1, attr2 Attribute
attr1.SetKey("key1")
attr1.SetValue("val1")
attr2.SetKey("key2")
attr2.SetValue("val2")
v2patch.SetNewAttributes([]v2object.Attribute{
*attr1.ToV2(), *attr2.ToV2(),
})
v2patch.SetReplaceAttributes(true)
var p Patch
p.FromV2(v2patch)
require.Equal(t, address, p.Address)
require.Equal(t, []Attribute{attr1, attr2}, p.NewAttributes)
require.Equal(t, true, p.ReplaceAttributes)
require.Nil(t, p.PayloadPatch)
})
t.Run("with payload patch", func(t *testing.T) {
v2patchReqBody := new(v2object.PatchRequestBody)
address := oidtest.Address()
v2addr := new(refs.Address)
address.WriteToV2(v2addr)
v2patchReqBody.SetAddress(v2addr)
var attr1, attr2 Attribute
attr1.SetKey("key1")
attr1.SetValue("val1")
attr2.SetKey("key2")
attr2.SetValue("val2")
v2patchReqBody.SetNewAttributes([]v2object.Attribute{
*attr1.ToV2(), *attr2.ToV2(),
})
v2patchReqBody.SetReplaceAttributes(true)
v2Rng := &v2object.Range{}
v2Rng.SetOffset(13)
v2Rng.SetLength(10)
v2Patch := &v2object.PatchRequestBodyPatch{
Range: v2Rng,
Chunk: []byte("payload_patch_chunk"),
}
v2patchReqBody.SetPatch(v2Patch)
var p Patch
p.FromV2(v2patchReqBody)
require.Equal(t, address, p.Address)
require.Equal(t, []Attribute{attr1, attr2}, p.NewAttributes)
require.Equal(t, true, p.ReplaceAttributes)
require.NotNil(t, p.PayloadPatch)
require.NotNil(t, p.PayloadPatch.Range)
require.Equal(t, uint64(13), p.PayloadPatch.Range.GetOffset())
require.Equal(t, uint64(10), p.PayloadPatch.Range.GetLength())
require.Equal(t, []byte("payload_patch_chunk"), p.PayloadPatch.Chunk)
})
})
}

244
object/patcher/patcher.go Normal file
View file

@ -0,0 +1,244 @@
package patcher
import (
"context"
"errors"
"fmt"
"io"
objectSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer"
)
var (
ErrOffsetExceedsSize = errors.New("patch offset exceeds object size")
ErrInvalidPatchOffsetOrder = errors.New("invalid patch offset order")
ErrPayloadPatchIsNil = errors.New("nil payload patch")
ErrAttrPatchAlreadyApplied = errors.New("attribute patch already applied")
)
// PatchRes is the result of patch application.
type PatchRes struct {
AccessIdentifiers *transformer.AccessIdentifiers
}
// PatchApplier is the interface that provides method to apply header and payload patches.
type PatchApplier interface {
// ApplyAttributesPatch applies the patch only for the object's attributes.
//
// ApplyAttributesPatch can't be invoked few times, otherwise it returns `ErrAttrPatchAlreadyApplied` error.
//
// The call is idempotent for the original header if it's invoked with empty `newAttrs` and
// `replaceAttrs = false`.
ApplyAttributesPatch(ctx context.Context, newAttrs []objectSDK.Attribute, replaceAttrs bool) error
// ApplyPayloadPatch applies the patch for the object's payload.
//
// ApplyPayloadPatch returns `ErrPayloadPatchIsNil` error if patch is nil.
ApplyPayloadPatch(ctx context.Context, payloadPatch *objectSDK.PayloadPatch) error
// Close closes PatchApplier when the patch stream is over.
Close(context.Context) (PatchRes, error)
}
// RangeProvider is the interface that provides a method to get original object payload
// by a given range.
type RangeProvider interface {
// GetRange reads an original object payload by the given range.
// The method returns io.Reader over the data range only. This means if the data is read out,
// then GetRange has to be invoked to provide reader over the next range.
GetRange(ctx context.Context, rng *objectSDK.Range) io.Reader
}
type patcher struct {
rangeProvider RangeProvider
objectWriter transformer.ChunkedObjectWriter
currOffset uint64
originalPayloadSize uint64
hdr *objectSDK.Object
attrPatchAlreadyApplied bool
readerBuffSize int
}
const (
DefaultReaderBufferSize = 64 * 1024
)
// Params is parameters to initialize patcher.
type Params struct {
// Original object header.
Header *objectSDK.Object
// Range provider.
RangeProvider RangeProvider
// ObjectWriter is the writer that writes the patched object.
ObjectWriter transformer.ChunkedObjectWriter
// The size of the buffer used by the original payload range reader.
// If it's set to <=0, then `DefaultReaderBufferSize` is used.
ReaderBufferSize int
}
func New(prm Params) PatchApplier {
readerBufferSize := prm.ReaderBufferSize
if readerBufferSize <= 0 {
readerBufferSize = DefaultReaderBufferSize
}
return &patcher{
rangeProvider: prm.RangeProvider,
objectWriter: prm.ObjectWriter,
hdr: prm.Header,
originalPayloadSize: prm.Header.PayloadSize(),
readerBuffSize: readerBufferSize,
}
}
func (p *patcher) ApplyAttributesPatch(ctx context.Context, newAttrs []objectSDK.Attribute, replaceAttrs bool) error {
defer func() {
p.attrPatchAlreadyApplied = true
}()
if p.attrPatchAlreadyApplied {
return ErrAttrPatchAlreadyApplied
}
if replaceAttrs {
p.hdr.SetAttributes(newAttrs...)
} else if len(newAttrs) > 0 {
mergedAttrs := mergeAttributes(newAttrs, p.hdr.Attributes())
p.hdr.SetAttributes(mergedAttrs...)
}
if err := p.objectWriter.WriteHeader(ctx, p.hdr); err != nil {
return fmt.Errorf("writer header: %w", err)
}
return nil
}
func (p *patcher) ApplyPayloadPatch(ctx context.Context, payloadPatch *objectSDK.PayloadPatch) error {
if payloadPatch == nil {
return ErrPayloadPatchIsNil
}
if payloadPatch.Range.GetOffset() < p.currOffset {
return fmt.Errorf("%w: current = %d, previous = %d", ErrInvalidPatchOffsetOrder, payloadPatch.Range.GetOffset(), p.currOffset)
}
if payloadPatch.Range.GetOffset() > p.originalPayloadSize {
return fmt.Errorf("%w: offset = %d, object size = %d", ErrOffsetExceedsSize, payloadPatch.Range.GetOffset(), p.originalPayloadSize)
}
var err error
if p.currOffset, err = p.applyPatch(ctx, payloadPatch, p.currOffset); err != nil {
return fmt.Errorf("apply patch: %w", err)
}
return nil
}
func (p *patcher) Close(ctx context.Context) (PatchRes, error) {
rng := new(objectSDK.Range)
rng.SetOffset(p.currOffset)
rng.SetLength(p.originalPayloadSize - p.currOffset)
// copy remaining originial payload
if err := p.copyRange(ctx, rng); err != nil {
return PatchRes{}, fmt.Errorf("copy payload: %w", err)
}
aid, err := p.objectWriter.Close(ctx)
if err != nil {
return PatchRes{}, fmt.Errorf("close object writer: %w", err)
}
return PatchRes{
AccessIdentifiers: aid,
}, nil
}
func (p *patcher) copyRange(ctx context.Context, rng *objectSDK.Range) error {
rdr := p.rangeProvider.GetRange(ctx, rng)
for {
buffOrigPayload := make([]byte, p.readerBuffSize)
n, readErr := rdr.Read(buffOrigPayload)
if readErr != nil {
if readErr != io.EOF {
return fmt.Errorf("read: %w", readErr)
}
}
_, wrErr := p.objectWriter.Write(ctx, buffOrigPayload[:n])
if wrErr != nil {
return fmt.Errorf("write: %w", wrErr)
}
if readErr == io.EOF {
break
}
}
return nil
}
func (p *patcher) applyPatch(ctx context.Context, payloadPatch *objectSDK.PayloadPatch, offset uint64) (newOffset uint64, err error) {
newOffset = offset
// write the original payload chunk before the start of the patch
if payloadPatch.Range.GetOffset() > offset {
rng := new(objectSDK.Range)
rng.SetOffset(offset)
rng.SetLength(payloadPatch.Range.GetOffset() - offset)
if err = p.copyRange(ctx, rng); err != nil {
err = fmt.Errorf("copy payload: %w", err)
return
}
newOffset = payloadPatch.Range.GetOffset()
}
// apply patch
if _, err = p.objectWriter.Write(ctx, payloadPatch.Chunk); err != nil {
return
}
if payloadPatch.Range.GetLength() > 0 {
newOffset += payloadPatch.Range.GetLength()
}
return
}
func mergeAttributes(newAttrs, oldAttrs []objectSDK.Attribute) []objectSDK.Attribute {
attrMap := make(map[string]string, len(newAttrs))
for _, attr := range newAttrs {
attrMap[attr.Key()] = attr.Value()
}
for i := range oldAttrs {
newVal, ok := attrMap[oldAttrs[i].Key()]
if !ok {
continue
}
oldAttrs[i].SetValue(newVal)
delete(attrMap, oldAttrs[i].Key())
}
for _, newAttr := range newAttrs {
if _, ok := attrMap[newAttr.Key()]; ok {
oldAttrs = append(oldAttrs, newAttr)
}
}
return oldAttrs
}

View file

@ -0,0 +1,626 @@
package patcher
import (
"bytes"
"context"
"io"
"testing"
objectSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
oidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id/test"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer"
"github.com/stretchr/testify/require"
)
type mockPatchedObjectWriter struct {
obj *objectSDK.Object
}
func (m *mockPatchedObjectWriter) Write(_ context.Context, chunk []byte) (int, error) {
res := append(m.obj.Payload(), chunk...)
m.obj.SetPayload(res)
m.obj.SetPayloadSize(uint64(len(res)))
return len(chunk), nil
}
func (m *mockPatchedObjectWriter) WriteHeader(_ context.Context, hdr *objectSDK.Object) error {
m.obj.ToV2().SetHeader(hdr.ToV2().GetHeader())
return nil
}
func (m *mockPatchedObjectWriter) Close(context.Context) (*transformer.AccessIdentifiers, error) {
return &transformer.AccessIdentifiers{}, nil
}
type mockRangeProvider struct {
originalObjectPayload []byte
}
var _ RangeProvider = (*mockRangeProvider)(nil)
func (m *mockRangeProvider) GetRange(_ context.Context, rng *objectSDK.Range) io.Reader {
offset := rng.GetOffset()
length := rng.GetLength()
if length == 0 {
return bytes.NewReader(m.originalObjectPayload[offset:])
}
return bytes.NewReader(m.originalObjectPayload[offset : offset+length])
}
func newTestObject() (*objectSDK.Object, oid.Address) {
obj := objectSDK.New()
addr := oidtest.Address()
obj.SetContainerID(addr.Container())
obj.SetID(addr.Object())
return obj, addr
}
func rangeWithOffestWithLength(offset, length uint64) *objectSDK.Range {
rng := new(objectSDK.Range)
rng.SetOffset(offset)
rng.SetLength(length)
return rng
}
func TestPatchRevert(t *testing.T) {
obj, _ := newTestObject()
modifPatch := &objectSDK.Patch{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(0, 0),
Chunk: []byte("inserted"),
},
}
originalObjectPayload := []byte("*******************")
obj.SetPayload(originalObjectPayload)
obj.SetPayloadSize(uint64(len(originalObjectPayload)))
exp := []byte("inserted*******************")
rangeProvider := &mockRangeProvider{
originalObjectPayload: originalObjectPayload,
}
patchedObj, _ := newTestObject()
wr := &mockPatchedObjectWriter{
obj: patchedObj,
}
prm := Params{
Header: obj.CutPayload(),
RangeProvider: rangeProvider,
ObjectWriter: wr,
}
patcher := New(prm)
err := patcher.ApplyAttributesPatch(context.Background(), modifPatch.NewAttributes, modifPatch.ReplaceAttributes)
require.NoError(t, err)
err = patcher.ApplyPayloadPatch(context.Background(), modifPatch.PayloadPatch)
require.NoError(t, err)
_, err = patcher.Close(context.Background())
require.NoError(t, err)
require.Equal(t, exp, patchedObj.Payload())
revertPatch := &objectSDK.Patch{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(0, uint64(len("inserted"))),
Chunk: []byte{},
},
}
rangeProvider = &mockRangeProvider{
originalObjectPayload: exp,
}
patchedPatchedObj, _ := newTestObject()
wr = &mockPatchedObjectWriter{
obj: patchedPatchedObj,
}
prm = Params{
Header: patchedObj.CutPayload(),
RangeProvider: rangeProvider,
ObjectWriter: wr,
}
patcher = New(prm)
err = patcher.ApplyAttributesPatch(context.Background(), revertPatch.NewAttributes, revertPatch.ReplaceAttributes)
require.NoError(t, err)
err = patcher.ApplyPayloadPatch(context.Background(), revertPatch.PayloadPatch)
require.NoError(t, err)
_, err = patcher.Close(context.Background())
require.NoError(t, err)
require.Equal(t, originalObjectPayload, patchedPatchedObj.Payload())
}
func TestPatchRepeatAttributePatch(t *testing.T) {
obj, _ := newTestObject()
modifPatch := &objectSDK.Patch{}
originalObjectPayload := []byte("*******************")
obj.SetPayload(originalObjectPayload)
obj.SetPayloadSize(uint64(len(originalObjectPayload)))
rangeProvider := &mockRangeProvider{
originalObjectPayload: originalObjectPayload,
}
patchedObj, _ := newTestObject()
wr := &mockPatchedObjectWriter{
obj: patchedObj,
}
prm := Params{
Header: obj.CutPayload(),
RangeProvider: rangeProvider,
ObjectWriter: wr,
}
patcher := New(prm)
err := patcher.ApplyAttributesPatch(context.Background(), modifPatch.NewAttributes, modifPatch.ReplaceAttributes)
require.NoError(t, err)
err = patcher.ApplyAttributesPatch(context.Background(), modifPatch.NewAttributes, modifPatch.ReplaceAttributes)
require.ErrorIs(t, err, ErrAttrPatchAlreadyApplied)
}
func TestPatchEmptyPayloadPatch(t *testing.T) {
obj, _ := newTestObject()
modifPatch := &objectSDK.Patch{}
originalObjectPayload := []byte("*******************")
obj.SetPayload(originalObjectPayload)
obj.SetPayloadSize(uint64(len(originalObjectPayload)))
rangeProvider := &mockRangeProvider{
originalObjectPayload: originalObjectPayload,
}
patchedObj, _ := newTestObject()
wr := &mockPatchedObjectWriter{
obj: patchedObj,
}
prm := Params{
Header: obj.CutPayload(),
RangeProvider: rangeProvider,
ObjectWriter: wr,
}
patcher := New(prm)
err := patcher.ApplyAttributesPatch(context.Background(), modifPatch.NewAttributes, modifPatch.ReplaceAttributes)
require.NoError(t, err)
err = patcher.ApplyPayloadPatch(context.Background(), nil)
require.ErrorIs(t, err, ErrPayloadPatchIsNil)
}
func newTestAttribute(key, val string) objectSDK.Attribute {
var attr objectSDK.Attribute
attr.SetKey(key)
attr.SetValue(val)
return attr
}
func TestPatch(t *testing.T) {
for _, test := range []struct {
name string
patches []objectSDK.Patch
originalObjectPayload []byte
patchedPayload []byte
originalHeaderAttributes []objectSDK.Attribute
patchedHeaderAttributes []objectSDK.Attribute
expectedPayloadPatchErr error
}{
{
name: "invalid offset",
patches: []objectSDK.Patch{
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(100, 0),
Chunk: []byte(""),
},
},
},
originalObjectPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnm"),
expectedPayloadPatchErr: ErrOffsetExceedsSize,
},
{
name: "invalid following patch offset",
patches: []objectSDK.Patch{
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(10, 0),
Chunk: []byte(""),
},
},
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(7, 0),
Chunk: []byte(""),
},
},
},
originalObjectPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnm"),
expectedPayloadPatchErr: ErrInvalidPatchOffsetOrder,
},
{
name: "only header patch",
patches: []objectSDK.Patch{
{
NewAttributes: []objectSDK.Attribute{
newTestAttribute("key1", "val2"),
newTestAttribute("key2", "val2"),
},
},
},
originalObjectPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnm"),
patchedPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnm"),
patchedHeaderAttributes: []objectSDK.Attribute{
newTestAttribute("key1", "val2"),
newTestAttribute("key2", "val2"),
},
},
{
name: "header and payload",
patches: []objectSDK.Patch{
{
NewAttributes: []objectSDK.Attribute{
newTestAttribute("key1", "val2"),
newTestAttribute("key2", "val2"),
},
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(0, 0),
Chunk: []byte("inserted at the beginning"),
},
},
},
originalObjectPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnm"),
patchedPayload: []byte("inserted at the beginning0123456789qwertyuiopasdfghjklzxcvbnm"),
patchedHeaderAttributes: []objectSDK.Attribute{
newTestAttribute("key1", "val2"),
newTestAttribute("key2", "val2"),
},
},
{
name: "header only merge attributes",
patches: []objectSDK.Patch{
{
NewAttributes: []objectSDK.Attribute{
newTestAttribute("key1", "val2"),
newTestAttribute("key2", "val2-incoming"),
},
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(0, 0),
Chunk: []byte("inserted at the beginning"),
},
},
},
originalHeaderAttributes: []objectSDK.Attribute{
newTestAttribute("key2", "to be popped out"),
newTestAttribute("key3", "val3"),
},
originalObjectPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnm"),
patchedPayload: []byte("inserted at the beginning0123456789qwertyuiopasdfghjklzxcvbnm"),
patchedHeaderAttributes: []objectSDK.Attribute{
newTestAttribute("key2", "val2-incoming"),
newTestAttribute("key3", "val3"),
newTestAttribute("key1", "val2"),
},
},
{
name: "header only then payload",
patches: []objectSDK.Patch{
{
NewAttributes: []objectSDK.Attribute{
newTestAttribute("key1", "val2"),
newTestAttribute("key2", "val2"),
},
},
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(0, 0),
Chunk: []byte("inserted at the beginning"),
},
},
},
originalObjectPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnm"),
patchedPayload: []byte("inserted at the beginning0123456789qwertyuiopasdfghjklzxcvbnm"),
patchedHeaderAttributes: []objectSDK.Attribute{
newTestAttribute("key1", "val2"),
newTestAttribute("key2", "val2"),
},
},
{
name: "no effect",
patches: []objectSDK.Patch{
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(0, 0),
Chunk: []byte(""),
},
},
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(12, 0),
Chunk: []byte(""),
},
},
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(20, 0),
Chunk: []byte(""),
},
},
},
originalObjectPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnm"),
patchedPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnm"),
},
{
name: "insert prefix",
patches: []objectSDK.Patch{
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(0, 0),
Chunk: []byte("inserted at the beginning"),
},
},
},
originalObjectPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnm"),
patchedPayload: []byte("inserted at the beginning0123456789qwertyuiopasdfghjklzxcvbnm"),
},
{
name: "insert in the middle",
patches: []objectSDK.Patch{
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(5, 0),
Chunk: []byte("inserted somewhere in the middle"),
},
},
},
originalObjectPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnm"),
patchedPayload: []byte("01234inserted somewhere in the middle56789qwertyuiopasdfghjklzxcvbnm"),
},
{
name: "insert at the end",
patches: []objectSDK.Patch{
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(36, 0),
Chunk: []byte("inserted somewhere at the end"),
},
},
},
originalObjectPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnm"),
patchedPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnminserted somewhere at the end"),
},
{
name: "replace by range",
patches: []objectSDK.Patch{
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(0, 12),
Chunk: []byte("just replace"),
},
},
},
originalObjectPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnm"),
patchedPayload: []byte("just replaceertyuiopasdfghjklzxcvbnm"),
},
{
name: "replace and insert some bytes",
patches: []objectSDK.Patch{
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(0, 11),
Chunk: []byte("replace and append in the middle"),
},
},
},
originalObjectPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnm"),
patchedPayload: []byte("replace and append in the middlewertyuiopasdfghjklzxcvbnm"),
},
{
name: "replace and insert some bytes in the middle",
patches: []objectSDK.Patch{
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(5, 3),
Chunk: []byte("@@@@@"),
},
},
},
originalObjectPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnm"),
patchedPayload: []byte("01234@@@@@89qwertyuiopasdfghjklzxcvbnm"),
},
{
name: "a few patches: prefix, suffix",
patches: []objectSDK.Patch{
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(0, 0),
Chunk: []byte("this_will_be_prefix"),
},
},
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(36, 0),
Chunk: []byte("this_will_be_suffix"),
},
},
},
originalObjectPayload: []byte("0123456789qwertyuiopasdfghjklzxcvbnm"),
patchedPayload: []byte("this_will_be_prefix0123456789qwertyuiopasdfghjklzxcvbnmthis_will_be_suffix"),
},
{
name: "a few patches: replace and insert some bytes",
patches: []objectSDK.Patch{
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(10, 3),
Chunk: []byte("aaaaa"),
},
},
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(16, 0),
Chunk: []byte("bbbbb"),
},
},
},
originalObjectPayload: []byte("0123456789ABCDEF"),
patchedPayload: []byte("0123456789aaaaaDEFbbbbb"),
},
{
name: "starting from the same offset",
patches: []objectSDK.Patch{
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(8, 3),
Chunk: []byte("1"),
},
},
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(11, 0),
Chunk: []byte("2"),
},
},
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(11, 0),
Chunk: []byte("3"),
},
},
},
originalObjectPayload: []byte("abcdefghijklmnop"),
patchedPayload: []byte("abcdefgh123lmnop"),
},
{
name: "a few patches: various modifiactions",
patches: []objectSDK.Patch{
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(4, 8),
Chunk: []byte("earliest"),
},
},
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(13, 0),
Chunk: []byte("known "),
},
},
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(35, 8),
Chunk: []byte("a small town"),
},
},
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(62, 6),
Chunk: []byte("tablet"),
},
},
{
PayloadPatch: &objectSDK.PayloadPatch{
Range: rangeWithOffestWithLength(87, 0),
Chunk: []byte("Shar-Kali-Sharri"),
},
},
},
originalObjectPayload: []byte("The ******** mention of Babylon as [insert] appears on a clay ****** from the reign of "),
patchedPayload: []byte("The earliest known mention of Babylon as a small town appears on a clay tablet from the reign of Shar-Kali-Sharri"),
},
} {
t.Run(test.name, func(t *testing.T) {
rangeProvider := &mockRangeProvider{
originalObjectPayload: test.originalObjectPayload,
}
originalObject, _ := newTestObject()
originalObject.SetPayload(test.originalObjectPayload)
originalObject.SetPayloadSize(uint64(len(test.originalObjectPayload)))
originalObject.SetAttributes(test.originalHeaderAttributes...)
patchedObject, _ := newTestObject()
wr := &mockPatchedObjectWriter{
obj: patchedObject,
}
prm := Params{
Header: originalObject.CutPayload(),
RangeProvider: rangeProvider,
ObjectWriter: wr,
}
patcher := New(prm)
for i, patch := range test.patches {
if i == 0 {
_ = patcher.ApplyAttributesPatch(context.Background(), patch.NewAttributes, patch.ReplaceAttributes)
}
if patch.PayloadPatch == nil {
continue
}
err := patcher.ApplyPayloadPatch(context.Background(), patch.PayloadPatch)
if err != nil && test.expectedPayloadPatchErr != nil {
require.ErrorIs(t, err, test.expectedPayloadPatchErr)
return
}
require.NoError(t, err)
}
_, err := patcher.Close(context.Background())
require.NoError(t, err)
require.Equal(t, test.patchedPayload, patchedObject.Payload())
patchedAttrs := append([]objectSDK.Attribute{}, test.patchedHeaderAttributes...)
require.Equal(t, patchedAttrs, patchedObject.Attributes())
})
}
}

View file

@ -120,6 +120,7 @@ const (
fKeySplitID fKeySplitID
fKeyPropRoot fKeyPropRoot
fKeyPropPhy fKeyPropPhy
fKeyECParent
) )
func (k filterKey) String() string { func (k filterKey) String() string {
@ -152,6 +153,8 @@ func (k filterKey) String() string {
return v2object.FilterPropertyRoot return v2object.FilterPropertyRoot
case fKeyPropPhy: case fKeyPropPhy:
return v2object.FilterPropertyPhy return v2object.FilterPropertyPhy
case fKeyECParent:
return v2object.FilterHeaderECParent
} }
} }
@ -277,6 +280,10 @@ func (f *SearchFilters) AddSplitIDFilter(m SearchMatchType, id *SplitID) {
f.addReservedFilter(m, fKeySplitID, staticStringer(id.String())) f.addReservedFilter(m, fKeySplitID, staticStringer(id.String()))
} }
func (f *SearchFilters) AddECParentFilter(m SearchMatchType, parentID oid.ID) {
f.addReservedFilter(m, fKeyECParent, staticStringer(parentID.String()))
}
// AddTypeFilter adds filter by object type. // AddTypeFilter adds filter by object type.
func (f *SearchFilters) AddTypeFilter(m SearchMatchType, typ Type) { func (f *SearchFilters) AddTypeFilter(m SearchMatchType, typ Type) {
f.addReservedFilter(m, fKeyType, staticStringer(typ.String())) f.addReservedFilter(m, fKeyType, staticStringer(typ.String()))

View file

@ -160,6 +160,23 @@ func TestSearchFilters_AddSplitIDFilter(t *testing.T) {
}) })
} }
func TestSearchFilters_AddECParentFilter(t *testing.T) {
id := testOID()
fs := new(object.SearchFilters)
fs.AddECParentFilter(object.MatchStringEqual, id)
t.Run("v2", func(t *testing.T) {
fsV2 := fs.ToV2()
require.Len(t, fsV2, 1)
require.Equal(t, v2object.FilterHeaderECParent, fsV2[0].GetKey())
require.Equal(t, id.String(), fsV2[0].GetValue())
require.Equal(t, v2object.MatchStringEqual, fsV2[0].GetMatchType())
})
}
func TestSearchFilters_AddTypeFilter(t *testing.T) { func TestSearchFilters_AddTypeFilter(t *testing.T) {
typ := object.TypeTombstone typ := object.TypeTombstone

View file

@ -14,7 +14,7 @@ func generateIDList(sz int) []oid.ID {
res := make([]oid.ID, sz) res := make([]oid.ID, sz)
cs := [sha256.Size]byte{} cs := [sha256.Size]byte{}
for i := 0; i < sz; i++ { for i := range sz {
var oID oid.ID var oID oid.ID
res[i] = oID res[i] = oID

View file

@ -6,6 +6,7 @@ import (
"crypto/sha256" "crypto/sha256"
"fmt" "fmt"
buffPool "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/util/pool"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/checksum" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/checksum"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id" oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
@ -45,6 +46,7 @@ type Params struct {
// functionality. Primary usecases are providing file size when putting an object // functionality. Primary usecases are providing file size when putting an object
// with the frostfs-cli or using Content-Length header in gateways. // with the frostfs-cli or using Content-Length header in gateways.
SizeHint uint64 SizeHint uint64
Pool *buffPool.BufferPool
} }
// NewPayloadSizeLimiter returns ObjectTarget instance that restricts payload length // NewPayloadSizeLimiter returns ObjectTarget instance that restricts payload length
@ -137,7 +139,13 @@ func (s *payloadSizeLimiter) initializeCurrent() {
payloadSize = remaining % s.MaxSize payloadSize = remaining % s.MaxSize
} }
} }
s.payload = make([]byte, 0, payloadSize)
if s.Pool == nil {
s.payload = make([]byte, 0, payloadSize)
} else {
buffer := s.Pool.Get(uint32(payloadSize))
s.payload = buffer.Data[:0]
}
} }
func (s *payloadSizeLimiter) initPayloadHashers() { func (s *payloadSizeLimiter) initPayloadHashers() {
@ -239,6 +247,7 @@ func (s *payloadSizeLimiter) fillHeader() (*AccessIdentifiers, error) {
ParentID: parID, ParentID: parID,
SelfID: id, SelfID: id,
ParentHeader: parHdr, ParentHeader: parHdr,
Epoch: curEpoch,
}, nil }, nil
} }

View file

@ -72,7 +72,7 @@ func TestTransformer(t *testing.T) {
require.Equal(t, ids.ParentID, &parID) require.Equal(t, ids.ParentID, &parID)
children := tt.objects[i].Children() children := tt.objects[i].Children()
for j := 0; j < i; j++ { for j := range i {
id, ok := tt.objects[j].ID() id, ok := tt.objects[j].ID()
require.True(t, ok) require.True(t, ok)
require.Equal(t, id, children[j]) require.Equal(t, id, children[j])
@ -152,7 +152,7 @@ func benchmarkTransformer(b *testing.B, header *objectSDK.Object, payloadSize, s
b.ReportAllocs() b.ReportAllocs()
b.ResetTimer() b.ResetTimer()
for i := 0; i < b.N; i++ { for range b.N {
f, _ := newPayloadSizeLimiter(maxSize, uint64(sizeHint), func() ObjectWriter { return benchTarget{} }) f, _ := newPayloadSizeLimiter(maxSize, uint64(sizeHint), func() ObjectWriter { return benchTarget{} })
if err := f.WriteHeader(ctx, header); err != nil { if err := f.WriteHeader(ctx, header); err != nil {
b.Fatalf("write header: %v", err) b.Fatalf("write header: %v", err)

View file

@ -14,6 +14,7 @@ type AccessIdentifiers struct {
ParentID *oid.ID ParentID *oid.ID
SelfID oid.ID SelfID oid.ID
ParentHeader *object.Object ParentHeader *object.Object
Epoch uint64
} }
// EpochSource is a source for the current epoch. // EpochSource is a source for the current epoch.

View file

@ -9,21 +9,22 @@ import (
) )
type sessionCache struct { type sessionCache struct {
cache *lru.Cache[string, *cacheValue] cache *lru.Cache[string, *cacheValue]
currentEpoch atomic.Uint64 currentEpoch atomic.Uint64
tokenDuration uint64
} }
type cacheValue struct { type cacheValue struct {
token session.Object token session.Object
} }
func newCache() (*sessionCache, error) { func newCache(tokenDuration uint64) (*sessionCache, error) {
cache, err := lru.New[string, *cacheValue](100) cache, err := lru.New[string, *cacheValue](100)
if err != nil { if err != nil {
return nil, err return nil, err
} }
return &sessionCache{cache: cache}, nil return &sessionCache{cache: cache, tokenDuration: tokenDuration}, nil
} }
// Get returns a copy of the session token from the cache without signature // Get returns a copy of the session token from the cache without signature
@ -66,8 +67,12 @@ func (c *sessionCache) updateEpoch(newEpoch uint64) {
func (c *sessionCache) expired(val *cacheValue) bool { func (c *sessionCache) expired(val *cacheValue) bool {
epoch := c.currentEpoch.Load() epoch := c.currentEpoch.Load()
// use epoch+1 (clear cache beforehand) to prevent 'expired session token' error right after epoch tick preExpiredDur := c.tokenDuration / 2
return val.token.ExpiredAt(epoch + 1) if preExpiredDur == 0 {
preExpiredDur = 1
}
return val.token.ExpiredAt(epoch + preExpiredDur)
} }
func (c *sessionCache) Epoch() uint64 { func (c *sessionCache) Epoch() uint64 {

View file

@ -20,7 +20,7 @@ func TestSessionCache_GetUnmodifiedToken(t *testing.T) {
require.False(t, tok.VerifySignature(), extra) require.False(t, tok.VerifySignature(), extra)
} }
cache, err := newCache() cache, err := newCache(0)
require.NoError(t, err) require.NoError(t, err)
cache.Put(key, target) cache.Put(key, target)

View file

@ -7,8 +7,7 @@ a weighted random selection of the underlying client to make requests.
Create pool instance with 3 nodes connection. Create pool instance with 3 nodes connection.
This InitParameters will make pool use 192.168.130.71 node while it is healthy. Otherwise, it will make the pool use This InitParameters will make pool use 192.168.130.71 node while it is healthy. Otherwise, it will make the pool use
192.168.130.72 for 90% of requests and 192.168.130.73 for remaining 10%. 192.168.130.72 for 90% of requests and 192.168.130.73 for remaining 10%:
:
var prm pool.InitParameters var prm pool.InitParameters
prm.SetKey(key) prm.SetKey(key)

View file

@ -9,14 +9,13 @@ import (
sessionv2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session" sessionv2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/accounting" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/accounting"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status" apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id" cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa" frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
"github.com/google/uuid" "github.com/google/uuid"
) )
@ -95,14 +94,18 @@ func (m *mockClient) containerDelete(context.Context, PrmContainerDelete) error
return nil return nil
} }
func (m *mockClient) containerEACL(context.Context, PrmContainerEACL) (eacl.Table, error) { func (c *mockClient) apeManagerAddChain(ctx context.Context, prm PrmAddAPEChain) error {
return eacl.Table{}, nil return nil
} }
func (m *mockClient) containerSetEACL(context.Context, PrmContainerSetEACL) error { func (c *mockClient) apeManagerRemoveChain(ctx context.Context, prm PrmRemoveAPEChain) error {
return nil return nil
} }
func (c *mockClient) apeManagerListChains(ctx context.Context, prm PrmListAPEChains) ([]ape.Chain, error) {
return []ape.Chain{}, nil
}
func (m *mockClient) endpointInfo(ctx context.Context, _ prmEndpointInfo) (netmap.NodeInfo, error) { func (m *mockClient) endpointInfo(ctx context.Context, _ prmEndpointInfo) (netmap.NodeInfo, error) {
var ni netmap.NodeInfo var ni netmap.NodeInfo
@ -129,8 +132,12 @@ func (m *mockClient) netMapSnapshot(context.Context, prmNetMapSnapshot) (netmap.
return nm, nil return nm, nil
} }
func (m *mockClient) objectPut(context.Context, PrmObjectPut) (oid.ID, error) { func (m *mockClient) objectPut(context.Context, PrmObjectPut) (ResPutObject, error) {
return oid.ID{}, nil return ResPutObject{}, nil
}
func (m *mockClient) objectPatch(context.Context, PrmObjectPatch) (ResPatchObject, error) {
return ResPatchObject{}, nil
} }
func (m *mockClient) objectDelete(context.Context, PrmObjectDelete) error { func (m *mockClient) objectDelete(context.Context, PrmObjectDelete) error {
@ -183,9 +190,9 @@ func (m *mockClient) dial(context.Context) error {
return nil return nil
} }
func (m *mockClient) restartIfUnhealthy(ctx context.Context) (healthy bool, changed bool) { func (m *mockClient) restartIfUnhealthy(ctx context.Context) (changed bool, err error) {
_, err := m.endpointInfo(ctx, prmEndpointInfo{}) _, err = m.endpointInfo(ctx, prmEndpointInfo{})
healthy = err == nil healthy := err == nil
changed = healthy != m.isHealthy() changed = healthy != m.isHealthy()
if healthy { if healthy {
m.setHealthy() m.setHealthy()

Some files were not shown because too many files have changed in this diff Show more