Compare commits

...

129 commits

Author SHA1 Message Date
fc4551b843 [#172] pool: Use priority of errors in tree pool
When retry happens, use priority map to decide
which error to return. Consider network errors
less desirable than business logic errors.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-11 12:00:34 +03:00
eb5288f4a5 [#172] pool: Do more retries on unexpected tree service responses
1. Try its best by looking for nodes during 'GetNodeByPath'
2. Retry on 'tree not found' and other not found errors

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-06 11:12:48 +03:00
60463871db [#171] pool: Add test for healthy status monitor
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-03 19:47:22 +03:00
8a04638749 [#171] pool: Close only dialed connections
To avoid panics during close operation, close
only dialed connections.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-03 19:47:22 +03:00
ddbfb758c9 [#171] pool: Use dial status to close connections during restarts
Every client restart, pool creates new client instance. If client
failed due to dial error, there was no prior connection and go
routine on a server side. If client failed due to communication
or business logic errors, then server side maintains connection and
client should close it to avoid routine and connection leak.

Dialing is a part of healthcheck, so health status is now a enum
of three values:
- unhealthy due to dial fail,
- unhealthy due to transmission fail,
- healthy.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-03 17:00:36 +03:00
d71a0e0755 [#88] netmap: use bool, fix hrw_sort tests
Signed-off-by: Andrew Danilin <andnilin@gmail.com>
2023-10-03 07:05:03 +00:00
163b3e1961 [#88] netmap: fix min aggregator bug, add tests
Signed-off-by: Andrew Danilin <andnilin@gmail.com>
2023-10-03 07:05:03 +00:00
84b9d29fc9 [#170] checksum: Use constant mapping for checksum types
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-28 17:20:24 +03:00
99c273f499 [#169] pool: Close inner pools during close routine
Some apps do not reuse pool instance and expect that
`pool.Close()` free resources. But it didn't actually
close inner SDK clients, so it leads to goroutine leak
in storage.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-09-20 12:16:13 +03:00
555ccc63b2 [#167] netmap: Allow to select insufficient number of nodes
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-15 14:47:54 +03:00
0550438b53 [#167] netmap/tests: Add replica to invalid tests
Make sure we fail exactly because of the reason specified.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-15 14:33:44 +03:00
c899163860 [#167] netmap/tests: Add json file name to the test output
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-15 14:33:44 +03:00
ac8fc6d440 [#162] netmap: Allow to parse single unnamed selectors
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-11 15:22:24 +03:00
0a0b590df3 [#162] Fix pre-commit warnings
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-11 15:21:35 +03:00
4df642e941 [#162] netmap: Fix possible panic
Placement policy is unvalidated external input.
Under no circumstances should we panic here.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-11 15:19:56 +03:00
8bc64e088e [#161] .golangci.yml: Reenable deprecated usage warnings
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-08 17:17:02 +03:00
49ad985cad [#161] *: Do not use math/rand.Read()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-08 17:17:02 +03:00
aa12d8c6a6 [#121] client: Make PrmObjectHash fields public
* Introduce buildRequest for PrmObjectHash

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-08 13:36:45 +00:00
303508328a [#121] pool: Refactor PrmSessionCreate usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-05 17:32:03 +03:00
55699d1480 [#121] client: Make PrmSessionCreate fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-05 17:30:42 +03:00
55a1f23e71 [#121] client: Make PrmEndpointInfo, PrmNetworkInfo fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-04 19:55:23 +03:00
291a71ba84 [#121] client: Make PrmAnnounceSpace fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-04 19:53:34 +03:00
5a471e5002 [#121] client: Make PrmObjectDelete fields public
* Introduce buildRequest for PrmObjectDelete
* Refactor the usage of these params in pool

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-04 14:14:22 +00:00
b5fe52d6bd [#150] policy: Check for redundant selectors and filters
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-29 14:16:57 +03:00
84e7e69f98 [#121] client: Make PrmObjectGet/Head/GetRange fields public
* Remove common PrmObjectRead structure
* Introduce buildRequest for PrmObjectGet/Head/GetRange
* Refactor the usage of these params in pool

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-28 11:26:57 +03:00
46a214d065 [#149] pool: Configure homomorphic hash and buffer size
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-25 09:45:15 +03:00
202412230a [#115] pool: Drop part buffer pool
Tests showed that using part buffer pool doesn't save memory a lot.
Especially on big parts.
Probably we can use pool only for small parts
after adding buffer in payloadSizeLimiter

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-21 12:03:03 +03:00
3cb3841073 [#115] pool: Try putSingle if possible
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-21 12:03:03 +03:00
faeeeab87a [#114] pool: Don't use part buffers when client cut is off
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-21 12:02:40 +03:00
cae215534f [#114] pool: Fix linter errors
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-21 12:02:40 +03:00
518fb79bc0 [#114] pool: Support client cut with memory limiter
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-21 12:02:40 +03:00
342524159a [#121] pool: Make PrmContainerSetEACL fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-21 10:33:19 +03:00
22978303f8 [#121] clientt: Make PrmContainerSetEACL fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-18 18:05:47 +03:00
6fdbe75517 [#121] pool: Make PrmContainerEACL fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-16 09:53:47 +00:00
3353940554 [#121] client: Make PrmContainerEACL fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-16 09:53:47 +00:00
a3b5d4d4f5 [#51] Add node addresses as debug information
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-08-15 09:51:40 +03:00
0314b326d3 [#51] Add current nodes as external statistics
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-08-15 09:51:13 +03:00
0382785763 [#146] .forgejo: Update DCO action
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-11 12:34:29 +03:00
548a81d3e6 [#48] client: Refactor accounting.Balance()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-10 11:05:30 +00:00
d48788c7a9 [#144] Bump required go version to go1.20
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-09 09:52:35 +03:00
6353df8bca [#142] Fix unwrapErr for go 1.20
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-05 15:58:25 +00:00
936e6d230b [#121] pool: Add wait params validation for containerPut method
* Add WaitParams.CheckValidity() check because SetWaitParams is deprecated,
  but parameters were checked within this setter with checkForPositive()

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-05 15:58:14 +00:00
be28b89312 [#121] pool: Make PrmContainerDelete fields public
* Refactor client PrmContainerDelete usage
* Introduce WaitParams CheckValidity method

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-05 15:58:14 +00:00
9e5faaf829 [#121] client: Make PrmContainerDelete fields public
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-05 15:58:14 +00:00
3dc8129ed7 [#135] Make all error status receivers pointers
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-04 08:35:01 +00:00
55c52c8d5d [#121] pool: Make PrmContainerGet fields public
* Also refactor client PrmContainerGet usage

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-03 10:54:13 +03:00
d376302a3b [#121] client: Make PrmContainerGet fields public
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-02 18:23:32 +03:00
363f153eaf [#136] pool: Set order field to get subtree
With new revision of tree service protocol, getSubTree
requires to use explicit order field.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-08-02 10:32:37 +00:00
95b987b818 [#137] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-02 12:04:10 +03:00
13d0b170d2 [#121] client: Make pool PrmContainerPut struct fields public
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-01 09:59:57 +00:00
18a9e4bceb [#121] client: Make PrmContainerPut struct fields public
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-01 09:59:57 +00:00
0fe0d71678 [#133] .forgejo: Add names to actions
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-31 11:33:39 +00:00
78d1439b2c [#133] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-31 11:33:39 +00:00
0886d80083 [#69] Add close for nns.Dial
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-07-31 11:05:26 +03:00
ecb1fef78c [#129] client: Do not override error status WriteObject()
Here is a scenario:
1. `resolveFrostFSErrors` is false.
2. The first object part was not written, which was signified in status.
3. The second part was written correctly.

Client now thinks that the object is written even though it was not.
In theory we could also return only status, but client-side splitting
is not a single RPC, so it makes sense.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-26 14:38:43 +03:00
5defed4ab4 [#126] go.mod: Update frostfs-api-go package version
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-07-19 16:03:56 +03:00
fb05f7dc5e [#112] Add basic documentation for placement policies
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-18 10:53:54 +00:00
b91f9d8c79 [#xx] Add support for SELECT-FILTER expressions
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-17 10:29:48 +00:00
b9afe7a2f9 [#42] Add ResolveContractHash method
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
2023-07-12 22:22:58 +03:00
998fe1a7ab [#102] netmap: properly process multiple REP
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-11 14:21:35 +00:00
c359a7465a [#64] transformer: Simplify interface
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-11 07:33:12 +00:00
d70ef2187b [#97] Add a method IterateUserAttributes in Container
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-07-10 09:05:01 +00:00
ac95b87e7c [#101] Add Equals for Address
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-07-10 09:05:01 +00:00
863be6034f [#104] Update neo-go/pkg/interop version
neo-go module uses broken commit of interop package.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-10 09:04:10 +00:00
35346a01c9 [#109] Bump neo-go version
Synced version with frostfs-node, see frostfs-node#417

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-10 09:04:10 +00:00
fe35373d8f [#107] go.mod: Tidy dependencies
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-07 14:57:16 +03:00
388d1ca1de [#107] pool/tree: Support grpc schemas
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-07 14:57:13 +03:00
14ed3e177d [#104] nemtap: Escape special symbols in filters
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-07 09:33:00 +00:00
fe28c33277 [#104] netmap: Add test with quote escaping
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-07 09:33:00 +00:00
98cab7ed61 [#103] object: Add PutSingle method code
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-06 17:06:17 +03:00
37e22b33ad [#103] sdk-go: Update api-go version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-05 18:10:42 +03:00
769f6eec05 [#105] client: Fix revive linter warning
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-05 15:52:06 +03:00
5d62cef27e [#98] Fix linter issues
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-28 12:13:02 +00:00
c0c0c588b5 [#98] Add forgejo workflows
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-28 12:13:02 +00:00
2f88460172 [#83] Allow to split objects in the client
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-28 12:12:37 +00:00
66cb5dcf34 [#83] Update .gitignore
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-28 12:12:37 +00:00
91e80ba743 [#73] pool/tree: Fix index in retry loop
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-28 13:06:23 +03:00
c243b443bc [#80] objectwriter: Allow custimize maxChunkLen
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-26 18:13:04 +03:00
aa8ffebc63 [#95] transformer: Set parent version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-23 13:48:02 +03:00
9d40228cec [#73] pool/tree: Fix client healthy status
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 17:01:55 +03:00
af40dc68f0 [#84] pool/tree: Allow to pass gRPC dial options
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 15:55:07 +03:00
981d24a493 [#84] pool: Allow to pass gRPC dial options
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 15:54:48 +03:00
19adb4dffa [#73] pool/tree: Add tests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 14:37:57 +03:00
51e022ab8c [#73] pool/tree: Add tree pool
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 14:37:57 +03:00
0d3dacb515 [#73] pool: Add getters for NodeParam
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 14:37:57 +03:00
b2e302624d [#73] pool/tree: Add proto tree service client
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 14:37:57 +03:00
fcbf96add6 [#76] Add UNIQUE keyword
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-06-06 13:54:07 +03:00
4f48f6c9e0 [#78] netmap: Add new keywords NOT and UNIQUE
* Add the rule for NOT operation to the policy parser grammar
* Regenerate query parse
* Implement NOT in filter
* Add unit-tests

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-06-02 17:47:20 +03:00
ec59ebfd88 [#87] go.mod: Update hrw
```
                                                              │      old      │                 new                  │
                                                              │    sec/op     │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                    13.07µ ± 12%   11.03µ ± 12%  -15.63% (p=0.003 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   11.383µ ±  9%   9.970µ ± 19%  -12.42% (p=0.005 n=10)
geomean                                                          12.20µ         10.49µ        -14.04%

                                                              │      old      │                 new                  │
                                                              │     B/op      │     B/op      vs base                │
Netmap_ContainerNodes/REP_2-8                                   10.203Ki ± 0%   7.711Ki ± 0%  -24.43% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8    9.641Ki ± 0%   7.148Ki ± 0%  -25.85% (p=0.000 n=10)
geomean                                                          9.918Ki        7.424Ki       -25.14%

                                                              │    old     │                new                 │
                                                              │ allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   216.0 ± 0%   131.0 ± 0%  -39.35% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   215.0 ± 0%   130.0 ± 0%  -39.53% (p=0.000 n=10)
geomean                                                         215.5        130.5       -39.44%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-02 14:54:40 +03:00
030ff2f122 [#87] netmap: Add benchmark for ContainerNodes()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-02 14:54:40 +03:00
0f7455ff7a [#75] Update antlr4 version to 4.13.0
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-01 13:15:11 +00:00
e6b662cfa6 [#75] Add make targets policy and docker/%
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-01 13:15:11 +00:00
406c2324d4 [#85] go.mod: Tidy
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-01 10:02:23 +03:00
10482ffbed [#82] client: Allow to pass gRPC dial options
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-30 16:51:22 +03:00
f5b23eb225 [#74] netmap/parser: Update antlr4 generator
Also, add -no-listener option, as we use visitor only.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 17:47:24 +03:00
70f23dd1ea [#74] pre-commit: Exclude auto-generated files
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 17:47:24 +03:00
57f874048b [#74] go.mod: Update antlr4 dependency
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 17:47:24 +03:00
a397d1fd15 [#74] go.mod: Update dependencies
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 17:47:23 +03:00
9803c2816a [#74] pool: move to sync/atomic
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 14:58:30 +03:00
d04d96b42e [#74] go.mod: Move to go1.19
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 14:53:47 +03:00
9a072a8f49 [#68] Replace interface{} with any
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-15 17:21:49 +03:00
15b4287092 [#49] bearer: Allow empty eacl if token is impersonated
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-05 12:45:39 +03:00
d4fe9a193d [#66] transformer: Accept constructor in NextTarget
The code of frostfs-node is not yet ready to reuse egress target for
multiple objects, let's postpone until #64.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 11:22:09 +03:00
c42a6119ff [#66] transformer: Extend tests
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 11:22:09 +03:00
29b188db57 [#52] Remove storage groups and audit
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 08:20:37 +00:00
38b03ff28b [#63] transformer: Resolve funlen linter
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-25 13:29:42 +03:00
0fa23a9b14 [#63] transformer: Add context to methods
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-25 13:29:42 +03:00
d0762d037d [#44] pool: Add copies number vector when putting object
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-04-20 13:41:05 +03:00
db5b89496d [#49] bearer: Consider impersonate in fillBody
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-18 14:54:05 +00:00
7c75db2f2d [#59] netmap: Remove unused param from getSelection()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 14:11:04 +00:00
dce55a436a [#9999] sdk: Up api-go version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-18 09:59:52 +00:00
cae2f37cdd client: rename formRequest() to buildRequest()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 09:58:34 +00:00
f60bea4be5 [#48] client: Refactor SessionCreate()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 09:58:34 +00:00
a16fc40c39 [#48] client: Refactor EndpointInfo()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 09:58:34 +00:00
40d966bec2 [#48] client: Refactor NetworkInfo()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 09:58:34 +00:00
Pavel Karpy
d0c5d837d2 [#56] *: Drop reputation system
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-18 07:52:16 +00:00
237b90f744 [#49] bearer: Add impersonate flag
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-17 11:51:38 +03:00
c8e620ad24 [#53] sdk-go: Drop subnet
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-14 16:31:08 +03:00
591dd1247d [#48] client: Refactor ContainerPut()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
57619fbbe4 [#48] client: Refactor ContainerSetEACL()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
8bc8f1f365 [#48] client: Refactor ContainerDelete()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
09ed2863fc [#48] client: Refactor ContainerEACL()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
8e2f77890f [#48] client: Refactor ContainerGet()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
8852b262f2 [#48] client: Refactor ContainerList()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
b2c66cb99e [#48] client: Refactor ContainerAnnounceUsedSpace()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
6c9b92c9dc [#48] client: Execute metainfo callback in processResponse()
It was lost in 8c5333ea55.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
772fa90983 [#48] client/container: Form requests in a separate function
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:06:52 +03:00
1bfa9ecdb0 [#48] client: Remove ctx == nil checks
Much less useful after we made context to be passed explicitly.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 09:04:09 +03:00
55b06cd764 [#48] client: Split container methods by files
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 09:04:09 +03:00
161 changed files with 11546 additions and 7179 deletions

View file

@ -0,0 +1,21 @@
name: DCO
on: [pull_request]
jobs:
dco:
name: DCO
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Setup Go
uses: actions/setup-go@v3
with:
go-version: '1.21'
- name: Run commit format checker
uses: https://git.frostfs.info/TrueCloudLab/dco-go@v2
with:
from: 'origin/${{ github.event.pull_request.base.ref }}'

View file

@ -0,0 +1,32 @@
name: Tests and linters
on: [pull_request]
jobs:
lint:
name: Lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: golangci-lint
uses: https://github.com/golangci/golangci-lint-action@v2
with:
version: latest
tests:
name: Tests
runs-on: ubuntu-latest
strategy:
matrix:
go_versions: [ '1.19', '1.20' ]
fail-fast: false
steps:
- uses: actions/checkout@v3
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: '${{ matrix.go_versions }}'
- name: Run tests
run: make test

View file

@ -1,21 +0,0 @@
name: DCO check
on:
pull_request:
branches:
- master
jobs:
commits_check_job:
runs-on: ubuntu-latest
name: Commits Check
steps:
- name: Get PR Commits
id: 'get-pr-commits'
uses: tim-actions/get-pr-commits@master
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: DCO Check
uses: tim-actions/dco@master
with:
commits: ${{ steps.get-pr-commits.outputs.commits }}

View file

@ -1,52 +0,0 @@
name: neofs-sdk-go tests
on:
pull_request:
branches:
- master
types: [opened, synchronize]
paths-ignore:
- '**/*.md'
workflow_dispatch:
jobs:
tests:
name: Tests
runs-on: ubuntu-20.04
strategy:
matrix:
go_versions: [ '1.18.x', '1.19.x', '1.20.x' ]
fail-fast: false
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: '${{ matrix.go_versions }}'
- name: Restore Go modules from cache
uses: actions/cache@v3
with:
path: /home/runner/go/pkg/mod
key: deps-${{ hashFiles('go.sum') }}
- name: Update Go modules
run: make dep
- name: Run tests
run: make test
lint:
runs-on: ubuntu-20.04
steps:
- name: Check out code
uses: actions/checkout@v3
- name: golangci-lint
uses: golangci/golangci-lint-action@v3
with:
version: latest
only-new-issues: true

6
.gitignore vendored
View file

@ -21,3 +21,9 @@ vendor/
# coverage
coverage.txt
coverage.html
# antlr tool jar
antlr-*.jar
# tempfiles
.cache

View file

@ -25,7 +25,7 @@ linters-settings:
# report about shadowed variables
check-shadowing: false
staticcheck:
checks: ["all", "-SA1019"] # TODO Enable SA1019 after deprecated warning are fixed.
checks: ["all"]
funlen:
lines: 80 # default 60
statements: 60 # default 40

View file

@ -16,7 +16,7 @@ repos:
- id: trailing-whitespace
args: [--markdown-linebreak-ext=md]
- id: end-of-file-fixer
exclude: ".key$"
exclude: "(.key|.interp|.tokens)$"
- repo: https://github.com/golangci/golangci-lint
rev: v1.51.2

4
Dockerfile Normal file
View file

@ -0,0 +1,4 @@
FROM golang:1.21
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install make openjdk-11-jre -y
WORKDIR /work

View file

@ -1,5 +1,7 @@
#!/usr/bin/make -f
ANTLR_VERSION="4.13.0"
# Run tests
test:
@go test ./... -cover
@ -29,6 +31,23 @@ format:
@echo "⇒ Processing goimports check"
@goimports -w ./
policy:
@wget -q https://www.antlr.org/download/antlr-${ANTLR_VERSION}-complete.jar -O antlr4-tool.jar
@java -Xmx500M -cp "`pwd`/antlr4-tool.jar" "org.antlr.v4.Tool" -o `pwd`/netmap/parser/ -Dlanguage=Go -no-listener -visitor `pwd`/netmap/parser/Query.g4 `pwd`/netmap/parser/QueryLexer.g4
# Run `make %` in truecloudlab/frostfs-sdk-go container(Golang+Java)
docker/%:
@docker build -t truecloudlab/frostfs-sdk-go --platform linux/amd64 . > /dev/null
@docker run --rm -t \
-v `pwd`:/work \
-u "$$(id -u):$$(id -g)" \
--env HOME=/work \
truecloudlab/frostfs-sdk-go make $*
# Synchronize tree service
sync-tree:
@./syncTree.sh
# Show this help prompt
help:
@echo ' Usage:'

View file

@ -1,26 +0,0 @@
/*
Package audit provides features to process data audit in FrostFS system.
Result type groups values which can be gathered during data audit process:
var res audit.Result
res.ForEpoch(32)
res.ForContainer(cnr)
// ...
res.Complete()
Result instances can be stored in a binary format. On reporter side:
data := res.Marshal()
// send data
On receiver side:
var res audit.Result
err := res.Unmarshal(data)
// ...
Using package types in an application is recommended to potentially work with
different protocol versions with which these types are compatible.
*/
package audit

View file

@ -1,377 +0,0 @@
package audit
import (
"errors"
"fmt"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/audit"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/version"
)
// Result represents report on the results of the data audit in FrostFS system.
//
// Result is mutually binary-compatible with git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/audit.DataAuditResult
// message. See Marshal / Unmarshal methods.
//
// Instances can be created using built-in var declaration.
type Result struct {
versionEncoded bool
v2 audit.DataAuditResult
}
// Marshal encodes Result into a canonical FrostFS binary format (Protocol Buffers
// with direct field order).
//
// Writes version.Current() protocol version into the resulting message if Result
// hasn't been already decoded from such a message using Unmarshal.
//
// See also Unmarshal.
func (r *Result) Marshal() []byte {
if !r.versionEncoded {
var verV2 refs.Version
version.Current().WriteToV2(&verV2)
r.v2.SetVersion(&verV2)
r.versionEncoded = true
}
return r.v2.StableMarshal(nil)
}
var errCIDNotSet = errors.New("container ID is not set")
// Unmarshal decodes Result from its canonical FrostFS binary format (Protocol Buffers
// with direct field order). Returns an error describing a format violation.
//
// See also Marshal.
func (r *Result) Unmarshal(data []byte) error {
err := r.v2.Unmarshal(data)
if err != nil {
return err
}
r.versionEncoded = true
// format checks
var cID cid.ID
cidV2 := r.v2.GetContainerID()
if cidV2 == nil {
return errCIDNotSet
}
err = cID.ReadFromV2(*cidV2)
if err != nil {
return fmt.Errorf("could not convert V2 container ID: %w", err)
}
var (
oID oid.ID
oidV2 refs.ObjectID
)
for _, oidV2 = range r.v2.GetPassSG() {
err = oID.ReadFromV2(oidV2)
if err != nil {
return fmt.Errorf("invalid passed storage group ID: %w", err)
}
}
for _, oidV2 = range r.v2.GetFailSG() {
err = oID.ReadFromV2(oidV2)
if err != nil {
return fmt.Errorf("invalid failed storage group ID: %w", err)
}
}
return nil
}
// Epoch returns FrostFS epoch when the data associated with the Result was audited.
//
// Zero Result has zero epoch.
//
// See also ForEpoch.
func (r Result) Epoch() uint64 {
return r.v2.GetAuditEpoch()
}
// ForEpoch specifies FrostFS epoch when the data associated with the Result was audited.
//
// See also Epoch.
func (r *Result) ForEpoch(epoch uint64) {
r.v2.SetAuditEpoch(epoch)
}
// Container returns identifier of the container with which the data audit Result
// is associated and a bool that indicates container ID field presence in the Result.
//
// Zero Result does not have container ID.
//
// See also ForContainer.
func (r Result) Container() (cid.ID, bool) {
var cID cid.ID
cidV2 := r.v2.GetContainerID()
if cidV2 != nil {
_ = cID.ReadFromV2(*cidV2)
return cID, true
}
return cID, false
}
// ForContainer sets identifier of the container with which the data audit Result
// is associated.
//
// See also Container.
func (r *Result) ForContainer(cnr cid.ID) {
var cidV2 refs.ContainerID
cnr.WriteToV2(&cidV2)
r.v2.SetContainerID(&cidV2)
}
// AuditorKey returns public key of the auditing FrostFS Inner Ring node in
// a FrostFS binary key format.
//
// Zero Result has nil key. Return value MUST NOT be mutated: to do this,
// first make a copy.
//
// See also SetAuditorPublicKey.
func (r Result) AuditorKey() []byte {
return r.v2.GetPublicKey()
}
// SetAuditorKey specifies public key of the auditing FrostFS Inner Ring node in
// a FrostFS binary key format.
//
// Argument MUST NOT be mutated at least until the end of using the Result.
//
// See also AuditorKey.
func (r *Result) SetAuditorKey(key []byte) {
r.v2.SetPublicKey(key)
}
// Completed returns completion state of the data audit associated with the Result.
//
// Zero Result corresponds to incomplete data audit.
//
// See also Complete.
func (r Result) Completed() bool {
return r.v2.GetComplete()
}
// Complete marks the data audit associated with the Result as completed.
//
// See also Completed.
func (r *Result) Complete() {
r.v2.SetComplete(true)
}
// RequestsPoR returns number of requests made by Proof-of-Retrievability
// audit check to get all headers of the objects inside storage groups.
//
// Zero Result has zero requests.
//
// See also SetRequestsPoR.
func (r Result) RequestsPoR() uint32 {
return r.v2.GetRequests()
}
// SetRequestsPoR sets number of requests made by Proof-of-Retrievability
// audit check to get all headers of the objects inside storage groups.
//
// See also RequestsPoR.
func (r *Result) SetRequestsPoR(v uint32) {
r.v2.SetRequests(v)
}
// RetriesPoR returns number of retries made by Proof-of-Retrievability
// audit check to get all headers of the objects inside storage groups.
//
// Zero Result has zero retries.
//
// See also SetRetriesPoR.
func (r Result) RetriesPoR() uint32 {
return r.v2.GetRetries()
}
// SetRetriesPoR sets number of retries made by Proof-of-Retrievability
// audit check to get all headers of the objects inside storage groups.
//
// See also RetriesPoR.
func (r *Result) SetRetriesPoR(v uint32) {
r.v2.SetRetries(v)
}
// IteratePassedStorageGroups iterates over all storage groups that passed
// Proof-of-Retrievability audit check and passes them into f. Breaks on f's
// false return, f MUST NOT be nil.
//
// Zero Result has no passed storage groups and doesn't call f.
//
// See also SubmitPassedStorageGroup.
func (r Result) IteratePassedStorageGroups(f func(oid.ID) bool) {
r2 := r.v2.GetPassSG()
var id oid.ID
for i := range r2 {
_ = id.ReadFromV2(r2[i])
if !f(id) {
return
}
}
}
// SubmitPassedStorageGroup marks storage group as passed Proof-of-Retrievability
// audit check.
//
// See also IteratePassedStorageGroups.
func (r *Result) SubmitPassedStorageGroup(sg oid.ID) {
var idV2 refs.ObjectID
sg.WriteToV2(&idV2)
r.v2.SetPassSG(append(r.v2.GetPassSG(), idV2))
}
// IterateFailedStorageGroups is similar to IteratePassedStorageGroups but for failed groups.
//
// See also SubmitFailedStorageGroup.
func (r Result) IterateFailedStorageGroups(f func(oid.ID) bool) {
v := r.v2.GetFailSG()
var id oid.ID
for i := range v {
_ = id.ReadFromV2(v[i])
if !f(id) {
return
}
}
}
// SubmitFailedStorageGroup is similar to SubmitPassedStorageGroup but for failed groups.
//
// See also IterateFailedStorageGroups.
func (r *Result) SubmitFailedStorageGroup(sg oid.ID) {
var idV2 refs.ObjectID
sg.WriteToV2(&idV2)
r.v2.SetFailSG(append(r.v2.GetFailSG(), idV2))
}
// Hits returns number of sampled objects under audit placed
// in an optimal way according to the container's placement policy
// when checking Proof-of-Placement.
//
// Zero result has zero hits.
//
// See also SetHits.
func (r Result) Hits() uint32 {
return r.v2.GetHit()
}
// SetHits sets number of sampled objects under audit placed
// in an optimal way according to the containers placement policy
// when checking Proof-of-Placement.
//
// See also Hits.
func (r *Result) SetHits(hit uint32) {
r.v2.SetHit(hit)
}
// Misses returns number of sampled objects under audit placed
// in suboptimal way according to the container's placement policy,
// but still at a satisfactory level when checking Proof-of-Placement.
//
// Zero Result has zero misses.
//
// See also SetMisses.
func (r Result) Misses() uint32 {
return r.v2.GetMiss()
}
// SetMisses sets number of sampled objects under audit placed
// in suboptimal way according to the container's placement policy,
// but still at a satisfactory level when checking Proof-of-Placement.
//
// See also Misses.
func (r *Result) SetMisses(miss uint32) {
r.v2.SetMiss(miss)
}
// Failures returns number of sampled objects under audit stored
// in a way not confirming placement policy or not found at all
// when checking Proof-of-Placement.
//
// Zero result has zero failures.
//
// See also SetFailures.
func (r Result) Failures() uint32 {
return r.v2.GetFail()
}
// SetFailures sets number of sampled objects under audit stored
// in a way not confirming placement policy or not found at all
// when checking Proof-of-Placement.
//
// See also Failures.
func (r *Result) SetFailures(fail uint32) {
r.v2.SetFail(fail)
}
// IteratePassedStorageNodes iterates over all storage nodes that passed at least one
// Proof-of-Data-Possession audit check and passes their public keys into f. Breaks on
// f's false return.
//
// f MUST NOT be nil and MUST NOT mutate parameter passed into it at least until
// the end of using the Result.
//
// Zero Result has no passed storage nodes and doesn't call f.
//
// See also SubmitPassedStorageNode.
func (r Result) IteratePassedStorageNodes(f func([]byte) bool) {
v := r.v2.GetPassNodes()
for i := range v {
if !f(v[i]) {
return
}
}
}
// SubmitPassedStorageNodes marks storage node list as passed Proof-of-Data-Possession
// audit check. The list contains public keys.
//
// Argument and its elements MUST NOT be mutated at least until the end of using the Result.
//
// See also IteratePassedStorageNodes.
func (r *Result) SubmitPassedStorageNodes(list [][]byte) {
r.v2.SetPassNodes(list)
}
// IterateFailedStorageNodes is similar to IteratePassedStorageNodes but for failed nodes.
//
// See also SubmitPassedStorageNodes.
func (r Result) IterateFailedStorageNodes(f func([]byte) bool) {
v := r.v2.GetFailNodes()
for i := range v {
if !f(v[i]) {
return
}
}
}
// SubmitFailedStorageNodes is similar to SubmitPassedStorageNodes but for failed nodes.
//
// See also IterateFailedStorageNodes.
func (r *Result) SubmitFailedStorageNodes(list [][]byte) {
r.v2.SetFailNodes(list)
}

View file

@ -1,191 +0,0 @@
package audit_test
import (
"bytes"
"testing"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/audit"
audittest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/audit/test"
cidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id/test"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
oidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id/test"
"github.com/stretchr/testify/require"
)
func TestResultData(t *testing.T) {
var r audit.Result
countSG := func(passed bool, f func(oid.ID)) int {
called := 0
ff := func(arg oid.ID) bool {
called++
if f != nil {
f(arg)
}
return true
}
if passed {
r.IteratePassedStorageGroups(ff)
} else {
r.IterateFailedStorageGroups(ff)
}
return called
}
countPassSG := func(f func(oid.ID)) int { return countSG(true, f) }
countFailSG := func(f func(oid.ID)) int { return countSG(false, f) }
countNodes := func(passed bool, f func([]byte)) int {
called := 0
ff := func(arg []byte) bool {
called++
if f != nil {
f(arg)
}
return true
}
if passed {
r.IteratePassedStorageNodes(ff)
} else {
r.IterateFailedStorageNodes(ff)
}
return called
}
countPassNodes := func(f func([]byte)) int { return countNodes(true, f) }
countFailNodes := func(f func([]byte)) int { return countNodes(false, f) }
require.Zero(t, r.Epoch())
_, set := r.Container()
require.False(t, set)
require.Nil(t, r.AuditorKey())
require.False(t, r.Completed())
require.Zero(t, r.RequestsPoR())
require.Zero(t, r.RetriesPoR())
require.Zero(t, countPassSG(nil))
require.Zero(t, countFailSG(nil))
require.Zero(t, countPassNodes(nil))
require.Zero(t, countFailNodes(nil))
epoch := uint64(13)
r.ForEpoch(epoch)
require.Equal(t, epoch, r.Epoch())
cnr := cidtest.ID()
r.ForContainer(cnr)
cID, set := r.Container()
require.True(t, set)
require.Equal(t, cnr, cID)
key := []byte{1, 2, 3}
r.SetAuditorKey(key)
require.Equal(t, key, r.AuditorKey())
r.Complete()
require.True(t, r.Completed())
requests := uint32(2)
r.SetRequestsPoR(requests)
require.Equal(t, requests, r.RequestsPoR())
retries := uint32(1)
r.SetRetriesPoR(retries)
require.Equal(t, retries, r.RetriesPoR())
passSG1, passSG2 := oidtest.ID(), oidtest.ID()
r.SubmitPassedStorageGroup(passSG1)
r.SubmitPassedStorageGroup(passSG2)
called1, called2 := false, false
require.EqualValues(t, 2, countPassSG(func(id oid.ID) {
if id.Equals(passSG1) {
called1 = true
} else if id.Equals(passSG2) {
called2 = true
}
}))
require.True(t, called1)
require.True(t, called2)
failSG1, failSG2 := oidtest.ID(), oidtest.ID()
r.SubmitFailedStorageGroup(failSG1)
r.SubmitFailedStorageGroup(failSG2)
called1, called2 = false, false
require.EqualValues(t, 2, countFailSG(func(id oid.ID) {
if id.Equals(failSG1) {
called1 = true
} else if id.Equals(failSG2) {
called2 = true
}
}))
require.True(t, called1)
require.True(t, called2)
hit := uint32(1)
r.SetHits(hit)
require.Equal(t, hit, r.Hits())
miss := uint32(2)
r.SetMisses(miss)
require.Equal(t, miss, r.Misses())
fail := uint32(3)
r.SetFailures(fail)
require.Equal(t, fail, r.Failures())
passNodes := [][]byte{{1}, {2}}
r.SubmitPassedStorageNodes(passNodes)
called1, called2 = false, false
require.EqualValues(t, 2, countPassNodes(func(arg []byte) {
if bytes.Equal(arg, passNodes[0]) {
called1 = true
} else if bytes.Equal(arg, passNodes[1]) {
called2 = true
}
}))
require.True(t, called1)
require.True(t, called2)
failNodes := [][]byte{{3}, {4}}
r.SubmitFailedStorageNodes(failNodes)
called1, called2 = false, false
require.EqualValues(t, 2, countFailNodes(func(arg []byte) {
if bytes.Equal(arg, failNodes[0]) {
called1 = true
} else if bytes.Equal(arg, failNodes[1]) {
called2 = true
}
}))
require.True(t, called1)
require.True(t, called2)
}
func TestResultEncoding(t *testing.T) {
r := *audittest.Result()
t.Run("binary", func(t *testing.T) {
data := r.Marshal()
var r2 audit.Result
require.NoError(t, r2.Unmarshal(data))
require.Equal(t, r, r2)
})
}

View file

@ -1,13 +0,0 @@
/*
Package audittest provides functions for convenient testing of audit package API.
Note that importing the package into source files is highly discouraged.
Random instance generation functions can be useful when testing expects any value, e.g.:
import audittest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/audit/test"
dec := audittest.Result()
// test the value
*/
package audittest

View file

@ -1,36 +0,0 @@
package audittest
import (
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/audit"
cidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id/test"
oidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id/test"
)
// Result returns random audit.Result.
func Result() *audit.Result {
var x audit.Result
x.ForContainer(cidtest.ID())
x.SetAuditorKey([]byte("key"))
x.Complete()
x.ForEpoch(44)
x.SetHits(55)
x.SetMisses(66)
x.SetFailures(77)
x.SetRequestsPoR(88)
x.SetRequestsPoR(99)
x.SubmitFailedStorageNodes([][]byte{
[]byte("node1"),
[]byte("node2"),
})
x.SubmitPassedStorageNodes([][]byte{
[]byte("node3"),
[]byte("node4"),
})
x.SubmitPassedStorageGroup(oidtest.ID())
x.SubmitPassedStorageGroup(oidtest.ID())
x.SubmitFailedStorageGroup(oidtest.ID())
x.SubmitFailedStorageGroup(oidtest.ID())
return &x
}

View file

@ -32,6 +32,8 @@ type Token struct {
sigSet bool
sig refs.Signature
impersonate bool
}
// reads Token from the acl.BearerToken message. If checkFieldPresence is set,
@ -44,10 +46,12 @@ func (b *Token) readFromV2(m acl.BearerToken, checkFieldPresence bool) error {
return errors.New("missing token body")
}
b.impersonate = body.GetImpersonate()
eaclTable := body.GetEACL()
if b.eaclTableSet = eaclTable != nil; b.eaclTableSet {
b.eaclTable = *eacl.NewTableFromV2(eaclTable)
} else if checkFieldPresence {
} else if checkFieldPresence && !b.impersonate {
return errors.New("missing eACL table")
}
@ -86,7 +90,7 @@ func (b *Token) ReadFromV2(m acl.BearerToken) error {
}
func (b Token) fillBody() *acl.BearerTokenBody {
if !b.eaclTableSet && !b.targetUserSet && !b.lifetimeSet {
if !b.eaclTableSet && !b.targetUserSet && !b.lifetimeSet && !b.impersonate {
return nil
}
@ -112,6 +116,8 @@ func (b Token) fillBody() *acl.BearerTokenBody {
body.SetLifetime(&lifetime)
}
body.SetImpersonate(b.impersonate)
return &body
}
@ -208,6 +214,17 @@ func (b Token) EACLTable() eacl.Table {
return eacl.Table{}
}
// SetImpersonate mark token as impersonate to consider token signer as request owner.
// If this field is true extended EACLTable in token body isn't processed.
func (b *Token) SetImpersonate(v bool) {
b.impersonate = v
}
// Impersonate returns true if token is impersonated.
func (b Token) Impersonate() bool {
return b.impersonate
}
// AssertContainer checks if the token is valid within the given container.
//
// Note: cnr is assumed to refer to the issuer's container, otherwise the check

View file

@ -323,6 +323,10 @@ func TestToken_ReadFromV2(t *testing.T) {
require.NoError(t, val.ReadFromV2(m))
body.SetEACL(nil)
body.SetImpersonate(true)
require.NoError(t, val.ReadFromV2(m))
var m2 acl.BearerToken
val.WriteToV2(&m2)

View file

@ -24,17 +24,17 @@ type Checksum refs.Checksum
// Type represents the enumeration
// of checksum types.
type Type uint8
type Type refs.ChecksumType
const (
// Unknown is an undefined checksum type.
Unknown Type = iota
Unknown Type = Type(refs.UnknownChecksum)
// SHA256 is a SHA256 checksum type.
SHA256
SHA256 = Type(refs.SHA256)
// TZ is a Tillich-Zémor checksum type.
TZ
TZ = Type(refs.TillichZemor)
)
// ReadFromV2 reads Checksum from the refs.Checksum message. Checks if the

View file

@ -2,9 +2,9 @@ package checksum
import (
"bytes"
"crypto/rand"
"crypto/sha256"
"fmt"
"math/rand"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
)

View file

@ -1,8 +1,8 @@
package checksumtest
import (
"crypto/rand"
"crypto/sha256"
"math/rand"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/checksum"
)
@ -11,7 +11,7 @@ import (
func Checksum() checksum.Checksum {
var cs [sha256.Size]byte
rand.Read(cs[:])
_, _ = rand.Read(cs[:])
var x checksum.Checksum

View file

@ -2,12 +2,16 @@ package client
import (
"context"
"fmt"
v2accounting "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/accounting"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/accounting"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
)
@ -26,6 +30,24 @@ func (x *PrmBalanceGet) SetAccount(id user.ID) {
x.accountSet = true
}
func (x *PrmBalanceGet) buildRequest(c *Client) (*v2accounting.BalanceRequest, error) {
if !x.accountSet {
return nil, errorAccountNotSet
}
var accountV2 refs.OwnerID
x.account.WriteToV2(&accountV2)
var body v2accounting.BalanceRequestBody
body.SetOwnerID(&accountV2)
var req v2accounting.BalanceRequest
req.SetBody(&body)
c.prepareRequest(&req, new(v2session.RequestMetaHeader))
return &req, nil
}
// ResBalanceGet groups resulting values of BalanceGet operation.
type ResBalanceGet struct {
statusRes
@ -52,60 +74,35 @@ func (x ResBalanceGet) Amount() accounting.Decimal {
// Return statuses:
// - global (see Client docs).
func (c *Client) BalanceGet(ctx context.Context, prm PrmBalanceGet) (*ResBalanceGet, error) {
switch {
case ctx == nil:
return nil, errorMissingContext
case !prm.accountSet:
return nil, errorAccountNotSet
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
// form request body
var accountV2 refs.OwnerID
prm.account.WriteToV2(&accountV2)
var body v2accounting.BalanceRequestBody
body.SetOwnerID(&accountV2)
// form request
var req v2accounting.BalanceRequest
req.SetBody(&body)
// init call context
var (
cc contextCall
res ResBalanceGet
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.Balance(&c.c, &req, client.WithContext(ctx))
if err := signature.SignServiceMessage(&c.prm.key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.Balance(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResBalanceGet
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
cc.result = func(r responseV2) {
resp := r.(*v2accounting.BalanceResponse)
const fieldBalance = "balance"
bal := resp.GetBody().GetBalance()
if bal == nil {
cc.err = newErrMissingResponseField(fieldBalance)
return
return &res, newErrMissingResponseField(fieldBalance)
}
cc.err = res.amount.ReadFromV2(*bal)
if cc.err != nil {
cc.err = newErrInvalidResponseField(fieldBalance, cc.err)
if err := res.amount.ReadFromV2(*bal); err != nil {
return &res, newErrInvalidResponseField(fieldBalance, err)
}
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}

View file

@ -10,6 +10,7 @@ import (
v2accounting "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/accounting"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
"google.golang.org/grpc"
)
// Client represents virtual connection to the FrostFS network to communicate
@ -101,6 +102,7 @@ func (c *Client) Dial(ctx context.Context, prm PrmDial) error {
client.WithNetworkURIAddress(prm.endpoint, prm.tlsConfig),
client.WithDialTimeout(prm.timeoutDial),
client.WithRWTimeout(prm.streamTimeout),
client.WithGRPCDialOptions(prm.dialOptions),
)...)
c.setFrostFSAPIServer((*coreServer)(&c.c))
@ -186,6 +188,8 @@ type PrmDial struct {
streamTimeoutSet bool
streamTimeout time.Duration
dialOptions []grpc.DialOption
}
// SetServerURI sets server URI in the FrostFS network.
@ -226,3 +230,8 @@ func (x *PrmDial) SetStreamTimeout(timeout time.Duration) {
x.streamTimeoutSet = true
x.streamTimeout = timeout
}
// SetGRPCDialOptions sets the gRPC dial options for new gRPC client connection.
func (x *PrmDial) SetGRPCDialOptions(opts ...grpc.DialOption) {
x.dialOptions = opts
}

View file

@ -1,7 +1,6 @@
package client
import (
"crypto/ecdsa"
"errors"
"fmt"
@ -13,21 +12,11 @@ import (
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/version"
)
// common interface of resulting structures with API status.
type resCommon interface {
setStatus(apistatus.Status)
}
// structure is embedded to all resulting types in order to inherit status-related methods.
type statusRes struct {
st apistatus.Status
}
// setStatus implements resCommon interface method.
func (x *statusRes) setStatus(st apistatus.Status) {
x.st = st
}
// Status returns server's status return.
//
// Use apistatus package functionality to handle the status.
@ -58,6 +47,8 @@ func writeXHeadersToMeta(xHeaders []string, h *v2session.RequestMetaHeader) {
return
}
// TODO (aarifullin): remove the panic when all client parameters will check XHeaders
// within buildRequest invocation.
if len(xHeaders)%2 != 0 {
panic("slice of X-Headers with odd length")
}
@ -73,7 +64,6 @@ func writeXHeadersToMeta(xHeaders []string, h *v2session.RequestMetaHeader) {
// error messages.
var (
errorMissingContext = errors.New("missing context")
errorMissingContainer = errors.New("missing container")
errorMissingObject = errors.New("missing object")
errorAccountNotSet = errors.New("account not set")
@ -83,94 +73,15 @@ var (
errorMissingAnnouncements = errors.New("missing announcements")
errorZeroRangeLength = errors.New("zero range length")
errorMissingRanges = errors.New("missing ranges")
errorZeroEpoch = errors.New("zero epoch")
errorMissingTrusts = errors.New("missing trusts")
errorTrustNotSet = errors.New("current trust value not set")
errorInvalidXHeaders = errors.New("xheaders must be presented only as key-value pairs")
)
// groups all the details required to send a single request and process a response to it.
type contextCall struct {
// ==================================================
// state vars that do not require explicit initialization
// final error to be returned from client method
err error
// received response
resp responseV2
// ==================================================
// shared parameters which are set uniformly on all calls
// request signing key
key ecdsa.PrivateKey
// callback prior to processing the response by the client
callbackResp func(ResponseMetaInfo) error
// if set, protocol errors will be expanded into a final error
resolveAPIFailures bool
// FrostFS network magic
netMagic uint64
// Meta parameters
meta prmCommonMeta
// ==================================================
// custom call parameters
// structure of the call result
statusRes resCommon
// request to be signed with a key and sent
req request
// function to send a request (unary) and receive a response
call func() (responseV2, error)
// function to send the request (req field)
wReq func() error
// function to recv the response (resp field)
rResp func() error
// function to close the message stream
closer func() error
// function of writing response fields to the resulting structure (optional)
result func(v2 responseV2)
}
type request interface {
GetMetaHeader() *v2session.RequestMetaHeader
SetMetaHeader(*v2session.RequestMetaHeader)
SetVerificationHeader(*v2session.RequestVerificationHeader)
}
// sets needed fields of the request meta header.
func (x contextCall) prepareRequest() {
meta := x.req.GetMetaHeader()
if meta == nil {
meta = new(v2session.RequestMetaHeader)
x.req.SetMetaHeader(meta)
}
if meta.GetTTL() == 0 {
meta.SetTTL(2)
}
if meta.GetVersion() == nil {
var verV2 refs.Version
version.Current().WriteToV2(&verV2)
meta.SetVersion(&verV2)
}
meta.SetNetworkMagic(x.netMagic)
writeXHeadersToMeta(x.meta.xHeaders, meta)
}
func (c *Client) prepareRequest(req request, meta *v2session.RequestMetaHeader) {
ttl := meta.GetTTL()
if ttl == 0 {
@ -190,77 +101,18 @@ func (c *Client) prepareRequest(req request, meta *v2session.RequestMetaHeader)
req.SetMetaHeader(meta)
}
// prepares, signs and writes the request. Result means success.
// If failed, contextCall.err contains the reason.
func (x *contextCall) writeRequest() bool {
x.prepareRequest()
x.req.SetVerificationHeader(nil)
// sign the request
x.err = signature.SignServiceMessage(&x.key, x.req)
if x.err != nil {
x.err = fmt.Errorf("sign request: %w", x.err)
return false
}
x.err = x.wReq()
if x.err != nil {
x.err = fmt.Errorf("write request: %w", x.err)
return false
}
return true
}
// performs common actions of response processing and writes any problem as a result status or client error
// (in both cases returns false).
//
// Actions:
// - verify signature (internal);
// - call response callback (internal);
// - unwrap status error (optional).
func (x *contextCall) processResponse() bool {
// call response callback if set
if x.callbackResp != nil {
x.err = x.callbackResp(ResponseMetaInfo{
key: x.resp.GetVerificationHeader().GetBodySignature().GetKey(),
epoch: x.resp.GetMetaHeader().GetEpoch(),
})
if x.err != nil {
x.err = fmt.Errorf("response callback error: %w", x.err)
return false
}
}
// note that we call response callback before signature check since it is expected more lightweight
// while verification needs marshaling
// verify response signature
x.err = signature.VerifyServiceMessage(x.resp)
if x.err != nil {
x.err = fmt.Errorf("invalid response signature: %w", x.err)
return false
}
// get result status
st := apistatus.FromStatusV2(x.resp.GetMetaHeader().GetStatus())
// unwrap unsuccessful status and return it
// as error if client has been configured so
successfulStatus := apistatus.IsSuccessful(st)
if x.resolveAPIFailures {
x.err = apistatus.ErrFromStatus(st)
} else {
x.statusRes.setStatus(st)
}
return successfulStatus
}
// processResponse verifies response signature and converts status to an error if needed.
func (c *Client) processResponse(resp responseV2) (apistatus.Status, error) {
if c.prm.cbRespInfo != nil {
rmi := ResponseMetaInfo{
key: resp.GetVerificationHeader().GetBodySignature().GetKey(),
epoch: resp.GetMetaHeader().GetEpoch(),
}
if err := c.prm.cbRespInfo(rmi); err != nil {
return nil, fmt.Errorf("response callback error: %w", err)
}
}
err := signature.VerifyServiceMessage(resp)
if err != nil {
return nil, fmt.Errorf("invalid response signature: %w", err)
@ -273,78 +125,6 @@ func (c *Client) processResponse(resp responseV2) (apistatus.Status, error) {
return st, nil
}
// reads response (if rResp is set) and processes it. Result means success.
// If failed, contextCall.err (or statusRes if resolveAPIFailures is set) contains the reason.
func (x *contextCall) readResponse() bool {
if x.rResp != nil {
x.err = x.rResp()
if x.err != nil {
x.err = fmt.Errorf("read response: %w", x.err)
return false
}
}
return x.processResponse()
}
// closes the message stream (if closer is set) and writes the results (if result is set).
// Return means success. If failed, contextCall.err contains the reason.
func (x *contextCall) close() bool {
if x.closer != nil {
x.err = x.closer()
if x.err != nil {
x.err = fmt.Errorf("close RPC: %w", x.err)
return false
}
}
// write response to resulting structure
if x.result != nil {
x.result(x.resp)
}
return x.err == nil
}
// goes through all stages of sending a request and processing a response. Returns true if successful.
// If failed, contextCall.err contains the reason.
func (x *contextCall) processCall() bool {
// set request writer
x.wReq = func() error {
var err error
x.resp, err = x.call()
return err
}
// write request
ok := x.writeRequest()
if !ok {
return false
}
// read response
ok = x.readResponse()
if !ok {
return x.err == nil
}
// close and write response to resulting structure
ok = x.close()
if !ok {
return false
}
return x.err == nil
}
// initializes static cross-call parameters inherited from client.
func (c *Client) initCallContext(ctx *contextCall) {
ctx.key = c.prm.key
ctx.resolveAPIFailures = c.prm.resolveFrostFSErrors
ctx.callbackResp = c.prm.cbRespInfo
ctx.netMagic = c.prm.netMagic
}
// ExecRaw executes f with underlying git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client.Client
// instance. Communicate over the Protocol Buffers protocol in a more flexible way:
// most often used to transmit data over a fixed version of the FrostFS protocol, as well

View file

@ -2,805 +2,11 @@ package client
import (
"context"
"errors"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
)
// PrmContainerPut groups parameters of ContainerPut operation.
type PrmContainerPut struct {
prmCommonMeta
cnrSet bool
cnr container.Container
sessionSet bool
session session.Container
}
// SetContainer sets structured information about new FrostFS container.
// Required parameter.
func (x *PrmContainerPut) SetContainer(cnr container.Container) {
x.cnr = cnr
x.cnrSet = true
}
// WithinSession specifies session within which container should be saved.
//
// Creator of the session acquires the authorship of the request. This affects
// the execution of an operation (e.g. access control).
//
// Session is optional, if set the following requirements apply:
// - session operation MUST be session.VerbContainerPut (ForVerb)
// - token MUST be signed using private key of the owner of the container to be saved
func (x *PrmContainerPut) WithinSession(s session.Container) {
x.session = s
x.sessionSet = true
}
// ResContainerPut groups resulting values of ContainerPut operation.
type ResContainerPut struct {
statusRes
id cid.ID
}
// ID returns identifier of the container declared to be stored in the system.
// Used as a link to information about the container (in particular, you can
// asynchronously check if the save was successful).
func (x ResContainerPut) ID() cid.ID {
return x.id
}
// ContainerPut sends request to save container in FrostFS.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see ResContainerPut.ID).
//
// Returns an error if parameters are set incorrectly (see PrmContainerPut docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
//
// nolint: funlen
func (c *Client) ContainerPut(ctx context.Context, prm PrmContainerPut) (*ResContainerPut, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case !prm.cnrSet:
return nil, errorMissingContainer
}
// TODO: check private key is set before forming the request
// sign container
var cnr v2container.Container
prm.cnr.WriteToV2(&cnr)
var sig frostfscrypto.Signature
err := container.CalculateSignature(&sig, prm.cnr, c.prm.key)
if err != nil {
return nil, fmt.Errorf("calculate container signature: %w", err)
}
var sigv2 refs.Signature
sig.WriteToV2(&sigv2)
// form request body
reqBody := new(v2container.PutRequestBody)
reqBody.SetContainer(&cnr)
reqBody.SetSignature(&sigv2)
// form meta header
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(prm.prmCommonMeta.xHeaders, &meta)
if prm.sessionSet {
var tokv2 v2session.Token
prm.session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
// form request
var req v2container.PutRequest
req.SetBody(reqBody)
req.SetMetaHeader(&meta)
// init call context
var (
cc contextCall
res ResContainerPut
)
c.initCallContext(&cc)
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.PutContainer(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2container.PutResponse)
const fieldCnrID = "container ID"
cidV2 := resp.GetBody().GetContainerID()
if cidV2 == nil {
cc.err = newErrMissingResponseField(fieldCnrID)
return
}
cc.err = res.id.ReadFromV2(*cidV2)
if cc.err != nil {
cc.err = newErrInvalidResponseField(fieldCnrID, cc.err)
}
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmContainerGet groups parameters of ContainerGet operation.
type PrmContainerGet struct {
prmCommonMeta
idSet bool
id cid.ID
}
// SetContainer sets identifier of the container to be read.
// Required parameter.
func (x *PrmContainerGet) SetContainer(id cid.ID) {
x.id = id
x.idSet = true
}
// ResContainerGet groups resulting values of ContainerGet operation.
type ResContainerGet struct {
statusRes
cnr container.Container
}
// Container returns structured information about the requested container.
//
// Client doesn't retain value so modification is safe.
func (x ResContainerGet) Container() container.Container {
return x.cnr
}
// ContainerGet reads FrostFS container by ID.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Returns an error if parameters are set incorrectly (see PrmContainerGet docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs);
// - *apistatus.ContainerNotFound.
func (c *Client) ContainerGet(ctx context.Context, prm PrmContainerGet) (*ResContainerGet, error) {
switch {
case ctx == nil:
return nil, errorMissingContext
case !prm.idSet:
return nil, errorMissingContainer
}
var cidV2 refs.ContainerID
prm.id.WriteToV2(&cidV2)
// form request body
reqBody := new(v2container.GetRequestBody)
reqBody.SetContainerID(&cidV2)
// form request
var req v2container.GetRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResContainerGet
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.GetContainer(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2container.GetResponse)
cnrV2 := resp.GetBody().GetContainer()
if cnrV2 == nil {
cc.err = errors.New("missing container in response")
return
}
cc.err = res.cnr.ReadFromV2(*cnrV2)
if cc.err != nil {
cc.err = fmt.Errorf("invalid container in response: %w", cc.err)
}
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmContainerList groups parameters of ContainerList operation.
type PrmContainerList struct {
prmCommonMeta
ownerSet bool
ownerID user.ID
}
// SetAccount sets identifier of the FrostFS account to list the containers.
// Required parameter.
func (x *PrmContainerList) SetAccount(id user.ID) {
x.ownerID = id
x.ownerSet = true
}
// ResContainerList groups resulting values of ContainerList operation.
type ResContainerList struct {
statusRes
ids []cid.ID
}
// Containers returns list of identifiers of the account-owned containers.
//
// Client doesn't retain value so modification is safe.
func (x ResContainerList) Containers() []cid.ID {
return x.ids
}
// ContainerList requests identifiers of the account-owned containers.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Returns an error if parameters are set incorrectly (see PrmContainerList docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerList(ctx context.Context, prm PrmContainerList) (*ResContainerList, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case !prm.ownerSet:
return nil, errorAccountNotSet
}
// form request body
var ownerV2 refs.OwnerID
prm.ownerID.WriteToV2(&ownerV2)
reqBody := new(v2container.ListRequestBody)
reqBody.SetOwnerID(&ownerV2)
// form request
var req v2container.ListRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResContainerList
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.ListContainers(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2container.ListResponse)
res.ids = make([]cid.ID, len(resp.GetBody().GetContainerIDs()))
for i, cidV2 := range resp.GetBody().GetContainerIDs() {
cc.err = res.ids[i].ReadFromV2(cidV2)
if cc.err != nil {
cc.err = fmt.Errorf("invalid ID in the response: %w", cc.err)
return
}
}
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmContainerDelete groups parameters of ContainerDelete operation.
type PrmContainerDelete struct {
prmCommonMeta
idSet bool
id cid.ID
tokSet bool
tok session.Container
}
// SetContainer sets identifier of the FrostFS container to be removed.
// Required parameter.
func (x *PrmContainerDelete) SetContainer(id cid.ID) {
x.id = id
x.idSet = true
}
// WithinSession specifies session within which container should be removed.
//
// Creator of the session acquires the authorship of the request.
// This may affect the execution of an operation (e.g. access control).
//
// Must be signed.
func (x *PrmContainerDelete) WithinSession(tok session.Container) {
x.tok = tok
x.tokSet = true
}
// ResContainerDelete groups resulting values of ContainerDelete operation.
type ResContainerDelete struct {
statusRes
}
// ContainerDelete sends request to remove the FrostFS container.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see GetContainer).
//
// Returns an error if parameters are set incorrectly (see PrmContainerDelete docs).
// Context is required and must not be nil. It is used for network communication.
//
// Exactly one return value is non-nil. Server status return is returned in ResContainerDelete.
// Reflects all internal errors in second return value (transport problems, response processing, etc.).
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerDelete(ctx context.Context, prm PrmContainerDelete) (*ResContainerDelete, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case !prm.idSet:
return nil, errorMissingContainer
}
// sign container ID
var cidV2 refs.ContainerID
prm.id.WriteToV2(&cidV2)
// container contract expects signature of container ID value
// don't get confused with stable marshaled protobuf container.ID structure
data := cidV2.GetValue()
var sig frostfscrypto.Signature
err := sig.Calculate(frostfsecdsa.SignerRFC6979(c.prm.key), data)
if err != nil {
return nil, fmt.Errorf("calculate signature: %w", err)
}
var sigv2 refs.Signature
sig.WriteToV2(&sigv2)
// form request body
reqBody := new(v2container.DeleteRequestBody)
reqBody.SetContainerID(&cidV2)
reqBody.SetSignature(&sigv2)
// form meta header
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(prm.prmCommonMeta.xHeaders, &meta)
if prm.tokSet {
var tokv2 v2session.Token
prm.tok.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
// form request
var req v2container.DeleteRequest
req.SetBody(reqBody)
req.SetMetaHeader(&meta)
// init call context
var (
cc contextCall
res ResContainerDelete
)
c.initCallContext(&cc)
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.DeleteContainer(&c.c, &req, client.WithContext(ctx))
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmContainerEACL groups parameters of ContainerEACL operation.
type PrmContainerEACL struct {
prmCommonMeta
idSet bool
id cid.ID
}
// SetContainer sets identifier of the FrostFS container to read the eACL table.
// Required parameter.
func (x *PrmContainerEACL) SetContainer(id cid.ID) {
x.id = id
x.idSet = true
}
// ResContainerEACL groups resulting values of ContainerEACL operation.
type ResContainerEACL struct {
statusRes
table eacl.Table
}
// Table returns eACL table of the requested container.
func (x ResContainerEACL) Table() eacl.Table {
return x.table
}
// ContainerEACL reads eACL table of the FrostFS container.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Returns an error if parameters are set incorrectly (see PrmContainerEACL docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs);
// - *apistatus.ContainerNotFound;
// - *apistatus.EACLNotFound.
func (c *Client) ContainerEACL(ctx context.Context, prm PrmContainerEACL) (*ResContainerEACL, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case !prm.idSet:
return nil, errorMissingContainer
}
var cidV2 refs.ContainerID
prm.id.WriteToV2(&cidV2)
// form request body
reqBody := new(v2container.GetExtendedACLRequestBody)
reqBody.SetContainerID(&cidV2)
// form request
var req v2container.GetExtendedACLRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResContainerEACL
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.GetEACL(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2container.GetExtendedACLResponse)
eACL := resp.GetBody().GetEACL()
if eACL == nil {
cc.err = newErrMissingResponseField("eACL")
return
}
res.table = *eacl.NewTableFromV2(eACL)
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmContainerSetEACL groups parameters of ContainerSetEACL operation.
type PrmContainerSetEACL struct {
prmCommonMeta
tableSet bool
table eacl.Table
sessionSet bool
session session.Container
}
// SetTable sets eACL table structure to be set for the container.
// Required parameter.
func (x *PrmContainerSetEACL) SetTable(table eacl.Table) {
x.table = table
x.tableSet = true
}
// WithinSession specifies session within which extended ACL of the container
// should be saved.
//
// Creator of the session acquires the authorship of the request. This affects
// the execution of an operation (e.g. access control).
//
// Session is optional, if set the following requirements apply:
// - if particular container is specified (ApplyOnlyTo), it MUST equal the container
// for which extended ACL is going to be set
// - session operation MUST be session.VerbContainerSetEACL (ForVerb)
// - token MUST be signed using private key of the owner of the container to be saved
func (x *PrmContainerSetEACL) WithinSession(s session.Container) {
x.session = s
x.sessionSet = true
}
// ResContainerSetEACL groups resulting values of ContainerSetEACL operation.
type ResContainerSetEACL struct {
statusRes
}
// ContainerSetEACL sends request to update eACL table of the FrostFS container.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see EACL).
//
// Returns an error if parameters are set incorrectly (see PrmContainerSetEACL docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerSetEACL(ctx context.Context, prm PrmContainerSetEACL) (*ResContainerSetEACL, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case !prm.tableSet:
return nil, errorEACLTableNotSet
}
// sign the eACL table
eaclV2 := prm.table.ToV2()
var sig frostfscrypto.Signature
err := sig.Calculate(frostfsecdsa.SignerRFC6979(c.prm.key), eaclV2.StableMarshal(nil))
if err != nil {
return nil, fmt.Errorf("calculate signature: %w", err)
}
var sigv2 refs.Signature
sig.WriteToV2(&sigv2)
// form request body
reqBody := new(v2container.SetExtendedACLRequestBody)
reqBody.SetEACL(eaclV2)
reqBody.SetSignature(&sigv2)
// form meta header
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(prm.prmCommonMeta.xHeaders, &meta)
if prm.sessionSet {
var tokv2 v2session.Token
prm.session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
// form request
var req v2container.SetExtendedACLRequest
req.SetBody(reqBody)
req.SetMetaHeader(&meta)
// init call context
var (
cc contextCall
res ResContainerSetEACL
)
c.initCallContext(&cc)
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.SetEACL(&c.c, &req, client.WithContext(ctx))
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmAnnounceSpace groups parameters of ContainerAnnounceUsedSpace operation.
type PrmAnnounceSpace struct {
prmCommonMeta
announcements []container.SizeEstimation
}
// SetValues sets values describing volume of space that is used for the container objects.
// Required parameter. Must not be empty.
//
// Must not be mutated before the end of the operation.
func (x *PrmAnnounceSpace) SetValues(vs []container.SizeEstimation) {
x.announcements = vs
}
// ResAnnounceSpace groups resulting values of ContainerAnnounceUsedSpace operation.
type ResAnnounceSpace struct {
statusRes
}
// ContainerAnnounceUsedSpace sends request to announce volume of the space used for the container objects.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// At this moment success can not be checked.
//
// Returns an error if parameters are set incorrectly (see PrmAnnounceSpace docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerAnnounceUsedSpace(ctx context.Context, prm PrmAnnounceSpace) (*ResAnnounceSpace, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case len(prm.announcements) == 0:
return nil, errorMissingAnnouncements
}
// convert list of SDK announcement structures into FrostFS-API v2 list
v2announce := make([]v2container.UsedSpaceAnnouncement, len(prm.announcements))
for i := range prm.announcements {
prm.announcements[i].WriteToV2(&v2announce[i])
}
// prepare body of the FrostFS-API v2 request and request itself
reqBody := new(v2container.AnnounceUsedSpaceRequestBody)
reqBody.SetAnnouncements(v2announce)
// form request
var req v2container.AnnounceUsedSpaceRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResAnnounceSpace
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.AnnounceUsedSpace(&c.c, &req, client.WithContext(ctx))
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// SyncContainerWithNetwork requests network configuration using passed client
// and applies it to the container. Container MUST not be nil.
//

139
client/container_delete.go Normal file
View file

@ -0,0 +1,139 @@
package client
import (
"context"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmContainerDelete groups parameters of ContainerDelete operation.
type PrmContainerDelete struct {
// FrostFS request X-Headers
XHeaders []string
ContainerID *cid.ID
Session *session.Container
}
// SetContainer sets identifier of the FrostFS container to be removed.
// Required parameter.
//
// Deprecated: Use PrmContainerDelete.Container instead.
func (prm *PrmContainerDelete) SetContainer(id cid.ID) {
prm.ContainerID = &id
}
func (prm *PrmContainerDelete) buildRequest(c *Client) (*v2container.DeleteRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
var cidV2 refs.ContainerID
prm.ContainerID.WriteToV2(&cidV2)
// Container contract expects signature of container ID value,
// don't get confused with stable marshaled protobuf container.ID structure.
data := cidV2.GetValue()
var sig frostfscrypto.Signature
err := sig.Calculate(frostfsecdsa.SignerRFC6979(c.prm.key), data)
if err != nil {
return nil, fmt.Errorf("calculate signature: %w", err)
}
var sigv2 refs.Signature
sig.WriteToV2(&sigv2)
reqBody := new(v2container.DeleteRequestBody)
reqBody.SetContainerID(&cidV2)
reqBody.SetSignature(&sigv2)
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(prm.XHeaders, &meta)
if prm.Session != nil {
var tokv2 v2session.Token
prm.Session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
var req v2container.DeleteRequest
req.SetBody(reqBody)
c.prepareRequest(&req, &meta)
return &req, nil
}
// WithinSession specifies session within which container should be removed.
//
// Creator of the session acquires the authorship of the request.
// This may affect the execution of an operation (e.g. access control).
//
// Must be signed.
//
// Deprecated: Use PrmContainerDelete.Session instead.
func (prm *PrmContainerDelete) WithinSession(tok session.Container) {
prm.Session = &tok
}
// ResContainerDelete groups resulting values of ContainerDelete operation.
type ResContainerDelete struct {
statusRes
}
// ContainerDelete sends request to remove the FrostFS container.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see GetContainer).
//
// Returns an error if parameters are set incorrectly (see PrmContainerDelete docs).
// Context is required and must not be nil. It is used for network communication.
//
// Exactly one return value is non-nil. Server status return is returned in ResContainerDelete.
// Reflects all internal errors in second return value (transport problems, response processing, etc.).
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerDelete(ctx context.Context, prm PrmContainerDelete) (*ResContainerDelete, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.DeleteContainer(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResContainerDelete
res.st, err = c.processResponse(resp)
return &res, err
}

110
client/container_eacl.go Normal file
View file

@ -0,0 +1,110 @@
package client
import (
"context"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl"
)
// PrmContainerEACL groups parameters of ContainerEACL operation.
type PrmContainerEACL struct {
// FrostFS request X-Headers.
XHeaders []string
ContainerID *cid.ID
}
// SetContainer sets identifier of the FrostFS container to read the eACL table.
// Required parameter.
//
// Deprecated: Use PrmContainerEACL.ContainerID instead.
func (x *PrmContainerEACL) SetContainer(id cid.ID) {
x.ContainerID = &id
}
func (x *PrmContainerEACL) buildRequest(c *Client) (*v2container.GetExtendedACLRequest, error) {
if x.ContainerID == nil {
return nil, errorMissingContainer
}
if len(x.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
var cidV2 refs.ContainerID
x.ContainerID.WriteToV2(&cidV2)
reqBody := new(v2container.GetExtendedACLRequestBody)
reqBody.SetContainerID(&cidV2)
var req v2container.GetExtendedACLRequest
req.SetBody(reqBody)
c.prepareRequest(&req, new(v2session.RequestMetaHeader))
return &req, nil
}
// ResContainerEACL groups resulting values of ContainerEACL operation.
type ResContainerEACL struct {
statusRes
table eacl.Table
}
// Table returns eACL table of the requested container.
func (x ResContainerEACL) Table() eacl.Table {
return x.table
}
// ContainerEACL reads eACL table of the FrostFS container.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Returns an error if parameters are set incorrectly (see PrmContainerEACL docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs);
// - *apistatus.ContainerNotFound;
// - *apistatus.EACLNotFound.
func (c *Client) ContainerEACL(ctx context.Context, prm PrmContainerEACL) (*ResContainerEACL, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.GetEACL(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResContainerEACL
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
eACL := resp.GetBody().GetEACL()
if eACL == nil {
return &res, newErrMissingResponseField("eACL")
}
res.table = *eacl.NewTableFromV2(eACL)
return &res, nil
}

113
client/container_get.go Normal file
View file

@ -0,0 +1,113 @@
package client
import (
"context"
"errors"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
)
// PrmContainerGet groups parameters of ContainerGet operation.
type PrmContainerGet struct {
// FrostFS request X-Headers.
XHeaders []string
ContainerID *cid.ID
}
// SetContainer sets identifier of the container to be read.
// Required parameter.
//
// Deprecated: Use PrmContainerGet.ContainerID instead.
func (prm *PrmContainerGet) SetContainer(cid cid.ID) {
prm.ContainerID = &cid
}
func (prm *PrmContainerGet) buildRequest(c *Client) (*v2container.GetRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
var cidV2 refs.ContainerID
prm.ContainerID.WriteToV2(&cidV2)
reqBody := new(v2container.GetRequestBody)
reqBody.SetContainerID(&cidV2)
var req v2container.GetRequest
req.SetBody(reqBody)
c.prepareRequest(&req, new(v2session.RequestMetaHeader))
return &req, nil
}
// ResContainerGet groups resulting values of ContainerGet operation.
type ResContainerGet struct {
statusRes
cnr container.Container
}
// Container returns structured information about the requested container.
//
// Client doesn't retain value so modification is safe.
func (x ResContainerGet) Container() container.Container {
return x.cnr
}
// ContainerGet reads FrostFS container by ID.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Returns an error if parameters are set incorrectly (see PrmContainerGet docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs);
// - *apistatus.ContainerNotFound.
func (c *Client) ContainerGet(ctx context.Context, prm PrmContainerGet) (*ResContainerGet, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.GetContainer(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResContainerGet
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
cnrV2 := resp.GetBody().GetContainer()
if cnrV2 == nil {
return &res, errors.New("missing container in response")
}
if err := res.cnr.ReadFromV2(*cnrV2); err != nil {
return &res, fmt.Errorf("invalid container in response: %w", err)
}
return &res, nil
}

106
client/container_list.go Normal file
View file

@ -0,0 +1,106 @@
package client
import (
"context"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
)
// PrmContainerList groups parameters of ContainerList operation.
type PrmContainerList struct {
prmCommonMeta
ownerSet bool
ownerID user.ID
}
// SetAccount sets identifier of the FrostFS account to list the containers.
// Required parameter.
func (x *PrmContainerList) SetAccount(id user.ID) {
x.ownerID = id
x.ownerSet = true
}
func (x *PrmContainerList) buildRequest(c *Client) (*v2container.ListRequest, error) {
if !x.ownerSet {
return nil, errorAccountNotSet
}
var ownerV2 refs.OwnerID
x.ownerID.WriteToV2(&ownerV2)
reqBody := new(v2container.ListRequestBody)
reqBody.SetOwnerID(&ownerV2)
var req v2container.ListRequest
req.SetBody(reqBody)
c.prepareRequest(&req, new(v2session.RequestMetaHeader))
return &req, nil
}
// ResContainerList groups resulting values of ContainerList operation.
type ResContainerList struct {
statusRes
ids []cid.ID
}
// Containers returns list of identifiers of the account-owned containers.
//
// Client doesn't retain value so modification is safe.
func (x ResContainerList) Containers() []cid.ID {
return x.ids
}
// ContainerList requests identifiers of the account-owned containers.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Returns an error if parameters are set incorrectly (see PrmContainerList docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerList(ctx context.Context, prm PrmContainerList) (*ResContainerList, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.ListContainers(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResContainerList
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
res.ids = make([]cid.ID, len(resp.GetBody().GetContainerIDs()))
for i, cidV2 := range resp.GetBody().GetContainerIDs() {
if err := res.ids[i].ReadFromV2(cidV2); err != nil {
return &res, fmt.Errorf("invalid ID in the response: %w", err)
}
}
return &res, nil
}

160
client/container_put.go Normal file
View file

@ -0,0 +1,160 @@
package client
import (
"context"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmContainerPut groups parameters of ContainerPut operation.
type PrmContainerPut struct {
// FrostFS request X-Headers
XHeaders []string
Container *container.Container
Session *session.Container
}
// SetContainer sets structured information about new FrostFS container.
// Required parameter.
//
// Deprecated: Use PrmContainerPut.Container instead.
func (x *PrmContainerPut) SetContainer(cnr container.Container) {
x.Container = &cnr
}
// WithinSession specifies session within which container should be saved.
//
// Creator of the session acquires the authorship of the request. This affects
// the execution of an operation (e.g. access control).
//
// Session is optional, if set the following requirements apply:
// - session operation MUST be session.VerbContainerPut (ForVerb)
// - token MUST be signed using private key of the owner of the container to be saved
//
// Deprecated: Use PrmContainerPut.Session instead.
func (x *PrmContainerPut) WithinSession(s session.Container) {
x.Session = &s
}
func (x *PrmContainerPut) buildRequest(c *Client) (*v2container.PutRequest, error) {
if x.Container == nil {
return nil, errorMissingContainer
}
if len(x.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
// TODO: check private key is set before forming the request
var cnr v2container.Container
x.Container.WriteToV2(&cnr)
var sig frostfscrypto.Signature
err := container.CalculateSignature(&sig, *x.Container, c.prm.key)
if err != nil {
return nil, fmt.Errorf("calculate container signature: %w", err)
}
var sigv2 refs.Signature
sig.WriteToV2(&sigv2)
reqBody := new(v2container.PutRequestBody)
reqBody.SetContainer(&cnr)
reqBody.SetSignature(&sigv2)
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(x.XHeaders, &meta)
if x.Session != nil {
var tokv2 v2session.Token
x.Session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
var req v2container.PutRequest
req.SetBody(reqBody)
c.prepareRequest(&req, &meta)
return &req, nil
}
// ResContainerPut groups resulting values of ContainerPut operation.
type ResContainerPut struct {
statusRes
id cid.ID
}
// ID returns identifier of the container declared to be stored in the system.
// Used as a link to information about the container (in particular, you can
// asynchronously check if the save was successful).
func (x ResContainerPut) ID() cid.ID {
return x.id
}
// ContainerPut sends request to save container in FrostFS.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see ResContainerPut.ID).
//
// Returns an error if parameters are set incorrectly (see PrmContainerPut docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
//
// nolint: funlen
func (c *Client) ContainerPut(ctx context.Context, prm PrmContainerPut) (*ResContainerPut, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.PutContainer(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResContainerPut
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
const fieldCnrID = "container ID"
cidV2 := resp.GetBody().GetContainerID()
if cidV2 == nil {
return &res, newErrMissingResponseField(fieldCnrID)
}
if err := res.id.ReadFromV2(*cidV2); err != nil {
return &res, newErrInvalidResponseField(fieldCnrID, err)
}
return &res, nil
}

View file

@ -0,0 +1,136 @@
package client
import (
"context"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmContainerSetEACL groups parameters of ContainerSetEACL operation.
type PrmContainerSetEACL struct {
// FrostFS request X-Headers.
XHeaders []string
Table *eacl.Table
Session *session.Container
}
// SetTable sets eACL table structure to be set for the container.
// Required parameter.
//
// Deprecated: Use PrmContainerSetEACL.Table instead.
func (x *PrmContainerSetEACL) SetTable(table eacl.Table) {
x.Table = &table
}
// WithinSession specifies session within which extended ACL of the container
// should be saved.
//
// Creator of the session acquires the authorship of the request. This affects
// the execution of an operation (e.g. access control).
//
// Session is optional, if set the following requirements apply:
// - if particular container is specified (ApplyOnlyTo), it MUST equal the container
// for which extended ACL is going to be set
// - session operation MUST be session.VerbContainerSetEACL (ForVerb)
// - token MUST be signed using private key of the owner of the container to be saved
//
// Deprecated: Use PrmContainerSetEACL.Session instead.
func (x *PrmContainerSetEACL) WithinSession(s session.Container) {
x.Session = &s
}
func (x *PrmContainerSetEACL) buildRequest(c *Client) (*v2container.SetExtendedACLRequest, error) {
if x.Table == nil {
return nil, errorEACLTableNotSet
}
if len(x.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
eaclV2 := x.Table.ToV2()
var sig frostfscrypto.Signature
err := sig.Calculate(frostfsecdsa.SignerRFC6979(c.prm.key), eaclV2.StableMarshal(nil))
if err != nil {
return nil, fmt.Errorf("calculate signature: %w", err)
}
var sigv2 refs.Signature
sig.WriteToV2(&sigv2)
reqBody := new(v2container.SetExtendedACLRequestBody)
reqBody.SetEACL(eaclV2)
reqBody.SetSignature(&sigv2)
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(x.XHeaders, &meta)
if x.Session != nil {
var tokv2 v2session.Token
x.Session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
var req v2container.SetExtendedACLRequest
req.SetBody(reqBody)
c.prepareRequest(&req, &meta)
return &req, nil
}
// ResContainerSetEACL groups resulting values of ContainerSetEACL operation.
type ResContainerSetEACL struct {
statusRes
}
// ContainerSetEACL sends request to update eACL table of the FrostFS container.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see EACL).
//
// Returns an error if parameters are set incorrectly (see PrmContainerSetEACL docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerSetEACL(ctx context.Context, prm PrmContainerSetEACL) (*ResContainerSetEACL, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.SetEACL(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResContainerSetEACL
res.st, err = c.processResponse(resp)
return &res, err
}

92
client/container_space.go Normal file
View file

@ -0,0 +1,92 @@
package client
import (
"context"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
)
// PrmAnnounceSpace groups parameters of ContainerAnnounceUsedSpace operation.
type PrmAnnounceSpace struct {
XHeaders []string
Announcements []container.SizeEstimation
}
// SetValues sets values describing volume of space that is used for the container objects.
// Required parameter. Must not be empty.
//
// Must not be mutated before the end of the operation.
//
// Deprecated: Use PrmAnnounceSpace.Announcements instead.
func (x *PrmAnnounceSpace) SetValues(vs []container.SizeEstimation) {
x.Announcements = vs
}
func (x *PrmAnnounceSpace) buildRequest(c *Client) (*v2container.AnnounceUsedSpaceRequest, error) {
if len(x.Announcements) == 0 {
return nil, errorMissingAnnouncements
}
v2announce := make([]v2container.UsedSpaceAnnouncement, len(x.Announcements))
for i := range x.Announcements {
x.Announcements[i].WriteToV2(&v2announce[i])
}
reqBody := new(v2container.AnnounceUsedSpaceRequestBody)
reqBody.SetAnnouncements(v2announce)
var req v2container.AnnounceUsedSpaceRequest
req.SetBody(reqBody)
c.prepareRequest(&req, new(v2session.RequestMetaHeader))
return &req, nil
}
// ResAnnounceSpace groups resulting values of ContainerAnnounceUsedSpace operation.
type ResAnnounceSpace struct {
statusRes
}
// ContainerAnnounceUsedSpace sends request to announce volume of the space used for the container objects.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// At this moment success can not be checked.
//
// Returns an error if parameters are set incorrectly (see PrmAnnounceSpace docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerAnnounceUsedSpace(ctx context.Context, prm PrmAnnounceSpace) (*ResAnnounceSpace, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.AnnounceUsedSpace(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResAnnounceSpace
res.st, err = c.processResponse(resp)
return &res, err
}

View file

@ -1,97 +1,64 @@
package client
import (
"errors"
"fmt"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
)
// unwraps err using errors.Unwrap and returns the result.
func unwrapErr(err error) error {
for e := errors.Unwrap(err); e != nil; e = errors.Unwrap(err) {
err = e
// wrapsErrType returns true if any error in the error tree of err is of type E.
func wrapsErrType[E error](err error) bool {
switch e := err.(type) {
case E:
return true
case interface{ Unwrap() error }:
return wrapsErrType[E](e.Unwrap())
case interface{ Unwrap() []error }:
for _, ei := range e.Unwrap() {
if wrapsErrType[E](ei) {
return true
}
}
return false
default:
return false
}
return err
}
// IsErrContainerNotFound checks if err corresponds to FrostFS status
// return corresponding to missing container. Supports wrapped errors.
func IsErrContainerNotFound(err error) bool {
switch unwrapErr(err).(type) {
default:
return false
case
apistatus.ContainerNotFound,
*apistatus.ContainerNotFound:
return true
}
return wrapsErrType[*apistatus.ContainerNotFound](err)
}
// IsErrEACLNotFound checks if err corresponds to FrostFS status
// return corresponding to missing eACL table. Supports wrapped errors.
func IsErrEACLNotFound(err error) bool {
switch unwrapErr(err).(type) {
default:
return false
case
apistatus.EACLNotFound,
*apistatus.EACLNotFound:
return true
}
return wrapsErrType[*apistatus.EACLNotFound](err)
}
// IsErrObjectNotFound checks if err corresponds to FrostFS status
// return corresponding to missing object. Supports wrapped errors.
func IsErrObjectNotFound(err error) bool {
switch unwrapErr(err).(type) {
default:
return false
case
apistatus.ObjectNotFound,
*apistatus.ObjectNotFound:
return true
}
return wrapsErrType[*apistatus.ObjectNotFound](err)
}
// IsErrObjectAlreadyRemoved checks if err corresponds to FrostFS status
// return corresponding to already removed object. Supports wrapped errors.
func IsErrObjectAlreadyRemoved(err error) bool {
switch unwrapErr(err).(type) {
default:
return false
case
apistatus.ObjectAlreadyRemoved,
*apistatus.ObjectAlreadyRemoved:
return true
}
return wrapsErrType[*apistatus.ObjectAlreadyRemoved](err)
}
// IsErrSessionExpired checks if err corresponds to FrostFS status return
// corresponding to expired session. Supports wrapped errors.
func IsErrSessionExpired(err error) bool {
switch unwrapErr(err).(type) {
default:
return false
case
apistatus.SessionTokenExpired,
*apistatus.SessionTokenExpired:
return true
}
return wrapsErrType[*apistatus.SessionTokenExpired](err)
}
// IsErrSessionNotFound checks if err corresponds to FrostFS status return
// corresponding to missing session. Supports wrapped errors.
func IsErrSessionNotFound(err error) bool {
switch unwrapErr(err).(type) {
default:
return false
case
apistatus.SessionTokenNotFound,
*apistatus.SessionTokenNotFound:
return true
}
return wrapsErrType[*apistatus.SessionTokenNotFound](err)
}
// returns error describing missing field with the given name.

View file

@ -10,59 +10,40 @@ import (
)
func TestErrors(t *testing.T) {
for _, tc := range []struct {
errs := []struct {
check func(error) bool
errs []error
err error
}{
{
check: client.IsErrContainerNotFound,
errs: []error{
apistatus.ContainerNotFound{},
new(apistatus.ContainerNotFound),
},
err: new(apistatus.ContainerNotFound),
},
{
check: client.IsErrEACLNotFound,
errs: []error{
apistatus.EACLNotFound{},
new(apistatus.EACLNotFound),
},
err: new(apistatus.EACLNotFound),
},
{
check: client.IsErrObjectNotFound,
errs: []error{
apistatus.ObjectNotFound{},
new(apistatus.ObjectNotFound),
},
err: new(apistatus.ObjectNotFound),
},
{
check: client.IsErrObjectAlreadyRemoved,
errs: []error{
apistatus.ObjectAlreadyRemoved{},
new(apistatus.ObjectAlreadyRemoved),
},
err: new(apistatus.ObjectAlreadyRemoved),
},
{
check: client.IsErrSessionExpired,
errs: []error{
apistatus.SessionTokenExpired{},
new(apistatus.SessionTokenExpired),
},
err: new(apistatus.SessionTokenExpired),
}, {
check: client.IsErrSessionNotFound,
errs: []error{
apistatus.SessionTokenNotFound{},
new(apistatus.SessionTokenNotFound),
err: new(apistatus.SessionTokenNotFound),
},
},
} {
require.NotEmpty(t, tc.errs)
}
for i := range tc.errs {
require.True(t, tc.check(tc.errs[i]), tc.errs[i])
require.True(t, tc.check(fmt.Errorf("top-level context: :%w",
fmt.Errorf("inner context: %w", tc.errs[i])),
), tc.errs[i])
for i := range errs {
for j := range errs {
nestedErr := fmt.Errorf("top-level context: :%w", fmt.Errorf("inner context: %w", errs[j].err))
require.Equal(t, i == j, errs[i].check(errs[j].err))
require.Equal(t, i == j, errs[i].check(nestedErr))
}
}
}

View file

@ -16,7 +16,17 @@ import (
// PrmEndpointInfo groups parameters of EndpointInfo operation.
type PrmEndpointInfo struct {
prmCommonMeta
XHeaders []string
}
func (x *PrmEndpointInfo) buildRequest(c *Client) (*v2netmap.LocalNodeInfoRequest, error) {
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(x.XHeaders, meta)
req := new(v2netmap.LocalNodeInfoRequest)
req.SetBody(new(v2netmap.LocalNodeInfoRequestBody))
c.prepareRequest(req, meta)
return req, nil
}
// ResEndpointInfo group resulting values of EndpointInfo operation.
@ -56,30 +66,25 @@ func (x ResEndpointInfo) NodeInfo() netmap.NodeInfo {
// Return statuses:
// - global (see Client docs).
func (c *Client) EndpointInfo(ctx context.Context, prm PrmEndpointInfo) (*ResEndpointInfo, error) {
// check context
if ctx == nil {
return nil, errorMissingContext
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
// form request
var req v2netmap.LocalNodeInfoRequest
// init call context
var (
cc contextCall
res ResEndpointInfo
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.LocalNodeInfo(&c.c, &req, client.WithContext(ctx))
if err := signature.SignServiceMessage(&c.prm.key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.LocalNodeInfo(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResEndpointInfo
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
cc.result = func(r responseV2) {
resp := r.(*v2netmap.LocalNodeInfoResponse)
body := resp.GetBody()
@ -87,42 +92,37 @@ func (c *Client) EndpointInfo(ctx context.Context, prm PrmEndpointInfo) (*ResEnd
verV2 := body.GetVersion()
if verV2 == nil {
cc.err = newErrMissingResponseField(fieldVersion)
return
return nil, newErrMissingResponseField(fieldVersion)
}
cc.err = res.version.ReadFromV2(*verV2)
if cc.err != nil {
cc.err = newErrInvalidResponseField(fieldVersion, cc.err)
return
if err := res.version.ReadFromV2(*verV2); err != nil {
return nil, newErrInvalidResponseField(fieldVersion, err)
}
const fieldNodeInfo = "node info"
nodeInfoV2 := body.GetNodeInfo()
if nodeInfoV2 == nil {
cc.err = newErrMissingResponseField(fieldNodeInfo)
return
return nil, newErrMissingResponseField(fieldNodeInfo)
}
cc.err = res.ni.ReadFromV2(*nodeInfoV2)
if cc.err != nil {
cc.err = newErrInvalidResponseField(fieldNodeInfo, cc.err)
return
if err := res.ni.ReadFromV2(*nodeInfoV2); err != nil {
return nil, newErrInvalidResponseField(fieldNodeInfo, err)
}
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmNetworkInfo groups parameters of NetworkInfo operation.
type PrmNetworkInfo struct {
prmCommonMeta
XHeaders []string
}
func (x PrmNetworkInfo) buildRequest(c *Client) (*v2netmap.NetworkInfoRequest, error) {
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(x.XHeaders, meta)
var req v2netmap.NetworkInfoRequest
req.SetBody(new(v2netmap.NetworkInfoRequestBody))
c.prepareRequest(&req, meta)
return &req, nil
}
// ResNetworkInfo groups resulting values of NetworkInfo operation.
@ -153,51 +153,35 @@ func (x ResNetworkInfo) Info() netmap.NetworkInfo {
// Return statuses:
// - global (see Client docs).
func (c *Client) NetworkInfo(ctx context.Context, prm PrmNetworkInfo) (*ResNetworkInfo, error) {
// check context
if ctx == nil {
return nil, errorMissingContext
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
// form request
var req v2netmap.NetworkInfoRequest
// init call context
var (
cc contextCall
res ResNetworkInfo
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.NetworkInfo(&c.c, &req, client.WithContext(ctx))
if err := signature.SignServiceMessage(&c.prm.key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.NetworkInfo(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResNetworkInfo
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
cc.result = func(r responseV2) {
resp := r.(*v2netmap.NetworkInfoResponse)
const fieldNetInfo = "network info"
netInfoV2 := resp.GetBody().GetNetworkInfo()
if netInfoV2 == nil {
cc.err = newErrMissingResponseField(fieldNetInfo)
return
return nil, newErrMissingResponseField(fieldNetInfo)
}
cc.err = res.info.ReadFromV2(*netInfoV2)
if cc.err != nil {
cc.err = newErrInvalidResponseField(fieldNetInfo, cc.err)
return
if err := res.info.ReadFromV2(*netInfoV2); err != nil {
return nil, newErrInvalidResponseField(fieldNetInfo, err)
}
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}

View file

@ -21,71 +21,25 @@ import (
// PrmObjectDelete groups parameters of ObjectDelete operation.
type PrmObjectDelete struct {
meta v2session.RequestMetaHeader
XHeaders []string
body v2object.DeleteRequestBody
BearerToken *bearer.Token
addr v2refs.Address
Session *session.Object
keySet bool
key ecdsa.PrivateKey
}
ContainerID *cid.ID
// WithinSession specifies session within which object should be read.
//
// Creator of the session acquires the authorship of the request.
// This may affect the execution of an operation (e.g. access control).
//
// Must be signed.
func (x *PrmObjectDelete) WithinSession(t session.Object) {
var tv2 v2session.Token
t.WriteToV2(&tv2)
ObjectID *oid.ID
x.meta.SetSessionToken(&tv2)
}
// WithBearerToken attaches bearer token to be used for the operation.
//
// If set, underlying eACL rules will be used in access control.
//
// Must be signed.
func (x *PrmObjectDelete) WithBearerToken(t bearer.Token) {
var v2token acl.BearerToken
t.WriteToV2(&v2token)
x.meta.SetBearerToken(&v2token)
}
// FromContainer specifies FrostFS container of the object.
// Required parameter.
func (x *PrmObjectDelete) FromContainer(id cid.ID) {
var cidV2 v2refs.ContainerID
id.WriteToV2(&cidV2)
x.addr.SetContainerID(&cidV2)
}
// ByID specifies identifier of the requested object.
// Required parameter.
func (x *PrmObjectDelete) ByID(id oid.ID) {
var idV2 v2refs.ObjectID
id.WriteToV2(&idV2)
x.addr.SetObjectID(&idV2)
Key *ecdsa.PrivateKey
}
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
func (x *PrmObjectDelete) UseKey(key ecdsa.PrivateKey) {
x.keySet = true
x.key = key
}
// WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length.
//
// Slice must not be mutated until the operation completes.
func (x *PrmObjectDelete) WithXHeaders(hs ...string) {
writeXHeadersToMeta(hs, &x.meta)
// Deprecated: Use PrmObjectDelete.Key instead.
func (prm *PrmObjectDelete) UseKey(key ecdsa.PrivateKey) {
prm.Key = &key
}
// ResObjectDelete groups resulting values of ObjectDelete operation.
@ -100,6 +54,54 @@ func (x ResObjectDelete) Tombstone() oid.ID {
return x.tomb
}
func (prm *PrmObjectDelete) buildRequest(c *Client) (*v2object.DeleteRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if prm.ObjectID == nil {
return nil, errorMissingObject
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
addr := new(v2refs.Address)
cnrV2 := new(v2refs.ContainerID)
prm.ContainerID.WriteToV2(cnrV2)
addr.SetContainerID(cnrV2)
objV2 := new(v2refs.ObjectID)
prm.ObjectID.WriteToV2(objV2)
addr.SetObjectID(objV2)
body := new(v2object.DeleteRequestBody)
body.SetAddress(addr)
req := new(v2object.DeleteRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// ObjectDelete marks an object for deletion from the container using FrostFS API protocol.
// As a marker, a special unit called a tombstone is placed in the container.
// It confirms the user's intent to delete the object, and is itself a container object.
@ -124,34 +126,22 @@ func (x ResObjectDelete) Tombstone() oid.ID {
// - *apistatus.ObjectLocked;
// - *apistatus.SessionTokenExpired.
func (c *Client) ObjectDelete(ctx context.Context, prm PrmObjectDelete) (*ResObjectDelete, error) {
switch {
case ctx == nil:
return nil, errorMissingContext
case prm.addr.GetContainerID() == nil:
return nil, errorMissingContainer
case prm.addr.GetObjectID() == nil:
return nil, errorMissingObject
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
// form request body
prm.body.SetAddress(&prm.addr)
// form request
var req v2object.DeleteRequest
req.SetBody(&prm.body)
c.prepareRequest(&req, &prm.meta)
key := c.prm.key
if prm.keySet {
key = prm.key
if prm.Key != nil {
key = *prm.Key
}
err := signature.SignServiceMessage(&key, &req)
err = signature.SignServiceMessage(&key, req)
if err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.DeleteObject(&c.c, &req, client.WithContext(ctx))
resp, err := rpcapi.DeleteObject(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}

View file

@ -22,77 +22,76 @@ import (
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// shared parameters of GET/HEAD/RANGE.
type prmObjectRead struct {
meta v2session.RequestMetaHeader
raw bool
addr v2refs.Address
}
// WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length.
//
// Slice must not be mutated until the operation completes.
func (x *prmObjectRead) WithXHeaders(hs ...string) {
writeXHeadersToMeta(hs, &x.meta)
}
// MarkRaw marks an intent to read physically stored object.
func (x *prmObjectRead) MarkRaw() {
x.raw = true
}
// MarkLocal tells the server to execute the operation locally.
func (x *prmObjectRead) MarkLocal() {
x.meta.SetTTL(1)
}
// WithinSession specifies session within which object should be read.
//
// Creator of the session acquires the authorship of the request.
// This may affect the execution of an operation (e.g. access control).
//
// Must be signed.
func (x *prmObjectRead) WithinSession(t session.Object) {
var tokv2 v2session.Token
t.WriteToV2(&tokv2)
x.meta.SetSessionToken(&tokv2)
}
// WithBearerToken attaches bearer token to be used for the operation.
//
// If set, underlying eACL rules will be used in access control.
//
// Must be signed.
func (x *prmObjectRead) WithBearerToken(t bearer.Token) {
var v2token acl.BearerToken
t.WriteToV2(&v2token)
x.meta.SetBearerToken(&v2token)
}
// FromContainer specifies FrostFS container of the object.
// Required parameter.
func (x *prmObjectRead) FromContainer(id cid.ID) {
var cnrV2 v2refs.ContainerID
id.WriteToV2(&cnrV2)
x.addr.SetContainerID(&cnrV2)
}
// ByID specifies identifier of the requested object.
// Required parameter.
func (x *prmObjectRead) ByID(id oid.ID) {
var objV2 v2refs.ObjectID
id.WriteToV2(&objV2)
x.addr.SetObjectID(&objV2)
}
// PrmObjectGet groups parameters of ObjectGetInit operation.
type PrmObjectGet struct {
prmObjectRead
XHeaders []string
key *ecdsa.PrivateKey
BearerToken *bearer.Token
Session *session.Object
Raw bool
Local bool
ContainerID *cid.ID
ObjectID *oid.ID
Key *ecdsa.PrivateKey
}
func (prm *PrmObjectGet) buildRequest(c *Client) (*v2object.GetRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if prm.ObjectID == nil {
return nil, errorMissingObject
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if prm.Local {
meta.SetTTL(1)
}
addr := new(v2refs.Address)
cnrV2 := new(v2refs.ContainerID)
prm.ContainerID.WriteToV2(cnrV2)
addr.SetContainerID(cnrV2)
objV2 := new(v2refs.ObjectID)
prm.ObjectID.WriteToV2(objV2)
addr.SetObjectID(objV2)
body := new(v2object.GetRequestBody)
body.SetRaw(prm.Raw)
body.SetAddress(addr)
req := new(v2object.GetRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// ResObjectGet groups the final result values of ObjectGetInit operation.
@ -122,8 +121,10 @@ type ObjectReader struct {
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
func (x *PrmObjectGet) UseKey(key ecdsa.PrivateKey) {
x.key = &key
//
// Deprecated: Use PrmObjectGet.Key instead.
func (prm *PrmObjectGet) UseKey(key ecdsa.PrivateKey) {
prm.Key = &key
}
// ReadHeader reads header of the object. Result means success.
@ -299,41 +300,24 @@ func (x *ObjectReader) Read(p []byte) (int, error) {
// Returns an error if parameters are set incorrectly (see PrmObjectGet docs).
// Context is required and must not be nil. It is used for network communication.
func (c *Client) ObjectGetInit(ctx context.Context, prm PrmObjectGet) (*ObjectReader, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case prm.addr.GetContainerID() == nil:
return nil, errorMissingContainer
case prm.addr.GetObjectID() == nil:
return nil, errorMissingObject
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
// form request body
var body v2object.GetRequestBody
body.SetRaw(prm.raw)
body.SetAddress(&prm.addr)
// form request
var req v2object.GetRequest
req.SetBody(&body)
c.prepareRequest(&req, &prm.meta)
key := prm.key
key := prm.Key
if key == nil {
key = &c.prm.key
}
err := signature.SignServiceMessage(key, &req)
err = signature.SignServiceMessage(key, req)
if err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
ctx, cancel := context.WithCancel(ctx)
stream, err := rpcapi.GetObject(&c.c, &req, client.WithContext(ctx))
stream, err := rpcapi.GetObject(&c.c, req, client.WithContext(ctx))
if err != nil {
cancel()
return nil, fmt.Errorf("open stream: %w", err)
@ -349,17 +333,29 @@ func (c *Client) ObjectGetInit(ctx context.Context, prm PrmObjectGet) (*ObjectRe
// PrmObjectHead groups parameters of ObjectHead operation.
type PrmObjectHead struct {
prmObjectRead
XHeaders []string
keySet bool
key ecdsa.PrivateKey
BearerToken *bearer.Token
Session *session.Object
Raw bool
Local bool
ContainerID *cid.ID
ObjectID *oid.ID
Key *ecdsa.PrivateKey
}
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
func (x *PrmObjectHead) UseKey(key ecdsa.PrivateKey) {
x.keySet = true
x.key = key
//
// Deprecated: Use PrmObjectHead.Key instead.
func (prm *PrmObjectHead) UseKey(key ecdsa.PrivateKey) {
prm.Key = &key
}
// ResObjectHead groups resulting values of ObjectHead operation.
@ -392,6 +388,58 @@ func (x *ResObjectHead) ReadHeader(dst *object.Object) bool {
return true
}
func (prm *PrmObjectHead) buildRequest(c *Client) (*v2object.HeadRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if prm.ObjectID == nil {
return nil, errorMissingObject
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if prm.Local {
meta.SetTTL(1)
}
addr := new(v2refs.Address)
cnrV2 := new(v2refs.ContainerID)
prm.ContainerID.WriteToV2(cnrV2)
addr.SetContainerID(cnrV2)
objV2 := new(v2refs.ObjectID)
prm.ObjectID.WriteToV2(objV2)
addr.SetObjectID(objV2)
body := new(v2object.HeadRequestBody)
body.SetRaw(prm.Raw)
body.SetAddress(addr)
req := new(v2object.HeadRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// ObjectHead reads object header through a remote server using FrostFS API protocol.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
@ -415,35 +463,24 @@ func (x *ResObjectHead) ReadHeader(dst *object.Object) bool {
// - *apistatus.ObjectAlreadyRemoved;
// - *apistatus.SessionTokenExpired.
func (c *Client) ObjectHead(ctx context.Context, prm PrmObjectHead) (*ResObjectHead, error) {
switch {
case ctx == nil:
return nil, errorMissingContext
case prm.addr.GetContainerID() == nil:
return nil, errorMissingContainer
case prm.addr.GetObjectID() == nil:
return nil, errorMissingObject
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
var body v2object.HeadRequestBody
body.SetRaw(prm.raw)
body.SetAddress(&prm.addr)
var req v2object.HeadRequest
req.SetBody(&body)
c.prepareRequest(&req, &prm.meta)
key := c.prm.key
if prm.keySet {
key = prm.key
if prm.Key != nil {
key = *prm.Key
}
// sign the request
err := signature.SignServiceMessage(&key, &req)
err = signature.SignServiceMessage(&key, req)
if err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.HeadObject(&c.c, &req, client.WithContext(ctx))
resp, err := rpcapi.HeadObject(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, fmt.Errorf("write request: %w", err)
}
@ -458,7 +495,7 @@ func (c *Client) ObjectHead(ctx context.Context, prm PrmObjectHead) (*ResObjectH
return &res, nil
}
_ = res.idObj.ReadFromV2(*prm.addr.GetObjectID())
res.idObj = *prm.ObjectID
switch v := resp.GetBody().GetHeaderPart().(type) {
default:
@ -474,29 +511,95 @@ func (c *Client) ObjectHead(ctx context.Context, prm PrmObjectHead) (*ResObjectH
// PrmObjectRange groups parameters of ObjectRange operation.
type PrmObjectRange struct {
prmObjectRead
XHeaders []string
key *ecdsa.PrivateKey
BearerToken *bearer.Token
rng v2object.Range
Session *session.Object
Raw bool
Local bool
ContainerID *cid.ID
ObjectID *oid.ID
Key *ecdsa.PrivateKey
Offset uint64
Length uint64
}
// SetOffset sets offset of the payload range to be read.
// Zero by default.
func (x *PrmObjectRange) SetOffset(off uint64) {
x.rng.SetOffset(off)
func (prm *PrmObjectRange) buildRequest(c *Client) (*v2object.GetRangeRequest, error) {
if prm.Length == 0 {
return nil, errorZeroRangeLength
}
// SetLength sets length of the payload range to be read.
// Must be positive.
func (x *PrmObjectRange) SetLength(ln uint64) {
x.rng.SetLength(ln)
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if prm.ObjectID == nil {
return nil, errorMissingObject
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if prm.Local {
meta.SetTTL(1)
}
addr := new(v2refs.Address)
cnrV2 := new(v2refs.ContainerID)
prm.ContainerID.WriteToV2(cnrV2)
addr.SetContainerID(cnrV2)
objV2 := new(v2refs.ObjectID)
prm.ObjectID.WriteToV2(objV2)
addr.SetObjectID(objV2)
rng := new(v2object.Range)
rng.SetLength(prm.Length)
rng.SetOffset(prm.Offset)
body := new(v2object.GetRangeRequestBody)
body.SetRaw(prm.Raw)
body.SetAddress(addr)
body.SetRange(rng)
req := new(v2object.GetRangeRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
func (x *PrmObjectRange) UseKey(key ecdsa.PrivateKey) {
x.key = &key
//
// Deprecated: Use PrmObjectRange.Key instead.
func (prm *PrmObjectRange) UseKey(key ecdsa.PrivateKey) {
prm.Key = &key
}
// ResObjectRange groups the final result values of ObjectRange operation.
@ -666,51 +769,31 @@ func (x *ObjectRangeReader) Read(p []byte) (int, error) {
// Returns an error if parameters are set incorrectly (see PrmObjectRange docs).
// Context is required and must not be nil. It is used for network communication.
func (c *Client) ObjectRangeInit(ctx context.Context, prm PrmObjectRange) (*ObjectRangeReader, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case prm.addr.GetContainerID() == nil:
return nil, errorMissingContainer
case prm.addr.GetObjectID() == nil:
return nil, errorMissingObject
case prm.rng.GetLength() == 0:
return nil, errorZeroRangeLength
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
// form request body
var body v2object.GetRangeRequestBody
body.SetRaw(prm.raw)
body.SetAddress(&prm.addr)
body.SetRange(&prm.rng)
// form request
var req v2object.GetRangeRequest
req.SetBody(&body)
c.prepareRequest(&req, &prm.meta)
key := prm.key
key := prm.Key
if key == nil {
key = &c.prm.key
}
err := signature.SignServiceMessage(key, &req)
err = signature.SignServiceMessage(key, req)
if err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
ctx, cancel := context.WithCancel(ctx)
stream, err := rpcapi.GetObjectRange(&c.c, &req, client.WithContext(ctx))
stream, err := rpcapi.GetObjectRange(&c.c, req, client.WithContext(ctx))
if err != nil {
cancel()
return nil, fmt.Errorf("open stream: %w", err)
}
var r ObjectRangeReader
r.remainingPayloadLen = int(prm.rng.GetLength())
r.remainingPayloadLen = int(prm.Length)
r.cancelCtxStream = cancel
r.stream = stream
r.client = c

View file

@ -13,121 +13,53 @@ import (
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/checksum"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmObjectHash groups parameters of ObjectHash operation.
type PrmObjectHash struct {
meta v2session.RequestMetaHeader
XHeaders []string
body v2object.GetRangeHashRequestBody
BearerToken *bearer.Token
csAlgo v2refs.ChecksumType
Session *session.Object
addr v2refs.Address
Local bool
keySet bool
key ecdsa.PrivateKey
Ranges []object.Range
Salt []byte
ChecksumType checksum.Type
ContainerID *cid.ID
ObjectID *oid.ID
Key *ecdsa.PrivateKey
}
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
func (x *PrmObjectHash) UseKey(key ecdsa.PrivateKey) {
x.keySet = true
x.key = key
}
// MarkLocal tells the server to execute the operation locally.
func (x *PrmObjectHash) MarkLocal() {
x.meta.SetTTL(1)
}
// WithinSession specifies session within which object should be read.
//
// Creator of the session acquires the authorship of the request.
// This may affect the execution of an operation (e.g. access control).
//
// Must be signed.
func (x *PrmObjectHash) WithinSession(t session.Object) {
var tv2 v2session.Token
t.WriteToV2(&tv2)
x.meta.SetSessionToken(&tv2)
}
// WithBearerToken attaches bearer token to be used for the operation.
//
// If set, underlying eACL rules will be used in access control.
//
// Must be signed.
func (x *PrmObjectHash) WithBearerToken(t bearer.Token) {
var v2token acl.BearerToken
t.WriteToV2(&v2token)
x.meta.SetBearerToken(&v2token)
}
// FromContainer specifies FrostFS container of the object.
// Required parameter.
func (x *PrmObjectHash) FromContainer(id cid.ID) {
var cidV2 v2refs.ContainerID
id.WriteToV2(&cidV2)
x.addr.SetContainerID(&cidV2)
}
// ByID specifies identifier of the requested object.
// Required parameter.
func (x *PrmObjectHash) ByID(id oid.ID) {
var idV2 v2refs.ObjectID
id.WriteToV2(&idV2)
x.addr.SetObjectID(&idV2)
}
// SetRangeList sets list of ranges in (offset, length) pair format.
// Required parameter.
//
// If passed as slice, then it must not be mutated before the operation completes.
func (x *PrmObjectHash) SetRangeList(r ...uint64) {
ln := len(r)
if ln%2 != 0 {
panic("odd number of range parameters")
}
rs := make([]v2object.Range, ln/2)
for i := 0; i < ln/2; i++ {
rs[i].SetOffset(r[2*i])
rs[i].SetLength(r[2*i+1])
}
x.body.SetRanges(rs)
// Deprecated: Use PrmObjectHash.Key instead.
func (prm *PrmObjectHash) UseKey(key ecdsa.PrivateKey) {
prm.Key = &key
}
// TillichZemorAlgo changes the hash function to Tillich-Zemor
// (https://link.springer.com/content/pdf/10.1007/3-540-48658-5_5.pdf).
//
// By default, SHA256 hash function is used.
func (x *PrmObjectHash) TillichZemorAlgo() {
x.csAlgo = v2refs.TillichZemor
}
// UseSalt sets the salt to XOR the data range before hashing.
// By default, SHA256 hash function is used/.
//
// Must not be mutated before the operation completes.
func (x *PrmObjectHash) UseSalt(salt []byte) {
x.body.SetSalt(salt)
}
// WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length.
//
// Slice must not be mutated until the operation completes.
func (x *PrmObjectHash) WithXHeaders(hs ...string) {
writeXHeadersToMeta(hs, &x.meta)
// Deprecated: Use PrmObjectHash.ChecksumType instead.
func (prm *PrmObjectHash) TillichZemorAlgo() {
prm.ChecksumType = checksum.TZ
}
// ResObjectHash groups resulting values of ObjectHash operation.
@ -142,6 +74,76 @@ func (x ResObjectHash) Checksums() [][]byte {
return x.checksums
}
func (prm *PrmObjectHash) buildRequest(c *Client) (*v2object.GetRangeHashRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if prm.ObjectID == nil {
return nil, errorMissingObject
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
if len(prm.Ranges) == 0 {
return nil, errorMissingRanges
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if prm.Local {
meta.SetTTL(1)
}
addr := new(v2refs.Address)
cnrV2 := new(v2refs.ContainerID)
prm.ContainerID.WriteToV2(cnrV2)
addr.SetContainerID(cnrV2)
objV2 := new(v2refs.ObjectID)
prm.ObjectID.WriteToV2(objV2)
addr.SetObjectID(objV2)
rs := make([]v2object.Range, len(prm.Ranges))
for i := range prm.Ranges {
rs[i].SetOffset(prm.Ranges[i].GetOffset())
rs[i].SetLength(prm.Ranges[i].GetLength())
}
body := new(v2object.GetRangeHashRequestBody)
body.SetAddress(addr)
body.SetRanges(rs)
body.SetSalt(prm.Salt)
if prm.ChecksumType == checksum.Unknown {
body.SetType(v2refs.SHA256)
} else {
body.SetType(v2refs.ChecksumType(prm.ChecksumType))
}
req := new(v2object.GetRangeHashRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// ObjectHash requests checksum of the range list of the object payload using
// FrostFS API protocol.
//
@ -165,39 +167,22 @@ func (x ResObjectHash) Checksums() [][]byte {
// - *apistatus.ObjectOutOfRange;
// - *apistatus.SessionTokenExpired.
func (c *Client) ObjectHash(ctx context.Context, prm PrmObjectHash) (*ResObjectHash, error) {
switch {
case ctx == nil:
return nil, errorMissingContext
case prm.addr.GetContainerID() == nil:
return nil, errorMissingContainer
case prm.addr.GetObjectID() == nil:
return nil, errorMissingObject
case len(prm.body.GetRanges()) == 0:
return nil, errorMissingRanges
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
prm.body.SetAddress(&prm.addr)
if prm.csAlgo == v2refs.UnknownChecksum {
prm.body.SetType(v2refs.SHA256)
} else {
prm.body.SetType(prm.csAlgo)
}
var req v2object.GetRangeHashRequest
c.prepareRequest(&req, &prm.meta)
req.SetBody(&prm.body)
key := c.prm.key
if prm.keySet {
key = prm.key
if prm.Key != nil {
key = *prm.Key
}
err := signature.SignServiceMessage(&key, &req)
err = signature.SignServiceMessage(&key, req)
if err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.HashObjectRange(&c.c, &req, client.WithContext(ctx))
resp, err := rpcapi.HashObjectRange(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, fmt.Errorf("write request: %w", err)
}

View file

@ -3,28 +3,29 @@ package client
import (
"context"
"crypto/ecdsa"
"errors"
"fmt"
"io"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// defaultGRPCPayloadChunkLen default value for maxChunkLen.
// See PrmObjectPutInit.SetGRPCPayloadChunkLen for details.
const defaultGRPCPayloadChunkLen = 3 << 20
// PrmObjectPutInit groups parameters of ObjectPutInit operation.
type PrmObjectPutInit struct {
copyNum []uint32
key *ecdsa.PrivateKey
meta v2session.RequestMetaHeader
maxChunkLen int
maxSize uint64
epochSource transformer.EpochSource
withoutHomomorphicHash bool
}
// SetCopiesNumber sets number of object copies that is enough to consider put successful.
@ -39,6 +40,15 @@ func (x *PrmObjectPutInit) SetCopiesNumberByVectors(copiesNumbers []uint32) {
x.copyNum = copiesNumbers
}
// SetGRPCPayloadChunkLen sets maximum chunk length value for gRPC Put request.
// Maximum chunk length restricts maximum byte length of the chunk
// transmitted in a single stream message. It depends on
// server settings and other message fields.
// If not specified or negative value set, default value of 3MiB will be used.
func (x *PrmObjectPutInit) SetGRPCPayloadChunkLen(v int) {
x.maxChunkLen = v
}
// ResObjectPut groups the final result values of ObjectPutInit operation.
type ResObjectPut struct {
statusRes
@ -51,29 +61,35 @@ func (x ResObjectPut) StoredObjectID() oid.ID {
return x.obj
}
// ObjectWriter is designed to write one object to FrostFS system.
// ObjectWriter is designed to write one object or
// multiple parts of one object to FrostFS system.
//
// Must be initialized using Client.ObjectPutInit, any other
// usage is unsafe.
type ObjectWriter struct {
cancelCtxStream context.CancelFunc
client *Client
stream interface {
Write(*v2object.PutRequest) error
Close() error
}
key *ecdsa.PrivateKey
res ResObjectPut
err error
chunkCalled bool
respV2 v2object.PutResponse
req v2object.PutRequest
partInit v2object.PutObjectPartInit
partChunk v2object.PutObjectPartChunk
type ObjectWriter interface {
// WriteHeader writes header of the object. Result means success.
// Failure reason can be received via Close.
WriteHeader(context.Context, object.Object) bool
// WritePayloadChunk writes chunk of the object payload. Result means success.
// Failure reason can be received via Close.
WritePayloadChunk(context.Context, []byte) bool
// Close ends writing the object and returns the result of the operation
// along with the final results. Must be called after using the ObjectWriter.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as Go built-in error.
// If Client is tuned to resolve FrostFS API statuses, then FrostFS failures
// codes are returned as error.
//
// Return statuses:
// - global (see Client docs);
// - *apistatus.ContainerNotFound;
// - *apistatus.ObjectAccessDenied;
// - *apistatus.ObjectLocked;
// - *apistatus.LockNonRegularObject;
// - *apistatus.SessionTokenNotFound;
// - *apistatus.SessionTokenExpired.
Close(context.Context) (*ResObjectPut, error)
}
// UseKey specifies private key to sign the requests.
@ -112,129 +128,21 @@ func (x *PrmObjectPutInit) WithXHeaders(hs ...string) {
writeXHeadersToMeta(hs, &x.meta)
}
// WriteHeader writes header of the object. Result means success.
// Failure reason can be received via Close.
func (x *ObjectWriter) WriteHeader(hdr object.Object) bool {
v2Hdr := hdr.ToV2()
x.partInit.SetObjectID(v2Hdr.GetObjectID())
x.partInit.SetHeader(v2Hdr.GetHeader())
x.partInit.SetSignature(v2Hdr.GetSignature())
x.req.GetBody().SetObjectPart(&x.partInit)
x.req.SetVerificationHeader(nil)
x.err = signature.SignServiceMessage(x.key, &x.req)
if x.err != nil {
x.err = fmt.Errorf("sign message: %w", x.err)
return false
// WithObjectMaxSize specifies max object size value and use it during object splitting.
// When specified, start writing to the stream only after the object is formed.
// Continue processing the input only when the previous formed object has been successfully written.
func (x *PrmObjectPutInit) WithObjectMaxSize(maxSize uint64) {
x.maxSize = maxSize
}
x.err = x.stream.Write(&x.req)
return x.err == nil
// WithoutHomomorphicHash if set to true do not use Tillich-Zémor hash for payload.
func (x *PrmObjectPutInit) WithoutHomomorphicHash(v bool) {
x.withoutHomomorphicHash = v
}
// WritePayloadChunk writes chunk of the object payload. Result means success.
// Failure reason can be received via Close.
func (x *ObjectWriter) WritePayloadChunk(chunk []byte) bool {
if !x.chunkCalled {
x.chunkCalled = true
x.req.GetBody().SetObjectPart(&x.partChunk)
}
for ln := len(chunk); ln > 0; ln = len(chunk) {
// maxChunkLen restricts maximum byte length of the chunk
// transmitted in a single stream message. It depends on
// server settings and other message fields, but for now
// we simply assume that 3MB is large enough to reduce the
// number of messages, and not to exceed the limit
// (4MB by default for gRPC servers).
const maxChunkLen = 3 << 20
if ln > maxChunkLen {
ln = maxChunkLen
}
// we deal with size limit overflow above, but there is another case:
// what if method is called with "small" chunk many times? We write
// a message to the stream on each call. Alternatively, we could use buffering.
// In most cases, the chunk length does not vary between calls. Given this
// assumption, as well as the length of the payload from the header, it is
// possible to buffer the data of intermediate chunks, and send a message when
// the allocated buffer is filled, or when the last chunk is received.
// It is mentally assumed that allocating and filling the buffer is better than
// synchronous sending, but this needs to be tested.
x.partChunk.SetChunk(chunk[:ln])
x.req.SetVerificationHeader(nil)
x.err = signature.SignServiceMessage(x.key, &x.req)
if x.err != nil {
x.err = fmt.Errorf("sign message: %w", x.err)
return false
}
x.err = x.stream.Write(&x.req)
if x.err != nil {
return false
}
chunk = chunk[ln:]
}
return true
}
// Close ends writing the object and returns the result of the operation
// along with the final results. Must be called after using the ObjectWriter.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as Go built-in error.
// If Client is tuned to resolve FrostFS API statuses, then FrostFS failures
// codes are returned as error.
//
// Return statuses:
// - global (see Client docs);
// - *apistatus.ContainerNotFound;
// - *apistatus.ObjectAccessDenied;
// - *apistatus.ObjectLocked;
// - *apistatus.LockNonRegularObject;
// - *apistatus.SessionTokenNotFound;
// - *apistatus.SessionTokenExpired.
func (x *ObjectWriter) Close() (*ResObjectPut, error) {
defer x.cancelCtxStream()
// Ignore io.EOF error, because it is expected error for client-side
// stream termination by the server. E.g. when stream contains invalid
// message. Server returns an error in response message (in status).
if x.err != nil && !errors.Is(x.err, io.EOF) {
return nil, x.err
}
if x.err = x.stream.Close(); x.err != nil {
return nil, x.err
}
x.res.st, x.err = x.client.processResponse(&x.respV2)
if x.err != nil {
return nil, x.err
}
if !apistatus.IsSuccessful(x.res.st) {
return &x.res, nil
}
const fieldID = "ID"
idV2 := x.respV2.GetBody().GetObjectID()
if idV2 == nil {
return nil, newErrMissingResponseField(fieldID)
}
x.err = x.res.obj.ReadFromV2(*idV2)
if x.err != nil {
x.err = newErrInvalidResponseField(fieldID, x.err)
}
return &x.res, nil
// WithEpochSource specifies epoch for object when split it on client side.
func (x *PrmObjectPutInit) WithEpochSource(es transformer.EpochSource) {
x.epochSource = es
}
// ObjectPutInit initiates writing an object through a remote server using FrostFS API protocol.
@ -244,31 +152,9 @@ func (x *ObjectWriter) Close() (*ResObjectPut, error) {
//
// Returns an error if parameters are set incorrectly.
// Context is required and must not be nil. It is used for network communication.
func (c *Client) ObjectPutInit(ctx context.Context, prm PrmObjectPutInit) (*ObjectWriter, error) {
// check parameters
if ctx == nil {
return nil, errorMissingContext
func (c *Client) ObjectPutInit(ctx context.Context, prm PrmObjectPutInit) (ObjectWriter, error) {
if prm.maxSize > 0 {
return c.objectPutInitTransformer(prm)
}
var w ObjectWriter
ctx, cancel := context.WithCancel(ctx)
stream, err := rpcapi.PutObject(&c.c, &w.respV2, client.WithContext(ctx))
if err != nil {
cancel()
return nil, fmt.Errorf("open stream: %w", err)
}
w.key = &c.prm.key
if prm.key != nil {
w.key = prm.key
}
w.cancelCtxStream = cancel
w.client = c
w.stream = stream
w.partInit.SetCopiesNumber(prm.copyNum)
w.req.SetBody(new(v2object.PutRequestBody))
c.prepareRequest(&w.req, &prm.meta)
return &w, nil
return c.objectPutInitRaw(ctx, prm)
}

154
client/object_put_raw.go Normal file
View file

@ -0,0 +1,154 @@
package client
import (
"context"
"crypto/ecdsa"
"errors"
"fmt"
"io"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
)
func (c *Client) objectPutInitRaw(ctx context.Context, prm PrmObjectPutInit) (*objectWriterRaw, error) {
var w objectWriterRaw
stream, err := rpcapi.PutObject(&c.c, &w.respV2, client.WithContext(ctx))
if err != nil {
return nil, fmt.Errorf("open stream: %w", err)
}
w.key = &c.prm.key
if prm.key != nil {
w.key = prm.key
}
w.client = c
w.stream = stream
w.partInit.SetCopiesNumber(prm.copyNum)
w.req.SetBody(new(v2object.PutRequestBody))
if prm.maxChunkLen > 0 {
w.maxChunkLen = prm.maxChunkLen
} else {
w.maxChunkLen = defaultGRPCPayloadChunkLen
}
c.prepareRequest(&w.req, &prm.meta)
return &w, nil
}
type objectWriterRaw struct {
client *Client
stream interface {
Write(*v2object.PutRequest) error
Close() error
}
key *ecdsa.PrivateKey
res ResObjectPut
err error
chunkCalled bool
respV2 v2object.PutResponse
req v2object.PutRequest
partInit v2object.PutObjectPartInit
partChunk v2object.PutObjectPartChunk
maxChunkLen int
}
func (x *objectWriterRaw) WriteHeader(_ context.Context, hdr object.Object) bool {
v2Hdr := hdr.ToV2()
x.partInit.SetObjectID(v2Hdr.GetObjectID())
x.partInit.SetHeader(v2Hdr.GetHeader())
x.partInit.SetSignature(v2Hdr.GetSignature())
x.req.GetBody().SetObjectPart(&x.partInit)
x.req.SetVerificationHeader(nil)
x.err = signature.SignServiceMessage(x.key, &x.req)
if x.err != nil {
x.err = fmt.Errorf("sign message: %w", x.err)
return false
}
x.err = x.stream.Write(&x.req)
return x.err == nil
}
func (x *objectWriterRaw) WritePayloadChunk(_ context.Context, chunk []byte) bool {
if !x.chunkCalled {
x.chunkCalled = true
x.req.GetBody().SetObjectPart(&x.partChunk)
}
for ln := len(chunk); ln > 0; ln = len(chunk) {
if ln > x.maxChunkLen {
ln = x.maxChunkLen
}
// we deal with size limit overflow above, but there is another case:
// what if method is called with "small" chunk many times? We write
// a message to the stream on each call. Alternatively, we could use buffering.
// In most cases, the chunk length does not vary between calls. Given this
// assumption, as well as the length of the payload from the header, it is
// possible to buffer the data of intermediate chunks, and send a message when
// the allocated buffer is filled, or when the last chunk is received.
// It is mentally assumed that allocating and filling the buffer is better than
// synchronous sending, but this needs to be tested.
x.partChunk.SetChunk(chunk[:ln])
x.req.SetVerificationHeader(nil)
x.err = signature.SignServiceMessage(x.key, &x.req)
if x.err != nil {
x.err = fmt.Errorf("sign message: %w", x.err)
return false
}
x.err = x.stream.Write(&x.req)
if x.err != nil {
return false
}
chunk = chunk[ln:]
}
return true
}
func (x *objectWriterRaw) Close(_ context.Context) (*ResObjectPut, error) {
// Ignore io.EOF error, because it is expected error for client-side
// stream termination by the server. E.g. when stream contains invalid
// message. Server returns an error in response message (in status).
if x.err != nil && !errors.Is(x.err, io.EOF) {
return nil, x.err
}
if x.err = x.stream.Close(); x.err != nil {
return nil, x.err
}
x.res.st, x.err = x.client.processResponse(&x.respV2)
if x.err != nil {
return nil, x.err
}
if !apistatus.IsSuccessful(x.res.st) {
return &x.res, nil
}
const fieldID = "ID"
idV2 := x.respV2.GetBody().GetObjectID()
if idV2 == nil {
return nil, newErrMissingResponseField(fieldID)
}
x.err = x.res.obj.ReadFromV2(*idV2)
if x.err != nil {
x.err = newErrInvalidResponseField(fieldID, x.err)
}
return &x.res, nil
}

116
client/object_put_single.go Normal file
View file

@ -0,0 +1,116 @@
package client
import (
"context"
"crypto/ecdsa"
"fmt"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmObjectPutSingle groups parameters of PutSingle operation.
type PrmObjectPutSingle struct {
copyNum []uint32
meta v2session.RequestMetaHeader
object *v2object.Object
key *ecdsa.PrivateKey
}
// SetCopiesNumber sets ordered list of minimal required object copies numbers
// per placement vector. List's length MUST equal container's placement vector number,
// otherwise request will fail.
func (x *PrmObjectPutSingle) SetCopiesNumber(v []uint32) {
x.copyNum = v
}
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
func (x *PrmObjectPutSingle) UseKey(key *ecdsa.PrivateKey) {
x.key = key
}
// WithBearerToken attaches bearer token to be used for the operation.
// Should be called once before any writing steps.
func (x *PrmObjectPutSingle) WithBearerToken(t bearer.Token) {
v2token := &acl.BearerToken{}
t.WriteToV2(v2token)
x.meta.SetBearerToken(v2token)
}
// WithinSession specifies session within which object should be stored.
// Should be called once before any writing steps.
func (x *PrmObjectPutSingle) WithinSession(t session.Object) {
tv2 := &v2session.Token{}
t.WriteToV2(tv2)
x.meta.SetSessionToken(tv2)
}
// ExecuteLocal tells the server to execute the operation locally.
func (x *PrmObjectPutSingle) ExecuteLocal() {
x.meta.SetTTL(1)
}
// WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length.
//
// Slice must not be mutated until the operation completes.
func (x *PrmObjectPutSingle) WithXHeaders(hs ...string) {
writeXHeadersToMeta(hs, &x.meta)
}
// SetObject specifies prepared object to put.
func (x *PrmObjectPutSingle) SetObject(o *v2object.Object) {
x.object = o
}
// ResObjectPutSingle groups resulting values of PutSingle operation.
type ResObjectPutSingle struct {
statusRes
}
// ObjectPutSingle writes prepared object to FrostFS.
// Object must have payload, also containerID, objectID, ownerID, payload hash, payload length of an object must be set.
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as Go built-in error.
// If Client is tuned to resolve FrostFS API statuses, then FrostFS failures
// codes are returned as error.
func (c *Client) ObjectPutSingle(ctx context.Context, prm PrmObjectPutSingle) (*ResObjectPutSingle, error) {
body := &v2object.PutSingleRequestBody{}
body.SetCopiesNumber(prm.copyNum)
body.SetObject(prm.object)
req := &v2object.PutSingleRequest{}
req.SetBody(body)
c.prepareRequest(req, &prm.meta)
key := &c.prm.key
if prm.key != nil {
key = prm.key
}
err := signature.SignServiceMessage(key, req)
if err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.PutSingleObject(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResObjectPutSingle
res.st, err = c.processResponse(resp)
if err != nil {
return nil, err
}
return &res, nil
}

View file

@ -0,0 +1,119 @@
package client
import (
"context"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
func (c *Client) objectPutInitTransformer(prm PrmObjectPutInit) (*objectWriterTransformer, error) {
var w objectWriterTransformer
w.it = internalTarget{
client: c,
prm: prm,
}
key := &c.prm.key
if prm.key != nil {
key = prm.key
}
w.ot = transformer.NewPayloadSizeLimiter(transformer.Params{
Key: key,
NextTargetInit: func() transformer.ObjectWriter { return &w.it },
MaxSize: prm.maxSize,
WithoutHomomorphicHash: prm.withoutHomomorphicHash,
NetworkState: prm.epochSource,
})
return &w, nil
}
type objectWriterTransformer struct {
ot transformer.ChunkedObjectWriter
it internalTarget
err error
}
func (x *objectWriterTransformer) WriteHeader(ctx context.Context, hdr object.Object) bool {
x.err = x.ot.WriteHeader(ctx, &hdr)
return x.err == nil
}
func (x *objectWriterTransformer) WritePayloadChunk(ctx context.Context, chunk []byte) bool {
_, x.err = x.ot.Write(ctx, chunk)
return x.err == nil
}
func (x *objectWriterTransformer) Close(ctx context.Context) (*ResObjectPut, error) {
ai, err := x.ot.Close(ctx)
if err != nil {
return nil, err
}
if ai != nil && ai.ParentID != nil {
x.it.res.obj = *ai.ParentID
}
return x.it.res, nil
}
type internalTarget struct {
client *Client
res *ResObjectPut
prm PrmObjectPutInit
useStream bool
}
func (it *internalTarget) WriteObject(ctx context.Context, o *object.Object) error {
putSingleImplemented, err := it.tryPutSingle(ctx, o)
if putSingleImplemented {
return err
}
it.useStream = true
return it.putAsStream(ctx, o)
}
func (it *internalTarget) putAsStream(ctx context.Context, o *object.Object) error {
wrt, err := it.client.objectPutInitRaw(ctx, it.prm)
if err != nil {
return err
}
if wrt.WriteHeader(ctx, *o) {
wrt.WritePayloadChunk(ctx, o.Payload())
}
it.res, err = wrt.Close(ctx)
if err == nil && !it.client.prm.resolveFrostFSErrors && !apistatus.IsSuccessful(it.res.st) {
err = apistatus.ErrFromStatus(it.res.st)
}
return err
}
func (it *internalTarget) tryPutSingle(ctx context.Context, o *object.Object) (bool, error) {
if it.useStream {
return false, nil
}
var prm PrmObjectPutSingle
prm.SetCopiesNumber(it.prm.copyNum)
prm.SetObject(o.ToV2())
prm.UseKey(prm.key)
prm.meta = it.prm.meta
res, err := it.client.ObjectPutSingle(ctx, prm)
if err != nil && status.Code(err) == codes.Unimplemented {
return false, err
}
if err == nil {
id, _ := o.ID()
it.res = &ResObjectPut{
statusRes: res.statusRes,
obj: id,
}
if !it.client.prm.resolveFrostFSErrors && !apistatus.IsSuccessful(it.res.st) {
return true, apistatus.ErrFromStatus(it.res.st)
}
return true, nil
}
return true, err
}

View file

@ -222,10 +222,7 @@ func (x *ObjectListReader) Close() (*ResObjectSearch, error) {
// Context is required and must not be nil. It is used for network communication.
func (c *Client) ObjectSearchInit(ctx context.Context, prm PrmObjectSearch) (*ObjectListReader, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case !prm.cnrSet:
if !prm.cnrSet {
return nil, errorMissingContainer
}

View file

@ -1,200 +0,0 @@
package client
import (
"context"
v2reputation "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/reputation"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/reputation"
)
// PrmAnnounceLocalTrust groups parameters of AnnounceLocalTrust operation.
type PrmAnnounceLocalTrust struct {
prmCommonMeta
epoch uint64
trusts []reputation.Trust
}
// SetEpoch sets number of FrostFS epoch in which the trust was assessed.
// Required parameter, must not be zero.
func (x *PrmAnnounceLocalTrust) SetEpoch(epoch uint64) {
x.epoch = epoch
}
// SetValues sets values describing trust of the client to the FrostFS network participants.
// Required parameter. Must not be empty.
//
// Must not be mutated before the end of the operation.
func (x *PrmAnnounceLocalTrust) SetValues(trusts []reputation.Trust) {
x.trusts = trusts
}
// ResAnnounceLocalTrust groups results of AnnounceLocalTrust operation.
type ResAnnounceLocalTrust struct {
statusRes
}
// AnnounceLocalTrust sends client's trust values to the FrostFS network participants.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Returns an error if parameters are set incorrectly (see PrmAnnounceLocalTrust docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) AnnounceLocalTrust(ctx context.Context, prm PrmAnnounceLocalTrust) (*ResAnnounceLocalTrust, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case prm.epoch == 0:
return nil, errorZeroEpoch
case len(prm.trusts) == 0:
return nil, errorMissingTrusts
}
// form request body
reqBody := new(v2reputation.AnnounceLocalTrustRequestBody)
reqBody.SetEpoch(prm.epoch)
trusts := make([]v2reputation.Trust, len(prm.trusts))
for i := range prm.trusts {
prm.trusts[i].WriteToV2(&trusts[i])
}
reqBody.SetTrusts(trusts)
// form request
var req v2reputation.AnnounceLocalTrustRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResAnnounceLocalTrust
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.AnnounceLocalTrust(&c.c, &req, client.WithContext(ctx))
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmAnnounceIntermediateTrust groups parameters of AnnounceIntermediateTrust operation.
type PrmAnnounceIntermediateTrust struct {
prmCommonMeta
epoch uint64
iter uint32
trustSet bool
trust reputation.PeerToPeerTrust
}
// SetEpoch sets number of FrostFS epoch with which client's calculation algorithm is initialized.
// Required parameter, must not be zero.
func (x *PrmAnnounceIntermediateTrust) SetEpoch(epoch uint64) {
x.epoch = epoch
}
// SetIteration sets current sequence number of the client's calculation algorithm.
// By default, corresponds to initial (zero) iteration.
func (x *PrmAnnounceIntermediateTrust) SetIteration(iter uint32) {
x.iter = iter
}
// SetCurrentValue sets current global trust value computed at the specified iteration
// of the client's calculation algorithm. Required parameter.
func (x *PrmAnnounceIntermediateTrust) SetCurrentValue(trust reputation.PeerToPeerTrust) {
x.trust = trust
x.trustSet = true
}
// ResAnnounceIntermediateTrust groups results of AnnounceIntermediateTrust operation.
type ResAnnounceIntermediateTrust struct {
statusRes
}
// AnnounceIntermediateTrust sends global trust values calculated for the specified FrostFS network participants
// at some stage of client's calculation algorithm.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Returns an error if parameters are set incorrectly (see PrmAnnounceIntermediateTrust docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) AnnounceIntermediateTrust(ctx context.Context, prm PrmAnnounceIntermediateTrust) (*ResAnnounceIntermediateTrust, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case prm.epoch == 0:
return nil, errorZeroEpoch
case !prm.trustSet:
return nil, errorTrustNotSet
}
var trust v2reputation.PeerToPeerTrust
prm.trust.WriteToV2(&trust)
// form request body
reqBody := new(v2reputation.AnnounceIntermediateResultRequestBody)
reqBody.SetEpoch(prm.epoch)
reqBody.SetIteration(prm.iter)
reqBody.SetTrust(&trust)
// form request
var req v2reputation.AnnounceIntermediateResultRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResAnnounceIntermediateTrust
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.AnnounceIntermediateResult(&c.c, &req, client.WithContext(ctx))
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}

View file

@ -3,34 +3,63 @@ package client
import (
"context"
"crypto/ecdsa"
"fmt"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
)
// PrmSessionCreate groups parameters of SessionCreate operation.
type PrmSessionCreate struct {
prmCommonMeta
XHeaders []string
exp uint64
Expiration uint64
keySet bool
key ecdsa.PrivateKey
Key *ecdsa.PrivateKey
}
// SetExp sets number of the last NepFS epoch in the lifetime of the session after which it will be expired.
//
// Deprecated: Use PrmSessionCreate.Expiration instead.
func (x *PrmSessionCreate) SetExp(exp uint64) {
x.exp = exp
x.Expiration = exp
}
// UseKey specifies private key to sign the requests and compute token owner.
// If key is not provided, then Client default key is used.
//
// Deprecated: Use PrmSessionCreate.Key instead.
func (x *PrmSessionCreate) UseKey(key ecdsa.PrivateKey) {
x.keySet = true
x.key = key
x.Key = &key
}
func (x *PrmSessionCreate) buildRequest(c *Client) (*v2session.CreateRequest, error) {
ownerKey := c.prm.key.PublicKey
if x.Key != nil {
ownerKey = x.Key.PublicKey
}
var ownerID user.ID
user.IDFromKey(&ownerID, ownerKey)
var ownerIDV2 refs.OwnerID
ownerID.WriteToV2(&ownerIDV2)
reqBody := new(v2session.CreateRequestBody)
reqBody.SetOwnerID(&ownerIDV2)
reqBody.SetExpiration(x.Expiration)
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(x.XHeaders, &meta)
var req v2session.CreateRequest
req.SetBody(reqBody)
c.prepareRequest(&req, &meta)
return &req, nil
}
// ResSessionCreate groups resulting values of SessionCreate operation.
@ -42,10 +71,6 @@ type ResSessionCreate struct {
sessionKey []byte
}
func (x *ResSessionCreate) setID(id []byte) {
x.id = id
}
// ID returns identifier of the opened session in a binary FrostFS API protocol format.
//
// Client doesn't retain value so modification is safe.
@ -53,10 +78,6 @@ func (x ResSessionCreate) ID() []byte {
return x.id
}
func (x *ResSessionCreate) setSessionKey(key []byte) {
x.sessionKey = key
}
// PublicKey returns public key of the opened session in a binary FrostFS API protocol format.
func (x ResSessionCreate) PublicKey() []byte {
return x.sessionKey
@ -78,62 +99,28 @@ func (x ResSessionCreate) PublicKey() []byte {
// Return statuses:
// - global (see Client docs).
func (c *Client) SessionCreate(ctx context.Context, prm PrmSessionCreate) (*ResSessionCreate, error) {
// check context
if ctx == nil {
return nil, errorMissingContext
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
ownerKey := c.prm.key.PublicKey
if prm.keySet {
ownerKey = prm.key.PublicKey
}
var ownerID user.ID
user.IDFromKey(&ownerID, ownerKey)
var ownerIDV2 refs.OwnerID
ownerID.WriteToV2(&ownerIDV2)
// form request body
reqBody := new(v2session.CreateRequestBody)
reqBody.SetOwnerID(&ownerIDV2)
reqBody.SetExpiration(prm.exp)
// for request
var req v2session.CreateRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResSessionCreate
)
c.initCallContext(&cc)
if prm.keySet {
cc.key = prm.key
if err := signature.SignServiceMessage(&c.prm.key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.CreateSession(&c.c, &req, client.WithContext(ctx))
resp, err := rpcapi.CreateSession(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResSessionCreate
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
cc.result = func(r responseV2) {
resp := r.(*v2session.CreateResponse)
body := resp.GetBody()
res.setID(body.GetID())
res.setSessionKey(body.GetSessionKey())
}
// process call
if !cc.processCall() {
return nil, cc.err
}
res.id = body.GetID()
res.sessionKey = body.GetSessionKey()
return &res, nil
}

View file

@ -14,7 +14,7 @@ type ServerInternal struct {
v2 status.Status
}
func (x ServerInternal) Error() string {
func (x *ServerInternal) Error() string {
return errMessageStatusV2(
globalizeCodeV2(status.Internal, status.GlobalizeCommonFail),
x.v2.Message(),
@ -62,7 +62,7 @@ type WrongMagicNumber struct {
v2 status.Status
}
func (x WrongMagicNumber) Error() string {
func (x *WrongMagicNumber) Error() string {
return errMessageStatusV2(
globalizeCodeV2(status.WrongMagicNumber, status.GlobalizeCommonFail),
x.v2.Message(),
@ -132,7 +132,7 @@ type SignatureVerification struct {
const defaultSignatureVerificationMsg = "signature verification failed"
func (x SignatureVerification) Error() string {
func (x *SignatureVerification) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultSignatureVerificationMsg
@ -191,7 +191,7 @@ type NodeUnderMaintenance struct {
const defaultNodeUnderMaintenanceMsg = "node is under maintenance"
// Error implements the error interface.
func (x NodeUnderMaintenance) Error() string {
func (x *NodeUnderMaintenance) Error() string {
msg := x.Message()
if msg == "" {
msg = defaultNodeUnderMaintenanceMsg

View file

@ -13,7 +13,7 @@ type ContainerNotFound struct {
const defaultContainerNotFoundMsg = "container not found"
func (x ContainerNotFound) Error() string {
func (x *ContainerNotFound) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultContainerNotFoundMsg
@ -51,7 +51,7 @@ type EACLNotFound struct {
const defaultEACLNotFoundMsg = "eACL not found"
func (x EACLNotFound) Error() string {
func (x *EACLNotFound) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultEACLNotFoundMsg

View file

@ -13,7 +13,7 @@ type ObjectLocked struct {
const defaultObjectLockedMsg = "object is locked"
func (x ObjectLocked) Error() string {
func (x *ObjectLocked) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultObjectLockedMsg
@ -50,7 +50,7 @@ type LockNonRegularObject struct {
const defaultLockNonRegularObjectMsg = "locking non-regular object is forbidden"
func (x LockNonRegularObject) Error() string {
func (x *LockNonRegularObject) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultLockNonRegularObjectMsg
@ -87,7 +87,7 @@ type ObjectAccessDenied struct {
const defaultObjectAccessDeniedMsg = "access to object operation denied"
func (x ObjectAccessDenied) Error() string {
func (x *ObjectAccessDenied) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultObjectAccessDeniedMsg
@ -135,7 +135,7 @@ type ObjectNotFound struct {
const defaultObjectNotFoundMsg = "object not found"
func (x ObjectNotFound) Error() string {
func (x *ObjectNotFound) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultObjectNotFoundMsg
@ -172,7 +172,7 @@ type ObjectAlreadyRemoved struct {
const defaultObjectAlreadyRemovedMsg = "object already removed"
func (x ObjectAlreadyRemoved) Error() string {
func (x *ObjectAlreadyRemoved) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultObjectAlreadyRemovedMsg
@ -210,7 +210,7 @@ type ObjectOutOfRange struct {
const defaultObjectOutOfRangeMsg = "out of range"
func (x ObjectOutOfRange) Error() string {
func (x *ObjectOutOfRange) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultObjectOutOfRangeMsg

View file

@ -13,7 +13,7 @@ type SessionTokenNotFound struct {
const defaultSessionTokenNotFoundMsg = "session token not found"
func (x SessionTokenNotFound) Error() string {
func (x *SessionTokenNotFound) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultSessionTokenNotFoundMsg
@ -50,7 +50,7 @@ type SessionTokenExpired struct {
const defaultSessionTokenExpiredMsg = "expired session token"
func (x SessionTokenExpired) Error() string {
func (x *SessionTokenExpired) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultSessionTokenExpiredMsg

View file

@ -15,7 +15,7 @@ package apistatus
// It should be noted that using direct typecasting is not a compatible approach.
//
// To transport statuses using the FrostFS API V2 protocol, see StatusV2 interface and FromStatusV2 and ToStatusV2 functions.
type Status interface{}
type Status any
// ErrFromStatus converts Status instance to error if it is failed. Returns nil on successful Status.
//

View file

@ -8,7 +8,7 @@ type unrecognizedStatusV2 struct {
v2 status.Status
}
func (x unrecognizedStatusV2) Error() string {
func (x *unrecognizedStatusV2) Error() string {
return errMessageStatusV2("unrecognized", x.v2.Message())
}

View file

@ -123,7 +123,7 @@ func ToStatusV2(st Status) *status.Status {
return internalErrorStatus
}
func errMessageStatusV2(code interface{}, msg string) string {
func errMessageStatusV2(code any, msg string) string {
const (
noMsgFmt = "status: code = %v"
msgFmt = noMsgFmt + " message = %s"

View file

@ -12,7 +12,7 @@ func TestToStatusV2(t *testing.T) {
type statusConstructor func() apistatus.Status
for _, testItem := range [...]struct {
status interface{} // Status or statusConstructor
status any // Status or statusConstructor
codeV2 uint64
messageV2 string
}{
@ -165,7 +165,7 @@ func TestFromStatusV2(t *testing.T) {
type statusConstructor func() apistatus.Status
for _, testItem := range [...]struct {
status interface{} // Status or statusConstructor
status any // Status or statusConstructor
codeV2 uint64
messageV2 string
}{

View file

@ -6,6 +6,7 @@ import (
"errors"
"fmt"
"strconv"
"strings"
"time"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
@ -16,7 +17,6 @@ import (
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap"
subnetid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/subnet/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/version"
"github.com/google/uuid"
@ -128,10 +128,7 @@ func checkAttributes(m container.Container) error {
}
var err error
switch key {
case container.SysAttributeSubnet:
err = new(subnetid.ID).DecodeString(val)
case attributeTimestamp:
if key == attributeTimestamp {
_, err = strconv.ParseInt(val, 10, 64)
}
@ -300,7 +297,7 @@ func (x Container) PlacementPolicy() (res netmap.PlacementPolicy) {
//
// SetAttribute overwrites existing attribute value.
//
// See also Attribute, IterateAttributes.
// See also Attribute, IterateAttributes, IterateUserAttributes.
func (x *Container) SetAttribute(key, value string) {
if key == "" {
panic("empty attribute key")
@ -328,7 +325,7 @@ func (x *Container) SetAttribute(key, value string) {
// Attribute reads value of the Container attribute by key. Empty result means
// attribute absence.
//
// See also SetAttribute, IterateAttributes.
// See also SetAttribute, IterateAttributes, IterateUserAttributes.
func (x Container) Attribute(key string) string {
attrs := x.v2.GetAttributes()
for i := range attrs {
@ -351,6 +348,21 @@ func (x Container) IterateAttributes(f func(key, val string)) {
}
}
// IterateUserAttributes iterates over user Container attributes and passes them
// into f. The handler MUST NOT be nil.
//
// See also SetAttribute, Attribute.
func (x Container) IterateUserAttributes(f func(key, val string)) {
attrs := x.v2.GetAttributes()
for _, attr := range attrs {
var key = attr.GetKey()
if !strings.HasPrefix(key, container.SysAttributePrefix) &&
!strings.HasPrefix(key, container.SysAttributePrefixNeoFS) {
f(key, attr.GetValue())
}
}
}
// SetName sets human-readable name of the Container. Name MUST NOT be empty.
//
// See also Name.
@ -391,28 +403,6 @@ func CreatedAt(cnr Container) time.Time {
return time.Unix(sec, 0)
}
// SetSubnet places the Container on the specified FrostFS subnet. If called,
// container nodes will only be selected from the given subnet, otherwise from
// the entire network.
func SetSubnet(cnr *Container, subNet subnetid.ID) {
cnr.SetAttribute(container.SysAttributeSubnet, subNet.EncodeToString())
}
// Subnet return container subnet set using SetSubnet.
//
// Zero Container is bound to zero subnet.
func Subnet(cnr Container) (res subnetid.ID) {
val := cnr.Attribute(container.SysAttributeSubnet)
if val != "" {
err := res.DecodeString(val)
if err != nil {
panic(fmt.Sprintf("invalid subnet attribute: %s (%v)", val, err))
}
}
return
}
const attributeHomoHashEnabled = "true"
// DisableHomomorphicHashing sets flag to disable homomorphic hashing of the

View file

@ -15,8 +15,6 @@ import (
containertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/test"
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
netmaptest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap/test"
subnetid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/subnet/id"
subnetidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/subnet/id/test"
usertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user/test"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/version"
"github.com/google/uuid"
@ -152,7 +150,7 @@ func assertContainsAttribute(t *testing.T, m v2container.Container, key, val str
}
func TestContainer_Attribute(t *testing.T) {
const attrKey1, attrKey2 = "key1", "key2"
const attrKey1, attrKey2 = v2container.SysAttributePrefix + "key1", v2container.SysAttributePrefixNeoFS + "key2"
const attrVal1, attrVal2 = "val1", "val2"
val := containertest.Container()
@ -160,6 +158,12 @@ func TestContainer_Attribute(t *testing.T) {
val.SetAttribute(attrKey1, attrVal1)
val.SetAttribute(attrKey2, attrVal2)
var i int
val.IterateUserAttributes(func(key, val string) {
i++
})
require.Equal(t, 1, i)
var msg v2container.Container
val.WriteToV2(&msg)
@ -233,28 +237,6 @@ func TestSetCreationTime(t *testing.T) {
require.Equal(t, creat.Unix(), container.CreatedAt(val2).Unix())
}
func TestSetSubnet(t *testing.T) {
var val container.Container
require.True(t, subnetid.IsZero(container.Subnet(val)))
val = containertest.Container()
sub := subnetidtest.ID()
container.SetSubnet(&val, sub)
var msg v2container.Container
val.WriteToV2(&msg)
assertContainsAttribute(t, msg, v2container.SysAttributeSubnet, sub.EncodeToString())
var val2 container.Container
require.NoError(t, val2.ReadFromV2(msg))
require.Equal(t, sub, container.Subnet(val))
}
func TestDisableHomomorphicHashing(t *testing.T) {
var val container.Container

View file

@ -1,8 +1,8 @@
package cid_test
import (
"crypto/rand"
"crypto/sha256"
"math/rand"
"testing"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"

View file

@ -1,8 +1,8 @@
package cidtest
import (
"crypto/rand"
"crypto/sha256"
"math/rand"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
)
@ -11,7 +11,7 @@ import (
func ID() cid.ID {
checksum := [sha256.Size]byte{}
rand.Read(checksum[:])
_, _ = rand.Read(checksum[:])
return IDWithChecksum(checksum)
}

View file

@ -1,7 +1,7 @@
package frostfscrypto_test
import (
"math/rand"
"crypto/rand"
"testing"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 12 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 23 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 8.8 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 12 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 11 KiB

446
doc/policy.md Normal file
View file

@ -0,0 +1,446 @@
# Placement Policy
This document describes placement policies, their purpose, syntax and semantics.
## Index
- [Introduction](#introduction)
- [Operations](#operations)
- [Basic Expressions](#basic-expressions)
- [`FILTER`](#filter)
- [`SELECT`](#select)
- [`REP`](#rep)
- [Policies](#policies)
- [The policy playground](#the-policy-playground)
- [`CBF`](#cbf)
- [`UNIQUE`](#unique)
- [More examples](#more-examples)
- [Appendix 1: Operators](#appendix-1-operators)
- [Appendix 2: Policy playground commands](#appendix-2-policy-playground-commands)
## Introduction
The purpose of a **placement policy** is to determine whichs node(s) of a frostfs system will store an object. Namely, given a **netmap** (a set of nodes) and a placement policy, a subset of those nodes is selected to store a given object. An important aspect is that since nodes in a netmap come and go due to distributed nature of the system, this selection must be deterministic and consistent, i.e. different nodes must come to the same conclusion as long as they share the same view of the netmap.
> Throughout this document, we will consider each node as a dictionary of attributes and a global unique ID.
One way to think about the placement policy, is as a pipeline of operations which begins with a set of nodes (the entire netmap) and gradually refine it into only the nodes that will be in charge of storing the object. More specifically, each operation in this pipeline takes a set of nodes and transforms it into a subset of those nodes. The transformation is done purely on the basis of the node attributes.
![Placement policy as a pipeline](./image/placement_policy.svg)
The three main operations are:
1. `FILTER`: filters a set of nodes based on their attributes.
2. `SELECT`: selects a specific amount of nodes from a set of nodes based on certain conditions.
3. `REP`: specifies how many nodes (and which ones) from a set of nodes are used to store an object.
In the next sections, we will explore each of them in detail.
## Operations
### Basic Expressions
Before exploring the operations in detail, we must get acquainted with the basic expressions that appear in a placement policy. As mentioned above, the placement policy operates solely on the basis of node attributes, and as such, basic expressions mostly revolve around node attribute comparison.
A comparison expression expresses whether a node attribute equals a specified value:
```
AttributeName Operation AttributeValue
```
For example, the following expression
```sql
City EQ 'Moscow'
```
asserts that the node attribute `City` equals the value `Moscow`.
Comparison expressions can be nested via boolean operators and parentheses. For example, the following expression:
```sql
(City EQ 'Moscow') AND (Disks GT 2)
```
asserts that the node attribute `City` equals the value `Moscow` and the node attribute `Disks` must be greater than `2`. Note that the arguments can be either a string or a number.
See [Appendix 1](#appendix-1-operators) for a complete list of supported operators.
### `FILTER`
A `FILTER` operation takes as input a set of nodes and returns a subset of those nodes. It's useful for selecting nodes that have (or lack) specific attributes. Its basic syntax is as follows:
```bnf
FILTER <expr> AS <id>
```
For example, the following filter
```sql
FILTER Color EQ 'Red' AS RedNodes
```
selects those nodes for which the `Color` attribute equals `Red`, and discards the rest. The filter's identifier is `RedNodes`, which can be used to reference it in other parts of the placement policy. For example, you could reference the above filter in another filter as follows
```sql
FILTER @RedNodes AND (City EQ 'Moscow') AS RedMoscowNodes
```
which would select those nodes for which the `Color` attribute equals `Red` and the `City` attribute equals `Moscow`. You can think of the `@` operator as embedding the referenced filter expression verbatim where it's used. This makes it easy to compose filters. However, filters can be referenced via `@` only within filter expressions. In other places you can simply use the filter identifier directly.
> ⚠️ Every filter requires a unique identifier. What would be the use of a filter that you cannot reference?
The following diagram illustrates the filter operation
![FILTER](./image/filter_illustration.svg)
where the nodes are represented as colored circles, with their color representing the value of their `Color` attribute, respectively.
> A filter referring to all nodes in the netmap always exists and can be referenced by `*`.
### `SELECT`
A `SELECT` operation specifies how many and which nodes from a subset previously obtained from a `FILTER` will be available to build replica groups for object storage. It's not that different from a `FILTER` in that it transforms a set of nodes into a subset of those, but while a `FILTER` cannot control the size of the resulting subset and other characteristics, a `SELECT` can.
Its basic syntax is as follows:
```bnf
SELECT <count> {IN (SAME|DISTINCT) <attribute>} FROM <filter> {AS <id>}
```
In a nutshell, a `SELECT` takes a filter result as input and outputs a specific number of nodes, optionally enforcing that all output nodes must either share or differ in a specific attribute. Note that only the output node count and the source filter are required.
Let's see some examples
```sql
-- Selects exactly one node from the entire netmap
SELECT 1 FROM *
-- Same as above, but with an identifier for the selection
SELECT 1 FROM * AS ONE
-- Selects two nodes from the RedOrBlueNodes filter, such that both selected nodes
-- share the same value for the Color attribute, i.e. both red or both blue.
SELECT 2 IN SAME Color FROM RedOrBlueNodes
-- Selects two nodes from the RedOrBlueNodes filter, such that the selected nodes
-- have distinct values for the Color attribute, i.e. one red and one blue.
-- The selection is also given an identifier.
SELECT 2 IN DISTINCT Color FROM RedOrBlueNodes AS MyNodes
```
The last example is illustrated in the following diagram:
![SELECT](./image/select_illustration.svg)
> At this point, notice that while `FILTER`'s output is always unique (namely, every node in the input is either filtered in or out), that is not always the case for `SELECT`. In the last example above, there is more than one way to select two nodes with distinct `Color` attribute. Because we require the output to be deterministic and consistent (so that all nodes agree on which nodes to store a given object without having to commmunicate with each other), we need a way to reach this consensus efficiently. Internally, the policy engine uses [Rendezvouz Hashing](https://en.wikipedia.org/wiki/Rendezvous_hashing) to ensure this. If you want more control over what nodes are actually selected, you can always use narrower filters/selections to ensure this.
### `REP`
A `REP` operation specifies how many copies of an object need to be stored (`REP` stands for "replica"). A placement policy can contain multiple replica operations, with each of them representing a replica group, i.e. a group of objects associated with the same replica. Following our analogy with a pipeline, `REP` operations are the sink or output nodes.
Its basic syntax is as follows:
```bnf
REP <count> {IN <select>}
```
If a select is not specified, then the entire netmap is used as input. The only exception to this rule is when exactly 1 replica and 1 selector are being present: in this case the only selector is being used instead of the whole netmap.
The resulting nodes will be used to actually store objects and they constitute a replica group (or simply, "a replica").
Examples
```sql
-- A replica consisting of a single copy, stored in an arbitrary node of the netmap.
REP 1
-- A replica consisting of three copies, each stored in a different node from the selection
-- identified as 'MyNodes'.
REP 3 IN MyNodes
```
The following diagram illustrates the `REP` operation:
![REP](./image/rep_illustration.svg)
> ⚠️ Notice that although we use `REP 1` in the examples, in real life scenarios you almost always want to have more than a single node in each replica for redundancy.
## Policies
In order to specify a complete placement policy, we just need to assemble it from the operations described above.
Its basic (simplified) syntax is as follows:
```bnf
<rep>+ <select>* <filter>*
```
We begin by stating all our `REP` operations, followed by all the `SELECT` operations and finally all the `FILTER` operations. Note that this is the reverse order in which they are applied. Also note that at least one `REP` operation is required.
Here's a complete example:
```sql
REP 1 IN MyNodes
SELECT 2 IN DISTINCT Color FROM RedOrBlueNodes AS MyNodes
FILTER Color EQ 'Red' AS RedNodes
FILTER Color EQ 'Blue' AS BlueNodes
FILTER @RedNodes OR @BlueNodes AS RedOrBlueNodes
```
In additional to this basic syntax, there are a couple of additional useful options to specify which nodes and how many nodes are actually selected to store objects. We explore these in the next sections.
### The policy playground
> This section assumes you have an up-to-date version of the `frostfs-cli`.
While simple placement policies have predictable results that can be understood at a glance, more complex ones need careful consideration before deployment. In order to simplify understanding a policy's outcome and experimenting while learning, a builtin tool is provided as part of the `frostfs-cli` for this purpose: the policy playground.
For the remainder of this guide, we will use the policy playground to setup a virtual netmap (that is, one that doesn't require any networking or deployment) and test various policies. In order to visualize this netmap easily, each node will have three attributes: a character, a shape and a color
![Sample Netmap](./image/sample_netmap.svg)
We can start the policy playground as follows:
```sh
$ frostfs-cli container policy-playground
>
```
Since we didn't pass any endpoint, the initial netmap is empty, which we can verify with the `ls` command (to list the nodes in the netmap):
```sh
> ls
>
```
Nows let's add virtual nodes to represent our test netmap in the figure above
```sh
> add 01 Char:A Shape:Circle Color:Blue
> add 02 Char:B Shape:Circle Color:Green
> add 03 Char:C Shape:Circle Color:Red
> add 04 Char:D Shape:Square Color:Blue
> add 05 Char:E Shape:Square Color:Green
> add 06 Char:F Shape:Square Color:Red
> add 07 Char:G Shape:Diamond Color:Blue
> add 08 Char:H Shape:Diamond Color:Green
> add 09 Char:I Shape:Diamond Color:Red
```
and verify that the netmap now contains what we expect
```sh
> ls
1: id=06 attrs={Char:F Shape:Square Color:Red}
2: id=08 attrs={Char:H Shape:Diamond Color:Green}
3: id=01 attrs={Char:A Shape:Circle Color:Blue}
4: id=04 attrs={Char:D Shape:Square Color:Blue}
5: id=05 attrs={Char:E Shape:Square Color:Green}
6: id=09 attrs={Char:I Shape:Diamond Color:Red}
7: id=02 attrs={Char:B Shape:Circle Color:Green}
8: id=03 attrs={Char:C Shape:Circle Color:Red}
9: id=07 attrs={Char:G Shape:Diamond Color:Blue}
```
With our sample netmap setup, we can now continue.
### `CBF`
Consider the following policy:
```sql
REP 1
```
It builds a replica consisting of one copy, selected from the entire netmap. If we evaluate this policy in our sample netmap, we obtain a result which is probably unexpected:
```sh
> eval REP 1
1: [06 05 02]
```
The `eval` commands evaluates a policy and lists in a separate line the nodes selected for each `REP` operation, in the order they appear in the policy. We were expecting a single node, but we got three instead. The reason is that there's a policy-wide parameter called **container backup factor** (or CBF). This parameter is a multiplier which controls the maximum number of storage nodes: for example, if a policy requires 10 nodes and the CBF is 3, it means that the policy can store an object in up to 10 × 3 nodes. However, if there are not enough nodes and fewer (but at least 10) are used, this is not considered an error.
The default value for CBF is `3`, which explains our result above, given than every node in the netmap agrees with the policy. The CBF can be explicitly set in the policy right after the `REP` operations. For example
```sh
> eval REP 1 CBF 1
1: [06]
```
results in what we expected in the first example. On the other hand
```sh
> eval REP 1 IN MyNodes SELECT 1 IN SAME Char FROM * AS MyNodes
1: [01]
```
results in a single node despite the default CBF, because there are not enough nodes compatible with the selection.
### `UNIQUE`
Consider the following policy:
```sql
REP 1
REP 1
CBF 2
```
If we evaluate it
```sh
> eval REP 1 REP 1 CBF 2
1: [06 05]
2: [06 05]
```
we find that each replica gets two nodes, in accordance with the CBF. However, these nodes are exactly the same and this might not be desirable. In order to force the policy engine to select different nodes for each replica, we can use the `UNIQUE` option, which is specified right before the `REP` operations.
In our example, if we change it to
```sql
UNIQUE
REP 1
REP 1
CBF 2
```
and evaluate it
```sh
> eval UNIQUE REP 1 REP 1 CBF 2
1: [06 05]
2: [02 03]
```
we now find that the nodes selected for each replica are now distinct from each other.
### More examples
This section presents some more examples of placement policies and their result when applied to the sample netmap. Try to figure out the result before looking at it or evaluating the policy.
#### Example #1
```sql
REP 1 IN TwoRedNodes
SELECT 2 FROM RedNodes AS TwoRedNodes
FILTER Color EQ 'Red' AS RedNodes
```
<details>
<summary>Result</summary>
```sh
> eval REP 1 IN TwoRedNodes SELECT 2 FROM RedNodes AS TwoRedNodes FILTER Color EQ 'Red' AS RedNodes
1: [06 09 03]
```
</details>
#### Example #2
```sql
REP 1 IN TwoRedNodes
REP 1 IN TwoRedNodes
SELECT 2 FROM RedNodes AS TwoRedNodes
FILTER Color EQ 'Red' AS RedNodes
```
<details>
<summary>Result</summary>
```sh
> eval REP 1 REP 1 IN TwoRedNodes SELECT 2 FROM RedNodes AS TwoRedNodes FILTER Color EQ 'Red' AS RedNodes
1: [06 09 03]
2: [06 09 03]
```
</details>
#### Example #3
```sql
REP 2 IN MyNodes
REP 2 IN MyNodes
SELECT 2 FROM RedOrBlueNodes AS MyNodes
FILTER Color EQ 'Red' AS RedNodes
FILTER Color EQ 'Blue' AS BlueNodes
FILTER @RedNodes OR @BlueNodes AS RedOrBlueNodes
```
<details>
<summary>Result</summary>
```sh
> eval REP 2 IN MyNodes REP 2 IN MyNodes SELECT 2 FROM RedOrBlueNodes AS MyNodes FILTER Color EQ 'Red' AS RedNodes FILTER Color EQ 'Blue' AS BlueNodes FILTER @RedNodes OR @BlueNodes AS RedOrBlueNodes
1: [06 01 04 03 09 07]
2: [06 01 04 03 09 07]
```
</details>
#### Example #4
```sql
REP 2 IN MyRedNodes
REP 2 IN MyBlueNodes
CBF 1
SELECT 2 FROM RedNodes AS MyRedNodes
SELECT 2 FROM BlueNodes AS MyBlueNodes
FILTER Color EQ 'Red' AS RedNodes
FILTER Color EQ 'Blue' AS BlueNodes
```
<details>
<summary>Result</summary>
```sh
> eval REP 2 IN MyRedNodes REP 2 IN MyBlueNodes CBF 1 SELECT 2 FROM RedNodes AS MyRedNodes SELECT 2 FROM BlueNodes AS MyBlueNodes FILTER Color EQ 'Red' AS RedNodes FILTER Color EQ 'Blue' AS BlueNodes
1: [06 03]
2: [01 04]
```
</details>
#### Example #5
```sql
UNIQUE
REP 1 IN MyGreenNodes
REP 1 IN MyGreenNodes
REP 1 IN MyGreenNodes
CBF 1
SELECT 1 FROM GreenNodes AS MyGreenNodes
FILTER Color EQ 'Green' AS GreenNodes
```
<details>
<summary>Result</summary>
```sh
> eval UNIQUE REP 1 IN MyGreenNodes REP 1 IN MyGreenNodes REP 1 IN MyGreenNodes CBF 1 SELECT 1 FROM GreenNodes AS MyGreenNodes FILTER Color EQ 'Green' AS GreenNodes
1: [05]
2: [02]
3: [08]
```
</details>
#### Example #6
```sql
REP 1 IN MyNodes
REP 2
CBF 2
SELECT 1 FROM CuteNodes AS MyNodes
FILTER (Color EQ 'Blue') AND NOT (Shape EQ 'Circle' OR Shape EQ 'Square') AS CuteNodes
```
<details>
<summary>Result</summary>
```sh
eval REP 1 IN MyNodes REP 2 CBF 2 SELECT 1 FROM CuteNodes AS MyNodes FILTER (Color EQ 'Blue') AND NOT (Shape EQ 'Circle' OR Shape EQ 'Square') AS CuteNodes
1: [07]
2: [06 05 02 03]
```
</details>
## Appendix 1: Operators
Comparison operators (all binary):
- `EQ`: equals
- `NE`: not equal
- `GE`: greater or equal
- `GT`: greater than
- `LE`: less or equal
- `LT`: less than
Logical operators:
- `NOT`: negation (unary)
- `AND`: conjunction (binary)
- `OR`: disjunction (binary)
Others:
- `@`: filter reference
- `(`: left parenthesis
- `)`: right parenthesis
## Appendix 2: Policy playground commands
- `ls`: list nodes in the current netmap and their attributes
- `add`: add a node to the current netmap. If it already exists, it will be overwritten.
- `remove`: remove a node from the current netmap.
- `eval`: evaluate a placement policy on the current netmap.

View file

@ -2,7 +2,7 @@ package eacltest
import (
"bytes"
"math/rand"
"crypto/rand"
"testing"
cidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id/test"

View file

@ -1,7 +1,7 @@
package eacl
import (
"math/rand"
"crypto/rand"
"testing"
"github.com/stretchr/testify/require"

50
go.mod
View file

@ -1,44 +1,46 @@
module git.frostfs.info/TrueCloudLab/frostfs-sdk-go
go 1.18
go 1.20
require (
git.frostfs.info/TrueCloudLab/frostfs-api-go/v2 v2.11.2-0.20230407123205-e6522d62a879
git.frostfs.info/TrueCloudLab/frostfs-api-go/v2 v2.15.1-0.20230802075510-964c3edb3f44
git.frostfs.info/TrueCloudLab/frostfs-contract v0.0.0-20230307110621-19a8ef2d02fb
git.frostfs.info/TrueCloudLab/hrw v1.2.0
git.frostfs.info/TrueCloudLab/frostfs-crypto v0.6.0
git.frostfs.info/TrueCloudLab/hrw v1.2.1
git.frostfs.info/TrueCloudLab/tzhash v1.8.0
github.com/antlr/antlr4/runtime/Go/antlr/v4 v4.0.0-20221202181307-76fa05c21b12
github.com/antlr4-go/antlr/v4 v4.13.0
github.com/google/uuid v1.3.0
github.com/hashicorp/golang-lru/v2 v2.0.1
github.com/hashicorp/golang-lru/v2 v2.0.2
github.com/mr-tron/base58 v1.2.0
github.com/nspcc-dev/neo-go v0.100.1
github.com/stretchr/testify v1.8.1
go.uber.org/atomic v1.10.0
github.com/nspcc-dev/neo-go v0.101.2-0.20230601131642-a0117042e8fc
github.com/stretchr/testify v1.8.3
go.uber.org/zap v1.24.0
google.golang.org/grpc v1.55.0
google.golang.org/protobuf v1.30.0
)
require (
git.frostfs.info/TrueCloudLab/frostfs-crypto v0.6.0 // indirect
git.frostfs.info/TrueCloudLab/rfc6979 v0.4.0 // indirect
github.com/benbjohnson/clock v1.1.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.0.1 // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/gorilla/websocket v1.4.2 // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0 // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/gorilla/websocket v1.5.0 // indirect
github.com/hashicorp/golang-lru v0.6.0 // indirect
github.com/nspcc-dev/go-ordered-json v0.0.0-20220111165707-25110be27d22 // indirect
github.com/nspcc-dev/neo-go/pkg/interop v0.0.0-20221202075445-cb5c18dc73eb // indirect
github.com/nspcc-dev/neo-go/pkg/interop v0.0.0-20230615193820-9185820289ce // indirect
github.com/nspcc-dev/rfc6979 v0.2.0 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/spaolacci/murmur3 v1.1.0 // indirect
go.uber.org/multierr v1.9.0 // indirect
golang.org/x/crypto v0.4.0 // indirect
golang.org/x/exp v0.0.0-20221227203929-1b447090c38c // indirect
golang.org/x/net v0.3.0 // indirect
golang.org/x/sync v0.1.0 // indirect
golang.org/x/sys v0.3.0 // indirect
golang.org/x/text v0.5.0 // indirect
google.golang.org/genproto v0.0.0-20200825200019-8632dd797987 // indirect
google.golang.org/grpc v1.48.0 // indirect
google.golang.org/protobuf v1.28.1 // indirect
github.com/twmb/murmur3 v1.1.8 // indirect
go.uber.org/atomic v1.10.0 // indirect
go.uber.org/goleak v1.2.1 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/crypto v0.9.0 // indirect
golang.org/x/exp v0.0.0-20230515195305-f3d0a9c9a5cc // indirect
golang.org/x/net v0.10.0 // indirect
golang.org/x/sync v0.2.0 // indirect
golang.org/x/sys v0.8.0 // indirect
golang.org/x/text v0.9.0 // indirect
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

95
go.sum
View file

@ -31,14 +31,14 @@ cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohl
cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=
cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
git.frostfs.info/TrueCloudLab/frostfs-api-go/v2 v2.11.2-0.20230407123205-e6522d62a879 h1:KijXOOWhXY50tMDoYusR7ZkZNxePd0tcQIqSw6cJbRk=
git.frostfs.info/TrueCloudLab/frostfs-api-go/v2 v2.11.2-0.20230407123205-e6522d62a879/go.mod h1:gRd5iE5A84viily6AcNBsSlTx2XgoWrwRDz7z0MayDQ=
git.frostfs.info/TrueCloudLab/frostfs-api-go/v2 v2.15.1-0.20230802075510-964c3edb3f44 h1:v6JqBD/VzZx3QSxbaXnUwnnJ1KEYheU4LzLGr3IhsAE=
git.frostfs.info/TrueCloudLab/frostfs-api-go/v2 v2.15.1-0.20230802075510-964c3edb3f44/go.mod h1:pKJJRLOChW4zDQsAt1e8k/snWKljJtpkiPfxV53ngjI=
git.frostfs.info/TrueCloudLab/frostfs-contract v0.0.0-20230307110621-19a8ef2d02fb h1:S/TrbOOu9qEXZRZ9/Ddw7crnxbBUQLo68PSzQWYrc9M=
git.frostfs.info/TrueCloudLab/frostfs-contract v0.0.0-20230307110621-19a8ef2d02fb/go.mod h1:nkR5gaGeez3Zv2SE7aceP0YwxG2FzIB5cGKpQO2vV2o=
git.frostfs.info/TrueCloudLab/frostfs-crypto v0.6.0 h1:FxqFDhQYYgpe41qsIHVOcdzSVCB8JNSfPG7Uk4r2oSk=
git.frostfs.info/TrueCloudLab/frostfs-crypto v0.6.0/go.mod h1:RUIKZATQLJ+TaYQa60X2fTDwfuhMfm8Ar60bQ5fr+vU=
git.frostfs.info/TrueCloudLab/hrw v1.2.0 h1:KvAES7xIqmQBGd2q8KanNosD9+4BhU/zqD5Kt5KSflk=
git.frostfs.info/TrueCloudLab/hrw v1.2.0/go.mod h1:mq2sbvYfO+BB6iFZwYBkgC0yc6mJNx+qZi4jW918m+Y=
git.frostfs.info/TrueCloudLab/hrw v1.2.1 h1:ccBRK21rFvY5R1WotI6LNoPlizk7qSvdfD8lNIRudVc=
git.frostfs.info/TrueCloudLab/hrw v1.2.1/go.mod h1:C1Ygde2n843yTZEQ0FP69jYiuaYV0kriLvP4zm8JuvM=
git.frostfs.info/TrueCloudLab/rfc6979 v0.4.0 h1:M2KR3iBj7WpY3hP10IevfIB9MURr4O9mwVfJ+SjT3HA=
git.frostfs.info/TrueCloudLab/rfc6979 v0.4.0/go.mod h1:okpbKfVYf/BpejtfFTfhZqFP+sZ8rsHrP8Rr/jYPNRc=
git.frostfs.info/TrueCloudLab/tzhash v1.8.0 h1:UFMnUIk0Zh17m8rjGHJMqku2hCgaXDqjqZzS4gsb4UA=
@ -63,8 +63,8 @@ github.com/alicebob/gopher-json v0.0.0-20180125190556-5a6b3ba71ee6/go.mod h1:SGn
github.com/alicebob/miniredis v2.5.0+incompatible/go.mod h1:8HZjEj4yU0dwhYHky+DxYx+6BMjkBbe5ONFIF1MXffk=
github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
github.com/antlr/antlr4/runtime/Go/antlr v0.0.0-20210521073959-f0d4d129b7f1/go.mod h1:F7bn7fEU90QkQ3tnmaTx3LTKLEDqnwWODIYppRQ5hnY=
github.com/antlr/antlr4/runtime/Go/antlr/v4 v4.0.0-20221202181307-76fa05c21b12 h1:npHgfD4Tl2WJS3AJaMUi5ynGDPUBfkg3U3fCzDyXZ+4=
github.com/antlr/antlr4/runtime/Go/antlr/v4 v4.0.0-20221202181307-76fa05c21b12/go.mod h1:pSwJ0fSY5KhvocuWSx4fz3BA8OrA1bQn+K1Eli3BRwM=
github.com/antlr4-go/antlr/v4 v4.13.0 h1:lxCg3LAv+EUK6t1i0y1V6/SLeUi0eKEKdhQAlS8TVTI=
github.com/antlr4-go/antlr/v4 v4.13.0/go.mod h1:pfChB/xh/Unjila75QW7+VU4TSnWnnk9UTnmpPaOR2g=
github.com/benbjohnson/clock v1.1.0 h1:Q92kusRqC1XV2MjkWETPvjJVqKetz1OzxZB7mHJLju8=
github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
@ -92,11 +92,7 @@ github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMn
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/cncf/udpa/go v0.0.0-20210930031921-04548b0d99d4/go.mod h1:6pvJx4me5XPnfI9Z40ddWsdw2W/uZgQLFXToKeRcDiI=
github.com/cncf/xds/go v0.0.0-20210805033703-aa0b78936158/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
github.com/cncf/xds/go v0.0.0-20210922020428-25de7278fc84/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
github.com/cncf/xds/go v0.0.0-20211001041855-01bcc9b48dfe/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
github.com/cncf/xds/go v0.0.0-20211011173535-cb28da3451f1/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
@ -104,16 +100,14 @@ github.com/davecgh/go-spew v0.0.0-20171005155431-ecdeabc65495/go.mod h1:J7Y8YcW2
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/decred/dcrd/crypto/blake256 v1.0.0/go.mod h1:sQl2p6Y26YV+ZOcSTP6thNdn47hh8kt6rqSlvmrXFAc=
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.0.1 h1:YLtO71vCjJRCBcrPMtQ9nqBsqpA1m5sE92cU+pd5Mcc=
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.0.1/go.mod h1:hyedUtir6IdtD/7lIxGeCxkaw7y45JueMRL4DIyJDKs=
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0 h1:8UrgZ3GkP4i/CLijOJx79Yu+etlyjdBU4sfcs2WYQMs=
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0/go.mod h1:v57UDF4pDQJcEfFUCRop3lJL149eHGSe9Jvczhzjo/0=
github.com/decred/dcrd/lru v1.0.0/go.mod h1:mxKOwFd7lFjN2GZYsiz/ecgqR6kkYAl+0pz0tEMk218=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=
github.com/envoyproxy/go-control-plane v0.9.10-0.20210907150352-cf90f659a021/go.mod h1:AFq3mo9L8Lqqiid3OhADV3RfLJnjiw63cSpi+fDTRC0=
github.com/envoyproxy/go-control-plane v0.10.2-0.20220325020618-49ff273808a1/go.mod h1:KJwIaB5Mv44NWtYuAOFCVOjcI94vtpEz2JU/D2v6IjE=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/etcd-io/bbolt v1.3.3/go.mod h1:ZF2nL25h33cCyBtcyWeZ2/I3HQOfTP+0PIEvHjkjCrw=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
@ -165,8 +159,9 @@ github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QD
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw=
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg=
github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/snappy v0.0.0-20170215233205-553a64147049/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/golang/snappy v0.0.1 h1:Qgr9rKW7uDUkrbSmQeiDsGa8SjGyCOGtuasMWwvp2P4=
github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
@ -183,8 +178,8 @@ github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.8 h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
github.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
@ -202,16 +197,17 @@ github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
github.com/gorilla/websocket v1.4.2 h1:+/TMaTYc4QFitKJxsQ7Yye35DkWvkdLcvGKqM+x0Ufc=
github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/gorilla/websocket v1.5.0 h1:PPwGk2jz7EePpoHN/+ClbZu8SPxiqlu12wZP/3sWmnc=
github.com/gorilla/websocket v1.5.0/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
github.com/hashicorp/golang-lru v0.6.0 h1:uL2shRDx7RTrOrTCUZEGP/wJUFiUI8QT6E7z5o8jga4=
github.com/hashicorp/golang-lru v0.6.0/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
github.com/hashicorp/golang-lru/v2 v2.0.1 h1:5pv5N1lT1fjLg2VQ5KWc7kmucp2x/kvFOnxuVTqZ6x4=
github.com/hashicorp/golang-lru/v2 v2.0.1/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/hashicorp/golang-lru/v2 v2.0.2 h1:Dwmkdr5Nc/oBiXgJS3CDHNhJtIHkuZ3DZF5twqnfBdU=
github.com/hashicorp/golang-lru/v2 v2.0.2/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/holiman/uint256 v1.2.0/go.mod h1:y4ga/t+u+Xwd7CpDgZESaRcWy0I7XMlTMA25ApIH5Jw=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
@ -272,11 +268,11 @@ github.com/nspcc-dev/hrw v1.0.9/go.mod h1:l/W2vx83vMQo6aStyx2AuZrJ+07lGv2JQGlVkP
github.com/nspcc-dev/neo-go v0.73.1-pre.0.20200303142215-f5a1b928ce09/go.mod h1:pPYwPZ2ks+uMnlRLUyXOpLieaDQSEaf4NM3zHVbRjmg=
github.com/nspcc-dev/neo-go v0.98.0/go.mod h1:E3cc1x6RXSXrJb2nDWXTXjnXk3rIqVN8YdFyWv+FrqM=
github.com/nspcc-dev/neo-go v0.99.4/go.mod h1:mKTolfRUfKjFso5HPvGSQtUZc70n0VKBMs16eGuC5gA=
github.com/nspcc-dev/neo-go v0.100.1 h1:yugxbQRdzM+ObVa5mtr9/n4rYjxSIrryne8MVr9NBwU=
github.com/nspcc-dev/neo-go v0.100.1/go.mod h1:Nnp7F4e9IBccsgtCeLtUWV+0T6gk1PtP5HRtA13hUfc=
github.com/nspcc-dev/neo-go v0.101.2-0.20230601131642-a0117042e8fc h1:fySIWvUQsitK5e5qYIHnTDCXuPpwzz89SEUEIyY11sg=
github.com/nspcc-dev/neo-go v0.101.2-0.20230601131642-a0117042e8fc/go.mod h1:s9QhjMC784MWqTURovMbyYduIJc86mnCruxcMiAebpc=
github.com/nspcc-dev/neo-go/pkg/interop v0.0.0-20220927123257-24c107e3a262/go.mod h1:23bBw0v6pBYcrWs8CBEEDIEDJNbcFoIh8pGGcf2Vv8s=
github.com/nspcc-dev/neo-go/pkg/interop v0.0.0-20221202075445-cb5c18dc73eb h1:GFxfkpXEYAbMIr69JpKOsQWeLOaGrd49HNAor8uDW+A=
github.com/nspcc-dev/neo-go/pkg/interop v0.0.0-20221202075445-cb5c18dc73eb/go.mod h1:23bBw0v6pBYcrWs8CBEEDIEDJNbcFoIh8pGGcf2Vv8s=
github.com/nspcc-dev/neo-go/pkg/interop v0.0.0-20230615193820-9185820289ce h1:vLGuUNDkmQrWMa4rr4vTd1u8ULqejWxVmNz1L7ocTEI=
github.com/nspcc-dev/neo-go/pkg/interop v0.0.0-20230615193820-9185820289ce/go.mod h1:ZUuXOkdtHZgaC13za/zMgXfQFncZ0jLzfQTe+OsDOtg=
github.com/nspcc-dev/neofs-api-go/v2 v2.11.0-pre.0.20211201134523-3604d96f3fe1/go.mod h1:oS8dycEh8PPf2Jjp6+8dlwWyEv2Dy77h/XhhcdxYEFs=
github.com/nspcc-dev/neofs-api-go/v2 v2.11.1/go.mod h1:oS8dycEh8PPf2Jjp6+8dlwWyEv2Dy77h/XhhcdxYEFs=
github.com/nspcc-dev/neofs-crypto v0.2.0/go.mod h1:F/96fUzPM3wR+UGsPi3faVNmFlA9KAEAUQR7dMxZmNA=
@ -339,25 +335,22 @@ github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeV
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88=
github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI=
github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.3 h1:RP3t2pwF7cMEbC1dqtB6poj3niw/9gnV4Cjg5oW5gtY=
github.com/stretchr/testify v1.8.3/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/syndtr/goleveldb v0.0.0-20180307113352-169b1b37be73/go.mod h1:Z4AUp2Km+PwemOoO/VB5AOx9XSsIItzFjoJlOSiYmn0=
github.com/syndtr/goleveldb v1.0.1-0.20210305035536-64b5b1c73954 h1:xQdMZ1WLrgkkvOZ/LDQxjVxMLdby7osSh4ZEVa5sIjs=
github.com/syndtr/goleveldb v1.0.1-0.20210305035536-64b5b1c73954/go.mod h1:u2MKkTVTVJWe5D1rCvame8WqhBd88EuIwODJZ1VHCPM=
github.com/twmb/murmur3 v1.1.5/go.mod h1:Qq/R7NUyOfr65zD+6Q5IHKsJLwP7exErjN6lyyq3OSQ=
github.com/twmb/murmur3 v1.1.8 h1:8Yt9taO/WN3l08xErzjeschgZU2QSrwm1kclYq+0aRg=
github.com/twmb/murmur3 v1.1.8/go.mod h1:Qq/R7NUyOfr65zD+6Q5IHKsJLwP7exErjN6lyyq3OSQ=
github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA=
github.com/urfave/cli v1.22.5/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
github.com/virtuald/go-ordered-json v0.0.0-20170621173500-b18e6e673d74 h1:JwtAtbp7r/7QSyGz8mKUbYJBg2+6Cd7OjM8o/GNOcVo=
@ -384,11 +377,12 @@ go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
go.uber.org/atomic v1.10.0 h1:9qC72Qh0+3MqyJbAn8YU5xVq1frD8bn3JtD2oXtafVQ=
go.uber.org/atomic v1.10.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
go.uber.org/goleak v1.1.10/go.mod h1:8a7PlsEVH3e/a/GLqe5IIrQx6GzcnRmZEufDUTk4A7A=
go.uber.org/goleak v1.1.11 h1:wy28qYRKZgnJTxGxvye5/wgWr1EKjmUDGYox5mGlRlI=
go.uber.org/goleak v1.2.1 h1:NBol2c7O1ZokfZ0LEU9K6Whx/KnwvepVetCUhtKja4A=
go.uber.org/goleak v1.2.1/go.mod h1:qlT2yGI9QafXHhZZLxlSuNsMw3FFLxBr+tBRlmO1xH4=
go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0=
go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU=
go.uber.org/multierr v1.9.0 h1:7fIwc/ZtS0q++VgcfqFDxSBZVv/Xo49/SYnDFupUwlI=
go.uber.org/multierr v1.9.0/go.mod h1:X2jQV1h+kxSjClGpnseKVIxpmcjrj7MNnI0bnlfKTVQ=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
go.uber.org/zap v1.18.1/go.mod h1:xg/QME4nWcxGxrpdeYfq7UvYrLh66cuVKdrbD1XF/NI=
go.uber.org/zap v1.24.0 h1:FiJd5l1UOLj0wCgbSE0rwwXHzEdAZS6hiiSnxJN/D60=
@ -404,8 +398,8 @@ golang.org/x/crypto v0.0.0-20200115085410-6d4e4cb37c7d/go.mod h1:LzIPMQfyMNhhGPh
golang.org/x/crypto v0.0.0-20200510223506-06a226fb4e37/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210711020723-a769d52b0f97/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.4.0 h1:UVQgzMY87xqpKNgb+kDsll2Igd33HszWHFLmpaRMq/8=
golang.org/x/crypto v0.4.0/go.mod h1:3quD/ATkf6oY+rnes5c3ExXTbLc8mueNue5/DoinL80=
golang.org/x/crypto v0.9.0 h1:LF6fAI+IutBocDJ2OT0Q1g8plpYljMZ4+lty+dsqw3g=
golang.org/x/crypto v0.9.0/go.mod h1:yrmDGqONDYtNj3tH8X9dzUun2m2lzPa9ngI6/RUPGR0=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@ -416,8 +410,8 @@ golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u0
golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
golang.org/x/exp v0.0.0-20221227203929-1b447090c38c h1:Govq2W3bnHJimHT2ium65kXcI7ZzTniZHcFATnLJM0Q=
golang.org/x/exp v0.0.0-20221227203929-1b447090c38c/go.mod h1:CxIveKay+FTh1D0yPZemJVgC/95VzuuOLq5Qi4xnoYc=
golang.org/x/exp v0.0.0-20230515195305-f3d0a9c9a5cc h1:mCRnTeVUjcrhlRmO0VK8a6k6Rrf6TF9htwo2pJVSjIU=
golang.org/x/exp v0.0.0-20230515195305-f3d0a9c9a5cc/go.mod h1:V1LtkGg67GoY2N1AnLN78QLrzxkLyJw7RJb1gzOOz9w=
golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
@ -477,8 +471,8 @@ golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qx
golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/net v0.3.0 h1:VWL6FNY2bEEmsGVKabSlHu5Irp34xmMRoqb/9lF9lxk=
golang.org/x/net v0.3.0/go.mod h1:MBQ8lrhLObU/6UmLb4fmbmk5OcyYmqtbGd/9yIeKjEE=
golang.org/x/net v0.10.0 h1:X2//UzNDwYmtCLn7To6G58Wr6f5ahEAQgKNzv9Y951M=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@ -498,8 +492,8 @@ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220601150217-0de741cfad7f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0 h1:wsuoTGHzEhffawBOhz5CYhcrV4IdKZbEyZjBMuTp12o=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.2.0 h1:PUR+T4wwASmuSTYdKjYHI5TD22Wy5ogLU5qZCOLxBrI=
golang.org/x/sync v0.2.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@ -546,7 +540,6 @@ golang.org/x/sys v0.0.0-20200814200057-3d37ad5750ed/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20200923182605-d9f96fdee20d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@ -555,8 +548,8 @@ golang.org/x/sys v0.0.0-20211019181941-9d821ace8654/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.3.0 h1:w8ZOecv6NaNa/zC8944JTU3vz4u6Lagfk4RPQxv92NQ=
golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0 h1:EBmGv8NaZBZTWvrbjNoL6HVt+IVy3QDQpJs7VRIw3tU=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210429154555-c04ba851c2a4/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
@ -567,8 +560,8 @@ golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.5.0 h1:OLmvp0KP+FVG99Ct/qFiL/Fhk4zp4QQnZ7b2U+5piUM=
golang.org/x/text v0.5.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0 h1:2sjJmO8cDvYveuX97RDLsxlyUxLl+GHoLxBiRdHllBE=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@ -671,8 +664,9 @@ google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEY
google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=
google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200825200019-8632dd797987 h1:PDIOdWxZ8eRizhKa1AAvY53xsvLB1cWorMjslvY3VA8=
google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 h1:KpwkzHKEF7B9Zxg18WzOa7djJ+Ha5DzthMyZYQfEn2A=
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1/go.mod h1:nKE/iIaLqn2bQwXBg8f1g2Ylh6r5MN5CmZvuzZCgsCU=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
@ -688,8 +682,8 @@ google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM
google.golang.org/grpc v1.33.1/go.mod h1:fr5YgcSWrqhRRxogOsw7RzIpsmvOZ6IcH4kBYTpR3n0=
google.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
google.golang.org/grpc v1.41.0/go.mod h1:U3l9uK9J0sini8mHphKoXyaqDA/8VyGnDee1zzIUK6k=
google.golang.org/grpc v1.48.0 h1:rQOsyJ/8+ufEDJd/Gdsz7HG220Mh9HAhFHRGnIjda0w=
google.golang.org/grpc v1.48.0/go.mod h1:vN9eftEi1UMyUsIF80+uQXhHjbXYbm0uXoFCACuMGWk=
google.golang.org/grpc v1.55.0 h1:3Oj82/tFSCeUrRTg/5E/7d/W5A1tj6Ky1ABAuZuv5ag=
google.golang.org/grpc v1.55.0/go.mod h1:iYEXKGkEBhg1PjZQvoYEVPTDkHo1/bjTnfwTeGONTY8=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
@ -703,8 +697,9 @@ google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlba
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.28.1 h1:d0NfwRgPtno5B1Wa6L2DAG+KivqkdutMf1UhdNx175w=
google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
google.golang.org/protobuf v1.30.0 h1:kPPoIgf3TsEvrm0PFe15JQ+570QVxYzEvvHqChK+cng=
google.golang.org/protobuf v1.30.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/abiosoft/ishell.v2 v2.0.0/go.mod h1:sFp+cGtH6o4s1FtpVPTMcHq2yue+c4DGOVohJCPUzwY=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=

View file

@ -24,6 +24,7 @@ type (
minAgg struct {
min float64
minFound bool
}
meanIQRAgg struct {
@ -102,7 +103,13 @@ func (a *meanAgg) Compute() float64 {
}
func (a *minAgg) Add(n float64) {
if a.min == 0 || n < a.min {
if !a.minFound {
a.min = n
a.minFound = true
return
}
if n < a.min {
a.min = n
}
}

44
netmap/aggregator_test.go Normal file
View file

@ -0,0 +1,44 @@
package netmap
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestMinAgg(t *testing.T) {
tests := []struct {
vals []float64
res float64
}{
{
vals: []float64{1, 2, 3, 0, 10},
res: 0,
},
{
vals: []float64{10, 0, 10, 0},
res: 0,
},
{
vals: []float64{0, 1, 2, 3, 10},
res: 0,
},
{
vals: []float64{0, 0, 0, 0},
res: 0,
},
{
vals: []float64{10, 10, 10, 10},
res: 10,
},
}
for _, test := range tests {
a := newMinAgg()
for _, val := range test.vals {
a.Add(val)
}
require.Equal(t, test.res, a.Compute())
}
}

59
netmap/bench_test.go Normal file
View file

@ -0,0 +1,59 @@
package netmap
import (
"testing"
cidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id/test"
"github.com/stretchr/testify/require"
)
func BenchmarkNetmap_ContainerNodes(b *testing.B) {
nodes := []NodeInfo{
nodeInfoFromAttributes("Country", "Russia", "Order", "1"),
nodeInfoFromAttributes("Country", "Germany", "Order", "2"),
nodeInfoFromAttributes("Country", "Russia", "Order", "3"),
nodeInfoFromAttributes("Country", "France", "Order", "4"),
nodeInfoFromAttributes("Country", "France", "Order", "5"),
nodeInfoFromAttributes("Country", "Russia", "Order", "6"),
nodeInfoFromAttributes("Country", "Russia", "Order", "7"),
nodeInfoFromAttributes("Country", "Germany", "Order", "8"),
nodeInfoFromAttributes("Country", "Germany", "Order", "9"),
nodeInfoFromAttributes("Country", "Russia", "Order", "10"),
nodeInfoFromAttributes("Country", "China", "Order", "11"),
nodeInfoFromAttributes("Country", "China", "Order", "12"),
nodeInfoFromAttributes("Country", "Finland", "Order", "13"),
nodeInfoFromAttributes("Country", "Finland", "Order", "14"),
nodeInfoFromAttributes("Country", "España", "Order", "15"),
nodeInfoFromAttributes("Country", "España", "Order", "16"),
}
var nm NetMap
nm.SetNodes(nodes)
policies := []string{
`REP 2`,
`REP 2 IN X CBF 2 SELECT 2 FROM * AS X`,
}
cnr := cidtest.ID()
pivot := make([]byte, 32)
cnr.Encode(pivot)
for i := range policies {
b.Run(policies[i], func(b *testing.B) {
var p PlacementPolicy
require.NoError(b, p.DecodeString(policies[i]))
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
_, err := nm.ContainerNodes(p, pivot)
if err != nil {
b.Fatal(err)
}
}
})
}
}

View file

@ -35,6 +35,15 @@ type context struct {
// container backup factor
cbf uint32
// nodes already used in previous selections, which is needed when the placement
// policy uses the UNIQUE flag. Nodes marked as used are not used in subsequent
// base selections.
usedNodes map[uint64]bool
// If true, returns an error when netmap does not contain enough nodes for selection.
// By default best effort is taken.
strict bool
}
// Various validation errors.
@ -58,6 +67,7 @@ func newContext(nm NetMap) *context {
numCache: make(map[string]uint64),
weightFunc: defaultWeightFunc(nm.nodes),
usedNodes: make(map[uint64]bool),
}
}
@ -76,6 +86,12 @@ func (c *context) setCBF(cbf uint32) {
}
}
func (c *context) addUsedNodes(ns ...NodeInfo) {
for _, n := range ns {
c.usedNodes[n.hash] = true
}
}
func defaultWeightFunc(ns nodes) weightFunc {
mean := newMeanAgg()
min := newMinAgg()

View file

@ -39,7 +39,7 @@ func (c *context) processFilter(f netmap.Filter, top bool) error {
inner := f.GetFilters()
switch op := f.GetOp(); op {
case netmap.AND, netmap.OR:
case netmap.AND, netmap.OR, netmap.NOT:
for i := range inner {
if err := c.processFilter(inner[i], false); err != nil {
return fmt.Errorf("process inner filter #%d: %w", i, err)
@ -79,6 +79,13 @@ func (c *context) processFilter(f netmap.Filter, top bool) error {
// and missing node properties are considered as a regular fail.
func (c *context) match(f *netmap.Filter, b NodeInfo) bool {
switch f.GetOp() {
case netmap.NOT:
inner := f.GetFilters()
fSub := &inner[0]
if name := inner[0].GetName(); name != "" {
fSub = c.processedFilters[name]
}
return !c.match(fSub, b)
case netmap.AND, netmap.OR:
inner := f.GetFilters()
for i := range inner {

View file

@ -2,6 +2,7 @@ package netmap
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"testing"
@ -47,7 +48,8 @@ func TestPlacementPolicy_Interopability(t *testing.T) {
require.NoError(t, err)
for i := range ds {
bs, err := os.ReadFile(filepath.Join(testsDir, ds[i].Name()))
filename := filepath.Join(testsDir, ds[i].Name())
bs, err := os.ReadFile(filename)
require.NoError(t, err)
var tc TestCase
@ -56,7 +58,7 @@ func TestPlacementPolicy_Interopability(t *testing.T) {
srcNodes := make([]NodeInfo, len(tc.Nodes))
copy(srcNodes, tc.Nodes)
t.Run(tc.Name, func(t *testing.T) {
t.Run(fmt.Sprintf("%s:%s", filename, tc.Name), func(t *testing.T) {
var nm NetMap
nm.SetNodes(tc.Nodes)

View file

@ -85,8 +85,7 @@
"filter": "*"
}
],
"filters": [],
"subnetId": null
"filters": []
},
"result": [
[

View file

@ -57,8 +57,7 @@
"filter": "*"
}
],
"filters": [],
"subnetId": null
"filters": []
},
"result": [
[
@ -86,8 +85,7 @@
"filter": "*"
}
],
"filters": [],
"subnetId": null
"filters": []
},
"result": [
[

View file

@ -61,8 +61,7 @@
],
"containerBackupFactor": 0,
"selectors": [],
"filters": [],
"subnetId": null
"filters": []
},
"result": [
[
@ -83,8 +82,7 @@
],
"containerBackupFactor": 3,
"selectors": [],
"filters": [],
"subnetId": null
"filters": []
},
"result": [
[
@ -113,8 +111,7 @@
"filter": "*"
}
],
"filters": [],
"subnetId": null
"filters": []
},
"result": [
[
@ -143,8 +140,7 @@
"filter": "*"
}
],
"filters": [],
"subnetId": null
"filters": []
},
"result": [
[

View file

@ -107,8 +107,7 @@
}
]
}
],
"subnetId": null
]
},
"result": [
[
@ -200,8 +199,7 @@
}
]
}
],
"subnetId": null
]
},
"error": "not enough nodes"
},
@ -289,8 +287,7 @@
}
]
}
],
"subnetId": null
]
},
"error": "not enough nodes"
},
@ -378,8 +375,7 @@
}
]
}
],
"subnetId": null
]
},
"error": "not enough nodes"
}

View file

@ -41,8 +41,7 @@
"value": "4",
"filters": []
}
],
"subnetId": null
]
},
"result": [
[
@ -76,8 +75,7 @@
"value": "5",
"filters": []
}
],
"subnetId": null
]
},
"error": "not enough nodes"
},
@ -107,8 +105,7 @@
"value": "3",
"filters": []
}
],
"subnetId": null
]
},
"result": [
[
@ -142,8 +139,7 @@
"value": "4",
"filters": []
}
],
"subnetId": null
]
},
"error": "not enough nodes"
},
@ -173,8 +169,7 @@
"value": "4",
"filters": []
}
],
"subnetId": null
]
},
"result": [
[
@ -208,8 +203,7 @@
"value": "3",
"filters": []
}
],
"subnetId": null
]
},
"error": "not enough nodes"
},
@ -239,8 +233,7 @@
"value": "5",
"filters": []
}
],
"subnetId": null
]
},
"result": [
[
@ -274,8 +267,7 @@
"value": "4",
"filters": []
}
],
"subnetId": null
]
},
"error": "not enough nodes"
},
@ -305,8 +297,7 @@
"value": "Germany",
"filters": []
}
],
"subnetId": null
]
},
"result": [
[
@ -340,8 +331,7 @@
"value": "China",
"filters": []
}
],
"subnetId": null
]
},
"error": "not enough nodes"
},
@ -371,8 +361,7 @@
"value": "France",
"filters": []
}
],
"subnetId": null
]
},
"result": [
[
@ -406,8 +395,7 @@
"value": "Germany",
"filters": []
}
],
"subnetId": null
]
},
"error": "not enough nodes"
}

View file

@ -144,21 +144,21 @@
],
"tests": {
"select 3 nodes in 3 distinct countries, same placement": {
"policy": {"replicas":[{"count":1,"selector":"Main"}],"containerBackupFactor":1,"selectors":[{"name":"Main","count":3,"clause":"DISTINCT","attribute":"Country","filter":"*"}],"filters":[],"subnetId":null},
"policy": {"replicas":[{"count":1,"selector":"Main"}],"containerBackupFactor":1,"selectors":[{"name":"Main","count":3,"clause":"DISTINCT","attribute":"Country","filter":"*"}],"filters":[]},
"pivot": "Y29udGFpbmVySUQ=",
"result": [[4, 0, 7]],
"result": [[0, 2, 3]],
"placement": {
"pivot": "b2JqZWN0SUQ=",
"result": [[4, 0, 7]]
"result": [[0, 2, 3]]
}
},
"select 6 nodes in 3 distinct countries, different placement": {
"policy": {"replicas":[{"count":1,"selector":"Main"}],"containerBackupFactor":2,"selectors":[{"name":"Main","count":3,"clause":"DISTINCT","attribute":"Country","filter":"*"}],"filters":[],"subnetId":null},
"policy": {"replicas":[{"count":1,"selector":"Main"}],"containerBackupFactor":2,"selectors":[{"name":"Main","count":3,"clause":"DISTINCT","attribute":"Country","filter":"*"}],"filters":[]},
"pivot": "Y29udGFpbmVySUQ=",
"result": [[4, 3, 0, 1, 7, 2]],
"result": [[0, 1, 2, 6, 3, 4]],
"placement": {
"pivot": "b2JqZWN0SUQ=",
"result": [[4, 3, 0, 7, 2, 1]]
"result": [[0, 1, 2, 6, 3, 4]]
}
}
}

View file

@ -93,8 +93,7 @@
"value": "Europe",
"filters": []
}
],
"subnetId": null
]
},
"result": [
[

View file

@ -79,8 +79,7 @@
"value": "Moscow",
"filters": []
}
],
"subnetId": null
]
},
"result": [
[

View file

@ -312,8 +312,7 @@
}
]
}
],
"subnetId": null
]
},
"result": [
[

View file

@ -0,0 +1,95 @@
{
"name": "non-strict selections",
"comment": "These test specify loose selection behaviour, to allow fetching already PUT objects even when there is not enough nodes to select from.",
"nodes": [
{
"attributes": [
{
"key": "Country",
"value": "Russia"
}
]
},
{
"attributes": [
{
"key": "Country",
"value": "Germany"
}
]
},
{
"attributes": [ ]
}
],
"tests": {
"not enough nodes (backup factor)": {
"policy": {
"replicas": [
{
"count": 1,
"selector": "MyStore"
}
],
"containerBackupFactor": 2,
"selectors": [
{
"name": "MyStore",
"count": 2,
"clause": "DISTINCT",
"attribute": "Country",
"filter": "FromRU"
}
],
"filters": [
{
"name": "FromRU",
"key": "Country",
"op": "EQ",
"value": "Russia",
"filters": [ ]
}
]
},
"result": [
[
0
]
]
},
"not enough nodes (buckets)": {
"policy": {
"replicas": [
{
"count": 1,
"selector": "MyStore"
}
],
"containerBackupFactor": 1,
"selectors": [
{
"name": "MyStore",
"count": 2,
"clause": "DISTINCT",
"attribute": "Country",
"filter": "FromRU"
}
],
"filters": [
{
"name": "FromRU",
"key": "Country",
"op": "EQ",
"value": "Russia",
"filters": [ ]
}
]
},
"result": [
[
0
]
]
}
}
}

View file

@ -61,8 +61,7 @@
],
"containerBackupFactor": 0,
"selectors": [],
"filters": [],
"subnetId": null
"filters": []
},
"result": [
[
@ -82,8 +81,7 @@
],
"containerBackupFactor": 0,
"selectors": [],
"filters": [],
"subnetId": null
"filters": []
},
"result": [
[
@ -104,10 +102,16 @@
],
"containerBackupFactor": 0,
"selectors": [],
"filters": [],
"subnetId": null
"filters": []
},
"error": "not enough nodes"
"result": [
[
0,
1,
2,
3
]
]
}
}
}

View file

@ -101,8 +101,7 @@
"filter": "*"
}
],
"filters": [],
"subnetId": null
"filters": []
},
"result": [
[

View file

@ -24,7 +24,12 @@
"tests": {
"missing filter": {
"policy": {
"replicas": [],
"replicas": [
{
"count": 1,
"selector": "MyStore"
}
],
"containerBackupFactor": 1,
"selectors": [
{
@ -43,14 +48,18 @@
"value": "Russia",
"filters": []
}
],
"subnetId": null
]
},
"error": "filter not found"
},
"not enough nodes (backup factor)": {
"not enough nodes (filter results in empty set)": {
"policy": {
"replicas": [],
"replicas": [
{
"count": 1,
"selector": "MyStore"
}
],
"containerBackupFactor": 2,
"selectors": [
{
@ -58,45 +67,18 @@
"count": 2,
"clause": "DISTINCT",
"attribute": "Country",
"filter": "FromRU"
"filter": "FromMoon"
}
],
"filters": [
{
"name": "FromRU",
"name": "FromMoon",
"key": "Country",
"op": "EQ",
"value": "Russia",
"value": "Moon",
"filters": []
}
],
"subnetId": null
},
"error": "not enough nodes"
},
"not enough nodes (buckets)": {
"policy": {
"replicas": [],
"containerBackupFactor": 1,
"selectors": [
{
"name": "MyStore",
"count": 2,
"clause": "DISTINCT",
"attribute": "Country",
"filter": "FromRU"
}
],
"filters": [
{
"name": "FromRU",
"key": "Country",
"op": "EQ",
"value": "Russia",
"filters": []
}
],
"subnetId": null
]
},
"error": "not enough nodes"
}

View file

@ -1,254 +0,0 @@
{
"name": "subnet tests",
"nodes": [
{
"attributes": [
{
"key": "ID",
"value": "0"
},
{
"key": "City",
"value": "Paris"
},
{
"key": "__SYSTEM__SUBNET_0",
"value": "False"
}
],
"state": "UNSPECIFIED"
},
{
"attributes": [
{
"key": "ID",
"value": "1"
},
{
"key": "City",
"value": "Paris"
}
],
"state": "UNSPECIFIED"
},
{
"attributes": [
{
"key": "ID",
"value": "2"
},
{
"key": "City",
"value": "London"
},
{
"key": "__SYSTEM__SUBNET_1",
"value": "True"
}
],
"state": "UNSPECIFIED"
},
{
"attributes": [
{
"key": "ID",
"value": "3"
},
{
"key": "City",
"value": "London"
}
],
"state": "UNSPECIFIED"
},
{
"attributes": [
{
"key": "ID",
"value": "4"
},
{
"key": "City",
"value": "Toronto"
},
{
"key": "__SYSTEM__SUBNET_1",
"value": "True"
}
],
"state": "UNSPECIFIED"
},
{
"attributes": [
{
"key": "ID",
"value": "5"
},
{
"key": "City",
"value": "Toronto"
},
{
"key": "__SYSTEM__SUBNET_2",
"value": "True"
}
],
"state": "UNSPECIFIED"
},
{
"attributes": [
{
"key": "ID",
"value": "6"
},
{
"key": "City",
"value": "Tokyo"
},
{
"key": "__SYSTEM__SUBNET_2",
"value": "True"
}
],
"state": "UNSPECIFIED"
},
{
"attributes": [
{
"key": "ID",
"value": "7"
},
{
"key": "City",
"value": "Tokyo"
},
{
"key": "__SYSTEM__SUBNET_2",
"value": "True"
}
],
"state": "UNSPECIFIED"
}
],
"tests": {
"select from default subnet, fail": {
"policy": {
"replicas": [
{
"count": 1,
"selector": "S"
}
],
"containerBackupFactor": 0,
"selectors": [
{
"name": "S",
"count": 2,
"clause": "SAME",
"attribute": "City",
"filter": "F"
}
],
"filters": [
{
"name": "F",
"key": "City",
"op": "EQ",
"value": "Paris",
"filters": []
}
],
"subnetId": null
},
"error": "not enough nodes"
},
"select from default subnet, success": {
"policy": {
"replicas": [
{
"count": 1,
"selector": "S"
}
],
"containerBackupFactor": 0,
"selectors": [
{
"name": "S",
"count": 2,
"clause": "SAME",
"attribute": "City",
"filter": "F"
}
],
"filters": [
{
"name": "F",
"key": "City",
"op": "EQ",
"value": "Toronto",
"filters": []
}
],
"subnetId": null
},
"result": [
[
4,
5
]
]
},
"select from non-default subnet, success": {
"policy": {
"replicas": [
{
"count": 3,
"selector": ""
}
],
"containerBackupFactor": 0,
"selectors": [],
"filters": [],
"subnetId": {
"value": 2
}
},
"result": [
[
5,
6,
7
]
]
},
"select subnet via filters": {
"policy": {
"replicas": [
{
"count": 1,
"selector": "S"
}
],
"containerBackupFactor": 1,
"selectors": [
{
"name": "S",
"count": 1,
"clause": "SAME",
"attribute": "City",
"filter": "F"
}
],
"filters": [
{
"name": "F",
"key": "__SYSTEM__SUBNET.2.ENABLED",
"op": "EQ",
"value": "True"
}
]
},
"error": "not enough nodes"
}
}
}

View file

@ -158,6 +158,57 @@ func (m NetMap) PlacementVectors(vectors [][]NodeInfo, pivot []byte) ([][]NodeIn
return result, nil
}
// SelectFilterNodes returns a two-dimensional list of nodes as a result of applying the
// given SelectFilterExpr to the NetMap.
// If the SelectFilterExpr contains only filters, the result contains a single row with the
// result of the last filter application.
// If the SelectFilterExpr contains only selectors, the result contains the selection rows
// of the last select application.
func (m NetMap) SelectFilterNodes(expr *SelectFilterExpr) ([][]NodeInfo, error) {
p := PlacementPolicy{
filters: expr.filters,
}
if expr.selector != nil {
p.selectors = append(p.selectors, *expr.selector)
}
c := newContext(m)
c.setCBF(expr.cbf)
if err := c.processFilters(p); err != nil {
return nil, err
}
if err := c.processSelectors(p); err != nil {
return nil, err
}
if expr.selector == nil {
var ret []NodeInfo
lastFilter := expr.filters[len(expr.filters)-1]
for _, ni := range m.nodes {
if c.match(c.processedFilters[lastFilter.GetName()], ni) {
ret = append(ret, ni)
}
}
return [][]NodeInfo{ret}, nil
}
sel, err := c.getSelection(*c.processedSelectors[expr.selector.GetName()])
if err != nil {
return nil, err
}
var ret [][]NodeInfo
for i, ns := range sel {
ret = append(ret, []NodeInfo{})
for _, n := range ns {
ret[i] = append(ret[i], n)
}
}
return ret, nil
}
// ContainerNodes returns two-dimensional list of nodes as a result of applying
// given PlacementPolicy to the NetMap. Each line of the list corresponds to a
// replica descriptor. Line order corresponds to order of ReplicaDescriptor list
@ -181,36 +232,49 @@ func (m NetMap) ContainerNodes(p PlacementPolicy, pivot []byte) ([][]NodeInfo, e
result := make([][]NodeInfo, len(p.replicas))
// Note that the cached selectors are not used when the policy contains the UNIQUE flag.
// This is necessary because each selection vector affects potentially the subsequent vectors
// and thus we call getSelection in such case, in order to take into account nodes previously
// marked as used by earlier replicas.
for i := range p.replicas {
sName := p.replicas[i].GetSelector()
if sName == "" {
if len(p.selectors) == 0 {
if sName == "" && !(len(p.replicas) == 1 && len(p.selectors) == 1) {
var s netmap.Selector
s.SetCount(p.replicas[i].GetCount())
s.SetFilter(mainFilterName)
nodes, err := c.getSelection(p, s)
nodes, err := c.getSelection(s)
if err != nil {
return nil, err
}
result[i] = flattenNodes(nodes)
}
result[i] = append(result[i], flattenNodes(nodes)...)
for i := range p.selectors {
result[i] = append(result[i], flattenNodes(c.selections[p.selectors[i].GetName()])...)
if p.unique {
c.addUsedNodes(result[i]...)
}
continue
}
if p.unique {
if c.processedSelectors[sName] == nil {
return nil, fmt.Errorf("selector not found: '%s'", sName)
}
nodes, err := c.getSelection(*c.processedSelectors[sName])
if err != nil {
return nil, err
}
result[i] = append(result[i], flattenNodes(nodes)...)
c.addUsedNodes(result[i]...)
} else {
nodes, ok := c.selections[sName]
if !ok {
return nil, fmt.Errorf("selector not found: REPLICA '%s'", sName)
}
result[i] = append(result[i], flattenNodes(nodes)...)
}
}
return result, nil
}

View file

@ -5,7 +5,6 @@ import (
"encoding/binary"
"errors"
"fmt"
"math"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
"github.com/nspcc-dev/neo-go/pkg/vm/stackitem"
@ -55,21 +54,11 @@ func (x *NetworkInfo) readFromV2(m netmap.NetworkInfo, checkFieldPresence bool)
err = fmt.Errorf("empty attribute value %s", name)
return true
}
case configEigenTrustAlpha:
var num uint64
num, err = decodeConfigValueUint64(prm.GetValue())
if err == nil {
if alpha := math.Float64frombits(num); alpha < 0 && alpha > 1 {
err = fmt.Errorf("EigenTrust alpha value %0.2f is out of range [0, 1]", alpha)
}
}
case
configAuditFee,
configStoragePrice,
configContainerFee,
configNamedContainerFee,
configEigenTrustNumberOfIterations,
configEpochDuration,
configIRCandidateFee,
configMaxObjSize,
@ -238,12 +227,10 @@ func (x *NetworkInfo) IterateRawNetworkParameters(f func(name string, value []by
default:
f(name, prm.GetValue())
case
configEigenTrustAlpha,
configAuditFee,
configStoragePrice,
configContainerFee,
configNamedContainerFee,
configEigenTrustNumberOfIterations,
configEpochDuration,
configIRCandidateFee,
configMaxObjSize,
@ -394,51 +381,6 @@ func (x NetworkInfo) NamedContainerFee() uint64 {
return x.configUint64(configNamedContainerFee)
}
const configEigenTrustAlpha = "EigenTrustAlpha"
// SetEigenTrustAlpha sets alpha parameter for EigenTrust algorithm used in
// reputation system of the storage nodes. Value MUST be in range [0, 1].
//
// See also EigenTrustAlpha.
func (x *NetworkInfo) SetEigenTrustAlpha(alpha float64) {
if alpha < 0 || alpha > 1 {
panic(fmt.Sprintf("EigenTrust alpha parameter MUST be in range [0, 1], got %.2f", alpha))
}
x.setConfigUint64(configEigenTrustAlpha, math.Float64bits(alpha))
}
// EigenTrustAlpha returns EigenTrust parameter set using SetEigenTrustAlpha.
//
// Zero NetworkInfo has zero alpha parameter.
func (x NetworkInfo) EigenTrustAlpha() float64 {
alpha := math.Float64frombits(x.configUint64(configEigenTrustAlpha))
if alpha < 0 || alpha > 1 {
panic(fmt.Sprintf("unexpected invalid %s parameter value %.2f", configEigenTrustAlpha, alpha))
}
return alpha
}
const configEigenTrustNumberOfIterations = "EigenTrustIterations"
// SetNumberOfEigenTrustIterations sets number of iterations of the EigenTrust
// algorithm to perform. The algorithm is used by the storage nodes for
// calculating the reputation values.
//
// See also NumberOfEigenTrustIterations.
func (x *NetworkInfo) SetNumberOfEigenTrustIterations(num uint64) {
x.setConfigUint64(configEigenTrustNumberOfIterations, num)
}
// NumberOfEigenTrustIterations returns number of EigenTrust iterations set
// using SetNumberOfEigenTrustIterations.
//
// Zero NetworkInfo has zero iteration number.
func (x NetworkInfo) NumberOfEigenTrustIterations() uint64 {
return x.configUint64(configEigenTrustNumberOfIterations)
}
const configEpochDuration = "EpochDuration"
// SetEpochDuration sets FrostFS epoch duration measured in number of blocks of

View file

@ -2,7 +2,6 @@ package netmap_test
import (
"encoding/binary"
"math"
"testing"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
@ -62,16 +61,16 @@ func TestNetworkInfo_MsPerBlock(t *testing.T) {
}
func testConfigValue(t *testing.T,
getter func(x NetworkInfo) interface{},
setter func(x *NetworkInfo, val interface{}),
val1, val2 interface{},
v2Key string, v2Val func(val interface{}) []byte,
getter func(x NetworkInfo) any,
setter func(x *NetworkInfo, val any),
val1, val2 any,
v2Key string, v2Val func(val any) []byte,
) {
var x NetworkInfo
require.Zero(t, getter(x))
checkVal := func(exp interface{}) {
checkVal := func(exp any) {
require.EqualValues(t, exp, getter(x))
var m netmap.NetworkInfo
@ -98,10 +97,10 @@ func testConfigValue(t *testing.T,
func TestNetworkInfo_AuditFee(t *testing.T) {
testConfigValue(t,
func(x NetworkInfo) interface{} { return x.AuditFee() },
func(info *NetworkInfo, val interface{}) { info.SetAuditFee(val.(uint64)) },
func(x NetworkInfo) any { return x.AuditFee() },
func(info *NetworkInfo, val any) { info.SetAuditFee(val.(uint64)) },
uint64(1), uint64(2),
"AuditFee", func(val interface{}) []byte {
"AuditFee", func(val any) []byte {
data := make([]byte, 8)
binary.LittleEndian.PutUint64(data, val.(uint64))
return data
@ -111,10 +110,10 @@ func TestNetworkInfo_AuditFee(t *testing.T) {
func TestNetworkInfo_StoragePrice(t *testing.T) {
testConfigValue(t,
func(x NetworkInfo) interface{} { return x.StoragePrice() },
func(info *NetworkInfo, val interface{}) { info.SetStoragePrice(val.(uint64)) },
func(x NetworkInfo) any { return x.StoragePrice() },
func(info *NetworkInfo, val any) { info.SetStoragePrice(val.(uint64)) },
uint64(1), uint64(2),
"BasicIncomeRate", func(val interface{}) []byte {
"BasicIncomeRate", func(val any) []byte {
data := make([]byte, 8)
binary.LittleEndian.PutUint64(data, val.(uint64))
return data
@ -124,10 +123,10 @@ func TestNetworkInfo_StoragePrice(t *testing.T) {
func TestNetworkInfo_ContainerFee(t *testing.T) {
testConfigValue(t,
func(x NetworkInfo) interface{} { return x.ContainerFee() },
func(info *NetworkInfo, val interface{}) { info.SetContainerFee(val.(uint64)) },
func(x NetworkInfo) any { return x.ContainerFee() },
func(info *NetworkInfo, val any) { info.SetContainerFee(val.(uint64)) },
uint64(1), uint64(2),
"ContainerFee", func(val interface{}) []byte {
"ContainerFee", func(val any) []byte {
data := make([]byte, 8)
binary.LittleEndian.PutUint64(data, val.(uint64))
return data
@ -137,36 +136,10 @@ func TestNetworkInfo_ContainerFee(t *testing.T) {
func TestNetworkInfo_NamedContainerFee(t *testing.T) {
testConfigValue(t,
func(x NetworkInfo) interface{} { return x.NamedContainerFee() },
func(info *NetworkInfo, val interface{}) { info.SetNamedContainerFee(val.(uint64)) },
func(x NetworkInfo) any { return x.NamedContainerFee() },
func(info *NetworkInfo, val any) { info.SetNamedContainerFee(val.(uint64)) },
uint64(1), uint64(2),
"ContainerAliasFee", func(val interface{}) []byte {
data := make([]byte, 8)
binary.LittleEndian.PutUint64(data, val.(uint64))
return data
},
)
}
func TestNetworkInfo_EigenTrustAlpha(t *testing.T) {
testConfigValue(t,
func(x NetworkInfo) interface{} { return x.EigenTrustAlpha() },
func(info *NetworkInfo, val interface{}) { info.SetEigenTrustAlpha(val.(float64)) },
0.1, 0.2,
"EigenTrustAlpha", func(val interface{}) []byte {
data := make([]byte, 8)
binary.LittleEndian.PutUint64(data, math.Float64bits(val.(float64)))
return data
},
)
}
func TestNetworkInfo_NumberOfEigenTrustIterations(t *testing.T) {
testConfigValue(t,
func(x NetworkInfo) interface{} { return x.NumberOfEigenTrustIterations() },
func(info *NetworkInfo, val interface{}) { info.SetNumberOfEigenTrustIterations(val.(uint64)) },
uint64(1), uint64(2),
"EigenTrustIterations", func(val interface{}) []byte {
"ContainerAliasFee", func(val any) []byte {
data := make([]byte, 8)
binary.LittleEndian.PutUint64(data, val.(uint64))
return data
@ -176,10 +149,10 @@ func TestNetworkInfo_NumberOfEigenTrustIterations(t *testing.T) {
func TestNetworkInfo_IRCandidateFee(t *testing.T) {
testConfigValue(t,
func(x NetworkInfo) interface{} { return x.IRCandidateFee() },
func(info *NetworkInfo, val interface{}) { info.SetIRCandidateFee(val.(uint64)) },
func(x NetworkInfo) any { return x.IRCandidateFee() },
func(info *NetworkInfo, val any) { info.SetIRCandidateFee(val.(uint64)) },
uint64(1), uint64(2),
"InnerRingCandidateFee", func(val interface{}) []byte {
"InnerRingCandidateFee", func(val any) []byte {
data := make([]byte, 8)
binary.LittleEndian.PutUint64(data, val.(uint64))
return data
@ -189,10 +162,10 @@ func TestNetworkInfo_IRCandidateFee(t *testing.T) {
func TestNetworkInfo_MaxObjectSize(t *testing.T) {
testConfigValue(t,
func(x NetworkInfo) interface{} { return x.MaxObjectSize() },
func(info *NetworkInfo, val interface{}) { info.SetMaxObjectSize(val.(uint64)) },
func(x NetworkInfo) any { return x.MaxObjectSize() },
func(info *NetworkInfo, val any) { info.SetMaxObjectSize(val.(uint64)) },
uint64(1), uint64(2),
"MaxObjectSize", func(val interface{}) []byte {
"MaxObjectSize", func(val any) []byte {
data := make([]byte, 8)
binary.LittleEndian.PutUint64(data, val.(uint64))
return data
@ -202,10 +175,10 @@ func TestNetworkInfo_MaxObjectSize(t *testing.T) {
func TestNetworkInfo_WithdrawalFee(t *testing.T) {
testConfigValue(t,
func(x NetworkInfo) interface{} { return x.WithdrawalFee() },
func(info *NetworkInfo, val interface{}) { info.SetWithdrawalFee(val.(uint64)) },
func(x NetworkInfo) any { return x.WithdrawalFee() },
func(info *NetworkInfo, val any) { info.SetWithdrawalFee(val.(uint64)) },
uint64(1), uint64(2),
"WithdrawFee", func(val interface{}) []byte {
"WithdrawFee", func(val any) []byte {
data := make([]byte, 8)
binary.LittleEndian.PutUint64(data, val.(uint64))
return data
@ -215,14 +188,14 @@ func TestNetworkInfo_WithdrawalFee(t *testing.T) {
func TestNetworkInfo_HomomorphicHashingDisabled(t *testing.T) {
testConfigValue(t,
func(x NetworkInfo) interface{} { return x.HomomorphicHashingDisabled() },
func(info *NetworkInfo, val interface{}) {
func(x NetworkInfo) any { return x.HomomorphicHashingDisabled() },
func(info *NetworkInfo, val any) {
if val.(bool) {
info.DisableHomomorphicHashing()
}
},
true, true, // it is impossible to enable hashing
"HomomorphicHashingDisabled", func(val interface{}) []byte {
"HomomorphicHashingDisabled", func(val any) []byte {
data := make([]byte, 1)
if val.(bool) {
@ -236,14 +209,14 @@ func TestNetworkInfo_HomomorphicHashingDisabled(t *testing.T) {
func TestNetworkInfo_MaintenanceModeAllowed(t *testing.T) {
testConfigValue(t,
func(x NetworkInfo) interface{} { return x.MaintenanceModeAllowed() },
func(info *NetworkInfo, val interface{}) {
func(x NetworkInfo) any { return x.MaintenanceModeAllowed() },
func(info *NetworkInfo, val any) {
if val.(bool) {
info.AllowMaintenanceMode()
}
},
true, true,
"MaintenanceModeAllowed", func(val interface{}) []byte {
"MaintenanceModeAllowed", func(val any) []byte {
if val.(bool) {
return []byte{1}
}

View file

@ -8,9 +8,7 @@ import (
"strings"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
subnetid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/subnet/id"
"git.frostfs.info/TrueCloudLab/hrw"
)
@ -54,8 +52,6 @@ func (x *NodeInfo) readFromV2(m netmap.NodeInfo, checkFieldPresence bool) error
return fmt.Errorf("duplicated attbiuted %s", key)
}
const subnetPrefix = "__SYSTEM__SUBNET_"
switch {
case key == attrCapacity:
_, err = strconv.ParseUint(attributes[i].GetValue(), 10, 64)
@ -68,17 +64,6 @@ func (x *NodeInfo) readFromV2(m netmap.NodeInfo, checkFieldPresence bool) error
if err != nil {
return fmt.Errorf("invalid %s attribute: %w", attrPrice, err)
}
case strings.HasPrefix(key, subnetPrefix):
var id subnetid.ID
err = id.DecodeString(strings.TrimPrefix(key, subnetPrefix))
if err != nil {
return fmt.Errorf("invalid key to the subnet attribute %s: %w", key, err)
}
if val := attributes[i].GetValue(); val != "True" && val != "False" {
return fmt.Errorf("invalid value of the subnet attribute %s: %w", val, err)
}
default:
if attributes[i].GetValue() == "" {
return fmt.Errorf("empty value of the attribute %s", key)
@ -484,81 +469,6 @@ func (x *NodeInfo) SortAttributes() {
x.m.SetAttributes(as)
}
// EnterSubnet writes storage node's intention to enter the given subnet.
//
// Zero NodeInfo belongs to zero subnet.
func (x *NodeInfo) EnterSubnet(id subnetid.ID) {
x.changeSubnet(id, true)
}
// ExitSubnet writes storage node's intention to exit the given subnet.
func (x *NodeInfo) ExitSubnet(id subnetid.ID) {
x.changeSubnet(id, false)
}
func (x *NodeInfo) changeSubnet(id subnetid.ID, isMember bool) {
var (
idv2 refs.SubnetID
info netmap.NodeSubnetInfo
)
id.WriteToV2(&idv2)
info.SetID(&idv2)
info.SetEntryFlag(isMember)
netmap.WriteSubnetInfo(&x.m, info)
}
// ErrRemoveSubnet is returned when a node needs to leave the subnet.
var ErrRemoveSubnet = netmap.ErrRemoveSubnet
// IterateSubnets iterates over all subnets the node belongs to and passes the IDs to f.
// Handler MUST NOT be nil.
//
// If f returns ErrRemoveSubnet, then removes subnet entry. Note that this leads to an
// instant mutation of NodeInfo. Breaks on any other non-nil error and returns it.
//
// Returns an error if subnet incorrectly enabled/disabled.
// Returns an error if the node is not included to any subnet by the end of the loop.
//
// See also EnterSubnet, ExitSubnet.
func (x NodeInfo) IterateSubnets(f func(subnetid.ID) error) error {
var id subnetid.ID
return netmap.IterateSubnets(&x.m, func(idv2 refs.SubnetID) error {
err := id.ReadFromV2(idv2)
if err != nil {
return fmt.Errorf("invalid subnet: %w", err)
}
err = f(id)
if errors.Is(err, ErrRemoveSubnet) {
return netmap.ErrRemoveSubnet
}
return err
})
}
var errAbortSubnetIter = errors.New("abort subnet iterator")
// BelongsToSubnet is a helper function over the IterateSubnets method which
// checks whether a node belongs to a subnet.
//
// Zero NodeInfo belongs to zero subnet only.
func BelongsToSubnet(node NodeInfo, id subnetid.ID) bool {
err := node.IterateSubnets(func(id_ subnetid.ID) error {
if id.Equals(id_) {
return errAbortSubnetIter
}
return nil
})
return errors.Is(err, errAbortSubnetIter)
}
// SetOffline sets the state of the node to "offline". When a node updates
// information about itself in the network map, this action is interpreted as
// an intention to leave the network.

View file

@ -4,7 +4,9 @@ options {
tokenVocab = QueryLexer;
}
policy: repStmt+ cbfStmt? selectStmt* filterStmt* EOF;
policy: UNIQUE? repStmt+ cbfStmt? selectStmt* filterStmt* EOF;
selectFilterExpr: cbfStmt? selectStmt? filterStmt* EOF;
repStmt:
REP Count = NUMBER1 // number of object replicas
@ -22,7 +24,8 @@ selectStmt:
clause: CLAUSE_SAME | CLAUSE_DISTINCT; // nodes from distinct buckets
filterExpr:
F1 = filterExpr Op = AND_OP F2 = filterExpr
Op = NOT_OP '(' F1 = filterExpr ')'
| F1 = filterExpr Op = AND_OP F2 = filterExpr
| F1 = filterExpr Op = OR_OP F2 = filterExpr
| '(' Inner = filterExpr ')'
| expr

Binary file not shown.

Binary file not shown.

View file

@ -1,9 +1,11 @@
lexer grammar QueryLexer;
NOT_OP : 'NOT';
AND_OP : 'AND';
OR_OP : 'OR';
SIMPLE_OP : 'EQ' | 'NE' | 'GE' | 'GT' | 'LT' | 'LE';
UNIQUE : 'UNIQUE';
REP : 'REP';
IN : 'IN';
AS : 'AS';

Binary file not shown.

Binary file not shown.

View file

@ -1,4 +1,4 @@
package parser
// ANTLR can be downloaded from https://www.antlr.org/download/antlr-4.11.1-complete.jar
//go:generate java -Xmx500M -cp "./antlr-4.11.1-complete.jar:$CLASSPATH" org.antlr.v4.Tool -Dlanguage=Go -visitor QueryLexer.g4 Query.g4
// ANTLR can be downloaded from https://www.antlr.org/download/antlr-4.13.0-complete.jar
//go:generate java -Xmx500M -cp "./antlr-4.13.0-complete.jar:$CLASSPATH" org.antlr.v4.Tool -Dlanguage=Go -no-listener -visitor QueryLexer.g4 Query.g4

View file

@ -1,106 +0,0 @@
// Code generated from java-escape by ANTLR 4.11.1. DO NOT EDIT.
package parser // Query
import "github.com/antlr/antlr4/runtime/Go/antlr/v4"
// BaseQueryListener is a complete listener for a parse tree produced by Query.
type BaseQueryListener struct{}
var _ QueryListener = &BaseQueryListener{}
// VisitTerminal is called when a terminal node is visited.
func (s *BaseQueryListener) VisitTerminal(node antlr.TerminalNode) {}
// VisitErrorNode is called when an error node is visited.
func (s *BaseQueryListener) VisitErrorNode(node antlr.ErrorNode) {}
// EnterEveryRule is called when any rule is entered.
func (s *BaseQueryListener) EnterEveryRule(ctx antlr.ParserRuleContext) {}
// ExitEveryRule is called when any rule is exited.
func (s *BaseQueryListener) ExitEveryRule(ctx antlr.ParserRuleContext) {}
// EnterPolicy is called when production policy is entered.
func (s *BaseQueryListener) EnterPolicy(ctx *PolicyContext) {}
// ExitPolicy is called when production policy is exited.
func (s *BaseQueryListener) ExitPolicy(ctx *PolicyContext) {}
// EnterRepStmt is called when production repStmt is entered.
func (s *BaseQueryListener) EnterRepStmt(ctx *RepStmtContext) {}
// ExitRepStmt is called when production repStmt is exited.
func (s *BaseQueryListener) ExitRepStmt(ctx *RepStmtContext) {}
// EnterCbfStmt is called when production cbfStmt is entered.
func (s *BaseQueryListener) EnterCbfStmt(ctx *CbfStmtContext) {}
// ExitCbfStmt is called when production cbfStmt is exited.
func (s *BaseQueryListener) ExitCbfStmt(ctx *CbfStmtContext) {}
// EnterSelectStmt is called when production selectStmt is entered.
func (s *BaseQueryListener) EnterSelectStmt(ctx *SelectStmtContext) {}
// ExitSelectStmt is called when production selectStmt is exited.
func (s *BaseQueryListener) ExitSelectStmt(ctx *SelectStmtContext) {}
// EnterClause is called when production clause is entered.
func (s *BaseQueryListener) EnterClause(ctx *ClauseContext) {}
// ExitClause is called when production clause is exited.
func (s *BaseQueryListener) ExitClause(ctx *ClauseContext) {}
// EnterFilterExpr is called when production filterExpr is entered.
func (s *BaseQueryListener) EnterFilterExpr(ctx *FilterExprContext) {}
// ExitFilterExpr is called when production filterExpr is exited.
func (s *BaseQueryListener) ExitFilterExpr(ctx *FilterExprContext) {}
// EnterFilterStmt is called when production filterStmt is entered.
func (s *BaseQueryListener) EnterFilterStmt(ctx *FilterStmtContext) {}
// ExitFilterStmt is called when production filterStmt is exited.
func (s *BaseQueryListener) ExitFilterStmt(ctx *FilterStmtContext) {}
// EnterExpr is called when production expr is entered.
func (s *BaseQueryListener) EnterExpr(ctx *ExprContext) {}
// ExitExpr is called when production expr is exited.
func (s *BaseQueryListener) ExitExpr(ctx *ExprContext) {}
// EnterFilterKey is called when production filterKey is entered.
func (s *BaseQueryListener) EnterFilterKey(ctx *FilterKeyContext) {}
// ExitFilterKey is called when production filterKey is exited.
func (s *BaseQueryListener) ExitFilterKey(ctx *FilterKeyContext) {}
// EnterFilterValue is called when production filterValue is entered.
func (s *BaseQueryListener) EnterFilterValue(ctx *FilterValueContext) {}
// ExitFilterValue is called when production filterValue is exited.
func (s *BaseQueryListener) ExitFilterValue(ctx *FilterValueContext) {}
// EnterNumber is called when production number is entered.
func (s *BaseQueryListener) EnterNumber(ctx *NumberContext) {}
// ExitNumber is called when production number is exited.
func (s *BaseQueryListener) ExitNumber(ctx *NumberContext) {}
// EnterKeyword is called when production keyword is entered.
func (s *BaseQueryListener) EnterKeyword(ctx *KeywordContext) {}
// ExitKeyword is called when production keyword is exited.
func (s *BaseQueryListener) ExitKeyword(ctx *KeywordContext) {}
// EnterIdent is called when production ident is entered.
func (s *BaseQueryListener) EnterIdent(ctx *IdentContext) {}
// ExitIdent is called when production ident is exited.
func (s *BaseQueryListener) ExitIdent(ctx *IdentContext) {}
// EnterIdentWC is called when production identWC is entered.
func (s *BaseQueryListener) EnterIdentWC(ctx *IdentWCContext) {}
// ExitIdentWC is called when production identWC is exited.
func (s *BaseQueryListener) ExitIdentWC(ctx *IdentWCContext) {}

View file

@ -1,8 +1,8 @@
// Code generated from java-escape by ANTLR 4.11.1. DO NOT EDIT.
// Code generated from Query.g4 by ANTLR 4.13.0. DO NOT EDIT.
package parser // Query
import "github.com/antlr/antlr4/runtime/Go/antlr/v4"
import "github.com/antlr4-go/antlr/v4"
type BaseQueryVisitor struct {
*antlr.BaseParseTreeVisitor
@ -12,6 +12,10 @@ func (v *BaseQueryVisitor) VisitPolicy(ctx *PolicyContext) interface{} {
return v.VisitChildren(ctx)
}
func (v *BaseQueryVisitor) VisitSelectFilterExpr(ctx *SelectFilterExprContext) interface{} {
return v.VisitChildren(ctx)
}
func (v *BaseQueryVisitor) VisitRepStmt(ctx *RepStmtContext) interface{} {
return v.VisitChildren(ctx)
}

View file

@ -1,13 +1,12 @@
// Code generated from java-escape by ANTLR 4.11.1. DO NOT EDIT.
// Code generated from QueryLexer.g4 by ANTLR 4.13.0. DO NOT EDIT.
package parser
import (
"fmt"
"github.com/antlr4-go/antlr/v4"
"sync"
"unicode"
"github.com/antlr/antlr4/runtime/Go/antlr/v4"
)
// Suppress unused import error
@ -22,134 +21,141 @@ type QueryLexer struct {
// TODO: EOF string
}
var querylexerLexerStaticData struct {
var QueryLexerLexerStaticData struct {
once sync.Once
serializedATN []int32
channelNames []string
modeNames []string
literalNames []string
symbolicNames []string
ruleNames []string
predictionContextCache *antlr.PredictionContextCache
ChannelNames []string
ModeNames []string
LiteralNames []string
SymbolicNames []string
RuleNames []string
PredictionContextCache *antlr.PredictionContextCache
atn *antlr.ATN
decisionToDFA []*antlr.DFA
}
func querylexerLexerInit() {
staticData := &querylexerLexerStaticData
staticData.channelNames = []string{
staticData := &QueryLexerLexerStaticData
staticData.ChannelNames = []string{
"DEFAULT_TOKEN_CHANNEL", "HIDDEN",
}
staticData.modeNames = []string{
staticData.ModeNames = []string{
"DEFAULT_MODE",
}
staticData.literalNames = []string{
"", "'AND'", "'OR'", "", "'REP'", "'IN'", "'AS'", "'CBF'", "'SELECT'",
"'FROM'", "'FILTER'", "'*'", "'SAME'", "'DISTINCT'", "'('", "')'", "'@'",
"", "", "'0'",
staticData.LiteralNames = []string{
"", "'NOT'", "'AND'", "'OR'", "", "'UNIQUE'", "'REP'", "'IN'", "'AS'",
"'CBF'", "'SELECT'", "'FROM'", "'FILTER'", "'*'", "'SAME'", "'DISTINCT'",
"'('", "')'", "'@'", "", "", "'0'",
}
staticData.symbolicNames = []string{
"", "AND_OP", "OR_OP", "SIMPLE_OP", "REP", "IN", "AS", "CBF", "SELECT",
"FROM", "FILTER", "WILDCARD", "CLAUSE_SAME", "CLAUSE_DISTINCT", "L_PAREN",
"R_PAREN", "AT", "IDENT", "NUMBER1", "ZERO", "STRING", "WS",
staticData.SymbolicNames = []string{
"", "NOT_OP", "AND_OP", "OR_OP", "SIMPLE_OP", "UNIQUE", "REP", "IN",
"AS", "CBF", "SELECT", "FROM", "FILTER", "WILDCARD", "CLAUSE_SAME",
"CLAUSE_DISTINCT", "L_PAREN", "R_PAREN", "AT", "IDENT", "NUMBER1", "ZERO",
"STRING", "WS",
}
staticData.ruleNames = []string{
"AND_OP", "OR_OP", "SIMPLE_OP", "REP", "IN", "AS", "CBF", "SELECT",
"FROM", "FILTER", "WILDCARD", "CLAUSE_SAME", "CLAUSE_DISTINCT", "L_PAREN",
"R_PAREN", "AT", "IDENT", "Digit", "Nondigit", "NUMBER1", "ZERO", "STRING",
"ESC", "UNICODE", "HEX", "SAFECODEPOINTSINGLE", "SAFECODEPOINTDOUBLE",
staticData.RuleNames = []string{
"NOT_OP", "AND_OP", "OR_OP", "SIMPLE_OP", "UNIQUE", "REP", "IN", "AS",
"CBF", "SELECT", "FROM", "FILTER", "WILDCARD", "CLAUSE_SAME", "CLAUSE_DISTINCT",
"L_PAREN", "R_PAREN", "AT", "IDENT", "Digit", "Nondigit", "NUMBER1",
"ZERO", "STRING", "ESC", "UNICODE", "HEX", "SAFECODEPOINTSINGLE", "SAFECODEPOINTDOUBLE",
"WS",
}
staticData.predictionContextCache = antlr.NewPredictionContextCache()
staticData.PredictionContextCache = antlr.NewPredictionContextCache()
staticData.serializedATN = []int32{
4, 0, 21, 198, 6, -1, 2, 0, 7, 0, 2, 1, 7, 1, 2, 2, 7, 2, 2, 3, 7, 3, 2,
4, 0, 23, 213, 6, -1, 2, 0, 7, 0, 2, 1, 7, 1, 2, 2, 7, 2, 2, 3, 7, 3, 2,
4, 7, 4, 2, 5, 7, 5, 2, 6, 7, 6, 2, 7, 7, 7, 2, 8, 7, 8, 2, 9, 7, 9, 2,
10, 7, 10, 2, 11, 7, 11, 2, 12, 7, 12, 2, 13, 7, 13, 2, 14, 7, 14, 2, 15,
7, 15, 2, 16, 7, 16, 2, 17, 7, 17, 2, 18, 7, 18, 2, 19, 7, 19, 2, 20, 7,
20, 2, 21, 7, 21, 2, 22, 7, 22, 2, 23, 7, 23, 2, 24, 7, 24, 2, 25, 7, 25,
2, 26, 7, 26, 2, 27, 7, 27, 1, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1,
2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 3,
2, 77, 8, 2, 1, 3, 1, 3, 1, 3, 1, 3, 1, 4, 1, 4, 1, 4, 1, 5, 1, 5, 1, 5,
1, 6, 1, 6, 1, 6, 1, 6, 1, 7, 1, 7, 1, 7, 1, 7, 1, 7, 1, 7, 1, 7, 1, 8,
1, 8, 1, 8, 1, 8, 1, 8, 1, 9, 1, 9, 1, 9, 1, 9, 1, 9, 1, 9, 1, 9, 1, 10,
1, 10, 1, 11, 1, 11, 1, 11, 1, 11, 1, 11, 1, 12, 1, 12, 1, 12, 1, 12, 1,
12, 1, 12, 1, 12, 1, 12, 1, 12, 1, 13, 1, 13, 1, 14, 1, 14, 1, 15, 1, 15,
1, 16, 1, 16, 1, 16, 5, 16, 137, 8, 16, 10, 16, 12, 16, 140, 9, 16, 1,
17, 1, 17, 1, 18, 1, 18, 1, 19, 1, 19, 5, 19, 148, 8, 19, 10, 19, 12, 19,
151, 9, 19, 1, 20, 1, 20, 1, 21, 1, 21, 1, 21, 5, 21, 158, 8, 21, 10, 21,
12, 21, 161, 9, 21, 1, 21, 1, 21, 1, 21, 1, 21, 5, 21, 167, 8, 21, 10,
21, 12, 21, 170, 9, 21, 1, 21, 3, 21, 173, 8, 21, 1, 22, 1, 22, 1, 22,
3, 22, 178, 8, 22, 1, 23, 1, 23, 1, 23, 1, 23, 1, 23, 1, 23, 1, 24, 1,
24, 1, 25, 1, 25, 1, 26, 1, 26, 1, 27, 4, 27, 193, 8, 27, 11, 27, 12, 27,
194, 1, 27, 1, 27, 0, 0, 28, 1, 1, 3, 2, 5, 3, 7, 4, 9, 5, 11, 6, 13, 7,
15, 8, 17, 9, 19, 10, 21, 11, 23, 12, 25, 13, 27, 14, 29, 15, 31, 16, 33,
17, 35, 0, 37, 0, 39, 18, 41, 19, 43, 20, 45, 0, 47, 0, 49, 0, 51, 0, 53,
0, 55, 21, 1, 0, 8, 1, 0, 48, 57, 3, 0, 65, 90, 95, 95, 97, 122, 1, 0,
49, 57, 9, 0, 34, 34, 39, 39, 47, 47, 92, 92, 98, 98, 102, 102, 110, 110,
114, 114, 116, 116, 3, 0, 48, 57, 65, 70, 97, 102, 3, 0, 0, 31, 39, 39,
92, 92, 3, 0, 0, 31, 34, 34, 92, 92, 3, 0, 9, 10, 13, 13, 32, 32, 205,
0, 1, 1, 0, 0, 0, 0, 3, 1, 0, 0, 0, 0, 5, 1, 0, 0, 0, 0, 7, 1, 0, 0, 0,
0, 9, 1, 0, 0, 0, 0, 11, 1, 0, 0, 0, 0, 13, 1, 0, 0, 0, 0, 15, 1, 0, 0,
0, 0, 17, 1, 0, 0, 0, 0, 19, 1, 0, 0, 0, 0, 21, 1, 0, 0, 0, 0, 23, 1, 0,
0, 0, 0, 25, 1, 0, 0, 0, 0, 27, 1, 0, 0, 0, 0, 29, 1, 0, 0, 0, 0, 31, 1,
0, 0, 0, 0, 33, 1, 0, 0, 0, 0, 39, 1, 0, 0, 0, 0, 41, 1, 0, 0, 0, 0, 43,
1, 0, 0, 0, 0, 55, 1, 0, 0, 0, 1, 57, 1, 0, 0, 0, 3, 61, 1, 0, 0, 0, 5,
76, 1, 0, 0, 0, 7, 78, 1, 0, 0, 0, 9, 82, 1, 0, 0, 0, 11, 85, 1, 0, 0,
0, 13, 88, 1, 0, 0, 0, 15, 92, 1, 0, 0, 0, 17, 99, 1, 0, 0, 0, 19, 104,
1, 0, 0, 0, 21, 111, 1, 0, 0, 0, 23, 113, 1, 0, 0, 0, 25, 118, 1, 0, 0,
0, 27, 127, 1, 0, 0, 0, 29, 129, 1, 0, 0, 0, 31, 131, 1, 0, 0, 0, 33, 133,
1, 0, 0, 0, 35, 141, 1, 0, 0, 0, 37, 143, 1, 0, 0, 0, 39, 145, 1, 0, 0,
0, 41, 152, 1, 0, 0, 0, 43, 172, 1, 0, 0, 0, 45, 174, 1, 0, 0, 0, 47, 179,
1, 0, 0, 0, 49, 185, 1, 0, 0, 0, 51, 187, 1, 0, 0, 0, 53, 189, 1, 0, 0,
0, 55, 192, 1, 0, 0, 0, 57, 58, 5, 65, 0, 0, 58, 59, 5, 78, 0, 0, 59, 60,
5, 68, 0, 0, 60, 2, 1, 0, 0, 0, 61, 62, 5, 79, 0, 0, 62, 63, 5, 82, 0,
0, 63, 4, 1, 0, 0, 0, 64, 65, 5, 69, 0, 0, 65, 77, 5, 81, 0, 0, 66, 67,
5, 78, 0, 0, 67, 77, 5, 69, 0, 0, 68, 69, 5, 71, 0, 0, 69, 77, 5, 69, 0,
0, 70, 71, 5, 71, 0, 0, 71, 77, 5, 84, 0, 0, 72, 73, 5, 76, 0, 0, 73, 77,
5, 84, 0, 0, 74, 75, 5, 76, 0, 0, 75, 77, 5, 69, 0, 0, 76, 64, 1, 0, 0,
0, 76, 66, 1, 0, 0, 0, 76, 68, 1, 0, 0, 0, 76, 70, 1, 0, 0, 0, 76, 72,
1, 0, 0, 0, 76, 74, 1, 0, 0, 0, 77, 6, 1, 0, 0, 0, 78, 79, 5, 82, 0, 0,
79, 80, 5, 69, 0, 0, 80, 81, 5, 80, 0, 0, 81, 8, 1, 0, 0, 0, 82, 83, 5,
73, 0, 0, 83, 84, 5, 78, 0, 0, 84, 10, 1, 0, 0, 0, 85, 86, 5, 65, 0, 0,
86, 87, 5, 83, 0, 0, 87, 12, 1, 0, 0, 0, 88, 89, 5, 67, 0, 0, 89, 90, 5,
66, 0, 0, 90, 91, 5, 70, 0, 0, 91, 14, 1, 0, 0, 0, 92, 93, 5, 83, 0, 0,
93, 94, 5, 69, 0, 0, 94, 95, 5, 76, 0, 0, 95, 96, 5, 69, 0, 0, 96, 97,
5, 67, 0, 0, 97, 98, 5, 84, 0, 0, 98, 16, 1, 0, 0, 0, 99, 100, 5, 70, 0,
0, 100, 101, 5, 82, 0, 0, 101, 102, 5, 79, 0, 0, 102, 103, 5, 77, 0, 0,
103, 18, 1, 0, 0, 0, 104, 105, 5, 70, 0, 0, 105, 106, 5, 73, 0, 0, 106,
107, 5, 76, 0, 0, 107, 108, 5, 84, 0, 0, 108, 109, 5, 69, 0, 0, 109, 110,
5, 82, 0, 0, 110, 20, 1, 0, 0, 0, 111, 112, 5, 42, 0, 0, 112, 22, 1, 0,
0, 0, 113, 114, 5, 83, 0, 0, 114, 115, 5, 65, 0, 0, 115, 116, 5, 77, 0,
0, 116, 117, 5, 69, 0, 0, 117, 24, 1, 0, 0, 0, 118, 119, 5, 68, 0, 0, 119,
120, 5, 73, 0, 0, 120, 121, 5, 83, 0, 0, 121, 122, 5, 84, 0, 0, 122, 123,
5, 73, 0, 0, 123, 124, 5, 78, 0, 0, 124, 125, 5, 67, 0, 0, 125, 126, 5,
84, 0, 0, 126, 26, 1, 0, 0, 0, 127, 128, 5, 40, 0, 0, 128, 28, 1, 0, 0,
0, 129, 130, 5, 41, 0, 0, 130, 30, 1, 0, 0, 0, 131, 132, 5, 64, 0, 0, 132,
32, 1, 0, 0, 0, 133, 138, 3, 37, 18, 0, 134, 137, 3, 35, 17, 0, 135, 137,
3, 37, 18, 0, 136, 134, 1, 0, 0, 0, 136, 135, 1, 0, 0, 0, 137, 140, 1,
0, 0, 0, 138, 136, 1, 0, 0, 0, 138, 139, 1, 0, 0, 0, 139, 34, 1, 0, 0,
0, 140, 138, 1, 0, 0, 0, 141, 142, 7, 0, 0, 0, 142, 36, 1, 0, 0, 0, 143,
144, 7, 1, 0, 0, 144, 38, 1, 0, 0, 0, 145, 149, 7, 2, 0, 0, 146, 148, 3,
35, 17, 0, 147, 146, 1, 0, 0, 0, 148, 151, 1, 0, 0, 0, 149, 147, 1, 0,
0, 0, 149, 150, 1, 0, 0, 0, 150, 40, 1, 0, 0, 0, 151, 149, 1, 0, 0, 0,
152, 153, 5, 48, 0, 0, 153, 42, 1, 0, 0, 0, 154, 159, 5, 34, 0, 0, 155,
158, 3, 45, 22, 0, 156, 158, 3, 53, 26, 0, 157, 155, 1, 0, 0, 0, 157, 156,
1, 0, 0, 0, 158, 161, 1, 0, 0, 0, 159, 157, 1, 0, 0, 0, 159, 160, 1, 0,
0, 0, 160, 162, 1, 0, 0, 0, 161, 159, 1, 0, 0, 0, 162, 173, 5, 34, 0, 0,
163, 168, 5, 39, 0, 0, 164, 167, 3, 45, 22, 0, 165, 167, 3, 51, 25, 0,
166, 164, 1, 0, 0, 0, 166, 165, 1, 0, 0, 0, 167, 170, 1, 0, 0, 0, 168,
166, 1, 0, 0, 0, 168, 169, 1, 0, 0, 0, 169, 171, 1, 0, 0, 0, 170, 168,
1, 0, 0, 0, 171, 173, 5, 39, 0, 0, 172, 154, 1, 0, 0, 0, 172, 163, 1, 0,
0, 0, 173, 44, 1, 0, 0, 0, 174, 177, 5, 92, 0, 0, 175, 178, 7, 3, 0, 0,
176, 178, 3, 47, 23, 0, 177, 175, 1, 0, 0, 0, 177, 176, 1, 0, 0, 0, 178,
46, 1, 0, 0, 0, 179, 180, 5, 117, 0, 0, 180, 181, 3, 49, 24, 0, 181, 182,
3, 49, 24, 0, 182, 183, 3, 49, 24, 0, 183, 184, 3, 49, 24, 0, 184, 48,
1, 0, 0, 0, 185, 186, 7, 4, 0, 0, 186, 50, 1, 0, 0, 0, 187, 188, 8, 5,
0, 0, 188, 52, 1, 0, 0, 0, 189, 190, 8, 6, 0, 0, 190, 54, 1, 0, 0, 0, 191,
193, 7, 7, 0, 0, 192, 191, 1, 0, 0, 0, 193, 194, 1, 0, 0, 0, 194, 192,
1, 0, 0, 0, 194, 195, 1, 0, 0, 0, 195, 196, 1, 0, 0, 0, 196, 197, 6, 27,
0, 0, 197, 56, 1, 0, 0, 0, 12, 0, 76, 136, 138, 149, 157, 159, 166, 168,
172, 177, 194, 1, 6, 0, 0,
2, 26, 7, 26, 2, 27, 7, 27, 2, 28, 7, 28, 2, 29, 7, 29, 1, 0, 1, 0, 1,
0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2, 1, 2, 1, 3, 1, 3, 1, 3, 1,
3, 1, 3, 1, 3, 1, 3, 1, 3, 1, 3, 1, 3, 1, 3, 1, 3, 3, 3, 85, 8, 3, 1, 4,
1, 4, 1, 4, 1, 4, 1, 4, 1, 4, 1, 4, 1, 5, 1, 5, 1, 5, 1, 5, 1, 6, 1, 6,
1, 6, 1, 7, 1, 7, 1, 7, 1, 8, 1, 8, 1, 8, 1, 8, 1, 9, 1, 9, 1, 9, 1, 9,
1, 9, 1, 9, 1, 9, 1, 10, 1, 10, 1, 10, 1, 10, 1, 10, 1, 11, 1, 11, 1, 11,
1, 11, 1, 11, 1, 11, 1, 11, 1, 12, 1, 12, 1, 13, 1, 13, 1, 13, 1, 13, 1,
13, 1, 14, 1, 14, 1, 14, 1, 14, 1, 14, 1, 14, 1, 14, 1, 14, 1, 14, 1, 15,
1, 15, 1, 16, 1, 16, 1, 17, 1, 17, 1, 18, 1, 18, 1, 18, 5, 18, 152, 8,
18, 10, 18, 12, 18, 155, 9, 18, 1, 19, 1, 19, 1, 20, 1, 20, 1, 21, 1, 21,
5, 21, 163, 8, 21, 10, 21, 12, 21, 166, 9, 21, 1, 22, 1, 22, 1, 23, 1,
23, 1, 23, 5, 23, 173, 8, 23, 10, 23, 12, 23, 176, 9, 23, 1, 23, 1, 23,
1, 23, 1, 23, 5, 23, 182, 8, 23, 10, 23, 12, 23, 185, 9, 23, 1, 23, 3,
23, 188, 8, 23, 1, 24, 1, 24, 1, 24, 3, 24, 193, 8, 24, 1, 25, 1, 25, 1,
25, 1, 25, 1, 25, 1, 25, 1, 26, 1, 26, 1, 27, 1, 27, 1, 28, 1, 28, 1, 29,
4, 29, 208, 8, 29, 11, 29, 12, 29, 209, 1, 29, 1, 29, 0, 0, 30, 1, 1, 3,
2, 5, 3, 7, 4, 9, 5, 11, 6, 13, 7, 15, 8, 17, 9, 19, 10, 21, 11, 23, 12,
25, 13, 27, 14, 29, 15, 31, 16, 33, 17, 35, 18, 37, 19, 39, 0, 41, 0, 43,
20, 45, 21, 47, 22, 49, 0, 51, 0, 53, 0, 55, 0, 57, 0, 59, 23, 1, 0, 8,
1, 0, 48, 57, 3, 0, 65, 90, 95, 95, 97, 122, 1, 0, 49, 57, 9, 0, 34, 34,
39, 39, 47, 47, 92, 92, 98, 98, 102, 102, 110, 110, 114, 114, 116, 116,
3, 0, 48, 57, 65, 70, 97, 102, 3, 0, 0, 31, 39, 39, 92, 92, 3, 0, 0, 31,
34, 34, 92, 92, 3, 0, 9, 10, 13, 13, 32, 32, 220, 0, 1, 1, 0, 0, 0, 0,
3, 1, 0, 0, 0, 0, 5, 1, 0, 0, 0, 0, 7, 1, 0, 0, 0, 0, 9, 1, 0, 0, 0, 0,
11, 1, 0, 0, 0, 0, 13, 1, 0, 0, 0, 0, 15, 1, 0, 0, 0, 0, 17, 1, 0, 0, 0,
0, 19, 1, 0, 0, 0, 0, 21, 1, 0, 0, 0, 0, 23, 1, 0, 0, 0, 0, 25, 1, 0, 0,
0, 0, 27, 1, 0, 0, 0, 0, 29, 1, 0, 0, 0, 0, 31, 1, 0, 0, 0, 0, 33, 1, 0,
0, 0, 0, 35, 1, 0, 0, 0, 0, 37, 1, 0, 0, 0, 0, 43, 1, 0, 0, 0, 0, 45, 1,
0, 0, 0, 0, 47, 1, 0, 0, 0, 0, 59, 1, 0, 0, 0, 1, 61, 1, 0, 0, 0, 3, 65,
1, 0, 0, 0, 5, 69, 1, 0, 0, 0, 7, 84, 1, 0, 0, 0, 9, 86, 1, 0, 0, 0, 11,
93, 1, 0, 0, 0, 13, 97, 1, 0, 0, 0, 15, 100, 1, 0, 0, 0, 17, 103, 1, 0,
0, 0, 19, 107, 1, 0, 0, 0, 21, 114, 1, 0, 0, 0, 23, 119, 1, 0, 0, 0, 25,
126, 1, 0, 0, 0, 27, 128, 1, 0, 0, 0, 29, 133, 1, 0, 0, 0, 31, 142, 1,
0, 0, 0, 33, 144, 1, 0, 0, 0, 35, 146, 1, 0, 0, 0, 37, 148, 1, 0, 0, 0,
39, 156, 1, 0, 0, 0, 41, 158, 1, 0, 0, 0, 43, 160, 1, 0, 0, 0, 45, 167,
1, 0, 0, 0, 47, 187, 1, 0, 0, 0, 49, 189, 1, 0, 0, 0, 51, 194, 1, 0, 0,
0, 53, 200, 1, 0, 0, 0, 55, 202, 1, 0, 0, 0, 57, 204, 1, 0, 0, 0, 59, 207,
1, 0, 0, 0, 61, 62, 5, 78, 0, 0, 62, 63, 5, 79, 0, 0, 63, 64, 5, 84, 0,
0, 64, 2, 1, 0, 0, 0, 65, 66, 5, 65, 0, 0, 66, 67, 5, 78, 0, 0, 67, 68,
5, 68, 0, 0, 68, 4, 1, 0, 0, 0, 69, 70, 5, 79, 0, 0, 70, 71, 5, 82, 0,
0, 71, 6, 1, 0, 0, 0, 72, 73, 5, 69, 0, 0, 73, 85, 5, 81, 0, 0, 74, 75,
5, 78, 0, 0, 75, 85, 5, 69, 0, 0, 76, 77, 5, 71, 0, 0, 77, 85, 5, 69, 0,
0, 78, 79, 5, 71, 0, 0, 79, 85, 5, 84, 0, 0, 80, 81, 5, 76, 0, 0, 81, 85,
5, 84, 0, 0, 82, 83, 5, 76, 0, 0, 83, 85, 5, 69, 0, 0, 84, 72, 1, 0, 0,
0, 84, 74, 1, 0, 0, 0, 84, 76, 1, 0, 0, 0, 84, 78, 1, 0, 0, 0, 84, 80,
1, 0, 0, 0, 84, 82, 1, 0, 0, 0, 85, 8, 1, 0, 0, 0, 86, 87, 5, 85, 0, 0,
87, 88, 5, 78, 0, 0, 88, 89, 5, 73, 0, 0, 89, 90, 5, 81, 0, 0, 90, 91,
5, 85, 0, 0, 91, 92, 5, 69, 0, 0, 92, 10, 1, 0, 0, 0, 93, 94, 5, 82, 0,
0, 94, 95, 5, 69, 0, 0, 95, 96, 5, 80, 0, 0, 96, 12, 1, 0, 0, 0, 97, 98,
5, 73, 0, 0, 98, 99, 5, 78, 0, 0, 99, 14, 1, 0, 0, 0, 100, 101, 5, 65,
0, 0, 101, 102, 5, 83, 0, 0, 102, 16, 1, 0, 0, 0, 103, 104, 5, 67, 0, 0,
104, 105, 5, 66, 0, 0, 105, 106, 5, 70, 0, 0, 106, 18, 1, 0, 0, 0, 107,
108, 5, 83, 0, 0, 108, 109, 5, 69, 0, 0, 109, 110, 5, 76, 0, 0, 110, 111,
5, 69, 0, 0, 111, 112, 5, 67, 0, 0, 112, 113, 5, 84, 0, 0, 113, 20, 1,
0, 0, 0, 114, 115, 5, 70, 0, 0, 115, 116, 5, 82, 0, 0, 116, 117, 5, 79,
0, 0, 117, 118, 5, 77, 0, 0, 118, 22, 1, 0, 0, 0, 119, 120, 5, 70, 0, 0,
120, 121, 5, 73, 0, 0, 121, 122, 5, 76, 0, 0, 122, 123, 5, 84, 0, 0, 123,
124, 5, 69, 0, 0, 124, 125, 5, 82, 0, 0, 125, 24, 1, 0, 0, 0, 126, 127,
5, 42, 0, 0, 127, 26, 1, 0, 0, 0, 128, 129, 5, 83, 0, 0, 129, 130, 5, 65,
0, 0, 130, 131, 5, 77, 0, 0, 131, 132, 5, 69, 0, 0, 132, 28, 1, 0, 0, 0,
133, 134, 5, 68, 0, 0, 134, 135, 5, 73, 0, 0, 135, 136, 5, 83, 0, 0, 136,
137, 5, 84, 0, 0, 137, 138, 5, 73, 0, 0, 138, 139, 5, 78, 0, 0, 139, 140,
5, 67, 0, 0, 140, 141, 5, 84, 0, 0, 141, 30, 1, 0, 0, 0, 142, 143, 5, 40,
0, 0, 143, 32, 1, 0, 0, 0, 144, 145, 5, 41, 0, 0, 145, 34, 1, 0, 0, 0,
146, 147, 5, 64, 0, 0, 147, 36, 1, 0, 0, 0, 148, 153, 3, 41, 20, 0, 149,
152, 3, 39, 19, 0, 150, 152, 3, 41, 20, 0, 151, 149, 1, 0, 0, 0, 151, 150,
1, 0, 0, 0, 152, 155, 1, 0, 0, 0, 153, 151, 1, 0, 0, 0, 153, 154, 1, 0,
0, 0, 154, 38, 1, 0, 0, 0, 155, 153, 1, 0, 0, 0, 156, 157, 7, 0, 0, 0,
157, 40, 1, 0, 0, 0, 158, 159, 7, 1, 0, 0, 159, 42, 1, 0, 0, 0, 160, 164,
7, 2, 0, 0, 161, 163, 3, 39, 19, 0, 162, 161, 1, 0, 0, 0, 163, 166, 1,
0, 0, 0, 164, 162, 1, 0, 0, 0, 164, 165, 1, 0, 0, 0, 165, 44, 1, 0, 0,
0, 166, 164, 1, 0, 0, 0, 167, 168, 5, 48, 0, 0, 168, 46, 1, 0, 0, 0, 169,
174, 5, 34, 0, 0, 170, 173, 3, 49, 24, 0, 171, 173, 3, 57, 28, 0, 172,
170, 1, 0, 0, 0, 172, 171, 1, 0, 0, 0, 173, 176, 1, 0, 0, 0, 174, 172,
1, 0, 0, 0, 174, 175, 1, 0, 0, 0, 175, 177, 1, 0, 0, 0, 176, 174, 1, 0,
0, 0, 177, 188, 5, 34, 0, 0, 178, 183, 5, 39, 0, 0, 179, 182, 3, 49, 24,
0, 180, 182, 3, 55, 27, 0, 181, 179, 1, 0, 0, 0, 181, 180, 1, 0, 0, 0,
182, 185, 1, 0, 0, 0, 183, 181, 1, 0, 0, 0, 183, 184, 1, 0, 0, 0, 184,
186, 1, 0, 0, 0, 185, 183, 1, 0, 0, 0, 186, 188, 5, 39, 0, 0, 187, 169,
1, 0, 0, 0, 187, 178, 1, 0, 0, 0, 188, 48, 1, 0, 0, 0, 189, 192, 5, 92,
0, 0, 190, 193, 7, 3, 0, 0, 191, 193, 3, 51, 25, 0, 192, 190, 1, 0, 0,
0, 192, 191, 1, 0, 0, 0, 193, 50, 1, 0, 0, 0, 194, 195, 5, 117, 0, 0, 195,
196, 3, 53, 26, 0, 196, 197, 3, 53, 26, 0, 197, 198, 3, 53, 26, 0, 198,
199, 3, 53, 26, 0, 199, 52, 1, 0, 0, 0, 200, 201, 7, 4, 0, 0, 201, 54,
1, 0, 0, 0, 202, 203, 8, 5, 0, 0, 203, 56, 1, 0, 0, 0, 204, 205, 8, 6,
0, 0, 205, 58, 1, 0, 0, 0, 206, 208, 7, 7, 0, 0, 207, 206, 1, 0, 0, 0,
208, 209, 1, 0, 0, 0, 209, 207, 1, 0, 0, 0, 209, 210, 1, 0, 0, 0, 210,
211, 1, 0, 0, 0, 211, 212, 6, 29, 0, 0, 212, 60, 1, 0, 0, 0, 12, 0, 84,
151, 153, 164, 172, 174, 181, 183, 187, 192, 209, 1, 6, 0, 0,
}
deserializer := antlr.NewATNDeserializer(nil)
staticData.atn = deserializer.Deserialize(staticData.serializedATN)
@ -166,7 +172,7 @@ func querylexerLexerInit() {
// NewQueryLexer(). You can call this function if you wish to initialize the static state ahead
// of time.
func QueryLexerInit() {
staticData := &querylexerLexerStaticData
staticData := &QueryLexerLexerStaticData
staticData.once.Do(querylexerLexerInit)
}
@ -175,13 +181,13 @@ func NewQueryLexer(input antlr.CharStream) *QueryLexer {
QueryLexerInit()
l := new(QueryLexer)
l.BaseLexer = antlr.NewBaseLexer(input)
staticData := &querylexerLexerStaticData
l.Interpreter = antlr.NewLexerATNSimulator(l, staticData.atn, staticData.decisionToDFA, staticData.predictionContextCache)
l.channelNames = staticData.channelNames
l.modeNames = staticData.modeNames
l.RuleNames = staticData.ruleNames
l.LiteralNames = staticData.literalNames
l.SymbolicNames = staticData.symbolicNames
staticData := &QueryLexerLexerStaticData
l.Interpreter = antlr.NewLexerATNSimulator(l, staticData.atn, staticData.decisionToDFA, staticData.PredictionContextCache)
l.channelNames = staticData.ChannelNames
l.modeNames = staticData.ModeNames
l.RuleNames = staticData.RuleNames
l.LiteralNames = staticData.LiteralNames
l.SymbolicNames = staticData.SymbolicNames
l.GrammarFileName = "QueryLexer.g4"
// TODO: l.EOF = antlr.TokenEOF
@ -190,25 +196,27 @@ func NewQueryLexer(input antlr.CharStream) *QueryLexer {
// QueryLexer tokens.
const (
QueryLexerAND_OP = 1
QueryLexerOR_OP = 2
QueryLexerSIMPLE_OP = 3
QueryLexerREP = 4
QueryLexerIN = 5
QueryLexerAS = 6
QueryLexerCBF = 7
QueryLexerSELECT = 8
QueryLexerFROM = 9
QueryLexerFILTER = 10
QueryLexerWILDCARD = 11
QueryLexerCLAUSE_SAME = 12
QueryLexerCLAUSE_DISTINCT = 13
QueryLexerL_PAREN = 14
QueryLexerR_PAREN = 15
QueryLexerAT = 16
QueryLexerIDENT = 17
QueryLexerNUMBER1 = 18
QueryLexerZERO = 19
QueryLexerSTRING = 20
QueryLexerWS = 21
QueryLexerNOT_OP = 1
QueryLexerAND_OP = 2
QueryLexerOR_OP = 3
QueryLexerSIMPLE_OP = 4
QueryLexerUNIQUE = 5
QueryLexerREP = 6
QueryLexerIN = 7
QueryLexerAS = 8
QueryLexerCBF = 9
QueryLexerSELECT = 10
QueryLexerFROM = 11
QueryLexerFILTER = 12
QueryLexerWILDCARD = 13
QueryLexerCLAUSE_SAME = 14
QueryLexerCLAUSE_DISTINCT = 15
QueryLexerL_PAREN = 16
QueryLexerR_PAREN = 17
QueryLexerAT = 18
QueryLexerIDENT = 19
QueryLexerNUMBER1 = 20
QueryLexerZERO = 21
QueryLexerSTRING = 22
QueryLexerWS = 23
)

Some files were not shown because too many files have changed in this diff Show more