Compare commits

...
Sign in to create a new pull request.

298 commits

Author SHA1 Message Date
7974eb9857 Add new event to read data from server
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-11-08 17:42:37 +03:00
9e55056842 Add pool trace events
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-11-06 21:10:44 +03:00
5361f0eceb [#279] pool: Count errors in object search
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-10-22 12:41:11 +00:00
05aa3becae [#278] pool: Don't make maintenance node healthy in rebalance
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-10-17 16:31:49 +03:00
79f387317a [#283] pool: Mark node unhealthy if node is under maintenance
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-10-16 15:22:05 +03:00
3ea4741231 [#283] go.mod: Tidy
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-10-16 15:20:53 +03:00
d7872061f8
[#284] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-11 15:17:23 +03:00
99c5c58365
[#282] client: Close connection on non-nil error in Dial
A particular status code does not imply that a connection has not been
established. However, `Dial()` requires user to call `Close()` only if
the error was nil. Thus, it is `Dial()` responsibility to close
everything if it returns an error.

Introduced after the gRPC update in #270 (6009d089fc).

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-10 14:03:44 +03:00
4c310ae1c7 [#280] client: Use DialTimeout for gRPC dial
After removing `grpc.Dial` from client `DialTimeout` used only if
custom dialer provided. Client uses `BalanceOf` instead of `grpc.Dial`,
so it is required to use `DialTimeout` to not to use RPC timeout.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-07 16:58:05 +03:00
997346ef95
[#274] client/status: Add missing test cases for commom statuses
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-10-03 17:05:15 +03:00
7f6eda566a
[#274] client/status: Fix check in TestNodeUnderMaintenance test
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-10-03 17:05:15 +03:00
d00892f418
[#274] client/status: Support INVALID_ARGUMENT status
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-10-03 17:05:08 +03:00
b9092aeb0c
[#274] go.mod: Update api-go
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-10-02 11:07:19 +03:00
1b67ab9608 [#275] client: Return status from all methods
Since status is checked first in handleError method, it should be returned from client methods

Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-09-24 18:29:32 +03:00
99d5bf913b [#269] netmap: Add tests for non-ascii attributes in SELECT IN
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-20 11:09:16 +00:00
e50838a33d [#269] netmap: Regenerate policy parser
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-20 11:09:16 +00:00
97cf56ba41 [#269] netmap: Support non-ascii attributes in SELECT IN
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-20 11:09:16 +00:00
07625e3bd1 [#269] Update ANTLR version 4.13.0 -> 4.13.1
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-20 11:09:16 +00:00
da2f0e7532 [#269] .gitignore: Ignore ANTLR jar file
The previous wildcard failed to properly match the ANTLR jar file.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-20 11:09:16 +00:00
114b4c14b5 [#269] Makefile: Update policy target
The previous policy target generated device-specific comments
(e.g., `/home/john_doe/repos/<...>`), which could result in
unnecessary file changes This behavior would confuse git and
require manual changes to resolve. The update ensures comments
are now device-agnostic, preventing unwanted changes.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-20 11:09:16 +00:00
e580ee991d [#271] Drop handling of system attributes with NeoFS prefix
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-18 09:59:38 +00:00
6821fe6fb2 [#271] object: Add UserAttributes method
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-18 09:59:38 +00:00
6009d089fc [#270] client: Use RPC call instead of Dial
After api-go upgrade created client doesn't establish connection after created,
so RPC call is required to establish and check connection.
RPC call returns status error, so conversion from status error to context error
is required to satisfy Dial contract and unit tests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-16 15:31:51 +03:00
3e455777fd [#270] go.mod: Upgrade api-go version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-16 15:10:02 +03:00
1dc3b77ac7 [#270] pool: Replace deprecated DialContext
`Healthcheck` request performed after client creation, so no extra RPC required.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-16 15:09:59 +03:00
88c6556c37 [#270] go.mod: Upgrade google.golang.org/grpc
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-16 15:09:54 +03:00
d342c0bc16 [#268] client: Make PayloadPatch correctly receive empty patch payload
* Make the method `PatchPayload` send a patch with empty
  payload patch if range's length is non-zero and if it's the
  first call.
* Empty payload patches just cut original object payload. So, these
  patches are also valid.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-12 15:12:52 +03:00
f0c599d06d [#268] client: Fix sequential PayloadPatch calls
* The flag 'firstPayloadPatch' keeps its state after first
  `PatchPayload` that make other calls incorrectly set patch
  ranges.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-12 15:09:01 +03:00
7d84d104fb [#260] *: Fix linter warnings
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:55:32 +00:00
812126a8ff [#260] .golangci.yml: Add protogetter linter
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:55:32 +00:00
d86223ed56 [#260] Makefile: Add pre-commit targets
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:55:32 +00:00
76a0cfdadb [#217] netmap: Return node netmap state directly
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:51:51 +00:00
46ee543899 [#265] go.mod: Use range over int
Since Go 1.22 a `for` statement with a `range` clause is able
to iterate through integer values from zero to an upper limit.

gopatch script:
@@
var i, e expression
@@
-for i := 0; i <= e - 1; i++ {
+for i := range e {
    ...
}

@@
var i, e expression
@@
-for i := 0; i <= e; i++ {
+for i := range e + 1 {
    ...
}

@@
var i, e expression
@@
-for i := 0; i < e; i++ {
+for i := range e {
    ...
}

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-09-04 12:37:46 +03:00
8f751d9dd0
[#263] go.mod: Update api-go
Remove `client.ContainerEACL` and related references. This change
was initiated by the removal of `ContainerService.GetExtendedACL`
from the API.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-03 12:36:28 +03:00
3c00f4eeac [#203] Add pool docs
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
2024-08-30 08:24:31 +03:00
f0b9493ce3 [#261] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-27 11:33:09 +03:00
28f140bf06 [#254] pool: Add parameter gracefulCloseOnSwitchTimeout
Add new param for waiting a little until current in-flight requests will be finished

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-22 08:02:51 +00:00
9115d3f281 [#1316] lint: Fix warnings
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-08-21 17:59:10 +03:00
cf225be0df [#1316] go.mod: Bump go version to 1.22
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-08-21 16:42:58 +03:00
338d1ef254 [#258] pool/tree: Add node address to error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-20 14:19:58 +03:00
6dd7be11d1 [#256] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-20 10:20:28 +03:00
203bba65a0 [#253] pool: Don't count regular FrostFS errors
Previously we count all frostfs errors like:
ObjectNotFound, EACLNotFound
because frostfs status is unconditionally resolved into built-in go errors
but handleError method handled built-in errors like internal network ones.
Since after resolving frostfs errors status is also returned we start check this first

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-14 16:48:51 +03:00
98aabc45a7 [#252] patcher: Fix applying patch from the same offset
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-13 18:58:21 +03:00
908c96a94d [#251] pool: Fix handlerError panic for objectPatch
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-13 15:28:39 +03:00
2077b35736 [#231] netmap: Add LIKE operation for filter
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-12 09:49:12 +03:00
92c7596157 [#231] go.mod: Update version for frostfs-api-go/v2
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-12 09:49:04 +03:00
a15b1264f5 [#231] policy: Bump jre version to 17
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-12 09:44:30 +03:00
5d58519253 [#249] pool: Introduce objectPatch method
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-08 18:26:40 +03:00
93171b3319 [#249] session: Support patch verb
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-08 18:26:40 +03:00
3ba7446157 [#249] client: Introduce ObjectPatch
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-08 18:26:35 +03:00
335aa18dc6 [#249] go.mod: Bump frostfs-api-go/v2 version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-06 16:57:42 +03:00
361739e860 [#247] object: Introduce patcher package
* Introduce `patcher` package that contains such interfaces to be
  implemented:
  - `PatchApplier` - the main patching engine that merges the stream
    of patches and the stream of original object payload divided by
    ranges. The merged streams result is output to `ChunkedObjectWriter`;
  - `RangeProvider` - provides the original object payload by ranges;
  - `HeaderProvider` - provides the original object header.
* Introduce `patcher` that implements `PatchApplier`;
* Cover all possible cases with unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-05 12:32:14 +00:00
6dd500def9 [#247] object: Introduce Patch type
* Make ToV2, FromV2 converters for Patch and PayloadPatch;
* Add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-05 12:32:14 +00:00
e83d6b7c6a [#244] pool/tree: Collect request duration statistic
After each request for tree pool statistic accumulated values are reset to zero.

Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-08-02 13:01:14 +03:00
9da46f566f [#243] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-26 14:13:49 +03:00
fa89999d91 [#242] pool: Log error that caused healthy status change
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-22 15:12:27 +03:00
7e94a6adf2 [#237] pool: Return creation epoch from object put
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-22 06:15:23 +00:00
ce8270568d [#239] go.mod: Update frostfs-contract
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-07-18 17:17:40 +03:00
7c06cdff2d [#239] pool/tree: Update tree service client
Update tree service to fix split tree problem.
Tree intermediate nodes can be duplicated so we must handle this.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-07-17 14:32:04 +03:00
e18b916231 [#238] go.mod: Update frostfs-contract to v0.19.3
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-07-16 11:36:21 +03:00
c4ff8a6cda [#236] go.mod: Update neo-go to v0.106.2
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 15:43:11 +03:00
fc7c524fcb [#236] *: Replace slice.Copy() with bytes.Clone()
The former is deprecated.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 15:43:07 +03:00
e977b8a94c [#236] netmap: Remove unused field from meanIQRAgg
It was there since the inception [1], but we never got to use it.

[1] 5931284e07

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:07 +03:00
6729f54c4e [#236] netmap: Reuse slice for weights in ContainerNodes()
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │    alloc     │               weights               │
                                                              │    sec/op    │    sec/op     vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.677µ ±  6%   8.384µ ± 10%       ~ (p=0.247 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.946µ ± 14%   7.998µ ±  6%       ~ (p=0.481 n=10)
geomean                                                         8.303µ         8.189µ        -1.38%

                                                              │    alloc     │               weights               │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   7.734Ki ± 0%   7.617Ki ± 0%  -1.52% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.156Ki ± 0%   7.039Ki ± 0%  -1.64% (p=0.000 n=10)
geomean                                                         7.440Ki        7.322Ki       -1.58%

                                                              │   alloc    │              weights               │
                                                              │ allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   92.00 ± 0%   77.00 ± 0%  -16.30% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   92.00 ± 0%   77.00 ± 0%  -16.30% (p=0.000 n=10)
geomean                                                         92.00        77.00       -16.30%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:07 +03:00
159a50fcf0 [#236] netmap: Reduce allocations in getSelection()
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │     new      │                alloc                 │
                                                              │    sec/op    │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                   9.227µ ± 13%   8.677µ ±  6%        ~ (p=0.165 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   9.189µ ±  7%   7.946µ ± 14%  -13.53% (p=0.001 n=10)
geomean                                                         9.208µ         8.303µ         -9.82%

                                                              │     new      │                alloc                │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.320Ki ± 0%   7.734Ki ± 0%  -7.04% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.742Ki ± 0%   7.156Ki ± 0%  -7.57% (p=0.000 n=10)
geomean                                                         8.026Ki        7.440Ki       -7.31%

                                                              │     new     │               alloc                │
                                                              │  allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   122.00 ± 0%   92.00 ± 0%  -24.59% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   122.00 ± 0%   92.00 ± 0%  -24.59% (p=0.000 n=10)
geomean                                                          122.0        92.00       -24.59%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:07 +03:00
a69f00903c [#236] netmap: Replace sort.Slice() with slices.Sort()
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │      old      │                 new                  │
                                                              │    sec/op     │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                   10.395µ ± 14%   9.227µ ± 13%  -11.24% (p=0.015 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   10.110µ ± 16%   9.189µ ±  7%        ~ (p=0.105 n=10)
geomean                                                          10.25µ         9.208µ        -10.18%

                                                              │     old      │                 new                 │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.695Ki ± 0%   8.320Ki ± 0%  -4.31% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   8.117Ki ± 0%   7.742Ki ± 0%  -4.62% (p=0.000 n=10)
geomean                                                         8.401Ki        8.026Ki       -4.47%

                                                              │    old     │                new                 │
                                                              │ allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   138.0 ± 0%   122.0 ± 0%  -11.59% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   138.0 ± 0%   122.0 ± 0%  -11.59% (p=0.000 n=10)
geomean                                                         138.0        122.0       -11.59%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:07 +03:00
9d89f08c7b [#236] go.mod: Bump min go version to go1.21
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:06 +03:00
51cefd4908 [#232] netmap: Allow empty values for unknown parameters in network config
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-07-09 12:19:25 +03:00
560cbbd1f1 [#234] pool: Update token expiration check in cache
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-05 12:36:17 +03:00
27e965007d [#233] pool: Introduce ape-manager methods
* Remove GetEACL, SetEACL methods as they are deprecated;
* Fix mock;
* Introduce add/remove/list methods to request ape-manager from
  the client.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-04 01:29:35 +03:00
1a5886e776 [#228] client: Move isClientErrMaintenance from node
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-06-17 17:07:30 +03:00
ebd8fcd168 [#224] object: Introduce parent attributes in EC-header
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 16:20:48 +03:00
717a7d00ef [#225] bearer: Introduce APEOverride field for the token
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 12:14:42 +00:00
dd23c6fd2b [#225] apemanager: Move apemanager to ape package
* Update go.mod;
* Fix packages;
* Fix client's `apemanager` methods.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 12:14:42 +00:00
6a52487edd [#226] pool/tree: Fix handling access denied error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-05-30 14:59:35 +03:00
c5c6272029 [#221] pool: Make sampler safe for concurrent using
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-05-20 14:14:14 +03:00
3de256d05e [#223] object: Introduce new fields for ECHeader
* Introduce `parentSplitID`, `parentSplitParentID` fields
  for `ECHeader`;
* Fix ECHeader's constructor;
* Fix `Split` and `Reconstruct`;
* Add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-17 15:16:28 +03:00
09b79d13f3 [#223] object: Introduce ec_parent search filter
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-17 15:03:40 +03:00
d4e6f4e125 [#223] go.mod: Update frosts-api-go/v2 version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-17 15:02:44 +03:00
b2ad1f3b3e [#215] client: Introduce apemanager rpc interface
* Introduce `APEManagerAddChain`, `APEManagerRemoveChain`, `APEManagerListChains`.
* Introduce reqeuest/response types for these handlers (Prm*, Res*).
* Inroduce status type for apemanager `APEManagerAccessDenied`; add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-14 10:59:52 +03:00
32a975a20d [#215] apemanager: Introduce apemanager types
* Introduce `Chain`, `ChainTarget` and `TargetType`.
* Implement api-v2 converters for the introduced types.
* Add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-14 10:59:49 +03:00
eaf36706a2 [#215] go.mod: Update frostfs-api-go/v2 package version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-08 11:18:18 +03:00
02c936f397 [#216] netmap: Add policy decode fuzz test
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-07 11:10:37 +00:00
99e02858af [#220] netmap: Fix setters for Replica.DataCount/ParityCount
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-07 09:34:14 +03:00
12ddefe078 [#218] object: Implement Range\RangeHash requests for EC object
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-02 11:01:21 +03:00
20ab57bf7e [#214] object: Implement Get\Head requests for EC object
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 11:07:26 +03:00
3790142b10 [#212] pool: Control sub tree nodes order
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-04-02 17:15:49 +03:00
ec0cb2169f [#211] object: Fix setIDWithSignature
* Calculate and set checksum before ID is calculated.
* Add header verification for parts in unit-test.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-29 13:48:04 +03:00
425d48f68b [#211] netmap: Introduce ReplicaDescriptor method
* Make ReplicaNumberByIndex deprecated.
* Introduce ReplicaDescriptor method that access i-th replica directly.
* Introduce new getters for ReplicaDescriptor.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-29 13:48:04 +03:00
6d0da3f861 [#211] go.mod: Update frostfs-api-go version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-29 13:48:04 +03:00
1af9b6d18b [#155] sdk-go: Add buffer support for payloadSizeLimiter
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-03-25 10:47:04 +03:00
bd2d350b09 [#205] object: Initial EC implementation
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
e9be3e6d94 [#205] netmap: Add well-known EC parameters to network config
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
70e9e40c7f [#205] netmap: Add EC statement to placement policy
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
d33b54d280 [#205] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
6f248436a5 [#210] client/put_transformer: Fix error handling
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-03-19 15:36:51 +03:00
edd40474e8 [#209] pre-commit: Add unit-test hook
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 13:35:48 +03:00
d9ec7c1988 [#209] Makefile: Allow to override test flags
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 13:35:48 +03:00
64b83f8220 [#209] Makefile: Update golangci-lint to 1.56.2
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 13:35:48 +03:00
7212f38115 [#209] pre-commit: Remove gitlint
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 13:35:48 +03:00
8081445ff2 [#208] go.mod: Bump frostfs-api-go version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-06 13:34:04 +03:00
6a7ef9d8c3 [#208] go.mod: Bump protobuf version
Found by vulncheck:
Vulnerability #1: GO-2024-2611
    Infinite loop in JSON unmarshaling in google.golang.org/protobuf
  More info: https://pkg.go.dev/vuln/GO-2024-2611
  Module: google.golang.org/protobuf
    Found in: google.golang.org/protobuf@v1.32.0
    Fixed in: google.golang.org/protobuf@v1.33.0

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-06 13:23:32 +03:00
6fe4e2541d [#207] netmap: Fix string escape in PlacementPolicy.String()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-01 15:02:05 +00:00
a5fab572ff [#206] Add session tokens for container read operations
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-03-01 15:18:53 +03:00
a86170f53a [#202] object: Reset marshal data on CutPayload
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 15:23:47 +03:00
aa41f71dcc [#202] eacl: Drop storage group test
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 15:23:47 +03:00
3a00fd51e4 [#202] go.mod: Update api-go version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 15:23:43 +03:00
65b4525b3b [#198] object/user: Add ScriptHash method
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-01-26 14:10:09 +00:00
7efff9d53d [#201] .forgejo: Update dco-go to v3
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-26 12:21:42 +03:00
110b7e4170 [#200] pre-commit: Fix linter invocation target
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-01-17 17:56:20 +03:00
56debcfa56 [#190] sdk-go: Gofumpt fixes
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 19:21:20 +03:00
157a9930e8 [#190] sdk-go: Pass user.ID by value
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 19:21:20 +03:00
1c07098740 [#194] make: Fix make test
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-17 11:48:51 +00:00
03d35dd1f3 [#166] netmap: Add support YML tests
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-17 14:43:03 +03:00
dea8759762 [#166] netmap: Move tests from JSON to YML
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-17 14:42:39 +03:00
3787477133 [#189] client: Make PrmDial fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-14 11:18:00 +03:00
e91d40e250 [#189] client: Make PrmInit fields public for client
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-13 15:18:07 +03:00
ab75edd709 [#191] pool/tree: Support limit request attempts
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-07 14:45:40 +03:00
8999d2f080 [#191] pool/tree: Support request id
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-07 14:45:35 +03:00
6fbe1595cb [#121] pool: Refactor PrmObjectSearch usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-01 17:45:15 +03:00
a9237aabd2 [#121] client: Make PrmObjectSearch fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-01 17:45:06 +03:00
a487033505 [#121] client: Nuke out unused prmCommonMeta
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-10-31 22:55:25 +03:00
51c3618850 [#121] client: Make PrmContainerList fields public
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-10-31 22:55:25 +03:00
665e5807bc [#188] transformer: Allow to provide size hint
For big objects with known size we can optimize allocation patterns
by providing size hint. As with any hint, it does not affect transformer
functionality: slices with capacity > MaxSize are never allocated.

```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                │     out     │
                                                │   sec/op    │
Transformer/small/no_size_hint-8                  65.44µ ± 3%
Transformer/small/no_size_hint,_with_buffer-8     64.24µ ± 5%
Transformer/small/with_size_hint,_with_buffer-8   58.70µ ± 5%
Transformer/big/no_size_hint-8                    367.8m ± 3%
Transformer/big/no_size_hint,_with_buffer-8       562.7m ± 0%
Transformer/big/with_size_hint,_with_buffer-8     385.6m ± 7%
geomean                                           5.197m

                                                │     out      │
                                                │     B/op     │
Transformer/small/no_size_hint-8                  13.40Ki ± 0%
Transformer/small/no_size_hint,_with_buffer-8     13.40Ki ± 0%
Transformer/small/with_size_hint,_with_buffer-8   13.39Ki ± 0%
Transformer/big/no_size_hint-8                    288.0Mi ± 0%
Transformer/big/no_size_hint,_with_buffer-8       1.390Gi ± 0%
Transformer/big/with_size_hint,_with_buffer-8     288.0Mi ± 0%
geomean                                           2.533Mi

                                                │    out     │
                                                │ allocs/op  │
Transformer/small/no_size_hint-8                  92.00 ± 0%
Transformer/small/no_size_hint,_with_buffer-8     92.00 ± 0%
Transformer/small/with_size_hint,_with_buffer-8   92.00 ± 0%
Transformer/big/no_size_hint-8                    546.5 ± 0%
Transformer/big/no_size_hint,_with_buffer-8       607.5 ± 0%
Transformer/big/with_size_hint,_with_buffer-8     545.5 ± 0%
geomean                                           228.1
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-30 12:13:04 +00:00
a02c0bfac8 [#186] netmap: Marshal policy with brackets
Brackets can be semantically important and must not be omitted,
otherwise the output is plain wrong.
We do not take the responsibility to preserve every bracket, though,
because parser does some optimizations related to grouping long chains
of filters combined with the same operation.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-27 10:54:45 +03:00
20d325e307 [#167] netmap: Fix reverse min agregator
The higher the price, the lower reverse min weight should be.
Previously nodes with 0 price had 0 weight which is a bit misleading.

Introduced in d71a0e0755.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-27 07:53:19 +00:00
670619d242 [#131] client: keep backwards-compatibility, update README.md, fix chore
Signed-off-by: Egor Olefirenko <egor.olefirenko892@gmail.com>
2023-10-26 14:35:49 +00:00
0d79d10482 [#131] client: rename option consistently and fix test
Signed-off-by: Egor Olefirenko <egor.olefirenko892@gmail.com>
2023-10-26 14:35:49 +00:00
9727beb47d [#131] client: Switch ResolveFrostFSFailures to DontResolveFrostFSFailures option
Signed-off-by: Egor Olefirenko <egor.olefirenko892@gmail.com>
2023-10-26 14:35:49 +00:00
84315fab6a [#121] client: Make PrmBalanceGet fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-23 18:53:14 +03:00
71335489ae [#183] forgejo: Make linter great again
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-23 18:13:40 +03:00
4c1feaf2cb [#182] pool: Fix linter error about deprecated methods
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-23 17:40:00 +03:00
5804128ff3 [#121] client: Make PrmObjectPutSingle fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-19 08:38:13 +00:00
abd38c918e [#177] pool: Support NetMapSnapshot method
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-18 11:31:53 +03:00
fc4551b843 [#172] pool: Use priority of errors in tree pool
When retry happens, use priority map to decide
which error to return. Consider network errors
less desirable than business logic errors.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-11 12:00:34 +03:00
eb5288f4a5 [#172] pool: Do more retries on unexpected tree service responses
1. Try its best by looking for nodes during 'GetNodeByPath'
2. Retry on 'tree not found' and other not found errors

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-06 11:12:48 +03:00
60463871db [#171] pool: Add test for healthy status monitor
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-03 19:47:22 +03:00
8a04638749 [#171] pool: Close only dialed connections
To avoid panics during close operation, close
only dialed connections.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-03 19:47:22 +03:00
ddbfb758c9 [#171] pool: Use dial status to close connections during restarts
Every client restart, pool creates new client instance. If client
failed due to dial error, there was no prior connection and go
routine on a server side. If client failed due to communication
or business logic errors, then server side maintains connection and
client should close it to avoid routine and connection leak.

Dialing is a part of healthcheck, so health status is now a enum
of three values:
- unhealthy due to dial fail,
- unhealthy due to transmission fail,
- healthy.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-03 17:00:36 +03:00
d71a0e0755 [#88] netmap: use bool, fix hrw_sort tests
Signed-off-by: Andrew Danilin <andnilin@gmail.com>
2023-10-03 07:05:03 +00:00
163b3e1961 [#88] netmap: fix min aggregator bug, add tests
Signed-off-by: Andrew Danilin <andnilin@gmail.com>
2023-10-03 07:05:03 +00:00
84b9d29fc9 [#170] checksum: Use constant mapping for checksum types
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-28 17:20:24 +03:00
99c273f499 [#169] pool: Close inner pools during close routine
Some apps do not reuse pool instance and expect that
`pool.Close()` free resources. But it didn't actually
close inner SDK clients, so it leads to goroutine leak
in storage.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-09-20 12:16:13 +03:00
555ccc63b2 [#167] netmap: Allow to select insufficient number of nodes
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-15 14:47:54 +03:00
0550438b53 [#167] netmap/tests: Add replica to invalid tests
Make sure we fail exactly because of the reason specified.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-15 14:33:44 +03:00
c899163860 [#167] netmap/tests: Add json file name to the test output
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-15 14:33:44 +03:00
ac8fc6d440 [#162] netmap: Allow to parse single unnamed selectors
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-11 15:22:24 +03:00
0a0b590df3 [#162] Fix pre-commit warnings
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-11 15:21:35 +03:00
4df642e941 [#162] netmap: Fix possible panic
Placement policy is unvalidated external input.
Under no circumstances should we panic here.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-11 15:19:56 +03:00
8bc64e088e [#161] .golangci.yml: Reenable deprecated usage warnings
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-08 17:17:02 +03:00
49ad985cad [#161] *: Do not use math/rand.Read()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-08 17:17:02 +03:00
aa12d8c6a6 [#121] client: Make PrmObjectHash fields public
* Introduce buildRequest for PrmObjectHash

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-08 13:36:45 +00:00
303508328a [#121] pool: Refactor PrmSessionCreate usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-05 17:32:03 +03:00
55699d1480 [#121] client: Make PrmSessionCreate fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-05 17:30:42 +03:00
55a1f23e71 [#121] client: Make PrmEndpointInfo, PrmNetworkInfo fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-04 19:55:23 +03:00
291a71ba84 [#121] client: Make PrmAnnounceSpace fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-04 19:53:34 +03:00
5a471e5002 [#121] client: Make PrmObjectDelete fields public
* Introduce buildRequest for PrmObjectDelete
* Refactor the usage of these params in pool

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-04 14:14:22 +00:00
b5fe52d6bd [#150] policy: Check for redundant selectors and filters
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-29 14:16:57 +03:00
84e7e69f98 [#121] client: Make PrmObjectGet/Head/GetRange fields public
* Remove common PrmObjectRead structure
* Introduce buildRequest for PrmObjectGet/Head/GetRange
* Refactor the usage of these params in pool

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-28 11:26:57 +03:00
46a214d065 [#149] pool: Configure homomorphic hash and buffer size
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-25 09:45:15 +03:00
202412230a [#115] pool: Drop part buffer pool
Tests showed that using part buffer pool doesn't save memory a lot.
Especially on big parts.
Probably we can use pool only for small parts
after adding buffer in payloadSizeLimiter

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-21 12:03:03 +03:00
3cb3841073 [#115] pool: Try putSingle if possible
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-21 12:03:03 +03:00
faeeeab87a [#114] pool: Don't use part buffers when client cut is off
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-21 12:02:40 +03:00
cae215534f [#114] pool: Fix linter errors
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-21 12:02:40 +03:00
518fb79bc0 [#114] pool: Support client cut with memory limiter
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-21 12:02:40 +03:00
342524159a [#121] pool: Make PrmContainerSetEACL fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-21 10:33:19 +03:00
22978303f8 [#121] clientt: Make PrmContainerSetEACL fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-18 18:05:47 +03:00
6fdbe75517 [#121] pool: Make PrmContainerEACL fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-16 09:53:47 +00:00
3353940554 [#121] client: Make PrmContainerEACL fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-16 09:53:47 +00:00
a3b5d4d4f5 [#51] Add node addresses as debug information
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-08-15 09:51:40 +03:00
0314b326d3 [#51] Add current nodes as external statistics
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-08-15 09:51:13 +03:00
0382785763 [#146] .forgejo: Update DCO action
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-11 12:34:29 +03:00
548a81d3e6 [#48] client: Refactor accounting.Balance()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-10 11:05:30 +00:00
d48788c7a9 [#144] Bump required go version to go1.20
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-09 09:52:35 +03:00
6353df8bca [#142] Fix unwrapErr for go 1.20
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-05 15:58:25 +00:00
936e6d230b [#121] pool: Add wait params validation for containerPut method
* Add WaitParams.CheckValidity() check because SetWaitParams is deprecated,
  but parameters were checked within this setter with checkForPositive()

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-05 15:58:14 +00:00
be28b89312 [#121] pool: Make PrmContainerDelete fields public
* Refactor client PrmContainerDelete usage
* Introduce WaitParams CheckValidity method

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-05 15:58:14 +00:00
9e5faaf829 [#121] client: Make PrmContainerDelete fields public
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-05 15:58:14 +00:00
3dc8129ed7 [#135] Make all error status receivers pointers
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-04 08:35:01 +00:00
55c52c8d5d [#121] pool: Make PrmContainerGet fields public
* Also refactor client PrmContainerGet usage

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-03 10:54:13 +03:00
d376302a3b [#121] client: Make PrmContainerGet fields public
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-02 18:23:32 +03:00
363f153eaf [#136] pool: Set order field to get subtree
With new revision of tree service protocol, getSubTree
requires to use explicit order field.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-08-02 10:32:37 +00:00
95b987b818 [#137] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-02 12:04:10 +03:00
13d0b170d2 [#121] client: Make pool PrmContainerPut struct fields public
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-01 09:59:57 +00:00
18a9e4bceb [#121] client: Make PrmContainerPut struct fields public
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-01 09:59:57 +00:00
0fe0d71678 [#133] .forgejo: Add names to actions
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-31 11:33:39 +00:00
78d1439b2c [#133] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-31 11:33:39 +00:00
0886d80083 [#69] Add close for nns.Dial
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-07-31 11:05:26 +03:00
ecb1fef78c [#129] client: Do not override error status WriteObject()
Here is a scenario:
1. `resolveFrostFSErrors` is false.
2. The first object part was not written, which was signified in status.
3. The second part was written correctly.

Client now thinks that the object is written even though it was not.
In theory we could also return only status, but client-side splitting
is not a single RPC, so it makes sense.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-26 14:38:43 +03:00
5defed4ab4 [#126] go.mod: Update frostfs-api-go package version
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-07-19 16:03:56 +03:00
fb05f7dc5e [#112] Add basic documentation for placement policies
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-18 10:53:54 +00:00
b91f9d8c79 [#xx] Add support for SELECT-FILTER expressions
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-17 10:29:48 +00:00
b9afe7a2f9 [#42] Add ResolveContractHash method
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
2023-07-12 22:22:58 +03:00
998fe1a7ab [#102] netmap: properly process multiple REP
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-11 14:21:35 +00:00
c359a7465a [#64] transformer: Simplify interface
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-11 07:33:12 +00:00
d70ef2187b [#97] Add a method IterateUserAttributes in Container
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-07-10 09:05:01 +00:00
ac95b87e7c [#101] Add Equals for Address
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-07-10 09:05:01 +00:00
863be6034f [#104] Update neo-go/pkg/interop version
neo-go module uses broken commit of interop package.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-10 09:04:10 +00:00
35346a01c9 [#109] Bump neo-go version
Synced version with frostfs-node, see frostfs-node#417

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-10 09:04:10 +00:00
fe35373d8f [#107] go.mod: Tidy dependencies
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-07 14:57:16 +03:00
388d1ca1de [#107] pool/tree: Support grpc schemas
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-07 14:57:13 +03:00
14ed3e177d [#104] nemtap: Escape special symbols in filters
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-07 09:33:00 +00:00
fe28c33277 [#104] netmap: Add test with quote escaping
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-07 09:33:00 +00:00
98cab7ed61 [#103] object: Add PutSingle method code
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-06 17:06:17 +03:00
37e22b33ad [#103] sdk-go: Update api-go version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-05 18:10:42 +03:00
769f6eec05 [#105] client: Fix revive linter warning
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-05 15:52:06 +03:00
5d62cef27e [#98] Fix linter issues
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-28 12:13:02 +00:00
c0c0c588b5 [#98] Add forgejo workflows
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-28 12:13:02 +00:00
2f88460172 [#83] Allow to split objects in the client
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-28 12:12:37 +00:00
66cb5dcf34 [#83] Update .gitignore
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-28 12:12:37 +00:00
91e80ba743 [#73] pool/tree: Fix index in retry loop
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-28 13:06:23 +03:00
c243b443bc [#80] objectwriter: Allow custimize maxChunkLen
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-26 18:13:04 +03:00
aa8ffebc63 [#95] transformer: Set parent version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-23 13:48:02 +03:00
9d40228cec [#73] pool/tree: Fix client healthy status
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 17:01:55 +03:00
af40dc68f0 [#84] pool/tree: Allow to pass gRPC dial options
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 15:55:07 +03:00
981d24a493 [#84] pool: Allow to pass gRPC dial options
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 15:54:48 +03:00
19adb4dffa [#73] pool/tree: Add tests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 14:37:57 +03:00
51e022ab8c [#73] pool/tree: Add tree pool
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 14:37:57 +03:00
0d3dacb515 [#73] pool: Add getters for NodeParam
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 14:37:57 +03:00
b2e302624d [#73] pool/tree: Add proto tree service client
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 14:37:57 +03:00
fcbf96add6 [#76] Add UNIQUE keyword
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-06-06 13:54:07 +03:00
4f48f6c9e0 [#78] netmap: Add new keywords NOT and UNIQUE
* Add the rule for NOT operation to the policy parser grammar
* Regenerate query parse
* Implement NOT in filter
* Add unit-tests

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-06-02 17:47:20 +03:00
ec59ebfd88 [#87] go.mod: Update hrw
```
                                                              │      old      │                 new                  │
                                                              │    sec/op     │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                    13.07µ ± 12%   11.03µ ± 12%  -15.63% (p=0.003 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   11.383µ ±  9%   9.970µ ± 19%  -12.42% (p=0.005 n=10)
geomean                                                          12.20µ         10.49µ        -14.04%

                                                              │      old      │                 new                  │
                                                              │     B/op      │     B/op      vs base                │
Netmap_ContainerNodes/REP_2-8                                   10.203Ki ± 0%   7.711Ki ± 0%  -24.43% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8    9.641Ki ± 0%   7.148Ki ± 0%  -25.85% (p=0.000 n=10)
geomean                                                          9.918Ki        7.424Ki       -25.14%

                                                              │    old     │                new                 │
                                                              │ allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   216.0 ± 0%   131.0 ± 0%  -39.35% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   215.0 ± 0%   130.0 ± 0%  -39.53% (p=0.000 n=10)
geomean                                                         215.5        130.5       -39.44%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-02 14:54:40 +03:00
030ff2f122 [#87] netmap: Add benchmark for ContainerNodes()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-02 14:54:40 +03:00
0f7455ff7a [#75] Update antlr4 version to 4.13.0
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-01 13:15:11 +00:00
e6b662cfa6 [#75] Add make targets policy and docker/%
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-01 13:15:11 +00:00
406c2324d4 [#85] go.mod: Tidy
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-01 10:02:23 +03:00
10482ffbed [#82] client: Allow to pass gRPC dial options
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-30 16:51:22 +03:00
f5b23eb225 [#74] netmap/parser: Update antlr4 generator
Also, add -no-listener option, as we use visitor only.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 17:47:24 +03:00
70f23dd1ea [#74] pre-commit: Exclude auto-generated files
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 17:47:24 +03:00
57f874048b [#74] go.mod: Update antlr4 dependency
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 17:47:24 +03:00
a397d1fd15 [#74] go.mod: Update dependencies
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 17:47:23 +03:00
9803c2816a [#74] pool: move to sync/atomic
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 14:58:30 +03:00
d04d96b42e [#74] go.mod: Move to go1.19
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 14:53:47 +03:00
9a072a8f49 [#68] Replace interface{} with any
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-15 17:21:49 +03:00
15b4287092 [#49] bearer: Allow empty eacl if token is impersonated
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-05 12:45:39 +03:00
d4fe9a193d [#66] transformer: Accept constructor in NextTarget
The code of frostfs-node is not yet ready to reuse egress target for
multiple objects, let's postpone until #64.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 11:22:09 +03:00
c42a6119ff [#66] transformer: Extend tests
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 11:22:09 +03:00
29b188db57 [#52] Remove storage groups and audit
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 08:20:37 +00:00
38b03ff28b [#63] transformer: Resolve funlen linter
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-25 13:29:42 +03:00
0fa23a9b14 [#63] transformer: Add context to methods
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-25 13:29:42 +03:00
d0762d037d [#44] pool: Add copies number vector when putting object
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-04-20 13:41:05 +03:00
db5b89496d [#49] bearer: Consider impersonate in fillBody
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-18 14:54:05 +00:00
7c75db2f2d [#59] netmap: Remove unused param from getSelection()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 14:11:04 +00:00
dce55a436a [#9999] sdk: Up api-go version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-18 09:59:52 +00:00
cae2f37cdd client: rename formRequest() to buildRequest()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 09:58:34 +00:00
f60bea4be5 [#48] client: Refactor SessionCreate()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 09:58:34 +00:00
a16fc40c39 [#48] client: Refactor EndpointInfo()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 09:58:34 +00:00
40d966bec2 [#48] client: Refactor NetworkInfo()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 09:58:34 +00:00
Pavel Karpy
d0c5d837d2 [#56] *: Drop reputation system
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-18 07:52:16 +00:00
237b90f744 [#49] bearer: Add impersonate flag
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-17 11:51:38 +03:00
c8e620ad24 [#53] sdk-go: Drop subnet
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-14 16:31:08 +03:00
591dd1247d [#48] client: Refactor ContainerPut()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
57619fbbe4 [#48] client: Refactor ContainerSetEACL()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
8bc8f1f365 [#48] client: Refactor ContainerDelete()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
09ed2863fc [#48] client: Refactor ContainerEACL()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
8e2f77890f [#48] client: Refactor ContainerGet()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
8852b262f2 [#48] client: Refactor ContainerList()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
b2c66cb99e [#48] client: Refactor ContainerAnnounceUsedSpace()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
6c9b92c9dc [#48] client: Execute metainfo callback in processResponse()
It was lost in 8c5333ea55.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
772fa90983 [#48] client/container: Form requests in a separate function
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:06:52 +03:00
1bfa9ecdb0 [#48] client: Remove ctx == nil checks
Much less useful after we made context to be passed explicitly.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 09:04:09 +03:00
55b06cd764 [#48] client: Split container methods by files
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 09:04:09 +03:00
Pavel Karpy
f41860f9bd [#46] client: Allow set copy_number for every placement vector
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-11 10:06:55 +00:00
Pavel Karpy
423b320f91 [#46] go.mod: Update api-go version
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-11 10:06:55 +00:00
f8c34b45f3 [#47] eacl: Fix duplicate imports
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
fa9573e857 [#47] client: Pass context to Dial() explicitly
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
bc62e2f712 [#47] pool: Resolve contextcheck and containedctx linters
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
31271ad8b1 [#47] netmap: Make PlacementPolicy.WriteStringTo() pass nofunlen, cogognit
REP statement is obligatory, thus insert newline before each statement
after REP.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
25e9336d68 [#47] container: Make readFromV2() pass funlen
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
4cd755877c [#47] client: Do not check context in NetmapSnapshot()
It is passed explicitly, non-nil by convention.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
1395b282fe [#47] acl: Remove unused resetBit()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
708d933fe3 [#47] .golangci.yml: Unify with other FrostFS repos
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
4fa52312c7 [#47] reputation: Fix misspelling
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
552219b8e1 [#16] pool: Fix counting context canceled error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-29 15:58:04 +03:00
cfb8a7b914 [#41] .gitlint: Synchronize settings across FrostFS repos
This change allows to use `[#xx]` placeholders for issue number.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-24 22:07:13 +03:00
4438f115fb [#39] Add Issue Template
Add bug report and feature request templates

Signed-off-by: Liza <e.chichindaeva@yadro.com>
2023-03-23 12:25:57 +03:00
bec77f280a [#37] container: Support legacy sys attributes
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-16 11:14:42 +03:00
df2090c2be [#37] netmap: Update tests for new sys attributes
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-16 11:14:36 +03:00
7e6592b28e [#37] go.mod: Update api-go for new sys attributes
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-16 11:14:27 +03:00
d589d51509 [#19] transformer: Fix dependencies
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-16 09:51:03 +03:00
25588ee3be [#19] transformer: Do not allocate intermeate slice for hashers
```
name                 old time/op    new time/op    delta
Transformer/small-8    73.7µs ±15%    72.4µs ±16%    ~     (p=0.604 n=10+9)
Transformer/big-8       1.36s ± 4%     1.36s ± 8%    ~     (p=0.579 n=10+10)

name                 old alloc/op   new alloc/op   delta
Transformer/small-8    7.67kB ± 0%    7.57kB ± 0%  -1.36%  (p=0.000 n=10+10)
Transformer/big-8      49.0kB ± 0%    48.3kB ± 0%  -1.48%  (p=0.000 n=10+10)

name                 old allocs/op  new allocs/op  delta
Transformer/small-8       101 ± 0%        98 ± 0%  -2.97%  (p=0.000 n=10+10)
Transformer/big-8         609 ± 0%       591 ± 1%  -3.00%  (p=0.000 n=10+9)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
9407f30248 [#19] transformer: Optimize payload hashers a bit
```
name                 old time/op    new time/op    delta
Transformer/small-8    74.8µs ±11%    73.7µs ±15%    ~     (p=0.529 n=10+10)
Transformer/big-8       1.38s ±11%     1.36s ± 4%    ~     (p=0.796 n=10+10)

name                 old alloc/op   new alloc/op   delta
Transformer/small-8    7.69kB ± 0%    7.67kB ± 0%  -0.21%  (p=0.000 n=10+10)
Transformer/big-8      49.2kB ± 0%    49.0kB ± 0%  -0.48%  (p=0.004 n=10+10)

name                 old allocs/op  new allocs/op  delta
Transformer/small-8       102 ± 0%       101 ± 0%  -0.98%  (p=0.000 n=9+10)
Transformer/big-8         620 ± 1%       609 ± 0%  -1.66%  (p=0.000 n=10+10)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
94c0a607b5 [#19] transformer: Add a target which sends parts to a channel
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
e45647de3c [#19] transformer: Do not reuse memory of sent objects
Slower, but more correct.
```
name                 old time/op    new time/op    delta
Transformer/small-8    72.4µs ± 8%    74.8µs ±11%     ~     (p=0.278 n=9+10)
Transformer/big-8       1.31s ± 8%     1.38s ±11%   +5.50%  (p=0.035 n=10+10)

name                 old alloc/op   new alloc/op   delta
Transformer/small-8    7.39kB ± 0%    7.69kB ± 0%   +4.04%  (p=0.000 n=10+10)
Transformer/big-8      46.9kB ± 0%    49.2kB ± 0%   +4.87%  (p=0.000 n=10+10)

name                 old allocs/op  new allocs/op  delta
Transformer/small-8      94.6 ± 1%     102.0 ± 0%   +7.82%  (p=0.000 n=10+9)
Transformer/big-8         560 ± 0%       620 ± 1%  +10.66%  (p=0.000 n=10+10)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
611e20587b [#19] transformer/test: Check owner ID and payload hash for parts
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
eba6831125 [#19] transformer/test: Add helper functions
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
7e3810d654 [#19] transformer: Move EpochSource to other types
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
cc0fef2c55 [#19] transformer: Merge formatter and payload splitter
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
b696d3c70e [#19] transformer: Make writeChunk non-recursive
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
1c94309d7a [#19] transformer: Simplify AccessIdentifiers
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
f43f18ecda [#19] transformer: Cover with unit-tests
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
ac8442bf99 [#19] object: Move transformer implementation from node
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
0ad877288e [TrueCloudLab#16] pool: Don't count grpc canceled error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-10 06:58:34 +00:00
0e1999c965 [#23] pre-commit: Add gitlint hook
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-09 22:53:16 +03:00
b461aa64b8 [#23] pre-commit: Add golangci-lint hook
Also, fix minor issues. Skip deprecated warning for now.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-09 22:53:14 +03:00
b761fd8070 [#23] pre-commit: Add initial configuration
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-09 22:52:51 +03:00
94476f9055 Rename package name
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 15:47:21 +03:00
310 changed files with 22839 additions and 11719 deletions

View file

@ -0,0 +1,21 @@
name: DCO
on: [pull_request]
jobs:
dco:
name: DCO
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Setup Go
uses: actions/setup-go@v3
with:
go-version: '1.23'
- name: Run commit format checker
uses: https://git.frostfs.info/TrueCloudLab/dco-go@v3
with:
from: 'origin/${{ github.event.pull_request.base.ref }}'

View file

@ -0,0 +1,39 @@
name: Tests and linters
on: [pull_request]
jobs:
lint:
name: Lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: '1.23'
cache: true
- name: Install linters
run: make lint-install
- name: Run linters
run: make lint
tests:
name: Tests
runs-on: ubuntu-latest
strategy:
matrix:
go_versions: [ '1.22', '1.23' ]
fail-fast: false
steps:
- uses: actions/checkout@v3
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: '${{ matrix.go_versions }}'
- name: Run tests
run: make test

45
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View file

@ -0,0 +1,45 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: community, triage, bug
assignees: ''
---
<!--- Provide a general summary of the issue in the Title above -->
## Expected Behavior
<!--- If you're describing a bug, tell us what should happen -->
<!--- If you're suggesting a change/improvement, tell us how it should work -->
## Current Behavior
<!--- If describing a bug, tell us what happens instead of the expected behavior -->
<!--- If suggesting a change/improvement, explain the difference from current behavior -->
## Possible Solution
<!--- Not obligatory -->
<!--- If no reason/fix/additions for the bug can be suggested, -->
<!--- uncomment the following phrase: -->
<!--- No fix can be suggested by a QA engineer. Further solutions shall be up to developers. -->
## Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to -->
<!--- reproduce this bug. -->
1.
## Context
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context helps us come up with a solution that is most useful in the real world -->
## Regression
<!-- Is this issue a regression? (Yes / No) -->
<!-- If Yes, optionally please include version or commit id or PR# that caused this regression, if you have these details. -->
## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used:
* Server setup and configuration:
* Operating System and version (`uname -a`):

1
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View file

@ -0,0 +1 @@
blank_issues_enabled: false

View file

@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: community, triage
assignees: ''
---
## Is your feature request related to a problem? Please describe.
<!--- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
## Describe the solution you'd like
<!--- A clear and concise description of what you want to happen. -->
## Describe alternatives you've considered
<!--- A clear and concise description of any alternative solutions or features you've considered. -->
## Additional context
<!--- Add any other context or screenshots about the feature request here. -->

View file

@ -1,21 +0,0 @@
name: DCO check
on:
pull_request:
branches:
- master
jobs:
commits_check_job:
runs-on: ubuntu-latest
name: Commits Check
steps:
- name: Get PR Commits
id: 'get-pr-commits'
uses: tim-actions/get-pr-commits@master
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: DCO Check
uses: tim-actions/dco@master
with:
commits: ${{ steps.get-pr-commits.outputs.commits }}

View file

@ -1,52 +0,0 @@
name: neofs-sdk-go tests
on:
pull_request:
branches:
- master
types: [opened, synchronize]
paths-ignore:
- '**/*.md'
workflow_dispatch:
jobs:
tests:
name: Tests
runs-on: ubuntu-20.04
strategy:
matrix:
go_versions: [ '1.18.x', '1.19.x', '1.20.x' ]
fail-fast: false
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: '${{ matrix.go_versions }}'
- name: Restore Go modules from cache
uses: actions/cache@v3
with:
path: /home/runner/go/pkg/mod
key: deps-${{ hashFiles('go.sum') }}
- name: Update Go modules
run: make dep
- name: Run tests
run: make test
lint:
runs-on: ubuntu-20.04
steps:
- name: Check out code
uses: actions/checkout@v3
- name: golangci-lint
uses: golangci/golangci-lint-action@v3
with:
version: latest
only-new-issues: true

12
.gitignore vendored
View file

@ -20,4 +20,14 @@ vendor/
# coverage # coverage
coverage.txt coverage.txt
coverage.html coverage.html
# antlr tool jar
antlr*.jar
# tempfiles
.cache
# binary
bin/
release/

View file

@ -4,15 +4,16 @@
# options for analysis running # options for analysis running
run: run:
# timeout for analysis, e.g. 30s, 5m, default is 1m # timeout for analysis, e.g. 30s, 5m, default is 1m
timeout: 5m timeout: 10m
# include test files or not, default is true # include test files or not, default is true
tests: true tests: false
# output configuration options # output configuration options
output: output:
# colored-line-number|line-number|json|tab|checkstyle|code-climate, default is "colored-line-number" # colored-line-number|line-number|json|tab|checkstyle|code-climate, default is "colored-line-number"
format: tab formats:
- format: tab
# all available settings of specific linters # all available settings of specific linters
linters-settings: linters-settings:
@ -24,6 +25,13 @@ linters-settings:
govet: govet:
# report about shadowed variables # report about shadowed variables
check-shadowing: false check-shadowing: false
staticcheck:
checks: ["all"]
funlen:
lines: 80 # default 60
statements: 60 # default 40
gocognit:
min-complexity: 40 # default 30
linters: linters:
enable: enable:
@ -34,23 +42,27 @@ linters:
# some default golangci-lint linters # some default golangci-lint linters
- errcheck - errcheck
- gosimple - gosimple
- godot
- ineffassign - ineffassign
- staticcheck - staticcheck
- typecheck - typecheck
- unused - unused
# extra linters # extra linters
- bidichk
- durationcheck
- exhaustive - exhaustive
- godot - exportloopref
- gofmt - gofmt
- whitespace
- goimports - goimports
- misspell
- predeclared
- reassign
- whitespace
- containedctx
- funlen
- gocognit
- contextcheck
- protogetter
disable-all: true disable-all: true
fast: false fast: false
issues:
include:
- EXC0002 # should have a comment
- EXC0003 # test/Test ... consider calling this
- EXC0004 # govet
- EXC0005 # C-style breaks

36
.pre-commit-config.yaml Normal file
View file

@ -0,0 +1,36 @@
ci:
autofix_prs: false
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: check-added-large-files
- id: check-case-conflict
- id: check-executables-have-shebangs
- id: check-shebang-scripts-are-executable
- id: check-merge-conflict
- id: check-json
- id: check-xml
- id: check-yaml
- id: trailing-whitespace
args: [--markdown-linebreak-ext=md]
- id: end-of-file-fixer
exclude: "(.key|.interp|.tokens)$"
- repo: local
hooks:
- id: go-unit-tests
name: go unit tests
entry: make test GOFLAGS=''
pass_filenames: false
types: [go]
language: system
- repo: local
hooks:
- id: make-lint
name: Run Make Lint
entry: make lint
language: system
pass_filenames: false

4
Dockerfile Normal file
View file

@ -0,0 +1,4 @@
FROM golang:1.22
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install make openjdk-17-jre -y
WORKDIR /work

58
Makefile Normal file → Executable file
View file

@ -1,8 +1,16 @@
#!/usr/bin/make -f #!/usr/bin/make -f
ANTLR_VERSION=4.13.1
TMP_DIR := .cache
LINT_VERSION ?= 1.60.1
TRUECLOUDLAB_LINT_VERSION ?= 0.0.6
OUTPUT_LINT_DIR ?= $(shell pwd)/bin
LINT_DIR = $(OUTPUT_LINT_DIR)/golangci-lint-$(LINT_VERSION)-v$(TRUECLOUDLAB_LINT_VERSION)
# Run tests # Run tests
test: GOFLAGS ?= "-cover -count=1"
test: test:
@go test ./... -cover @GOFLAGS=$(GOFLAGS) go test ./...
# Pull go dependencies # Pull go dependencies
dep: dep:
@ -13,9 +21,23 @@ dep:
@CGO_ENABLED=0 \ @CGO_ENABLED=0 \
go mod tidy -v && echo OK go mod tidy -v && echo OK
# Install linters
lint-install:
@mkdir -p $(TMP_DIR)
@rm -rf $(TMP_DIR)/linters
@git -c advice.detachedHead=false clone --branch v$(TRUECLOUDLAB_LINT_VERSION) https://git.frostfs.info/TrueCloudLab/linters.git $(TMP_DIR)/linters
@@make -C $(TMP_DIR)/linters lib CGO_ENABLED=1 OUT_DIR=$(OUTPUT_LINT_DIR)
@rm -rf $(TMP_DIR)/linters
@rmdir $(TMP_DIR) 2>/dev/null || true
@CGO_ENABLED=1 GOBIN=$(LINT_DIR) go install github.com/golangci/golangci-lint/cmd/golangci-lint@v$(LINT_VERSION)
# Run linters # Run linters
lint: lint:
@golangci-lint --timeout=5m run @if [ ! -d "$(LINT_DIR)" ]; then \
echo "Run make lint-install"; \
exit 1; \
fi
$(LINT_DIR)/golangci-lint run
# Run tests with race detection and produce coverage output # Run tests with race detection and produce coverage output
cover: cover:
@ -29,6 +51,24 @@ format:
@echo "⇒ Processing goimports check" @echo "⇒ Processing goimports check"
@goimports -w ./ @goimports -w ./
policy:
@wget -q https://www.antlr.org/download/antlr-${ANTLR_VERSION}-complete.jar -O antlr4-tool.jar
@java -Xmx500M -cp antlr4-tool.jar org.antlr.v4.Tool -Dlanguage=Go \
-no-listener -visitor netmap/parser/Query.g4 netmap/parser/QueryLexer.g4
# Run `make %` in truecloudlab/frostfs-sdk-go container(Golang+Java)
docker/%:
@docker build -t truecloudlab/frostfs-sdk-go --platform linux/amd64 . > /dev/null
@docker run --rm -t \
-v `pwd`:/work \
-u "$$(id -u):$$(id -g)" \
--env HOME=/work \
truecloudlab/frostfs-sdk-go make $*
# Synchronize tree service
sync-tree:
@./syncTree.sh
# Show this help prompt # Show this help prompt
help: help:
@echo ' Usage:' @echo ' Usage:'
@ -37,4 +77,16 @@ help:
@echo '' @echo ''
@echo ' Targets:' @echo ' Targets:'
@echo '' @echo ''
@awk '/^#/{ comment = substr($$0,3) } comment && /^[a-zA-Z][a-zA-Z0-9_-]+ ?:/{ print " ", $$1, comment }' $(MAKEFILE_LIST) | column -t -s ':' | grep -v 'IGNORE' | sort -u @awk '/^#/{ comment = substr($$0,3) } comment && /^[a-zA-Z][a-zA-Z0-9_-]+ ?:/{ print " ", $$1, comment }' $(MAKEFILE_LIST) | column -t -s ':' | grep -v 'IGNORE' | sort -u
# Activate pre-commit hooks
pre-commit:
pre-commit install --hook-type pre-commit
# Deactivate pre-commit hooks
unpre-commit:
pre-commit uninstall --hook-type pre-commit
# Run pre-commit hooks
pre-commit-run:
@pre-commit run --all-files --hook-stage manual

View file

@ -1,6 +1,6 @@
# frostfs-sdk-go # frostfs-sdk-go
Go implementation of FrostFS SDK. It contains high-level version-independent wrappers Go implementation of FrostFS SDK. It contains high-level version-independent wrappers
for structures from [frostfs-api-go](https://github.com/TrueCloudLab/frostfs-api-go) as well as for structures from [frostfs-api-go](https://git.frostfs.info/TrueCloudLab/frostfs-api-go) as well as
helper functions for simplifying node/dApp implementations. helper functions for simplifying node/dApp implementations.
## Repository structure ## Repository structure
@ -14,7 +14,7 @@ There is also a reference implementation of checking algorithm which is used in
### checksum ### checksum
Contains `Checksum` type encapsulating checksum as well as it's kind. Contains `Checksum` type encapsulating checksum as well as it's kind.
Currently Sha256 and [Tillich-Zemor hashsum](https://github.com/TrueCloudLab/tzhash) are in use. Currently Sha256 and [Tillich-Zemor hashsum](https://git.frostfs.info/TrueCloudLab/tzhash) are in use.
### owner ### owner
`owner.ID` type represents single account interacting with FrostFS. In v2 version of protocol `owner.ID` type represents single account interacting with FrostFS. In v2 version of protocol
@ -27,7 +27,7 @@ Contains Bearer token type with several FrostFS-specific methods.
### ns ### ns
In FrostFS there are 2 types of name resolution: DNS and NNS. NNS stands for Neo Name Service In FrostFS there are 2 types of name resolution: DNS and NNS. NNS stands for Neo Name Service
is just a [contract](https://github.com/TrueCloudLab/frostfs-contract) deployed on a Neo blockchain. is just a [contract](https://git.frostfs.info/TrueCloudLab/frostfs-contract) deployed on a Neo blockchain.
Basically, NNS is just a DNS-on-chain which can be used for resolving container nice-names as well Basically, NNS is just a DNS-on-chain which can be used for resolving container nice-names as well
as any other name in dApps. See our [CoreDNS plugin](https://github.com/nspcc-dev/coredns/tree/master/plugin/nns) as any other name in dApps. See our [CoreDNS plugin](https://github.com/nspcc-dev/coredns/tree/master/plugin/nns)
for the example of how NNS can be integrated in DNS. for the example of how NNS can be integrated in DNS.
@ -42,7 +42,6 @@ Contains client for working with FrostFS.
```go ```go
var prmInit client.PrmInit var prmInit client.PrmInit
prmInit.SetDefaultPrivateKey(key) // private key for request signing prmInit.SetDefaultPrivateKey(key) // private key for request signing
prmInit.ResolveFrostFSFailures() // enable erroneous status parsing
var c client.Client var c client.Client
c.Init(prmInit) c.Init(prmInit)
@ -54,7 +53,7 @@ err := c.Dial(prmDial)
if err != nil { if err != nil {
return return
} }
ctx, cancel := context.WithTimeout(context.Background(), 5 * time.Second) ctx, cancel := context.WithTimeout(context.Background(), 5 * time.Second)
defer cancel() defer cancel()
@ -77,8 +76,7 @@ if needed and perform any desired action. In the case above we may want to repor
these details to the user as well as retry an operation, possibly with different parameters. these details to the user as well as retry an operation, possibly with different parameters.
Status wire-format is extendable and each node can report any set of details it wants. Status wire-format is extendable and each node can report any set of details it wants.
The set of reserved status codes can be found in The set of reserved status codes can be found in
[FrostFS API](https://github.com/TrueCloudLab/frostfs-api/blob/master/status/types.proto). There is also [FrostFS API](https://git.frostfs.info/TrueCloudLab/frostfs-api/src/branch/master/status/types.proto).
a `client.PrmInit.ResolveFrostFSFailures()` to seamlessly convert erroneous statuses into Go error type.
### policy ### policy
Contains helpers allowing conversion of placing policy from/to JSON representation Contains helpers allowing conversion of placing policy from/to JSON representation
@ -98,19 +96,19 @@ Contains CRUSH-like implementation of container node selection algorithm. Releva
are described in this paper http://ceur-ws.org/Vol-2344/short10.pdf . Note that it can be are described in this paper http://ceur-ws.org/Vol-2344/short10.pdf . Note that it can be
outdated in some details. outdated in some details.
`netmap/json_tests` subfolder contains language-agnostic tests for selection algorithm. `netmap/json_tests` subfolder contains language-agnostic tests for selection algorithm.
```go ```go
import ( import (
"github.com/TrueCloudLab/frostfs-sdk-go/netmap" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap"
"github.com/TrueCloudLab/frostfs-sdk-go/object" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
) )
func placementNodes(addr *object.Address, p *netmap.PlacementPolicy, frostfsNodes []netmap.NodeInfo) { func placementNodes(addr *object.Address, p *netmap.PlacementPolicy, frostfsNodes []netmap.NodeInfo) {
// Convert list of nodes in FrostFS API format to the intermediate representation. // Convert list of nodes in FrostFS API format to the intermediate representation.
nodes := netmap.NodesFromInfo(nodes) nodes := netmap.NodesFromInfo(nodes)
// Create new netmap (errors are skipped for the sake of clarity). // Create new netmap (errors are skipped for the sake of clarity).
nm, _ := NewNetmap(nodes) nm, _ := NewNetmap(nodes)
// Calculate nodes of container. // Calculate nodes of container.
@ -131,4 +129,4 @@ Contain simple API wrappers.
Wrapper over `zap.Logger` which is used across FrostFS codebase. Wrapper over `zap.Logger` which is used across FrostFS codebase.
### util ### util
Utilities for working with signature-related code. Utilities for working with signature-related code.

View file

@ -1,10 +1,10 @@
package accounting package accounting
import "github.com/TrueCloudLab/frostfs-api-go/v2/accounting" import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/accounting"
// Decimal represents decimal number for accounting operations. // Decimal represents decimal number for accounting operations.
// //
// Decimal is mutually compatible with github.com/TrueCloudLab/frostfs-api-go/v2/accounting.Decimal // Decimal is mutually compatible with git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/accounting.Decimal
// message. See ReadFromV2 / WriteToV2 methods. // message. See ReadFromV2 / WriteToV2 methods.
// //
// Instances can be created using built-in var declaration. // Instances can be created using built-in var declaration.

View file

@ -3,8 +3,8 @@ package accounting_test
import ( import (
"testing" "testing"
v2accounting "github.com/TrueCloudLab/frostfs-api-go/v2/accounting" v2accounting "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/accounting"
"github.com/TrueCloudLab/frostfs-sdk-go/accounting" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/accounting"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )

View file

@ -9,11 +9,11 @@ working with Fixed8 balance precision:
dec.SetPrecision(8) dec.SetPrecision(8)
Instances can be also used to process FrostFS API V2 protocol messages Instances can be also used to process FrostFS API V2 protocol messages
(see neo.fs.v2.accounting package in https://github.com/TrueCloudLab/frostfs-api). (see neo.fs.v2.accounting package in https://git.frostfs.info/TrueCloudLab/frostfs-api).
On client side: On client side:
import "github.com/TrueCloudLab/frostfs-api-go/v2/accounting" import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/accounting"
var msg accounting.Decimal var msg accounting.Decimal
dec.WriteToV2(&msg) dec.WriteToV2(&msg)

View file

@ -3,7 +3,7 @@ package accountingtest
import ( import (
"math/rand" "math/rand"
"github.com/TrueCloudLab/frostfs-sdk-go/accounting" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/accounting"
) )
// Decimal returns random accounting.Decimal. // Decimal returns random accounting.Decimal.

View file

@ -5,7 +5,7 @@ Note that importing the package into source files is highly discouraged.
Random instance generation functions can be useful when testing expects any value, e.g.: Random instance generation functions can be useful when testing expects any value, e.g.:
import accountingtest "github.com/TrueCloudLab/frostfs-sdk-go/accounting/test" import accountingtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/accounting/test"
dec := accountingtest.Decimal() dec := accountingtest.Decimal()
// test the value // test the value

52
ape/chain.go Normal file
View file

@ -0,0 +1,52 @@
package ape
import (
"errors"
"fmt"
apeV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/ape"
)
var (
ErrInvalidChainRepresentation = errors.New("invalid chain representation")
)
// ChainID is Chain's identifier.
type ChainID []byte
// Chain is an SDK representation for v2's Chain.
//
// Note that Chain (as well as v2's Chain) and all related entities
// are NOT operated by Access-Policy-Engine (APE). The client is responsible
// to convert these types to policy-engine entities.
type Chain struct {
// Raw is the encoded chain kind.
// It assumes that Raw's bytes are the result of encoding provided by
// policy-engine package.
Raw []byte
}
// ToV2 converts Chain to v2.
func (c *Chain) ToV2() *apeV2.Chain {
v2ct := new(apeV2.Chain)
if c.Raw != nil {
v2Raw := new(apeV2.ChainRaw)
v2Raw.SetRaw(c.Raw)
v2ct.SetKind(v2Raw)
}
return v2ct
}
// ReadFromV2 fills Chain from v2.
func (c *Chain) ReadFromV2(v2ct *apeV2.Chain) error {
switch v := v2ct.GetKind().(type) {
default:
return fmt.Errorf("unsupported chain kind: %T", v)
case *apeV2.ChainRaw:
raw := v.GetRaw()
c.Raw = raw
}
return nil
}

53
ape/chain_target.go Normal file
View file

@ -0,0 +1,53 @@
package ape
import (
apeV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/ape"
)
// TargetType is an SDK representation for v2's TargetType.
type TargetType apeV2.TargetType
const (
TargetTypeUndefined TargetType = iota
TargetTypeNamespace
TargetTypeContainer
TargetTypeUser
TargetTypeGroup
)
// ToV2 converts TargetType to v2.
func (targetType TargetType) ToV2() apeV2.TargetType {
return apeV2.TargetType(targetType)
}
// FromV2 reads TargetType to v2.
func (targetType *TargetType) FromV2(v2targetType apeV2.TargetType) {
*targetType = TargetType(v2targetType)
}
// ChainTarget is an SDK representation for v2's ChainTarget.
//
// Note that ChainTarget (as well as v2's ChainTarget) and all related entities
// are NOT operated by Access-Policy-Engine (APE). The client is responsible
// to convert these types to policy-engine entities.
type ChainTarget struct {
TargetType TargetType
Name string
}
// ToV2 converts ChainTarget to v2.
func (ct *ChainTarget) ToV2() *apeV2.ChainTarget {
v2ct := new(apeV2.ChainTarget)
v2ct.SetTargetType(ct.TargetType.ToV2())
v2ct.SetName(ct.Name)
return v2ct
}
// FromV2 reads ChainTarget frpm v2.
func (ct *ChainTarget) FromV2(v2ct *apeV2.ChainTarget) {
ct.TargetType.FromV2(v2ct.GetTargetType())
ct.Name = v2ct.GetName()
}

65
ape/chain_target_test.go Normal file
View file

@ -0,0 +1,65 @@
package ape_test
import (
"testing"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
"github.com/stretchr/testify/require"
apeV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/ape"
)
var (
m = map[ape.TargetType]apeV2.TargetType{
ape.TargetTypeUndefined: apeV2.TargetTypeUndefined,
ape.TargetTypeNamespace: apeV2.TargetTypeNamespace,
ape.TargetTypeContainer: apeV2.TargetTypeContainer,
ape.TargetTypeUser: apeV2.TargetTypeUser,
ape.TargetTypeGroup: apeV2.TargetTypeGroup,
}
)
func TestTargetType(t *testing.T) {
for typesdk, typev2 := range m {
t.Run("from sdk to v2 "+typev2.String(), func(t *testing.T) {
v2 := typesdk.ToV2()
require.Equal(t, v2, typev2)
})
t.Run("from v2 to sdk "+typev2.String(), func(t *testing.T) {
var typ ape.TargetType
typ.FromV2(typev2)
require.Equal(t, typesdk, typ)
})
}
}
func TestChainTarget(t *testing.T) {
var (
typ = ape.TargetTypeNamespace
name = "namespaceXXYYZZ"
)
t.Run("from sdk to v2", func(t *testing.T) {
ct := ape.ChainTarget{
TargetType: typ,
Name: name,
}
v2 := ct.ToV2()
require.Equal(t, m[typ], v2.GetTargetType())
require.Equal(t, name, v2.GetName())
})
t.Run("from v2 to sdk", func(t *testing.T) {
v2 := &apeV2.ChainTarget{}
v2.SetTargetType(m[typ])
v2.SetName(name)
var ct ape.ChainTarget
ct.FromV2(v2)
require.Equal(t, typ, ct.TargetType)
require.Equal(t, name, ct.Name)
})
}

43
ape/chain_test.go Normal file
View file

@ -0,0 +1,43 @@
package ape_test
import (
"testing"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
"github.com/stretchr/testify/require"
apeV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/ape"
)
const (
encoded = `{"ID":"","Rules":[{"Status":"Allow","Actions":{"Inverted":false,"Names":["GetObject"]},"Resources":{"Inverted":false,"Names":["native:object/*"]},"Any":false,"Condition":[{"Op":"StringEquals","Object":"Resource","Key":"Department","Value":"HR"}]}],"MatchType":"DenyPriority"}`
)
func TestChainData(t *testing.T) {
t.Run("raw chain", func(t *testing.T) {
var c ape.Chain
b := []byte(encoded)
c.Raw = b
v2, ok := c.ToV2().GetKind().(*apeV2.ChainRaw)
require.True(t, ok)
require.Equal(t, b, v2.Raw)
})
}
func TestChainMessageV2(t *testing.T) {
b := []byte(encoded)
v2Raw := &apeV2.ChainRaw{}
v2Raw.SetRaw(b)
v2 := &apeV2.Chain{}
v2.SetKind(v2Raw)
var c ape.Chain
c.ReadFromV2(v2)
require.NotNil(t, c.Raw)
require.Equal(t, b, c.Raw)
}

View file

@ -1,26 +0,0 @@
/*
Package audit provides features to process data audit in FrostFS system.
Result type groups values which can be gathered during data audit process:
var res audit.Result
res.ForEpoch(32)
res.ForContainer(cnr)
// ...
res.Complete()
Result instances can be stored in a binary format. On reporter side:
data := res.Marshal()
// send data
On receiver side:
var res audit.Result
err := res.Unmarshal(data)
// ...
Using package types in an application is recommended to potentially work with
different protocol versions with which these types are compatible.
*/
package audit

View file

@ -1,377 +0,0 @@
package audit
import (
"errors"
"fmt"
"github.com/TrueCloudLab/frostfs-api-go/v2/audit"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs"
cid "github.com/TrueCloudLab/frostfs-sdk-go/container/id"
oid "github.com/TrueCloudLab/frostfs-sdk-go/object/id"
"github.com/TrueCloudLab/frostfs-sdk-go/version"
)
// Result represents report on the results of the data audit in FrostFS system.
//
// Result is mutually binary-compatible with github.com/TrueCloudLab/frostfs-api-go/v2/audit.DataAuditResult
// message. See Marshal / Unmarshal methods.
//
// Instances can be created using built-in var declaration.
type Result struct {
versionEncoded bool
v2 audit.DataAuditResult
}
// Marshal encodes Result into a canonical FrostFS binary format (Protocol Buffers
// with direct field order).
//
// Writes version.Current() protocol version into the resulting message if Result
// hasn't been already decoded from such a message using Unmarshal.
//
// See also Unmarshal.
func (r *Result) Marshal() []byte {
if !r.versionEncoded {
var verV2 refs.Version
version.Current().WriteToV2(&verV2)
r.v2.SetVersion(&verV2)
r.versionEncoded = true
}
return r.v2.StableMarshal(nil)
}
var errCIDNotSet = errors.New("container ID is not set")
// Unmarshal decodes Result from its canonical FrostFS binary format (Protocol Buffers
// with direct field order). Returns an error describing a format violation.
//
// See also Marshal.
func (r *Result) Unmarshal(data []byte) error {
err := r.v2.Unmarshal(data)
if err != nil {
return err
}
r.versionEncoded = true
// format checks
var cID cid.ID
cidV2 := r.v2.GetContainerID()
if cidV2 == nil {
return errCIDNotSet
}
err = cID.ReadFromV2(*cidV2)
if err != nil {
return fmt.Errorf("could not convert V2 container ID: %w", err)
}
var (
oID oid.ID
oidV2 refs.ObjectID
)
for _, oidV2 = range r.v2.GetPassSG() {
err = oID.ReadFromV2(oidV2)
if err != nil {
return fmt.Errorf("invalid passed storage group ID: %w", err)
}
}
for _, oidV2 = range r.v2.GetFailSG() {
err = oID.ReadFromV2(oidV2)
if err != nil {
return fmt.Errorf("invalid failed storage group ID: %w", err)
}
}
return nil
}
// Epoch returns FrostFS epoch when the data associated with the Result was audited.
//
// Zero Result has zero epoch.
//
// See also ForEpoch.
func (r Result) Epoch() uint64 {
return r.v2.GetAuditEpoch()
}
// ForEpoch specifies FrostFS epoch when the data associated with the Result was audited.
//
// See also Epoch.
func (r *Result) ForEpoch(epoch uint64) {
r.v2.SetAuditEpoch(epoch)
}
// Container returns identifier of the container with which the data audit Result
// is associated and a bool that indicates container ID field presence in the Result.
//
// Zero Result does not have container ID.
//
// See also ForContainer.
func (r Result) Container() (cid.ID, bool) {
var cID cid.ID
cidV2 := r.v2.GetContainerID()
if cidV2 != nil {
_ = cID.ReadFromV2(*cidV2)
return cID, true
}
return cID, false
}
// ForContainer sets identifier of the container with which the data audit Result
// is associated.
//
// See also Container.
func (r *Result) ForContainer(cnr cid.ID) {
var cidV2 refs.ContainerID
cnr.WriteToV2(&cidV2)
r.v2.SetContainerID(&cidV2)
}
// AuditorKey returns public key of the auditing FrostFS Inner Ring node in
// a FrostFS binary key format.
//
// Zero Result has nil key. Return value MUST NOT be mutated: to do this,
// first make a copy.
//
// See also SetAuditorPublicKey.
func (r Result) AuditorKey() []byte {
return r.v2.GetPublicKey()
}
// SetAuditorKey specifies public key of the auditing FrostFS Inner Ring node in
// a FrostFS binary key format.
//
// Argument MUST NOT be mutated at least until the end of using the Result.
//
// See also AuditorKey.
func (r *Result) SetAuditorKey(key []byte) {
r.v2.SetPublicKey(key)
}
// Completed returns completion state of the data audit associated with the Result.
//
// Zero Result corresponds to incomplete data audit.
//
// See also Complete.
func (r Result) Completed() bool {
return r.v2.GetComplete()
}
// Complete marks the data audit associated with the Result as completed.
//
// See also Completed.
func (r *Result) Complete() {
r.v2.SetComplete(true)
}
// RequestsPoR returns number of requests made by Proof-of-Retrievability
// audit check to get all headers of the objects inside storage groups.
//
// Zero Result has zero requests.
//
// See also SetRequestsPoR.
func (r Result) RequestsPoR() uint32 {
return r.v2.GetRequests()
}
// SetRequestsPoR sets number of requests made by Proof-of-Retrievability
// audit check to get all headers of the objects inside storage groups.
//
// See also RequestsPoR.
func (r *Result) SetRequestsPoR(v uint32) {
r.v2.SetRequests(v)
}
// RetriesPoR returns number of retries made by Proof-of-Retrievability
// audit check to get all headers of the objects inside storage groups.
//
// Zero Result has zero retries.
//
// See also SetRetriesPoR.
func (r Result) RetriesPoR() uint32 {
return r.v2.GetRetries()
}
// SetRetriesPoR sets number of retries made by Proof-of-Retrievability
// audit check to get all headers of the objects inside storage groups.
//
// See also RetriesPoR.
func (r *Result) SetRetriesPoR(v uint32) {
r.v2.SetRetries(v)
}
// IteratePassedStorageGroups iterates over all storage groups that passed
// Proof-of-Retrievability audit check and passes them into f. Breaks on f's
// false return, f MUST NOT be nil.
//
// Zero Result has no passed storage groups and doesn't call f.
//
// See also SubmitPassedStorageGroup.
func (r Result) IteratePassedStorageGroups(f func(oid.ID) bool) {
r2 := r.v2.GetPassSG()
var id oid.ID
for i := range r2 {
_ = id.ReadFromV2(r2[i])
if !f(id) {
return
}
}
}
// SubmitPassedStorageGroup marks storage group as passed Proof-of-Retrievability
// audit check.
//
// See also IteratePassedStorageGroups.
func (r *Result) SubmitPassedStorageGroup(sg oid.ID) {
var idV2 refs.ObjectID
sg.WriteToV2(&idV2)
r.v2.SetPassSG(append(r.v2.GetPassSG(), idV2))
}
// IterateFailedStorageGroups is similar to IteratePassedStorageGroups but for failed groups.
//
// See also SubmitFailedStorageGroup.
func (r Result) IterateFailedStorageGroups(f func(oid.ID) bool) {
v := r.v2.GetFailSG()
var id oid.ID
for i := range v {
_ = id.ReadFromV2(v[i])
if !f(id) {
return
}
}
}
// SubmitFailedStorageGroup is similar to SubmitPassedStorageGroup but for failed groups.
//
// See also IterateFailedStorageGroups.
func (r *Result) SubmitFailedStorageGroup(sg oid.ID) {
var idV2 refs.ObjectID
sg.WriteToV2(&idV2)
r.v2.SetFailSG(append(r.v2.GetFailSG(), idV2))
}
// Hits returns number of sampled objects under audit placed
// in an optimal way according to the container's placement policy
// when checking Proof-of-Placement.
//
// Zero result has zero hits.
//
// See also SetHits.
func (r Result) Hits() uint32 {
return r.v2.GetHit()
}
// SetHits sets number of sampled objects under audit placed
// in an optimal way according to the containers placement policy
// when checking Proof-of-Placement.
//
// See also Hits.
func (r *Result) SetHits(hit uint32) {
r.v2.SetHit(hit)
}
// Misses returns number of sampled objects under audit placed
// in suboptimal way according to the container's placement policy,
// but still at a satisfactory level when checking Proof-of-Placement.
//
// Zero Result has zero misses.
//
// See also SetMisses.
func (r Result) Misses() uint32 {
return r.v2.GetMiss()
}
// SetMisses sets number of sampled objects under audit placed
// in suboptimal way according to the container's placement policy,
// but still at a satisfactory level when checking Proof-of-Placement.
//
// See also Misses.
func (r *Result) SetMisses(miss uint32) {
r.v2.SetMiss(miss)
}
// Failures returns number of sampled objects under audit stored
// in a way not confirming placement policy or not found at all
// when checking Proof-of-Placement.
//
// Zero result has zero failures.
//
// See also SetFailures.
func (r Result) Failures() uint32 {
return r.v2.GetFail()
}
// SetFailures sets number of sampled objects under audit stored
// in a way not confirming placement policy or not found at all
// when checking Proof-of-Placement.
//
// See also Failures.
func (r *Result) SetFailures(fail uint32) {
r.v2.SetFail(fail)
}
// IteratePassedStorageNodes iterates over all storage nodes that passed at least one
// Proof-of-Data-Possession audit check and passes their public keys into f. Breaks on
// f's false return.
//
// f MUST NOT be nil and MUST NOT mutate parameter passed into it at least until
// the end of using the Result.
//
// Zero Result has no passed storage nodes and doesn't call f.
//
// See also SubmitPassedStorageNode.
func (r Result) IteratePassedStorageNodes(f func([]byte) bool) {
v := r.v2.GetPassNodes()
for i := range v {
if !f(v[i]) {
return
}
}
}
// SubmitPassedStorageNodes marks storage node list as passed Proof-of-Data-Possession
// audit check. The list contains public keys.
//
// Argument and its elements MUST NOT be mutated at least until the end of using the Result.
//
// See also IteratePassedStorageNodes.
func (r *Result) SubmitPassedStorageNodes(list [][]byte) {
r.v2.SetPassNodes(list)
}
// IterateFailedStorageNodes is similar to IteratePassedStorageNodes but for failed nodes.
//
// See also SubmitPassedStorageNodes.
func (r Result) IterateFailedStorageNodes(f func([]byte) bool) {
v := r.v2.GetFailNodes()
for i := range v {
if !f(v[i]) {
return
}
}
}
// SubmitFailedStorageNodes is similar to SubmitPassedStorageNodes but for failed nodes.
//
// See also IterateFailedStorageNodes.
func (r *Result) SubmitFailedStorageNodes(list [][]byte) {
r.v2.SetFailNodes(list)
}

View file

@ -1,191 +0,0 @@
package audit_test
import (
"bytes"
"testing"
"github.com/TrueCloudLab/frostfs-sdk-go/audit"
audittest "github.com/TrueCloudLab/frostfs-sdk-go/audit/test"
cidtest "github.com/TrueCloudLab/frostfs-sdk-go/container/id/test"
oid "github.com/TrueCloudLab/frostfs-sdk-go/object/id"
oidtest "github.com/TrueCloudLab/frostfs-sdk-go/object/id/test"
"github.com/stretchr/testify/require"
)
func TestResultData(t *testing.T) {
var r audit.Result
countSG := func(passed bool, f func(oid.ID)) int {
called := 0
ff := func(arg oid.ID) bool {
called++
if f != nil {
f(arg)
}
return true
}
if passed {
r.IteratePassedStorageGroups(ff)
} else {
r.IterateFailedStorageGroups(ff)
}
return called
}
countPassSG := func(f func(oid.ID)) int { return countSG(true, f) }
countFailSG := func(f func(oid.ID)) int { return countSG(false, f) }
countNodes := func(passed bool, f func([]byte)) int {
called := 0
ff := func(arg []byte) bool {
called++
if f != nil {
f(arg)
}
return true
}
if passed {
r.IteratePassedStorageNodes(ff)
} else {
r.IterateFailedStorageNodes(ff)
}
return called
}
countPassNodes := func(f func([]byte)) int { return countNodes(true, f) }
countFailNodes := func(f func([]byte)) int { return countNodes(false, f) }
require.Zero(t, r.Epoch())
_, set := r.Container()
require.False(t, set)
require.Nil(t, r.AuditorKey())
require.False(t, r.Completed())
require.Zero(t, r.RequestsPoR())
require.Zero(t, r.RetriesPoR())
require.Zero(t, countPassSG(nil))
require.Zero(t, countFailSG(nil))
require.Zero(t, countPassNodes(nil))
require.Zero(t, countFailNodes(nil))
epoch := uint64(13)
r.ForEpoch(epoch)
require.Equal(t, epoch, r.Epoch())
cnr := cidtest.ID()
r.ForContainer(cnr)
cID, set := r.Container()
require.True(t, set)
require.Equal(t, cnr, cID)
key := []byte{1, 2, 3}
r.SetAuditorKey(key)
require.Equal(t, key, r.AuditorKey())
r.Complete()
require.True(t, r.Completed())
requests := uint32(2)
r.SetRequestsPoR(requests)
require.Equal(t, requests, r.RequestsPoR())
retries := uint32(1)
r.SetRetriesPoR(retries)
require.Equal(t, retries, r.RetriesPoR())
passSG1, passSG2 := oidtest.ID(), oidtest.ID()
r.SubmitPassedStorageGroup(passSG1)
r.SubmitPassedStorageGroup(passSG2)
called1, called2 := false, false
require.EqualValues(t, 2, countPassSG(func(id oid.ID) {
if id.Equals(passSG1) {
called1 = true
} else if id.Equals(passSG2) {
called2 = true
}
}))
require.True(t, called1)
require.True(t, called2)
failSG1, failSG2 := oidtest.ID(), oidtest.ID()
r.SubmitFailedStorageGroup(failSG1)
r.SubmitFailedStorageGroup(failSG2)
called1, called2 = false, false
require.EqualValues(t, 2, countFailSG(func(id oid.ID) {
if id.Equals(failSG1) {
called1 = true
} else if id.Equals(failSG2) {
called2 = true
}
}))
require.True(t, called1)
require.True(t, called2)
hit := uint32(1)
r.SetHits(hit)
require.Equal(t, hit, r.Hits())
miss := uint32(2)
r.SetMisses(miss)
require.Equal(t, miss, r.Misses())
fail := uint32(3)
r.SetFailures(fail)
require.Equal(t, fail, r.Failures())
passNodes := [][]byte{{1}, {2}}
r.SubmitPassedStorageNodes(passNodes)
called1, called2 = false, false
require.EqualValues(t, 2, countPassNodes(func(arg []byte) {
if bytes.Equal(arg, passNodes[0]) {
called1 = true
} else if bytes.Equal(arg, passNodes[1]) {
called2 = true
}
}))
require.True(t, called1)
require.True(t, called2)
failNodes := [][]byte{{3}, {4}}
r.SubmitFailedStorageNodes(failNodes)
called1, called2 = false, false
require.EqualValues(t, 2, countFailNodes(func(arg []byte) {
if bytes.Equal(arg, failNodes[0]) {
called1 = true
} else if bytes.Equal(arg, failNodes[1]) {
called2 = true
}
}))
require.True(t, called1)
require.True(t, called2)
}
func TestResultEncoding(t *testing.T) {
r := *audittest.Result()
t.Run("binary", func(t *testing.T) {
data := r.Marshal()
var r2 audit.Result
require.NoError(t, r2.Unmarshal(data))
require.Equal(t, r, r2)
})
}

View file

@ -1,13 +0,0 @@
/*
Package audittest provides functions for convenient testing of audit package API.
Note that importing the package into source files is highly discouraged.
Random instance generation functions can be useful when testing expects any value, e.g.:
import audittest "github.com/TrueCloudLab/frostfs-sdk-go/audit/test"
dec := audittest.Result()
// test the value
*/
package audittest

View file

@ -1,36 +0,0 @@
package audittest
import (
"github.com/TrueCloudLab/frostfs-sdk-go/audit"
cidtest "github.com/TrueCloudLab/frostfs-sdk-go/container/id/test"
oidtest "github.com/TrueCloudLab/frostfs-sdk-go/object/id/test"
)
// Result returns random audit.Result.
func Result() *audit.Result {
var x audit.Result
x.ForContainer(cidtest.ID())
x.SetAuditorKey([]byte("key"))
x.Complete()
x.ForEpoch(44)
x.SetHits(55)
x.SetMisses(66)
x.SetFailures(77)
x.SetRequestsPoR(88)
x.SetRequestsPoR(99)
x.SubmitFailedStorageNodes([][]byte{
[]byte("node1"),
[]byte("node2"),
})
x.SubmitPassedStorageNodes([][]byte{
[]byte("node3"),
[]byte("node4"),
})
x.SubmitPassedStorageGroup(oidtest.ID())
x.SubmitPassedStorageGroup(oidtest.ID())
x.SubmitFailedStorageGroup(oidtest.ID())
x.SubmitFailedStorageGroup(oidtest.ID())
return &x
}

View file

@ -5,18 +5,20 @@ import (
"errors" "errors"
"fmt" "fmt"
"github.com/TrueCloudLab/frostfs-api-go/v2/acl" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs" apeV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/ape"
cid "github.com/TrueCloudLab/frostfs-sdk-go/container/id" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
frostfscrypto "github.com/TrueCloudLab/frostfs-sdk-go/crypto" apeSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
frostfsecdsa "github.com/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa" cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"github.com/TrueCloudLab/frostfs-sdk-go/eacl" frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
"github.com/TrueCloudLab/frostfs-sdk-go/user" frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
) )
// Token represents bearer token for object service operations. // Token represents bearer token for object service operations.
// //
// Token is mutually compatible with github.com/TrueCloudLab/frostfs-api-go/v2/acl.BearerToken // Token is mutually compatible with git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl.BearerToken
// message. See ReadFromV2 / WriteToV2 methods. // message. See ReadFromV2 / WriteToV2 methods.
// //
// Instances can be created using built-in var declaration. // Instances can be created using built-in var declaration.
@ -32,6 +34,84 @@ type Token struct {
sigSet bool sigSet bool
sig refs.Signature sig refs.Signature
apeOverrideSet bool
apeOverride APEOverride
impersonate bool
}
// APEOverride is the list of APE chains defined for a target.
// These chains are meant to serve as overrides to the already defined (or even undefined)
// APE chains for the target (see contract `Policy`).
//
// The server-side processing of the bearer token with set APE overrides must verify if a client is permitted
// to override chains for the target, preventing unauthorized access through the APE mechanism.
type APEOverride struct {
// Target for which chains are applied.
Target apeSDK.ChainTarget
// The list of APE chains.
Chains []apeSDK.Chain
}
// Marshal marshals APEOverride into a protobuf binary form.
func (c *APEOverride) Marshal() ([]byte, error) {
return c.ToV2().StableMarshal(nil), nil
}
// Unmarshal unmarshals protobuf binary representation of APEOverride.
func (c *APEOverride) Unmarshal(data []byte) error {
overrideV2 := new(acl.APEOverride)
if err := overrideV2.Unmarshal(data); err != nil {
return err
}
return c.FromV2(overrideV2)
}
// MarshalJSON encodes APEOverride to protobuf JSON format.
func (c *APEOverride) MarshalJSON() ([]byte, error) {
return c.ToV2().MarshalJSON()
}
// UnmarshalJSON decodes APEOverride from protobuf JSON format.
func (c *APEOverride) UnmarshalJSON(data []byte) error {
overrideV2 := new(acl.APEOverride)
if err := overrideV2.UnmarshalJSON(data); err != nil {
return err
}
return c.FromV2(overrideV2)
}
func (c *APEOverride) FromV2(tokenAPEChains *acl.APEOverride) error {
c.Target.FromV2(tokenAPEChains.GetTarget())
if chains := tokenAPEChains.GetChains(); len(chains) > 0 {
c.Chains = make([]apeSDK.Chain, len(chains))
for i := range chains {
if err := c.Chains[i].ReadFromV2(chains[i]); err != nil {
return fmt.Errorf("invalid APE chain: %w", err)
}
}
}
return nil
}
func (c *APEOverride) ToV2() *acl.APEOverride {
if c == nil {
return nil
}
apeOverride := new(acl.APEOverride)
apeOverride.SetTarget(c.Target.ToV2())
chains := make([]*apeV2.Chain, len(c.Chains))
for i := range c.Chains {
chains[i] = c.Chains[i].ToV2()
}
apeOverride.SetChains(chains)
return apeOverride
} }
// reads Token from the acl.BearerToken message. If checkFieldPresence is set, // reads Token from the acl.BearerToken message. If checkFieldPresence is set,
@ -44,10 +124,13 @@ func (b *Token) readFromV2(m acl.BearerToken, checkFieldPresence bool) error {
return errors.New("missing token body") return errors.New("missing token body")
} }
b.impersonate = body.GetImpersonate()
apeOverrides := body.GetAPEOverride()
eaclTable := body.GetEACL() eaclTable := body.GetEACL()
if b.eaclTableSet = eaclTable != nil; b.eaclTableSet { if b.eaclTableSet = eaclTable != nil; b.eaclTableSet {
b.eaclTable = *eacl.NewTableFromV2(eaclTable) b.eaclTable = *eacl.NewTableFromV2(eaclTable)
} else if checkFieldPresence { } else if checkFieldPresence && !b.impersonate && apeOverrides == nil {
return errors.New("missing eACL table") return errors.New("missing eACL table")
} }
@ -68,6 +151,14 @@ func (b *Token) readFromV2(m acl.BearerToken, checkFieldPresence bool) error {
return errors.New("missing token lifetime") return errors.New("missing token lifetime")
} }
if b.apeOverrideSet = apeOverrides != nil; b.apeOverrideSet {
if err = b.apeOverride.FromV2(apeOverrides); err != nil {
return err
}
} else if checkFieldPresence && !b.impersonate && !b.eaclTableSet {
return errors.New("missing APE override")
}
sig := m.GetSignature() sig := m.GetSignature()
if b.sigSet = sig != nil; sig != nil { if b.sigSet = sig != nil; sig != nil {
b.sig = *sig b.sig = *sig
@ -86,7 +177,7 @@ func (b *Token) ReadFromV2(m acl.BearerToken) error {
} }
func (b Token) fillBody() *acl.BearerTokenBody { func (b Token) fillBody() *acl.BearerTokenBody {
if !b.eaclTableSet && !b.targetUserSet && !b.lifetimeSet { if !b.eaclTableSet && !b.targetUserSet && !b.lifetimeSet && !b.impersonate && !b.apeOverrideSet {
return nil return nil
} }
@ -112,6 +203,12 @@ func (b Token) fillBody() *acl.BearerTokenBody {
body.SetLifetime(&lifetime) body.SetLifetime(&lifetime)
} }
if b.apeOverrideSet {
body.SetAPEOverride(b.apeOverride.ToV2())
}
body.SetImpersonate(b.impersonate)
return &body return &body
} }
@ -208,6 +305,36 @@ func (b Token) EACLTable() eacl.Table {
return eacl.Table{} return eacl.Table{}
} }
// SetAPEOverride sets APE override to the bearer token.
//
// See also: APEOverride.
func (b *Token) SetAPEOverride(v APEOverride) {
b.apeOverride = v
b.apeOverrideSet = true
}
// APEOverride returns APE override set by SetAPEOverride.
//
// Zero Token has zero APEOverride.
func (b *Token) APEOverride() APEOverride {
if b.apeOverrideSet {
return b.apeOverride
}
return APEOverride{}
}
// SetImpersonate mark token as impersonate to consider token signer as request owner.
// If this field is true extended EACLTable in token body isn't processed.
func (b *Token) SetImpersonate(v bool) {
b.impersonate = v
}
// Impersonate returns true if token is impersonated.
func (b Token) Impersonate() bool {
return b.impersonate
}
// AssertContainer checks if the token is valid within the given container. // AssertContainer checks if the token is valid within the given container.
// //
// Note: cnr is assumed to refer to the issuer's container, otherwise the check // Note: cnr is assumed to refer to the issuer's container, otherwise the check

View file

@ -3,19 +3,20 @@ package bearer_test
import ( import (
"bytes" "bytes"
"math/rand" "math/rand"
"reflect"
"testing" "testing"
"github.com/TrueCloudLab/frostfs-api-go/v2/acl" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
"github.com/TrueCloudLab/frostfs-sdk-go/bearer" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
bearertest "github.com/TrueCloudLab/frostfs-sdk-go/bearer/test" bearertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer/test"
cidtest "github.com/TrueCloudLab/frostfs-sdk-go/container/id/test" cidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id/test"
frostfscrypto "github.com/TrueCloudLab/frostfs-sdk-go/crypto" frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
frostfsecdsa "github.com/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa" frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
"github.com/TrueCloudLab/frostfs-sdk-go/eacl" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl"
eacltest "github.com/TrueCloudLab/frostfs-sdk-go/eacl/test" eacltest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl/test"
"github.com/TrueCloudLab/frostfs-sdk-go/user" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
usertest "github.com/TrueCloudLab/frostfs-sdk-go/user/test" usertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user/test"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
@ -81,6 +82,58 @@ func TestToken_SetEACLTable(t *testing.T) {
require.True(t, isEqualEACLTables(eaclTable, val.EACLTable())) require.True(t, isEqualEACLTables(eaclTable, val.EACLTable()))
} }
func TestToken_SetAPEOverrides(t *testing.T) {
var val bearer.Token
var m acl.BearerToken
filled := bearertest.Token()
val.WriteToV2(&m)
require.Zero(t, m.GetBody())
val2 := filled
require.NoError(t, val2.Unmarshal(val.Marshal()))
require.Zero(t, val2.APEOverride())
val2 = filled
jd, err := val.MarshalJSON()
require.NoError(t, err)
require.NoError(t, val2.UnmarshalJSON(jd))
require.Zero(t, val2.APEOverride())
// set value
tApe := bearertest.APEOverride()
val.SetAPEOverride(tApe)
require.Equal(t, tApe, val.APEOverride())
val.WriteToV2(&m)
require.NotNil(t, m.GetBody().GetAPEOverride())
require.True(t, tokenAPEOverridesEqual(tApe.ToV2(), m.GetBody().GetAPEOverride()))
val2 = filled
require.NoError(t, val2.Unmarshal(val.Marshal()))
apeOverride := val2.APEOverride()
require.True(t, tokenAPEOverridesEqual(tApe.ToV2(), apeOverride.ToV2()))
val2 = filled
jd, err = val.MarshalJSON()
require.NoError(t, err)
require.NoError(t, val2.UnmarshalJSON(jd))
apeOverride = val.APEOverride()
require.True(t, tokenAPEOverridesEqual(tApe.ToV2(), apeOverride.ToV2()))
}
func tokenAPEOverridesEqual(lhs, rhs *acl.APEOverride) bool {
return reflect.DeepEqual(lhs, rhs)
}
func TestToken_ForUser(t *testing.T) { func TestToken_ForUser(t *testing.T) {
var val bearer.Token var val bearer.Token
var m acl.BearerToken var m acl.BearerToken
@ -107,7 +160,7 @@ func TestToken_ForUser(t *testing.T) {
require.Zero(t, m.GetBody()) require.Zero(t, m.GetBody())
// set value // set value
usr := *usertest.ID() usr := usertest.ID()
var usrV2 refs.OwnerID var usrV2 refs.OwnerID
usr.WriteToV2(&usrV2) usr.WriteToV2(&usrV2)
@ -243,11 +296,11 @@ func TestToken_AssertContainer(t *testing.T) {
func TestToken_AssertUser(t *testing.T) { func TestToken_AssertUser(t *testing.T) {
var val bearer.Token var val bearer.Token
usr := *usertest.ID() usr := usertest.ID()
require.True(t, val.AssertUser(usr)) require.True(t, val.AssertUser(usr))
val.ForUser(*usertest.ID()) val.ForUser(usertest.ID())
require.False(t, val.AssertUser(usr)) require.False(t, val.AssertUser(usr))
val.ForUser(usr) val.ForUser(usr)
@ -323,12 +376,16 @@ func TestToken_ReadFromV2(t *testing.T) {
require.NoError(t, val.ReadFromV2(m)) require.NoError(t, val.ReadFromV2(m))
body.SetEACL(nil)
body.SetImpersonate(true)
require.NoError(t, val.ReadFromV2(m))
var m2 acl.BearerToken var m2 acl.BearerToken
val.WriteToV2(&m2) val.WriteToV2(&m2)
require.Equal(t, m, m2) require.Equal(t, m, m2)
usr, usr2 := *usertest.ID(), *usertest.ID() usr, usr2 := usertest.ID(), usertest.ID()
require.True(t, val.AssertUser(usr)) require.True(t, val.AssertUser(usr))
require.True(t, val.AssertUser(usr2)) require.True(t, val.AssertUser(usr2))

View file

@ -22,7 +22,7 @@ Bearer token must be signed by owner of the container.
Provide signed token in JSON or binary format to the request sender. Request Provide signed token in JSON or binary format to the request sender. Request
sender can attach this bearer token to the object service requests: sender can attach this bearer token to the object service requests:
import sdkClient "github.com/TrueCloudLab/frostfs-sdk-go/client" import sdkClient "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client"
var headParams sdkClient.PrmObjectHead var headParams sdkClient.PrmObjectHead
headParams.WithBearerToken(bearerToken) headParams.WithBearerToken(bearerToken)

View file

@ -1,9 +1,10 @@
package bearertest package bearertest
import ( import (
"github.com/TrueCloudLab/frostfs-sdk-go/bearer" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
eacltest "github.com/TrueCloudLab/frostfs-sdk-go/eacl/test" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
usertest "github.com/TrueCloudLab/frostfs-sdk-go/user/test" eacltest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl/test"
usertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user/test"
) )
// Token returns random bearer.Token. // Token returns random bearer.Token.
@ -13,8 +14,19 @@ func Token() (t bearer.Token) {
t.SetExp(3) t.SetExp(3)
t.SetNbf(2) t.SetNbf(2)
t.SetIat(1) t.SetIat(1)
t.ForUser(*usertest.ID()) t.ForUser(usertest.ID())
t.SetEACLTable(*eacltest.Table()) t.SetEACLTable(*eacltest.Table())
t.SetAPEOverride(APEOverride())
return t return t
} }
func APEOverride() bearer.APEOverride {
return bearer.APEOverride{
Target: ape.ChainTarget{
TargetType: ape.TargetTypeContainer,
Name: "F8JsMnChywiPvbDvpxMbjTjx5KhWHHp6gCDt8BhzL9kF",
},
Chains: []ape.Chain{{Raw: []byte("{}")}},
}
}

View file

@ -6,13 +6,13 @@ import (
"errors" "errors"
"fmt" "fmt"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
"github.com/TrueCloudLab/tzhash/tz" "git.frostfs.info/TrueCloudLab/tzhash/tz"
) )
// Checksum represents checksum of some digital data. // Checksum represents checksum of some digital data.
// //
// Checksum is mutually compatible with github.com/TrueCloudLab/frostfs-api-go/v2/refs.Checksum // Checksum is mutually compatible with git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs.Checksum
// message. See ReadFromV2 / WriteToV2 methods. // message. See ReadFromV2 / WriteToV2 methods.
// //
// Instances can be created using built-in var declaration. // Instances can be created using built-in var declaration.
@ -24,17 +24,17 @@ type Checksum refs.Checksum
// Type represents the enumeration // Type represents the enumeration
// of checksum types. // of checksum types.
type Type uint8 type Type refs.ChecksumType
const ( const (
// Unknown is an undefined checksum type. // Unknown is an undefined checksum type.
Unknown Type = iota Unknown Type = Type(refs.UnknownChecksum)
// SHA256 is a SHA256 checksum type. // SHA256 is a SHA256 checksum type.
SHA256 SHA256 = Type(refs.SHA256)
// TZ is a Tillich-Zémor checksum type. // TZ is a Tillich-Zémor checksum type.
TZ TZ = Type(refs.TillichZemor)
) )
// ReadFromV2 reads Checksum from the refs.Checksum message. Checks if the // ReadFromV2 reads Checksum from the refs.Checksum message. Checks if the

View file

@ -5,8 +5,8 @@ import (
"crypto/sha256" "crypto/sha256"
"testing" "testing"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
"github.com/TrueCloudLab/tzhash/tz" "git.frostfs.info/TrueCloudLab/tzhash/tz"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )

View file

@ -2,11 +2,11 @@ package checksum
import ( import (
"bytes" "bytes"
"crypto/rand"
"crypto/sha256" "crypto/sha256"
"fmt" "fmt"
"math/rand"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
) )
func ExampleCalculate() { func ExampleCalculate() {

View file

@ -5,7 +5,7 @@ Note that importing the package into source files is highly discouraged.
Random instance generation functions can be useful when testing expects any value, e.g.: Random instance generation functions can be useful when testing expects any value, e.g.:
import checksumtest "github.com/TrueCloudLab/frostfs-sdk-go/checksum/test" import checksumtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/checksum/test"
cs := checksumtest.Checksum() cs := checksumtest.Checksum()
// test the value // test the value

View file

@ -1,17 +1,17 @@
package checksumtest package checksumtest
import ( import (
"crypto/rand"
"crypto/sha256" "crypto/sha256"
"math/rand"
"github.com/TrueCloudLab/frostfs-sdk-go/checksum" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/checksum"
) )
// Checksum returns random checksum.Checksum. // Checksum returns random checksum.Checksum.
func Checksum() checksum.Checksum { func Checksum() checksum.Checksum {
var cs [sha256.Size]byte var cs [sha256.Size]byte
rand.Read(cs[:]) _, _ = rand.Read(cs[:])
var x checksum.Checksum var x checksum.Checksum

View file

@ -2,28 +2,50 @@ package client
import ( import (
"context" "context"
"fmt"
v2accounting "github.com/TrueCloudLab/frostfs-api-go/v2/accounting" v2accounting "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/accounting"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "github.com/TrueCloudLab/frostfs-api-go/v2/rpc" rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
"github.com/TrueCloudLab/frostfs-sdk-go/accounting" v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"github.com/TrueCloudLab/frostfs-sdk-go/user" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/accounting"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
) )
// PrmBalanceGet groups parameters of BalanceGet operation. // PrmBalanceGet groups parameters of BalanceGet operation.
type PrmBalanceGet struct { type PrmBalanceGet struct {
prmCommonMeta XHeaders []string
accountSet bool Account user.ID
account user.ID
} }
// SetAccount sets identifier of the FrostFS account for which the balance is requested. // SetAccount sets identifier of the FrostFS account for which the balance is requested.
// Required parameter. // Required parameter.
//
// Deprecated: Use PrmBalanceGet.Account instead.
func (x *PrmBalanceGet) SetAccount(id user.ID) { func (x *PrmBalanceGet) SetAccount(id user.ID) {
x.account = id x.Account = id
x.accountSet = true }
func (x *PrmBalanceGet) buildRequest(c *Client) (*v2accounting.BalanceRequest, error) {
if x.Account.IsEmpty() {
return nil, errorAccountNotSet
}
var accountV2 refs.OwnerID
x.Account.WriteToV2(&accountV2)
var body v2accounting.BalanceRequestBody
body.SetOwnerID(&accountV2)
var req v2accounting.BalanceRequest
req.SetBody(&body)
c.prepareRequest(&req, new(v2session.RequestMetaHeader))
return &req, nil
} }
// ResBalanceGet groups resulting values of BalanceGet operation. // ResBalanceGet groups resulting values of BalanceGet operation.
@ -42,9 +64,9 @@ func (x ResBalanceGet) Amount() accounting.Decimal {
// //
// Exactly one return value is non-nil. By default, server status is returned in res structure. // Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`, // Any client's internal or transport errors are returned as `error`,
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful // If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included // FrostFS status codes are included in the returned result structure,
// in the returned result structure. // otherwise, are also returned as `error`.
// //
// Returns an error if parameters are set incorrectly (see PrmBalanceGet docs). // Returns an error if parameters are set incorrectly (see PrmBalanceGet docs).
// Context is required and must not be nil. It is used for network communication. // Context is required and must not be nil. It is used for network communication.
@ -52,60 +74,35 @@ func (x ResBalanceGet) Amount() accounting.Decimal {
// Return statuses: // Return statuses:
// - global (see Client docs). // - global (see Client docs).
func (c *Client) BalanceGet(ctx context.Context, prm PrmBalanceGet) (*ResBalanceGet, error) { func (c *Client) BalanceGet(ctx context.Context, prm PrmBalanceGet) (*ResBalanceGet, error) {
switch { req, err := prm.buildRequest(c)
case ctx == nil: if err != nil {
return nil, errorMissingContext return nil, err
case !prm.accountSet:
return nil, errorAccountNotSet
} }
// form request body if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
var accountV2 refs.OwnerID return nil, fmt.Errorf("sign request: %w", err)
prm.account.WriteToV2(&accountV2)
var body v2accounting.BalanceRequestBody
body.SetOwnerID(&accountV2)
// form request
var req v2accounting.BalanceRequest
req.SetBody(&body)
// init call context
var (
cc contextCall
res ResBalanceGet
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.Balance(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2accounting.BalanceResponse)
const fieldBalance = "balance"
bal := resp.GetBody().GetBalance()
if bal == nil {
cc.err = newErrMissingResponseField(fieldBalance)
return
}
cc.err = res.amount.ReadFromV2(*bal)
if cc.err != nil {
cc.err = newErrInvalidResponseField(fieldBalance, cc.err)
}
} }
// process call resp, err := rpcapi.Balance(&c.c, req, client.WithContext(ctx))
if !cc.processCall() { if err != nil {
return nil, cc.err return nil, err
} }
var res ResBalanceGet
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
const fieldBalance = "balance"
bal := resp.GetBody().GetBalance()
if bal == nil {
return &res, newErrMissingResponseField(fieldBalance)
}
if err := res.amount.ReadFromV2(*bal); err != nil {
return &res, newErrInvalidResponseField(fieldBalance, err)
}
return &res, nil return &res, nil
} }

View file

@ -0,0 +1,79 @@
package client
import (
"context"
"fmt"
apemanagerV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/apemanager"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
sessionV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apeSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
)
// PrmAPEManagerAddChain groups parameters of APEManagerAddChain operation.
type PrmAPEManagerAddChain struct {
XHeaders []string
ChainTarget apeSDK.ChainTarget
Chain apeSDK.Chain
}
func (prm *PrmAPEManagerAddChain) buildRequest(c *Client) (*apemanagerV2.AddChainRequest, error) {
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
req := new(apemanagerV2.AddChainRequest)
reqBody := new(apemanagerV2.AddChainRequestBody)
reqBody.SetTarget(prm.ChainTarget.ToV2())
reqBody.SetChain(prm.Chain.ToV2())
req.SetBody(reqBody)
var meta sessionV2.RequestMetaHeader
writeXHeadersToMeta(prm.XHeaders, &meta)
c.prepareRequest(req, &meta)
return req, nil
}
type ResAPEManagerAddChain struct {
statusRes
// ChainID of set Chain. If Chain does not contain chainID before request, then
// ChainID is generated.
ChainID apeSDK.ChainID
}
// APEManagerAddChain sets Chain for ChainTarget.
func (c *Client) APEManagerAddChain(ctx context.Context, prm PrmAPEManagerAddChain) (*ResAPEManagerAddChain, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.AddChain(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResAPEManagerAddChain
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
res.ChainID = resp.GetBody().GetChainID()
return &res, nil
}

View file

@ -0,0 +1,80 @@
package client
import (
"context"
"fmt"
apemanagerV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/apemanager"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
sessionV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apeSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
)
// PrmAPEManagerListChains groups parameters of APEManagerListChains operation.
type PrmAPEManagerListChains struct {
XHeaders []string
ChainTarget apeSDK.ChainTarget
}
func (prm *PrmAPEManagerListChains) buildRequest(c *Client) (*apemanagerV2.ListChainsRequest, error) {
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
req := new(apemanagerV2.ListChainsRequest)
reqBody := new(apemanagerV2.ListChainsRequestBody)
reqBody.SetTarget(prm.ChainTarget.ToV2())
req.SetBody(reqBody)
var meta sessionV2.RequestMetaHeader
writeXHeadersToMeta(prm.XHeaders, &meta)
c.prepareRequest(req, &meta)
return req, nil
}
type ResAPEManagerListChains struct {
statusRes
Chains []apeSDK.Chain
}
// APEManagerListChains lists Chains for ChainTarget.
func (c *Client) APEManagerListChains(ctx context.Context, prm PrmAPEManagerListChains) (*ResAPEManagerListChains, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.ListChains(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResAPEManagerListChains
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
for _, ch := range resp.GetBody().GetChains() {
var chSDK apeSDK.Chain
if err := chSDK.ReadFromV2(ch); err != nil {
return nil, err
}
res.Chains = append(res.Chains, chSDK)
}
return &res, nil
}

View file

@ -0,0 +1,73 @@
package client
import (
"context"
"fmt"
apemanagerV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/apemanager"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
sessionV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apeSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
)
// PrmAPEManagerRemoveChain groups parameters of APEManagerRemoveChain operation.
type PrmAPEManagerRemoveChain struct {
XHeaders []string
ChainTarget apeSDK.ChainTarget
ChainID apeSDK.ChainID
}
func (prm *PrmAPEManagerRemoveChain) buildRequest(c *Client) (*apemanagerV2.RemoveChainRequest, error) {
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
req := new(apemanagerV2.RemoveChainRequest)
reqBody := new(apemanagerV2.RemoveChainRequestBody)
reqBody.SetTarget(prm.ChainTarget.ToV2())
reqBody.SetChainID(prm.ChainID)
req.SetBody(reqBody)
var meta sessionV2.RequestMetaHeader
writeXHeadersToMeta(prm.XHeaders, &meta)
c.prepareRequest(req, &meta)
return req, nil
}
type ResAPEManagerRemoveChain struct {
statusRes
}
// APEManagerRemoveChain removes Chain with ChainID defined for ChainTarget.
func (c *Client) APEManagerRemoveChain(ctx context.Context, prm PrmAPEManagerRemoveChain) (*ResAPEManagerRemoveChain, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.RemoveChain(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResAPEManagerRemoveChain
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
return &res, nil
}

View file

@ -4,9 +4,9 @@ import (
"context" "context"
"fmt" "fmt"
v2netmap "github.com/TrueCloudLab/frostfs-api-go/v2/netmap" v2netmap "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
rpcapi "github.com/TrueCloudLab/frostfs-api-go/v2/rpc" rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
) )
// interface of FrostFS API server. Exists for test purposes only. // interface of FrostFS API server. Exists for test purposes only.

View file

@ -7,9 +7,12 @@ import (
"errors" "errors"
"time" "time"
v2accounting "github.com/TrueCloudLab/frostfs-api-go/v2/accounting" v2accounting "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/accounting"
"github.com/TrueCloudLab/frostfs-api-go/v2/rpc" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
) )
// Client represents virtual connection to the FrostFS network to communicate // Client represents virtual connection to the FrostFS network to communicate
@ -76,46 +79,49 @@ func (c *Client) Init(prm PrmInit) {
// Calling multiple times leads to undefined behavior. // Calling multiple times leads to undefined behavior.
// //
// See also Init / Close. // See also Init / Close.
func (c *Client) Dial(prm PrmDial) error { func (c *Client) Dial(ctx context.Context, prm PrmDial) error {
if prm.endpoint == "" { if prm.Endpoint == "" {
return errorServerAddrUnset return errorServerAddrUnset
} }
if prm.timeoutDialSet { if prm.DialTimeout <= 0 {
if prm.timeoutDial <= 0 { prm.DialTimeout = defaultDialTimeout
return errorNonPositiveTimeout
}
} else {
prm.timeoutDial = 5 * time.Second
} }
if prm.StreamTimeout <= 0 {
if prm.streamTimeoutSet { prm.StreamTimeout = defaultStreamTimeout
if prm.streamTimeout <= 0 {
return errorNonPositiveTimeout
}
} else {
prm.streamTimeout = 10 * time.Second
} }
c.c = *client.New(append( c.c = *client.New(append(
client.WithNetworkURIAddress(prm.endpoint, prm.tlsConfig), client.WithNetworkURIAddress(prm.Endpoint, prm.TLSConfig),
client.WithDialTimeout(prm.timeoutDial), client.WithDialTimeout(prm.DialTimeout),
client.WithRWTimeout(prm.streamTimeout), client.WithRWTimeout(prm.StreamTimeout),
client.WithGRPCDialOptions(prm.GRPCDialOptions),
)...) )...)
c.setFrostFSAPIServer((*coreServer)(&c.c)) c.setFrostFSAPIServer((*coreServer)(&c.c))
if prm.parentCtx == nil { ctx, cancel := context.WithTimeout(ctx, prm.DialTimeout)
prm.parentCtx = context.Background() defer cancel()
}
// TODO: (neofs-api-go#382) perform generic dial stage of the client.Client
_, err := rpc.Balance(&c.c, new(v2accounting.BalanceRequest), _, err := rpc.Balance(&c.c, new(v2accounting.BalanceRequest),
client.WithContext(prm.parentCtx), client.WithContext(ctx),
) )
// return context errors since they signal about dial problem if err != nil {
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) { var ctxErr error
return err
// return context errors since they signal about dial problem
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
ctxErr = err
} else if st, ok := status.FromError(err); ok && st.Code() == codes.Canceled {
ctxErr = context.Canceled
} else if ok && st.Code() == codes.DeadlineExceeded {
ctxErr = context.DeadlineExceeded
}
if ctxErr != nil {
if conn := c.c.Conn(); conn != nil {
_ = conn.Close()
}
return ctxErr
}
} }
return nil return nil
@ -123,7 +129,7 @@ func (c *Client) Dial(prm PrmDial) error {
// sets underlying provider of frostFSAPIServer. The method is used for testing as an approach // sets underlying provider of frostFSAPIServer. The method is used for testing as an approach
// to skip Dial stage and override FrostFS API server. MUST NOT be used outside test code. // to skip Dial stage and override FrostFS API server. MUST NOT be used outside test code.
// In real applications wrapper over github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client // In real applications wrapper over git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client
// is statically used. // is statically used.
func (c *Client) setFrostFSAPIServer(server frostFSAPIServer) { func (c *Client) setFrostFSAPIServer(server frostFSAPIServer) {
c.server = server c.server = server
@ -146,52 +152,67 @@ func (c *Client) Close() error {
// //
// See also Init. // See also Init.
type PrmInit struct { type PrmInit struct {
resolveFrostFSErrors bool DisableFrostFSErrorResolution bool
key ecdsa.PrivateKey Key ecdsa.PrivateKey
cbRespInfo func(ResponseMetaInfo) error ResponseInfoCallback func(ResponseMetaInfo) error
netMagic uint64 NetMagic uint64
} }
// SetDefaultPrivateKey sets Client private key to be used for the protocol // SetDefaultPrivateKey sets Client private key to be used for the protocol
// communication by default. // communication by default.
// //
// Required for operations without custom key parametrization (see corresponding Prm* docs). // Required for operations without custom key parametrization (see corresponding Prm* docs).
//
// Deprecated: Use PrmInit.Key instead.
func (x *PrmInit) SetDefaultPrivateKey(key ecdsa.PrivateKey) { func (x *PrmInit) SetDefaultPrivateKey(key ecdsa.PrivateKey) {
x.key = key x.Key = key
} }
// ResolveFrostFSFailures makes the Client to resolve failure statuses of the // Deprecated: method is no-op. Option is default.
// FrostFS protocol into Go built-in errors. These errors are returned from
// each protocol operation. By default, statuses aren't resolved and written
// to the resulting structure (see corresponding Res* docs).
func (x *PrmInit) ResolveFrostFSFailures() { func (x *PrmInit) ResolveFrostFSFailures() {
x.resolveFrostFSErrors = true }
// DisableFrostFSFailuresResolution makes the Client to preserve failure statuses of the
// FrostFS protocol only in resulting structure (see corresponding Res* docs).
// These errors are returned from each protocol operation. By default, statuses
// are resolved and returned as a Go built-in errors.
//
// Deprecated: Use PrmInit.DisableFrostFSErrorResolution instead.
func (x *PrmInit) DisableFrostFSFailuresResolution() {
x.DisableFrostFSErrorResolution = true
} }
// SetResponseInfoCallback makes the Client to pass ResponseMetaInfo from each // SetResponseInfoCallback makes the Client to pass ResponseMetaInfo from each
// FrostFS server response to f. Nil (default) means ignore response meta info. // FrostFS server response to f. Nil (default) means ignore response meta info.
//
// Deprecated: Use PrmInit.ResponseInfoCallback instead.
func (x *PrmInit) SetResponseInfoCallback(f func(ResponseMetaInfo) error) { func (x *PrmInit) SetResponseInfoCallback(f func(ResponseMetaInfo) error) {
x.cbRespInfo = f x.ResponseInfoCallback = f
} }
const (
defaultDialTimeout = 5 * time.Second
defaultStreamTimeout = 10 * time.Second
)
// PrmDial groups connection parameters for the Client. // PrmDial groups connection parameters for the Client.
// //
// See also Dial. // See also Dial.
type PrmDial struct { type PrmDial struct {
endpoint string Endpoint string
tlsConfig *tls.Config TLSConfig *tls.Config
timeoutDialSet bool // If DialTimeout is non-positive, then it's set to defaultDialTimeout.
timeoutDial time.Duration DialTimeout time.Duration
streamTimeoutSet bool // If StreamTimeout is non-positive, then it's set to defaultStreamTimeout.
streamTimeout time.Duration StreamTimeout time.Duration
parentCtx context.Context GRPCDialOptions []grpc.DialOption
} }
// SetServerURI sets server URI in the FrostFS network. // SetServerURI sets server URI in the FrostFS network.
@ -207,36 +228,41 @@ type PrmDial struct {
// grpcs // grpcs
// //
// See also SetTLSConfig. // See also SetTLSConfig.
//
// Deprecated: Use PrmDial.Endpoint instead.
func (x *PrmDial) SetServerURI(endpoint string) { func (x *PrmDial) SetServerURI(endpoint string) {
x.endpoint = endpoint x.Endpoint = endpoint
} }
// SetTLSConfig sets tls.Config to open TLS client connection // SetTLSConfig sets tls.Config to open TLS client connection
// to the FrostFS server. Nil (default) means insecure connection. // to the FrostFS server. Nil (default) means insecure connection.
// //
// See also SetServerURI. // See also SetServerURI.
//
// Depreacted: Use PrmDial.TLSConfig instead.
func (x *PrmDial) SetTLSConfig(tlsConfig *tls.Config) { func (x *PrmDial) SetTLSConfig(tlsConfig *tls.Config) {
x.tlsConfig = tlsConfig x.TLSConfig = tlsConfig
} }
// SetTimeout sets the timeout for connection to be established. // SetTimeout sets the timeout for connection to be established.
// MUST BE positive. If not called, 5s timeout will be used by default. // MUST BE positive. If not called, 5s timeout will be used by default.
//
// Deprecated: Use PrmDial.DialTimeout instead.
func (x *PrmDial) SetTimeout(timeout time.Duration) { func (x *PrmDial) SetTimeout(timeout time.Duration) {
x.timeoutDialSet = true x.DialTimeout = timeout
x.timeoutDial = timeout
} }
// SetStreamTimeout sets the timeout for individual operations in streaming RPC. // SetStreamTimeout sets the timeout for individual operations in streaming RPC.
// MUST BE positive. If not called, 10s timeout will be used by default. // MUST BE positive. If not called, 10s timeout will be used by default.
//
// Deprecated: Use PrmDial.StreamTimeout instead.
func (x *PrmDial) SetStreamTimeout(timeout time.Duration) { func (x *PrmDial) SetStreamTimeout(timeout time.Duration) {
x.streamTimeoutSet = true x.StreamTimeout = timeout
x.streamTimeout = timeout
} }
// SetContext allows to specify optional base context within which connection // SetGRPCDialOptions sets the gRPC dial options for new gRPC client connection.
// should be established.
// //
// Context SHOULD NOT be nil. // Deprecated: Use PrmDial.GRPCDialOptions instead.
func (x *PrmDial) SetContext(ctx context.Context) { func (x *PrmDial) SetGRPCDialOptions(opts ...grpc.DialOption) {
x.parentCtx = ctx x.GRPCDialOptions = opts
} }

View file

@ -7,7 +7,7 @@ import (
"crypto/rand" "crypto/rand"
"testing" "testing"
apistatus "github.com/TrueCloudLab/frostfs-sdk-go/client/status" apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
@ -29,8 +29,9 @@ func assertStatusErr(tb testing.TB, res interface{ Status() apistatus.Status })
} }
func newClient(server frostFSAPIServer) *Client { func newClient(server frostFSAPIServer) *Client {
var prm PrmInit prm := PrmInit{
prm.SetDefaultPrivateKey(*key) Key: *key,
}
var c Client var c Client
c.Init(prm) c.Init(prm)
@ -43,15 +44,13 @@ func TestClient_DialContext(t *testing.T) {
var c Client var c Client
// try to connect to any host // try to connect to any host
var prm PrmDial prm := PrmDial{
prm.SetServerURI("localhost:8080") Endpoint: "localhost:8080",
}
assert := func(ctx context.Context, errExpected error) { assert := func(ctx context.Context, errExpected error) {
// use the particular context
prm.SetContext(ctx)
// expect particular context error according to Dial docs // expect particular context error according to Dial docs
require.ErrorIs(t, c.Dial(prm), errExpected) require.ErrorIs(t, c.Dial(ctx, prm), errExpected)
} }
// create pre-abandoned context // create pre-abandoned context

View file

@ -1,33 +1,22 @@
package client package client
import ( import (
"crypto/ecdsa"
"errors" "errors"
"fmt" "fmt"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
"github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "github.com/TrueCloudLab/frostfs-api-go/v2/session" v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"github.com/TrueCloudLab/frostfs-api-go/v2/signature" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "github.com/TrueCloudLab/frostfs-sdk-go/client/status" apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"github.com/TrueCloudLab/frostfs-sdk-go/version" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/version"
) )
// common interface of resulting structures with API status.
type resCommon interface {
setStatus(apistatus.Status)
}
// structure is embedded to all resulting types in order to inherit status-related methods. // structure is embedded to all resulting types in order to inherit status-related methods.
type statusRes struct { type statusRes struct {
st apistatus.Status st apistatus.Status
} }
// setStatus implements resCommon interface method.
func (x *statusRes) setStatus(st apistatus.Status) {
x.st = st
}
// Status returns server's status return. // Status returns server's status return.
// //
// Use apistatus package functionality to handle the status. // Use apistatus package functionality to handle the status.
@ -35,29 +24,13 @@ func (x statusRes) Status() apistatus.Status {
return x.st return x.st
} }
// groups meta parameters shared between all Client operations.
type prmCommonMeta struct {
// FrostFS request X-Headers
xHeaders []string
}
// WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length.
//
// Slice must not be mutated until the operation completes.
func (x *prmCommonMeta) WithXHeaders(hs ...string) {
if len(hs)%2 != 0 {
panic("slice of X-Headers with odd length")
}
x.xHeaders = hs
}
func writeXHeadersToMeta(xHeaders []string, h *v2session.RequestMetaHeader) { func writeXHeadersToMeta(xHeaders []string, h *v2session.RequestMetaHeader) {
if len(xHeaders) == 0 { if len(xHeaders) == 0 {
return return
} }
// TODO (aarifullin): remove the panic when all client parameters will check XHeaders
// within buildRequest invocation.
if len(xHeaders)%2 != 0 { if len(xHeaders)%2 != 0 {
panic("slice of X-Headers with odd length") panic("slice of X-Headers with odd length")
} }
@ -73,104 +46,21 @@ func writeXHeadersToMeta(xHeaders []string, h *v2session.RequestMetaHeader) {
// error messages. // error messages.
var ( var (
errorMissingContext = errors.New("missing context") errorMissingContainer = errors.New("missing container")
errorMissingContainer = errors.New("missing container") errorMissingObject = errors.New("missing object")
errorMissingObject = errors.New("missing object") errorAccountNotSet = errors.New("account not set")
errorAccountNotSet = errors.New("account not set") errorServerAddrUnset = errors.New("server address is unset or empty")
errorServerAddrUnset = errors.New("server address is unset or empty") errorZeroRangeLength = errors.New("zero range length")
errorNonPositiveTimeout = errors.New("non-positive timeout") errorMissingRanges = errors.New("missing ranges")
errorEACLTableNotSet = errors.New("eACL table not set") errorInvalidXHeaders = errors.New("xheaders must be presented only as key-value pairs")
errorMissingAnnouncements = errors.New("missing announcements")
errorZeroRangeLength = errors.New("zero range length")
errorMissingRanges = errors.New("missing ranges")
errorZeroEpoch = errors.New("zero epoch")
errorMissingTrusts = errors.New("missing trusts")
errorTrustNotSet = errors.New("current trust value not set")
) )
// groups all the details required to send a single request and process a response to it.
type contextCall struct {
// ==================================================
// state vars that do not require explicit initialization
// final error to be returned from client method
err error
// received response
resp responseV2
// ==================================================
// shared parameters which are set uniformly on all calls
// request signing key
key ecdsa.PrivateKey
// callback prior to processing the response by the client
callbackResp func(ResponseMetaInfo) error
// if set, protocol errors will be expanded into a final error
resolveAPIFailures bool
// FrostFS network magic
netMagic uint64
// Meta parameters
meta prmCommonMeta
// ==================================================
// custom call parameters
// structure of the call result
statusRes resCommon
// request to be signed with a key and sent
req request
// function to send a request (unary) and receive a response
call func() (responseV2, error)
// function to send the request (req field)
wReq func() error
// function to recv the response (resp field)
rResp func() error
// function to close the message stream
closer func() error
// function of writing response fields to the resulting structure (optional)
result func(v2 responseV2)
}
type request interface { type request interface {
GetMetaHeader() *v2session.RequestMetaHeader GetMetaHeader() *v2session.RequestMetaHeader
SetMetaHeader(*v2session.RequestMetaHeader) SetMetaHeader(*v2session.RequestMetaHeader)
SetVerificationHeader(*v2session.RequestVerificationHeader) SetVerificationHeader(*v2session.RequestVerificationHeader)
} }
// sets needed fields of the request meta header.
func (x contextCall) prepareRequest() {
meta := x.req.GetMetaHeader()
if meta == nil {
meta = new(v2session.RequestMetaHeader)
x.req.SetMetaHeader(meta)
}
if meta.GetTTL() == 0 {
meta.SetTTL(2)
}
if meta.GetVersion() == nil {
var verV2 refs.Version
version.Current().WriteToV2(&verV2)
meta.SetVersion(&verV2)
}
meta.SetNetworkMagic(x.netMagic)
writeXHeadersToMeta(x.meta.xHeaders, meta)
}
func (c *Client) prepareRequest(req request, meta *v2session.RequestMetaHeader) { func (c *Client) prepareRequest(req request, meta *v2session.RequestMetaHeader) {
ttl := meta.GetTTL() ttl := meta.GetTTL()
if ttl == 0 { if ttl == 0 {
@ -185,167 +75,36 @@ func (c *Client) prepareRequest(req request, meta *v2session.RequestMetaHeader)
meta.SetTTL(ttl) meta.SetTTL(ttl)
meta.SetVersion(verV2) meta.SetVersion(verV2)
meta.SetNetworkMagic(c.prm.netMagic) meta.SetNetworkMagic(c.prm.NetMagic)
req.SetMetaHeader(meta) req.SetMetaHeader(meta)
} }
// prepares, signs and writes the request. Result means success. // processResponse verifies response signature and converts status to an error if needed.
// If failed, contextCall.err contains the reason. func (c *Client) processResponse(resp responseV2) (apistatus.Status, error) {
func (x *contextCall) writeRequest() bool { if c.prm.ResponseInfoCallback != nil {
x.prepareRequest() rmi := ResponseMetaInfo{
key: resp.GetVerificationHeader().GetBodySignature().GetKey(),
x.req.SetVerificationHeader(nil) epoch: resp.GetMetaHeader().GetEpoch(),
}
// sign the request if err := c.prm.ResponseInfoCallback(rmi); err != nil {
x.err = signature.SignServiceMessage(&x.key, x.req) return nil, fmt.Errorf("response callback error: %w", err)
if x.err != nil {
x.err = fmt.Errorf("sign request: %w", x.err)
return false
}
x.err = x.wReq()
if x.err != nil {
x.err = fmt.Errorf("write request: %w", x.err)
return false
}
return true
}
// performs common actions of response processing and writes any problem as a result status or client error
// (in both cases returns false).
//
// Actions:
// - verify signature (internal);
// - call response callback (internal);
// - unwrap status error (optional).
func (x *contextCall) processResponse() bool {
// call response callback if set
if x.callbackResp != nil {
x.err = x.callbackResp(ResponseMetaInfo{
key: x.resp.GetVerificationHeader().GetBodySignature().GetKey(),
epoch: x.resp.GetMetaHeader().GetEpoch(),
})
if x.err != nil {
x.err = fmt.Errorf("response callback error: %w", x.err)
return false
} }
} }
// note that we call response callback before signature check since it is expected more lightweight
// while verification needs marshaling
// verify response signature
x.err = signature.VerifyServiceMessage(x.resp)
if x.err != nil {
x.err = fmt.Errorf("invalid response signature: %w", x.err)
return false
}
// get result status
st := apistatus.FromStatusV2(x.resp.GetMetaHeader().GetStatus())
// unwrap unsuccessful status and return it
// as error if client has been configured so
successfulStatus := apistatus.IsSuccessful(st)
if x.resolveAPIFailures {
x.err = apistatus.ErrFromStatus(st)
} else {
x.statusRes.setStatus(st)
}
return successfulStatus
}
// processResponse verifies response signature and converts status to an error if needed.
func (c *Client) processResponse(resp responseV2) (apistatus.Status, error) {
err := signature.VerifyServiceMessage(resp) err := signature.VerifyServiceMessage(resp)
if err != nil { if err != nil {
return nil, fmt.Errorf("invalid response signature: %w", err) return nil, fmt.Errorf("invalid response signature: %w", err)
} }
st := apistatus.FromStatusV2(resp.GetMetaHeader().GetStatus()) st := apistatus.FromStatusV2(resp.GetMetaHeader().GetStatus())
if c.prm.resolveFrostFSErrors { if !c.prm.DisableFrostFSErrorResolution {
return st, apistatus.ErrFromStatus(st) return st, apistatus.ErrFromStatus(st)
} }
return st, nil return st, nil
} }
// reads response (if rResp is set) and processes it. Result means success. // ExecRaw executes f with underlying git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client.Client
// If failed, contextCall.err (or statusRes if resolveAPIFailures is set) contains the reason.
func (x *contextCall) readResponse() bool {
if x.rResp != nil {
x.err = x.rResp()
if x.err != nil {
x.err = fmt.Errorf("read response: %w", x.err)
return false
}
}
return x.processResponse()
}
// closes the message stream (if closer is set) and writes the results (if result is set).
// Return means success. If failed, contextCall.err contains the reason.
func (x *contextCall) close() bool {
if x.closer != nil {
x.err = x.closer()
if x.err != nil {
x.err = fmt.Errorf("close RPC: %w", x.err)
return false
}
}
// write response to resulting structure
if x.result != nil {
x.result(x.resp)
}
return x.err == nil
}
// goes through all stages of sending a request and processing a response. Returns true if successful.
// If failed, contextCall.err contains the reason.
func (x *contextCall) processCall() bool {
// set request writer
x.wReq = func() error {
var err error
x.resp, err = x.call()
return err
}
// write request
ok := x.writeRequest()
if !ok {
return false
}
// read response
ok = x.readResponse()
if !ok {
return x.err == nil
}
// close and write response to resulting structure
ok = x.close()
if !ok {
return false
}
return x.err == nil
}
// initializes static cross-call parameters inherited from client.
func (c *Client) initCallContext(ctx *contextCall) {
ctx.key = c.prm.key
ctx.resolveAPIFailures = c.prm.resolveFrostFSErrors
ctx.callbackResp = c.prm.cbRespInfo
ctx.netMagic = c.prm.netMagic
}
// ExecRaw executes f with underlying github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client.Client
// instance. Communicate over the Protocol Buffers protocol in a more flexible way: // instance. Communicate over the Protocol Buffers protocol in a more flexible way:
// most often used to transmit data over a fixed version of the FrostFS protocol, as well // most often used to transmit data over a fixed version of the FrostFS protocol, as well
// as to support custom services. // as to support custom services.
@ -356,7 +115,7 @@ func (c *Client) initCallContext(ctx *contextCall) {
// before closing the connection. // before closing the connection.
// //
// See also Dial and Close. // See also Dial and Close.
// See also github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client package docs. // See also git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client package docs.
func (c *Client) ExecRaw(f func(client *client.Client) error) error { func (c *Client) ExecRaw(f func(client *client.Client) error) error {
return f(&c.c) return f(&c.c)
} }

View file

@ -2,803 +2,11 @@ package client
import ( import (
"context" "context"
"errors"
"fmt" "fmt"
v2container "github.com/TrueCloudLab/frostfs-api-go/v2/container" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "github.com/TrueCloudLab/frostfs-api-go/v2/rpc"
"github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "github.com/TrueCloudLab/frostfs-api-go/v2/session"
"github.com/TrueCloudLab/frostfs-sdk-go/container"
cid "github.com/TrueCloudLab/frostfs-sdk-go/container/id"
frostfscrypto "github.com/TrueCloudLab/frostfs-sdk-go/crypto"
frostfsecdsa "github.com/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
"github.com/TrueCloudLab/frostfs-sdk-go/eacl"
"github.com/TrueCloudLab/frostfs-sdk-go/session"
"github.com/TrueCloudLab/frostfs-sdk-go/user"
) )
// PrmContainerPut groups parameters of ContainerPut operation.
type PrmContainerPut struct {
prmCommonMeta
cnrSet bool
cnr container.Container
sessionSet bool
session session.Container
}
// SetContainer sets structured information about new FrostFS container.
// Required parameter.
func (x *PrmContainerPut) SetContainer(cnr container.Container) {
x.cnr = cnr
x.cnrSet = true
}
// WithinSession specifies session within which container should be saved.
//
// Creator of the session acquires the authorship of the request. This affects
// the execution of an operation (e.g. access control).
//
// Session is optional, if set the following requirements apply:
// - session operation MUST be session.VerbContainerPut (ForVerb)
// - token MUST be signed using private key of the owner of the container to be saved
func (x *PrmContainerPut) WithinSession(s session.Container) {
x.session = s
x.sessionSet = true
}
// ResContainerPut groups resulting values of ContainerPut operation.
type ResContainerPut struct {
statusRes
id cid.ID
}
// ID returns identifier of the container declared to be stored in the system.
// Used as a link to information about the container (in particular, you can
// asynchronously check if the save was successful).
func (x ResContainerPut) ID() cid.ID {
return x.id
}
// ContainerPut sends request to save container in FrostFS.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see ResContainerPut.ID).
//
// Returns an error if parameters are set incorrectly (see PrmContainerPut docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerPut(ctx context.Context, prm PrmContainerPut) (*ResContainerPut, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case !prm.cnrSet:
return nil, errorMissingContainer
}
// TODO: check private key is set before forming the request
// sign container
var cnr v2container.Container
prm.cnr.WriteToV2(&cnr)
var sig frostfscrypto.Signature
err := container.CalculateSignature(&sig, prm.cnr, c.prm.key)
if err != nil {
return nil, fmt.Errorf("calculate container signature: %w", err)
}
var sigv2 refs.Signature
sig.WriteToV2(&sigv2)
// form request body
reqBody := new(v2container.PutRequestBody)
reqBody.SetContainer(&cnr)
reqBody.SetSignature(&sigv2)
// form meta header
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(prm.prmCommonMeta.xHeaders, &meta)
if prm.sessionSet {
var tokv2 v2session.Token
prm.session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
// form request
var req v2container.PutRequest
req.SetBody(reqBody)
req.SetMetaHeader(&meta)
// init call context
var (
cc contextCall
res ResContainerPut
)
c.initCallContext(&cc)
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.PutContainer(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2container.PutResponse)
const fieldCnrID = "container ID"
cidV2 := resp.GetBody().GetContainerID()
if cidV2 == nil {
cc.err = newErrMissingResponseField(fieldCnrID)
return
}
cc.err = res.id.ReadFromV2(*cidV2)
if cc.err != nil {
cc.err = newErrInvalidResponseField(fieldCnrID, cc.err)
}
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmContainerGet groups parameters of ContainerGet operation.
type PrmContainerGet struct {
prmCommonMeta
idSet bool
id cid.ID
}
// SetContainer sets identifier of the container to be read.
// Required parameter.
func (x *PrmContainerGet) SetContainer(id cid.ID) {
x.id = id
x.idSet = true
}
// ResContainerGet groups resulting values of ContainerGet operation.
type ResContainerGet struct {
statusRes
cnr container.Container
}
// Container returns structured information about the requested container.
//
// Client doesn't retain value so modification is safe.
func (x ResContainerGet) Container() container.Container {
return x.cnr
}
// ContainerGet reads FrostFS container by ID.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Returns an error if parameters are set incorrectly (see PrmContainerGet docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs);
// - *apistatus.ContainerNotFound.
func (c *Client) ContainerGet(ctx context.Context, prm PrmContainerGet) (*ResContainerGet, error) {
switch {
case ctx == nil:
return nil, errorMissingContext
case !prm.idSet:
return nil, errorMissingContainer
}
var cidV2 refs.ContainerID
prm.id.WriteToV2(&cidV2)
// form request body
reqBody := new(v2container.GetRequestBody)
reqBody.SetContainerID(&cidV2)
// form request
var req v2container.GetRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResContainerGet
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.GetContainer(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2container.GetResponse)
cnrV2 := resp.GetBody().GetContainer()
if cnrV2 == nil {
cc.err = errors.New("missing container in response")
return
}
cc.err = res.cnr.ReadFromV2(*cnrV2)
if cc.err != nil {
cc.err = fmt.Errorf("invalid container in response: %w", cc.err)
}
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmContainerList groups parameters of ContainerList operation.
type PrmContainerList struct {
prmCommonMeta
ownerSet bool
ownerID user.ID
}
// SetAccount sets identifier of the FrostFS account to list the containers.
// Required parameter.
func (x *PrmContainerList) SetAccount(id user.ID) {
x.ownerID = id
x.ownerSet = true
}
// ResContainerList groups resulting values of ContainerList operation.
type ResContainerList struct {
statusRes
ids []cid.ID
}
// Containers returns list of identifiers of the account-owned containers.
//
// Client doesn't retain value so modification is safe.
func (x ResContainerList) Containers() []cid.ID {
return x.ids
}
// ContainerList requests identifiers of the account-owned containers.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Returns an error if parameters are set incorrectly (see PrmContainerList docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerList(ctx context.Context, prm PrmContainerList) (*ResContainerList, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case !prm.ownerSet:
return nil, errorAccountNotSet
}
// form request body
var ownerV2 refs.OwnerID
prm.ownerID.WriteToV2(&ownerV2)
reqBody := new(v2container.ListRequestBody)
reqBody.SetOwnerID(&ownerV2)
// form request
var req v2container.ListRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResContainerList
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.ListContainers(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2container.ListResponse)
res.ids = make([]cid.ID, len(resp.GetBody().GetContainerIDs()))
for i, cidV2 := range resp.GetBody().GetContainerIDs() {
cc.err = res.ids[i].ReadFromV2(cidV2)
if cc.err != nil {
cc.err = fmt.Errorf("invalid ID in the response: %w", cc.err)
return
}
}
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmContainerDelete groups parameters of ContainerDelete operation.
type PrmContainerDelete struct {
prmCommonMeta
idSet bool
id cid.ID
tokSet bool
tok session.Container
}
// SetContainer sets identifier of the FrostFS container to be removed.
// Required parameter.
func (x *PrmContainerDelete) SetContainer(id cid.ID) {
x.id = id
x.idSet = true
}
// WithinSession specifies session within which container should be removed.
//
// Creator of the session acquires the authorship of the request.
// This may affect the execution of an operation (e.g. access control).
//
// Must be signed.
func (x *PrmContainerDelete) WithinSession(tok session.Container) {
x.tok = tok
x.tokSet = true
}
// ResContainerDelete groups resulting values of ContainerDelete operation.
type ResContainerDelete struct {
statusRes
}
// ContainerDelete sends request to remove the FrostFS container.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see GetContainer).
//
// Returns an error if parameters are set incorrectly (see PrmContainerDelete docs).
// Context is required and must not be nil. It is used for network communication.
//
// Exactly one return value is non-nil. Server status return is returned in ResContainerDelete.
// Reflects all internal errors in second return value (transport problems, response processing, etc.).
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerDelete(ctx context.Context, prm PrmContainerDelete) (*ResContainerDelete, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case !prm.idSet:
return nil, errorMissingContainer
}
// sign container ID
var cidV2 refs.ContainerID
prm.id.WriteToV2(&cidV2)
// container contract expects signature of container ID value
// don't get confused with stable marshaled protobuf container.ID structure
data := cidV2.GetValue()
var sig frostfscrypto.Signature
err := sig.Calculate(frostfsecdsa.SignerRFC6979(c.prm.key), data)
if err != nil {
return nil, fmt.Errorf("calculate signature: %w", err)
}
var sigv2 refs.Signature
sig.WriteToV2(&sigv2)
// form request body
reqBody := new(v2container.DeleteRequestBody)
reqBody.SetContainerID(&cidV2)
reqBody.SetSignature(&sigv2)
// form meta header
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(prm.prmCommonMeta.xHeaders, &meta)
if prm.tokSet {
var tokv2 v2session.Token
prm.tok.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
// form request
var req v2container.DeleteRequest
req.SetBody(reqBody)
req.SetMetaHeader(&meta)
// init call context
var (
cc contextCall
res ResContainerDelete
)
c.initCallContext(&cc)
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.DeleteContainer(&c.c, &req, client.WithContext(ctx))
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmContainerEACL groups parameters of ContainerEACL operation.
type PrmContainerEACL struct {
prmCommonMeta
idSet bool
id cid.ID
}
// SetContainer sets identifier of the FrostFS container to read the eACL table.
// Required parameter.
func (x *PrmContainerEACL) SetContainer(id cid.ID) {
x.id = id
x.idSet = true
}
// ResContainerEACL groups resulting values of ContainerEACL operation.
type ResContainerEACL struct {
statusRes
table eacl.Table
}
// Table returns eACL table of the requested container.
func (x ResContainerEACL) Table() eacl.Table {
return x.table
}
// ContainerEACL reads eACL table of the FrostFS container.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Returns an error if parameters are set incorrectly (see PrmContainerEACL docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs);
// - *apistatus.ContainerNotFound;
// - *apistatus.EACLNotFound.
func (c *Client) ContainerEACL(ctx context.Context, prm PrmContainerEACL) (*ResContainerEACL, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case !prm.idSet:
return nil, errorMissingContainer
}
var cidV2 refs.ContainerID
prm.id.WriteToV2(&cidV2)
// form request body
reqBody := new(v2container.GetExtendedACLRequestBody)
reqBody.SetContainerID(&cidV2)
// form request
var req v2container.GetExtendedACLRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResContainerEACL
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.GetEACL(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2container.GetExtendedACLResponse)
eACL := resp.GetBody().GetEACL()
if eACL == nil {
cc.err = newErrMissingResponseField("eACL")
return
}
res.table = *eacl.NewTableFromV2(eACL)
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmContainerSetEACL groups parameters of ContainerSetEACL operation.
type PrmContainerSetEACL struct {
prmCommonMeta
tableSet bool
table eacl.Table
sessionSet bool
session session.Container
}
// SetTable sets eACL table structure to be set for the container.
// Required parameter.
func (x *PrmContainerSetEACL) SetTable(table eacl.Table) {
x.table = table
x.tableSet = true
}
// WithinSession specifies session within which extended ACL of the container
// should be saved.
//
// Creator of the session acquires the authorship of the request. This affects
// the execution of an operation (e.g. access control).
//
// Session is optional, if set the following requirements apply:
// - if particular container is specified (ApplyOnlyTo), it MUST equal the container
// for which extended ACL is going to be set
// - session operation MUST be session.VerbContainerSetEACL (ForVerb)
// - token MUST be signed using private key of the owner of the container to be saved
func (x *PrmContainerSetEACL) WithinSession(s session.Container) {
x.session = s
x.sessionSet = true
}
// ResContainerSetEACL groups resulting values of ContainerSetEACL operation.
type ResContainerSetEACL struct {
statusRes
}
// ContainerSetEACL sends request to update eACL table of the FrostFS container.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see EACL).
//
// Returns an error if parameters are set incorrectly (see PrmContainerSetEACL docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerSetEACL(ctx context.Context, prm PrmContainerSetEACL) (*ResContainerSetEACL, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case !prm.tableSet:
return nil, errorEACLTableNotSet
}
// sign the eACL table
eaclV2 := prm.table.ToV2()
var sig frostfscrypto.Signature
err := sig.Calculate(frostfsecdsa.SignerRFC6979(c.prm.key), eaclV2.StableMarshal(nil))
if err != nil {
return nil, fmt.Errorf("calculate signature: %w", err)
}
var sigv2 refs.Signature
sig.WriteToV2(&sigv2)
// form request body
reqBody := new(v2container.SetExtendedACLRequestBody)
reqBody.SetEACL(eaclV2)
reqBody.SetSignature(&sigv2)
// form meta header
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(prm.prmCommonMeta.xHeaders, &meta)
if prm.sessionSet {
var tokv2 v2session.Token
prm.session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
// form request
var req v2container.SetExtendedACLRequest
req.SetBody(reqBody)
req.SetMetaHeader(&meta)
// init call context
var (
cc contextCall
res ResContainerSetEACL
)
c.initCallContext(&cc)
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.SetEACL(&c.c, &req, client.WithContext(ctx))
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmAnnounceSpace groups parameters of ContainerAnnounceUsedSpace operation.
type PrmAnnounceSpace struct {
prmCommonMeta
announcements []container.SizeEstimation
}
// SetValues sets values describing volume of space that is used for the container objects.
// Required parameter. Must not be empty.
//
// Must not be mutated before the end of the operation.
func (x *PrmAnnounceSpace) SetValues(vs []container.SizeEstimation) {
x.announcements = vs
}
// ResAnnounceSpace groups resulting values of ContainerAnnounceUsedSpace operation.
type ResAnnounceSpace struct {
statusRes
}
// ContainerAnnounceUsedSpace sends request to announce volume of the space used for the container objects.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// At this moment success can not be checked.
//
// Returns an error if parameters are set incorrectly (see PrmAnnounceSpace docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerAnnounceUsedSpace(ctx context.Context, prm PrmAnnounceSpace) (*ResAnnounceSpace, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case len(prm.announcements) == 0:
return nil, errorMissingAnnouncements
}
// convert list of SDK announcement structures into FrostFS-API v2 list
v2announce := make([]v2container.UsedSpaceAnnouncement, len(prm.announcements))
for i := range prm.announcements {
prm.announcements[i].WriteToV2(&v2announce[i])
}
// prepare body of the FrostFS-API v2 request and request itself
reqBody := new(v2container.AnnounceUsedSpaceRequestBody)
reqBody.SetAnnouncements(v2announce)
// form request
var req v2container.AnnounceUsedSpaceRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResAnnounceSpace
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.AnnounceUsedSpace(&c.c, &req, client.WithContext(ctx))
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// SyncContainerWithNetwork requests network configuration using passed client // SyncContainerWithNetwork requests network configuration using passed client
// and applies it to the container. Container MUST not be nil. // and applies it to the container. Container MUST not be nil.
// //

139
client/container_delete.go Normal file
View file

@ -0,0 +1,139 @@
package client
import (
"context"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmContainerDelete groups parameters of ContainerDelete operation.
type PrmContainerDelete struct {
// FrostFS request X-Headers
XHeaders []string
ContainerID *cid.ID
Session *session.Container
}
// SetContainer sets identifier of the FrostFS container to be removed.
// Required parameter.
//
// Deprecated: Use PrmContainerDelete.Container instead.
func (prm *PrmContainerDelete) SetContainer(id cid.ID) {
prm.ContainerID = &id
}
func (prm *PrmContainerDelete) buildRequest(c *Client) (*v2container.DeleteRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
var cidV2 refs.ContainerID
prm.ContainerID.WriteToV2(&cidV2)
// Container contract expects signature of container ID value,
// don't get confused with stable marshaled protobuf container.ID structure.
data := cidV2.GetValue()
var sig frostfscrypto.Signature
err := sig.Calculate(frostfsecdsa.SignerRFC6979(c.prm.Key), data)
if err != nil {
return nil, fmt.Errorf("calculate signature: %w", err)
}
var sigv2 refs.Signature
sig.WriteToV2(&sigv2)
reqBody := new(v2container.DeleteRequestBody)
reqBody.SetContainerID(&cidV2)
reqBody.SetSignature(&sigv2)
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(prm.XHeaders, &meta)
if prm.Session != nil {
var tokv2 v2session.Token
prm.Session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
var req v2container.DeleteRequest
req.SetBody(reqBody)
c.prepareRequest(&req, &meta)
return &req, nil
}
// WithinSession specifies session within which container should be removed.
//
// Creator of the session acquires the authorship of the request.
// This may affect the execution of an operation (e.g. access control).
//
// Must be signed.
//
// Deprecated: Use PrmContainerDelete.Session instead.
func (prm *PrmContainerDelete) WithinSession(tok session.Container) {
prm.Session = &tok
}
// ResContainerDelete groups resulting values of ContainerDelete operation.
type ResContainerDelete struct {
statusRes
}
// ContainerDelete sends request to remove the FrostFS container.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see GetContainer).
//
// Returns an error if parameters are set incorrectly (see PrmContainerDelete docs).
// Context is required and must not be nil. It is used for network communication.
//
// Exactly one return value is non-nil. Server status return is returned in ResContainerDelete.
// Reflects all internal errors in second return value (transport problems, response processing, etc.).
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerDelete(ctx context.Context, prm PrmContainerDelete) (*ResContainerDelete, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.DeleteContainer(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResContainerDelete
res.st, err = c.processResponse(resp)
return &res, err
}

126
client/container_get.go Normal file
View file

@ -0,0 +1,126 @@
package client
import (
"context"
"errors"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmContainerGet groups parameters of ContainerGet operation.
type PrmContainerGet struct {
// FrostFS request X-Headers.
XHeaders []string
ContainerID *cid.ID
Session *session.Container
}
// SetContainer sets identifier of the container to be read.
// Required parameter.
//
// Deprecated: Use PrmContainerGet.ContainerID instead.
func (prm *PrmContainerGet) SetContainer(cid cid.ID) {
prm.ContainerID = &cid
}
func (prm *PrmContainerGet) buildRequest(c *Client) (*v2container.GetRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
var cidV2 refs.ContainerID
prm.ContainerID.WriteToV2(&cidV2)
reqBody := new(v2container.GetRequestBody)
reqBody.SetContainerID(&cidV2)
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(prm.XHeaders, &meta)
if prm.Session != nil {
var tokv2 v2session.Token
prm.Session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
var req v2container.GetRequest
req.SetBody(reqBody)
c.prepareRequest(&req, &meta)
return &req, nil
}
// ResContainerGet groups resulting values of ContainerGet operation.
type ResContainerGet struct {
statusRes
cnr container.Container
}
// Container returns structured information about the requested container.
//
// Client doesn't retain value so modification is safe.
func (x ResContainerGet) Container() container.Container {
return x.cnr
}
// ContainerGet reads FrostFS container by ID.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Returns an error if parameters are set incorrectly (see PrmContainerGet docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs);
// - *apistatus.ContainerNotFound.
func (c *Client) ContainerGet(ctx context.Context, prm PrmContainerGet) (*ResContainerGet, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.GetContainer(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResContainerGet
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
cnrV2 := resp.GetBody().GetContainer()
if cnrV2 == nil {
return &res, errors.New("missing container in response")
}
if err := res.cnr.ReadFromV2(*cnrV2); err != nil {
return &res, fmt.Errorf("invalid container in response: %w", err)
}
return &res, nil
}

119
client/container_list.go Normal file
View file

@ -0,0 +1,119 @@
package client
import (
"context"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
)
// PrmContainerList groups parameters of ContainerList operation.
type PrmContainerList struct {
XHeaders []string
Account user.ID
Session *session.Container
}
// SetAccount sets identifier of the FrostFS account to list the containers.
// Required parameter.
//
// Deprecated: Use PrmContainerList.Account instead.
func (x *PrmContainerList) SetAccount(id user.ID) {
x.Account = id
}
func (x *PrmContainerList) buildRequest(c *Client) (*v2container.ListRequest, error) {
if x.Account.IsEmpty() {
return nil, errorAccountNotSet
}
var ownerV2 refs.OwnerID
x.Account.WriteToV2(&ownerV2)
reqBody := new(v2container.ListRequestBody)
reqBody.SetOwnerID(&ownerV2)
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(x.XHeaders, &meta)
if x.Session != nil {
var tokv2 v2session.Token
x.Session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
var req v2container.ListRequest
req.SetBody(reqBody)
c.prepareRequest(&req, &meta)
return &req, nil
}
// ResContainerList groups resulting values of ContainerList operation.
type ResContainerList struct {
statusRes
ids []cid.ID
}
// Containers returns list of identifiers of the account-owned containers.
//
// Client doesn't retain value so modification is safe.
func (x ResContainerList) Containers() []cid.ID {
return x.ids
}
// ContainerList requests identifiers of the account-owned containers.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Returns an error if parameters are set incorrectly (see PrmContainerList docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerList(ctx context.Context, prm PrmContainerList) (*ResContainerList, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.ListContainers(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResContainerList
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
res.ids = make([]cid.ID, len(resp.GetBody().GetContainerIDs()))
for i, cidV2 := range resp.GetBody().GetContainerIDs() {
if err := res.ids[i].ReadFromV2(cidV2); err != nil {
return &res, fmt.Errorf("invalid ID in the response: %w", err)
}
}
return &res, nil
}

160
client/container_put.go Normal file
View file

@ -0,0 +1,160 @@
package client
import (
"context"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmContainerPut groups parameters of ContainerPut operation.
type PrmContainerPut struct {
// FrostFS request X-Headers
XHeaders []string
Container *container.Container
Session *session.Container
}
// SetContainer sets structured information about new FrostFS container.
// Required parameter.
//
// Deprecated: Use PrmContainerPut.Container instead.
func (x *PrmContainerPut) SetContainer(cnr container.Container) {
x.Container = &cnr
}
// WithinSession specifies session within which container should be saved.
//
// Creator of the session acquires the authorship of the request. This affects
// the execution of an operation (e.g. access control).
//
// Session is optional, if set the following requirements apply:
// - session operation MUST be session.VerbContainerPut (ForVerb)
// - token MUST be signed using private key of the owner of the container to be saved
//
// Deprecated: Use PrmContainerPut.Session instead.
func (x *PrmContainerPut) WithinSession(s session.Container) {
x.Session = &s
}
func (x *PrmContainerPut) buildRequest(c *Client) (*v2container.PutRequest, error) {
if x.Container == nil {
return nil, errorMissingContainer
}
if len(x.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
// TODO: check private key is set before forming the request
var cnr v2container.Container
x.Container.WriteToV2(&cnr)
var sig frostfscrypto.Signature
err := container.CalculateSignature(&sig, *x.Container, c.prm.Key)
if err != nil {
return nil, fmt.Errorf("calculate container signature: %w", err)
}
var sigv2 refs.Signature
sig.WriteToV2(&sigv2)
reqBody := new(v2container.PutRequestBody)
reqBody.SetContainer(&cnr)
reqBody.SetSignature(&sigv2)
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(x.XHeaders, &meta)
if x.Session != nil {
var tokv2 v2session.Token
x.Session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
var req v2container.PutRequest
req.SetBody(reqBody)
c.prepareRequest(&req, &meta)
return &req, nil
}
// ResContainerPut groups resulting values of ContainerPut operation.
type ResContainerPut struct {
statusRes
id cid.ID
}
// ID returns identifier of the container declared to be stored in the system.
// Used as a link to information about the container (in particular, you can
// asynchronously check if the save was successful).
func (x ResContainerPut) ID() cid.ID {
return x.id
}
// ContainerPut sends request to save container in FrostFS.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see ResContainerPut.ID).
//
// Returns an error if parameters are set incorrectly (see PrmContainerPut docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
//
// nolint: funlen
func (c *Client) ContainerPut(ctx context.Context, prm PrmContainerPut) (*ResContainerPut, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.PutContainer(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResContainerPut
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
const fieldCnrID = "container ID"
cidV2 := resp.GetBody().GetContainerID()
if cidV2 == nil {
return &res, newErrMissingResponseField(fieldCnrID)
}
if err := res.id.ReadFromV2(*cidV2); err != nil {
return &res, newErrInvalidResponseField(fieldCnrID, err)
}
return &res, nil
}

View file

@ -47,8 +47,8 @@ Consume custom service of the server:
rpc CustomRPC(CustomRPCRequest) returns (CustomRPCResponse); rpc CustomRPC(CustomRPCRequest) returns (CustomRPCResponse);
} }
import "github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client" import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
import "github.com/TrueCloudLab/frostfs-api-go/v2/rpc/common" import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/common"
req := new(CustomRPCRequest) req := new(CustomRPCRequest)
// ... // ...
@ -72,7 +72,7 @@ for the all operations are write-only and the results of the all operations are
read-only. To be able to override client behavior (e.g. for tests), abstract it read-only. To be able to override client behavior (e.g. for tests), abstract it
with an interface: with an interface:
import "github.com/TrueCloudLab/frostfs-sdk-go/client" import "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client"
type FrostFSClient interface { type FrostFSClient interface {
// Operations according to the application needs // Operations according to the application needs

View file

@ -1,97 +1,76 @@
package client package client
import ( import (
"errors"
"fmt" "fmt"
apistatus "github.com/TrueCloudLab/frostfs-sdk-go/client/status" apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
) )
// unwraps err using errors.Unwrap and returns the result. // wrapsErrType returns true if any error in the error tree of err is of type E.
func unwrapErr(err error) error { func wrapsErrType[E error](err error) bool {
for e := errors.Unwrap(err); e != nil; e = errors.Unwrap(err) { switch e := err.(type) {
err = e case E:
return true
case interface{ Unwrap() error }:
return wrapsErrType[E](e.Unwrap())
case interface{ Unwrap() []error }:
for _, ei := range e.Unwrap() {
if wrapsErrType[E](ei) {
return true
}
}
return false
default:
return false
} }
return err
} }
// IsErrContainerNotFound checks if err corresponds to FrostFS status // IsErrContainerNotFound checks if err corresponds to FrostFS status
// return corresponding to missing container. Supports wrapped errors. // return corresponding to missing container. Supports wrapped errors.
func IsErrContainerNotFound(err error) bool { func IsErrContainerNotFound(err error) bool {
switch unwrapErr(err).(type) { return wrapsErrType[*apistatus.ContainerNotFound](err)
default:
return false
case
apistatus.ContainerNotFound,
*apistatus.ContainerNotFound:
return true
}
} }
// IsErrEACLNotFound checks if err corresponds to FrostFS status // IsErrEACLNotFound checks if err corresponds to FrostFS status
// return corresponding to missing eACL table. Supports wrapped errors. // return corresponding to missing eACL table. Supports wrapped errors.
func IsErrEACLNotFound(err error) bool { func IsErrEACLNotFound(err error) bool {
switch unwrapErr(err).(type) { return wrapsErrType[*apistatus.EACLNotFound](err)
default:
return false
case
apistatus.EACLNotFound,
*apistatus.EACLNotFound:
return true
}
} }
// IsErrObjectNotFound checks if err corresponds to FrostFS status // IsErrObjectNotFound checks if err corresponds to FrostFS status
// return corresponding to missing object. Supports wrapped errors. // return corresponding to missing object. Supports wrapped errors.
func IsErrObjectNotFound(err error) bool { func IsErrObjectNotFound(err error) bool {
switch unwrapErr(err).(type) { return wrapsErrType[*apistatus.ObjectNotFound](err)
default:
return false
case
apistatus.ObjectNotFound,
*apistatus.ObjectNotFound:
return true
}
} }
// IsErrObjectAlreadyRemoved checks if err corresponds to FrostFS status // IsErrObjectAlreadyRemoved checks if err corresponds to FrostFS status
// return corresponding to already removed object. Supports wrapped errors. // return corresponding to already removed object. Supports wrapped errors.
func IsErrObjectAlreadyRemoved(err error) bool { func IsErrObjectAlreadyRemoved(err error) bool {
switch unwrapErr(err).(type) { return wrapsErrType[*apistatus.ObjectAlreadyRemoved](err)
default:
return false
case
apistatus.ObjectAlreadyRemoved,
*apistatus.ObjectAlreadyRemoved:
return true
}
} }
// IsErrSessionExpired checks if err corresponds to FrostFS status return // IsErrSessionExpired checks if err corresponds to FrostFS status return
// corresponding to expired session. Supports wrapped errors. // corresponding to expired session. Supports wrapped errors.
func IsErrSessionExpired(err error) bool { func IsErrSessionExpired(err error) bool {
switch unwrapErr(err).(type) { return wrapsErrType[*apistatus.SessionTokenExpired](err)
default:
return false
case
apistatus.SessionTokenExpired,
*apistatus.SessionTokenExpired:
return true
}
} }
// IsErrSessionNotFound checks if err corresponds to FrostFS status return // IsErrSessionNotFound checks if err corresponds to FrostFS status return
// corresponding to missing session. Supports wrapped errors. // corresponding to missing session. Supports wrapped errors.
func IsErrSessionNotFound(err error) bool { func IsErrSessionNotFound(err error) bool {
switch unwrapErr(err).(type) { return wrapsErrType[*apistatus.SessionTokenNotFound](err)
default: }
return false
case // IsErrAPEManagerAccessDenied checks if err corresponds to FrostFS status return
apistatus.SessionTokenNotFound, // corresponding to apemanager access deny. Supports wrapped errors.
*apistatus.SessionTokenNotFound: func IsErrAPEManagerAccessDenied(err error) bool {
return true return wrapsErrType[*apistatus.APEManagerAccessDenied](err)
} }
// IsErrNodeUnderMaintenance checks if err corresponds to FrostFS status return
// corresponding to nodes being under maintenance. Supports wrapped errors.
func IsErrNodeUnderMaintenance(err error) bool {
return wrapsErrType[*apistatus.NodeUnderMaintenance](err)
} }
// returns error describing missing field with the given name. // returns error describing missing field with the given name.

View file

@ -4,65 +4,51 @@ import (
"fmt" "fmt"
"testing" "testing"
"github.com/TrueCloudLab/frostfs-sdk-go/client" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client"
apistatus "github.com/TrueCloudLab/frostfs-sdk-go/client/status" apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
func TestErrors(t *testing.T) { func TestErrors(t *testing.T) {
for _, tc := range []struct { errs := []struct {
check func(error) bool check func(error) bool
errs []error err error
}{ }{
{ {
check: client.IsErrContainerNotFound, check: client.IsErrContainerNotFound,
errs: []error{ err: new(apistatus.ContainerNotFound),
apistatus.ContainerNotFound{},
new(apistatus.ContainerNotFound),
},
}, },
{ {
check: client.IsErrEACLNotFound, check: client.IsErrEACLNotFound,
errs: []error{ err: new(apistatus.EACLNotFound),
apistatus.EACLNotFound{},
new(apistatus.EACLNotFound),
},
}, },
{ {
check: client.IsErrObjectNotFound, check: client.IsErrObjectNotFound,
errs: []error{ err: new(apistatus.ObjectNotFound),
apistatus.ObjectNotFound{},
new(apistatus.ObjectNotFound),
},
}, },
{ {
check: client.IsErrObjectAlreadyRemoved, check: client.IsErrObjectAlreadyRemoved,
errs: []error{ err: new(apistatus.ObjectAlreadyRemoved),
apistatus.ObjectAlreadyRemoved{},
new(apistatus.ObjectAlreadyRemoved),
},
}, },
{ {
check: client.IsErrSessionExpired, check: client.IsErrSessionExpired,
errs: []error{ err: new(apistatus.SessionTokenExpired),
apistatus.SessionTokenExpired{},
new(apistatus.SessionTokenExpired),
},
}, {
check: client.IsErrSessionNotFound,
errs: []error{
apistatus.SessionTokenNotFound{},
new(apistatus.SessionTokenNotFound),
},
}, },
} { {
require.NotEmpty(t, tc.errs) check: client.IsErrSessionNotFound,
err: new(apistatus.SessionTokenNotFound),
},
{
check: client.IsErrNodeUnderMaintenance,
err: new(apistatus.NodeUnderMaintenance),
},
}
for i := range tc.errs { for i := range errs {
require.True(t, tc.check(tc.errs[i]), tc.errs[i]) for j := range errs {
require.True(t, tc.check(fmt.Errorf("top-level context: :%w", nestedErr := fmt.Errorf("top-level context: :%w", fmt.Errorf("inner context: %w", errs[j].err))
fmt.Errorf("inner context: %w", tc.errs[i])), require.Equal(t, i == j, errs[i].check(errs[j].err))
), tc.errs[i]) require.Equal(t, i == j, errs[i].check(nestedErr))
} }
} }
} }

View file

@ -4,19 +4,29 @@ import (
"context" "context"
"fmt" "fmt"
v2netmap "github.com/TrueCloudLab/frostfs-api-go/v2/netmap" v2netmap "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
rpcapi "github.com/TrueCloudLab/frostfs-api-go/v2/rpc" rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "github.com/TrueCloudLab/frostfs-api-go/v2/session" v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"github.com/TrueCloudLab/frostfs-api-go/v2/signature" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "github.com/TrueCloudLab/frostfs-sdk-go/client/status" apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"github.com/TrueCloudLab/frostfs-sdk-go/netmap" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap"
"github.com/TrueCloudLab/frostfs-sdk-go/version" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/version"
) )
// PrmEndpointInfo groups parameters of EndpointInfo operation. // PrmEndpointInfo groups parameters of EndpointInfo operation.
type PrmEndpointInfo struct { type PrmEndpointInfo struct {
prmCommonMeta XHeaders []string
}
func (x *PrmEndpointInfo) buildRequest(c *Client) (*v2netmap.LocalNodeInfoRequest, error) {
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(x.XHeaders, meta)
req := new(v2netmap.LocalNodeInfoRequest)
req.SetBody(new(v2netmap.LocalNodeInfoRequestBody))
c.prepareRequest(req, meta)
return req, nil
} }
// ResEndpointInfo group resulting values of EndpointInfo operation. // ResEndpointInfo group resulting values of EndpointInfo operation.
@ -43,9 +53,9 @@ func (x ResEndpointInfo) NodeInfo() netmap.NodeInfo {
// Method can be used as a health check to see if node is alive and responds to requests. // Method can be used as a health check to see if node is alive and responds to requests.
// //
// Any client's internal or transport errors are returned as `error`. // Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful // If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included // FrostFS status codes are included in the returned result structure,
// in the returned result structure. // otherwise, are also returned as `error`.
// //
// Returns an error if parameters are set incorrectly (see PrmEndpointInfo docs). // Returns an error if parameters are set incorrectly (see PrmEndpointInfo docs).
// Context is required and must not be nil. It is used for network communication. // Context is required and must not be nil. It is used for network communication.
@ -56,73 +66,63 @@ func (x ResEndpointInfo) NodeInfo() netmap.NodeInfo {
// Return statuses: // Return statuses:
// - global (see Client docs). // - global (see Client docs).
func (c *Client) EndpointInfo(ctx context.Context, prm PrmEndpointInfo) (*ResEndpointInfo, error) { func (c *Client) EndpointInfo(ctx context.Context, prm PrmEndpointInfo) (*ResEndpointInfo, error) {
// check context req, err := prm.buildRequest(c)
if ctx == nil { if err != nil {
return nil, errorMissingContext return nil, err
} }
// form request if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
var req v2netmap.LocalNodeInfoRequest return nil, fmt.Errorf("sign request: %w", err)
// init call context
var (
cc contextCall
res ResEndpointInfo
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.LocalNodeInfo(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2netmap.LocalNodeInfoResponse)
body := resp.GetBody()
const fieldVersion = "version"
verV2 := body.GetVersion()
if verV2 == nil {
cc.err = newErrMissingResponseField(fieldVersion)
return
}
cc.err = res.version.ReadFromV2(*verV2)
if cc.err != nil {
cc.err = newErrInvalidResponseField(fieldVersion, cc.err)
return
}
const fieldNodeInfo = "node info"
nodeInfoV2 := body.GetNodeInfo()
if nodeInfoV2 == nil {
cc.err = newErrMissingResponseField(fieldNodeInfo)
return
}
cc.err = res.ni.ReadFromV2(*nodeInfoV2)
if cc.err != nil {
cc.err = newErrInvalidResponseField(fieldNodeInfo, cc.err)
return
}
} }
// process call resp, err := rpcapi.LocalNodeInfo(&c.c, req, client.WithContext(ctx))
if !cc.processCall() { if err != nil {
return nil, cc.err return nil, err
} }
var res ResEndpointInfo
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
body := resp.GetBody()
const fieldVersion = "version"
verV2 := body.GetVersion()
if verV2 == nil {
return nil, newErrMissingResponseField(fieldVersion)
}
if err := res.version.ReadFromV2(*verV2); err != nil {
return nil, newErrInvalidResponseField(fieldVersion, err)
}
const fieldNodeInfo = "node info"
nodeInfoV2 := body.GetNodeInfo()
if nodeInfoV2 == nil {
return nil, newErrMissingResponseField(fieldNodeInfo)
}
if err := res.ni.ReadFromV2(*nodeInfoV2); err != nil {
return nil, newErrInvalidResponseField(fieldNodeInfo, err)
}
return &res, nil return &res, nil
} }
// PrmNetworkInfo groups parameters of NetworkInfo operation. // PrmNetworkInfo groups parameters of NetworkInfo operation.
type PrmNetworkInfo struct { type PrmNetworkInfo struct {
prmCommonMeta XHeaders []string
}
func (x PrmNetworkInfo) buildRequest(c *Client) (*v2netmap.NetworkInfoRequest, error) {
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(x.XHeaders, meta)
var req v2netmap.NetworkInfoRequest
req.SetBody(new(v2netmap.NetworkInfoRequestBody))
c.prepareRequest(&req, meta)
return &req, nil
} }
// ResNetworkInfo groups resulting values of NetworkInfo operation. // ResNetworkInfo groups resulting values of NetworkInfo operation.
@ -140,9 +140,9 @@ func (x ResNetworkInfo) Info() netmap.NetworkInfo {
// NetworkInfo requests information about the FrostFS network of which the remote server is a part. // NetworkInfo requests information about the FrostFS network of which the remote server is a part.
// //
// Any client's internal or transport errors are returned as `error`. // Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful // If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included // FrostFS status codes are included in the returned result structure,
// in the returned result structure. // otherwise, are also returned as `error`.
// //
// Returns an error if parameters are set incorrectly (see PrmNetworkInfo docs). // Returns an error if parameters are set incorrectly (see PrmNetworkInfo docs).
// Context is required and must not be nil. It is used for network communication. // Context is required and must not be nil. It is used for network communication.
@ -153,57 +153,40 @@ func (x ResNetworkInfo) Info() netmap.NetworkInfo {
// Return statuses: // Return statuses:
// - global (see Client docs). // - global (see Client docs).
func (c *Client) NetworkInfo(ctx context.Context, prm PrmNetworkInfo) (*ResNetworkInfo, error) { func (c *Client) NetworkInfo(ctx context.Context, prm PrmNetworkInfo) (*ResNetworkInfo, error) {
// check context req, err := prm.buildRequest(c)
if ctx == nil { if err != nil {
return nil, errorMissingContext return nil, err
} }
// form request if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
var req v2netmap.NetworkInfoRequest return nil, fmt.Errorf("sign request: %w", err)
// init call context
var (
cc contextCall
res ResNetworkInfo
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.NetworkInfo(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2netmap.NetworkInfoResponse)
const fieldNetInfo = "network info"
netInfoV2 := resp.GetBody().GetNetworkInfo()
if netInfoV2 == nil {
cc.err = newErrMissingResponseField(fieldNetInfo)
return
}
cc.err = res.info.ReadFromV2(*netInfoV2)
if cc.err != nil {
cc.err = newErrInvalidResponseField(fieldNetInfo, cc.err)
return
}
} }
// process call resp, err := rpcapi.NetworkInfo(&c.c, req, client.WithContext(ctx))
if !cc.processCall() { if err != nil {
return nil, cc.err return nil, err
} }
var res ResNetworkInfo
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
const fieldNetInfo = "network info"
netInfoV2 := resp.GetBody().GetNetworkInfo()
if netInfoV2 == nil {
return nil, newErrMissingResponseField(fieldNetInfo)
}
if err := res.info.ReadFromV2(*netInfoV2); err != nil {
return nil, newErrInvalidResponseField(fieldNetInfo, err)
}
return &res, nil return &res, nil
} }
// PrmNetMapSnapshot groups parameters of NetMapSnapshot operation. // PrmNetMapSnapshot groups parameters of NetMapSnapshot operation.
type PrmNetMapSnapshot struct { type PrmNetMapSnapshot struct{}
}
// ResNetMapSnapshot groups resulting values of NetMapSnapshot operation. // ResNetMapSnapshot groups resulting values of NetMapSnapshot operation.
type ResNetMapSnapshot struct { type ResNetMapSnapshot struct {
@ -220,9 +203,9 @@ func (x ResNetMapSnapshot) NetMap() netmap.NetMap {
// NetMapSnapshot requests current network view of the remote server. // NetMapSnapshot requests current network view of the remote server.
// //
// Any client's internal or transport errors are returned as `error`. // Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful // If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included // FrostFS status codes are included in the returned result structure,
// in the returned result structure. // otherwise, are also returned as `error`.
// //
// Returns an error if parameters are set incorrectly. // Returns an error if parameters are set incorrectly.
// Context is required and MUST NOT be nil. It is used for network communication. // Context is required and MUST NOT be nil. It is used for network communication.
@ -233,11 +216,6 @@ func (x ResNetMapSnapshot) NetMap() netmap.NetMap {
// Return statuses: // Return statuses:
// - global (see Client docs). // - global (see Client docs).
func (c *Client) NetMapSnapshot(ctx context.Context, _ PrmNetMapSnapshot) (*ResNetMapSnapshot, error) { func (c *Client) NetMapSnapshot(ctx context.Context, _ PrmNetMapSnapshot) (*ResNetMapSnapshot, error) {
// check context
if ctx == nil {
return nil, errorMissingContext
}
// form request body // form request body
var body v2netmap.SnapshotRequestBody var body v2netmap.SnapshotRequestBody
@ -249,7 +227,7 @@ func (c *Client) NetMapSnapshot(ctx context.Context, _ PrmNetMapSnapshot) (*ResN
req.SetBody(&body) req.SetBody(&body)
c.prepareRequest(&req, &meta) c.prepareRequest(&req, &meta)
err := signature.SignServiceMessage(&c.prm.key, &req) err := signature.SignServiceMessage(&c.prm.Key, &req)
if err != nil { if err != nil {
return nil, fmt.Errorf("sign request: %w", err) return nil, fmt.Errorf("sign request: %w", err)
} }
@ -261,12 +239,8 @@ func (c *Client) NetMapSnapshot(ctx context.Context, _ PrmNetMapSnapshot) (*ResN
var res ResNetMapSnapshot var res ResNetMapSnapshot
res.st, err = c.processResponse(resp) res.st, err = c.processResponse(resp)
if err != nil { if err != nil || !apistatus.IsSuccessful(res.st) {
return nil, err return &res, err
}
if !apistatus.IsSuccessful(res.st) {
return &res, nil
} }
const fieldNetMap = "network map" const fieldNetMap = "network map"

View file

@ -6,11 +6,11 @@ import (
"fmt" "fmt"
"testing" "testing"
v2netmap "github.com/TrueCloudLab/frostfs-api-go/v2/netmap" v2netmap "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
"github.com/TrueCloudLab/frostfs-api-go/v2/session" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"github.com/TrueCloudLab/frostfs-api-go/v2/signature" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "github.com/TrueCloudLab/frostfs-sdk-go/client/status" apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"github.com/TrueCloudLab/frostfs-sdk-go/netmap" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
@ -67,13 +67,9 @@ func TestClient_NetMapSnapshot(t *testing.T) {
var res *ResNetMapSnapshot var res *ResNetMapSnapshot
var srv serverNetMap var srv serverNetMap
c := newClient(&srv) c := newClient(&srv)
c.prm.DisableFrostFSFailuresResolution()
ctx := context.Background() ctx := context.Background()
// missing context
//nolint:staticcheck
_, err = c.NetMapSnapshot(nil, prm)
require.ErrorIs(t, err, errorMissingContext, "")
// request signature // request signature
srv.errTransport = errors.New("any error") srv.errTransport = errors.New("any error")

View file

@ -5,87 +5,41 @@ import (
"crypto/ecdsa" "crypto/ecdsa"
"fmt" "fmt"
"github.com/TrueCloudLab/frostfs-api-go/v2/acl" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "github.com/TrueCloudLab/frostfs-api-go/v2/object" v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
v2refs "github.com/TrueCloudLab/frostfs-api-go/v2/refs" v2refs "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "github.com/TrueCloudLab/frostfs-api-go/v2/rpc" rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "github.com/TrueCloudLab/frostfs-api-go/v2/session" v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"github.com/TrueCloudLab/frostfs-api-go/v2/signature" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"github.com/TrueCloudLab/frostfs-sdk-go/bearer" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
apistatus "github.com/TrueCloudLab/frostfs-sdk-go/client/status" apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
cid "github.com/TrueCloudLab/frostfs-sdk-go/container/id" cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
oid "github.com/TrueCloudLab/frostfs-sdk-go/object/id" oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"github.com/TrueCloudLab/frostfs-sdk-go/session" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
) )
// PrmObjectDelete groups parameters of ObjectDelete operation. // PrmObjectDelete groups parameters of ObjectDelete operation.
type PrmObjectDelete struct { type PrmObjectDelete struct {
meta v2session.RequestMetaHeader XHeaders []string
body v2object.DeleteRequestBody BearerToken *bearer.Token
addr v2refs.Address Session *session.Object
keySet bool ContainerID *cid.ID
key ecdsa.PrivateKey
}
// WithinSession specifies session within which object should be read. ObjectID *oid.ID
//
// Creator of the session acquires the authorship of the request.
// This may affect the execution of an operation (e.g. access control).
//
// Must be signed.
func (x *PrmObjectDelete) WithinSession(t session.Object) {
var tv2 v2session.Token
t.WriteToV2(&tv2)
x.meta.SetSessionToken(&tv2) Key *ecdsa.PrivateKey
}
// WithBearerToken attaches bearer token to be used for the operation.
//
// If set, underlying eACL rules will be used in access control.
//
// Must be signed.
func (x *PrmObjectDelete) WithBearerToken(t bearer.Token) {
var v2token acl.BearerToken
t.WriteToV2(&v2token)
x.meta.SetBearerToken(&v2token)
}
// FromContainer specifies FrostFS container of the object.
// Required parameter.
func (x *PrmObjectDelete) FromContainer(id cid.ID) {
var cidV2 v2refs.ContainerID
id.WriteToV2(&cidV2)
x.addr.SetContainerID(&cidV2)
}
// ByID specifies identifier of the requested object.
// Required parameter.
func (x *PrmObjectDelete) ByID(id oid.ID) {
var idV2 v2refs.ObjectID
id.WriteToV2(&idV2)
x.addr.SetObjectID(&idV2)
} }
// UseKey specifies private key to sign the requests. // UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used. // If key is not provided, then Client default key is used.
func (x *PrmObjectDelete) UseKey(key ecdsa.PrivateKey) {
x.keySet = true
x.key = key
}
// WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length.
// //
// Slice must not be mutated until the operation completes. // Deprecated: Use PrmObjectDelete.Key instead.
func (x *PrmObjectDelete) WithXHeaders(hs ...string) { func (prm *PrmObjectDelete) UseKey(key ecdsa.PrivateKey) {
writeXHeadersToMeta(hs, &x.meta) prm.Key = &key
} }
// ResObjectDelete groups resulting values of ObjectDelete operation. // ResObjectDelete groups resulting values of ObjectDelete operation.
@ -100,6 +54,54 @@ func (x ResObjectDelete) Tombstone() oid.ID {
return x.tomb return x.tomb
} }
func (prm *PrmObjectDelete) buildRequest(c *Client) (*v2object.DeleteRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if prm.ObjectID == nil {
return nil, errorMissingObject
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
addr := new(v2refs.Address)
cnrV2 := new(v2refs.ContainerID)
prm.ContainerID.WriteToV2(cnrV2)
addr.SetContainerID(cnrV2)
objV2 := new(v2refs.ObjectID)
prm.ObjectID.WriteToV2(objV2)
addr.SetObjectID(objV2)
body := new(v2object.DeleteRequestBody)
body.SetAddress(addr)
req := new(v2object.DeleteRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// ObjectDelete marks an object for deletion from the container using FrostFS API protocol. // ObjectDelete marks an object for deletion from the container using FrostFS API protocol.
// As a marker, a special unit called a tombstone is placed in the container. // As a marker, a special unit called a tombstone is placed in the container.
// It confirms the user's intent to delete the object, and is itself a container object. // It confirms the user's intent to delete the object, and is itself a container object.
@ -110,9 +112,9 @@ func (x ResObjectDelete) Tombstone() oid.ID {
// //
// Exactly one return value is non-nil. By default, server status is returned in res structure. // Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`, // Any client's internal or transport errors are returned as `error`,
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful // If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included // FrostFS status codes are included in the returned result structure,
// in the returned result structure. // otherwise, are also returned as `error`.
// //
// Returns an error if parameters are set incorrectly (see PrmObjectDelete docs). // Returns an error if parameters are set incorrectly (see PrmObjectDelete docs).
// Context is required and must not be nil. It is used for network communication. // Context is required and must not be nil. It is used for network communication.
@ -124,46 +126,30 @@ func (x ResObjectDelete) Tombstone() oid.ID {
// - *apistatus.ObjectLocked; // - *apistatus.ObjectLocked;
// - *apistatus.SessionTokenExpired. // - *apistatus.SessionTokenExpired.
func (c *Client) ObjectDelete(ctx context.Context, prm PrmObjectDelete) (*ResObjectDelete, error) { func (c *Client) ObjectDelete(ctx context.Context, prm PrmObjectDelete) (*ResObjectDelete, error) {
switch { req, err := prm.buildRequest(c)
case ctx == nil: if err != nil {
return nil, errorMissingContext return nil, err
case prm.addr.GetContainerID() == nil:
return nil, errorMissingContainer
case prm.addr.GetObjectID() == nil:
return nil, errorMissingObject
} }
// form request body key := c.prm.Key
prm.body.SetAddress(&prm.addr) if prm.Key != nil {
key = *prm.Key
// form request
var req v2object.DeleteRequest
req.SetBody(&prm.body)
c.prepareRequest(&req, &prm.meta)
key := c.prm.key
if prm.keySet {
key = prm.key
} }
err := signature.SignServiceMessage(&key, &req) err = signature.SignServiceMessage(&key, req)
if err != nil { if err != nil {
return nil, fmt.Errorf("sign request: %w", err) return nil, fmt.Errorf("sign request: %w", err)
} }
resp, err := rpcapi.DeleteObject(&c.c, &req, client.WithContext(ctx)) resp, err := rpcapi.DeleteObject(&c.c, req, client.WithContext(ctx))
if err != nil { if err != nil {
return nil, err return nil, err
} }
var res ResObjectDelete var res ResObjectDelete
res.st, err = c.processResponse(resp) res.st, err = c.processResponse(resp)
if err != nil { if err != nil || !apistatus.IsSuccessful(res.st) {
return nil, err return &res, err
}
if !apistatus.IsSuccessful(res.st) {
return &res, nil
} }
const fieldTombstone = "tombstone" const fieldTombstone = "tombstone"

View file

@ -7,92 +7,91 @@ import (
"fmt" "fmt"
"io" "io"
"github.com/TrueCloudLab/frostfs-api-go/v2/acl" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "github.com/TrueCloudLab/frostfs-api-go/v2/object" v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
v2refs "github.com/TrueCloudLab/frostfs-api-go/v2/refs" v2refs "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "github.com/TrueCloudLab/frostfs-api-go/v2/rpc" rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "github.com/TrueCloudLab/frostfs-api-go/v2/session" v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"github.com/TrueCloudLab/frostfs-api-go/v2/signature" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"github.com/TrueCloudLab/frostfs-sdk-go/bearer" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
apistatus "github.com/TrueCloudLab/frostfs-sdk-go/client/status" apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
cid "github.com/TrueCloudLab/frostfs-sdk-go/container/id" cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"github.com/TrueCloudLab/frostfs-sdk-go/object" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "github.com/TrueCloudLab/frostfs-sdk-go/object/id" oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"github.com/TrueCloudLab/frostfs-sdk-go/session" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
) )
// shared parameters of GET/HEAD/RANGE.
type prmObjectRead struct {
meta v2session.RequestMetaHeader
raw bool
addr v2refs.Address
}
// WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length.
//
// Slice must not be mutated until the operation completes.
func (x *prmObjectRead) WithXHeaders(hs ...string) {
writeXHeadersToMeta(hs, &x.meta)
}
// MarkRaw marks an intent to read physically stored object.
func (x *prmObjectRead) MarkRaw() {
x.raw = true
}
// MarkLocal tells the server to execute the operation locally.
func (x *prmObjectRead) MarkLocal() {
x.meta.SetTTL(1)
}
// WithinSession specifies session within which object should be read.
//
// Creator of the session acquires the authorship of the request.
// This may affect the execution of an operation (e.g. access control).
//
// Must be signed.
func (x *prmObjectRead) WithinSession(t session.Object) {
var tokv2 v2session.Token
t.WriteToV2(&tokv2)
x.meta.SetSessionToken(&tokv2)
}
// WithBearerToken attaches bearer token to be used for the operation.
//
// If set, underlying eACL rules will be used in access control.
//
// Must be signed.
func (x *prmObjectRead) WithBearerToken(t bearer.Token) {
var v2token acl.BearerToken
t.WriteToV2(&v2token)
x.meta.SetBearerToken(&v2token)
}
// FromContainer specifies FrostFS container of the object.
// Required parameter.
func (x *prmObjectRead) FromContainer(id cid.ID) {
var cnrV2 v2refs.ContainerID
id.WriteToV2(&cnrV2)
x.addr.SetContainerID(&cnrV2)
}
// ByID specifies identifier of the requested object.
// Required parameter.
func (x *prmObjectRead) ByID(id oid.ID) {
var objV2 v2refs.ObjectID
id.WriteToV2(&objV2)
x.addr.SetObjectID(&objV2)
}
// PrmObjectGet groups parameters of ObjectGetInit operation. // PrmObjectGet groups parameters of ObjectGetInit operation.
type PrmObjectGet struct { type PrmObjectGet struct {
prmObjectRead XHeaders []string
key *ecdsa.PrivateKey BearerToken *bearer.Token
Session *session.Object
Raw bool
Local bool
ContainerID *cid.ID
ObjectID *oid.ID
Key *ecdsa.PrivateKey
}
func (prm *PrmObjectGet) buildRequest(c *Client) (*v2object.GetRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if prm.ObjectID == nil {
return nil, errorMissingObject
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if prm.Local {
meta.SetTTL(1)
}
addr := new(v2refs.Address)
cnrV2 := new(v2refs.ContainerID)
prm.ContainerID.WriteToV2(cnrV2)
addr.SetContainerID(cnrV2)
objV2 := new(v2refs.ObjectID)
prm.ObjectID.WriteToV2(objV2)
addr.SetObjectID(objV2)
body := new(v2object.GetRequestBody)
body.SetRaw(prm.Raw)
body.SetAddress(addr)
req := new(v2object.GetRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
} }
// ResObjectGet groups the final result values of ObjectGetInit operation. // ResObjectGet groups the final result values of ObjectGetInit operation.
@ -122,8 +121,10 @@ type ObjectReader struct {
// UseKey specifies private key to sign the requests. // UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used. // If key is not provided, then Client default key is used.
func (x *PrmObjectGet) UseKey(key ecdsa.PrivateKey) { //
x.key = &key // Deprecated: Use PrmObjectGet.Key instead.
func (prm *PrmObjectGet) UseKey(key ecdsa.PrivateKey) {
prm.Key = &key
} }
// ReadHeader reads header of the object. Result means success. // ReadHeader reads header of the object. Result means success.
@ -149,6 +150,9 @@ func (x *ObjectReader) ReadHeader(dst *object.Object) bool {
case *v2object.SplitInfo: case *v2object.SplitInfo:
x.err = object.NewSplitInfoError(object.NewSplitInfoFromV2(v)) x.err = object.NewSplitInfoError(object.NewSplitInfoFromV2(v))
return false return false
case *v2object.ECInfo:
x.err = object.NewECInfoError(object.NewECInfoFromV2(v))
return false
case *v2object.GetObjectPartInit: case *v2object.GetObjectPartInit:
partInit = v partInit = v
} }
@ -257,6 +261,7 @@ func (x *ObjectReader) close(ignoreEOF bool) (*ResObjectGet, error) {
// Return errors: // Return errors:
// //
// *object.SplitInfoError (returned on virtual objects with PrmObjectGet.MakeRaw). // *object.SplitInfoError (returned on virtual objects with PrmObjectGet.MakeRaw).
// *object.ECInfoError (returned on erasure-coded objects with PrmObjectGet.MakeRaw).
// //
// Return statuses: // Return statuses:
// - global (see Client docs); // - global (see Client docs);
@ -299,41 +304,24 @@ func (x *ObjectReader) Read(p []byte) (int, error) {
// Returns an error if parameters are set incorrectly (see PrmObjectGet docs). // Returns an error if parameters are set incorrectly (see PrmObjectGet docs).
// Context is required and must not be nil. It is used for network communication. // Context is required and must not be nil. It is used for network communication.
func (c *Client) ObjectGetInit(ctx context.Context, prm PrmObjectGet) (*ObjectReader, error) { func (c *Client) ObjectGetInit(ctx context.Context, prm PrmObjectGet) (*ObjectReader, error) {
// check parameters req, err := prm.buildRequest(c)
switch { if err != nil {
case ctx == nil: return nil, err
return nil, errorMissingContext
case prm.addr.GetContainerID() == nil:
return nil, errorMissingContainer
case prm.addr.GetObjectID() == nil:
return nil, errorMissingObject
} }
// form request body key := prm.Key
var body v2object.GetRequestBody
body.SetRaw(prm.raw)
body.SetAddress(&prm.addr)
// form request
var req v2object.GetRequest
req.SetBody(&body)
c.prepareRequest(&req, &prm.meta)
key := prm.key
if key == nil { if key == nil {
key = &c.prm.key key = &c.prm.Key
} }
err := signature.SignServiceMessage(key, &req) err = signature.SignServiceMessage(key, req)
if err != nil { if err != nil {
return nil, fmt.Errorf("sign request: %w", err) return nil, fmt.Errorf("sign request: %w", err)
} }
ctx, cancel := context.WithCancel(ctx) ctx, cancel := context.WithCancel(ctx)
stream, err := rpcapi.GetObject(&c.c, &req, client.WithContext(ctx)) stream, err := rpcapi.GetObject(&c.c, req, client.WithContext(ctx))
if err != nil { if err != nil {
cancel() cancel()
return nil, fmt.Errorf("open stream: %w", err) return nil, fmt.Errorf("open stream: %w", err)
@ -349,17 +337,29 @@ func (c *Client) ObjectGetInit(ctx context.Context, prm PrmObjectGet) (*ObjectRe
// PrmObjectHead groups parameters of ObjectHead operation. // PrmObjectHead groups parameters of ObjectHead operation.
type PrmObjectHead struct { type PrmObjectHead struct {
prmObjectRead XHeaders []string
keySet bool BearerToken *bearer.Token
key ecdsa.PrivateKey
Session *session.Object
Raw bool
Local bool
ContainerID *cid.ID
ObjectID *oid.ID
Key *ecdsa.PrivateKey
} }
// UseKey specifies private key to sign the requests. // UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used. // If key is not provided, then Client default key is used.
func (x *PrmObjectHead) UseKey(key ecdsa.PrivateKey) { //
x.keySet = true // Deprecated: Use PrmObjectHead.Key instead.
x.key = key func (prm *PrmObjectHead) UseKey(key ecdsa.PrivateKey) {
prm.Key = &key
} }
// ResObjectHead groups resulting values of ObjectHead operation. // ResObjectHead groups resulting values of ObjectHead operation.
@ -392,13 +392,65 @@ func (x *ResObjectHead) ReadHeader(dst *object.Object) bool {
return true return true
} }
func (prm *PrmObjectHead) buildRequest(c *Client) (*v2object.HeadRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if prm.ObjectID == nil {
return nil, errorMissingObject
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if prm.Local {
meta.SetTTL(1)
}
addr := new(v2refs.Address)
cnrV2 := new(v2refs.ContainerID)
prm.ContainerID.WriteToV2(cnrV2)
addr.SetContainerID(cnrV2)
objV2 := new(v2refs.ObjectID)
prm.ObjectID.WriteToV2(objV2)
addr.SetObjectID(objV2)
body := new(v2object.HeadRequestBody)
body.SetRaw(prm.Raw)
body.SetAddress(addr)
req := new(v2object.HeadRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// ObjectHead reads object header through a remote server using FrostFS API protocol. // ObjectHead reads object header through a remote server using FrostFS API protocol.
// //
// Exactly one return value is non-nil. By default, server status is returned in res structure. // Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`, // Any client's internal or transport errors are returned as `error`,
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful // If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included // FrostFS status codes are included in the returned result structure,
// in the returned result structure. // otherwise, are also returned as `error`.
// //
// Returns an error if parameters are set incorrectly (see PrmObjectHead docs). // Returns an error if parameters are set incorrectly (see PrmObjectHead docs).
// Context is required and must not be nil. It is used for network communication. // Context is required and must not be nil. It is used for network communication.
@ -406,6 +458,7 @@ func (x *ResObjectHead) ReadHeader(dst *object.Object) bool {
// Return errors: // Return errors:
// //
// *object.SplitInfoError (returned on virtual objects with PrmObjectHead.MakeRaw). // *object.SplitInfoError (returned on virtual objects with PrmObjectHead.MakeRaw).
// *object.ECInfoError (returned on erasure-coded objects with PrmObjectHead.MakeRaw).
// //
// Return statuses: // Return statuses:
// - global (see Client docs); // - global (see Client docs);
@ -415,56 +468,43 @@ func (x *ResObjectHead) ReadHeader(dst *object.Object) bool {
// - *apistatus.ObjectAlreadyRemoved; // - *apistatus.ObjectAlreadyRemoved;
// - *apistatus.SessionTokenExpired. // - *apistatus.SessionTokenExpired.
func (c *Client) ObjectHead(ctx context.Context, prm PrmObjectHead) (*ResObjectHead, error) { func (c *Client) ObjectHead(ctx context.Context, prm PrmObjectHead) (*ResObjectHead, error) {
switch { req, err := prm.buildRequest(c)
case ctx == nil: if err != nil {
return nil, errorMissingContext return nil, err
case prm.addr.GetContainerID() == nil:
return nil, errorMissingContainer
case prm.addr.GetObjectID() == nil:
return nil, errorMissingObject
} }
var body v2object.HeadRequestBody key := c.prm.Key
body.SetRaw(prm.raw) if prm.Key != nil {
body.SetAddress(&prm.addr) key = *prm.Key
var req v2object.HeadRequest
req.SetBody(&body)
c.prepareRequest(&req, &prm.meta)
key := c.prm.key
if prm.keySet {
key = prm.key
} }
// sign the request // sign the request
err := signature.SignServiceMessage(&key, &req)
err = signature.SignServiceMessage(&key, req)
if err != nil { if err != nil {
return nil, fmt.Errorf("sign request: %w", err) return nil, fmt.Errorf("sign request: %w", err)
} }
resp, err := rpcapi.HeadObject(&c.c, &req, client.WithContext(ctx)) resp, err := rpcapi.HeadObject(&c.c, req, client.WithContext(ctx))
if err != nil { if err != nil {
return nil, fmt.Errorf("write request: %w", err) return nil, fmt.Errorf("write request: %w", err)
} }
var res ResObjectHead var res ResObjectHead
res.st, err = c.processResponse(resp) res.st, err = c.processResponse(resp)
if err != nil { if err != nil || !apistatus.IsSuccessful(res.st) {
return nil, err return &res, err
} }
if !apistatus.IsSuccessful(res.st) { res.idObj = *prm.ObjectID
return &res, nil
}
_ = res.idObj.ReadFromV2(*prm.addr.GetObjectID())
switch v := resp.GetBody().GetHeaderPart().(type) { switch v := resp.GetBody().GetHeaderPart().(type) {
default: default:
return nil, fmt.Errorf("unexpected header type %T", v) return nil, fmt.Errorf("unexpected header type %T", v)
case *v2object.SplitInfo: case *v2object.SplitInfo:
return nil, object.NewSplitInfoError(object.NewSplitInfoFromV2(v)) return nil, object.NewSplitInfoError(object.NewSplitInfoFromV2(v))
case *v2object.ECInfo:
return nil, object.NewECInfoError(object.NewECInfoFromV2(v))
case *v2object.HeaderWithSignature: case *v2object.HeaderWithSignature:
res.hdr = v res.hdr = v
} }
@ -474,29 +514,95 @@ func (c *Client) ObjectHead(ctx context.Context, prm PrmObjectHead) (*ResObjectH
// PrmObjectRange groups parameters of ObjectRange operation. // PrmObjectRange groups parameters of ObjectRange operation.
type PrmObjectRange struct { type PrmObjectRange struct {
prmObjectRead XHeaders []string
key *ecdsa.PrivateKey BearerToken *bearer.Token
rng v2object.Range Session *session.Object
Raw bool
Local bool
ContainerID *cid.ID
ObjectID *oid.ID
Key *ecdsa.PrivateKey
Offset uint64
Length uint64
} }
// SetOffset sets offset of the payload range to be read. func (prm *PrmObjectRange) buildRequest(c *Client) (*v2object.GetRangeRequest, error) {
// Zero by default. if prm.Length == 0 {
func (x *PrmObjectRange) SetOffset(off uint64) { return nil, errorZeroRangeLength
x.rng.SetOffset(off) }
}
// SetLength sets length of the payload range to be read. if prm.ContainerID == nil {
// Must be positive. return nil, errorMissingContainer
func (x *PrmObjectRange) SetLength(ln uint64) { }
x.rng.SetLength(ln)
if prm.ObjectID == nil {
return nil, errorMissingObject
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if prm.Local {
meta.SetTTL(1)
}
addr := new(v2refs.Address)
cnrV2 := new(v2refs.ContainerID)
prm.ContainerID.WriteToV2(cnrV2)
addr.SetContainerID(cnrV2)
objV2 := new(v2refs.ObjectID)
prm.ObjectID.WriteToV2(objV2)
addr.SetObjectID(objV2)
rng := new(v2object.Range)
rng.SetLength(prm.Length)
rng.SetOffset(prm.Offset)
body := new(v2object.GetRangeRequestBody)
body.SetRaw(prm.Raw)
body.SetAddress(addr)
body.SetRange(rng)
req := new(v2object.GetRangeRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
} }
// UseKey specifies private key to sign the requests. // UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used. // If key is not provided, then Client default key is used.
func (x *PrmObjectRange) UseKey(key ecdsa.PrivateKey) { //
x.key = &key // Deprecated: Use PrmObjectRange.Key instead.
func (prm *PrmObjectRange) UseKey(key ecdsa.PrivateKey) {
prm.Key = &key
} }
// ResObjectRange groups the final result values of ObjectRange operation. // ResObjectRange groups the final result values of ObjectRange operation.
@ -562,6 +668,9 @@ func (x *ObjectRangeReader) readChunk(buf []byte) (int, bool) {
case *v2object.SplitInfo: case *v2object.SplitInfo:
x.err = object.NewSplitInfoError(object.NewSplitInfoFromV2(v)) x.err = object.NewSplitInfoError(object.NewSplitInfoFromV2(v))
return read, false return read, false
case *v2object.ECInfo:
x.err = object.NewECInfoError(object.NewECInfoFromV2(v))
return read, false
case *v2object.GetRangePartChunk: case *v2object.GetRangePartChunk:
partChunk = v partChunk = v
} }
@ -622,6 +731,7 @@ func (x *ObjectRangeReader) close(ignoreEOF bool) (*ResObjectRange, error) {
// Return errors: // Return errors:
// //
// *object.SplitInfoError (returned on virtual objects with PrmObjectRange.MakeRaw). // *object.SplitInfoError (returned on virtual objects with PrmObjectRange.MakeRaw).
// *object.ECInfoError (returned on erasure-coded objects with PrmObjectRange.MakeRaw).
// //
// Return statuses: // Return statuses:
// - global (see Client docs); // - global (see Client docs);
@ -666,51 +776,31 @@ func (x *ObjectRangeReader) Read(p []byte) (int, error) {
// Returns an error if parameters are set incorrectly (see PrmObjectRange docs). // Returns an error if parameters are set incorrectly (see PrmObjectRange docs).
// Context is required and must not be nil. It is used for network communication. // Context is required and must not be nil. It is used for network communication.
func (c *Client) ObjectRangeInit(ctx context.Context, prm PrmObjectRange) (*ObjectRangeReader, error) { func (c *Client) ObjectRangeInit(ctx context.Context, prm PrmObjectRange) (*ObjectRangeReader, error) {
// check parameters req, err := prm.buildRequest(c)
switch { if err != nil {
case ctx == nil: return nil, err
return nil, errorMissingContext
case prm.addr.GetContainerID() == nil:
return nil, errorMissingContainer
case prm.addr.GetObjectID() == nil:
return nil, errorMissingObject
case prm.rng.GetLength() == 0:
return nil, errorZeroRangeLength
} }
// form request body key := prm.Key
var body v2object.GetRangeRequestBody
body.SetRaw(prm.raw)
body.SetAddress(&prm.addr)
body.SetRange(&prm.rng)
// form request
var req v2object.GetRangeRequest
req.SetBody(&body)
c.prepareRequest(&req, &prm.meta)
key := prm.key
if key == nil { if key == nil {
key = &c.prm.key key = &c.prm.Key
} }
err := signature.SignServiceMessage(key, &req) err = signature.SignServiceMessage(key, req)
if err != nil { if err != nil {
return nil, fmt.Errorf("sign request: %w", err) return nil, fmt.Errorf("sign request: %w", err)
} }
ctx, cancel := context.WithCancel(ctx) ctx, cancel := context.WithCancel(ctx)
stream, err := rpcapi.GetObjectRange(&c.c, &req, client.WithContext(ctx)) stream, err := rpcapi.GetObjectRange(&c.c, req, client.WithContext(ctx))
if err != nil { if err != nil {
cancel() cancel()
return nil, fmt.Errorf("open stream: %w", err) return nil, fmt.Errorf("open stream: %w", err)
} }
var r ObjectRangeReader var r ObjectRangeReader
r.remainingPayloadLen = int(prm.rng.GetLength()) r.remainingPayloadLen = int(prm.Length)
r.cancelCtxStream = cancel r.cancelCtxStream = cancel
r.stream = stream r.stream = stream
r.client = c r.client = c

View file

@ -5,129 +5,61 @@ import (
"crypto/ecdsa" "crypto/ecdsa"
"fmt" "fmt"
"github.com/TrueCloudLab/frostfs-api-go/v2/acl" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "github.com/TrueCloudLab/frostfs-api-go/v2/object" v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
v2refs "github.com/TrueCloudLab/frostfs-api-go/v2/refs" v2refs "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "github.com/TrueCloudLab/frostfs-api-go/v2/rpc" rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "github.com/TrueCloudLab/frostfs-api-go/v2/session" v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"github.com/TrueCloudLab/frostfs-api-go/v2/signature" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"github.com/TrueCloudLab/frostfs-sdk-go/bearer" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
apistatus "github.com/TrueCloudLab/frostfs-sdk-go/client/status" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/checksum"
cid "github.com/TrueCloudLab/frostfs-sdk-go/container/id" apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
oid "github.com/TrueCloudLab/frostfs-sdk-go/object/id" cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"github.com/TrueCloudLab/frostfs-sdk-go/session" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
) )
// PrmObjectHash groups parameters of ObjectHash operation. // PrmObjectHash groups parameters of ObjectHash operation.
type PrmObjectHash struct { type PrmObjectHash struct {
meta v2session.RequestMetaHeader XHeaders []string
body v2object.GetRangeHashRequestBody BearerToken *bearer.Token
csAlgo v2refs.ChecksumType Session *session.Object
addr v2refs.Address Local bool
keySet bool Ranges []object.Range
key ecdsa.PrivateKey
Salt []byte
ChecksumType checksum.Type
ContainerID *cid.ID
ObjectID *oid.ID
Key *ecdsa.PrivateKey
} }
// UseKey specifies private key to sign the requests. // UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used. // If key is not provided, then Client default key is used.
func (x *PrmObjectHash) UseKey(key ecdsa.PrivateKey) {
x.keySet = true
x.key = key
}
// MarkLocal tells the server to execute the operation locally.
func (x *PrmObjectHash) MarkLocal() {
x.meta.SetTTL(1)
}
// WithinSession specifies session within which object should be read.
// //
// Creator of the session acquires the authorship of the request. // Deprecated: Use PrmObjectHash.Key instead.
// This may affect the execution of an operation (e.g. access control). func (prm *PrmObjectHash) UseKey(key ecdsa.PrivateKey) {
// prm.Key = &key
// Must be signed.
func (x *PrmObjectHash) WithinSession(t session.Object) {
var tv2 v2session.Token
t.WriteToV2(&tv2)
x.meta.SetSessionToken(&tv2)
}
// WithBearerToken attaches bearer token to be used for the operation.
//
// If set, underlying eACL rules will be used in access control.
//
// Must be signed.
func (x *PrmObjectHash) WithBearerToken(t bearer.Token) {
var v2token acl.BearerToken
t.WriteToV2(&v2token)
x.meta.SetBearerToken(&v2token)
}
// FromContainer specifies FrostFS container of the object.
// Required parameter.
func (x *PrmObjectHash) FromContainer(id cid.ID) {
var cidV2 v2refs.ContainerID
id.WriteToV2(&cidV2)
x.addr.SetContainerID(&cidV2)
}
// ByID specifies identifier of the requested object.
// Required parameter.
func (x *PrmObjectHash) ByID(id oid.ID) {
var idV2 v2refs.ObjectID
id.WriteToV2(&idV2)
x.addr.SetObjectID(&idV2)
}
// SetRangeList sets list of ranges in (offset, length) pair format.
// Required parameter.
//
// If passed as slice, then it must not be mutated before the operation completes.
func (x *PrmObjectHash) SetRangeList(r ...uint64) {
ln := len(r)
if ln%2 != 0 {
panic("odd number of range parameters")
}
rs := make([]v2object.Range, ln/2)
for i := 0; i < ln/2; i++ {
rs[i].SetOffset(r[2*i])
rs[i].SetLength(r[2*i+1])
}
x.body.SetRanges(rs)
} }
// TillichZemorAlgo changes the hash function to Tillich-Zemor // TillichZemorAlgo changes the hash function to Tillich-Zemor
// (https://link.springer.com/content/pdf/10.1007/3-540-48658-5_5.pdf). // (https://link.springer.com/content/pdf/10.1007/3-540-48658-5_5.pdf).
// //
// By default, SHA256 hash function is used. // By default, SHA256 hash function is used/.
func (x *PrmObjectHash) TillichZemorAlgo() {
x.csAlgo = v2refs.TillichZemor
}
// UseSalt sets the salt to XOR the data range before hashing.
// //
// Must not be mutated before the operation completes. // Deprecated: Use PrmObjectHash.ChecksumType instead.
func (x *PrmObjectHash) UseSalt(salt []byte) { func (prm *PrmObjectHash) TillichZemorAlgo() {
x.body.SetSalt(salt) prm.ChecksumType = checksum.TZ
}
// WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length.
//
// Slice must not be mutated until the operation completes.
func (x *PrmObjectHash) WithXHeaders(hs ...string) {
writeXHeadersToMeta(hs, &x.meta)
} }
// ResObjectHash groups resulting values of ObjectHash operation. // ResObjectHash groups resulting values of ObjectHash operation.
@ -142,6 +74,76 @@ func (x ResObjectHash) Checksums() [][]byte {
return x.checksums return x.checksums
} }
func (prm *PrmObjectHash) buildRequest(c *Client) (*v2object.GetRangeHashRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if prm.ObjectID == nil {
return nil, errorMissingObject
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
if len(prm.Ranges) == 0 {
return nil, errorMissingRanges
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if prm.Local {
meta.SetTTL(1)
}
addr := new(v2refs.Address)
cnrV2 := new(v2refs.ContainerID)
prm.ContainerID.WriteToV2(cnrV2)
addr.SetContainerID(cnrV2)
objV2 := new(v2refs.ObjectID)
prm.ObjectID.WriteToV2(objV2)
addr.SetObjectID(objV2)
rs := make([]v2object.Range, len(prm.Ranges))
for i := range prm.Ranges {
rs[i].SetOffset(prm.Ranges[i].GetOffset())
rs[i].SetLength(prm.Ranges[i].GetLength())
}
body := new(v2object.GetRangeHashRequestBody)
body.SetAddress(addr)
body.SetRanges(rs)
body.SetSalt(prm.Salt)
if prm.ChecksumType == checksum.Unknown {
body.SetType(v2refs.SHA256)
} else {
body.SetType(v2refs.ChecksumType(prm.ChecksumType))
}
req := new(v2object.GetRangeHashRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// ObjectHash requests checksum of the range list of the object payload using // ObjectHash requests checksum of the range list of the object payload using
// FrostFS API protocol. // FrostFS API protocol.
// //
@ -150,9 +152,9 @@ func (x ResObjectHash) Checksums() [][]byte {
// //
// Exactly one return value is non-nil. By default, server status is returned in res structure. // Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`, // Any client's internal or transport errors are returned as `error`,
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful // If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included // FrostFS status codes are included in the returned result structure,
// in the returned result structure. // otherwise, are also returned as `error`.
// //
// Returns an error if parameters are set incorrectly (see PrmObjectHash docs). // Returns an error if parameters are set incorrectly (see PrmObjectHash docs).
// Context is required and must not be nil. It is used for network communication. // Context is required and must not be nil. It is used for network communication.
@ -165,51 +167,30 @@ func (x ResObjectHash) Checksums() [][]byte {
// - *apistatus.ObjectOutOfRange; // - *apistatus.ObjectOutOfRange;
// - *apistatus.SessionTokenExpired. // - *apistatus.SessionTokenExpired.
func (c *Client) ObjectHash(ctx context.Context, prm PrmObjectHash) (*ResObjectHash, error) { func (c *Client) ObjectHash(ctx context.Context, prm PrmObjectHash) (*ResObjectHash, error) {
switch { req, err := prm.buildRequest(c)
case ctx == nil: if err != nil {
return nil, errorMissingContext return nil, err
case prm.addr.GetContainerID() == nil:
return nil, errorMissingContainer
case prm.addr.GetObjectID() == nil:
return nil, errorMissingObject
case len(prm.body.GetRanges()) == 0:
return nil, errorMissingRanges
} }
prm.body.SetAddress(&prm.addr) key := c.prm.Key
if prm.csAlgo == v2refs.UnknownChecksum { if prm.Key != nil {
prm.body.SetType(v2refs.SHA256) key = *prm.Key
} else {
prm.body.SetType(prm.csAlgo)
} }
var req v2object.GetRangeHashRequest err = signature.SignServiceMessage(&key, req)
c.prepareRequest(&req, &prm.meta)
req.SetBody(&prm.body)
key := c.prm.key
if prm.keySet {
key = prm.key
}
err := signature.SignServiceMessage(&key, &req)
if err != nil { if err != nil {
return nil, fmt.Errorf("sign request: %w", err) return nil, fmt.Errorf("sign request: %w", err)
} }
resp, err := rpcapi.HashObjectRange(&c.c, &req, client.WithContext(ctx)) resp, err := rpcapi.HashObjectRange(&c.c, req, client.WithContext(ctx))
if err != nil { if err != nil {
return nil, fmt.Errorf("write request: %w", err) return nil, fmt.Errorf("write request: %w", err)
} }
var res ResObjectHash var res ResObjectHash
res.st, err = c.processResponse(resp) res.st, err = c.processResponse(resp)
if err != nil { if err != nil || !apistatus.IsSuccessful(res.st) {
return nil, err return &res, err
}
if !apistatus.IsSuccessful(res.st) {
return &res, nil
} }
res.checksums = resp.GetBody().GetHashList() res.checksums = resp.GetBody().GetHashList()

266
client/object_patch.go Normal file
View file

@ -0,0 +1,266 @@
package client
import (
"context"
"crypto/ecdsa"
"errors"
"fmt"
"io"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// ObjectPatcher is designed to patch an object.
//
// Must be initialized using Client.ObjectPatchInit, any other
// usage is unsafe.
type ObjectPatcher interface {
// PatchAttributes patches attributes. Attributes can be patched no more than once,
// otherwise, the server returns an error.
//
// Result means success. Failure reason can be received via Close.
PatchAttributes(ctx context.Context, newAttrs []object.Attribute, replace bool) bool
// PatchPayload patches the object's payload.
//
// PatchPayload receives `payloadReader` and thus the payload of the patch is read and sent by chunks of
// `MaxChunkLength` length.
//
// Result means success. Failure reason can be received via Close.
PatchPayload(ctx context.Context, rng *object.Range, payloadReader io.Reader) bool
// Close ends patching the object and returns the result of the operation
// along with the final results. Must be called after using the ObjectPatcher.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as Go built-in error.
// If Client is tuned to resolve FrostFS API statuses, then FrostFS failures
// codes are returned as error.
//
// Return statuses:
// - global (see Client docs);
// - *apistatus.ContainerNotFound;
// - *apistatus.ContainerAccessDenied;
// - *apistatus.ObjectAccessDenied;
// - *apistatus.ObjectAlreadyRemoved;
// - *apistatus.ObjectLocked;
// - *apistatus.ObjectOutOfRange;
// - *apistatus.SessionTokenNotFound;
// - *apistatus.SessionTokenExpired.
Close(_ context.Context) (*ResObjectPatch, error)
}
// ResObjectPatch groups resulting values of ObjectPatch operation.
type ResObjectPatch struct {
statusRes
obj oid.ID
}
// ObjectID returns an object ID of the patched object.
func (r ResObjectPatch) ObjectID() oid.ID {
return r.obj
}
// PrmObjectPatch groups parameters of ObjectPatch operation.
type PrmObjectPatch struct {
XHeaders []string
Address oid.Address
BearerToken *bearer.Token
Session *session.Object
Key *ecdsa.PrivateKey
MaxChunkLength int
}
// ObjectPatchInit initializes object patcher.
func (c *Client) ObjectPatchInit(ctx context.Context, prm PrmObjectPatch) (ObjectPatcher, error) {
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
var objectPatcher objectPatcher
stream, err := rpcapi.Patch(&c.c, &objectPatcher.respV2, client.WithContext(ctx))
if err != nil {
return nil, fmt.Errorf("open stream: %w", err)
}
objectPatcher.addr = prm.Address
objectPatcher.key = &c.prm.Key
if prm.Key != nil {
objectPatcher.key = prm.Key
}
objectPatcher.client = c
objectPatcher.stream = stream
if prm.MaxChunkLength > 0 {
objectPatcher.maxChunkLen = prm.MaxChunkLength
} else {
objectPatcher.maxChunkLen = defaultGRPCPayloadChunkLen
}
objectPatcher.req.SetBody(&v2object.PatchRequestBody{})
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
c.prepareRequest(&objectPatcher.req, meta)
return &objectPatcher, nil
}
type objectPatcher struct {
client *Client
stream interface {
Write(*v2object.PatchRequest) error
Close() error
}
key *ecdsa.PrivateKey
res ResObjectPatch
err error
addr oid.Address
req v2object.PatchRequest
respV2 v2object.PatchResponse
maxChunkLen int
}
func (x *objectPatcher) PatchAttributes(_ context.Context, newAttrs []object.Attribute, replace bool) bool {
return x.patch(&object.Patch{
Address: x.addr,
NewAttributes: newAttrs,
ReplaceAttributes: replace,
})
}
func (x *objectPatcher) PatchPayload(_ context.Context, rng *object.Range, payloadReader io.Reader) bool {
offset := rng.GetOffset()
buf := make([]byte, x.maxChunkLen)
for patchIter := 0; ; patchIter++ {
n, err := payloadReader.Read(buf)
if err != nil && err != io.EOF {
x.err = fmt.Errorf("read payload: %w", err)
return false
}
if n == 0 {
if patchIter == 0 {
if rng.GetLength() == 0 {
x.err = errors.New("zero-length empty payload patch can't be applied")
return false
}
if !x.patch(&object.Patch{
Address: x.addr,
PayloadPatch: &object.PayloadPatch{
Range: rng,
Chunk: []byte{},
},
}) {
return false
}
}
break
}
rngPart := object.NewRange()
if patchIter == 0 {
rngPart.SetOffset(offset)
rngPart.SetLength(rng.GetLength())
} else {
rngPart.SetOffset(offset + rng.GetLength())
}
if !x.patch(&object.Patch{
Address: x.addr,
PayloadPatch: &object.PayloadPatch{
Range: rngPart,
Chunk: buf[:n],
},
}) {
return false
}
if err == io.EOF {
break
}
}
return true
}
func (x *objectPatcher) patch(patch *object.Patch) bool {
x.req.SetBody(patch.ToV2())
x.req.SetVerificationHeader(nil)
x.err = signature.SignServiceMessage(x.key, &x.req)
if x.err != nil {
x.err = fmt.Errorf("sign message: %w", x.err)
return false
}
x.err = x.stream.Write(&x.req)
return x.err == nil
}
func (x *objectPatcher) Close(_ context.Context) (*ResObjectPatch, error) {
// Ignore io.EOF error, because it is expected error for client-side
// stream termination by the server. E.g. when stream contains invalid
// message. Server returns an error in response message (in status).
if x.err != nil && !errors.Is(x.err, io.EOF) {
return nil, x.err
}
if x.err = x.stream.Close(); x.err != nil {
return nil, x.err
}
x.res.st, x.err = x.client.processResponse(&x.respV2)
if x.err != nil || !apistatus.IsSuccessful(x.res.st) {
return &x.res, x.err
}
const fieldID = "ID"
idV2 := x.respV2.Body.ObjectID
if idV2 == nil {
return nil, newErrMissingResponseField(fieldID)
}
x.err = x.res.obj.ReadFromV2(*idV2)
if x.err != nil {
x.err = newErrInvalidResponseField(fieldID, x.err)
}
return &x.res, nil
}

295
client/object_patch_test.go Normal file
View file

@ -0,0 +1,295 @@
package client
import (
"bytes"
"context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"testing"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id/test"
"github.com/stretchr/testify/require"
)
type mockPatchStream struct {
streamedPayloadPatches []*object.PayloadPatch
}
func (m *mockPatchStream) Write(r *v2object.PatchRequest) error {
pp := new(object.PayloadPatch)
pp.FromV2(r.GetBody().GetPatch())
if r.GetBody().GetPatch() != nil {
bodyChunk := r.GetBody().GetPatch().Chunk
pp.Chunk = make([]byte, len(bodyChunk))
copy(pp.Chunk, bodyChunk)
}
m.streamedPayloadPatches = append(m.streamedPayloadPatches, pp)
return nil
}
func (m *mockPatchStream) Close() error {
return nil
}
func TestObjectPatcher(t *testing.T) {
type part struct {
offset int
length int
chunk string
}
for _, test := range []struct {
name string
patchPayload string
rng *object.Range
maxChunkLen int
expectParts []part
}{
{
name: "no split payload patch",
patchPayload: "011111",
rng: newRange(0, 6),
maxChunkLen: defaultGRPCPayloadChunkLen,
expectParts: []part{
{
offset: 0,
length: 6,
chunk: "011111",
},
},
},
{
name: "splitted payload patch",
patchPayload: "012345",
rng: newRange(0, 6),
maxChunkLen: 2,
expectParts: []part{
{
offset: 0,
length: 6,
chunk: "01",
},
{
offset: 6,
length: 0,
chunk: "23",
},
{
offset: 6,
length: 0,
chunk: "45",
},
},
},
{
name: "splitted payload patch with zero-length subpatches",
patchPayload: "0123456789!@",
rng: newRange(0, 4),
maxChunkLen: 2,
expectParts: []part{
{
offset: 0,
length: 4,
chunk: "01",
},
{
offset: 4,
length: 0,
chunk: "23",
},
{
offset: 4,
length: 0,
chunk: "45",
},
{
offset: 4,
length: 0,
chunk: "67",
},
{
offset: 4,
length: 0,
chunk: "89",
},
{
offset: 4,
length: 0,
chunk: "!@",
},
},
},
{
name: "splitted payload patch with zero-length subpatches only",
patchPayload: "0123456789!@",
rng: newRange(0, 0),
maxChunkLen: 2,
expectParts: []part{
{
offset: 0,
length: 0,
chunk: "01",
},
{
offset: 0,
length: 0,
chunk: "23",
},
{
offset: 0,
length: 0,
chunk: "45",
},
{
offset: 0,
length: 0,
chunk: "67",
},
{
offset: 0,
length: 0,
chunk: "89",
},
{
offset: 0,
length: 0,
chunk: "!@",
},
},
},
} {
t.Run(test.name, func(t *testing.T) {
m := &mockPatchStream{}
pk, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
patcher := objectPatcher{
client: &Client{},
stream: m,
addr: oidtest.Address(),
key: pk,
maxChunkLen: test.maxChunkLen,
}
success := patcher.PatchAttributes(context.Background(), nil, false)
require.True(t, success)
success = patcher.PatchPayload(context.Background(), test.rng, bytes.NewReader([]byte(test.patchPayload)))
require.True(t, success)
require.Len(t, m.streamedPayloadPatches, len(test.expectParts)+1)
// m.streamedPayloadPatches[0] is attribute patch, so skip it
for i, part := range test.expectParts {
requireRangeChunk(t, m.streamedPayloadPatches[i+1], part.offset, part.length, part.chunk)
}
})
}
}
func TestRepeatPayloadPatch(t *testing.T) {
t.Run("no payload patch partioning", func(t *testing.T) {
m := &mockPatchStream{}
pk, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
const maxChunkLen = 20
patcher := objectPatcher{
client: &Client{},
stream: m,
addr: oidtest.Address(),
key: pk,
maxChunkLen: maxChunkLen,
}
for _, pp := range []struct {
patchPayload string
rng *object.Range
}{
{
patchPayload: "xxxxxxxxxx",
rng: newRange(1, 6),
},
{
patchPayload: "yyyyyyyyyy",
rng: newRange(5, 9),
},
{
patchPayload: "zzzzzzzzzz",
rng: newRange(10, 0),
},
} {
success := patcher.PatchPayload(context.Background(), pp.rng, bytes.NewReader([]byte(pp.patchPayload)))
require.True(t, success)
}
requireRangeChunk(t, m.streamedPayloadPatches[0], 1, 6, "xxxxxxxxxx")
requireRangeChunk(t, m.streamedPayloadPatches[1], 5, 9, "yyyyyyyyyy")
requireRangeChunk(t, m.streamedPayloadPatches[2], 10, 0, "zzzzzzzzzz")
})
t.Run("payload patch partioning", func(t *testing.T) {
m := &mockPatchStream{}
pk, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
const maxChunkLen = 5
patcher := objectPatcher{
client: &Client{},
stream: m,
addr: oidtest.Address(),
key: pk,
maxChunkLen: maxChunkLen,
}
for _, pp := range []struct {
patchPayload string
rng *object.Range
}{
{
patchPayload: "xxxxxxxxxx",
rng: newRange(1, 6),
},
{
patchPayload: "yyyyyyyyyy",
rng: newRange(5, 9),
},
{
patchPayload: "zzzzzzzzzz",
rng: newRange(10, 0),
},
} {
success := patcher.PatchPayload(context.Background(), pp.rng, bytes.NewReader([]byte(pp.patchPayload)))
require.True(t, success)
}
requireRangeChunk(t, m.streamedPayloadPatches[0], 1, 6, "xxxxx")
requireRangeChunk(t, m.streamedPayloadPatches[1], 7, 0, "xxxxx")
requireRangeChunk(t, m.streamedPayloadPatches[2], 5, 9, "yyyyy")
requireRangeChunk(t, m.streamedPayloadPatches[3], 14, 0, "yyyyy")
requireRangeChunk(t, m.streamedPayloadPatches[4], 10, 0, "zzzzz")
requireRangeChunk(t, m.streamedPayloadPatches[5], 10, 0, "zzzzz")
})
}
func requireRangeChunk(t *testing.T, pp *object.PayloadPatch, offset, length int, chunk string) {
require.NotNil(t, pp)
require.Equal(t, uint64(offset), pp.Range.GetOffset())
require.Equal(t, uint64(length), pp.Range.GetLength())
require.Equal(t, []byte(chunk), pp.Chunk)
}
func newRange(offest, length uint64) *object.Range {
rng := &object.Range{}
rng.SetOffset(offest)
rng.SetLength(length)
return rng
}

View file

@ -3,40 +3,77 @@ package client
import ( import (
"context" "context"
"crypto/ecdsa" "crypto/ecdsa"
"errors"
"fmt"
"io"
"github.com/TrueCloudLab/frostfs-api-go/v2/acl" buffPool "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/util/pool"
v2object "github.com/TrueCloudLab/frostfs-api-go/v2/object" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
rpcapi "github.com/TrueCloudLab/frostfs-api-go/v2/rpc" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
"github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client" oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
v2session "github.com/TrueCloudLab/frostfs-api-go/v2/session" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer"
"github.com/TrueCloudLab/frostfs-api-go/v2/signature" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
"github.com/TrueCloudLab/frostfs-sdk-go/bearer"
apistatus "github.com/TrueCloudLab/frostfs-sdk-go/client/status"
"github.com/TrueCloudLab/frostfs-sdk-go/object"
oid "github.com/TrueCloudLab/frostfs-sdk-go/object/id"
"github.com/TrueCloudLab/frostfs-sdk-go/session"
) )
// defaultGRPCPayloadChunkLen default value for maxChunkLen.
// See PrmObjectPutInit.SetGRPCPayloadChunkLen for details.
const defaultGRPCPayloadChunkLen = 3 << 20
// PrmObjectPutInit groups parameters of ObjectPutInit operation. // PrmObjectPutInit groups parameters of ObjectPutInit operation.
type PrmObjectPutInit struct { type PrmObjectPutInit struct {
copyNum uint32 XHeaders []string
key *ecdsa.PrivateKey
meta v2session.RequestMetaHeader BearerToken *bearer.Token
Session *session.Object
Local bool
CopiesNumber []uint32
MaxChunkLength int
MaxSize uint64
EpochSource transformer.EpochSource
WithoutHomomorphHash bool
Key *ecdsa.PrivateKey
Pool *buffPool.BufferPool
} }
// SetCopiesNumber sets number of object copies that is enough to consider put successful. // SetCopiesNumber sets number of object copies that is enough to consider put successful.
//
// Deprecated: Use PrmObjectPutInit.CopiesNumber instead.
func (x *PrmObjectPutInit) SetCopiesNumber(copiesNumber uint32) { func (x *PrmObjectPutInit) SetCopiesNumber(copiesNumber uint32) {
x.copyNum = copiesNumber x.CopiesNumber = []uint32{copiesNumber}
}
// SetCopiesNumberByVectors sets ordered list of minimal required object copies numbers
// per placement vector. List's length MUST equal container's placement vector number,
// otherwise request will fail.
//
// Deprecated: Use PrmObjectPutInit.CopiesNumber instead.
func (x *PrmObjectPutInit) SetCopiesNumberByVectors(copiesNumbers []uint32) {
x.CopiesNumber = copiesNumbers
}
// SetGRPCPayloadChunkLen sets maximum chunk length value for gRPC Put request.
// Maximum chunk length restricts maximum byte length of the chunk
// transmitted in a single stream message. It depends on
// server settings and other message fields.
// If not specified or negative value set, default value of 3MiB will be used.
//
// Deprecated: Use PrmObjectPutInit.MaxChunkLength instead.
func (x *PrmObjectPutInit) SetGRPCPayloadChunkLen(v int) {
x.MaxChunkLength = v
} }
// ResObjectPut groups the final result values of ObjectPutInit operation. // ResObjectPut groups the final result values of ObjectPutInit operation.
type ResObjectPut struct { type ResObjectPut struct {
statusRes statusRes
obj oid.ID obj oid.ID
epoch uint64
} }
// StoredObjectID returns identifier of the saved object. // StoredObjectID returns identifier of the saved object.
@ -44,190 +81,104 @@ func (x ResObjectPut) StoredObjectID() oid.ID {
return x.obj return x.obj
} }
// ObjectWriter is designed to write one object to FrostFS system. // StoredEpoch returns creation epoch of the saved object.
func (x ResObjectPut) StoredEpoch() uint64 {
return x.epoch
}
// ObjectWriter is designed to write one object or
// multiple parts of one object to FrostFS system.
// //
// Must be initialized using Client.ObjectPutInit, any other // Must be initialized using Client.ObjectPutInit, any other
// usage is unsafe. // usage is unsafe.
type ObjectWriter struct { type ObjectWriter interface {
cancelCtxStream context.CancelFunc // WriteHeader writes header of the object. Result means success.
// Failure reason can be received via Close.
client *Client WriteHeader(context.Context, object.Object) bool
stream interface { // WritePayloadChunk writes chunk of the object payload. Result means success.
Write(*v2object.PutRequest) error // Failure reason can be received via Close.
Close() error WritePayloadChunk(context.Context, []byte) bool
} // Close ends writing the object and returns the result of the operation
// along with the final results. Must be called after using the ObjectWriter.
key *ecdsa.PrivateKey //
res ResObjectPut // Exactly one return value is non-nil. By default, server status is returned in res structure.
err error // Any client's internal or transport errors are returned as Go built-in error.
// If Client is tuned to resolve FrostFS API statuses, then FrostFS failures
chunkCalled bool // codes are returned as error.
//
respV2 v2object.PutResponse // Return statuses:
req v2object.PutRequest // - global (see Client docs);
partInit v2object.PutObjectPartInit // - *apistatus.ContainerNotFound;
partChunk v2object.PutObjectPartChunk // - *apistatus.ObjectAccessDenied;
// - *apistatus.ObjectLocked;
// - *apistatus.LockNonRegularObject;
// - *apistatus.SessionTokenNotFound;
// - *apistatus.SessionTokenExpired.
Close(context.Context) (*ResObjectPut, error)
} }
// UseKey specifies private key to sign the requests. // UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used. // If key is not provided, then Client default key is used.
//
// Deprecated: Use PrmObjectPutInit.Key instead.
func (x *PrmObjectPutInit) UseKey(key ecdsa.PrivateKey) { func (x *PrmObjectPutInit) UseKey(key ecdsa.PrivateKey) {
x.key = &key x.Key = &key
} }
// WithBearerToken attaches bearer token to be used for the operation. // WithBearerToken attaches bearer token to be used for the operation.
// Should be called once before any writing steps. // Should be called once before any writing steps.
//
// Deprecated: Use PrmObjectPutInit.BearerToken instead.
func (x *PrmObjectPutInit) WithBearerToken(t bearer.Token) { func (x *PrmObjectPutInit) WithBearerToken(t bearer.Token) {
var v2token acl.BearerToken x.BearerToken = &t
t.WriteToV2(&v2token)
x.meta.SetBearerToken(&v2token)
} }
// WithinSession specifies session within which object should be stored. // WithinSession specifies session within which object should be stored.
// Should be called once before any writing steps. // Should be called once before any writing steps.
//
// Deprecated: Use PrmObjectPutInit.Session instead.
func (x *PrmObjectPutInit) WithinSession(t session.Object) { func (x *PrmObjectPutInit) WithinSession(t session.Object) {
var tv2 v2session.Token x.Session = &t
t.WriteToV2(&tv2)
x.meta.SetSessionToken(&tv2)
} }
// MarkLocal tells the server to execute the operation locally. // MarkLocal tells the server to execute the operation locally.
//
// Deprecated: Use PrmObjectPutInit.Local instead.
func (x *PrmObjectPutInit) MarkLocal() { func (x *PrmObjectPutInit) MarkLocal() {
x.meta.SetTTL(1) x.Local = true
} }
// WithXHeaders specifies list of extended headers (string key-value pairs) // WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length. // to be attached to the request. Must have an even length.
// //
// Slice must not be mutated until the operation completes. // Slice must not be mutated until the operation completes.
//
// Deprecated: Use PrmObjectPutInit.XHeaders instead.
func (x *PrmObjectPutInit) WithXHeaders(hs ...string) { func (x *PrmObjectPutInit) WithXHeaders(hs ...string) {
writeXHeadersToMeta(hs, &x.meta) x.XHeaders = hs
} }
// WriteHeader writes header of the object. Result means success. // WithObjectMaxSize specifies max object size value and use it during object splitting.
// Failure reason can be received via Close. // When specified, start writing to the stream only after the object is formed.
func (x *ObjectWriter) WriteHeader(hdr object.Object) bool { // Continue processing the input only when the previous formed object has been successfully written.
v2Hdr := hdr.ToV2()
x.partInit.SetObjectID(v2Hdr.GetObjectID())
x.partInit.SetHeader(v2Hdr.GetHeader())
x.partInit.SetSignature(v2Hdr.GetSignature())
x.req.GetBody().SetObjectPart(&x.partInit)
x.req.SetVerificationHeader(nil)
x.err = signature.SignServiceMessage(x.key, &x.req)
if x.err != nil {
x.err = fmt.Errorf("sign message: %w", x.err)
return false
}
x.err = x.stream.Write(&x.req)
return x.err == nil
}
// WritePayloadChunk writes chunk of the object payload. Result means success.
// Failure reason can be received via Close.
func (x *ObjectWriter) WritePayloadChunk(chunk []byte) bool {
if !x.chunkCalled {
x.chunkCalled = true
x.req.GetBody().SetObjectPart(&x.partChunk)
}
for ln := len(chunk); ln > 0; ln = len(chunk) {
// maxChunkLen restricts maximum byte length of the chunk
// transmitted in a single stream message. It depends on
// server settings and other message fields, but for now
// we simply assume that 3MB is large enough to reduce the
// number of messages, and not to exceed the limit
// (4MB by default for gRPC servers).
const maxChunkLen = 3 << 20
if ln > maxChunkLen {
ln = maxChunkLen
}
// we deal with size limit overflow above, but there is another case:
// what if method is called with "small" chunk many times? We write
// a message to the stream on each call. Alternatively, we could use buffering.
// In most cases, the chunk length does not vary between calls. Given this
// assumption, as well as the length of the payload from the header, it is
// possible to buffer the data of intermediate chunks, and send a message when
// the allocated buffer is filled, or when the last chunk is received.
// It is mentally assumed that allocating and filling the buffer is better than
// synchronous sending, but this needs to be tested.
x.partChunk.SetChunk(chunk[:ln])
x.req.SetVerificationHeader(nil)
x.err = signature.SignServiceMessage(x.key, &x.req)
if x.err != nil {
x.err = fmt.Errorf("sign message: %w", x.err)
return false
}
x.err = x.stream.Write(&x.req)
if x.err != nil {
return false
}
chunk = chunk[ln:]
}
return true
}
// Close ends writing the object and returns the result of the operation
// along with the final results. Must be called after using the ObjectWriter.
// //
// Exactly one return value is non-nil. By default, server status is returned in res structure. // Deprecated: Use PrmObjectPutInit.MaxSize instead.
// Any client's internal or transport errors are returned as Go built-in error. func (x *PrmObjectPutInit) WithObjectMaxSize(maxSize uint64) {
// If Client is tuned to resolve FrostFS API statuses, then FrostFS failures x.MaxSize = maxSize
// codes are returned as error. }
// WithoutHomomorphicHash if set to true do not use Tillich-Zémor hash for payload.
// //
// Return statuses: // Deprecated: Use PrmObjectPutInit.WithoutHomomorphHash instead.
// - global (see Client docs); func (x *PrmObjectPutInit) WithoutHomomorphicHash(v bool) {
// - *apistatus.ContainerNotFound; x.WithoutHomomorphHash = v
// - *apistatus.ObjectAccessDenied; }
// - *apistatus.ObjectLocked;
// - *apistatus.LockNonRegularObject;
// - *apistatus.SessionTokenNotFound;
// - *apistatus.SessionTokenExpired.
func (x *ObjectWriter) Close() (*ResObjectPut, error) {
defer x.cancelCtxStream()
// Ignore io.EOF error, because it is expected error for client-side // WithEpochSource specifies epoch for object when split it on client side.
// stream termination by the server. E.g. when stream contains invalid //
// message. Server returns an error in response message (in status). // Deprecated: Use PrmObjectPutInit.EpochSource instead.
if x.err != nil && !errors.Is(x.err, io.EOF) { func (x *PrmObjectPutInit) WithEpochSource(es transformer.EpochSource) {
return nil, x.err x.EpochSource = es
}
if x.err = x.stream.Close(); x.err != nil {
return nil, x.err
}
x.res.st, x.err = x.client.processResponse(&x.respV2)
if x.err != nil {
return nil, x.err
}
if !apistatus.IsSuccessful(x.res.st) {
return &x.res, nil
}
const fieldID = "ID"
idV2 := x.respV2.GetBody().GetObjectID()
if idV2 == nil {
return nil, newErrMissingResponseField(fieldID)
}
x.err = x.res.obj.ReadFromV2(*idV2)
if x.err != nil {
x.err = newErrInvalidResponseField(fieldID, x.err)
}
return &x.res, nil
} }
// ObjectPutInit initiates writing an object through a remote server using FrostFS API protocol. // ObjectPutInit initiates writing an object through a remote server using FrostFS API protocol.
@ -237,31 +188,9 @@ func (x *ObjectWriter) Close() (*ResObjectPut, error) {
// //
// Returns an error if parameters are set incorrectly. // Returns an error if parameters are set incorrectly.
// Context is required and must not be nil. It is used for network communication. // Context is required and must not be nil. It is used for network communication.
func (c *Client) ObjectPutInit(ctx context.Context, prm PrmObjectPutInit) (*ObjectWriter, error) { func (c *Client) ObjectPutInit(ctx context.Context, prm PrmObjectPutInit) (ObjectWriter, error) {
// check parameters if prm.MaxSize > 0 {
if ctx == nil { return c.objectPutInitTransformer(prm)
return nil, errorMissingContext
} }
return c.objectPutInitRaw(ctx, prm)
var w ObjectWriter
ctx, cancel := context.WithCancel(ctx)
stream, err := rpcapi.PutObject(&c.c, &w.respV2, client.WithContext(ctx))
if err != nil {
cancel()
return nil, fmt.Errorf("open stream: %w", err)
}
w.key = &c.prm.key
if prm.key != nil {
w.key = prm.key
}
w.cancelCtxStream = cancel
w.client = c
w.stream = stream
w.partInit.SetCopiesNumber(prm.copyNum)
w.req.SetBody(new(v2object.PutRequestBody))
c.prepareRequest(&w.req, &prm.meta)
return &w, nil
} }

177
client/object_put_raw.go Normal file
View file

@ -0,0 +1,177 @@
package client
import (
"context"
"crypto/ecdsa"
"errors"
"fmt"
"io"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
)
func (c *Client) objectPutInitRaw(ctx context.Context, prm PrmObjectPutInit) (*objectWriterRaw, error) {
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
var w objectWriterRaw
stream, err := rpcapi.PutObject(&c.c, &w.respV2, client.WithContext(ctx))
if err != nil {
return nil, fmt.Errorf("open stream: %w", err)
}
w.key = &c.prm.Key
if prm.Key != nil {
w.key = prm.Key
}
w.client = c
w.stream = stream
w.partInit.SetCopiesNumber(prm.CopiesNumber)
w.req.SetBody(new(v2object.PutRequestBody))
if prm.MaxChunkLength > 0 {
w.maxChunkLen = prm.MaxChunkLength
} else {
w.maxChunkLen = defaultGRPCPayloadChunkLen
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if prm.Local {
meta.SetTTL(1)
}
c.prepareRequest(&w.req, meta)
return &w, nil
}
type objectWriterRaw struct {
client *Client
stream interface {
Write(*v2object.PutRequest) error
Close() error
}
key *ecdsa.PrivateKey
res ResObjectPut
err error
chunkCalled bool
respV2 v2object.PutResponse
req v2object.PutRequest
partInit v2object.PutObjectPartInit
partChunk v2object.PutObjectPartChunk
maxChunkLen int
}
func (x *objectWriterRaw) WriteHeader(_ context.Context, hdr object.Object) bool {
v2Hdr := hdr.ToV2()
x.partInit.SetObjectID(v2Hdr.GetObjectID())
x.partInit.SetHeader(v2Hdr.GetHeader())
x.partInit.SetSignature(v2Hdr.GetSignature())
x.req.GetBody().SetObjectPart(&x.partInit)
x.req.SetVerificationHeader(nil)
x.err = signature.SignServiceMessage(x.key, &x.req)
if x.err != nil {
x.err = fmt.Errorf("sign message: %w", x.err)
return false
}
x.err = x.stream.Write(&x.req)
return x.err == nil
}
func (x *objectWriterRaw) WritePayloadChunk(_ context.Context, chunk []byte) bool {
if !x.chunkCalled {
x.chunkCalled = true
x.req.GetBody().SetObjectPart(&x.partChunk)
}
for ln := len(chunk); ln > 0; ln = len(chunk) {
if ln > x.maxChunkLen {
ln = x.maxChunkLen
}
// we deal with size limit overflow above, but there is another case:
// what if method is called with "small" chunk many times? We write
// a message to the stream on each call. Alternatively, we could use buffering.
// In most cases, the chunk length does not vary between calls. Given this
// assumption, as well as the length of the payload from the header, it is
// possible to buffer the data of intermediate chunks, and send a message when
// the allocated buffer is filled, or when the last chunk is received.
// It is mentally assumed that allocating and filling the buffer is better than
// synchronous sending, but this needs to be tested.
x.partChunk.SetChunk(chunk[:ln])
x.req.SetVerificationHeader(nil)
x.err = signature.SignServiceMessage(x.key, &x.req)
if x.err != nil {
x.err = fmt.Errorf("sign message: %w", x.err)
return false
}
x.err = x.stream.Write(&x.req)
if x.err != nil {
return false
}
chunk = chunk[ln:]
}
return true
}
func (x *objectWriterRaw) Close(_ context.Context) (*ResObjectPut, error) {
// Ignore io.EOF error, because it is expected error for client-side
// stream termination by the server. E.g. when stream contains invalid
// message. Server returns an error in response message (in status).
if x.err != nil && !errors.Is(x.err, io.EOF) {
return nil, x.err
}
if x.err = x.stream.Close(); x.err != nil {
return nil, x.err
}
x.res.st, x.err = x.client.processResponse(&x.respV2)
if x.err != nil || !apistatus.IsSuccessful(x.res.st) {
return &x.res, x.err
}
const fieldID = "ID"
idV2 := x.respV2.GetBody().GetObjectID()
if idV2 == nil {
return nil, newErrMissingResponseField(fieldID)
}
x.err = x.res.obj.ReadFromV2(*idV2)
if x.err != nil {
x.err = newErrInvalidResponseField(fieldID, x.err)
}
x.res.epoch = x.respV2.GetMetaHeader().GetEpoch()
return &x.res, nil
}

175
client/object_put_single.go Normal file
View file

@ -0,0 +1,175 @@
package client
import (
"context"
"crypto/ecdsa"
"fmt"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmObjectPutSingle groups parameters of PutSingle operation.
type PrmObjectPutSingle struct {
XHeaders []string
BearerToken *bearer.Token
Session *session.Object
Local bool
CopiesNumber []uint32
Object *object.Object
Key *ecdsa.PrivateKey
}
// SetCopiesNumber sets ordered list of minimal required object copies numbers
// per placement vector. List's length MUST equal container's placement vector number,
// otherwise request will fail.
//
// Deprecated: Use PrmObjectPutSingle.CopiesNumber instead.
func (prm *PrmObjectPutSingle) SetCopiesNumber(v []uint32) {
prm.CopiesNumber = v
}
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
//
// Deprecated: Use PrmObjectPutSingle.Key instead.
func (prm *PrmObjectPutSingle) UseKey(key *ecdsa.PrivateKey) {
prm.Key = key
}
// WithBearerToken attaches bearer token to be used for the operation.
// Should be called once before any writing steps.
//
// Deprecated: Use PrmObjectPutSingle.BearerToken instead.
func (prm *PrmObjectPutSingle) WithBearerToken(t bearer.Token) {
prm.BearerToken = &t
}
// WithinSession specifies session within which object should be stored.
// Should be called once before any writing steps.
//
// Deprecated: Use PrmObjectPutSingle.Session instead.
func (prm *PrmObjectPutSingle) WithinSession(t session.Object) {
prm.Session = &t
}
// ExecuteLocal tells the server to execute the operation locally.
//
// Deprecated: Use PrmObjectPutSingle.Local instead.
func (prm *PrmObjectPutSingle) ExecuteLocal() {
prm.Local = true
}
// WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length.
//
// Slice must not be mutated until the operation completes.
//
// Deprecated: Use PrmObjectPutSingle.XHeaders instead.
func (prm *PrmObjectPutSingle) WithXHeaders(hs ...string) {
prm.XHeaders = hs
}
// SetObject specifies prepared object to put.
//
// Deprecated: Use PrmObjectPutSingle.Object instead.
func (prm *PrmObjectPutSingle) SetObject(o *v2object.Object) {
prm.Object = object.NewFromV2(o)
}
// ResObjectPutSingle groups resulting values of PutSingle operation.
type ResObjectPutSingle struct {
statusRes
epoch uint64
}
// Epoch returns creation epoch of the saved object.
func (r *ResObjectPutSingle) Epoch() uint64 {
return r.epoch
}
func (prm *PrmObjectPutSingle) buildRequest(c *Client) (*v2object.PutSingleRequest, error) {
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
body := new(v2object.PutSingleRequestBody)
body.SetCopiesNumber(prm.CopiesNumber)
body.SetObject(prm.Object.ToV2())
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if prm.Local {
meta.SetTTL(1)
}
req := &v2object.PutSingleRequest{}
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// ObjectPutSingle writes prepared object to FrostFS.
// Object must have payload, also containerID, objectID, ownerID, payload hash, payload length of an object must be set.
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as Go built-in error.
// If Client is tuned to resolve FrostFS API statuses, then FrostFS failures
// codes are returned as error.
func (c *Client) ObjectPutSingle(ctx context.Context, prm PrmObjectPutSingle) (*ResObjectPutSingle, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
key := &c.prm.Key
if prm.Key != nil {
key = prm.Key
}
err = signature.SignServiceMessage(key, req)
if err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.PutSingleObject(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResObjectPutSingle
res.st, err = c.processResponse(resp)
if err != nil {
return &res, err
}
res.epoch = resp.GetMetaHeader().GetEpoch()
return &res, nil
}

View file

@ -0,0 +1,144 @@
package client
import (
"context"
buffPool "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/util/pool"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
func (c *Client) objectPutInitTransformer(prm PrmObjectPutInit) (*objectWriterTransformer, error) {
var w objectWriterTransformer
w.it = internalTarget{
client: c,
prm: prm,
}
key := &c.prm.Key
if prm.Key != nil {
key = prm.Key
}
w.ot = transformer.NewPayloadSizeLimiter(transformer.Params{
Key: key,
NextTargetInit: func() transformer.ObjectWriter { return &w.it },
MaxSize: prm.MaxSize,
WithoutHomomorphicHash: prm.WithoutHomomorphHash,
NetworkState: prm.EpochSource,
Pool: prm.Pool,
})
return &w, nil
}
type objectWriterTransformer struct {
ot transformer.ChunkedObjectWriter
it internalTarget
err error
}
func (x *objectWriterTransformer) WriteHeader(ctx context.Context, hdr object.Object) bool {
x.err = x.ot.WriteHeader(ctx, &hdr)
return x.err == nil
}
func (x *objectWriterTransformer) WritePayloadChunk(ctx context.Context, chunk []byte) bool {
_, x.err = x.ot.Write(ctx, chunk)
return x.err == nil
}
func (x *objectWriterTransformer) Close(ctx context.Context) (*ResObjectPut, error) {
if x.err != nil {
return nil, x.err
}
ai, err := x.ot.Close(ctx)
if err != nil {
return nil, err
}
if ai != nil {
x.it.res.epoch = ai.Epoch
if ai.ParentID != nil {
x.it.res.obj = *ai.ParentID
}
}
return x.it.res, nil
}
type internalTarget struct {
client *Client
res *ResObjectPut
prm PrmObjectPutInit
useStream bool
}
func (it *internalTarget) WriteObject(ctx context.Context, o *object.Object) error {
putSingleImplemented, err := it.tryPutSingle(ctx, o)
if putSingleImplemented {
return err
}
it.useStream = true
return it.putAsStream(ctx, o)
}
func (it *internalTarget) putAsStream(ctx context.Context, o *object.Object) error {
wrt, err := it.client.objectPutInitRaw(ctx, it.prm)
if err != nil {
return err
}
if wrt.WriteHeader(ctx, *o) {
wrt.WritePayloadChunk(ctx, o.Payload())
}
it.res, err = wrt.Close(ctx)
if err == nil && it.client.prm.DisableFrostFSErrorResolution && !apistatus.IsSuccessful(it.res.st) {
err = apistatus.ErrFromStatus(it.res.st)
}
return err
}
func (it *internalTarget) tryPutSingle(ctx context.Context, o *object.Object) (bool, error) {
if it.useStream {
return false, nil
}
prm := PrmObjectPutSingle{
XHeaders: it.prm.XHeaders,
BearerToken: it.prm.BearerToken,
Session: it.prm.Session,
Local: it.prm.Local,
CopiesNumber: it.prm.CopiesNumber,
Object: o,
Key: it.prm.Key,
}
res, err := it.client.ObjectPutSingle(ctx, prm)
if err != nil && status.Code(err) == codes.Unimplemented {
return false, err
}
if err == nil {
it.returnBuffPool(o.Payload())
id, _ := o.ID()
it.res = &ResObjectPut{
statusRes: res.statusRes,
obj: id,
epoch: res.epoch,
}
if it.client.prm.DisableFrostFSErrorResolution && !apistatus.IsSuccessful(it.res.st) {
return true, apistatus.ErrFromStatus(it.res.st)
}
return true, nil
}
return true, err
}
func (it *internalTarget) returnBuffPool(playback []byte) {
if it.prm.Pool == nil {
return
}
var buffer buffPool.Buffer
buffer.Data = playback
it.prm.Pool.Put(&buffer)
}

View file

@ -7,36 +7,43 @@ import (
"fmt" "fmt"
"io" "io"
"github.com/TrueCloudLab/frostfs-api-go/v2/acl" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "github.com/TrueCloudLab/frostfs-api-go/v2/object" v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
v2refs "github.com/TrueCloudLab/frostfs-api-go/v2/refs" v2refs "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "github.com/TrueCloudLab/frostfs-api-go/v2/rpc" rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "github.com/TrueCloudLab/frostfs-api-go/v2/session" v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"github.com/TrueCloudLab/frostfs-api-go/v2/signature" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"github.com/TrueCloudLab/frostfs-sdk-go/bearer" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
apistatus "github.com/TrueCloudLab/frostfs-sdk-go/client/status" apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
cid "github.com/TrueCloudLab/frostfs-sdk-go/container/id" cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"github.com/TrueCloudLab/frostfs-sdk-go/object" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "github.com/TrueCloudLab/frostfs-sdk-go/object/id" oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"github.com/TrueCloudLab/frostfs-sdk-go/session" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
) )
// PrmObjectSearch groups parameters of ObjectSearch operation. // PrmObjectSearch groups parameters of ObjectSearch operation.
type PrmObjectSearch struct { type PrmObjectSearch struct {
meta v2session.RequestMetaHeader XHeaders []string
key *ecdsa.PrivateKey Local bool
cnrSet bool BearerToken *bearer.Token
cnrID cid.ID
filters object.SearchFilters Session *session.Object
ContainerID *cid.ID
Key *ecdsa.PrivateKey
Filters object.SearchFilters
} }
// MarkLocal tells the server to execute the operation locally. // MarkLocal tells the server to execute the operation locally.
//
// Deprecated: Use PrmObjectSearch.Local instead.
func (x *PrmObjectSearch) MarkLocal() { func (x *PrmObjectSearch) MarkLocal() {
x.meta.SetTTL(1) x.Local = true
} }
// WithinSession specifies session within which the search query must be executed. // WithinSession specifies session within which the search query must be executed.
@ -45,10 +52,10 @@ func (x *PrmObjectSearch) MarkLocal() {
// This may affect the execution of an operation (e.g. access control). // This may affect the execution of an operation (e.g. access control).
// //
// Must be signed. // Must be signed.
//
// Deprecated: Use PrmObjectSearch.Session instead.
func (x *PrmObjectSearch) WithinSession(t session.Object) { func (x *PrmObjectSearch) WithinSession(t session.Object) {
var tokv2 v2session.Token x.Session = &t
t.WriteToV2(&tokv2)
x.meta.SetSessionToken(&tokv2)
} }
// WithBearerToken attaches bearer token to be used for the operation. // WithBearerToken attaches bearer token to be used for the operation.
@ -56,37 +63,44 @@ func (x *PrmObjectSearch) WithinSession(t session.Object) {
// If set, underlying eACL rules will be used in access control. // If set, underlying eACL rules will be used in access control.
// //
// Must be signed. // Must be signed.
//
// Deprecated: Use PrmObjectSearch.BearerToken instead.
func (x *PrmObjectSearch) WithBearerToken(t bearer.Token) { func (x *PrmObjectSearch) WithBearerToken(t bearer.Token) {
var v2token acl.BearerToken x.BearerToken = &t
t.WriteToV2(&v2token)
x.meta.SetBearerToken(&v2token)
} }
// WithXHeaders specifies list of extended headers (string key-value pairs) // WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length. // to be attached to the request. Must have an even length.
// //
// Slice must not be mutated until the operation completes. // Slice must not be mutated until the operation completes.
//
// Deprecated: Use PrmObjectSearch.XHeaders instead.
func (x *PrmObjectSearch) WithXHeaders(hs ...string) { func (x *PrmObjectSearch) WithXHeaders(hs ...string) {
writeXHeadersToMeta(hs, &x.meta) x.XHeaders = hs
} }
// UseKey specifies private key to sign the requests. // UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used. // If key is not provided, then Client default key is used.
//
// Deprecated: Use PrmObjectSearch.Key instead.
func (x *PrmObjectSearch) UseKey(key ecdsa.PrivateKey) { func (x *PrmObjectSearch) UseKey(key ecdsa.PrivateKey) {
x.key = &key x.Key = &key
} }
// InContainer specifies the container in which to look for objects. // InContainer specifies the container in which to look for objects.
// Required parameter. // Required parameter.
//
// Deprecated: Use PrmObjectSearch.ContainerID instead.
func (x *PrmObjectSearch) InContainer(id cid.ID) { func (x *PrmObjectSearch) InContainer(id cid.ID) {
x.cnrID = id x.ContainerID = &id
x.cnrSet = true
} }
// SetFilters sets filters by which to select objects. All container objects // SetFilters sets filters by which to select objects. All container objects
// match unset/empty filters. // match unset/empty filters.
//
// Deprecated: Use PrmObjectSearch.Filters instead.
func (x *PrmObjectSearch) SetFilters(filters object.SearchFilters) { func (x *PrmObjectSearch) SetFilters(filters object.SearchFilters) {
x.filters = filters x.Filters = filters
} }
// ResObjectSearch groups the final result values of ObjectSearch operation. // ResObjectSearch groups the final result values of ObjectSearch operation.
@ -206,12 +220,54 @@ func (x *ObjectListReader) Close() (*ResObjectSearch, error) {
defer x.cancelCtxStream() defer x.cancelCtxStream()
if x.err != nil && !errors.Is(x.err, io.EOF) { if x.err != nil && !errors.Is(x.err, io.EOF) {
return nil, x.err return &x.res, x.err
} }
return &x.res, nil return &x.res, nil
} }
func (x *PrmObjectSearch) buildRequest(c *Client) (*v2object.SearchRequest, error) {
if x.ContainerID == nil {
return nil, errorMissingContainer
}
if len(x.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(x.XHeaders, meta)
if x.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
x.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if x.Session != nil {
v2SessionToken := new(v2session.Token)
x.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if x.Local {
meta.SetTTL(1)
}
cnrV2 := new(v2refs.ContainerID)
x.ContainerID.WriteToV2(cnrV2)
body := new(v2object.SearchRequestBody)
body.SetVersion(1)
body.SetContainerID(cnrV2)
body.SetFilters(x.Filters.ToV2())
req := new(v2object.SearchRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// ObjectSearchInit initiates object selection through a remote server using FrostFS API protocol. // ObjectSearchInit initiates object selection through a remote server using FrostFS API protocol.
// //
// The call only opens the transmission channel, explicit fetching of matched objects // The call only opens the transmission channel, explicit fetching of matched objects
@ -221,33 +277,17 @@ func (x *ObjectListReader) Close() (*ResObjectSearch, error) {
// Returns an error if parameters are set incorrectly (see PrmObjectSearch docs). // Returns an error if parameters are set incorrectly (see PrmObjectSearch docs).
// Context is required and must not be nil. It is used for network communication. // Context is required and must not be nil. It is used for network communication.
func (c *Client) ObjectSearchInit(ctx context.Context, prm PrmObjectSearch) (*ObjectListReader, error) { func (c *Client) ObjectSearchInit(ctx context.Context, prm PrmObjectSearch) (*ObjectListReader, error) {
// check parameters req, err := prm.buildRequest(c)
switch { if err != nil {
case ctx == nil: return nil, err
return nil, errorMissingContext
case !prm.cnrSet:
return nil, errorMissingContainer
} }
var cidV2 v2refs.ContainerID key := prm.Key
prm.cnrID.WriteToV2(&cidV2)
var body v2object.SearchRequestBody
body.SetVersion(1)
body.SetContainerID(&cidV2)
body.SetFilters(prm.filters.ToV2())
// init reader
var req v2object.SearchRequest
req.SetBody(&body)
c.prepareRequest(&req, &prm.meta)
key := prm.key
if key == nil { if key == nil {
key = &c.prm.key key = &c.prm.Key
} }
err := signature.SignServiceMessage(key, &req) err = signature.SignServiceMessage(key, req)
if err != nil { if err != nil {
return nil, fmt.Errorf("sign request: %w", err) return nil, fmt.Errorf("sign request: %w", err)
} }
@ -255,7 +295,7 @@ func (c *Client) ObjectSearchInit(ctx context.Context, prm PrmObjectSearch) (*Ob
var r ObjectListReader var r ObjectListReader
ctx, r.cancelCtxStream = context.WithCancel(ctx) ctx, r.cancelCtxStream = context.WithCancel(ctx)
r.stream, err = rpcapi.SearchObjects(&c.c, &req, client.WithContext(ctx)) r.stream, err = rpcapi.SearchObjects(&c.c, req, client.WithContext(ctx))
if err != nil { if err != nil {
return nil, fmt.Errorf("open stream: %w", err) return nil, fmt.Errorf("open stream: %w", err)
} }

View file

@ -7,11 +7,11 @@ import (
"io" "io"
"testing" "testing"
v2object "github.com/TrueCloudLab/frostfs-api-go/v2/object" v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
signatureV2 "github.com/TrueCloudLab/frostfs-api-go/v2/signature" signatureV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
oid "github.com/TrueCloudLab/frostfs-sdk-go/object/id" oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
oidtest "github.com/TrueCloudLab/frostfs-sdk-go/object/id/test" oidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id/test"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )

View file

@ -1,200 +0,0 @@
package client
import (
"context"
v2reputation "github.com/TrueCloudLab/frostfs-api-go/v2/reputation"
rpcapi "github.com/TrueCloudLab/frostfs-api-go/v2/rpc"
"github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client"
"github.com/TrueCloudLab/frostfs-sdk-go/reputation"
)
// PrmAnnounceLocalTrust groups parameters of AnnounceLocalTrust operation.
type PrmAnnounceLocalTrust struct {
prmCommonMeta
epoch uint64
trusts []reputation.Trust
}
// SetEpoch sets number of FrostFS epoch in which the trust was assessed.
// Required parameter, must not be zero.
func (x *PrmAnnounceLocalTrust) SetEpoch(epoch uint64) {
x.epoch = epoch
}
// SetValues sets values describing trust of the client to the FrostFS network participants.
// Required parameter. Must not be empty.
//
// Must not be mutated before the end of the operation.
func (x *PrmAnnounceLocalTrust) SetValues(trusts []reputation.Trust) {
x.trusts = trusts
}
// ResAnnounceLocalTrust groups results of AnnounceLocalTrust operation.
type ResAnnounceLocalTrust struct {
statusRes
}
// AnnounceLocalTrust sends client's trust values to the FrostFS network participants.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Returns an error if parameters are set incorrectly (see PrmAnnounceLocalTrust docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) AnnounceLocalTrust(ctx context.Context, prm PrmAnnounceLocalTrust) (*ResAnnounceLocalTrust, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case prm.epoch == 0:
return nil, errorZeroEpoch
case len(prm.trusts) == 0:
return nil, errorMissingTrusts
}
// form request body
reqBody := new(v2reputation.AnnounceLocalTrustRequestBody)
reqBody.SetEpoch(prm.epoch)
trusts := make([]v2reputation.Trust, len(prm.trusts))
for i := range prm.trusts {
prm.trusts[i].WriteToV2(&trusts[i])
}
reqBody.SetTrusts(trusts)
// form request
var req v2reputation.AnnounceLocalTrustRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResAnnounceLocalTrust
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.AnnounceLocalTrust(&c.c, &req, client.WithContext(ctx))
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmAnnounceIntermediateTrust groups parameters of AnnounceIntermediateTrust operation.
type PrmAnnounceIntermediateTrust struct {
prmCommonMeta
epoch uint64
iter uint32
trustSet bool
trust reputation.PeerToPeerTrust
}
// SetEpoch sets number of FrostFS epoch with which client's calculation algorithm is initialized.
// Required parameter, must not be zero.
func (x *PrmAnnounceIntermediateTrust) SetEpoch(epoch uint64) {
x.epoch = epoch
}
// SetIteration sets current sequence number of the client's calculation algorithm.
// By default, corresponds to initial (zero) iteration.
func (x *PrmAnnounceIntermediateTrust) SetIteration(iter uint32) {
x.iter = iter
}
// SetCurrentValue sets current global trust value computed at the specified iteration
// of the client's calculation algorithm. Required parameter.
func (x *PrmAnnounceIntermediateTrust) SetCurrentValue(trust reputation.PeerToPeerTrust) {
x.trust = trust
x.trustSet = true
}
// ResAnnounceIntermediateTrust groups results of AnnounceIntermediateTrust operation.
type ResAnnounceIntermediateTrust struct {
statusRes
}
// AnnounceIntermediateTrust sends global trust values calculated for the specified FrostFS network participants
// at some stage of client's calculation algorithm.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Returns an error if parameters are set incorrectly (see PrmAnnounceIntermediateTrust docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) AnnounceIntermediateTrust(ctx context.Context, prm PrmAnnounceIntermediateTrust) (*ResAnnounceIntermediateTrust, error) {
// check parameters
switch {
case ctx == nil:
return nil, errorMissingContext
case prm.epoch == 0:
return nil, errorZeroEpoch
case !prm.trustSet:
return nil, errorTrustNotSet
}
var trust v2reputation.PeerToPeerTrust
prm.trust.WriteToV2(&trust)
// form request body
reqBody := new(v2reputation.AnnounceIntermediateResultRequestBody)
reqBody.SetEpoch(prm.epoch)
reqBody.SetIteration(prm.iter)
reqBody.SetTrust(&trust)
// form request
var req v2reputation.AnnounceIntermediateResultRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResAnnounceIntermediateTrust
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.AnnounceIntermediateResult(&c.c, &req, client.WithContext(ctx))
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}

View file

@ -1,6 +1,6 @@
package client package client
import "github.com/TrueCloudLab/frostfs-api-go/v2/session" import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
// ResponseMetaInfo groups meta information about any FrostFS API response. // ResponseMetaInfo groups meta information about any FrostFS API response.
type ResponseMetaInfo struct { type ResponseMetaInfo struct {

View file

@ -3,34 +3,63 @@ package client
import ( import (
"context" "context"
"crypto/ecdsa" "crypto/ecdsa"
"fmt"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "github.com/TrueCloudLab/frostfs-api-go/v2/rpc" rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"github.com/TrueCloudLab/frostfs-api-go/v2/rpc/client" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "github.com/TrueCloudLab/frostfs-api-go/v2/session" v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"github.com/TrueCloudLab/frostfs-sdk-go/user" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
) )
// PrmSessionCreate groups parameters of SessionCreate operation. // PrmSessionCreate groups parameters of SessionCreate operation.
type PrmSessionCreate struct { type PrmSessionCreate struct {
prmCommonMeta XHeaders []string
exp uint64 Expiration uint64
keySet bool Key *ecdsa.PrivateKey
key ecdsa.PrivateKey
} }
// SetExp sets number of the last NepFS epoch in the lifetime of the session after which it will be expired. // SetExp sets number of the last NepFS epoch in the lifetime of the session after which it will be expired.
//
// Deprecated: Use PrmSessionCreate.Expiration instead.
func (x *PrmSessionCreate) SetExp(exp uint64) { func (x *PrmSessionCreate) SetExp(exp uint64) {
x.exp = exp x.Expiration = exp
} }
// UseKey specifies private key to sign the requests and compute token owner. // UseKey specifies private key to sign the requests and compute token owner.
// If key is not provided, then Client default key is used. // If key is not provided, then Client default key is used.
//
// Deprecated: Use PrmSessionCreate.Key instead.
func (x *PrmSessionCreate) UseKey(key ecdsa.PrivateKey) { func (x *PrmSessionCreate) UseKey(key ecdsa.PrivateKey) {
x.keySet = true x.Key = &key
x.key = key }
func (x *PrmSessionCreate) buildRequest(c *Client) (*v2session.CreateRequest, error) {
ownerKey := c.prm.Key.PublicKey
if x.Key != nil {
ownerKey = x.Key.PublicKey
}
var ownerID user.ID
user.IDFromKey(&ownerID, ownerKey)
var ownerIDV2 refs.OwnerID
ownerID.WriteToV2(&ownerIDV2)
reqBody := new(v2session.CreateRequestBody)
reqBody.SetOwnerID(&ownerIDV2)
reqBody.SetExpiration(x.Expiration)
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(x.XHeaders, &meta)
var req v2session.CreateRequest
req.SetBody(reqBody)
c.prepareRequest(&req, &meta)
return &req, nil
} }
// ResSessionCreate groups resulting values of SessionCreate operation. // ResSessionCreate groups resulting values of SessionCreate operation.
@ -42,10 +71,6 @@ type ResSessionCreate struct {
sessionKey []byte sessionKey []byte
} }
func (x *ResSessionCreate) setID(id []byte) {
x.id = id
}
// ID returns identifier of the opened session in a binary FrostFS API protocol format. // ID returns identifier of the opened session in a binary FrostFS API protocol format.
// //
// Client doesn't retain value so modification is safe. // Client doesn't retain value so modification is safe.
@ -53,10 +78,6 @@ func (x ResSessionCreate) ID() []byte {
return x.id return x.id
} }
func (x *ResSessionCreate) setSessionKey(key []byte) {
x.sessionKey = key
}
// PublicKey returns public key of the opened session in a binary FrostFS API protocol format. // PublicKey returns public key of the opened session in a binary FrostFS API protocol format.
func (x ResSessionCreate) PublicKey() []byte { func (x ResSessionCreate) PublicKey() []byte {
return x.sessionKey return x.sessionKey
@ -68,9 +89,9 @@ func (x ResSessionCreate) PublicKey() []byte {
// //
// Exactly one return value is non-nil. By default, server status is returned in res structure. // Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`. // Any client's internal or transport errors are returned as `error`.
// If PrmInit.ResolveFrostFSFailures has been called, unsuccessful // If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are returned as `error`, otherwise, are included // FrostFS status codes are included in the returned result structure,
// in the returned result structure. // otherwise, are also returned as `error`.
// //
// Returns an error if parameters are set incorrectly (see PrmSessionCreate docs). // Returns an error if parameters are set incorrectly (see PrmSessionCreate docs).
// Context is required and must not be nil. It is used for network communication. // Context is required and must not be nil. It is used for network communication.
@ -78,62 +99,28 @@ func (x ResSessionCreate) PublicKey() []byte {
// Return statuses: // Return statuses:
// - global (see Client docs). // - global (see Client docs).
func (c *Client) SessionCreate(ctx context.Context, prm PrmSessionCreate) (*ResSessionCreate, error) { func (c *Client) SessionCreate(ctx context.Context, prm PrmSessionCreate) (*ResSessionCreate, error) {
// check context req, err := prm.buildRequest(c)
if ctx == nil { if err != nil {
return nil, errorMissingContext return nil, err
} }
ownerKey := c.prm.key.PublicKey if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
if prm.keySet { return nil, fmt.Errorf("sign request: %w", err)
ownerKey = prm.key.PublicKey
}
var ownerID user.ID
user.IDFromKey(&ownerID, ownerKey)
var ownerIDV2 refs.OwnerID
ownerID.WriteToV2(&ownerIDV2)
// form request body
reqBody := new(v2session.CreateRequestBody)
reqBody.SetOwnerID(&ownerIDV2)
reqBody.SetExpiration(prm.exp)
// for request
var req v2session.CreateRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResSessionCreate
)
c.initCallContext(&cc)
if prm.keySet {
cc.key = prm.key
} }
cc.meta = prm.prmCommonMeta resp, err := rpcapi.CreateSession(&c.c, req, client.WithContext(ctx))
cc.req = &req if err != nil {
cc.statusRes = &res return nil, err
cc.call = func() (responseV2, error) {
return rpcapi.CreateSession(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2session.CreateResponse)
body := resp.GetBody()
res.setID(body.GetID())
res.setSessionKey(body.GetSessionKey())
} }
// process call var res ResSessionCreate
if !cc.processCall() { res.st, err = c.processResponse(resp)
return nil, cc.err if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
} }
body := resp.GetBody()
res.id = body.GetID()
res.sessionKey = body.GetSessionKey()
return &res, nil return &res, nil
} }

View file

@ -0,0 +1,53 @@
package apistatus
import (
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/apemanager"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
)
// APEManagerAccessDenied describes status of the failure because of the access control violation.
// Instances provide Status and StatusV2 interfaces.
type APEManagerAccessDenied struct {
v2 status.Status
}
const defaultAPEManagerAccessDeniedMsg = "apemanager access denied"
func (x *APEManagerAccessDenied) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultAPEManagerAccessDeniedMsg
}
return errMessageStatusV2(
globalizeCodeV2(apemanager.StatusAPEManagerAccessDenied, apemanager.GlobalizeFail),
msg,
)
}
func (x *APEManagerAccessDenied) fromStatusV2(st *status.Status) {
x.v2 = *st
}
// ToStatusV2 converts APEManagerAccessDenied to v2's Status.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// - code: APE_MANAGER_ACCESS_DENIED;
// - string message: "apemanager access denied";
// - details: empty.
func (x APEManagerAccessDenied) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(apemanager.StatusAPEManagerAccessDenied, apemanager.GlobalizeFail))
x.v2.SetMessage(defaultAPEManagerAccessDeniedMsg)
return &x.v2
}
// WriteReason writes human-readable access rejection reason.
func (x *APEManagerAccessDenied) WriteReason(reason string) {
apemanager.WriteAccessDeniedDesc(&x.v2, reason)
}
// Reason returns human-readable access rejection reason returned by the server.
// Returns empty value is reason is not presented.
func (x APEManagerAccessDenied) Reason() string {
return apemanager.ReadAccessDeniedDesc(x.v2)
}

View file

@ -3,7 +3,7 @@ package apistatus
import ( import (
"encoding/binary" "encoding/binary"
"github.com/TrueCloudLab/frostfs-api-go/v2/status" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
) )
// ServerInternal describes failure statuses related to internal server errors. // ServerInternal describes failure statuses related to internal server errors.
@ -14,7 +14,7 @@ type ServerInternal struct {
v2 status.Status v2 status.Status
} }
func (x ServerInternal) Error() string { func (x *ServerInternal) Error() string {
return errMessageStatusV2( return errMessageStatusV2(
globalizeCodeV2(status.Internal, status.GlobalizeCommonFail), globalizeCodeV2(status.Internal, status.GlobalizeCommonFail),
x.v2.Message(), x.v2.Message(),
@ -62,7 +62,7 @@ type WrongMagicNumber struct {
v2 status.Status v2 status.Status
} }
func (x WrongMagicNumber) Error() string { func (x *WrongMagicNumber) Error() string {
return errMessageStatusV2( return errMessageStatusV2(
globalizeCodeV2(status.WrongMagicNumber, status.GlobalizeCommonFail), globalizeCodeV2(status.WrongMagicNumber, status.GlobalizeCommonFail),
x.v2.Message(), x.v2.Message(),
@ -132,7 +132,7 @@ type SignatureVerification struct {
const defaultSignatureVerificationMsg = "signature verification failed" const defaultSignatureVerificationMsg = "signature verification failed"
func (x SignatureVerification) Error() string { func (x *SignatureVerification) Error() string {
msg := x.v2.Message() msg := x.v2.Message()
if msg == "" { if msg == "" {
msg = defaultSignatureVerificationMsg msg = defaultSignatureVerificationMsg
@ -191,7 +191,7 @@ type NodeUnderMaintenance struct {
const defaultNodeUnderMaintenanceMsg = "node is under maintenance" const defaultNodeUnderMaintenanceMsg = "node is under maintenance"
// Error implements the error interface. // Error implements the error interface.
func (x NodeUnderMaintenance) Error() string { func (x *NodeUnderMaintenance) Error() string {
msg := x.Message() msg := x.Message()
if msg == "" { if msg == "" {
msg = defaultNodeUnderMaintenanceMsg msg = defaultNodeUnderMaintenanceMsg
@ -238,3 +238,61 @@ func (x *NodeUnderMaintenance) SetMessage(v string) {
func (x NodeUnderMaintenance) Message() string { func (x NodeUnderMaintenance) Message() string {
return x.v2.Message() return x.v2.Message()
} }
// InvalidArgument describes failure status related to invalid argument.
// Instances provide Status and StatusV2 interfaces.
type InvalidArgument struct {
v2 status.Status
}
const defaultInvalidArgumentMsg = "argument is invalid"
// Error implements the error interface.
func (x *InvalidArgument) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultInvalidArgumentMsg
}
return errMessageStatusV2(
globalizeCodeV2(status.InvalidArgument, status.GlobalizeCommonFail),
msg,
)
}
// implements local interface defined in FromStatusV2 func.
func (x *InvalidArgument) fromStatusV2(st *status.Status) {
x.v2 = *st
}
// ToStatusV2 implements StatusV2 interface method.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// - code: INVALID_ARGUMENT;
// - string message: written message via SetMessage or
// "argument is invalid" as a default message;
// - details: empty.
func (x InvalidArgument) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(status.InvalidArgument, status.GlobalizeCommonFail))
if x.v2.Message() == "" {
x.v2.SetMessage(defaultInvalidArgumentMsg)
}
return &x.v2
}
// SetMessage writes invalid argument failure message.
// Message should be used for debug purposes only.
//
// See also Message.
func (x *InvalidArgument) SetMessage(v string) {
x.v2.SetMessage(v)
}
// Message returns status message. Zero status returns empty message.
// Message should be used for debug purposes only.
//
// See also SetMessage.
func (x InvalidArgument) Message() string {
return x.v2.Message()
}

View file

@ -3,8 +3,8 @@ package apistatus_test
import ( import (
"testing" "testing"
"github.com/TrueCloudLab/frostfs-api-go/v2/status" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
apistatus "github.com/TrueCloudLab/frostfs-sdk-go/client/status" apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
@ -114,7 +114,7 @@ func TestNodeUnderMaintenance(t *testing.T) {
stV2 := st.ToStatusV2() stV2 := st.ToStatusV2()
require.Empty(t, "", stV2.Message()) require.Equal(t, "node is under maintenance", stV2.Message())
}) })
t.Run("non-empty to V2", func(t *testing.T) { t.Run("non-empty to V2", func(t *testing.T) {
@ -128,3 +128,42 @@ func TestNodeUnderMaintenance(t *testing.T) {
require.Equal(t, msg, stV2.Message()) require.Equal(t, msg, stV2.Message())
}) })
} }
func TestInvalidArgument(t *testing.T) {
t.Run("default", func(t *testing.T) {
var st apistatus.InvalidArgument
require.Empty(t, st.Message())
})
t.Run("custom message", func(t *testing.T) {
var st apistatus.InvalidArgument
msg := "some message"
st.SetMessage(msg)
stV2 := st.ToStatusV2()
require.Equal(t, msg, st.Message())
require.Equal(t, msg, stV2.Message())
})
t.Run("empty to V2", func(t *testing.T) {
var st apistatus.InvalidArgument
stV2 := st.ToStatusV2()
require.Equal(t, "argument is invalid", stV2.Message())
})
t.Run("non-empty to V2", func(t *testing.T) {
var st apistatus.InvalidArgument
msg := "some other msg"
st.SetMessage(msg)
stV2 := st.ToStatusV2()
require.Equal(t, msg, stV2.Message())
})
}

View file

@ -1,8 +1,8 @@
package apistatus package apistatus
import ( import (
"github.com/TrueCloudLab/frostfs-api-go/v2/container" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"github.com/TrueCloudLab/frostfs-api-go/v2/status" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
) )
// ContainerNotFound describes status of the failure because of the missing container. // ContainerNotFound describes status of the failure because of the missing container.
@ -13,7 +13,7 @@ type ContainerNotFound struct {
const defaultContainerNotFoundMsg = "container not found" const defaultContainerNotFoundMsg = "container not found"
func (x ContainerNotFound) Error() string { func (x *ContainerNotFound) Error() string {
msg := x.v2.Message() msg := x.v2.Message()
if msg == "" { if msg == "" {
msg = defaultContainerNotFoundMsg msg = defaultContainerNotFoundMsg
@ -51,7 +51,7 @@ type EACLNotFound struct {
const defaultEACLNotFoundMsg = "eACL not found" const defaultEACLNotFoundMsg = "eACL not found"
func (x EACLNotFound) Error() string { func (x *EACLNotFound) Error() string {
msg := x.v2.Message() msg := x.v2.Message()
if msg == "" { if msg == "" {
msg = defaultEACLNotFoundMsg msg = defaultEACLNotFoundMsg

View file

@ -1,8 +1,8 @@
package apistatus package apistatus
import ( import (
"github.com/TrueCloudLab/frostfs-api-go/v2/object" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
"github.com/TrueCloudLab/frostfs-api-go/v2/status" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
) )
// ObjectLocked describes status of the failure because of the locked object. // ObjectLocked describes status of the failure because of the locked object.
@ -13,7 +13,7 @@ type ObjectLocked struct {
const defaultObjectLockedMsg = "object is locked" const defaultObjectLockedMsg = "object is locked"
func (x ObjectLocked) Error() string { func (x *ObjectLocked) Error() string {
msg := x.v2.Message() msg := x.v2.Message()
if msg == "" { if msg == "" {
msg = defaultObjectLockedMsg msg = defaultObjectLockedMsg
@ -50,7 +50,7 @@ type LockNonRegularObject struct {
const defaultLockNonRegularObjectMsg = "locking non-regular object is forbidden" const defaultLockNonRegularObjectMsg = "locking non-regular object is forbidden"
func (x LockNonRegularObject) Error() string { func (x *LockNonRegularObject) Error() string {
msg := x.v2.Message() msg := x.v2.Message()
if msg == "" { if msg == "" {
msg = defaultLockNonRegularObjectMsg msg = defaultLockNonRegularObjectMsg
@ -87,7 +87,7 @@ type ObjectAccessDenied struct {
const defaultObjectAccessDeniedMsg = "access to object operation denied" const defaultObjectAccessDeniedMsg = "access to object operation denied"
func (x ObjectAccessDenied) Error() string { func (x *ObjectAccessDenied) Error() string {
msg := x.v2.Message() msg := x.v2.Message()
if msg == "" { if msg == "" {
msg = defaultObjectAccessDeniedMsg msg = defaultObjectAccessDeniedMsg
@ -135,7 +135,7 @@ type ObjectNotFound struct {
const defaultObjectNotFoundMsg = "object not found" const defaultObjectNotFoundMsg = "object not found"
func (x ObjectNotFound) Error() string { func (x *ObjectNotFound) Error() string {
msg := x.v2.Message() msg := x.v2.Message()
if msg == "" { if msg == "" {
msg = defaultObjectNotFoundMsg msg = defaultObjectNotFoundMsg
@ -172,7 +172,7 @@ type ObjectAlreadyRemoved struct {
const defaultObjectAlreadyRemovedMsg = "object already removed" const defaultObjectAlreadyRemovedMsg = "object already removed"
func (x ObjectAlreadyRemoved) Error() string { func (x *ObjectAlreadyRemoved) Error() string {
msg := x.v2.Message() msg := x.v2.Message()
if msg == "" { if msg == "" {
msg = defaultObjectAlreadyRemovedMsg msg = defaultObjectAlreadyRemovedMsg
@ -210,7 +210,7 @@ type ObjectOutOfRange struct {
const defaultObjectOutOfRangeMsg = "out of range" const defaultObjectOutOfRangeMsg = "out of range"
func (x ObjectOutOfRange) Error() string { func (x *ObjectOutOfRange) Error() string {
msg := x.v2.Message() msg := x.v2.Message()
if msg == "" { if msg == "" {
msg = defaultObjectOutOfRangeMsg msg = defaultObjectOutOfRangeMsg

View file

@ -3,7 +3,7 @@ package apistatus_test
import ( import (
"testing" "testing"
apistatus "github.com/TrueCloudLab/frostfs-sdk-go/client/status" apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )

View file

@ -1,8 +1,8 @@
package apistatus package apistatus
import ( import (
"github.com/TrueCloudLab/frostfs-api-go/v2/session" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"github.com/TrueCloudLab/frostfs-api-go/v2/status" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
) )
// SessionTokenNotFound describes status of the failure because of the missing session token. // SessionTokenNotFound describes status of the failure because of the missing session token.
@ -13,7 +13,7 @@ type SessionTokenNotFound struct {
const defaultSessionTokenNotFoundMsg = "session token not found" const defaultSessionTokenNotFoundMsg = "session token not found"
func (x SessionTokenNotFound) Error() string { func (x *SessionTokenNotFound) Error() string {
msg := x.v2.Message() msg := x.v2.Message()
if msg == "" { if msg == "" {
msg = defaultSessionTokenNotFoundMsg msg = defaultSessionTokenNotFoundMsg
@ -50,7 +50,7 @@ type SessionTokenExpired struct {
const defaultSessionTokenExpiredMsg = "expired session token" const defaultSessionTokenExpiredMsg = "expired session token"
func (x SessionTokenExpired) Error() string { func (x *SessionTokenExpired) Error() string {
msg := x.v2.Message() msg := x.v2.Message()
if msg == "" { if msg == "" {
msg = defaultSessionTokenExpiredMsg msg = defaultSessionTokenExpiredMsg

View file

@ -15,7 +15,7 @@ package apistatus
// It should be noted that using direct typecasting is not a compatible approach. // It should be noted that using direct typecasting is not a compatible approach.
// //
// To transport statuses using the FrostFS API V2 protocol, see StatusV2 interface and FromStatusV2 and ToStatusV2 functions. // To transport statuses using the FrostFS API V2 protocol, see StatusV2 interface and FromStatusV2 and ToStatusV2 functions.
type Status interface{} type Status any
// ErrFromStatus converts Status instance to error if it is failed. Returns nil on successful Status. // ErrFromStatus converts Status instance to error if it is failed. Returns nil on successful Status.
// //

View file

@ -4,7 +4,7 @@ import (
"errors" "errors"
"testing" "testing"
apistatus "github.com/TrueCloudLab/frostfs-sdk-go/client/status" apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )

View file

@ -1,7 +1,7 @@
package apistatus package apistatus
import ( import (
"github.com/TrueCloudLab/frostfs-api-go/v2/status" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
) )
// SuccessDefaultV2 represents Status instance of default success. Implements StatusV2. // SuccessDefaultV2 represents Status instance of default success. Implements StatusV2.

View file

@ -1,14 +1,14 @@
package apistatus package apistatus
import ( import (
"github.com/TrueCloudLab/frostfs-api-go/v2/status" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
) )
type unrecognizedStatusV2 struct { type unrecognizedStatusV2 struct {
v2 status.Status v2 status.Status
} }
func (x unrecognizedStatusV2) Error() string { func (x *unrecognizedStatusV2) Error() string {
return errMessageStatusV2("unrecognized", x.v2.Message()) return errMessageStatusV2("unrecognized", x.v2.Message())
} }

View file

@ -3,10 +3,11 @@ package apistatus
import ( import (
"fmt" "fmt"
"github.com/TrueCloudLab/frostfs-api-go/v2/container" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/apemanager"
"github.com/TrueCloudLab/frostfs-api-go/v2/object" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"github.com/TrueCloudLab/frostfs-api-go/v2/session" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
"github.com/TrueCloudLab/frostfs-api-go/v2/status" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
) )
// StatusV2 defines a variety of Status instances compatible with FrostFS API V2 protocol. // StatusV2 defines a variety of Status instances compatible with FrostFS API V2 protocol.
@ -15,7 +16,7 @@ import (
type StatusV2 interface { type StatusV2 interface {
Status Status
// ToStatusV2 returns the status as github.com/TrueCloudLab/frostfs-api-go/v2/status.Status message structure. // ToStatusV2 returns the status as git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status.Status message structure.
ToStatusV2() *status.Status ToStatusV2() *status.Status
} }
@ -32,12 +33,29 @@ type StatusV2 interface {
// //
// Common failures: // Common failures:
// - status.Internal: *ServerInternal; // - status.Internal: *ServerInternal;
// - status.SignatureVerificationFail: *SignatureVerification. // - status.WrongMagicNumber: *WrongMagicNumber;
// - status.SignatureVerificationFail: *SignatureVerification;
// - status.NodeUnderMaintenance: *NodeUnderMaintenance;
// - status.InvalidArgument: *InvalidArgument.
// //
// Object failures: // Object failures:
// - object.StatusLocked: *ObjectLocked; // - object.StatusLocked: *ObjectLocked;
// - object.StatusLockNonRegularObject: *LockNonRegularObject. // - object.StatusLockNonRegularObject: *LockNonRegularObject;
// - object.StatusAccessDenied: *ObjectAccessDenied. // - object.StatusAccessDenied: *ObjectAccessDenied;
// - object.StatusNotFound: *ObjectNotFound;
// - object.StatusAlreadyRemoved: *ObjectAlreadyRemoved;
// - object.StatusOutOfRange: *ObjectOutOfRange.
//
// Container failures:
// - container.StatusNotFound: *ContainerNotFound;
// - container.StatusEACLNotFound: *EACLNotFound.
//
// Session failures:
// - session.StatusTokenNotFound: *SessionTokenNotFound;
// - session.StatusTokenExpired: *SessionTokenExpired.
//
// APE Manager failures
// - apemanager.StatusAPEManagerAccessDenied: *APEManagerAccessDenied.
func FromStatusV2(st *status.Status) Status { func FromStatusV2(st *status.Status) Status {
var decoder interface { var decoder interface {
fromStatusV2(*status.Status) fromStatusV2(*status.Status)
@ -60,6 +78,8 @@ func FromStatusV2(st *status.Status) Status {
decoder = new(SignatureVerification) decoder = new(SignatureVerification)
case status.NodeUnderMaintenance: case status.NodeUnderMaintenance:
decoder = new(NodeUnderMaintenance) decoder = new(NodeUnderMaintenance)
case status.InvalidArgument:
decoder = new(InvalidArgument)
} }
case object.LocalizeFailStatus(&code): case object.LocalizeFailStatus(&code):
switch code { switch code {
@ -92,6 +112,12 @@ func FromStatusV2(st *status.Status) Status {
case session.StatusTokenExpired: case session.StatusTokenExpired:
decoder = new(SessionTokenExpired) decoder = new(SessionTokenExpired)
} }
case apemanager.LocalizeFailStatus(&code):
//nolint:exhaustive
switch code {
case apemanager.StatusAPEManagerAccessDenied:
decoder = new(APEManagerAccessDenied)
}
} }
if decoder == nil { if decoder == nil {
@ -123,7 +149,7 @@ func ToStatusV2(st Status) *status.Status {
return internalErrorStatus return internalErrorStatus
} }
func errMessageStatusV2(code interface{}, msg string) string { func errMessageStatusV2(code any, msg string) string {
const ( const (
noMsgFmt = "status: code = %v" noMsgFmt = "status: code = %v"
msgFmt = noMsgFmt + " message = %s" msgFmt = noMsgFmt + " message = %s"

View file

@ -4,7 +4,7 @@ import (
"errors" "errors"
"testing" "testing"
apistatus "github.com/TrueCloudLab/frostfs-sdk-go/client/status" apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
@ -12,7 +12,7 @@ func TestToStatusV2(t *testing.T) {
type statusConstructor func() apistatus.Status type statusConstructor func() apistatus.Status
for _, testItem := range [...]struct { for _, testItem := range [...]struct {
status interface{} // Status or statusConstructor status any // Status or statusConstructor
codeV2 uint64 codeV2 uint64
messageV2 string messageV2 string
}{ }{
@ -61,6 +61,24 @@ func TestToStatusV2(t *testing.T) {
}), }),
codeV2: 1025, codeV2: 1025,
}, },
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.SignatureVerification)
}),
codeV2: 1026,
},
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.NodeUnderMaintenance)
}),
codeV2: 1027,
},
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.InvalidArgument)
}),
codeV2: 1028,
},
{ {
status: (statusConstructor)(func() apistatus.Status { status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.ObjectLocked) return new(apistatus.ObjectLocked)
@ -127,9 +145,9 @@ func TestToStatusV2(t *testing.T) {
}, },
{ {
status: (statusConstructor)(func() apistatus.Status { status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.NodeUnderMaintenance) return new(apistatus.APEManagerAccessDenied)
}), }),
codeV2: 1027, codeV2: 5120,
}, },
} { } {
var st apistatus.Status var st apistatus.Status
@ -165,7 +183,7 @@ func TestFromStatusV2(t *testing.T) {
type statusConstructor func() apistatus.Status type statusConstructor func() apistatus.Status
for _, testItem := range [...]struct { for _, testItem := range [...]struct {
status interface{} // Status or statusConstructor status any // Status or statusConstructor
codeV2 uint64 codeV2 uint64
messageV2 string messageV2 string
}{ }{
@ -278,6 +296,12 @@ func TestFromStatusV2(t *testing.T) {
}), }),
codeV2: 4097, codeV2: 4097,
}, },
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.APEManagerAccessDenied)
}),
codeV2: 5120,
},
{ {
status: (statusConstructor)(func() apistatus.Status { status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.NodeUnderMaintenance) return new(apistatus.NodeUnderMaintenance)

View file

@ -9,6 +9,7 @@ import "strconv"
// use corresponding constants and/or methods instead. // use corresponding constants and/or methods instead.
type Op uint32 type Op uint32
// nolint: unused
const ( const (
opZero Op = iota // extreme value for testing opZero Op = iota // extreme value for testing
@ -53,6 +54,7 @@ func (x Op) String() string {
// use corresponding constants and/or methods instead. // use corresponding constants and/or methods instead.
type Role uint32 type Role uint32
// nolint: unused
const ( const (
roleZero Role = iota // extreme value for testing roleZero Role = iota // extreme value for testing

View file

@ -7,14 +7,6 @@ func setBit(num *uint32, n uint8) {
*num |= 1 << n *num |= 1 << n
} }
// resets n-th bit in num (starting at 0).
func resetBit(num *uint32, n uint8) {
var mask uint32
setBit(&mask, n)
*num &= ^mask
}
// checks if n-th bit in num is set (starting at 0). // checks if n-th bit in num is set (starting at 0).
func isBitSet(num uint32, n uint8) bool { func isBitSet(num uint32, n uint8) bool {
mask := uint32(1 << n) mask := uint32(1 << n)

View file

@ -22,9 +22,6 @@ func TestBits(t *testing.T) {
setBit(&num, 6) setBit(&num, 6)
require.EqualValues(t, 0b1011110, num) require.EqualValues(t, 0b1011110, num)
resetBit(&num, 1)
require.EqualValues(t, 0b1011100, num)
} }
func TestOpBits(t *testing.T) { func TestOpBits(t *testing.T) {

View file

@ -6,19 +6,19 @@ import (
"errors" "errors"
"fmt" "fmt"
"strconv" "strconv"
"strings"
"time" "time"
"github.com/TrueCloudLab/frostfs-api-go/v2/container" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
v2netmap "github.com/TrueCloudLab/frostfs-api-go/v2/netmap" v2netmap "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
"github.com/TrueCloudLab/frostfs-sdk-go/container/acl" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/acl"
cid "github.com/TrueCloudLab/frostfs-sdk-go/container/id" cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
frostfscrypto "github.com/TrueCloudLab/frostfs-sdk-go/crypto" frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
frostfsecdsa "github.com/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa" frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
"github.com/TrueCloudLab/frostfs-sdk-go/netmap" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap"
subnetid "github.com/TrueCloudLab/frostfs-sdk-go/subnet/id" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
"github.com/TrueCloudLab/frostfs-sdk-go/user" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/version"
"github.com/TrueCloudLab/frostfs-sdk-go/version"
"github.com/google/uuid" "github.com/google/uuid"
) )
@ -37,7 +37,7 @@ import (
// Instances for existing containers can be initialized using decoding methods // Instances for existing containers can be initialized using decoding methods
// (e.g Unmarshal). // (e.g Unmarshal).
// //
// Container is mutually compatible with github.com/TrueCloudLab/frostfs-api-go/v2/container.Container // Container is mutually compatible with git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container.Container
// message. See ReadFromV2 / WriteToV2 methods. // message. See ReadFromV2 / WriteToV2 methods.
type Container struct { type Container struct {
v2 container.Container v2 container.Container
@ -96,6 +96,16 @@ func (x *Container) readFromV2(m container.Container, checkFieldPresence bool) e
return errors.New("missing placement policy") return errors.New("missing placement policy")
} }
if err := checkAttributes(m); err != nil {
return err
}
x.v2 = m
return nil
}
func checkAttributes(m container.Container) error {
attrs := m.GetAttributes() attrs := m.GetAttributes()
mAttr := make(map[string]struct{}, len(attrs)) mAttr := make(map[string]struct{}, len(attrs))
var key, val string var key, val string
@ -117,10 +127,8 @@ func (x *Container) readFromV2(m container.Container, checkFieldPresence bool) e
return fmt.Errorf("empty attribute value %s", key) return fmt.Errorf("empty attribute value %s", key)
} }
switch key { var err error
case container.SysAttributeSubnet: if key == attributeTimestamp {
err = new(subnetid.ID).DecodeString(val)
case attributeTimestamp:
_, err = strconv.ParseInt(val, 10, 64) _, err = strconv.ParseInt(val, 10, 64)
} }
@ -130,9 +138,6 @@ func (x *Container) readFromV2(m container.Container, checkFieldPresence bool) e
mAttr[key] = struct{}{} mAttr[key] = struct{}{}
} }
x.v2 = m
return nil return nil
} }
@ -292,7 +297,7 @@ func (x Container) PlacementPolicy() (res netmap.PlacementPolicy) {
// //
// SetAttribute overwrites existing attribute value. // SetAttribute overwrites existing attribute value.
// //
// See also Attribute, IterateAttributes. // See also Attribute, IterateAttributes, IterateUserAttributes.
func (x *Container) SetAttribute(key, value string) { func (x *Container) SetAttribute(key, value string) {
if key == "" { if key == "" {
panic("empty attribute key") panic("empty attribute key")
@ -303,7 +308,7 @@ func (x *Container) SetAttribute(key, value string) {
attrs := x.v2.GetAttributes() attrs := x.v2.GetAttributes()
ln := len(attrs) ln := len(attrs)
for i := 0; i < ln; i++ { for i := range ln {
if attrs[i].GetKey() == key { if attrs[i].GetKey() == key {
attrs[i].SetValue(value) attrs[i].SetValue(value)
return return
@ -320,7 +325,7 @@ func (x *Container) SetAttribute(key, value string) {
// Attribute reads value of the Container attribute by key. Empty result means // Attribute reads value of the Container attribute by key. Empty result means
// attribute absence. // attribute absence.
// //
// See also SetAttribute, IterateAttributes. // See also SetAttribute, IterateAttributes, IterateUserAttributes.
func (x Container) Attribute(key string) string { func (x Container) Attribute(key string) string {
attrs := x.v2.GetAttributes() attrs := x.v2.GetAttributes()
for i := range attrs { for i := range attrs {
@ -343,6 +348,20 @@ func (x Container) IterateAttributes(f func(key, val string)) {
} }
} }
// IterateUserAttributes iterates over user Container attributes and passes them
// into f. The handler MUST NOT be nil.
//
// See also SetAttribute, Attribute.
func (x Container) IterateUserAttributes(f func(key, val string)) {
attrs := x.v2.GetAttributes()
for _, attr := range attrs {
key := attr.GetKey()
if !strings.HasPrefix(key, container.SysAttributePrefix) {
f(key, attr.GetValue())
}
}
}
// SetName sets human-readable name of the Container. Name MUST NOT be empty. // SetName sets human-readable name of the Container. Name MUST NOT be empty.
// //
// See also Name. // See also Name.
@ -383,28 +402,6 @@ func CreatedAt(cnr Container) time.Time {
return time.Unix(sec, 0) return time.Unix(sec, 0)
} }
// SetSubnet places the Container on the specified FrostFS subnet. If called,
// container nodes will only be selected from the given subnet, otherwise from
// the entire network.
func SetSubnet(cnr *Container, subNet subnetid.ID) {
cnr.SetAttribute(container.SysAttributeSubnet, subNet.EncodeToString())
}
// Subnet return container subnet set using SetSubnet.
//
// Zero Container is bound to zero subnet.
func Subnet(cnr Container) (res subnetid.ID) {
val := cnr.Attribute(container.SysAttributeSubnet)
if val != "" {
err := res.DecodeString(val)
if err != nil {
panic(fmt.Sprintf("invalid subnet attribute: %s (%v)", val, err))
}
}
return
}
const attributeHomoHashEnabled = "true" const attributeHomoHashEnabled = "true"
// DisableHomomorphicHashing sets flag to disable homomorphic hashing of the // DisableHomomorphicHashing sets flag to disable homomorphic hashing of the
@ -465,8 +462,7 @@ func WriteDomain(cnr *Container, domain Domain) {
// ReadDomain reads Domain from the Container. Returns value with empty name // ReadDomain reads Domain from the Container. Returns value with empty name
// if domain is not specified. // if domain is not specified.
func ReadDomain(cnr Container) (res Domain) { func ReadDomain(cnr Container) (res Domain) {
name := cnr.Attribute(container.SysAttributeName) if name := cnr.Attribute(container.SysAttributeName); name != "" {
if name != "" {
res.SetName(name) res.SetName(name)
res.SetZone(cnr.Attribute(container.SysAttributeZone)) res.SetZone(cnr.Attribute(container.SysAttributeZone))
} }

View file

@ -6,19 +6,17 @@ import (
"testing" "testing"
"time" "time"
v2container "github.com/TrueCloudLab/frostfs-api-go/v2/container" v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
v2netmap "github.com/TrueCloudLab/frostfs-api-go/v2/netmap" v2netmap "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
"github.com/TrueCloudLab/frostfs-sdk-go/container" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
cid "github.com/TrueCloudLab/frostfs-sdk-go/container/id" cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
cidtest "github.com/TrueCloudLab/frostfs-sdk-go/container/id/test" cidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id/test"
containertest "github.com/TrueCloudLab/frostfs-sdk-go/container/test" containertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/test"
frostfscrypto "github.com/TrueCloudLab/frostfs-sdk-go/crypto" frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
netmaptest "github.com/TrueCloudLab/frostfs-sdk-go/netmap/test" netmaptest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap/test"
subnetid "github.com/TrueCloudLab/frostfs-sdk-go/subnet/id" usertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user/test"
subnetidtest "github.com/TrueCloudLab/frostfs-sdk-go/subnet/id/test" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/version"
usertest "github.com/TrueCloudLab/frostfs-sdk-go/user/test"
"github.com/TrueCloudLab/frostfs-sdk-go/version"
"github.com/google/uuid" "github.com/google/uuid"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
@ -80,7 +78,7 @@ func TestContainer_Owner(t *testing.T) {
val = containertest.Container() val = containertest.Container()
owner := *usertest.ID() owner := usertest.ID()
val.SetOwner(owner) val.SetOwner(owner)
@ -152,7 +150,7 @@ func assertContainsAttribute(t *testing.T, m v2container.Container, key, val str
} }
func TestContainer_Attribute(t *testing.T) { func TestContainer_Attribute(t *testing.T) {
const attrKey1, attrKey2 = "key1", "key2" const attrKey1, attrKey2 = v2container.SysAttributePrefix + "key1", v2container.SysAttributePrefix + "key2"
const attrVal1, attrVal2 = "val1", "val2" const attrVal1, attrVal2 = "val1", "val2"
val := containertest.Container() val := containertest.Container()
@ -160,6 +158,12 @@ func TestContainer_Attribute(t *testing.T) {
val.SetAttribute(attrKey1, attrVal1) val.SetAttribute(attrKey1, attrVal1)
val.SetAttribute(attrKey2, attrVal2) val.SetAttribute(attrKey2, attrVal2)
var i int
val.IterateUserAttributes(func(key, val string) {
i++
})
require.Equal(t, 1, i)
var msg v2container.Container var msg v2container.Container
val.WriteToV2(&msg) val.WriteToV2(&msg)
@ -233,28 +237,6 @@ func TestSetCreationTime(t *testing.T) {
require.Equal(t, creat.Unix(), container.CreatedAt(val2).Unix()) require.Equal(t, creat.Unix(), container.CreatedAt(val2).Unix())
} }
func TestSetSubnet(t *testing.T) {
var val container.Container
require.True(t, subnetid.IsZero(container.Subnet(val)))
val = containertest.Container()
sub := subnetidtest.ID()
container.SetSubnet(&val, sub)
var msg v2container.Container
val.WriteToV2(&msg)
assertContainsAttribute(t, msg, v2container.SysAttributeSubnet, sub.EncodeToString())
var val2 container.Container
require.NoError(t, val2.ReadFromV2(msg))
require.Equal(t, sub, container.Subnet(val))
}
func TestDisableHomomorphicHashing(t *testing.T) { func TestDisableHomomorphicHashing(t *testing.T) {
var val container.Container var val container.Container

View file

@ -23,11 +23,11 @@ it using the instance of Container types
// process the container data // process the container data
Instances can be also used to process FrostFS API V2 protocol messages Instances can be also used to process FrostFS API V2 protocol messages
(see neo.fs.v2.container package in https://github.com/TrueCloudLab/frostfs-api). (see neo.fs.v2.container package in https://git.frostfs.info/TrueCloudLab/frostfs-api).
On client side: On client side:
import "github.com/TrueCloudLab/frostfs-api-go/v2/container" import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
var msg container.Container var msg container.Container
cnr.WriteToV2(&msg) cnr.WriteToV2(&msg)

View file

@ -4,13 +4,13 @@ import (
"crypto/sha256" "crypto/sha256"
"fmt" "fmt"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
"github.com/mr-tron/base58" "github.com/mr-tron/base58"
) )
// ID represents FrostFS container identifier. // ID represents FrostFS container identifier.
// //
// ID is mutually compatible with github.com/TrueCloudLab/frostfs-api-go/v2/refs.ContainerID // ID is mutually compatible with git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs.ContainerID
// message. See ReadFromV2 / WriteToV2 methods. // message. See ReadFromV2 / WriteToV2 methods.
// //
// Instances can be created using built-in var declaration. // Instances can be created using built-in var declaration.

View file

@ -1,13 +1,13 @@
package cid_test package cid_test
import ( import (
"crypto/rand"
"crypto/sha256" "crypto/sha256"
"math/rand"
"testing" "testing"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
cid "github.com/TrueCloudLab/frostfs-sdk-go/container/id" cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
cidtest "github.com/TrueCloudLab/frostfs-sdk-go/container/id/test" cidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id/test"
"github.com/mr-tron/base58" "github.com/mr-tron/base58"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )

View file

@ -5,7 +5,7 @@ Note that importing the package into source files is highly discouraged.
Random instance generation functions can be useful when testing expects any value, e.g.: Random instance generation functions can be useful when testing expects any value, e.g.:
import cidtest "github.com/TrueCloudLab/frostfs-sdk-go/container/id/test" import cidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id/test"
cid := cidtest.ID() cid := cidtest.ID()
// test the value // test the value

View file

@ -1,17 +1,17 @@
package cidtest package cidtest
import ( import (
"crypto/rand"
"crypto/sha256" "crypto/sha256"
"math/rand"
cid "github.com/TrueCloudLab/frostfs-sdk-go/container/id" cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
) )
// ID returns random cid.ID. // ID returns random cid.ID.
func ID() cid.ID { func ID() cid.ID {
checksum := [sha256.Size]byte{} checksum := [sha256.Size]byte{}
rand.Read(checksum[:]) _, _ = rand.Read(checksum[:])
return IDWithChecksum(checksum) return IDWithChecksum(checksum)
} }

View file

@ -1,7 +1,7 @@
package container package container
import ( import (
"github.com/TrueCloudLab/frostfs-sdk-go/netmap" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap"
) )
// ApplyNetworkConfig applies network configuration to the // ApplyNetworkConfig applies network configuration to the

View file

@ -3,9 +3,9 @@ package container_test
import ( import (
"testing" "testing"
"github.com/TrueCloudLab/frostfs-sdk-go/container" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
containertest "github.com/TrueCloudLab/frostfs-sdk-go/container/test" containertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/test"
netmaptest "github.com/TrueCloudLab/frostfs-sdk-go/netmap/test" netmaptest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap/test"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )

View file

@ -1,104 +0,0 @@
package container
import (
"errors"
"fmt"
"github.com/TrueCloudLab/frostfs-api-go/v2/container"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs"
cid "github.com/TrueCloudLab/frostfs-sdk-go/container/id"
)
// SizeEstimation groups information about estimation of the size of the data
// stored in the FrostFS container.
//
// SizeEstimation is mutually compatible with github.com/TrueCloudLab/frostfs-api-go/v2/container.UsedSpaceAnnouncement
// message. See ReadFromV2 / WriteToV2 methods.
type SizeEstimation struct {
m container.UsedSpaceAnnouncement
}
// ReadFromV2 reads SizeEstimation from the container.UsedSpaceAnnouncement message.
// Checks if the message conforms to FrostFS API V2 protocol.
//
// See also WriteToV2.
func (x *SizeEstimation) ReadFromV2(m container.UsedSpaceAnnouncement) error {
cnrV2 := m.GetContainerID()
if cnrV2 == nil {
return errors.New("missing container")
}
var cnr cid.ID
err := cnr.ReadFromV2(*cnrV2)
if err != nil {
return fmt.Errorf("invalid container: %w", err)
}
x.m = m
return nil
}
// WriteToV2 writes SizeEstimation into the container.UsedSpaceAnnouncement message.
// The message MUST NOT be nil.
//
// See also ReadFromV2.
func (x SizeEstimation) WriteToV2(m *container.UsedSpaceAnnouncement) {
*m = x.m
}
// SetEpoch sets epoch when estimation of the container data size was calculated.
//
// See also Epoch.
func (x *SizeEstimation) SetEpoch(epoch uint64) {
x.m.SetEpoch(epoch)
}
// Epoch return epoch set using SetEpoch.
//
// Zero SizeEstimation represents estimation in zero epoch.
func (x SizeEstimation) Epoch() uint64 {
return x.m.GetEpoch()
}
// SetContainer specifies the container for which the amount of data is estimated.
// Required by the FrostFS API protocol.
//
// See also Container.
func (x *SizeEstimation) SetContainer(cnr cid.ID) {
var cidV2 refs.ContainerID
cnr.WriteToV2(&cidV2)
x.m.SetContainerID(&cidV2)
}
// Container returns container set using SetContainer.
//
// Zero SizeEstimation is not bound to any container (returns zero) which is
// incorrect according to FrostFS API protocol.
func (x SizeEstimation) Container() (res cid.ID) {
m := x.m.GetContainerID()
if m != nil {
err := res.ReadFromV2(*m)
if err != nil {
panic(fmt.Errorf("unexpected error from cid.ID.ReadFromV2: %w", err))
}
}
return
}
// SetValue sets estimated amount of data (in bytes) in the specified container.
//
// See also Value.
func (x *SizeEstimation) SetValue(value uint64) {
x.m.SetUsedSpace(value)
}
// Value returns data size estimation set using SetValue.
//
// Zero SizeEstimation has zero value.
func (x SizeEstimation) Value() uint64 {
return x.m.GetUsedSpace()
}

View file

@ -1,94 +0,0 @@
package container_test
import (
"crypto/sha256"
"testing"
v2container "github.com/TrueCloudLab/frostfs-api-go/v2/container"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs"
"github.com/TrueCloudLab/frostfs-sdk-go/container"
cid "github.com/TrueCloudLab/frostfs-sdk-go/container/id"
cidtest "github.com/TrueCloudLab/frostfs-sdk-go/container/id/test"
"github.com/stretchr/testify/require"
)
func TestSizeEstimation_Epoch(t *testing.T) {
var val container.SizeEstimation
require.Zero(t, val.Epoch())
const epoch = 123
val.SetEpoch(epoch)
require.EqualValues(t, epoch, val.Epoch())
var msg v2container.UsedSpaceAnnouncement
val.WriteToV2(&msg)
require.EqualValues(t, epoch, msg.GetEpoch())
}
func TestSizeEstimation_Container(t *testing.T) {
var val container.SizeEstimation
require.Zero(t, val.Container())
cnr := cidtest.ID()
val.SetContainer(cnr)
require.True(t, val.Container().Equals(cnr))
var msg v2container.UsedSpaceAnnouncement
val.WriteToV2(&msg)
var msgCnr refs.ContainerID
cnr.WriteToV2(&msgCnr)
require.Equal(t, &msgCnr, msg.GetContainerID())
}
func TestSizeEstimation_Value(t *testing.T) {
var val container.SizeEstimation
require.Zero(t, val.Value())
const value = 876
val.SetValue(value)
require.EqualValues(t, value, val.Value())
var msg v2container.UsedSpaceAnnouncement
val.WriteToV2(&msg)
require.EqualValues(t, value, msg.GetUsedSpace())
}
func TestSizeEstimation_ReadFromV2(t *testing.T) {
const epoch = 654
const value = 903
var cnrMsg refs.ContainerID
var msg v2container.UsedSpaceAnnouncement
var val container.SizeEstimation
require.Error(t, val.ReadFromV2(msg))
msg.SetContainerID(&cnrMsg)
require.Error(t, val.ReadFromV2(msg))
cnrMsg.SetValue(make([]byte, sha256.Size))
var cnr cid.ID
require.NoError(t, cnr.ReadFromV2(cnrMsg))
msg.SetEpoch(epoch)
msg.SetUsedSpace(value)
require.NoError(t, val.ReadFromV2(msg))
require.EqualValues(t, epoch, val.Epoch())
require.EqualValues(t, value, val.Value())
require.EqualValues(t, cnr, val.Container())
}

View file

@ -3,11 +3,10 @@ package containertest
import ( import (
"math/rand" "math/rand"
"github.com/TrueCloudLab/frostfs-sdk-go/container" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
"github.com/TrueCloudLab/frostfs-sdk-go/container/acl" "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/acl"
cidtest "github.com/TrueCloudLab/frostfs-sdk-go/container/id/test" netmaptest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap/test"
netmaptest "github.com/TrueCloudLab/frostfs-sdk-go/netmap/test" usertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user/test"
usertest "github.com/TrueCloudLab/frostfs-sdk-go/user/test"
) )
// Container returns random container.Container. // Container returns random container.Container.
@ -16,22 +15,13 @@ func Container() (x container.Container) {
x.Init() x.Init()
x.SetAttribute("some attribute", "value") x.SetAttribute("some attribute", "value")
x.SetOwner(*owner) x.SetOwner(owner)
x.SetBasicACL(BasicACL()) x.SetBasicACL(BasicACL())
x.SetPlacementPolicy(netmaptest.PlacementPolicy()) x.SetPlacementPolicy(netmaptest.PlacementPolicy())
return x return x
} }
// SizeEstimation returns random container.SizeEstimation.
func SizeEstimation() (x container.SizeEstimation) {
x.SetContainer(cidtest.ID())
x.SetEpoch(rand.Uint64())
x.SetValue(rand.Uint64())
return x
}
// BasicACL returns random acl.Basic. // BasicACL returns random acl.Basic.
func BasicACL() (x acl.Basic) { func BasicACL() (x acl.Basic) {
x.FromBits(rand.Uint32()) x.FromBits(rand.Uint32())

View file

@ -1,12 +1,12 @@
package frostfscrypto_test package frostfscrypto_test
import ( import (
"math/rand" "crypto/rand"
"testing" "testing"
"github.com/TrueCloudLab/frostfs-api-go/v2/refs" "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
frostfscrypto "github.com/TrueCloudLab/frostfs-sdk-go/crypto" frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
frostfsecdsa "github.com/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa" frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )

View file

@ -25,11 +25,11 @@ PublicKey allows to verify signatures.
// ... // ...
Signature can be also used to process FrostFS API V2 protocol messages Signature can be also used to process FrostFS API V2 protocol messages
(see neo.fs.v2.refs package in https://github.com/TrueCloudLab/frostfs-api). (see neo.fs.v2.refs package in https://git.frostfs.info/TrueCloudLab/frostfs-api).
On client side: On client side:
import "github.com/TrueCloudLab/frostfs-api-go/v2/refs" import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
var msg refs.Signature var msg refs.Signature
sig.WriteToV2(&msg) sig.WriteToV2(&msg)

View file

@ -1,6 +1,6 @@
package frostfsecdsa package frostfsecdsa
import frostfscrypto "github.com/TrueCloudLab/frostfs-sdk-go/crypto" import frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
func init() { func init() {
frostfscrypto.RegisterScheme(frostfscrypto.ECDSA_SHA512, func() frostfscrypto.PublicKey { frostfscrypto.RegisterScheme(frostfscrypto.ECDSA_SHA512, func() frostfscrypto.PublicKey {

View file

@ -6,7 +6,7 @@ import (
"crypto/rand" "crypto/rand"
"crypto/sha512" "crypto/sha512"
frostfscrypto "github.com/TrueCloudLab/frostfs-sdk-go/crypto" frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys" "github.com/nspcc-dev/neo-go/pkg/crypto/keys"
) )

Some files were not shown because too many files have changed in this diff Show more