Compare commits

...

518 commits

Author SHA1 Message Date
d342c0bc16 [#268] client: Make PayloadPatch correctly receive empty patch payload
* Make the method `PatchPayload` send a patch with empty
  payload patch if range's length is non-zero and if it's the
  first call.
* Empty payload patches just cut original object payload. So, these
  patches are also valid.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-12 15:12:52 +03:00
f0c599d06d [#268] client: Fix sequential PayloadPatch calls
* The flag 'firstPayloadPatch' keeps its state after first
  `PatchPayload` that make other calls incorrectly set patch
  ranges.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-12 15:09:01 +03:00
7d84d104fb [#260] *: Fix linter warnings
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:55:32 +00:00
812126a8ff [#260] .golangci.yml: Add protogetter linter
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:55:32 +00:00
d86223ed56 [#260] Makefile: Add pre-commit targets
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:55:32 +00:00
76a0cfdadb [#217] netmap: Return node netmap state directly
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:51:51 +00:00
46ee543899 [#265] go.mod: Use range over int
Since Go 1.22 a `for` statement with a `range` clause is able
to iterate through integer values from zero to an upper limit.

gopatch script:
@@
var i, e expression
@@
-for i := 0; i <= e - 1; i++ {
+for i := range e {
    ...
}

@@
var i, e expression
@@
-for i := 0; i <= e; i++ {
+for i := range e + 1 {
    ...
}

@@
var i, e expression
@@
-for i := 0; i < e; i++ {
+for i := range e {
    ...
}

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-09-04 12:37:46 +03:00
8f751d9dd0
[#263] go.mod: Update api-go
Remove `client.ContainerEACL` and related references. This change
was initiated by the removal of `ContainerService.GetExtendedACL`
from the API.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-03 12:36:28 +03:00
3c00f4eeac [#203] Add pool docs
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
2024-08-30 08:24:31 +03:00
f0b9493ce3 [#261] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-27 11:33:09 +03:00
28f140bf06 [#254] pool: Add parameter gracefulCloseOnSwitchTimeout
Add new param for waiting a little until current in-flight requests will be finished

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-22 08:02:51 +00:00
9115d3f281 [#1316] lint: Fix warnings
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-08-21 17:59:10 +03:00
cf225be0df [#1316] go.mod: Bump go version to 1.22
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-08-21 16:42:58 +03:00
338d1ef254 [#258] pool/tree: Add node address to error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-20 14:19:58 +03:00
6dd7be11d1 [#256] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-20 10:20:28 +03:00
203bba65a0 [#253] pool: Don't count regular FrostFS errors
Previously we count all frostfs errors like:
ObjectNotFound, EACLNotFound
because frostfs status is unconditionally resolved into built-in go errors
but handleError method handled built-in errors like internal network ones.
Since after resolving frostfs errors status is also returned we start check this first

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-14 16:48:51 +03:00
98aabc45a7 [#252] patcher: Fix applying patch from the same offset
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-13 18:58:21 +03:00
908c96a94d [#251] pool: Fix handlerError panic for objectPatch
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-13 15:28:39 +03:00
2077b35736 [#231] netmap: Add LIKE operation for filter
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-12 09:49:12 +03:00
92c7596157 [#231] go.mod: Update version for frostfs-api-go/v2
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-12 09:49:04 +03:00
a15b1264f5 [#231] policy: Bump jre version to 17
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-12 09:44:30 +03:00
5d58519253 [#249] pool: Introduce objectPatch method
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-08 18:26:40 +03:00
93171b3319 [#249] session: Support patch verb
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-08 18:26:40 +03:00
3ba7446157 [#249] client: Introduce ObjectPatch
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-08 18:26:35 +03:00
335aa18dc6 [#249] go.mod: Bump frostfs-api-go/v2 version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-06 16:57:42 +03:00
361739e860 [#247] object: Introduce patcher package
* Introduce `patcher` package that contains such interfaces to be
  implemented:
  - `PatchApplier` - the main patching engine that merges the stream
    of patches and the stream of original object payload divided by
    ranges. The merged streams result is output to `ChunkedObjectWriter`;
  - `RangeProvider` - provides the original object payload by ranges;
  - `HeaderProvider` - provides the original object header.
* Introduce `patcher` that implements `PatchApplier`;
* Cover all possible cases with unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-05 12:32:14 +00:00
6dd500def9 [#247] object: Introduce Patch type
* Make ToV2, FromV2 converters for Patch and PayloadPatch;
* Add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-05 12:32:14 +00:00
e83d6b7c6a [#244] pool/tree: Collect request duration statistic
After each request for tree pool statistic accumulated values are reset to zero.

Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-08-02 13:01:14 +03:00
9da46f566f [#243] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-26 14:13:49 +03:00
fa89999d91 [#242] pool: Log error that caused healthy status change
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-22 15:12:27 +03:00
7e94a6adf2 [#237] pool: Return creation epoch from object put
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-22 06:15:23 +00:00
ce8270568d [#239] go.mod: Update frostfs-contract
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-07-18 17:17:40 +03:00
7c06cdff2d [#239] pool/tree: Update tree service client
Update tree service to fix split tree problem.
Tree intermediate nodes can be duplicated so we must handle this.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-07-17 14:32:04 +03:00
e18b916231 [#238] go.mod: Update frostfs-contract to v0.19.3
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-07-16 11:36:21 +03:00
c4ff8a6cda [#236] go.mod: Update neo-go to v0.106.2
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 15:43:11 +03:00
fc7c524fcb [#236] *: Replace slice.Copy() with bytes.Clone()
The former is deprecated.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 15:43:07 +03:00
e977b8a94c [#236] netmap: Remove unused field from meanIQRAgg
It was there since the inception [1], but we never got to use it.

[1] 5931284e07

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:07 +03:00
6729f54c4e [#236] netmap: Reuse slice for weights in ContainerNodes()
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │    alloc     │               weights               │
                                                              │    sec/op    │    sec/op     vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.677µ ±  6%   8.384µ ± 10%       ~ (p=0.247 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.946µ ± 14%   7.998µ ±  6%       ~ (p=0.481 n=10)
geomean                                                         8.303µ         8.189µ        -1.38%

                                                              │    alloc     │               weights               │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   7.734Ki ± 0%   7.617Ki ± 0%  -1.52% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.156Ki ± 0%   7.039Ki ± 0%  -1.64% (p=0.000 n=10)
geomean                                                         7.440Ki        7.322Ki       -1.58%

                                                              │   alloc    │              weights               │
                                                              │ allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   92.00 ± 0%   77.00 ± 0%  -16.30% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   92.00 ± 0%   77.00 ± 0%  -16.30% (p=0.000 n=10)
geomean                                                         92.00        77.00       -16.30%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:07 +03:00
159a50fcf0 [#236] netmap: Reduce allocations in getSelection()
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │     new      │                alloc                 │
                                                              │    sec/op    │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                   9.227µ ± 13%   8.677µ ±  6%        ~ (p=0.165 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   9.189µ ±  7%   7.946µ ± 14%  -13.53% (p=0.001 n=10)
geomean                                                         9.208µ         8.303µ         -9.82%

                                                              │     new      │                alloc                │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.320Ki ± 0%   7.734Ki ± 0%  -7.04% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   7.742Ki ± 0%   7.156Ki ± 0%  -7.57% (p=0.000 n=10)
geomean                                                         8.026Ki        7.440Ki       -7.31%

                                                              │     new     │               alloc                │
                                                              │  allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   122.00 ± 0%   92.00 ± 0%  -24.59% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   122.00 ± 0%   92.00 ± 0%  -24.59% (p=0.000 n=10)
geomean                                                          122.0        92.00       -24.59%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:07 +03:00
a69f00903c [#236] netmap: Replace sort.Slice() with slices.Sort()
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                              │      old      │                 new                  │
                                                              │    sec/op     │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                   10.395µ ± 14%   9.227µ ± 13%  -11.24% (p=0.015 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   10.110µ ± 16%   9.189µ ±  7%        ~ (p=0.105 n=10)
geomean                                                          10.25µ         9.208µ        -10.18%

                                                              │     old      │                 new                 │
                                                              │     B/op     │     B/op      vs base               │
Netmap_ContainerNodes/REP_2-8                                   8.695Ki ± 0%   8.320Ki ± 0%  -4.31% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   8.117Ki ± 0%   7.742Ki ± 0%  -4.62% (p=0.000 n=10)
geomean                                                         8.401Ki        8.026Ki       -4.47%

                                                              │    old     │                new                 │
                                                              │ allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   138.0 ± 0%   122.0 ± 0%  -11.59% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   138.0 ± 0%   122.0 ± 0%  -11.59% (p=0.000 n=10)
geomean                                                         138.0        122.0       -11.59%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:07 +03:00
9d89f08c7b [#236] go.mod: Bump min go version to go1.21
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-12 14:25:06 +03:00
51cefd4908 [#232] netmap: Allow empty values for unknown parameters in network config
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-07-09 12:19:25 +03:00
560cbbd1f1 [#234] pool: Update token expiration check in cache
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-05 12:36:17 +03:00
27e965007d [#233] pool: Introduce ape-manager methods
* Remove GetEACL, SetEACL methods as they are deprecated;
* Fix mock;
* Introduce add/remove/list methods to request ape-manager from
  the client.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-04 01:29:35 +03:00
1a5886e776 [#228] client: Move isClientErrMaintenance from node
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-06-17 17:07:30 +03:00
ebd8fcd168 [#224] object: Introduce parent attributes in EC-header
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 16:20:48 +03:00
717a7d00ef [#225] bearer: Introduce APEOverride field for the token
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 12:14:42 +00:00
dd23c6fd2b [#225] apemanager: Move apemanager to ape package
* Update go.mod;
* Fix packages;
* Fix client's `apemanager` methods.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 12:14:42 +00:00
6a52487edd [#226] pool/tree: Fix handling access denied error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-05-30 14:59:35 +03:00
c5c6272029 [#221] pool: Make sampler safe for concurrent using
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-05-20 14:14:14 +03:00
3de256d05e [#223] object: Introduce new fields for ECHeader
* Introduce `parentSplitID`, `parentSplitParentID` fields
  for `ECHeader`;
* Fix ECHeader's constructor;
* Fix `Split` and `Reconstruct`;
* Add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-17 15:16:28 +03:00
09b79d13f3 [#223] object: Introduce ec_parent search filter
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-17 15:03:40 +03:00
d4e6f4e125 [#223] go.mod: Update frosts-api-go/v2 version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-17 15:02:44 +03:00
b2ad1f3b3e [#215] client: Introduce apemanager rpc interface
* Introduce `APEManagerAddChain`, `APEManagerRemoveChain`, `APEManagerListChains`.
* Introduce reqeuest/response types for these handlers (Prm*, Res*).
* Inroduce status type for apemanager `APEManagerAccessDenied`; add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-14 10:59:52 +03:00
32a975a20d [#215] apemanager: Introduce apemanager types
* Introduce `Chain`, `ChainTarget` and `TargetType`.
* Implement api-v2 converters for the introduced types.
* Add unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-14 10:59:49 +03:00
eaf36706a2 [#215] go.mod: Update frostfs-api-go/v2 package version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-08 11:18:18 +03:00
02c936f397 [#216] netmap: Add policy decode fuzz test
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-07 11:10:37 +00:00
99e02858af [#220] netmap: Fix setters for Replica.DataCount/ParityCount
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-07 09:34:14 +03:00
12ddefe078 [#218] object: Implement Range\RangeHash requests for EC object
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-02 11:01:21 +03:00
20ab57bf7e [#214] object: Implement Get\Head requests for EC object
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 11:07:26 +03:00
3790142b10 [#212] pool: Control sub tree nodes order
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-04-02 17:15:49 +03:00
ec0cb2169f [#211] object: Fix setIDWithSignature
* Calculate and set checksum before ID is calculated.
* Add header verification for parts in unit-test.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-29 13:48:04 +03:00
425d48f68b [#211] netmap: Introduce ReplicaDescriptor method
* Make ReplicaNumberByIndex deprecated.
* Introduce ReplicaDescriptor method that access i-th replica directly.
* Introduce new getters for ReplicaDescriptor.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-29 13:48:04 +03:00
6d0da3f861 [#211] go.mod: Update frostfs-api-go version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-29 13:48:04 +03:00
1af9b6d18b [#155] sdk-go: Add buffer support for payloadSizeLimiter
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-03-25 10:47:04 +03:00
bd2d350b09 [#205] object: Initial EC implementation
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
e9be3e6d94 [#205] netmap: Add well-known EC parameters to network config
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
70e9e40c7f [#205] netmap: Add EC statement to placement policy
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
d33b54d280 [#205] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-22 10:14:12 +00:00
6f248436a5 [#210] client/put_transformer: Fix error handling
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-03-19 15:36:51 +03:00
edd40474e8 [#209] pre-commit: Add unit-test hook
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 13:35:48 +03:00
d9ec7c1988 [#209] Makefile: Allow to override test flags
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 13:35:48 +03:00
64b83f8220 [#209] Makefile: Update golangci-lint to 1.56.2
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 13:35:48 +03:00
7212f38115 [#209] pre-commit: Remove gitlint
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 13:35:48 +03:00
8081445ff2 [#208] go.mod: Bump frostfs-api-go version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-06 13:34:04 +03:00
6a7ef9d8c3 [#208] go.mod: Bump protobuf version
Found by vulncheck:
Vulnerability #1: GO-2024-2611
    Infinite loop in JSON unmarshaling in google.golang.org/protobuf
  More info: https://pkg.go.dev/vuln/GO-2024-2611
  Module: google.golang.org/protobuf
    Found in: google.golang.org/protobuf@v1.32.0
    Fixed in: google.golang.org/protobuf@v1.33.0

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-06 13:23:32 +03:00
6fe4e2541d [#207] netmap: Fix string escape in PlacementPolicy.String()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-01 15:02:05 +00:00
a5fab572ff [#206] Add session tokens for container read operations
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-03-01 15:18:53 +03:00
a86170f53a [#202] object: Reset marshal data on CutPayload
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 15:23:47 +03:00
aa41f71dcc [#202] eacl: Drop storage group test
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 15:23:47 +03:00
3a00fd51e4 [#202] go.mod: Update api-go version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 15:23:43 +03:00
65b4525b3b [#198] object/user: Add ScriptHash method
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-01-26 14:10:09 +00:00
7efff9d53d [#201] .forgejo: Update dco-go to v3
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-26 12:21:42 +03:00
110b7e4170 [#200] pre-commit: Fix linter invocation target
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-01-17 17:56:20 +03:00
56debcfa56 [#190] sdk-go: Gofumpt fixes
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 19:21:20 +03:00
157a9930e8 [#190] sdk-go: Pass user.ID by value
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 19:21:20 +03:00
1c07098740 [#194] make: Fix make test
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-17 11:48:51 +00:00
03d35dd1f3 [#166] netmap: Add support YML tests
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-17 14:43:03 +03:00
dea8759762 [#166] netmap: Move tests from JSON to YML
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-17 14:42:39 +03:00
3787477133 [#189] client: Make PrmDial fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-14 11:18:00 +03:00
e91d40e250 [#189] client: Make PrmInit fields public for client
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-13 15:18:07 +03:00
ab75edd709 [#191] pool/tree: Support limit request attempts
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-07 14:45:40 +03:00
8999d2f080 [#191] pool/tree: Support request id
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-07 14:45:35 +03:00
6fbe1595cb [#121] pool: Refactor PrmObjectSearch usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-01 17:45:15 +03:00
a9237aabd2 [#121] client: Make PrmObjectSearch fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-01 17:45:06 +03:00
a487033505 [#121] client: Nuke out unused prmCommonMeta
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-10-31 22:55:25 +03:00
51c3618850 [#121] client: Make PrmContainerList fields public
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-10-31 22:55:25 +03:00
665e5807bc [#188] transformer: Allow to provide size hint
For big objects with known size we can optimize allocation patterns
by providing size hint. As with any hint, it does not affect transformer
functionality: slices with capacity > MaxSize are never allocated.

```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                                │     out     │
                                                │   sec/op    │
Transformer/small/no_size_hint-8                  65.44µ ± 3%
Transformer/small/no_size_hint,_with_buffer-8     64.24µ ± 5%
Transformer/small/with_size_hint,_with_buffer-8   58.70µ ± 5%
Transformer/big/no_size_hint-8                    367.8m ± 3%
Transformer/big/no_size_hint,_with_buffer-8       562.7m ± 0%
Transformer/big/with_size_hint,_with_buffer-8     385.6m ± 7%
geomean                                           5.197m

                                                │     out      │
                                                │     B/op     │
Transformer/small/no_size_hint-8                  13.40Ki ± 0%
Transformer/small/no_size_hint,_with_buffer-8     13.40Ki ± 0%
Transformer/small/with_size_hint,_with_buffer-8   13.39Ki ± 0%
Transformer/big/no_size_hint-8                    288.0Mi ± 0%
Transformer/big/no_size_hint,_with_buffer-8       1.390Gi ± 0%
Transformer/big/with_size_hint,_with_buffer-8     288.0Mi ± 0%
geomean                                           2.533Mi

                                                │    out     │
                                                │ allocs/op  │
Transformer/small/no_size_hint-8                  92.00 ± 0%
Transformer/small/no_size_hint,_with_buffer-8     92.00 ± 0%
Transformer/small/with_size_hint,_with_buffer-8   92.00 ± 0%
Transformer/big/no_size_hint-8                    546.5 ± 0%
Transformer/big/no_size_hint,_with_buffer-8       607.5 ± 0%
Transformer/big/with_size_hint,_with_buffer-8     545.5 ± 0%
geomean                                           228.1
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-30 12:13:04 +00:00
a02c0bfac8 [#186] netmap: Marshal policy with brackets
Brackets can be semantically important and must not be omitted,
otherwise the output is plain wrong.
We do not take the responsibility to preserve every bracket, though,
because parser does some optimizations related to grouping long chains
of filters combined with the same operation.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-27 10:54:45 +03:00
20d325e307 [#167] netmap: Fix reverse min agregator
The higher the price, the lower reverse min weight should be.
Previously nodes with 0 price had 0 weight which is a bit misleading.

Introduced in d71a0e0755.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-27 07:53:19 +00:00
670619d242 [#131] client: keep backwards-compatibility, update README.md, fix chore
Signed-off-by: Egor Olefirenko <egor.olefirenko892@gmail.com>
2023-10-26 14:35:49 +00:00
0d79d10482 [#131] client: rename option consistently and fix test
Signed-off-by: Egor Olefirenko <egor.olefirenko892@gmail.com>
2023-10-26 14:35:49 +00:00
9727beb47d [#131] client: Switch ResolveFrostFSFailures to DontResolveFrostFSFailures option
Signed-off-by: Egor Olefirenko <egor.olefirenko892@gmail.com>
2023-10-26 14:35:49 +00:00
84315fab6a [#121] client: Make PrmBalanceGet fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-23 18:53:14 +03:00
71335489ae [#183] forgejo: Make linter great again
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-23 18:13:40 +03:00
4c1feaf2cb [#182] pool: Fix linter error about deprecated methods
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-23 17:40:00 +03:00
5804128ff3 [#121] client: Make PrmObjectPutSingle fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-19 08:38:13 +00:00
abd38c918e [#177] pool: Support NetMapSnapshot method
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-18 11:31:53 +03:00
fc4551b843 [#172] pool: Use priority of errors in tree pool
When retry happens, use priority map to decide
which error to return. Consider network errors
less desirable than business logic errors.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-11 12:00:34 +03:00
eb5288f4a5 [#172] pool: Do more retries on unexpected tree service responses
1. Try its best by looking for nodes during 'GetNodeByPath'
2. Retry on 'tree not found' and other not found errors

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-06 11:12:48 +03:00
60463871db [#171] pool: Add test for healthy status monitor
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-03 19:47:22 +03:00
8a04638749 [#171] pool: Close only dialed connections
To avoid panics during close operation, close
only dialed connections.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-03 19:47:22 +03:00
ddbfb758c9 [#171] pool: Use dial status to close connections during restarts
Every client restart, pool creates new client instance. If client
failed due to dial error, there was no prior connection and go
routine on a server side. If client failed due to communication
or business logic errors, then server side maintains connection and
client should close it to avoid routine and connection leak.

Dialing is a part of healthcheck, so health status is now a enum
of three values:
- unhealthy due to dial fail,
- unhealthy due to transmission fail,
- healthy.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-03 17:00:36 +03:00
d71a0e0755 [#88] netmap: use bool, fix hrw_sort tests
Signed-off-by: Andrew Danilin <andnilin@gmail.com>
2023-10-03 07:05:03 +00:00
163b3e1961 [#88] netmap: fix min aggregator bug, add tests
Signed-off-by: Andrew Danilin <andnilin@gmail.com>
2023-10-03 07:05:03 +00:00
84b9d29fc9 [#170] checksum: Use constant mapping for checksum types
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-28 17:20:24 +03:00
99c273f499 [#169] pool: Close inner pools during close routine
Some apps do not reuse pool instance and expect that
`pool.Close()` free resources. But it didn't actually
close inner SDK clients, so it leads to goroutine leak
in storage.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-09-20 12:16:13 +03:00
555ccc63b2 [#167] netmap: Allow to select insufficient number of nodes
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-15 14:47:54 +03:00
0550438b53 [#167] netmap/tests: Add replica to invalid tests
Make sure we fail exactly because of the reason specified.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-15 14:33:44 +03:00
c899163860 [#167] netmap/tests: Add json file name to the test output
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-15 14:33:44 +03:00
ac8fc6d440 [#162] netmap: Allow to parse single unnamed selectors
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-11 15:22:24 +03:00
0a0b590df3 [#162] Fix pre-commit warnings
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-11 15:21:35 +03:00
4df642e941 [#162] netmap: Fix possible panic
Placement policy is unvalidated external input.
Under no circumstances should we panic here.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-11 15:19:56 +03:00
8bc64e088e [#161] .golangci.yml: Reenable deprecated usage warnings
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-08 17:17:02 +03:00
49ad985cad [#161] *: Do not use math/rand.Read()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-08 17:17:02 +03:00
aa12d8c6a6 [#121] client: Make PrmObjectHash fields public
* Introduce buildRequest for PrmObjectHash

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-08 13:36:45 +00:00
303508328a [#121] pool: Refactor PrmSessionCreate usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-05 17:32:03 +03:00
55699d1480 [#121] client: Make PrmSessionCreate fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-05 17:30:42 +03:00
55a1f23e71 [#121] client: Make PrmEndpointInfo, PrmNetworkInfo fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-04 19:55:23 +03:00
291a71ba84 [#121] client: Make PrmAnnounceSpace fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-04 19:53:34 +03:00
5a471e5002 [#121] client: Make PrmObjectDelete fields public
* Introduce buildRequest for PrmObjectDelete
* Refactor the usage of these params in pool

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-04 14:14:22 +00:00
b5fe52d6bd [#150] policy: Check for redundant selectors and filters
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-29 14:16:57 +03:00
84e7e69f98 [#121] client: Make PrmObjectGet/Head/GetRange fields public
* Remove common PrmObjectRead structure
* Introduce buildRequest for PrmObjectGet/Head/GetRange
* Refactor the usage of these params in pool

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-28 11:26:57 +03:00
46a214d065 [#149] pool: Configure homomorphic hash and buffer size
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-25 09:45:15 +03:00
202412230a [#115] pool: Drop part buffer pool
Tests showed that using part buffer pool doesn't save memory a lot.
Especially on big parts.
Probably we can use pool only for small parts
after adding buffer in payloadSizeLimiter

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-21 12:03:03 +03:00
3cb3841073 [#115] pool: Try putSingle if possible
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-21 12:03:03 +03:00
faeeeab87a [#114] pool: Don't use part buffers when client cut is off
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-21 12:02:40 +03:00
cae215534f [#114] pool: Fix linter errors
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-21 12:02:40 +03:00
518fb79bc0 [#114] pool: Support client cut with memory limiter
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-21 12:02:40 +03:00
342524159a [#121] pool: Make PrmContainerSetEACL fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-21 10:33:19 +03:00
22978303f8 [#121] clientt: Make PrmContainerSetEACL fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-18 18:05:47 +03:00
6fdbe75517 [#121] pool: Make PrmContainerEACL fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-16 09:53:47 +00:00
3353940554 [#121] client: Make PrmContainerEACL fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-16 09:53:47 +00:00
a3b5d4d4f5 [#51] Add node addresses as debug information
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-08-15 09:51:40 +03:00
0314b326d3 [#51] Add current nodes as external statistics
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-08-15 09:51:13 +03:00
0382785763 [#146] .forgejo: Update DCO action
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-11 12:34:29 +03:00
548a81d3e6 [#48] client: Refactor accounting.Balance()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-10 11:05:30 +00:00
d48788c7a9 [#144] Bump required go version to go1.20
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-09 09:52:35 +03:00
6353df8bca [#142] Fix unwrapErr for go 1.20
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-05 15:58:25 +00:00
936e6d230b [#121] pool: Add wait params validation for containerPut method
* Add WaitParams.CheckValidity() check because SetWaitParams is deprecated,
  but parameters were checked within this setter with checkForPositive()

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-05 15:58:14 +00:00
be28b89312 [#121] pool: Make PrmContainerDelete fields public
* Refactor client PrmContainerDelete usage
* Introduce WaitParams CheckValidity method

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-05 15:58:14 +00:00
9e5faaf829 [#121] client: Make PrmContainerDelete fields public
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-05 15:58:14 +00:00
3dc8129ed7 [#135] Make all error status receivers pointers
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-04 08:35:01 +00:00
55c52c8d5d [#121] pool: Make PrmContainerGet fields public
* Also refactor client PrmContainerGet usage

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-03 10:54:13 +03:00
d376302a3b [#121] client: Make PrmContainerGet fields public
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-02 18:23:32 +03:00
363f153eaf [#136] pool: Set order field to get subtree
With new revision of tree service protocol, getSubTree
requires to use explicit order field.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-08-02 10:32:37 +00:00
95b987b818 [#137] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-02 12:04:10 +03:00
13d0b170d2 [#121] client: Make pool PrmContainerPut struct fields public
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-01 09:59:57 +00:00
18a9e4bceb [#121] client: Make PrmContainerPut struct fields public
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-01 09:59:57 +00:00
0fe0d71678 [#133] .forgejo: Add names to actions
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-31 11:33:39 +00:00
78d1439b2c [#133] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-31 11:33:39 +00:00
0886d80083 [#69] Add close for nns.Dial
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2023-07-31 11:05:26 +03:00
ecb1fef78c [#129] client: Do not override error status WriteObject()
Here is a scenario:
1. `resolveFrostFSErrors` is false.
2. The first object part was not written, which was signified in status.
3. The second part was written correctly.

Client now thinks that the object is written even though it was not.
In theory we could also return only status, but client-side splitting
is not a single RPC, so it makes sense.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-26 14:38:43 +03:00
5defed4ab4 [#126] go.mod: Update frostfs-api-go package version
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-07-19 16:03:56 +03:00
fb05f7dc5e [#112] Add basic documentation for placement policies
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-18 10:53:54 +00:00
b91f9d8c79 [#xx] Add support for SELECT-FILTER expressions
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-17 10:29:48 +00:00
b9afe7a2f9 [#42] Add ResolveContractHash method
Signed-off-by: Pavel Pogodaev <p.pogodaev@yadro.com>
2023-07-12 22:22:58 +03:00
998fe1a7ab [#102] netmap: properly process multiple REP
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-11 14:21:35 +00:00
c359a7465a [#64] transformer: Simplify interface
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-11 07:33:12 +00:00
d70ef2187b [#97] Add a method IterateUserAttributes in Container
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-07-10 09:05:01 +00:00
ac95b87e7c [#101] Add Equals for Address
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-07-10 09:05:01 +00:00
863be6034f [#104] Update neo-go/pkg/interop version
neo-go module uses broken commit of interop package.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-10 09:04:10 +00:00
35346a01c9 [#109] Bump neo-go version
Synced version with frostfs-node, see frostfs-node#417

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-10 09:04:10 +00:00
fe35373d8f [#107] go.mod: Tidy dependencies
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-07 14:57:16 +03:00
388d1ca1de [#107] pool/tree: Support grpc schemas
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-07-07 14:57:13 +03:00
14ed3e177d [#104] nemtap: Escape special symbols in filters
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-07 09:33:00 +00:00
fe28c33277 [#104] netmap: Add test with quote escaping
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-07-07 09:33:00 +00:00
98cab7ed61 [#103] object: Add PutSingle method code
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-06 17:06:17 +03:00
37e22b33ad [#103] sdk-go: Update api-go version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-05 18:10:42 +03:00
769f6eec05 [#105] client: Fix revive linter warning
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-05 15:52:06 +03:00
5d62cef27e [#98] Fix linter issues
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-28 12:13:02 +00:00
c0c0c588b5 [#98] Add forgejo workflows
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-06-28 12:13:02 +00:00
2f88460172 [#83] Allow to split objects in the client
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-28 12:12:37 +00:00
66cb5dcf34 [#83] Update .gitignore
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-28 12:12:37 +00:00
91e80ba743 [#73] pool/tree: Fix index in retry loop
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-28 13:06:23 +03:00
c243b443bc [#80] objectwriter: Allow custimize maxChunkLen
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-26 18:13:04 +03:00
aa8ffebc63 [#95] transformer: Set parent version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-23 13:48:02 +03:00
9d40228cec [#73] pool/tree: Fix client healthy status
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 17:01:55 +03:00
af40dc68f0 [#84] pool/tree: Allow to pass gRPC dial options
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 15:55:07 +03:00
981d24a493 [#84] pool: Allow to pass gRPC dial options
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 15:54:48 +03:00
19adb4dffa [#73] pool/tree: Add tests
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 14:37:57 +03:00
51e022ab8c [#73] pool/tree: Add tree pool
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 14:37:57 +03:00
0d3dacb515 [#73] pool: Add getters for NodeParam
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 14:37:57 +03:00
b2e302624d [#73] pool/tree: Add proto tree service client
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-06-08 14:37:57 +03:00
fcbf96add6 [#76] Add UNIQUE keyword
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-06-06 13:54:07 +03:00
4f48f6c9e0 [#78] netmap: Add new keywords NOT and UNIQUE
* Add the rule for NOT operation to the policy parser grammar
* Regenerate query parse
* Implement NOT in filter
* Add unit-tests

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-06-02 17:47:20 +03:00
ec59ebfd88 [#87] go.mod: Update hrw
```
                                                              │      old      │                 new                  │
                                                              │    sec/op     │    sec/op     vs base                │
Netmap_ContainerNodes/REP_2-8                                    13.07µ ± 12%   11.03µ ± 12%  -15.63% (p=0.003 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   11.383µ ±  9%   9.970µ ± 19%  -12.42% (p=0.005 n=10)
geomean                                                          12.20µ         10.49µ        -14.04%

                                                              │      old      │                 new                  │
                                                              │     B/op      │     B/op      vs base                │
Netmap_ContainerNodes/REP_2-8                                   10.203Ki ± 0%   7.711Ki ± 0%  -24.43% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8    9.641Ki ± 0%   7.148Ki ± 0%  -25.85% (p=0.000 n=10)
geomean                                                          9.918Ki        7.424Ki       -25.14%

                                                              │    old     │                new                 │
                                                              │ allocs/op  │ allocs/op   vs base                │
Netmap_ContainerNodes/REP_2-8                                   216.0 ± 0%   131.0 ± 0%  -39.35% (p=0.000 n=10)
Netmap_ContainerNodes/REP_2_IN_X_CBF_2_SELECT_2_FROM_*_AS_X-8   215.0 ± 0%   130.0 ± 0%  -39.53% (p=0.000 n=10)
geomean                                                         215.5        130.5       -39.44%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-02 14:54:40 +03:00
030ff2f122 [#87] netmap: Add benchmark for ContainerNodes()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-02 14:54:40 +03:00
0f7455ff7a [#75] Update antlr4 version to 4.13.0
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-01 13:15:11 +00:00
e6b662cfa6 [#75] Add make targets policy and docker/%
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-06-01 13:15:11 +00:00
406c2324d4 [#85] go.mod: Tidy
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-01 10:02:23 +03:00
10482ffbed [#82] client: Allow to pass gRPC dial options
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-30 16:51:22 +03:00
f5b23eb225 [#74] netmap/parser: Update antlr4 generator
Also, add -no-listener option, as we use visitor only.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 17:47:24 +03:00
70f23dd1ea [#74] pre-commit: Exclude auto-generated files
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 17:47:24 +03:00
57f874048b [#74] go.mod: Update antlr4 dependency
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 17:47:24 +03:00
a397d1fd15 [#74] go.mod: Update dependencies
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 17:47:23 +03:00
9803c2816a [#74] pool: move to sync/atomic
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 14:58:30 +03:00
d04d96b42e [#74] go.mod: Move to go1.19
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 14:53:47 +03:00
9a072a8f49 [#68] Replace interface{} with any
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-15 17:21:49 +03:00
15b4287092 [#49] bearer: Allow empty eacl if token is impersonated
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-05-05 12:45:39 +03:00
d4fe9a193d [#66] transformer: Accept constructor in NextTarget
The code of frostfs-node is not yet ready to reuse egress target for
multiple objects, let's postpone until #64.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 11:22:09 +03:00
c42a6119ff [#66] transformer: Extend tests
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 11:22:09 +03:00
29b188db57 [#52] Remove storage groups and audit
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-03 08:20:37 +00:00
38b03ff28b [#63] transformer: Resolve funlen linter
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-25 13:29:42 +03:00
0fa23a9b14 [#63] transformer: Add context to methods
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-25 13:29:42 +03:00
d0762d037d [#44] pool: Add copies number vector when putting object
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
2023-04-20 13:41:05 +03:00
db5b89496d [#49] bearer: Consider impersonate in fillBody
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-18 14:54:05 +00:00
7c75db2f2d [#59] netmap: Remove unused param from getSelection()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 14:11:04 +00:00
dce55a436a [#9999] sdk: Up api-go version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-18 09:59:52 +00:00
cae2f37cdd client: rename formRequest() to buildRequest()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 09:58:34 +00:00
f60bea4be5 [#48] client: Refactor SessionCreate()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 09:58:34 +00:00
a16fc40c39 [#48] client: Refactor EndpointInfo()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 09:58:34 +00:00
40d966bec2 [#48] client: Refactor NetworkInfo()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-18 09:58:34 +00:00
Pavel Karpy
d0c5d837d2 [#56] *: Drop reputation system
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-18 07:52:16 +00:00
237b90f744 [#49] bearer: Add impersonate flag
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-04-17 11:51:38 +03:00
c8e620ad24 [#53] sdk-go: Drop subnet
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-14 16:31:08 +03:00
591dd1247d [#48] client: Refactor ContainerPut()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
57619fbbe4 [#48] client: Refactor ContainerSetEACL()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
8bc8f1f365 [#48] client: Refactor ContainerDelete()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
09ed2863fc [#48] client: Refactor ContainerEACL()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
8e2f77890f [#48] client: Refactor ContainerGet()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
8852b262f2 [#48] client: Refactor ContainerList()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
b2c66cb99e [#48] client: Refactor ContainerAnnounceUsedSpace()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
6c9b92c9dc [#48] client: Execute metainfo callback in processResponse()
It was lost in 8c5333ea55.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:07:18 +03:00
772fa90983 [#48] client/container: Form requests in a separate function
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 09:06:52 +03:00
1bfa9ecdb0 [#48] client: Remove ctx == nil checks
Much less useful after we made context to be passed explicitly.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 09:04:09 +03:00
55b06cd764 [#48] client: Split container methods by files
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-13 09:04:09 +03:00
Pavel Karpy
f41860f9bd [#46] client: Allow set copy_number for every placement vector
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-11 10:06:55 +00:00
Pavel Karpy
423b320f91 [#46] go.mod: Update api-go version
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-11 10:06:55 +00:00
f8c34b45f3 [#47] eacl: Fix duplicate imports
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
fa9573e857 [#47] client: Pass context to Dial() explicitly
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
bc62e2f712 [#47] pool: Resolve contextcheck and containedctx linters
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
31271ad8b1 [#47] netmap: Make PlacementPolicy.WriteStringTo() pass nofunlen, cogognit
REP statement is obligatory, thus insert newline before each statement
after REP.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
25e9336d68 [#47] container: Make readFromV2() pass funlen
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
4cd755877c [#47] client: Do not check context in NetmapSnapshot()
It is passed explicitly, non-nil by convention.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
1395b282fe [#47] acl: Remove unused resetBit()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
708d933fe3 [#47] .golangci.yml: Unify with other FrostFS repos
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
4fa52312c7 [#47] reputation: Fix misspelling
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-07 08:59:19 +03:00
552219b8e1 [#16] pool: Fix counting context canceled error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-29 15:58:04 +03:00
cfb8a7b914 [#41] .gitlint: Synchronize settings across FrostFS repos
This change allows to use `[#xx]` placeholders for issue number.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-24 22:07:13 +03:00
4438f115fb [#39] Add Issue Template
Add bug report and feature request templates

Signed-off-by: Liza <e.chichindaeva@yadro.com>
2023-03-23 12:25:57 +03:00
bec77f280a [#37] container: Support legacy sys attributes
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-16 11:14:42 +03:00
df2090c2be [#37] netmap: Update tests for new sys attributes
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-16 11:14:36 +03:00
7e6592b28e [#37] go.mod: Update api-go for new sys attributes
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-16 11:14:27 +03:00
d589d51509 [#19] transformer: Fix dependencies
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-16 09:51:03 +03:00
25588ee3be [#19] transformer: Do not allocate intermeate slice for hashers
```
name                 old time/op    new time/op    delta
Transformer/small-8    73.7µs ±15%    72.4µs ±16%    ~     (p=0.604 n=10+9)
Transformer/big-8       1.36s ± 4%     1.36s ± 8%    ~     (p=0.579 n=10+10)

name                 old alloc/op   new alloc/op   delta
Transformer/small-8    7.67kB ± 0%    7.57kB ± 0%  -1.36%  (p=0.000 n=10+10)
Transformer/big-8      49.0kB ± 0%    48.3kB ± 0%  -1.48%  (p=0.000 n=10+10)

name                 old allocs/op  new allocs/op  delta
Transformer/small-8       101 ± 0%        98 ± 0%  -2.97%  (p=0.000 n=10+10)
Transformer/big-8         609 ± 0%       591 ± 1%  -3.00%  (p=0.000 n=10+9)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
9407f30248 [#19] transformer: Optimize payload hashers a bit
```
name                 old time/op    new time/op    delta
Transformer/small-8    74.8µs ±11%    73.7µs ±15%    ~     (p=0.529 n=10+10)
Transformer/big-8       1.38s ±11%     1.36s ± 4%    ~     (p=0.796 n=10+10)

name                 old alloc/op   new alloc/op   delta
Transformer/small-8    7.69kB ± 0%    7.67kB ± 0%  -0.21%  (p=0.000 n=10+10)
Transformer/big-8      49.2kB ± 0%    49.0kB ± 0%  -0.48%  (p=0.004 n=10+10)

name                 old allocs/op  new allocs/op  delta
Transformer/small-8       102 ± 0%       101 ± 0%  -0.98%  (p=0.000 n=9+10)
Transformer/big-8         620 ± 1%       609 ± 0%  -1.66%  (p=0.000 n=10+10)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
94c0a607b5 [#19] transformer: Add a target which sends parts to a channel
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
e45647de3c [#19] transformer: Do not reuse memory of sent objects
Slower, but more correct.
```
name                 old time/op    new time/op    delta
Transformer/small-8    72.4µs ± 8%    74.8µs ±11%     ~     (p=0.278 n=9+10)
Transformer/big-8       1.31s ± 8%     1.38s ±11%   +5.50%  (p=0.035 n=10+10)

name                 old alloc/op   new alloc/op   delta
Transformer/small-8    7.39kB ± 0%    7.69kB ± 0%   +4.04%  (p=0.000 n=10+10)
Transformer/big-8      46.9kB ± 0%    49.2kB ± 0%   +4.87%  (p=0.000 n=10+10)

name                 old allocs/op  new allocs/op  delta
Transformer/small-8      94.6 ± 1%     102.0 ± 0%   +7.82%  (p=0.000 n=10+9)
Transformer/big-8         560 ± 0%       620 ± 1%  +10.66%  (p=0.000 n=10+10)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
611e20587b [#19] transformer/test: Check owner ID and payload hash for parts
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
eba6831125 [#19] transformer/test: Add helper functions
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
7e3810d654 [#19] transformer: Move EpochSource to other types
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
cc0fef2c55 [#19] transformer: Merge formatter and payload splitter
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
b696d3c70e [#19] transformer: Make writeChunk non-recursive
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
1c94309d7a [#19] transformer: Simplify AccessIdentifiers
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
f43f18ecda [#19] transformer: Cover with unit-tests
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
ac8442bf99 [#19] object: Move transformer implementation from node
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-15 05:39:09 +00:00
0ad877288e [TrueCloudLab#16] pool: Don't count grpc canceled error
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-03-10 06:58:34 +00:00
0e1999c965 [#23] pre-commit: Add gitlint hook
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-09 22:53:16 +03:00
b461aa64b8 [#23] pre-commit: Add golangci-lint hook
Also, fix minor issues. Skip deprecated warning for now.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-09 22:53:14 +03:00
b761fd8070 [#23] pre-commit: Add initial configuration
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-03-09 22:52:51 +03:00
94476f9055 Rename package name
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 15:47:21 +03:00
5e759bf089 [#2] Remove panic from RPCs
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-03-01 10:29:23 +03:00
d4f5bba459 [#2] Update lint config, fix lint errors
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-03-01 10:29:23 +03:00
e9c1a2ab2b [TrueCloudLab/hrw#2] sdk-go: Optimize node hash
Compute node hash by node initialization

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-02-28 18:07:14 +03:00
2cbc585edd [TrueCloudLab/hrw#2] sdk-go: Use typed HRW methods
Update HRW pkg and use typed HRW methods to sort nodes

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-02-28 18:07:14 +03:00
e355e5eeba [TrueCloudLab#22] .github: Fix CODEOWNERS
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-28 16:35:38 +03:00
f08069ceeb [#20] .github: Update CODEOWNERS
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-27 17:47:21 +03:00
dad99bad48 [#20] .github: Drop go1.17, add go1.20
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-27 17:47:21 +03:00
0d3a238d9c [#5] pool: Update hashicorp/lru to v2
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-24 17:32:40 +03:00
cf64ddfb14 [TrueCloudLab#13] pool: Renew token before it expired
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-01-30 15:06:02 +03:00
cf9a54dcda [TrueCloudLab#11] pool: Fix handling SplitInfoError
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-01-23 11:07:29 +03:00
Denis Kirillov
b2a37543d3 [#362] pool: Don't use default session token for read
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2023-01-13 13:21:29 +03:00
f0ac49b8f0 [TrueCloudLab#6] pool: Decrease rebalanceInterval
Set default rebalance interval to 15s

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-01-13 13:16:22 +03:00
54696acf48 [TrueCloudLab#6] pool: Log node healthy changing
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-01-13 13:16:22 +03:00
dd88a5c5e0 [#4] go.mod: Bump supported go version to 1.18
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 13:40:50 +03:00
4ff9c00de3 [#4] Rename NeoFS mentions in comments and method names
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 13:40:50 +03:00
b204a62da1 [#4] go.mod: Update ANTLR
Current `go:generate` command was, probably, executed only on my laptop.
Replace it with explicit version, because package in the generated code
depends on in.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 13:40:50 +03:00
377a5a0517 [#4] go.mod: Update dependencies
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 13:40:50 +03:00
43c046f343 [#4] go.mod: Update neo-go to v0.100.1
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 13:40:50 +03:00
Denis Kirillov
8c0c7789ca [#365] pool: Add request callback
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-29 16:34:13 +03:00
Evgenii Stratonikov
339e2702f8 [#367] client: Allow to override key in Object.Hash RPC
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-29 16:34:13 +03:00
Pavel Karpy
a1748ae0e7 [#370] bearer, session: Clarify expiration epoch
The expiration epoch is the _last_ valid epoch for an entity. Also, clarify
the expiration epoch meaning for tombstones and regular objects.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-29 16:34:13 +03:00
Pavel Karpy
4c779423f5 Move to frostfs-sdk-go
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-14 09:59:29 +03:00
Pavel Karpy
45a6e7a7c2 [#369] status: Make errors return default messages
Use default messages in `Error` methods like in `ToStatusV2`.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-14 09:59:29 +03:00
Denis Kirillov
b4b07a3c4e [#364] pool: Adopt client stream timeout param
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-15 17:08:20 +03:00
Denis Kirillov
d047289182 [#360] object: Add new package relations
Package relations provides feature to process inner object structure.

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-14 17:46:17 +03:00
Denis Kirillov
1cacf472a3 [#360] pool: Fix removing virtual object
Include phy objects in session token scope

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-14 17:46:17 +03:00
Denis Kirillov
e35f0df1ca [#358] pool: Make ErrPoolClientUnhealthy unexported
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-09 10:22:33 +03:00
Denis Kirillov
5f9d846fb4 [#358] pool: Don't init session with dead node
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-09 10:22:33 +03:00
Denis Kirillov
2eefdab0e4 [#358] pool: Start even if not all node healthy
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-11-09 10:22:33 +03:00
Denis Kirillov
7a2a76af95 [#356] ns: Use domain instead of name in NNS resolver
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-10-28 12:24:56 +04:00
Evgenii Stratonikov
da4ddcf337 [#353] netmap: Get rid of ioutil
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-10-19 12:05:49 +04:00
Evgenii Stratonikov
231ec66054 [#353] go.mod: Update external dependencies
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-10-19 12:05:49 +04:00
Evgenii Stratonikov
a02a3870a5 [#353] go.mod: Update neofs-api-go to v2.14.0
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-10-19 12:05:49 +04:00
Evgenii Stratonikov
9d4b36a8e6 [#353] go.mod: Update neo-contract to v0.16.0
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-10-19 12:05:49 +04:00
Evgenii Stratonikov
d3a09cbe7f [#353] go.mod: Update neo-go to v0.99.4
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-10-19 12:05:49 +04:00
anastasia prasolova
f21d63bded Add CODEOWNERS file
Signed-off-by: anastasia prasolova <anastasia@nspcc.ru>
2022-10-16 21:06:24 +03:00
21eef1ae7f [#351] Add StringifyPublicKey method for NodeInfo
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2022-10-13 10:27:18 +03:00
Leonard Lyubich
d2f3929b51 [#349] client: Use context.Background in Dial by default
Passing `nil` context to `rpc.Balance` leads to panic. To prevent panic,
we need to provide some default context in `Dial`.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
2022-10-12 17:50:55 +04:00
Leonard Lyubich
8c682641bf [#343] pool: Provide dial context to clients
After recent changes `client.Client` accepts dial context. There is a
need to forward the context passed into `Pool.Dial` to the underlying
`Client` instances.

Define type aliases of different client constructors: context-based and
non-context. Use context-based constructor in `Pool`. Pass `ctx`
parameter of `Pool.Dial` method to the client builder.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
2022-10-07 13:24:02 +03:00
Leonard Lyubich
452a50e9d5 [#343] client: Accept context parameter in Dial
In previous implementation of `Client.Dial` there was no ability to
specify parent context (e.g. global application context).

Add `PrmDial.SetContext` method which accepts optional base dial
context. Use the context to open client connection or fall back to using
`context.Background()`.

Upgraded version of `github.com/nspcc-dev/neofs-api-go/v2` module
also fixes the problem when dial timeout didn't work properly.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
2022-10-07 13:24:02 +03:00
Leonard Lyubich
1325b4f272 [#336] session: Support group object sessions
Rename `LimitByObject` method of the `session.Object` type to
`LimitByObjects` and make it to accept variadic parameter.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
2022-10-05 13:39:51 +04:00
Evgenii Stratonikov
c6576c8112 [#344] netmap: Support ExternalAddr well-known attribute
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-09-26 13:28:39 +03:00
Leonard Lyubich
8e3173eacd [#270] client/netmap: Cover NetMapSnapshot with unit tests
There is a need to test each `Client` operation. In previous
implementation `Client` was based on real socket connection. This didn't
allow to test the `Client` without OS resources. In order to write
convenient and useful unit tests we need to slightly refactor the code.

Introduce `neoFSAPIServer` interface as a provider of `Client` type's
abstraction from the exact NeoFS API server. Add `netMapSnapshot` method
for initial implementation. Define core interface provider used in real
code. Set `coreServer` as an underlying `neoFSAPIServer` in
`Client.Dial`. Cover `Client.NetMapSnapshot` method with unit tests
using the opportunity to override the server.

From now client library can be tested not only with real physical
listeners but with imitations.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-09-22 10:51:07 +04:00
Leonard Lyubich
89124d442d [#312] client/netmap: Simplify NetMapSnapshot implementaiton
Make flat set of instructions similar to `ObjectDelete` operation.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-09-22 10:51:07 +04:00
Leonard Lyubich
664392afc2 [#312] netmap: Support NetmapService.NetmapSnapshot RPC
Extend functionality of `NetMap` type. Add `NetMapSnapshot` operation to
`client` package.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-09-22 10:51:07 +04:00
Leonard Lyubich
f2f97f656d [#315] apistatus: Write default message in NodeUnderMaintenance.Error
Use `node is under maintenance` message in `NodeUnderMaintenance.Error`
is `message` field is unset in the status message.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-09-20 11:23:22 +04:00
Leonard Lyubich
be9a1aca90 [#315] apistatus: Support NodeUnderMaintenance in FromStatusV2
Support decoding `NodeUnderMaintenance` status errors in `FromStatusV2`
function.

From now `NodeUnderMaintenance` instance can be decoded from
`status.Status` message of NeoFS API V2 protocol.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-09-20 11:23:22 +04:00
Evgenii Stratonikov
3d6b5d807b [#315] netmap: Add maintenance node state
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-09-19 16:49:27 +04:00
Evgenii Stratonikov
3dad44232e [#315] netmap: Add maintenance mode network setting
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-09-19 16:49:27 +04:00
Evgenii Stratonikov
4662d39886 [#315] client/status: Add NodeUnderMaintenance status
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-09-19 16:49:27 +04:00
Evgenii Stratonikov
f75a5feba3 [#334] client: Use parameter key in object.Get
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-09-14 11:34:56 +04:00
Evgenii Stratonikov
1929b634a1 [#332] netmap/parser: Disallow trailing garbage
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-09-12 10:45:37 +03:00
Evgenii Stratonikov
402d72e629 [#332] go.mod: Update antlr dependency
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-09-12 10:45:37 +03:00
Evgenii Stratonikov
43a57d42dd [#331] client: Allow to set timeout for streaming operations
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-09-07 11:51:28 +04:00
Pavel Karpy
71891029da [#327] nns: Support Null returned value
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-09-06 17:27:07 +04:00
Pavel Karpy
d808f72c38 [#327] Add go 1.19 tests
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-09-06 17:27:07 +04:00
Pavel Karpy
511886b8d2 [#327] Update go to 1.17
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-09-06 17:27:07 +04:00
Pavel Karpy
01c238ddc0 [#327] nns: Simplify code using neo-go v0.99.2
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-09-06 17:27:07 +04:00
Pavel Karpy
2e5c66934c [#327] go.mod: Update neo-go to v0.99.2
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-09-06 17:27:07 +04:00
Denis Kirillov
74234623b2 [#215] object: Add well-known FilePath attribute
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-09-06 17:21:37 +04:00
Evgenii Stratonikov
ee92df3203 [#326] client: Sign message in object.GetRange'
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-29 14:45:50 +03:00
Evgenii Stratonikov
cf7bee3087 [#326] client: Provide pointers requests for signing
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-29 14:45:50 +03:00
Evgenii Stratonikov
456167e777 [#325] go.mod: Update neofs-api-go
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-29 11:53:34 +03:00
Evgenii Stratonikov
0e4d07fb06 [#323] client: Remove xheaders length check in a common helper
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-26 17:34:27 +03:00
Evgenii Stratonikov
d6d6a41f5d [#323] client: Refactor object.Put
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-26 17:34:27 +03:00
Evgenii Stratonikov
724d30db1a [#323] client: Refactor object.Get
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-26 17:34:27 +03:00
Evgenii Stratonikov
1f593d0fb2 [#323] client: Refactor object.GetRange
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-26 17:34:27 +03:00
Evgenii Stratonikov
f543ba68d3 [#323] client: Refactor object.Delete
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-26 17:34:27 +03:00
Evgenii Stratonikov
dd5826d071 [#323] client: Refactor object.Head
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-26 17:34:27 +03:00
Evgenii Stratonikov
02bc2bc236 [#323] client: Unify parameter processing for Get* methods
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-26 17:34:27 +03:00
Evgenii Stratonikov
992b26a3ff [#323] client: Form meta header during parameter initialization
Unify with other client methods.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-26 17:34:27 +03:00
Evgenii Stratonikov
6a43accf96 [#323] client: Refactor object.Search
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-26 17:34:27 +03:00
Evgenii Stratonikov
5d7650c3e7 [#323] client: Use checksum parameter directly in object.Hash
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-26 17:34:27 +03:00
Evgenii Stratonikov
a926e5a1de [#323] client: Replace writeToMetaHeader method with function
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-26 17:34:27 +03:00
Evgenii Stratonikov
8c5333ea55 [#323] client: Refactor object.Hash
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-26 17:34:27 +03:00
Evgenii Stratonikov
0236b03fa7 [#323] client: Use constant for error message
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-26 17:34:27 +03:00
Evgenii Stratonikov
2f843de3ed [#323] client: Remove initCallContextWithoutKey
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-26 17:34:27 +03:00
Evgenii Stratonikov
84888854ab [#322] *: Go fmt -s
go1.19 rewrites comments to proper render them in docs.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-24 18:58:59 +03:00
Evgenii Stratonikov
7537fa0dec [#321] client: Set object ID field in Put
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-24 17:04:10 +03:00
Pavel Karpy
7578b54fac [#319] crypto: Fix signing in go v1.19
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-22 13:19:10 +03:00
Leonard Lyubich
737e690482 [#299] pool: Do not use pointers to the resulting values
In previous implementation `pool` package provided access to resulting
values as pointers to them. This caused clients to handle nil
cases even when the field presence in the response is required.

Avoid returning pointers to values in result getters. This also reduces
reference counter load and allows values from `client.Client` to be
forwarded directly without additional assignment.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-08-16 15:31:57 +04:00
Leonard Lyubich
f8148c954b [#317] client: Fix processing of the Object PUT failed response
In previous implementation `ObjectWrite.Close` called `close` method
which in turn called `result` callback. Thus failed statuses could lead
to false-positive result processing.

Replace calling `close` method with direct `closer` method's call in
`ObjectWrite.Close`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-08-16 15:31:37 +04:00
Leonard Lyubich
4e31b4f231 [#299] client: Do not use pointers to required response fields
In previous implementation `client` package provided access to nested
response fields as pointers to them. This caused clients to handle nil
cases even when the field presence in the response is required.

Avoid returning pointers to required fields in response getters. This
also reduces reference counter load and allows fields to be decoded
directly without additional assignment.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-08-08 12:55:25 +03:00
Denis Kirillov
30bf79f075 [#302] client: Adopt SetCopiesNumber
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-08-03 15:22:13 +03:00
Pavel Karpy
7a99cc916c [#307] status: Support EACL_NOT_FOUND status code
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-01 19:57:07 +03:00
Pavel Karpy
a0f7c903d3 [#307] go.mod: Update neofs-api-go to v2.13.1
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-01 19:57:07 +03:00
Evgenii Stratonikov
7de66159d4 [#304] go.mod: Update neo-go to v0.99.1
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-29 11:43:51 +03:00
Denis Kirillov
90255e9efa [#291] pool: Use more robust way to init methods
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-26 21:47:06 +03:00
Denis Kirillov
828cfdc5bf [#291] pool: Improve docs
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-26 21:47:06 +03:00
Denis Kirillov
90f1cc7a1a [#301] ns: Adopt neo-go int parameter
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-25 13:14:11 +03:00
Evgenii Stratonikov
3a95686aab [#300] go.mod: Update neofs-contract
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-22 17:02:58 +03:00
Denis Kirillov
f4ac75423c [#283] pool: Store methods status in slice
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:57:37 +03:00
Denis Kirillov
b4f4ee4f79 [#283] pool: Compute avg time for every method
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:57:37 +03:00
Denis Kirillov
54145916a9 [#283] pool: Change batch of atomics to mutex
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:57:37 +03:00
Denis Kirillov
0d54757545 [#283] pool: Add res nil check
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:57:37 +03:00
Denis Kirillov
58fe1768cc [#283] pool: Fix incErrorRate
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:57:37 +03:00
Denis Kirillov
0b8c53ebc9 [#283] pool: Add number of requests to statistic
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:57:37 +03:00
Denis Kirillov
1b30d228da [#283] pool: Expose statistic
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:57:37 +03:00
Denis Kirillov
423804de84 [#283] pool: Add latency calculating
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:57:37 +03:00
Denis Kirillov
99e185690e [#283] pool: Add error threshold
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:57:37 +03:00
Denis Kirillov
9d3a1835d1 [#283] pool: Drop gomock
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:57:37 +03:00
Denis Kirillov
e6cb5f2ee1 [#283] pool: Add counter for errors
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-07-22 15:57:37 +03:00
Leonard Lyubich
7d10b432d1 [#284] *: Return error from all ReadFromV2 methods
Return `error` from all `ReadFromV2` methods in order to support
backward compatibility if message will be extended with some formatted
field.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-07-07 14:20:18 +03:00
Evgenii Stratonikov
0d862d8568 [#294] go.mod: Update tzhash to v1.6.1
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-06 18:10:41 +03:00
Pavel Karpy
48ba86f4dd [#293] version: Update API version constants
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-06 12:05:58 +03:00
Evgenii Stratonikov
f91b1facd5 [#292] go.mod: Update tzhash to v1.6.0
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-05 20:44:56 +03:00
Pavel Karpy
c6fee6d01a [#240] pool: Add SyncContainerWithNetwork
It is a helper function that uses a `Pool` to fetch network information and
apply it to the container (change the container if it does not fit the
network). Currently, only applies the homomorphic hashing parameter.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-05 15:37:49 +03:00
Pavel Karpy
a6ecf6b881 [#240] client: Add SyncContainerWithNetwork func
It requests network configuration and syncs it (changes if required) with
passed container.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-05 15:37:49 +03:00
Pavel Karpy
9996b3be01 [#240] container: Support disabling homomorphic hashing
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-05 15:37:49 +03:00
Pavel Karpy
0cd790cfe0 [#240] netmap: Support HomomorphicHashingDisabled network config
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-05 15:37:49 +03:00
Pavel Karpy
df6538c68c [#282] apistatus: Support OUT_OF_RANGE error
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-05 11:53:29 +03:00
Pavel Karpy
27fe9c19a7 [#273] status: Support signature status
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-05 11:50:44 +03:00
Leonard Lyubich
6994eb0e55 [#222] client: Fix docs of resolving the failure statuses
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-07-05 11:43:38 +03:00
Leonard Lyubich
cec2373b50 [#222] reputation: Clarify docs of GlobalTrust formation
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-07-05 11:42:58 +03:00
Leonard Lyubich
ff09b1cfe1 Upgrade NeoFS API Go to v2.13.0
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-07-05 11:42:58 +03:00
Leonard Lyubich
041e1ef2b6 [#286] reputation: Refactor and document package functionality
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-07-05 11:42:58 +03:00
Leonard Lyubich
30d27c3050 [#289] oid: Add JSON encoding of Address type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-07-04 14:58:00 +03:00
Leonard Lyubich
2ad89085a3 [#225] container: Clarify docs of Container type initialization
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-07-04 11:21:16 +03:00
Leonard Lyubich
70845147f6 [#225] container: Refactor and document package functionality
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-07-04 11:21:16 +03:00
Angira Kekteeva
86a447bc80 [#288] pool: Fix lost err in fmt.Errorf
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-07-01 15:09:26 +03:00
Leonard Lyubich
ab4d1e34a8 [#280] policy: Use policyVisitor as ErrorListener for QueryLexer
Share `policyVisitor` instance as an `ErrorListener` provider for both
`QueryLexer` and `Query`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-28 14:47:22 +03:00
Leonard Lyubich
dea3d9c419 [#280] policy: Remove error listeners from query lexer
In previous implementation `PlacementPolicy.DecodeString` produced
undesired console logs.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-28 14:47:22 +03:00
Leonard Lyubich
d2cd9ebfbd [#278] client: Increase depth of error wrapping in IsErr* tests
Wrap test error twice just to be sure error unwrapping goes deep.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-28 09:06:13 +03:00
Leonard Lyubich
1e1139f305 [#278] pool: Avoid using strings.Contains for error assertion
`client` package now provides functions for error checking.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-28 09:06:13 +03:00
Leonard Lyubich
dcaf454c1d [#278] client: Add session error checkers
Add `IsErrSessionExpired` and `IsErrSessionNotFound` functions which
assert corresponding session errors.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-28 09:06:13 +03:00
Leonard Lyubich
09ed6077f9 [#278] client: Support wrapped errors in error checkers
Client errors are quite often wrapped in context. When checking a
specific error, it is required not to lose the ability to determine it,
regardless of the attached context. In previous implementation `IsErr*`
functions didn't support wrapped errors.

Make `IsErr*` functions to preliminarily unwrap `error` argument before
type assertion. Use `errors.Unwrap` for this instead of `errors.As`
since the latter has the overhead of filling in an error.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-28 09:06:13 +03:00
Leonard Lyubich
40942affe9 [#278] client: Add unit test for error checkers
`IsErr*` functions assert `error` argument corresponds to particular
typed error.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-28 09:06:13 +03:00
Leonard Lyubich
aa5ee1dcde [#225] container/acl: Make Basic type numeric
Also make well-known values constant not variables.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-23 11:05:32 +03:00
Leonard Lyubich
3b15a01327 [#225] container/acl: Fix minor places
Make test value generator to be random as documented. Add `default`
keyword to switch-case to avoid linter comments.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-23 11:05:32 +03:00
Leonard Lyubich
c4ebe8d854 [#225] container: Replace basic ACL code to a separate package
Create `acl` package inside `container` path. Replace basic ACL
functionality into it.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-23 11:05:32 +03:00
Leonard Lyubich
e82a2d86ef [#225] container: Refactor and document basic ACL
Replace basic ACL functionality from `acl` package to the `container`
one. Create `BasicACL` type and provide convenient interface to work
with it.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-23 11:05:32 +03:00
Evgenii Stratonikov
af7e20073b [#277] netmap: Allow more uint64 config values
NEO VM uses little-endian format for integers,
however the resulting byte slice can contain less than 8 bytes.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-22 15:51:36 +03:00
Evgenii Stratonikov
596f43a540 [#274] crypto: Add WalletConnect API support
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-22 11:20:54 +03:00
Leonard Lyubich
721df386c5 [#276] container: Remove session token and signature from Container/eACL
Session token and signature isn't presented in `Container` and
`EACLTable` messages of NeoFS API V2 protocol. These entities are needed
for access control and doesn't carry payload of these messages.

Remove `SetSessionToken` / `SessionToken` methods of
`container.Container` and `eacl.Table` types. Provide methods to specify
these components in corresponding `Client` operations.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-21 20:03:07 +03:00
Leonard Lyubich
e986f47807 [#227] netmap: Add more encoding methods for PlacementPolicy
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-16 11:23:21 +03:00
Leonard Lyubich
e999fb00c3 [#227] netmap: Remove unnecessary unit tests
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-16 11:23:21 +03:00
Leonard Lyubich
d51a324147 [#227] netmap: Clarify some fields of PlacementPolicy
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-15 20:50:32 +03:00
Leonard Lyubich
d648b86776 [#227] netmap: Fix minor blots in documentation/code
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-15 20:50:32 +03:00
Leonard Lyubich
ca523f1ff1 [#227] netmap: Refactor and document package functionality
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-15 20:50:32 +03:00
Leonard Lyubich
5bfdb64251 [#227] netmap: Do not export bucket weight calculator
`GetBucketWeight` is not usable outside placement context.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-15 20:50:32 +03:00
Leonard Lyubich
86bdc670d5 [#227] netmap: Do not export default weightFunc initializer
`GetDefaultWeightFunc` is not usable outside placement context.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-15 20:50:32 +03:00
Leonard Lyubich
6796b4a29a [#227] netmap: Remove unused aggregators and normalizers
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-15 20:50:32 +03:00
Leonard Lyubich
723ba5ee45 [#227] netmap: Do not use intermediate types for placement
Support preprocessing within the `NodeInfo` type. Provide methods for
placement directly from the `NodeInfo` type. Returns slice of slices of
`NodeInfo` from placement methods of `Netmap`. Remove no longer needed
`Node` and `Nodes` types.

```
name            old time/op    new time/op    delta
ManySelects-12    19.7µs ±14%    15.8µs ±15%  -19.70%  (p=0.000 n=20+20)

name            old alloc/op   new alloc/op   delta
ManySelects-12    8.65kB ± 0%    6.22kB ± 0%  -28.03%  (p=0.000 n=20+20)

name            old allocs/op  new allocs/op  delta
ManySelects-12      82.0 ± 0%      81.0 ± 0%   -1.22%  (p=0.000 n=20+20)
```

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-15 20:50:32 +03:00
Leonard Lyubich
2b21146185 [#227] netmap/test: Assert source placement builder is pure for nodes
`GetContainerNodes` / `GetPlacementVectors` must not mutate source slice
of nodes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-15 20:50:32 +03:00
Leonard Lyubich
9c47fca7c2 [#227] netmap: Pre-calculate resulting slice capacity in flattenNodes
```
name           old time/op  new time/op  delta
ManySelects-12  17.5µs ±27%   15.5µs ±24%  -11.57%  (p=0.024 n=20+20)

name           old alloc/op  new alloc/op  delta
ManySelects-12  9.03kB ± 0%   8.65kB ± 0%  -4.25%  (p=0.000 n=20+20)

name           old allocs/op  new allocs/op  delta
ManySelects-12  86.0 ± 0%     82.0 ± 0%     -4.65%  (p=0.000 n=20+20)
```

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-15 20:50:32 +03:00
Leonard Lyubich
a8fd2ef954 [#227] netmap: Benchmark JSON tests
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-15 20:50:32 +03:00
Leonard Lyubich
ea21cdf731 [#269] client: Fix response processing of Object.Put rpc
After 85e3c7b087 `processResponse` method
sets `err` field or sets status in `statusRes` field of `contextCall`.
In previous implementation `ObjectWriter.Close` method returned
`ctxCall.err` on `false` return of `processResponse` method. This could
cause NPE-panic if status failure resolving was disabled.

Make `ObjectWriter.Close` to return internal `err` field only if it is
set, otherwise return status response.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-15 15:30:39 +03:00
Leonard Lyubich
eb3b990812 [#210] subnet: Refactor and document package functionality
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-15 11:52:07 +03:00
Pavel Karpy
6709b00c89 [#245] storagegroup: Add Object <-> SG converters
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-15 10:33:47 +03:00
Evgenii Stratonikov
517d7a1e4a [#266] eacl: Allow lossless conversion of reserved filters
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-09 09:45:32 +03:00
Pavel Karpy
6ac9deabb8 [#264] storagegroup: Fix out of range
It panicked when the previous members slice had capacity more than a new one
because of incorrect slicing that led to `out of range`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-09 09:38:02 +03:00
Evgenii Stratonikov
5518b63432 [#243] eacl: Return success flag in CalculateAction
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-09 09:35:32 +03:00
Leonard Lyubich
031eac2f48 [#265] session: Implement method to verify session data signature
There is a need to verify session data signatures calculated using
private session key. `Container` token encapsulates public session key,
so we need to provide method for signature check.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-07 21:53:39 +03:00
Leonard Lyubich
67ff996dc3 Revert "[#261] session: Make failed Sign pure on failed signature calculation"
This reverts commit 458c882ff4.

If `issuerSet` is set after signature calculation then issuer ID isn't
written to signed data, in other words not signed.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-06 10:02:17 +03:00
Leonard Lyubich
bf8312a547 [#263] session: Add Sign / VerifySignature test
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-06 10:02:17 +03:00
Leonard Lyubich
0ef49cf851 Upgrade NeoFS API Go to v2.12.2
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-03 17:07:45 +03:00
Leonard Lyubich
458c882ff4 [#261] session: Make failed Sign pure on failed signature calculation
Make `Sign` method implementation to not modify `issuerSet` state
variable after signature calculation's failures.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-03 14:46:18 +03:00
Leonard Lyubich
0bb40b3245 [#261] session: Share common code between Container and Object
`Container` and `Object` types are transmitted in single `session.Token`
message. They differ only by session context.

Share common parts of the message in `commonData` struct. Embed struct
into `Container` and `Object`. Make `ReadFromV2` methods to check
protocol compliance. Make `Unmarshal`/`UmarshalJSON` to check field
format in case of presence only.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-03 14:46:18 +03:00
Pavel Karpy
d3b998d672 [#250] eacl: Do not require CID in eACL table
Container ID of extended ACL table can be omitted in bearer token according
to API.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-03 14:19:57 +03:00
Leonard Lyubich
c65be6d469 [#259] bearer: Align with the protocol
Sync lifetime names with `session` package. Add `AssertContainer` and
`AssertUser` methods which cover specific cases describe in protocol.
Increase test coverage.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-01 18:32:49 +03:00
Leonard Lyubich
82d762f536 [#258] Fix string encoding of identifiers
Use `EncodeToString` method to get protocol string according to type
docs.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-31 12:14:04 +03:00
Leonard Lyubich
6cb513c976 [#257] Upgrade NeoFS API Go module
New version contains fix for `object.GetRangeResponse` message type.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-31 09:47:55 +03:00
Denis Kirillov
3953c2166e [#255] Create session token using arbitrary key
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-05-30 20:09:43 +03:00
Leonard Lyubich
85e3c7b087 [#254] client: Fix incorrect response processing case
In previous implementation `processResponse` returned `true` if
`resolveAPIFailures` is set and status failure is received. This led to
post-processing of the results, which no longer pays attention to the
status. For example, `ObjectHead` returned `unexpected header type`
error due to empty body.

Make `contextCall.processResponse` to return success flag regardless of
`resolveAPIFailures` setting. Make `contextCall.processCall` to return
`err` field presence flag on `processResponse` false return.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-30 17:23:56 +03:00
Leonard Lyubich
3bbf7ee15d [#252] object/id: Fix typos in docs
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-27 12:41:35 +03:00
Leonard Lyubich
85affc3c93 object: Remove debug printline
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-27 12:41:35 +03:00
Leonard Lyubich
f0a5eb6dbc [#251] object/address: Refactor and document package functionality
Merge `address` package into `oid` one. Bring `session.Object`
implementation into conformity with the NeoFS API protocol.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-27 12:41:35 +03:00
Leonard Lyubich
bef4618cd6 [#252] session: Add Object.Issuer method
Add method to get `Object` session's issuer similar to `Container`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-26 09:54:57 +03:00
Evgenii Stratonikov
1f7fe6864d [#229] netmap: make context private
It isn't used anywhere outside of the package.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-25 11:02:51 +03:00
Leonard Lyubich
60ef026923 [#248] session: Add Issuer method
There is a need to duplicate session token owner, e.g. in container
created within the session. For such cases we need to have the ability
to receive session issuer.

Add `Container.Issuer` method. Transform `IssuedBy` to helper function.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-05-25 10:57:53 +03:00
Evgenii Stratonikov
c976332e20 [#242] object: Require only 1 field to be set for SplitInfo
Also write more tests.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-05-23 19:09:53 +03:00
Alex Vanin
12ea1e8d74 [#238] session: Add container.IssuedBy method
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-04 22:24:02 +03:00
Alex Vanin
539ac9915e [#237] client: Sign container ID value at container removal
Container contract expects signature of container ID value,
which is SHA256 of container body. Not the signature of stable
marshaled container.ID structure.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-05-04 15:22:48 +03:00
Leonard Lyubich
526b45e207 [#197] session: Link issues for further TODO resolving
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-27 11:09:29 +03:00
Leonard Lyubich
5fe6d96bf1 [#197] session: Fix Container.AppliedTo method docs
Method checks propagation, not limit.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-27 11:09:29 +03:00
Leonard Lyubich
4cbbbdd3e2 [#197] pool: Change function signature
Make `createSessionTokenForDuration` to accept a pointer to
`session.Object` and write the response on session creation request
through the pointer. Rename function to `initSessionForDuration`. As a
consequence, problem in `openDefaultSession` with uncached sessions is
fixed.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-27 11:09:29 +03:00
Leonard Lyubich
f0134ef26e [#197] pool: Remove no longer needed test func
`TestCopySessionTokenWithoutSignatureAndContext` was based on already
removed function.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-27 11:09:29 +03:00
Leonard Lyubich
e468f409d7 [#197] pool: Remove unused function parameter
`owner.ID` parameter of `createSessionTokenForDuration` function is no
longer used since session owner is set automatically during the sign
operation. As a consequence, remove `Pool.sessionOwner` field and its
getter.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-27 11:09:29 +03:00
Leonard Lyubich
4dc3a7669e [#197] client: Link FIXME with an issue
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-27 11:09:29 +03:00
Leonard Lyubich
48434d9999 [#197] client: Remove no longer used type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-27 11:09:29 +03:00
Leonard Lyubich
552c7875bf [#197] session: Refactor and document the package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-27 11:09:29 +03:00
Denis Kirillov
497053c785 [#228] ns: use neofs-contract for nns constants
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-24 14:11:16 +03:00
Alex Vanin
6e81e13e1b [#218] client: Decode status at io.EOF in the client-side stream
Object service of NeoFS API contains one client-side stream method:
object.Put. In client-side streams, server can return an error after
processing stream message. In this case write method returns `io.EOF`
and actual error reason is encoded in response status, which is
obtained after `Close()`. Client library should process such case.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-04-21 15:57:37 +03:00
Leonard Lyubich
1ed426b8a6 [#199] owner: Rename to user, refactor and doc
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-19 18:44:02 +03:00
Leonard Lyubich
d20999113a [#190] crypto/ecdsa: Actualize package docs
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-19 12:55:11 +03:00
Leonard Lyubich
6554c681c8 [#190] crypto: Doc RegisterScheme isn't thread-unsafe
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-19 12:55:11 +03:00
Leonard Lyubich
9f20d74d76 [#190] crypto/ecdsa: Use separate types for RFC-6979 signature algo
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-19 12:55:11 +03:00
Leonard Lyubich
7fe75d2cd9 [#190] crypto: Replace public key encoding methods to PublicKey
This make `neofscrypto.Signer` interface more similar to the one from
standard `crypto` package.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-19 12:55:11 +03:00
Leonard Lyubich
bcbffd516a [#190] crypto: Provide fmt.Stringer of the Scheme type
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-19 12:55:11 +03:00
Leonard Lyubich
2f9cc50fec [#190] crypto: Start Scheme enum from zero
Enumerate signature schemes of `Scheme` type from 0 in order to sync
with NeoFS API protocol by values.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-19 12:55:11 +03:00
Leonard Lyubich
ea043f4ca3 [#190] Refactor cryptographic functionality
Remove `signature` and `util/signature` packages. Re-implement their
functionality in new `crypto` package. Generalize the approach of
digital signature computation and verification by adding `Signer` and
`PublicKey` primitives similar to standard `crypto` package. Support
already exising in protocol signature schemes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-19 12:55:11 +03:00
Pavel Karpy
2deaaeef05 [#170] object: Fix comments about checksums
Delete "must not be nil" where there are no pointers.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-13 18:33:33 +03:00
Pavel Karpy
9b63c07c59 [#170] storagegroup: Refactor and document package functionality
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-13 18:33:33 +03:00
Pavel Karpy
1186f2f703 [#170] oid, cid: Add marshal format checks
Also add checking presence of the `oid`, `cid` fields via `set` flag.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-13 17:32:25 +03:00
Pavel Karpy
f7172adf18 [#170] oid, cid: Refactor and document package functionality
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-13 17:32:25 +03:00
Alex Vanin
24d6c2221f [#170] bearer: Return presence flag in getters of optional fields
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-04-13 11:24:15 +03:00
Alex Vanin
27cd721422 [#170] bearer: Add docs, refactor
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-04-13 11:24:15 +03:00
Pavel Karpy
96892d7bc4 [#170] checksum: Drop Empty method
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-13 10:53:57 +03:00
Pavel Karpy
168b3ee7a4 [#170] checksum: Add examples
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-13 10:53:57 +03:00
Pavel Karpy
caa055236b [#170] checksum: Do not use pointers
Do not return pointers from getters. Do not pass pointers to the methods
that does not modify the checksum. Add `Empty` method.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-13 10:53:57 +03:00
Pavel Karpy
fd13e61266 [#170] checksum: Drop Empty method
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-13 10:53:57 +03:00
Leonard Lyubich
9c502a9cae [#179] resolver: Support wss connection scheme in NNS.Dial
WebSocket Secure scheme should also lead to using `neoclient.WSClient`
by `NNS`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-13 10:46:27 +03:00
Leonard Lyubich
d51d18d5f3 [#179] ns: Select Neo connection scheme dynamically in NNS.Dial
WebSocket protocol is optionally supported by Neo servers.

Make `NNS.Dial` to select between HTTP and WebSocket connections
according to URL address scheme.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-13 10:46:27 +03:00
Leonard Lyubich
55283d3c91 [#179] Rename resolver package to ns
`resolver` isn't suitable name for the package with NeoFS name system
functionality.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-13 10:46:27 +03:00
Leonard Lyubich
75a5b6588d [#179] resolver: Use Client with WSClient in NNS
WebSocket client (`WSClient` type) from Neo Go library shows better
performance than HTTP one (`Client` type).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-13 10:46:27 +03:00
Leonard Lyubich
146fc4f07a [#179] resolver: Refactor and document the package
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-04-13 10:46:27 +03:00
Alex Vanin
c961aea144 [#170] version: Add specification string encoder
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-04-13 10:28:12 +03:00
Alex Vanin
ade8822a2f [#170] version: Add docs, refactor
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-04-13 10:28:12 +03:00
Denis Kirillov
3e75660802 [#205] pool: fix using default key for requests
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-12 18:12:50 +03:00
Denis Kirillov
53e064e556 [#206] pool: fix waitForContainerRemoved
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-12 18:12:39 +03:00
Denis Kirillov
385f1b10f9 [#203] pool: fix PrmContainerDelete
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-12 14:26:08 +03:00
Denis Kirillov
1e8aa6a99f [#85] pool: update tests
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-12 10:30:28 +03:00
Denis Kirillov
97c8274dc0 [#85] pool: use non-pointer client inside wrapper
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-12 10:30:28 +03:00
Denis Kirillov
9e8e12f6ae [#85] pool: make client interface methods private
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-12 10:30:28 +03:00
Denis Kirillov
aa6a4950d5 [#85] pool: fix skipped tests
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-12 10:30:28 +03:00
Denis Kirillov
e56eef495d [#85] pool: refactor client interface
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-12 10:30:28 +03:00
Denis Kirillov
e50e6d2828 [#137] pool: simplify session creation
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-07 13:33:16 +03:00
Denis Kirillov
df0573d521 [#137] pool: drop sessionTokenThreshold
Description

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-07 13:33:16 +03:00
Denis Kirillov
c4adb03f8e [#137] pool: drop retry for object operations
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-07 13:33:16 +03:00
Denis Kirillov
fcfae4a249 [#137] Use callback response to update epoch
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-07 13:33:16 +03:00
Denis Kirillov
9814748958 [#137] pool: fix tests
Description

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-07 13:33:16 +03:00
Denis Kirillov
7df00fb0eb [#137] Add exp to ResSessionCreate
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-07 13:33:16 +03:00
Denis Kirillov
11a25bb413 [#137] Expire session tokens locally
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-07 13:33:16 +03:00
Denis Kirillov
b8d2158acd [#180] eacl: make equal methods functions
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-07 09:50:02 +03:00
Denis Kirillov
a709cf5444 [#180] pool: make some operations synchronous
Operations: PutContainer, DeleteContainer, SetEACL

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-07 09:50:02 +03:00
Denis Kirillov
0dbea5452a [#180] pool: add WaitForContainerRemoved
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-07 09:50:02 +03:00
Denis Kirillov
47345a33da [#180] pool: add WaitForEACLPresence
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-07 09:50:02 +03:00
Denis Kirillov
d568458fab [#180] eacl: add EqualTo for table
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-07 09:50:02 +03:00
Alex Vanin
2104945f9e [#193] pool: Return copy of session token in cache
To avoid side effects after token re-sign.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-04-06 12:06:24 +03:00
Alex Vanin
f38a24e8b5 [#193] pool: Add test to check side effects in token cache
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-04-06 12:06:24 +03:00
Denis Kirillov
e0281c3b34 [#191] pool: use bearer token
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-04-06 10:27:58 +04:00
Evgenii Stratonikov
f18b4a2a75 [#177] *: Remove logger package
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-29 09:54:46 +03:00
Pavel Karpy
58d4f4a55f [#170] acl: Document package functionality
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-03-25 19:15:22 +03:00
Denis Kirillov
030bbce2cf [#165] pool: drop prmCommon from container params
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-21 12:41:08 +03:00
Denis Kirillov
59b49dd7e6 [#165] pool: update WaitForContainerPresence
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-21 12:41:08 +03:00
Denis Kirillov
191d85e607 [#165] pool: change exported fields to setters
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-21 12:41:08 +03:00
Denis Kirillov
9be9697856 [#165] pool: make client interface unexported
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-21 12:41:08 +03:00
Denis Kirillov
52548fe176 [#165] pool: add docs
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-21 12:41:08 +03:00
Denis Kirillov
7811d8eefc [#165] pool: distinguish init step and run
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-21 12:41:08 +03:00
Denis Kirillov
d03523a3bc [#165] pool: replace options with parameters
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-21 12:41:08 +03:00
Denis Kirillov
ec5c223f29 [#165] pool: make private inner structs
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-21 12:41:08 +03:00
Denis Kirillov
f5cabe26cb [#165] pool: drop builder
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-03-21 12:41:08 +03:00
Leonard Lyubich
aeb4ac638a [#170] accounting: Remove message sub-string from V2-methods
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-21 12:37:53 +03:00
Leonard Lyubich
7d31de57ec [#170] audit: Refactor and document package functionality
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-21 12:37:53 +03:00
Leonard Lyubich
48150852f3 [#170] accounting: Refactor and document package functionality
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-21 12:37:53 +03:00
Alex Vanin
a55ffa4796 [#176] status: Do not lose built-in error text message
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2022-03-16 17:16:20 +03:00
380 changed files with 38483 additions and 21603 deletions

View file

@ -0,0 +1,21 @@
name: DCO
on: [pull_request]
jobs:
dco:
name: DCO
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Setup Go
uses: actions/setup-go@v3
with:
go-version: '1.23'
- name: Run commit format checker
uses: https://git.frostfs.info/TrueCloudLab/dco-go@v3
with:
from: 'origin/${{ github.event.pull_request.base.ref }}'

View file

@ -0,0 +1,39 @@
name: Tests and linters
on: [pull_request]
jobs:
lint:
name: Lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: '1.23'
cache: true
- name: Install linters
run: make lint-install
- name: Run linters
run: make lint
tests:
name: Tests
runs-on: ubuntu-latest
strategy:
matrix:
go_versions: [ '1.22', '1.23' ]
fail-fast: false
steps:
- uses: actions/checkout@v3
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: '${{ matrix.go_versions }}'
- name: Run tests
run: make test

1
.github/CODEOWNERS vendored Normal file
View file

@ -0,0 +1 @@
* @TrueCloudLab/storage-core @TrueCloudLab/storage-services

45
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View file

@ -0,0 +1,45 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: community, triage, bug
assignees: ''
---
<!--- Provide a general summary of the issue in the Title above -->
## Expected Behavior
<!--- If you're describing a bug, tell us what should happen -->
<!--- If you're suggesting a change/improvement, tell us how it should work -->
## Current Behavior
<!--- If describing a bug, tell us what happens instead of the expected behavior -->
<!--- If suggesting a change/improvement, explain the difference from current behavior -->
## Possible Solution
<!--- Not obligatory -->
<!--- If no reason/fix/additions for the bug can be suggested, -->
<!--- uncomment the following phrase: -->
<!--- No fix can be suggested by a QA engineer. Further solutions shall be up to developers. -->
## Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to -->
<!--- reproduce this bug. -->
1.
## Context
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context helps us come up with a solution that is most useful in the real world -->
## Regression
<!-- Is this issue a regression? (Yes / No) -->
<!-- If Yes, optionally please include version or commit id or PR# that caused this regression, if you have these details. -->
## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used:
* Server setup and configuration:
* Operating System and version (`uname -a`):

1
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View file

@ -0,0 +1 @@
blank_issues_enabled: false

View file

@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: community, triage
assignees: ''
---
## Is your feature request related to a problem? Please describe.
<!--- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
## Describe the solution you'd like
<!--- A clear and concise description of what you want to happen. -->
## Describe alternatives you've considered
<!--- A clear and concise description of any alternative solutions or features you've considered. -->
## Additional context
<!--- Add any other context or screenshots about the feature request here. -->

View file

@ -1,21 +0,0 @@
name: DCO check
on:
pull_request:
branches:
- master
jobs:
commits_check_job:
runs-on: ubuntu-latest
name: Commits Check
steps:
- name: Get PR Commits
id: 'get-pr-commits'
uses: tim-actions/get-pr-commits@master
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: DCO Check
uses: tim-actions/dco@master
with:
commits: ${{ steps.get-pr-commits.outputs.commits }}

View file

@ -1,52 +0,0 @@
name: neofs-sdk-go tests
on:
pull_request:
branches:
- master
types: [opened, synchronize]
paths-ignore:
- '**/*.md'
workflow_dispatch:
jobs:
tests:
name: Tests
runs-on: ubuntu-20.04
strategy:
matrix:
go_versions: [ '1.16.x', '1.17.x' ]
fail-fast: false
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: '${{ matrix.go_versions }}'
- name: Restore Go modules from cache
uses: actions/cache@v2
with:
path: /home/runner/go/pkg/mod
key: deps-${{ hashFiles('go.sum') }}
- name: Update Go modules
run: make dep
- name: Run tests
run: make test
lint:
runs-on: ubuntu-20.04
steps:
- name: Check out code
uses: actions/checkout@v2
- name: golangci-lint
uses: golangci/golangci-lint-action@v2
with:
version: latest
only-new-issues: true

10
.gitignore vendored
View file

@ -21,3 +21,13 @@ vendor/
# coverage
coverage.txt
coverage.html
# antlr tool jar
antlr-*.jar
# tempfiles
.cache
# binary
bin/
release/

View file

@ -4,15 +4,16 @@
# options for analysis running
run:
# timeout for analysis, e.g. 30s, 5m, default is 1m
timeout: 5m
timeout: 10m
# include test files or not, default is true
tests: true
tests: false
# output configuration options
output:
# colored-line-number|line-number|json|tab|checkstyle|code-climate, default is "colored-line-number"
format: tab
formats:
- format: tab
# all available settings of specific linters
linters-settings:
@ -24,6 +25,13 @@ linters-settings:
govet:
# report about shadowed variables
check-shadowing: false
staticcheck:
checks: ["all"]
funlen:
lines: 80 # default 60
statements: 60 # default 40
gocognit:
min-complexity: 40 # default 30
linters:
enable:
@ -32,28 +40,29 @@ linters:
- revive
# some default golangci-lint linters
- deadcode
- errcheck
- gosimple
- godot
- ineffassign
- staticcheck
- structcheck
- typecheck
- unused
- varcheck
# extra linters
- bidichk
- durationcheck
- exhaustive
- godot
- exportloopref
- gofmt
- whitespace
- goimports
- misspell
- predeclared
- reassign
- whitespace
- containedctx
- funlen
- gocognit
- contextcheck
- protogetter
disable-all: true
fast: false
issues:
include:
- EXC0002 # should have a comment
- EXC0003 # test/Test ... consider calling this
- EXC0004 # govet
- EXC0005 # C-style breaks

36
.pre-commit-config.yaml Normal file
View file

@ -0,0 +1,36 @@
ci:
autofix_prs: false
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: check-added-large-files
- id: check-case-conflict
- id: check-executables-have-shebangs
- id: check-shebang-scripts-are-executable
- id: check-merge-conflict
- id: check-json
- id: check-xml
- id: check-yaml
- id: trailing-whitespace
args: [--markdown-linebreak-ext=md]
- id: end-of-file-fixer
exclude: "(.key|.interp|.tokens)$"
- repo: local
hooks:
- id: go-unit-tests
name: go unit tests
entry: make test GOFLAGS=''
pass_filenames: false
types: [go]
language: system
- repo: local
hooks:
- id: make-lint
name: Run Make Lint
entry: make lint
language: system
pass_filenames: false

4
Dockerfile Normal file
View file

@ -0,0 +1,4 @@
FROM golang:1.22
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install make openjdk-17-jre -y
WORKDIR /work

55
Makefile Normal file → Executable file
View file

@ -1,8 +1,16 @@
#!/usr/bin/make -f
ANTLR_VERSION="4.13.0"
TMP_DIR := .cache
LINT_VERSION ?= 1.60.1
TRUECLOUDLAB_LINT_VERSION ?= 0.0.6
OUTPUT_LINT_DIR ?= $(shell pwd)/bin
LINT_DIR = $(OUTPUT_LINT_DIR)/golangci-lint-$(LINT_VERSION)-v$(TRUECLOUDLAB_LINT_VERSION)
# Run tests
test: GOFLAGS ?= "-cover -count=1"
test:
@go test ./... -cover
@GOFLAGS=$(GOFLAGS) go test ./...
# Pull go dependencies
dep:
@ -13,9 +21,23 @@ dep:
@CGO_ENABLED=0 \
go mod tidy -v && echo OK
# Install linters
lint-install:
@mkdir -p $(TMP_DIR)
@rm -rf $(TMP_DIR)/linters
@git -c advice.detachedHead=false clone --branch v$(TRUECLOUDLAB_LINT_VERSION) https://git.frostfs.info/TrueCloudLab/linters.git $(TMP_DIR)/linters
@@make -C $(TMP_DIR)/linters lib CGO_ENABLED=1 OUT_DIR=$(OUTPUT_LINT_DIR)
@rm -rf $(TMP_DIR)/linters
@rmdir $(TMP_DIR) 2>/dev/null || true
@CGO_ENABLED=1 GOBIN=$(LINT_DIR) go install github.com/golangci/golangci-lint/cmd/golangci-lint@v$(LINT_VERSION)
# Run linters
lint:
@golangci-lint --timeout=5m run
@if [ ! -d "$(LINT_DIR)" ]; then \
echo "Run make lint-install"; \
exit 1; \
fi
$(LINT_DIR)/golangci-lint run
# Run tests with race detection and produce coverage output
cover:
@ -29,6 +51,23 @@ format:
@echo "⇒ Processing goimports check"
@goimports -w ./
policy:
@wget -q https://www.antlr.org/download/antlr-${ANTLR_VERSION}-complete.jar -O antlr4-tool.jar
@java -Xmx500M -cp "`pwd`/antlr4-tool.jar" "org.antlr.v4.Tool" -o `pwd`/netmap/parser/ -Dlanguage=Go -no-listener -visitor `pwd`/netmap/parser/Query.g4 `pwd`/netmap/parser/QueryLexer.g4
# Run `make %` in truecloudlab/frostfs-sdk-go container(Golang+Java)
docker/%:
@docker build -t truecloudlab/frostfs-sdk-go --platform linux/amd64 . > /dev/null
@docker run --rm -t \
-v `pwd`:/work \
-u "$$(id -u):$$(id -g)" \
--env HOME=/work \
truecloudlab/frostfs-sdk-go make $*
# Synchronize tree service
sync-tree:
@./syncTree.sh
# Show this help prompt
help:
@echo ' Usage:'
@ -38,3 +77,15 @@ help:
@echo ' Targets:'
@echo ''
@awk '/^#/{ comment = substr($$0,3) } comment && /^[a-zA-Z][a-zA-Z0-9_-]+ ?:/{ print " ", $$1, comment }' $(MAKEFILE_LIST) | column -t -s ':' | grep -v 'IGNORE' | sort -u
# Activate pre-commit hooks
pre-commit:
pre-commit install --hook-type pre-commit
# Deactivate pre-commit hooks
unpre-commit:
pre-commit uninstall --hook-type pre-commit
# Run pre-commit hooks
pre-commit-run:
@pre-commit run --all-files --hook-stage manual

View file

@ -1,6 +1,6 @@
# neofs-sdk-go
Go implementation of NeoFS SDK. It contains high-level version-independent wrappers
for structures from [neofs-api-go](https://github.com/nspcc-dev/neofs-api-go) as well as
# frostfs-sdk-go
Go implementation of FrostFS SDK. It contains high-level version-independent wrappers
for structures from [frostfs-api-go](https://git.frostfs.info/TrueCloudLab/frostfs-api-go) as well as
helper functions for simplifying node/dApp implementations.
## Repository structure
@ -10,63 +10,73 @@ Contains fixed-point `Decimal` type for performing balance calculations.
### eacl
Contains Extended ACL types for fine-grained access control.
There is also a reference implementation of checking algorithm which is used in NeoFS node.
There is also a reference implementation of checking algorithm which is used in FrostFS node.
### checksum
Contains `Checksum` type encapsulating checksum as well as it's kind.
Currently Sha256 and [Tillich-Zemor hashsum](https://github.com/nspcc-dev/tzhash) are in use.
Currently Sha256 and [Tillich-Zemor hashsum](https://git.frostfs.info/TrueCloudLab/tzhash) are in use.
### owner
`owner.ID` type represents single account interacting with NeoFS. In v2 version of protocol
`owner.ID` type represents single account interacting with FrostFS. In v2 version of protocol
it is just raw bytes behing [base58-encoded address](https://docs.neo.org/docs/en-us/basic/concept/wallets.html#address)
in Neo blockchain. Note that for historical reasons it contains
version prefix and checksum in addition to script-hash.
### token
Contains Bearer token type with several NeoFS-specific methods.
Contains Bearer token type with several FrostFS-specific methods.
### resolver
In NeoFS there are 2 types of name resolution: DNS and NNS. NNS stands for Neo Name Service
is just a [contract](https://github.com/nspcc-dev/neofs-contract/) deployed on a Neo blockchain.
### ns
In FrostFS there are 2 types of name resolution: DNS and NNS. NNS stands for Neo Name Service
is just a [contract](https://git.frostfs.info/TrueCloudLab/frostfs-contract) deployed on a Neo blockchain.
Basically, NNS is just a DNS-on-chain which can be used for resolving container nice-names as well
as any other name in dApps. See our [CoreDNS plugin](https://github.com/nspcc-dev/coredns/tree/master/plugin/nns)
for the example of how NNS can be integrated in DNS.
### session
To help lightweight clients interact with NeoFS without sacrificing trust, NeoFS has a concept
To help lightweight clients interact with FrostFS without sacrificing trust, FrostFS has a concept
of session token. It is signed by client and allows any node with which a session is established
to perform certain actions on behalf of the user.
### client
Contains client for working with NeoFS.
Contains client for working with FrostFS.
```go
c, _ := client.New(
client.WithAddress("localhost:40005"), // endpoint address
client.WithDefaultPrivateKey(key), // private key for request signing
client.WithNeoFSErrorHandling(), // enable erroneous status parsing
client.WithTLSConfig(&tls.Config{})) // custom TLS configuration
var prmInit client.PrmInit
prmInit.SetDefaultPrivateKey(key) // private key for request signing
ctx, cancel := context.WithTimeout(context.Background(), 5 * time.Second)
defer cancel()
var c client.Client
c.Init(prmInit)
res, err := c.BalanceGet(ctx, owner)
var prmDial client.PrmDial
prmDial.SetServerURI("grpcs://localhost:40005") // endpoint address
err := c.Dial(prmDial)
if err != nil {
return
}
fmt.Printf("Balance for %s: %s\n", owner, res.Amount())
ctx, cancel := context.WithTimeout(context.Background(), 5 * time.Second)
defer cancel()
var prm client.PrmBalanceGet
prm.SetAccount(acc)
res, err := c.BalanceGet(ctx, prm)
if err != nil {
return
}
fmt.Printf("Balance for %s: %v\n", acc, res.Amount())
```
#### Response status
In NeoFS every operation can fail on multiple levels, so a single `error` doesn't suffice,
In FrostFS every operation can fail on multiple levels, so a single `error` doesn't suffice,
e.g. consider a case when object was put on 4 out of 5 replicas. Thus, all request execution
details are contained in `Status` returned from every RPC call. dApp can inspect them
if needed and perform any desired action. In the case above we may want to report
these details to the user as well as retry an operation, possibly with different parameters.
Status wire-format is extendable and each node can report any set of details it wants.
The set of reserved status codes can be found in
[NeoFS API](https://github.com/nspcc-dev/neofs-api/blob/master/status/types.proto). There is also
a `client.WithNeoFSErrorHandling()` to seamlessly convert erroneous statuses into Go error type.
[FrostFS API](https://git.frostfs.info/TrueCloudLab/frostfs-api/src/branch/master/status/types.proto).
### policy
Contains helpers allowing conversion of placing policy from/to JSON representation
@ -90,12 +100,12 @@ outdated in some details.
```go
import (
"github.com/nspcc-dev/neofs-sdk-go/netmap"
"github.com/nspcc-dev/neofs-sdk-go/object"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
)
func placementNodes(addr *object.Address, p *netmap.PlacementPolicy, neofsNodes []netmap.NodeInfo) {
// Convert list of nodes in NeoFS API format to the intermediate representation.
func placementNodes(addr *object.Address, p *netmap.PlacementPolicy, frostfsNodes []netmap.NodeInfo) {
// Convert list of nodes in FrostFS API format to the intermediate representation.
nodes := netmap.NodesFromInfo(nodes)
// Create new netmap (errors are skipped for the sake of clarity).
@ -110,13 +120,13 @@ func placementNodes(addr *object.Address, p *netmap.PlacementPolicy, neofsNodes
```
### pool
Simple pool for managing connections to NeoFS nodes.
Simple pool for managing connections to FrostFS nodes.
### acl, checksum, version, signature
Contain simple API wrappers.
### logger
Wrapper over `zap.Logger` which is used across NeoFS codebase.
Wrapper over `zap.Logger` which is used across FrostFS codebase.
### util
Utilities for working with signature-related code.

View file

@ -1,69 +1,64 @@
package accounting
import "github.com/nspcc-dev/neofs-api-go/v2/accounting"
import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/accounting"
// Decimal represents decimal number for accounting operations.
//
// Decimal is mutually compatible with git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/accounting.Decimal
// message. See ReadFromV2 / WriteToV2 methods.
//
// Instances can be created using built-in var declaration.
//
// Note that direct typecast is not safe and may result in loss of compatibility:
//
// _ = Decimal(accounting.Decimal{}) // not recommended
type Decimal accounting.Decimal
// NewDecimal creates, initializes and returns empty Decimal instance.
// ReadFromV2 reads Decimal from the accounting.Decimal message. Checks if the
// message conforms to FrostFS API V2 protocol.
//
// Defaults:
// - value: 0
// - precision: 0
func NewDecimal() *Decimal {
return NewDecimalFromV2(new(accounting.Decimal))
// See also WriteToV2.
func (d *Decimal) ReadFromV2(m accounting.Decimal) error {
*d = Decimal(m)
return nil
}
// NewDecimalFromV2 converts v2 Decimal to Decimal.
// WriteToV2 writes Decimal to the accounting.Decimal message.
// The message must not be nil.
//
// Nil Decimal converts to nil.
func NewDecimalFromV2(d *accounting.Decimal) *Decimal {
return (*Decimal)(d)
}
// ToV2 returns the v2 Decimal message.
//
// Nil Decimal converts to nil.
func (d *Decimal) ToV2() *accounting.Decimal {
return (*accounting.Decimal)(d)
// See also ReadFromV2.
func (d Decimal) WriteToV2(m *accounting.Decimal) {
*m = (accounting.Decimal)(d)
}
// Value returns value of the decimal number.
func (d *Decimal) Value() int64 {
return (*accounting.Decimal)(d).GetValue()
//
// Zero Decimal has zero value.
//
// See also SetValue.
func (d Decimal) Value() int64 {
return (*accounting.Decimal)(&d).GetValue()
}
// SetValue sets value of the decimal number.
//
// See also Value.
func (d *Decimal) SetValue(v int64) {
(*accounting.Decimal)(d).SetValue(v)
}
// Precision returns precision of the decimal number.
func (d *Decimal) Precision() uint32 {
return (*accounting.Decimal)(d).GetPrecision()
//
// Zero Decimal has zero precision.
//
// See also SetPrecision.
func (d Decimal) Precision() uint32 {
return (*accounting.Decimal)(&d).GetPrecision()
}
// SetPrecision sets precision of the decimal number.
//
// See also Precision.
func (d *Decimal) SetPrecision(p uint32) {
(*accounting.Decimal)(d).SetPrecision(p)
}
// Marshal marshals Decimal into a protobuf binary form.
func (d *Decimal) Marshal() ([]byte, error) {
return (*accounting.Decimal)(d).StableMarshal(nil)
}
// Unmarshal unmarshalls protobuf binary representation of Decimal.
func (d *Decimal) Unmarshal(data []byte) error {
return (*accounting.Decimal)(d).Unmarshal(data)
}
// MarshalJSON encodes Decimal to protobuf JSON format.
func (d *Decimal) MarshalJSON() ([]byte, error) {
return (*accounting.Decimal)(d).MarshalJSON()
}
// UnmarshalJSON decodes Decimal from protobuf JSON format.
func (d *Decimal) UnmarshalJSON(data []byte) error {
return (*accounting.Decimal)(d).UnmarshalJSON(data)
}

View file

@ -3,15 +3,19 @@ package accounting_test
import (
"testing"
"github.com/nspcc-dev/neofs-sdk-go/accounting"
accountingtest "github.com/nspcc-dev/neofs-sdk-go/accounting/test"
v2accounting "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/accounting"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/accounting"
"github.com/stretchr/testify/require"
)
func TestDecimal(t *testing.T) {
func TestDecimalData(t *testing.T) {
const v, p = 4, 2
d := accounting.NewDecimal()
var d accounting.Decimal
require.Zero(t, d.Value())
require.Zero(t, d.Precision())
d.SetValue(v)
d.SetPrecision(p)
@ -19,26 +23,24 @@ func TestDecimal(t *testing.T) {
require.EqualValues(t, p, d.Precision())
}
func TestDecimalEncoding(t *testing.T) {
d := accountingtest.Decimal()
func TestDecimalMessageV2(t *testing.T) {
var (
d accounting.Decimal
m v2accounting.Decimal
)
t.Run("binary", func(t *testing.T) {
data, err := d.Marshal()
require.NoError(t, err)
m.SetValue(7)
m.SetPrecision(8)
d2 := accounting.NewDecimal()
require.NoError(t, d2.Unmarshal(data))
require.NoError(t, d.ReadFromV2(m))
require.Equal(t, d, d2)
})
require.EqualValues(t, m.GetValue(), d.Value())
require.EqualValues(t, m.GetPrecision(), d.Precision())
t.Run("json", func(t *testing.T) {
data, err := d.MarshalJSON()
require.NoError(t, err)
var m2 v2accounting.Decimal
d2 := accounting.NewDecimal()
require.NoError(t, d2.UnmarshalJSON(data))
d.WriteToV2(&m2)
require.Equal(t, d, d2)
})
require.EqualValues(t, d.Value(), m2.GetValue())
require.EqualValues(t, d.Precision(), m2.GetPrecision())
}

35
accounting/doc.go Normal file
View file

@ -0,0 +1,35 @@
/*
Package accounting provides primitives to perform accounting operations in FrostFS.
Decimal type provides functionality to process user balances. For example, when
working with Fixed8 balance precision:
var dec accounting.Decimal
dec.SetValue(val)
dec.SetPrecision(8)
Instances can be also used to process FrostFS API V2 protocol messages
(see neo.fs.v2.accounting package in https://git.frostfs.info/TrueCloudLab/frostfs-api).
On client side:
import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/accounting"
var msg accounting.Decimal
dec.WriteToV2(&msg)
// send msg
On server side:
// recv msg
var dec accounting.Decimal
dec.ReadFromV2(msg)
// process dec
Using package types in an application is recommended to potentially work with
different protocol versions with which these types are compatible.
*/
package accounting

View file

@ -3,14 +3,14 @@ package accountingtest
import (
"math/rand"
"github.com/nspcc-dev/neofs-sdk-go/accounting"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/accounting"
)
// Decimal returns random accounting.Decimal.
func Decimal() *accounting.Decimal {
d := accounting.NewDecimal()
var d accounting.Decimal
d.SetValue(rand.Int63())
d.SetPrecision(rand.Uint32())
return d
return &d
}

13
accounting/test/doc.go Normal file
View file

@ -0,0 +1,13 @@
/*
Package accountingtest provides functions for convenient testing of accounting package API.
Note that importing the package into source files is highly discouraged.
Random instance generation functions can be useful when testing expects any value, e.g.:
import accountingtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/accounting/test"
dec := accountingtest.Decimal()
// test the value
*/
package accountingtest

View file

@ -1,105 +0,0 @@
package acl
import (
"fmt"
"strconv"
"strings"
)
// BasicACL is Access Control List that defines who can interact with containers and what exactly they can do.
type BasicACL uint32
func (a BasicACL) String() string {
return fmt.Sprintf("0x%08x", uint32(a))
}
const (
// PublicBasicRule is a basic ACL value for final public-read-write container for which extended ACL CANNOT be set.
PublicBasicRule BasicACL = 0x1FBFBFFF
// PrivateBasicRule is a basic ACL value for final private container for which extended ACL CANNOT be set.
PrivateBasicRule BasicACL = 0x1C8C8CCC
// ReadOnlyBasicRule is a basic ACL value for final public-read container for which extended ACL CANNOT be set.
ReadOnlyBasicRule BasicACL = 0x1FBF8CFF
// PublicAppendRule is a basic ACL value for final public-append container for which extended ACL CANNOT be set.
PublicAppendRule BasicACL = 0x1FBF9FFF
// EACLPublicBasicRule is a basic ACL value for non-final public-read-write container for which extended ACL CAN be set.
EACLPublicBasicRule BasicACL = 0x0FBFBFFF
// EACLPrivateBasicRule is a basic ACL value for non-final private container for which extended ACL CAN be set.
EACLPrivateBasicRule BasicACL = 0x0C8C8CCC
// EACLReadOnlyBasicRule is a basic ACL value for non-final public-read container for which extended ACL CAN be set.
EACLReadOnlyBasicRule BasicACL = 0x0FBF8CFF
// EACLPublicAppendRule is a basic ACL value for non-final public-append container for which extended ACL CAN be set.
EACLPublicAppendRule BasicACL = 0x0FBF9FFF
)
const (
// PublicBasicName is a well-known name for 0x1FBFBFFF basic ACL.
// It represents fully-public container without eACL.
PublicBasicName = "public-read-write"
// PrivateBasicName is a well-known name for 0x1C8C8CCC basic ACL.
// It represents fully-private container without eACL.
PrivateBasicName = "private"
// ReadOnlyBasicName is a well-known name for 0x1FBF8CFF basic ACL.
// It represents public read-only container without eACL.
ReadOnlyBasicName = "public-read"
// PublicAppendName is a well-known name for 0x1FBF9FFF basic ACL.
// It represents fully-public container without eACL except DELETE operation is only allowed on the owner.
PublicAppendName = "public-append"
// EACLPublicBasicName is a well-known name for 0x0FBFBFFF basic ACL.
// It represents fully-public container that allows eACL.
EACLPublicBasicName = "eacl-public-read-write"
// EACLPrivateBasicName is a well-known name for 0x0C8C8CCC basic ACL.
// It represents fully-private container that allows eACL.
EACLPrivateBasicName = "eacl-private"
// EACLReadOnlyBasicName is a well-known name for 0x0FBF8CFF basic ACL.
// It represents public read-only container that allows eACL.
EACLReadOnlyBasicName = "eacl-public-read"
// EACLPublicAppendName is a well-known name for 0x0FBF9FFF basic ACL.
// It represents fully-public container that allows eACL except DELETE operation is only allowed on the owner.
EACLPublicAppendName = "eacl-public-append"
)
// ParseBasicACL parse string ACL (well-known names or hex representation).
func ParseBasicACL(basicACL string) (BasicACL, error) {
switch basicACL {
case PublicBasicName:
return PublicBasicRule, nil
case PrivateBasicName:
return PrivateBasicRule, nil
case ReadOnlyBasicName:
return ReadOnlyBasicRule, nil
case PublicAppendName:
return PublicAppendRule, nil
case EACLPublicBasicName:
return EACLPublicBasicRule, nil
case EACLPrivateBasicName:
return EACLPrivateBasicRule, nil
case EACLReadOnlyBasicName:
return EACLReadOnlyBasicRule, nil
case EACLPublicAppendName:
return EACLPublicAppendRule, nil
default:
basicACL = strings.TrimPrefix(strings.ToLower(basicACL), "0x")
value, err := strconv.ParseUint(basicACL, 16, 32)
if err != nil {
return 0, fmt.Errorf("can't parse basic ACL: %s", basicACL)
}
return BasicACL(value), nil
}
}

View file

@ -1,82 +0,0 @@
package acl
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestParser(t *testing.T) {
for _, tc := range []struct {
acl string
expected BasicACL
err bool
}{
{
acl: PublicBasicName,
expected: PublicBasicRule,
},
{
acl: PrivateBasicName,
expected: PrivateBasicRule,
},
{
acl: ReadOnlyBasicName,
expected: ReadOnlyBasicRule,
},
{
acl: PublicAppendName,
expected: PublicAppendRule,
},
{
acl: EACLPublicBasicName,
expected: EACLPublicBasicRule,
},
{
acl: EACLPrivateBasicName,
expected: EACLPrivateBasicRule,
},
{
acl: EACLReadOnlyBasicName,
expected: EACLReadOnlyBasicRule,
},
{
acl: EACLPublicAppendName,
expected: EACLPublicAppendRule,
},
{
acl: "0x1C8C8CCC",
expected: 0x1C8C8CCC,
},
{
acl: "1C8C8CCC",
expected: 0x1C8C8CCC,
},
{
acl: "123456789",
err: true,
},
{
acl: "0x1C8C8CCG",
err: true,
},
} {
actual, err := ParseBasicACL(tc.acl)
if tc.err {
require.Error(t, err)
continue
}
require.NoError(t, err)
require.Equal(t, tc.expected, actual)
}
}
func TestString(t *testing.T) {
acl := BasicACL(0x1fbfbfff)
require.Equal(t, "0x1fbfbfff", acl.String())
acl2, err := ParseBasicACL(PrivateBasicName)
require.NoError(t, err)
require.Equal(t, "0x1c8c8ccc", acl2.String())
}

52
ape/chain.go Normal file
View file

@ -0,0 +1,52 @@
package ape
import (
"errors"
"fmt"
apeV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/ape"
)
var (
ErrInvalidChainRepresentation = errors.New("invalid chain representation")
)
// ChainID is Chain's identifier.
type ChainID []byte
// Chain is an SDK representation for v2's Chain.
//
// Note that Chain (as well as v2's Chain) and all related entities
// are NOT operated by Access-Policy-Engine (APE). The client is responsible
// to convert these types to policy-engine entities.
type Chain struct {
// Raw is the encoded chain kind.
// It assumes that Raw's bytes are the result of encoding provided by
// policy-engine package.
Raw []byte
}
// ToV2 converts Chain to v2.
func (c *Chain) ToV2() *apeV2.Chain {
v2ct := new(apeV2.Chain)
if c.Raw != nil {
v2Raw := new(apeV2.ChainRaw)
v2Raw.SetRaw(c.Raw)
v2ct.SetKind(v2Raw)
}
return v2ct
}
// ReadFromV2 fills Chain from v2.
func (c *Chain) ReadFromV2(v2ct *apeV2.Chain) error {
switch v := v2ct.GetKind().(type) {
default:
return fmt.Errorf("unsupported chain kind: %T", v)
case *apeV2.ChainRaw:
raw := v.GetRaw()
c.Raw = raw
}
return nil
}

53
ape/chain_target.go Normal file
View file

@ -0,0 +1,53 @@
package ape
import (
apeV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/ape"
)
// TargetType is an SDK representation for v2's TargetType.
type TargetType apeV2.TargetType
const (
TargetTypeUndefined TargetType = iota
TargetTypeNamespace
TargetTypeContainer
TargetTypeUser
TargetTypeGroup
)
// ToV2 converts TargetType to v2.
func (targetType TargetType) ToV2() apeV2.TargetType {
return apeV2.TargetType(targetType)
}
// FromV2 reads TargetType to v2.
func (targetType *TargetType) FromV2(v2targetType apeV2.TargetType) {
*targetType = TargetType(v2targetType)
}
// ChainTarget is an SDK representation for v2's ChainTarget.
//
// Note that ChainTarget (as well as v2's ChainTarget) and all related entities
// are NOT operated by Access-Policy-Engine (APE). The client is responsible
// to convert these types to policy-engine entities.
type ChainTarget struct {
TargetType TargetType
Name string
}
// ToV2 converts ChainTarget to v2.
func (ct *ChainTarget) ToV2() *apeV2.ChainTarget {
v2ct := new(apeV2.ChainTarget)
v2ct.SetTargetType(ct.TargetType.ToV2())
v2ct.SetName(ct.Name)
return v2ct
}
// FromV2 reads ChainTarget frpm v2.
func (ct *ChainTarget) FromV2(v2ct *apeV2.ChainTarget) {
ct.TargetType.FromV2(v2ct.GetTargetType())
ct.Name = v2ct.GetName()
}

65
ape/chain_target_test.go Normal file
View file

@ -0,0 +1,65 @@
package ape_test
import (
"testing"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
"github.com/stretchr/testify/require"
apeV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/ape"
)
var (
m = map[ape.TargetType]apeV2.TargetType{
ape.TargetTypeUndefined: apeV2.TargetTypeUndefined,
ape.TargetTypeNamespace: apeV2.TargetTypeNamespace,
ape.TargetTypeContainer: apeV2.TargetTypeContainer,
ape.TargetTypeUser: apeV2.TargetTypeUser,
ape.TargetTypeGroup: apeV2.TargetTypeGroup,
}
)
func TestTargetType(t *testing.T) {
for typesdk, typev2 := range m {
t.Run("from sdk to v2 "+typev2.String(), func(t *testing.T) {
v2 := typesdk.ToV2()
require.Equal(t, v2, typev2)
})
t.Run("from v2 to sdk "+typev2.String(), func(t *testing.T) {
var typ ape.TargetType
typ.FromV2(typev2)
require.Equal(t, typesdk, typ)
})
}
}
func TestChainTarget(t *testing.T) {
var (
typ = ape.TargetTypeNamespace
name = "namespaceXXYYZZ"
)
t.Run("from sdk to v2", func(t *testing.T) {
ct := ape.ChainTarget{
TargetType: typ,
Name: name,
}
v2 := ct.ToV2()
require.Equal(t, m[typ], v2.GetTargetType())
require.Equal(t, name, v2.GetName())
})
t.Run("from v2 to sdk", func(t *testing.T) {
v2 := &apeV2.ChainTarget{}
v2.SetTargetType(m[typ])
v2.SetName(name)
var ct ape.ChainTarget
ct.FromV2(v2)
require.Equal(t, typ, ct.TargetType)
require.Equal(t, name, ct.Name)
})
}

43
ape/chain_test.go Normal file
View file

@ -0,0 +1,43 @@
package ape_test
import (
"testing"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
"github.com/stretchr/testify/require"
apeV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/ape"
)
const (
encoded = `{"ID":"","Rules":[{"Status":"Allow","Actions":{"Inverted":false,"Names":["GetObject"]},"Resources":{"Inverted":false,"Names":["native:object/*"]},"Any":false,"Condition":[{"Op":"StringEquals","Object":"Resource","Key":"Department","Value":"HR"}]}],"MatchType":"DenyPriority"}`
)
func TestChainData(t *testing.T) {
t.Run("raw chain", func(t *testing.T) {
var c ape.Chain
b := []byte(encoded)
c.Raw = b
v2, ok := c.ToV2().GetKind().(*apeV2.ChainRaw)
require.True(t, ok)
require.Equal(t, b, v2.Raw)
})
}
func TestChainMessageV2(t *testing.T) {
b := []byte(encoded)
v2Raw := &apeV2.ChainRaw{}
v2Raw.SetRaw(b)
v2 := &apeV2.Chain{}
v2.SetKind(v2Raw)
var c ape.Chain
c.ReadFromV2(v2)
require.NotNil(t, c.Raw)
require.Equal(t, b, c.Raw)
}

View file

@ -1,291 +0,0 @@
package audit
import (
"github.com/nspcc-dev/neofs-api-go/v2/audit"
"github.com/nspcc-dev/neofs-api-go/v2/refs"
cid "github.com/nspcc-dev/neofs-sdk-go/container/id"
oid "github.com/nspcc-dev/neofs-sdk-go/object/id"
"github.com/nspcc-dev/neofs-sdk-go/version"
)
// Result represents v2-compatible data audit result.
type Result audit.DataAuditResult
// NewFromV2 wraps v2 DataAuditResult message to Result.
//
// Nil audit.DataAuditResult converts to nil.
func NewResultFromV2(aV2 *audit.DataAuditResult) *Result {
return (*Result)(aV2)
}
// New creates and initializes blank Result.
//
// Defaults:
// - version: version.Current();
// - complete: false;
// - cid: nil;
// - pubKey: nil;
// - passSG, failSG: nil;
// - failNodes, passNodes: nil;
// - hit, miss, fail: 0;
// - requests, retries: 0;
// - auditEpoch: 0.
func NewResult() *Result {
r := NewResultFromV2(new(audit.DataAuditResult))
r.SetVersion(version.Current())
return r
}
// ToV2 converts Result to v2 DataAuditResult message.
//
// Nil Result converts to nil.
func (r *Result) ToV2() *audit.DataAuditResult {
return (*audit.DataAuditResult)(r)
}
// Marshal marshals Result into a protobuf binary form.
func (r *Result) Marshal() ([]byte, error) {
return (*audit.DataAuditResult)(r).StableMarshal(nil)
}
// Unmarshal unmarshals protobuf binary representation of Result.
func (r *Result) Unmarshal(data []byte) error {
return (*audit.DataAuditResult)(r).Unmarshal(data)
}
// MarshalJSON encodes Result to protobuf JSON format.
func (r *Result) MarshalJSON() ([]byte, error) {
return (*audit.DataAuditResult)(r).MarshalJSON()
}
// UnmarshalJSON decodes Result from protobuf JSON format.
func (r *Result) UnmarshalJSON(data []byte) error {
return (*audit.DataAuditResult)(r).UnmarshalJSON(data)
}
// Version returns Data Audit structure version.
func (r *Result) Version() *version.Version {
return version.NewFromV2(
(*audit.DataAuditResult)(r).GetVersion())
}
// SetVersion sets Data Audit structure version.
func (r *Result) SetVersion(v *version.Version) {
(*audit.DataAuditResult)(r).SetVersion(v.ToV2())
}
// AuditEpoch returns epoch number when the Data Audit was conducted.
func (r *Result) AuditEpoch() uint64 {
return (*audit.DataAuditResult)(r).GetAuditEpoch()
}
// SetAuditEpoch sets epoch number when the Data Audit was conducted.
func (r *Result) SetAuditEpoch(epoch uint64) {
(*audit.DataAuditResult)(r).SetAuditEpoch(epoch)
}
// ContainerID returns container under audit.
func (r *Result) ContainerID() *cid.ID {
return cid.NewFromV2(
(*audit.DataAuditResult)(r).GetContainerID())
}
// SetContainerID sets container under audit.
func (r *Result) SetContainerID(id *cid.ID) {
(*audit.DataAuditResult)(r).SetContainerID(id.ToV2())
}
// PublicKey returns public key of the auditing InnerRing node in a binary format.
func (r *Result) PublicKey() []byte {
return (*audit.DataAuditResult)(r).GetPublicKey()
}
// SetPublicKey sets public key of the auditing InnerRing node in a binary format.
func (r *Result) SetPublicKey(key []byte) {
(*audit.DataAuditResult)(r).SetPublicKey(key)
}
// Complete returns completion state of audit result.
func (r *Result) Complete() bool {
return (*audit.DataAuditResult)(r).GetComplete()
}
// SetComplete sets completion state of audit result.
func (r *Result) SetComplete(v bool) {
(*audit.DataAuditResult)(r).SetComplete(v)
}
// Requests returns number of requests made by PoR audit check to get
// all headers of the objects inside storage groups.
func (r *Result) Requests() uint32 {
return (*audit.DataAuditResult)(r).GetRequests()
}
// SetRequests sets number of requests made by PoR audit check to get
// all headers of the objects inside storage groups.
func (r *Result) SetRequests(v uint32) {
(*audit.DataAuditResult)(r).SetRequests(v)
}
// Retries returns number of retries made by PoR audit check to get
// all headers of the objects inside storage groups.
func (r *Result) Retries() uint32 {
return (*audit.DataAuditResult)(r).GetRetries()
}
// SetRetries sets number of retries made by PoR audit check to get
// all headers of the objects inside storage groups.
func (r *Result) SetRetries(v uint32) {
(*audit.DataAuditResult)(r).SetRetries(v)
}
// PassSG returns list of Storage Groups that passed audit PoR stage.
func (r *Result) PassSG() []oid.ID {
mV2 := (*audit.DataAuditResult)(r).
GetPassSG()
if mV2 == nil {
return nil
}
m := make([]oid.ID, len(mV2))
for i := range mV2 {
m[i] = *oid.NewIDFromV2(&mV2[i])
}
return m
}
// SetPassSG sets list of Storage Groups that passed audit PoR stage.
func (r *Result) SetPassSG(list []oid.ID) {
mV2 := (*audit.DataAuditResult)(r).
GetPassSG()
if list == nil {
mV2 = nil
} else {
ln := len(list)
if cap(mV2) >= ln {
mV2 = mV2[:0]
} else {
mV2 = make([]refs.ObjectID, ln)
}
for i := 0; i < ln; i++ {
mV2[i] = *list[i].ToV2()
}
}
(*audit.DataAuditResult)(r).SetPassSG(mV2)
}
// FailSG returns list of Storage Groups that failed audit PoR stage.
func (r *Result) FailSG() []oid.ID {
mV2 := (*audit.DataAuditResult)(r).
GetFailSG()
if mV2 == nil {
return nil
}
m := make([]oid.ID, len(mV2))
for i := range mV2 {
m[i] = *oid.NewIDFromV2(&mV2[i])
}
return m
}
// SetFailSG sets list of Storage Groups that failed audit PoR stage.
func (r *Result) SetFailSG(list []oid.ID) {
mV2 := (*audit.DataAuditResult)(r).
GetFailSG()
if list == nil {
mV2 = nil
} else {
ln := len(list)
if cap(mV2) >= ln {
mV2 = mV2[:0]
} else {
mV2 = make([]refs.ObjectID, ln)
}
for i := 0; i < ln; i++ {
mV2[i] = *list[i].ToV2()
}
}
(*audit.DataAuditResult)(r).SetFailSG(mV2)
}
// Hit returns number of sampled objects under audit placed
// in an optimal way according to the containers placement policy
// when checking PoP.
func (r *Result) Hit() uint32 {
return (*audit.DataAuditResult)(r).GetHit()
}
// SetHit sets number of sampled objects under audit placed
// in an optimal way according to the containers placement policy
// when checking PoP.
func (r *Result) SetHit(hit uint32) {
(*audit.DataAuditResult)(r).SetHit(hit)
}
// Miss returns number of sampled objects under audit placed
// in suboptimal way according to the containers placement policy,
// but still at a satisfactory level when checking PoP.
func (r *Result) Miss() uint32 {
return (*audit.DataAuditResult)(r).GetMiss()
}
// SetMiss sets number of sampled objects under audit placed
// in suboptimal way according to the containers placement policy,
// but still at a satisfactory level when checking PoP.
func (r *Result) SetMiss(miss uint32) {
(*audit.DataAuditResult)(r).SetMiss(miss)
}
// Fail returns number of sampled objects under audit stored
// in a way not confirming placement policy or not found at all
// when checking PoP.
func (r *Result) Fail() uint32 {
return (*audit.DataAuditResult)(r).GetFail()
}
// SetFail sets number of sampled objects under audit stored
// in a way not confirming placement policy or not found at all
// when checking PoP.
func (r *Result) SetFail(fail uint32) {
(*audit.DataAuditResult)(r).SetFail(fail)
}
// PassNodes returns list of storage node public keys that
// passed at least one PDP.
func (r *Result) PassNodes() [][]byte {
return (*audit.DataAuditResult)(r).GetPassNodes()
}
// SetPassNodes sets list of storage node public keys that
// passed at least one PDP.
func (r *Result) SetPassNodes(list [][]byte) {
(*audit.DataAuditResult)(r).SetPassNodes(list)
}
// FailNodes returns list of storage node public keys that
// failed at least one PDP.
func (r *Result) FailNodes() [][]byte {
return (*audit.DataAuditResult)(r).GetFailNodes()
}
// SetFailNodes sets list of storage node public keys that
// failed at least one PDP.
func (r *Result) SetFailNodes(list [][]byte) {
(*audit.DataAuditResult)(r).SetFailNodes(list)
}

View file

@ -1,154 +0,0 @@
package audit_test
import (
"testing"
auditv2 "github.com/nspcc-dev/neofs-api-go/v2/audit"
"github.com/nspcc-dev/neofs-sdk-go/audit"
audittest "github.com/nspcc-dev/neofs-sdk-go/audit/test"
cidtest "github.com/nspcc-dev/neofs-sdk-go/container/id/test"
oid "github.com/nspcc-dev/neofs-sdk-go/object/id"
oidtest "github.com/nspcc-dev/neofs-sdk-go/object/id/test"
"github.com/nspcc-dev/neofs-sdk-go/version"
"github.com/stretchr/testify/require"
)
func TestResult(t *testing.T) {
r := audit.NewResult()
require.Equal(t, version.Current(), r.Version())
epoch := uint64(13)
r.SetAuditEpoch(epoch)
require.Equal(t, epoch, r.AuditEpoch())
cid := cidtest.ID()
r.SetContainerID(cid)
require.Equal(t, cid, r.ContainerID())
key := []byte{1, 2, 3}
r.SetPublicKey(key)
require.Equal(t, key, r.PublicKey())
r.SetComplete(true)
require.True(t, r.Complete())
requests := uint32(2)
r.SetRequests(requests)
require.Equal(t, requests, r.Requests())
retries := uint32(1)
r.SetRetries(retries)
require.Equal(t, retries, r.Retries())
passSG := []oid.ID{*oidtest.ID(), *oidtest.ID()}
r.SetPassSG(passSG)
require.Equal(t, passSG, r.PassSG())
failSG := []oid.ID{*oidtest.ID(), *oidtest.ID()}
r.SetFailSG(failSG)
require.Equal(t, failSG, r.FailSG())
hit := uint32(1)
r.SetHit(hit)
require.Equal(t, hit, r.Hit())
miss := uint32(2)
r.SetMiss(miss)
require.Equal(t, miss, r.Miss())
fail := uint32(3)
r.SetFail(fail)
require.Equal(t, fail, r.Fail())
passNodes := [][]byte{{1}, {2}}
r.SetPassNodes(passNodes)
require.Equal(t, passNodes, r.PassNodes())
failNodes := [][]byte{{3}, {4}}
r.SetFailNodes(failNodes)
require.Equal(t, failNodes, r.FailNodes())
}
func TestStorageGroupEncoding(t *testing.T) {
r := audittest.Result()
t.Run("binary", func(t *testing.T) {
data, err := r.Marshal()
require.NoError(t, err)
r2 := audit.NewResult()
require.NoError(t, r2.Unmarshal(data))
require.Equal(t, r, r2)
})
t.Run("json", func(t *testing.T) {
data, err := r.MarshalJSON()
require.NoError(t, err)
r2 := audit.NewResult()
require.NoError(t, r2.UnmarshalJSON(data))
require.Equal(t, r, r2)
})
}
func TestResult_ToV2(t *testing.T) {
t.Run("nil", func(t *testing.T) {
var x *audit.Result
require.Nil(t, x.ToV2())
})
t.Run("default values", func(t *testing.T) {
result := audit.NewResult()
// check initial values
require.Equal(t, version.Current(), result.Version())
require.False(t, result.Complete())
require.Nil(t, result.ContainerID())
require.Nil(t, result.PublicKey())
require.Nil(t, result.PassSG())
require.Nil(t, result.FailSG())
require.Nil(t, result.PassNodes())
require.Nil(t, result.FailNodes())
require.Zero(t, result.Hit())
require.Zero(t, result.Miss())
require.Zero(t, result.Fail())
require.Zero(t, result.Requests())
require.Zero(t, result.Retries())
require.Zero(t, result.AuditEpoch())
// convert to v2 message
resultV2 := result.ToV2()
require.Equal(t, version.Current().ToV2(), resultV2.GetVersion())
require.False(t, resultV2.GetComplete())
require.Nil(t, resultV2.GetContainerID())
require.Nil(t, resultV2.GetPublicKey())
require.Nil(t, resultV2.GetPassSG())
require.Nil(t, resultV2.GetFailSG())
require.Nil(t, resultV2.GetPassNodes())
require.Nil(t, resultV2.GetFailNodes())
require.Zero(t, resultV2.GetHit())
require.Zero(t, resultV2.GetMiss())
require.Zero(t, resultV2.GetFail())
require.Zero(t, resultV2.GetRequests())
require.Zero(t, resultV2.GetRetries())
require.Zero(t, resultV2.GetAuditEpoch())
})
}
func TestNewResultFromV2(t *testing.T) {
t.Run("from nil", func(t *testing.T) {
var x *auditv2.DataAuditResult
require.Nil(t, audit.NewResultFromV2(x))
})
}

View file

@ -1,37 +0,0 @@
package audittest
import (
"github.com/nspcc-dev/neofs-sdk-go/audit"
cidtest "github.com/nspcc-dev/neofs-sdk-go/container/id/test"
oid "github.com/nspcc-dev/neofs-sdk-go/object/id"
oidtest "github.com/nspcc-dev/neofs-sdk-go/object/id/test"
versiontest "github.com/nspcc-dev/neofs-sdk-go/version/test"
)
// Result returns random audit.Result.
func Result() *audit.Result {
x := audit.NewResult()
x.SetVersion(versiontest.Version())
x.SetContainerID(cidtest.ID())
x.SetPublicKey([]byte("key"))
x.SetComplete(true)
x.SetAuditEpoch(44)
x.SetHit(55)
x.SetMiss(66)
x.SetFail(77)
x.SetRetries(88)
x.SetRequests(99)
x.SetFailNodes([][]byte{
[]byte("node1"),
[]byte("node2"),
})
x.SetPassNodes([][]byte{
[]byte("node3"),
[]byte("node4"),
})
x.SetPassSG([]oid.ID{*oidtest.ID(), *oidtest.ID()})
x.SetFailSG([]oid.ID{*oidtest.ID(), *oidtest.ID()})
return x
}

497
bearer/bearer.go Normal file
View file

@ -0,0 +1,497 @@
package bearer
import (
"crypto/ecdsa"
"errors"
"fmt"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
apeV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/ape"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
apeSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
)
// Token represents bearer token for object service operations.
//
// Token is mutually compatible with git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl.BearerToken
// message. See ReadFromV2 / WriteToV2 methods.
//
// Instances can be created using built-in var declaration.
type Token struct {
targetUserSet bool
targetUser user.ID
eaclTableSet bool
eaclTable eacl.Table
lifetimeSet bool
iat, nbf, exp uint64
sigSet bool
sig refs.Signature
apeOverrideSet bool
apeOverride APEOverride
impersonate bool
}
// APEOverride is the list of APE chains defined for a target.
// These chains are meant to serve as overrides to the already defined (or even undefined)
// APE chains for the target (see contract `Policy`).
//
// The server-side processing of the bearer token with set APE overrides must verify if a client is permitted
// to override chains for the target, preventing unauthorized access through the APE mechanism.
type APEOverride struct {
// Target for which chains are applied.
Target apeSDK.ChainTarget
// The list of APE chains.
Chains []apeSDK.Chain
}
// Marshal marshals APEOverride into a protobuf binary form.
func (c *APEOverride) Marshal() ([]byte, error) {
return c.ToV2().StableMarshal(nil), nil
}
// Unmarshal unmarshals protobuf binary representation of APEOverride.
func (c *APEOverride) Unmarshal(data []byte) error {
overrideV2 := new(acl.APEOverride)
if err := overrideV2.Unmarshal(data); err != nil {
return err
}
return c.FromV2(overrideV2)
}
// MarshalJSON encodes APEOverride to protobuf JSON format.
func (c *APEOverride) MarshalJSON() ([]byte, error) {
return c.ToV2().MarshalJSON()
}
// UnmarshalJSON decodes APEOverride from protobuf JSON format.
func (c *APEOverride) UnmarshalJSON(data []byte) error {
overrideV2 := new(acl.APEOverride)
if err := overrideV2.UnmarshalJSON(data); err != nil {
return err
}
return c.FromV2(overrideV2)
}
func (c *APEOverride) FromV2(tokenAPEChains *acl.APEOverride) error {
c.Target.FromV2(tokenAPEChains.GetTarget())
if chains := tokenAPEChains.GetChains(); len(chains) > 0 {
c.Chains = make([]apeSDK.Chain, len(chains))
for i := range chains {
if err := c.Chains[i].ReadFromV2(chains[i]); err != nil {
return fmt.Errorf("invalid APE chain: %w", err)
}
}
}
return nil
}
func (c *APEOverride) ToV2() *acl.APEOverride {
if c == nil {
return nil
}
apeOverride := new(acl.APEOverride)
apeOverride.SetTarget(c.Target.ToV2())
chains := make([]*apeV2.Chain, len(c.Chains))
for i := range c.Chains {
chains[i] = c.Chains[i].ToV2()
}
apeOverride.SetChains(chains)
return apeOverride
}
// reads Token from the acl.BearerToken message. If checkFieldPresence is set,
// returns an error on absence of any protocol-required field.
func (b *Token) readFromV2(m acl.BearerToken, checkFieldPresence bool) error {
var err error
body := m.GetBody()
if checkFieldPresence && body == nil {
return errors.New("missing token body")
}
b.impersonate = body.GetImpersonate()
apeOverrides := body.GetAPEOverride()
eaclTable := body.GetEACL()
if b.eaclTableSet = eaclTable != nil; b.eaclTableSet {
b.eaclTable = *eacl.NewTableFromV2(eaclTable)
} else if checkFieldPresence && !b.impersonate && apeOverrides == nil {
return errors.New("missing eACL table")
}
targetUser := body.GetOwnerID()
if b.targetUserSet = targetUser != nil; b.targetUserSet {
err = b.targetUser.ReadFromV2(*targetUser)
if err != nil {
return fmt.Errorf("invalid target user: %w", err)
}
}
lifetime := body.GetLifetime()
if b.lifetimeSet = lifetime != nil; b.lifetimeSet {
b.iat = lifetime.GetIat()
b.nbf = lifetime.GetNbf()
b.exp = lifetime.GetExp()
} else if checkFieldPresence {
return errors.New("missing token lifetime")
}
if b.apeOverrideSet = apeOverrides != nil; b.apeOverrideSet {
if err = b.apeOverride.FromV2(apeOverrides); err != nil {
return err
}
} else if checkFieldPresence && !b.impersonate && !b.eaclTableSet {
return errors.New("missing APE override")
}
sig := m.GetSignature()
if b.sigSet = sig != nil; sig != nil {
b.sig = *sig
} else if checkFieldPresence {
return errors.New("missing body signature")
}
return nil
}
// ReadFromV2 reads Token from the acl.BearerToken message.
//
// See also WriteToV2.
func (b *Token) ReadFromV2(m acl.BearerToken) error {
return b.readFromV2(m, true)
}
func (b Token) fillBody() *acl.BearerTokenBody {
if !b.eaclTableSet && !b.targetUserSet && !b.lifetimeSet && !b.impersonate && !b.apeOverrideSet {
return nil
}
var body acl.BearerTokenBody
if b.eaclTableSet {
body.SetEACL(b.eaclTable.ToV2())
}
if b.targetUserSet {
var targetUser refs.OwnerID
b.targetUser.WriteToV2(&targetUser)
body.SetOwnerID(&targetUser)
}
if b.lifetimeSet {
var lifetime acl.TokenLifetime
lifetime.SetIat(b.iat)
lifetime.SetNbf(b.nbf)
lifetime.SetExp(b.exp)
body.SetLifetime(&lifetime)
}
if b.apeOverrideSet {
body.SetAPEOverride(b.apeOverride.ToV2())
}
body.SetImpersonate(b.impersonate)
return &body
}
func (b Token) signedData() []byte {
return b.fillBody().StableMarshal(nil)
}
// WriteToV2 writes Token to the acl.BearerToken message.
// The message must not be nil.
//
// See also ReadFromV2.
func (b Token) WriteToV2(m *acl.BearerToken) {
m.SetBody(b.fillBody())
var sig *refs.Signature
if b.sigSet {
sig = &b.sig
}
m.SetSignature(sig)
}
// SetExp sets "exp" (expiration time) claim which identifies the
// expiration time (in FrostFS epochs) after which the Token MUST NOT be
// accepted for processing. The processing of the "exp" claim requires
// that the current epoch MUST be before or equal to the expiration epoch
// listed in the "exp" claim.
//
// Naming is inspired by https://datatracker.ietf.org/doc/html/rfc7519#section-4.1.4.
//
// See also InvalidAt.
func (b *Token) SetExp(exp uint64) {
b.exp = exp
b.lifetimeSet = true
}
// SetNbf sets "nbf" (not before) claim which identifies the time (in
// FrostFS epochs) before which the Token MUST NOT be accepted for processing. The
// processing of the "nbf" claim requires that the current epoch MUST be
// after or equal to the not-before epoch listed in the "nbf" claim.
//
// Naming is inspired by https://datatracker.ietf.org/doc/html/rfc7519#section-4.1.5.
//
// See also InvalidAt.
func (b *Token) SetNbf(nbf uint64) {
b.nbf = nbf
b.lifetimeSet = true
}
// SetIat sets "iat" (issued at) claim which identifies the time (in FrostFS
// epochs) at which the Token was issued. This claim can be used to determine
// the age of the Token.
//
// Naming is inspired by https://datatracker.ietf.org/doc/html/rfc7519#section-4.1.6.
//
// See also InvalidAt.
func (b *Token) SetIat(iat uint64) {
b.iat = iat
b.lifetimeSet = true
}
// InvalidAt asserts "exp", "nbf" and "iat" claims for the given epoch.
//
// Zero Container is invalid in any epoch.
//
// See also SetExp, SetNbf, SetIat.
func (b Token) InvalidAt(epoch uint64) bool {
return !b.lifetimeSet || b.nbf > epoch || b.iat > epoch || b.exp < epoch
}
// SetEACLTable sets eacl.Table that replaces the one from the issuer's
// container. If table has specified container, bearer token can be used only
// for operations within this specific container. Otherwise, Token can be used
// within any issuer's container.
//
// SetEACLTable MUST be called if Token is going to be transmitted over
// FrostFS API V2 protocol.
//
// See also EACLTable, AssertContainer.
func (b *Token) SetEACLTable(table eacl.Table) {
b.eaclTable = table
b.eaclTableSet = true
}
// EACLTable returns extended ACL table set by SetEACLTable.
//
// Zero Token has zero eacl.Table.
func (b Token) EACLTable() eacl.Table {
if b.eaclTableSet {
return b.eaclTable
}
return eacl.Table{}
}
// SetAPEOverride sets APE override to the bearer token.
//
// See also: APEOverride.
func (b *Token) SetAPEOverride(v APEOverride) {
b.apeOverride = v
b.apeOverrideSet = true
}
// APEOverride returns APE override set by SetAPEOverride.
//
// Zero Token has zero APEOverride.
func (b *Token) APEOverride() APEOverride {
if b.apeOverrideSet {
return b.apeOverride
}
return APEOverride{}
}
// SetImpersonate mark token as impersonate to consider token signer as request owner.
// If this field is true extended EACLTable in token body isn't processed.
func (b *Token) SetImpersonate(v bool) {
b.impersonate = v
}
// Impersonate returns true if token is impersonated.
func (b Token) Impersonate() bool {
return b.impersonate
}
// AssertContainer checks if the token is valid within the given container.
//
// Note: cnr is assumed to refer to the issuer's container, otherwise the check
// is meaningless.
//
// Zero Token is valid in any container.
//
// See also SetEACLTable.
func (b Token) AssertContainer(cnr cid.ID) bool {
if !b.eaclTableSet {
return true
}
cnrTable, set := b.eaclTable.CID()
return !set || cnrTable.Equals(cnr)
}
// ForUser specifies ID of the user who can use the Token for the operations
// within issuer's container(s).
//
// Optional: by default, any user has access to Token usage.
//
// See also AssertUser.
func (b *Token) ForUser(id user.ID) {
b.targetUser = id
b.targetUserSet = true
}
// AssertUser checks if the Token is issued to the given user.
//
// Zero Token is available to any user.
//
// See also ForUser.
func (b Token) AssertUser(id user.ID) bool {
return !b.targetUserSet || b.targetUser.Equals(id)
}
// Sign calculates and writes signature of the Token data using issuer's secret.
// Returns signature calculation errors.
//
// Sign MUST be called if Token is going to be transmitted over
// FrostFS API V2 protocol.
//
// Note that any Token mutation is likely to break the signature, so it is
// expected to be calculated as a final stage of Token formation.
//
// See also VerifySignature, Issuer.
func (b *Token) Sign(key ecdsa.PrivateKey) error {
var sig frostfscrypto.Signature
err := sig.Calculate(frostfsecdsa.Signer(key), b.signedData())
if err != nil {
return err
}
sig.WriteToV2(&b.sig)
b.sigSet = true
return nil
}
// VerifySignature checks if Token signature is presented and valid.
//
// Zero Token fails the check.
//
// See also Sign.
func (b Token) VerifySignature() bool {
if !b.sigSet {
return false
}
var sig frostfscrypto.Signature
// TODO: (#233) check owner<->key relation
return sig.ReadFromV2(b.sig) == nil && sig.Verify(b.signedData())
}
// Marshal encodes Token into a binary format of the FrostFS API protocol
// (Protocol Buffers V3 with direct field order).
//
// See also Unmarshal.
func (b Token) Marshal() []byte {
var m acl.BearerToken
b.WriteToV2(&m)
return m.StableMarshal(nil)
}
// Unmarshal decodes FrostFS API protocol binary data into the Token
// (Protocol Buffers V3 with direct field order). Returns an error describing
// a format violation.
//
// See also Marshal.
func (b *Token) Unmarshal(data []byte) error {
var m acl.BearerToken
err := m.Unmarshal(data)
if err != nil {
return err
}
return b.readFromV2(m, false)
}
// MarshalJSON encodes Token into a JSON format of the FrostFS API protocol
// (Protocol Buffers V3 JSON).
//
// See also UnmarshalJSON.
func (b Token) MarshalJSON() ([]byte, error) {
var m acl.BearerToken
b.WriteToV2(&m)
return m.MarshalJSON()
}
// UnmarshalJSON decodes FrostFS API protocol JSON data into the Token
// (Protocol Buffers V3 JSON). Returns an error describing a format violation.
//
// See also MarshalJSON.
func (b *Token) UnmarshalJSON(data []byte) error {
var m acl.BearerToken
err := m.UnmarshalJSON(data)
if err != nil {
return err
}
return b.readFromV2(m, false)
}
// SigningKeyBytes returns issuer's public key in a binary format of
// FrostFS API protocol.
//
// Unsigned Token has empty key.
//
// See also ResolveIssuer.
func (b Token) SigningKeyBytes() []byte {
if b.sigSet {
return b.sig.GetKey()
}
return nil
}
// ResolveIssuer resolves issuer's user.ID from the key used for Token signing.
// Returns zero user.ID if Token is unsigned or key has incorrect format.
//
// See also SigningKeyBytes.
func ResolveIssuer(b Token) (usr user.ID) {
binKey := b.SigningKeyBytes()
if len(binKey) != 0 {
var key frostfsecdsa.PublicKey
if key.Decode(binKey) == nil {
user.IDFromKey(&usr, ecdsa.PublicKey(key))
}
}
return
}

447
bearer/bearer_test.go Normal file
View file

@ -0,0 +1,447 @@
package bearer_test
import (
"bytes"
"math/rand"
"reflect"
"testing"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
bearertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer/test"
cidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id/test"
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl"
eacltest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl/test"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
usertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user/test"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/stretchr/testify/require"
)
// compares binary representations of two eacl.Table instances.
func isEqualEACLTables(t1, t2 eacl.Table) bool {
d1, err := t1.Marshal()
if err != nil {
panic(err)
}
d2, err := t2.Marshal()
if err != nil {
panic(err)
}
return bytes.Equal(d1, d2)
}
func TestToken_SetEACLTable(t *testing.T) {
var val bearer.Token
var m acl.BearerToken
filled := bearertest.Token()
val.WriteToV2(&m)
require.Zero(t, m.GetBody())
val2 := filled
require.NoError(t, val2.Unmarshal(val.Marshal()))
require.Zero(t, val2.EACLTable())
val2 = filled
jd, err := val.MarshalJSON()
require.NoError(t, err)
require.NoError(t, val2.UnmarshalJSON(jd))
require.Zero(t, val2.EACLTable())
// set value
eaclTable := *eacltest.Table()
val.SetEACLTable(eaclTable)
require.True(t, isEqualEACLTables(eaclTable, val.EACLTable()))
val.WriteToV2(&m)
eaclTableV2 := eaclTable.ToV2()
require.Equal(t, eaclTableV2, m.GetBody().GetEACL())
val2 = filled
require.NoError(t, val2.Unmarshal(val.Marshal()))
require.True(t, isEqualEACLTables(eaclTable, val.EACLTable()))
val2 = filled
jd, err = val.MarshalJSON()
require.NoError(t, err)
require.NoError(t, val2.UnmarshalJSON(jd))
require.True(t, isEqualEACLTables(eaclTable, val.EACLTable()))
}
func TestToken_SetAPEOverrides(t *testing.T) {
var val bearer.Token
var m acl.BearerToken
filled := bearertest.Token()
val.WriteToV2(&m)
require.Zero(t, m.GetBody())
val2 := filled
require.NoError(t, val2.Unmarshal(val.Marshal()))
require.Zero(t, val2.APEOverride())
val2 = filled
jd, err := val.MarshalJSON()
require.NoError(t, err)
require.NoError(t, val2.UnmarshalJSON(jd))
require.Zero(t, val2.APEOverride())
// set value
tApe := bearertest.APEOverride()
val.SetAPEOverride(tApe)
require.Equal(t, tApe, val.APEOverride())
val.WriteToV2(&m)
require.NotNil(t, m.GetBody().GetAPEOverride())
require.True(t, tokenAPEOverridesEqual(tApe.ToV2(), m.GetBody().GetAPEOverride()))
val2 = filled
require.NoError(t, val2.Unmarshal(val.Marshal()))
apeOverride := val2.APEOverride()
require.True(t, tokenAPEOverridesEqual(tApe.ToV2(), apeOverride.ToV2()))
val2 = filled
jd, err = val.MarshalJSON()
require.NoError(t, err)
require.NoError(t, val2.UnmarshalJSON(jd))
apeOverride = val.APEOverride()
require.True(t, tokenAPEOverridesEqual(tApe.ToV2(), apeOverride.ToV2()))
}
func tokenAPEOverridesEqual(lhs, rhs *acl.APEOverride) bool {
return reflect.DeepEqual(lhs, rhs)
}
func TestToken_ForUser(t *testing.T) {
var val bearer.Token
var m acl.BearerToken
filled := bearertest.Token()
val.WriteToV2(&m)
require.Zero(t, m.GetBody())
val2 := filled
require.NoError(t, val2.Unmarshal(val.Marshal()))
val2.WriteToV2(&m)
require.Zero(t, m.GetBody())
val2 = filled
jd, err := val.MarshalJSON()
require.NoError(t, err)
require.NoError(t, val2.UnmarshalJSON(jd))
val2.WriteToV2(&m)
require.Zero(t, m.GetBody())
// set value
usr := usertest.ID()
var usrV2 refs.OwnerID
usr.WriteToV2(&usrV2)
val.ForUser(usr)
val.WriteToV2(&m)
require.Equal(t, usrV2, *m.GetBody().GetOwnerID())
val2 = filled
require.NoError(t, val2.Unmarshal(val.Marshal()))
val2.WriteToV2(&m)
require.Equal(t, usrV2, *m.GetBody().GetOwnerID())
val2 = filled
jd, err = val.MarshalJSON()
require.NoError(t, err)
require.NoError(t, val2.UnmarshalJSON(jd))
val2.WriteToV2(&m)
require.Equal(t, usrV2, *m.GetBody().GetOwnerID())
}
func testLifetimeClaim(t *testing.T, setter func(*bearer.Token, uint64), getter func(*acl.BearerToken) uint64) {
var val bearer.Token
var m acl.BearerToken
filled := bearertest.Token()
val.WriteToV2(&m)
require.Zero(t, m.GetBody())
val2 := filled
require.NoError(t, val2.Unmarshal(val.Marshal()))
val2.WriteToV2(&m)
require.Zero(t, m.GetBody())
val2 = filled
jd, err := val.MarshalJSON()
require.NoError(t, err)
require.NoError(t, val2.UnmarshalJSON(jd))
val2.WriteToV2(&m)
require.Zero(t, m.GetBody())
// set value
exp := rand.Uint64()
setter(&val, exp)
val.WriteToV2(&m)
require.Equal(t, exp, getter(&m))
val2 = filled
require.NoError(t, val2.Unmarshal(val.Marshal()))
val2.WriteToV2(&m)
require.Equal(t, exp, getter(&m))
val2 = filled
jd, err = val.MarshalJSON()
require.NoError(t, err)
require.NoError(t, val2.UnmarshalJSON(jd))
val2.WriteToV2(&m)
require.Equal(t, exp, getter(&m))
}
func TestToken_SetLifetime(t *testing.T) {
t.Run("iat", func(t *testing.T) {
testLifetimeClaim(t, (*bearer.Token).SetIat, func(token *acl.BearerToken) uint64 {
return token.GetBody().GetLifetime().GetIat()
})
})
t.Run("nbf", func(t *testing.T) {
testLifetimeClaim(t, (*bearer.Token).SetNbf, func(token *acl.BearerToken) uint64 {
return token.GetBody().GetLifetime().GetNbf()
})
})
t.Run("exp", func(t *testing.T) {
testLifetimeClaim(t, (*bearer.Token).SetExp, func(token *acl.BearerToken) uint64 {
return token.GetBody().GetLifetime().GetExp()
})
})
}
func TestToken_InvalidAt(t *testing.T) {
var val bearer.Token
require.True(t, val.InvalidAt(0))
require.True(t, val.InvalidAt(1))
val.SetIat(1)
val.SetNbf(2)
val.SetExp(4)
require.True(t, val.InvalidAt(0))
require.True(t, val.InvalidAt(1))
require.False(t, val.InvalidAt(2))
require.False(t, val.InvalidAt(3))
require.False(t, val.InvalidAt(4))
require.True(t, val.InvalidAt(5))
}
func TestToken_AssertContainer(t *testing.T) {
var val bearer.Token
cnr := cidtest.ID()
require.True(t, val.AssertContainer(cnr))
eaclTable := *eacltest.Table()
eaclTable.SetCID(cidtest.ID())
val.SetEACLTable(eaclTable)
require.False(t, val.AssertContainer(cnr))
eaclTable.SetCID(cnr)
val.SetEACLTable(eaclTable)
require.True(t, val.AssertContainer(cnr))
}
func TestToken_AssertUser(t *testing.T) {
var val bearer.Token
usr := usertest.ID()
require.True(t, val.AssertUser(usr))
val.ForUser(usertest.ID())
require.False(t, val.AssertUser(usr))
val.ForUser(usr)
require.True(t, val.AssertUser(usr))
}
func TestToken_Sign(t *testing.T) {
var val bearer.Token
require.False(t, val.VerifySignature())
k, err := keys.NewPrivateKey()
require.NoError(t, err)
key := k.PrivateKey
val = bearertest.Token()
require.NoError(t, val.Sign(key))
require.True(t, val.VerifySignature())
var m acl.BearerToken
val.WriteToV2(&m)
require.NotZero(t, m.GetSignature().GetKey())
require.NotZero(t, m.GetSignature().GetSign())
val2 := bearertest.Token()
require.NoError(t, val2.Unmarshal(val.Marshal()))
require.True(t, val2.VerifySignature())
jd, err := val.MarshalJSON()
require.NoError(t, err)
val2 = bearertest.Token()
require.NoError(t, val2.UnmarshalJSON(jd))
require.True(t, val2.VerifySignature())
}
func TestToken_ReadFromV2(t *testing.T) {
var val bearer.Token
var m acl.BearerToken
require.Error(t, val.ReadFromV2(m))
var body acl.BearerTokenBody
m.SetBody(&body)
require.Error(t, val.ReadFromV2(m))
eaclTable := eacltest.Table().ToV2()
body.SetEACL(eaclTable)
require.Error(t, val.ReadFromV2(m))
var lifetime acl.TokenLifetime
body.SetLifetime(&lifetime)
require.Error(t, val.ReadFromV2(m))
const iat, nbf, exp = 1, 2, 3
lifetime.SetIat(iat)
lifetime.SetNbf(nbf)
lifetime.SetExp(exp)
body.SetLifetime(&lifetime)
require.Error(t, val.ReadFromV2(m))
var sig refs.Signature
m.SetSignature(&sig)
require.NoError(t, val.ReadFromV2(m))
body.SetEACL(nil)
body.SetImpersonate(true)
require.NoError(t, val.ReadFromV2(m))
var m2 acl.BearerToken
val.WriteToV2(&m2)
require.Equal(t, m, m2)
usr, usr2 := usertest.ID(), usertest.ID()
require.True(t, val.AssertUser(usr))
require.True(t, val.AssertUser(usr2))
var usrV2 refs.OwnerID
usr.WriteToV2(&usrV2)
body.SetOwnerID(&usrV2)
require.NoError(t, val.ReadFromV2(m))
val.WriteToV2(&m2)
require.Equal(t, m, m2)
require.True(t, val.AssertUser(usr))
require.False(t, val.AssertUser(usr2))
k, err := keys.NewPrivateKey()
require.NoError(t, err)
signer := frostfsecdsa.Signer(k.PrivateKey)
var s frostfscrypto.Signature
require.NoError(t, s.Calculate(signer, body.StableMarshal(nil)))
s.WriteToV2(&sig)
require.NoError(t, val.ReadFromV2(m))
require.True(t, val.VerifySignature())
require.Equal(t, sig.GetKey(), val.SigningKeyBytes())
}
func TestResolveIssuer(t *testing.T) {
k, err := keys.NewPrivateKey()
require.NoError(t, err)
var val bearer.Token
require.Zero(t, bearer.ResolveIssuer(val))
var m acl.BearerToken
var sig refs.Signature
sig.SetKey([]byte("invalid key"))
m.SetSignature(&sig)
require.NoError(t, val.Unmarshal(m.StableMarshal(nil)))
require.Zero(t, bearer.ResolveIssuer(val))
require.NoError(t, val.Sign(k.PrivateKey))
var usr user.ID
user.IDFromKey(&usr, k.PrivateKey.PublicKey)
require.Equal(t, usr, bearer.ResolveIssuer(val))
}

31
bearer/doc.go Normal file
View file

@ -0,0 +1,31 @@
/*
Package bearer provides bearer token definition.
Bearer token is attached to the object service requests, and it overwrites
extended ACL of the container. Mainly it is used to provide access of private
data for specific user. Therefore, it must be signed by owner of the container.
Define bearer token by setting correct lifetime, extended ACL and owner ID of
the user that will attach token to its requests.
var bearerToken bearer.Token
bearerToken.SetExpiration(500)
bearerToken.SetIssuedAt(10)
bearerToken.SetNotBefore(10)
bearerToken.SetEACL(eaclTable)
bearerToken.SetOwner(ownerID)
Bearer token must be signed by owner of the container.
err := bearerToken.Sign(privateKey)
Provide signed token in JSON or binary format to the request sender. Request
sender can attach this bearer token to the object service requests:
import sdkClient "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client"
var headParams sdkClient.PrmObjectHead
headParams.WithBearerToken(bearerToken)
response, err := client.ObjectHead(ctx, headParams)
*/
package bearer

6
bearer/test/doc.go Normal file
View file

@ -0,0 +1,6 @@
/*
Package bearertest provides functions for testing bearer package.
Note that this package intended only for tests.
*/
package bearertest

32
bearer/test/generate.go Normal file
View file

@ -0,0 +1,32 @@
package bearertest
import (
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
eacltest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/eacl/test"
usertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user/test"
)
// Token returns random bearer.Token.
//
// Resulting token is unsigned.
func Token() (t bearer.Token) {
t.SetExp(3)
t.SetNbf(2)
t.SetIat(1)
t.ForUser(usertest.ID())
t.SetEACLTable(*eacltest.Table())
t.SetAPEOverride(APEOverride())
return t
}
func APEOverride() bearer.APEOverride {
return bearer.APEOverride{
Target: ape.ChainTarget{
TargetType: ape.TargetTypeContainer,
Name: "F8JsMnChywiPvbDvpxMbjTjx5KhWHHp6gCDt8BhzL9kF",
},
Chains: []ape.Chain{{Raw: []byte("{}")}},
}
}

View file

@ -1,51 +1,78 @@
package checksum
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"github.com/nspcc-dev/neofs-api-go/v2/refs"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
"git.frostfs.info/TrueCloudLab/tzhash/tz"
)
// Checksum represents v2-compatible checksum.
// Checksum represents checksum of some digital data.
//
// Checksum is mutually compatible with git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs.Checksum
// message. See ReadFromV2 / WriteToV2 methods.
//
// Instances can be created using built-in var declaration.
//
// Note that direct typecast is not safe and may result in loss of compatibility:
//
// _ = Checksum(refs.Checksum{}) // not recommended
type Checksum refs.Checksum
// Type represents the enumeration
// of checksum types.
type Type uint8
type Type refs.ChecksumType
const (
// Unknown is an undefined checksum type.
Unknown Type = iota
Unknown Type = Type(refs.UnknownChecksum)
// SHA256 is a SHA256 checksum type.
SHA256
SHA256 = Type(refs.SHA256)
// TZ is a Tillich-Zemor checksum type.
TZ
// TZ is a Tillich-Zémor checksum type.
TZ = Type(refs.TillichZemor)
)
// NewFromV2 wraps v2 Checksum message to Checksum.
// ReadFromV2 reads Checksum from the refs.Checksum message. Checks if the
// message conforms to FrostFS API V2 protocol.
//
// Nil refs.Checksum converts to nil.
func NewFromV2(cV2 *refs.Checksum) *Checksum {
return (*Checksum)(cV2)
// See also WriteToV2.
func (c *Checksum) ReadFromV2(m refs.Checksum) error {
if len(m.GetSum()) == 0 {
return errors.New("missing value")
}
// New creates and initializes blank Checksum.
switch m.GetType() {
default:
return fmt.Errorf("unsupported type %v", m.GetType())
case refs.SHA256, refs.TillichZemor:
}
*c = Checksum(m)
return nil
}
// WriteToV2 writes Checksum to the refs.Checksum message.
// The message must not be nil.
//
// Defaults:
// - sum: nil;
// - type: Unknown.
func New() *Checksum {
return NewFromV2(new(refs.Checksum))
// See also ReadFromV2.
func (c Checksum) WriteToV2(m *refs.Checksum) {
*m = (refs.Checksum)(c)
}
// Type returns checksum type.
func (c *Checksum) Type() Type {
switch (*refs.Checksum)(c).GetType() {
//
// Zero Checksum has Unknown checksum type.
//
// See also SetTillichZemor and SetSHA256.
func (c Checksum) Type() Type {
v2 := (refs.Checksum)(c)
switch v2.GetType() {
case refs.SHA256:
return SHA256
case refs.TillichZemor:
@ -55,93 +82,70 @@ func (c *Checksum) Type() Type {
}
}
// Sum returns checksum bytes.
func (c *Checksum) Sum() []byte {
return (*refs.Checksum)(c).GetSum()
// Value returns checksum bytes. Return value
// MUST NOT be mutated.
//
// Zero Checksum has nil sum.
//
// See also SetTillichZemor and SetSHA256.
func (c Checksum) Value() []byte {
v2 := (refs.Checksum)(c)
return v2.GetSum()
}
// SetSHA256 sets checksum to SHA256 hash.
//
// See also Calculate.
func (c *Checksum) SetSHA256(v [sha256.Size]byte) {
checksum := (*refs.Checksum)(c)
v2 := (*refs.Checksum)(c)
checksum.SetType(refs.SHA256)
checksum.SetSum(v[:])
v2.SetType(refs.SHA256)
v2.SetSum(v[:])
}
// SetTillichZemor sets checksum to Tillich-Zemor hash.
func (c *Checksum) SetTillichZemor(v [64]byte) {
checksum := (*refs.Checksum)(c)
checksum.SetType(refs.TillichZemor)
checksum.SetSum(v[:])
}
// ToV2 converts Checksum to v2 Checksum message.
// Calculate calculates checksum and sets it
// to the passed checksum. Checksum must not be nil.
//
// Nil Checksum converts to nil.
func (c *Checksum) ToV2() *refs.Checksum {
return (*refs.Checksum)(c)
}
func Equal(cs1, cs2 *Checksum) bool {
return cs1.Type() == cs2.Type() && bytes.Equal(cs1.Sum(), cs2.Sum())
}
// Marshal marshals Checksum into a protobuf binary form.
func (c *Checksum) Marshal() ([]byte, error) {
return (*refs.Checksum)(c).StableMarshal(nil)
}
// Unmarshal unmarshals protobuf binary representation of Checksum.
func (c *Checksum) Unmarshal(data []byte) error {
return (*refs.Checksum)(c).Unmarshal(data)
}
// MarshalJSON encodes Checksum to protobuf JSON format.
func (c *Checksum) MarshalJSON() ([]byte, error) {
return (*refs.Checksum)(c).MarshalJSON()
}
// UnmarshalJSON decodes Checksum from protobuf JSON format.
func (c *Checksum) UnmarshalJSON(data []byte) error {
return (*refs.Checksum)(c).UnmarshalJSON(data)
}
func (c *Checksum) String() string {
return hex.EncodeToString((*refs.Checksum)(c).GetSum())
}
// Parse parses Checksum from its string representation.
func (c *Checksum) Parse(s string) error {
data, err := hex.DecodeString(s)
if err != nil {
return err
}
var typ refs.ChecksumType
switch ln := len(data); ln {
// Does nothing if the passed type is not one of the:
// - SHA256;
// - TZ.
//
// Does not mutate the passed value.
//
// See also SetSHA256, SetTillichZemor.
func Calculate(c *Checksum, t Type, v []byte) {
switch t {
case SHA256:
c.SetSHA256(sha256.Sum256(v))
case TZ:
c.SetTillichZemor(tz.Sum(v))
default:
return fmt.Errorf("unsupported checksum length %d", ln)
case sha256.Size:
typ = refs.SHA256
case 64:
typ = refs.TillichZemor
}
}
cV2 := (*refs.Checksum)(c)
cV2.SetType(typ)
cV2.SetSum(data)
return nil
}
// String returns string representation of Type.
// SetTillichZemor sets checksum to Tillich-Zémor hash.
//
// String mapping:
// * TZ: TZ;
// * SHA256: SHA256;
// * Unknown, default: CHECKSUM_TYPE_UNSPECIFIED.
// See also Calculate.
func (c *Checksum) SetTillichZemor(v [tz.Size]byte) {
v2 := (*refs.Checksum)(c)
v2.SetType(refs.TillichZemor)
v2.SetSum(v[:])
}
// String implements fmt.Stringer.
//
// String is designed to be human-readable, and its format MAY differ between
// SDK versions.
func (c Checksum) String() string {
v2 := (refs.Checksum)(c)
return fmt.Sprintf("%s:%s", c.Type(), hex.EncodeToString(v2.GetSum()))
}
// String implements fmt.Stringer.
//
// String is designed to be human-readable, and its format MAY differ between
// SDK versions.
func (m Type) String() string {
var m2 refs.ChecksumType
@ -156,26 +160,3 @@ func (m Type) String() string {
return m2.String()
}
// FromString parses Type from a string representation.
// It is a reverse action to String().
//
// Returns true if s was parsed successfully.
func (m *Type) FromString(s string) bool {
var g refs.ChecksumType
ok := g.FromString(s)
if ok {
switch g {
default:
*m = Unknown
case refs.TillichZemor:
*m = TZ
case refs.SHA256:
*m = SHA256
}
}
return ok
}

View file

@ -5,20 +5,13 @@ import (
"crypto/sha256"
"testing"
"github.com/nspcc-dev/neofs-api-go/v2/refs"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
"git.frostfs.info/TrueCloudLab/tzhash/tz"
"github.com/stretchr/testify/require"
)
func randSHA256(t *testing.T) [sha256.Size]byte {
cSHA256 := [sha256.Size]byte{}
_, err := rand.Read(cSHA256[:])
require.NoError(t, err)
return cSHA256
}
func TestChecksum(t *testing.T) {
c := New()
var c Checksum
cSHA256 := [sha256.Size]byte{}
_, _ = rand.Read(cSHA256[:])
@ -26,150 +19,62 @@ func TestChecksum(t *testing.T) {
c.SetSHA256(cSHA256)
require.Equal(t, SHA256, c.Type())
require.Equal(t, cSHA256[:], c.Sum())
require.Equal(t, cSHA256[:], c.Value())
cV2 := c.ToV2()
var cV2 refs.Checksum
c.WriteToV2(&cV2)
require.Equal(t, refs.SHA256, cV2.GetType())
require.Equal(t, cSHA256[:], cV2.GetSum())
cTZ := [64]byte{}
cTZ := [tz.Size]byte{}
_, _ = rand.Read(cSHA256[:])
c.SetTillichZemor(cTZ)
require.Equal(t, TZ, c.Type())
require.Equal(t, cTZ[:], c.Sum())
require.Equal(t, cTZ[:], c.Value())
cV2 = c.ToV2()
c.WriteToV2(&cV2)
require.Equal(t, refs.TillichZemor, cV2.GetType())
require.Equal(t, cTZ[:], cV2.GetSum())
}
func TestEqualChecksums(t *testing.T) {
require.True(t, Equal(nil, nil))
csSHA := [sha256.Size]byte{}
_, _ = rand.Read(csSHA[:])
cs1 := New()
cs1.SetSHA256(csSHA)
cs2 := New()
cs2.SetSHA256(csSHA)
require.True(t, Equal(cs1, cs2))
csSHA[0]++
cs2.SetSHA256(csSHA)
require.False(t, Equal(cs1, cs2))
}
func TestChecksumEncoding(t *testing.T) {
cs := New()
cs.SetSHA256(randSHA256(t))
t.Run("binary", func(t *testing.T) {
data, err := cs.Marshal()
require.NoError(t, err)
c2 := New()
require.NoError(t, c2.Unmarshal(data))
require.Equal(t, cs, c2)
})
t.Run("json", func(t *testing.T) {
data, err := cs.MarshalJSON()
require.NoError(t, err)
cs2 := New()
require.NoError(t, cs2.UnmarshalJSON(data))
require.Equal(t, cs, cs2)
})
t.Run("string", func(t *testing.T) {
cs2 := New()
require.NoError(t, cs2.Parse(cs.String()))
require.Equal(t, cs, cs2)
})
}
func TestNewChecksumFromV2(t *testing.T) {
t.Run("from nil", func(t *testing.T) {
var x *refs.Checksum
require.Nil(t, NewFromV2(x))
})
}
func TestChecksum_ToV2(t *testing.T) {
t.Run("nil", func(t *testing.T) {
var x *Checksum
require.Nil(t, x.ToV2())
})
}
func TestNewChecksum(t *testing.T) {
t.Run("default values", func(t *testing.T) {
chs := New()
var chs Checksum
// check initial values
require.Equal(t, Unknown, chs.Type())
require.Nil(t, chs.Sum())
require.Nil(t, chs.Value())
// convert to v2 message
chsV2 := chs.ToV2()
var chsV2 refs.Checksum
chs.WriteToV2(&chsV2)
require.Equal(t, refs.UnknownChecksum, chsV2.GetType())
require.Nil(t, chsV2.GetSum())
})
}
type enumIface interface {
FromString(string) bool
String() string
}
func TestCalculation(t *testing.T) {
var c Checksum
payload := []byte{0, 1, 2, 3, 4, 5}
type enumStringItem struct {
val enumIface
str string
}
t.Run("SHA256", func(t *testing.T) {
orig := sha256.Sum256(payload)
func testEnumStrings(t *testing.T, e enumIface, items []enumStringItem) {
for _, item := range items {
require.Equal(t, item.str, item.val.String())
Calculate(&c, SHA256, payload)
s := item.val.String()
require.Equal(t, orig[:], c.Value())
})
require.True(t, e.FromString(s), s)
t.Run("TZ", func(t *testing.T) {
orig := tz.Sum(payload)
require.EqualValues(t, item.val, e, item.val)
}
Calculate(&c, TZ, payload)
// incorrect strings
for _, str := range []string{
"some string",
"undefined",
} {
require.False(t, e.FromString(str))
}
}
func TestChecksumType_String(t *testing.T) {
toPtr := func(v Type) *Type {
return &v
}
testEnumStrings(t, new(Type), []enumStringItem{
{val: toPtr(TZ), str: "TZ"},
{val: toPtr(SHA256), str: "SHA256"},
{val: toPtr(Unknown), str: "CHECKSUM_TYPE_UNSPECIFIED"},
require.Equal(t, orig[:], c.Value())
})
}

18
checksum/doc.go Normal file
View file

@ -0,0 +1,18 @@
/*
Package checksum provides primitives to work with checksums.
Checksum is a basic type of data checksums.
For example, calculating checksums:
// retrieving any payload for hashing
var sha256Sum Checksum
Calculate(&sha256Sum, SHA256, payload) // sha256Sum contains SHA256 hash of the payload
var tzSum Checksum
Calculate(&tzSum, TZ, payload) // tzSum contains TZ hash of the payload
Using package types in an application is recommended to potentially work with
different protocol versions with which these types are compatible.
*/
package checksum

34
checksum/example_test.go Normal file
View file

@ -0,0 +1,34 @@
package checksum
import (
"bytes"
"crypto/rand"
"crypto/sha256"
"fmt"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
)
func ExampleCalculate() {
payload := []byte{0, 1, 2, 3, 4, 5, 6}
var cs Checksum
Calculate(&cs, SHA256, payload)
Calculate(&cs, TZ, payload)
}
func ExampleChecksum_WriteToV2() {
var (
csRaw [sha256.Size]byte
csV2 refs.Checksum
cs Checksum
)
rand.Read(csRaw[:])
cs.SetSHA256(csRaw)
cs.WriteToV2(&csV2)
fmt.Println(bytes.Equal(cs.Value(), csV2.GetSum()))
// Output: true
}

13
checksum/test/doc.go Normal file
View file

@ -0,0 +1,13 @@
/*
Package checksumtest provides functions for convenient testing of checksum package API.
Note that importing the package into source files is highly discouraged.
Random instance generation functions can be useful when testing expects any value, e.g.:
import checksumtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/checksum/test"
cs := checksumtest.Checksum()
// test the value
*/
package checksumtest

View file

@ -1,19 +1,19 @@
package checksumtest
import (
"crypto/rand"
"crypto/sha256"
"math/rand"
"github.com/nspcc-dev/neofs-sdk-go/checksum"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/checksum"
)
// Checksum returns random checksum.Checksum.
func Checksum() *checksum.Checksum {
func Checksum() checksum.Checksum {
var cs [sha256.Size]byte
rand.Read(cs[:])
_, _ = rand.Read(cs[:])
x := checksum.New()
var x checksum.Checksum
x.SetSHA256(cs)

View file

@ -2,103 +2,107 @@ package client
import (
"context"
"fmt"
v2accounting "github.com/nspcc-dev/neofs-api-go/v2/accounting"
rpcapi "github.com/nspcc-dev/neofs-api-go/v2/rpc"
"github.com/nspcc-dev/neofs-api-go/v2/rpc/client"
"github.com/nspcc-dev/neofs-sdk-go/accounting"
"github.com/nspcc-dev/neofs-sdk-go/owner"
v2accounting "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/accounting"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/accounting"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
)
// PrmBalanceGet groups parameters of BalanceGet operation.
type PrmBalanceGet struct {
prmCommonMeta
XHeaders []string
ownerSet bool
ownerID owner.ID
Account user.ID
}
// SetAccount sets identifier of the NeoFS account for which the balance is requested.
// Required parameter. Must be a valid ID according to NeoFS API protocol.
func (x *PrmBalanceGet) SetAccount(id owner.ID) {
x.ownerID = id
x.ownerSet = true
// SetAccount sets identifier of the FrostFS account for which the balance is requested.
// Required parameter.
//
// Deprecated: Use PrmBalanceGet.Account instead.
func (x *PrmBalanceGet) SetAccount(id user.ID) {
x.Account = id
}
func (x *PrmBalanceGet) buildRequest(c *Client) (*v2accounting.BalanceRequest, error) {
if x.Account.IsEmpty() {
return nil, errorAccountNotSet
}
var accountV2 refs.OwnerID
x.Account.WriteToV2(&accountV2)
var body v2accounting.BalanceRequestBody
body.SetOwnerID(&accountV2)
var req v2accounting.BalanceRequest
req.SetBody(&body)
c.prepareRequest(&req, new(v2session.RequestMetaHeader))
return &req, nil
}
// ResBalanceGet groups resulting values of BalanceGet operation.
type ResBalanceGet struct {
statusRes
amount *accounting.Decimal
amount accounting.Decimal
}
func (x *ResBalanceGet) setAmount(v *accounting.Decimal) {
x.amount = v
}
// Amount returns current amount of funds on the NeoFS account as decimal number.
//
// Client doesn't retain value so modification is safe.
func (x ResBalanceGet) Amount() *accounting.Decimal {
// Amount returns current amount of funds on the FrostFS account as decimal number.
func (x ResBalanceGet) Amount() accounting.Decimal {
return x.amount
}
// BalanceGet requests current balance of the NeoFS account.
// BalanceGet requests current balance of the FrostFS account.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`,
// If WithNeoFSErrorParsing option has been provided, unsuccessful
// NeoFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Immediately panics if parameters are set incorrectly (see PrmBalanceGet docs).
// Returns an error if parameters are set incorrectly (see PrmBalanceGet docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) BalanceGet(ctx context.Context, prm PrmBalanceGet) (*ResBalanceGet, error) {
switch {
case ctx == nil:
panic(panicMsgMissingContext)
case !prm.ownerSet:
panic("account not set")
case !prm.ownerID.Valid():
panic("invalid account ID")
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
// form request body
var body v2accounting.BalanceRequestBody
body.SetOwnerID(prm.ownerID.ToV2())
// form request
var req v2accounting.BalanceRequest
req.SetBody(&body)
// init call context
var (
cc contextCall
res ResBalanceGet
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.Balance(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2accounting.BalanceResponse)
res.setAmount(accounting.NewDecimalFromV2(resp.GetBody().GetBalance()))
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
// process call
if !cc.processCall() {
return nil, cc.err
resp, err := rpcapi.Balance(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResBalanceGet
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
const fieldBalance = "balance"
bal := resp.GetBody().GetBalance()
if bal == nil {
return &res, newErrMissingResponseField(fieldBalance)
}
if err := res.amount.ReadFromV2(*bal); err != nil {
return &res, newErrInvalidResponseField(fieldBalance, err)
}
return &res, nil
}

View file

@ -0,0 +1,79 @@
package client
import (
"context"
"fmt"
apemanagerV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/apemanager"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
sessionV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apeSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
)
// PrmAPEManagerAddChain groups parameters of APEManagerAddChain operation.
type PrmAPEManagerAddChain struct {
XHeaders []string
ChainTarget apeSDK.ChainTarget
Chain apeSDK.Chain
}
func (prm *PrmAPEManagerAddChain) buildRequest(c *Client) (*apemanagerV2.AddChainRequest, error) {
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
req := new(apemanagerV2.AddChainRequest)
reqBody := new(apemanagerV2.AddChainRequestBody)
reqBody.SetTarget(prm.ChainTarget.ToV2())
reqBody.SetChain(prm.Chain.ToV2())
req.SetBody(reqBody)
var meta sessionV2.RequestMetaHeader
writeXHeadersToMeta(prm.XHeaders, &meta)
c.prepareRequest(req, &meta)
return req, nil
}
type ResAPEManagerAddChain struct {
statusRes
// ChainID of set Chain. If Chain does not contain chainID before request, then
// ChainID is generated.
ChainID apeSDK.ChainID
}
// APEManagerAddChain sets Chain for ChainTarget.
func (c *Client) APEManagerAddChain(ctx context.Context, prm PrmAPEManagerAddChain) (*ResAPEManagerAddChain, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.AddChain(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResAPEManagerAddChain
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
res.ChainID = resp.GetBody().GetChainID()
return &res, nil
}

View file

@ -0,0 +1,80 @@
package client
import (
"context"
"fmt"
apemanagerV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/apemanager"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
sessionV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apeSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
)
// PrmAPEManagerListChains groups parameters of APEManagerListChains operation.
type PrmAPEManagerListChains struct {
XHeaders []string
ChainTarget apeSDK.ChainTarget
}
func (prm *PrmAPEManagerListChains) buildRequest(c *Client) (*apemanagerV2.ListChainsRequest, error) {
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
req := new(apemanagerV2.ListChainsRequest)
reqBody := new(apemanagerV2.ListChainsRequestBody)
reqBody.SetTarget(prm.ChainTarget.ToV2())
req.SetBody(reqBody)
var meta sessionV2.RequestMetaHeader
writeXHeadersToMeta(prm.XHeaders, &meta)
c.prepareRequest(req, &meta)
return req, nil
}
type ResAPEManagerListChains struct {
statusRes
Chains []apeSDK.Chain
}
// APEManagerListChains lists Chains for ChainTarget.
func (c *Client) APEManagerListChains(ctx context.Context, prm PrmAPEManagerListChains) (*ResAPEManagerListChains, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.ListChains(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResAPEManagerListChains
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return nil, err
}
for _, ch := range resp.GetBody().GetChains() {
var chSDK apeSDK.Chain
if err := chSDK.ReadFromV2(ch); err != nil {
return nil, err
}
res.Chains = append(res.Chains, chSDK)
}
return &res, nil
}

View file

@ -0,0 +1,73 @@
package client
import (
"context"
"fmt"
apemanagerV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/apemanager"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
sessionV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apeSDK "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/ape"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
)
// PrmAPEManagerRemoveChain groups parameters of APEManagerRemoveChain operation.
type PrmAPEManagerRemoveChain struct {
XHeaders []string
ChainTarget apeSDK.ChainTarget
ChainID apeSDK.ChainID
}
func (prm *PrmAPEManagerRemoveChain) buildRequest(c *Client) (*apemanagerV2.RemoveChainRequest, error) {
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
req := new(apemanagerV2.RemoveChainRequest)
reqBody := new(apemanagerV2.RemoveChainRequestBody)
reqBody.SetTarget(prm.ChainTarget.ToV2())
reqBody.SetChainID(prm.ChainID)
req.SetBody(reqBody)
var meta sessionV2.RequestMetaHeader
writeXHeadersToMeta(prm.XHeaders, &meta)
c.prepareRequest(req, &meta)
return req, nil
}
type ResAPEManagerRemoveChain struct {
statusRes
}
// APEManagerRemoveChain removes Chain with ChainID defined for ChainTarget.
func (c *Client) APEManagerRemoveChain(ctx context.Context, prm PrmAPEManagerRemoveChain) (*ResAPEManagerRemoveChain, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.RemoveChain(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResAPEManagerRemoveChain
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
return &res, nil
}

35
client/api.go Normal file
View file

@ -0,0 +1,35 @@
package client
import (
"context"
"fmt"
v2netmap "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
)
// interface of FrostFS API server. Exists for test purposes only.
type frostFSAPIServer interface {
netMapSnapshot(context.Context, v2netmap.SnapshotRequest) (*v2netmap.SnapshotResponse, error)
}
// wrapper over real client connection which communicates over FrostFS API protocol.
// Provides frostFSAPIServer for Client instances used in real applications.
type coreServer client.Client
// unifies errors of all RPC.
func rpcErr(e error) error {
return fmt.Errorf("rpc failure: %w", e)
}
// executes NetmapService.NetmapSnapshot RPC declared in FrostFS API protocol
// using underlying client.Client.
func (x *coreServer) netMapSnapshot(ctx context.Context, req v2netmap.SnapshotRequest) (*v2netmap.SnapshotResponse, error) {
resp, err := rpcapi.NetMapSnapshot((*client.Client)(x), &req, client.WithContext(ctx))
if err != nil {
return nil, rpcErr(err)
}
return resp, nil
}

View file

@ -1,23 +1,26 @@
package client
import (
"context"
"crypto/ecdsa"
"crypto/tls"
"errors"
"time"
v2accounting "github.com/nspcc-dev/neofs-api-go/v2/accounting"
"github.com/nspcc-dev/neofs-api-go/v2/rpc"
"github.com/nspcc-dev/neofs-api-go/v2/rpc/client"
v2accounting "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/accounting"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
"google.golang.org/grpc"
)
// Client represents virtual connection to the NeoFS network to communicate
// with NeoFS server using NeoFS API protocol. It is designed to provide
// Client represents virtual connection to the FrostFS network to communicate
// with FrostFS server using FrostFS API protocol. It is designed to provide
// an abstraction interface from the protocol details of data transfer over
// a network in NeoFS.
// a network in FrostFS.
//
// Client can be created using simple Go variable declaration. Before starting
// work with the Client, it SHOULD BE correctly initialized (see Init method).
// Before executing the NeoFS operations using the Client, connection to the
// Before executing the FrostFS operations using the Client, connection to the
// server MUST BE correctly established (see Dial method and pay attention
// to the mandatory parameters). Using the Client before connecting have
// been established can lead to a panic. After the work, the Client SHOULD BE
@ -26,7 +29,7 @@ import (
// during the communication process step strongly discouraged as it leads to
// undefined behavior.
//
// Each method which produces a NeoFS API call may return a server response.
// Each method which produces a FrostFS API call may return a server response.
// Status responses are returned in the result structure, and can be cast
// to built-in error instance (or in the returned error if the client is
// configured accordingly). Certain statuses can be checked using `apistatus`
@ -35,6 +38,7 @@ import (
// All possible responses are documented in methods, however, some may be
// returned from all of them (pay attention to the presence of the pointer sign):
// - *apistatus.ServerInternal on internal server error;
// - *apistatus.NodeUnderMaintenance if a server is under maintenance;
// - *apistatus.SuccessDefaultV2 on default success.
//
// Client MUST NOT be copied by value: use pointer to Client instead.
@ -44,6 +48,8 @@ type Client struct {
prm PrmInit
c client.Client
server frostFSAPIServer
}
// Init brings the Client instance to its initial state.
@ -56,42 +62,63 @@ func (c *Client) Init(prm PrmInit) {
c.prm = prm
}
// Dial establishes a connection to the server from the NeoFS network.
// Dial establishes a connection to the server from the FrostFS network.
// Returns an error describing failure reason. If failed, the Client
// SHOULD NOT be used.
//
// Panics if required parameters are set incorrectly, look carefully
// Uses the context specified by SetContext if it was called with non-nil
// argument, otherwise context.Background() is used. Dial returns context
// errors, see context package docs for details.
//
// Returns an error if required parameters are set incorrectly, look carefully
// at the method documentation.
//
// One-time method call during application start-up stage (after Init ) is expected.
// Calling multiple times leads to undefined behavior.
//
// See also Init / Close.
func (c *Client) Dial(prm PrmDial) error {
if prm.endpoint == "" {
panic("server address is unset or empty")
func (c *Client) Dial(ctx context.Context, prm PrmDial) error {
if prm.Endpoint == "" {
return errorServerAddrUnset
}
if prm.timeoutDialSet {
if prm.timeoutDial <= 0 {
panic("non-positive timeout")
if prm.DialTimeout <= 0 {
prm.DialTimeout = defaultDialTimeout
}
} else {
prm.timeoutDial = 5 * time.Second
if prm.StreamTimeout <= 0 {
prm.StreamTimeout = defaultStreamTimeout
}
c.c = *client.New(append(
client.WithNetworkURIAddress(prm.endpoint, prm.tlsConfig),
client.WithDialTimeout(prm.timeoutDial),
client.WithNetworkURIAddress(prm.Endpoint, prm.TLSConfig),
client.WithDialTimeout(prm.DialTimeout),
client.WithRWTimeout(prm.StreamTimeout),
client.WithGRPCDialOptions(prm.GRPCDialOptions),
)...)
c.setFrostFSAPIServer((*coreServer)(&c.c))
// TODO: (neofs-api-go#382) perform generic dial stage of the client.Client
_, _ = rpc.Balance(&c.c, new(v2accounting.BalanceRequest))
_, err := rpc.Balance(&c.c, new(v2accounting.BalanceRequest),
client.WithContext(ctx),
)
// return context errors since they signal about dial problem
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
return err
}
return nil
}
// Close closes underlying connection to the NeoFS server. Implements io.Closer.
// sets underlying provider of frostFSAPIServer. The method is used for testing as an approach
// to skip Dial stage and override FrostFS API server. MUST NOT be used outside test code.
// In real applications wrapper over git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client
// is statically used.
func (c *Client) setFrostFSAPIServer(server frostFSAPIServer) {
c.server = server
}
// Close closes underlying connection to the FrostFS server. Implements io.Closer.
// MUST NOT be called before successful Dial. Can be called concurrently
// with server operations processing on running goroutines: in this case
// they are likely to fail due to a connection error.
@ -108,75 +135,117 @@ func (c *Client) Close() error {
//
// See also Init.
type PrmInit struct {
resolveNeoFSErrors bool
DisableFrostFSErrorResolution bool
key ecdsa.PrivateKey
Key ecdsa.PrivateKey
cbRespInfo func(ResponseMetaInfo) error
ResponseInfoCallback func(ResponseMetaInfo) error
netMagic uint64
NetMagic uint64
}
// SetDefaultPrivateKey sets Client private key to be used for the protocol
// communication by default.
//
// Required for operations without custom key parametrization (see corresponding Prm* docs).
//
// Deprecated: Use PrmInit.Key instead.
func (x *PrmInit) SetDefaultPrivateKey(key ecdsa.PrivateKey) {
x.key = key
x.Key = key
}
// ResolveNeoFSFailures makes the Client to resolve failure statuses of the
// NeoFS protocol into Go built-in errors. These errors are returned from
// each protocol operation. By default, statuses aren't resolved and written
// to the resulting structure (see corresponding Res* docs).
func (x *PrmInit) ResolveNeoFSFailures() {
x.resolveNeoFSErrors = true
// Deprecated: method is no-op. Option is default.
func (x *PrmInit) ResolveFrostFSFailures() {
}
// DisableFrostFSFailuresResolution makes the Client to preserve failure statuses of the
// FrostFS protocol only in resulting structure (see corresponding Res* docs).
// These errors are returned from each protocol operation. By default, statuses
// are resolved and returned as a Go built-in errors.
//
// Deprecated: Use PrmInit.DisableFrostFSErrorResolution instead.
func (x *PrmInit) DisableFrostFSFailuresResolution() {
x.DisableFrostFSErrorResolution = true
}
// SetResponseInfoCallback makes the Client to pass ResponseMetaInfo from each
// NeoFS server response to f. Nil (default) means ignore response meta info.
// FrostFS server response to f. Nil (default) means ignore response meta info.
//
// Deprecated: Use PrmInit.ResponseInfoCallback instead.
func (x *PrmInit) SetResponseInfoCallback(f func(ResponseMetaInfo) error) {
x.cbRespInfo = f
x.ResponseInfoCallback = f
}
const (
defaultDialTimeout = 5 * time.Second
defaultStreamTimeout = 10 * time.Second
)
// PrmDial groups connection parameters for the Client.
//
// See also Dial.
type PrmDial struct {
endpoint string
Endpoint string
tlsConfig *tls.Config
TLSConfig *tls.Config
timeoutDialSet bool
timeoutDial time.Duration
// If DialTimeout is non-positive, then it's set to defaultDialTimeout.
DialTimeout time.Duration
// If StreamTimeout is non-positive, then it's set to defaultStreamTimeout.
StreamTimeout time.Duration
GRPCDialOptions []grpc.DialOption
}
// SetServerURI sets server URI in the NeoFS network.
// SetServerURI sets server URI in the FrostFS network.
// Required parameter.
//
// Format of the URI:
//
// [scheme://]host:port
//
// Supported schemes:
//
// grpc
// grpcs
//
// See also SetTLSConfig.
//
// Deprecated: Use PrmDial.Endpoint instead.
func (x *PrmDial) SetServerURI(endpoint string) {
x.endpoint = endpoint
x.Endpoint = endpoint
}
// SetTLSConfig sets tls.Config to open TLS client connection
// to the NeoFS server. Nil (default) means insecure connection.
// to the FrostFS server. Nil (default) means insecure connection.
//
// See also SetServerURI.
//
// Depreacted: Use PrmDial.TLSConfig instead.
func (x *PrmDial) SetTLSConfig(tlsConfig *tls.Config) {
x.tlsConfig = tlsConfig
x.TLSConfig = tlsConfig
}
// SetTimeout sets the timeout for connection to be established.
// MUST BE positive. If not called, 5s timeout will be used by default.
//
// Deprecated: Use PrmDial.DialTimeout instead.
func (x *PrmDial) SetTimeout(timeout time.Duration) {
x.timeoutDialSet = true
x.timeoutDial = timeout
x.DialTimeout = timeout
}
// SetStreamTimeout sets the timeout for individual operations in streaming RPC.
// MUST BE positive. If not called, 10s timeout will be used by default.
//
// Deprecated: Use PrmDial.StreamTimeout instead.
func (x *PrmDial) SetStreamTimeout(timeout time.Duration) {
x.StreamTimeout = timeout
}
// SetGRPCDialOptions sets the gRPC dial options for new gRPC client connection.
//
// Deprecated: Use PrmDial.GRPCDialOptions instead.
func (x *PrmDial) SetGRPCDialOptions(opts ...grpc.DialOption) {
x.GRPCDialOptions = opts
}

67
client/client_test.go Normal file
View file

@ -0,0 +1,67 @@
package client
import (
"context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"testing"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"github.com/stretchr/testify/require"
)
/*
File contains common functionality used for client package testing.
*/
var key, _ = ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
var statusErr apistatus.ServerInternal
func init() {
statusErr.SetMessage("test status error")
}
func assertStatusErr(tb testing.TB, res interface{ Status() apistatus.Status }) {
require.IsType(tb, &statusErr, res.Status())
require.Equal(tb, statusErr.Message(), res.Status().(*apistatus.ServerInternal).Message())
}
func newClient(server frostFSAPIServer) *Client {
prm := PrmInit{
Key: *key,
}
var c Client
c.Init(prm)
c.setFrostFSAPIServer(server)
return &c
}
func TestClient_DialContext(t *testing.T) {
var c Client
// try to connect to any host
prm := PrmDial{
Endpoint: "localhost:8080",
}
assert := func(ctx context.Context, errExpected error) {
// expect particular context error according to Dial docs
require.ErrorIs(t, c.Dial(ctx, prm), errExpected)
}
// create pre-abandoned context
ctx, cancel := context.WithCancel(context.Background())
cancel()
assert(ctx, context.Canceled)
// create "pre-deadlined" context
ctx, cancel = context.WithTimeout(context.Background(), 0)
defer cancel()
assert(ctx, context.DeadlineExceeded)
}

View file

@ -1,32 +1,22 @@
package client
import (
"crypto/ecdsa"
"errors"
"fmt"
"github.com/nspcc-dev/neofs-api-go/v2/rpc/client"
v2session "github.com/nspcc-dev/neofs-api-go/v2/session"
"github.com/nspcc-dev/neofs-api-go/v2/signature"
apistatus "github.com/nspcc-dev/neofs-sdk-go/client/status"
"github.com/nspcc-dev/neofs-sdk-go/session"
"github.com/nspcc-dev/neofs-sdk-go/version"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/version"
)
// common interface of resulting structures with API status.
type resCommon interface {
setStatus(apistatus.Status)
}
// structure is embedded to all resulting types in order to inherit status-related methods.
type statusRes struct {
st apistatus.Status
}
// setStatus implements resCommon interface method.
func (x *statusRes) setStatus(st apistatus.Status) {
x.st = st
}
// Status returns server's status return.
//
// Use apistatus package functionality to handle the status.
@ -34,287 +24,89 @@ func (x statusRes) Status() apistatus.Status {
return x.st
}
type prmSession struct {
tokenSessionSet bool
tokenSession session.Token
func writeXHeadersToMeta(xHeaders []string, h *v2session.RequestMetaHeader) {
if len(xHeaders) == 0 {
return
}
// SetSessionToken sets token of the session within which request should be sent.
func (x *prmSession) SetSessionToken(tok session.Token) {
x.tokenSession = tok
x.tokenSessionSet = true
}
func (x prmSession) writeToMetaHeader(meta *v2session.RequestMetaHeader) {
if x.tokenSessionSet {
meta.SetSessionToken(x.tokenSession.ToV2())
}
}
// groups meta parameters shared between all Client operations.
type prmCommonMeta struct {
// NeoFS request X-Headers
xHeaders []string
}
// WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length.
//
// Slice must not be mutated until the operation completes.
func (x *prmCommonMeta) WithXHeaders(hs ...string) {
if len(hs)%2 != 0 {
// TODO (aarifullin): remove the panic when all client parameters will check XHeaders
// within buildRequest invocation.
if len(xHeaders)%2 != 0 {
panic("slice of X-Headers with odd length")
}
x.xHeaders = hs
}
func (x prmCommonMeta) writeToMetaHeader(h *v2session.RequestMetaHeader) {
if len(x.xHeaders) > 0 {
hs := make([]v2session.XHeader, len(x.xHeaders)/2)
for i := 0; i < len(x.xHeaders); i += 2 {
hs[i].SetKey(x.xHeaders[i])
hs[i].SetValue(x.xHeaders[i+1])
hs := make([]v2session.XHeader, len(xHeaders)/2)
for i := 0; i < len(xHeaders); i += 2 {
hs[i].SetKey(xHeaders[i])
hs[i].SetValue(xHeaders[i+1])
}
h.SetXHeaders(hs)
}
}
// panic messages.
const (
panicMsgMissingContext = "missing context"
panicMsgMissingContainer = "missing container"
// error messages.
var (
errorMissingContainer = errors.New("missing container")
errorMissingObject = errors.New("missing object")
errorAccountNotSet = errors.New("account not set")
errorServerAddrUnset = errors.New("server address is unset or empty")
errorZeroRangeLength = errors.New("zero range length")
errorMissingRanges = errors.New("missing ranges")
errorInvalidXHeaders = errors.New("xheaders must be presented only as key-value pairs")
)
// groups all the details required to send a single request and process a response to it.
type contextCall struct {
// ==================================================
// state vars that do not require explicit initialization
// final error to be returned from client method
err error
// received response
resp responseV2
// ==================================================
// shared parameters which are set uniformly on all calls
// request signing key
key ecdsa.PrivateKey
// callback prior to processing the response by the client
callbackResp func(ResponseMetaInfo) error
// if set, protocol errors will be expanded into a final error
resolveAPIFailures bool
// NeoFS network magic
netMagic uint64
// Meta parameters
meta prmCommonMeta
// ==================================================
// custom call parameters
// structure of the call result
statusRes resCommon
// request to be signed with a key and sent
req interface {
type request interface {
GetMetaHeader() *v2session.RequestMetaHeader
SetMetaHeader(*v2session.RequestMetaHeader)
SetVerificationHeader(*v2session.RequestVerificationHeader)
}
// function to send a request (unary) and receive a response
call func() (responseV2, error)
// function to send the request (req field)
wReq func() error
// function to recv the response (resp field)
rResp func() error
// function to close the message stream
closer func() error
// function of writing response fields to the resulting structure (optional)
result func(v2 responseV2)
func (c *Client) prepareRequest(req request, meta *v2session.RequestMetaHeader) {
ttl := meta.GetTTL()
if ttl == 0 {
ttl = 2
}
// sets needed fields of the request meta header.
func (x contextCall) prepareRequest() {
meta := x.req.GetMetaHeader()
if meta == nil {
meta = new(v2session.RequestMetaHeader)
x.req.SetMetaHeader(meta)
verV2 := meta.GetVersion()
if verV2 == nil {
verV2 = new(refs.Version)
version.Current().WriteToV2(verV2)
}
if meta.GetTTL() == 0 {
meta.SetTTL(2)
meta.SetTTL(ttl)
meta.SetVersion(verV2)
meta.SetNetworkMagic(c.prm.NetMagic)
req.SetMetaHeader(meta)
}
if meta.GetVersion() == nil {
meta.SetVersion(version.Current().ToV2())
// processResponse verifies response signature and converts status to an error if needed.
func (c *Client) processResponse(resp responseV2) (apistatus.Status, error) {
if c.prm.ResponseInfoCallback != nil {
rmi := ResponseMetaInfo{
key: resp.GetVerificationHeader().GetBodySignature().GetKey(),
epoch: resp.GetMetaHeader().GetEpoch(),
}
meta.SetNetworkMagic(x.netMagic)
x.meta.writeToMetaHeader(meta)
}
// prepares, signs and writes the request. Result means success.
// If failed, contextCall.err contains the reason.
func (x *contextCall) writeRequest() bool {
x.prepareRequest()
x.req.SetVerificationHeader(nil)
// sign the request
x.err = signature.SignServiceMessage(&x.key, x.req)
if x.err != nil {
x.err = fmt.Errorf("sign request: %w", x.err)
return false
}
x.err = x.wReq()
if x.err != nil {
x.err = fmt.Errorf("write request: %w", x.err)
return false
}
return true
}
// performs common actions of response processing and writes any problem as a result status or client error
// (in both cases returns false).
//
// Actions:
// * verify signature (internal);
// * call response callback (internal);
// * unwrap status error (optional).
func (x *contextCall) processResponse() bool {
// call response callback if set
if x.callbackResp != nil {
x.err = x.callbackResp(ResponseMetaInfo{
key: x.resp.GetVerificationHeader().GetBodySignature().GetKey(),
})
if x.err != nil {
x.err = fmt.Errorf("response callback error: %w", x.err)
return false
if err := c.prm.ResponseInfoCallback(rmi); err != nil {
return nil, fmt.Errorf("response callback error: %w", err)
}
}
// note that we call response callback before signature check since it is expected more lightweight
// while verification needs marshaling
// verify response signature
x.err = signature.VerifyServiceMessage(x.resp)
if x.err != nil {
x.err = fmt.Errorf("invalid response signature: %w", x.err)
return false
err := signature.VerifyServiceMessage(resp)
if err != nil {
return nil, fmt.Errorf("invalid response signature: %w", err)
}
// get result status
st := apistatus.FromStatusV2(x.resp.GetMetaHeader().GetStatus())
// unwrap unsuccessful status and return it
// as error if client has been configured so
successfulStatus := apistatus.IsSuccessful(st)
if !successfulStatus && x.resolveAPIFailures {
x.err = apistatus.ErrFromStatus(st)
return false
st := apistatus.FromStatusV2(resp.GetMetaHeader().GetStatus())
if !c.prm.DisableFrostFSErrorResolution {
return st, apistatus.ErrFromStatus(st)
}
return st, nil
}
x.statusRes.setStatus(st)
return successfulStatus || !x.resolveAPIFailures
}
// reads response (if rResp is set) and processes it. Result means success.
// If failed, contextCall.err contains the reason.
func (x *contextCall) readResponse() bool {
if x.rResp != nil {
x.err = x.rResp()
if x.err != nil {
x.err = fmt.Errorf("read response: %w", x.err)
return false
}
}
return x.processResponse()
}
// closes the message stream (if closer is set) and writes the results (if result is set).
// Return means success. If failed, contextCall.err contains the reason.
func (x *contextCall) close() bool {
if x.closer != nil {
x.err = x.closer()
if x.err != nil {
x.err = fmt.Errorf("close RPC: %w", x.err)
return false
}
}
// write response to resulting structure
if x.result != nil {
x.result(x.resp)
}
return true
}
// goes through all stages of sending a request and processing a response. Returns true if successful.
// If failed, contextCall.err contains the reason.
func (x *contextCall) processCall() bool {
// set request writer
x.wReq = func() error {
var err error
x.resp, err = x.call()
return err
}
// write request
ok := x.writeRequest()
if !ok {
return false
}
// read response
ok = x.readResponse()
if !ok {
return false
}
// close and write response to resulting structure
ok = x.close()
if !ok {
return false
}
return x.err == nil
}
// initializes static cross-call parameters inherited from client.
func (c *Client) initCallContext(ctx *contextCall) {
ctx.key = c.prm.key
c.initCallContextWithoutKey(ctx)
}
// initializes static cross-call parameters inherited from client except private key.
func (c *Client) initCallContextWithoutKey(ctx *contextCall) {
ctx.resolveAPIFailures = c.prm.resolveNeoFSErrors
ctx.callbackResp = c.prm.cbRespInfo
ctx.netMagic = c.prm.netMagic
}
// ExecRaw executes f with underlying github.com/nspcc-dev/neofs-api-go/v2/rpc/client.Client
// ExecRaw executes f with underlying git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client.Client
// instance. Communicate over the Protocol Buffers protocol in a more flexible way:
// most often used to transmit data over a fixed version of the NeoFS protocol, as well
// most often used to transmit data over a fixed version of the FrostFS protocol, as well
// as to support custom services.
//
// The f must not manipulate the client connection passed into it.
@ -323,7 +115,7 @@ func (c *Client) initCallContextWithoutKey(ctx *contextCall) {
// before closing the connection.
//
// See also Dial and Close.
// See also github.com/nspcc-dev/neofs-api-go/v2/rpc/client package docs.
// See also git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client package docs.
func (c *Client) ExecRaw(f func(client *client.Client) error) error {
return f(&c.c)
}

View file

@ -2,732 +2,27 @@ package client
import (
"context"
"fmt"
v2container "github.com/nspcc-dev/neofs-api-go/v2/container"
rpcapi "github.com/nspcc-dev/neofs-api-go/v2/rpc"
"github.com/nspcc-dev/neofs-api-go/v2/rpc/client"
v2session "github.com/nspcc-dev/neofs-api-go/v2/session"
v2signature "github.com/nspcc-dev/neofs-api-go/v2/signature"
"github.com/nspcc-dev/neofs-sdk-go/container"
cid "github.com/nspcc-dev/neofs-sdk-go/container/id"
"github.com/nspcc-dev/neofs-sdk-go/eacl"
"github.com/nspcc-dev/neofs-sdk-go/owner"
"github.com/nspcc-dev/neofs-sdk-go/session"
"github.com/nspcc-dev/neofs-sdk-go/signature"
sigutil "github.com/nspcc-dev/neofs-sdk-go/util/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
)
// PrmContainerPut groups parameters of ContainerPut operation.
type PrmContainerPut struct {
prmCommonMeta
cnrSet bool
cnr container.Container
}
// SetContainer sets structured information about new NeoFS container.
// Required parameter.
func (x *PrmContainerPut) SetContainer(cnr container.Container) {
x.cnr = cnr
x.cnrSet = true
}
// ResContainerPut groups resulting values of ContainerPut operation.
type ResContainerPut struct {
statusRes
id *cid.ID
}
// ID returns identifier of the container declared to be stored in the system.
// Used as a link to information about the container (in particular, you can
// asynchronously check if the save was successful).
// SyncContainerWithNetwork requests network configuration using passed client
// and applies it to the container. Container MUST not be nil.
//
// Client doesn't retain value so modification is safe.
func (x ResContainerPut) ID() *cid.ID {
return x.id
}
func (x *ResContainerPut) setID(id *cid.ID) {
x.id = id
}
// ContainerPut sends request to save container in NeoFS.
// Note: if container does not match network configuration, SyncContainerWithNetwork
// changes it.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If WithNeoFSErrorParsing option has been provided, unsuccessful
// NeoFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
// Returns any network/parsing config errors.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see ResContainerPut.ID).
//
// Immediately panics if parameters are set incorrectly (see PrmContainerPut docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerPut(ctx context.Context, prm PrmContainerPut) (*ResContainerPut, error) {
// check parameters
switch {
case ctx == nil:
panic(panicMsgMissingContext)
case !prm.cnrSet:
panic(panicMsgMissingContainer)
}
// TODO: check private key is set before forming the request
// form request body
reqBody := new(v2container.PutRequestBody)
reqBody.SetContainer(prm.cnr.ToV2())
// sign container
signWrapper := v2signature.StableMarshalerWrapper{SM: reqBody.GetContainer()}
sig, err := sigutil.SignData(&c.prm.key, signWrapper, sigutil.SignWithRFC6979())
// See also NetworkInfo, container.ApplyNetworkConfig.
func SyncContainerWithNetwork(ctx context.Context, cnr *container.Container, c *Client) error {
res, err := c.NetworkInfo(ctx, PrmNetworkInfo{})
if err != nil {
return nil, err
return fmt.Errorf("network info call: %w", err)
}
reqBody.SetSignature(sig.ToV2())
container.ApplyNetworkConfig(cnr, res.Info())
// form meta header
var meta v2session.RequestMetaHeader
meta.SetSessionToken(prm.cnr.SessionToken().ToV2())
prm.prmCommonMeta.writeToMetaHeader(&meta)
// form request
var req v2container.PutRequest
req.SetBody(reqBody)
req.SetMetaHeader(&meta)
// init call context
var (
cc contextCall
res ResContainerPut
)
c.initCallContext(&cc)
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.PutContainer(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2container.PutResponse)
res.setID(cid.NewFromV2(resp.GetBody().GetContainerID()))
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmContainerGet groups parameters of ContainerGet operation.
type PrmContainerGet struct {
prmCommonMeta
idSet bool
id cid.ID
}
// SetContainer sets identifier of the container to be read.
// Required parameter.
func (x *PrmContainerGet) SetContainer(id cid.ID) {
x.id = id
x.idSet = true
}
// ResContainerGet groups resulting values of ContainerGet operation.
type ResContainerGet struct {
statusRes
cnr *container.Container
}
// Container returns structured information about the requested container.
//
// Client doesn't retain value so modification is safe.
func (x ResContainerGet) Container() *container.Container {
return x.cnr
}
func (x *ResContainerGet) setContainer(cnr *container.Container) {
x.cnr = cnr
}
// ContainerGet reads NeoFS container by ID.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If WithNeoFSErrorParsing option has been provided, unsuccessful
// NeoFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Immediately panics if parameters are set incorrectly (see PrmContainerGet docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs);
// - *apistatus.ContainerNotFound.
func (c *Client) ContainerGet(ctx context.Context, prm PrmContainerGet) (*ResContainerGet, error) {
switch {
case ctx == nil:
panic(panicMsgMissingContext)
case !prm.idSet:
panic(panicMsgMissingContainer)
}
// form request body
reqBody := new(v2container.GetRequestBody)
reqBody.SetContainerID(prm.id.ToV2())
// form request
var req v2container.GetRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResContainerGet
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.GetContainer(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2container.GetResponse)
body := resp.GetBody()
cnr := container.NewContainerFromV2(body.GetContainer())
cnr.SetSessionToken(
session.NewTokenFromV2(body.GetSessionToken()),
)
cnr.SetSignature(
signature.NewFromV2(body.GetSignature()),
)
res.setContainer(cnr)
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmContainerList groups parameters of ContainerList operation.
type PrmContainerList struct {
prmCommonMeta
ownerSet bool
ownerID owner.ID
}
// SetAccount sets identifier of the NeoFS account to list the containers.
// Required parameter. Must be a valid ID according to NeoFS API protocol.
func (x *PrmContainerList) SetAccount(id owner.ID) {
x.ownerID = id
x.ownerSet = true
}
// ResContainerList groups resulting values of ContainerList operation.
type ResContainerList struct {
statusRes
ids []cid.ID
}
// Containers returns list of identifiers of the account-owned containers.
//
// Client doesn't retain value so modification is safe.
func (x ResContainerList) Containers() []cid.ID {
return x.ids
}
func (x *ResContainerList) setContainers(ids []cid.ID) {
x.ids = ids
}
// ContainerList requests identifiers of the account-owned containers.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If WithNeoFSErrorParsing option has been provided, unsuccessful
// NeoFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Immediately panics if parameters are set incorrectly (see PrmContainerList docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerList(ctx context.Context, prm PrmContainerList) (*ResContainerList, error) {
// check parameters
switch {
case ctx == nil:
panic(panicMsgMissingContext)
case !prm.ownerSet:
panic("account not set")
case !prm.ownerID.Valid():
panic("invalid account")
}
// form request body
reqBody := new(v2container.ListRequestBody)
reqBody.SetOwnerID(prm.ownerID.ToV2())
// form request
var req v2container.ListRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResContainerList
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.ListContainers(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2container.ListResponse)
ids := make([]cid.ID, len(resp.GetBody().GetContainerIDs()))
for i, cidV2 := range resp.GetBody().GetContainerIDs() {
ids[i] = *cid.NewFromV2(&cidV2)
}
res.setContainers(ids)
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmContainerDelete groups parameters of ContainerDelete operation.
type PrmContainerDelete struct {
prmCommonMeta
prmSession
idSet bool
id cid.ID
}
// SetContainer sets identifier of the NeoFS container to be removed.
// Required parameter.
func (x *PrmContainerDelete) SetContainer(id cid.ID) {
x.id = id
x.idSet = true
}
// ResContainerDelete groups resulting values of ContainerDelete operation.
type ResContainerDelete struct {
statusRes
}
// implements github.com/nspcc-dev/neofs-sdk-go/util/signature.DataSource.
type delContainerSignWrapper struct {
body *v2container.DeleteRequestBody
}
func (c delContainerSignWrapper) ReadSignedData([]byte) ([]byte, error) {
return c.body.GetContainerID().GetValue(), nil
}
func (c delContainerSignWrapper) SignedDataSize() int {
return len(c.body.GetContainerID().GetValue())
}
// ContainerDelete sends request to remove the NeoFS container.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If WithNeoFSErrorParsing option has been provided, unsuccessful
// NeoFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see GetContainer).
//
// Immediately panics if parameters are set incorrectly (see PrmContainerDelete docs).
// Context is required and must not be nil. It is used for network communication.
//
// Exactly one return value is non-nil. Server status return is returned in ResContainerDelete.
// Reflects all internal errors in second return value (transport problems, response processing, etc.).
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerDelete(ctx context.Context, prm PrmContainerDelete) (*ResContainerDelete, error) {
// check parameters
switch {
case ctx == nil:
panic(panicMsgMissingContext)
case !prm.idSet:
panic(panicMsgMissingContainer)
}
// form request body
reqBody := new(v2container.DeleteRequestBody)
reqBody.SetContainerID(prm.id.ToV2())
signWrapper := delContainerSignWrapper{body: reqBody}
// sign container
sig, err := sigutil.SignData(&c.prm.key, signWrapper, sigutil.SignWithRFC6979())
if err != nil {
return nil, err
}
reqBody.SetSignature(sig.ToV2())
// form meta header
var meta v2session.RequestMetaHeader
prm.prmSession.writeToMetaHeader(&meta)
prm.prmCommonMeta.writeToMetaHeader(&meta)
// form request
var req v2container.DeleteRequest
req.SetBody(reqBody)
req.SetMetaHeader(&meta)
// init call context
var (
cc contextCall
res ResContainerDelete
)
c.initCallContext(&cc)
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.DeleteContainer(&c.c, &req, client.WithContext(ctx))
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmContainerEACL groups parameters of ContainerEACL operation.
type PrmContainerEACL struct {
prmCommonMeta
idSet bool
id cid.ID
}
// SetContainer sets identifier of the NeoFS container to read the eACL table.
// Required parameter.
func (x *PrmContainerEACL) SetContainer(id cid.ID) {
x.id = id
x.idSet = true
}
// ResContainerEACL groups resulting values of ContainerEACL operation.
type ResContainerEACL struct {
statusRes
table *eacl.Table
}
// Table returns eACL table of the requested container.
//
// Client doesn't retain value so modification is safe.
func (x ResContainerEACL) Table() *eacl.Table {
return x.table
}
func (x *ResContainerEACL) setTable(table *eacl.Table) {
x.table = table
}
// ContainerEACL reads eACL table of the NeoFS container.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If WithNeoFSErrorParsing option has been provided, unsuccessful
// NeoFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Immediately panics if parameters are set incorrectly (see PrmContainerEACL docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs);
// - *apistatus.ContainerNotFound.
func (c *Client) ContainerEACL(ctx context.Context, prm PrmContainerEACL) (*ResContainerEACL, error) {
// check parameters
switch {
case ctx == nil:
panic(panicMsgMissingContext)
case !prm.idSet:
panic(panicMsgMissingContainer)
}
// form request body
reqBody := new(v2container.GetExtendedACLRequestBody)
reqBody.SetContainerID(prm.id.ToV2())
// form request
var req v2container.GetExtendedACLRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResContainerEACL
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.GetEACL(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2container.GetExtendedACLResponse)
body := resp.GetBody()
table := eacl.NewTableFromV2(body.GetEACL())
table.SetSessionToken(
session.NewTokenFromV2(body.GetSessionToken()),
)
table.SetSignature(
signature.NewFromV2(body.GetSignature()),
)
res.setTable(table)
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmContainerSetEACL groups parameters of ContainerSetEACL operation.
type PrmContainerSetEACL struct {
prmCommonMeta
tableSet bool
table eacl.Table
}
// SetTable sets eACL table structure to be set for the container.
// Required parameter.
func (x *PrmContainerSetEACL) SetTable(table eacl.Table) {
x.table = table
x.tableSet = true
}
// ResContainerSetEACL groups resulting values of ContainerSetEACL operation.
type ResContainerSetEACL struct {
statusRes
}
// ContainerSetEACL sends request to update eACL table of the NeoFS container.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If WithNeoFSErrorParsing option has been provided, unsuccessful
// NeoFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see EACL).
//
// Immediately panics if parameters are set incorrectly (see PrmContainerSetEACL docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerSetEACL(ctx context.Context, prm PrmContainerSetEACL) (*ResContainerSetEACL, error) {
// check parameters
switch {
case ctx == nil:
panic(panicMsgMissingContext)
case !prm.tableSet:
panic("eACL table not set")
}
// form request body
reqBody := new(v2container.SetExtendedACLRequestBody)
reqBody.SetEACL(prm.table.ToV2())
// sign the eACL table
signWrapper := v2signature.StableMarshalerWrapper{SM: reqBody.GetEACL()}
sig, err := sigutil.SignData(&c.prm.key, signWrapper, sigutil.SignWithRFC6979())
if err != nil {
return nil, err
}
reqBody.SetSignature(sig.ToV2())
// form meta header
var meta v2session.RequestMetaHeader
meta.SetSessionToken(prm.table.SessionToken().ToV2())
prm.prmCommonMeta.writeToMetaHeader(&meta)
// form request
var req v2container.SetExtendedACLRequest
req.SetBody(reqBody)
req.SetMetaHeader(&meta)
// init call context
var (
cc contextCall
res ResContainerSetEACL
)
c.initCallContext(&cc)
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.SetEACL(&c.c, &req, client.WithContext(ctx))
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmAnnounceSpace groups parameters of ContainerAnnounceUsedSpace operation.
type PrmAnnounceSpace struct {
prmCommonMeta
announcements []container.UsedSpaceAnnouncement
}
// SetValues sets values describing volume of space that is used for the container objects.
// Required parameter. Must not be empty.
//
// Must not be mutated before the end of the operation.
func (x *PrmAnnounceSpace) SetValues(announcements []container.UsedSpaceAnnouncement) {
x.announcements = announcements
}
// ResAnnounceSpace groups resulting values of ContainerAnnounceUsedSpace operation.
type ResAnnounceSpace struct {
statusRes
}
// ContainerAnnounceUsedSpace sends request to announce volume of the space used for the container objects.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If WithNeoFSErrorParsing option has been provided, unsuccessful
// NeoFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// At this moment success can not be checked.
//
// Immediately panics if parameters are set incorrectly (see PrmAnnounceSpace docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerAnnounceUsedSpace(ctx context.Context, prm PrmAnnounceSpace) (*ResAnnounceSpace, error) {
// check parameters
switch {
case ctx == nil:
panic(panicMsgMissingContext)
case len(prm.announcements) == 0:
panic("missing announcements")
}
// convert list of SDK announcement structures into NeoFS-API v2 list
v2announce := make([]v2container.UsedSpaceAnnouncement, len(prm.announcements))
for i := range prm.announcements {
v2announce[i] = *prm.announcements[i].ToV2()
}
// prepare body of the NeoFS-API v2 request and request itself
reqBody := new(v2container.AnnounceUsedSpaceRequestBody)
reqBody.SetAnnouncements(v2announce)
// form request
var req v2container.AnnounceUsedSpaceRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResAnnounceSpace
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.AnnounceUsedSpace(&c.c, &req, client.WithContext(ctx))
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
return nil
}

139
client/container_delete.go Normal file
View file

@ -0,0 +1,139 @@
package client
import (
"context"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmContainerDelete groups parameters of ContainerDelete operation.
type PrmContainerDelete struct {
// FrostFS request X-Headers
XHeaders []string
ContainerID *cid.ID
Session *session.Container
}
// SetContainer sets identifier of the FrostFS container to be removed.
// Required parameter.
//
// Deprecated: Use PrmContainerDelete.Container instead.
func (prm *PrmContainerDelete) SetContainer(id cid.ID) {
prm.ContainerID = &id
}
func (prm *PrmContainerDelete) buildRequest(c *Client) (*v2container.DeleteRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
var cidV2 refs.ContainerID
prm.ContainerID.WriteToV2(&cidV2)
// Container contract expects signature of container ID value,
// don't get confused with stable marshaled protobuf container.ID structure.
data := cidV2.GetValue()
var sig frostfscrypto.Signature
err := sig.Calculate(frostfsecdsa.SignerRFC6979(c.prm.Key), data)
if err != nil {
return nil, fmt.Errorf("calculate signature: %w", err)
}
var sigv2 refs.Signature
sig.WriteToV2(&sigv2)
reqBody := new(v2container.DeleteRequestBody)
reqBody.SetContainerID(&cidV2)
reqBody.SetSignature(&sigv2)
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(prm.XHeaders, &meta)
if prm.Session != nil {
var tokv2 v2session.Token
prm.Session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
var req v2container.DeleteRequest
req.SetBody(reqBody)
c.prepareRequest(&req, &meta)
return &req, nil
}
// WithinSession specifies session within which container should be removed.
//
// Creator of the session acquires the authorship of the request.
// This may affect the execution of an operation (e.g. access control).
//
// Must be signed.
//
// Deprecated: Use PrmContainerDelete.Session instead.
func (prm *PrmContainerDelete) WithinSession(tok session.Container) {
prm.Session = &tok
}
// ResContainerDelete groups resulting values of ContainerDelete operation.
type ResContainerDelete struct {
statusRes
}
// ContainerDelete sends request to remove the FrostFS container.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see GetContainer).
//
// Returns an error if parameters are set incorrectly (see PrmContainerDelete docs).
// Context is required and must not be nil. It is used for network communication.
//
// Exactly one return value is non-nil. Server status return is returned in ResContainerDelete.
// Reflects all internal errors in second return value (transport problems, response processing, etc.).
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerDelete(ctx context.Context, prm PrmContainerDelete) (*ResContainerDelete, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.DeleteContainer(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResContainerDelete
res.st, err = c.processResponse(resp)
return &res, err
}

126
client/container_get.go Normal file
View file

@ -0,0 +1,126 @@
package client
import (
"context"
"errors"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmContainerGet groups parameters of ContainerGet operation.
type PrmContainerGet struct {
// FrostFS request X-Headers.
XHeaders []string
ContainerID *cid.ID
Session *session.Container
}
// SetContainer sets identifier of the container to be read.
// Required parameter.
//
// Deprecated: Use PrmContainerGet.ContainerID instead.
func (prm *PrmContainerGet) SetContainer(cid cid.ID) {
prm.ContainerID = &cid
}
func (prm *PrmContainerGet) buildRequest(c *Client) (*v2container.GetRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
var cidV2 refs.ContainerID
prm.ContainerID.WriteToV2(&cidV2)
reqBody := new(v2container.GetRequestBody)
reqBody.SetContainerID(&cidV2)
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(prm.XHeaders, &meta)
if prm.Session != nil {
var tokv2 v2session.Token
prm.Session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
var req v2container.GetRequest
req.SetBody(reqBody)
c.prepareRequest(&req, &meta)
return &req, nil
}
// ResContainerGet groups resulting values of ContainerGet operation.
type ResContainerGet struct {
statusRes
cnr container.Container
}
// Container returns structured information about the requested container.
//
// Client doesn't retain value so modification is safe.
func (x ResContainerGet) Container() container.Container {
return x.cnr
}
// ContainerGet reads FrostFS container by ID.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Returns an error if parameters are set incorrectly (see PrmContainerGet docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs);
// - *apistatus.ContainerNotFound.
func (c *Client) ContainerGet(ctx context.Context, prm PrmContainerGet) (*ResContainerGet, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.GetContainer(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResContainerGet
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
cnrV2 := resp.GetBody().GetContainer()
if cnrV2 == nil {
return &res, errors.New("missing container in response")
}
if err := res.cnr.ReadFromV2(*cnrV2); err != nil {
return &res, fmt.Errorf("invalid container in response: %w", err)
}
return &res, nil
}

119
client/container_list.go Normal file
View file

@ -0,0 +1,119 @@
package client
import (
"context"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
)
// PrmContainerList groups parameters of ContainerList operation.
type PrmContainerList struct {
XHeaders []string
Account user.ID
Session *session.Container
}
// SetAccount sets identifier of the FrostFS account to list the containers.
// Required parameter.
//
// Deprecated: Use PrmContainerList.Account instead.
func (x *PrmContainerList) SetAccount(id user.ID) {
x.Account = id
}
func (x *PrmContainerList) buildRequest(c *Client) (*v2container.ListRequest, error) {
if x.Account.IsEmpty() {
return nil, errorAccountNotSet
}
var ownerV2 refs.OwnerID
x.Account.WriteToV2(&ownerV2)
reqBody := new(v2container.ListRequestBody)
reqBody.SetOwnerID(&ownerV2)
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(x.XHeaders, &meta)
if x.Session != nil {
var tokv2 v2session.Token
x.Session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
var req v2container.ListRequest
req.SetBody(reqBody)
c.prepareRequest(&req, &meta)
return &req, nil
}
// ResContainerList groups resulting values of ContainerList operation.
type ResContainerList struct {
statusRes
ids []cid.ID
}
// Containers returns list of identifiers of the account-owned containers.
//
// Client doesn't retain value so modification is safe.
func (x ResContainerList) Containers() []cid.ID {
return x.ids
}
// ContainerList requests identifiers of the account-owned containers.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Returns an error if parameters are set incorrectly (see PrmContainerList docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) ContainerList(ctx context.Context, prm PrmContainerList) (*ResContainerList, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.ListContainers(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResContainerList
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
res.ids = make([]cid.ID, len(resp.GetBody().GetContainerIDs()))
for i, cidV2 := range resp.GetBody().GetContainerIDs() {
if err := res.ids[i].ReadFromV2(cidV2); err != nil {
return &res, fmt.Errorf("invalid ID in the response: %w", err)
}
}
return &res, nil
}

160
client/container_put.go Normal file
View file

@ -0,0 +1,160 @@
package client
import (
"context"
"fmt"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmContainerPut groups parameters of ContainerPut operation.
type PrmContainerPut struct {
// FrostFS request X-Headers
XHeaders []string
Container *container.Container
Session *session.Container
}
// SetContainer sets structured information about new FrostFS container.
// Required parameter.
//
// Deprecated: Use PrmContainerPut.Container instead.
func (x *PrmContainerPut) SetContainer(cnr container.Container) {
x.Container = &cnr
}
// WithinSession specifies session within which container should be saved.
//
// Creator of the session acquires the authorship of the request. This affects
// the execution of an operation (e.g. access control).
//
// Session is optional, if set the following requirements apply:
// - session operation MUST be session.VerbContainerPut (ForVerb)
// - token MUST be signed using private key of the owner of the container to be saved
//
// Deprecated: Use PrmContainerPut.Session instead.
func (x *PrmContainerPut) WithinSession(s session.Container) {
x.Session = &s
}
func (x *PrmContainerPut) buildRequest(c *Client) (*v2container.PutRequest, error) {
if x.Container == nil {
return nil, errorMissingContainer
}
if len(x.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
// TODO: check private key is set before forming the request
var cnr v2container.Container
x.Container.WriteToV2(&cnr)
var sig frostfscrypto.Signature
err := container.CalculateSignature(&sig, *x.Container, c.prm.Key)
if err != nil {
return nil, fmt.Errorf("calculate container signature: %w", err)
}
var sigv2 refs.Signature
sig.WriteToV2(&sigv2)
reqBody := new(v2container.PutRequestBody)
reqBody.SetContainer(&cnr)
reqBody.SetSignature(&sigv2)
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(x.XHeaders, &meta)
if x.Session != nil {
var tokv2 v2session.Token
x.Session.WriteToV2(&tokv2)
meta.SetSessionToken(&tokv2)
}
var req v2container.PutRequest
req.SetBody(reqBody)
c.prepareRequest(&req, &meta)
return &req, nil
}
// ResContainerPut groups resulting values of ContainerPut operation.
type ResContainerPut struct {
statusRes
id cid.ID
}
// ID returns identifier of the container declared to be stored in the system.
// Used as a link to information about the container (in particular, you can
// asynchronously check if the save was successful).
func (x ResContainerPut) ID() cid.ID {
return x.id
}
// ContainerPut sends request to save container in FrostFS.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Operation is asynchronous and no guaranteed even in the absence of errors.
// The required time is also not predictable.
//
// Success can be verified by reading by identifier (see ResContainerPut.ID).
//
// Returns an error if parameters are set incorrectly (see PrmContainerPut docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
//
// nolint: funlen
func (c *Client) ContainerPut(ctx context.Context, prm PrmContainerPut) (*ResContainerPut, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.PutContainer(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResContainerPut
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
const fieldCnrID = "container ID"
cidV2 := resp.GetBody().GetContainerID()
if cidV2 == nil {
return &res, newErrMissingResponseField(fieldCnrID)
}
if err := res.id.ReadFromV2(*cidV2); err != nil {
return &res, newErrInvalidResponseField(fieldCnrID, err)
}
return &res, nil
}

View file

@ -1,20 +1,23 @@
/*
Package client provides NeoFS API client implementation.
Package client provides FrostFS API client implementation.
The main component is Client type. It is a virtual connection to the network
and provides methods for executing operations on the server.
Create client instance:
var c client.Client
Initialize client state:
var prm client.PrmInit
prm.SetDefaultPrivateKey(key)
// ...
c.Init(prm)
Connect to the NeoFS server:
Connect to the FrostFS server:
var prm client.PrmDial
prm.SetServerURI("localhost:8080")
prm.SetDefaultPrivateKey(key)
@ -23,7 +26,8 @@ Connect to the NeoFS server:
err := c.Dial(prm)
// ...
Execute NeoFS operation on the server:
Execute FrostFS operation on the server:
var prm client.PrmContainerPut
prm.SetContainer(cnr)
// ...
@ -36,14 +40,15 @@ Execute NeoFS operation on the server:
// ...
Consume custom service of the server:
syntax = "proto3";
service CustomService {
rpc CustomRPC(CustomRPCRequest) returns (CustomRPCResponse);
}
import "github.com/nspcc-dev/neofs-api-go/v2/rpc/client"
import "github.com/nspcc-dev/neofs-api-go/v2/rpc/common"
import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/common"
req := new(CustomRPCRequest)
// ...
@ -58,6 +63,7 @@ Consume custom service of the server:
// ...
Close the connection:
err := c.Close()
// ...
@ -65,9 +71,10 @@ Note that it's not allowed to override Client behaviour directly: the parameters
for the all operations are write-only and the results of the all operations are
read-only. To be able to override client behavior (e.g. for tests), abstract it
with an interface:
import "github.com/nspcc-dev/neofs-sdk-go/client"
type NeoFSClient interface {
import "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client"
type FrostFSClient interface {
// Operations according to the application needs
CreateContainer(context.Context, container.Container) error
// ...
@ -80,6 +87,5 @@ with an interface:
func (x *client) CreateContainer(context.Context, container.Container) error {
// ...
}
*/
package client

View file

@ -1,42 +1,85 @@
package client
import apistatus "github.com/nspcc-dev/neofs-sdk-go/client/status"
import (
"fmt"
// IsErrContainerNotFound checks if err corresponds to NeoFS status
// return corresponding to missing container.
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
)
// wrapsErrType returns true if any error in the error tree of err is of type E.
func wrapsErrType[E error](err error) bool {
switch e := err.(type) {
case E:
return true
case interface{ Unwrap() error }:
return wrapsErrType[E](e.Unwrap())
case interface{ Unwrap() []error }:
for _, ei := range e.Unwrap() {
if wrapsErrType[E](ei) {
return true
}
}
return false
default:
return false
}
}
// IsErrContainerNotFound checks if err corresponds to FrostFS status
// return corresponding to missing container. Supports wrapped errors.
func IsErrContainerNotFound(err error) bool {
switch err.(type) {
default:
return false
case
apistatus.ContainerNotFound,
*apistatus.ContainerNotFound:
return true
}
return wrapsErrType[*apistatus.ContainerNotFound](err)
}
// IsErrObjectNotFound checks if err corresponds to NeoFS status
// return corresponding to missing object.
// IsErrEACLNotFound checks if err corresponds to FrostFS status
// return corresponding to missing eACL table. Supports wrapped errors.
func IsErrEACLNotFound(err error) bool {
return wrapsErrType[*apistatus.EACLNotFound](err)
}
// IsErrObjectNotFound checks if err corresponds to FrostFS status
// return corresponding to missing object. Supports wrapped errors.
func IsErrObjectNotFound(err error) bool {
switch err.(type) {
default:
return false
case
apistatus.ObjectNotFound,
*apistatus.ObjectNotFound:
return true
}
return wrapsErrType[*apistatus.ObjectNotFound](err)
}
// IsErrObjectAlreadyRemoved checks if err corresponds to NeoFS status
// return corresponding to already removed object.
// IsErrObjectAlreadyRemoved checks if err corresponds to FrostFS status
// return corresponding to already removed object. Supports wrapped errors.
func IsErrObjectAlreadyRemoved(err error) bool {
switch err.(type) {
default:
return false
case
apistatus.ObjectAlreadyRemoved,
*apistatus.ObjectAlreadyRemoved:
return true
return wrapsErrType[*apistatus.ObjectAlreadyRemoved](err)
}
// IsErrSessionExpired checks if err corresponds to FrostFS status return
// corresponding to expired session. Supports wrapped errors.
func IsErrSessionExpired(err error) bool {
return wrapsErrType[*apistatus.SessionTokenExpired](err)
}
// IsErrSessionNotFound checks if err corresponds to FrostFS status return
// corresponding to missing session. Supports wrapped errors.
func IsErrSessionNotFound(err error) bool {
return wrapsErrType[*apistatus.SessionTokenNotFound](err)
}
// IsErrAPEManagerAccessDenied checks if err corresponds to FrostFS status return
// corresponding to apemanager access deny. Supports wrapped errors.
func IsErrAPEManagerAccessDenied(err error) bool {
return wrapsErrType[*apistatus.APEManagerAccessDenied](err)
}
// IsErrNodeUnderMaintenance checks if err corresponds to FrostFS status return
// corresponding to nodes being under maintenance. Supports wrapped errors.
func IsErrNodeUnderMaintenance(err error) bool {
return wrapsErrType[*apistatus.NodeUnderMaintenance](err)
}
// returns error describing missing field with the given name.
func newErrMissingResponseField(name string) error {
return fmt.Errorf("missing %s field in the response", name)
}
// returns error describing invalid field (according to the FrostFS protocol)
// with the given name and format violation err.
func newErrInvalidResponseField(name string, err error) error {
return fmt.Errorf("invalid %s field in the response: %w", name, err)
}

54
client/errors_test.go Normal file
View file

@ -0,0 +1,54 @@
package client_test
import (
"fmt"
"testing"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"github.com/stretchr/testify/require"
)
func TestErrors(t *testing.T) {
errs := []struct {
check func(error) bool
err error
}{
{
check: client.IsErrContainerNotFound,
err: new(apistatus.ContainerNotFound),
},
{
check: client.IsErrEACLNotFound,
err: new(apistatus.EACLNotFound),
},
{
check: client.IsErrObjectNotFound,
err: new(apistatus.ObjectNotFound),
},
{
check: client.IsErrObjectAlreadyRemoved,
err: new(apistatus.ObjectAlreadyRemoved),
},
{
check: client.IsErrSessionExpired,
err: new(apistatus.SessionTokenExpired),
},
{
check: client.IsErrSessionNotFound,
err: new(apistatus.SessionTokenNotFound),
},
{
check: client.IsErrNodeUnderMaintenance,
err: new(apistatus.NodeUnderMaintenance),
},
}
for i := range errs {
for j := range errs {
nestedErr := fmt.Errorf("top-level context: :%w", fmt.Errorf("inner context: %w", errs[j].err))
require.Equal(t, i == j, errs[i].check(errs[j].err))
require.Equal(t, i == j, errs[i].check(nestedErr))
}
}
}

View file

@ -2,60 +2,62 @@ package client
import (
"context"
"fmt"
v2netmap "github.com/nspcc-dev/neofs-api-go/v2/netmap"
rpcapi "github.com/nspcc-dev/neofs-api-go/v2/rpc"
"github.com/nspcc-dev/neofs-api-go/v2/rpc/client"
"github.com/nspcc-dev/neofs-sdk-go/netmap"
"github.com/nspcc-dev/neofs-sdk-go/version"
v2netmap "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/version"
)
// PrmEndpointInfo groups parameters of EndpointInfo operation.
type PrmEndpointInfo struct {
prmCommonMeta
XHeaders []string
}
func (x *PrmEndpointInfo) buildRequest(c *Client) (*v2netmap.LocalNodeInfoRequest, error) {
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(x.XHeaders, meta)
req := new(v2netmap.LocalNodeInfoRequest)
req.SetBody(new(v2netmap.LocalNodeInfoRequestBody))
c.prepareRequest(req, meta)
return req, nil
}
// ResEndpointInfo group resulting values of EndpointInfo operation.
type ResEndpointInfo struct {
statusRes
version *version.Version
version version.Version
ni *netmap.NodeInfo
ni netmap.NodeInfo
}
// LatestVersion returns latest NeoFS API protocol's version in use.
//
// Client doesn't retain value so modification is safe.
func (x ResEndpointInfo) LatestVersion() *version.Version {
// LatestVersion returns latest FrostFS API protocol's version in use.
func (x ResEndpointInfo) LatestVersion() version.Version {
return x.version
}
func (x *ResEndpointInfo) setLatestVersion(ver *version.Version) {
x.version = ver
}
// NodeInfo returns information about the NeoFS node served on the remote endpoint.
//
// Client doesn't retain value so modification is safe.
func (x ResEndpointInfo) NodeInfo() *netmap.NodeInfo {
// NodeInfo returns information about the FrostFS node served on the remote endpoint.
func (x ResEndpointInfo) NodeInfo() netmap.NodeInfo {
return x.ni
}
func (x *ResEndpointInfo) setNodeInfo(info *netmap.NodeInfo) {
x.ni = info
}
// EndpointInfo requests information about the storage node served on the remote endpoint.
//
// Method can be used as a health check to see if node is alive and responds to requests.
//
// Any client's internal or transport errors are returned as `error`.
// If WithNeoFSErrorParsing option has been provided, unsuccessful
// NeoFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Immediately panics if parameters are set incorrectly (see PrmEndpointInfo docs).
// Returns an error if parameters are set incorrectly (see PrmEndpointInfo docs).
// Context is required and must not be nil. It is used for network communication.
//
// Exactly one return value is non-nil. Server status return is returned in ResEndpointInfo.
@ -64,76 +66,85 @@ func (x *ResEndpointInfo) setNodeInfo(info *netmap.NodeInfo) {
// Return statuses:
// - global (see Client docs).
func (c *Client) EndpointInfo(ctx context.Context, prm PrmEndpointInfo) (*ResEndpointInfo, error) {
// check context
if ctx == nil {
panic(panicMsgMissingContext)
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
// form request
var req v2netmap.LocalNodeInfoRequest
// init call context
var (
cc contextCall
res ResEndpointInfo
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.LocalNodeInfo(&c.c, &req, client.WithContext(ctx))
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.LocalNodeInfo(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResEndpointInfo
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
cc.result = func(r responseV2) {
resp := r.(*v2netmap.LocalNodeInfoResponse)
body := resp.GetBody()
res.setLatestVersion(version.NewFromV2(body.GetVersion()))
res.setNodeInfo(netmap.NewNodeInfoFromV2(body.GetNodeInfo()))
const fieldVersion = "version"
verV2 := body.GetVersion()
if verV2 == nil {
return nil, newErrMissingResponseField(fieldVersion)
}
if err := res.version.ReadFromV2(*verV2); err != nil {
return nil, newErrInvalidResponseField(fieldVersion, err)
}
// process call
if !cc.processCall() {
return nil, cc.err
}
const fieldNodeInfo = "node info"
nodeInfoV2 := body.GetNodeInfo()
if nodeInfoV2 == nil {
return nil, newErrMissingResponseField(fieldNodeInfo)
}
if err := res.ni.ReadFromV2(*nodeInfoV2); err != nil {
return nil, newErrInvalidResponseField(fieldNodeInfo, err)
}
return &res, nil
}
// PrmNetworkInfo groups parameters of NetworkInfo operation.
type PrmNetworkInfo struct {
prmCommonMeta
XHeaders []string
}
func (x PrmNetworkInfo) buildRequest(c *Client) (*v2netmap.NetworkInfoRequest, error) {
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(x.XHeaders, meta)
var req v2netmap.NetworkInfoRequest
req.SetBody(new(v2netmap.NetworkInfoRequestBody))
c.prepareRequest(&req, meta)
return &req, nil
}
// ResNetworkInfo groups resulting values of NetworkInfo operation.
type ResNetworkInfo struct {
statusRes
info *netmap.NetworkInfo
info netmap.NetworkInfo
}
// Info returns structured information about the NeoFS network.
//
// Client doesn't retain value so modification is safe.
func (x ResNetworkInfo) Info() *netmap.NetworkInfo {
// Info returns structured information about the FrostFS network.
func (x ResNetworkInfo) Info() netmap.NetworkInfo {
return x.info
}
func (x *ResNetworkInfo) setInfo(info *netmap.NetworkInfo) {
x.info = info
}
// NetworkInfo requests information about the NeoFS network of which the remote server is a part.
// NetworkInfo requests information about the FrostFS network of which the remote server is a part.
//
// Any client's internal or transport errors are returned as `error`.
// If WithNeoFSErrorParsing option has been provided, unsuccessful
// NeoFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Immediately panics if parameters are set incorrectly (see PrmNetworkInfo docs).
// Returns an error if parameters are set incorrectly (see PrmNetworkInfo docs).
// Context is required and must not be nil. It is used for network communication.
//
// Exactly one return value is non-nil. Server status return is returned in ResNetworkInfo.
@ -142,37 +153,110 @@ func (x *ResNetworkInfo) setInfo(info *netmap.NetworkInfo) {
// Return statuses:
// - global (see Client docs).
func (c *Client) NetworkInfo(ctx context.Context, prm PrmNetworkInfo) (*ResNetworkInfo, error) {
// check context
if ctx == nil {
panic(panicMsgMissingContext)
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.NetworkInfo(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResNetworkInfo
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
const fieldNetInfo = "network info"
netInfoV2 := resp.GetBody().GetNetworkInfo()
if netInfoV2 == nil {
return nil, newErrMissingResponseField(fieldNetInfo)
}
if err := res.info.ReadFromV2(*netInfoV2); err != nil {
return nil, newErrInvalidResponseField(fieldNetInfo, err)
}
return &res, nil
}
// PrmNetMapSnapshot groups parameters of NetMapSnapshot operation.
type PrmNetMapSnapshot struct{}
// ResNetMapSnapshot groups resulting values of NetMapSnapshot operation.
type ResNetMapSnapshot struct {
statusRes
netMap netmap.NetMap
}
// NetMap returns current server's local network map.
func (x ResNetMapSnapshot) NetMap() netmap.NetMap {
return x.netMap
}
// NetMapSnapshot requests current network view of the remote server.
//
// Any client's internal or transport errors are returned as `error`.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Returns an error if parameters are set incorrectly.
// Context is required and MUST NOT be nil. It is used for network communication.
//
// Exactly one return value is non-nil. Server status return is returned in ResNetMapSnapshot.
// Reflects all internal errors in second return value (transport problems, response processing, etc.).
//
// Return statuses:
// - global (see Client docs).
func (c *Client) NetMapSnapshot(ctx context.Context, _ PrmNetMapSnapshot) (*ResNetMapSnapshot, error) {
// form request body
var body v2netmap.SnapshotRequestBody
// form meta header
var meta v2session.RequestMetaHeader
// form request
var req v2netmap.NetworkInfoRequest
var req v2netmap.SnapshotRequest
req.SetBody(&body)
c.prepareRequest(&req, &meta)
// init call context
var (
cc contextCall
res ResNetworkInfo
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.NetworkInfo(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
resp := r.(*v2netmap.NetworkInfoResponse)
res.setInfo(netmap.NewNetworkInfoFromV2(resp.GetBody().GetNetworkInfo()))
err := signature.SignServiceMessage(&c.prm.Key, &req)
if err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
// process call
if !cc.processCall() {
return nil, cc.err
resp, err := c.server.netMapSnapshot(ctx, req)
if err != nil {
return nil, err
}
var res ResNetMapSnapshot
res.st, err = c.processResponse(resp)
if err != nil {
return nil, err
}
if !apistatus.IsSuccessful(res.st) {
return &res, nil
}
const fieldNetMap = "network map"
netMapV2 := resp.GetBody().NetMap()
if netMapV2 == nil {
return nil, newErrMissingResponseField(fieldNetMap)
}
err = res.netMap.ReadFromV2(*netMapV2)
if err != nil {
return nil, newErrInvalidResponseField(fieldNetMap, err)
}
return &res, nil

132
client/netmap_test.go Normal file
View file

@ -0,0 +1,132 @@
package client
import (
"context"
"errors"
"fmt"
"testing"
v2netmap "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap"
"github.com/stretchr/testify/require"
)
type serverNetMap struct {
errTransport error
signResponse bool
statusOK bool
setNetMap bool
netMap v2netmap.NetMap
}
func (x *serverNetMap) netMapSnapshot(ctx context.Context, req v2netmap.SnapshotRequest) (*v2netmap.SnapshotResponse, error) {
err := signature.VerifyServiceMessage(&req)
if err != nil {
return nil, err
}
if x.errTransport != nil {
return nil, x.errTransport
}
var body v2netmap.SnapshotResponseBody
if x.setNetMap {
body.SetNetMap(&x.netMap)
}
var meta session.ResponseMetaHeader
if !x.statusOK {
meta.SetStatus(statusErr.ToStatusV2())
}
var resp v2netmap.SnapshotResponse
resp.SetBody(&body)
resp.SetMetaHeader(&meta)
if x.signResponse {
err = signature.SignServiceMessage(key, &resp)
if err != nil {
panic(fmt.Sprintf("sign response: %v", err))
}
}
return &resp, nil
}
func TestClient_NetMapSnapshot(t *testing.T) {
var err error
var prm PrmNetMapSnapshot
var res *ResNetMapSnapshot
var srv serverNetMap
c := newClient(&srv)
c.prm.DisableFrostFSFailuresResolution()
ctx := context.Background()
// request signature
srv.errTransport = errors.New("any error")
_, err = c.NetMapSnapshot(ctx, prm)
require.ErrorIs(t, err, srv.errTransport)
srv.errTransport = nil
// unsigned response
_, err = c.NetMapSnapshot(ctx, prm)
require.Error(t, err)
srv.signResponse = true
// status failure
res, err = c.NetMapSnapshot(ctx, prm)
require.NoError(t, err)
assertStatusErr(t, res)
srv.statusOK = true
// missing netmap field
_, err = c.NetMapSnapshot(ctx, prm)
require.Error(t, err)
srv.setNetMap = true
// invalid network map
var netMap netmap.NetMap
var node netmap.NodeInfo
// TODO: #260 use instance corrupter
var nodeV2 v2netmap.NodeInfo
node.WriteToV2(&nodeV2)
require.Error(t, new(netmap.NodeInfo).ReadFromV2(nodeV2))
netMap.SetNodes([]netmap.NodeInfo{node})
netMap.WriteToV2(&srv.netMap)
_, err = c.NetMapSnapshot(ctx, prm)
require.Error(t, err)
// correct network map
// TODO: #260 use instance normalizer
node.SetPublicKey([]byte{1, 2, 3})
node.SetNetworkEndpoints("1", "2", "3")
node.WriteToV2(&nodeV2)
require.NoError(t, new(netmap.NodeInfo).ReadFromV2(nodeV2))
netMap.SetNodes([]netmap.NodeInfo{node})
netMap.WriteToV2(&srv.netMap)
res, err = c.NetMapSnapshot(ctx, prm)
require.NoError(t, err)
require.True(t, apistatus.IsSuccessful(res.Status()))
require.Equal(t, netMap, res.NetMap())
}

View file

@ -3,99 +3,106 @@ package client
import (
"context"
"crypto/ecdsa"
"fmt"
v2object "github.com/nspcc-dev/neofs-api-go/v2/object"
v2refs "github.com/nspcc-dev/neofs-api-go/v2/refs"
rpcapi "github.com/nspcc-dev/neofs-api-go/v2/rpc"
"github.com/nspcc-dev/neofs-api-go/v2/rpc/client"
v2session "github.com/nspcc-dev/neofs-api-go/v2/session"
cid "github.com/nspcc-dev/neofs-sdk-go/container/id"
oid "github.com/nspcc-dev/neofs-sdk-go/object/id"
"github.com/nspcc-dev/neofs-sdk-go/session"
"github.com/nspcc-dev/neofs-sdk-go/token"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
v2refs "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmObjectDelete groups parameters of ObjectDelete operation.
type PrmObjectDelete struct {
meta v2session.RequestMetaHeader
XHeaders []string
body v2object.DeleteRequestBody
BearerToken *bearer.Token
addr v2refs.Address
Session *session.Object
keySet bool
key ecdsa.PrivateKey
}
ContainerID *cid.ID
// WithinSession specifies session within which object should be read.
//
// Creator of the session acquires the authorship of the request.
// This may affect the execution of an operation (e.g. access control).
//
// Must be signed.
func (x *PrmObjectDelete) WithinSession(t session.Token) {
x.meta.SetSessionToken(t.ToV2())
}
ObjectID *oid.ID
// WithBearerToken attaches bearer token to be used for the operation.
//
// If set, underlying eACL rules will be used in access control.
//
// Must be signed.
func (x *PrmObjectDelete) WithBearerToken(t token.BearerToken) {
x.meta.SetBearerToken(t.ToV2())
}
// FromContainer specifies NeoFS container of the object.
// Required parameter.
func (x *PrmObjectDelete) FromContainer(id cid.ID) {
x.addr.SetContainerID(id.ToV2())
}
// ByID specifies identifier of the requested object.
// Required parameter.
func (x *PrmObjectDelete) ByID(id oid.ID) {
x.addr.SetObjectID(id.ToV2())
Key *ecdsa.PrivateKey
}
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
func (x *PrmObjectDelete) UseKey(key ecdsa.PrivateKey) {
x.keySet = true
x.key = key
}
// WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length.
//
// Slice must not be mutated until the operation completes.
func (x *PrmObjectDelete) WithXHeaders(hs ...string) {
if len(hs)%2 != 0 {
panic("slice of X-Headers with odd length")
}
prmCommonMeta{xHeaders: hs}.writeToMetaHeader(&x.meta)
// Deprecated: Use PrmObjectDelete.Key instead.
func (prm *PrmObjectDelete) UseKey(key ecdsa.PrivateKey) {
prm.Key = &key
}
// ResObjectDelete groups resulting values of ObjectDelete operation.
type ResObjectDelete struct {
statusRes
idTomb *v2refs.ObjectID
tomb oid.ID
}
// ReadTombstoneID reads identifier of the created tombstone object.
// Returns false if ID is missing (not read).
func (x ResObjectDelete) ReadTombstoneID(dst *oid.ID) bool {
if x.idTomb != nil {
*dst = *oid.NewIDFromV2(x.idTomb) // need smth better
return true
// Tombstone returns identifier of the created tombstone object.
func (x ResObjectDelete) Tombstone() oid.ID {
return x.tomb
}
return false
func (prm *PrmObjectDelete) buildRequest(c *Client) (*v2object.DeleteRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
// ObjectDelete marks an object for deletion from the container using NeoFS API protocol.
if prm.ObjectID == nil {
return nil, errorMissingObject
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
addr := new(v2refs.Address)
cnrV2 := new(v2refs.ContainerID)
prm.ContainerID.WriteToV2(cnrV2)
addr.SetContainerID(cnrV2)
objV2 := new(v2refs.ObjectID)
prm.ObjectID.WriteToV2(objV2)
addr.SetObjectID(objV2)
body := new(v2object.DeleteRequestBody)
body.SetAddress(addr)
req := new(v2object.DeleteRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// ObjectDelete marks an object for deletion from the container using FrostFS API protocol.
// As a marker, a special unit called a tombstone is placed in the container.
// It confirms the user's intent to delete the object, and is itself a container object.
// Explicit deletion is done asynchronously, and is generally not guaranteed.
@ -105,11 +112,11 @@ func (x ResObjectDelete) ReadTombstoneID(dst *oid.ID) bool {
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`,
// If WithNeoFSErrorParsing option has been provided, unsuccessful
// NeoFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Immediately panics if parameters are set incorrectly (see PrmObjectDelete docs).
// Returns an error if parameters are set incorrectly (see PrmObjectDelete docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
@ -119,48 +126,46 @@ func (x ResObjectDelete) ReadTombstoneID(dst *oid.ID) bool {
// - *apistatus.ObjectLocked;
// - *apistatus.SessionTokenExpired.
func (c *Client) ObjectDelete(ctx context.Context, prm PrmObjectDelete) (*ResObjectDelete, error) {
switch {
case ctx == nil:
panic(panicMsgMissingContext)
case prm.addr.GetContainerID() == nil:
panic(panicMsgMissingContainer)
case prm.addr.GetObjectID() == nil:
panic("missing object")
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
// form request body
prm.body.SetAddress(&prm.addr)
// form request
var req v2object.DeleteRequest
req.SetBody(&prm.body)
req.SetMetaHeader(&prm.meta)
// init call context
var (
cc contextCall
res ResObjectDelete
)
if prm.keySet {
c.initCallContextWithoutKey(&cc)
cc.key = prm.key
} else {
c.initCallContext(&cc)
key := c.prm.Key
if prm.Key != nil {
key = *prm.Key
}
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.DeleteObject(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
res.idTomb = r.(*v2object.DeleteResponse).GetBody().GetTombstone().GetObjectID()
err = signature.SignServiceMessage(&key, req)
if err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
// process call
if !cc.processCall() {
return nil, cc.err
resp, err := rpcapi.DeleteObject(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResObjectDelete
res.st, err = c.processResponse(resp)
if err != nil {
return nil, err
}
if !apistatus.IsSuccessful(res.st) {
return &res, nil
}
const fieldTombstone = "tombstone"
idTombV2 := resp.GetBody().GetTombstone().GetObjectID()
if idTombV2 == nil {
return nil, newErrMissingResponseField(fieldTombstone)
}
err = res.tomb.ReadFromV2(*idTombV2)
if err != nil {
return nil, newErrInvalidResponseField(fieldTombstone, err)
}
return &res, nil

View file

@ -7,104 +7,91 @@ import (
"fmt"
"io"
v2object "github.com/nspcc-dev/neofs-api-go/v2/object"
v2refs "github.com/nspcc-dev/neofs-api-go/v2/refs"
rpcapi "github.com/nspcc-dev/neofs-api-go/v2/rpc"
"github.com/nspcc-dev/neofs-api-go/v2/rpc/client"
v2session "github.com/nspcc-dev/neofs-api-go/v2/session"
apistatus "github.com/nspcc-dev/neofs-sdk-go/client/status"
cid "github.com/nspcc-dev/neofs-sdk-go/container/id"
"github.com/nspcc-dev/neofs-sdk-go/object"
oid "github.com/nspcc-dev/neofs-sdk-go/object/id"
"github.com/nspcc-dev/neofs-sdk-go/session"
"github.com/nspcc-dev/neofs-sdk-go/token"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
v2refs "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// shared parameters of GET/HEAD/RANGE.
type prmObjectRead struct {
prmCommonMeta
raw bool
local bool
sessionSet bool
session session.Token
bearerSet bool
bearer token.BearerToken
cnrSet bool
cnr cid.ID
objSet bool
obj oid.ID
}
func (x prmObjectRead) writeToMetaHeader(h *v2session.RequestMetaHeader) {
if x.local {
h.SetTTL(1)
}
if x.bearerSet {
h.SetBearerToken(x.bearer.ToV2())
}
if x.sessionSet {
h.SetSessionToken(x.session.ToV2())
}
x.prmCommonMeta.writeToMetaHeader(h)
}
// MarkRaw marks an intent to read physically stored object.
func (x *prmObjectRead) MarkRaw() {
x.raw = true
}
// MarkLocal tells the server to execute the operation locally.
func (x *prmObjectRead) MarkLocal() {
x.local = true
}
// WithinSession specifies session within which object should be read.
//
// Creator of the session acquires the authorship of the request.
// This may affect the execution of an operation (e.g. access control).
//
// Must be signed.
func (x *prmObjectRead) WithinSession(t session.Token) {
x.session = t
x.sessionSet = true
}
// WithBearerToken attaches bearer token to be used for the operation.
//
// If set, underlying eACL rules will be used in access control.
//
// Must be signed.
func (x *prmObjectRead) WithBearerToken(t token.BearerToken) {
x.bearer = t
x.bearerSet = true
}
// FromContainer specifies NeoFS container of the object.
// Required parameter.
func (x *prmObjectRead) FromContainer(id cid.ID) {
x.cnr = id
x.cnrSet = true
}
// ByID specifies identifier of the requested object.
// Required parameter.
func (x *prmObjectRead) ByID(id oid.ID) {
x.obj = id
x.objSet = true
}
// PrmObjectGet groups parameters of ObjectGetInit operation.
type PrmObjectGet struct {
prmObjectRead
XHeaders []string
BearerToken *bearer.Token
Session *session.Object
Raw bool
Local bool
ContainerID *cid.ID
ObjectID *oid.ID
Key *ecdsa.PrivateKey
}
func (prm *PrmObjectGet) buildRequest(c *Client) (*v2object.GetRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if prm.ObjectID == nil {
return nil, errorMissingObject
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if prm.Local {
meta.SetTTL(1)
}
addr := new(v2refs.Address)
cnrV2 := new(v2refs.ContainerID)
prm.ContainerID.WriteToV2(cnrV2)
addr.SetContainerID(cnrV2)
objV2 := new(v2refs.ObjectID)
prm.ObjectID.WriteToV2(objV2)
addr.SetObjectID(objV2)
body := new(v2object.GetRequestBody)
body.SetRaw(prm.Raw)
body.SetAddress(addr)
req := new(v2object.GetRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// ResObjectGet groups the final result values of ObjectGetInit operation.
@ -112,17 +99,20 @@ type ResObjectGet struct {
statusRes
}
// ObjectReader is designed to read one object from NeoFS system.
// ObjectReader is designed to read one object from FrostFS system.
//
// Must be initialized using Client.ObjectGetInit, any other
// usage is unsafe.
type ObjectReader struct {
cancelCtxStream context.CancelFunc
ctxCall contextCall
client *Client
stream interface {
Read(resp *v2object.GetResponse) error
}
// initially bound to contextCall
bodyResp v2object.GetResponseBody
res ResObjectGet
err error
tailPayload []byte
@ -131,29 +121,37 @@ type ObjectReader struct {
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
func (x *ObjectReader) UseKey(key ecdsa.PrivateKey) {
x.ctxCall.key = key
}
func handleSplitInfo(ctx *contextCall, i *v2object.SplitInfo) {
ctx.err = object.NewSplitInfoError(object.NewSplitInfoFromV2(i))
//
// Deprecated: Use PrmObjectGet.Key instead.
func (prm *PrmObjectGet) UseKey(key ecdsa.PrivateKey) {
prm.Key = &key
}
// ReadHeader reads header of the object. Result means success.
// Failure reason can be received via Close.
func (x *ObjectReader) ReadHeader(dst *object.Object) bool {
if !x.ctxCall.writeRequest() || !x.ctxCall.readResponse() {
var resp v2object.GetResponse
x.err = x.stream.Read(&resp)
if x.err != nil {
return false
}
x.res.st, x.err = x.client.processResponse(&resp)
if x.err != nil || !apistatus.IsSuccessful(x.res.st) {
return false
}
var partInit *v2object.GetObjectPartInit
switch v := x.bodyResp.GetObjectPart().(type) {
switch v := resp.GetBody().GetObjectPart().(type) {
default:
x.ctxCall.err = fmt.Errorf("unexpected message instead of heading part: %T", v)
x.err = fmt.Errorf("unexpected message instead of heading part: %T", v)
return false
case *v2object.SplitInfo:
handleSplitInfo(&x.ctxCall, v)
x.err = object.NewSplitInfoError(object.NewSplitInfoFromV2(v))
return false
case *v2object.ECInfo:
x.err = object.NewECInfoError(object.NewECInfoFromV2(v))
return false
case *v2object.GetObjectPartInit:
partInit = v
@ -184,26 +182,25 @@ func (x *ObjectReader) readChunk(buf []byte) (int, bool) {
return read, true
}
var ok bool
var part v2object.GetObjectPart
var chunk []byte
var lastRead int
for {
// receive next message
ok = x.ctxCall.readResponse()
if !ok {
var resp v2object.GetResponse
x.err = x.stream.Read(&resp)
if x.err != nil {
return read, false
}
// get chunk part message
part = x.bodyResp.GetObjectPart()
x.res.st, x.err = x.client.processResponse(&resp)
if x.err != nil || !apistatus.IsSuccessful(x.res.st) {
return read, false
}
var partChunk *v2object.GetObjectPartChunk
partChunk, ok = part.(*v2object.GetObjectPartChunk)
part := resp.GetBody().GetObjectPart()
partChunk, ok := part.(*v2object.GetObjectPartChunk)
if !ok {
x.ctxCall.err = fmt.Errorf("unexpected message instead of chunk part: %T", part)
x.err = fmt.Errorf("unexpected message instead of chunk part: %T", part)
return read, false
}
@ -238,9 +235,9 @@ func (x *ObjectReader) ReadChunk(buf []byte) (int, bool) {
func (x *ObjectReader) close(ignoreEOF bool) (*ResObjectGet, error) {
defer x.cancelCtxStream()
if x.ctxCall.err != nil {
if !errors.Is(x.ctxCall.err, io.EOF) {
return nil, x.ctxCall.err
if x.err != nil {
if !errors.Is(x.err, io.EOF) {
return nil, x.err
} else if !ignoreEOF {
if x.remainingPayloadLen > 0 {
return nil, io.ErrUnexpectedEOF
@ -250,7 +247,7 @@ func (x *ObjectReader) close(ignoreEOF bool) (*ResObjectGet, error) {
}
}
return x.ctxCall.statusRes.(*ResObjectGet), nil
return &x.res, nil
}
// Close ends reading the object and returns the result of the operation
@ -258,11 +255,13 @@ func (x *ObjectReader) close(ignoreEOF bool) (*ResObjectGet, error) {
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as Go built-in error.
// If Client is tuned to resolve NeoFS API statuses, then NeoFS failures
// If Client is tuned to resolve FrostFS API statuses, then FrostFS failures
// codes are returned as error.
//
// Return errors:
//
// *object.SplitInfoError (returned on virtual objects with PrmObjectGet.MakeRaw).
// *object.ECInfoError (returned on erasure-coded objects with PrmObjectGet.MakeRaw).
//
// Return statuses:
// - global (see Client docs);
@ -297,92 +296,70 @@ func (x *ObjectReader) Read(p []byte) (int, error) {
return n, nil
}
// ObjectGetInit initiates reading an object through a remote server using NeoFS API protocol.
// ObjectGetInit initiates reading an object through a remote server using FrostFS API protocol.
//
// The call only opens the transmission channel, explicit fetching is done using the ObjectReader.
// Exactly one return value is non-nil. Resulting reader must be finally closed.
//
// Immediately panics if parameters are set incorrectly (see PrmObjectGet docs).
// Returns an error if parameters are set incorrectly (see PrmObjectGet docs).
// Context is required and must not be nil. It is used for network communication.
func (c *Client) ObjectGetInit(ctx context.Context, prm PrmObjectGet) (*ObjectReader, error) {
// check parameters
switch {
case ctx == nil:
panic(panicMsgMissingContext)
case !prm.cnrSet:
panic(panicMsgMissingContainer)
case !prm.objSet:
panic("missing object")
}
var addr v2refs.Address
addr.SetContainerID(prm.cnr.ToV2())
addr.SetObjectID(prm.obj.ToV2())
// form request body
var body v2object.GetRequestBody
body.SetRaw(prm.raw)
body.SetAddress(&addr)
// form meta header
var meta v2session.RequestMetaHeader
prm.prmObjectRead.writeToMetaHeader(&meta)
// form request
var req v2object.GetRequest
req.SetBody(&body)
req.SetMetaHeader(&meta)
// init reader
var (
r ObjectReader
resp v2object.GetResponse
stream *rpcapi.GetResponseReader
)
ctx, r.cancelCtxStream = context.WithCancel(ctx)
resp.SetBody(&r.bodyResp)
// init call context
c.initCallContext(&r.ctxCall)
r.ctxCall.req = &req
r.ctxCall.statusRes = new(ResObjectGet)
r.ctxCall.resp = &resp
r.ctxCall.wReq = func() error {
var err error
stream, err = rpcapi.GetObject(&c.c, &req, client.WithContext(ctx))
req, err := prm.buildRequest(c)
if err != nil {
return fmt.Errorf("open stream: %w", err)
return nil, err
}
return nil
key := prm.Key
if key == nil {
key = &c.prm.Key
}
r.ctxCall.rResp = func() error {
return stream.Read(&resp)
err = signature.SignServiceMessage(key, req)
if err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
ctx, cancel := context.WithCancel(ctx)
stream, err := rpcapi.GetObject(&c.c, req, client.WithContext(ctx))
if err != nil {
cancel()
return nil, fmt.Errorf("open stream: %w", err)
}
var r ObjectReader
r.cancelCtxStream = cancel
r.stream = stream
r.client = c
return &r, nil
}
// PrmObjectHead groups parameters of ObjectHead operation.
type PrmObjectHead struct {
prmObjectRead
XHeaders []string
keySet bool
key ecdsa.PrivateKey
BearerToken *bearer.Token
Session *session.Object
Raw bool
Local bool
ContainerID *cid.ID
ObjectID *oid.ID
Key *ecdsa.PrivateKey
}
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
func (x *PrmObjectHead) UseKey(key ecdsa.PrivateKey) {
x.keySet = true
x.key = key
//
// Deprecated: Use PrmObjectHead.Key instead.
func (prm *PrmObjectHead) UseKey(key ecdsa.PrivateKey) {
prm.Key = &key
}
// ResObjectHead groups resulting values of ObjectHead operation.
@ -408,26 +385,80 @@ func (x *ResObjectHead) ReadHeader(dst *object.Object) bool {
objv2.SetSignature(x.hdr.GetSignature())
obj := object.NewFromV2(&objv2)
obj.SetID(&x.idObj)
obj.SetID(x.idObj)
*dst = *obj
return true
}
// ObjectHead reads object header through a remote server using NeoFS API protocol.
func (prm *PrmObjectHead) buildRequest(c *Client) (*v2object.HeadRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if prm.ObjectID == nil {
return nil, errorMissingObject
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if prm.Local {
meta.SetTTL(1)
}
addr := new(v2refs.Address)
cnrV2 := new(v2refs.ContainerID)
prm.ContainerID.WriteToV2(cnrV2)
addr.SetContainerID(cnrV2)
objV2 := new(v2refs.ObjectID)
prm.ObjectID.WriteToV2(objV2)
addr.SetObjectID(objV2)
body := new(v2object.HeadRequestBody)
body.SetRaw(prm.Raw)
body.SetAddress(addr)
req := new(v2object.HeadRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// ObjectHead reads object header through a remote server using FrostFS API protocol.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`,
// If WithNeoFSErrorParsing option has been provided, unsuccessful
// NeoFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Immediately panics if parameters are set incorrectly (see PrmObjectHead docs).
// Returns an error if parameters are set incorrectly (see PrmObjectHead docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return errors:
//
// *object.SplitInfoError (returned on virtual objects with PrmObjectHead.MakeRaw).
// *object.ECInfoError (returned on erasure-coded objects with PrmObjectHead.MakeRaw).
//
// Return statuses:
// - global (see Client docs);
@ -437,94 +468,145 @@ func (x *ResObjectHead) ReadHeader(dst *object.Object) bool {
// - *apistatus.ObjectAlreadyRemoved;
// - *apistatus.SessionTokenExpired.
func (c *Client) ObjectHead(ctx context.Context, prm PrmObjectHead) (*ResObjectHead, error) {
switch {
case ctx == nil:
panic(panicMsgMissingContext)
case !prm.cnrSet:
panic(panicMsgMissingContainer)
case !prm.objSet:
panic("missing object")
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
var addr v2refs.Address
addr.SetContainerID(prm.cnr.ToV2())
addr.SetObjectID(prm.obj.ToV2())
// form request body
var body v2object.HeadRequestBody
body.SetRaw(prm.raw)
body.SetAddress(&addr)
// form meta header
var meta v2session.RequestMetaHeader
prm.prmObjectRead.writeToMetaHeader(&meta)
// form request
var req v2object.HeadRequest
req.SetBody(&body)
req.SetMetaHeader(&meta)
// init call context
var (
cc contextCall
res ResObjectHead
)
res.idObj = prm.obj
if prm.keySet {
c.initCallContextWithoutKey(&cc)
cc.key = prm.key
} else {
c.initCallContext(&cc)
key := c.prm.Key
if prm.Key != nil {
key = *prm.Key
}
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.HeadObject(&c.c, &req, client.WithContext(ctx))
// sign the request
err = signature.SignServiceMessage(&key, req)
if err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
cc.result = func(r responseV2) {
switch v := r.(*v2object.HeadResponse).GetBody().GetHeaderPart().(type) {
resp, err := rpcapi.HeadObject(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, fmt.Errorf("write request: %w", err)
}
var res ResObjectHead
res.st, err = c.processResponse(resp)
if err != nil {
return nil, err
}
if !apistatus.IsSuccessful(res.st) {
return &res, nil
}
res.idObj = *prm.ObjectID
switch v := resp.GetBody().GetHeaderPart().(type) {
default:
cc.err = fmt.Errorf("unexpected header type %T", v)
return nil, fmt.Errorf("unexpected header type %T", v)
case *v2object.SplitInfo:
handleSplitInfo(&cc, v)
return nil, object.NewSplitInfoError(object.NewSplitInfoFromV2(v))
case *v2object.ECInfo:
return nil, object.NewECInfoError(object.NewECInfoFromV2(v))
case *v2object.HeaderWithSignature:
res.hdr = v
}
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmObjectRange groups parameters of ObjectRange operation.
type PrmObjectRange struct {
prmObjectRead
XHeaders []string
off, ln uint64
BearerToken *bearer.Token
Session *session.Object
Raw bool
Local bool
ContainerID *cid.ID
ObjectID *oid.ID
Key *ecdsa.PrivateKey
Offset uint64
Length uint64
}
// SetOffset sets offset of the payload range to be read.
// Zero by default.
func (x *PrmObjectRange) SetOffset(off uint64) {
x.off = off
func (prm *PrmObjectRange) buildRequest(c *Client) (*v2object.GetRangeRequest, error) {
if prm.Length == 0 {
return nil, errorZeroRangeLength
}
// SetLength sets length of the payload range to be read.
// Must be positive.
func (x *PrmObjectRange) SetLength(ln uint64) {
x.ln = ln
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if prm.ObjectID == nil {
return nil, errorMissingObject
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if prm.Local {
meta.SetTTL(1)
}
addr := new(v2refs.Address)
cnrV2 := new(v2refs.ContainerID)
prm.ContainerID.WriteToV2(cnrV2)
addr.SetContainerID(cnrV2)
objV2 := new(v2refs.ObjectID)
prm.ObjectID.WriteToV2(objV2)
addr.SetObjectID(objV2)
rng := new(v2object.Range)
rng.SetLength(prm.Length)
rng.SetOffset(prm.Offset)
body := new(v2object.GetRangeRequestBody)
body.SetRaw(prm.Raw)
body.SetAddress(addr)
body.SetRange(rng)
req := new(v2object.GetRangeRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
//
// Deprecated: Use PrmObjectRange.Key instead.
func (prm *PrmObjectRange) UseKey(key ecdsa.PrivateKey) {
prm.Key = &key
}
// ResObjectRange groups the final result values of ObjectRange operation.
@ -533,40 +615,28 @@ type ResObjectRange struct {
}
// ObjectRangeReader is designed to read payload range of one object
// from NeoFS system.
// from FrostFS system.
//
// Must be initialized using Client.ObjectRangeInit, any other
// usage is unsafe.
type ObjectRangeReader struct {
cancelCtxStream context.CancelFunc
ctxCall contextCall
client *Client
reqWritten bool
res ResObjectRange
err error
// initially bound to contextCall
bodyResp v2object.GetRangeResponseBody
stream interface {
Read(resp *v2object.GetRangeResponse) error
}
tailPayload []byte
remainingPayloadLen int
}
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
func (x *ObjectRangeReader) UseKey(key ecdsa.PrivateKey) {
x.ctxCall.key = key
}
func (x *ObjectRangeReader) readChunk(buf []byte) (int, bool) {
if !x.reqWritten {
if !x.ctxCall.writeRequest() {
return 0, false
}
x.reqWritten = true
}
var read int
// read remaining tail
@ -578,25 +648,32 @@ func (x *ObjectRangeReader) readChunk(buf []byte) (int, bool) {
return read, true
}
var ok bool
var partChunk *v2object.GetRangePartChunk
var chunk []byte
var lastRead int
for {
// receive next message
ok = x.ctxCall.readResponse()
if !ok {
var resp v2object.GetRangeResponse
x.err = x.stream.Read(&resp)
if x.err != nil {
return read, false
}
x.res.st, x.err = x.client.processResponse(&resp)
if x.err != nil || !apistatus.IsSuccessful(x.res.st) {
return read, false
}
// get chunk message
switch v := x.bodyResp.GetRangePart().(type) {
switch v := resp.GetBody().GetRangePart().(type) {
default:
x.ctxCall.err = fmt.Errorf("unexpected message received: %T", v)
x.err = fmt.Errorf("unexpected message received: %T", v)
return read, false
case *v2object.SplitInfo:
handleSplitInfo(&x.ctxCall, v)
x.err = object.NewSplitInfoError(object.NewSplitInfoFromV2(v))
return read, false
case *v2object.ECInfo:
x.err = object.NewECInfoError(object.NewECInfoFromV2(v))
return read, false
case *v2object.GetRangePartChunk:
partChunk = v
@ -632,9 +709,9 @@ func (x *ObjectRangeReader) ReadChunk(buf []byte) (int, bool) {
func (x *ObjectRangeReader) close(ignoreEOF bool) (*ResObjectRange, error) {
defer x.cancelCtxStream()
if x.ctxCall.err != nil {
if !errors.Is(x.ctxCall.err, io.EOF) {
return nil, x.ctxCall.err
if x.err != nil {
if !errors.Is(x.err, io.EOF) {
return nil, x.err
} else if !ignoreEOF {
if x.remainingPayloadLen > 0 {
return nil, io.ErrUnexpectedEOF
@ -644,7 +721,7 @@ func (x *ObjectRangeReader) close(ignoreEOF bool) (*ResObjectRange, error) {
}
}
return x.ctxCall.statusRes.(*ResObjectRange), nil
return &x.res, nil
}
// Close ends reading the payload range and returns the result of the operation
@ -652,11 +729,13 @@ func (x *ObjectRangeReader) close(ignoreEOF bool) (*ResObjectRange, error) {
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as Go built-in error.
// If Client is tuned to resolve NeoFS API statuses, then NeoFS failures
// If Client is tuned to resolve FrostFS API statuses, then FrostFS failures
// codes are returned as error.
//
// Return errors:
//
// *object.SplitInfoError (returned on virtual objects with PrmObjectRange.MakeRaw).
// *object.ECInfoError (returned on erasure-coded objects with PrmObjectRange.MakeRaw).
//
// Return statuses:
// - global (see Client docs);
@ -664,6 +743,7 @@ func (x *ObjectRangeReader) close(ignoreEOF bool) (*ResObjectRange, error) {
// - *apistatus.ObjectNotFound;
// - *apistatus.ObjectAccessDenied;
// - *apistatus.ObjectAlreadyRemoved;
// - *apistatus.ObjectOutOfRange;
// - *apistatus.SessionTokenExpired.
func (x *ObjectRangeReader) Close() (*ResObjectRange, error) {
return x.close(true)
@ -692,85 +772,42 @@ func (x *ObjectRangeReader) Read(p []byte) (int, error) {
}
// ObjectRangeInit initiates reading an object's payload range through a remote
// server using NeoFS API protocol.
// server using FrostFS API protocol.
//
// The call only opens the transmission channel, explicit fetching is done using the ObjectRangeReader.
// Exactly one return value is non-nil. Resulting reader must be finally closed.
//
// Immediately panics if parameters are set incorrectly (see PrmObjectRange docs).
// Returns an error if parameters are set incorrectly (see PrmObjectRange docs).
// Context is required and must not be nil. It is used for network communication.
func (c *Client) ObjectRangeInit(ctx context.Context, prm PrmObjectRange) (*ObjectRangeReader, error) {
// check parameters
switch {
case ctx == nil:
panic(panicMsgMissingContext)
case !prm.cnrSet:
panic(panicMsgMissingContainer)
case !prm.objSet:
panic("missing object")
case prm.ln == 0:
panic("zero range length")
}
var addr v2refs.Address
addr.SetContainerID(prm.cnr.ToV2())
addr.SetObjectID(prm.obj.ToV2())
var rng v2object.Range
rng.SetOffset(prm.off)
rng.SetLength(prm.ln)
// form request body
var body v2object.GetRangeRequestBody
body.SetRaw(prm.raw)
body.SetAddress(&addr)
body.SetRange(&rng)
// form meta header
var meta v2session.RequestMetaHeader
prm.prmObjectRead.writeToMetaHeader(&meta)
// form request
var req v2object.GetRangeRequest
req.SetBody(&body)
req.SetMetaHeader(&meta)
// init reader
var (
r ObjectRangeReader
resp v2object.GetRangeResponse
stream *rpcapi.ObjectRangeResponseReader
)
r.remainingPayloadLen = int(prm.ln)
ctx, r.cancelCtxStream = context.WithCancel(ctx)
resp.SetBody(&r.bodyResp)
// init call context
c.initCallContext(&r.ctxCall)
r.ctxCall.req = &req
r.ctxCall.statusRes = new(ResObjectRange)
r.ctxCall.resp = &resp
r.ctxCall.wReq = func() error {
var err error
stream, err = rpcapi.GetObjectRange(&c.c, &req, client.WithContext(ctx))
req, err := prm.buildRequest(c)
if err != nil {
return fmt.Errorf("open stream: %w", err)
return nil, err
}
return nil
key := prm.Key
if key == nil {
key = &c.prm.Key
}
r.ctxCall.rResp = func() error {
return stream.Read(&resp)
err = signature.SignServiceMessage(key, req)
if err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
ctx, cancel := context.WithCancel(ctx)
stream, err := rpcapi.GetObjectRange(&c.c, req, client.WithContext(ctx))
if err != nil {
cancel()
return nil, fmt.Errorf("open stream: %w", err)
}
var r ObjectRangeReader
r.remainingPayloadLen = int(prm.Length)
r.cancelCtxStream = cancel
r.stream = stream
r.client = c
return &r, nil
}

View file

@ -2,110 +2,64 @@ package client
import (
"context"
"crypto/ecdsa"
"fmt"
v2object "github.com/nspcc-dev/neofs-api-go/v2/object"
v2refs "github.com/nspcc-dev/neofs-api-go/v2/refs"
rpcapi "github.com/nspcc-dev/neofs-api-go/v2/rpc"
"github.com/nspcc-dev/neofs-api-go/v2/rpc/client"
v2session "github.com/nspcc-dev/neofs-api-go/v2/session"
cid "github.com/nspcc-dev/neofs-sdk-go/container/id"
oid "github.com/nspcc-dev/neofs-sdk-go/object/id"
"github.com/nspcc-dev/neofs-sdk-go/session"
"github.com/nspcc-dev/neofs-sdk-go/token"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
v2refs "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/checksum"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmObjectHash groups parameters of ObjectHash operation.
type PrmObjectHash struct {
meta v2session.RequestMetaHeader
XHeaders []string
body v2object.GetRangeHashRequestBody
BearerToken *bearer.Token
tillichZemor bool
Session *session.Object
addr v2refs.Address
Local bool
Ranges []object.Range
Salt []byte
ChecksumType checksum.Type
ContainerID *cid.ID
ObjectID *oid.ID
Key *ecdsa.PrivateKey
}
// MarkLocal tells the server to execute the operation locally.
func (x *PrmObjectHash) MarkLocal() {
x.meta.SetTTL(1)
}
// WithinSession specifies session within which object should be read.
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
//
// Creator of the session acquires the authorship of the request.
// This may affect the execution of an operation (e.g. access control).
//
// Must be signed.
func (x *PrmObjectHash) WithinSession(t session.Token) {
x.meta.SetSessionToken(t.ToV2())
}
// WithBearerToken attaches bearer token to be used for the operation.
//
// If set, underlying eACL rules will be used in access control.
//
// Must be signed.
func (x *PrmObjectHash) WithBearerToken(t token.BearerToken) {
x.meta.SetBearerToken(t.ToV2())
}
// FromContainer specifies NeoFS container of the object.
// Required parameter.
func (x *PrmObjectHash) FromContainer(id cid.ID) {
x.addr.SetContainerID(id.ToV2())
}
// ByID specifies identifier of the requested object.
// Required parameter.
func (x *PrmObjectHash) ByID(id oid.ID) {
x.addr.SetObjectID(id.ToV2())
}
// SetRangeList sets list of ranges in (offset, length) pair format.
// Required parameter.
//
// If passed as slice, then it must not be mutated before the operation completes.
func (x *PrmObjectHash) SetRangeList(r ...uint64) {
ln := len(r)
if ln%2 != 0 {
panic("odd number of range parameters")
}
rs := make([]v2object.Range, ln/2)
for i := 0; i < ln/2; i++ {
rs[i].SetOffset(r[2*i])
rs[i].SetLength(r[2*i+1])
}
x.body.SetRanges(rs)
// Deprecated: Use PrmObjectHash.Key instead.
func (prm *PrmObjectHash) UseKey(key ecdsa.PrivateKey) {
prm.Key = &key
}
// TillichZemorAlgo changes the hash function to Tillich-Zemor
// (https://link.springer.com/content/pdf/10.1007/3-540-48658-5_5.pdf).
//
// By default, SHA256 hash function is used.
func (x *PrmObjectHash) TillichZemorAlgo() {
x.tillichZemor = true
}
// UseSalt sets the salt to XOR the data range before hashing.
// By default, SHA256 hash function is used/.
//
// Must not be mutated before the operation completes.
func (x *PrmObjectHash) UseSalt(salt []byte) {
x.body.SetSalt(salt)
}
// WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length.
//
// Slice must not be mutated until the operation completes.
func (x *PrmObjectHash) WithXHeaders(hs ...string) {
if len(hs)%2 != 0 {
panic("slice of X-Headers with odd length")
}
prmCommonMeta{xHeaders: hs}.writeToMetaHeader(&x.meta)
// Deprecated: Use PrmObjectHash.ChecksumType instead.
func (prm *PrmObjectHash) TillichZemorAlgo() {
prm.ChecksumType = checksum.TZ
}
// ResObjectHash groups resulting values of ObjectHash operation.
@ -120,19 +74,89 @@ func (x ResObjectHash) Checksums() [][]byte {
return x.checksums
}
func (prm *PrmObjectHash) buildRequest(c *Client) (*v2object.GetRangeHashRequest, error) {
if prm.ContainerID == nil {
return nil, errorMissingContainer
}
if prm.ObjectID == nil {
return nil, errorMissingObject
}
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
if len(prm.Ranges) == 0 {
return nil, errorMissingRanges
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if prm.Local {
meta.SetTTL(1)
}
addr := new(v2refs.Address)
cnrV2 := new(v2refs.ContainerID)
prm.ContainerID.WriteToV2(cnrV2)
addr.SetContainerID(cnrV2)
objV2 := new(v2refs.ObjectID)
prm.ObjectID.WriteToV2(objV2)
addr.SetObjectID(objV2)
rs := make([]v2object.Range, len(prm.Ranges))
for i := range prm.Ranges {
rs[i].SetOffset(prm.Ranges[i].GetOffset())
rs[i].SetLength(prm.Ranges[i].GetLength())
}
body := new(v2object.GetRangeHashRequestBody)
body.SetAddress(addr)
body.SetRanges(rs)
body.SetSalt(prm.Salt)
if prm.ChecksumType == checksum.Unknown {
body.SetType(v2refs.SHA256)
} else {
body.SetType(v2refs.ChecksumType(prm.ChecksumType))
}
req := new(v2object.GetRangeHashRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// ObjectHash requests checksum of the range list of the object payload using
// NeoFS API protocol.
// FrostFS API protocol.
//
// Returns a list of checksums in raw form: the format of hashes and their number
// is left for the caller to check. Client preserves the order of the server's response.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`,
// If WithNeoFSErrorParsing option has been provided, unsuccessful
// NeoFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Immediately panics if parameters are set incorrectly (see PrmObjectHash docs).
// Returns an error if parameters are set incorrectly (see PrmObjectHash docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
@ -140,53 +164,42 @@ func (x ResObjectHash) Checksums() [][]byte {
// - *apistatus.ContainerNotFound;
// - *apistatus.ObjectNotFound;
// - *apistatus.ObjectAccessDenied;
// - *apistatus.ObjectOutOfRange;
// - *apistatus.SessionTokenExpired.
func (c *Client) ObjectHash(ctx context.Context, prm PrmObjectHash) (*ResObjectHash, error) {
switch {
case ctx == nil:
panic(panicMsgMissingContext)
case prm.addr.GetContainerID() == nil:
panic(panicMsgMissingContainer)
case prm.addr.GetObjectID() == nil:
panic("missing object")
case len(prm.body.GetRanges()) == 0:
panic("missing ranges")
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
// form request body
prm.body.SetAddress(&prm.addr)
// ranges and salt are already by prm setters
if prm.tillichZemor {
prm.body.SetType(v2refs.TillichZemor)
} else {
prm.body.SetType(v2refs.SHA256)
key := c.prm.Key
if prm.Key != nil {
key = *prm.Key
}
// form request
var req v2object.GetRangeHashRequest
req.SetBody(&prm.body)
req.SetMetaHeader(&prm.meta)
// init call context
var (
cc contextCall
res ResObjectHash
)
c.initCallContext(&cc)
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.HashObjectRange(&c.c, &req, client.WithContext(ctx))
}
cc.result = func(r responseV2) {
res.checksums = r.(*v2object.GetRangeHashResponse).GetBody().GetHashList()
err = signature.SignServiceMessage(&key, req)
if err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
// process call
if !cc.processCall() {
return nil, cc.err
resp, err := rpcapi.HashObjectRange(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, fmt.Errorf("write request: %w", err)
}
var res ResObjectHash
res.st, err = c.processResponse(resp)
if err != nil {
return nil, err
}
if !apistatus.IsSuccessful(res.st) {
return &res, nil
}
res.checksums = resp.GetBody().GetHashList()
if len(res.checksums) == 0 {
return nil, newErrMissingResponseField("hash list")
}
return &res, nil

270
client/object_patch.go Normal file
View file

@ -0,0 +1,270 @@
package client
import (
"context"
"crypto/ecdsa"
"errors"
"fmt"
"io"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// ObjectPatcher is designed to patch an object.
//
// Must be initialized using Client.ObjectPatchInit, any other
// usage is unsafe.
type ObjectPatcher interface {
// PatchAttributes patches attributes. Attributes can be patched no more than once,
// otherwise, the server returns an error.
//
// Result means success. Failure reason can be received via Close.
PatchAttributes(ctx context.Context, newAttrs []object.Attribute, replace bool) bool
// PatchPayload patches the object's payload.
//
// PatchPayload receives `payloadReader` and thus the payload of the patch is read and sent by chunks of
// `MaxChunkLength` length.
//
// Result means success. Failure reason can be received via Close.
PatchPayload(ctx context.Context, rng *object.Range, payloadReader io.Reader) bool
// Close ends patching the object and returns the result of the operation
// along with the final results. Must be called after using the ObjectPatcher.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as Go built-in error.
// If Client is tuned to resolve FrostFS API statuses, then FrostFS failures
// codes are returned as error.
//
// Return statuses:
// - global (see Client docs);
// - *apistatus.ContainerNotFound;
// - *apistatus.ContainerAccessDenied;
// - *apistatus.ObjectAccessDenied;
// - *apistatus.ObjectAlreadyRemoved;
// - *apistatus.ObjectLocked;
// - *apistatus.ObjectOutOfRange;
// - *apistatus.SessionTokenNotFound;
// - *apistatus.SessionTokenExpired.
Close(_ context.Context) (*ResObjectPatch, error)
}
// ResObjectPatch groups resulting values of ObjectPatch operation.
type ResObjectPatch struct {
statusRes
obj oid.ID
}
// ObjectID returns an object ID of the patched object.
func (r ResObjectPatch) ObjectID() oid.ID {
return r.obj
}
// PrmObjectPatch groups parameters of ObjectPatch operation.
type PrmObjectPatch struct {
XHeaders []string
Address oid.Address
BearerToken *bearer.Token
Session *session.Object
Key *ecdsa.PrivateKey
MaxChunkLength int
}
// ObjectPatchInit initializes object patcher.
func (c *Client) ObjectPatchInit(ctx context.Context, prm PrmObjectPatch) (ObjectPatcher, error) {
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
var objectPatcher objectPatcher
stream, err := rpcapi.Patch(&c.c, &objectPatcher.respV2, client.WithContext(ctx))
if err != nil {
return nil, fmt.Errorf("open stream: %w", err)
}
objectPatcher.addr = prm.Address
objectPatcher.key = &c.prm.Key
if prm.Key != nil {
objectPatcher.key = prm.Key
}
objectPatcher.client = c
objectPatcher.stream = stream
if prm.MaxChunkLength > 0 {
objectPatcher.maxChunkLen = prm.MaxChunkLength
} else {
objectPatcher.maxChunkLen = defaultGRPCPayloadChunkLen
}
objectPatcher.req.SetBody(&v2object.PatchRequestBody{})
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
c.prepareRequest(&objectPatcher.req, meta)
return &objectPatcher, nil
}
type objectPatcher struct {
client *Client
stream interface {
Write(*v2object.PatchRequest) error
Close() error
}
key *ecdsa.PrivateKey
res ResObjectPatch
err error
addr oid.Address
req v2object.PatchRequest
respV2 v2object.PatchResponse
maxChunkLen int
}
func (x *objectPatcher) PatchAttributes(_ context.Context, newAttrs []object.Attribute, replace bool) bool {
return x.patch(&object.Patch{
Address: x.addr,
NewAttributes: newAttrs,
ReplaceAttributes: replace,
})
}
func (x *objectPatcher) PatchPayload(_ context.Context, rng *object.Range, payloadReader io.Reader) bool {
offset := rng.GetOffset()
buf := make([]byte, x.maxChunkLen)
for patchIter := 0; ; patchIter++ {
n, err := payloadReader.Read(buf)
if err != nil && err != io.EOF {
x.err = fmt.Errorf("read payload: %w", err)
return false
}
if n == 0 {
if patchIter == 0 {
if rng.GetLength() == 0 {
x.err = errors.New("zero-length empty payload patch can't be applied")
return false
}
if !x.patch(&object.Patch{
Address: x.addr,
PayloadPatch: &object.PayloadPatch{
Range: rng,
Chunk: []byte{},
},
}) {
return false
}
}
break
}
rngPart := object.NewRange()
if patchIter == 0 {
rngPart.SetOffset(offset)
rngPart.SetLength(rng.GetLength())
} else {
rngPart.SetOffset(offset + rng.GetLength())
}
if !x.patch(&object.Patch{
Address: x.addr,
PayloadPatch: &object.PayloadPatch{
Range: rngPart,
Chunk: buf[:n],
},
}) {
return false
}
if err == io.EOF {
break
}
}
return true
}
func (x *objectPatcher) patch(patch *object.Patch) bool {
x.req.SetBody(patch.ToV2())
x.req.SetVerificationHeader(nil)
x.err = signature.SignServiceMessage(x.key, &x.req)
if x.err != nil {
x.err = fmt.Errorf("sign message: %w", x.err)
return false
}
x.err = x.stream.Write(&x.req)
return x.err == nil
}
func (x *objectPatcher) Close(_ context.Context) (*ResObjectPatch, error) {
// Ignore io.EOF error, because it is expected error for client-side
// stream termination by the server. E.g. when stream contains invalid
// message. Server returns an error in response message (in status).
if x.err != nil && !errors.Is(x.err, io.EOF) {
return nil, x.err
}
if x.err = x.stream.Close(); x.err != nil {
return nil, x.err
}
x.res.st, x.err = x.client.processResponse(&x.respV2)
if x.err != nil {
return nil, x.err
}
if !apistatus.IsSuccessful(x.res.st) {
return &x.res, nil
}
const fieldID = "ID"
idV2 := x.respV2.Body.ObjectID
if idV2 == nil {
return nil, newErrMissingResponseField(fieldID)
}
x.err = x.res.obj.ReadFromV2(*idV2)
if x.err != nil {
x.err = newErrInvalidResponseField(fieldID, x.err)
}
return &x.res, nil
}

295
client/object_patch_test.go Normal file
View file

@ -0,0 +1,295 @@
package client
import (
"bytes"
"context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"testing"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id/test"
"github.com/stretchr/testify/require"
)
type mockPatchStream struct {
streamedPayloadPatches []*object.PayloadPatch
}
func (m *mockPatchStream) Write(r *v2object.PatchRequest) error {
pp := new(object.PayloadPatch)
pp.FromV2(r.GetBody().GetPatch())
if r.GetBody().GetPatch() != nil {
bodyChunk := r.GetBody().GetPatch().Chunk
pp.Chunk = make([]byte, len(bodyChunk))
copy(pp.Chunk, bodyChunk)
}
m.streamedPayloadPatches = append(m.streamedPayloadPatches, pp)
return nil
}
func (m *mockPatchStream) Close() error {
return nil
}
func TestObjectPatcher(t *testing.T) {
type part struct {
offset int
length int
chunk string
}
for _, test := range []struct {
name string
patchPayload string
rng *object.Range
maxChunkLen int
expectParts []part
}{
{
name: "no split payload patch",
patchPayload: "011111",
rng: newRange(0, 6),
maxChunkLen: defaultGRPCPayloadChunkLen,
expectParts: []part{
{
offset: 0,
length: 6,
chunk: "011111",
},
},
},
{
name: "splitted payload patch",
patchPayload: "012345",
rng: newRange(0, 6),
maxChunkLen: 2,
expectParts: []part{
{
offset: 0,
length: 6,
chunk: "01",
},
{
offset: 6,
length: 0,
chunk: "23",
},
{
offset: 6,
length: 0,
chunk: "45",
},
},
},
{
name: "splitted payload patch with zero-length subpatches",
patchPayload: "0123456789!@",
rng: newRange(0, 4),
maxChunkLen: 2,
expectParts: []part{
{
offset: 0,
length: 4,
chunk: "01",
},
{
offset: 4,
length: 0,
chunk: "23",
},
{
offset: 4,
length: 0,
chunk: "45",
},
{
offset: 4,
length: 0,
chunk: "67",
},
{
offset: 4,
length: 0,
chunk: "89",
},
{
offset: 4,
length: 0,
chunk: "!@",
},
},
},
{
name: "splitted payload patch with zero-length subpatches only",
patchPayload: "0123456789!@",
rng: newRange(0, 0),
maxChunkLen: 2,
expectParts: []part{
{
offset: 0,
length: 0,
chunk: "01",
},
{
offset: 0,
length: 0,
chunk: "23",
},
{
offset: 0,
length: 0,
chunk: "45",
},
{
offset: 0,
length: 0,
chunk: "67",
},
{
offset: 0,
length: 0,
chunk: "89",
},
{
offset: 0,
length: 0,
chunk: "!@",
},
},
},
} {
t.Run(test.name, func(t *testing.T) {
m := &mockPatchStream{}
pk, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
patcher := objectPatcher{
client: &Client{},
stream: m,
addr: oidtest.Address(),
key: pk,
maxChunkLen: test.maxChunkLen,
}
success := patcher.PatchAttributes(context.Background(), nil, false)
require.True(t, success)
success = patcher.PatchPayload(context.Background(), test.rng, bytes.NewReader([]byte(test.patchPayload)))
require.True(t, success)
require.Len(t, m.streamedPayloadPatches, len(test.expectParts)+1)
// m.streamedPayloadPatches[0] is attribute patch, so skip it
for i, part := range test.expectParts {
requireRangeChunk(t, m.streamedPayloadPatches[i+1], part.offset, part.length, part.chunk)
}
})
}
}
func TestRepeatPayloadPatch(t *testing.T) {
t.Run("no payload patch partioning", func(t *testing.T) {
m := &mockPatchStream{}
pk, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
const maxChunkLen = 20
patcher := objectPatcher{
client: &Client{},
stream: m,
addr: oidtest.Address(),
key: pk,
maxChunkLen: maxChunkLen,
}
for _, pp := range []struct {
patchPayload string
rng *object.Range
}{
{
patchPayload: "xxxxxxxxxx",
rng: newRange(1, 6),
},
{
patchPayload: "yyyyyyyyyy",
rng: newRange(5, 9),
},
{
patchPayload: "zzzzzzzzzz",
rng: newRange(10, 0),
},
} {
success := patcher.PatchPayload(context.Background(), pp.rng, bytes.NewReader([]byte(pp.patchPayload)))
require.True(t, success)
}
requireRangeChunk(t, m.streamedPayloadPatches[0], 1, 6, "xxxxxxxxxx")
requireRangeChunk(t, m.streamedPayloadPatches[1], 5, 9, "yyyyyyyyyy")
requireRangeChunk(t, m.streamedPayloadPatches[2], 10, 0, "zzzzzzzzzz")
})
t.Run("payload patch partioning", func(t *testing.T) {
m := &mockPatchStream{}
pk, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
const maxChunkLen = 5
patcher := objectPatcher{
client: &Client{},
stream: m,
addr: oidtest.Address(),
key: pk,
maxChunkLen: maxChunkLen,
}
for _, pp := range []struct {
patchPayload string
rng *object.Range
}{
{
patchPayload: "xxxxxxxxxx",
rng: newRange(1, 6),
},
{
patchPayload: "yyyyyyyyyy",
rng: newRange(5, 9),
},
{
patchPayload: "zzzzzzzzzz",
rng: newRange(10, 0),
},
} {
success := patcher.PatchPayload(context.Background(), pp.rng, bytes.NewReader([]byte(pp.patchPayload)))
require.True(t, success)
}
requireRangeChunk(t, m.streamedPayloadPatches[0], 1, 6, "xxxxx")
requireRangeChunk(t, m.streamedPayloadPatches[1], 7, 0, "xxxxx")
requireRangeChunk(t, m.streamedPayloadPatches[2], 5, 9, "yyyyy")
requireRangeChunk(t, m.streamedPayloadPatches[3], 14, 0, "yyyyy")
requireRangeChunk(t, m.streamedPayloadPatches[4], 10, 0, "zzzzz")
requireRangeChunk(t, m.streamedPayloadPatches[5], 10, 0, "zzzzz")
})
}
func requireRangeChunk(t *testing.T, pp *object.PayloadPatch, offset, length int, chunk string) {
require.NotNil(t, pp)
require.Equal(t, uint64(offset), pp.Range.GetOffset())
require.Equal(t, uint64(length), pp.Range.GetLength())
require.Equal(t, []byte(chunk), pp.Chunk)
}
func newRange(offest, length uint64) *object.Range {
rng := &object.Range{}
rng.SetOffset(offest)
rng.SetLength(length)
return rng
}

View file

@ -3,158 +3,107 @@ package client
import (
"context"
"crypto/ecdsa"
"fmt"
v2object "github.com/nspcc-dev/neofs-api-go/v2/object"
rpcapi "github.com/nspcc-dev/neofs-api-go/v2/rpc"
"github.com/nspcc-dev/neofs-api-go/v2/rpc/client"
v2session "github.com/nspcc-dev/neofs-api-go/v2/session"
"github.com/nspcc-dev/neofs-sdk-go/object"
oid "github.com/nspcc-dev/neofs-sdk-go/object/id"
"github.com/nspcc-dev/neofs-sdk-go/session"
"github.com/nspcc-dev/neofs-sdk-go/token"
buffPool "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/util/pool"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// defaultGRPCPayloadChunkLen default value for maxChunkLen.
// See PrmObjectPutInit.SetGRPCPayloadChunkLen for details.
const defaultGRPCPayloadChunkLen = 3 << 20
// PrmObjectPutInit groups parameters of ObjectPutInit operation.
type PrmObjectPutInit struct {
XHeaders []string
BearerToken *bearer.Token
Session *session.Object
Local bool
CopiesNumber []uint32
MaxChunkLength int
MaxSize uint64
EpochSource transformer.EpochSource
WithoutHomomorphHash bool
Key *ecdsa.PrivateKey
Pool *buffPool.BufferPool
}
// SetCopiesNumber sets number of object copies that is enough to consider put successful.
//
// At the moment the operation is not parameterized, however,
// the structure is still declared for backward compatibility.
type PrmObjectPutInit struct{}
// Deprecated: Use PrmObjectPutInit.CopiesNumber instead.
func (x *PrmObjectPutInit) SetCopiesNumber(copiesNumber uint32) {
x.CopiesNumber = []uint32{copiesNumber}
}
// SetCopiesNumberByVectors sets ordered list of minimal required object copies numbers
// per placement vector. List's length MUST equal container's placement vector number,
// otherwise request will fail.
//
// Deprecated: Use PrmObjectPutInit.CopiesNumber instead.
func (x *PrmObjectPutInit) SetCopiesNumberByVectors(copiesNumbers []uint32) {
x.CopiesNumber = copiesNumbers
}
// SetGRPCPayloadChunkLen sets maximum chunk length value for gRPC Put request.
// Maximum chunk length restricts maximum byte length of the chunk
// transmitted in a single stream message. It depends on
// server settings and other message fields.
// If not specified or negative value set, default value of 3MiB will be used.
//
// Deprecated: Use PrmObjectPutInit.MaxChunkLength instead.
func (x *PrmObjectPutInit) SetGRPCPayloadChunkLen(v int) {
x.MaxChunkLength = v
}
// ResObjectPut groups the final result values of ObjectPutInit operation.
type ResObjectPut struct {
statusRes
resp v2object.PutResponse
obj oid.ID
epoch uint64
}
// ReadStoredObjectID reads identifier of the saved object.
// Returns false if ID is missing (not read).
func (x *ResObjectPut) ReadStoredObjectID(id *oid.ID) bool {
idv2 := x.resp.GetBody().GetObjectID()
if idv2 == nil {
return false
// StoredObjectID returns identifier of the saved object.
func (x ResObjectPut) StoredObjectID() oid.ID {
return x.obj
}
*id = *oid.NewIDFromV2(idv2) // need smth better
return true
// StoredEpoch returns creation epoch of the saved object.
func (x ResObjectPut) StoredEpoch() uint64 {
return x.epoch
}
// ObjectWriter is designed to write one object to NeoFS system.
// ObjectWriter is designed to write one object or
// multiple parts of one object to FrostFS system.
//
// Must be initialized using Client.ObjectPutInit, any other
// usage is unsafe.
type ObjectWriter struct {
cancelCtxStream context.CancelFunc
ctxCall contextCall
// initially bound tp contextCall
metaHdr v2session.RequestMetaHeader
// initially bound to contextCall
partInit v2object.PutObjectPartInit
chunkCalled bool
partChunk v2object.PutObjectPartChunk
}
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
func (x *ObjectWriter) UseKey(key ecdsa.PrivateKey) {
x.ctxCall.key = key
}
// WithBearerToken attaches bearer token to be used for the operation.
// Should be called once before any writing steps.
func (x *ObjectWriter) WithBearerToken(t token.BearerToken) {
x.metaHdr.SetBearerToken(t.ToV2())
}
// WithinSession specifies session within which object should be stored.
// Should be called once before any writing steps.
func (x *ObjectWriter) WithinSession(t session.Token) {
x.metaHdr.SetSessionToken(t.ToV2())
}
// MarkLocal tells the server to execute the operation locally.
func (x *ObjectWriter) MarkLocal() {
x.metaHdr.SetTTL(1)
}
// WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length.
//
// Slice must not be mutated until the operation completes.
func (x *ObjectWriter) WithXHeaders(hs ...string) {
if len(hs)%2 != 0 {
panic("slice of X-Headers with odd length")
}
prmCommonMeta{xHeaders: hs}.writeToMetaHeader(&x.metaHdr)
}
type ObjectWriter interface {
// WriteHeader writes header of the object. Result means success.
// Failure reason can be received via Close.
func (x *ObjectWriter) WriteHeader(hdr object.Object) bool {
v2Hdr := hdr.ToV2()
x.partInit.SetObjectID(v2Hdr.GetObjectID())
x.partInit.SetHeader(v2Hdr.GetHeader())
x.partInit.SetSignature(v2Hdr.GetSignature())
return x.ctxCall.writeRequest()
}
WriteHeader(context.Context, object.Object) bool
// WritePayloadChunk writes chunk of the object payload. Result means success.
// Failure reason can be received via Close.
func (x *ObjectWriter) WritePayloadChunk(chunk []byte) bool {
if !x.chunkCalled {
x.chunkCalled = true
x.ctxCall.req.(*v2object.PutRequest).GetBody().SetObjectPart(&x.partChunk)
}
for ln := len(chunk); ln > 0; ln = len(chunk) {
// maxChunkLen restricts maximum byte length of the chunk
// transmitted in a single stream message. It depends on
// server settings and other message fields, but for now
// we simply assume that 3MB is large enough to reduce the
// number of messages, and not to exceed the limit
// (4MB by default for gRPC servers).
const maxChunkLen = 3 << 20
if ln > maxChunkLen {
ln = maxChunkLen
}
// we deal with size limit overflow above, but there is another case:
// what if method is called with "small" chunk many times? We write
// a message to the stream on each call. Alternatively, we could use buffering.
// In most cases, the chunk length does not vary between calls. Given this
// assumption, as well as the length of the payload from the header, it is
// possible to buffer the data of intermediate chunks, and send a message when
// the allocated buffer is filled, or when the last chunk is received.
// It is mentally assumed that allocating and filling the buffer is better than
// synchronous sending, but this needs to be tested.
x.partChunk.SetChunk(chunk[:ln])
if !x.ctxCall.writeRequest() {
return false
}
chunk = chunk[ln:]
}
return true
}
WritePayloadChunk(context.Context, []byte) bool
// Close ends writing the object and returns the result of the operation
// along with the final results. Must be called after using the ObjectWriter.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as Go built-in error.
// If Client is tuned to resolve NeoFS API statuses, then NeoFS failures
// If Client is tuned to resolve FrostFS API statuses, then FrostFS failures
// codes are returned as error.
//
// Return statuses:
@ -165,69 +114,83 @@ func (x *ObjectWriter) WritePayloadChunk(chunk []byte) bool {
// - *apistatus.LockNonRegularObject;
// - *apistatus.SessionTokenNotFound;
// - *apistatus.SessionTokenExpired.
func (x *ObjectWriter) Close() (*ResObjectPut, error) {
defer x.cancelCtxStream()
if x.ctxCall.err != nil {
return nil, x.ctxCall.err
Close(context.Context) (*ResObjectPut, error)
}
if !x.ctxCall.close() {
return nil, x.ctxCall.err
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
//
// Deprecated: Use PrmObjectPutInit.Key instead.
func (x *PrmObjectPutInit) UseKey(key ecdsa.PrivateKey) {
x.Key = &key
}
if !x.ctxCall.processResponse() {
return nil, x.ctxCall.err
// WithBearerToken attaches bearer token to be used for the operation.
// Should be called once before any writing steps.
//
// Deprecated: Use PrmObjectPutInit.BearerToken instead.
func (x *PrmObjectPutInit) WithBearerToken(t bearer.Token) {
x.BearerToken = &t
}
return x.ctxCall.statusRes.(*ResObjectPut), nil
// WithinSession specifies session within which object should be stored.
// Should be called once before any writing steps.
//
// Deprecated: Use PrmObjectPutInit.Session instead.
func (x *PrmObjectPutInit) WithinSession(t session.Object) {
x.Session = &t
}
// ObjectPutInit initiates writing an object through a remote server using NeoFS API protocol.
// MarkLocal tells the server to execute the operation locally.
//
// Deprecated: Use PrmObjectPutInit.Local instead.
func (x *PrmObjectPutInit) MarkLocal() {
x.Local = true
}
// WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length.
//
// Slice must not be mutated until the operation completes.
//
// Deprecated: Use PrmObjectPutInit.XHeaders instead.
func (x *PrmObjectPutInit) WithXHeaders(hs ...string) {
x.XHeaders = hs
}
// WithObjectMaxSize specifies max object size value and use it during object splitting.
// When specified, start writing to the stream only after the object is formed.
// Continue processing the input only when the previous formed object has been successfully written.
//
// Deprecated: Use PrmObjectPutInit.MaxSize instead.
func (x *PrmObjectPutInit) WithObjectMaxSize(maxSize uint64) {
x.MaxSize = maxSize
}
// WithoutHomomorphicHash if set to true do not use Tillich-Zémor hash for payload.
//
// Deprecated: Use PrmObjectPutInit.WithoutHomomorphHash instead.
func (x *PrmObjectPutInit) WithoutHomomorphicHash(v bool) {
x.WithoutHomomorphHash = v
}
// WithEpochSource specifies epoch for object when split it on client side.
//
// Deprecated: Use PrmObjectPutInit.EpochSource instead.
func (x *PrmObjectPutInit) WithEpochSource(es transformer.EpochSource) {
x.EpochSource = es
}
// ObjectPutInit initiates writing an object through a remote server using FrostFS API protocol.
//
// The call only opens the transmission channel, explicit recording is done using the ObjectWriter.
// Exactly one return value is non-nil. Resulting writer must be finally closed.
//
// Returns an error if parameters are set incorrectly.
// Context is required and must not be nil. It is used for network communication.
func (c *Client) ObjectPutInit(ctx context.Context, _ PrmObjectPutInit) (*ObjectWriter, error) {
// check parameters
if ctx == nil {
panic(panicMsgMissingContext)
func (c *Client) ObjectPutInit(ctx context.Context, prm PrmObjectPutInit) (ObjectWriter, error) {
if prm.MaxSize > 0 {
return c.objectPutInitTransformer(prm)
}
// open stream
var (
res ResObjectPut
w ObjectWriter
)
ctx, w.cancelCtxStream = context.WithCancel(ctx)
stream, err := rpcapi.PutObject(&c.c, &res.resp, client.WithContext(ctx))
if err != nil {
return nil, fmt.Errorf("open stream: %w", err)
}
// form request body
var body v2object.PutRequestBody
// form request
var req v2object.PutRequest
req.SetBody(&body)
req.SetMetaHeader(&w.metaHdr)
body.SetObjectPart(&w.partInit)
// init call context
c.initCallContext(&w.ctxCall)
w.ctxCall.req = &req
w.ctxCall.statusRes = &res
w.ctxCall.resp = &res.resp
w.ctxCall.wReq = func() error {
return stream.Write(&req)
}
w.ctxCall.closer = stream.Close
return &w, nil
return c.objectPutInitRaw(ctx, prm)
}

181
client/object_put_raw.go Normal file
View file

@ -0,0 +1,181 @@
package client
import (
"context"
"crypto/ecdsa"
"errors"
"fmt"
"io"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
)
func (c *Client) objectPutInitRaw(ctx context.Context, prm PrmObjectPutInit) (*objectWriterRaw, error) {
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
var w objectWriterRaw
stream, err := rpcapi.PutObject(&c.c, &w.respV2, client.WithContext(ctx))
if err != nil {
return nil, fmt.Errorf("open stream: %w", err)
}
w.key = &c.prm.Key
if prm.Key != nil {
w.key = prm.Key
}
w.client = c
w.stream = stream
w.partInit.SetCopiesNumber(prm.CopiesNumber)
w.req.SetBody(new(v2object.PutRequestBody))
if prm.MaxChunkLength > 0 {
w.maxChunkLen = prm.MaxChunkLength
} else {
w.maxChunkLen = defaultGRPCPayloadChunkLen
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if prm.Local {
meta.SetTTL(1)
}
c.prepareRequest(&w.req, meta)
return &w, nil
}
type objectWriterRaw struct {
client *Client
stream interface {
Write(*v2object.PutRequest) error
Close() error
}
key *ecdsa.PrivateKey
res ResObjectPut
err error
chunkCalled bool
respV2 v2object.PutResponse
req v2object.PutRequest
partInit v2object.PutObjectPartInit
partChunk v2object.PutObjectPartChunk
maxChunkLen int
}
func (x *objectWriterRaw) WriteHeader(_ context.Context, hdr object.Object) bool {
v2Hdr := hdr.ToV2()
x.partInit.SetObjectID(v2Hdr.GetObjectID())
x.partInit.SetHeader(v2Hdr.GetHeader())
x.partInit.SetSignature(v2Hdr.GetSignature())
x.req.GetBody().SetObjectPart(&x.partInit)
x.req.SetVerificationHeader(nil)
x.err = signature.SignServiceMessage(x.key, &x.req)
if x.err != nil {
x.err = fmt.Errorf("sign message: %w", x.err)
return false
}
x.err = x.stream.Write(&x.req)
return x.err == nil
}
func (x *objectWriterRaw) WritePayloadChunk(_ context.Context, chunk []byte) bool {
if !x.chunkCalled {
x.chunkCalled = true
x.req.GetBody().SetObjectPart(&x.partChunk)
}
for ln := len(chunk); ln > 0; ln = len(chunk) {
if ln > x.maxChunkLen {
ln = x.maxChunkLen
}
// we deal with size limit overflow above, but there is another case:
// what if method is called with "small" chunk many times? We write
// a message to the stream on each call. Alternatively, we could use buffering.
// In most cases, the chunk length does not vary between calls. Given this
// assumption, as well as the length of the payload from the header, it is
// possible to buffer the data of intermediate chunks, and send a message when
// the allocated buffer is filled, or when the last chunk is received.
// It is mentally assumed that allocating and filling the buffer is better than
// synchronous sending, but this needs to be tested.
x.partChunk.SetChunk(chunk[:ln])
x.req.SetVerificationHeader(nil)
x.err = signature.SignServiceMessage(x.key, &x.req)
if x.err != nil {
x.err = fmt.Errorf("sign message: %w", x.err)
return false
}
x.err = x.stream.Write(&x.req)
if x.err != nil {
return false
}
chunk = chunk[ln:]
}
return true
}
func (x *objectWriterRaw) Close(_ context.Context) (*ResObjectPut, error) {
// Ignore io.EOF error, because it is expected error for client-side
// stream termination by the server. E.g. when stream contains invalid
// message. Server returns an error in response message (in status).
if x.err != nil && !errors.Is(x.err, io.EOF) {
return nil, x.err
}
if x.err = x.stream.Close(); x.err != nil {
return nil, x.err
}
x.res.st, x.err = x.client.processResponse(&x.respV2)
if x.err != nil {
return nil, x.err
}
if !apistatus.IsSuccessful(x.res.st) {
return &x.res, nil
}
const fieldID = "ID"
idV2 := x.respV2.GetBody().GetObjectID()
if idV2 == nil {
return nil, newErrMissingResponseField(fieldID)
}
x.err = x.res.obj.ReadFromV2(*idV2)
if x.err != nil {
x.err = newErrInvalidResponseField(fieldID, x.err)
}
x.res.epoch = x.respV2.GetMetaHeader().GetEpoch()
return &x.res, nil
}

175
client/object_put_single.go Normal file
View file

@ -0,0 +1,175 @@
package client
import (
"context"
"crypto/ecdsa"
"fmt"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmObjectPutSingle groups parameters of PutSingle operation.
type PrmObjectPutSingle struct {
XHeaders []string
BearerToken *bearer.Token
Session *session.Object
Local bool
CopiesNumber []uint32
Object *object.Object
Key *ecdsa.PrivateKey
}
// SetCopiesNumber sets ordered list of minimal required object copies numbers
// per placement vector. List's length MUST equal container's placement vector number,
// otherwise request will fail.
//
// Deprecated: Use PrmObjectPutSingle.CopiesNumber instead.
func (prm *PrmObjectPutSingle) SetCopiesNumber(v []uint32) {
prm.CopiesNumber = v
}
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
//
// Deprecated: Use PrmObjectPutSingle.Key instead.
func (prm *PrmObjectPutSingle) UseKey(key *ecdsa.PrivateKey) {
prm.Key = key
}
// WithBearerToken attaches bearer token to be used for the operation.
// Should be called once before any writing steps.
//
// Deprecated: Use PrmObjectPutSingle.BearerToken instead.
func (prm *PrmObjectPutSingle) WithBearerToken(t bearer.Token) {
prm.BearerToken = &t
}
// WithinSession specifies session within which object should be stored.
// Should be called once before any writing steps.
//
// Deprecated: Use PrmObjectPutSingle.Session instead.
func (prm *PrmObjectPutSingle) WithinSession(t session.Object) {
prm.Session = &t
}
// ExecuteLocal tells the server to execute the operation locally.
//
// Deprecated: Use PrmObjectPutSingle.Local instead.
func (prm *PrmObjectPutSingle) ExecuteLocal() {
prm.Local = true
}
// WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length.
//
// Slice must not be mutated until the operation completes.
//
// Deprecated: Use PrmObjectPutSingle.XHeaders instead.
func (prm *PrmObjectPutSingle) WithXHeaders(hs ...string) {
prm.XHeaders = hs
}
// SetObject specifies prepared object to put.
//
// Deprecated: Use PrmObjectPutSingle.Object instead.
func (prm *PrmObjectPutSingle) SetObject(o *v2object.Object) {
prm.Object = object.NewFromV2(o)
}
// ResObjectPutSingle groups resulting values of PutSingle operation.
type ResObjectPutSingle struct {
statusRes
epoch uint64
}
// Epoch returns creation epoch of the saved object.
func (r *ResObjectPutSingle) Epoch() uint64 {
return r.epoch
}
func (prm *PrmObjectPutSingle) buildRequest(c *Client) (*v2object.PutSingleRequest, error) {
if len(prm.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
body := new(v2object.PutSingleRequestBody)
body.SetCopiesNumber(prm.CopiesNumber)
body.SetObject(prm.Object.ToV2())
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(prm.XHeaders, meta)
if prm.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
prm.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if prm.Session != nil {
v2SessionToken := new(v2session.Token)
prm.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if prm.Local {
meta.SetTTL(1)
}
req := &v2object.PutSingleRequest{}
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// ObjectPutSingle writes prepared object to FrostFS.
// Object must have payload, also containerID, objectID, ownerID, payload hash, payload length of an object must be set.
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as Go built-in error.
// If Client is tuned to resolve FrostFS API statuses, then FrostFS failures
// codes are returned as error.
func (c *Client) ObjectPutSingle(ctx context.Context, prm PrmObjectPutSingle) (*ResObjectPutSingle, error) {
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
key := &c.prm.Key
if prm.Key != nil {
key = prm.Key
}
err = signature.SignServiceMessage(key, req)
if err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.PutSingleObject(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResObjectPutSingle
res.st, err = c.processResponse(resp)
if err != nil {
return nil, err
}
res.epoch = resp.GetMetaHeader().GetEpoch()
return &res, nil
}

View file

@ -0,0 +1,144 @@
package client
import (
"context"
buffPool "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/util/pool"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/transformer"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
func (c *Client) objectPutInitTransformer(prm PrmObjectPutInit) (*objectWriterTransformer, error) {
var w objectWriterTransformer
w.it = internalTarget{
client: c,
prm: prm,
}
key := &c.prm.Key
if prm.Key != nil {
key = prm.Key
}
w.ot = transformer.NewPayloadSizeLimiter(transformer.Params{
Key: key,
NextTargetInit: func() transformer.ObjectWriter { return &w.it },
MaxSize: prm.MaxSize,
WithoutHomomorphicHash: prm.WithoutHomomorphHash,
NetworkState: prm.EpochSource,
Pool: prm.Pool,
})
return &w, nil
}
type objectWriterTransformer struct {
ot transformer.ChunkedObjectWriter
it internalTarget
err error
}
func (x *objectWriterTransformer) WriteHeader(ctx context.Context, hdr object.Object) bool {
x.err = x.ot.WriteHeader(ctx, &hdr)
return x.err == nil
}
func (x *objectWriterTransformer) WritePayloadChunk(ctx context.Context, chunk []byte) bool {
_, x.err = x.ot.Write(ctx, chunk)
return x.err == nil
}
func (x *objectWriterTransformer) Close(ctx context.Context) (*ResObjectPut, error) {
if x.err != nil {
return nil, x.err
}
ai, err := x.ot.Close(ctx)
if err != nil {
return nil, err
}
if ai != nil {
x.it.res.epoch = ai.Epoch
if ai.ParentID != nil {
x.it.res.obj = *ai.ParentID
}
}
return x.it.res, nil
}
type internalTarget struct {
client *Client
res *ResObjectPut
prm PrmObjectPutInit
useStream bool
}
func (it *internalTarget) WriteObject(ctx context.Context, o *object.Object) error {
putSingleImplemented, err := it.tryPutSingle(ctx, o)
if putSingleImplemented {
return err
}
it.useStream = true
return it.putAsStream(ctx, o)
}
func (it *internalTarget) putAsStream(ctx context.Context, o *object.Object) error {
wrt, err := it.client.objectPutInitRaw(ctx, it.prm)
if err != nil {
return err
}
if wrt.WriteHeader(ctx, *o) {
wrt.WritePayloadChunk(ctx, o.Payload())
}
it.res, err = wrt.Close(ctx)
if err == nil && it.client.prm.DisableFrostFSErrorResolution && !apistatus.IsSuccessful(it.res.st) {
err = apistatus.ErrFromStatus(it.res.st)
}
return err
}
func (it *internalTarget) tryPutSingle(ctx context.Context, o *object.Object) (bool, error) {
if it.useStream {
return false, nil
}
prm := PrmObjectPutSingle{
XHeaders: it.prm.XHeaders,
BearerToken: it.prm.BearerToken,
Session: it.prm.Session,
Local: it.prm.Local,
CopiesNumber: it.prm.CopiesNumber,
Object: o,
Key: it.prm.Key,
}
res, err := it.client.ObjectPutSingle(ctx, prm)
if err != nil && status.Code(err) == codes.Unimplemented {
return false, err
}
if err == nil {
it.returnBuffPool(o.Payload())
id, _ := o.ID()
it.res = &ResObjectPut{
statusRes: res.statusRes,
obj: id,
epoch: res.epoch,
}
if it.client.prm.DisableFrostFSErrorResolution && !apistatus.IsSuccessful(it.res.st) {
return true, apistatus.ErrFromStatus(it.res.st)
}
return true, nil
}
return true, err
}
func (it *internalTarget) returnBuffPool(playback []byte) {
if it.prm.Pool == nil {
return
}
var buffer buffPool.Buffer
buffer.Data = playback
it.prm.Pool.Put(&buffer)
}

View file

@ -7,40 +7,43 @@ import (
"fmt"
"io"
v2object "github.com/nspcc-dev/neofs-api-go/v2/object"
v2refs "github.com/nspcc-dev/neofs-api-go/v2/refs"
rpcapi "github.com/nspcc-dev/neofs-api-go/v2/rpc"
"github.com/nspcc-dev/neofs-api-go/v2/rpc/client"
v2session "github.com/nspcc-dev/neofs-api-go/v2/session"
apistatus "github.com/nspcc-dev/neofs-sdk-go/client/status"
cid "github.com/nspcc-dev/neofs-sdk-go/container/id"
"github.com/nspcc-dev/neofs-sdk-go/object"
oid "github.com/nspcc-dev/neofs-sdk-go/object/id"
"github.com/nspcc-dev/neofs-sdk-go/session"
"github.com/nspcc-dev/neofs-sdk-go/token"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/acl"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
v2refs "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/bearer"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/session"
)
// PrmObjectSearch groups parameters of ObjectSearch operation.
type PrmObjectSearch struct {
prmCommonMeta
XHeaders []string
local bool
Local bool
sessionSet bool
session session.Token
BearerToken *bearer.Token
bearerSet bool
bearer token.BearerToken
Session *session.Object
cnrSet bool
cnr cid.ID
ContainerID *cid.ID
filters object.SearchFilters
Key *ecdsa.PrivateKey
Filters object.SearchFilters
}
// MarkLocal tells the server to execute the operation locally.
//
// Deprecated: Use PrmObjectSearch.Local instead.
func (x *PrmObjectSearch) MarkLocal() {
x.local = true
x.Local = true
}
// WithinSession specifies session within which the search query must be executed.
@ -49,9 +52,10 @@ func (x *PrmObjectSearch) MarkLocal() {
// This may affect the execution of an operation (e.g. access control).
//
// Must be signed.
func (x *PrmObjectSearch) WithinSession(t session.Token) {
x.session = t
x.sessionSet = true
//
// Deprecated: Use PrmObjectSearch.Session instead.
func (x *PrmObjectSearch) WithinSession(t session.Object) {
x.Session = &t
}
// WithBearerToken attaches bearer token to be used for the operation.
@ -59,22 +63,44 @@ func (x *PrmObjectSearch) WithinSession(t session.Token) {
// If set, underlying eACL rules will be used in access control.
//
// Must be signed.
func (x *PrmObjectSearch) WithBearerToken(t token.BearerToken) {
x.bearer = t
x.bearerSet = true
//
// Deprecated: Use PrmObjectSearch.BearerToken instead.
func (x *PrmObjectSearch) WithBearerToken(t bearer.Token) {
x.BearerToken = &t
}
// WithXHeaders specifies list of extended headers (string key-value pairs)
// to be attached to the request. Must have an even length.
//
// Slice must not be mutated until the operation completes.
//
// Deprecated: Use PrmObjectSearch.XHeaders instead.
func (x *PrmObjectSearch) WithXHeaders(hs ...string) {
x.XHeaders = hs
}
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
//
// Deprecated: Use PrmObjectSearch.Key instead.
func (x *PrmObjectSearch) UseKey(key ecdsa.PrivateKey) {
x.Key = &key
}
// InContainer specifies the container in which to look for objects.
// Required parameter.
//
// Deprecated: Use PrmObjectSearch.ContainerID instead.
func (x *PrmObjectSearch) InContainer(id cid.ID) {
x.cnr = id
x.cnrSet = true
x.ContainerID = &id
}
// SetFilters sets filters by which to select objects. All container objects
// match unset/empty filters.
//
// Deprecated: Use PrmObjectSearch.Filters instead.
func (x *PrmObjectSearch) SetFilters(filters object.SearchFilters) {
x.filters = filters
x.Filters = filters
}
// ResObjectSearch groups the final result values of ObjectSearch operation.
@ -82,26 +108,18 @@ type ResObjectSearch struct {
statusRes
}
// ObjectListReader is designed to read list of object identifiers from NeoFS system.
// ObjectListReader is designed to read list of object identifiers from FrostFS system.
//
// Must be initialized using Client.ObjectSearch, any other usage is unsafe.
type ObjectListReader struct {
client *Client
cancelCtxStream context.CancelFunc
ctxCall contextCall
reqWritten bool
// initially bound to contextCall
bodyResp v2object.SearchResponseBody
tail []v2refs.ObjectID
err error
res ResObjectSearch
stream interface {
Read(resp *v2object.SearchResponse) error
}
// UseKey specifies private key to sign the requests.
// If key is not provided, then Client default key is used.
func (x *ObjectListReader) UseKey(key ecdsa.PrivateKey) {
x.ctxCall.key = key
tail []v2refs.ObjectID
}
// Read reads another list of the object identifiers. Works similar to
@ -115,58 +133,33 @@ func (x *ObjectListReader) Read(buf []oid.ID) (int, bool) {
panic("empty buffer in ObjectListReader.ReadList")
}
if !x.reqWritten {
if !x.ctxCall.writeRequest() {
return 0, false
}
x.reqWritten = true
}
// read remaining tail
read := len(x.tail)
if read > len(buf) {
read = len(buf)
}
for i := 0; i < read; i++ {
buf[i] = *oid.NewIDFromV2(&x.tail[i]) // need smth better
}
read := copyIDBuffers(buf, x.tail)
x.tail = x.tail[read:]
if len(buf) == read {
return read, true
}
var ok bool
var ids []v2refs.ObjectID
var i, ln, rem int
for {
// receive next message
ok = x.ctxCall.readResponse()
if !ok {
var resp v2object.SearchResponse
x.err = x.stream.Read(&resp)
if x.err != nil {
return read, false
}
x.res.st, x.err = x.client.processResponse(&resp)
if x.err != nil || !apistatus.IsSuccessful(x.res.st) {
return read, false
}
// read new chunk of objects
ids = x.bodyResp.GetIDList()
ln = len(ids)
if ln == 0 {
ids := resp.GetBody().GetIDList()
if len(ids) == 0 {
// just skip empty lists since they are not prohibited by protocol
continue
}
if rem = len(buf) - read; ln > rem {
ln = rem
}
for i = 0; i < ln; i++ {
buf[read+i] = *oid.NewIDFromV2(&ids[i]) // need smth better
}
ln := copyIDBuffers(buf[read:], ids)
read += ln
if read == len(buf) {
@ -178,6 +171,14 @@ func (x *ObjectListReader) Read(buf []oid.ID) (int, bool) {
}
}
func copyIDBuffers(dst []oid.ID, src []v2refs.ObjectID) int {
var i int
for ; i < len(dst) && i < len(src); i++ {
_ = dst[i].ReadFromV2(src[i])
}
return i
}
// Iterate iterates over the list of found object identifiers.
// f can return true to stop iteration earlier.
//
@ -207,7 +208,7 @@ func (x *ObjectListReader) Iterate(f func(oid.ID) bool) error {
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as Go built-in error.
// If Client is tuned to resolve NeoFS API statuses, then NeoFS failures
// If Client is tuned to resolve FrostFS API statuses, then FrostFS failures
// codes are returned as error.
//
// Return statuses:
@ -218,89 +219,87 @@ func (x *ObjectListReader) Iterate(f func(oid.ID) bool) error {
func (x *ObjectListReader) Close() (*ResObjectSearch, error) {
defer x.cancelCtxStream()
if x.ctxCall.err != nil && !errors.Is(x.ctxCall.err, io.EOF) {
return nil, x.ctxCall.err
if x.err != nil && !errors.Is(x.err, io.EOF) {
return nil, x.err
}
return x.ctxCall.statusRes.(*ResObjectSearch), nil
return &x.res, nil
}
// ObjectSearchInit initiates object selection through a remote server using NeoFS API protocol.
func (x *PrmObjectSearch) buildRequest(c *Client) (*v2object.SearchRequest, error) {
if x.ContainerID == nil {
return nil, errorMissingContainer
}
if len(x.XHeaders)%2 != 0 {
return nil, errorInvalidXHeaders
}
meta := new(v2session.RequestMetaHeader)
writeXHeadersToMeta(x.XHeaders, meta)
if x.BearerToken != nil {
v2BearerToken := new(acl.BearerToken)
x.BearerToken.WriteToV2(v2BearerToken)
meta.SetBearerToken(v2BearerToken)
}
if x.Session != nil {
v2SessionToken := new(v2session.Token)
x.Session.WriteToV2(v2SessionToken)
meta.SetSessionToken(v2SessionToken)
}
if x.Local {
meta.SetTTL(1)
}
cnrV2 := new(v2refs.ContainerID)
x.ContainerID.WriteToV2(cnrV2)
body := new(v2object.SearchRequestBody)
body.SetVersion(1)
body.SetContainerID(cnrV2)
body.SetFilters(x.Filters.ToV2())
req := new(v2object.SearchRequest)
req.SetBody(body)
c.prepareRequest(req, meta)
return req, nil
}
// ObjectSearchInit initiates object selection through a remote server using FrostFS API protocol.
//
// The call only opens the transmission channel, explicit fetching of matched objects
// is done using the ObjectListReader. Exactly one return value is non-nil.
// Resulting reader must be finally closed.
//
// Immediately panics if parameters are set incorrectly (see PrmObjectSearch docs).
// Returns an error if parameters are set incorrectly (see PrmObjectSearch docs).
// Context is required and must not be nil. It is used for network communication.
func (c *Client) ObjectSearchInit(ctx context.Context, prm PrmObjectSearch) (*ObjectListReader, error) {
// check parameters
switch {
case ctx == nil:
panic(panicMsgMissingContext)
case !prm.cnrSet:
panic(panicMsgMissingContainer)
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
// form request body
var body v2object.SearchRequestBody
body.SetVersion(1)
body.SetContainerID(prm.cnr.ToV2())
body.SetFilters(prm.filters.ToV2())
// form meta header
var meta v2session.RequestMetaHeader
if prm.local {
meta.SetTTL(1)
key := prm.Key
if key == nil {
key = &c.prm.Key
}
if prm.bearerSet {
meta.SetBearerToken(prm.bearer.ToV2())
err = signature.SignServiceMessage(key, req)
if err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
if prm.sessionSet {
meta.SetSessionToken(prm.session.ToV2())
}
prm.prmCommonMeta.writeToMetaHeader(&meta)
// form request
var req v2object.SearchRequest
req.SetBody(&body)
req.SetMetaHeader(&meta)
// init reader
var (
r ObjectListReader
resp v2object.SearchResponse
stream *rpcapi.SearchResponseReader
)
var r ObjectListReader
ctx, r.cancelCtxStream = context.WithCancel(ctx)
resp.SetBody(&r.bodyResp)
// init call context
c.initCallContext(&r.ctxCall)
r.ctxCall.req = &req
r.ctxCall.statusRes = new(ResObjectSearch)
r.ctxCall.resp = &resp
r.ctxCall.wReq = func() error {
var err error
stream, err = rpcapi.SearchObjects(&c.c, &req, client.WithContext(ctx))
r.stream, err = rpcapi.SearchObjects(&c.c, req, client.WithContext(ctx))
if err != nil {
return fmt.Errorf("open stream: %w", err)
}
return nil
}
r.ctxCall.rResp = func() error {
return stream.Read(&resp)
return nil, fmt.Errorf("open stream: %w", err)
}
r.client = c
return &r, nil
}

View file

@ -1,26 +1,28 @@
package client
import (
"crypto/ecdsa"
"errors"
"fmt"
"io"
"testing"
v2object "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
signatureV2 "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
oid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id"
oidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/object/id/test"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/nspcc-dev/neofs-api-go/v2/object"
"github.com/nspcc-dev/neofs-api-go/v2/refs"
signatureV2 "github.com/nspcc-dev/neofs-api-go/v2/signature"
oid "github.com/nspcc-dev/neofs-sdk-go/object/id"
oidtest "github.com/nspcc-dev/neofs-sdk-go/object/id/test"
"github.com/stretchr/testify/require"
)
func TestObjectSearch(t *testing.T) {
ids := make([]oid.ID, 20)
for i := range ids {
ids[i] = *oidtest.ID()
ids[i] = oidtest.ID()
}
resp, setID := testListReaderResponse(t)
p, resp := testListReaderResponse(t)
buf := make([]oid.ID, 2)
checkRead := func(t *testing.T, expected []oid.ID) {
@ -34,38 +36,23 @@ func TestObjectSearch(t *testing.T) {
require.Panics(t, func() { resp.Read(nil) })
// both ID fetched
setID(ids[:3])
resp.stream = newSearchStream(p, nil, ids[:3])
checkRead(t, ids[:2])
// one ID cached, second fetched
setID(ids[3:6])
resp.stream = newSearchStream(p, nil, ids[3:6])
checkRead(t, ids[2:4])
// both ID cached
resp.ctxCall.resp = nil
resp.stream = nil // shouldn't be called, panic if so
checkRead(t, ids[4:6])
// both ID fetched in 2 requests, with empty one in the middle
var n int
resp.ctxCall.rResp = func() error {
switch n {
case 0:
setID(ids[6:7])
case 1:
setID(nil)
case 2:
setID(ids[7:8])
default:
t.FailNow()
}
n++
return nil
}
resp.stream = newSearchStream(p, nil, ids[6:7], nil, ids[7:8])
checkRead(t, ids[6:8])
// read from tail multiple times
resp.ctxCall.rResp = nil
setID(ids[8:11])
resp.stream = newSearchStream(p, nil, ids[8:11])
buf = buf[:1]
checkRead(t, ids[8:9])
checkRead(t, ids[9:10])
@ -73,43 +60,20 @@ func TestObjectSearch(t *testing.T) {
// handle EOF
buf = buf[:2]
n = 0
resp.ctxCall.rResp = func() error {
if n > 0 {
return io.EOF
}
n++
setID(ids[11:12])
return nil
}
resp.stream = newSearchStream(p, io.EOF, ids[11:12])
checkRead(t, ids[11:12])
}
func TestObjectIterate(t *testing.T) {
ids := make([]oid.ID, 3)
for i := range ids {
ids[i] = *oidtest.ID()
ids[i] = oidtest.ID()
}
t.Run("iterate all sequence", func(t *testing.T) {
resp, setID := testListReaderResponse(t)
p, resp := testListReaderResponse(t)
// Iterate over all sequence
var n int
resp.ctxCall.rResp = func() error {
switch n {
case 0:
setID(ids[0:2])
case 1:
setID(nil)
case 2:
setID(ids[2:3])
default:
return io.EOF
}
n++
return nil
}
resp.stream = newSearchStream(p, io.EOF, ids[0:2], nil, ids[2:3])
var actual []oid.ID
require.NoError(t, resp.Iterate(func(id oid.ID) bool {
@ -119,10 +83,10 @@ func TestObjectIterate(t *testing.T) {
require.Equal(t, ids[:3], actual)
})
t.Run("stop by return value", func(t *testing.T) {
resp, setID := testListReaderResponse(t)
p, resp := testListReaderResponse(t)
var actual []oid.ID
setID(ids)
resp.stream = &singleStreamResponder{key: p, idList: [][]oid.ID{ids}}
require.NoError(t, resp.Iterate(func(id oid.ID) bool {
actual = append(actual, id)
return len(actual) == 2
@ -130,22 +94,12 @@ func TestObjectIterate(t *testing.T) {
require.Equal(t, ids[:2], actual)
})
t.Run("stop after error", func(t *testing.T) {
resp, setID := testListReaderResponse(t)
p, resp := testListReaderResponse(t)
expectedErr := errors.New("test error")
var actual []oid.ID
var n int
resp.ctxCall.rResp = func() error {
switch n {
case 0:
setID(ids[:2])
default:
return expectedErr
}
n++
return nil
}
resp.stream = newSearchStream(p, expectedErr, ids[:2])
var actual []oid.ID
err := resp.Iterate(func(id oid.ID) bool {
actual = append(actual, id)
return false
@ -155,37 +109,56 @@ func TestObjectIterate(t *testing.T) {
})
}
func testListReaderResponse(t *testing.T) (*ObjectListReader, func(id []oid.ID) *object.SearchResponse) {
func testListReaderResponse(t *testing.T) (*ecdsa.PrivateKey, *ObjectListReader) {
p, err := keys.NewPrivateKey()
require.NoError(t, err)
obj := &ObjectListReader{
return &p.PrivateKey, &ObjectListReader{
cancelCtxStream: func() {},
ctxCall: contextCall{
closer: func() error { return nil },
result: func(v2 responseV2) {},
statusRes: new(ResObjectSearch),
},
reqWritten: true,
bodyResp: object.SearchResponseBody{},
client: &Client{},
tail: nil,
}
return obj, func(id []oid.ID) *object.SearchResponse {
resp := new(object.SearchResponse)
resp.SetBody(new(object.SearchResponseBody))
v2id := make([]refs.ObjectID, len(id))
for i := range id {
v2id[i] = *id[i].ToV2()
}
resp.GetBody().SetIDList(v2id)
err := signatureV2.SignServiceMessage(&p.PrivateKey, resp)
func newSearchStream(key *ecdsa.PrivateKey, endError error, idList ...[]oid.ID) *singleStreamResponder {
return &singleStreamResponder{
key: key,
endError: endError,
idList: idList,
}
}
type singleStreamResponder struct {
key *ecdsa.PrivateKey
n int
endError error
idList [][]oid.ID
}
func (s *singleStreamResponder) Read(resp *v2object.SearchResponse) error {
if s.n >= len(s.idList) {
if s.endError != nil {
return s.endError
}
panic("unexpected call to `Read`")
}
var body v2object.SearchResponseBody
if s.idList[s.n] != nil {
ids := make([]refs.ObjectID, len(s.idList[s.n]))
for i := range s.idList[s.n] {
s.idList[s.n][i].WriteToV2(&ids[i])
}
body.SetIDList(ids)
}
resp.SetBody(&body)
err := signatureV2.SignServiceMessage(s.key, resp)
if err != nil {
t.Fatalf("error: %v", err)
}
obj.ctxCall.resp = resp
obj.bodyResp = *resp.GetBody()
return resp
panic(fmt.Errorf("error: %w", err))
}
s.n++
return nil
}

View file

@ -1,194 +0,0 @@
package client
import (
"context"
v2reputation "github.com/nspcc-dev/neofs-api-go/v2/reputation"
rpcapi "github.com/nspcc-dev/neofs-api-go/v2/rpc"
"github.com/nspcc-dev/neofs-api-go/v2/rpc/client"
"github.com/nspcc-dev/neofs-sdk-go/reputation"
)
// PrmAnnounceLocalTrust groups parameters of AnnounceLocalTrust operation.
type PrmAnnounceLocalTrust struct {
prmCommonMeta
epoch uint64
trusts []reputation.Trust
}
// SetEpoch sets number of NeoFS epoch in which the trust was assessed.
// Required parameter, must not be zero.
func (x *PrmAnnounceLocalTrust) SetEpoch(epoch uint64) {
x.epoch = epoch
}
// SetValues sets values describing trust of the client to the NeoFS network participants.
// Required parameter. Must not be empty.
//
// Must not be mutated before the end of the operation.
func (x *PrmAnnounceLocalTrust) SetValues(trusts []reputation.Trust) {
x.trusts = trusts
}
// ResAnnounceLocalTrust groups results of AnnounceLocalTrust operation.
type ResAnnounceLocalTrust struct {
statusRes
}
// AnnounceLocalTrust sends client's trust values to the NeoFS network participants.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If WithNeoFSErrorParsing option has been provided, unsuccessful
// NeoFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Immediately panics if parameters are set incorrectly (see PrmAnnounceLocalTrust docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) AnnounceLocalTrust(ctx context.Context, prm PrmAnnounceLocalTrust) (*ResAnnounceLocalTrust, error) {
// check parameters
switch {
case ctx == nil:
panic(panicMsgMissingContext)
case prm.epoch == 0:
panic("zero epoch")
case len(prm.trusts) == 0:
panic("missing trusts")
}
// form request body
reqBody := new(v2reputation.AnnounceLocalTrustRequestBody)
reqBody.SetEpoch(prm.epoch)
trusts := make([]reputation.Trust, len(prm.trusts))
copy(trusts, prm.trusts)
reqBody.SetTrusts(reputation.TrustsToV2(trusts))
// form request
var req v2reputation.AnnounceLocalTrustRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResAnnounceLocalTrust
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.AnnounceLocalTrust(&c.c, &req, client.WithContext(ctx))
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}
// PrmAnnounceIntermediateTrust groups parameters of AnnounceIntermediateTrust operation.
type PrmAnnounceIntermediateTrust struct {
prmCommonMeta
epoch uint64
iter uint32
trustSet bool
trust reputation.PeerToPeerTrust
}
// SetEpoch sets number of NeoFS epoch with which client's calculation algorithm is initialized.
// Required parameter, must not be zero.
func (x *PrmAnnounceIntermediateTrust) SetEpoch(epoch uint64) {
x.epoch = epoch
}
// SetIteration sets current sequence number of the client's calculation algorithm.
// By default, corresponds to initial (zero) iteration.
func (x *PrmAnnounceIntermediateTrust) SetIteration(iter uint32) {
x.iter = iter
}
// SetCurrentValue sets current global trust value computed at the specified iteration
// of the client's calculation algorithm. Required parameter.
func (x *PrmAnnounceIntermediateTrust) SetCurrentValue(trust reputation.PeerToPeerTrust) {
x.trust = trust
x.trustSet = true
}
// ResAnnounceIntermediateTrust groups results of AnnounceIntermediateTrust operation.
type ResAnnounceIntermediateTrust struct {
statusRes
}
// AnnounceIntermediateTrust sends global trust values calculated for the specified NeoFS network participants
// at some stage of client's calculation algorithm.
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If WithNeoFSErrorParsing option has been provided, unsuccessful
// NeoFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
//
// Immediately panics if parameters are set incorrectly (see PrmAnnounceIntermediateTrust docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) AnnounceIntermediateTrust(ctx context.Context, prm PrmAnnounceIntermediateTrust) (*ResAnnounceIntermediateTrust, error) {
// check parameters
switch {
case ctx == nil:
panic(panicMsgMissingContext)
case prm.epoch == 0:
panic("zero epoch")
case !prm.trustSet:
panic("current trust value not set")
}
// form request body
reqBody := new(v2reputation.AnnounceIntermediateResultRequestBody)
reqBody.SetEpoch(prm.epoch)
reqBody.SetIteration(prm.iter)
reqBody.SetTrust(prm.trust.ToV2())
// form request
var req v2reputation.AnnounceIntermediateResultRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResAnnounceIntermediateTrust
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.AnnounceIntermediateResult(&c.c, &req, client.WithContext(ctx))
}
// process call
if !cc.processCall() {
return nil, cc.err
}
return &res, nil
}

View file

@ -1,10 +1,12 @@
package client
import "github.com/nspcc-dev/neofs-api-go/v2/session"
import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
// ResponseMetaInfo groups meta information about any NeoFS API response.
// ResponseMetaInfo groups meta information about any FrostFS API response.
type ResponseMetaInfo struct {
key []byte
epoch uint64
}
type responseV2 interface {
@ -18,3 +20,8 @@ type responseV2 interface {
func (x ResponseMetaInfo) ResponderKey() []byte {
return x.key
}
// Epoch returns local FrostFS epoch of the server.
func (x ResponseMetaInfo) Epoch() uint64 {
return x.epoch
}

View file

@ -2,23 +2,64 @@ package client
import (
"context"
"crypto/ecdsa"
"fmt"
rpcapi "github.com/nspcc-dev/neofs-api-go/v2/rpc"
"github.com/nspcc-dev/neofs-api-go/v2/rpc/client"
v2session "github.com/nspcc-dev/neofs-api-go/v2/session"
"github.com/nspcc-dev/neofs-sdk-go/owner"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
rpcapi "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/rpc/client"
v2session "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/signature"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
)
// PrmSessionCreate groups parameters of SessionCreate operation.
type PrmSessionCreate struct {
prmCommonMeta
XHeaders []string
exp uint64
Expiration uint64
Key *ecdsa.PrivateKey
}
// SetExp sets number of the last NepFS epoch in the lifetime of the session after which it will be expired.
//
// Deprecated: Use PrmSessionCreate.Expiration instead.
func (x *PrmSessionCreate) SetExp(exp uint64) {
x.exp = exp
x.Expiration = exp
}
// UseKey specifies private key to sign the requests and compute token owner.
// If key is not provided, then Client default key is used.
//
// Deprecated: Use PrmSessionCreate.Key instead.
func (x *PrmSessionCreate) UseKey(key ecdsa.PrivateKey) {
x.Key = &key
}
func (x *PrmSessionCreate) buildRequest(c *Client) (*v2session.CreateRequest, error) {
ownerKey := c.prm.Key.PublicKey
if x.Key != nil {
ownerKey = x.Key.PublicKey
}
var ownerID user.ID
user.IDFromKey(&ownerID, ownerKey)
var ownerIDV2 refs.OwnerID
ownerID.WriteToV2(&ownerIDV2)
reqBody := new(v2session.CreateRequestBody)
reqBody.SetOwnerID(&ownerIDV2)
reqBody.SetExpiration(x.Expiration)
var meta v2session.RequestMetaHeader
writeXHeadersToMeta(x.XHeaders, &meta)
var req v2session.CreateRequest
req.SetBody(reqBody)
c.prepareRequest(&req, &meta)
return &req, nil
}
// ResSessionCreate groups resulting values of SessionCreate operation.
@ -30,22 +71,14 @@ type ResSessionCreate struct {
sessionKey []byte
}
func (x *ResSessionCreate) setID(id []byte) {
x.id = id
}
// ID returns identifier of the opened session in a binary NeoFS API protocol format.
// ID returns identifier of the opened session in a binary FrostFS API protocol format.
//
// Client doesn't retain value so modification is safe.
func (x ResSessionCreate) ID() []byte {
return x.id
}
func (x *ResSessionCreate) setSessionKey(key []byte) {
x.sessionKey = key
}
// PublicKey returns public key of the opened session in a binary NeoFS API protocol format.
// PublicKey returns public key of the opened session in a binary FrostFS API protocol format.
func (x ResSessionCreate) PublicKey() []byte {
return x.sessionKey
}
@ -56,60 +89,38 @@ func (x ResSessionCreate) PublicKey() []byte {
//
// Exactly one return value is non-nil. By default, server status is returned in res structure.
// Any client's internal or transport errors are returned as `error`.
// If WithNeoFSErrorParsing option has been provided, unsuccessful
// NeoFS status codes are returned as `error`, otherwise, are included
// in the returned result structure.
// If PrmInit.DisableFrostFSFailuresResolution has been called, unsuccessful
// FrostFS status codes are included in the returned result structure,
// otherwise, are also returned as `error`.
//
// Immediately panics if parameters are set incorrectly (see PrmSessionCreate docs).
// Returns an error if parameters are set incorrectly (see PrmSessionCreate docs).
// Context is required and must not be nil. It is used for network communication.
//
// Return statuses:
// - global (see Client docs).
func (c *Client) SessionCreate(ctx context.Context, prm PrmSessionCreate) (*ResSessionCreate, error) {
// check context
if ctx == nil {
panic(panicMsgMissingContext)
req, err := prm.buildRequest(c)
if err != nil {
return nil, err
}
ownerID := owner.NewIDFromPublicKey(&c.prm.key.PublicKey)
// form request body
reqBody := new(v2session.CreateRequestBody)
reqBody.SetOwnerID(ownerID.ToV2())
reqBody.SetExpiration(prm.exp)
// for request
var req v2session.CreateRequest
req.SetBody(reqBody)
// init call context
var (
cc contextCall
res ResSessionCreate
)
c.initCallContext(&cc)
cc.meta = prm.prmCommonMeta
cc.req = &req
cc.statusRes = &res
cc.call = func() (responseV2, error) {
return rpcapi.CreateSession(&c.c, &req, client.WithContext(ctx))
if err := signature.SignServiceMessage(&c.prm.Key, req); err != nil {
return nil, fmt.Errorf("sign request: %w", err)
}
resp, err := rpcapi.CreateSession(&c.c, req, client.WithContext(ctx))
if err != nil {
return nil, err
}
var res ResSessionCreate
res.st, err = c.processResponse(resp)
if err != nil || !apistatus.IsSuccessful(res.st) {
return &res, err
}
cc.result = func(r responseV2) {
resp := r.(*v2session.CreateResponse)
body := resp.GetBody()
res.setID(body.GetID())
res.setSessionKey(body.GetSessionKey())
}
// process call
if !cc.processCall() {
return nil, cc.err
}
res.id = body.GetID()
res.sessionKey = body.GetSessionKey()
return &res, nil
}

View file

@ -0,0 +1,53 @@
package apistatus
import (
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/apemanager"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
)
// APEManagerAccessDenied describes status of the failure because of the access control violation.
// Instances provide Status and StatusV2 interfaces.
type APEManagerAccessDenied struct {
v2 status.Status
}
const defaultAPEManagerAccessDeniedMsg = "apemanager access denied"
func (x *APEManagerAccessDenied) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultAPEManagerAccessDeniedMsg
}
return errMessageStatusV2(
globalizeCodeV2(apemanager.StatusAPEManagerAccessDenied, apemanager.GlobalizeFail),
msg,
)
}
func (x *APEManagerAccessDenied) fromStatusV2(st *status.Status) {
x.v2 = *st
}
// ToStatusV2 converts APEManagerAccessDenied to v2's Status.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// - code: APE_MANAGER_ACCESS_DENIED;
// - string message: "apemanager access denied";
// - details: empty.
func (x APEManagerAccessDenied) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(apemanager.StatusAPEManagerAccessDenied, apemanager.GlobalizeFail))
x.v2.SetMessage(defaultAPEManagerAccessDeniedMsg)
return &x.v2
}
// WriteReason writes human-readable access rejection reason.
func (x *APEManagerAccessDenied) WriteReason(reason string) {
apemanager.WriteAccessDeniedDesc(&x.v2, reason)
}
// Reason returns human-readable access rejection reason returned by the server.
// Returns empty value is reason is not presented.
func (x APEManagerAccessDenied) Reason() string {
return apemanager.ReadAccessDeniedDesc(x.v2)
}

View file

@ -3,7 +3,7 @@ package apistatus
import (
"encoding/binary"
"github.com/nspcc-dev/neofs-api-go/v2/status"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
)
// ServerInternal describes failure statuses related to internal server errors.
@ -14,7 +14,7 @@ type ServerInternal struct {
v2 status.Status
}
func (x ServerInternal) Error() string {
func (x *ServerInternal) Error() string {
return errMessageStatusV2(
globalizeCodeV2(status.Internal, status.GlobalizeCommonFail),
x.v2.Message(),
@ -29,9 +29,9 @@ func (x *ServerInternal) fromStatusV2(st *status.Status) {
// ToStatusV2 implements StatusV2 interface method.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// * code: INTERNAL;
// * string message: empty;
// * details: empty.
// - code: INTERNAL;
// - string message: empty;
// - details: empty.
func (x ServerInternal) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(status.Internal, status.GlobalizeCommonFail))
return &x.v2
@ -62,7 +62,7 @@ type WrongMagicNumber struct {
v2 status.Status
}
func (x WrongMagicNumber) Error() string {
func (x *WrongMagicNumber) Error() string {
return errMessageStatusV2(
globalizeCodeV2(status.WrongMagicNumber, status.GlobalizeCommonFail),
x.v2.Message(),
@ -77,9 +77,9 @@ func (x *WrongMagicNumber) fromStatusV2(st *status.Status) {
// ToStatusV2 implements StatusV2 interface method.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// * code: WRONG_MAGIC_NUMBER;
// * string message: empty;
// * details: empty.
// - code: WRONG_MAGIC_NUMBER;
// - string message: empty;
// - details: empty.
func (x WrongMagicNumber) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(status.WrongMagicNumber, status.GlobalizeCommonFail))
return &x.v2
@ -104,9 +104,9 @@ func (x *WrongMagicNumber) WriteCorrectMagic(magic uint64) {
// CorrectMagic returns network magic returned by the server.
// Second value indicates presence status:
// * -1 if number is presented in incorrect format
// * 0 if number is not presented
// * +1 otherwise
// - -1 if number is presented in incorrect format
// - 0 if number is not presented
// - +1 otherwise
func (x WrongMagicNumber) CorrectMagic() (magic uint64, ok int8) {
x.v2.IterateDetails(func(d *status.Detail) bool {
if d.ID() == status.DetailIDCorrectMagic {
@ -123,3 +123,118 @@ func (x WrongMagicNumber) CorrectMagic() (magic uint64, ok int8) {
return
}
// SignatureVerification describes failure status related to signature verification.
// Instances provide Status and StatusV2 interfaces.
type SignatureVerification struct {
v2 status.Status
}
const defaultSignatureVerificationMsg = "signature verification failed"
func (x *SignatureVerification) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultSignatureVerificationMsg
}
return errMessageStatusV2(
globalizeCodeV2(status.SignatureVerificationFail, status.GlobalizeCommonFail),
msg,
)
}
// implements local interface defined in FromStatusV2 func.
func (x *SignatureVerification) fromStatusV2(st *status.Status) {
x.v2 = *st
}
// ToStatusV2 implements StatusV2 interface method.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// - code: SIGNATURE_VERIFICATION_FAIL;
// - string message: written message via SetMessage or
// "signature verification failed" as a default message;
// - details: empty.
func (x SignatureVerification) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(status.SignatureVerificationFail, status.GlobalizeCommonFail))
if x.v2.Message() == "" {
x.v2.SetMessage(defaultSignatureVerificationMsg)
}
return &x.v2
}
// SetMessage writes signature verification failure message.
// Message should be used for debug purposes only.
//
// See also Message.
func (x *SignatureVerification) SetMessage(v string) {
x.v2.SetMessage(v)
}
// Message returns status message. Zero status returns empty message.
// Message should be used for debug purposes only.
//
// See also SetMessage.
func (x SignatureVerification) Message() string {
return x.v2.Message()
}
// NodeUnderMaintenance describes failure status for nodes being under maintenance.
// Instances provide Status and StatusV2 interfaces.
type NodeUnderMaintenance struct {
v2 status.Status
}
const defaultNodeUnderMaintenanceMsg = "node is under maintenance"
// Error implements the error interface.
func (x *NodeUnderMaintenance) Error() string {
msg := x.Message()
if msg == "" {
msg = defaultNodeUnderMaintenanceMsg
}
return errMessageStatusV2(
globalizeCodeV2(status.NodeUnderMaintenance, status.GlobalizeCommonFail),
msg,
)
}
func (x *NodeUnderMaintenance) fromStatusV2(st *status.Status) {
x.v2 = *st
}
// ToStatusV2 implements StatusV2 interface method.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// - code: NODE_UNDER_MAINTENANCE;
// - string message: written message via SetMessage or
// "node is under maintenance" as a default message;
// - details: empty.
func (x NodeUnderMaintenance) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(status.NodeUnderMaintenance, status.GlobalizeCommonFail))
if x.v2.Message() == "" {
x.v2.SetMessage(defaultNodeUnderMaintenanceMsg)
}
return &x.v2
}
// SetMessage writes signature verification failure message.
// Message should be used for debug purposes only.
//
// See also Message.
func (x *NodeUnderMaintenance) SetMessage(v string) {
x.v2.SetMessage(v)
}
// Message returns status message. Zero status returns empty message.
// Message should be used for debug purposes only.
//
// See also SetMessage.
func (x NodeUnderMaintenance) Message() string {
return x.v2.Message()
}

View file

@ -3,8 +3,8 @@ package apistatus_test
import (
"testing"
"github.com/nspcc-dev/neofs-api-go/v2/status"
apistatus "github.com/nspcc-dev/neofs-sdk-go/client/status"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"github.com/stretchr/testify/require"
)
@ -50,3 +50,81 @@ func TestWrongMagicNumber_CorrectMagic(t *testing.T) {
_, ok = st.CorrectMagic()
require.EqualValues(t, -1, ok)
}
func TestSignatureVerification(t *testing.T) {
t.Run("default", func(t *testing.T) {
var st apistatus.SignatureVerification
require.Empty(t, st.Message())
})
t.Run("custom message", func(t *testing.T) {
var st apistatus.SignatureVerification
msg := "some message"
st.SetMessage(msg)
stV2 := st.ToStatusV2()
require.Equal(t, msg, st.Message())
require.Equal(t, msg, stV2.Message())
})
t.Run("empty to V2", func(t *testing.T) {
var st apistatus.SignatureVerification
stV2 := st.ToStatusV2()
require.Equal(t, "signature verification failed", stV2.Message())
})
t.Run("non-empty to V2", func(t *testing.T) {
var st apistatus.SignatureVerification
msg := "some other msg"
st.SetMessage(msg)
stV2 := st.ToStatusV2()
require.Equal(t, msg, stV2.Message())
})
}
func TestNodeUnderMaintenance(t *testing.T) {
t.Run("default", func(t *testing.T) {
var st apistatus.NodeUnderMaintenance
require.Empty(t, st.Message())
})
t.Run("custom message", func(t *testing.T) {
var st apistatus.NodeUnderMaintenance
msg := "some message"
st.SetMessage(msg)
stV2 := st.ToStatusV2()
require.Equal(t, msg, st.Message())
require.Equal(t, msg, stV2.Message())
})
t.Run("empty to V2", func(t *testing.T) {
var st apistatus.NodeUnderMaintenance
stV2 := st.ToStatusV2()
require.Empty(t, "", stV2.Message())
})
t.Run("non-empty to V2", func(t *testing.T) {
var st apistatus.NodeUnderMaintenance
msg := "some other msg"
st.SetMessage(msg)
stV2 := st.ToStatusV2()
require.Equal(t, msg, stV2.Message())
})
}

View file

@ -1,8 +1,8 @@
package apistatus
import (
"github.com/nspcc-dev/neofs-api-go/v2/container"
"github.com/nspcc-dev/neofs-api-go/v2/status"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
)
// ContainerNotFound describes status of the failure because of the missing container.
@ -11,10 +11,17 @@ type ContainerNotFound struct {
v2 status.Status
}
func (x ContainerNotFound) Error() string {
const defaultContainerNotFoundMsg = "container not found"
func (x *ContainerNotFound) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultContainerNotFoundMsg
}
return errMessageStatusV2(
globalizeCodeV2(container.StatusNotFound, container.GlobalizeFail),
x.v2.Message(),
msg,
)
}
@ -26,11 +33,49 @@ func (x *ContainerNotFound) fromStatusV2(st *status.Status) {
// ToStatusV2 implements StatusV2 interface method.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// * code: CONTAINER_NOT_FOUND;
// * string message: "container not found";
// * details: empty.
// - code: CONTAINER_NOT_FOUND;
// - string message: "container not found";
// - details: empty.
func (x ContainerNotFound) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(container.StatusNotFound, container.GlobalizeFail))
x.v2.SetMessage("container not found")
x.v2.SetMessage(defaultContainerNotFoundMsg)
return &x.v2
}
// EACLNotFound describes status of the failure because of the missing eACL
// table.
// Instances provide Status and StatusV2 interfaces.
type EACLNotFound struct {
v2 status.Status
}
const defaultEACLNotFoundMsg = "eACL not found"
func (x *EACLNotFound) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultEACLNotFoundMsg
}
return errMessageStatusV2(
globalizeCodeV2(container.StatusEACLNotFound, container.GlobalizeFail),
msg,
)
}
// implements local interface defined in FromStatusV2 func.
func (x *EACLNotFound) fromStatusV2(st *status.Status) {
x.v2 = *st
}
// ToStatusV2 implements StatusV2 interface method.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// - code: EACL_NOT_FOUND;
// - string message: "eACL not found";
// - details: empty.
func (x EACLNotFound) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(container.StatusEACLNotFound, container.GlobalizeFail))
x.v2.SetMessage(defaultEACLNotFoundMsg)
return &x.v2
}

View file

@ -1,8 +1,8 @@
package apistatus
import (
"github.com/nspcc-dev/neofs-api-go/v2/object"
"github.com/nspcc-dev/neofs-api-go/v2/status"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
)
// ObjectLocked describes status of the failure because of the locked object.
@ -11,10 +11,17 @@ type ObjectLocked struct {
v2 status.Status
}
func (x ObjectLocked) Error() string {
const defaultObjectLockedMsg = "object is locked"
func (x *ObjectLocked) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultObjectLockedMsg
}
return errMessageStatusV2(
globalizeCodeV2(object.StatusLocked, object.GlobalizeFail),
x.v2.Message(),
msg,
)
}
@ -26,12 +33,12 @@ func (x *ObjectLocked) fromStatusV2(st *status.Status) {
// ToStatusV2 implements StatusV2 interface method.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// * code: LOCKED;
// * string message: "object is locked";
// * details: empty.
// - code: LOCKED;
// - string message: "object is locked";
// - details: empty.
func (x ObjectLocked) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(object.StatusLocked, object.GlobalizeFail))
x.v2.SetMessage("object is locked")
x.v2.SetMessage(defaultObjectLockedMsg)
return &x.v2
}
@ -41,10 +48,17 @@ type LockNonRegularObject struct {
v2 status.Status
}
func (x LockNonRegularObject) Error() string {
const defaultLockNonRegularObjectMsg = "locking non-regular object is forbidden"
func (x *LockNonRegularObject) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultLockNonRegularObjectMsg
}
return errMessageStatusV2(
globalizeCodeV2(object.StatusLockNonRegularObject, object.GlobalizeFail),
x.v2.Message(),
msg,
)
}
@ -56,12 +70,12 @@ func (x *LockNonRegularObject) fromStatusV2(st *status.Status) {
// ToStatusV2 implements StatusV2 interface method.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// * code: LOCK_NON_REGULAR_OBJECT;
// * string message: "locking non-regular object is forbidden";
// * details: empty.
// - code: LOCK_NON_REGULAR_OBJECT;
// - string message: "locking non-regular object is forbidden";
// - details: empty.
func (x LockNonRegularObject) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(object.StatusLockNonRegularObject, object.GlobalizeFail))
x.v2.SetMessage("locking non-regular object is forbidden")
x.v2.SetMessage(defaultLockNonRegularObjectMsg)
return &x.v2
}
@ -71,10 +85,17 @@ type ObjectAccessDenied struct {
v2 status.Status
}
func (x ObjectAccessDenied) Error() string {
const defaultObjectAccessDeniedMsg = "access to object operation denied"
func (x *ObjectAccessDenied) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultObjectAccessDeniedMsg
}
return errMessageStatusV2(
globalizeCodeV2(object.StatusAccessDenied, object.GlobalizeFail),
x.v2.Message(),
msg,
)
}
@ -86,12 +107,12 @@ func (x *ObjectAccessDenied) fromStatusV2(st *status.Status) {
// ToStatusV2 implements StatusV2 interface method.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// * code: ACCESS_DENIED;
// * string message: "access to object operation denied";
// * details: empty.
// - code: ACCESS_DENIED;
// - string message: "access to object operation denied";
// - details: empty.
func (x ObjectAccessDenied) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(object.StatusAccessDenied, object.GlobalizeFail))
x.v2.SetMessage("access to object operation denied")
x.v2.SetMessage(defaultObjectAccessDeniedMsg)
return &x.v2
}
@ -112,10 +133,17 @@ type ObjectNotFound struct {
v2 status.Status
}
func (x ObjectNotFound) Error() string {
const defaultObjectNotFoundMsg = "object not found"
func (x *ObjectNotFound) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultObjectNotFoundMsg
}
return errMessageStatusV2(
globalizeCodeV2(object.StatusNotFound, object.GlobalizeFail),
x.v2.Message(),
msg,
)
}
@ -127,12 +155,12 @@ func (x *ObjectNotFound) fromStatusV2(st *status.Status) {
// ToStatusV2 implements StatusV2 interface method.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// * code: OBJECT_NOT_FOUND;
// * string message: "object not found";
// * details: empty.
// - code: OBJECT_NOT_FOUND;
// - string message: "object not found";
// - details: empty.
func (x ObjectNotFound) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(object.StatusNotFound, object.GlobalizeFail))
x.v2.SetMessage("object not found")
x.v2.SetMessage(defaultObjectNotFoundMsg)
return &x.v2
}
@ -142,10 +170,17 @@ type ObjectAlreadyRemoved struct {
v2 status.Status
}
func (x ObjectAlreadyRemoved) Error() string {
const defaultObjectAlreadyRemovedMsg = "object already removed"
func (x *ObjectAlreadyRemoved) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultObjectAlreadyRemovedMsg
}
return errMessageStatusV2(
globalizeCodeV2(object.StatusAlreadyRemoved, object.GlobalizeFail),
x.v2.Message(),
msg,
)
}
@ -157,11 +192,49 @@ func (x *ObjectAlreadyRemoved) fromStatusV2(st *status.Status) {
// ToStatusV2 implements StatusV2 interface method.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// * code: OBJECT_ALREADY_REMOVED;
// * string message: "object already removed";
// * details: empty.
// - code: OBJECT_ALREADY_REMOVED;
// - string message: "object already removed";
// - details: empty.
func (x ObjectAlreadyRemoved) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(object.StatusAlreadyRemoved, object.GlobalizeFail))
x.v2.SetMessage("object already removed")
x.v2.SetMessage(defaultObjectAlreadyRemovedMsg)
return &x.v2
}
// ObjectOutOfRange describes status of the failure because of the incorrect
// provided object ranges.
// Instances provide Status and StatusV2 interfaces.
type ObjectOutOfRange struct {
v2 status.Status
}
const defaultObjectOutOfRangeMsg = "out of range"
func (x *ObjectOutOfRange) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultObjectOutOfRangeMsg
}
return errMessageStatusV2(
globalizeCodeV2(object.StatusOutOfRange, object.GlobalizeFail),
msg,
)
}
// implements local interface defined in FromStatusV2 func.
func (x *ObjectOutOfRange) fromStatusV2(st *status.Status) {
x.v2 = *st
}
// ToStatusV2 implements StatusV2 interface method.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// - code: OUT_OF_RANGE;
// - string message: "out of range";
// - details: empty.
func (x ObjectOutOfRange) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(object.StatusOutOfRange, object.GlobalizeFail))
x.v2.SetMessage(defaultObjectOutOfRangeMsg)
return &x.v2
}

View file

@ -3,7 +3,7 @@ package apistatus_test
import (
"testing"
apistatus "github.com/nspcc-dev/neofs-sdk-go/client/status"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"github.com/stretchr/testify/require"
)

View file

@ -1,8 +1,8 @@
package apistatus
import (
"github.com/nspcc-dev/neofs-api-go/v2/session"
"github.com/nspcc-dev/neofs-api-go/v2/status"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
)
// SessionTokenNotFound describes status of the failure because of the missing session token.
@ -11,10 +11,17 @@ type SessionTokenNotFound struct {
v2 status.Status
}
func (x SessionTokenNotFound) Error() string {
const defaultSessionTokenNotFoundMsg = "session token not found"
func (x *SessionTokenNotFound) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultSessionTokenNotFoundMsg
}
return errMessageStatusV2(
globalizeCodeV2(session.StatusTokenNotFound, session.GlobalizeFail),
x.v2.Message(),
msg,
)
}
@ -26,12 +33,12 @@ func (x *SessionTokenNotFound) fromStatusV2(st *status.Status) {
// ToStatusV2 implements StatusV2 interface method.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// * code: TOKEN_NOT_FOUND;
// * string message: "session token not found";
// * details: empty.
// - code: TOKEN_NOT_FOUND;
// - string message: "session token not found";
// - details: empty.
func (x SessionTokenNotFound) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(session.StatusTokenNotFound, session.GlobalizeFail))
x.v2.SetMessage("session token not found")
x.v2.SetMessage(defaultSessionTokenNotFoundMsg)
return &x.v2
}
@ -41,10 +48,17 @@ type SessionTokenExpired struct {
v2 status.Status
}
func (x SessionTokenExpired) Error() string {
const defaultSessionTokenExpiredMsg = "expired session token"
func (x *SessionTokenExpired) Error() string {
msg := x.v2.Message()
if msg == "" {
msg = defaultSessionTokenExpiredMsg
}
return errMessageStatusV2(
globalizeCodeV2(session.StatusTokenExpired, session.GlobalizeFail),
x.v2.Message(),
msg,
)
}
@ -56,11 +70,11 @@ func (x *SessionTokenExpired) fromStatusV2(st *status.Status) {
// ToStatusV2 implements StatusV2 interface method.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// * code: TOKEN_EXPIRED;
// * string message: "expired session token";
// * details: empty.
// - code: TOKEN_EXPIRED;
// - string message: "expired session token";
// - details: empty.
func (x SessionTokenExpired) ToStatusV2() *status.Status {
x.v2.SetCode(globalizeCodeV2(session.StatusTokenExpired, session.GlobalizeFail))
x.v2.SetMessage("expired session token")
x.v2.SetMessage(defaultSessionTokenExpiredMsg)
return &x.v2
}

View file

@ -1,10 +1,10 @@
package apistatus
// Status defines a variety of NeoFS API status returns.
// Status defines a variety of FrostFS API status returns.
//
// All statuses are split into two disjoint subsets: successful and failed, and:
// * statuses that implement the build-in error interface are considered failed statuses;
// * all other value types are considered successes (nil is a default success).
// - statuses that implement the build-in error interface are considered failed statuses;
// - all other value types are considered successes (nil is a default success).
//
// In Go code type of success can be determined by a type switch, failure - by a switch with errors.As calls.
// Nil should be considered as a success, and default switch section - as an unrecognized Status.
@ -14,8 +14,8 @@ package apistatus
// IsSuccessful function should be used (try to avoid nil comparison).
// It should be noted that using direct typecasting is not a compatible approach.
//
// To transport statuses using the NeoFS API V2 protocol, see StatusV2 interface and FromStatusV2 and ToStatusV2 functions.
type Status interface{}
// To transport statuses using the FrostFS API V2 protocol, see StatusV2 interface and FromStatusV2 and ToStatusV2 functions.
type Status any
// ErrFromStatus converts Status instance to error if it is failed. Returns nil on successful Status.
//

View file

@ -4,7 +4,7 @@ import (
"errors"
"testing"
apistatus "github.com/nspcc-dev/neofs-sdk-go/client/status"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"github.com/stretchr/testify/require"
)

View file

@ -1,7 +1,7 @@
package apistatus
import (
"github.com/nspcc-dev/neofs-api-go/v2/status"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
)
// SuccessDefaultV2 represents Status instance of default success. Implements StatusV2.
@ -20,9 +20,9 @@ func (x *SuccessDefaultV2) fromStatusV2(st *status.Status) {
// ToStatusV2 implements StatusV2 interface method.
// If the value was returned by FromStatusV2, returns the source message.
// Otherwise, returns message with
// * code: OK;
// * string message: empty;
// * details: empty.
// - code: OK;
// - string message: empty;
// - details: empty.
func (x SuccessDefaultV2) ToStatusV2() *status.Status {
if x.isNil || x.v2 != nil {
return x.v2

View file

@ -1,14 +1,14 @@
package apistatus
import (
"github.com/nspcc-dev/neofs-api-go/v2/status"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
)
type unrecognizedStatusV2 struct {
v2 status.Status
}
func (x unrecognizedStatusV2) Error() string {
func (x *unrecognizedStatusV2) Error() string {
return errMessageStatusV2("unrecognized", x.v2.Message())
}

View file

@ -3,19 +3,20 @@ package apistatus
import (
"fmt"
"github.com/nspcc-dev/neofs-api-go/v2/container"
"github.com/nspcc-dev/neofs-api-go/v2/object"
"github.com/nspcc-dev/neofs-api-go/v2/session"
"github.com/nspcc-dev/neofs-api-go/v2/status"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/apemanager"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/object"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/session"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status"
)
// StatusV2 defines a variety of Status instances compatible with NeoFS API V2 protocol.
// StatusV2 defines a variety of Status instances compatible with FrostFS API V2 protocol.
//
// Note: it is not recommended to use this type directly, it is intended for documentation of the library functionality.
type StatusV2 interface {
Status
// ToStatusV2 returns the status as github.com/nspcc-dev/neofs-api-go/v2/status.Status message structure.
// ToStatusV2 returns the status as git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/status.Status message structure.
ToStatusV2() *status.Status
}
@ -28,15 +29,16 @@ type StatusV2 interface {
// Note: notice if the return type is a pointer.
//
// Successes:
// * status.OK: *SuccessDefaultV2 (this also includes nil argument).
// - status.OK: *SuccessDefaultV2 (this also includes nil argument).
//
// Common failures:
// * status.Internal: *ServerInternal.
// - status.Internal: *ServerInternal;
// - status.SignatureVerificationFail: *SignatureVerification.
//
// Object failures:
// * object.StatusLocked: *ObjectLocked;
// * object.StatusLockNonRegularObject: *LockNonRegularObject.
// * object.StatusAccessDenied: *ObjectAccessDenied.
// - object.StatusLocked: *ObjectLocked;
// - object.StatusLockNonRegularObject: *LockNonRegularObject.
// - object.StatusAccessDenied: *ObjectAccessDenied.
func FromStatusV2(st *status.Status) Status {
var decoder interface {
fromStatusV2(*status.Status)
@ -55,6 +57,10 @@ func FromStatusV2(st *status.Status) Status {
decoder = new(ServerInternal)
case status.WrongMagicNumber:
decoder = new(WrongMagicNumber)
case status.SignatureVerificationFail:
decoder = new(SignatureVerification)
case status.NodeUnderMaintenance:
decoder = new(NodeUnderMaintenance)
}
case object.LocalizeFailStatus(&code):
switch code {
@ -68,12 +74,16 @@ func FromStatusV2(st *status.Status) Status {
decoder = new(ObjectNotFound)
case object.StatusAlreadyRemoved:
decoder = new(ObjectAlreadyRemoved)
case object.StatusOutOfRange:
decoder = new(ObjectOutOfRange)
}
case container.LocalizeFailStatus(&code):
//nolint:exhaustive
switch code {
case container.StatusNotFound:
decoder = new(ContainerNotFound)
case container.StatusEACLNotFound:
decoder = new(EACLNotFound)
}
case session.LocalizeFailStatus(&code):
//nolint:exhaustive
@ -83,6 +93,12 @@ func FromStatusV2(st *status.Status) Status {
case session.StatusTokenExpired:
decoder = new(SessionTokenExpired)
}
case apemanager.LocalizeFailStatus(&code):
//nolint:exhaustive
switch code {
case apemanager.StatusAPEManagerAccessDenied:
decoder = new(APEManagerAccessDenied)
}
}
if decoder == nil {
@ -97,7 +113,8 @@ func FromStatusV2(st *status.Status) Status {
// ToStatusV2 converts Status instance to status.Status message structure. Inverse to FromStatusV2 operation.
//
// If argument is the StatusV2 instance, it is converted directly.
// Otherwise, successes are converted with status.OK code w/o details and message, failures - with status.Internal.
// Otherwise, successes are converted with status.OK code w/o details and message,
// failures - with status.Internal and error text message w/o details.
func ToStatusV2(st Status) *status.Status {
if v, ok := st.(StatusV2); ok {
return v.ToStatusV2()
@ -107,10 +124,13 @@ func ToStatusV2(st Status) *status.Status {
return newStatusV2WithLocalCode(status.OK, status.GlobalizeSuccess)
}
return newStatusV2WithLocalCode(status.Internal, status.GlobalizeCommonFail)
internalErrorStatus := newStatusV2WithLocalCode(status.Internal, status.GlobalizeCommonFail)
internalErrorStatus.SetMessage(st.(error).Error()) // type cast never panics because IsSuccessful() checks cast
return internalErrorStatus
}
func errMessageStatusV2(code interface{}, msg string) string {
func errMessageStatusV2(code any, msg string) string {
const (
noMsgFmt = "status: code = %v"
msgFmt = noMsgFmt + " message = %s"

View file

@ -4,7 +4,7 @@ import (
"errors"
"testing"
apistatus "github.com/nspcc-dev/neofs-sdk-go/client/status"
apistatus "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/client/status"
"github.com/stretchr/testify/require"
)
@ -12,12 +12,14 @@ func TestToStatusV2(t *testing.T) {
type statusConstructor func() apistatus.Status
for _, testItem := range [...]struct {
status interface{} // Status or statusConstructor
status any // Status or statusConstructor
codeV2 uint64
messageV2 string
}{
{
status: errors.New("some error"),
codeV2: 1024,
messageV2: "some error",
},
{
status: 1,
@ -93,12 +95,24 @@ func TestToStatusV2(t *testing.T) {
}),
codeV2: 2052,
},
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.ObjectOutOfRange)
}),
codeV2: 2053,
},
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.ContainerNotFound)
}),
codeV2: 3072,
},
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.EACLNotFound)
}),
codeV2: 3073,
},
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.SessionTokenNotFound)
@ -111,6 +125,18 @@ func TestToStatusV2(t *testing.T) {
}),
codeV2: 4097,
},
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.APEManagerAccessDenied)
}),
codeV2: 5120,
},
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.NodeUnderMaintenance)
}),
codeV2: 1027,
},
} {
var st apistatus.Status
@ -124,6 +150,9 @@ func TestToStatusV2(t *testing.T) {
// must generate the same status.Status message
require.EqualValues(t, testItem.codeV2, stv2.Code())
if len(testItem.messageV2) > 0 {
require.Equal(t, testItem.messageV2, stv2.Message())
}
_, ok := st.(apistatus.StatusV2)
if ok {
@ -142,12 +171,14 @@ func TestFromStatusV2(t *testing.T) {
type statusConstructor func() apistatus.Status
for _, testItem := range [...]struct {
status interface{} // Status or statusConstructor
status any // Status or statusConstructor
codeV2 uint64
messageV2 string
}{
{
status: errors.New("some error"),
codeV2: 1024,
messageV2: "some error",
},
{
status: 1,
@ -223,12 +254,24 @@ func TestFromStatusV2(t *testing.T) {
}),
codeV2: 2052,
},
{
status: statusConstructor(func() apistatus.Status {
return new(apistatus.ObjectOutOfRange)
}),
codeV2: 2053,
},
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.ContainerNotFound)
}),
codeV2: 3072,
},
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.EACLNotFound)
}),
codeV2: 3073,
},
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.SessionTokenNotFound)
@ -241,6 +284,18 @@ func TestFromStatusV2(t *testing.T) {
}),
codeV2: 4097,
},
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.APEManagerAccessDenied)
}),
codeV2: 5120,
},
{
status: (statusConstructor)(func() apistatus.Status {
return new(apistatus.NodeUnderMaintenance)
}),
codeV2: 1027,
},
} {
var st apistatus.Status
@ -254,6 +309,9 @@ func TestFromStatusV2(t *testing.T) {
// must generate the same status.Status message
require.EqualValues(t, testItem.codeV2, stv2.Code())
if len(testItem.messageV2) > 0 {
require.Equal(t, testItem.messageV2, stv2.Message())
}
_, ok := st.(apistatus.StatusV2)
if ok {

83
container/acl/acl.go Normal file
View file

@ -0,0 +1,83 @@
package acl
import "strconv"
// Op enumerates operations under access control inside container.
// Non-positive values are reserved and depend on context (e.g. unsupported op).
//
// Note that type conversion from- and to numerical types is not recommended,
// use corresponding constants and/or methods instead.
type Op uint32
// nolint: unused
const (
opZero Op = iota // extreme value for testing
OpObjectGet // Object.Get rpc
OpObjectHead // Object.Head rpc
OpObjectPut // Object.Put rpc
OpObjectDelete // Object.Delete rpc
OpObjectSearch // Object.Search rpc
OpObjectRange // Object.GetRange rpc
OpObjectHash // Object.GetRangeHash rpc
opLast // extreme value for testing
)
// String implements fmt.Stringer.
func (x Op) String() string {
switch x {
default:
return "UNKNOWN#" + strconv.FormatUint(uint64(x), 10)
case OpObjectGet:
return "OBJECT_GET"
case OpObjectHead:
return "OBJECT_HEAD"
case OpObjectPut:
return "OBJECT_PUT"
case OpObjectDelete:
return "OBJECT_DELETE"
case OpObjectSearch:
return "OBJECT_SEARCH"
case OpObjectRange:
return "OBJECT_RANGE"
case OpObjectHash:
return "OBJECT_HASH"
}
}
// Role enumerates roles covered by container ACL. Each role represents
// some party which can be authenticated during container op execution.
// Non-positive values are reserved and depend on context (e.g. unsupported role).
//
// Note that type conversion from- and to numerical types is not recommended,
// use corresponding constants and/or methods instead.
type Role uint32
// nolint: unused
const (
roleZero Role = iota // extreme value for testing
RoleOwner // container owner
RoleContainer // nodes of the related container
RoleInnerRing // Inner Ring nodes
RoleOthers // all others
roleLast // extreme value for testing
)
// String implements fmt.Stringer.
func (x Role) String() string {
switch x {
default:
return "UNKNOWN#" + strconv.FormatUint(uint64(x), 10)
case RoleOwner:
return "OWNER"
case RoleContainer:
return "CONTAINER"
case RoleInnerRing:
return "INNER_RING"
case RoleOthers:
return "OTHERS"
}
}

283
container/acl/acl_basic.go Normal file
View file

@ -0,0 +1,283 @@
package acl
import (
"fmt"
"strconv"
"strings"
)
// Basic represents basic part of the FrostFS container's ACL. It includes
// common (pretty simple) access rules for operations inside the container.
// See FrostFS Specification for details.
//
// One can find some similarities with the traditional Unix permission, such as
//
// division into scopes: user, group, others
// op-permissions: read, write, etc.
// sticky bit
//
// However, these similarities should only be used for better understanding,
// in general these mechanisms are different.
//
// Instances can be created using built-in var declaration, but look carefully
// at the default values, and how individual permissions are regulated.
// Some frequently used values are presented in exported variables.
//
// Basic instances are comparable: values can be compared directly using
// == operator.
//
// Note that type conversion from- and to numerical types is not recommended,
// use corresponding constants and/or methods instead.
type Basic uint32
// FromBits decodes Basic from the numerical representation.
//
// See also Bits.
func (x *Basic) FromBits(bits uint32) {
*x = Basic(bits)
}
// Bits returns numerical encoding of Basic.
//
// See also FromBits.
func (x Basic) Bits() uint32 {
return uint32(x)
}
// common bit sections.
const (
opAmount = 7
bitsPerOp = 4
bitPosFinal = opAmount * bitsPerOp
bitPosSticky = bitPosFinal + 1
)
// per-op bit order.
const (
opBitPosBearer uint8 = iota
opBitPosOthers
opBitPosContainer
opBitPosOwner
)
// DisableExtension makes Basic FINAL. FINAL indicates the ACL non-extendability
// in the related container.
//
// See also Extendable.
func (x *Basic) DisableExtension() {
setBit((*uint32)(x), bitPosFinal)
}
// Extendable checks if Basic is NOT made FINAL using DisableExtension.
//
// Zero Basic is extendable.
func (x Basic) Extendable() bool {
return !isBitSet(uint32(x), bitPosFinal)
}
// MakeSticky makes Basic STICKY. STICKY indicates that only the owner of any
// particular object is allowed to operate on it.
//
// See also Sticky.
func (x *Basic) MakeSticky() {
setBit((*uint32)(x), bitPosSticky)
}
// Sticky checks if Basic is made STICKY using MakeSticky.
//
// Zero Basic is NOT STICKY.
func (x Basic) Sticky() bool {
return isBitSet(uint32(x), bitPosSticky)
}
// checks if op is used by the storage nodes within replication mechanism.
func isReplicationOp(op Op) bool {
switch op {
default:
return false
case
OpObjectGet,
OpObjectHead,
OpObjectPut,
OpObjectSearch,
OpObjectHash:
return true
}
}
// AllowOp allows the parties with the given role to the given operation.
// Op MUST be one of the Op enumeration. Role MUST be one of:
//
// RoleOwner
// RoleContainer
// RoleOthers
//
// and if role is RoleContainer, op MUST NOT be:
//
// OpObjectGet
// OpObjectHead
// OpObjectPut
// OpObjectSearch
// OpObjectHash
//
// See also IsOpAllowed.
func (x *Basic) AllowOp(op Op, role Role) {
var bitPos uint8
switch role {
default:
panic(fmt.Sprintf("unable to set rules for unsupported role %v", role))
case RoleInnerRing:
panic("basic ACL MUST NOT be modified for Inner Ring")
case RoleOwner:
bitPos = opBitPosOwner
case RoleContainer:
if isReplicationOp(op) {
panic("basic ACL for container replication ops MUST NOT be modified")
}
bitPos = opBitPosContainer
case RoleOthers:
bitPos = opBitPosOthers
}
setOpBit((*uint32)(x), op, bitPos)
}
// IsOpAllowed checks if parties with the given role are allowed to the given op
// according to the Basic rules. Op MUST be one of the Op enumeration.
// Role MUST be one of the Role enumeration.
//
// Members with RoleContainer role have exclusive default access to the
// operations of the data replication mechanism:
//
// OpObjectGet
// OpObjectHead
// OpObjectPut
// OpObjectSearch
// OpObjectHash
//
// RoleInnerRing members are allowed to data audit ops only:
//
// OpObjectGet
// OpObjectHead
// OpObjectHash
// OpObjectSearch
//
// Zero Basic prevents any role from accessing any operation in the absence
// of default rights.
//
// See also AllowOp.
func (x Basic) IsOpAllowed(op Op, role Role) bool {
var bitPos uint8
switch role {
default:
panic(fmt.Sprintf("role is unsupported %v", role))
case RoleInnerRing:
switch op {
case
OpObjectGet,
OpObjectHead,
OpObjectHash,
OpObjectSearch:
return true
default:
return false
}
case RoleOwner:
bitPos = opBitPosOwner
case RoleContainer:
if isReplicationOp(op) {
return true
}
bitPos = opBitPosContainer
case RoleOthers:
bitPos = opBitPosOthers
}
return isOpBitSet(uint32(x), op, bitPos)
}
// AllowBearerRules allows bearer to provide extended ACL rules for the given
// operation. Bearer rules doesn't depend on container ACL extensibility.
//
// See also AllowedBearerRules.
func (x *Basic) AllowBearerRules(op Op) {
setOpBit((*uint32)(x), op, opBitPosBearer)
}
// AllowedBearerRules checks if bearer rules are allowed using AllowBearerRules.
// Op MUST be one of the Op enumeration.
//
// Zero Basic disallows bearer rules for any op.
func (x Basic) AllowedBearerRules(op Op) bool {
return isOpBitSet(uint32(x), op, opBitPosBearer)
}
// EncodeToString encodes Basic into hexadecimal string.
//
// See also DecodeString.
func (x Basic) EncodeToString() string {
return strconv.FormatUint(uint64(x), 16)
}
// Names of the frequently used Basic values.
const (
NamePrivate = "private"
NamePrivateExtended = "eacl-private"
NamePublicRO = "public-read"
NamePublicROExtended = "eacl-public-read"
NamePublicRW = "public-read-write"
NamePublicRWExtended = "eacl-public-read-write"
NamePublicAppend = "public-append"
NamePublicAppendExtended = "eacl-public-append"
)
// Frequently used Basic values. Bitmasks are taken from the FrostFS Specification.
const (
Private = Basic(0x1C8C8CCC) // private
PrivateExtended = Basic(0x0C8C8CCC) // eacl-private
PublicRO = Basic(0x1FBF8CFF) // public-read
PublicROExtended = Basic(0x0FBF8CFF) // eacl-public-read
PublicRW = Basic(0x1FBFBFFF) // public-read-write
PublicRWExtended = Basic(0x0FBFBFFF) // eacl-public-read-write
PublicAppend = Basic(0x1FBF9FFF) // public-append
PublicAppendExtended = Basic(0x0FBF9FFF) // eacl-public-append
)
// DecodeString decodes string calculated using EncodeToString. Also supports
// human-readable names (Name* constants).
func (x *Basic) DecodeString(s string) (e error) {
switch s {
case NamePrivate:
*x = Private
case NamePrivateExtended:
*x = PrivateExtended
case NamePublicRO:
*x = PublicRO
case NamePublicROExtended:
*x = PublicROExtended
case NamePublicRW:
*x = PublicRW
case NamePublicRWExtended:
*x = PublicRWExtended
case NamePublicAppend:
*x = PublicAppend
case NamePublicAppendExtended:
*x = PublicAppendExtended
default:
s = strings.TrimPrefix(strings.ToLower(s), "0x")
v, err := strconv.ParseUint(s, 16, 32)
if err != nil {
return fmt.Errorf("parse hex: %w", err)
}
*x = Basic(v)
}
return nil
}

View file

@ -0,0 +1,384 @@
package acl
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestBasic_DisableExtension(t *testing.T) {
var val, val2 Basic
require.True(t, val.Extendable())
val2.FromBits(val.Bits())
require.True(t, val2.Extendable())
val.DisableExtension()
require.False(t, val.Extendable())
val2.FromBits(val.Bits())
require.False(t, val2.Extendable())
}
func TestBasic_MakeSticky(t *testing.T) {
var val, val2 Basic
require.False(t, val.Sticky())
val2.FromBits(val.Bits())
require.False(t, val2.Sticky())
val.MakeSticky()
require.True(t, val.Sticky())
val2.FromBits(val.Bits())
require.True(t, val2.Sticky())
}
func TestBasic_AllowBearerRules(t *testing.T) {
var val Basic
require.Panics(t, func() { val.AllowBearerRules(opZero) })
require.Panics(t, func() { val.AllowBearerRules(opLast) })
require.Panics(t, func() { val.AllowedBearerRules(opZero) })
require.Panics(t, func() { val.AllowedBearerRules(opLast) })
for op := opZero + 1; op < opLast; op++ {
val := val
require.False(t, val.AllowedBearerRules(op))
val.AllowBearerRules(op)
for j := opZero + 1; j < opLast; j++ {
if j == op {
require.True(t, val.AllowedBearerRules(j), op)
} else {
require.False(t, val.AllowedBearerRules(j), op)
}
}
}
}
func TestBasic_AllowOp(t *testing.T) {
var val, val2 Basic
require.Panics(t, func() { val.IsOpAllowed(opZero, roleZero+1) })
require.Panics(t, func() { val.IsOpAllowed(opLast, roleZero+1) })
require.Panics(t, func() { val.IsOpAllowed(opZero+1, roleZero) })
require.Panics(t, func() { val.IsOpAllowed(opZero+1, roleLast) })
for op := opZero + 1; op < opLast; op++ {
require.Panics(t, func() { val.AllowOp(op, RoleInnerRing) })
if isReplicationOp(op) {
require.Panics(t, func() { val.AllowOp(op, RoleContainer) })
require.True(t, val.IsOpAllowed(op, RoleContainer))
}
}
require.True(t, val.IsOpAllowed(OpObjectGet, RoleInnerRing))
require.True(t, val.IsOpAllowed(OpObjectHead, RoleInnerRing))
require.True(t, val.IsOpAllowed(OpObjectSearch, RoleInnerRing))
require.True(t, val.IsOpAllowed(OpObjectHash, RoleInnerRing))
const op = opZero + 1
const role = RoleOthers
require.False(t, val.IsOpAllowed(op, role))
val2.FromBits(val.Bits())
require.False(t, val2.IsOpAllowed(op, role))
val.AllowOp(op, role)
require.True(t, val.IsOpAllowed(op, role))
val2.FromBits(val.Bits())
require.True(t, val2.IsOpAllowed(op, role))
}
type opsExpected struct {
owner, container, innerRing, others, bearer bool
}
func testOp(t *testing.T, v Basic, op Op, exp opsExpected) {
require.Equal(t, exp.owner, v.IsOpAllowed(op, RoleOwner), op)
require.Equal(t, exp.container, v.IsOpAllowed(op, RoleContainer), op)
require.Equal(t, exp.innerRing, v.IsOpAllowed(op, RoleInnerRing), op)
require.Equal(t, exp.others, v.IsOpAllowed(op, RoleOthers), op)
require.Equal(t, exp.bearer, v.AllowedBearerRules(op), op)
}
type expected struct {
extendable, sticky bool
mOps map[Op]opsExpected
}
func testBasicPredefined(t *testing.T, val Basic, name string, exp expected) {
require.Equal(t, exp.sticky, val.Sticky())
require.Equal(t, exp.extendable, val.Extendable())
for op, exp := range exp.mOps {
testOp(t, val, op, exp)
}
s := val.EncodeToString()
var val2 Basic
require.NoError(t, val2.DecodeString(s))
require.Equal(t, val, val2)
require.NoError(t, val2.DecodeString(name))
require.Equal(t, val, val2)
}
func TestBasicPredefined(t *testing.T) {
t.Run("private", func(t *testing.T) {
exp := expected{
extendable: false,
sticky: false,
mOps: map[Op]opsExpected{
OpObjectHash: {
owner: true,
container: true,
innerRing: true,
others: false,
bearer: false,
},
OpObjectRange: {
owner: true,
container: false,
innerRing: false,
others: false,
bearer: false,
},
OpObjectSearch: {
owner: true,
container: true,
innerRing: true,
others: false,
bearer: false,
},
OpObjectDelete: {
owner: true,
container: false,
innerRing: false,
others: false,
bearer: false,
},
OpObjectPut: {
owner: true,
container: true,
innerRing: false,
others: false,
bearer: false,
},
OpObjectHead: {
owner: true,
container: true,
innerRing: true,
others: false,
bearer: false,
},
OpObjectGet: {
owner: true,
container: true,
innerRing: true,
others: false,
bearer: false,
},
},
}
testBasicPredefined(t, Private, NamePrivate, exp)
exp.extendable = true
testBasicPredefined(t, PrivateExtended, NamePrivateExtended, exp)
})
t.Run("public-read", func(t *testing.T) {
exp := expected{
extendable: false,
sticky: false,
mOps: map[Op]opsExpected{
OpObjectHash: {
owner: true,
container: true,
innerRing: true,
others: true,
bearer: true,
},
OpObjectRange: {
owner: true,
container: false,
innerRing: false,
others: true,
bearer: true,
},
OpObjectSearch: {
owner: true,
container: true,
innerRing: true,
others: true,
bearer: true,
},
OpObjectDelete: {
owner: true,
container: false,
innerRing: false,
others: false,
bearer: false,
},
OpObjectPut: {
owner: true,
container: true,
innerRing: false,
others: false,
bearer: false,
},
OpObjectHead: {
owner: true,
container: true,
innerRing: true,
others: true,
bearer: true,
},
OpObjectGet: {
owner: true,
container: true,
innerRing: true,
others: true,
bearer: true,
},
},
}
testBasicPredefined(t, PublicRO, NamePublicRO, exp)
exp.extendable = true
testBasicPredefined(t, PublicROExtended, NamePublicROExtended, exp)
})
t.Run("public-read-write", func(t *testing.T) {
exp := expected{
extendable: false,
sticky: false,
mOps: map[Op]opsExpected{
OpObjectHash: {
owner: true,
container: true,
innerRing: true,
others: true,
bearer: true,
},
OpObjectRange: {
owner: true,
container: false,
innerRing: false,
others: true,
bearer: true,
},
OpObjectSearch: {
owner: true,
container: true,
innerRing: true,
others: true,
bearer: true,
},
OpObjectDelete: {
owner: true,
container: false,
innerRing: false,
others: true,
bearer: true,
},
OpObjectPut: {
owner: true,
container: true,
innerRing: false,
others: true,
bearer: true,
},
OpObjectHead: {
owner: true,
container: true,
innerRing: true,
others: true,
bearer: true,
},
OpObjectGet: {
owner: true,
container: true,
innerRing: true,
others: true,
bearer: true,
},
},
}
testBasicPredefined(t, PublicRW, NamePublicRW, exp)
exp.extendable = true
testBasicPredefined(t, PublicRWExtended, NamePublicRWExtended, exp)
})
t.Run("public-append", func(t *testing.T) {
exp := expected{
extendable: false,
sticky: false,
mOps: map[Op]opsExpected{
OpObjectHash: {
owner: true,
container: true,
innerRing: true,
others: true,
bearer: true,
},
OpObjectRange: {
owner: true,
container: false,
innerRing: false,
others: true,
bearer: true,
},
OpObjectSearch: {
owner: true,
container: true,
innerRing: true,
others: true,
bearer: true,
},
OpObjectDelete: {
owner: true,
container: false,
innerRing: false,
others: false,
bearer: true,
},
OpObjectPut: {
owner: true,
container: true,
innerRing: false,
others: true,
bearer: true,
},
OpObjectHead: {
owner: true,
container: true,
innerRing: true,
others: true,
bearer: true,
},
OpObjectGet: {
owner: true,
container: true,
innerRing: true,
others: true,
bearer: true,
},
},
}
testBasicPredefined(t, PublicAppend, NamePublicAppend, exp)
exp.extendable = true
testBasicPredefined(t, PublicAppendExtended, NamePublicAppendExtended, exp)
})
}

8
container/acl/doc.go Normal file
View file

@ -0,0 +1,8 @@
/*
Package acl provides functionality to control access to data and operations on them in FrostFS containers.
Type Basic represents basic ACL of the FrostFS container which specifies the general order of data access.
Basic provides interface of rule composition. Package acl also exports some frequently used settings like
private or public.
*/
package acl

20
container/acl/init.go Normal file
View file

@ -0,0 +1,20 @@
package acl
func init() {
// left-to-right order of the object operations
orderedOps := [...]Op{
OpObjectGet,
OpObjectHead,
OpObjectPut,
OpObjectDelete,
OpObjectSearch,
OpObjectRange,
OpObjectHash,
}
mOrder = make(map[Op]uint8, len(orderedOps))
for i := range orderedOps {
mOrder[orderedOps[i]] = uint8(i)
}
}

37
container/acl/util.go Normal file
View file

@ -0,0 +1,37 @@
package acl
import "fmt"
// sets n-th bit in num (starting at 0).
func setBit(num *uint32, n uint8) {
*num |= 1 << n
}
// checks if n-th bit in num is set (starting at 0).
func isBitSet(num uint32, n uint8) bool {
mask := uint32(1 << n)
return mask != 0 && num&mask == mask
}
// maps Op to op-section index in Basic. Filled on init.
var mOrder map[Op]uint8
// sets n-th bit in num for the given op. Panics if op is unsupported.
func setOpBit(num *uint32, op Op, opBitPos uint8) {
n, ok := mOrder[op]
if !ok {
panic(fmt.Sprintf("op is unsupported %v", op))
}
setBit(num, n*bitsPerOp+opBitPos)
}
// checks if n-th bit in num for the given op is set. Panics if op is unsupported.
func isOpBitSet(num uint32, op Op, n uint8) bool {
off, ok := mOrder[op]
if !ok {
panic(fmt.Sprintf("op is unsupported %v", op))
}
return isBitSet(num, bitsPerOp*off+n)
}

130
container/acl/util_test.go Normal file
View file

@ -0,0 +1,130 @@
package acl
import (
"fmt"
"testing"
"github.com/stretchr/testify/require"
)
func TestBits(t *testing.T) {
num := uint32(0b10110)
require.False(t, isBitSet(num, 0))
require.True(t, isBitSet(num, 1))
require.True(t, isBitSet(num, 2))
require.False(t, isBitSet(num, 3))
require.True(t, isBitSet(num, 4))
require.False(t, isBitSet(num, 5))
setBit(&num, 3)
require.EqualValues(t, 0b11110, num)
setBit(&num, 6)
require.EqualValues(t, 0b1011110, num)
}
func TestOpBits(t *testing.T) {
num := uint32(0b_1001_0101_1100_0011_0110_0111_1000_1111)
require.Panics(t, func() { isOpBitSet(num, opZero, 0) })
require.Panics(t, func() { isOpBitSet(num, opLast, 0) })
cpNum := num
require.Panics(t, func() { setOpBit(&num, opZero, 0) })
require.EqualValues(t, cpNum, num)
require.Panics(t, func() { setOpBit(&num, opLast, 0) })
require.EqualValues(t, cpNum, num)
for _, tc := range []struct {
op Op
set [4]bool // is bit set (left-to-right)
bits [4]uint32 // result of setting i-th bit (left-to-right) to zero num
}{
{
op: OpObjectHash,
set: [4]bool{false, true, false, true},
bits: [4]uint32{
0b_0000_1000_0000_0000_0000_0000_0000_0000,
0b_0000_0100_0000_0000_0000_0000_0000_0000,
0b_0000_0010_0000_0000_0000_0000_0000_0000,
0b_0000_0001_0000_0000_0000_0000_0000_0000,
},
},
{
op: OpObjectRange,
set: [4]bool{true, true, false, false},
bits: [4]uint32{
0b_0000_0000_1000_0000_0000_0000_0000_0000,
0b_0000_0000_0100_0000_0000_0000_0000_0000,
0b_0000_0000_0010_0000_0000_0000_0000_0000,
0b_0000_0000_0001_0000_0000_0000_0000_0000,
},
},
{
op: OpObjectSearch,
set: [4]bool{false, false, true, true},
bits: [4]uint32{
0b_0000_0000_0000_1000_0000_0000_0000_0000,
0b_0000_0000_0000_0100_0000_0000_0000_0000,
0b_0000_0000_0000_0010_0000_0000_0000_0000,
0b_0000_0000_0000_0001_0000_0000_0000_0000,
},
},
{
op: OpObjectDelete,
set: [4]bool{false, true, true, false},
bits: [4]uint32{
0b_0000_0000_0000_0000_1000_0000_0000_0000,
0b_0000_0000_0000_0000_0100_0000_0000_0000,
0b_0000_0000_0000_0000_0010_0000_0000_0000,
0b_0000_0000_0000_0000_0001_0000_0000_0000,
},
},
{
op: OpObjectPut,
set: [4]bool{false, true, true, true},
bits: [4]uint32{
0b_0000_0000_0000_0000_0000_1000_0000_0000,
0b_0000_0000_0000_0000_0000_0100_0000_0000,
0b_0000_0000_0000_0000_0000_0010_0000_0000,
0b_0000_0000_0000_0000_0000_0001_0000_0000,
},
},
{
op: OpObjectHead,
set: [4]bool{true, false, false, false},
bits: [4]uint32{
0b_0000_0000_0000_0000_0000_0000_1000_0000,
0b_0000_0000_0000_0000_0000_0000_0100_0000,
0b_0000_0000_0000_0000_0000_0000_0010_0000,
0b_0000_0000_0000_0000_0000_0000_0001_0000,
},
},
{
op: OpObjectGet,
set: [4]bool{true, true, true, true},
bits: [4]uint32{
0b_0000_0000_0000_0000_0000_0000_0000_1000,
0b_0000_0000_0000_0000_0000_0000_0000_0100,
0b_0000_0000_0000_0000_0000_0000_0000_0010,
0b_0000_0000_0000_0000_0000_0000_0000_0001,
},
},
} {
for i := range tc.set {
require.EqualValues(t, tc.set[i], isOpBitSet(num, tc.op, uint8(len(tc.set)-1-i)),
fmt.Sprintf("op %s, left bit #%d", tc.op, i),
)
}
for i := range tc.bits {
num := uint32(0)
setOpBit(&num, tc.op, uint8(len(tc.bits)-1-i))
require.EqualValues(t, tc.bits[i], num)
}
}
}

View file

@ -1,77 +0,0 @@
package container
import (
"github.com/nspcc-dev/neofs-api-go/v2/container"
cid "github.com/nspcc-dev/neofs-sdk-go/container/id"
)
// UsedSpaceAnnouncement is an announcement message used by storage nodes to
// estimate actual container sizes.
type UsedSpaceAnnouncement container.UsedSpaceAnnouncement
// NewAnnouncement initialize empty UsedSpaceAnnouncement message.
//
// Defaults:
// - epoch: 0;
// - usedSpace: 0;
// - cid: nil.
func NewAnnouncement() *UsedSpaceAnnouncement {
return NewAnnouncementFromV2(new(container.UsedSpaceAnnouncement))
}
// NewAnnouncementFromV2 wraps protocol dependent version of
// UsedSpaceAnnouncement message.
//
// Nil container.UsedSpaceAnnouncement converts to nil.
func NewAnnouncementFromV2(v *container.UsedSpaceAnnouncement) *UsedSpaceAnnouncement {
return (*UsedSpaceAnnouncement)(v)
}
// Epoch of the announcement.
func (a *UsedSpaceAnnouncement) Epoch() uint64 {
return (*container.UsedSpaceAnnouncement)(a).GetEpoch()
}
// SetEpoch sets announcement epoch value.
func (a *UsedSpaceAnnouncement) SetEpoch(epoch uint64) {
(*container.UsedSpaceAnnouncement)(a).SetEpoch(epoch)
}
// ContainerID of the announcement.
func (a *UsedSpaceAnnouncement) ContainerID() *cid.ID {
return cid.NewFromV2(
(*container.UsedSpaceAnnouncement)(a).GetContainerID(),
)
}
// SetContainerID sets announcement container value.
func (a *UsedSpaceAnnouncement) SetContainerID(cid *cid.ID) {
(*container.UsedSpaceAnnouncement)(a).SetContainerID(cid.ToV2())
}
// UsedSpace in container.
func (a *UsedSpaceAnnouncement) UsedSpace() uint64 {
return (*container.UsedSpaceAnnouncement)(a).GetUsedSpace()
}
// SetUsedSpace sets used space value by specified container.
func (a *UsedSpaceAnnouncement) SetUsedSpace(value uint64) {
(*container.UsedSpaceAnnouncement)(a).SetUsedSpace(value)
}
// ToV2 returns protocol dependent version of UsedSpaceAnnouncement message.
//
// Nil UsedSpaceAnnouncement converts to nil.
func (a *UsedSpaceAnnouncement) ToV2() *container.UsedSpaceAnnouncement {
return (*container.UsedSpaceAnnouncement)(a)
}
// Marshal marshals UsedSpaceAnnouncement into a protobuf binary form.
func (a *UsedSpaceAnnouncement) Marshal() ([]byte, error) {
return a.ToV2().StableMarshal(nil)
}
// Unmarshal unmarshals protobuf binary representation of UsedSpaceAnnouncement.
func (a *UsedSpaceAnnouncement) Unmarshal(data []byte) error {
return a.ToV2().Unmarshal(data)
}

View file

@ -1,99 +0,0 @@
package container_test
import (
"crypto/sha256"
"testing"
containerv2 "github.com/nspcc-dev/neofs-api-go/v2/container"
"github.com/nspcc-dev/neofs-api-go/v2/refs"
"github.com/nspcc-dev/neofs-sdk-go/container"
cid "github.com/nspcc-dev/neofs-sdk-go/container/id"
cidtest "github.com/nspcc-dev/neofs-sdk-go/container/id/test"
containertest "github.com/nspcc-dev/neofs-sdk-go/container/test"
"github.com/stretchr/testify/require"
)
func TestAnnouncement(t *testing.T) {
const epoch, usedSpace uint64 = 10, 100
cidValue := [sha256.Size]byte{1, 2, 3}
id := cidtest.IDWithChecksum(cidValue)
a := container.NewAnnouncement()
a.SetEpoch(epoch)
a.SetContainerID(id)
a.SetUsedSpace(usedSpace)
require.Equal(t, epoch, a.Epoch())
require.Equal(t, usedSpace, a.UsedSpace())
require.Equal(t, id, a.ContainerID())
t.Run("test v2", func(t *testing.T) {
const newEpoch, newUsedSpace uint64 = 20, 200
newCidValue := [32]byte{4, 5, 6}
newCID := new(refs.ContainerID)
newCID.SetValue(newCidValue[:])
v2 := a.ToV2()
require.Equal(t, usedSpace, v2.GetUsedSpace())
require.Equal(t, epoch, v2.GetEpoch())
require.Equal(t, cidValue[:], v2.GetContainerID().GetValue())
v2.SetEpoch(newEpoch)
v2.SetUsedSpace(newUsedSpace)
v2.SetContainerID(newCID)
newA := container.NewAnnouncementFromV2(v2)
require.Equal(t, newEpoch, newA.Epoch())
require.Equal(t, newUsedSpace, newA.UsedSpace())
require.Equal(t, cid.NewFromV2(newCID), newA.ContainerID())
})
}
func TestUsedSpaceEncoding(t *testing.T) {
a := containertest.UsedSpaceAnnouncement()
t.Run("binary", func(t *testing.T) {
data, err := a.Marshal()
require.NoError(t, err)
a2 := container.NewAnnouncement()
require.NoError(t, a2.Unmarshal(data))
require.Equal(t, a, a2)
})
}
func TestUsedSpaceAnnouncement_ToV2(t *testing.T) {
t.Run("nil", func(t *testing.T) {
var x *container.UsedSpaceAnnouncement
require.Nil(t, x.ToV2())
})
t.Run("default values", func(t *testing.T) {
announcement := container.NewAnnouncement()
// check initial values
require.Zero(t, announcement.Epoch())
require.Zero(t, announcement.UsedSpace())
require.Nil(t, announcement.ContainerID())
// convert to v2 message
announcementV2 := announcement.ToV2()
require.Zero(t, announcementV2.GetEpoch())
require.Zero(t, announcementV2.GetUsedSpace())
require.Nil(t, announcementV2.GetContainerID())
})
}
func TestNewAnnouncementFromV2(t *testing.T) {
t.Run("from nil", func(t *testing.T) {
var x *containerv2.UsedSpaceAnnouncement
require.Nil(t, container.NewAnnouncementFromV2(x))
})
}

View file

@ -1,139 +0,0 @@
package container
import (
"github.com/nspcc-dev/neofs-api-go/v2/container"
)
type (
Attribute container.Attribute
Attributes []Attribute
)
// NewAttribute creates and initializes blank Attribute.
//
// Defaults:
// - key: "";
// - value: "".
func NewAttribute() *Attribute {
return NewAttributeFromV2(new(container.Attribute))
}
func (a *Attribute) SetKey(v string) {
(*container.Attribute)(a).SetKey(v)
}
func (a *Attribute) SetValue(v string) {
(*container.Attribute)(a).SetValue(v)
}
func (a *Attribute) Key() string {
return (*container.Attribute)(a).GetKey()
}
func (a *Attribute) Value() string {
return (*container.Attribute)(a).GetValue()
}
// NewAttributeFromV2 wraps protocol dependent version of
// Attribute message.
//
// Nil container.Attribute converts to nil.
func NewAttributeFromV2(v *container.Attribute) *Attribute {
return (*Attribute)(v)
}
// ToV2 converts Attribute to v2 Attribute message.
//
// Nil Attribute converts to nil.
func (a *Attribute) ToV2() *container.Attribute {
return (*container.Attribute)(a)
}
func NewAttributesFromV2(v []container.Attribute) Attributes {
if v == nil {
return nil
}
attrs := make(Attributes, len(v))
for i := range v {
attrs[i] = *NewAttributeFromV2(&v[i])
}
return attrs
}
func (a Attributes) ToV2() []container.Attribute {
if a == nil {
return nil
}
attrs := make([]container.Attribute, len(a))
for i := range a {
attrs[i] = *a[i].ToV2()
}
return attrs
}
// sets value of the attribute by key.
func setAttribute(c *Container, key, value string) {
attrs := c.Attributes()
found := false
for i := range attrs {
if attrs[i].Key() == key {
attrs[i].SetValue(value)
found = true
break
}
}
if !found {
index := len(attrs)
attrs = append(attrs, Attribute{})
attrs[index].SetKey(key)
attrs[index].SetValue(value)
}
c.SetAttributes(attrs)
}
// iterates over container attributes. Stops at f's true return.
//
// Handler must not be nil.
func iterateAttributes(c *Container, f func(*Attribute) bool) {
attrs := c.Attributes()
for i := range attrs {
if f(&attrs[i]) {
return
}
}
}
// SetNativeNameWithZone sets container native name and its zone.
//
// Use SetNativeName to set default zone.
func SetNativeNameWithZone(c *Container, name, zone string) {
setAttribute(c, container.SysAttributeName, name)
setAttribute(c, container.SysAttributeZone, zone)
}
// SetNativeName sets container native name with default zone (container).
func SetNativeName(c *Container, name string) {
SetNativeNameWithZone(c, name, container.SysAttributeZoneDefault)
}
// GetNativeNameWithZone returns container native name and its zone.
func GetNativeNameWithZone(c *Container) (name string, zone string) {
iterateAttributes(c, func(a *Attribute) bool {
if key := a.Key(); key == container.SysAttributeName {
name = a.Value()
} else if key == container.SysAttributeZone {
zone = a.Value()
}
return name != "" && zone != ""
})
return
}

View file

@ -1,152 +0,0 @@
package container_test
import (
"testing"
containerv2 "github.com/nspcc-dev/neofs-api-go/v2/container"
"github.com/nspcc-dev/neofs-sdk-go/container"
"github.com/stretchr/testify/require"
)
func TestAttribute(t *testing.T) {
t.Run("nil", func(t *testing.T) {
var x *container.Attribute
require.Nil(t, x.ToV2())
})
t.Run("default values", func(t *testing.T) {
attr := container.NewAttribute()
// check initial values
require.Empty(t, attr.Key())
require.Empty(t, attr.Value())
// convert to v2 message
attrV2 := attr.ToV2()
require.Empty(t, attrV2.GetKey())
require.Empty(t, attrV2.GetValue())
})
const (
key = "key"
value = "value"
)
attr := container.NewAttribute()
attr.SetKey(key)
attr.SetValue(value)
require.Equal(t, key, attr.Key())
require.Equal(t, value, attr.Value())
t.Run("test v2", func(t *testing.T) {
const (
newKey = "newKey"
newValue = "newValue"
)
v2 := attr.ToV2()
require.Equal(t, key, v2.GetKey())
require.Equal(t, value, v2.GetValue())
v2.SetKey(newKey)
v2.SetValue(newValue)
newAttr := container.NewAttributeFromV2(v2)
require.Equal(t, newKey, newAttr.Key())
require.Equal(t, newValue, newAttr.Value())
})
}
func TestAttributes(t *testing.T) {
t.Run("nil", func(t *testing.T) {
var x container.Attributes
require.Nil(t, x.ToV2())
require.Nil(t, container.NewAttributesFromV2(nil))
})
var (
keys = []string{"key1", "key2", "key3"}
vals = []string{"val1", "val2", "val3"}
)
attrs := make(container.Attributes, len(keys))
for i := range keys {
attrs[i].SetKey(keys[i])
attrs[i].SetValue(vals[i])
}
t.Run("test v2", func(t *testing.T) {
const postfix = "x"
v2 := attrs.ToV2()
require.Len(t, v2, len(keys))
for i := range v2 {
k := v2[i].GetKey()
v := v2[i].GetValue()
require.Equal(t, keys[i], k)
require.Equal(t, vals[i], v)
v2[i].SetKey(k + postfix)
v2[i].SetValue(v + postfix)
}
newAttrs := container.NewAttributesFromV2(v2)
require.Len(t, newAttrs, len(keys))
for i := range newAttrs {
require.Equal(t, keys[i]+postfix, newAttrs[i].Key())
require.Equal(t, vals[i]+postfix, newAttrs[i].Value())
}
})
}
func TestNewAttributeFromV2(t *testing.T) {
t.Run("from nil", func(t *testing.T) {
var x *containerv2.Attribute
require.Nil(t, container.NewAttributeFromV2(x))
})
}
func TestGetNameWithZone(t *testing.T) {
c := container.New()
for _, item := range [...]struct {
name, zone string
}{
{"name1", ""},
{"name1", "zone1"},
{"name2", "zone1"},
{"name2", "zone2"},
{"", "zone2"},
{"", ""},
} {
container.SetNativeNameWithZone(c, item.name, item.zone)
name, zone := container.GetNativeNameWithZone(c)
require.Equal(t, item.name, name, item.name)
require.Equal(t, item.zone, zone, item.zone)
}
}
func TestSetNativeName(t *testing.T) {
c := container.New()
const nameDefZone = "some name"
container.SetNativeName(c, nameDefZone)
name, zone := container.GetNativeNameWithZone(c)
require.Equal(t, nameDefZone, name)
require.Equal(t, containerv2.SysAttributeZoneDefault, zone)
}

View file

@ -1,193 +1,521 @@
package container
import (
"crypto/ecdsa"
"crypto/sha256"
"errors"
"fmt"
"strconv"
"strings"
"time"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
v2netmap "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/acl"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
frostfsecdsa "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto/ecdsa"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/version"
"github.com/google/uuid"
"github.com/nspcc-dev/neofs-api-go/v2/container"
"github.com/nspcc-dev/neofs-sdk-go/acl"
cid "github.com/nspcc-dev/neofs-sdk-go/container/id"
"github.com/nspcc-dev/neofs-sdk-go/netmap"
"github.com/nspcc-dev/neofs-sdk-go/owner"
"github.com/nspcc-dev/neofs-sdk-go/session"
"github.com/nspcc-dev/neofs-sdk-go/signature"
"github.com/nspcc-dev/neofs-sdk-go/version"
)
// Container represents descriptor of the FrostFS container. Container logically
// stores FrostFS objects. Container is one of the basic and at the same time
// necessary data storage units in the FrostFS. Container includes data about the
// owner, rules for placing objects and other information necessary for the
// system functioning.
//
// Container type instances can represent different container states in the
// system, depending on the context. To create new container in FrostFS zero
// instance SHOULD be declared, initialized using Init method and filled using
// dedicated methods. Once container is saved in the FrostFS network, it can't be
// changed: containers stored in the system are immutable, and FrostFS is a CAS
// of containers that are identified by a fixed length value (see cid.ID type).
// Instances for existing containers can be initialized using decoding methods
// (e.g Unmarshal).
//
// Container is mutually compatible with git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container.Container
// message. See ReadFromV2 / WriteToV2 methods.
type Container struct {
v2 container.Container
token *session.Token
sig *signature.Signature
}
// New creates, initializes and returns blank Container instance.
//
// Defaults:
// - token: nil;
// - sig: nil;
// - basicACL: acl.PrivateBasicRule;
// - version: version.Current;
// - nonce: random UUID;
// - attr: nil;
// - policy: nil;
// - ownerID: nil.
func New(opts ...Option) *Container {
cnrOptions := defaultContainerOptions()
const (
attributeName = "Name"
attributeTimestamp = "Timestamp"
)
for i := range opts {
opts[i](&cnrOptions)
// reads Container from the container.Container message. If checkFieldPresence is set,
// returns an error on absence of any protocol-required field.
func (x *Container) readFromV2(m container.Container, checkFieldPresence bool) error {
var err error
ownerV2 := m.GetOwnerID()
if ownerV2 != nil {
var owner user.ID
err = owner.ReadFromV2(*ownerV2)
if err != nil {
return fmt.Errorf("invalid owner: %w", err)
}
} else if checkFieldPresence {
return errors.New("missing owner")
}
cnr := new(Container)
cnr.SetNonceUUID(cnrOptions.nonce)
cnr.SetBasicACL(cnrOptions.acl)
binNonce := m.GetNonce()
if len(binNonce) > 0 {
var nonce uuid.UUID
if cnrOptions.owner != nil {
cnr.SetOwnerID(cnrOptions.owner)
err = nonce.UnmarshalBinary(binNonce)
if err != nil {
return fmt.Errorf("invalid nonce: %w", err)
} else if ver := nonce.Version(); ver != 4 {
return fmt.Errorf("invalid nonce UUID version %d", ver)
}
} else if checkFieldPresence {
return errors.New("missing nonce")
}
if cnrOptions.policy != nil {
cnr.SetPlacementPolicy(cnrOptions.policy)
ver := m.GetVersion()
if checkFieldPresence && ver == nil {
return errors.New("missing version")
}
cnr.SetAttributes(cnrOptions.attributes)
cnr.SetVersion(version.Current())
policyV2 := m.GetPlacementPolicy()
if policyV2 != nil {
var policy netmap.PlacementPolicy
return cnr
err = policy.ReadFromV2(*policyV2)
if err != nil {
return fmt.Errorf("invalid placement policy: %w", err)
}
} else if checkFieldPresence {
return errors.New("missing placement policy")
}
// ToV2 returns the v2 Container message.
//
// Nil Container converts to nil.
func (c *Container) ToV2() *container.Container {
if c == nil {
if err := checkAttributes(m); err != nil {
return err
}
x.v2 = m
return nil
}
return &c.v2
func checkAttributes(m container.Container) error {
attrs := m.GetAttributes()
mAttr := make(map[string]struct{}, len(attrs))
var key, val string
var was bool
for i := range attrs {
key = attrs[i].GetKey()
if key == "" {
return errors.New("empty attribute key")
}
// NewVerifiedFromV2 constructs Container from NeoFS API V2 Container message.
//
// Does not perform if message meets NeoFS API V2 specification. To do this
// use NewVerifiedFromV2 constructor.
func NewContainerFromV2(c *container.Container) *Container {
cnr := new(Container)
if c != nil {
cnr.v2 = *c
_, was = mAttr[key]
if was {
return fmt.Errorf("duplicated attribute %s", key)
}
return cnr
val = attrs[i].GetValue()
if val == "" {
return fmt.Errorf("empty attribute value %s", key)
}
var err error
if key == attributeTimestamp {
_, err = strconv.ParseInt(val, 10, 64)
}
// CalculateID calculates container identifier
// based on its structure.
func CalculateID(c *Container) *cid.ID {
data, err := c.ToV2().StableMarshal(nil)
if err != nil {
panic(err)
return fmt.Errorf("invalid attribute value %s: %s (%w)", key, val, err)
}
id := cid.New()
id.SetSHA256(sha256.Sum256(data))
return id
mAttr[key] = struct{}{}
}
return nil
}
func (c *Container) Version() *version.Version {
return version.NewFromV2(c.v2.GetVersion())
}
func (c *Container) SetVersion(v *version.Version) {
c.v2.SetVersion(v.ToV2())
}
func (c *Container) OwnerID() *owner.ID {
return owner.NewIDFromV2(c.v2.GetOwnerID())
}
func (c *Container) SetOwnerID(v *owner.ID) {
c.v2.SetOwnerID(v.ToV2())
}
// Returns container nonce in UUID format.
// ReadFromV2 reads Container from the container.Container message. Checks if the
// message conforms to FrostFS API V2 protocol.
//
// Returns error if container nonce is not a valid UUID.
func (c *Container) NonceUUID() (uuid.UUID, error) {
return uuid.FromBytes(c.v2.GetNonce())
// See also WriteToV2.
func (x *Container) ReadFromV2(m container.Container) error {
return x.readFromV2(m, true)
}
// SetNonceUUID sets container nonce as UUID.
func (c *Container) SetNonceUUID(v uuid.UUID) {
data, _ := v.MarshalBinary()
c.v2.SetNonce(data)
// WriteToV2 writes Container into the container.Container message.
// The message MUST NOT be nil.
//
// See also ReadFromV2.
func (x Container) WriteToV2(m *container.Container) {
*m = x.v2
}
func (c *Container) BasicACL() uint32 {
return c.v2.GetBasicACL()
// Marshal encodes Container into a binary format of the FrostFS API protocol
// (Protocol Buffers with direct field order).
//
// See also Unmarshal.
func (x Container) Marshal() []byte {
return x.v2.StableMarshal(nil)
}
func (c *Container) SetBasicACL(v acl.BasicACL) {
c.v2.SetBasicACL(uint32(v))
// Unmarshal decodes FrostFS API protocol binary format into the Container
// (Protocol Buffers with direct field order). Returns an error describing
// a format violation.
//
// See also Marshal.
func (x *Container) Unmarshal(data []byte) error {
var m container.Container
err := m.Unmarshal(data)
if err != nil {
return err
}
func (c *Container) Attributes() Attributes {
return NewAttributesFromV2(c.v2.GetAttributes())
return x.readFromV2(m, false)
}
func (c *Container) SetAttributes(v Attributes) {
c.v2.SetAttributes(v.ToV2())
// MarshalJSON encodes Container into a JSON format of the FrostFS API protocol
// (Protocol Buffers JSON).
//
// See also UnmarshalJSON.
func (x Container) MarshalJSON() ([]byte, error) {
return x.v2.MarshalJSON()
}
func (c *Container) PlacementPolicy() *netmap.PlacementPolicy {
return netmap.NewPlacementPolicyFromV2(c.v2.GetPlacementPolicy())
// UnmarshalJSON decodes FrostFS API protocol JSON format into the Container
// (Protocol Buffers JSON). Returns an error describing a format violation.
//
// See also MarshalJSON.
func (x *Container) UnmarshalJSON(data []byte) error {
return x.v2.UnmarshalJSON(data)
}
func (c *Container) SetPlacementPolicy(v *netmap.PlacementPolicy) {
c.v2.SetPlacementPolicy(v.ToV2())
// Init initializes all internal data of the Container required by FrostFS API
// protocol. Init MUST be called when creating a new container. Init SHOULD NOT
// be called multiple times. Init SHOULD NOT be called if the Container instance
// is used for decoding only.
func (x *Container) Init() {
var ver refs.Version
version.Current().WriteToV2(&ver)
x.v2.SetVersion(&ver)
nonce, err := uuid.New().MarshalBinary()
if err != nil {
panic(fmt.Sprintf("unexpected error from UUID.MarshalBinary: %v", err))
}
// SessionToken returns token of the session within
// which container was created.
func (c Container) SessionToken() *session.Token {
return c.token
x.v2.SetNonce(nonce)
}
// SetSessionToken sets token of the session within
// which container was created.
func (c *Container) SetSessionToken(t *session.Token) {
c.token = t
// SetOwner specifies the owner of the Container. Each Container has exactly
// one owner, so SetOwner MUST be called for instances to be saved in the
// FrostFS.
//
// See also Owner.
func (x *Container) SetOwner(owner user.ID) {
var m refs.OwnerID
owner.WriteToV2(&m)
x.v2.SetOwnerID(&m)
}
// Signature returns signature of the marshaled container.
func (c Container) Signature() *signature.Signature {
return c.sig
// Owner returns owner of the Container set using SetOwner.
//
// Zero Container has no owner which is incorrect according to FrostFS API
// protocol.
func (x Container) Owner() (res user.ID) {
m := x.v2.GetOwnerID()
if m != nil {
err := res.ReadFromV2(*m)
if err != nil {
panic(fmt.Sprintf("unexpected error from user.ID.ReadFromV2: %v", err))
}
}
// SetSignature sets signature of the marshaled container.
func (c *Container) SetSignature(sig *signature.Signature) {
c.sig = sig
return
}
// Marshal marshals Container into a protobuf binary form.
func (c *Container) Marshal() ([]byte, error) {
return c.v2.StableMarshal(nil)
// SetBasicACL specifies basic part of the Container ACL. Basic ACL is used
// to control access inside container storage.
//
// See also BasicACL.
func (x *Container) SetBasicACL(basicACL acl.Basic) {
x.v2.SetBasicACL(basicACL.Bits())
}
// Unmarshal unmarshals protobuf binary representation of Container.
func (c *Container) Unmarshal(data []byte) error {
return c.v2.Unmarshal(data)
// BasicACL returns basic ACL set using SetBasicACL.
//
// Zero Container has zero basic ACL which structurally correct but doesn't
// make sense since it denies any access to any party.
func (x Container) BasicACL() (res acl.Basic) {
res.FromBits(x.v2.GetBasicACL())
return
}
// MarshalJSON encodes Container to protobuf JSON format.
func (c *Container) MarshalJSON() ([]byte, error) {
return c.v2.MarshalJSON()
// SetPlacementPolicy sets placement policy for the objects within the Container.
// FrostFS storage layer strives to follow the specified policy.
//
// See also PlacementPolicy.
func (x *Container) SetPlacementPolicy(policy netmap.PlacementPolicy) {
var m v2netmap.PlacementPolicy
policy.WriteToV2(&m)
x.v2.SetPlacementPolicy(&m)
}
// UnmarshalJSON decodes Container from protobuf JSON format.
func (c *Container) UnmarshalJSON(data []byte) error {
return c.v2.UnmarshalJSON(data)
// PlacementPolicy returns placement policy set using SetPlacementPolicy.
//
// Zero Container has no placement policy which is incorrect according to
// FrostFS API protocol.
func (x Container) PlacementPolicy() (res netmap.PlacementPolicy) {
m := x.v2.GetPlacementPolicy()
if m != nil {
err := res.ReadFromV2(*m)
if err != nil {
panic(fmt.Sprintf("unexpected error from PlacementPolicy.ReadFromV2: %v", err))
}
}
return
}
// SetAttribute sets Container attribute value by key. Both key and value
// MUST NOT be empty. Attributes set by the creator (owner) are most commonly
// ignored by the FrostFS system and used for application layer. Some attributes
// are so-called system or well-known attributes: they are reserved for system
// needs. System attributes SHOULD NOT be modified using SetAttribute, use
// corresponding methods/functions. List of the reserved keys is documented
// in the particular protocol version.
//
// SetAttribute overwrites existing attribute value.
//
// See also Attribute, IterateAttributes, IterateUserAttributes.
func (x *Container) SetAttribute(key, value string) {
if key == "" {
panic("empty attribute key")
} else if value == "" {
panic("empty attribute value")
}
attrs := x.v2.GetAttributes()
ln := len(attrs)
for i := range ln {
if attrs[i].GetKey() == key {
attrs[i].SetValue(value)
return
}
}
attrs = append(attrs, container.Attribute{})
attrs[ln].SetKey(key)
attrs[ln].SetValue(value)
x.v2.SetAttributes(attrs)
}
// Attribute reads value of the Container attribute by key. Empty result means
// attribute absence.
//
// See also SetAttribute, IterateAttributes, IterateUserAttributes.
func (x Container) Attribute(key string) string {
attrs := x.v2.GetAttributes()
for i := range attrs {
if attrs[i].GetKey() == key {
return attrs[i].GetValue()
}
}
return ""
}
// IterateAttributes iterates over all Container attributes and passes them
// into f. The handler MUST NOT be nil.
//
// See also SetAttribute, Attribute.
func (x Container) IterateAttributes(f func(key, val string)) {
attrs := x.v2.GetAttributes()
for i := range attrs {
f(attrs[i].GetKey(), attrs[i].GetValue())
}
}
// IterateUserAttributes iterates over user Container attributes and passes them
// into f. The handler MUST NOT be nil.
//
// See also SetAttribute, Attribute.
func (x Container) IterateUserAttributes(f func(key, val string)) {
attrs := x.v2.GetAttributes()
for _, attr := range attrs {
key := attr.GetKey()
if !strings.HasPrefix(key, container.SysAttributePrefix) &&
!strings.HasPrefix(key, container.SysAttributePrefixNeoFS) {
f(key, attr.GetValue())
}
}
}
// SetName sets human-readable name of the Container. Name MUST NOT be empty.
//
// See also Name.
func SetName(cnr *Container, name string) {
cnr.SetAttribute(attributeName, name)
}
// Name returns container name set using SetName.
//
// Zero Container has no name.
func Name(cnr Container) string {
return cnr.Attribute(attributeName)
}
// SetCreationTime writes container's creation time in Unix Timestamp format.
//
// See also CreatedAt.
func SetCreationTime(cnr *Container, t time.Time) {
cnr.SetAttribute(attributeTimestamp, strconv.FormatInt(t.Unix(), 10))
}
// CreatedAt returns container's creation time set using SetCreationTime.
//
// Zero Container has zero timestamp (in seconds).
func CreatedAt(cnr Container) time.Time {
var sec int64
attr := cnr.Attribute(attributeTimestamp)
if attr != "" {
var err error
sec, err = strconv.ParseInt(cnr.Attribute(attributeTimestamp), 10, 64)
if err != nil {
panic(fmt.Sprintf("parse container timestamp: %v", err))
}
}
return time.Unix(sec, 0)
}
const attributeHomoHashEnabled = "true"
// DisableHomomorphicHashing sets flag to disable homomorphic hashing of the
// Container data.
//
// See also IsHomomorphicHashingDisabled.
func DisableHomomorphicHashing(cnr *Container) {
cnr.SetAttribute(container.SysAttributeHomomorphicHashing, attributeHomoHashEnabled)
}
// IsHomomorphicHashingDisabled checks if DisableHomomorphicHashing was called.
//
// Zero Container has enabled hashing.
func IsHomomorphicHashingDisabled(cnr Container) bool {
return cnr.Attribute(container.SysAttributeHomomorphicHashing) == attributeHomoHashEnabled ||
cnr.Attribute(container.SysAttributeHomomorphicHashingNeoFS) == attributeHomoHashEnabled
}
// Domain represents information about container domain registered in the NNS
// contract deployed in the FrostFS network.
type Domain struct {
name, zone string
}
// SetName sets human-friendly container domain name.
func (x *Domain) SetName(name string) {
x.name = name
}
// Name returns name set using SetName.
//
// Zero Domain has zero name.
func (x Domain) Name() string {
return x.name
}
// SetZone sets zone which is used as a TLD of a domain name in NNS contract.
func (x *Domain) SetZone(zone string) {
x.zone = zone
}
// Zone returns domain zone set using SetZone.
//
// Zero Domain has "container" zone.
func (x Domain) Zone() string {
if x.zone != "" {
return x.zone
}
return "container"
}
// WriteDomain writes Domain into the Container. Name MUST NOT be empty.
func WriteDomain(cnr *Container, domain Domain) {
cnr.SetAttribute(container.SysAttributeName, domain.Name())
cnr.SetAttribute(container.SysAttributeZone, domain.Zone())
}
// ReadDomain reads Domain from the Container. Returns value with empty name
// if domain is not specified.
func ReadDomain(cnr Container) (res Domain) {
if name := cnr.Attribute(container.SysAttributeName); name != "" {
res.SetName(name)
res.SetZone(cnr.Attribute(container.SysAttributeZone))
} else if name = cnr.Attribute(container.SysAttributeNameNeoFS); name != "" {
res.SetName(name)
res.SetZone(cnr.Attribute(container.SysAttributeZoneNeoFS))
}
return
}
// CalculateSignature calculates signature of the Container using provided signer
// and writes it into dst. Signature instance MUST NOT be nil. CalculateSignature
// is expected to be called after all the Container data is filled and before
// saving the Container in the FrostFS network. Note that мany subsequent change
// will most likely break the signature.
//
// See also VerifySignature.
func CalculateSignature(dst *frostfscrypto.Signature, cnr Container, signer ecdsa.PrivateKey) error {
return dst.Calculate(frostfsecdsa.SignerRFC6979(signer), cnr.Marshal())
}
// VerifySignature verifies Container signature calculated using CalculateSignature.
// Result means signature correctness.
func VerifySignature(sig frostfscrypto.Signature, cnr Container) bool {
return sig.Verify(cnr.Marshal())
}
// CalculateIDFromBinary calculates identifier of the binary-encoded container
// in CAS of the FrostFS containers and writes it into dst. ID instance MUST NOT
// be nil.
//
// See also CalculateID, AssertID.
func CalculateIDFromBinary(dst *cid.ID, cnr []byte) {
dst.SetSHA256(sha256.Sum256(cnr))
}
// CalculateID encodes the given Container and passes the result into
// CalculateIDFromBinary.
//
// See also Container.Marshal, AssertID.
func CalculateID(dst *cid.ID, cnr Container) {
CalculateIDFromBinary(dst, cnr.Marshal())
}
// AssertID checks if the given Container matches its identifier in CAS of the
// FrostFS containers.
//
// See also CalculateID.
func AssertID(id cid.ID, cnr Container) bool {
var id2 cid.ID
CalculateID(&id2, cnr)
return id2.Equals(id)
}

View file

@ -1,136 +1,333 @@
package container_test
import (
"crypto/sha256"
"strconv"
"testing"
"time"
v2container "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
v2netmap "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/netmap"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container"
cid "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id"
cidtest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/id/test"
containertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/container/test"
frostfscrypto "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/crypto"
netmaptest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/netmap/test"
usertest "git.frostfs.info/TrueCloudLab/frostfs-sdk-go/user/test"
"git.frostfs.info/TrueCloudLab/frostfs-sdk-go/version"
"github.com/google/uuid"
"github.com/nspcc-dev/neofs-sdk-go/acl"
"github.com/nspcc-dev/neofs-sdk-go/container"
containertest "github.com/nspcc-dev/neofs-sdk-go/container/test"
netmaptest "github.com/nspcc-dev/neofs-sdk-go/netmap/test"
ownertest "github.com/nspcc-dev/neofs-sdk-go/owner/test"
sessiontest "github.com/nspcc-dev/neofs-sdk-go/session/test"
sigtest "github.com/nspcc-dev/neofs-sdk-go/signature/test"
"github.com/nspcc-dev/neofs-sdk-go/version"
versiontest "github.com/nspcc-dev/neofs-sdk-go/version/test"
"github.com/nspcc-dev/neo-go/pkg/crypto/keys"
"github.com/stretchr/testify/require"
)
func TestNewContainer(t *testing.T) {
c := container.New()
nonce := uuid.New()
ownerID := ownertest.ID()
policy := netmaptest.PlacementPolicy()
c.SetBasicACL(acl.PublicBasicRule)
attrs := containertest.Attributes()
c.SetAttributes(attrs)
c.SetPlacementPolicy(policy)
c.SetNonceUUID(nonce)
c.SetOwnerID(ownerID)
ver := versiontest.Version()
c.SetVersion(ver)
v2 := c.ToV2()
newContainer := container.NewContainerFromV2(v2)
require.EqualValues(t, newContainer.PlacementPolicy(), policy)
require.EqualValues(t, newContainer.Attributes(), attrs)
require.EqualValues(t, newContainer.BasicACL(), acl.PublicBasicRule)
newNonce, err := newContainer.NonceUUID()
require.NoError(t, err)
require.EqualValues(t, newNonce, nonce)
require.EqualValues(t, newContainer.OwnerID(), ownerID)
require.EqualValues(t, newContainer.Version(), ver)
}
func TestContainerEncoding(t *testing.T) {
c := containertest.Container()
func TestPlacementPolicyEncoding(t *testing.T) {
v := containertest.Container()
t.Run("binary", func(t *testing.T) {
data, err := c.Marshal()
require.NoError(t, err)
var v2 container.Container
require.NoError(t, v2.Unmarshal(v.Marshal()))
c2 := container.New()
require.NoError(t, c2.Unmarshal(data))
require.Equal(t, c, c2)
require.Equal(t, v, v2)
})
t.Run("json", func(t *testing.T) {
data, err := c.MarshalJSON()
data, err := v.MarshalJSON()
require.NoError(t, err)
c2 := container.New()
require.NoError(t, c2.UnmarshalJSON(data))
var v2 container.Container
require.NoError(t, v2.UnmarshalJSON(data))
require.Equal(t, c, c2)
require.Equal(t, v, v2)
})
}
func TestContainer_SessionToken(t *testing.T) {
tok := sessiontest.Token()
func TestContainer_Init(t *testing.T) {
val := containertest.Container()
cnr := container.New()
val.Init()
cnr.SetSessionToken(tok)
var msg v2container.Container
val.WriteToV2(&msg)
require.Equal(t, tok, cnr.SessionToken())
binNonce := msg.GetNonce()
var nonce uuid.UUID
require.NoError(t, nonce.UnmarshalBinary(binNonce))
require.EqualValues(t, 4, nonce.Version())
verV2 := msg.GetVersion()
require.NotNil(t, verV2)
var ver version.Version
require.NoError(t, ver.ReadFromV2(*verV2))
require.Equal(t, version.Current(), ver)
var val2 container.Container
require.NoError(t, val2.ReadFromV2(msg))
require.Equal(t, val, val2)
}
func TestContainer_Signature(t *testing.T) {
sig := sigtest.Signature()
func TestContainer_Owner(t *testing.T) {
var val container.Container
cnr := container.New()
cnr.SetSignature(sig)
require.Zero(t, val.Owner())
require.Equal(t, sig, cnr.Signature())
val = containertest.Container()
owner := usertest.ID()
val.SetOwner(owner)
var msg v2container.Container
val.WriteToV2(&msg)
var msgOwner refs.OwnerID
owner.WriteToV2(&msgOwner)
require.Equal(t, &msgOwner, msg.GetOwnerID())
var val2 container.Container
require.NoError(t, val2.ReadFromV2(msg))
require.True(t, val2.Owner().Equals(owner))
}
func TestContainer_ToV2(t *testing.T) {
t.Run("nil", func(t *testing.T) {
var x *container.Container
func TestContainer_BasicACL(t *testing.T) {
var val container.Container
require.Nil(t, x.ToV2())
require.Zero(t, val.BasicACL())
val = containertest.Container()
basicACL := containertest.BasicACL()
val.SetBasicACL(basicACL)
var msg v2container.Container
val.WriteToV2(&msg)
require.EqualValues(t, basicACL.Bits(), msg.GetBasicACL())
var val2 container.Container
require.NoError(t, val2.ReadFromV2(msg))
require.Equal(t, basicACL, val2.BasicACL())
}
func TestContainer_PlacementPolicy(t *testing.T) {
var val container.Container
require.Zero(t, val.PlacementPolicy())
val = containertest.Container()
pp := netmaptest.PlacementPolicy()
val.SetPlacementPolicy(pp)
var msgPolicy v2netmap.PlacementPolicy
pp.WriteToV2(&msgPolicy)
var msg v2container.Container
val.WriteToV2(&msg)
require.Equal(t, &msgPolicy, msg.GetPlacementPolicy())
var val2 container.Container
require.NoError(t, val2.ReadFromV2(msg))
require.Equal(t, pp, val2.PlacementPolicy())
}
func assertContainsAttribute(t *testing.T, m v2container.Container, key, val string) {
var msgAttr v2container.Attribute
msgAttr.SetKey(key)
msgAttr.SetValue(val)
require.Contains(t, m.GetAttributes(), msgAttr)
}
func TestContainer_Attribute(t *testing.T) {
const attrKey1, attrKey2 = v2container.SysAttributePrefix + "key1", v2container.SysAttributePrefixNeoFS + "key2"
const attrVal1, attrVal2 = "val1", "val2"
val := containertest.Container()
val.SetAttribute(attrKey1, attrVal1)
val.SetAttribute(attrKey2, attrVal2)
var i int
val.IterateUserAttributes(func(key, val string) {
i++
})
require.Equal(t, 1, i)
var msg v2container.Container
val.WriteToV2(&msg)
require.GreaterOrEqual(t, len(msg.GetAttributes()), 2)
assertContainsAttribute(t, msg, attrKey1, attrVal1)
assertContainsAttribute(t, msg, attrKey2, attrVal2)
var val2 container.Container
require.NoError(t, val2.ReadFromV2(msg))
require.Equal(t, attrVal1, val2.Attribute(attrKey1))
require.Equal(t, attrVal2, val2.Attribute(attrKey2))
m := map[string]string{}
val2.IterateAttributes(func(key, val string) {
m[key] = val
})
t.Run("default values", func(t *testing.T) {
cnt := container.New()
require.GreaterOrEqual(t, len(m), 2)
require.Equal(t, attrVal1, m[attrKey1])
require.Equal(t, attrVal2, m[attrKey2])
// check initial values
require.Nil(t, cnt.SessionToken())
require.Nil(t, cnt.Signature())
require.Nil(t, cnt.Attributes())
require.Nil(t, cnt.PlacementPolicy())
require.Nil(t, cnt.OwnerID())
require.EqualValues(t, acl.PrivateBasicRule, cnt.BasicACL())
require.Equal(t, version.Current(), cnt.Version())
nonce, err := cnt.NonceUUID()
require.NoError(t, err)
require.NotNil(t, nonce)
// convert to v2 message
cntV2 := cnt.ToV2()
nonceV2, err := uuid.FromBytes(cntV2.GetNonce())
require.NoError(t, err)
require.Equal(t, nonce.String(), nonceV2.String())
require.Nil(t, cntV2.GetAttributes())
require.Nil(t, cntV2.GetPlacementPolicy())
require.Nil(t, cntV2.GetOwnerID())
require.Equal(t, uint32(acl.PrivateBasicRule), cntV2.GetBasicACL())
require.Equal(t, version.Current().ToV2(), cntV2.GetVersion())
})
val2.SetAttribute(attrKey1, attrVal1+"_")
require.Equal(t, attrVal1+"_", val2.Attribute(attrKey1))
}
func TestSetName(t *testing.T) {
var val container.Container
require.Panics(t, func() {
container.SetName(&val, "")
})
val = containertest.Container()
const name = "some name"
container.SetName(&val, name)
var msg v2container.Container
val.WriteToV2(&msg)
assertContainsAttribute(t, msg, "Name", name)
var val2 container.Container
require.NoError(t, val2.ReadFromV2(msg))
require.Equal(t, name, container.Name(val2))
}
func TestSetCreationTime(t *testing.T) {
var val container.Container
require.Zero(t, container.CreatedAt(val).Unix())
val = containertest.Container()
creat := time.Now()
container.SetCreationTime(&val, creat)
var msg v2container.Container
val.WriteToV2(&msg)
assertContainsAttribute(t, msg, "Timestamp", strconv.FormatInt(creat.Unix(), 10))
var val2 container.Container
require.NoError(t, val2.ReadFromV2(msg))
require.Equal(t, creat.Unix(), container.CreatedAt(val2).Unix())
}
func TestDisableHomomorphicHashing(t *testing.T) {
var val container.Container
require.False(t, container.IsHomomorphicHashingDisabled(val))
val = containertest.Container()
container.DisableHomomorphicHashing(&val)
var msg v2container.Container
val.WriteToV2(&msg)
assertContainsAttribute(t, msg, v2container.SysAttributePrefix+"DISABLE_HOMOMORPHIC_HASHING", "true")
var val2 container.Container
require.NoError(t, val2.ReadFromV2(msg))
require.True(t, container.IsHomomorphicHashingDisabled(val2))
}
func TestWriteDomain(t *testing.T) {
var val container.Container
require.Zero(t, container.ReadDomain(val).Name())
val = containertest.Container()
const name = "domain name"
var d container.Domain
d.SetName(name)
container.WriteDomain(&val, d)
var msg v2container.Container
val.WriteToV2(&msg)
assertContainsAttribute(t, msg, v2container.SysAttributeName, name)
assertContainsAttribute(t, msg, v2container.SysAttributeZone, "container")
const zone = "domain zone"
d.SetZone(zone)
container.WriteDomain(&val, d)
val.WriteToV2(&msg)
assertContainsAttribute(t, msg, v2container.SysAttributeZone, zone)
var val2 container.Container
require.NoError(t, val2.ReadFromV2(msg))
require.Equal(t, d, container.ReadDomain(val2))
}
func TestCalculateID(t *testing.T) {
val := containertest.Container()
require.False(t, container.AssertID(cidtest.ID(), val))
var id cid.ID
container.CalculateID(&id, val)
var msg refs.ContainerID
id.WriteToV2(&msg)
h := sha256.Sum256(val.Marshal())
require.Equal(t, h[:], msg.GetValue())
var id2 cid.ID
require.NoError(t, id2.ReadFromV2(msg))
require.True(t, container.AssertID(id2, val))
}
func TestCalculateSignature(t *testing.T) {
key, err := keys.NewPrivateKey()
require.NoError(t, err)
val := containertest.Container()
var sig frostfscrypto.Signature
require.NoError(t, container.CalculateSignature(&sig, val, key.PrivateKey))
var msg refs.Signature
sig.WriteToV2(&msg)
var sig2 frostfscrypto.Signature
require.NoError(t, sig2.ReadFromV2(msg))
require.True(t, container.VerifySignature(sig2, val))
}

49
container/doc.go Normal file
View file

@ -0,0 +1,49 @@
/*
Package container provides functionality related to the FrostFS containers.
The base type is Container. To create new container in the FrostFS network
Container instance should be initialized
var cnr Container
cnr.Init()
// fill all the fields
// encode cnr and send
After the container is persisted in the FrostFS network, applications can process
it using the instance of Container types
// recv binary container
var cnr Container
err := cnr.Unmarshal(bin)
// ...
// process the container data
Instances can be also used to process FrostFS API V2 protocol messages
(see neo.fs.v2.container package in https://git.frostfs.info/TrueCloudLab/frostfs-api).
On client side:
import "git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/container"
var msg container.Container
cnr.WriteToV2(&msg)
// send msg
On server side:
// recv msg
var cnr Container
cnr.ReadFromV2(msg)
// process cnr
Using package types in an application is recommended to potentially work with
different protocol versions with which these types are compatible.
*/
package container

7
container/id/doc.go Normal file
View file

@ -0,0 +1,7 @@
/*
Package cid provides primitives to work with container identification in FrostFS.
Using package types in an application is recommended to potentially work with
different protocol versions with which these types are compatible.
*/
package cid

View file

@ -1,90 +1,115 @@
package cid
import (
"bytes"
"crypto/sha256"
"errors"
"fmt"
"git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs"
"github.com/mr-tron/base58"
"github.com/nspcc-dev/neofs-api-go/v2/refs"
)
// ID represents v2-compatible container identifier.
type ID refs.ContainerID
// NewFromV2 wraps v2 ContainerID message to ID.
// ID represents FrostFS container identifier.
//
// Nil refs.ContainerID converts to nil.
func NewFromV2(idV2 *refs.ContainerID) *ID {
return (*ID)(idV2)
}
// New creates and initializes blank ID.
// ID is mutually compatible with git.frostfs.info/TrueCloudLab/frostfs-api-go/v2/refs.ContainerID
// message. See ReadFromV2 / WriteToV2 methods.
//
// Defaults:
// - value: nil.
func New() *ID {
return NewFromV2(new(refs.ContainerID))
}
// SetSHA256 sets container identifier value to SHA256 checksum of container body.
func (id *ID) SetSHA256(v [sha256.Size]byte) {
(*refs.ContainerID)(id).SetValue(v[:])
}
// ToV2 returns the v2 container ID message.
// Instances can be created using built-in var declaration.
//
// Nil ID converts to nil.
func (id *ID) ToV2() *refs.ContainerID {
return (*refs.ContainerID)(id)
}
// Equal returns true if identifiers are identical.
func (id *ID) Equal(id2 *ID) bool {
return bytes.Equal(
(*refs.ContainerID)(id).GetValue(),
(*refs.ContainerID)(id2).GetValue(),
)
}
// Parse parses string representation of ID.
// Note that direct typecast is not safe and may result in loss of compatibility:
//
// Returns error if s is not a base58 encoded
// ID data.
func (id *ID) Parse(s string) error {
data, err := base58.Decode(s)
if err != nil {
return err
} else if len(data) != sha256.Size {
return errors.New("incorrect format of the string container ID")
// _ = ID([32]byte) // not recommended
type ID [sha256.Size]byte
// ReadFromV2 reads ID from the refs.ContainerID message.
// Returns an error if the message is malformed according
// to the FrostFS API V2 protocol.
//
// See also WriteToV2.
func (id *ID) ReadFromV2(m refs.ContainerID) error {
return id.Decode(m.GetValue())
}
(*refs.ContainerID)(id).SetValue(data)
// WriteToV2 writes ID to the refs.ContainerID message.
// The message must not be nil.
//
// See also ReadFromV2.
func (id ID) WriteToV2(m *refs.ContainerID) {
m.SetValue(id[:])
}
// Encode encodes ID into 32 bytes of dst. Panics if
// dst length is less than 32.
//
// Zero ID is all zeros.
//
// See also Decode.
func (id ID) Encode(dst []byte) {
if l := len(dst); l < sha256.Size {
panic(fmt.Sprintf("destination length is less than %d bytes: %d", sha256.Size, l))
}
copy(dst, id[:])
}
// Decode decodes src bytes into ID.
//
// Decode expects that src has 32 bytes length. If the input is malformed,
// Decode returns an error describing format violation. In this case ID
// remains unchanged.
//
// Decode doesn't mutate src.
//
// See also Encode.
func (id *ID) Decode(src []byte) error {
if len(src) != sha256.Size {
return fmt.Errorf("invalid length %d", len(src))
}
copy(id[:], src)
return nil
}
// String returns base58 string representation of ID.
func (id *ID) String() string {
return base58.Encode((*refs.ContainerID)(id).GetValue())
// SetSHA256 sets container identifier value to SHA256 checksum of container structure.
func (id *ID) SetSHA256(v [sha256.Size]byte) {
copy(id[:], v[:])
}
// Marshal marshals ID into a protobuf binary form.
func (id *ID) Marshal() ([]byte, error) {
return (*refs.ContainerID)(id).StableMarshal(nil)
// Equals defines a comparison relation between two ID instances.
//
// Note that comparison using '==' operator is not recommended since it MAY result
// in loss of compatibility.
func (id ID) Equals(id2 ID) bool {
return id == id2
}
// Unmarshal unmarshals protobuf binary representation of ID.
func (id *ID) Unmarshal(data []byte) error {
return (*refs.ContainerID)(id).Unmarshal(data)
// EncodeToString encodes ID into FrostFS API protocol string.
//
// Zero ID is base58 encoding of 32 zeros.
//
// See also DecodeString.
func (id ID) EncodeToString() string {
return base58.Encode(id[:])
}
// MarshalJSON encodes ID to protobuf JSON format.
func (id *ID) MarshalJSON() ([]byte, error) {
return (*refs.ContainerID)(id).MarshalJSON()
// DecodeString decodes string into ID according to FrostFS API protocol. Returns
// an error if s is malformed.
//
// See also DecodeString.
func (id *ID) DecodeString(s string) error {
data, err := base58.Decode(s)
if err != nil {
return fmt.Errorf("decode base58: %w", err)
}
// UnmarshalJSON decodes ID from protobuf JSON format.
func (id *ID) UnmarshalJSON(data []byte) error {
return (*refs.ContainerID)(id).UnmarshalJSON(data)
return id.Decode(data)
}
// String implements fmt.Stringer.
//
// String is designed to be human-readable, and its format MAY differ between
// SDK versions. String MAY return same result as EncodeToString. String MUST NOT
// be used to encode ID into FrostFS protocol string.
func (id ID) String() string {
return id.EncodeToString()
}

Some files were not shown because too many files have changed in this diff Show more