olefirenque/frostfs-sdk-go
1
0
Fork 0
forked from TrueCloudLab/frostfs-sdk-go
Code Pull requests Activity
478 commits 4 branches 0 tags 2.3 MiB
Commit graph

140 commits

Author SHA1 Message Date
Alex Vanin
27cd721422 [#170] bearer: Add docs, refactor 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-04-13 11:24:15 +03:00
Denis Kirillov
3e75660802 [#205] pool: fix using default key for requests 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-12 18:12:50 +03:00
Denis Kirillov
53e064e556 [#206] pool: fix waitForContainerRemoved 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-12 18:12:39 +03:00
Denis Kirillov
385f1b10f9 [#203] pool: fix PrmContainerDelete 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-12 14:26:08 +03:00
Denis Kirillov
1e8aa6a99f [#85] pool: update tests 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-12 10:30:28 +03:00
Denis Kirillov
97c8274dc0 [#85] pool: use non-pointer client inside wrapper 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-12 10:30:28 +03:00
Denis Kirillov
9e8e12f6ae [#85] pool: make client interface methods private 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-12 10:30:28 +03:00
Denis Kirillov
aa6a4950d5 [#85] pool: fix skipped tests 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-12 10:30:28 +03:00
Denis Kirillov
e56eef495d [#85] pool: refactor client interface 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-12 10:30:28 +03:00
Denis Kirillov
e50e6d2828 [#137] pool: simplify session creation 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-07 13:33:16 +03:00
Denis Kirillov
df0573d521 [#137] pool: drop sessionTokenThreshold 
Description

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-07 13:33:16 +03:00
Denis Kirillov
c4adb03f8e [#137] pool: drop retry for object operations 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-07 13:33:16 +03:00
Denis Kirillov
fcfae4a249 [#137] Use callback response to update epoch 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-07 13:33:16 +03:00
Denis Kirillov
9814748958 [#137] pool: fix tests 
Description

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-07 13:33:16 +03:00
Denis Kirillov
7df00fb0eb [#137] Add exp to ResSessionCreate 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-07 13:33:16 +03:00
Denis Kirillov
11a25bb413 [#137] Expire session tokens locally 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-07 13:33:16 +03:00
Denis Kirillov
b8d2158acd [#180] eacl: make equal methods functions 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-07 09:50:02 +03:00
Denis Kirillov
a709cf5444 [#180] pool: make some operations synchronous 
Operations: PutContainer, DeleteContainer, SetEACL

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-07 09:50:02 +03:00
Denis Kirillov
0dbea5452a [#180] pool: add WaitForContainerRemoved 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-07 09:50:02 +03:00
Denis Kirillov
47345a33da [#180] pool: add WaitForEACLPresence 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-07 09:50:02 +03:00
Alex Vanin
2104945f9e [#193] pool: Return copy of session token in cache 
To avoid side effects after token re-sign.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-04-06 12:06:24 +03:00
Alex Vanin
f38a24e8b5 [#193] pool: Add test to check side effects in token cache 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-04-06 12:06:24 +03:00
Denis Kirillov
e0281c3b34 [#191] pool: use bearer token 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-04-06 10:27:58 +04:00
Denis Kirillov
030bbce2cf [#165] pool: drop prmCommon from container params 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-03-21 12:41:08 +03:00
Denis Kirillov
59b49dd7e6 [#165] pool: update WaitForContainerPresence 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-03-21 12:41:08 +03:00
Denis Kirillov
191d85e607 [#165] pool: change exported fields to setters 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-03-21 12:41:08 +03:00
Denis Kirillov
9be9697856 [#165] pool: make client interface unexported 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-03-21 12:41:08 +03:00
Denis Kirillov
52548fe176 [#165] pool: add docs 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-03-21 12:41:08 +03:00
Denis Kirillov
7811d8eefc [#165] pool: distinguish init step and run 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-03-21 12:41:08 +03:00
Denis Kirillov
d03523a3bc [#165] pool: replace options with parameters 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-03-21 12:41:08 +03:00
Denis Kirillov
ec5c223f29 [#165] pool: make private inner structs 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-03-21 12:41:08 +03:00
Denis Kirillov
f5cabe26cb [#165] pool: drop builder 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-03-21 12:41:08 +03:00
Alex Vanin
b91e25f782 [#168] pool: Adopt replacement of pointer slices with struct slices 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-03-15 16:59:59 +03:00
Leonard Lyubich
b81122740a [#164] pool: Make pool.Pool a struct 
Connection pool package should not define `Pool` type as an interface
since it provides single particular implementation.

Make `pool.Pool` type a struct instead of interface. Also remove
`Object`, `Container` and `Accounting` interfaces.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-10 15:02:10 +03:00
Leonard Lyubich
22dad0573d [#164] client: Refactor and document package functionality 
Get rid of `Option` pattern. Define `Init`, `Dial` and `Close` methods
for the corresponding stages of use.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-10 15:02:10 +03:00
Denis Kirillov
61f9c3392b [#159] Add object context to session tokens 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-03-03 15:08:47 +03:00
Evgenii Stratonikov
07817fb403 [#148] client: Allow to iterate over search results 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-02-28 10:19:35 +03:00
Evgenii Stratonikov
fa670ab57a *: Fix linter warnings 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-02-24 18:49:55 +03:00
Leonard Lyubich
29c057272c [#147] pool: Limit buffer length by 3MB in PutObject 
In previous implementation we wrote up to 4KB payload chunk per-call.
This led to sending a large number of messages with a large
amount of payload.

Increase buffer length limit to 3MB.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-22 09:09:22 +03:00
Denis Kirillov
5386694a1a [#142] Drop retry for container operations 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-02-21 16:48:28 +03:00
Leonard Lyubich
66bc59da5c [#146] pool: Set private key in object HEAD/DELETE 
Use call private key parameterized in `HeadObject` / `DeleteObject`
methods of `Pool` as corresponding key for `ObjectHead` / `ObjectDelete`
methods of `Client`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-21 15:21:37 +03:00
Leonard Lyubich
69ffface78 [#131] client: Name all methods and types the same way 
Inherit name format of object operations in all other ones.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-18 17:01:08 +03:00
Leonard Lyubich
998bae3f1c [#131] pool: Sign session token before caching in newPool 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-18 17:01:08 +03:00
Leonard Lyubich
37f0cde78d [#131] client: Remove SetEACLPrm.SetSessionToken 
From now session token for `SetEACL` operation should be written into
eACL table structure (similar to `PutContainer`).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-18 17:01:08 +03:00
Leonard Lyubich
d90fe8fbab [#131] client: Remove no longer needed code 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-18 17:01:08 +03:00
Leonard Lyubich
1130c1f5a6 [#131] pool: Limit session lifetime in openDefaultSession method 
Call `createSessionTokenForDuration` function for session opening in
`pool.openDefaultSession` method in order to limit session lifetime
according to pool configuration (`SessionExpirationDuration`).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-18 17:01:08 +03:00
Leonard Lyubich
28b49419ab [#131] client: Re-implement Object Delete and Hash methods 
Also remove unused methods of `pool.Client` and `pool.Pool`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-18 17:01:08 +03:00
Leonard Lyubich
fb3501752d [#131] client: Re-implement Object.Search method 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-18 17:01:08 +03:00
Leonard Lyubich
ff4febff6f [#131] client: Re-implement Object.Range method 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-18 17:01:08 +03:00
Leonard Lyubich
7de81cd585 [#131] pool: Fix ineffectual error assignments 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-18 17:01:08 +03:00
Leonard Lyubich
caf31a928c [#131] client: Re-implement Object.Head method 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-18 17:01:08 +03:00
Leonard Lyubich
8d734d1020 [#131] client: Add ID suffix to ResObjectPut.ReadStoredObject 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-18 17:01:08 +03:00
Leonard Lyubich
b508f3b71e [#131] pool: Refactor retry сalls 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-18 17:01:08 +03:00
Leonard Lyubich
d5ce5d63b4 [#131] pool: Remove redundant var from pool.PutObject 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-18 17:01:08 +03:00
Leonard Lyubich
2624347d9b [#131] client: Change interface of object PUT and GET ops 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-18 17:01:08 +03:00
Alex Vanin
40aaaafc73 [#126] pool: Do not resend object Put requests 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-02-11 14:21:59 +03:00
Alex Vanin
de4eedcd61 [#126] pool: Store access time in the session token cache 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-02-11 14:21:59 +03:00
Alex Vanin
ac66db59e0 [#130] pool: Set default session token lifetime before initial requests 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-02-10 16:20:06 +03:00
Alex Vanin
b6abb02ace [#130] pool: Add default session token duration 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-02-09 10:10:00 +03:00
Alex Vanin
bfff21965d [#130] pool: Process expired session tokens as invalid 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-02-09 10:10:00 +03:00
Alex Vanin
a86c08b3ee [#130] pool: Replace expiration value with expiration duration 
Based on the applications that use pool, most of them simply
set session token duration to MaxUint64 value. It is completely
understandable, because epochs are incrementing and expiration
value will be surpassed sooner or later, unless it is MaxUint64.

As an alternative I suggest specifying duration instead of
absolute epoch values. Now apps can set duration of 100-200
epochs and pool automatically calculated expiration epoch
base on the network info.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-02-09 10:10:00 +03:00
Alex Vanin
2a72f180dc [#133] client: Don't accept session token params in PutContainer 
PutContainer method takes `container.Container` structure
as an argument. This structure already contains session
token field, so there is no need in `prmSession` because it
duplicates session token definition.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-02-08 10:42:58 +03:00
Pavel Karpy
e8eac3997c [#128] object: Move ID and Address in subpkg 
This is done to prevent import cycles when `object` package needs any other
that requires `object.ID` or `object.Address`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-02-01 17:03:12 +03:00
Leonard Lyubich
213d20e3fb [#92] client: Accept structured parameters in non-object operations 
Define `XPrm` type for each `X` client operation which structures
parameters. Export setters of each parameterized value. Emphasize that
some parameters are required. Make the client panic when the parameters
are incorrectly set. Get rid of vadiadic call options and `CallOption`
type. Improve documentation of client behavior.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-02-01 17:02:43 +03:00
Evgenii Stratonikov
596774ce5b [#103] owner: remove NEO3Wallet type 
Allow to use public keys and N3 wallet accounts instead.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-01-21 11:01:44 +03:00
Denis Kirillov
dd91fbd6ef [#118] Use correct owner in session token 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2022-01-18 13:21:40 +03:00
Pavel Karpy
7f31621106 [#114] pool: Use "simple" error handling in pool 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-01-13 15:37:43 +03:00
Pavel Karpy
b50bff53e7 [#111] pool: Adopt new client.New signature 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-01-11 11:03:14 +03:00
Alex Vanin
2fd5802c48 [#105] pool: Add balance command 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2021-12-17 15:11:23 +03:00
Alex Vanin
b1770ecb92 [#70] pool: Ignore default session token in non object service requests 
Default session token is created for object service requests
and should not be reused in container or any other service requests.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2021-12-16 19:14:45 +03:00
Denis Kirillov
2806d90089 [#93] Remove golang.org/x/crypto dependency 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-12-14 15:27:04 +03:00
Denis Kirillov
fa7c01bd0b [#74] Fix some tests 
For some tests enough empty results.

Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-11-26 16:17:45 +03:00
Denis Kirillov
618eeb8172 [#74] Add priority to nodes 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-11-26 16:17:45 +03:00
Leonard Lyubich
d9317cbea1 [#83] pool: Temporary skip tests with mocked client 
After update of `client.Client` interface it can't be mocked.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-11-23 13:03:40 +03:00
Leonard Lyubich
639ab1e443 [#83] pool: Support changes of client.Client interface 
Handle status results and return status failures in `error` return.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-11-23 13:03:40 +03:00
Leonard Lyubich
9dcff95a29 [#83] client: Implement status library 
Define base `Status` interface. Provide the functionality to distinguish
success and failure returns. Provide functionality to transport statuses
over NeoFS API V2 protocol. Support success `OK` and failure `INTERNAL`
returns.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-11-23 13:03:40 +03:00
Evgenii Stratonikov
a2d342e928 [#87] go.mod: remove neo-go dependency 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-11-22 15:40:13 +03:00
Denis Kirillov
e2c740ae8d [#38] Refactoring 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-11-15 14:04:06 +03:00
Denis Kirillov
818f38b811 [#38] Replace gcache with golang-lru 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-11-15 14:04:06 +03:00
Denis Kirillov
1d546711e5 [#38] Add retrying when session token error 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-11-15 14:04:06 +03:00
Denis Kirillov
e08e3d6c00 [#38] Add session cache tests 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-11-15 14:04:06 +03:00
Denis Kirillov
c651a714fd [#38] Add session token cache 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-11-15 14:04:06 +03:00
Evgenii Stratonikov
10b78e74af [#62] pool: replace neofs-api-go imports with neofs-sdk-go 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-11-10 18:29:19 +03:00
Evgenii Stratonikov
46353456f1 [#48] pool/test: add missing EXPECT calls 
Another possible panic (though quite rare).
```
--- FAIL: TestTwoNodes (0.00s)
    pool.go:185: Unexpected call to *pool.MockClient.EndpointInfo([context.Background.WithCancel.WithDeadline(2021-10-26 15:50:50.005585875 +0300 MSK m=+1.216577808 [-940.323µs])]) at /home/dzeta/repo/neofs-sdk-go/pool/pool.go:185 because: there are no expected calls of the method "EndpointInfo" for that receiver
    pool.go:185: Unexpected call to *pool.MockClient.EndpointInfo([context.Background.WithCancel.WithDeadline(2021-10-26 15:50:50.005715623 +0300 MSK m=+1.216707547 [-1.351991ms])]) at /home/dzeta/repo/neofs-sdk-go/pool/pool.go:185 because: there are no expected calls of the method "EndpointInfo" for that receiver
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-11-01 17:51:21 +03:00
Evgenii Stratonikov
f3f8ce01d3 [#48] pool/test: add go:generate line for mocks generation 
We also don't use mocks for `Pool`.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-11-01 17:51:21 +03:00
Evgenii Stratonikov
cd5e08d725 [#48] pool: add Close method 
Fix occasional panic in tests:
```
> for i in (seq 1 100); go test -race -count=1 ./pool/... ; end
...
{"level":"warn","ts":1635251466.567485,"caller":"pool/pool.go:122","msg":"failed to create neofs session token for client","address":"peer0","error":"error session"}
panic: Fail in goroutine after TestTwoNodes has completed

goroutine 6 [running]:
testing.(*common).Fail(0xc0002e1380)
        /usr/lib/go/src/testing/testing.go:710 +0x1b4
testing.(*common).FailNow(0xc0002e1380)
        /usr/lib/go/src/testing/testing.go:732 +0x2f
testing.(*common).Fatalf(0xc000074070, {0xd9d816, 0x2e}, {0xc000094050, 0x5, 0x5})
        /usr/lib/go/src/testing/testing.go:830 +0x85
github.com/golang/mock/gomock.(*Controller).Call.func1(0xc0002f4120, {0xd68380, 0xc0002dac30}, {0xd8847f, 0xc}, {0xc000074020, 0x1, 0x1})
        /home/dzeta/go/pkg/mod/github.com/golang/mock@v1.6.0/gomock/controller.go:231 +0x44d
github.com/golang/mock/gomock.(*Controller).Call(0xc0002f4120, {0xd68380, 0xc0002dac30}, {0xd8847f, 0xc}, {0xc000074020, 0x1, 0x1})
        /home/dzeta/go/pkg/mod/github.com/golang/mock@v1.6.0/gomock/controller.go:247 +0xce
github.com/nspcc-dev/neofs-sdk-go/pool.(*MockClient).EndpointInfo(0xc0002dac30, {0xe85528, 0xc00008a120}, {0x0, 0x0, 0x0})
        /home/dzeta/repo/neofs-sdk-go/pool/mock_test.go:186 +0x298
github.com/nspcc-dev/neofs-sdk-go/pool.updateNodesHealth.func1(0x1, {0xe950d8, 0xc0002dac30})
        /home/dzeta/repo/neofs-sdk-go/pool/pool.go:183 +0x188
created by github.com/nspcc-dev/neofs-sdk-go/pool.updateNodesHealth
        /home/dzeta/repo/neofs-sdk-go/pool/pool.go:174 +0x233
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-11-01 17:51:21 +03:00
Evgenii Stratonikov
d541aab1ff [#48] pool/test: remove defer for controller close 
We have go1.16 in go.mod and `gomock.NewController` comment indicates:
```
New in go1.14+, if you are passing a *testing.T into this function you no
longer need to call ctrl.Finish() in your test methods.
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-11-01 17:51:21 +03:00
Denis Kirillov
880f3a61e5 [#32] Require at least one healthy node 
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
 2021-09-09 14:46:46 +03:00
Alex Vanin
234373f249 [#31] pool: Do not invoke EndpointInfo to find the address 
Since we transformed the pool of gRPC connection into the pool of
neofs clients, we don't need to call EndpointInfo to fetch the
address of the node for the beautiful error output.

Moreover it is done incorrectly, because c.CreateSession may return
context.Cancelled error (graceful shutdown). We will reuse the same
context for the EndpointInfo call which will lead to a
timeout freeze.

Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2021-09-07 13:54:34 +03:00
Angira Kekteeva
628ae20c72 [#26] *: Move pkg content to root 
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
 2021-07-28 13:47:27 +03:00