Commit graph

229 commits

Author SHA1 Message Date
55f5409203
[#185] tree/pool: Control timeout of tree service operations
All checks were successful
DCO / DCO (pull_request) Successful in 1m2s
Tests and linters / Tests (pull_request) Successful in 1m16s
Tests and linters / Lint (pull_request) Successful in 2m10s
Implemented context timeout for all tree service operations except those that return a GRPC stream

Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
2024-11-01 09:34:40 +03:00
5361f0eceb [#279] pool: Count errors in object search
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-10-22 12:41:11 +00:00
05aa3becae [#278] pool: Don't make maintenance node healthy in rebalance
All checks were successful
DCO / DCO (pull_request) Successful in 41s
Tests and linters / Tests (pull_request) Successful in 1m8s
Tests and linters / Lint (pull_request) Successful in 2m36s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-10-17 16:31:49 +03:00
79f387317a [#283] pool: Mark node unhealthy if node is under maintenance
All checks were successful
DCO / DCO (pull_request) Successful in 59s
Tests and linters / Tests (pull_request) Successful in 1m13s
Tests and linters / Lint (pull_request) Successful in 2m4s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-10-16 15:22:05 +03:00
d00892f418
[#274] client/status: Support INVALID_ARGUMENT status
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-10-03 17:05:08 +03:00
1dc3b77ac7 [#270] pool: Replace deprecated DialContext
`Healthcheck` request performed after client creation, so no extra RPC required.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-16 15:09:59 +03:00
7d84d104fb [#260] *: Fix linter warnings
All checks were successful
DCO / DCO (pull_request) Successful in 1m2s
Tests and linters / Tests (pull_request) Successful in 1m12s
Tests and linters / Lint (pull_request) Successful in 1m26s
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-09 08:55:32 +00:00
46ee543899 [#265] go.mod: Use range over int
All checks were successful
DCO / DCO (pull_request) Successful in 47s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m7s
Tests and linters / Tests (1.23) (pull_request) Successful in 1m7s
Tests and linters / Lint (pull_request) Successful in 1m42s
Since Go 1.22 a `for` statement with a `range` clause is able
to iterate through integer values from zero to an upper limit.

gopatch script:
@@
var i, e expression
@@
-for i := 0; i <= e - 1; i++ {
+for i := range e {
    ...
}

@@
var i, e expression
@@
-for i := 0; i <= e; i++ {
+for i := range e + 1 {
    ...
}

@@
var i, e expression
@@
-for i := 0; i < e; i++ {
+for i := range e {
    ...
}

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-09-04 12:37:46 +03:00
8f751d9dd0
[#263] go.mod: Update api-go
All checks were successful
DCO / DCO (pull_request) Successful in 42s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m5s
Tests and linters / Tests (1.23) (pull_request) Successful in 1m4s
Tests and linters / Lint (pull_request) Successful in 1m44s
Remove `client.ContainerEACL` and related references. This change
was initiated by the removal of `ContainerService.GetExtendedACL`
from the API.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-03 12:36:28 +03:00
3c00f4eeac [#203] Add pool docs
All checks were successful
DCO / DCO (pull_request) Successful in 48s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m3s
Tests and linters / Tests (1.23) (pull_request) Successful in 1m2s
Tests and linters / Lint (pull_request) Successful in 1m41s
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
2024-08-30 08:24:31 +03:00
28f140bf06 [#254] pool: Add parameter gracefulCloseOnSwitchTimeout
Add new param for waiting a little until current in-flight requests will be finished

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-22 08:02:51 +00:00
338d1ef254 [#258] pool/tree: Add node address to error
All checks were successful
DCO / DCO (pull_request) Successful in 39s
Tests and linters / Tests (1.22) (pull_request) Successful in 50s
Tests and linters / Tests (1.21) (pull_request) Successful in 54s
Tests and linters / Lint (pull_request) Successful in 1m8s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-20 14:19:58 +03:00
203bba65a0 [#253] pool: Don't count regular FrostFS errors
All checks were successful
DCO / DCO (pull_request) Successful in 54s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m13s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m11s
Tests and linters / Lint (pull_request) Successful in 1m36s
Previously we count all frostfs errors like:
ObjectNotFound, EACLNotFound
because frostfs status is unconditionally resolved into built-in go errors
but handleError method handled built-in errors like internal network ones.
Since after resolving frostfs errors status is also returned we start check this first

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-08-14 16:48:51 +03:00
908c96a94d [#251] pool: Fix handlerError panic for objectPatch
All checks were successful
DCO / DCO (pull_request) Successful in 55s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m0s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m0s
Tests and linters / Lint (pull_request) Successful in 1m56s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-13 15:28:39 +03:00
5d58519253 [#249] pool: Introduce objectPatch method
All checks were successful
DCO / DCO (pull_request) Successful in 38s
Tests and linters / Tests (1.22) (pull_request) Successful in 53s
Tests and linters / Tests (1.21) (pull_request) Successful in 55s
Tests and linters / Lint (pull_request) Successful in 1m17s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-08 18:26:40 +03:00
e83d6b7c6a [#244] pool/tree: Collect request duration statistic
All checks were successful
DCO / DCO (pull_request) Successful in 38s
Tests and linters / Tests (1.22) (pull_request) Successful in 50s
Tests and linters / Tests (1.21) (pull_request) Successful in 56s
Tests and linters / Lint (pull_request) Successful in 1m22s
After each request for tree pool statistic accumulated values are reset to zero.

Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-08-02 13:01:14 +03:00
9da46f566f [#243] go.mod: Update api-go
All checks were successful
DCO / DCO (pull_request) Successful in 1m5s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m18s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m24s
Tests and linters / Lint (pull_request) Successful in 2m51s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-26 14:13:49 +03:00
fa89999d91 [#242] pool: Log error that caused healthy status change
All checks were successful
DCO / DCO (pull_request) Successful in 56s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m21s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m21s
Tests and linters / Lint (pull_request) Successful in 2m4s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-22 15:12:27 +03:00
7e94a6adf2 [#237] pool: Return creation epoch from object put
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-22 06:15:23 +00:00
7c06cdff2d [#239] pool/tree: Update tree service client
All checks were successful
DCO / DCO (pull_request) Successful in 3m45s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m46s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m50s
Tests and linters / Lint (pull_request) Successful in 4m53s
Update tree service to fix split tree problem.
Tree intermediate nodes can be duplicated so we must handle this.

Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-07-17 14:32:04 +03:00
560cbbd1f1 [#234] pool: Update token expiration check in cache
All checks were successful
DCO / DCO (pull_request) Successful in 1m51s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m5s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m1s
Tests and linters / Lint (pull_request) Successful in 3m14s
Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-07-05 12:36:17 +03:00
27e965007d [#233] pool: Introduce ape-manager methods
All checks were successful
DCO / DCO (pull_request) Successful in 1m26s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m21s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m33s
Tests and linters / Lint (pull_request) Successful in 2m20s
* Remove GetEACL, SetEACL methods as they are deprecated;
* Fix mock;
* Introduce add/remove/list methods to request ape-manager from
  the client.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-04 01:29:35 +03:00
6a52487edd [#226] pool/tree: Fix handling access denied error
All checks were successful
DCO / DCO (pull_request) Successful in 1m18s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m45s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m52s
Tests and linters / Lint (pull_request) Successful in 3m19s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-05-30 14:59:35 +03:00
c5c6272029 [#221] pool: Make sampler safe for concurrent using
All checks were successful
DCO / DCO (pull_request) Successful in 1m4s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m20s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m31s
Tests and linters / Lint (pull_request) Successful in 2m25s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-05-20 14:14:14 +03:00
20ab57bf7e [#214] object: Implement Get\Head requests for EC object
All checks were successful
DCO / DCO (pull_request) Successful in 1m3s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m15s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m27s
Tests and linters / Lint (pull_request) Successful in 2m13s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 11:07:26 +03:00
3790142b10 [#212] pool: Control sub tree nodes order
All checks were successful
DCO / DCO (pull_request) Successful in 1m37s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m53s
Tests and linters / Tests (1.20) (pull_request) Successful in 2m4s
Tests and linters / Lint (pull_request) Successful in 3m22s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-04-02 17:15:49 +03:00
64b83f8220 [#209] Makefile: Update golangci-lint to 1.56.2
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-03-14 13:35:48 +03:00
a5fab572ff [#206] Add session tokens for container read operations
All checks were successful
Tests and linters / Tests (1.21) (pull_request) Successful in 1m29s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m37s
Tests and linters / Lint (pull_request) Successful in 2m45s
DCO / DCO (pull_request) Successful in 42s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-03-01 15:18:53 +03:00
56debcfa56 [#190] sdk-go: Gofumpt fixes
All checks were successful
DCO / DCO (pull_request) Successful in 56s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m16s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m13s
Tests and linters / Lint (pull_request) Successful in 2m20s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 19:21:20 +03:00
157a9930e8 [#190] sdk-go: Pass user.ID by value
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 19:21:20 +03:00
3787477133 [#189] client: Make PrmDial fields public
All checks were successful
DCO / DCO (pull_request) Successful in 42s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m19s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m29s
Tests and linters / Lint (pull_request) Successful in 2m58s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-14 11:18:00 +03:00
e91d40e250 [#189] client: Make PrmInit fields public for client
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-13 15:18:07 +03:00
ab75edd709 [#191] pool/tree: Support limit request attempts
All checks were successful
DCO / DCO (pull_request) Successful in 1m8s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m17s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m25s
Tests and linters / Lint (pull_request) Successful in 2m35s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-07 14:45:40 +03:00
8999d2f080 [#191] pool/tree: Support request id
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-11-07 14:45:35 +03:00
6fbe1595cb [#121] pool: Refactor PrmObjectSearch usage
All checks were successful
DCO / DCO (pull_request) Successful in 1m8s
Tests and linters / Tests (1.21) (pull_request) Successful in 1m16s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m24s
Tests and linters / Lint (pull_request) Successful in 2m33s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-01 17:45:15 +03:00
51c3618850 [#121] client: Make PrmContainerList fields public
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-10-31 22:55:25 +03:00
84315fab6a [#121] client: Make PrmBalanceGet fields public
Some checks failed
Tests and linters / Tests (1.21) (pull_request) Successful in 1m1s
Tests and linters / Lint (pull_request) Successful in 2m29s
DCO / DCO (pull_request) Successful in 4m13s
Tests and linters / Tests (1.20) (pull_request) Failing after 5m47s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-23 18:53:14 +03:00
4c1feaf2cb [#182] pool: Fix linter error about deprecated methods
All checks were successful
DCO / DCO (pull_request) Successful in 1m11s
Tests and linters / Lint (pull_request) Successful in 1m30s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m25s
Tests and linters / Tests (1.19) (pull_request) Successful in 1m1s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-23 17:40:00 +03:00
5804128ff3 [#121] client: Make PrmObjectPutSingle fields public
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-19 08:38:13 +00:00
abd38c918e [#177] pool: Support NetMapSnapshot method
All checks were successful
DCO / DCO (pull_request) Successful in 1m1s
Tests and linters / Lint (pull_request) Successful in 1m20s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m27s
Tests and linters / Tests (1.19) (pull_request) Successful in 1m36s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-10-18 11:31:53 +03:00
fc4551b843 [#172] pool: Use priority of errors in tree pool
All checks were successful
Tests and linters / Tests (1.20) (pull_request) Successful in 1m8s
Tests and linters / Lint (pull_request) Successful in 1m33s
DCO / DCO (pull_request) Successful in 5m8s
Tests and linters / Tests (1.19) (pull_request) Successful in 1m2s
When retry happens, use priority map to decide
which error to return. Consider network errors
less desirable than business logic errors.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-11 12:00:34 +03:00
eb5288f4a5 [#172] pool: Do more retries on unexpected tree service responses
All checks were successful
DCO / DCO (pull_request) Successful in 2m12s
Tests and linters / Lint (pull_request) Successful in 3m33s
Tests and linters / Tests (1.20) (pull_request) Successful in 3m35s
Tests and linters / Tests (1.19) (pull_request) Successful in 1m3s
1. Try its best by looking for nodes during 'GetNodeByPath'
2. Retry on 'tree not found' and other not found errors

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-06 11:12:48 +03:00
60463871db [#171] pool: Add test for healthy status monitor
All checks were successful
Tests and linters / Tests (1.20) (pull_request) Successful in 1m10s
Tests and linters / Lint (pull_request) Successful in 1m31s
DCO / DCO (pull_request) Successful in 4m12s
Tests and linters / Tests (1.19) (pull_request) Successful in 5m9s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-03 19:47:22 +03:00
8a04638749 [#171] pool: Close only dialed connections
To avoid panics during close operation, close
only dialed connections.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-03 19:47:22 +03:00
ddbfb758c9 [#171] pool: Use dial status to close connections during restarts
Every client restart, pool creates new client instance. If client
failed due to dial error, there was no prior connection and go
routine on a server side. If client failed due to communication
or business logic errors, then server side maintains connection and
client should close it to avoid routine and connection leak.

Dialing is a part of healthcheck, so health status is now a enum
of three values:
- unhealthy due to dial fail,
- unhealthy due to transmission fail,
- healthy.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-10-03 17:00:36 +03:00
99c273f499 [#169] pool: Close inner pools during close routine
All checks were successful
DCO / DCO (pull_request) Successful in 1m9s
Tests and linters / Lint (pull_request) Successful in 1m26s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m27s
Tests and linters / Tests (1.19) (pull_request) Successful in 1m44s
Some apps do not reuse pool instance and expect that
`pool.Close()` free resources. But it didn't actually
close inner SDK clients, so it leads to goroutine leak
in storage.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-09-20 12:16:13 +03:00
303508328a [#121] pool: Refactor PrmSessionCreate usage
All checks were successful
Tests and linters / Lint (pull_request) Successful in 1m2s
Tests and linters / Tests (1.20) (pull_request) Successful in 1m8s
DCO / DCO (pull_request) Successful in 3m55s
Tests and linters / Tests (1.19) (pull_request) Successful in 5m3s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-05 17:32:03 +03:00
5a471e5002 [#121] client: Make PrmObjectDelete fields public
* Introduce buildRequest for PrmObjectDelete
* Refactor the usage of these params in pool

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-04 14:14:22 +00:00
84e7e69f98 [#121] client: Make PrmObjectGet/Head/GetRange fields public
All checks were successful
DCO / DCO (pull_request) Successful in 45s
Tests and linters / Tests (1.19) (pull_request) Successful in 1m10s
Tests and linters / Lint (pull_request) Successful in 2m13s
Tests and linters / Tests (1.20) (pull_request) Successful in 4m49s
* Remove common PrmObjectRead structure
* Introduce buildRequest for PrmObjectGet/Head/GetRange
* Refactor the usage of these params in pool

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-28 11:26:57 +03:00
46a214d065 [#149] pool: Configure homomorphic hash and buffer size
All checks were successful
DCO / DCO (pull_request) Successful in 2m39s
Tests and linters / Lint (pull_request) Successful in 3m23s
Tests and linters / Tests (1.19) (pull_request) Successful in 4m1s
Tests and linters / Tests (1.20) (pull_request) Successful in 4m25s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2023-08-25 09:45:15 +03:00