Commit graph

845 commits

Author SHA1 Message Date
749b4e9ab5 [#344] netmap: Add method Clone
All checks were successful
DCO / DCO (pull_request) Successful in 26s
Code generation / Generate proto (pull_request) Successful in 30s
Tests and linters / Tests (pull_request) Successful in 41s
Tests and linters / Lint (pull_request) Successful in 2m25s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2025-03-07 18:02:02 +03:00
f70c0c9081 [#300] pool: Remove obvious comments
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-03-07 11:45:31 +00:00
17177697b5 [#300] pool/cm: Remove unused mutex in 'statistics'
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-03-07 11:45:31 +00:00
56046935b0 [#300] pool: Move 'clientWrapper' to separate file
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-03-07 11:45:31 +00:00
cee7f9de47 [#300] pool: Move 'connectionManager' to separate file
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-03-07 11:45:31 +00:00
d77a8742bc [#300] pool: Move 'healthCheck' to separate file
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-03-07 11:45:31 +00:00
b083bf8546 [#300] pool: Extract healthCheck functionality from 'connectionManager' to 'healthCheck'
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-03-07 11:45:31 +00:00
b480df99ca [#300] pool: Extract connection handler functionality to 'connectionManager'
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-03-07 11:45:31 +00:00
69b0711d12 [#342] client: Disable service config query
By default, gRPC fetches TXT report while resolving a domain.
0914bba6c5/internal/resolver/dns/dns_resolver.go (L336)

This leads to a hanging dial if DNS is unavailable, even though the host
may be specified in `/etc/hosts` (hello, localhost!).

Use `grpc.WithDisableServiceConfig()` to override the default.
This option seems impossible to override with `WithGRPCDialOpts()`,
but we do not use service config anyway.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-03-06 09:24:16 +00:00
7a37613988 [#339] pool/tree: Improve code after review
All checks were successful
DCO / DCO (pull_request) Successful in 29s
Code generation / Generate proto (pull_request) Successful in 39s
Tests and linters / Tests (pull_request) Successful in 46s
Tests and linters / Lint (pull_request) Successful in 1m34s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2025-03-05 14:40:45 +03:00
d592bb931e [#339] pool/tree: Configure circuit breaker parameters
Some checks failed
Code generation / Generate proto (pull_request) Successful in 29s
DCO / DCO (pull_request) Successful in 32s
Tests and linters / Lint (pull_request) Failing after 42s
Tests and linters / Tests (pull_request) Successful in 42s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2025-03-04 14:49:07 +03:00
06ef257ddc [#339] pool/tree: Do not lock mutex on circuit break function
Some checks failed
DCO / DCO (pull_request) Successful in 29s
Code generation / Generate proto (pull_request) Successful in 30s
Tests and linters / Lint (pull_request) Failing after 39s
Tests and linters / Tests (pull_request) Successful in 44s
Circuit break function may take some time to execute so it should
not be executed when lock is enabled.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2025-03-04 14:41:01 +03:00
2d08fa5240 [#339] pool/tree: Close replaced connection in client map
There is a race condition: multiple clients are created and dialled,
but only one is stored in the map. Others are remaining active but not
used.

With this change, new connection replaces old connection and closes
it.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2025-03-04 14:41:01 +03:00
c5991fc66d [#339] pool/tree: Fix linter issues
All checks were successful
DCO / DCO (pull_request) Successful in 28s
Code generation / Generate proto (pull_request) Successful in 32s
Tests and linters / Tests (pull_request) Successful in 44s
Tests and linters / Lint (pull_request) Successful in 1m27s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2025-03-04 12:22:38 +03:00
f78fb6dcb0 [#339] pool/tree: Make circuit breaker more generic
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2025-03-04 12:21:11 +03:00
c8d71c450a [#339] pool/tree: Add circuit breaker
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2025-03-04 12:20:58 +03:00
2b8329e026 [#336] pool/tree: Increase test coverage in TestStreamRetry
All checks were successful
DCO / DCO (pull_request) Successful in 26s
Code generation / Generate proto (pull_request) Successful in 33s
Tests and linters / Tests (pull_request) Successful in 44s
Tests and linters / Lint (pull_request) Successful in 1m36s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2025-02-28 12:32:56 +03:00
ada0513504 [#336] pool/tree: Do probe in getSubTree to handle error in advance
All checks were successful
DCO / DCO (pull_request) Successful in 28s
Code generation / Generate proto (pull_request) Successful in 33s
Tests and linters / Tests (pull_request) Successful in 45s
Tests and linters / Lint (pull_request) Successful in 1m35s
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2025-02-28 10:46:01 +03:00
c3f7378887 [#294] user: Implement Cmp() function for ID struct
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-02-17 15:22:55 +00:00
bcb5fd22d4 [#294] container: Implement Cmp() function for ID struct
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-02-17 15:22:55 +00:00
148a341b6f [#294] object: Implement Cmp() function for ID struct
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-02-17 15:22:55 +00:00
56892e48ac
[#334] rpc: Use DeadlineExceeded error when creating stream failed by timeout
All checks were successful
DCO / DCO (pull_request) Successful in 28s
Code generation / Generate proto (pull_request) Successful in 32s
Tests and linters / Tests (pull_request) Successful in 45s
Tests and linters / Lint (pull_request) Successful in 1m53s
According to https://pkg.go.dev/context#pkg-variables, ContextCancelled
should be returned when the context is canceled for some reason other
than its deadline passing. So creating gRPC stream with dial timeout fits
better for context.DeadlineExceeded.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-02-17 16:06:11 +03:00
7bdc78f2b5
[#332] client/status: Support RESOURCE_EXHAUSTED status
All checks were successful
DCO / DCO (pull_request) Successful in 30s
Code generation / Generate proto (pull_request) Successful in 34s
Tests and linters / Tests (pull_request) Successful in 46s
Tests and linters / Lint (pull_request) Successful in 1m53s
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2025-02-06 11:38:57 +03:00
cf0bbd03ae
[#332] api/status: Regenerate common statuses
Introduced a new common status `RESOURCE_EXHAUSTED`.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2025-02-06 11:38:45 +03:00
5a35fa4353 [#331] pool: Fix 'sortServers' in tree pool server test
All checks were successful
DCO / DCO (pull_request) Successful in 30s
Code generation / Generate proto (pull_request) Successful in 36s
Tests and linters / Tests (pull_request) Successful in 47s
Tests and linters / Lint (pull_request) Successful in 1m37s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2025-02-05 10:44:03 +03:00
4ecbfb0edf [#331] pool: Add mocked test tree service and check goroutine leak
All checks were successful
DCO / DCO (pull_request) Successful in 29s
Code generation / Generate proto (pull_request) Successful in 32s
Tests and linters / Tests (pull_request) Successful in 46s
Tests and linters / Lint (pull_request) Successful in 1m35s
Use real gRPC connection in new mocked tree service.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2025-02-04 21:15:15 +03:00
f08a7f0b3c [#331] pool: Avoid connection leak in tree pool with netmap support
To avoid connection leak, call `close()` immediately after connection
is established. In regular tree pool, unhealthy connections are handled
by background goroutine which calls `redialIfNecessary()` to reestablish
connection. Here it is not viable so connection must be close.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2025-02-04 21:15:10 +03:00
8389887a34
[#319] object/transformer: Add expiration epoch to each part
All checks were successful
DCO / DCO (pull_request) Successful in 32s
Code generation / Generate proto (pull_request) Successful in 42s
Tests and linters / Tests (pull_request) Successful in 46s
Tests and linters / Lint (pull_request) Successful in 1m47s
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2025-02-02 18:14:21 +03:00
593dd77d84
[#327] rpc: Fix mem leak
All checks were successful
DCO / DCO (pull_request) Successful in 26s
Tests and linters / Tests (pull_request) Successful in 1m20s
Code generation / Generate proto (pull_request) Successful in 33s
Tests and linters / Lint (pull_request) Successful in 2m16s
gRPC stream must be closed by `cancel` to prevent memleak.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-01-30 12:53:43 +03:00
2786fadb25 [#326] pool: Add test for concurrent client deletion
All checks were successful
DCO / DCO (pull_request) Successful in 26s
Code generation / Generate proto (pull_request) Successful in 33s
Tests and linters / Tests (pull_request) Successful in 42s
Tests and linters / Lint (pull_request) Successful in 1m52s
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2025-01-29 20:16:59 +03:00
37350dbb1e [#326] pool: Fix panic that causes mutex deadlock
Two concurrent 'deleteClientFromMap' calls for
the same client may produce panic and deadlock.

First goroutine acquires lock, removes element
from the map, releases lock.

Second goroutine acquires lock, and throws panic
while trying to call 'close()' on empty struct.
Lock is never released and it causes deadlock for
other goroutines.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2025-01-29 20:14:55 +03:00
d195cb5104
[#324] rpc: Fix mem leak
All checks were successful
DCO / DCO (pull_request) Successful in 25s
Code generation / Generate proto (pull_request) Successful in 30s
Tests and linters / Tests (pull_request) Successful in 46s
Tests and linters / Lint (pull_request) Successful in 1m50s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-01-29 16:34:30 +03:00
00cebd297f [#303] user: Make user.ID a util.Uint160
All checks were successful
DCO / DCO (pull_request) Successful in 1m36s
Code generation / Generate proto (pull_request) Successful in 1m25s
Tests and linters / Tests (pull_request) Successful in 2m16s
Tests and linters / Lint (pull_request) Successful in 3m6s
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2025-01-28 18:01:21 +03:00
761d087b93 [#302] user: Make ScriptHash return no error
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2025-01-27 19:13:52 +03:00
9d7f7bd04f
[#320] .forgejo: Add generate-proto action
All checks were successful
DCO / DCO (pull_request) Successful in 28s
Code generation / Generate proto (pull_request) Successful in 34s
Tests and linters / Tests (pull_request) Successful in 44s
Tests and linters / Lint (pull_request) Successful in 2m4s
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-01-21 14:35:18 +03:00
7c577ecaeb
[#320] api: Run 'make proto'
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-01-21 14:33:23 +03:00
4b4f35bbac
[#320] make/proto: Add installing protoc-gen-go-grpc
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-01-21 14:32:02 +03:00
328d214d2d
[#317] api: Revert easyproto marshaler usage
All checks were successful
DCO / DCO (pull_request) Successful in 26s
Tests and linters / Tests (pull_request) Successful in 46s
Tests and linters / Lint (pull_request) Successful in 1m52s
It has caused a noticeable degradation in node.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-01-09 11:46:09 +03:00
bab4d5a692
[#317] go.mod: Update protobuf version
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-01-09 11:46:08 +03:00
42a0fc8c13 [#305] tree/pool: Add flag to use net map to prioritize tree services
All checks were successful
DCO / DCO (pull_request) Successful in 3m38s
Tests and linters / Tests (pull_request) Successful in 3m59s
Tests and linters / Lint (pull_request) Successful in 5m30s
New version of tree/pool selects tree service connection to make request based on the current net map and container placement policy

Signed-off-by: Marina Biryukova <m.biryukova@yadro.com>
2024-12-18 09:23:44 +03:00
dcd4ea334e
[#315] api: Restore test.proto file
All checks were successful
Tests and linters / Lint (pull_request) Successful in 3m6s
DCO / DCO (pull_request) Successful in 4m10s
Tests and linters / Tests (pull_request) Successful in 4m33s
Introduced in #281.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-16 10:42:16 +03:00
c757365752 [#309] Refine CODEOWNERS settings
Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
2024-12-12 12:34:51 +00:00
902f32eeab [#312] pool: Ignore ECInfoErr errors in GetSplitInfo
* Ignore `ECInfoError` errors after raw `HeadObject` request

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-12-12 10:12:24 +00:00
d86cf85b39 [#311] client/container: Remove outdated comment
All checks were successful
DCO / DCO (pull_request) Successful in 1m34s
Tests and linters / Tests (pull_request) Successful in 2m3s
Tests and linters / Lint (pull_request) Successful in 2m16s
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-12-11 15:53:53 +03:00
3dbae11efa
[#310] client/container: Remove commented code
All checks were successful
DCO / DCO (pull_request) Successful in 1m35s
Tests and linters / Tests (pull_request) Successful in 2m5s
Tests and linters / Lint (pull_request) Successful in 2m15s
Refs #291.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-11 14:23:44 +03:00
c4463df8d4 [#291] container: Rename field Account to OwnerID in PrmContainerList
All checks were successful
DCO / DCO (pull_request) Successful in 1m27s
Tests and linters / Tests (pull_request) Successful in 1m47s
Tests and linters / Lint (pull_request) Successful in 2m12s
Renaming this field so that it matches PrmContainerListStream.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-12-10 13:49:38 +03:00
852dac1476 [#291] container: Add ListStream method
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-12-10 13:49:38 +03:00
e3026d3f4c [#291] container: Add ListStream to api
Since api was moved from `TrueCloudLab/frostfs-api-go` to sdk,
moving changes from TrueCloudLab/frostfs-api-go#125 here.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-12-10 12:05:56 +03:00
81c423e709 [#301] rpc: Make client stream initialization get cancelled by dial timeout
* `c.conn` may be already invalidated but the rpc client can't detect this.
  `NewStream` may hang trying to open a stream with invalidated connection.
  Using timer with `dialTimeout` for `NewStream` fixes this problem.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-12-06 09:49:44 +00:00
f7da6ba99c
[#306] netmap: Allow to select empty set of nodes
All checks were successful
Tests and linters / Lint (pull_request) Successful in 2m18s
DCO / DCO (pull_request) Successful in 2m19s
Tests and linters / Tests (pull_request) Successful in 2m21s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-05 15:33:29 +03:00