Commit Graph

982 Commits (v0.37.0-rc.1)

Author SHA1 Message Date
Anton Nikiforov aeeb8193d2 [#676] node: Fix header source creation when checking eacl
DCO action / DCO (pull_request) Successful in 2m55s Details
Build / Build Components (1.20) (pull_request) Successful in 4m53s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m36s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m35s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m7s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m47s Details
Tests and linters / Tests with -race (pull_request) Failing after 10m7s Details
Build / Build Components (1.21) (pull_request) Successful in 11m3s Details
Tests and linters / Lint (pull_request) Successful in 17m34s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-09-06 17:06:54 +03:00
Anton Nikiforov 88d50e4c77 [#656] policer: Add "bad" testcase
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-09-06 08:04:59 +00:00
Airat Arifullin 806cc13d9f [#658] client: Refactor PrmObjectGet/Head/Range usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-30 17:13:23 +00:00
Anton Nikiforov fe5aa06a75 [#665] node: Bind length of copies number to number of replicas
Allow to use one digit in copies number array for backward compatibility.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-30 17:11:55 +00:00
Dmitrii Stepanov 55b82e744b [#529] objectcore: Use common sender classifier
DCO action / DCO (pull_request) Successful in 2m19s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m5s Details
Build / Build Components (1.21) (pull_request) Successful in 4m8s Details
Build / Build Components (1.20) (pull_request) Successful in 4m24s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 4m57s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m43s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m2s Details
Tests and linters / Lint (pull_request) Successful in 5m21s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m17s Details
Use common sender classifier for ACL service and format validator.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-29 10:33:06 +03:00
Dmitrii Stepanov ae81d6660a [#529] objectcore: Fix object content validation
There are old objects where the owner of the object
may not match the one who issued the token.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-29 10:33:06 +03:00
Airat Arifullin 4ea0df77d0 [#574] policer: Check if the container was really removed
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-28 14:21:38 +00:00
Airat Arifullin 554ff2c06b [#574] core: Extend Source interface with DeletionInfo method
* Introduce common method EverExisted
* Define DeletionInfo for struct that must implement Source
* Refactor tree srv

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-28 14:21:38 +00:00
Evgenii Stratonikov f8ba60aa0c [#648] objsvc/delete: Handle errors in Go style
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 09:45:35 +00:00
Evgenii Stratonikov d2084ece41 [#648] objsvc/delete: Remove redundant logs
We never propagate delete requests to the container node, because
tombstone broadcast is done via PUT. No need to pollute logs.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 09:45:35 +00:00
Evgenii Stratonikov 40b556fc19 [#647] objsvc/search: Improve testing coverage
DCO action / DCO (pull_request) Successful in 1m32s Details
Build / Build Components (1.20) (pull_request) Successful in 3m54s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m44s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 4m18s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 4m35s Details
Tests and linters / Lint (pull_request) Successful in 4m42s Details
Vulncheck / Vulncheck (pull_request) Successful in 5m11s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m23s Details
Build / Build Components (1.21) (pull_request) Successful in 8m38s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
Evgenii Stratonikov 4db2cbc927 [#647] objsvc/search: Wrap in uniqueIDWriter during parameter setting
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
Evgenii Stratonikov 966ad22abf [#647] objsvc/search: Simplify error handling
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
Evgenii Stratonikov 56f841b022 [#647] objsvc/search: Remove TraverserGenerator wrapper
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
Evgenii Stratonikov ba58144de1 [#647] objsvc/search: Remove netmap.Source wrapper
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
Evgenii Stratonikov c9e3c9956e [#643] objsvc/put: Unify extraBroadcastEnabled usage
DCO action / DCO (pull_request) Successful in 1m26s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m44s Details
Build / Build Components (1.20) (pull_request) Successful in 3m26s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m51s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m4s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m46s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m23s Details
Build / Build Components (1.21) (pull_request) Successful in 13m1s Details
Tests and linters / Lint (pull_request) Successful in 19m46s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-24 11:03:17 +03:00
Evgenii Stratonikov facd3b2c4b [#643] objsvc/put: Unify placement iterators
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-24 11:03:17 +03:00
Evgenii Stratonikov 3fcf56f2fb [#643] objsvc/put: Copy config to `distributedTarget`
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-24 11:03:17 +03:00
Evgenii Stratonikov 96e690883f [#638] Unify test loggers
In some places we have debug=false, in others debug=true.
Let's be consistent.

Semantic patch:
```
@@
@@
-test.NewLogger(..., false)
+test.NewLogger(..., true)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-23 11:21:05 +00:00
Evgenii Stratonikov 322c1dc273 [#638] Use test.NewLogger() in tests
Semantic patch (restricted to **/*_test.go):
```
@@
@@
+import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger/test"
-import "go.uber.org/zap"
-import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger"

-&logger.Logger{Logger: zap.L()}
+test.NewLogger(t, false)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-23 11:21:05 +00:00
Airat Arifullin 23be3eb627 [#574] tree: Check if container is really removed
Vulncheck / Vulncheck (pull_request) Successful in 2m37s Details
Build / Build Components (1.21) (pull_request) Successful in 3m36s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m36s Details
DCO action / DCO (pull_request) Successful in 4m16s Details
Tests and linters / Lint (pull_request) Successful in 5m34s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m36s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m46s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m21s Details
Build / Build Components (1.20) (pull_request) Successful in 8m1s Details
* Use DeletionInfo method from morph client to check if
  the container has been really removed from neo-go

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-21 12:50:20 +03:00
Evgenii Stratonikov f7042c5a6f [#609] Replace zaptest.NewLogger() with zap.L()
Semantic patch:
```
@@
@@
-import "go.uber.org/zap/zaptest"
+import "go.uber.org/zap"

-zaptest.NewLogger(t)
+zap.L()
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 16:51:10 +00:00
Evgenii Stratonikov e604a3d749 [#607] *: Use zap.Stringer() where possible
Semantic patch:
```
@@
var f expression
var t expression
var a expression
@@
 f(
    ...,
-    zap.String(t, a.String()),
+    zap.Stringer(t, a),
    ...,
)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 16:50:02 +00:00
Evgenii Stratonikov a8de37c8a2 [#607] *: Remove redundant if on error returns
Semantic patch:
```
@@
@@
-if err != nil { return err }
-return nil
+return err
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 16:50:02 +00:00
Evgenii Stratonikov 5a51b78946 [#620] object: Send status response for server-side streams
Previously status responses were wrapped in the gRPC error and thus
couldn't be correctly handled on client.

Introduced in c2617baf63, thanks @ale64bit for having found.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 12:52:38 +00:00
Anton Nikiforov 6407bb5bd1 [#619] node: Fix object put when copies numbers contains only zeros
DCO action / DCO (pull_request) Successful in 3m59s Details
Build / Build Components (1.21) (pull_request) Successful in 4m53s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m43s Details
Build / Build Components (1.20) (pull_request) Successful in 5m57s Details
Tests and linters / Staticcheck (pull_request) Successful in 8m18s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m33s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m41s Details
Tests and linters / Lint (pull_request) Successful in 8m58s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m53s Details
In this case object should placement according to replicas.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-17 11:31:34 +03:00
Alexander Chuprov 21800e9fcc [#162] core: Move literals to constants
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-08-11 15:48:42 +03:00
Alejandro Lopez 5b7e4a51b7 [#481] Update frostfs-sdk-go and error pointer receivers
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-09 10:26:53 +00:00
Anton Nikiforov 8d589314b5 [#560] node: Fix `Put` in multi REP with intersecting sets of nodes
Once the node was processed it skipped, at the step of forming
result in case when all nodes skipped, because processed for
previous REP, service mark the whole request as incomplete.

Example of policies which are unblocked:
- REP 1 REP 1 CBF 1
- REP 4 IN X REP 4 IN Y
  CBF 4
  SELECT 2 FROM FX AS X SELECT 2 FROM FY AS Y
  FILTER Country EQ Russia OR Country EQ Sweden OR Country EQ Finland AS FY
  FILTER Price GE 0 AS FX

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-08 10:22:53 +00:00
Evgenii Stratonikov 1e8b4b8a17 [#557] services: Regenerate stable marshalers
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-02 13:37:40 +00:00
Airat Arifullin b3695411d9 [#553] eacl: Fix bug with casting to ObjectAccessDenied error
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-02 07:22:48 +00:00
Evgenii Stratonikov 05ac9e3637 [#547] objectsvc: Work with `traversal` struct from a single thread
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-31 15:35:15 +00:00
Evgenii Stratonikov 7b0fdf0202 [#533] services: Assume API supports status codes
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-31 15:35:15 +00:00
Dmitrii Stepanov ea32913430 [#543] putsvc: Fix PutSingle implementation
Add Lock and Delete handlers to local PutSingle.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-28 12:09:41 +00:00
Dmitrii Stepanov 99bb488ebd [#539] getsvc: Write payload direct to out stream
Build / Build Components (1.20) (pull_request) Failing after 1s Details
Build / Build Components (1.19) (pull_request) Failing after 3s Details
Tests and linters / Lint (pull_request) Failing after 2s Details
Tests and linters / Tests (1.19) (pull_request) Failing after 1s Details
Tests and linters / Tests (1.20) (pull_request) Failing after 2s Details
Tests and linters / Tests with -race (pull_request) Failing after 1s Details
Vulncheck / Vulncheck (pull_request) Failing after 2s Details
Tests and linters / Staticcheck (pull_request) Failing after 2s Details
To reduce memory allocations.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-27 17:02:08 +03:00
Dmitrii Stepanov 286242cad0 [#539] getsvc: Use buffer to assemble object
To reduce memory consumption.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-27 17:02:08 +03:00
Evgenii Stratonikov 5ff82ff04f [#6] services/object: Simplify local/remote targets
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Tests (1.19) (pull_request) Successful in 14m18s Details
Vulncheck / Vulncheck (pull_request) Failing after 2m23s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m8s Details
Build / Build Components (1.19) (pull_request) Successful in 3m26s Details
Build / Build Components (1.20) (pull_request) Successful in 2m56s Details
Tests and linters / Tests with -race (pull_request) Successful in 21m39s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 22m36s Details
Tests and linters / Lint (pull_request) Successful in 1m39s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
We do not use the return result from Close() and we always execute both
methods in succession. It makes sense to unite them.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-21 18:39:12 +03:00
Evgenii Stratonikov 448b48287c [#6] services/util: Do not panic in sign function
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-21 18:39:12 +03:00
Evgenii Stratonikov c2617baf63 [#6] services/util: Remove remaining stream wrappers
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-21 18:39:09 +03:00
Evgenii Stratonikov 372160d048 [#6] services/util: Remove `SignService.HandleUnaryRequest`
There is no need in a wrapper with many from-`interface{}` conversions.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-21 18:36:08 +03:00
Evgenii Stratonikov fef172c5b0 [#6] services/util: Simplify `response.Service`
It has only 1 parameter, which is obligatory.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-21 18:29:57 +03:00
Dmitrii Stepanov eed594431f [#335] treesvc: Add GetSubTree ordering unit test
Build / Build Components (1.20) (pull_request) Successful in 3m47s Details
Build / Build Components (1.19) (pull_request) Successful in 4m2s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 3m8s Details
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Staticcheck (pull_request) Successful in 3m48s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m59s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 15m36s Details
Tests and linters / Lint (pull_request) Successful in 18m12s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-20 10:14:10 +03:00
Dmitrii Stepanov b4e72a2dfd [#335] treesvc: Sort nodes by Filename in GetSubTree
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-20 10:14:10 +03:00
Dmitrii Stepanov f0355a453e [#463] policer: Remove capacity rebalance logic
ci/woodpecker/push/pre-commit Pipeline was successful Details
Current implementation has some quirks. For example,
using only half of object.put.pool_size_remote threads
tells replicator that is node is 50% loaded,
but in reality we could be putting lot's of big objects.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-18 10:52:12 +00:00
Dmitrii Stepanov 8b78db74bc [#463] replicator: Add tracing span
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-18 10:52:12 +00:00
Dmitrii Stepanov 8966dd8e35 [#463] putsvc: Use PutSingle RPC for remote target
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-18 10:52:12 +00:00
Alejandro Lopez 3b66f98f27 [#519] Use Address.Equals in policer tests
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-17 10:45:07 +00:00
Evgenii Stratonikov 8a9fc2c372 [#510] treesvc: Rename `tableFromBearer` to `useBearer`
ci/woodpecker/push/pre-commit Pipeline was successful Details
With impersonation, the old name is no longer descriptive.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-12 10:23:21 +00:00
Evgenii Stratonikov b8bcfac531 [#510] treesvc: Fix panic in bearer token processing
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-12 10:23:21 +00:00
Dmitrii Stepanov 24eb988897 [#294] deletesvc: Drop cast
ci/woodpecker/push/pre-commit Pipeline was successful Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00