Commit graph

1115 commits

Author SHA1 Message Date
75a1a95c2c [#986] tree: Skip ACL checks if basicACL mask is unset
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:05:57 +00:00
b1d171c261 [#986] container: Interpret APE NoRuleFound as request deny
* If APE check returns NoRuleFound, then it is taken for request deny.
* Add more unit-test for ape container middleware.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:05:57 +00:00
7cc368e188 [#986] object: Introduce soft ape checks
* Soft APE check means that APE should allow request even
  it gets status NoRuleFound for a request. Otherwise,
  it is interpreted as Deny.
* Soft APE check is performed if basic ACL mask is not set.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:05:57 +00:00
dad56d2e98 Revert "[#972] Use min/max builtins"
This reverts commit 89784b2e0a.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
3359349acb Revert "[#972] Use slices.Sort* when useful"
This reverts commit b871d7a5e8.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
b871d7a5e8 [#972] Use slices.Sort* when useful
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
89784b2e0a [#972] Use min/max builtins
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
2680192ba0 [#988] objectSvc: Fix SetMarshalData for PutSingle
All checks were successful
DCO action / DCO (pull_request) Successful in 5m10s
Vulncheck / Vulncheck (pull_request) Successful in 5m24s
Build / Build Components (1.21) (pull_request) Successful in 7m40s
Build / Build Components (1.20) (pull_request) Successful in 7m50s
Tests and linters / Staticcheck (pull_request) Successful in 7m52s
Tests and linters / Lint (pull_request) Successful in 9m8s
Tests and linters / Tests with -race (pull_request) Successful in 10m54s
Tests and linters / Tests (1.20) (pull_request) Successful in 11m12s
Tests and linters / Tests (1.21) (pull_request) Successful in 11m7s
After api-go update it is required to pass marshal data
to `SetMarshalData`.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 17:21:08 +03:00
db67c21d55 [#947] engine: Evacuate trees to remote nodes
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:33:15 +03:00
15d853ea22 [#947] controlSvc: Return tree evacuation stat
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:39 +03:00
b3f3505ada [#947] cli: Allow to specify evacuation scope
It may be required to evacuate only objects or only tree or all, so
now it spossible to specify.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00
a6eb66bf9c [#947] evacuate: Refactor evacuate parameters
Drop methods to make it easier to extend.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00
edbe06e07e [#956] policer/test: Reuse testPool helper
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 15:49:43 +00:00
cbfeb72466 [#956] policer: Remove WithMaxCapacity option
We already provide the pool and this argument is used only for
preallocation. No functional changes.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 15:49:43 +00:00
c3fa902780 [#969] policer: Restrict the number of remembered errors
All checks were successful
DCO action / DCO (pull_request) Successful in 4m23s
Vulncheck / Vulncheck (pull_request) Successful in 4m57s
Build / Build Components (1.21) (pull_request) Successful in 6m3s
Build / Build Components (1.20) (pull_request) Successful in 9m2s
Tests and linters / Staticcheck (pull_request) Successful in 10m4s
Tests and linters / Lint (pull_request) Successful in 10m51s
Tests and linters / Tests (1.21) (pull_request) Successful in 10m44s
Tests and linters / Tests (1.20) (pull_request) Successful in 10m56s
Tests and linters / Tests with -race (pull_request) Successful in 10m57s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 10:10:41 +03:00
6010dfdf3d [#969] policer: Make error skip thread-safe
Introduces in afd2ba9a66.
Refs #914

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 10:10:41 +03:00
a6c9a337cd [#965] morph: Get rid of container.List invocations
ContainersOf() is better in almost every aspect, besides creating a
session when the containers number is between 1024 and 2048 (prefetch
script does limited unwrapping). Making List() private helps to ensure
it is no longer used and can be safely removed in future.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-07 08:56:27 +00:00
9ba48c582d [#917] engine: Allow to detach shards
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-06 14:49:47 +03:00
4358d3c423 [#917] controlSvc: Add DetachShards handler
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-06 14:47:52 +03:00
afd2ba9a66 [#110] Add check for repeated error log in policer
All checks were successful
DCO action / DCO (pull_request) Successful in 2m55s
Vulncheck / Vulncheck (pull_request) Successful in 3m22s
Tests and linters / Staticcheck (pull_request) Successful in 4m10s
Build / Build Components (1.20) (pull_request) Successful in 4m58s
Build / Build Components (1.21) (pull_request) Successful in 4m54s
Tests and linters / Lint (pull_request) Successful in 5m47s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m45s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m58s
Tests and linters / Tests with -race (pull_request) Successful in 7m42s
processObject() returns 3 types of errors: container not found errors,
could not get container error and placement vector building error. Every
error will occur for all objects in container simultaneously, so we can
log each error once and safely ignore the rest.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-02-06 00:56:41 +03:00
602ee11123 [#934] containersvc: Marhal public key in short format for APE
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-02 17:51:38 +00:00
9916598dfb [#922] control: Extend api with ListOverrideDefinedTargets
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
95e15f499f [#922] Update files generated by protoc
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
2cb04379a4 [#922] go.mod: Update APE
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
a5446bc17d [#952] object: Pass namespace within context in ACL service
All checks were successful
DCO action / DCO (pull_request) Successful in 6m23s
Vulncheck / Vulncheck (pull_request) Successful in 7m3s
Build / Build Components (1.21) (pull_request) Successful in 8m21s
Build / Build Components (1.20) (pull_request) Successful in 8m31s
Tests and linters / Staticcheck (pull_request) Successful in 11m1s
Tests and linters / Lint (pull_request) Successful in 11m26s
Tests and linters / Tests (1.20) (pull_request) Successful in 12m51s
Tests and linters / Tests (1.21) (pull_request) Successful in 13m14s
Tests and linters / Tests with -race (pull_request) Successful in 13m31s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-02 14:48:11 +03:00
6534252c22 [#799] policer: Refactor method processNodes
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-01 17:49:22 +00:00
5be2af881a [#934] container: Make container APE middleware read namespaces
* Those methods that can access already existing containers and thus
  can get container properties should read namespace from Zone
  property. If Zone is not set, take a namespace for root.
* Otherwise, define namespaces by owner ID via frostfs-id contract.
* Improve unit-tests, consider more cases.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-01 17:38:24 +00:00
4352bd0e8e [#934] ape: Transform empty namespace within chainbase
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-01 17:38:24 +00:00
483a67b170 [#937] ape: Validate chain resource name
All checks were successful
DCO action / DCO (pull_request) Successful in 2m13s
Build / Build Components (1.20) (pull_request) Successful in 3m38s
Vulncheck / Vulncheck (pull_request) Successful in 3m13s
Tests and linters / Staticcheck (pull_request) Successful in 6m15s
Build / Build Components (1.21) (pull_request) Successful in 7m11s
Tests and linters / Lint (pull_request) Successful in 9m35s
Tests and linters / Tests (1.21) (pull_request) Successful in 13m56s
Tests and linters / Tests (1.20) (pull_request) Successful in 14m14s
Tests and linters / Tests with -race (pull_request) Successful in 6m2s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-31 11:34:35 +03:00
c441296592 [#930] policer: Release task pool when context cancelled
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-31 08:30:34 +00:00
6e2cc32768 [#681] objsvc: Validate session token owner for local sessions
Previously, the check was in place only when session token was missing.
Format validator checks are applied only to fully-prepared object, so
this lead to the following situation:
1. Object is put locally with malformed token, because there are no
   checks.
2. Object cannot be replicated, because the token is malformed.

This is now fixed and token check is done before any payload receival.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-26 08:52:29 +00:00
b6fc3321c5 [#876] Fix linters
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-25 20:26:13 +03:00
f2f3294fc3 [#919] ape: Improve error messages in ape service
* Wrap all APE middleware errors in apeErr that
  makes errors more explicit with status AccessDenied.
* Use denyingRuleErr for denying status from chain router.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-23 08:11:24 +00:00
be33070550 [#910] control: Take empty namespace for 'root'
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-19 06:22:10 +00:00
96b020626f [#915] ape: Fix method name in getStreamBasicChecker
All checks were successful
DCO action / DCO (pull_request) Successful in 2m2s
Build / Build Components (1.21) (pull_request) Successful in 2m17s
Build / Build Components (1.20) (pull_request) Successful in 3m2s
Vulncheck / Vulncheck (pull_request) Successful in 2m39s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m54s
Tests and linters / Staticcheck (pull_request) Successful in 5m49s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m11s
Tests and linters / Lint (pull_request) Successful in 6m44s
Tests and linters / Tests with -race (pull_request) Successful in 6m32s
* Replace incorrect MethodGetContainer by MethodGetObject constant.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-16 23:52:37 +03:00
c8baf76fae [#872] object: Introduce APE middlewar for object service
All checks were successful
DCO action / DCO (pull_request) Successful in 2m4s
Vulncheck / Vulncheck (pull_request) Successful in 3m12s
Build / Build Components (1.21) (pull_request) Successful in 4m1s
Build / Build Components (1.20) (pull_request) Successful in 4m13s
Tests and linters / Staticcheck (pull_request) Successful in 4m3s
Tests and linters / Lint (pull_request) Successful in 8m7s
Tests and linters / Tests (1.20) (pull_request) Successful in 8m14s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m18s
Tests and linters / Tests with -race (pull_request) Successful in 8m24s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-12 18:41:35 +03:00
52ffa9f164 [#891] getSvc: Refactor Get service V2 creation
All checks were successful
DCO action / DCO (pull_request) Successful in 2m46s
Vulncheck / Vulncheck (pull_request) Successful in 2m53s
Build / Build Components (1.20) (pull_request) Successful in 4m31s
Build / Build Components (1.21) (pull_request) Successful in 4m26s
Tests and linters / Lint (pull_request) Successful in 6m9s
Tests and linters / Staticcheck (pull_request) Successful in 6m7s
Tests and linters / Tests (1.20) (pull_request) Successful in 8m27s
Tests and linters / Tests (1.21) (pull_request) Successful in 9m4s
Tests and linters / Tests with -race (pull_request) Successful in 9m48s
Use arguments for mandatory fields.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-12 13:35:38 +03:00
394f086fe2 [#891] getSvc: Fix get range hash implementation
Get range can perform GET request, so this request must be done
from container node to not to get access denied error.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-12 13:35:38 +03:00
f1b2b8bffa [#895] test: Fix NewLogger arguments list
`debug` is always true.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-11 12:32:09 +00:00
a8e52ef7aa [#898] control: Fix codes for returning APE errors
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-11 12:31:31 +00:00
c19396d203 [#885] control: Make chain id bytes in grpc
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-01-11 07:24:22 +00:00
5c0a736a25 [#899] containerSvc: Fix invalid session token type
Some checks failed
DCO action / DCO (pull_request) Successful in 1m23s
Vulncheck / Vulncheck (pull_request) Successful in 3m29s
Tests and linters / Tests (1.21) (pull_request) Failing after 3m58s
Build / Build Components (1.21) (pull_request) Successful in 3m46s
Build / Build Components (1.20) (pull_request) Successful in 3m52s
Tests and linters / Lint (pull_request) Successful in 4m48s
Tests and linters / Staticcheck (pull_request) Successful in 5m5s
Tests and linters / Tests (1.20) (pull_request) Successful in 7m4s
Tests and linters / Tests with -race (pull_request) Successful in 8m36s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-10 18:37:54 +03:00
581887148a [#569] cli: Add control shards writecache seal command
All checks were successful
DCO action / DCO (pull_request) Successful in 2m46s
Vulncheck / Vulncheck (pull_request) Successful in 3m3s
Build / Build Components (1.21) (pull_request) Successful in 3m53s
Build / Build Components (1.20) (pull_request) Successful in 4m0s
Tests and linters / Staticcheck (pull_request) Successful in 5m31s
Tests and linters / Lint (pull_request) Successful in 6m17s
Tests and linters / Tests (1.20) (pull_request) Successful in 12m47s
Tests and linters / Tests (1.21) (pull_request) Successful in 13m14s
Tests and linters / Tests with -race (pull_request) Successful in 13m12s
It does the same as `control shards flush-writecache --seal`, but
has better name.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 16:05:37 +03:00
0cb0fc1735 [#569] writecache: Allow to seal writecache after flush
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 16:05:37 +03:00
b118734909 [#890] getsvc: Log node PK
Some checks failed
DCO action / DCO (pull_request) Successful in 13m55s
Vulncheck / Vulncheck (pull_request) Successful in 16m51s
Build / Build Components (1.20) (pull_request) Successful in 17m52s
Build / Build Components (1.21) (pull_request) Successful in 18m9s
Tests and linters / Tests (1.20) (pull_request) Failing after 24m34s
Tests and linters / Tests (1.21) (pull_request) Failing after 25m56s
Tests and linters / Staticcheck (pull_request) Successful in 26m46s
Tests and linters / Lint (pull_request) Successful in 28m6s
Tests and linters / Tests with -race (pull_request) Failing after 33m35s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 14:24:13 +03:00
764f70634d [#881] containerSvc: Add APE validation
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 2m40s
DCO action / DCO (pull_request) Successful in 2m27s
Build / Build Components (1.21) (pull_request) Successful in 3m22s
Tests and linters / Lint (pull_request) Successful in 4m52s
Tests and linters / Staticcheck (pull_request) Successful in 4m46s
Build / Build Components (1.20) (pull_request) Successful in 4m54s
Tests and linters / Tests (1.20) (pull_request) Successful in 11m59s
Tests and linters / Tests (1.21) (pull_request) Successful in 12m38s
Tests and linters / Tests with -race (pull_request) Successful in 13m10s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-27 11:05:34 +03:00
eab981bf1a [#876] Fix linter error
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-12-21 18:21:37 +03:00
bdd43f6211 [#869] object: Pass just CID to chain router
* Do not convert CID from request to native-schema resource
  format - this step is unneccessary for APE.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-14 11:01:20 +00:00
11add38e87 [#857] golangci: Add protogetter linter
All checks were successful
DCO action / DCO (pull_request) Successful in 1m38s
Vulncheck / Vulncheck (pull_request) Successful in 3m2s
Build / Build Components (1.21) (pull_request) Successful in 3m51s
Build / Build Components (1.20) (pull_request) Successful in 4m3s
Tests and linters / Staticcheck (pull_request) Successful in 5m17s
Tests and linters / Lint (pull_request) Successful in 6m11s
Tests and linters / Tests (1.20) (pull_request) Successful in 11m7s
Tests and linters / Tests (1.21) (pull_request) Successful in 11m32s
Tests and linters / Tests with -race (pull_request) Successful in 12m22s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-12 16:27:02 +03:00
681b2c5fd4 [#825] policer: Do not drop required linking objects
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-12 11:04:03 +00:00
0f45e3d344 [#804] ape: Implement boltdb storage for local overrides
All checks were successful
DCO action / DCO (pull_request) Successful in 2m10s
Vulncheck / Vulncheck (pull_request) Successful in 3m26s
Build / Build Components (1.20) (pull_request) Successful in 5m41s
Build / Build Components (1.21) (pull_request) Successful in 5m44s
Tests and linters / Staticcheck (pull_request) Successful in 7m10s
Tests and linters / Lint (pull_request) Successful in 8m14s
Tests and linters / Tests (1.21) (pull_request) Successful in 14m24s
Tests and linters / Tests (1.20) (pull_request) Successful in 14m41s
Tests and linters / Tests with -race (pull_request) Successful in 14m38s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 19:08:41 +03:00
e361e017f3 [#842] control: Pass target instead resource name
* Update policy-engine package version in go.mod, go.sum.
* Refactor CheckIfRequestPermitted: pass container target
  instead container ID.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 14:21:55 +00:00
39060382a1 [#842] control: Recieve target in gRPC methods for APE managing
* Introduce Target type and pass it to all gRPC methods
  for APE chain managing instead CID.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 14:21:55 +00:00
26e4f7005c [#741] treesvc: Refactor tree sync
Fix linter issues.
Add error logging.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-30 12:45:02 +00:00
b21be1abdd [#741] treesvc: Do not update sync height if some node is unavailable
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-30 12:45:02 +00:00
b215817e14 [#741] treesvc: Remove unused height variables
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-30 12:45:02 +00:00
c516c7c5f4 [#821] node: Pass user.ID by value
All checks were successful
DCO action / DCO (pull_request) Successful in 3m45s
Build / Build Components (1.21) (pull_request) Successful in 5m18s
Build / Build Components (1.20) (pull_request) Successful in 5m28s
Tests and linters / Tests (1.20) (pull_request) Successful in 7m30s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m42s
Tests and linters / Lint (pull_request) Successful in 8m25s
Vulncheck / Vulncheck (pull_request) Successful in 9m22s
Tests and linters / Staticcheck (pull_request) Successful in 10m57s
Tests and linters / Tests with -race (pull_request) Successful in 16m53s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-23 10:21:07 +03:00
4d5be5ccb5 [#811] ape: Update policy-engine module version and rebase
All checks were successful
DCO action / DCO (pull_request) Successful in 4m23s
Vulncheck / Vulncheck (pull_request) Successful in 5m31s
Build / Build Components (1.21) (pull_request) Successful in 7m33s
Build / Build Components (1.20) (pull_request) Successful in 7m40s
Tests and linters / Staticcheck (pull_request) Successful in 8m22s
Tests and linters / Lint (pull_request) Successful in 9m23s
Tests and linters / Tests with -race (pull_request) Successful in 11m20s
Tests and linters / Tests (1.21) (pull_request) Successful in 11m32s
Tests and linters / Tests (1.20) (pull_request) Successful in 11m41s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-16 11:31:37 +03:00
8088063195 [#787] netmap: Refactor NewEpoch method
All checks were successful
DCO action / DCO (pull_request) Successful in 3m58s
Vulncheck / Vulncheck (pull_request) Successful in 4m51s
Build / Build Components (1.21) (pull_request) Successful in 5m47s
Build / Build Components (1.20) (pull_request) Successful in 5m57s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m25s
Tests and linters / Staticcheck (pull_request) Successful in 6m31s
Tests and linters / Lint (pull_request) Successful in 7m15s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m16s
Tests and linters / Tests with -race (pull_request) Successful in 8m30s
Split for user and control methods.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:22:31 +03:00
2393d13e4d [#787] morph: Return VUB for IR service calls
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:13:03 +03:00
bdfa523487 [#787] proto: Add VUB field for IR service
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:13:03 +03:00
0f75e48138 [#796] policer: Fix tombstone objects replication
Tombstone objects must be replicated to all container nodes.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 13:39:33 +03:00
7cdae4f660 [#792] proto: Regenerate with fixed version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 10:09:13 +00:00
9133b4389e [#788] objectsvc: Fix formatting (gofumpt)
All checks were successful
DCO action / DCO (pull_request) Successful in 3m19s
Vulncheck / Vulncheck (pull_request) Successful in 3m40s
Build / Build Components (1.21) (pull_request) Successful in 4m17s
Build / Build Components (1.20) (pull_request) Successful in 4m32s
Tests and linters / Staticcheck (pull_request) Successful in 4m46s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m9s
Tests and linters / Lint (pull_request) Successful in 5m28s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m24s
Tests and linters / Tests with -race (pull_request) Successful in 7m38s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 10:27:32 +03:00
3534d6d05b [#794] objectsvc: Return accidentally removed acl checks for Head
All checks were successful
DCO action / DCO (pull_request) Successful in 1m42s
Vulncheck / Vulncheck (pull_request) Successful in 3m23s
Build / Build Components (1.21) (pull_request) Successful in 4m22s
Build / Build Components (1.20) (pull_request) Successful in 5m44s
Tests and linters / Staticcheck (pull_request) Successful in 6m3s
Tests and linters / Lint (pull_request) Successful in 6m35s
Tests and linters / Tests (1.20) (pull_request) Successful in 8m32s
Tests and linters / Tests with -race (pull_request) Successful in 8m47s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m54s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 17:13:58 +03:00
66848d3288 [#770] cli: Add methods to work with APE rules via control svc
* Add methods to frostfs-cli
* Implement rpc in control service

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
8e11ef46b8 [#770] object: Introduce ape chain checker for object svc
* Introduce Request type converted from RequestInfo type
  to implement policy-engine's Request interface
* Implement basic ape checker to check if a request is
  permitted to be performed
* Make put handlers use APE checker instead EACL

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
5ec73fe8a0 [#770] node: Introduce ape chain source
* Provide methods to access rule chains with access
  policy engine (APE) chain source
* Initialize apeChainSource within object service
  initialization
* Share apeChainSource with control service
* Implement dummy apeChainSource instance based on
  in-memory implementation

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
3a2c319b87 [#770] control: Generate gRPC methods to manipulate APE chains
* Define new types and gRPC methods to manipulate APE chains
  in control service.
* Stub gRPC handlers for the generated methods.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
226e84d782 [#684] node: Add skipped objects count to evacuation result
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-07 12:17:11 +00:00
523fb3ca51 [#684] proto: Add skipped count to evacuation status response
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-07 12:17:11 +00:00
74c91eeef5 [#777] client: Refactor PrmContainerList, PrmObjectSearch usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-06 06:50:11 +00:00
20d6132f31 [#531] signSvc: Add SetMarshaledData method call
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 1m25s
DCO action / DCO (pull_request) Successful in 1m36s
Build / Build Components (1.21) (pull_request) Successful in 2m40s
Build / Build Components (1.20) (pull_request) Successful in 3m2s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m41s
Tests and linters / Staticcheck (pull_request) Successful in 5m32s
Tests and linters / Lint (pull_request) Successful in 5m59s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m3s
Tests and linters / Tests with -race (pull_request) Successful in 6m6s
To reduce memory allocations add `SetMarshaledData` method call
to return already marshalled data in next `StableMarshal` calls.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-02 17:34:33 +03:00
05b508f79a [#772] proto: Fix file ending
All checks were successful
DCO action / DCO (pull_request) Successful in 3m47s
Vulncheck / Vulncheck (pull_request) Successful in 4m47s
Build / Build Components (1.21) (pull_request) Successful in 6m5s
Build / Build Components (1.20) (pull_request) Successful in 6m13s
Tests and linters / Tests (1.20) (pull_request) Successful in 13m9s
Tests and linters / Staticcheck (pull_request) Successful in 13m17s
Tests and linters / Lint (pull_request) Successful in 13m49s
Tests and linters / Tests (1.21) (pull_request) Successful in 14m12s
Tests and linters / Tests with -race (pull_request) Successful in 4m30s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-31 17:03:04 +03:00
79088baa06 [#772] node: Apply gofumpt
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-31 17:03:03 +03:00
b0cf100427 [#49] node: React on SIGHUP only when node in READY state
Add more info in logs when node is going to shut down,
but initialization process still in progress.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-24 15:55:29 +00:00
58b6224dd8 [#747] client: Refactor PrmObjectPutInit usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-20 11:55:40 +00:00
12b7cf2533 [#747] client: Refactor PrmObjectPutSingle usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-20 11:55:40 +00:00
189dbb01be [#733] frostfs-cli: Add control ir remove-container
All checks were successful
DCO action / DCO (pull_request) Successful in 3m51s
Vulncheck / Vulncheck (pull_request) Successful in 3m49s
Build / Build Components (1.21) (pull_request) Successful in 4m34s
Build / Build Components (1.20) (pull_request) Successful in 6m21s
Tests and linters / Tests (1.21) (pull_request) Successful in 9m1s
Tests and linters / Tests (1.20) (pull_request) Successful in 9m12s
Tests and linters / Staticcheck (pull_request) Successful in 6m40s
Tests and linters / Tests with -race (pull_request) Successful in 9m21s
Tests and linters / Lint (pull_request) Successful in 9m39s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-19 16:22:18 +03:00
aca11d7474 [#735] policer: Allow to provide metrics from the outside
Some checks failed
DCO action / DCO (pull_request) Successful in 3m9s
Vulncheck / Vulncheck (pull_request) Successful in 3m28s
Build / Build Components (1.21) (pull_request) Successful in 4m17s
Build / Build Components (1.20) (pull_request) Successful in 4m35s
Tests and linters / Lint (pull_request) Successful in 7m19s
Tests and linters / Tests with -race (pull_request) Failing after 9m47s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m0s
Tests and linters / Tests (1.20) (pull_request) Successful in 11m19s
Tests and linters / Staticcheck (pull_request) Successful in 11m4s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-11 15:14:13 +03:00
5e229dc248 [#701] metrics: add metric to evaluate policer performance
Add processed objects counter in policerMetrics,
add policer field to NodeMetrics

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-10-09 19:02:08 +00:00
4caa934eea [#729] containersvc: Remove load announcement
IR code was removed in 8879c6ea.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-09 19:01:13 +00:00
c1e4130020 [#146] node: Add trace_id to logs
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 3m7s
DCO action / DCO (pull_request) Successful in 3m36s
Build / Build Components (1.21) (pull_request) Successful in 3m29s
Build / Build Components (1.20) (pull_request) Successful in 3m37s
Tests and linters / Staticcheck (pull_request) Successful in 4m39s
Tests and linters / Lint (pull_request) Successful in 5m2s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m27s
Tests and linters / Tests with -race (pull_request) Successful in 6m29s
Tests and linters / Tests (1.20) (pull_request) Successful in 9m19s
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-09-27 11:05:27 +03:00
aeeb8193d2 [#676] node: Fix header source creation when checking eacl
Some checks failed
DCO action / DCO (pull_request) Successful in 2m55s
Build / Build Components (1.20) (pull_request) Successful in 4m53s
Vulncheck / Vulncheck (pull_request) Successful in 4m36s
Tests and linters / Staticcheck (pull_request) Successful in 6m35s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m7s
Tests and linters / Tests (1.20) (pull_request) Successful in 7m47s
Tests and linters / Tests with -race (pull_request) Failing after 10m7s
Build / Build Components (1.21) (pull_request) Successful in 11m3s
Tests and linters / Lint (pull_request) Successful in 17m34s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-09-06 17:06:54 +03:00
88d50e4c77 [#656] policer: Add "bad" testcase
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-09-06 08:04:59 +00:00
806cc13d9f [#658] client: Refactor PrmObjectGet/Head/Range usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-30 17:13:23 +00:00
fe5aa06a75 [#665] node: Bind length of copies number to number of replicas
Allow to use one digit in copies number array for backward compatibility.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-30 17:11:55 +00:00
55b82e744b [#529] objectcore: Use common sender classifier
All checks were successful
DCO action / DCO (pull_request) Successful in 2m19s
Vulncheck / Vulncheck (pull_request) Successful in 3m5s
Build / Build Components (1.21) (pull_request) Successful in 4m8s
Build / Build Components (1.20) (pull_request) Successful in 4m24s
Tests and linters / Tests (1.20) (pull_request) Successful in 4m57s
Tests and linters / Staticcheck (pull_request) Successful in 4m43s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m2s
Tests and linters / Lint (pull_request) Successful in 5m21s
Tests and linters / Tests with -race (pull_request) Successful in 6m17s
Use common sender classifier for ACL service and format validator.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-29 10:33:06 +03:00
ae81d6660a [#529] objectcore: Fix object content validation
There are old objects where the owner of the object
may not match the one who issued the token.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-29 10:33:06 +03:00
4ea0df77d0 [#574] policer: Check if the container was really removed
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-28 14:21:38 +00:00
554ff2c06b [#574] core: Extend Source interface with DeletionInfo method
* Introduce common method EverExisted
* Define DeletionInfo for struct that must implement Source
* Refactor tree srv

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-28 14:21:38 +00:00
f8ba60aa0c [#648] objsvc/delete: Handle errors in Go style
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 09:45:35 +00:00
d2084ece41 [#648] objsvc/delete: Remove redundant logs
We never propagate delete requests to the container node, because
tombstone broadcast is done via PUT. No need to pollute logs.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 09:45:35 +00:00
40b556fc19 [#647] objsvc/search: Improve testing coverage
All checks were successful
DCO action / DCO (pull_request) Successful in 1m32s
Build / Build Components (1.20) (pull_request) Successful in 3m54s
Tests and linters / Staticcheck (pull_request) Successful in 3m44s
Tests and linters / Tests (1.21) (pull_request) Successful in 4m18s
Tests and linters / Tests (1.20) (pull_request) Successful in 4m35s
Tests and linters / Lint (pull_request) Successful in 4m42s
Vulncheck / Vulncheck (pull_request) Successful in 5m11s
Tests and linters / Tests with -race (pull_request) Successful in 6m23s
Build / Build Components (1.21) (pull_request) Successful in 8m38s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
4db2cbc927 [#647] objsvc/search: Wrap in uniqueIDWriter during parameter setting
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
966ad22abf [#647] objsvc/search: Simplify error handling
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
56f841b022 [#647] objsvc/search: Remove TraverserGenerator wrapper
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
ba58144de1 [#647] objsvc/search: Remove netmap.Source wrapper
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
c9e3c9956e [#643] objsvc/put: Unify extraBroadcastEnabled usage
All checks were successful
DCO action / DCO (pull_request) Successful in 1m26s
Vulncheck / Vulncheck (pull_request) Successful in 2m44s
Build / Build Components (1.20) (pull_request) Successful in 3m26s
Tests and linters / Staticcheck (pull_request) Successful in 3m51s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m4s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m46s
Tests and linters / Tests with -race (pull_request) Successful in 6m23s
Build / Build Components (1.21) (pull_request) Successful in 13m1s
Tests and linters / Lint (pull_request) Successful in 19m46s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-24 11:03:17 +03:00
facd3b2c4b [#643] objsvc/put: Unify placement iterators
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-24 11:03:17 +03:00