Commit Graph

1056 Commits (e4064c43941098ee639d8efc0b7f75480732830d)

Author SHA1 Message Date
Dmitrii Stepanov 15d853ea22 [#947] controlSvc: Return tree evacuation stat
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:39 +03:00
Dmitrii Stepanov b3f3505ada [#947] cli: Allow to specify evacuation scope
It may be required to evacuate only objects or only tree or all, so
now it spossible to specify.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00
Dmitrii Stepanov a6eb66bf9c [#947] evacuate: Refactor evacuate parameters
Drop methods to make it easier to extend.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00
Evgenii Stratonikov edbe06e07e [#956] policer/test: Reuse testPool helper
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 15:49:43 +00:00
Evgenii Stratonikov cbfeb72466 [#956] policer: Remove WithMaxCapacity option
We already provide the pool and this argument is used only for
preallocation. No functional changes.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 15:49:43 +00:00
Evgenii Stratonikov c3fa902780 [#969] policer: Restrict the number of remembered errors
DCO action / DCO (pull_request) Successful in 4m23s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m57s Details
Build / Build Components (1.21) (pull_request) Successful in 6m3s Details
Build / Build Components (1.20) (pull_request) Successful in 9m2s Details
Tests and linters / Staticcheck (pull_request) Successful in 10m4s Details
Tests and linters / Lint (pull_request) Successful in 10m51s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 10m44s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 10m56s Details
Tests and linters / Tests with -race (pull_request) Successful in 10m57s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 10:10:41 +03:00
Evgenii Stratonikov 6010dfdf3d [#969] policer: Make error skip thread-safe
Introduces in afd2ba9a66.
Refs #914

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 10:10:41 +03:00
Evgenii Stratonikov a6c9a337cd [#965] morph: Get rid of container.List invocations
ContainersOf() is better in almost every aspect, besides creating a
session when the containers number is between 1024 and 2048 (prefetch
script does limited unwrapping). Making List() private helps to ensure
it is no longer used and can be safely removed in future.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-07 08:56:27 +00:00
Dmitrii Stepanov 9ba48c582d [#917] engine: Allow to detach shards
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-06 14:49:47 +03:00
Dmitrii Stepanov 4358d3c423 [#917] controlSvc: Add DetachShards handler
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-06 14:47:52 +03:00
Ekaterina Lebedeva afd2ba9a66 [#110] Add check for repeated error log in policer
DCO action / DCO (pull_request) Successful in 2m55s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m22s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m10s Details
Build / Build Components (1.20) (pull_request) Successful in 4m58s Details
Build / Build Components (1.21) (pull_request) Successful in 4m54s Details
Tests and linters / Lint (pull_request) Successful in 5m47s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m45s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m58s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m42s Details
processObject() returns 3 types of errors: container not found errors,
could not get container error and placement vector building error. Every
error will occur for all objects in container simultaneously, so we can
log each error once and safely ignore the rest.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-02-06 00:56:41 +03:00
Evgenii Stratonikov 602ee11123 [#934] containersvc: Marhal public key in short format for APE
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-02 17:51:38 +00:00
Anton Nikiforov 9916598dfb [#922] control: Extend api with `ListOverrideDefinedTargets`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
Anton Nikiforov 95e15f499f [#922] Update files generated by protoc
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
Anton Nikiforov 2cb04379a4 [#922] go.mod: Update APE
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
Airat Arifullin a5446bc17d [#952] object: Pass namespace within context in ACL service
DCO action / DCO (pull_request) Successful in 6m23s Details
Vulncheck / Vulncheck (pull_request) Successful in 7m3s Details
Build / Build Components (1.21) (pull_request) Successful in 8m21s Details
Build / Build Components (1.20) (pull_request) Successful in 8m31s Details
Tests and linters / Staticcheck (pull_request) Successful in 11m1s Details
Tests and linters / Lint (pull_request) Successful in 11m26s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m51s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m14s Details
Tests and linters / Tests with -race (pull_request) Successful in 13m31s Details
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-02 14:48:11 +03:00
Anton Nikiforov 6534252c22 [#799] policer: Refactor method `processNodes`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-01 17:49:22 +00:00
Airat Arifullin 5be2af881a [#934] container: Make container APE middleware read namespaces
* Those methods that can access already existing containers and thus
  can get container properties should read namespace from Zone
  property. If Zone is not set, take a namespace for root.
* Otherwise, define namespaces by owner ID via frostfs-id contract.
* Improve unit-tests, consider more cases.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-01 17:38:24 +00:00
Airat Arifullin 4352bd0e8e [#934] ape: Transform empty namespace within chainbase
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-01 17:38:24 +00:00
Anton Nikiforov 483a67b170 [#937] ape: Validate chain resource name
DCO action / DCO (pull_request) Successful in 2m13s Details
Build / Build Components (1.20) (pull_request) Successful in 3m38s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m13s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m15s Details
Build / Build Components (1.21) (pull_request) Successful in 7m11s Details
Tests and linters / Lint (pull_request) Successful in 9m35s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m56s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 14m14s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m2s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-31 11:34:35 +03:00
Anton Nikiforov c441296592 [#930] policer: Release task pool when context cancelled
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-31 08:30:34 +00:00
Evgenii Stratonikov 6e2cc32768 [#681] objsvc: Validate session token owner for local sessions
Previously, the check was in place only when session token was missing.
Format validator checks are applied only to fully-prepared object, so
this lead to the following situation:
1. Object is put locally with malformed token, because there are no
   checks.
2. Object cannot be replicated, because the token is malformed.

This is now fixed and token check is done before any payload receival.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-26 08:52:29 +00:00
Anton Nikiforov b6fc3321c5 [#876] Fix linters
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-25 20:26:13 +03:00
Airat Arifullin f2f3294fc3 [#919] ape: Improve error messages in ape service
* Wrap all APE middleware errors in apeErr that
  makes errors more explicit with status AccessDenied.
* Use denyingRuleErr for denying status from chain router.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-23 08:11:24 +00:00
Airat Arifullin be33070550 [#910] control: Take empty namespace for 'root'
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-19 06:22:10 +00:00
Airat Arifullin 96b020626f [#915] ape: Fix method name in getStreamBasicChecker
DCO action / DCO (pull_request) Successful in 2m2s Details
Build / Build Components (1.21) (pull_request) Successful in 2m17s Details
Build / Build Components (1.20) (pull_request) Successful in 3m2s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m39s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m54s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m49s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m11s Details
Tests and linters / Lint (pull_request) Successful in 6m44s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m32s Details
* Replace incorrect MethodGetContainer by MethodGetObject constant.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-16 23:52:37 +03:00
Airat Arifullin c8baf76fae [#872] object: Introduce APE middlewar for object service
DCO action / DCO (pull_request) Successful in 2m4s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m12s Details
Build / Build Components (1.21) (pull_request) Successful in 4m1s Details
Build / Build Components (1.20) (pull_request) Successful in 4m13s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m3s Details
Tests and linters / Lint (pull_request) Successful in 8m7s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m14s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m18s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m24s Details
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-12 18:41:35 +03:00
Dmitrii Stepanov 52ffa9f164 [#891] getSvc: Refactor Get service V2 creation
DCO action / DCO (pull_request) Successful in 2m46s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m53s Details
Build / Build Components (1.20) (pull_request) Successful in 4m31s Details
Build / Build Components (1.21) (pull_request) Successful in 4m26s Details
Tests and linters / Lint (pull_request) Successful in 6m9s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m7s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m27s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 9m4s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m48s Details
Use arguments for mandatory fields.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-12 13:35:38 +03:00
Dmitrii Stepanov 394f086fe2 [#891] getSvc: Fix get range hash implementation
Get range can perform GET request, so this request must be done
from container node to not to get access denied error.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-12 13:35:38 +03:00
Dmitrii Stepanov f1b2b8bffa [#895] test: Fix NewLogger arguments list
`debug` is always true.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-11 12:32:09 +00:00
Airat Arifullin a8e52ef7aa [#898] control: Fix codes for returning APE errors
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-11 12:31:31 +00:00
Denis Kirillov c19396d203 [#885] control: Make chain id bytes in grpc
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-01-11 07:24:22 +00:00
Dmitrii Stepanov 5c0a736a25 [#899] containerSvc: Fix invalid session token type
DCO action / DCO (pull_request) Successful in 1m23s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m29s Details
Tests and linters / Tests (1.21) (pull_request) Failing after 3m58s Details
Build / Build Components (1.21) (pull_request) Successful in 3m46s Details
Build / Build Components (1.20) (pull_request) Successful in 3m52s Details
Tests and linters / Lint (pull_request) Successful in 4m48s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m5s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m4s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m36s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-10 18:37:54 +03:00
Dmitrii Stepanov 581887148a [#569] cli: Add `control shards writecache seal` command
DCO action / DCO (pull_request) Successful in 2m46s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m3s Details
Build / Build Components (1.21) (pull_request) Successful in 3m53s Details
Build / Build Components (1.20) (pull_request) Successful in 4m0s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m31s Details
Tests and linters / Lint (pull_request) Successful in 6m17s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m47s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m14s Details
Tests and linters / Tests with -race (pull_request) Successful in 13m12s Details
It does the same as `control shards flush-writecache --seal`, but
has better name.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 16:05:37 +03:00
Dmitrii Stepanov 0cb0fc1735 [#569] writecache: Allow to seal writecache after flush
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 16:05:37 +03:00
Dmitrii Stepanov b118734909 [#890] getsvc: Log node PK
DCO action / DCO (pull_request) Successful in 13m55s Details
Vulncheck / Vulncheck (pull_request) Successful in 16m51s Details
Build / Build Components (1.20) (pull_request) Successful in 17m52s Details
Build / Build Components (1.21) (pull_request) Successful in 18m9s Details
Tests and linters / Tests (1.20) (pull_request) Failing after 24m34s Details
Tests and linters / Tests (1.21) (pull_request) Failing after 25m56s Details
Tests and linters / Staticcheck (pull_request) Successful in 26m46s Details
Tests and linters / Lint (pull_request) Successful in 28m6s Details
Tests and linters / Tests with -race (pull_request) Failing after 33m35s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 14:24:13 +03:00
Dmitrii Stepanov 764f70634d [#881] containerSvc: Add APE validation
Vulncheck / Vulncheck (pull_request) Successful in 2m40s Details
DCO action / DCO (pull_request) Successful in 2m27s Details
Build / Build Components (1.21) (pull_request) Successful in 3m22s Details
Tests and linters / Lint (pull_request) Successful in 4m52s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m46s Details
Build / Build Components (1.20) (pull_request) Successful in 4m54s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m59s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 12m38s Details
Tests and linters / Tests with -race (pull_request) Successful in 13m10s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-27 11:05:34 +03:00
Anton Nikiforov eab981bf1a [#876] Fix linter error
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-12-21 18:21:37 +03:00
Airat Arifullin bdd43f6211 [#869] object: Pass just CID to chain router
* Do not convert CID from request to native-schema resource
  format - this step is unneccessary for APE.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-14 11:01:20 +00:00
Dmitrii Stepanov 11add38e87 [#857] golangci: Add protogetter linter
DCO action / DCO (pull_request) Successful in 1m38s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m2s Details
Build / Build Components (1.21) (pull_request) Successful in 3m51s Details
Build / Build Components (1.20) (pull_request) Successful in 4m3s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m17s Details
Tests and linters / Lint (pull_request) Successful in 6m11s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m7s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 11m32s Details
Tests and linters / Tests with -race (pull_request) Successful in 12m22s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-12 16:27:02 +03:00
Dmitrii Stepanov 681b2c5fd4 [#825] policer: Do not drop required linking objects
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-12 11:04:03 +00:00
Airat Arifullin 0f45e3d344 [#804] ape: Implement boltdb storage for local overrides
DCO action / DCO (pull_request) Successful in 2m10s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m26s Details
Build / Build Components (1.20) (pull_request) Successful in 5m41s Details
Build / Build Components (1.21) (pull_request) Successful in 5m44s Details
Tests and linters / Staticcheck (pull_request) Successful in 7m10s Details
Tests and linters / Lint (pull_request) Successful in 8m14s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 14m24s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 14m41s Details
Tests and linters / Tests with -race (pull_request) Successful in 14m38s Details
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 19:08:41 +03:00
Airat Arifullin e361e017f3 [#842] control: Pass target instead resource name
* Update policy-engine package version in go.mod, go.sum.
* Refactor CheckIfRequestPermitted: pass container target
  instead container ID.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 14:21:55 +00:00
Airat Arifullin 39060382a1 [#842] control: Recieve target in gRPC methods for APE managing
* Introduce Target type and pass it to all gRPC methods
  for APE chain managing instead CID.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 14:21:55 +00:00
Dmitrii Stepanov 26e4f7005c [#741] treesvc: Refactor tree sync
Fix linter issues.
Add error logging.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-30 12:45:02 +00:00
Dmitrii Stepanov b21be1abdd [#741] treesvc: Do not update sync height if some node is unavailable
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-30 12:45:02 +00:00
Dmitrii Stepanov b215817e14 [#741] treesvc: Remove unused height variables
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-30 12:45:02 +00:00
Dmitrii Stepanov c516c7c5f4 [#821] node: Pass user.ID by value
DCO action / DCO (pull_request) Successful in 3m45s Details
Build / Build Components (1.21) (pull_request) Successful in 5m18s Details
Build / Build Components (1.20) (pull_request) Successful in 5m28s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m30s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m42s Details
Tests and linters / Lint (pull_request) Successful in 8m25s Details
Vulncheck / Vulncheck (pull_request) Successful in 9m22s Details
Tests and linters / Staticcheck (pull_request) Successful in 10m57s Details
Tests and linters / Tests with -race (pull_request) Successful in 16m53s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-23 10:21:07 +03:00
Airat Arifullin 4d5be5ccb5 [#811] ape: Update policy-engine module version and rebase
DCO action / DCO (pull_request) Successful in 4m23s Details
Vulncheck / Vulncheck (pull_request) Successful in 5m31s Details
Build / Build Components (1.21) (pull_request) Successful in 7m33s Details
Build / Build Components (1.20) (pull_request) Successful in 7m40s Details
Tests and linters / Staticcheck (pull_request) Successful in 8m22s Details
Tests and linters / Lint (pull_request) Successful in 9m23s Details
Tests and linters / Tests with -race (pull_request) Successful in 11m20s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 11m32s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m41s Details
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-16 11:31:37 +03:00
Dmitrii Stepanov 8088063195 [#787] netmap: Refactor `NewEpoch` method
DCO action / DCO (pull_request) Successful in 3m58s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m51s Details
Build / Build Components (1.21) (pull_request) Successful in 5m47s Details
Build / Build Components (1.20) (pull_request) Successful in 5m57s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m25s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m31s Details
Tests and linters / Lint (pull_request) Successful in 7m15s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m16s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m30s Details
Split for user and control methods.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:22:31 +03:00