Commit Graph

1080 Commits (v0.38.3)

Author SHA1 Message Date
Evgenii Stratonikov c1d90f018b [#1074] pilorama: Allow empty filenames in SortedByFilename()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-04 10:27:45 +00:00
Anton Nikiforov 748da78dc7 [#1072] Fix gofumpt issues
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-04 11:24:27 +03:00
Evgenii Stratonikov ff4c23f59a [#1070] services/tree: Fix fast listing depth processing
For unsorted `GetSubTree()` we return a single node for depth=1.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-02 14:41:31 +00:00
Evgenii Stratonikov e12fcc041d [#1059] services/tree: Fast sorted listing
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-01 12:37:34 +00:00
Airat Arifullin 6959e617c4 [#1047] object: Set container owner ID property to ape request
* Introduce ContainerOwner field in RequestContext.
* Set ContainerOwner in aclv2 middleware.
* Set PropertyKeyContainerOwnerID for object ape request.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-18 15:39:50 +00:00
Airat Arifullin d7be70e93f [#1040] object: Wrap CheckAPE errors to status errors
* All methods should wrap CheckAPE error, if it occurs, to
  status error.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-14 07:34:03 +00:00
Anton Nikiforov 5ee5f1df42 [#976] control: Introduce new method `RemoveChainLocalOverridesByTarget`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-03-13 15:33:19 +03:00
Dmitrii Stepanov 17f5463389 [#1043] cli: Add reset evacuation status command
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-13 10:29:45 +00:00
Dmitrii Stepanov 31e2396a5f [#1043] control: Add ResetEvacuationStatus implementation
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-13 10:29:45 +00:00
Dmitrii Stepanov 926cdeb072 [#1043] services: Regenerate proto
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-13 10:29:45 +00:00
Airat Arifullin 5c252c9193 [#1039] object: Skip APE check for certain request roles
DCO action / DCO (pull_request) Successful in 1m31s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m52s Details
Build / Build Components (1.21) (pull_request) Successful in 3m52s Details
Build / Build Components (1.20) (pull_request) Successful in 4m16s Details
Tests and linters / gopls check (pull_request) Successful in 11m54s Details
Tests and linters / Staticcheck (pull_request) Successful in 12m31s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 12m49s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 13m8s Details
Tests and linters / Tests with -race (pull_request) Successful in 13m14s Details
Tests and linters / Lint (pull_request) Successful in 13m31s Details
* Skip APE check if a role is Container.
* Skip APE check if a role is IR and methods are get-like.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-12 16:15:20 +03:00
Dmitrii Stepanov d433b49265 [#973] node: Resolve perfsprint linter
DCO action / DCO (pull_request) Successful in 2m40s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m41s Details
Build / Build Components (1.20) (pull_request) Successful in 4m27s Details
Build / Build Components (1.21) (pull_request) Successful in 5m6s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m16s Details
Tests and linters / gopls check (pull_request) Successful in 6m23s Details
Tests and linters / Lint (pull_request) Successful in 6m48s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m4s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m9s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 9m23s Details
`fmt.Errorf can be replaced with errors.New` and `fmt.Sprintf can be replaced with string addition`

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-11 17:55:50 +03:00
Dmitrii Stepanov 66a26b7775 [#973] node: Resolve revive: unused-parameter linter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-11 17:11:49 +03:00
Dmitrii Stepanov d6534fd755 [#1016] frostfs-node: Fix gopls issues
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-01 12:13:43 +03:00
Airat Arifullin 93bf9acbc2 [#898] control: Remove removed flag from RemoveChainLocalOverrideResponse
* Remove removed flag in service.proto for RemoveChainLocalOverrideResponse.
* Regenerate control API.
* Return error only if RemoveOverride returns non-NotFound code.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:07:07 +00:00
Airat Arifullin 75a1a95c2c [#986] tree: Skip ACL checks if basicACL mask is unset
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:05:57 +00:00
Airat Arifullin b1d171c261 [#986] container: Interpret APE NoRuleFound as request deny
* If APE check returns NoRuleFound, then it is taken for request deny.
* Add more unit-test for ape container middleware.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:05:57 +00:00
Airat Arifullin 7cc368e188 [#986] object: Introduce soft ape checks
* Soft APE check means that APE should allow request even
  it gets status NoRuleFound for a request. Otherwise,
  it is interpreted as Deny.
* Soft APE check is performed if basic ACL mask is not set.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:05:57 +00:00
Evgenii Stratonikov dad56d2e98 Revert "[#972] Use min/max builtins"
This reverts commit 89784b2e0a.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov 3359349acb Revert "[#972] Use slices.Sort* when useful"
This reverts commit b871d7a5e8.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov b871d7a5e8 [#972] Use slices.Sort* when useful
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Evgenii Stratonikov 89784b2e0a [#972] Use min/max builtins
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Dmitrii Stepanov 2680192ba0 [#988] objectSvc: Fix `SetMarshalData` for PutSingle
DCO action / DCO (pull_request) Successful in 5m10s Details
Vulncheck / Vulncheck (pull_request) Successful in 5m24s Details
Build / Build Components (1.21) (pull_request) Successful in 7m40s Details
Build / Build Components (1.20) (pull_request) Successful in 7m50s Details
Tests and linters / Staticcheck (pull_request) Successful in 7m52s Details
Tests and linters / Lint (pull_request) Successful in 9m8s Details
Tests and linters / Tests with -race (pull_request) Successful in 10m54s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m12s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 11m7s Details
After api-go update it is required to pass marshal data
to `SetMarshalData`.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 17:21:08 +03:00
Dmitrii Stepanov db67c21d55 [#947] engine: Evacuate trees to remote nodes
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:33:15 +03:00
Dmitrii Stepanov 15d853ea22 [#947] controlSvc: Return tree evacuation stat
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:39 +03:00
Dmitrii Stepanov b3f3505ada [#947] cli: Allow to specify evacuation scope
It may be required to evacuate only objects or only tree or all, so
now it spossible to specify.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00
Dmitrii Stepanov a6eb66bf9c [#947] evacuate: Refactor evacuate parameters
Drop methods to make it easier to extend.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-09 11:20:38 +03:00
Evgenii Stratonikov edbe06e07e [#956] policer/test: Reuse testPool helper
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 15:49:43 +00:00
Evgenii Stratonikov cbfeb72466 [#956] policer: Remove WithMaxCapacity option
We already provide the pool and this argument is used only for
preallocation. No functional changes.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 15:49:43 +00:00
Evgenii Stratonikov c3fa902780 [#969] policer: Restrict the number of remembered errors
DCO action / DCO (pull_request) Successful in 4m23s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m57s Details
Build / Build Components (1.21) (pull_request) Successful in 6m3s Details
Build / Build Components (1.20) (pull_request) Successful in 9m2s Details
Tests and linters / Staticcheck (pull_request) Successful in 10m4s Details
Tests and linters / Lint (pull_request) Successful in 10m51s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 10m44s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 10m56s Details
Tests and linters / Tests with -race (pull_request) Successful in 10m57s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 10:10:41 +03:00
Evgenii Stratonikov 6010dfdf3d [#969] policer: Make error skip thread-safe
Introduces in afd2ba9a66.
Refs #914

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-08 10:10:41 +03:00
Evgenii Stratonikov a6c9a337cd [#965] morph: Get rid of container.List invocations
ContainersOf() is better in almost every aspect, besides creating a
session when the containers number is between 1024 and 2048 (prefetch
script does limited unwrapping). Making List() private helps to ensure
it is no longer used and can be safely removed in future.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-07 08:56:27 +00:00
Dmitrii Stepanov 9ba48c582d [#917] engine: Allow to detach shards
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-06 14:49:47 +03:00
Dmitrii Stepanov 4358d3c423 [#917] controlSvc: Add DetachShards handler
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-06 14:47:52 +03:00
Ekaterina Lebedeva afd2ba9a66 [#110] Add check for repeated error log in policer
DCO action / DCO (pull_request) Successful in 2m55s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m22s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m10s Details
Build / Build Components (1.20) (pull_request) Successful in 4m58s Details
Build / Build Components (1.21) (pull_request) Successful in 4m54s Details
Tests and linters / Lint (pull_request) Successful in 5m47s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m45s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m58s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m42s Details
processObject() returns 3 types of errors: container not found errors,
could not get container error and placement vector building error. Every
error will occur for all objects in container simultaneously, so we can
log each error once and safely ignore the rest.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-02-06 00:56:41 +03:00
Evgenii Stratonikov 602ee11123 [#934] containersvc: Marhal public key in short format for APE
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-02 17:51:38 +00:00
Anton Nikiforov 9916598dfb [#922] control: Extend api with `ListOverrideDefinedTargets`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
Anton Nikiforov 95e15f499f [#922] Update files generated by protoc
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
Anton Nikiforov 2cb04379a4 [#922] go.mod: Update APE
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
Airat Arifullin a5446bc17d [#952] object: Pass namespace within context in ACL service
DCO action / DCO (pull_request) Successful in 6m23s Details
Vulncheck / Vulncheck (pull_request) Successful in 7m3s Details
Build / Build Components (1.21) (pull_request) Successful in 8m21s Details
Build / Build Components (1.20) (pull_request) Successful in 8m31s Details
Tests and linters / Staticcheck (pull_request) Successful in 11m1s Details
Tests and linters / Lint (pull_request) Successful in 11m26s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m51s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m14s Details
Tests and linters / Tests with -race (pull_request) Successful in 13m31s Details
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-02 14:48:11 +03:00
Anton Nikiforov 6534252c22 [#799] policer: Refactor method `processNodes`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-01 17:49:22 +00:00
Airat Arifullin 5be2af881a [#934] container: Make container APE middleware read namespaces
* Those methods that can access already existing containers and thus
  can get container properties should read namespace from Zone
  property. If Zone is not set, take a namespace for root.
* Otherwise, define namespaces by owner ID via frostfs-id contract.
* Improve unit-tests, consider more cases.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-01 17:38:24 +00:00
Airat Arifullin 4352bd0e8e [#934] ape: Transform empty namespace within chainbase
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-01 17:38:24 +00:00
Anton Nikiforov 483a67b170 [#937] ape: Validate chain resource name
DCO action / DCO (pull_request) Successful in 2m13s Details
Build / Build Components (1.20) (pull_request) Successful in 3m38s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m13s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m15s Details
Build / Build Components (1.21) (pull_request) Successful in 7m11s Details
Tests and linters / Lint (pull_request) Successful in 9m35s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m56s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 14m14s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m2s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-31 11:34:35 +03:00
Anton Nikiforov c441296592 [#930] policer: Release task pool when context cancelled
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-31 08:30:34 +00:00
Evgenii Stratonikov 6e2cc32768 [#681] objsvc: Validate session token owner for local sessions
Previously, the check was in place only when session token was missing.
Format validator checks are applied only to fully-prepared object, so
this lead to the following situation:
1. Object is put locally with malformed token, because there are no
   checks.
2. Object cannot be replicated, because the token is malformed.

This is now fixed and token check is done before any payload receival.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-26 08:52:29 +00:00
Anton Nikiforov b6fc3321c5 [#876] Fix linters
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-25 20:26:13 +03:00
Airat Arifullin f2f3294fc3 [#919] ape: Improve error messages in ape service
* Wrap all APE middleware errors in apeErr that
  makes errors more explicit with status AccessDenied.
* Use denyingRuleErr for denying status from chain router.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-23 08:11:24 +00:00
Airat Arifullin be33070550 [#910] control: Take empty namespace for 'root'
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-19 06:22:10 +00:00
Airat Arifullin 96b020626f [#915] ape: Fix method name in getStreamBasicChecker
DCO action / DCO (pull_request) Successful in 2m2s Details
Build / Build Components (1.21) (pull_request) Successful in 2m17s Details
Build / Build Components (1.20) (pull_request) Successful in 3m2s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m39s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m54s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m49s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m11s Details
Tests and linters / Lint (pull_request) Successful in 6m44s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m32s Details
* Replace incorrect MethodGetContainer by MethodGetObject constant.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-16 23:52:37 +03:00