dstepanov-yadro
/
frostfs-node
forked from TrueCloudLab/frostfs-node
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,614 Commits
213 Branches
26 Tags
29 MiB
Commit Graph

1086 Commits (feat/resync_speedup)

Author SHA1 Message Date
Dmitrii Stepanov e74bdaa5d5 [#1080] ape: Use value for APE request 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-09 18:42:03 +03:00
Dmitrii Stepanov 338d8cbebd [#1080] ape: Do not read object headers before Head/Get 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-09 15:27:40 +03:00
Anton Nikiforov 2b88361849 [#1062] object: Fix buffer allocation for `PayloadRange` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-09 11:59:07 +03:00
Dmitrii Stepanov f5b67c6735 [#1064] policer: Disable EC processing 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-09 07:08:53 +00:00
Dmitrii Stepanov 1c5e0f90aa [#1064] putsvc: Add EC put 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-09 07:08:53 +00:00
Dmitrii Stepanov 39da643354 [#1064] putsvc: Refactor distributed target 
Extract object builder.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-09 07:08:53 +00:00
Anton Nikiforov d614f04a0a [#1072] Fix gofumpt issues 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-03 22:21:14 +03:00
Airat Arifullin 4738508ce2 [#1063] go.mod: Update SDK version 
* Update frostfs-sdk and frostfs-api-go versions.
* Refactor depreacted method ReplicaNumberByIndex.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-04-03 10:33:51 +00:00
Evgenii Stratonikov ff4c23f59a [#1070] services/tree: Fix fast listing depth processing 
For unsorted `GetSubTree()` we return a single node for depth=1.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-02 14:41:31 +00:00
Evgenii Stratonikov e12fcc041d [#1059] services/tree: Fast sorted listing 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-01 12:37:34 +00:00
Airat Arifullin 6959e617c4 [#1047] object: Set container owner ID property to ape request 
* Introduce ContainerOwner field in RequestContext.
* Set ContainerOwner in aclv2 middleware.
* Set PropertyKeyContainerOwnerID for object ape request.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-03-18 15:39:50 +00:00
Airat Arifullin d7be70e93f [#1040] object: Wrap CheckAPE errors to status errors 
* All methods should wrap CheckAPE error, if it occurs, to
  status error.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-03-14 07:34:03 +00:00
Anton Nikiforov 5ee5f1df42 [#976] control: Introduce new method `RemoveChainLocalOverridesByTarget` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-03-13 15:33:19 +03:00
Dmitrii Stepanov 17f5463389 [#1043] cli: Add reset evacuation status command 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-13 10:29:45 +00:00
Dmitrii Stepanov 31e2396a5f [#1043] control: Add ResetEvacuationStatus implementation 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-13 10:29:45 +00:00
Dmitrii Stepanov 926cdeb072 [#1043] services: Regenerate proto 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-13 10:29:45 +00:00
Airat Arifullin 5c252c9193 [#1039] object: Skip APE check for certain request roles 
* Skip APE check if a role is Container.
* Skip APE check if a role is IR and methods are get-like.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-03-12 16:15:20 +03:00
Dmitrii Stepanov d433b49265 [#973] node: Resolve perfsprint linter 
`fmt.Errorf can be replaced with errors.New` and `fmt.Sprintf can be replaced with string addition`

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-11 17:55:50 +03:00
Dmitrii Stepanov 66a26b7775 [#973] node: Resolve revive: unused-parameter linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-11 17:11:49 +03:00
Dmitrii Stepanov d6534fd755 [#1016] frostfs-node: Fix gopls issues 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-01 12:13:43 +03:00
Airat Arifullin 93bf9acbc2 [#898] control: Remove removed flag from RemoveChainLocalOverrideResponse 
* Remove removed flag in service.proto for RemoveChainLocalOverrideResponse.
* Regenerate control API.
* Return error only if RemoveOverride returns non-NotFound code.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-02-28 19:07:07 +00:00
Airat Arifullin 75a1a95c2c [#986] tree: Skip ACL checks if basicACL mask is unset 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-02-28 19:05:57 +00:00
Airat Arifullin b1d171c261 [#986] container: Interpret APE NoRuleFound as request deny 
* If APE check returns NoRuleFound, then it is taken for request deny.
* Add more unit-test for ape container middleware.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-02-28 19:05:57 +00:00
Airat Arifullin 7cc368e188 [#986] object: Introduce soft ape checks 
* Soft APE check means that APE should allow request even
  it gets status NoRuleFound for a request. Otherwise,
  it is interpreted as Deny.
* Soft APE check is performed if basic ACL mask is not set.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-02-28 19:05:57 +00:00
Evgenii Stratonikov dad56d2e98 Revert "[#972] Use min/max builtins" 
This reverts commit 89784b2e0a.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-19 15:36:01 +00:00
Evgenii Stratonikov 3359349acb Revert "[#972] Use slices.Sort* when useful" 
This reverts commit b871d7a5e8.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-19 15:36:01 +00:00
Evgenii Stratonikov b871d7a5e8 [#972] Use slices.Sort* when useful 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-19 13:13:09 +00:00
Evgenii Stratonikov 89784b2e0a [#972] Use min/max builtins 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-19 13:13:09 +00:00
Dmitrii Stepanov 2680192ba0 [#988] objectSvc: Fix `SetMarshalData` for PutSingle 
After api-go update it is required to pass marshal data
to `SetMarshalData`.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-15 17:21:08 +03:00
Dmitrii Stepanov db67c21d55 [#947] engine: Evacuate trees to remote nodes 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-09 11:33:15 +03:00
Dmitrii Stepanov 15d853ea22 [#947] controlSvc: Return tree evacuation stat 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-09 11:20:39 +03:00
Dmitrii Stepanov b3f3505ada [#947] cli: Allow to specify evacuation scope 
It may be required to evacuate only objects or only tree or all, so
now it spossible to specify.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-09 11:20:38 +03:00
Dmitrii Stepanov a6eb66bf9c [#947] evacuate: Refactor evacuate parameters 
Drop methods to make it easier to extend.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-09 11:20:38 +03:00
Evgenii Stratonikov edbe06e07e [#956] policer/test: Reuse testPool helper 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-08 15:49:43 +00:00
Evgenii Stratonikov cbfeb72466 [#956] policer: Remove WithMaxCapacity option 
We already provide the pool and this argument is used only for
preallocation. No functional changes.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-08 15:49:43 +00:00
Evgenii Stratonikov c3fa902780 [#969] policer: Restrict the number of remembered errors 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-08 10:10:41 +03:00
Evgenii Stratonikov 6010dfdf3d [#969] policer: Make error skip thread-safe 
Introduces in afd2ba9a66.
Refs #914

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-08 10:10:41 +03:00
Evgenii Stratonikov a6c9a337cd [#965] morph: Get rid of container.List invocations 
ContainersOf() is better in almost every aspect, besides creating a
session when the containers number is between 1024 and 2048 (prefetch
script does limited unwrapping). Making List() private helps to ensure
it is no longer used and can be safely removed in future.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-07 08:56:27 +00:00
Dmitrii Stepanov 9ba48c582d [#917] engine: Allow to detach shards 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-06 14:49:47 +03:00
Dmitrii Stepanov 4358d3c423 [#917] controlSvc: Add DetachShards handler 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-06 14:47:52 +03:00
Ekaterina Lebedeva afd2ba9a66 [#110] Add check for repeated error log in policer 
processObject() returns 3 types of errors: container not found errors,
could not get container error and placement vector building error. Every
error will occur for all objects in container simultaneously, so we can
log each error once and safely ignore the rest.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-02-06 00:56:41 +03:00
Evgenii Stratonikov 602ee11123 [#934] containersvc: Marhal public key in short format for APE 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-02 17:51:38 +00:00
Anton Nikiforov 9916598dfb [#922] control: Extend api with `ListOverrideDefinedTargets` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-02 12:09:51 +00:00
Anton Nikiforov 95e15f499f [#922] Update files generated by protoc 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-02 12:09:51 +00:00
Anton Nikiforov 2cb04379a4 [#922] go.mod: Update APE 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-02 12:09:51 +00:00
Airat Arifullin a5446bc17d [#952] object: Pass namespace within context in ACL service 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-02-02 14:48:11 +03:00
Anton Nikiforov 6534252c22 [#799] policer: Refactor method `processNodes` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-01 17:49:22 +00:00
Airat Arifullin 5be2af881a [#934] container: Make container APE middleware read namespaces 
* Those methods that can access already existing containers and thus
  can get container properties should read namespace from Zone
  property. If Zone is not set, take a namespace for root.
* Otherwise, define namespaces by owner ID via frostfs-id contract.
* Improve unit-tests, consider more cases.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-02-01 17:38:24 +00:00
Airat Arifullin 4352bd0e8e [#934] ape: Transform empty namespace within chainbase 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-02-01 17:38:24 +00:00
Anton Nikiforov 483a67b170 [#937] ape: Validate chain resource name 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-01-31 11:34:35 +03:00