dkirillov/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Issues Pull requests Projects Releases Packages Wiki Activity
4815 commits 15 branches 26 tags 16 MiB
Commit graph

1159 commits

Author SHA1 Message Date
Evgenii Stratonikov
89784b2e0a [#972] Use min/max builtins 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-19 13:13:09 +00:00
Dmitrii Stepanov
2680192ba0 [#988] objectSvc: Fix SetMarshalData for PutSingle 
After api-go update it is required to pass marshal data
to `SetMarshalData`.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-15 17:21:08 +03:00
Dmitrii Stepanov
db67c21d55 [#947] engine: Evacuate trees to remote nodes 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-09 11:33:15 +03:00
Dmitrii Stepanov
15d853ea22 [#947] controlSvc: Return tree evacuation stat 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-09 11:20:39 +03:00
Dmitrii Stepanov
b3f3505ada [#947] cli: Allow to specify evacuation scope 
It may be required to evacuate only objects or only tree or all, so
now it spossible to specify.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-09 11:20:38 +03:00
Dmitrii Stepanov
a6eb66bf9c [#947] evacuate: Refactor evacuate parameters 
Drop methods to make it easier to extend.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-09 11:20:38 +03:00
Evgenii Stratonikov
edbe06e07e [#956] policer/test: Reuse testPool helper 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-08 15:49:43 +00:00
Evgenii Stratonikov
cbfeb72466 [#956] policer: Remove WithMaxCapacity option 
We already provide the pool and this argument is used only for
preallocation. No functional changes.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-08 15:49:43 +00:00
Evgenii Stratonikov
c3fa902780 [#969] policer: Restrict the number of remembered errors 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-08 10:10:41 +03:00
Evgenii Stratonikov
6010dfdf3d [#969] policer: Make error skip thread-safe 
Introduces in afd2ba9a66.
Refs #914

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-08 10:10:41 +03:00
Evgenii Stratonikov
a6c9a337cd [#965] morph: Get rid of container.List invocations 
ContainersOf() is better in almost every aspect, besides creating a
session when the containers number is between 1024 and 2048 (prefetch
script does limited unwrapping). Making List() private helps to ensure
it is no longer used and can be safely removed in future.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-07 08:56:27 +00:00
Dmitrii Stepanov
9ba48c582d [#917] engine: Allow to detach shards 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-06 14:49:47 +03:00
Dmitrii Stepanov
4358d3c423 [#917] controlSvc: Add DetachShards handler 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-02-06 14:47:52 +03:00
Ekaterina Lebedeva
afd2ba9a66 [#110] Add check for repeated error log in policer 
processObject() returns 3 types of errors: container not found errors,
could not get container error and placement vector building error. Every
error will occur for all objects in container simultaneously, so we can
log each error once and safely ignore the rest.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-02-06 00:56:41 +03:00
Evgenii Stratonikov
602ee11123 [#934] containersvc: Marhal public key in short format for APE 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-02-02 17:51:38 +00:00
Anton Nikiforov
9916598dfb [#922] control: Extend api with ListOverrideDefinedTargets 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-02 12:09:51 +00:00
Anton Nikiforov
95e15f499f [#922] Update files generated by protoc 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-02 12:09:51 +00:00
Anton Nikiforov
2cb04379a4 [#922] go.mod: Update APE 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-02 12:09:51 +00:00
Airat Arifullin
a5446bc17d [#952] object: Pass namespace within context in ACL service 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-02-02 14:48:11 +03:00
Anton Nikiforov
6534252c22 [#799] policer: Refactor method processNodes 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-01 17:49:22 +00:00
Airat Arifullin
5be2af881a [#934] container: Make container APE middleware read namespaces 
* Those methods that can access already existing containers and thus
  can get container properties should read namespace from Zone
  property. If Zone is not set, take a namespace for root.
* Otherwise, define namespaces by owner ID via frostfs-id contract.
* Improve unit-tests, consider more cases.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-02-01 17:38:24 +00:00
Airat Arifullin
4352bd0e8e [#934] ape: Transform empty namespace within chainbase 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-02-01 17:38:24 +00:00
Anton Nikiforov
483a67b170 [#937] ape: Validate chain resource name 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-01-31 11:34:35 +03:00
Anton Nikiforov
c441296592 [#930] policer: Release task pool when context cancelled 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-01-31 08:30:34 +00:00
Evgenii Stratonikov
6e2cc32768 [#681] objsvc: Validate session token owner for local sessions 
Previously, the check was in place only when session token was missing.
Format validator checks are applied only to fully-prepared object, so
this lead to the following situation:
1. Object is put locally with malformed token, because there are no
   checks.
2. Object cannot be replicated, because the token is malformed.

This is now fixed and token check is done before any payload receival.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-01-26 08:52:29 +00:00
Anton Nikiforov
b6fc3321c5 [#876] Fix linters 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-01-25 20:26:13 +03:00
Airat Arifullin
f2f3294fc3 [#919] ape: Improve error messages in ape service 
* Wrap all APE middleware errors in apeErr that
  makes errors more explicit with status AccessDenied.
* Use denyingRuleErr for denying status from chain router.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-01-23 08:11:24 +00:00
Airat Arifullin
be33070550 [#910] control: Take empty namespace for 'root' 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-01-19 06:22:10 +00:00
Airat Arifullin
96b020626f [#915] ape: Fix method name in getStreamBasicChecker 
* Replace incorrect MethodGetContainer by MethodGetObject constant.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-01-16 23:52:37 +03:00
Airat Arifullin
c8baf76fae [#872] object: Introduce APE middlewar for object service 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-01-12 18:41:35 +03:00
Dmitrii Stepanov
52ffa9f164 [#891] getSvc: Refactor Get service V2 creation 
Use arguments for mandatory fields.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-12 13:35:38 +03:00
Dmitrii Stepanov
394f086fe2 [#891] getSvc: Fix get range hash implementation 
Get range can perform GET request, so this request must be done
from container node to not to get access denied error.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-12 13:35:38 +03:00
Dmitrii Stepanov
f1b2b8bffa [#895] test: Fix NewLogger arguments list 
`debug` is always true.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Airat Arifullin
a8e52ef7aa [#898] control: Fix codes for returning APE errors 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-01-11 12:31:31 +00:00
Denis Kirillov
c19396d203 [#885] control: Make chain id bytes in grpc 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
 2024-01-11 07:24:22 +00:00
Dmitrii Stepanov
5c0a736a25 [#899] containerSvc: Fix invalid session token type 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-10 18:37:54 +03:00
Dmitrii Stepanov
581887148a [#569] cli: Add control shards writecache seal command 
It does the same as `control shards flush-writecache --seal`, but
has better name.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 16:05:37 +03:00
Dmitrii Stepanov
0cb0fc1735 [#569] writecache: Allow to seal writecache after flush 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 16:05:37 +03:00
Dmitrii Stepanov
b118734909 [#890] getsvc: Log node PK 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-29 14:24:13 +03:00
Dmitrii Stepanov
764f70634d [#881] containerSvc: Add APE validation 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-27 11:05:34 +03:00
Anton Nikiforov
eab981bf1a [#876] Fix linter error 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-12-21 18:21:37 +03:00
Airat Arifullin
bdd43f6211 [#869] object: Pass just CID to chain router 
* Do not convert CID from request to native-schema resource
  format - this step is unneccessary for APE.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-12-14 11:01:20 +00:00
Dmitrii Stepanov
11add38e87 [#857] golangci: Add protogetter linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 16:27:02 +03:00
Dmitrii Stepanov
681b2c5fd4 [#825] policer: Do not drop required linking objects 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 11:04:03 +00:00
aarifullin
0f45e3d344 [#804] ape: Implement boltdb storage for local overrides 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-12-07 19:08:41 +03:00
aarifullin
e361e017f3 [#842] control: Pass target instead resource name 
* Update policy-engine package version in go.mod, go.sum.
* Refactor CheckIfRequestPermitted: pass container target
  instead container ID.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-12-07 14:21:55 +00:00
aarifullin
39060382a1 [#842] control: Recieve target in gRPC methods for APE managing 
* Introduce Target type and pass it to all gRPC methods
  for APE chain managing instead CID.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-12-07 14:21:55 +00:00
Dmitrii Stepanov
26e4f7005c [#741] treesvc: Refactor tree sync 
Fix linter issues.
Add error logging.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-30 12:45:02 +00:00
Dmitrii Stepanov
b21be1abdd [#741] treesvc: Do not update sync height if some node is unavailable 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-30 12:45:02 +00:00
Dmitrii Stepanov
b215817e14 [#741] treesvc: Remove unused height variables 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-30 12:45:02 +00:00
Dmitrii Stepanov
c516c7c5f4 [#821] node: Pass user.ID by value 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-23 10:21:07 +03:00
aarifullin
4d5be5ccb5 [#811] ape: Update policy-engine module version and rebase 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-11-16 11:31:37 +03:00
Dmitrii Stepanov
8088063195 [#787] netmap: Refactor NewEpoch method 
Split for user and control methods.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-13 17:22:31 +03:00
Dmitrii Stepanov
2393d13e4d [#787] morph: Return VUB for IR service calls 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-13 17:13:03 +03:00
Dmitrii Stepanov
bdfa523487 [#787] proto: Add VUB field for IR service 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-13 17:13:03 +03:00
Dmitrii Stepanov
0f75e48138 [#796] policer: Fix tombstone objects replication 
Tombstone objects must be replicated to all container nodes.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-09 13:39:33 +03:00
Dmitrii Stepanov
7cdae4f660 [#792] proto: Regenerate with fixed version 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-09 10:09:13 +00:00
Dmitrii Stepanov
9133b4389e [#788] objectsvc: Fix formatting (gofumpt) 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-09 10:27:32 +03:00
aarifullin
3534d6d05b [#794] objectsvc: Return accidentally removed acl checks for Head 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-11-08 17:13:58 +03:00
Airat Arifullin
66848d3288 [#770] cli: Add methods to work with APE rules via control svc 
* Add methods to frostfs-cli
* Implement rpc in control service

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-11-08 13:34:03 +00:00
Airat Arifullin
8e11ef46b8 [#770] object: Introduce ape chain checker for object svc 
* Introduce Request type converted from RequestInfo type
  to implement policy-engine's Request interface
* Implement basic ape checker to check if a request is
  permitted to be performed
* Make put handlers use APE checker instead EACL

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-11-08 13:34:03 +00:00
Airat Arifullin
5ec73fe8a0 [#770] node: Introduce ape chain source 
* Provide methods to access rule chains with access
  policy engine (APE) chain source
* Initialize apeChainSource within object service
  initialization
* Share apeChainSource with control service
* Implement dummy apeChainSource instance based on
  in-memory implementation

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-11-08 13:34:03 +00:00
Airat Arifullin
3a2c319b87 [#770] control: Generate gRPC methods to manipulate APE chains 
* Define new types and gRPC methods to manipulate APE chains
  in control service.
* Stub gRPC handlers for the generated methods.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-11-08 13:34:03 +00:00
Dmitrii Stepanov
226e84d782 [#684] node: Add skipped objects count to evacuation result 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-07 12:17:11 +00:00
Dmitrii Stepanov
523fb3ca51 [#684] proto: Add skipped count to evacuation status response 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-07 12:17:11 +00:00
aarifullin
74c91eeef5 [#777] client: Refactor PrmContainerList, PrmObjectSearch usage 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-11-06 06:50:11 +00:00
Dmitrii Stepanov
20d6132f31 [#531] signSvc: Add SetMarshaledData method call 
To reduce memory allocations add `SetMarshaledData` method call
to return already marshalled data in next `StableMarshal` calls.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-02 17:34:33 +03:00
Dmitrii Stepanov
05b508f79a [#772] proto: Fix file ending 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-31 17:03:04 +03:00
Dmitrii Stepanov
79088baa06 [#772] node: Apply gofumpt 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-31 17:03:03 +03:00
Anton Nikiforov
b0cf100427 [#49] node: React on SIGHUP only when node in READY state 
Add more info in logs when node is going to shut down,
but initialization process still in progress.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-10-24 15:55:29 +00:00
aarifullin
58b6224dd8 [#747] client: Refactor PrmObjectPutInit usage 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-10-20 11:55:40 +00:00
aarifullin
12b7cf2533 [#747] client: Refactor PrmObjectPutSingle usage 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-10-20 11:55:40 +00:00
Dmitrii Stepanov
189dbb01be [#733] frostfs-cli: Add control ir remove-container 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-19 16:22:18 +03:00
Evgenii Stratonikov
aca11d7474 [#735] policer: Allow to provide metrics from the outside 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-10-11 15:14:13 +03:00
Ekaterina Lebedeva
5e229dc248 [#701] metrics: add metric to evaluate policer performance 
Add processed objects counter in policerMetrics,
add policer field to NodeMetrics

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2023-10-09 19:02:08 +00:00
Evgenii Stratonikov
4caa934eea [#729] containersvc: Remove load announcement 
IR code was removed in 8879c6ea.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-10-09 19:01:13 +00:00
Alexander Chuprov
c1e4130020 [#146] node: Add trace_id to logs 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-09-27 11:05:27 +03:00
Anton Nikiforov
aeeb8193d2 [#676] node: Fix header source creation when checking eacl 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-09-06 17:06:54 +03:00
Anton Nikiforov
88d50e4c77 [#656] policer: Add "bad" testcase 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-09-06 08:04:59 +00:00
Airat Arifullin
806cc13d9f [#658] client: Refactor PrmObjectGet/Head/Range usage 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-08-30 17:13:23 +00:00
Anton Nikiforov
fe5aa06a75 [#665] node: Bind length of copies number to number of replicas 
Allow to use one digit in copies number array for backward compatibility.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-08-30 17:11:55 +00:00
Dmitrii Stepanov
55b82e744b [#529] objectcore: Use common sender classifier 
Use common sender classifier for ACL service and format validator.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-29 10:33:06 +03:00
Dmitrii Stepanov
ae81d6660a [#529] objectcore: Fix object content validation 
There are old objects where the owner of the object
may not match the one who issued the token.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-29 10:33:06 +03:00
Airat Arifullin
4ea0df77d0 [#574] policer: Check if the container was really removed 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-08-28 14:21:38 +00:00
Airat Arifullin
554ff2c06b [#574] core: Extend Source interface with DeletionInfo method 
* Introduce common method EverExisted
* Define DeletionInfo for struct that must implement Source
* Refactor tree srv

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-08-28 14:21:38 +00:00
Evgenii Stratonikov
f8ba60aa0c [#648] objsvc/delete: Handle errors in Go style 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-25 09:45:35 +00:00
Evgenii Stratonikov
d2084ece41 [#648] objsvc/delete: Remove redundant logs 
We never propagate delete requests to the container node, because
tombstone broadcast is done via PUT. No need to pollute logs.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-25 09:45:35 +00:00
Evgenii Stratonikov
40b556fc19 [#647] objsvc/search: Improve testing coverage 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-25 10:40:01 +03:00
Evgenii Stratonikov
4db2cbc927 [#647] objsvc/search: Wrap in uniqueIDWriter during parameter setting 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-25 10:40:01 +03:00
Evgenii Stratonikov
966ad22abf [#647] objsvc/search: Simplify error handling 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-25 10:40:01 +03:00
Evgenii Stratonikov
56f841b022 [#647] objsvc/search: Remove TraverserGenerator wrapper 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-25 10:40:01 +03:00
Evgenii Stratonikov
ba58144de1 [#647] objsvc/search: Remove netmap.Source wrapper 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-25 10:40:01 +03:00
Evgenii Stratonikov
c9e3c9956e [#643] objsvc/put: Unify extraBroadcastEnabled usage 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-24 11:03:17 +03:00
Evgenii Stratonikov
facd3b2c4b [#643] objsvc/put: Unify placement iterators 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-24 11:03:17 +03:00
Evgenii Stratonikov
3fcf56f2fb [#643] objsvc/put: Copy config to distributedTarget 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-24 11:03:17 +03:00
Evgenii Stratonikov
96e690883f [#638] Unify test loggers 
In some places we have debug=false, in others debug=true.
Let's be consistent.

Semantic patch:
```
@@
@@
-test.NewLogger(..., false)
+test.NewLogger(..., true)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-23 11:21:05 +00:00
Evgenii Stratonikov
322c1dc273 [#638] Use test.NewLogger() in tests 
Semantic patch (restricted to **/*_test.go):
```
@@
@@
+import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger/test"
-import "go.uber.org/zap"
-import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger"

-&logger.Logger{Logger: zap.L()}
+test.NewLogger(t, false)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-23 11:21:05 +00:00
Airat Arifullin
23be3eb627 [#574] tree: Check if container is really removed 
* Use DeletionInfo method from morph client to check if
  the container has been really removed from neo-go

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-08-21 12:50:20 +03:00
Evgenii Stratonikov
f7042c5a6f [#609] Replace zaptest.NewLogger() with zap.L() 
Semantic patch:
```
@@
@@
-import "go.uber.org/zap/zaptest"
+import "go.uber.org/zap"

-zaptest.NewLogger(t)
+zap.L()
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-17 16:51:10 +00:00
Evgenii Stratonikov
e604a3d749 [#607] *: Use zap.Stringer() where possible 
Semantic patch:
```
@@
var f expression
var t expression
var a expression
@@
 f(
    ...,
-    zap.String(t, a.String()),
+    zap.Stringer(t, a),
    ...,
)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-17 16:50:02 +00:00
Evgenii Stratonikov
a8de37c8a2 [#607] *: Remove redundant if on error returns 
Semantic patch:
```
@@
@@
-if err != nil { return err }
-return nil
+return err
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-17 16:50:02 +00:00
Evgenii Stratonikov
5a51b78946 [#620] object: Send status response for server-side streams 
Previously status responses were wrapped in the gRPC error and thus
couldn't be correctly handled on client.

Introduced in c2617baf63, thanks @ale64bit for having found.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-17 12:52:38 +00:00
Anton Nikiforov
6407bb5bd1 [#619] node: Fix object put when copies numbers contains only zeros 
In this case object should placement according to replicas.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-08-17 11:31:34 +03:00
Alexander Chuprov
21800e9fcc [#162] core: Move literals to constants 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-08-11 15:48:42 +03:00
Alejandro Lopez
5b7e4a51b7 [#481] Update frostfs-sdk-go and error pointer receivers 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-09 10:26:53 +00:00
Anton Nikiforov
8d589314b5 [#560] node: Fix Put in multi REP with intersecting sets of nodes 
Once the node was processed it skipped, at the step of forming
result in case when all nodes skipped, because processed for
previous REP, service mark the whole request as incomplete.

Example of policies which are unblocked:
- REP 1 REP 1 CBF 1
- REP 4 IN X REP 4 IN Y
  CBF 4
  SELECT 2 FROM FX AS X SELECT 2 FROM FY AS Y
  FILTER Country EQ Russia OR Country EQ Sweden OR Country EQ Finland AS FY
  FILTER Price GE 0 AS FX

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-08-08 10:22:53 +00:00
Evgenii Stratonikov
1e8b4b8a17 [#557] services: Regenerate stable marshalers 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-02 13:37:40 +00:00
Airat Arifullin
b3695411d9 [#553] eacl: Fix bug with casting to ObjectAccessDenied error 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-08-02 07:22:48 +00:00
Evgenii Stratonikov
05ac9e3637 [#547] objectsvc: Work with traversal struct from a single thread 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-31 15:35:15 +00:00
Evgenii Stratonikov
7b0fdf0202 [#533] services: Assume API supports status codes 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-31 15:35:15 +00:00
Dmitrii Stepanov
ea32913430 [#543] putsvc: Fix PutSingle implementation 
Add Lock and Delete handlers to local PutSingle.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-28 12:09:41 +00:00
Dmitrii Stepanov
99bb488ebd [#539] getsvc: Write payload direct to out stream 
To reduce memory allocations.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-27 17:02:08 +03:00
Dmitrii Stepanov
286242cad0 [#539] getsvc: Use buffer to assemble object 
To reduce memory consumption.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-27 17:02:08 +03:00
Evgenii Stratonikov
5ff82ff04f [#6] services/object: Simplify local/remote targets 
We do not use the return result from Close() and we always execute both
methods in succession. It makes sense to unite them.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-21 18:39:12 +03:00
Evgenii Stratonikov
448b48287c [#6] services/util: Do not panic in sign function 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-21 18:39:12 +03:00
Evgenii Stratonikov
c2617baf63 [#6] services/util: Remove remaining stream wrappers 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-21 18:39:09 +03:00
Evgenii Stratonikov
372160d048 [#6] services/util: Remove SignService.HandleUnaryRequest 
There is no need in a wrapper with many from-`interface{}` conversions.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-21 18:36:08 +03:00
Evgenii Stratonikov
fef172c5b0 [#6] services/util: Simplify response.Service 
It has only 1 parameter, which is obligatory.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-21 18:29:57 +03:00
Dmitrii Stepanov
eed594431f [#335] treesvc: Add GetSubTree ordering unit test 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-20 10:14:10 +03:00
Dmitrii Stepanov
b4e72a2dfd [#335] treesvc: Sort nodes by Filename in GetSubTree 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-20 10:14:10 +03:00
Dmitrii Stepanov
f0355a453e [#463] policer: Remove capacity rebalance logic 
Current implementation has some quirks. For example,
using only half of object.put.pool_size_remote threads
tells replicator that is node is 50% loaded,
but in reality we could be putting lot's of big objects.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-18 10:52:12 +00:00
Dmitrii Stepanov
8b78db74bc [#463] replicator: Add tracing span 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-18 10:52:12 +00:00
Dmitrii Stepanov
8966dd8e35 [#463] putsvc: Use PutSingle RPC for remote target 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-18 10:52:12 +00:00
Alejandro Lopez
3b66f98f27 [#519] Use Address.Equals in policer tests 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-07-17 10:45:07 +00:00
Evgenii Stratonikov
8a9fc2c372 [#510] treesvc: Rename tableFromBearer to useBearer 
With impersonation, the old name is no longer descriptive.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-12 10:23:21 +00:00
Evgenii Stratonikov
b8bcfac531 [#510] treesvc: Fix panic in bearer token processing 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-12 10:23:21 +00:00
Dmitrii Stepanov
24eb988897 [#294] deletesvc: Drop cast 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov
c83e7c875f [#294] searchsvcv2: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov
e8091101c7 [#294] searchsvc: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov
ec9b738465 [#294] putsvcv2: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov
800a685e84 [#294] putsvc: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov
1420b8b9ea [#294] deletesvc: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov
a476d8285a [#294] deletesvcv2: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov
70a1081988 [#294] aclsvcv2: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov
18d8898b00 [#294] aclsvc: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov
61541eaec2 [#294] aclsvc: Refactor checker constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Evgenii Stratonikov
7da284f3e8 [#509] go.mod: Update sdk-go 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-11 17:38:00 +03:00
Evgenii Stratonikov
80481c015c [#509] putsvc: Omit AccessIdentifiers from iteratePlacement() 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-11 17:38:00 +03:00
Dmitrii Stepanov
a65e26878b [#486] put service: Fix error typo 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-10 15:49:21 +03:00
Dmitrii Stepanov
fcbf90d31b [#486] node: Add PutSingle implemetation 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-10 15:49:21 +03:00
Dmitrii Stepanov
7b76527759 [#486] node: Add PutSingle wrappers 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-10 15:49:21 +03:00
Evgenii Stratonikov
140d970a95 [#498] policer: Fix objectSDK import 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-07 18:52:40 +03:00
Evgenii Stratonikov
2310a5c7ba [#498] policer: Allow to set sleep duration between iterations 
Speed up tests on CI.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-07 14:52:33 +00:00
Evgenii Stratonikov
e858479a74 [#498] policer: Explicitly Rewind() iterator after finish 
Previously, we can continue to return `EndOfListing` infinitely.
Reflect iterator reuse via Rewind() method.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-07 14:52:33 +00:00
Alexander Chuprov
033eaf77e1 [#496] node: Fix linter importas 
Standardize the alias of the
import frostfs-sdk-go/object as objectSDK.

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-07-06 15:36:41 +03:00
Evgenii Stratonikov
6eefe9747e treesvc: Do not provide credentials unless TLS is used 
4f413fe86e was perfect, except it did the opposite of what we needed.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-06 08:04:28 +00:00
Alex Vanin
f354b8a270 [#473] Add tests for rem values of traverser 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-07-05 20:13:23 +03:00
Alex Vanin
c6df6c84ae [#473] Do not modify traverse plan when [0] copies number vector is provided 
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-07-05 19:54:43 +03:00
Anton Nikiforov
90e9a85acc [#483] Update dependencies 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-07-03 15:44:59 +03:00
Alejandro Lopez
26acf5689e [#92] Ensure policer objects cannot be worked on concurrently 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-07-03 07:05:31 +00:00