5be2af881a
[ #934 ] container: Make container APE middleware read namespaces
...
* Those methods that can access already existing containers and thus
can get container properties should read namespace from Zone
property. If Zone is not set, take a namespace for root.
* Otherwise, define namespaces by owner ID via frostfs-id contract.
* Improve unit-tests, consider more cases.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-01 17:38:24 +00:00
6e2cc32768
[ #681 ] objsvc: Validate session token owner for local sessions
...
Previously, the check was in place only when session token was missing.
Format validator checks are applied only to fully-prepared object, so
this lead to the following situation:
1. Object is put locally with malformed token, because there are no
checks.
2. Object cannot be replicated, because the token is malformed.
This is now fixed and token check is done before any payload receival.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-26 08:52:29 +00:00
b6fc3321c5
[ #876 ] Fix linters
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-25 20:26:13 +03:00
f2f3294fc3
[ #919 ] ape: Improve error messages in ape service
...
* Wrap all APE middleware errors in apeErr that
makes errors more explicit with status AccessDenied.
* Use denyingRuleErr for denying status from chain router.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-23 08:11:24 +00:00
96b020626f
[ #915 ] ape: Fix method name in getStreamBasicChecker
...
* Replace incorrect MethodGetContainer by MethodGetObject constant.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-16 23:52:37 +03:00
c8baf76fae
[ #872 ] object: Introduce APE middlewar for object service
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-12 18:41:35 +03:00
52ffa9f164
[ #891 ] getSvc: Refactor Get service V2 creation
...
Use arguments for mandatory fields.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-12 13:35:38 +03:00
394f086fe2
[ #891 ] getSvc: Fix get range hash implementation
...
Get range can perform GET request, so this request must be done
from container node to not to get access denied error.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-12 13:35:38 +03:00
f1b2b8bffa
[ #895 ] test: Fix NewLogger arguments list
...
`debug` is always true.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-11 12:32:09 +00:00
b118734909
[ #890 ] getsvc: Log node PK
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 14:24:13 +03:00
bdd43f6211
[ #869 ] object: Pass just CID to chain router
...
* Do not convert CID from request to native-schema resource
format - this step is unneccessary for APE.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-14 11:01:20 +00:00
0f45e3d344
[ #804 ] ape: Implement boltdb storage for local overrides
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 19:08:41 +03:00
e361e017f3
[ #842 ] control: Pass target instead resource name
...
* Update policy-engine package version in go.mod, go.sum.
* Refactor CheckIfRequestPermitted: pass container target
instead container ID.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 14:21:55 +00:00
c516c7c5f4
[ #821 ] node: Pass user.ID by value
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-23 10:21:07 +03:00
4d5be5ccb5
[ #811 ] ape: Update policy-engine module version and rebase
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-16 11:31:37 +03:00
9133b4389e
[ #788 ] objectsvc: Fix formatting (gofumpt)
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 10:27:32 +03:00
3534d6d05b
[ #794 ] objectsvc: Return accidentally removed acl checks for Head
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 17:13:58 +03:00
66848d3288
[ #770 ] cli: Add methods to work with APE rules via control svc
...
* Add methods to frostfs-cli
* Implement rpc in control service
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
8e11ef46b8
[ #770 ] object: Introduce ape chain checker for object svc
...
* Introduce Request type converted from RequestInfo type
to implement policy-engine's Request interface
* Implement basic ape checker to check if a request is
permitted to be performed
* Make put handlers use APE checker instead EACL
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
74c91eeef5
[ #777 ] client: Refactor PrmContainerList, PrmObjectSearch usage
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-06 06:50:11 +00:00
20d6132f31
[ #531 ] signSvc: Add SetMarshaledData method call
...
To reduce memory allocations add `SetMarshaledData` method call
to return already marshalled data in next `StableMarshal` calls.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-02 17:34:33 +03:00
79088baa06
[ #772 ] node: Apply gofumpt
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-31 17:03:03 +03:00
58b6224dd8
[ #747 ] client: Refactor PrmObjectPutInit usage
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-20 11:55:40 +00:00
12b7cf2533
[ #747 ] client: Refactor PrmObjectPutSingle usage
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-20 11:55:40 +00:00
c1e4130020
[ #146 ] node: Add trace_id to logs
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-09-27 11:05:27 +03:00
aeeb8193d2
[ #676 ] node: Fix header source creation when checking eacl
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-09-06 17:06:54 +03:00
806cc13d9f
[ #658 ] client: Refactor PrmObjectGet/Head/Range usage
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-30 17:13:23 +00:00
55b82e744b
[ #529 ] objectcore: Use common sender classifier
...
Use common sender classifier for ACL service and format validator.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-29 10:33:06 +03:00
ae81d6660a
[ #529 ] objectcore: Fix object content validation
...
There are old objects where the owner of the object
may not match the one who issued the token.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-29 10:33:06 +03:00
f8ba60aa0c
[ #648 ] objsvc/delete: Handle errors in Go style
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 09:45:35 +00:00
d2084ece41
[ #648 ] objsvc/delete: Remove redundant logs
...
We never propagate delete requests to the container node, because
tombstone broadcast is done via PUT. No need to pollute logs.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 09:45:35 +00:00
40b556fc19
[ #647 ] objsvc/search: Improve testing coverage
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
4db2cbc927
[ #647 ] objsvc/search: Wrap in uniqueIDWriter during parameter setting
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
966ad22abf
[ #647 ] objsvc/search: Simplify error handling
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
56f841b022
[ #647 ] objsvc/search: Remove TraverserGenerator wrapper
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
ba58144de1
[ #647 ] objsvc/search: Remove netmap.Source wrapper
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-25 10:40:01 +03:00
c9e3c9956e
[ #643 ] objsvc/put: Unify extraBroadcastEnabled usage
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-24 11:03:17 +03:00
facd3b2c4b
[ #643 ] objsvc/put: Unify placement iterators
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-24 11:03:17 +03:00
3fcf56f2fb
[ #643 ] objsvc/put: Copy config to distributedTarget
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-24 11:03:17 +03:00
96e690883f
[ #638 ] Unify test loggers
...
In some places we have debug=false, in others debug=true.
Let's be consistent.
Semantic patch:
```
@@
@@
-test.NewLogger(..., false)
+test.NewLogger(..., true)
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-23 11:21:05 +00:00
5a51b78946
[ #620 ] object: Send status response for server-side streams
...
Previously status responses were wrapped in the gRPC error and thus
couldn't be correctly handled on client.
Introduced in c2617baf63@ale64bit for having found.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-17 12:52:38 +00:00
21800e9fcc
[ #162 ] core: Move literals to constants
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-08-11 15:48:42 +03:00
5b7e4a51b7
[ #481 ] Update frostfs-sdk-go and error pointer receivers
...
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-09 10:26:53 +00:00
8d589314b5
[ #560 ] node: Fix Put in multi REP with intersecting sets of nodes
...
Once the node was processed it skipped, at the step of forming
result in case when all nodes skipped, because processed for
previous REP, service mark the whole request as incomplete.
Example of policies which are unblocked:
- REP 1 REP 1 CBF 1
- REP 4 IN X REP 4 IN Y
CBF 4
SELECT 2 FROM FX AS X SELECT 2 FROM FY AS Y
FILTER Country EQ Russia OR Country EQ Sweden OR Country EQ Finland AS FY
FILTER Price GE 0 AS FX
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-08 10:22:53 +00:00
b3695411d9
[ #553 ] eacl: Fix bug with casting to ObjectAccessDenied error
...
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-08-02 07:22:48 +00:00
05ac9e3637
[ #547 ] objectsvc: Work with traversal struct from a single thread
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-31 15:35:15 +00:00
7b0fdf0202
[ #533 ] services: Assume API supports status codes
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-31 15:35:15 +00:00
ea32913430
[ #543 ] putsvc: Fix PutSingle implementation
...
Add Lock and Delete handlers to local PutSingle.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-28 12:09:41 +00:00
99bb488ebd
[ #539 ] getsvc: Write payload direct to out stream
...
To reduce memory allocations.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-27 17:02:08 +03:00
286242cad0
[ #539 ] getsvc: Use buffer to assemble object
...
To reduce memory consumption.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-27 17:02:08 +03:00
5ff82ff04f
[ #6 ] services/object: Simplify local/remote targets
...
We do not use the return result from Close() and we always execute both
methods in succession. It makes sense to unite them.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-21 18:39:12 +03:00
c2617baf63
[ #6 ] services/util: Remove remaining stream wrappers
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-21 18:39:09 +03:00
372160d048
[ #6 ] services/util: Remove SignService.HandleUnaryRequest
...
There is no need in a wrapper with many from-`interface{}` conversions.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-21 18:36:08 +03:00
8966dd8e35
[ #463 ] putsvc: Use PutSingle RPC for remote target
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-18 10:52:12 +00:00
24eb988897
[ #294 ] deletesvc: Drop cast
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
c83e7c875f
[ #294 ] searchsvcv2: Refactor service constructor
...
Pass required deps as args.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
e8091101c7
[ #294 ] searchsvc: Refactor service constructor
...
Pass required deps as args.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
ec9b738465
[ #294 ] putsvcv2: Refactor service constructor
...
Pass required deps as args.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
800a685e84
[ #294 ] putsvc: Refactor service constructor
...
Pass required deps as args.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
1420b8b9ea
[ #294 ] deletesvc: Refactor service constructor
...
Pass required deps as args.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
a476d8285a
[ #294 ] deletesvcv2: Refactor service constructor
...
Pass required deps as args.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
70a1081988
[ #294 ] aclsvcv2: Refactor service constructor
...
Pass required deps as args.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
18d8898b00
[ #294 ] aclsvc: Refactor service constructor
...
Pass required deps as args.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
61541eaec2
[ #294 ] aclsvc: Refactor checker constructor
...
Pass required deps as args.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-12 07:42:10 +00:00
7da284f3e8
[ #509 ] go.mod: Update sdk-go
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-11 17:38:00 +03:00
80481c015c
[ #509 ] putsvc: Omit AccessIdentifiers from iteratePlacement()
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-11 17:38:00 +03:00
a65e26878b
[ #486 ] put service: Fix error typo
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-10 15:49:21 +03:00
fcbf90d31b
[ #486 ] node: Add PutSingle implemetation
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-10 15:49:21 +03:00
7b76527759
[ #486 ] node: Add PutSingle wrappers
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-10 15:49:21 +03:00
033eaf77e1
[ #496 ] node: Fix linter importas
...
Standardize the alias of the
import frostfs-sdk-go/object as objectSDK.
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-07-06 15:36:41 +03:00
90e9a85acc
[ #483 ] Update dependencies
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-07-03 15:44:59 +03:00
f437ab8f15
[ #197 ] object: Make Delete method return correct status
...
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
2023-06-30 12:58:56 +00:00
8a4e250dae
[ #468 ] *: replace outdated TODO crypto-related links
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
f4c71cea65
[ #1 ] *: Replace outdated FIXME/TODO links
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-28 12:13:20 +00:00
40eae22109
[ #460 ] services/util: Remove CreateRequestStreamer
...
There is no need in a wrapper with many from-`interface{}` conversions.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-21 17:08:04 +03:00
a64dc9ad70
[ #460 ] services/util: Remove HandleServerStreamRequest
...
There is no need in a wrapper with many from-`interface{}` conversions.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-21 17:08:01 +03:00
167a67f0b8
[ #460 ] services/util: Remove HandleUnaryRequest
...
There is no need in a wrapper with many from-`interface{}` conversions.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-21 17:07:56 +03:00
785d81a68a
[ #460 ] services/object: Reduce distibutedTarget memory footprint
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-21 17:07:50 +03:00
c348ae35b0
[ #424 ] metrics: Drop embedded metrics
...
It was not obvious where metrics are used.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 14:53:32 +03:00
0400153b7d
[ #424 ] object: Do not store large slices in pool
...
Dynamically growing an unbounded buffers can cause a large
amount of memory to be pinned and never be freed.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-13 07:19:55 +00:00
74578052f9
[ #412 ] node: Replace tracing package
...
Use observability module.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-01 13:23:11 +00:00
faca861451
[ #411 ] Remove unnecessary pointers for sync objects
...
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-05-31 10:19:14 +00:00
b2ffd7df53
[ #291 ] object: Use PayloadSizeLimiter from SDK
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-05 19:07:06 +03:00
35ea207df6
[ #291 ] object: Split validating target in two
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-05 19:07:06 +03:00
Pavel Karpy
ee58b390bb
[ #221 ] node: Allow using vector copies_number
...
Also, take into account that value in general (it was not used before at
all).
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-05-05 16:07:13 +00:00
22d47376a6
[ #277 ] getsvc: Refactor errors
...
Move errors to separate files.
Use zap.Error for error logging.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-28 14:03:12 +00:00
45438e7b06
[ #277 ] getsvc: Rename and reorder code
...
Rename execCtx to request.
Move code to appropriate files.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-28 14:03:12 +00:00
1440450606
[ #277 ] getsvc: Drop cyclic struct dependency
...
Drop cyclic dependency between execCtx and Service.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-28 14:03:12 +00:00
591c4e7d50
[ #277 ] getsvc: Move headOnly to request params
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-28 14:03:12 +00:00
265d2326a0
[ #277 ] getsvc: Extract remote storage
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-28 14:03:12 +00:00
30e1b62b67
[ #277 ] getsvc: Fix service deps
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-28 14:03:12 +00:00
8fc082b688
[ #277 ] getsvc: Do not return status error
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-28 14:03:12 +00:00
dfe4ada838
[ #285 ] lint: Resolve revive/if-return
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-26 17:35:08 +03:00
c04f6c5e59
[ #229 ] acl: Allow Impersonate
...
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-04-26 10:23:33 +03:00
3d43b0f7f9
[ #265 ] node: Fix after SDK & API-Go version up
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-18 12:09:19 +03:00
Pavel Karpy
560f73ab7e
[ #247 ] node, ir: Drop reputation related code
...
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-14 14:44:14 +03:00
d62c6e4ce6
[ #242 ] node: Add tracing spans
...
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-14 10:25:53 +00:00
200fc8b882
[ #242 ] put: Pass context to relay function
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-14 10:25:53 +00:00
cffcc7745e
[ #240 ] logs: Factor out common service log messages
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 05:06:09 +00:00
0e31c12e63
[ #240 ] logs: Move log messages to constants
...
Drop duplicate entities.
Format entities.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 05:06:09 +00:00
