Commit Graph

589 Commits (v0.41.0)

Author SHA1 Message Date
Airat Arifullin 04a3f891fd [#1157] object: Make APE checker use Bearer-token's APE overrides
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-06-07 12:11:11 +00:00
Anton Nikiforov c1af13b47e [#1147] node: Fix issue from `gopls`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
Anton Nikiforov 6130650bb6 [#1147] node: Implement `Lock\Delete` requests for EC object
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
Anton Nikiforov d355274cd0 [#1147] object: Use methods on pointer for `searchsvc.execCtx`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
Anton Nikiforov 3555c73225 [#1147] object: Use methods on pointer for `deletesvc.execCtx`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
Anton Nikiforov e43e7bec3a [#1147] log: Remove redundant `address` field from log
Filled when logger created for `request` object from package `getsvc`.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
Airat Arifullin 482c5129ac [#1142] object: Fill APE-request with source IP property
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-27 10:17:17 +00:00
Dmitrii Stepanov 436c9f5558 [#1129] policer: Restore EC object
Vulncheck / Vulncheck (pull_request) Successful in 1m43s Details
DCO action / DCO (pull_request) Successful in 2m53s Details
Build / Build Components (1.21) (pull_request) Successful in 4m6s Details
Build / Build Components (1.22) (pull_request) Successful in 4m31s Details
Tests and linters / gopls check (pull_request) Successful in 4m57s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m14s Details
Tests and linters / Lint (pull_request) Successful in 6m42s Details
Pre-commit hooks / Pre-commit (pull_request) Successful in 9m19s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 10m15s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 10m36s Details
Tests and linters / Tests with -race (pull_request) Successful in 10m36s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-17 14:36:18 +03:00
Dmitrii Stepanov 44f2e8f27f [#1129] putSvc: Allow to put single unprepared object to EC container
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-16 16:28:49 +03:00
Dmitrii Stepanov cbe9757490 [#1129] policer: Pull required EC chunks
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-16 16:28:49 +03:00
Evgenii Stratonikov b3eaa8a9bc [#1083] objsvc/v2: Check response status in RANGE_HASH forwarder
DCO action / DCO (pull_request) Successful in 1m45s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m11s Details
Pre-commit hooks / Pre-commit (pull_request) Successful in 5m44s Details
Build / Build Components (1.22) (pull_request) Successful in 5m25s Details
Build / Build Components (1.21) (pull_request) Successful in 5m30s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m50s Details
Tests and linters / gopls check (pull_request) Successful in 5m51s Details
Tests and linters / Lint (pull_request) Successful in 7m44s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 11m36s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 12m13s Details
Tests and linters / Tests with -race (pull_request) Successful in 12m25s Details
Fixes #1083

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-15 12:04:21 +03:00
Evgenii Stratonikov 0924b62a95 [#1083] objsvc/v2: Unify response verification after forwarding
1. Use the same routine for HEAD/GET_RANGE methods.
2. Make error message similar.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-15 12:04:06 +03:00
Evgenii Stratonikov 300654b045 [#1083] objsvc/v2: Properly check response status after forwarding
Previously we had cryptic error:
```
debug   get/remote.go:38        remote call failed      {"component": "Object.Get service", "request": "HEAD", "address": "9sTxoVrhJ7WBtXQfK2NJ7zDV5yCF7BPLKK1XTxYPdGsP/BbHV4KZZ8y2BPqAT5kyjdHRLkfbtY2xf5uYoMVqxACn1", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "unexpected header type <nil>"}
```
Now we have and expected error:
```
debug   get/remote.go:38        remote call failed      {"component": "Object.Get service", "request": "HEAD", "address": "D2rqaMG4D2VHdv3HKky8UYSYmwQFH2v9oXXqtyRZPTMy/BbHV4KZZ8y2BPqAT5kyjdHRLkfbtY2xf5uYoMVqxACn1", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"}
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-15 12:04:06 +03:00
Airat Arifullin 952d13cd2b [#1124] cli: Improve APE rule parsing
Vulncheck / Vulncheck (pull_request) Successful in 1m25s Details
DCO action / DCO (pull_request) Successful in 1m59s Details
Build / Build Components (1.21) (pull_request) Successful in 2m27s Details
Build / Build Components (1.22) (pull_request) Successful in 4m25s Details
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m57s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m38s Details
Tests and linters / gopls check (pull_request) Successful in 5m57s Details
Tests and linters / Lint (pull_request) Successful in 6m26s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 9m5s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 9m11s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m4s Details
* Make APE rule parser to read condition's kind in unambiguous using lexemes
`ResourceCondition`, `RequestCondition` instead confusing `Object.Request`, `Object.Resource`.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-14 12:23:26 +03:00
Dmitrii Stepanov 0144117cc9 [#1125] objectSvc: Add EC header APE check
Build / Build Components (1.21) (pull_request) Successful in 6m27s Details
DCO action / DCO (pull_request) Successful in 6m38s Details
Build / Build Components (1.22) (pull_request) Successful in 8m54s Details
Vulncheck / Vulncheck (pull_request) Successful in 8m37s Details
Tests and linters / gopls check (pull_request) Successful in 10m32s Details
Tests and linters / Staticcheck (pull_request) Successful in 11m3s Details
Tests and linters / Lint (pull_request) Successful in 11m27s Details
Pre-commit hooks / Pre-commit (pull_request) Successful in 14m16s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 14m26s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 15m14s Details
Tests and linters / Tests with -race (pull_request) Successful in 15m45s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-08 16:25:55 +03:00
Dmitrii Stepanov ada1b9f737 [#1120] objectSvc: Fix EC put placement
Use parent object ID to compute placement.
Fix too many copies saving.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-08 15:23:57 +03:00
Anton Nikiforov fe2c1c926f [#1112] node: Fix race warning for `GetObjectAndWritePayload`
DCO action / DCO (pull_request) Successful in 2m10s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m2s Details
Build / Build Components (1.22) (pull_request) Successful in 3m44s Details
Build / Build Components (1.21) (pull_request) Successful in 3m52s Details
Pre-commit hooks / Pre-commit (pull_request) Successful in 5m40s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m40s Details
Tests and linters / Lint (pull_request) Successful in 7m11s Details
Tests and linters / gopls check (pull_request) Successful in 9m16s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 10m58s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 11m2s Details
Tests and linters / Tests with -race (pull_request) Successful in 11m35s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-07 14:47:21 +03:00
Anton Nikiforov 3e782527b8 [#1112] node: Add test for `Range` request for EC object
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-07 14:47:21 +03:00
Anton Nikiforov 21a490da8f [#1112] Fix issue from `gofumpt`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-07 14:47:21 +03:00
Evgenii Stratonikov 93c0ccad4f [#1077] objectsvc: Fix possible panic in GetRange()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-07 14:47:21 +03:00
Anton Nikiforov 00b2b77b26 [#1112] node: Implement `Range\RangeHash` requests for EC object
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-07 14:47:21 +03:00
Airat Arifullin b60a51b862 [#1117] ape: Introduce `FormFrostfsIDRequestProperties` method
* `FormFrostfsIDRequestProperties` gets user claim tags and group id and sets them
  as ape request properties.
* Make tree, container and object service use the method.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-07 10:01:21 +00:00
Airat Arifullin 6c76c9b457 [#1117] core: Introduce SubjectProvider interface for FrostfsID
* Make tree, object and container services use SubjectProvider interface.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-07 10:01:21 +00:00
Ekaterina Lebedeva e07869a8cf [#1100] Remove unused fields
DCO action / DCO (pull_request) Successful in 2m47s Details
Build / Build Components (1.21) (pull_request) Successful in 3m21s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m33s Details
Build / Build Components (1.22) (pull_request) Successful in 5m25s Details
Tests and linters / gopls check (pull_request) Successful in 5m13s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m13s Details
Tests and linters / Lint (pull_request) Successful in 7m2s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 10m2s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m55s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 10m11s Details
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-05-06 10:14:36 +03:00
Evgenii Stratonikov 71789676d5 [#1114] aclsvc: Add tests for request ownership
DCO action / DCO (pull_request) Successful in 5m13s Details
Build / Build Components (1.21) (pull_request) Successful in 9m51s Details
Build / Build Components (1.22) (pull_request) Successful in 10m21s Details
Vulncheck / Vulncheck (pull_request) Successful in 12m9s Details
Tests and linters / Lint (pull_request) Successful in 16m50s Details
Tests and linters / gopls check (pull_request) Successful in 18m8s Details
Tests and linters / Staticcheck (pull_request) Successful in 19m1s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 19m59s Details
Tests and linters / Tests with -race (pull_request) Successful in 20m5s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 2m52s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-02 11:57:39 +03:00
Anton Nikiforov 112a7c690f [#1103] node: Implement `Get\Head` requests for EC object
DCO action / DCO (pull_request) Successful in 1m44s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m3s Details
Build / Build Components (1.21) (pull_request) Successful in 4m0s Details
Build / Build Components (1.22) (pull_request) Successful in 3m57s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m46s Details
Tests and linters / gopls check (pull_request) Successful in 4m48s Details
Tests and linters / Lint (pull_request) Successful in 5m45s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m57s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m10s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 9m20s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 18:15:53 +03:00
Airat Arifullin c21d72ac23 [#1096] object: Make ape middleware fill request with user claim tags
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-16 15:12:44 +03:00
Evgenii Stratonikov 91e79c98ba [#1089] ape: Provide request actor as an additional target
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-16 11:03:50 +00:00
Airat Arifullin f4dcb418f2 [#1090] ape: Move ape request and resource implementations to common package
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-15 07:45:45 +00:00
Evgenii Stratonikov 3dc81cb4fc Reapply "[#972] Use min/max builtins"
This reverts commit dad56d2e98.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
Dmitrii Stepanov e74bdaa5d5 [#1080] ape: Use value for APE request
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-09 18:42:03 +03:00
Dmitrii Stepanov 338d8cbebd [#1080] ape: Do not read object headers before Head/Get
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-09 15:27:40 +03:00
Anton Nikiforov 2b88361849 [#1062] object: Fix buffer allocation for `PayloadRange`
DCO action / DCO (pull_request) Successful in 5m34s Details
Vulncheck / Vulncheck (pull_request) Successful in 5m14s Details
Build / Build Components (1.20) (pull_request) Successful in 8m56s Details
Build / Build Components (1.21) (pull_request) Successful in 8m56s Details
Tests and linters / gopls check (pull_request) Successful in 9m3s Details
Tests and linters / Staticcheck (pull_request) Successful in 9m39s Details
Tests and linters / Lint (pull_request) Successful in 10m9s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 13m43s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 14m4s Details
Tests and linters / Tests with -race (pull_request) Successful in 14m40s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-09 11:59:07 +03:00
Dmitrii Stepanov 1c5e0f90aa [#1064] putsvc: Add EC put
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-09 07:08:53 +00:00
Dmitrii Stepanov 39da643354 [#1064] putsvc: Refactor distributed target
Extract object builder.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-09 07:08:53 +00:00
Airat Arifullin 6959e617c4 [#1047] object: Set container owner ID property to ape request
* Introduce ContainerOwner field in RequestContext.
* Set ContainerOwner in aclv2 middleware.
* Set PropertyKeyContainerOwnerID for object ape request.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-18 15:39:50 +00:00
Airat Arifullin d7be70e93f [#1040] object: Wrap CheckAPE errors to status errors
* All methods should wrap CheckAPE error, if it occurs, to
  status error.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-14 07:34:03 +00:00
Airat Arifullin 5c252c9193 [#1039] object: Skip APE check for certain request roles
DCO action / DCO (pull_request) Successful in 1m31s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m52s Details
Build / Build Components (1.21) (pull_request) Successful in 3m52s Details
Build / Build Components (1.20) (pull_request) Successful in 4m16s Details
Tests and linters / gopls check (pull_request) Successful in 11m54s Details
Tests and linters / Staticcheck (pull_request) Successful in 12m31s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 12m49s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 13m8s Details
Tests and linters / Tests with -race (pull_request) Successful in 13m14s Details
Tests and linters / Lint (pull_request) Successful in 13m31s Details
* Skip APE check if a role is Container.
* Skip APE check if a role is IR and methods are get-like.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-03-12 16:15:20 +03:00
Dmitrii Stepanov d433b49265 [#973] node: Resolve perfsprint linter
DCO action / DCO (pull_request) Successful in 2m40s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m41s Details
Build / Build Components (1.20) (pull_request) Successful in 4m27s Details
Build / Build Components (1.21) (pull_request) Successful in 5m6s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m16s Details
Tests and linters / gopls check (pull_request) Successful in 6m23s Details
Tests and linters / Lint (pull_request) Successful in 6m48s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m4s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m9s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 9m23s Details
`fmt.Errorf can be replaced with errors.New` and `fmt.Sprintf can be replaced with string addition`

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-11 17:55:50 +03:00
Dmitrii Stepanov d6534fd755 [#1016] frostfs-node: Fix gopls issues
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-01 12:13:43 +03:00
Airat Arifullin 7cc368e188 [#986] object: Introduce soft ape checks
* Soft APE check means that APE should allow request even
  it gets status NoRuleFound for a request. Otherwise,
  it is interpreted as Deny.
* Soft APE check is performed if basic ACL mask is not set.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-28 19:05:57 +00:00
Evgenii Stratonikov dad56d2e98 Revert "[#972] Use min/max builtins"
This reverts commit 89784b2e0a.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 15:36:01 +00:00
Evgenii Stratonikov 89784b2e0a [#972] Use min/max builtins
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-19 13:13:09 +00:00
Dmitrii Stepanov 2680192ba0 [#988] objectSvc: Fix `SetMarshalData` for PutSingle
DCO action / DCO (pull_request) Successful in 5m10s Details
Vulncheck / Vulncheck (pull_request) Successful in 5m24s Details
Build / Build Components (1.21) (pull_request) Successful in 7m40s Details
Build / Build Components (1.20) (pull_request) Successful in 7m50s Details
Tests and linters / Staticcheck (pull_request) Successful in 7m52s Details
Tests and linters / Lint (pull_request) Successful in 9m8s Details
Tests and linters / Tests with -race (pull_request) Successful in 10m54s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m12s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 11m7s Details
After api-go update it is required to pass marshal data
to `SetMarshalData`.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-15 17:21:08 +03:00
Airat Arifullin a5446bc17d [#952] object: Pass namespace within context in ACL service
DCO action / DCO (pull_request) Successful in 6m23s Details
Vulncheck / Vulncheck (pull_request) Successful in 7m3s Details
Build / Build Components (1.21) (pull_request) Successful in 8m21s Details
Build / Build Components (1.20) (pull_request) Successful in 8m31s Details
Tests and linters / Staticcheck (pull_request) Successful in 11m1s Details
Tests and linters / Lint (pull_request) Successful in 11m26s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m51s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m14s Details
Tests and linters / Tests with -race (pull_request) Successful in 13m31s Details
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-02 14:48:11 +03:00
Airat Arifullin 5be2af881a [#934] container: Make container APE middleware read namespaces
* Those methods that can access already existing containers and thus
  can get container properties should read namespace from Zone
  property. If Zone is not set, take a namespace for root.
* Otherwise, define namespaces by owner ID via frostfs-id contract.
* Improve unit-tests, consider more cases.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-01 17:38:24 +00:00
Evgenii Stratonikov 6e2cc32768 [#681] objsvc: Validate session token owner for local sessions
Previously, the check was in place only when session token was missing.
Format validator checks are applied only to fully-prepared object, so
this lead to the following situation:
1. Object is put locally with malformed token, because there are no
   checks.
2. Object cannot be replicated, because the token is malformed.

This is now fixed and token check is done before any payload receival.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-26 08:52:29 +00:00
Anton Nikiforov b6fc3321c5 [#876] Fix linters
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-25 20:26:13 +03:00
Airat Arifullin f2f3294fc3 [#919] ape: Improve error messages in ape service
* Wrap all APE middleware errors in apeErr that
  makes errors more explicit with status AccessDenied.
* Use denyingRuleErr for denying status from chain router.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-23 08:11:24 +00:00
Airat Arifullin 96b020626f [#915] ape: Fix method name in getStreamBasicChecker
DCO action / DCO (pull_request) Successful in 2m2s Details
Build / Build Components (1.21) (pull_request) Successful in 2m17s Details
Build / Build Components (1.20) (pull_request) Successful in 3m2s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m39s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m54s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m49s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m11s Details
Tests and linters / Lint (pull_request) Successful in 6m44s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m32s Details
* Replace incorrect MethodGetContainer by MethodGetObject constant.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-16 23:52:37 +03:00