Commit Graph

1033 Commits (cc2da73b20a9aefd9fcab5ce9bb9312ffd41153d)

Author SHA1 Message Date
Airat Arifullin f2f3294fc3 [#919] ape: Improve error messages in ape service
* Wrap all APE middleware errors in apeErr that
  makes errors more explicit with status AccessDenied.
* Use denyingRuleErr for denying status from chain router.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-23 08:11:24 +00:00
Airat Arifullin be33070550 [#910] control: Take empty namespace for 'root'
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-19 06:22:10 +00:00
Airat Arifullin 96b020626f [#915] ape: Fix method name in getStreamBasicChecker
DCO action / DCO (pull_request) Successful in 2m2s Details
Build / Build Components (1.21) (pull_request) Successful in 2m17s Details
Build / Build Components (1.20) (pull_request) Successful in 3m2s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m39s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m54s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m49s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m11s Details
Tests and linters / Lint (pull_request) Successful in 6m44s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m32s Details
* Replace incorrect MethodGetContainer by MethodGetObject constant.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-16 23:52:37 +03:00
Airat Arifullin c8baf76fae [#872] object: Introduce APE middlewar for object service
DCO action / DCO (pull_request) Successful in 2m4s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m12s Details
Build / Build Components (1.21) (pull_request) Successful in 4m1s Details
Build / Build Components (1.20) (pull_request) Successful in 4m13s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m3s Details
Tests and linters / Lint (pull_request) Successful in 8m7s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m14s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m18s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m24s Details
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-12 18:41:35 +03:00
Dmitrii Stepanov 52ffa9f164 [#891] getSvc: Refactor Get service V2 creation
DCO action / DCO (pull_request) Successful in 2m46s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m53s Details
Build / Build Components (1.20) (pull_request) Successful in 4m31s Details
Build / Build Components (1.21) (pull_request) Successful in 4m26s Details
Tests and linters / Lint (pull_request) Successful in 6m9s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m7s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m27s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 9m4s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m48s Details
Use arguments for mandatory fields.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-12 13:35:38 +03:00
Dmitrii Stepanov 394f086fe2 [#891] getSvc: Fix get range hash implementation
Get range can perform GET request, so this request must be done
from container node to not to get access denied error.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-12 13:35:38 +03:00
Dmitrii Stepanov f1b2b8bffa [#895] test: Fix NewLogger arguments list
`debug` is always true.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-11 12:32:09 +00:00
Airat Arifullin a8e52ef7aa [#898] control: Fix codes for returning APE errors
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-11 12:31:31 +00:00
Denis Kirillov c19396d203 [#885] control: Make chain id bytes in grpc
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-01-11 07:24:22 +00:00
Dmitrii Stepanov 5c0a736a25 [#899] containerSvc: Fix invalid session token type
DCO action / DCO (pull_request) Successful in 1m23s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m29s Details
Tests and linters / Tests (1.21) (pull_request) Failing after 3m58s Details
Build / Build Components (1.21) (pull_request) Successful in 3m46s Details
Build / Build Components (1.20) (pull_request) Successful in 3m52s Details
Tests and linters / Lint (pull_request) Successful in 4m48s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m5s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m4s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m36s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-10 18:37:54 +03:00
Dmitrii Stepanov 581887148a [#569] cli: Add `control shards writecache seal` command
DCO action / DCO (pull_request) Successful in 2m46s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m3s Details
Build / Build Components (1.21) (pull_request) Successful in 3m53s Details
Build / Build Components (1.20) (pull_request) Successful in 4m0s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m31s Details
Tests and linters / Lint (pull_request) Successful in 6m17s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m47s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m14s Details
Tests and linters / Tests with -race (pull_request) Successful in 13m12s Details
It does the same as `control shards flush-writecache --seal`, but
has better name.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 16:05:37 +03:00
Dmitrii Stepanov 0cb0fc1735 [#569] writecache: Allow to seal writecache after flush
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 16:05:37 +03:00
Dmitrii Stepanov b118734909 [#890] getsvc: Log node PK
DCO action / DCO (pull_request) Successful in 13m55s Details
Vulncheck / Vulncheck (pull_request) Successful in 16m51s Details
Build / Build Components (1.20) (pull_request) Successful in 17m52s Details
Build / Build Components (1.21) (pull_request) Successful in 18m9s Details
Tests and linters / Tests (1.20) (pull_request) Failing after 24m34s Details
Tests and linters / Tests (1.21) (pull_request) Failing after 25m56s Details
Tests and linters / Staticcheck (pull_request) Successful in 26m46s Details
Tests and linters / Lint (pull_request) Successful in 28m6s Details
Tests and linters / Tests with -race (pull_request) Failing after 33m35s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-29 14:24:13 +03:00
Dmitrii Stepanov 764f70634d [#881] containerSvc: Add APE validation
Vulncheck / Vulncheck (pull_request) Successful in 2m40s Details
DCO action / DCO (pull_request) Successful in 2m27s Details
Build / Build Components (1.21) (pull_request) Successful in 3m22s Details
Tests and linters / Lint (pull_request) Successful in 4m52s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m46s Details
Build / Build Components (1.20) (pull_request) Successful in 4m54s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m59s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 12m38s Details
Tests and linters / Tests with -race (pull_request) Successful in 13m10s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-27 11:05:34 +03:00
Anton Nikiforov eab981bf1a [#876] Fix linter error
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-12-21 18:21:37 +03:00
Airat Arifullin bdd43f6211 [#869] object: Pass just CID to chain router
* Do not convert CID from request to native-schema resource
  format - this step is unneccessary for APE.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-14 11:01:20 +00:00
Dmitrii Stepanov 11add38e87 [#857] golangci: Add protogetter linter
DCO action / DCO (pull_request) Successful in 1m38s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m2s Details
Build / Build Components (1.21) (pull_request) Successful in 3m51s Details
Build / Build Components (1.20) (pull_request) Successful in 4m3s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m17s Details
Tests and linters / Lint (pull_request) Successful in 6m11s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m7s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 11m32s Details
Tests and linters / Tests with -race (pull_request) Successful in 12m22s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-12 16:27:02 +03:00
Dmitrii Stepanov 681b2c5fd4 [#825] policer: Do not drop required linking objects
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-12 11:04:03 +00:00
Airat Arifullin 0f45e3d344 [#804] ape: Implement boltdb storage for local overrides
DCO action / DCO (pull_request) Successful in 2m10s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m26s Details
Build / Build Components (1.20) (pull_request) Successful in 5m41s Details
Build / Build Components (1.21) (pull_request) Successful in 5m44s Details
Tests and linters / Staticcheck (pull_request) Successful in 7m10s Details
Tests and linters / Lint (pull_request) Successful in 8m14s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 14m24s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 14m41s Details
Tests and linters / Tests with -race (pull_request) Successful in 14m38s Details
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 19:08:41 +03:00
Airat Arifullin e361e017f3 [#842] control: Pass target instead resource name
* Update policy-engine package version in go.mod, go.sum.
* Refactor CheckIfRequestPermitted: pass container target
  instead container ID.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 14:21:55 +00:00
Airat Arifullin 39060382a1 [#842] control: Recieve target in gRPC methods for APE managing
* Introduce Target type and pass it to all gRPC methods
  for APE chain managing instead CID.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 14:21:55 +00:00
Dmitrii Stepanov 26e4f7005c [#741] treesvc: Refactor tree sync
Fix linter issues.
Add error logging.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-30 12:45:02 +00:00
Dmitrii Stepanov b21be1abdd [#741] treesvc: Do not update sync height if some node is unavailable
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-30 12:45:02 +00:00
Dmitrii Stepanov b215817e14 [#741] treesvc: Remove unused height variables
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-30 12:45:02 +00:00
Dmitrii Stepanov c516c7c5f4 [#821] node: Pass user.ID by value
DCO action / DCO (pull_request) Successful in 3m45s Details
Build / Build Components (1.21) (pull_request) Successful in 5m18s Details
Build / Build Components (1.20) (pull_request) Successful in 5m28s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m30s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m42s Details
Tests and linters / Lint (pull_request) Successful in 8m25s Details
Vulncheck / Vulncheck (pull_request) Successful in 9m22s Details
Tests and linters / Staticcheck (pull_request) Successful in 10m57s Details
Tests and linters / Tests with -race (pull_request) Successful in 16m53s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-23 10:21:07 +03:00
Airat Arifullin 4d5be5ccb5 [#811] ape: Update policy-engine module version and rebase
DCO action / DCO (pull_request) Successful in 4m23s Details
Vulncheck / Vulncheck (pull_request) Successful in 5m31s Details
Build / Build Components (1.21) (pull_request) Successful in 7m33s Details
Build / Build Components (1.20) (pull_request) Successful in 7m40s Details
Tests and linters / Staticcheck (pull_request) Successful in 8m22s Details
Tests and linters / Lint (pull_request) Successful in 9m23s Details
Tests and linters / Tests with -race (pull_request) Successful in 11m20s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 11m32s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m41s Details
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-16 11:31:37 +03:00
Dmitrii Stepanov 8088063195 [#787] netmap: Refactor `NewEpoch` method
DCO action / DCO (pull_request) Successful in 3m58s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m51s Details
Build / Build Components (1.21) (pull_request) Successful in 5m47s Details
Build / Build Components (1.20) (pull_request) Successful in 5m57s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m25s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m31s Details
Tests and linters / Lint (pull_request) Successful in 7m15s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m16s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m30s Details
Split for user and control methods.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:22:31 +03:00
Dmitrii Stepanov 2393d13e4d [#787] morph: Return VUB for IR service calls
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:13:03 +03:00
Dmitrii Stepanov bdfa523487 [#787] proto: Add VUB field for IR service
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-13 17:13:03 +03:00
Dmitrii Stepanov 0f75e48138 [#796] policer: Fix tombstone objects replication
Tombstone objects must be replicated to all container nodes.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 13:39:33 +03:00
Dmitrii Stepanov 7cdae4f660 [#792] proto: Regenerate with fixed version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 10:09:13 +00:00
Dmitrii Stepanov 9133b4389e [#788] objectsvc: Fix formatting (gofumpt)
DCO action / DCO (pull_request) Successful in 3m19s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m40s Details
Build / Build Components (1.21) (pull_request) Successful in 4m17s Details
Build / Build Components (1.20) (pull_request) Successful in 4m32s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m46s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m9s Details
Tests and linters / Lint (pull_request) Successful in 5m28s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m24s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m38s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-09 10:27:32 +03:00
Airat Arifullin 3534d6d05b [#794] objectsvc: Return accidentally removed acl checks for Head
DCO action / DCO (pull_request) Successful in 1m42s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m23s Details
Build / Build Components (1.21) (pull_request) Successful in 4m22s Details
Build / Build Components (1.20) (pull_request) Successful in 5m44s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m3s Details
Tests and linters / Lint (pull_request) Successful in 6m35s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m32s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m47s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m54s Details
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 17:13:58 +03:00
Airat Arifullin 66848d3288 [#770] cli: Add methods to work with APE rules via control svc
* Add methods to frostfs-cli
* Implement rpc in control service

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
Airat Arifullin 8e11ef46b8 [#770] object: Introduce ape chain checker for object svc
* Introduce Request type converted from RequestInfo type
  to implement policy-engine's Request interface
* Implement basic ape checker to check if a request is
  permitted to be performed
* Make put handlers use APE checker instead EACL

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
Airat Arifullin 5ec73fe8a0 [#770] node: Introduce ape chain source
* Provide methods to access rule chains with access
  policy engine (APE) chain source
* Initialize apeChainSource within object service
  initialization
* Share apeChainSource with control service
* Implement dummy apeChainSource instance based on
  in-memory implementation

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
Airat Arifullin 3a2c319b87 [#770] control: Generate gRPC methods to manipulate APE chains
* Define new types and gRPC methods to manipulate APE chains
  in control service.
* Stub gRPC handlers for the generated methods.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-08 13:34:03 +00:00
Dmitrii Stepanov 226e84d782 [#684] node: Add skipped objects count to evacuation result
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-07 12:17:11 +00:00
Dmitrii Stepanov 523fb3ca51 [#684] proto: Add skipped count to evacuation status response
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-07 12:17:11 +00:00
Airat Arifullin 74c91eeef5 [#777] client: Refactor PrmContainerList, PrmObjectSearch usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-06 06:50:11 +00:00
Dmitrii Stepanov 20d6132f31 [#531] signSvc: Add SetMarshaledData method call
Vulncheck / Vulncheck (pull_request) Successful in 1m25s Details
DCO action / DCO (pull_request) Successful in 1m36s Details
Build / Build Components (1.21) (pull_request) Successful in 2m40s Details
Build / Build Components (1.20) (pull_request) Successful in 3m2s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m41s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m32s Details
Tests and linters / Lint (pull_request) Successful in 5m59s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m3s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m6s Details
To reduce memory allocations add `SetMarshaledData` method call
to return already marshalled data in next `StableMarshal` calls.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-02 17:34:33 +03:00
Dmitrii Stepanov 05b508f79a [#772] proto: Fix file ending
DCO action / DCO (pull_request) Successful in 3m47s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m47s Details
Build / Build Components (1.21) (pull_request) Successful in 6m5s Details
Build / Build Components (1.20) (pull_request) Successful in 6m13s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 13m9s Details
Tests and linters / Staticcheck (pull_request) Successful in 13m17s Details
Tests and linters / Lint (pull_request) Successful in 13m49s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 14m12s Details
Tests and linters / Tests with -race (pull_request) Successful in 4m30s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-31 17:03:04 +03:00
Dmitrii Stepanov 79088baa06 [#772] node: Apply gofumpt
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-31 17:03:03 +03:00
Anton Nikiforov b0cf100427 [#49] node: React on SIGHUP only when node in READY state
Add more info in logs when node is going to shut down,
but initialization process still in progress.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-24 15:55:29 +00:00
Airat Arifullin 58b6224dd8 [#747] client: Refactor PrmObjectPutInit usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-20 11:55:40 +00:00
Airat Arifullin 12b7cf2533 [#747] client: Refactor PrmObjectPutSingle usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-10-20 11:55:40 +00:00
Dmitrii Stepanov 189dbb01be [#733] frostfs-cli: Add `control ir remove-container`
DCO action / DCO (pull_request) Successful in 3m51s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m49s Details
Build / Build Components (1.21) (pull_request) Successful in 4m34s Details
Build / Build Components (1.20) (pull_request) Successful in 6m21s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 9m1s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m12s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m40s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m21s Details
Tests and linters / Lint (pull_request) Successful in 9m39s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-19 16:22:18 +03:00
Evgenii Stratonikov aca11d7474 [#735] policer: Allow to provide metrics from the outside
DCO action / DCO (pull_request) Successful in 3m9s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m28s Details
Build / Build Components (1.21) (pull_request) Successful in 4m17s Details
Build / Build Components (1.20) (pull_request) Successful in 4m35s Details
Tests and linters / Lint (pull_request) Successful in 7m19s Details
Tests and linters / Tests with -race (pull_request) Failing after 9m47s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m0s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m19s Details
Tests and linters / Staticcheck (pull_request) Successful in 11m4s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-11 15:14:13 +03:00
Ekaterina Lebedeva 5e229dc248 [#701] metrics: add metric to evaluate policer performance
Add processed objects counter in policerMetrics,
add policer field to NodeMetrics

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-10-09 19:02:08 +00:00
Evgenii Stratonikov 4caa934eea [#729] containersvc: Remove load announcement
IR code was removed in 8879c6ea.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-09 19:01:13 +00:00