Commit Graph

4532 Commits (7cc368e188dc03e761cad0801e7f269a81b1ed83)

Author SHA1 Message Date
Dmitrii Stepanov 4358d3c423 [#917] controlSvc: Add DetachShards handler
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-06 14:47:52 +03:00
Ekaterina Lebedeva afd2ba9a66 [#110] Add check for repeated error log in policer
DCO action / DCO (pull_request) Successful in 2m55s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m22s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m10s Details
Build / Build Components (1.20) (pull_request) Successful in 4m58s Details
Build / Build Components (1.21) (pull_request) Successful in 4m54s Details
Tests and linters / Lint (pull_request) Successful in 5m47s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m45s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m58s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m42s Details
processObject() returns 3 types of errors: container not found errors,
could not get container error and placement vector building error. Every
error will occur for all objects in container simultaneously, so we can
log each error once and safely ignore the rest.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-02-06 00:56:41 +03:00
Evgenii Stratonikov 602ee11123 [#934] containersvc: Marhal public key in short format for APE
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-02 17:51:38 +00:00
Airat Arifullin c1a5b831b6 [#955] chainbase: Fix rule chain unmarshalling
DCO action / DCO (pull_request) Successful in 4m41s Details
Vulncheck / Vulncheck (pull_request) Successful in 4m46s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m30s Details
Tests and linters / Lint (pull_request) Successful in 7m36s Details
Build / Build Components (1.21) (pull_request) Successful in 8m57s Details
Build / Build Components (1.20) (pull_request) Successful in 9m8s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 10m32s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 10m42s Details
Tests and linters / Tests with -race (pull_request) Successful in 11m23s Details
* Use correct way DecodeBytes instead unmarshalling by json.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-02 20:28:04 +03:00
Anton Nikiforov befbaf9d56 [#922] cli: Add new command `control list-targets`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
Anton Nikiforov f64c48b157 [#922] Fix linter issue
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
Anton Nikiforov 9916598dfb [#922] control: Extend api with `ListOverrideDefinedTargets`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
Anton Nikiforov 95e15f499f [#922] Update files generated by protoc
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
Anton Nikiforov 2cb04379a4 [#922] go.mod: Update APE
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-02 12:09:51 +00:00
Airat Arifullin a5446bc17d [#952] object: Pass namespace within context in ACL service
DCO action / DCO (pull_request) Successful in 6m23s Details
Vulncheck / Vulncheck (pull_request) Successful in 7m3s Details
Build / Build Components (1.21) (pull_request) Successful in 8m21s Details
Build / Build Components (1.20) (pull_request) Successful in 8m31s Details
Tests and linters / Staticcheck (pull_request) Successful in 11m1s Details
Tests and linters / Lint (pull_request) Successful in 11m26s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m51s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m14s Details
Tests and linters / Tests with -race (pull_request) Successful in 13m31s Details
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-02 14:48:11 +03:00
Anton Nikiforov d0eadf7ea2 [#799] engine: Skip put when object removed from shard
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-01 17:49:22 +00:00
Anton Nikiforov 6534252c22 [#799] policer: Refactor method `processNodes`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-02-01 17:49:22 +00:00
Airat Arifullin 5be2af881a [#934] container: Make container APE middleware read namespaces
* Those methods that can access already existing containers and thus
  can get container properties should read namespace from Zone
  property. If Zone is not set, take a namespace for root.
* Otherwise, define namespaces by owner ID via frostfs-id contract.
* Improve unit-tests, consider more cases.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-01 17:38:24 +00:00
Airat Arifullin 96c86c4637 [#934] adm: Make frostfsid commands read alphabet wallets
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-01 17:38:24 +00:00
Airat Arifullin 4352bd0e8e [#934] ape: Transform empty namespace within chainbase
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-02-01 17:38:24 +00:00
Anton Nikiforov 483a67b170 [#937] ape: Validate chain resource name
DCO action / DCO (pull_request) Successful in 2m13s Details
Build / Build Components (1.20) (pull_request) Successful in 3m38s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m13s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m15s Details
Build / Build Components (1.21) (pull_request) Successful in 7m11s Details
Tests and linters / Lint (pull_request) Successful in 9m35s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m56s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 14m14s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m2s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-31 11:34:35 +03:00
Anton Nikiforov e3573de6db [#930] gc: Stop internal activity by context
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-31 08:30:34 +00:00
Anton Nikiforov c441296592 [#930] policer: Release task pool when context cancelled
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-31 08:30:34 +00:00
Dmitrii Stepanov 675eec91f3 [#938] shard: Update only changed counters
DCO action / DCO (pull_request) Successful in 5m40s Details
Vulncheck / Vulncheck (pull_request) Successful in 5m33s Details
Tests and linters / Staticcheck (pull_request) Successful in 7m12s Details
Build / Build Components (1.21) (pull_request) Successful in 7m25s Details
Build / Build Components (1.20) (pull_request) Successful in 7m29s Details
Tests and linters / Lint (pull_request) Successful in 8m26s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m48s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m56s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 10m10s Details
If metric value hasn't changed, but we update metric, then
non existed metric will apear with zero value.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-30 12:37:48 +03:00
Dmitrii Stepanov c681354afd [#938] engine: Fix container count removal
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-30 12:37:48 +03:00
Evgenii Stratonikov df055fead5 [#931] morph: Provide batch size for container listing explicitly
Besides VM stack item limit we also have restrictions on the size of
JSON for a stackitem, which is 128k. This limit is much harder to
calculate, because JSON representation includes type and the encoding is
different for different items. Thus is makes no sense to invent our own
default, so use the one provided by neo-go. But for container listing we
know exactly what we process, so use big enough value, which is tested.

Introduced in be8607a1f6.

Refs #902
Refs https://github.com/nspcc-dev/neo-go/blob/v0.105.0/pkg/vm/stackitem/json.go#L353

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-29 14:00:11 +00:00
Evgenii Stratonikov c916a75948 [#935] .forgejo: Update dco-go to v3
DCO action / DCO (pull_request) Successful in 3m23s Details
Vulncheck / Vulncheck (pull_request) Successful in 5m22s Details
Tests and linters / Staticcheck (pull_request) Successful in 8m5s Details
Build / Build Components (1.20) (pull_request) Successful in 10m33s Details
Build / Build Components (1.21) (pull_request) Successful in 10m28s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 14m25s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 20m17s Details
Tests and linters / Lint (pull_request) Successful in 17m12s Details
Tests and linters / Tests with -race (pull_request) Successful in 21m33s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-26 12:19:20 +03:00
Evgenii Stratonikov 6e2cc32768 [#681] objsvc: Validate session token owner for local sessions
Previously, the check was in place only when session token was missing.
Format validator checks are applied only to fully-prepared object, so
this lead to the following situation:
1. Object is put locally with malformed token, because there are no
   checks.
2. Object cannot be replicated, because the token is malformed.

This is now fixed and token check is done before any payload receival.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-26 08:52:29 +00:00
Anton Nikiforov 417f8fc2c2 [#876] cli: Add doc for commands `control *-rule`
DCO action / DCO (pull_request) Successful in 7m54s Details
Tests and linters / Lint (pull_request) Successful in 9m7s Details
Vulncheck / Vulncheck (pull_request) Successful in 8m51s Details
Build / Build Components (1.20) (pull_request) Successful in 11m57s Details
Build / Build Components (1.21) (pull_request) Successful in 11m52s Details
Tests and linters / Staticcheck (pull_request) Successful in 14m15s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 14m46s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 15m9s Details
Tests and linters / Tests with -race (pull_request) Successful in 15m17s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-25 20:26:13 +03:00
Anton Nikiforov 51d1d935ef [#876] cli: Add support for `container` in local rules
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-25 20:26:13 +03:00
Anton Nikiforov b6fc3321c5 [#876] Fix linters
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-01-25 20:26:13 +03:00
Airat Arifullin 1fe7736d92 [#925] morph: Introduce switch rpc guarded rpc actor
* Introduce switch rpc guarded rpc actor in morph client
  to prevent using invalidated rpc actor when RPC switch
  happens.
* Initialize NewContractStorage with SwitchRPCGuardedActor.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-25 13:24:13 +00:00
Evgenii Stratonikov d13e37f70b [#768] adm: Do not change frostfsid admin on update
Behave similarly to the netmap contract.
Close #768

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-25 10:54:42 +00:00
Evgenii Stratonikov d2d850786d [#923] adm: Move netconfig merge to a separate function
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-25 10:54:42 +00:00
Evgenii Stratonikov 61a3afbf9b [#768] adm: Rephrase error text for NNS fetch
Error could come from the network, we must not assume everything.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-25 10:54:42 +00:00
Evgenii Stratonikov c00eb7ccee [#768] adm: Return error from getContractDeployData()
Make code more idiomatic and unify in style with other error reporting.
Retain panics for code errors.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-25 10:54:42 +00:00
Evgenii Stratonikov cc2da73b20 [#929] go.mod: Update dependencies
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-01-25 07:24:11 +00:00
Dmitrii Stepanov 5ed330e436 [#927] metabase: Delete GC marks
DCO action / DCO (pull_request) Successful in 6m34s Details
Vulncheck / Vulncheck (pull_request) Successful in 7m19s Details
Build / Build Components (1.20) (pull_request) Successful in 7m48s Details
Build / Build Components (1.21) (pull_request) Successful in 9m39s Details
Tests and linters / Staticcheck (pull_request) Successful in 10m36s Details
Tests and linters / Lint (pull_request) Successful in 11m26s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m57s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m8s Details
Tests and linters / Tests with -race (pull_request) Successful in 13m7s Details
`key` is changed inside `db.get`, so encode address again after get.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-24 18:51:16 +03:00
Dmitrii Stepanov 931a5e9aaf [#918] engine: Move shard to degraded mode if metabase open failed
DCO action / DCO (pull_request) Successful in 1m55s Details
Build / Build Components (1.20) (pull_request) Successful in 2m14s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m15s Details
Build / Build Components (1.21) (pull_request) Successful in 4m6s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m58s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m24s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m11s Details
Tests and linters / Lint (pull_request) Successful in 6m37s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m35s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-23 11:16:40 +03:00
Airat Arifullin f2f3294fc3 [#919] ape: Improve error messages in ape service
* Wrap all APE middleware errors in apeErr that
  makes errors more explicit with status AccessDenied.
* Use denyingRuleErr for denying status from chain router.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-23 08:11:24 +00:00
Dmitrii Stepanov f526f49995 [#874] engine: Check object existance concurrently
DCO action / DCO (pull_request) Successful in 1m36s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m33s Details
Build / Build Components (1.21) (pull_request) Successful in 3m9s Details
Build / Build Components (1.20) (pull_request) Successful in 4m49s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m5s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m6s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m39s Details
Tests and linters / Lint (pull_request) Successful in 7m0s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m46s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-23 09:28:29 +03:00
Dmitrii Stepanov f5160b27fc [#920] tests: Fix data races
DCO action / DCO (pull_request) Successful in 2m2s Details
Build / Build Components (1.21) (pull_request) Successful in 3m35s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m3s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m23s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m33s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m41s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m47s Details
Tests and linters / Lint (pull_request) Successful in 8m12s Details
Build / Build Components (1.20) (pull_request) Successful in 1m32s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-19 14:06:05 +03:00
Denis Kirillov e42262a863 [#908] adm/frostfsid: Add parameter validations
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-01-19 11:04:03 +00:00
Denis Kirillov 136acdba21 [#908] adm/frostfsid: Use client for write operations
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-01-19 11:04:03 +00:00
Denis Kirillov 6ebd61298e [#908] adm/frostfsid: Support subject/group management
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-01-19 11:04:03 +00:00
Denis Kirillov 0e3d144695 [#908] adm: Support frostfsid basic operations
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-01-19 11:04:03 +00:00
Airat Arifullin be33070550 [#910] control: Take empty namespace for 'root'
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-19 06:22:10 +00:00
Dmitrii Stepanov 63d3ed1ad8 [#904] tests: Close test engine after test
DCO action / DCO (pull_request) Successful in 2m7s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m37s Details
Build / Build Components (1.20) (pull_request) Successful in 4m0s Details
Build / Build Components (1.21) (pull_request) Successful in 3m56s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m25s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m22s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m43s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m47s Details
Tests and linters / Lint (pull_request) Successful in 7m5s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-17 19:04:39 +03:00
Dmitrii Stepanov 57171907e3 [#904] metabase: Return if object was actuall inserted
This requires to count metrics properly.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-17 19:04:39 +03:00
Dmitrii Stepanov c1a80235db [#904] metabase: Log Inhume operation
It will be very useful for troubleshooting.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-17 18:42:52 +03:00
Airat Arifullin 96b020626f [#915] ape: Fix method name in getStreamBasicChecker
DCO action / DCO (pull_request) Successful in 2m2s Details
Build / Build Components (1.21) (pull_request) Successful in 2m17s Details
Build / Build Components (1.20) (pull_request) Successful in 3m2s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m39s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m54s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m49s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m11s Details
Tests and linters / Lint (pull_request) Successful in 6m44s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m32s Details
* Replace incorrect MethodGetContainer by MethodGetObject constant.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-16 23:52:37 +03:00
Airat Arifullin c8baf76fae [#872] object: Introduce APE middlewar for object service
DCO action / DCO (pull_request) Successful in 2m4s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m12s Details
Build / Build Components (1.21) (pull_request) Successful in 4m1s Details
Build / Build Components (1.20) (pull_request) Successful in 4m13s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m3s Details
Tests and linters / Lint (pull_request) Successful in 8m7s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m14s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m18s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m24s Details
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-01-12 18:41:35 +03:00
Denis Kirillov e43609c616 [#885] go.mod: Update APE
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-01-12 15:38:37 +00:00
Dmitrii Stepanov 52ffa9f164 [#891] getSvc: Refactor Get service V2 creation
DCO action / DCO (pull_request) Successful in 2m46s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m53s Details
Build / Build Components (1.20) (pull_request) Successful in 4m31s Details
Build / Build Components (1.21) (pull_request) Successful in 4m26s Details
Tests and linters / Lint (pull_request) Successful in 6m9s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m7s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m27s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 9m4s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m48s Details
Use arguments for mandatory fields.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-12 13:35:38 +03:00
Dmitrii Stepanov 394f086fe2 [#891] getSvc: Fix get range hash implementation
Get range can perform GET request, so this request must be done
from container node to not to get access denied error.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-12 13:35:38 +03:00