Commit graph

3233 commits

Author SHA1 Message Date
8377372a40 [#1276] go.mod: Update api-go
All checks were successful
Tests and linters / Staticcheck (pull_request) Successful in 2m54s
Build / Build Components (1.22) (pull_request) Successful in 3m28s
Build / Build Components (1.21) (pull_request) Successful in 3m16s
Tests and linters / Lint (pull_request) Successful in 4m14s
Tests and linters / Tests with -race (pull_request) Successful in 8m2s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m41s
Tests and linters / Tests (1.22) (pull_request) Successful in 8m54s
Tests and linters / gopls check (pull_request) Successful in 2m53s
DCO action / DCO (pull_request) Successful in 46s
Vulncheck / Vulncheck (pull_request) Successful in 1m4s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m47s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-26 16:44:19 +03:00
7fd7961dfa [#1271] getSvc: Fix local EC chunk get
All checks were successful
Build / Build Components (1.21) (pull_request) Successful in 4m22s
Tests and linters / Staticcheck (pull_request) Successful in 4m56s
Tests and linters / gopls check (pull_request) Successful in 5m59s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m20s
DCO action / DCO (pull_request) Successful in 35s
Build / Build Components (1.22) (pull_request) Successful in 1m15s
Vulncheck / Vulncheck (pull_request) Successful in 59s
Tests and linters / Lint (pull_request) Successful in 2m24s
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m1s
Tests and linters / Tests (1.22) (pull_request) Successful in 6m59s
Tests and linters / Tests with -race (pull_request) Successful in 7m32s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-24 08:52:16 +03:00
8398a8b4b3 [#1271] getSvc: Fix head --raw assemble for EC
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-24 08:52:08 +03:00
9c2c76ca32
[#1259] Move pkg/util/locode to frostfs-locode-db
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 4m47s
DCO action / DCO (pull_request) Successful in 4m40s
Build / Build Components (1.21) (pull_request) Successful in 7m27s
Tests and linters / Staticcheck (pull_request) Successful in 8m22s
Tests and linters / gopls check (pull_request) Successful in 10m56s
Build / Build Components (1.22) (pull_request) Successful in 11m48s
Tests and linters / Tests (1.22) (pull_request) Successful in 16m11s
Tests and linters / Lint (pull_request) Successful in 16m27s
Pre-commit hooks / Pre-commit (pull_request) Successful in 19m12s
Tests and linters / Tests (1.21) (pull_request) Successful in 20m31s
Tests and linters / Tests with -race (pull_request) Successful in 8m56s
Removed pkg/util/locode package, added
git.frostfs.info/TrueCloudLab/frostfs-locode-db/pkg/locode dependency.

Signed-off-by: George Bartolomey <george@bh4.ru>
2024-07-23 14:59:14 +03:00
9ee1bd3669
[#1265] *: Run gofumpt
All checks were successful
DCO action / DCO (pull_request) Successful in 7m50s
Vulncheck / Vulncheck (pull_request) Successful in 11m12s
Build / Build Components (1.22) (pull_request) Successful in 15m55s
Build / Build Components (1.21) (pull_request) Successful in 16m2s
Pre-commit hooks / Pre-commit (pull_request) Successful in 20m47s
Tests and linters / Staticcheck (pull_request) Successful in 4m8s
Tests and linters / gopls check (pull_request) Successful in 4m16s
Tests and linters / Lint (pull_request) Successful in 5m35s
Tests and linters / Tests with -race (pull_request) Successful in 7m58s
Tests and linters / Tests (1.22) (pull_request) Successful in 8m12s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m59s
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-07-23 10:36:32 +03:00
18182e578e [#1261] shard: Fix delete objects from FSTree
All checks were successful
DCO action / DCO (pull_request) Successful in 6m56s
Vulncheck / Vulncheck (pull_request) Successful in 7m38s
Build / Build Components (1.22) (pull_request) Successful in 9m57s
Build / Build Components (1.21) (pull_request) Successful in 10m38s
Tests and linters / gopls check (pull_request) Successful in 10m43s
Tests and linters / Staticcheck (pull_request) Successful in 11m1s
Tests and linters / Lint (pull_request) Successful in 13m35s
Pre-commit hooks / Pre-commit (pull_request) Successful in 14m53s
Tests and linters / Tests (1.21) (pull_request) Successful in 15m55s
Tests and linters / Tests with -race (pull_request) Successful in 15m49s
Tests and linters / Tests (1.22) (pull_request) Successful in 16m10s
Replace nil storageID with empty like by shard.Get.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-22 14:38:09 +03:00
3119f2fbc3 [#1257] metabase: Delete EC gc marks and split info
All checks were successful
DCO action / DCO (pull_request) Successful in 2m12s
Vulncheck / Vulncheck (pull_request) Successful in 4m9s
Pre-commit hooks / Pre-commit (pull_request) Successful in 5m37s
Tests and linters / gopls check (pull_request) Successful in 6m7s
Build / Build Components (1.21) (pull_request) Successful in 6m2s
Build / Build Components (1.22) (pull_request) Successful in 5m55s
Tests and linters / Staticcheck (pull_request) Successful in 6m55s
Tests and linters / Lint (pull_request) Successful in 9m2s
Tests and linters / Tests (1.21) (pull_request) Successful in 11m56s
Tests and linters / Tests with -race (pull_request) Successful in 11m47s
Tests and linters / Tests (1.22) (pull_request) Successful in 11m55s
EC parent and split gc marks should be deleted after last EC chunk delete.
Also should delete split info for EC parent and split parent.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-19 17:14:14 +03:00
eadcea8df0 [#1249] object: Remove all APE pre-checks in handlers
* Methods `Head`, `Get`, `GetRangeHash` should no longer use APE pre-checks
  as that leads only to incorrect rule chain processing for requests:
  1. Immediate return with `NoRuleFound` may be unexpected as some `Allow`
     rule is actually defined but can't be matched yet as it gets no object
     attributes;
  2. Immdediate return with `Allow` may be incorrect as some `Deny` rule
     is actually defined but can't bet matched yet as it gets no object
     attirbutes;
  3. Pre-check breaks compatibility for converted EACL-tables.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-18 13:52:43 +00:00
5e5ee545b6 [#1254] policer: Fix svacer warning
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-18 11:30:39 +00:00
27caa8a72f [#1256] metabase: Put split parent object ID for EC chunks
All checks were successful
DCO action / DCO (pull_request) Successful in 6m13s
Vulncheck / Vulncheck (pull_request) Successful in 6m44s
Tests and linters / gopls check (pull_request) Successful in 9m52s
Build / Build Components (1.22) (pull_request) Successful in 10m1s
Build / Build Components (1.21) (pull_request) Successful in 10m33s
Tests and linters / Staticcheck (pull_request) Successful in 11m5s
Pre-commit hooks / Pre-commit (pull_request) Successful in 14m47s
Tests and linters / Lint (pull_request) Successful in 16m21s
Tests and linters / Tests (1.22) (pull_request) Successful in 16m22s
Tests and linters / Tests with -race (pull_request) Successful in 16m32s
Tests and linters / Tests (1.21) (pull_request) Successful in 16m47s
It is required to save split parent ID too, not only split ID.
Otherwise inhume operation works incorrect: shard with last part may be skipped
and parent object will be available.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-17 18:08:07 +03:00
e83d39e33f [#1253] deleteSvc: Use copy of common parameters
All checks were successful
DCO action / DCO (pull_request) Successful in 3m29s
Vulncheck / Vulncheck (pull_request) Successful in 3m31s
Build / Build Components (1.21) (pull_request) Successful in 4m56s
Build / Build Components (1.22) (pull_request) Successful in 4m57s
Tests and linters / Staticcheck (pull_request) Successful in 5m14s
Tests and linters / gopls check (pull_request) Successful in 5m11s
Tests and linters / Lint (pull_request) Successful in 6m23s
Tests and linters / Tests (1.22) (pull_request) Successful in 12m42s
Tests and linters / Tests (1.21) (pull_request) Successful in 12m54s
Tests and linters / Tests with -race (pull_request) Successful in 16m31s
Pre-commit hooks / Pre-commit (pull_request) Successful in 21m57s
getSvc may change the values of some fields, so Head will affect Delete
or Put. In this case, the change is necessary so that the session token
is stored in the tombstone object (EC assemble calls `ForgetTokens`).

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-17 14:24:27 +03:00
fc383ea6ae [#1253] getSvc: Fix EC objects get
Now EC objects assembling is performed concurrently.
Also fixed issue with an error in case of getting
EC object via non-container node.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-17 14:24:27 +03:00
00a88b9936 [#1251] *: Run gofumpt
All checks were successful
DCO action / DCO (pull_request) Successful in 3m50s
Vulncheck / Vulncheck (pull_request) Successful in 3m37s
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m45s
Build / Build Components (1.21) (pull_request) Successful in 5m7s
Build / Build Components (1.22) (pull_request) Successful in 5m1s
Tests and linters / Staticcheck (pull_request) Successful in 5m13s
Tests and linters / gopls check (pull_request) Successful in 5m30s
Tests and linters / Lint (pull_request) Successful in 6m31s
Tests and linters / Tests (1.21) (pull_request) Successful in 11m39s
Tests and linters / Tests with -race (pull_request) Successful in 11m37s
Tests and linters / Tests (1.22) (pull_request) Successful in 13m11s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-17 11:25:07 +03:00
3940bc17c1 [#1251] pilorama: Allow traversing multiple branches in parallel
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-17 11:25:07 +03:00
e5767c9002 [#1250] *: Reformat proto filets with clang-format
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-16 15:13:08 +00:00
286df198c9 [#1248] placement: Decouple ContainerNodes() cache from the placement builder.
Also, write tests.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-16 12:21:30 +00:00
3a48b282b6 [#1248] placement: Use epoch to track netmap versions
Previously we used pointer, this could have worked,
because most of the time, the netmap is cached.
This didn't work, however, because `lastNm` field was always nil.

Rework the mechanism completely:
1. Use epoch to track netmap versions, as it it simpler and
   is unrelated to the TTL of an underlying cache.
2. Fix a bug where the epoch could change while mutex was unlocked.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-16 12:21:30 +00:00
21431f22c0 [#1248] placement: Use cid.ID as key in the cache
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-16 12:21:30 +00:00
d5dc14c639 [#1243] object: Make APE checker set x-headers to request properties
* Update go.mod, go.sum;
* Add x-headers to request properties;
* Add a unit-test.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-16 07:28:42 +00:00
39866a957c [#1196] morph/client: Fix ArrayFromStackItem() description
All checks were successful
DCO action / DCO (pull_request) Successful in 2m38s
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m26s
Vulncheck / Vulncheck (pull_request) Successful in 3m44s
Build / Build Components (1.21) (pull_request) Successful in 6m6s
Build / Build Components (1.22) (pull_request) Successful in 6m1s
Tests and linters / gopls check (pull_request) Successful in 7m47s
Tests and linters / Staticcheck (pull_request) Successful in 8m23s
Tests and linters / Lint (pull_request) Successful in 8m53s
Tests and linters / Tests (1.21) (pull_request) Successful in 11m5s
Tests and linters / Tests with -race (pull_request) Successful in 11m10s
Tests and linters / Tests (1.22) (pull_request) Successful in 11m36s
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-07-15 16:24:58 +03:00
d02a7c2d38 [#1196] morph/client: Remove duplicate utility functions
* We used several utility functions to parse frostfsid client
  subject and extended subject. However, following the changes
  in TrueCloudLab/frostfs-contract#97, these utility functions
  have become public. So there is no more need to have them here.

* There was a mismatch of slice parameter required length between
  frostfs-node's and frostfs-contract's utility functions,
  `checkStackItem()` solves this problem.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-07-15 16:24:48 +03:00
3bf6e6dde6 [#1238] engine: Add non-blocking send in the shard's notification channel
All checks were successful
DCO action / DCO (pull_request) Successful in 17m26s
Build / Build Components (1.21) (pull_request) Successful in 19m19s
Vulncheck / Vulncheck (pull_request) Successful in 19m46s
Build / Build Components (1.22) (pull_request) Successful in 23m58s
Pre-commit hooks / Pre-commit (pull_request) Successful in 32m59s
Tests and linters / Staticcheck (pull_request) Successful in 5m9s
Tests and linters / Lint (pull_request) Successful in 5m47s
Tests and linters / gopls check (pull_request) Successful in 5m22s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m37s
Tests and linters / Tests with -race (pull_request) Successful in 8m38s
Tests and linters / Tests (1.22) (pull_request) Successful in 8m56s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-07-10 11:37:11 +03:00
b027a7f91e [#1234] pilorama: Fix GetByPath() on duplicate directories
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-10 06:11:38 +00:00
6ace2f597e [#1234] pilorama: Add test for duplicate directory behaviour
When AddByPath() is called concurrently on 2 different nodes,
internal path components may be created twice. This violates some
of our assumptions in GetByPath() and, indirectly, in S3 handling of
GetSubTree() results.

Add a test for the correct behaviour, fixes will follow.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-10 06:11:38 +00:00
84ecd61dfd [#1233] putSvc: Try to put EC chunk to any node
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-09 07:54:29 +00:00
d90aab5454 [#1229] util: Fix session token expiration check
* Make session token expired at `current_epoch + 1` but
  not at `current_epoch` when it's still valid.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-08 08:15:56 +00:00
0c2b6f3dac [#1216] ape: Make services use bearer chains fed router
* Refactor object and tree service - they should instantiate
  chain router cheking the bearer token. If there are no bearer
  token rules, then defaul chain router is used.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-05 18:26:48 +00:00
47bcd346d3 [#1216] ape: Introduce BearerChainFeedRouter
* Unlike default chain router, `BearerChainFedRouter` performs checks for
  overrides defined in the bearer token;
* Add unit-test for the introduced router.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-05 18:26:48 +00:00
8eb591d668 [#1231] policer: Add EC node-off unit test
Some checks failed
Build / Build Components (1.21) (pull_request) Successful in 10m29s
Build / Build Components (1.22) (pull_request) Successful in 10m24s
DCO action / DCO (pull_request) Successful in 10m29s
Vulncheck / Vulncheck (pull_request) Successful in 12m18s
Tests and linters / gopls check (pull_request) Successful in 21m40s
Tests and linters / Tests (1.21) (pull_request) Failing after 22m2s
Pre-commit hooks / Pre-commit (pull_request) Successful in 23m40s
Tests and linters / Staticcheck (pull_request) Successful in 24m10s
Tests and linters / Tests (1.22) (pull_request) Successful in 24m19s
Tests and linters / Lint (pull_request) Successful in 25m30s
Tests and linters / Tests with -race (pull_request) Successful in 25m15s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-05 12:11:36 +03:00
cfd5e3d403 [#1227] morph/event: Release ants.Pool on listener stopping
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
2024-07-04 10:55:05 +00:00
62cbb72a5e [#1226] blobovniczatree: Delete fix db extensions in Init()
Some checks failed
DCO action / DCO (pull_request) Successful in 14m27s
Vulncheck / Vulncheck (pull_request) Successful in 16m45s
Build / Build Components (1.21) (pull_request) Successful in 18m53s
Build / Build Components (1.22) (pull_request) Successful in 20m30s
Tests and linters / gopls check (pull_request) Successful in 25m12s
Tests and linters / Tests (1.21) (pull_request) Failing after 27m52s
Tests and linters / Tests with -race (pull_request) Failing after 27m46s
Tests and linters / Tests (1.22) (pull_request) Failing after 28m13s
Tests and linters / Lint (pull_request) Successful in 30m25s
Tests and linters / Staticcheck (pull_request) Successful in 30m23s
Pre-commit hooks / Pre-commit (pull_request) Successful in 31m27s
Since several releases have been released, this code is not relevant.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-04 12:22:06 +03:00
78b1d9b18d [#1226] blobovniczatree: Drop leaf width limitation
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-04 12:22:06 +03:00
40c9ddb6ba [#1226] blobovniczatree: Drop init in advance option
To make blobovniczatree unlimited.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-04 12:22:06 +03:00
3a797e4682 [#1222] engine: Fix tree evacuation
All checks were successful
DCO action / DCO (pull_request) Successful in 13m4s
Vulncheck / Vulncheck (pull_request) Successful in 15m32s
Build / Build Components (1.22) (pull_request) Successful in 16m30s
Build / Build Components (1.21) (pull_request) Successful in 16m37s
Tests and linters / gopls check (pull_request) Successful in 20m33s
Tests and linters / Staticcheck (pull_request) Successful in 21m17s
Tests and linters / Lint (pull_request) Successful in 21m37s
Tests and linters / Tests with -race (pull_request) Successful in 26m40s
Tests and linters / Tests (1.21) (pull_request) Successful in 26m54s
Tests and linters / Tests (1.22) (pull_request) Successful in 27m59s
Pre-commit hooks / Pre-commit (pull_request) Successful in 39m39s
Do not fail if it is unable to evacuate tree to other node.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-04 10:38:10 +03:00
2bac82cd6f [#1222] engine: Fix object evacuation
Do not fail evacuation if it unable to evacuate object to other node.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-04 10:38:10 +03:00
bbe95dac8b [#1225] engine: Log the error when check object existence
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-07-04 07:22:54 +00:00
80d7459560 Revert "[#1196] morph/client: Remove duplicate utility functions"
Some checks failed
Tests and linters / Tests (1.21) (pull_request) Failing after 27s
DCO action / DCO (pull_request) Successful in 3m38s
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m16s
Vulncheck / Vulncheck (pull_request) Failing after 3m43s
Build / Build Components (1.22) (pull_request) Successful in 4m45s
Build / Build Components (1.21) (pull_request) Successful in 4m54s
Tests and linters / Staticcheck (pull_request) Successful in 5m10s
Tests and linters / gopls check (pull_request) Successful in 5m15s
Tests and linters / Lint (pull_request) Successful in 5m53s
Tests and linters / Tests with -race (pull_request) Successful in 9m12s
Tests and linters / Tests (1.22) (pull_request) Successful in 9m43s
This reverts commit 259007540f.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-03 15:44:35 +03:00
4bd4667791 [#1218] tree: Fix bearer token validation
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-03 07:22:11 +00:00
f3a861806e [#1218] object: Fix bearer token validation
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-03 07:22:11 +00:00
a378ff9cf6 [#1218] object: Pass container owner for backward get method check
* `getStreamBasicChecker` must define `containerOwner` for backward checks,
  otherwise bearer token cannot be validated for the token issuer.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-03 07:22:11 +00:00
91bed3b0ba [#1219] Remove Container.SetEACL method
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-02 13:05:40 +00:00
259007540f [#1196] morph/client: Remove duplicate utility functions
All checks were successful
Build / Build Components (1.21) (pull_request) Successful in 4m1s
DCO action / DCO (pull_request) Successful in 4m3s
Vulncheck / Vulncheck (pull_request) Successful in 3m51s
Pre-commit hooks / Pre-commit (pull_request) Successful in 5m15s
Tests and linters / Lint (pull_request) Successful in 6m59s
Build / Build Components (1.22) (pull_request) Successful in 8m54s
Tests and linters / gopls check (pull_request) Successful in 10m46s
Tests and linters / Staticcheck (pull_request) Successful in 11m30s
Tests and linters / Tests (1.21) (pull_request) Successful in 13m37s
Tests and linters / Tests (1.22) (pull_request) Successful in 13m43s
Tests and linters / Tests with -race (pull_request) Successful in 13m44s
We used several utility functions to parse frostfsid client
subject and extended subject. However, following the changes
in TrueCloudLab/frostfs-contract#97, these utility functions
have become public. So there is no more need to have them here.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-07-02 15:19:59 +03:00
56eeb630b6 [#1217] Fix grammar mistakes and misspelling
All checks were successful
DCO action / DCO (pull_request) Successful in 13m23s
Tests and linters / gopls check (pull_request) Successful in 16m29s
Tests and linters / Lint (pull_request) Successful in 17m2s
Build / Build Components (1.21) (pull_request) Successful in 16m26s
Build / Build Components (1.22) (pull_request) Successful in 16m21s
Vulncheck / Vulncheck (pull_request) Successful in 16m40s
Tests and linters / Staticcheck (pull_request) Successful in 19m9s
Pre-commit hooks / Pre-commit (pull_request) Successful in 21m37s
Tests and linters / Tests with -race (pull_request) Successful in 22m33s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m46s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m51s
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-07-01 19:14:25 +03:00
7a8ac4907a [#1213] engine: Drop unused
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-01 06:49:35 +00:00
dc2867682f [#1213] deleteSvc: Do not allow to delete EC chunks
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-01 06:49:35 +00:00
dc6778f385 [#1213] fmt: Drop unused interfaces
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-01 06:49:35 +00:00
7085723c6b [#1074] pilorama: Allow empty filenames in SortedByFilename()
All checks were successful
DCO action / DCO (pull_request) Successful in 5m38s
Vulncheck / Vulncheck (pull_request) Successful in 5m57s
Tests and linters / gopls check (pull_request) Successful in 6m57s
Build / Build Components (1.22) (pull_request) Successful in 6m48s
Build / Build Components (1.21) (pull_request) Successful in 6m52s
Tests and linters / Lint (pull_request) Successful in 8m26s
Tests and linters / Staticcheck (pull_request) Successful in 8m48s
Tests and linters / Tests (1.22) (pull_request) Successful in 11m39s
Tests and linters / Tests (1.21) (pull_request) Successful in 11m47s
Tests and linters / Tests with -race (pull_request) Successful in 13m49s
Pre-commit hooks / Pre-commit (pull_request) Successful in 17m24s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-28 17:46:24 +03:00
4c7ff159ec [#1201] writecache: Cancel background flush without lock
All checks were successful
DCO action / DCO (pull_request) Successful in 3m14s
Vulncheck / Vulncheck (pull_request) Successful in 5m14s
Tests and linters / gopls check (pull_request) Successful in 6m26s
Tests and linters / Lint (pull_request) Successful in 7m29s
Tests and linters / Staticcheck (pull_request) Successful in 7m22s
Build / Build Components (1.22) (pull_request) Successful in 7m55s
Build / Build Components (1.21) (pull_request) Successful in 8m1s
Pre-commit hooks / Pre-commit (pull_request) Successful in 12m8s
Tests and linters / Tests with -race (pull_request) Successful in 12m41s
Tests and linters / Tests (1.21) (pull_request) Successful in 13m3s
Tests and linters / Tests (1.22) (pull_request) Successful in 13m32s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-28 09:02:36 +03:00
4f7d76c9ef [#1206] audit: Drop not required events
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-27 10:54:31 +00:00
4951babd5f [#1208] blobstor: Fix delete without storage id
All checks were successful
DCO action / DCO (pull_request) Successful in 7m14s
Vulncheck / Vulncheck (pull_request) Successful in 15m56s
Build / Build Components (1.22) (pull_request) Successful in 17m0s
Build / Build Components (1.21) (pull_request) Successful in 17m5s
Tests and linters / gopls check (pull_request) Successful in 18m17s
Tests and linters / Lint (pull_request) Successful in 21m1s
Tests and linters / Staticcheck (pull_request) Successful in 20m41s
Pre-commit hooks / Pre-commit (pull_request) Successful in 22m18s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m0s
Tests and linters / Tests (1.22) (pull_request) Successful in 2m55s
Tests and linters / Tests with -race (pull_request) Successful in 3m42s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-27 11:59:08 +03:00
81ea91de52 [#451] metrics: Move to internal
`metrics` don't look like something others want to import.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-25 08:52:37 +00:00
11a38a0a84 [#1190] tree: GroupIDs must also be target of APE checks
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-06-25 08:49:20 +00:00
0b87388c18 [#1190] object: GroupIDs must also be target of APE checks
* Also add new test case for ape middleware in container service.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-06-25 08:49:20 +00:00
621dbf58ab [#1190] container: GroupIDs must also be target of APE checks
* Also add new test case for ape middleware in container service.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-06-25 08:49:20 +00:00
a1f7615b7e [#1190] ape: Introduce Groups util function to retrieve actor's groupIDs
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-06-25 08:49:20 +00:00
a83eeddb1d [#60] control: Add GetNetmapStatus method
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-20 16:28:42 +03:00
9ac74efc41 [#1173] shard: Use mode from config on reload
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-06-20 11:29:10 +00:00
40b68bcb6c [#1109] object: Validate attribute EXPIRATION_EPOCH on put
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-06-19 17:19:27 +00:00
fd28461def [#1184] ir: Add grpc middleware for control service
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 3m14s
DCO action / DCO (pull_request) Successful in 3m46s
Build / Build Components (1.21) (pull_request) Successful in 4m20s
Build / Build Components (1.22) (pull_request) Successful in 4m21s
Tests and linters / gopls check (pull_request) Successful in 4m45s
Tests and linters / Lint (pull_request) Successful in 5m16s
Tests and linters / Staticcheck (pull_request) Successful in 6m24s
Pre-commit hooks / Pre-commit (pull_request) Successful in 8m54s
Tests and linters / Tests (1.21) (pull_request) Successful in 10m57s
Tests and linters / Tests with -race (pull_request) Successful in 10m52s
Tests and linters / Tests (1.22) (pull_request) Successful in 11m20s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-19 16:05:53 +03:00
ecd1ed7a5e [#1184] node: Add audit middleware for grpc services
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-19 16:05:53 +03:00
75eedf71f3 [#1187] pilorama/test: Remove debug print
Some checks failed
DCO action / DCO (pull_request) Successful in 19m36s
Vulncheck / Vulncheck (pull_request) Successful in 30m13s
Tests and linters / gopls check (pull_request) Successful in 33m40s
Build / Build Components (1.22) (pull_request) Successful in 34m44s
Tests and linters / Lint (pull_request) Successful in 37m6s
Tests and linters / Tests (1.22) (pull_request) Failing after 40m59s
Tests and linters / Tests (1.21) (pull_request) Failing after 41m56s
Tests and linters / Tests with -race (pull_request) Failing after 45m4s
Build / Build Components (1.21) (pull_request) Successful in 44m45s
Tests and linters / Staticcheck (pull_request) Successful in 15m42s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1h0m46s
Introduced in e12fcc041d.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-18 15:09:44 +03:00
5b100699d7 [#566] policer: Move isClientErrMaintenance to frostfs-sdk-go
All checks were successful
DCO action / DCO (pull_request) Successful in 1m5s
Vulncheck / Vulncheck (pull_request) Successful in 4m8s
Pre-commit hooks / Pre-commit (pull_request) Successful in 5m5s
Build / Build Components (1.21) (pull_request) Successful in 4m47s
Build / Build Components (1.22) (pull_request) Successful in 4m44s
Tests and linters / Lint (pull_request) Successful in 6m13s
Tests and linters / Staticcheck (pull_request) Successful in 6m3s
Tests and linters / gopls check (pull_request) Successful in 6m16s
Tests and linters / Tests (1.21) (pull_request) Successful in 9m19s
Tests and linters / Tests with -race (pull_request) Successful in 9m15s
Tests and linters / Tests (1.22) (pull_request) Successful in 9m22s
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-06-18 10:20:45 +03:00
76cf7a051b [#1178] shard: Check metabase existence before read shard ID
All checks were successful
DCO action / DCO (pull_request) Successful in 6m15s
Build / Build Components (1.22) (pull_request) Successful in 11m1s
Build / Build Components (1.21) (pull_request) Successful in 11m6s
Tests and linters / Staticcheck (pull_request) Successful in 16m15s
Tests and linters / gopls check (pull_request) Successful in 16m11s
Tests and linters / Lint (pull_request) Successful in 17m34s
Vulncheck / Vulncheck (pull_request) Successful in 7m47s
Pre-commit hooks / Pre-commit (pull_request) Successful in 21m25s
Tests and linters / Tests (1.21) (pull_request) Successful in 21m22s
Tests and linters / Tests with -race (pull_request) Successful in 21m51s
Tests and linters / Tests (1.22) (pull_request) Successful in 2m46s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-17 09:59:15 +03:00
96fe271bab [#1170] innerring: Support morph mTLS
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-14 09:28:32 +03:00
42ecc2f2b9 [#1170] morph: Support mTLS
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-14 09:26:27 +03:00
68ac490729 [#1174] shard: Update metric mode_info on Init
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-06-13 08:32:59 +00:00
6a39c3d15e [#1086] engine: Do not use metabase if shard looks bad
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-13 07:35:22 +00:00
9d73f9c2c6 Reapply "[#446] engine: Move to read-only on blobstor errors"
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-13 07:35:22 +00:00
b9fcaad21f [#1168] shard: Set Disabled as default mode for components
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-06-11 15:13:38 +00:00
6cf512e574 [#1166] blobovniczatree: Handle blobovnicza's NoSpaceLeft error
All checks were successful
DCO action / DCO (pull_request) Successful in 10m57s
Vulncheck / Vulncheck (pull_request) Successful in 11m50s
Build / Build Components (1.21) (pull_request) Successful in 14m43s
Build / Build Components (1.22) (pull_request) Successful in 16m38s
Tests and linters / gopls check (pull_request) Successful in 18m17s
Tests and linters / Staticcheck (pull_request) Successful in 23m58s
Tests and linters / Lint (pull_request) Successful in 25m50s
Pre-commit hooks / Pre-commit (pull_request) Successful in 27m13s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m5s
Tests and linters / Tests (1.22) (pull_request) Successful in 4m3s
Tests and linters / Tests with -race (pull_request) Successful in 4m23s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-07 17:15:43 +03:00
e7d479f4c2 [#1166] blobovnicza: Return NoSpaceLeft error instead of syscall.ENOSPC
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-07 17:15:43 +03:00
239323eeef [#1157] tree: Make tree service use Bearer token's APE overrides
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-06-07 12:11:11 +00:00
04a3f891fd [#1157] object: Make APE checker use Bearer-token's APE overrides
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-06-07 12:11:11 +00:00
4edff5eea6 [#1157] ape: Introduce single-run chain router
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-06-07 12:11:11 +00:00
a90310335d [#1156] ape: Return not found when removing local overrides
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-06-07 12:10:57 +00:00
a849236b68 [#1161] node: Remove notification functionality
It is unused and will be reworked in future.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-07 12:10:51 +00:00
3f1961157e [#1163] metabase: Handle multiple splitInfos for EC
All checks were successful
DCO action / DCO (pull_request) Successful in 3m5s
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m41s
Vulncheck / Vulncheck (pull_request) Successful in 4m10s
Build / Build Components (1.21) (pull_request) Successful in 5m49s
Build / Build Components (1.22) (pull_request) Successful in 5m46s
Tests and linters / Staticcheck (pull_request) Successful in 6m37s
Tests and linters / Lint (pull_request) Successful in 7m12s
Tests and linters / gopls check (pull_request) Successful in 7m16s
Tests and linters / Tests (1.21) (pull_request) Successful in 10m11s
Tests and linters / Tests (1.22) (pull_request) Successful in 10m20s
Tests and linters / Tests with -race (pull_request) Successful in 10m21s
For REP updating split info is handled explicitly by a high-level PUT logic.
For EC it is trickier, because the address of an object we put is only
distantly related to a split info.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-06 16:26:29 +03:00
2e074d3846 [#1163] metabase: Properly save EC parent split ID
Search by SplitID should return all parts of a complex object.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-05 12:40:16 +03:00
806236da78 [#1121] node: Change mode of shard components
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-06-05 05:55:24 +00:00
6f2187a420 [#1121] node: Refactor mods of shard
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-06-05 05:55:24 +00:00
cc2449beaf [#1158] metabase: Fix EC storage schema
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 3m5s
Build / Build Components (1.21) (pull_request) Successful in 4m0s
Build / Build Components (1.22) (pull_request) Successful in 3m56s
Tests and linters / Staticcheck (pull_request) Successful in 4m54s
Tests and linters / gopls check (pull_request) Successful in 4m55s
Tests and linters / Lint (pull_request) Successful in 5m54s
Pre-commit hooks / Pre-commit (pull_request) Successful in 6m29s
DCO action / DCO (pull_request) Successful in 3m34s
Tests and linters / Tests (1.22) (pull_request) Successful in 7m28s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m50s
Tests and linters / Tests with -race (pull_request) Successful in 9m25s
Do not store EC info twice.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-04 17:24:40 +03:00
5aacb8fc86 [#1144] metabase: Save parent attributes for ec-chunks
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 4m15s
DCO action / DCO (pull_request) Successful in 4m4s
Build / Build Components (1.22) (pull_request) Successful in 5m17s
Build / Build Components (1.21) (pull_request) Successful in 5m22s
Tests and linters / Lint (pull_request) Successful in 7m6s
Tests and linters / Staticcheck (pull_request) Successful in 7m25s
Tests and linters / gopls check (pull_request) Successful in 7m19s
Pre-commit hooks / Pre-commit (pull_request) Successful in 9m39s
Tests and linters / Tests (1.22) (pull_request) Successful in 10m36s
Tests and linters / Tests (1.21) (pull_request) Successful in 10m50s
Tests and linters / Tests with -race (pull_request) Successful in 10m51s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 19:55:32 +03:00
f8e33f8e3a [#1144] metabase: Proprely choose root OID for EC-splitted objects
* If EC-parent is a part of Split itself, then save to root bucket
  its parent;
* If EC-parent is not a part of Split itself, then save to root bucket
  OID of this EC-parent.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 19:53:32 +03:00
f0edebea18 [#1144] metabase: Support ec parent filter for Search
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 19:53:32 +03:00
0b367007fc [#1152] go.mod: Update api-go and sdk versions
All checks were successful
DCO action / DCO (pull_request) Successful in 5m23s
Build / Build Components (1.22) (pull_request) Successful in 7m33s
Build / Build Components (1.21) (pull_request) Successful in 7m43s
Tests and linters / Lint (pull_request) Successful in 8m51s
Tests and linters / Tests (1.22) (pull_request) Successful in 14m33s
Pre-commit hooks / Pre-commit (pull_request) Successful in 15m48s
Vulncheck / Vulncheck (pull_request) Successful in 15m12s
Tests and linters / Tests (1.21) (pull_request) Successful in 16m38s
Tests and linters / gopls check (pull_request) Successful in 19m24s
Tests and linters / Staticcheck (pull_request) Successful in 21m29s
Tests and linters / Tests with -race (pull_request) Successful in 23m16s
* Resolve conflicts for apemanager since api-go
  contains ape and apemanager packages and SDK only
  ape package.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 15:39:09 +03:00
92e19feb57 [#1147] node: Use public fields for shard.ExistsPrm
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
c1af13b47e [#1147] node: Fix issue from gopls
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
6130650bb6 [#1147] node: Implement Lock\Delete requests for EC object
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
a82c8cc5b8 [#1147] gc: Execute callback for expired tombstones when they exists
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
d355274cd0 [#1147] object: Use methods on pointer for searchsvc.execCtx
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
3555c73225 [#1147] object: Use methods on pointer for deletesvc.execCtx
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
e43e7bec3a [#1147] log: Remove redundant address field from log
Filled when logger created for `request` object from package `getsvc`.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
50923ed81c [#1147] Fix gofumpt issue
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
4a34d0d40e [#1149] go.mod: Bump neo-go up to v0.106.0
Required to work with neo-go v0.106.0 node
with default hardfork configuration. Without
neo-go client version bump, it throws error.

  failed to get network magic: unexpected hardfork: Cockatrice

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2024-05-30 08:11:58 +00:00
3627b44e92 [#1142] tree: Fill APE-request with source IP property
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-27 10:17:17 +00:00
482c5129ac [#1142] object: Fill APE-request with source IP property
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-27 10:17:17 +00:00
43625e7536 [#1142] container: Fill APE-request property with source IP
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-27 10:17:17 +00:00
542d3adcb2 [#1105] apemanager: Implement apemanager service
* Introduce grpc server for apemanager service and
  its implementation in `pkg/services/apemanager`.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-27 09:34:21 +00:00
51ade979e8 [#1105] ape: Introduce contract storage with proxy contract verification
* `ProxyVerificationContractStorage` uses Proxy contract as a cosigner.
* `ProxyVerificationContractStorage` recreates a contract storage for each handler
  invocation because of an issue: rpc-actor from morph client may be expired. This
  way won't create a bottlenecks because it is expected that this contract storage
  implementation will be used not so often.
* Make morph client return `RPCActor` (that is websocket client in fact).
* Make `SwitchRPCGuardedActor` return `RPCActor` as it will be used for
  `ProxyVerificationContractStorage`.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-27 09:34:21 +00:00
40b04c00ef [#1141] metabase: Fix IsUserObject method
All checks were successful
DCO action / DCO (pull_request) Successful in 6m57s
Build / Build Components (1.22) (pull_request) Successful in 10m4s
Build / Build Components (1.21) (pull_request) Successful in 10m12s
Vulncheck / Vulncheck (pull_request) Successful in 10m4s
Tests and linters / gopls check (pull_request) Successful in 13m13s
Tests and linters / Staticcheck (pull_request) Successful in 13m45s
Tests and linters / Lint (pull_request) Successful in 14m21s
Tests and linters / Tests (1.21) (pull_request) Successful in 16m3s
Pre-commit hooks / Pre-commit (pull_request) Successful in 16m26s
Tests and linters / Tests (1.22) (pull_request) Successful in 16m27s
Tests and linters / Tests with -race (pull_request) Successful in 16m29s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-20 13:24:15 +03:00