Commit graph

3211 commits

Author SHA1 Message Date
2ca4d4ca6b [#9999] shard: Fix reload test
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:48 +03:00
3e00dc304c [#9999] metabase: Fix EC delete tests
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:47 +03:00
13234a6e4f [#9999] shard: Resolve linters
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:47 +03:00
343662773f [#9999] metabase: Fix SetMode after db engine change to pebble
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:47 +03:00
9bd273d2f9 [#9999] engine: Fix methods after db engine change
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:47 +03:00
88a8738259 [#9999] shard: Fix metabase usage after db engine change
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:47 +03:00
743819134c [#9999] metabase: Fix unit test after db engine change
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:47 +03:00
d0a5d96089 [#9999] metabase: Fix db engine to pebble in db.go and control.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:47 +03:00
c229b08f46 [#9999] metabase: Fix db engine to pebble in version.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:47 +03:00
3db4112366 [#9999] metabase: Fix db engine to pebble in shard_id.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:47 +03:00
c73bf08b72 [#9999] metabase: Fix db engine to pebble in select.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:47 +03:00
e5b9ad6465 [#9999] metabase: Fix db engine to pebble in put.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:47 +03:00
8bfa2fe106 [#9999] metabase: Fix db engine to pebble in storage_id.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:47 +03:00
4bac5644f1 [#9999] metabase: Fix db engine to pebble in lock.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:47 +03:00
b12c1f1b23 [#9999] metabase: Fix db engine to pebble in list.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:47 +03:00
06b0ad5b89 [#9999] metabase: Fix db engine to pebble in iterators.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:46 +03:00
691af2b2eb [#9999] metabase: Fix db engine to pebble in inhume.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:46 +03:00
6bb7330a46 [#9999] metabase: Fix db engine to pebble in graveyard.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:46 +03:00
b1d0b71be7 [#9999] metabase: Fix db engine to pebble in get.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:46 +03:00
1ff0f37267 [#9999] metabase: Fix db engine to pebble in expired.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:46 +03:00
956a64e06b [#9999] metabase: Fix db engine to pebble in delete.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:46 +03:00
eae6b4f7e8 [#9999] metabase: Fix db engine to pebble in counter.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:46 +03:00
b2bd6650ba [#9999] metabase: Fix db engine to pebble in containers.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:46 +03:00
5b33d84180 [#9999] metabase: Fix db engine to pebble in children.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:46 +03:00
b8599d9826 [#9999] metabase: Fix db engine to pebble in exists.go
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:46 +03:00
1aaf3346ae [#9999] metabase: Change DB engine to pebble
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-30 12:14:46 +03:00
7e04083c27 [#1278] containerSvc: Validate FrostFSID subject exitence on Put
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-29 16:01:37 +03:00
a12c39667d [#1278] ir: Do not allow to create container without FrostFSID record
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-29 16:01:37 +03:00
8377372a40 [#1276] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-26 16:44:19 +03:00
7fd7961dfa [#1271] getSvc: Fix local EC chunk get
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-24 08:52:16 +03:00
8398a8b4b3 [#1271] getSvc: Fix head --raw assemble for EC
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-24 08:52:08 +03:00
9c2c76ca32
[#1259] Move pkg/util/locode to frostfs-locode-db
Removed pkg/util/locode package, added
git.frostfs.info/TrueCloudLab/frostfs-locode-db/pkg/locode dependency.

Signed-off-by: George Bartolomey <george@bh4.ru>
2024-07-23 14:59:14 +03:00
9ee1bd3669
[#1265] *: Run gofumpt
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-07-23 10:36:32 +03:00
18182e578e [#1261] shard: Fix delete objects from FSTree
Replace nil storageID with empty like by shard.Get.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-22 14:38:09 +03:00
3119f2fbc3 [#1257] metabase: Delete EC gc marks and split info
EC parent and split gc marks should be deleted after last EC chunk delete.
Also should delete split info for EC parent and split parent.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-19 17:14:14 +03:00
eadcea8df0 [#1249] object: Remove all APE pre-checks in handlers
* Methods `Head`, `Get`, `GetRangeHash` should no longer use APE pre-checks
  as that leads only to incorrect rule chain processing for requests:
  1. Immediate return with `NoRuleFound` may be unexpected as some `Allow`
     rule is actually defined but can't be matched yet as it gets no object
     attributes;
  2. Immdediate return with `Allow` may be incorrect as some `Deny` rule
     is actually defined but can't bet matched yet as it gets no object
     attirbutes;
  3. Pre-check breaks compatibility for converted EACL-tables.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-18 13:52:43 +00:00
5e5ee545b6 [#1254] policer: Fix svacer warning
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-18 11:30:39 +00:00
27caa8a72f [#1256] metabase: Put split parent object ID for EC chunks
It is required to save split parent ID too, not only split ID.
Otherwise inhume operation works incorrect: shard with last part may be skipped
and parent object will be available.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-17 18:08:07 +03:00
e83d39e33f [#1253] deleteSvc: Use copy of common parameters
getSvc may change the values of some fields, so Head will affect Delete
or Put. In this case, the change is necessary so that the session token
is stored in the tombstone object (EC assemble calls `ForgetTokens`).

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-17 14:24:27 +03:00
fc383ea6ae [#1253] getSvc: Fix EC objects get
Now EC objects assembling is performed concurrently.
Also fixed issue with an error in case of getting
EC object via non-container node.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-17 14:24:27 +03:00
00a88b9936 [#1251] *: Run gofumpt
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-17 11:25:07 +03:00
3940bc17c1 [#1251] pilorama: Allow traversing multiple branches in parallel
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-17 11:25:07 +03:00
e5767c9002 [#1250] *: Reformat proto filets with clang-format
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-16 15:13:08 +00:00
286df198c9 [#1248] placement: Decouple ContainerNodes() cache from the placement builder.
Also, write tests.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-16 12:21:30 +00:00
3a48b282b6 [#1248] placement: Use epoch to track netmap versions
Previously we used pointer, this could have worked,
because most of the time, the netmap is cached.
This didn't work, however, because `lastNm` field was always nil.

Rework the mechanism completely:
1. Use epoch to track netmap versions, as it it simpler and
   is unrelated to the TTL of an underlying cache.
2. Fix a bug where the epoch could change while mutex was unlocked.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-16 12:21:30 +00:00
21431f22c0 [#1248] placement: Use cid.ID as key in the cache
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-16 12:21:30 +00:00
d5dc14c639 [#1243] object: Make APE checker set x-headers to request properties
* Update go.mod, go.sum;
* Add x-headers to request properties;
* Add a unit-test.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-16 07:28:42 +00:00
39866a957c [#1196] morph/client: Fix ArrayFromStackItem() description
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-07-15 16:24:58 +03:00
d02a7c2d38 [#1196] morph/client: Remove duplicate utility functions
* We used several utility functions to parse frostfsid client
  subject and extended subject. However, following the changes
  in TrueCloudLab/frostfs-contract#97, these utility functions
  have become public. So there is no more need to have them here.

* There was a mismatch of slice parameter required length between
  frostfs-node's and frostfs-contract's utility functions,
  `checkStackItem()` solves this problem.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-07-15 16:24:48 +03:00
3bf6e6dde6 [#1238] engine: Add non-blocking send in the shard's notification channel
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-07-10 11:37:11 +03:00