Commit graph

5175 commits

Author SHA1 Message Date
74e11b3cf3
[] objsvc: Allow to send search response in multiple messages
Previously, `ln` was only set once, so search has really worked for
small number of objects.

Fix panic:
```
panic: runtime error: slice bounds out of range [:43690] with capacity 21238
goroutine 6859775 [running]:
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object.(*searchStreamMsgSizeCtrl).Send(0xc001eec8d0, 0xc005734000)
        git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/transport_splitter.go:173 +0x1f0
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/v2.(*streamWriter).WriteIDs(0xc000520320, {0xc00eb1a000, 0x4fd9c, 0x7fd6475a9a68?})
        git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/v2/streamer.go:28 +0x155
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search.(*uniqueIDWriter).WriteIDs(0xc001386420, {0xc00eb1a000?, 0xc0013ea9c0?, 0x113eef3?})
        git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/util.go:62 +0x202
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search.(*execCtx).writeIDList(0xc00011aa38?, {0xc00eb1a000?, 0xc001eec9f0?, 0xc0008f4380?})
        git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/exec.go:68 +0x91
git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search.(*execCtx).executeLocal(0xc0008f4380, {0x176c538, 0xc001eec9f0})
        git.frostfs.info/TrueCloudLab/frostfs-node/pkg/services/object/search/local.go:18 +0x16b
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-28 09:23:54 +03:00
f527556735
Revert "[] metabase: Ensure Unmarshal() is called on a cloned slice"
This reverts commit 8ed7a676d5.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-26 16:25:13 +03:00
23b426ec4a
[] go.mod: Update sdk-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-26 15:37:21 +03:00
7e3f2fca47 [] tree: Wrap only ChainRouterError erros with ObjectAccessDenied
* Such wrapping helps to differentiate logical check errors and server internal
  errors.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-12-16 16:19:04 +03:00
25ed595354 [] object: Wrap only ChainRouterError erros with ObjectAccessDenied
* Such wrapping helps to differentiate logical check errors and server internal
  errors.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-12-16 16:18:58 +03:00
fb2f40aea1 [] ape: Introduce ChainRouterError error type
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-12-16 16:18:51 +03:00
15b4288d80
[] metabase: Check if EC parent is removed or expired
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-12-11 10:42:12 +03:00
6ea8f2b23c
[] engine: Rename parent -> ecParent
Parent could mean split parent or EC parent. In this case it is EC parent only.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-12-11 10:41:32 +03:00
47f12f8440
[] policer: Do not replicate EC chunk if object already removed
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-12-11 10:41:00 +03:00
edc1824c23
[] go.mod: Update sdk-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-06 12:50:39 +03:00
cf48dfd55e [] tree: Make check APE error get wrapped to api status
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-12-04 09:57:46 +03:00
274ac61236 [] ape: Make APE checker return error without status
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-12-04 09:57:41 +03:00
892542d6e3 [] node: Allow to omit metabase.path if shard is disabled
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-12-03 13:11:16 +00:00
af3d6368b0 [] cli: Keep order for required nodes in the result of object nodes
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-12-03 11:45:52 +03:00
36fe470956 [] node: Keep order for equal elements when sort priority metrics
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-12-03 11:45:38 +03:00
7eac5fb18b
Release v0.44.0
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-25 14:41:36 +03:00
0e5524dac7 [] adm: Print address in base58 format in morph ape get-admin
Signed-off-by: George Bartolomey <george@bh4.ru>
2024-11-25 10:38:05 +00:00
3ebd560f42 [] cli: Make descriptive help for--rule option
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-25 07:21:05 +00:00
1ed7ab75fb [] cli: Print the reason of ape manager error
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-22 15:24:32 +03:00
99f9e59de9 [] adm: Remove --alphabet-wallets flag from readonly commands
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-21 14:37:34 +00:00
256f96e252 [] adm/nns: Rename getRPCClient() to nnsWriter()
Make it more specific and similar to nnsReader().

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-21 14:37:34 +00:00
e5ea95c045 [] adm/nns: Do not return hash from getRPCClient()
It was unused and we employ better abstractions now.
gopatch:
```
@@
var a, b expression
@@
-a, b, _ := getRPCClient(...)
+a, b := getRPCClient(...)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-21 14:37:34 +00:00
9073e555db [] adm/nns: Do not create actor for readonly commands
`nns get-records` and `nns tokens` command do not need to sign anything,
so remove useless actor and use invoker directly.
`NewLocalActor()` is only used in `ape` and `nns` packages. `ape`
package seem to use it correctly, only when alphabet wallets are
provided, so no changes there.
Also, remove --alphabet-wallets flag from commands that do not need it.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-21 14:37:34 +00:00
2771fdb8c7 [] adm/nns: Use nns.GetAllRecords() wrapper
It was not possible previously, because GetAllRecords() was not declared
safe in frostfs-contract.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-21 14:37:34 +00:00
efa4ce00b8 [] go.mod: Update frostfs-contract to v0.21.0-rc.3
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-21 14:37:34 +00:00
f12f04199e [] traverser: Check for placement vector out of range
Placement vector may contain fewer nodes count than it required by policy
due to the outage of the one of the node.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-11-21 14:18:55 +03:00
2e2c62147d
[] adm: Move ProtoConfigPath from constants to commonflags package
Refs 

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-21 09:39:06 +03:00
6ae8667fb4
[] .forgejo: Run actions on push to master
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-20 11:42:12 +03:00
49a4e727fd [] timer/test: Use const for constants
Make it easy to see what the test is about.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-20 08:36:25 +00:00
2e974f734c [] timer/test: Improve test coverage
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-20 08:36:25 +00:00
3042490340 [] timer: Remove unused OnDelta() method
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-20 08:36:25 +00:00
a339b52a60 [] adm: Refactor APE-chains managing subcommands
* Use `cmd/internal/common/ape` parser commands within `ape`
  subcommands
* Use flag names from `cmd/internal/common/ape

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-20 07:58:32 +00:00
4ab4ed6f96 [] cli: Refactor bearer subcommand
* Use `cmd/internal/common/ape` parser commands within `generate-ape-override`
  subcommand
* Use flag names from `cmd/internal/common/ape`

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-20 07:58:32 +00:00
3b1364e4cf [] cli: Refactor ape-manager subcommands
* Refactor ape-manager subcommands
* Use `cmd/internal/common/ape` parser commands within ape-manager subcommands
* Use flag names from `cmd/internal/common/ape`

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-20 07:58:32 +00:00
daff77b273 [] cli: Refactor local override managing subcommands
* Refactor local override managing subcommands
* Use `cmd/internal/common/ape` parser commands within local
  override subcommands
* Use flag names from `cmd/internal/common/ape`

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-20 07:58:32 +00:00
7a7ee71a4d [] cmd: Introduce common APE-chain parser commands
* Introduce common parsing commands to use them in `frostfs-cli`
  and `frostfs-adm` APE-related subcommands
* Introduce common flags for these parsing commands

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-20 07:58:32 +00:00
ffe9906266 [] cli: Move APE-chain parser methods to pkg/util
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-20 07:58:32 +00:00
ae31ef3602 [] cli: Move PrintHumanReadableAPEChain to a common package
* Both `frostfs-cli` and `frostfs-adm` APE-related subcommands use
  `PrintHumanReadableAPEChain` to print a parsed APE-chain. So, it's
  more correct to have it in a common package over `frostfs-cli` and
  `frostfs-adm` folders.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-20 07:58:32 +00:00
e2cb0640f1 [] util: Move eACL-to-APE converter to pkg/util
* `ConvertEACLToAPE` is useful method which couldn't be imported
  out of frostfs-node so far as it has been in `internal`
* Since `ConvertEACLToAPE` and related structures and unit-tests
  are placed in `pkg/util`

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-11-20 07:58:32 +00:00
9f4ce600ac
[] adm: Allow to manage additional keys in frostfsid
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-19 16:25:16 +03:00
d82f0d1926
[] node/control: Await until SetNetmapStatus() persists
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-15 16:36:07 +03:00
acd5babd86
[] morph: Merge InvokeRes and WaitParams
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-15 16:36:07 +03:00
b65874d1c3
[] morph: Return InvokeRes from all invoke*() methods
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-15 16:36:07 +03:00
69c63006da
[] morph: Move tx waiter to morph package
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-15 16:36:07 +03:00
d77a218f7c [] metabase: Merge Inhume() and DropGraves() for tombstones
DropGraves() is only used to drop gravemarks after a tombstone
removal. Thus, it makes sense to do Inhume() and DropGraves() in one
transaction. It has less overhead and no unexpected problems in case
of sudden power failure.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-14 06:47:04 +00:00
44df67492f [] metabase: Split inhumeTx() into 2 functions
No functional changes.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-14 06:47:04 +00:00
1e6f132b4e [] metabase: Pass InhumePrm by value
Unify with the other code, no functional changes.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-14 06:47:04 +00:00
6dc0dc6691 [] shard: Take mode mutex in HandleExpiredTombstones()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-11-14 06:47:04 +00:00
f7cb6b4d87 [] Makefile: Update golangci-lint
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-11-13 15:01:41 +00:00
7fc6101bec
[] engine/test: Rework engine test utils
- Remove `testNewShard` and `setInitializedShards` because they
violated the default engine workflow. The correct workflow is:
first use `New()`, followed by `Open()`, and then `Init()`. As a
result, adding new logic to `(*StorageEngine).Init` caused several
tests to fail with a panic when attempting to access uninitialized
resources. Now, all engines created with the test utils must be
initialized manually. The new helper method `prepare` can be used
for that purpose.
- Additionally, `setInitializedShards` hardcoded the shard worker
pool size, which prevented it from being configured in tests and
benchmarks. This has been fixed as well.
- Ensure engine initialization is done wherever it was missing.
- Refactor `setShardsNumOpts`, `setShardsNumAdditionalOpts`, and
`setShardsNum`. Make them all depend on `setShardsNumOpts`.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-11-13 14:42:53 +03:00