george.bartolomey/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Pull requests Activity
5227 commits 7 branches 53 tags 20 MiB
Commit graph

65 commits

Author SHA1 Message Date
Aleksey Savchuk
9cabca9dfe
[#1527] engine/test: Move default metabase options to separate function 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-12-02 16:13:37 +03:00
Aleksey Savchuk
60feed3b5f
[#1527] engine/test: Allow to specify current epoch in epochState 
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-12-02 15:37:25 +03:00
Aleksey Savchuk
7fc6101bec
[#1491] engine/test: Rework engine test utils 
- Remove `testNewShard` and `setInitializedShards` because they
violated the default engine workflow. The correct workflow is:
first use `New()`, followed by `Open()`, and then `Init()`. As a
result, adding new logic to `(*StorageEngine).Init` caused several
tests to fail with a panic when attempting to access uninitialized
resources. Now, all engines created with the test utils must be
initialized manually. The new helper method `prepare` can be used
for that purpose.
- Additionally, `setInitializedShards` hardcoded the shard worker
pool size, which prevented it from being configured in tests and
benchmarks. This has been fixed as well.
- Ensure engine initialization is done wherever it was missing.
- Refactor `setShardsNumOpts`, `setShardsNumAdditionalOpts`, and
`setShardsNum`. Make them all depend on `setShardsNumOpts`.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-11-13 14:42:53 +03:00
Aleksey Savchuk
7ef36749d0
[#1491] engine/test: Move BenchmarkExists to exists_test.go 
Move `BenchmarkExists` from `engine_test.go` to `exists_test.go`
for better organization and clarity.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-11-13 14:09:29 +03:00
Aleksey Savchuk
c6066d6ee4
[#1491] engine/test: Use more suitable testing utils here and there 
Use `setShardsNum` instead of `setInitializedShards` wherever possible.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
 2024-11-13 14:09:29 +03:00
Dmitrii Stepanov
1b520f7973
[#1412] engine: Add IsIndexedContainer flag 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-10-08 11:41:49 +03:00
Ekaterina Lebedeva
a685fcdc96 [#1317] go.mod: Use range over int 
Since Go 1.22 a "for" statement with a "range" clause is able
to iterate through integer values from zero to an upper limit.

gopatch script:
@@
var i, e expression
@@
-for i := 0; i <= e - 1; i++ {
+for i := range e {
    ...
}

@@
var i, e expression
@@
-for i := 0; i <= e; i++ {
+for i := range e + 1 {
    ...
}

@@
var i, e expression
@@
-for i := 0; i < e; i++ {
+for i := range e {
    ...
}

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-09-03 13:00:54 +03:00
Anton Nikiforov
92e19feb57 [#1147] node: Use public fields for shard.ExistsPrm 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
6130650bb6 [#1147] node: Implement Lock\Delete requests for EC object 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
411a8d0245 [#1004] blobovnicza: Use TTL for blobovnicza tree cache 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-26 19:54:29 +03:00
Dmitrii Stepanov
942d83611b [#874] engine: Revert Check object existance concurrently 
This reverts commit f526f49995.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-04-01 08:42:34 +00:00
Dmitrii Stepanov
d6534fd755 [#1016] frostfs-node: Fix gopls issues 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-03-01 12:13:43 +03:00
Dmitrii Stepanov
f526f49995 [#874] engine: Check object existance concurrently 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-23 09:28:29 +03:00
Dmitrii Stepanov
a2ab373a0a [#895] metabase: Do not delete GC mark for virtual objects 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Dmitrii Stepanov
7166e77c2b [#895] test: Add logger to test shard 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Dmitrii Stepanov
47dcfa20f3 [#895] test: Use t.Cleanup only for external resources 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Dmitrii Stepanov
f1b2b8bffa [#895] test: Fix NewLogger arguments list 
`debug` is always true.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-11 12:32:09 +00:00
Dmitrii Stepanov
79088baa06 [#772] node: Apply gofumpt 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-31 17:03:03 +03:00
Ekaterina Lebedeva
8a81af5a3b [#653] Add context parameter to Open functions 
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2023-09-07 18:03:29 +03:00
Evgenii Stratonikov
96e690883f [#638] Unify test loggers 
In some places we have debug=false, in others debug=true.
Let's be consistent.

Semantic patch:
```
@@
@@
-test.NewLogger(..., false)
+test.NewLogger(..., true)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-23 11:21:05 +00:00
Evgenii Stratonikov
322c1dc273 [#638] Use test.NewLogger() in tests 
Semantic patch (restricted to **/*_test.go):
```
@@
@@
+import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger/test"
-import "go.uber.org/zap"
-import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger"

-&logger.Logger{Logger: zap.L()}
+test.NewLogger(t, false)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-23 11:21:05 +00:00
Evgenii Stratonikov
f7042c5a6f [#609] Replace zaptest.NewLogger() with zap.L() 
Semantic patch:
```
@@
@@
-import "go.uber.org/zap/zaptest"
+import "go.uber.org/zap"

-zaptest.NewLogger(t)
+zap.L()
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-17 16:51:10 +00:00
Alejandro Lopez
ae8be495c8 [#xx] Avoid manual management of files in tests 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-14 14:01:39 +03:00
Alexander Chuprov
033eaf77e1 [#496] node: Fix linter importas 
Standardize the alias of the
import frostfs-sdk-go/object as objectSDK.

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-07-06 15:36:41 +03:00
Evgenii Stratonikov
41ab4d070e [#423] *: Use hrw.StringHash() where possible 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-13 07:18:25 +00:00
Evgenii Stratonikov
4b768fd115 [#381] *: Move to sync/atomic 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-23 08:18:01 +03:00
Dmitrii Stepanov
d62c6e4ce6 [#242] node: Add tracing spans 
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-14 10:25:53 +00:00
Airat Arifullin
6f7b6a8813 [#116] node: Improve shard/engine construction in tests 
* Introduce testEngineWrapper that can be constructed with different options

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-05 14:36:40 +00:00
Dmitrii Stepanov
a7c79c773a [#168] node: Refactor node config 
Resolve containedctx linter for cfg

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-31 09:32:59 +03:00
Alejandro Lopez
341fe1688f [#139] test: Add test storage implementation 
This aims to reduce the usage of chmod hackery to induce or simulate
OS-related failures.

Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-03-29 14:28:49 +00:00
Airat Arifullin
221203beeb [#180] node: Refactor panics in unit test 
* Replace panics in unit tests by require.NoError and t.Fatalf

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-03-29 12:39:07 +03:00
Ayrat Arifullin
9808dec591 [#86] node: Move testing utils to one package 
Move testing utils from tests in local_object_storage package to
unified testutil package

Signed-off-by: Airat Arifullin <aarifullin@yadro.com>
 2023-03-23 08:19:15 +00:00
Alex Vanin
20de74a505 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 16:38:26 +03:00
Dmitrii Stepanov
c3a7039801 [TrueCloudLab/hrw#2] node: Optimize shard hash 
Compute shard hash only once

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-02-28 13:36:25 +03:00
Pavel Karpy
923f84722a Move to frostfs-node 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-28 15:04:29 +03:00
Pavel Karpy
f037022a7a [#1770] logger: Refactor Logger component 
Make it store its internal `zap.Logger`'s level. Also, make all the
components to accept internal `logger.Logger` instead of `zap.Logger`; it
will simplify future refactor.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-12 18:11:05 +03:00
Pavel Karpy
2d7166f8d0 [#1770] shard: Move NewEpoch event routing on SE level 
It will allow dynamic shard management. Closing a shard does not allow
removing event handlers.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
beb7f2a048 [#1634] node: Change default epoch in tests 
Do not treat objects with expiration as expired by default.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 21:32:37 +04:00
Evgenii Stratonikov
26b4a258e0 [#1523] neofs-node: Refactor configuration 
1. Move compression parameters to the `shard` section.
2. Allow to use multiple sub-storage components in the blobstor.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Pavel Karpy
9aba0ba512 [#1634] meta: Add epoch state 
It allows performing expiration checks on the stored objects.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-04 16:31:49 +03:00
Evgenii Stratonikov
26041f18bf [#1505] pilorama: Allow to customize database parameters 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 15:08:24 +03:00
Leonard Lyubich
1c30414a6c [#1454] Upgrade NeoFS SDK Go module with new IDs 
Core changes:
 * avoid package-colliding variable naming
 * avoid using pointers to IDs where unnecessary
 * avoid using `idSDK` import alias pattern
 * use `EncodeToString` for protocol string calculation and `String` for
  printing

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-01 17:41:45 +03:00
Leonard Lyubich
bb25ecbd15 [#1400] owner: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-23 15:33:14 +03:00
Leonard Lyubich
f15e6e888f [#1377] oid, cid: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-16 15:33:22 +03:00
Leonard Lyubich
ae92074272 [#1369] checksum: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-12 09:48:31 +03:00
Leonard Lyubich
088df0e2a9 [#1369] version: Upgrade SDK package 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-05-12 09:48:31 +03:00
Alex Vanin
8f476f3c4d [#1238] Adopt neofs-node for non pointer slices in SDK 
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
 2022-03-16 15:29:31 +03:00
Leonard Lyubich
9feb5f9405 [#1175] engine: Fix AddShard implementation 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-15 13:03:23 +03:00
Leonard Lyubich
9dff07200c [#1181] local storage: Process expired locks similar to tombstones 
There is a need to process expired `LOCK` objects similar to `TOMBSTONE`
ones: we collect them on `Shard`, notify all other shards about
expiration so they could unlock the objects, and only after that mark
lockers as garbage.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-15 13:03:23 +03:00
Leonard Lyubich
323dea95c6 [#1181] Do not use deprecated elements 
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-15 13:03:23 +03:00