Commit graph

4199 commits

Author SHA1 Message Date
888f966eb4 [#661] blobovniczatree: Make Rebuild concurrent
Different DBs can be rebuild concurrently.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:36:04 +03:00
f325b9557c [#661] metrics: Add blobovniczatree rebuild metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:36:04 +03:00
4f7f67d136 [#661] blobovniczatree: Make Rebuild failover safe
Now move info stores in blobovnicza, so in case of failover
rebuild completes previous operation first.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:36:04 +03:00
8afb42aae8 [#698] blobovniczatree: Init blobovniczas concurrently
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:36:04 +03:00
71fac2b9c3 [#661] shard: Fix Delete method
Due to the flushing data from the writecache to the storage
and simultaneous deletion, a partial deletion situation is possible.
So as a solution, deletion is allowed only when the object is in storage,
because object will be deleted from writecache by flush goroutine.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:36:04 +03:00
9ef4a885de [#661] blobovniczatree: Add Rebuild implementation
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:36:04 +03:00
b22b703325 [#661] blobstor: Add Rebuild implementation
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:36:04 +03:00
784efb6155 [#661] blobovniczatree: Allow to change depth or width
Now it is possible to change depth or with of blobovniczatree.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:36:04 +03:00
df11b40dfd [#661] blobovniczatree: Use .db extension for db files
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:36:04 +03:00
4e05ce1c3a [#661] shard: Add blobstor rebuilder
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:36:04 +03:00
1f07e8b375 Release v0.37.0
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-12-07 13:36:34 +03:00
2be938f3cd [#823] go.mod: Update frostfs-api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-21 14:46:50 +03:00
eac61fe9b2 [#733] frostfs-cli: Add control ir remove-container
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-03 15:42:58 +03:00
dc81b4b50c [#725] writecache: Fix metric values
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-27 16:25:04 +03:00
98da032324 [#758] ir: Do not exclude node in maintenance mode from netmap
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-10-26 16:05:02 +03:00
5acc13fa94 [#732] containersvc: Remove load announcement
IR code was removed in 8879c6ea.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-10-09 15:17:10 +03:00
e0f0b93b5e [#723] netmap: Drop already bootstraped check
Because of this check, under certain conditions,
the node could be removed from the network map,
although the node was functioning normally.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-05 15:31:18 +03:00
eb5248621a [#723] netmap: Send bootstrap at each epoch tick
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-05 15:31:10 +03:00
02be6a4341 [#702] node: Update SDK version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-09-29 18:46:06 +03:00
368774be95 [#691] node: Compare node info during initial bootstrap properly
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-18 07:30:15 +00:00
b9ef294b99 [#692] go.mod: Update sdk-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-15 16:54:15 +03:00
10570fc035 [#690] go.mod: Update contract and api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-14 14:50:30 +03:00
c6af4a3ec8 [#679] engine: Do not increase error counter on meta mismatch
It was introduced in 69e1e6ca to help node determine faulty shards.
However, the situation is possible in a real-life scenario:
1. Object O is evacuated from shard A to B.
2. Shard A is unmounted because of lower-level errors.
3. We now have object in meta on A and in blobstor on B. Technically we
   have it in meta on shard B too, but we still got the error if B goes
   to a degraded mode.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-14 10:39:18 +03:00
58239d1b2c [#683] cli: Add context to policy parsing errors
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-12 09:47:21 +00:00
3c76884182 [#682] cli: Unify array of ranges type
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-12 10:42:34 +03:00
f435ab1b26 [#682] go.mod: Update sdk-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-12 10:42:34 +03:00
aa9f8dce3d [#677] client: Refactor PrmAnnounceSpace/EndpointInfo/NetworkInfo usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-08 09:42:28 +03:00
8a81af5a3b [#653] Add context parameter to Open functions
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-09-07 18:03:29 +03:00
a716db99db [#668] shard/test: Do not alter rootPath option
Supposedly, this was added to allow creating 2 different shards without
subtest. Now we use t.TempDir() everywhere, so this should not be a
problem.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
36759f8434 [#668] shard/test: Properly check event processing
See https://git.frostfs.info/TrueCloudLab/frostfs-node/actions/runs/1594/jobs/2

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
39879fa868 [#668] shard/test: Add dontRelease options
Most of the time we would like to close shard with minor exceptions.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
c661ba1312 [#668] shard/test: Use sane defaults in the test constructor
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
268adb79cb [#668] shard/test: Simplify shard construction
newCustomShard() has many parameters but only the first is obligatory.
`enableWriteCache` is left as-is, because it directly affects the
functionality.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
429f941cda [#668] shard/test: Release shard in t.Cleanup()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
382eb8a485 [#668] shard/test: Disable GC where it is not needed
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
42696016de [#668] shard: Close stopChannel in GC
It is done once, but now we could read it from multiple places.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
bdecfbc1be [#668] shard/test: Move tests to the main package
Semantic patch (also, duplicate definitions are removed):
```
@@
var e identifier
@@
-import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/shard"

-shard.e
+e
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
aa23c6a83a [#668] shard/test: Remove subtest from TestCounters
Otherwise, individual tests cannot be run.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
da8f384324 [#668] shard/test: Fix typo in existence
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-07 07:39:39 +00:00
aeeb8193d2 [#676] node: Fix header source creation when checking eacl
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-09-06 17:06:54 +03:00
d9de9e2bbb [#675] client: Refactor PrmObjectDelete usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-09-06 08:05:47 +00:00
88d50e4c77 [#656] policer: Add "bad" testcase
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-09-06 08:04:59 +00:00
054e3ef3d3 [#674] pre-commit: Update shellcheck-py
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-04 15:34:24 +03:00
a54b4472de [#674] network: Close connections on address updates
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-09-04 15:34:24 +03:00
7456c8556a [#536] blobovnicza: Add blobovniczatree DB cache
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-09-01 13:53:11 +03:00
c672f59ab8 [#536] blobovnicza: Drop cache
Each blobovnicza instance is opened
while is in use.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-09-01 13:51:26 +03:00
b9b86d2ec8 [#666] shard/test: Fix data race in metrics tests
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-08-31 08:39:42 +00:00
4dff9555f1 [#568] writecache: Improve flushing scheme for badger
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-08-30 17:22:28 +00:00
806cc13d9f [#658] client: Refactor PrmObjectGet/Head/Range usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-08-30 17:13:23 +00:00
1daef2ceeb [#660] writecache: Fix remaining addr2key uses
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-30 17:12:33 +00:00