TrueCloudLab/frostfs-node
19
1
Fork 27
Code Issues 91 Pull requests 7 Projects Releases 1 Activity Actions
3217 commits 8 branches 83 tags 53 MiB
Commit graph

633 commits

Author SHA1 Message Date
Evgenii Stratonikov
a56927e3d4 [#1818] writecache: Remove unused variable 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-02 14:24:02 +03:00
Evgenii Stratonikov
98a152256b [#1992] writecache: Allow to open in NOSYNC mode 
Applicable only to FSTree as we cannot handle corrupted databases
properly yet.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-01 09:42:26 +03:00
Evgenii Stratonikov
f564430b90 [#1992] fstree: Allow working in SYNC mode 
Make O_SYNC the default and allow to opt-out explicitly.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-01 09:42:26 +03:00
Evgenii Stratonikov
34501685b7 [#1969] local_object_storage: Move ErrObjectIsExpired to another package 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-31 11:41:24 +03:00
Evgenii Stratonikov
56de2f1363 [#1969] local_object_storage: Simplify logic error construction 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-31 11:41:24 +03:00
Evgenii Stratonikov
fcdbf5e509 [#1969] local_object_storage: Add a type for logical errors 
All logic errors are wrapped in `logicerr.Logical` type and do not
affect shard error counter.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-31 11:41:24 +03:00
Leonard Lyubich
e8c5f03c30 [#1905] shard: Don't log read-only errors of write-cache 
There is no need to log `writecache.ErrReadOnly` errors in `Delete`
method of the `Shard`.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 18:30:45 +03:00
Leonard Lyubich
b1fa084756 [#1905] shard: Decrease severity level of write-cache failure logs 
In previous implementation `Shard.Delete` logged writecache's removal
failures in `error` level. There is a need to decrease severity of these
log records since they aren't critical and don't require individual
review.

Change level of the message to `info`.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 18:30:45 +03:00
Evgenii Stratonikov
1e6588e761 [#1974] shard: Do not panic in degraded mode 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 12:41:12 +03:00
Evgenii Stratonikov
7395ab8ef7 [#1944] metabase: Assume static buckets are created on Init 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 12:41:12 +03:00
Evgenii Stratonikov
e4bc3d0e9d [#1944] metabase: Recreate static buckets instead of resetting 
From the `Bucket.ForEach` doc:
```
The provided function must not modify the bucket; this will result in undefined behavior.
```

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 12:41:12 +03:00
Evgenii Stratonikov
3b939d190c [#1957] engine: Move shard to read-only if cannot move to degraded 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-26 08:20:53 +03:00
Evgenii Stratonikov
713fdab177 [#1907] shard: Return from Close after GC has stopped 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-25 11:54:45 +03:00
Pavel Karpy
41f93dcc1d [#1953] blobstor: Log any object removal operation 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-24 21:52:18 +03:00
Pavel Karpy
942cc38146 [#1953] engine: Add refactored storage helpers 
After the refactor there are new storage characteristics: a type and
a general storage id (that could be stringified).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-24 21:52:18 +03:00
Pavel Karpy
0371d15b2f [#1953] shard: Fix debug log messages 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-24 21:52:18 +03:00
Pavel Karpy
f8180447a1 [#1938] meta: Make version error messages more descriptive 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-24 21:46:18 +03:00
Evgenii Stratonikov
1beafea0b5 [#1869] shard: Add logs for SetMode operations on reload 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 13:28:26 +03:00
Evgenii Stratonikov
87be4f1629 [#1869] shard: Restore shard mode on failed reloads 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 13:28:26 +03:00
Evgenii Stratonikov
c785e11b20 [#1869] shard: Allow to reload metabase on SIGHUP 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 13:28:26 +03:00
Evgenii Stratonikov
f769fc83fc [#1869] shard: Embed gcCfg as raw struct 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-24 13:28:26 +03:00
Pavel Karpy
49c38d642d [#1902] engine: Search for the tree IDs in every shard 
Iterate over every shard and search for the container's trees. Final result
is a concatenation of shards' results. It is considered that one fixed tree
is placed on one fixed shard but the different trees of a fixed container
could be placed on different shards.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-20 16:17:57 +03:00
Pavel Karpy
24e9e3f3bf [#1902] engine, shard: Implement TreeList method 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-20 16:17:57 +03:00
Pavel Karpy
2e199c7ab1 [#1902] shard: Fix pilorama disabled err message 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-20 16:17:57 +03:00
Pavel Karpy
19850ef157 [#1902] pilorama: Add TreeList method 
To both `bolt` and `memory` forests; extend `Forest` interface.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-20 16:17:57 +03:00
Evgenii Stratonikov
1cb892c579 [#1910] .golangci.yml: Add misspell linker 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-18 15:08:26 +03:00
Evgenii Stratonikov
d772e35aba [#1910] .golangci.yml: Add godot linker 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-18 15:08:26 +03:00
Pavel Karpy
f76516a883 [#1860] meta: Fix 0,1 -> 2+ version migration 
In the 2nd version, there was a database format change: buckets have changed
their keys, so it becomes impossible to check the version in the 1 -> 2+
migrations because of different buckets that store info about the version.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-17 13:42:05 +03:00
Pavel Karpy
31c623636d [#1863] node: Fix shard id in the object counter metrics 
If shard ID is stored in metabase (it is not the first time boot), read it,
set it, use it (not a generated one) in the metrics writer.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-13 13:06:41 +03:00
Pavel Karpy
f037022a7a [#1770] logger: Refactor Logger component 
Make it store its internal `zap.Logger`'s level. Also, make all the
components to accept internal `logger.Logger` instead of `zap.Logger`; it
will simplify future refactor.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-12 18:11:05 +03:00
Evgenii Stratonikov
19c0a74e94 [#1867] services/control: Allow to provide multiple shard IDs to some commands 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-12 11:20:48 +03:00
Evgenii Stratonikov
2d43892fc9 [#1840] neofs-node: Use blobstor paths to identify shard 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Evgenii Stratonikov
4b005d3178 [#1840] blobstor: Return info about all components 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Evgenii Stratonikov
6d7ffefec5 [#1840] blobstor/test: Add tests for auxiliary functions 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Evgenii Stratonikov
328691c94f [#1840] blobstor: Return Path from components 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Evgenii Stratonikov
9b241e4a17 [#1840] neofs-node: Allow to use mode: disabled in config 
Currently, when removing shard special care must be taken with respect
to shard numbering. `mode: disabled` allows to leave shard configuration
in place while also ignoring it during initialization. This makes
disk replacement much more convenient.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Evgenii Stratonikov
b2aa9947c2 [#1829] engine: Delete split objects properly 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-07 16:35:46 +03:00
Evgenii Stratonikov
6557f5d249 [#1839] engine: Handle Inhume errors properly 
If shard is in read-only or degraded mode, there is no need to increase
error counter.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-07 14:25:52 +03:00
Pavel Karpy
4eb0ed11f8 [#1809] node: Do not boot up if metabase is outdated 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 12:32:10 +03:00
Evgenii Stratonikov
8b3b16fe62 [#1825] writecache: Flush cache when moving to the DEGRADED mode 
Degraded mode allows us to operate without an SSD,
thus writecache should be unavailable in this mode.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-04 12:13:09 +03:00
Evgenii Stratonikov
2e3ef817f4 [#1819] engine: Increase error counter for PUT errors 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-04 10:11:52 +03:00
Evgenii Stratonikov
b89e71fa78 [#1819] common: Add ErrNoSpace 
Add a common error for this case because it is not an error
which should increase error counter. Single error simplifies checks on
the call-site.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-04 10:11:52 +03:00
Evgenii Stratonikov
af56574849 [#1819] engine: Fix error counter in Inhume 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-04 10:11:52 +03:00
Pavel Karpy
8ebe95747e [#1770] node: Do not lock on shard's Close call 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
887afeaddb [#1770] engine: Do not lock on shard init 
Init can take a lot of time. Because the mutex is taken, all new operations
are blocked.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
fbd5bc8c38 [#1770] engine: Support configuration reload 
Currently, it only supports changing the compound of the shards.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
5c69e19016 [#1770] fstree: Depth parameter int -> uint64 
Negative values have no sense. On the other hand it differs from the
blobovnicza's configuration and prevents unification.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Pavel Karpy
2d7166f8d0 [#1770] shard: Move NewEpoch event routing on SE level 
It will allow dynamic shard management. Closing a shard does not allow
removing event handlers.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Evgenii Stratonikov
0a411908ee [#1806] writecache: Allow to ignore read errors during flush 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-28 09:28:01 +03:00
Evgenii Stratonikov
f2045c10d7 [#1806] shard: Check each component mode when setting mode 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-28 09:28:01 +03:00