d772e35aba
[ #1910 ] .golangci.yml: Add `godot` linker
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-18 15:08:26 +03:00
31c623636d
[ #1863 ] node: Fix shard id in the object counter metrics
...
If shard ID is stored in metabase (it is not the first time boot), read it,
set it, use it (not a generated one) in the metrics writer.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-10-13 13:06:41 +03:00
f037022a7a
[ #1770 ] logger: Refactor `Logger` component
...
Make it store its internal `zap.Logger`'s level. Also, make all the
components to accept internal `logger.Logger` instead of `zap.Logger`; it
will simplify future refactor.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-10-12 18:11:05 +03:00
4b005d3178
[ #1840 ] blobstor: Return info about all components
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-10 11:14:55 +03:00
9b241e4a17
[ #1840 ] neofs-node: Allow to use `mode: disabled` in config
...
Currently, when removing shard special care must be taken with respect
to shard numbering. `mode: disabled` allows to leave shard configuration
in place while also ignoring it during initialization. This makes
disk replacement much more convenient.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-10 11:14:55 +03:00
4eb0ed11f8
[ #1809 ] node: Do not boot up if metabase is outdated
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-10-04 12:32:10 +03:00
8b3b16fe62
[ #1825 ] writecache: Flush cache when moving to the DEGRADED mode
...
Degraded mode allows us to operate without an SSD,
thus writecache should be unavailable in this mode.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-04 12:13:09 +03:00
2d7166f8d0
[ #1770 ] shard: Move NewEpoch event routing on SE level
...
It will allow dynamic shard management. Closing a shard does not allow
removing event handlers.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-10-04 10:08:55 +03:00
0a411908ee
[ #1806 ] writecache: Allow to ignore read errors during flush
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-28 09:28:01 +03:00
f2045c10d7
[ #1806 ] shard: Check each component mode when setting mode
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-28 09:28:01 +03:00
3d882e9f47
[ #1806 ] engine: Allow to flush write-cache
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-28 09:28:01 +03:00
431e331373
[ #1658 ] shard: Update metric counters
...
Use meta's operation results to change the metrics. Support typed object
counters.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-09-13 21:32:37 +04:00
d872862710
[ #1658 ] meta: Force counters resync process
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-09-13 21:32:37 +04:00
beb7f2a048
[ #1634 ] node: Change default epoch in tests
...
Do not treat objects with expiration as expired by default.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-09-13 21:32:37 +04:00
a2bb3a2a96
[ #1630 ] pilorama: Support dropping trees
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-12 09:54:15 +03:00
c04126f35e
[ #1745 ] writecache: Remove `IsErrNotFound`
...
We specify the error in the doc-comment, and it is the same for all our
components.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-02 11:59:53 +03:00
177e8e01b1
[ #1745 ] neofs-node: Remove `memcache_capacity` from the configuration
...
It is unused since ddaed283e9
2022-09-02 11:59:53 +03:00
bc20851cd1
[ #1686 ] shard: Open all components in case metabase has failed
...
All components must be opened first, before any other operation is
performed.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-08-30 10:02:43 +03:00
9c7b3ce799
[ #1658 ] node: Read metrics from meta on startup
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-25 19:20:33 +03:00
745f72fff0
[ #1658 ] node: Support object counter metrics
...
Increment shard's object counter on successful `Put` calls and decrement on
`Delete`.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-25 19:20:33 +03:00
30341f2192
[ #1687 ] *: Perform `go fmt` using go `v1.19`
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-22 18:59:57 +03:00
26b4a258e0
[ #1523 ] neofs-node: Refactor configuration
...
1. Move compression parameters to the `shard` section.
2. Allow to use multiple sub-storage components in the blobstor.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
ddaed283e9
[ #1523 ] writecache: Simplify logic
...
1. Remove in-memory cache. It doesn't persist objects and if we want
more speed, `NoSync` option can be used for the bolt DB.
2. Put to the metabase in a synchronous fashion. This considerably
simplifies overall logic and plays nicely with the metabase bolt DB
batch settings.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
4176b2a1bc
[ #1523 ] local_object_storage: Unify parameters for the `Iterate` operation
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
73f8bb3e5f
[ #1523 ] shard: Store generic storage ID in metabase
...
Allow to extend blobstor with more storage sub-systems. Currently
objects stored in the FSTree have empty byte slice descriptor and object
from blobovnicza tree have the same id as earlier. Each such change in
the identifier formation should be accompanied with metabase version
increase.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
9eb018672c
[ #1523 ] blobstor: Unify parameters for `Put` operation
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
7d6df543d7
[ #1523 ] blobstor: Export single `Get`, `GetRange` and `Delete` methods
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
ca15083a50
[ #1523 ] local_object_storage: Unify parameters for the `Exists` operation
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
d75d030a90
[ #1523 ] local_object_storage: Unify parameters for the `Delete` operation
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
36c88f0dc8
[ #1523 ] local_object_storage: Unify parameters for the `GetRange` operation
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
d9a2f280c9
[ #1523 ] local_object_storage: Unify parameters for the `Get` operation
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
b621f5983a
[ #1523 ] local_object_storage: Move blobovnicza tree to a separate package
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
156ba85326
[ #1634 ] node: Do not return expired objects
...
If an object has not been marked for removal by the GC in the current epoch
yet but has already expired, respond with `ErrObjectNotFound` api status.
Also, optimize shard iteration: a node must stop any iteration if the object
is found but gonna be removed soon.
All the checks are performed by the Metabase.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-04 16:31:49 +03:00
9aba0ba512
[ #1634 ] meta: Add epoch state
...
It allows performing expiration checks on the stored objects.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-04 16:31:49 +03:00
4abe5a7245
shard: add more checks for `GetRange` parameters
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-07-29 11:22:04 +03:00
72586f17d4
shard: fix `GetRange` for objects stored in the write-cache
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-07-29 11:22:04 +03:00
1691364653
[ #1559 ] local_object_storage: Fix tests and some data races
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
50e28f22f9
[ #1559 ] shard: Change `Degraded` mode string representation
...
It is a flag, but is a `degraded-read-write` mode for a user.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
fabe717d32
[ #1549 ] shard: Turn to read-only mode on metabase failure
...
If metabase can't be opened in the default mode, try opening shard
first in `ReadOnly` mode and then in `DegradedReadOnly`.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
4944490ffb
[ #1559 ] local_object_storage: Move shard to the `DegradedReadOnly` mode
...
`Degraded` mode can be set by the administrator if needed.
Modifying operations in this mode can lead node into an inconsistent state
because metabase checks such as lock checking are not performed.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
3df62769c0
[ #1559 ] local_object_storage: Allow to set mode for all components
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
1e786233bf
[ #1559 ] local_object_storage: Provide readOnly flag to `Open`
...
We should be able to reopen storage in readonly in runtime.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
c8911d72d0
[ #1559 ] shard: Do not consult metabase in a degraded mode
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
339864b720
[ #1559 ] local_object_storage: Move `shard.Mode` to a separate package
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
d8ba954aff
[ #1559 ] shard: Use `Set` prefix for parameter setting
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
7b882b26d8
[ #1559 ] shard: Remove public functions
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
34d319fed2
[ #1559 ] metabase: Use `Set` prefix for parameter setting
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
f58234aa2f
[ #1559 ] metabase: Remove public functions
...
Reduce public interface of this package. Later each result will contain
an additional status, so it makes more sense to use the same functions
and result processing everywhere.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
a4adb79db7
[ #1607 ] pilorama: Enable tree service explicitly
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
26041f18bf
[ #1505 ] pilorama: Allow to customize database parameters
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 15:08:24 +03:00
Evgenii Stratonikov
Pavel Karpy
Evgenii Stratonikov
Leonard Lyubich