Commit graph

173 commits

Author SHA1 Message Date
ad5f527bd3 [#541] writecache/test: Remove initWC()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-07-29 10:57:25 +00:00
11027945d8 [#479] writecache: Fix writecache fstree flush premature ctx cancel
All checks were successful
Build / Build Components (1.20) (pull_request) Successful in 3m49s
Build / Build Components (1.19) (pull_request) Successful in 4m2s
ci/woodpecker/pr/pre-commit Pipeline was successful
Tests and linters / Tests (1.20) (pull_request) Successful in 6m13s
Tests and linters / Lint (pull_request) Successful in 11m25s
Tests and linters / Tests with -race (pull_request) Successful in 9m4s
Tests and linters / Staticcheck (pull_request) Successful in 6m0s
Tests and linters / Tests (1.19) (pull_request) Successful in 25m11s
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-07-14 10:25:52 +03:00
033eaf77e1 [#496] node: Fix linter importas
All checks were successful
Build / Build Components (1.20) (pull_request) Successful in 3m52s
Build / Build Components (1.19) (pull_request) Successful in 4m1s
ci/woodpecker/pr/pre-commit Pipeline was successful
Tests and linters / Tests with -race (pull_request) Successful in 5m36s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m55s
Tests and linters / Lint (pull_request) Successful in 14m40s
Tests and linters / Tests (1.19) (pull_request) Successful in 15m29s
ci/woodpecker/push/pre-commit Pipeline was successful
Standardize the alias of the
import frostfs-sdk-go/object as objectSDK.

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-07-06 15:36:41 +03:00
fe01781811 [#446] los: Wrap SSD errors in a separate type
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-06-16 14:53:31 +03:00
85deb12f4d [#424] writecache: Drop metrics when close
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 14:52:40 +03:00
74578052f9 [#412] node: Replace tracing package
Use observability module.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-01 13:23:11 +00:00
802168c0c6 [#364] node: Stop flushing big object when termination signal received
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-26 16:46:58 +03:00
2ce43935f9 [#312] metrics: Add writecache metrcis
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 10:18:39 +00:00
d212d908b5 [#312] wc: Add metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 10:18:39 +00:00
4503a61997 [#312] wc: Delete unused Iterate method
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-05-24 10:18:39 +00:00
4b768fd115 [#381] *: Move to sync/atomic
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-23 08:18:01 +03:00
35c9b6b26d [#314] writecache: remove objects right after they are flushed
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-19 09:37:05 +00:00
Pavel Karpy
bf79d06f03 [#314] writecache: Do not lose small objects on disk errors
Do return error if an object could not been stored on WC's disk.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-05-19 09:37:05 +00:00
Pavel Karpy
9e56592be3 [#314] writecache: Simplify background workers naming
Also, drop not used arg.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-05-19 09:37:05 +00:00
Pavel Karpy
e9461686b8 [#274] wc: Resolve possible deadlock
If operation with WC are _fast enough_ (e.g. `Init` failed and `Close` is
called immediately) there is a race and a deadlock that do not allow finish
(and start, in fact) an initialization routine because of taken `modeMtx`
and also do not allow finish `Close` call because of awaiting initialization
finish. So do stop initialization _before_ any mutex is taken.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-20 19:22:54 +03:00
Pavel Karpy
6b6f33ed71 [#274] wc: Make wait groups work more explicit
Do not run routine that calls `wg.Done()` inside, it is hard to read.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-20 19:22:54 +03:00
Pavel Karpy
20cd080323 [#255] write-cache: Fix init race condition
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Do not use WC's internals in the initialization routines without mode
protection. WC should be able to change its mode even if the initialization
is not finished yet.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-19 09:29:07 +00:00
d62c6e4ce6 [#242] node: Add tracing spans
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-14 10:25:53 +00:00
0e31c12e63 [#240] logs: Move log messages to constants
Drop duplicate entities.
Format entities.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 05:06:09 +00:00
0920d848d0 [#135] get-object: Add tracing spans
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-12 06:52:00 +00:00
341fe1688f [#139] test: Add test storage implementation
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
This aims to reduce the usage of chmod hackery to induce or simulate
OS-related failures.

Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-03-29 14:28:49 +00:00
Pavel Karpy
f1f3c80dbf [#32] node: Init write-cache asynchronously
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-03-09 11:07:33 +00:00
20de74a505 Rename package name
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 16:38:26 +03:00
d1d123d180 [#2234] writecache: Fix possible panic in initFlushMarks
In case we have many small objects in the write-cache, `indices` should
not be reused between iterations.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
e0309e398c [#2239] writecache: Fix possible deadlock
LRU `Peek`/`Contains` take LRU mutex _inside_ of a `View` transaction.
`View` transaction itself takes `mmapLock` [1], which is lifted after tx
finishes (in `tx.Commit()` -> `tx.close()` -> `tx.db.removeTx`)

When we evict items from LRU cache mutex order is different:
first we take LRU mutex and then execute `Batch` which _does_ take
`mmapLock` in case we need to remap. Thus the deadlock.

[1] 8f4a7e1f92/db.go (L708)

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
4155c1bdff [#5] writecache: Use generic LRU cache
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-31 23:04:06 +03:00
Pavel Karpy
1608fd1c07 [#2167] write-cache: Add "write-cache" to its logs
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-30 11:07:35 +03:00
Pavel Karpy
923f84722a Move to frostfs-node
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-28 15:04:29 +03:00
Evgenii Stratonikov
42554a9298 [#2068] writecache: Remove deleted objects from the writecache
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-12-02 11:52:05 +03:00
Evgenii Stratonikov
4a49ea0855 [#2068] writecache: Allow to open FSTree in read-only mode
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-12-02 11:52:05 +03:00
Evgenii Stratonikov
857d2dc3f5 [#2068] writecache: Optimize initial flush existence checking
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-12-02 11:52:05 +03:00
Pavel Karpy
ed4351aab0 [#2074] write-cache: Do not flush same object twice
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Pavel Karpy
dd225906a0 [#2074] write-cache: Remove unused variables
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
b0e94b6a6b [#1906] writecache: Do not require read-only mode in Flush
It was needed before we started to flush during transition to
`degraded` mode. Now it is confusing.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
777fd32d4f [#1818] writecache: Increase error counter on background errors
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-02 14:24:02 +03:00
Evgenii Stratonikov
bffb0f894c [#1818] writecache: Update storage ID during flush
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-02 14:24:02 +03:00
Evgenii Stratonikov
b64b14eb54 [#1818] writecache: Reuse FSTree flushing code between flushes
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-02 14:24:02 +03:00
Evgenii Stratonikov
a56927e3d4 [#1818] writecache: Remove unused variable
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-02 14:24:02 +03:00
Evgenii Stratonikov
98a152256b [#1992] writecache: Allow to open in NOSYNC mode
Applicable only to FSTree as we cannot handle corrupted databases
properly yet.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-01 09:42:26 +03:00
Evgenii Stratonikov
34501685b7 [#1969] local_object_storage: Move ErrObjectIsExpired to another package
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-31 11:41:24 +03:00
Evgenii Stratonikov
56de2f1363 [#1969] local_object_storage: Simplify logic error construction
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-31 11:41:24 +03:00
Evgenii Stratonikov
fcdbf5e509 [#1969] local_object_storage: Add a type for logical errors
All logic errors are wrapped in `logicerr.Logical` type and do not
affect shard error counter.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-31 11:41:24 +03:00
Leonard Lyubich
e8c5f03c30 [#1905] shard: Don't log read-only errors of write-cache
There is no need to log `writecache.ErrReadOnly` errors in `Delete`
method of the `Shard`.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
2022-10-28 18:30:45 +03:00
Pavel Karpy
f037022a7a [#1770] logger: Refactor Logger component
Make it store its internal `zap.Logger`'s level. Also, make all the
components to accept internal `logger.Logger` instead of `zap.Logger`; it
will simplify future refactor.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-10-12 18:11:05 +03:00
Evgenii Stratonikov
8b3b16fe62 [#1825] writecache: Flush cache when moving to the DEGRADED mode
Degraded mode allows us to operate without an SSD,
thus writecache should be unavailable in this mode.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-04 12:13:09 +03:00
Evgenii Stratonikov
0a411908ee [#1806] writecache: Allow to ignore read errors during flush
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-28 09:28:01 +03:00
Evgenii Stratonikov
0b4c867ef1 [#1806] writecache: Allow to start flush manually
Allow user to initiate flushing objects from a writecache.
We need this in 2 cases:
1. During writecache storage schema update, it should be flushed with
   the old version of node and started clean with a new one.
2. During SSD replacement, to avoid data loss.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-28 09:28:01 +03:00
Evgenii Stratonikov
55148404ee [#1806] writecache: Use interfaces for blobstor and metabase
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-28 09:28:01 +03:00
Evgenii Stratonikov
e6cf0b3d42 [#1745] writecache: Set flush parameters based on max size
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-02 11:59:53 +03:00
Evgenii Stratonikov
c04126f35e [#1745] writecache: Remove IsErrNotFound
We specify the error in the doc-comment, and it is the same for all our
components.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-02 11:59:53 +03:00
Evgenii Stratonikov
9e41e85295 [#1745] writecache: Actualize docs
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-02 11:59:53 +03:00
Evgenii Stratonikov
e9c6ee2623 [#1745] writecache: Rename constants for default values
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-02 11:59:53 +03:00
Evgenii Stratonikov
177e8e01b1 [#1745] neofs-node: Remove memcache_capacity from the configuration
It is unused since ddaed283e9 .

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-02 11:59:53 +03:00
Evgenii Stratonikov
bda084f331 [#1745] writecache: Simplify object counters
Remove unused option and additional pointers to db/fstree.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-02 11:59:53 +03:00
Evgenii Stratonikov
20abdaeed4 [#1745] writecache: Set flush mark only on success
Set flush mark in the inside the flush worker because writing to the blobstor
can fail. Because each evicted object must be deleted, it is reasonable
to do this in the evict callback.

The evict callback is protected by LRU mutex and thus potentially interferes
with `Get` and `Iterate` methods. This problem will be addressed in the
future.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-02 11:59:53 +03:00
Evgenii Stratonikov
b8b9d25f9d [#1686] local_object_storage: Add generic tests
Use them for writecache as a simple example.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-08-30 10:02:43 +03:00
Pavel Karpy
30341f2192 [#1687] *: Perform go fmt using go v1.19
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-22 18:59:57 +03:00
Evgenii Stratonikov
26b4a258e0 [#1523] neofs-node: Refactor configuration
1. Move compression parameters to the `shard` section.
2. Allow to use multiple sub-storage components in the blobstor.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
266458fe5c [#1523] blobstor: Unify fstree and blobovnicza interfaces
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
5aa3defc67 [#1523] writecache: Remove cloneBytes
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
ddaed283e9 [#1523] writecache: Simplify logic
1. Remove in-memory cache. It doesn't persist objects and if we want
   more speed, `NoSync` option can be used for the bolt DB.
2. Put to the metabase in a synchronous fashion. This considerably
   simplifies overall logic and plays nicely with the metabase bolt DB
   batch settings.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
4176b2a1bc [#1523] local_object_storage: Unify parameters for the Iterate operation
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
73f8bb3e5f [#1523] shard: Store generic storage ID in metabase
Allow to extend blobstor with more storage sub-systems. Currently
objects stored in the FSTree have empty byte slice descriptor and object
from blobovnicza tree have the same id as earlier. Each such change in
the identifier formation should be accompanied with metabase version
increase.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
9eb018672c [#1523] blobstor: Unify parameters for Put operation
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
ca15083a50 [#1523] local_object_storage: Unify parameters for the Exists operation
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
d9a2f280c9 [#1523] local_object_storage: Unify parameters for the Get operation
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
4e361e2ab4 [#1647] writecache: Open DB in sync mode
Close #1647.

Initially, `Sync: false` was provided because we can already lose
objects cached in memory. However, future changes in writecache will
remove inmemory cache and speed up it via other means.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-08-16 15:27:18 +04:00
Pavel Karpy
da2975a2f9 [#1664] write-cache: Fix panic on Delete operation
If an object is found in the Write-cache and is placed at the end of
the in-memory cache, the memory counter update operation tries to
dereference the index that is out of the sliced array. Moreover, even if
panic does not appear, the counter is updated with the wrong value.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-09 20:18:26 +03:00
Pavel Karpy
156ba85326 [#1634] node: Do not return expired objects
If an object has not been marked for removal by the GC in the current epoch
yet but has already expired, respond with `ErrObjectNotFound` api status.
Also, optimize shard iteration: a node must stop any iteration if the object
 is found but gonna be removed soon.
All the checks are performed by the Metabase.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-04 16:31:49 +03:00
Evgenii Stratonikov
1691364653 [#1559] local_object_storage: Fix tests and some data races
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
3df62769c0 [#1559] local_object_storage: Allow to set mode for all components
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
1e786233bf [#1559] local_object_storage: Provide readOnly flag to Open
We should be able to reopen storage in readonly in runtime.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
339864b720 [#1559] local_object_storage: Move shard.Mode to a separate package
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
34d319fed2 [#1559] metabase: Use Set prefix for parameter setting
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
f58234aa2f [#1559] metabase: Remove public functions
Reduce public interface of this package. Later each result will contain
an additional status, so it makes more sense to use the same functions
and result processing everywhere.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
11c9df6f00 [#1611] shard: Print shard ID in component logs
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-19 15:57:53 +03:00
Evgenii Stratonikov
972ca83e23 [#1524] writecache: Add some bolt parameters to the configuration
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-20 17:04:35 +03:00
Evgenii Stratonikov
38772e1a2e writecache: provide timeout to bbolt.Open
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-09 14:19:47 +03:00
Evgenii Stratonikov
feef9a98f7 [#1462] writecache: remove useless logs
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-03 14:30:00 +03:00
Evgenii Stratonikov
43e776dfb1 [#1462] writecache: Persist objects in batches
`Update` becomes a botleneck under a heavy load.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-03 14:30:00 +03:00
Evgenii Stratonikov
6ad87e7959 [#1462] writecache: Fill flush marks during startup
Some of the objects are already flushed, don't do it twice.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-03 14:30:00 +03:00
Evgenii Stratonikov
f2a7503964 [#1462] fstree: Allow to fetch file content lazily
If we should process address based on some condition, there is no need
to read file content in memory.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-03 14:30:00 +03:00
Evgenii Stratonikov
54d4503701 [#1462] writecache: Rename flushWorkersCount
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-03 14:30:00 +03:00
Evgenii Stratonikov
0148209168 [#1462] writecache: Use Timer instead of Ticker in flush
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-03 14:30:00 +03:00
Pavel Karpy
281befec67 [#1418] blobstor: Do not use pointers as parameters
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-03 07:35:17 +03:00
Pavel Karpy
1c100fb4b0 [#1418] writecache: Do not use pointers as parameters
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-03 07:35:17 +03:00
Leonard Lyubich
1c30414a6c [#1454] Upgrade NeoFS SDK Go module with new IDs
Core changes:
 * avoid package-colliding variable naming
 * avoid using pointers to IDs where unnecessary
 * avoid using `idSDK` import alias pattern
 * use `EncodeToString` for protocol string calculation and `String` for
  printing

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-01 17:41:45 +03:00
Elizaveta Chichindaeva
cc7a723d77 [#1320] English Check
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
2022-05-11 10:40:02 +03:00
Evgenii Stratonikov
48eb87d32c [#1149] writecache: Prevent corruption in Head
Also, remove optimization comments:
1. Having to maintain an execute the same logic for headers as for
   objects is quite inefficient, as it increases memory footprint.
2. Unmarshaling object is a cheap operation if data slice is in memory.
3. For unmarshaling header-only, I think we need SDK support.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-31 15:40:25 +03:00
Evgenii Stratonikov
6472a170eb [#1143] shard: Introduce explicit Degraded mode
`Degraded` mode is set automatically after error counter is over the
threshold. `ReadOnly` mode can still be set by an administrator.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-03-31 15:33:22 +03:00
Leonard Lyubich
318639e5bf [#1247] *: Clarify docs about returned errors from apistatus package
`apistatus` package provides types which implement build-in `error`
interface. Add `error of type` pattern when documenting these errors in
order to clarify how these errors should be handled (e.g. `errors.Is` is
not good).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-17 16:34:00 +03:00
Leonard Lyubich
70ffdf3478 [#1247] object: Return NOT_FOUND and ALREADY_REMOVED statuses
Replace `ErrNotFound`/`ErrAlreadyRemoved` error from
`pkg/core/object` package with `ObjectNotFound`/`ObjectAlreadyRemoved`
one from `apistatus` package. These errors are returned by storage
node's server as NeoFS API statuses.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-17 16:34:00 +03:00
Leonard Lyubich
7ccd1625af [#1214] *: Use single Object type in whole project
Remove `Object` and `RawObject` types from `pkg/core/object` package.
Use `Object` type from NeoFS SDK Go library everywhere. Avoid using the
deprecated elements.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-04 17:45:00 +03:00
Evgenii Stratonikov
050a4bb2b0 [#1115] *: link TODOs to corresponding issues
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-11 12:58:59 +03:00
Pavel Karpy
1667ec9e6d [#1131] *: Adopt SDK changes
`object.Address` has been moved to `object/address`
`object.ID` has been moved to `object/id`

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-02-08 09:45:38 +03:00
Evgenii Stratonikov
674f520da7 [#1132] *: Use path/filepath package when working with files
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-03 10:12:07 +03:00
Evgenii Stratonikov
36eebb5932 [#1085] writecache: allow to ignore errors during iteration
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00
Evgenii Stratonikov
c954f0e71b [#1085] fstree: allow to ignore errors during iteration
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00
Evgenii Stratonikov
0ef3d5ab03 [#1085] writecache: persist in-memory objects before shutdown
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00
Evgenii Stratonikov
c0f6c988f0 [#1085] shard: dump data from write-cache
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-01-26 15:28:13 +03:00