Commit graph

12 commits

Author SHA1 Message Date
Evgenii Stratonikov
d2cce62934 [#1818] writecache: Increase error counter on background errors
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-07 16:07:20 +03:00
Evgenii Stratonikov
a4a6d547a8 [#1818] writecache: Update storage ID during flush
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-07 16:07:20 +03:00
Evgenii Stratonikov
245da1e50e [#1992] writecache: Allow to open in NOSYNC mode
Applicable only to FSTree as we cannot handle corrupted databases
properly yet.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-01 15:29:01 +03:00
Pavel Karpy
f037022a7a [#1770] logger: Refactor Logger component
Make it store its internal `zap.Logger`'s level. Also, make all the
components to accept internal `logger.Logger` instead of `zap.Logger`; it
will simplify future refactor.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-10-12 18:11:05 +03:00
Evgenii Stratonikov
55148404ee [#1806] writecache: Use interfaces for blobstor and metabase
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-28 09:28:01 +03:00
Evgenii Stratonikov
177e8e01b1 [#1745] neofs-node: Remove memcache_capacity from the configuration
It is unused since ddaed283e9 .

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-02 11:59:53 +03:00
Evgenii Stratonikov
bda084f331 [#1745] writecache: Simplify object counters
Remove unused option and additional pointers to db/fstree.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-02 11:59:53 +03:00
Evgenii Stratonikov
972ca83e23 [#1524] writecache: Add some bolt parameters to the configuration
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-20 17:04:35 +03:00
Leonard Lyubich
a1696a81b6 [#776] writecache: Limit size of used disk space
There is a need to limit disk space used by write-cache. It is almost
impossible to calculate the value exactly. It is proposed to estimate the
size of the cache by the number of objects stored in it.

Track amounts of objects saved in DB and FSTree separately. To do this,
`ObjectCounters` interface is defined. It is generalized to a store of
numbers that can be made persistent (new option `WithObjectCounters`). By
default DB number is calculated as key number in default bucket, and FS
number is set same to DB since it is currently hard to read the actual value
from `FSTree` instance. Each PUT/DELETE operation to DB or FS
increases/decreases corresponding counter. Before each PUT op an overflow
check is performed with the following formula for evaluating the occupied
space: `NumDB * MaxDBSize + NumFS * MaxFSSize`. If next PUT can cause
write-cache overflow, object is written to the main storage.

By default maximum write-cache size is set to 1GB.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-15 18:07:36 +03:00
Leonard Lyubich
3258d9c616 [#790] writecache: Log each writing operation
Call `storagelog.Write` in all places after the successful writing op.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2021-09-13 15:02:13 +03:00
Pavel Karpy
c54f524df9 [#773] writecache: Delete unused dbSize param
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2021-08-27 15:05:53 +03:00
Evgenii Stratonikov
59de521fd1 [#472] blobstor: implement write-cache
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-04-29 17:36:36 +03:00