e39378b1c3
[ #1367 ] writecache: Add background flushing objects limiter
...
To limit memory usage by background flush.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 15:06:33 +03:00
d3b209c8e1
[ #1337 ] shard: Disable background rebuild
...
Vulncheck / Vulncheck (pull_request) Successful in 1m3s
DCO action / DCO (pull_request) Successful in 1m28s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m2s
Build / Build Components (pull_request) Successful in 2m15s
Tests and linters / Run gofumpt (pull_request) Successful in 2m10s
Tests and linters / gopls check (pull_request) Successful in 2m21s
Tests and linters / Staticcheck (pull_request) Successful in 2m55s
Tests and linters / Lint (pull_request) Successful in 3m29s
Tests and linters / Tests (pull_request) Successful in 3m47s
Tests and linters / Tests with -race (pull_request) Successful in 3m58s
Since `frostfs-cli control shards rebuild` command was added,
there is no need for background rebuild now.
For failover tests used used value 1 to rebuild only schema change.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-06 15:19:55 +03:00
a61201a987
[ #1337 ] config: Move rebuild_worker_count to shard section
...
This makes it simple to limit performance degradation for every shard
because of rebuild.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-06 13:57:27 +03:00
68029d756e
[ #1302 ] writecache: Allow to specify custom page size
...
DCO action / DCO (pull_request) Successful in 1m57s
Vulncheck / Vulncheck (pull_request) Successful in 2m6s
Tests and linters / Run gofumpt (pull_request) Successful in 2m9s
Build / Build Components (1.22) (pull_request) Successful in 2m37s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m49s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m1s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m4s
Tests and linters / Staticcheck (pull_request) Successful in 3m0s
Tests and linters / Tests with -race (pull_request) Successful in 3m9s
Tests and linters / Lint (pull_request) Successful in 3m30s
Tests and linters / gopls check (pull_request) Successful in 3m40s
Build / Build Components (1.21) (pull_request) Successful in 1m12s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-09 12:12:07 +03:00
08b1f18bca
[ #1296 ] writecache: Add count limit
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-09 06:30:32 +00:00
78b1d9b18d
[ #1226 ] blobovniczatree: Drop leaf width limitation
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-04 12:22:06 +03:00
40c9ddb6ba
[ #1226 ] blobovniczatree: Drop init in advance option
...
To make blobovniczatree unlimited.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-04 12:22:06 +03:00
7b8937ec35
[ #1184 ] config: Add audit.enabled parameter for node
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-19 16:05:53 +03:00
dbd3b238f7
[ #1170 ] node: Support morph mTLS
...
Vulncheck / Vulncheck (pull_request) Successful in 3m18s
DCO action / DCO (pull_request) Successful in 3m33s
Build / Build Components (1.21) (pull_request) Successful in 4m15s
Build / Build Components (1.22) (pull_request) Successful in 4m28s
Tests and linters / gopls check (pull_request) Successful in 4m58s
Tests and linters / Staticcheck (pull_request) Successful in 5m8s
Tests and linters / Lint (pull_request) Successful in 6m14s
Pre-commit hooks / Pre-commit (pull_request) Successful in 8m13s
Tests and linters / Tests (1.21) (pull_request) Successful in 9m59s
Tests and linters / Tests with -race (pull_request) Successful in 9m52s
Tests and linters / Tests (1.22) (pull_request) Successful in 10m6s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-14 09:30:46 +03:00
a849236b68
[ #1161 ] node: Remove notification functionality
...
It is unused and will be reworked in future.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-07 12:10:51 +00:00
b4cfc80579
[ #1133 ] node: Improve tests for shards.default config section
...
DCO action / DCO (pull_request) Successful in 4m58s
Vulncheck / Vulncheck (pull_request) Successful in 7m43s
Tests and linters / gopls check (pull_request) Successful in 9m21s
Tests and linters / Lint (pull_request) Successful in 10m3s
Tests and linters / Staticcheck (pull_request) Successful in 9m59s
Tests and linters / Tests (1.22) (pull_request) Successful in 12m27s
Tests and linters / Tests (1.21) (pull_request) Successful in 13m13s
Build / Build Components (1.21) (pull_request) Successful in 13m8s
Build / Build Components (1.22) (pull_request) Successful in 13m2s
Tests and linters / Tests with -race (pull_request) Successful in 13m46s
Pre-commit hooks / Pre-commit (pull_request) Successful in 14m38s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-15 09:48:21 +03:00
411a8d0245
[ #1004 ] blobovnicza: Use TTL for blobovnicza tree cache
...
DCO action / DCO (pull_request) Successful in 12m46s
Vulncheck / Vulncheck (pull_request) Successful in 14m10s
Build / Build Components (1.22) (pull_request) Successful in 15m23s
Build / Build Components (1.21) (pull_request) Successful in 15m38s
Tests and linters / Staticcheck (pull_request) Successful in 3m36s
Tests and linters / gopls check (pull_request) Successful in 4m0s
Tests and linters / Lint (pull_request) Successful in 5m8s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m41s
Tests and linters / Tests with -race (pull_request) Successful in 7m46s
Tests and linters / Tests (1.22) (pull_request) Successful in 7m58s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-26 19:54:29 +03:00
57466594fb
[ #1024 ] shard: Resync metabase concurrently
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:21:10 +03:00
6bf77cabd4
[ #1044 ] ape: Add morph chain cache
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-18 18:44:10 +03:00
702351a5d1
[ #983 ] blobstor: Allow to specify wait before drop time
...
DCO action / DCO (pull_request) Successful in 1m11s
Build / Build Components (1.21) (pull_request) Successful in 3m29s
Vulncheck / Vulncheck (pull_request) Successful in 2m53s
Build / Build Components (1.20) (pull_request) Successful in 4m8s
Tests and linters / Staticcheck (pull_request) Successful in 4m19s
Tests and linters / Lint (pull_request) Successful in 5m50s
Tests and linters / gopls check (pull_request) Successful in 5m40s
Tests and linters / Tests (1.20) (pull_request) Successful in 7m46s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m6s
Tests and linters / Tests with -race (pull_request) Successful in 8m50s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-03-05 09:23:57 +03:00
ae5bb87e70
Revert "[ #866 ] Use TTL for blobovnicza tree cache"
...
DCO action / DCO (pull_request) Successful in 2m7s
Vulncheck / Vulncheck (pull_request) Successful in 3m7s
Tests and linters / Staticcheck (pull_request) Successful in 5m12s
Tests and linters / Lint (pull_request) Successful in 6m23s
Tests and linters / Tests with -race (pull_request) Successful in 7m11s
Tests and linters / gopls check (pull_request) Successful in 7m27s
Tests and linters / Tests (1.20) (pull_request) Successful in 8m5s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m22s
Build / Build Components (1.21) (pull_request) Successful in 1m41s
Build / Build Components (1.20) (pull_request) Successful in 1m56s
This reverts commit d9cbb16bd3
2024-03-01 19:29:33 +03:00
962e5a9c19
[ #736 ] logger: Add journald support
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-02-12 06:17:22 +00:00
8180a0664f
[ #887 ] node: Drop badger writecache implementation
...
Vulncheck / Vulncheck (pull_request) Successful in 6m1s
Build / Build Components (1.21) (pull_request) Successful in 7m37s
Build / Build Components (1.20) (pull_request) Successful in 7m52s
Tests and linters / Staticcheck (pull_request) Successful in 8m56s
Tests and linters / Lint (pull_request) Successful in 9m26s
Tests and linters / Tests (1.21) (pull_request) Successful in 15m5s
Tests and linters / Tests with -race (pull_request) Successful in 15m7s
DCO action / DCO (pull_request) Successful in 1m1s
Tests and linters / Tests (1.20) (pull_request) Successful in 4m1s
Badger implementation isn't tested and works not well,
but requires human resources to maintain.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-22 13:00:54 +03:00
d9cbb16bd3
[ #866 ] Use TTL for blobovnicza tree cache
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-12-19 16:36:28 +00:00
ef07c1a3c9
[ #552 ] Add sysd notifications to storage service
...
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2023-12-13 17:51:41 +03:00
db49ad16cc
[ #826 ] blobovniczatree: Do not create DB's on init
...
DCO action / DCO (pull_request) Successful in 4m10s
Build / Build Components (1.21) (pull_request) Successful in 7m30s
Vulncheck / Vulncheck (pull_request) Successful in 7m5s
Tests and linters / Staticcheck (pull_request) Successful in 10m6s
Tests and linters / Lint (pull_request) Successful in 10m26s
Build / Build Components (1.20) (pull_request) Successful in 12m39s
Tests and linters / Tests (1.20) (pull_request) Successful in 16m55s
Tests and linters / Tests (1.21) (pull_request) Successful in 17m13s
Tests and linters / Tests with -race (pull_request) Successful in 17m14s
Blobovniczas will be created on write requests.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
f1c7905263
[ #661 ] blobovniczatree: Make Rebuild concurrent
...
Different DBs can be rebuild concurrently.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
da4fee2d0b
[ #698 ] blobovniczatree: Init blobovniczas concurrently
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
07390ad4e3
[ #715 ] node: Unify config parameter names
...
DCO action / DCO (pull_request) Successful in 2m54s
Vulncheck / Vulncheck (pull_request) Successful in 3m11s
Tests and linters / Staticcheck (pull_request) Successful in 3m55s
Build / Build Components (1.21) (pull_request) Successful in 3m51s
Build / Build Components (1.20) (pull_request) Successful in 4m6s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m10s
Tests and linters / Lint (pull_request) Successful in 5m24s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m56s
Tests and linters / Tests with -race (pull_request) Successful in 7m46s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 17:13:50 +03:00
c80b46fad3
[ #754 ] blobstor: Estimate compressability
...
DCO action / DCO (pull_request) Successful in 1m59s
Vulncheck / Vulncheck (pull_request) Successful in 3m31s
Build / Build Components (1.20) (pull_request) Successful in 4m37s
Build / Build Components (1.21) (pull_request) Successful in 4m33s
Tests and linters / Tests (1.20) (pull_request) Successful in 4m54s
Tests and linters / Staticcheck (pull_request) Successful in 4m49s
Tests and linters / Tests with -race (pull_request) Successful in 5m9s
Tests and linters / Lint (pull_request) Successful in 6m4s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m9s
Now it is possible to enable compressability estimation.
If data is likely uncompressable, it should reduce CPU time and memory.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-01 11:24:32 +03:00
ae81d6660a
[ #529 ] objectcore: Fix object content validation
...
There are old objects where the owner of the object
may not match the one who issued the token.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-29 10:33:06 +03:00
d3904ec599
[ #602 ] config: Add blobovnicza leaf width parameter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-21 10:27:32 +03:00
1a0cb0f34a
[ #421 ] Try using badger for the write-cache
...
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-07 08:16:57 +00:00
32c77f3a23
[ #537 ] node: Add runtime.memory_limit config parameter
...
Build / Build Components (1.20) (pull_request) Successful in 4m6s
Build / Build Components (1.19) (pull_request) Successful in 4m18s
ci/woodpecker/pr/pre-commit Pipeline failed
Tests and linters / Lint (pull_request) Failing after 2m10s
Tests and linters / Tests with -race (pull_request) Has started running
Tests and linters / Tests (1.19) (pull_request) Successful in 3m58s
Tests and linters / Tests (1.20) (pull_request) Successful in 3m44s
Vulncheck / Vulncheck (pull_request) Failing after 3m51s
Tests and linters / Staticcheck (pull_request) Successful in 8m3s
ci/woodpecker/push/pre-commit Pipeline was successful
This parameter allows to set soft memory limit for Go GC.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-25 17:27:09 +03:00
d8e37a827f
[ #497 ] config: Add examples and unit tests
...
ci/woodpecker/push/pre-commit Pipeline was successful
Add examples and unit tests for tree.authorized_keys section.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-07-17 08:23:28 +00:00
Pavel Karpy
14c35d776e
[ #39 ] node: Add optional profilers
...
Include settings for block and mutex profilers.
They are disabled by default, as in Go runtime itself.
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-05-12 15:49:04 +03:00
d757d881d0
[ #249 ] node: Drop subnet from config
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-17 09:46:02 +03:00
Pavel Karpy
960e3c219e
[ #247 ] config, doc: Drop reputation references
...
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-04-14 14:44:14 +03:00
8d2f443868
[ #238 ] Fix linter error
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-04-14 10:19:10 +00:00
5af9f58469
[ #135 ] tracing: Add tracing to node gRPC endpoints
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-12 06:52:00 +00:00
5059dcc19d
[ #145 ] shard-gc: Delete expired objects after locks
...
GC deletes expired locks and objects sequentially. Expired locks and
objects are now being deleted concurrently in batches. Added a config
parameter that controls the number of concurrent workers and batch size.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-21 11:31:08 +03:00
73bb590cb1
[ #64 ] node: Use pool_size_local and separate pool for local puts
...
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-02-22 13:43:19 +03:00
351fdd9fa2
[ #2246 ] node: Allow to configure tombsone lifetime
...
Currently, DELETE service sets tombstone expiration epoch to
`current epoch + 5`. This works less than ideal in private networks
where an epoch can be e.g. 10 minutes. In this case, after a node is
unavailable for more than 1 hour, already deleted objects have a chance
to reappear.
After this commit tombstone lifetime can be configured.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
1858f11146
[ #1 ] Fix viper env prefix in cli tools
...
Changing env prefix and corresponding example config files.
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
2023-02-06 17:41:14 +03:00
741482c26f
[ #2164 ] neofs-node: Allow to set reconnect interval
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
d3054e577a
[ #2165 ] services/tree: Allow to set custom synchronization interval
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2022-12-30 11:07:35 +03:00
Pavel Karpy
306609030a
[ #2159 ] node: Add tree replication timeout configuration
...
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-30 11:07:35 +03:00
Pavel Karpy
923f84722a
Move to frostfs-node
...
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-28 15:04:29 +03:00
Evgenii Stratonikov
7ef0303e13
[ #2003 ] neofs-node: Allow to configure replicator pool size
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
98a152256b
[ #1992 ] writecache: Allow to open in NOSYNC mode
...
Applicable only to FSTree as we cannot handle corrupted databases
properly yet.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-01 09:42:26 +03:00
Evgenii Stratonikov
b6930f2219
[ #1992 ] neofs-node: Allow to open fstree in NOSYNC mode
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-01 09:42:26 +03:00
Leonard Lyubich
eb206d6e59
[ #1883 ] config/example: Actualize desc of shard modes
...
Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
2022-10-26 08:24:40 +03:00
Pavel Karpy
9a0824766c
[ #1713 ] node: Add Neo RPC switch interval to the config
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-10-25 17:38:48 +03:00
Evgenii Stratonikov
ca8dc872b2
[ #1846 ] neofs-node: Make morph.cache_ttl equal to block time by default
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-07 10:34:55 +03:00
Evgenii Stratonikov
236414df49
[ #1817 ] network: Allow to use network addresses from the iterator
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-04 12:12:18 +03:00
