TrueCloudLab
/
frostfs-node
18
1
Fork 20
Code Issues 88 Pull Requests 8 Actions Releases Activity
4,676 Commits
7 Branches
19 Tags
47 MiB
Commit Graph

254 Commits (v0.39.0)

Author SHA1 Message Date
Dmitrii Stepanov c4e1d8eb07 [#602] node: Fix blobovnicza typos 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-18 11:14:10 +03:00
Dmitrii Stepanov 2e49d7ea7e [#602] blobovnicza: Init before using 
Fix blobovnicza size: after restart size metric resets.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-17 19:17:35 +00:00
Evgenii Stratonikov f7042c5a6f [#609] Replace zaptest.NewLogger() with zap.L() 
Semantic patch:
```
@@
@@
-import "go.uber.org/zap/zaptest"
+import "go.uber.org/zap"

-zaptest.NewLogger(t)
+zap.L()
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-17 16:51:10 +00:00
Evgenii Stratonikov a8de37c8a2 [#607] *: Remove redundant if on error returns 
Semantic patch:
```
@@
@@
-if err != nil { return err }
-return nil
+return err
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-17 16:50:02 +00:00
Dmitrii Stepanov 58c8722c81 [#585] fstree: Add optional file counter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-16 14:47:44 +03:00
Dmitrii Stepanov baad49990c [#585] fstree: Return logical error if object deleted 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-16 14:47:44 +03:00
Dmitrii Stepanov 0c52186572 [#585] fstree: Remove unused method 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-16 14:47:44 +03:00
Alejandro Lopez ae8be495c8 [#xx] Avoid manual management of files in tests
Vulncheck / Vulncheck (pull_request) Successful in 2m36s Details
Build / Build Components (1.21) (pull_request) Successful in 3m33s Details
DCO action / DCO (pull_request) Successful in 3m33s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m15s Details
Tests and linters / Lint (pull_request) Successful in 5m31s Details
Build / Build Components (1.20) (pull_request) Successful in 7m2s Details
Tests and linters / Tests with -race (pull_request) Failing after 8m51s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m55s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 10m6s Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-14 14:01:39 +03:00
Evgenii Stratonikov b44a8dd46c [#597] *: Fix linter warnings
Vulncheck / Vulncheck (pull_request) Successful in 6m26s Details
Tests and linters / Staticcheck (pull_request) Successful in 7m20s Details
Build / Build Components (1.21) (pull_request) Successful in 8m31s Details
Build / Build Components (1.20) (pull_request) Successful in 8m43s Details
Tests and linters / Tests (1.21) (pull_request) Failing after 15m48s Details
Tests and linters / Tests (1.20) (pull_request) Failing after 16m2s Details
Tests and linters / Tests with -race (pull_request) Failing after 16m28s Details
Tests and linters / Lint (pull_request) Successful in 19m8s Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-11 09:55:51 +03:00
Alejandro Lopez 8f994163ee [#586] Fix writecache benchmarks and refactor hacky NeedsCompression 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-10 08:05:18 +00:00
Evgenii Stratonikov d641cba2fc [#587] Do not use math/rand.Read
Build / Build Components (1.20) (pull_request) Failing after 3s Details
Build / Build Components (1.21) (pull_request) Failing after 3s Details
Tests and linters / Lint (pull_request) Failing after 3s Details
Tests and linters / Tests (1.20) (pull_request) Failing after 3s Details
Tests and linters / Tests (1.21) (pull_request) Failing after 3s Details
Tests and linters / Tests with -race (pull_request) Failing after 5s Details
Tests and linters / Staticcheck (pull_request) Failing after 8s Details
Vulncheck / Vulncheck (pull_request) Failing after 2s Details 
Fix staticcheck warnings after go1.20 update.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-09 16:02:44 +03:00
Alejandro Lopez 5b7e4a51b7 [#481] Update frostfs-sdk-go and error pointer receivers 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-09 10:26:53 +00:00
Alejandro Lopez 65c72f3e0b [#559] Remove manual path handling in fstree tests
Build / Build Components (1.19) (pull_request) Successful in 2m8s Details
Build / Build Components (1.20) (pull_request) Successful in 1m54s Details
Tests and linters / Lint (pull_request) Successful in 2m48s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 1m53s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 2m4s Details
Tests and linters / Tests with -race (pull_request) Successful in 5m21s Details
Tests and linters / Staticcheck (pull_request) Successful in 2m16s Details
Vulncheck / Vulncheck (pull_request) Failing after 16m3s Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-03 10:03:41 +03:00
Dmitrii Stepanov 4d48377cec [#459] blobovniczatree: Fix get error
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 16:44:53 +03:00
Dmitrii Stepanov af608da952 [#373] metrics: Add blobovnizca metrics 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 15:13:26 +03:00
Dmitrii Stepanov d8ecc69d00 [#373] local storage: Pass parent ID 
This is required to add shard ID as metric label.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 15:13:26 +03:00
Dmitrii Stepanov f54cc0b607 [#373] blobstor: Add metrics 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 15:13:26 +03:00
Dmitrii Stepanov 8318d90ad0 [#373] blobovniczatree: Add metrics 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 15:13:26 +03:00
Dmitrii Stepanov 3ae3c8dfdb [#373] fstree: Add metrics 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 15:13:26 +03:00
Dmitrii Stepanov a8526d45e9 [#373] blobovnizca: Add missed/fix tracing spans 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 15:13:26 +03:00
Evgenii Stratonikov 41ab4d070e [#423] *: Use hrw.StringHash() where possible
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-13 07:18:25 +00:00
Dmitrii Stepanov 74578052f9 [#412] node: Replace tracing package 
Use observability module.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-01 13:23:11 +00:00
Anton Nikiforov 8dcd06c587 [#394] node: Use `Context` in `Blobovniczas.Iterate()`
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-05-31 10:09:18 +00:00
Anton Nikiforov 802168c0c6 [#364] node: Stop flushing big object when termination signal received
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-05-26 16:46:58 +03:00
Alejandro Lopez 973af12854 [#327] tests: replace os.MkdirTemp with t.TempDir
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-05-05 11:24:51 +00:00
Evgenii Stratonikov a1b4ba9980 [#209] compression: Do not store uncompressible data 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-03 16:28:29 +03:00
Evgenii Stratonikov f07e2d4812 [#285] lint: Fix revive/unused-parameter 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-26 17:35:04 +03:00
Airat Arifullin 6c90bb87f1 [#118] node: add ctx for unit tests for blobstor
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-19 18:09:33 +03:00
Airat Arifullin 13c8afcb02 [#118] node: add unit concurrent tests for blobstor
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-19 10:22:50 +00:00
Pavel Karpy 262c9c2b93 [#256] blobovniczaTree: Make `Exists` test stable
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Corrupt and request _the same_ file.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-14 16:46:24 +03:00
Dmitrii Stepanov d62c6e4ce6 [#242] node: Add tracing spans 
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-14 10:25:53 +00:00
Alejandro Lopez 41eb3129ae [#139] Refactor blobovnicza exist test to not use chmod
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-04-14 12:16:14 +03:00
Evgenii Stratonikov 0e31c12e63 [#240] logs: Move log messages to constants 
Drop duplicate entities.
Format entities.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-14 05:06:09 +00:00
Dmitrii Stepanov 0920d848d0 [#135] get-object: Add tracing spans 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Evgenii Stratonikov c4865783fc [#236] blobstor/test: Prefill storage in parallel in read benchmark
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
`blobovniczatree` takes a really long time to prefill, because each
batch takes at least 10ms, so for 10k iterations we have at least 100s of
prefill.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-11 20:52:11 +03:00
Evgenii Stratonikov c85a0bc866 [#236] blobstor/test: Reduce test descriptions 
I tried to add 4 more tests and suddenly, it became harder to navigate in
code. Move directory creation in a common function.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-11 20:52:04 +03:00
Evgenii Stratonikov 760af6b912 [#211] fstree: Consider ENOSPC a logical error 
We already do this for file writing, however directory creation can also
fail.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-05 11:10:39 +00:00
Dmitrii Stepanov 8273a3dfb2 [#188] blobstor: Refactor blobstor test 
Resolve funlen linter for TestIterate function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 14:50:43 +03:00
Dmitrii Stepanov 594b5821ed [#188] blobstor: Refactor put data to blobovniczas 
Resolve funlen linter for Blobovniczas.Put method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 14:50:43 +03:00
Alejandro Lopez 341fe1688f [#139] test: Add test storage implementation
ci/woodpecker/push/pre-commit Pipeline was successful Details 
This aims to reduce the usage of chmod hackery to induce or simulate
OS-related failures.

Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-03-29 14:28:49 +00:00
Airat Arifullin 221203beeb [#180] node: Refactor panics in unit test
ci/woodpecker/push/pre-commit Pipeline was successful Details 
* Replace panics in unit tests by require.NoError and t.Fatalf

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-03-29 12:39:07 +03:00
Airat Arifullin 9808dec591 [#86] node: Move testing utils to one package
ci/woodpecker/push/pre-commit Pipeline failed Details 
Move testing utils from tests in local_object_storage package to
unified testutil package

Signed-off-by: Airat Arifullin <aarifullin@yadro.com>
 2023-03-23 08:19:15 +00:00
Pavel Karpy da8da1c63a [#98] fstree: Do not fail iteration over just removed files
ci/woodpecker/push/pre-commit Pipeline was successful Details 
A directory is read and files are saved to a local variable. The iteration
over such files may lead to a non-existing files reading due to a normal SN
operation cycle and, therefore, may lead to a returning the OS error to a
caller. Skip just removed (or lost) files as the golang std library does in
similar situations:
5f1a0320b9/src/os/dir_unix.go (L128-L133).

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-22 07:26:27 +00:00
Dmitrii Stepanov 97c36ed3ec [#148] linter: Add funlen linter 
Long functions are hard to understand and source of errors

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:54:41 +03:00
Dmitrii Stepanov 2dc86058c3 [#148] memstore: Drop space line 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:52:39 +03:00
Alejandro Lopez 724debfdcd [#81] node: Add basic read/write benchmarks for substorages 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-03-15 16:37:04 +00:00
Alexey Vanin 20de74a505 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 16:38:26 +03:00
Anton Nikiforov e9f3c24229 [#65] Use `strings.Cut` instead of `strings.Split*` where possible 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-28 13:39:14 +03:00
Evgenii Stratonikov 315141dc2c [#2252] fstree: Allow concurrent writes 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-20 13:53:27 +03:00
Evgenii Stratonikov 204cd3a11c [#31] fstree: Optimize `treePath` 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-10 12:49:31 +03:00
Evgenii Stratonikov dee4498c1e [#31] fstree: Do not check for a file existence twice 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-10 12:49:31 +03:00
Evgenii Stratonikov abbecf49d6 [#31] fstree: Speedup string-to-address conversion 
```
name                  old time/op    new time/op    delta
_addressFromString-8    1.25µs ±30%    1.02µs ± 6%  -18.49%  (p=0.000 n=9+9)

name                  old alloc/op   new alloc/op   delta
_addressFromString-8      352B ± 0%      256B ± 0%  -27.27%  (p=0.000 n=9+10)

name                  old allocs/op  new allocs/op  delta
_addressFromString-8      6.00 ± 0%      4.00 ± 0%  -33.33%  (p=0.000
n=10+10)
```

Also, assure compiler that `s` doesn't escape:
Before this commit:
```
./fstree.go:74:24: leaking param: s
./fstree.go:90:6: moved to heap: addr
```

After this commit:
```
./fstree.go:74:24: s does not escape
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-10 12:49:31 +03:00
Stanislav Bogatyrev cb016d53a6 [#1] Fix comments and error messages 
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
 2023-02-06 17:41:14 +03:00
Pavel Karpy 73bc1b0b68 [#38] node: Fix linter warnings 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-02-06 17:27:54 +03:00
Evgenii Stratonikov 87f0e3ea25 [#2208] fstree: Rename file after write 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 792319a044 [#2208] fstree: Remove file if there was an error during write 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 9009612a82 [#2198] blobovniczatree: Properly handle concurrent active blobovnicza update 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov 9936b112b8 [#5] blobstor: Use generic LRU cache 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-31 23:04:06 +03:00
Evgenii Stratonikov 0272218eb9 [#2184] compression: Properly calculate upper bound 
If the data is not compressible allocating `len(data)` will lead to a
slice reallocation. For a compressible data the results for small size
are flaky and we allocate a bit more. However, it feels right to use a
provided function if we need to pick any size at all.

```
name                                                           old time/op    new time/op    delta
Compression/size=128/zeroed_slice-8                              2.23µs ±12%    2.06µs ± 6%   -7.35%  (p=0.009 n=10+10)
Compression/size=128/not_so_random_slice_(block_=_123)-8         19.0µs ±10%    15.8µs ±16%  -17.09%  (p=0.000 n=9+10)
Compression/size=128/random_slice-8                              17.6µs ±15%    16.1µs ±16%     ~     (p=0.075 n=10+10)
Compression/size=1024/zeroed_slice-8                             3.05µs ±11%    2.84µs ±10%     ~     (p=0.089 n=10+10)
Compression/size=1024/not_so_random_slice_(block_=_123)-8        18.1µs ± 6%    18.2µs ±12%     ~     (p=0.971 n=10+10)
Compression/size=1024/random_slice-8                             48.6µs ± 6%    45.6µs ± 5%   -6.07%  (p=0.006 n=10+9)
Compression/size=32768/zeroed_slice-8                            26.8µs ± 3%    28.7µs ± 8%   +7.23%  (p=0.001 n=10+10)
Compression/size=32768/not_so_random_slice_(block_=_123)-8       44.3µs ± 8%    43.7µs ±13%     ~     (p=0.762 n=8+10)
Compression/size=32768/random_slice-8                            97.3µs ±32%    68.9µs ±15%  -29.13%  (p=0.000 n=10+10)
Compression/size=33554432/zeroed_slice-8                         29.8ms ± 9%    30.3ms ±17%     ~     (p=1.000 n=9+9)
Compression/size=33554432/not_so_random_slice_(block_=_123)-8    33.1ms ±14%    30.3ms ±11%   -8.61%  (p=0.043 n=10+10)
Compression/size=33554432/random_slice-8                         41.7ms ± 3%    30.1ms ± 8%  -27.72%  (p=0.000 n=9+10)

name                                                           old alloc/op   new alloc/op   delta
Compression/size=128/zeroed_slice-8                                128B ± 0%      144B ± 0%  +12.50%  (p=0.000 n=10+10)
Compression/size=128/not_so_random_slice_(block_=_123)-8           384B ± 0%      144B ± 0%  -62.50%  (p=0.000 n=10+10)
Compression/size=128/random_slice-8                                384B ± 0%      144B ± 0%  -62.50%  (p=0.000 n=10+10)
Compression/size=1024/zeroed_slice-8                             1.02kB ± 0%    1.15kB ± 0%  +12.50%  (p=0.000 n=10+10)
Compression/size=1024/not_so_random_slice_(block_=_123)-8        1.02kB ± 0%    1.15kB ± 0%  +12.50%  (p=0.000 n=10+10)
Compression/size=1024/random_slice-8                             2.56kB ± 0%    1.15kB ± 0%  -55.00%  (p=0.000 n=10+10)
Compression/size=32768/zeroed_slice-8                            32.8kB ± 0%    41.0kB ± 0%  +25.00%  (p=0.000 n=10+10)
Compression/size=32768/not_so_random_slice_(block_=_123)-8       32.8kB ± 0%    41.0kB ± 0%  +25.00%  (p=0.000 n=10+10)
Compression/size=32768/random_slice-8                            81.9kB ± 0%    41.0kB ± 0%  -50.00%  (p=0.000 n=10+10)
Compression/size=33554432/zeroed_slice-8                         33.6MB ± 0%    33.6MB ± 0%   +0.02%  (p=0.000 n=9+9)
Compression/size=33554432/not_so_random_slice_(block_=_123)-8    33.6MB ± 0%    33.6MB ± 0%   +0.02%  (p=0.000 n=8+10)
Compression/size=33554432/random_slice-8                         75.5MB ± 0%    33.6MB ± 0%  -55.55%  (p=0.000 n=10+10)

name                                                           old allocs/op  new allocs/op  delta
Compression/size=128/zeroed_slice-8                                1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=128/not_so_random_slice_(block_=_123)-8           2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
Compression/size=128/random_slice-8                                2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
Compression/size=1024/zeroed_slice-8                               1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=1024/not_so_random_slice_(block_=_123)-8          1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=1024/random_slice-8                               2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
Compression/size=32768/zeroed_slice-8                              1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=32768/not_so_random_slice_(block_=_123)-8         1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=32768/random_slice-8                              2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
Compression/size=33554432/zeroed_slice-8                           1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=33554432/not_so_random_slice_(block_=_123)-8      1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=33554432/random_slice-8                           2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov 0ace28e43d [#2175] blobovniczatree: Close all non-active blobovniczas 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov c1cf418956 [#2175] blobovniczatree: Make function parameters more descriptive 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Pavel Karpy 923f84722a Move to frostfs-node 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-28 15:04:29 +03:00
Evgenii Stratonikov 63f604e948 [#2068] blobstor: Allow to provide storage ID in `Exists` 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-12-02 11:52:05 +03:00
Evgenii Stratonikov ae7b473768 [#2064] blobovniczatree: Remove `index too big` log 
There is no need to log about a situation which is expected.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-30 16:53:18 +03:00
Pavel Karpy fa231b8c56 [#2057] blobstor: Block operations on a mode change 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00
Evgenii Stratonikov d65604ad30 [#1985] blobstor: Allow to report multiple errors to caller 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-19 11:01:04 +03:00
Evgenii Stratonikov f564430b90 [#1992] fstree: Allow working in SYNC mode 
Make O_SYNC the default and allow to opt-out explicitly.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-01 09:42:26 +03:00
Evgenii Stratonikov 56de2f1363 [#1969] local_object_storage: Simplify logic error construction 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-31 11:41:24 +03:00
Evgenii Stratonikov fcdbf5e509 [#1969] local_object_storage: Add a type for logical errors 
All logic errors are wrapped in `logicerr.Logical` type and do not
affect shard error counter.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-31 11:41:24 +03:00
Pavel Karpy 41f93dcc1d [#1953] blobstor: Log any object removal operation 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-24 21:52:18 +03:00
Evgenii Stratonikov 1cb892c579 [#1910] .golangci.yml: Add `misspell` linker 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-18 15:08:26 +03:00
Pavel Karpy f037022a7a [#1770] logger: Refactor `Logger` component 
Make it store its internal `zap.Logger`'s level. Also, make all the
components to accept internal `logger.Logger` instead of `zap.Logger`; it
will simplify future refactor.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-12 18:11:05 +03:00
Evgenii Stratonikov 4b005d3178 [#1840] blobstor: Return info about all components 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Evgenii Stratonikov 6d7ffefec5 [#1840] blobstor/test: Add tests for auxiliary functions 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Evgenii Stratonikov 328691c94f [#1840] blobstor: Return `Path` from components 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Evgenii Stratonikov 2e3ef817f4 [#1819] engine: Increase error counter for PUT errors 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-04 10:11:52 +03:00
Evgenii Stratonikov b89e71fa78 [#1819] common: Add `ErrNoSpace` 
Add a common error for this case because it is not an error
which should increase error counter. Single error simplifies checks on
the call-site.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-04 10:11:52 +03:00
Pavel Karpy 5c69e19016 [#1770] fstree: Depth parameter `int` -> `uint64` 
Negative values have no sense. On the other hand it differs from the
blobovnicza's configuration and prevents unification.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Evgenii Stratonikov 6f45cc81fc [#1764] neofs-node: Use constants for storage types 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-27 08:57:20 +03:00
Evgenii Stratonikov 7cc68cf4d4 [#1810] blobovniczatree: Add a test for blobovnicva usage 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-23 10:19:26 +04:00
Evgenii Stratonikov 60b2930417 [#1778] neofs-node: Fix blobovnicza config type 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-12 17:06:00 +03:00
Evgenii Stratonikov d8a00c365a [#1691] blobovniczatree: Fix active blobovnicza caching 
Maintain an invariant that any blobovnicza is present either
in `opened` or in `active` map. Otherwise, the logic becomes too
complicate because it is not obvious when we should close the blobovnicza.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-31 19:11:48 +03:00
Evgenii Stratonikov dfac4e1c0b [#1691] blobovniczatree: Do not fill cache during initialization 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-31 19:11:48 +03:00
Evgenii Stratonikov 78bf17c0b0 [#1686] blobstor/*: Remove unneeded tests 
The blobovniczatree test also checks the amount of objects we can put,
so leave it here.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov b9a2055e1c [#1686] blobstor/*: Return ErrReadOnly for modifying operations 
This check should occur on the shard level, but because
blobstor components expose `Open(readOnly bool)` interface,
it is reasonable to expect an error here.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov 6c2d3b020f [#1686] fstree: Return proper error for concurrent `Delete` 
If the file doesn't exist, return `apistatus.ObjectNotFound`.
First check is still there as a shortcut.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov 0b95a21701 [#1686] blobstor: Add generic tests 
This tests check that each blobstor component behaves similarly when
same methods are being used. It is intended to serve as a specification
for all future components.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov b2d4cc556e [#1686] blobovnicza: Remove `IterateObjects` 
It is used only once, makes sense to inline.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov 5f95498364 [#1686] blobstor: Add generic tests 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov e8c6dce466 [#1523] blobstor: Rename `compression.CConfig` to `Config` 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov 0042c1d4a6 [#1584] blobovniczatree: move public methods to different files 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov 26b4a258e0 [#1523] neofs-node: Refactor configuration 
1. Move compression parameters to the `shard` section.
2. Allow to use multiple sub-storage components in the blobstor.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov 13cdbde2e2 [#1523] blobstor: Remove unused types 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov 0c9e4e6a35 [#1523] blobstor: Unify request dispatch logic 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov 266458fe5c [#1523] blobstor: Unify fstree and blobovnicza interfaces 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov 4176b2a1bc [#1523] local_object_storage: Unify parameters for the `Iterate` operation 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov 73f8bb3e5f [#1523] shard: Store generic storage ID in metabase 
Allow to extend blobstor with more storage sub-systems. Currently
objects stored in the FSTree have empty byte slice descriptor and object
from blobovnicza tree have the same id as earlier. Each such change in
the identifier formation should be accompanied with metabase version
increase.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov 9eb018672c [#1523] blobstor: Unify parameters for `Put` operation 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov 7d6df543d7 [#1523] blobstor: Export single `Get`, `GetRange` and `Delete` methods 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov ca15083a50 [#1523] local_object_storage: Unify parameters for the `Exists` operation 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00