AlekseySVTN/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Pull requests Activity
4749 commits 9 branches 70 tags 42 MiB
Commit graph

257 commits

Author SHA1 Message Date
Evgenii Stratonikov
96e690883f [#638] Unify test loggers 
In some places we have debug=false, in others debug=true.
Let's be consistent.

Semantic patch:
```
@@
@@
-test.NewLogger(..., false)
+test.NewLogger(..., true)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-23 11:21:05 +00:00
Evgenii Stratonikov
322c1dc273 [#638] Use test.NewLogger() in tests 
Semantic patch (restricted to **/*_test.go):
```
@@
@@
+import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger/test"
-import "go.uber.org/zap"
-import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger"

-&logger.Logger{Logger: zap.L()}
+test.NewLogger(t, false)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-23 11:21:05 +00:00
Dmitrii Stepanov
354a92ea2c [#602] blobovnicza: Add leaf width implementation 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-21 10:27:32 +03:00
Dmitrii Stepanov
c4e1d8eb07 [#602] node: Fix blobovnicza typos 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-18 11:14:10 +03:00
Dmitrii Stepanov
2e49d7ea7e [#602] blobovnicza: Init before using 
Fix blobovnicza size: after restart size metric resets.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-17 19:17:35 +00:00
Evgenii Stratonikov
f7042c5a6f [#609] Replace zaptest.NewLogger() with zap.L() 
Semantic patch:
```
@@
@@
-import "go.uber.org/zap/zaptest"
+import "go.uber.org/zap"

-zaptest.NewLogger(t)
+zap.L()
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-17 16:51:10 +00:00
Evgenii Stratonikov
a8de37c8a2 [#607] *: Remove redundant if on error returns 
Semantic patch:
```
@@
@@
-if err != nil { return err }
-return nil
+return err
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-17 16:50:02 +00:00
Dmitrii Stepanov
58c8722c81 [#585] fstree: Add optional file counter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-16 14:47:44 +03:00
Dmitrii Stepanov
baad49990c [#585] fstree: Return logical error if object deleted 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-16 14:47:44 +03:00
Dmitrii Stepanov
0c52186572 [#585] fstree: Remove unused method 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-16 14:47:44 +03:00
Alejandro Lopez
ae8be495c8 [#xx] Avoid manual management of files in tests 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-14 14:01:39 +03:00
Evgenii Stratonikov
b44a8dd46c [#597] *: Fix linter warnings 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-11 09:55:51 +03:00
Alejandro Lopez
8f994163ee [#586] Fix writecache benchmarks and refactor hacky NeedsCompression 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-10 08:05:18 +00:00
Evgenii Stratonikov
d641cba2fc [#587] Do not use math/rand.Read 
Fix staticcheck warnings after go1.20 update.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-09 16:02:44 +03:00
Alejandro Lopez
5b7e4a51b7 [#481] Update frostfs-sdk-go and error pointer receivers 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-09 10:26:53 +00:00
Alejandro Lopez
65c72f3e0b [#559] Remove manual path handling in fstree tests 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-03 10:03:41 +03:00
Dmitrii Stepanov
4d48377cec [#459] blobovniczatree: Fix get error 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 16:44:53 +03:00
Dmitrii Stepanov
af608da952 [#373] metrics: Add blobovnizca metrics 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
d8ecc69d00 [#373] local storage: Pass parent ID 
This is required to add shard ID as metric label.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
f54cc0b607 [#373] blobstor: Add metrics 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
8318d90ad0 [#373] blobovniczatree: Add metrics 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
3ae3c8dfdb [#373] fstree: Add metrics 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 15:13:26 +03:00
Dmitrii Stepanov
a8526d45e9 [#373] blobovnizca: Add missed/fix tracing spans 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-21 15:13:26 +03:00
Evgenii Stratonikov
41ab4d070e [#423] *: Use hrw.StringHash() where possible 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-13 07:18:25 +00:00
Dmitrii Stepanov
74578052f9 [#412] node: Replace tracing package 
Use observability module.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-01 13:23:11 +00:00
Anton Nikiforov
8dcd06c587 [#394] node: Use Context in Blobovniczas.Iterate() 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-05-31 10:09:18 +00:00
Anton Nikiforov
802168c0c6 [#364] node: Stop flushing big object when termination signal received 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-05-26 16:46:58 +03:00
Alejandro Lopez
973af12854 [#327] tests: replace os.MkdirTemp with t.TempDir 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-05-05 11:24:51 +00:00
Evgenii Stratonikov
a1b4ba9980 [#209] compression: Do not store uncompressible data 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-03 16:28:29 +03:00
Evgenii Stratonikov
f07e2d4812 [#285] lint: Fix revive/unused-parameter 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-26 17:35:04 +03:00
Airat Arifullin
6c90bb87f1 [#118] node: add ctx for unit tests for blobstor 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-19 18:09:33 +03:00
Airat Arifullin
13c8afcb02 [#118] node: add unit concurrent tests for blobstor 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
 2023-04-19 10:22:50 +00:00
Pavel Karpy
262c9c2b93 [#256] blobovniczaTree: Make Exists test stable 
Corrupt and request _the same_ file.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-04-14 16:46:24 +03:00
Dmitrii Stepanov
d62c6e4ce6 [#242] node: Add tracing spans 
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-14 10:25:53 +00:00
Alejandro Lopez
41eb3129ae [#139] Refactor blobovnicza exist test to not use chmod 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-04-14 12:16:14 +03:00
Evgenii Stratonikov
0e31c12e63 [#240] logs: Move log messages to constants 
Drop duplicate entities.
Format entities.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-14 05:06:09 +00:00
Dmitrii Stepanov
0920d848d0 [#135] get-object: Add tracing spans 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Evgenii Stratonikov
c4865783fc [#236] blobstor/test: Prefill storage in parallel in read benchmark 
`blobovniczatree` takes a really long time to prefill, because each
batch takes at least 10ms, so for 10k iterations we have at least 100s of
prefill.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-11 20:52:11 +03:00
Evgenii Stratonikov
c85a0bc866 [#236] blobstor/test: Reduce test descriptions 
I tried to add 4 more tests and suddenly, it became harder to navigate in
code. Move directory creation in a common function.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-11 20:52:04 +03:00
Evgenii Stratonikov
760af6b912 [#211] fstree: Consider ENOSPC a logical error 
We already do this for file writing, however directory creation can also
fail.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-05 11:10:39 +00:00
Dmitrii Stepanov
8273a3dfb2 [#188] blobstor: Refactor blobstor test 
Resolve funlen linter for TestIterate function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 14:50:43 +03:00
Dmitrii Stepanov
594b5821ed [#188] blobstor: Refactor put data to blobovniczas 
Resolve funlen linter for Blobovniczas.Put method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 14:50:43 +03:00
Alejandro Lopez
341fe1688f [#139] test: Add test storage implementation 
This aims to reduce the usage of chmod hackery to induce or simulate
OS-related failures.

Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-03-29 14:28:49 +00:00
Airat Arifullin
221203beeb [#180] node: Refactor panics in unit test 
* Replace panics in unit tests by require.NoError and t.Fatalf

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-03-29 12:39:07 +03:00
Ayrat Arifullin
9808dec591 [#86] node: Move testing utils to one package 
Move testing utils from tests in local_object_storage package to
unified testutil package

Signed-off-by: Airat Arifullin <aarifullin@yadro.com>
 2023-03-23 08:19:15 +00:00
Pavel Karpy
da8da1c63a [#98] fstree: Do not fail iteration over just removed files 
A directory is read and files are saved to a local variable. The iteration
over such files may lead to a non-existing files reading due to a normal SN
operation cycle and, therefore, may lead to a returning the OS error to a
caller. Skip just removed (or lost) files as the golang std library does in
similar situations:
5f1a0320b9/src/os/dir_unix.go (L128-L133).

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-22 07:26:27 +00:00
Dmitrii Stepanov
97c36ed3ec [#148] linter: Add funlen linter 
Long functions are hard to understand and source of errors

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:54:41 +03:00
Dmitrii Stepanov
2dc86058c3 [#148] memstore: Drop space line 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:52:39 +03:00
Alejandro Lopez
724debfdcd [#81] node: Add basic read/write benchmarks for substorages 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-03-15 16:37:04 +00:00
Alex Vanin
20de74a505 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 16:38:26 +03:00
Anton Nikiforov
e9f3c24229 [#65] Use strings.Cut instead of strings.Split* where possible 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-28 13:39:14 +03:00
Evgenii Stratonikov
315141dc2c [#2252] fstree: Allow concurrent writes 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-20 13:53:27 +03:00
Evgenii Stratonikov
204cd3a11c [#31] fstree: Optimize treePath 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-10 12:49:31 +03:00
Evgenii Stratonikov
dee4498c1e [#31] fstree: Do not check for a file existence twice 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-10 12:49:31 +03:00
Evgenii Stratonikov
abbecf49d6 [#31] fstree: Speedup string-to-address conversion 
```
name                  old time/op    new time/op    delta
_addressFromString-8    1.25µs ±30%    1.02µs ± 6%  -18.49%  (p=0.000 n=9+9)

name                  old alloc/op   new alloc/op   delta
_addressFromString-8      352B ± 0%      256B ± 0%  -27.27%  (p=0.000 n=9+10)

name                  old allocs/op  new allocs/op  delta
_addressFromString-8      6.00 ± 0%      4.00 ± 0%  -33.33%  (p=0.000
n=10+10)
```

Also, assure compiler that `s` doesn't escape:
Before this commit:
```
./fstree.go:74:24: leaking param: s
./fstree.go:90:6: moved to heap: addr
```

After this commit:
```
./fstree.go:74:24: s does not escape
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-10 12:49:31 +03:00
Stanislav Bogatyrev
cb016d53a6 [#1] Fix comments and error messages 
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
 2023-02-06 17:41:14 +03:00
Pavel Karpy
73bc1b0b68 [#38] node: Fix linter warnings 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-02-06 17:27:54 +03:00
Evgenii Stratonikov
87f0e3ea25 [#2208] fstree: Rename file after write 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov
792319a044 [#2208] fstree: Remove file if there was an error during write 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov
9009612a82 [#2198] blobovniczatree: Properly handle concurrent active blobovnicza update 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov
9936b112b8 [#5] blobstor: Use generic LRU cache 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-31 23:04:06 +03:00
Evgenii Stratonikov
0272218eb9 [#2184] compression: Properly calculate upper bound 
If the data is not compressible allocating `len(data)` will lead to a
slice reallocation. For a compressible data the results for small size
are flaky and we allocate a bit more. However, it feels right to use a
provided function if we need to pick any size at all.

```
name                                                           old time/op    new time/op    delta
Compression/size=128/zeroed_slice-8                              2.23µs ±12%    2.06µs ± 6%   -7.35%  (p=0.009 n=10+10)
Compression/size=128/not_so_random_slice_(block_=_123)-8         19.0µs ±10%    15.8µs ±16%  -17.09%  (p=0.000 n=9+10)
Compression/size=128/random_slice-8                              17.6µs ±15%    16.1µs ±16%     ~     (p=0.075 n=10+10)
Compression/size=1024/zeroed_slice-8                             3.05µs ±11%    2.84µs ±10%     ~     (p=0.089 n=10+10)
Compression/size=1024/not_so_random_slice_(block_=_123)-8        18.1µs ± 6%    18.2µs ±12%     ~     (p=0.971 n=10+10)
Compression/size=1024/random_slice-8                             48.6µs ± 6%    45.6µs ± 5%   -6.07%  (p=0.006 n=10+9)
Compression/size=32768/zeroed_slice-8                            26.8µs ± 3%    28.7µs ± 8%   +7.23%  (p=0.001 n=10+10)
Compression/size=32768/not_so_random_slice_(block_=_123)-8       44.3µs ± 8%    43.7µs ±13%     ~     (p=0.762 n=8+10)
Compression/size=32768/random_slice-8                            97.3µs ±32%    68.9µs ±15%  -29.13%  (p=0.000 n=10+10)
Compression/size=33554432/zeroed_slice-8                         29.8ms ± 9%    30.3ms ±17%     ~     (p=1.000 n=9+9)
Compression/size=33554432/not_so_random_slice_(block_=_123)-8    33.1ms ±14%    30.3ms ±11%   -8.61%  (p=0.043 n=10+10)
Compression/size=33554432/random_slice-8                         41.7ms ± 3%    30.1ms ± 8%  -27.72%  (p=0.000 n=9+10)

name                                                           old alloc/op   new alloc/op   delta
Compression/size=128/zeroed_slice-8                                128B ± 0%      144B ± 0%  +12.50%  (p=0.000 n=10+10)
Compression/size=128/not_so_random_slice_(block_=_123)-8           384B ± 0%      144B ± 0%  -62.50%  (p=0.000 n=10+10)
Compression/size=128/random_slice-8                                384B ± 0%      144B ± 0%  -62.50%  (p=0.000 n=10+10)
Compression/size=1024/zeroed_slice-8                             1.02kB ± 0%    1.15kB ± 0%  +12.50%  (p=0.000 n=10+10)
Compression/size=1024/not_so_random_slice_(block_=_123)-8        1.02kB ± 0%    1.15kB ± 0%  +12.50%  (p=0.000 n=10+10)
Compression/size=1024/random_slice-8                             2.56kB ± 0%    1.15kB ± 0%  -55.00%  (p=0.000 n=10+10)
Compression/size=32768/zeroed_slice-8                            32.8kB ± 0%    41.0kB ± 0%  +25.00%  (p=0.000 n=10+10)
Compression/size=32768/not_so_random_slice_(block_=_123)-8       32.8kB ± 0%    41.0kB ± 0%  +25.00%  (p=0.000 n=10+10)
Compression/size=32768/random_slice-8                            81.9kB ± 0%    41.0kB ± 0%  -50.00%  (p=0.000 n=10+10)
Compression/size=33554432/zeroed_slice-8                         33.6MB ± 0%    33.6MB ± 0%   +0.02%  (p=0.000 n=9+9)
Compression/size=33554432/not_so_random_slice_(block_=_123)-8    33.6MB ± 0%    33.6MB ± 0%   +0.02%  (p=0.000 n=8+10)
Compression/size=33554432/random_slice-8                         75.5MB ± 0%    33.6MB ± 0%  -55.55%  (p=0.000 n=10+10)

name                                                           old allocs/op  new allocs/op  delta
Compression/size=128/zeroed_slice-8                                1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=128/not_so_random_slice_(block_=_123)-8           2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
Compression/size=128/random_slice-8                                2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
Compression/size=1024/zeroed_slice-8                               1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=1024/not_so_random_slice_(block_=_123)-8          1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=1024/random_slice-8                               2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
Compression/size=32768/zeroed_slice-8                              1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=32768/not_so_random_slice_(block_=_123)-8         1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=32768/random_slice-8                              2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
Compression/size=33554432/zeroed_slice-8                           1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=33554432/not_so_random_slice_(block_=_123)-8      1.00 ± 0%      1.00 ± 0%     ~     (all equal)
Compression/size=33554432/random_slice-8                           2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov
0ace28e43d [#2175] blobovniczatree: Close all non-active blobovniczas 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov
c1cf418956 [#2175] blobovniczatree: Make function parameters more descriptive 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Pavel Karpy
923f84722a Move to frostfs-node 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-28 15:04:29 +03:00
Evgenii Stratonikov
63f604e948 [#2068] blobstor: Allow to provide storage ID in Exists 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-12-02 11:52:05 +03:00
Evgenii Stratonikov
ae7b473768 [#2064] blobovniczatree: Remove index too big log 
There is no need to log about a situation which is expected.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-30 16:53:18 +03:00
Pavel Karpy
fa231b8c56 [#2057] blobstor: Block operations on a mode change 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
d65604ad30 [#1985] blobstor: Allow to report multiple errors to caller 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
f564430b90 [#1992] fstree: Allow working in SYNC mode 
Make O_SYNC the default and allow to opt-out explicitly.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-01 09:42:26 +03:00
Evgenii Stratonikov
56de2f1363 [#1969] local_object_storage: Simplify logic error construction 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-31 11:41:24 +03:00
Evgenii Stratonikov
fcdbf5e509 [#1969] local_object_storage: Add a type for logical errors 
All logic errors are wrapped in `logicerr.Logical` type and do not
affect shard error counter.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-31 11:41:24 +03:00
Pavel Karpy
41f93dcc1d [#1953] blobstor: Log any object removal operation 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-24 21:52:18 +03:00
Evgenii Stratonikov
1cb892c579 [#1910] .golangci.yml: Add misspell linker 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-18 15:08:26 +03:00
Pavel Karpy
f037022a7a [#1770] logger: Refactor Logger component 
Make it store its internal `zap.Logger`'s level. Also, make all the
components to accept internal `logger.Logger` instead of `zap.Logger`; it
will simplify future refactor.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-12 18:11:05 +03:00
Evgenii Stratonikov
4b005d3178 [#1840] blobstor: Return info about all components 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Evgenii Stratonikov
6d7ffefec5 [#1840] blobstor/test: Add tests for auxiliary functions 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Evgenii Stratonikov
328691c94f [#1840] blobstor: Return Path from components 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-10 11:14:55 +03:00
Evgenii Stratonikov
2e3ef817f4 [#1819] engine: Increase error counter for PUT errors 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-04 10:11:52 +03:00
Evgenii Stratonikov
b89e71fa78 [#1819] common: Add ErrNoSpace 
Add a common error for this case because it is not an error
which should increase error counter. Single error simplifies checks on
the call-site.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-04 10:11:52 +03:00
Pavel Karpy
5c69e19016 [#1770] fstree: Depth parameter int -> uint64 
Negative values have no sense. On the other hand it differs from the
blobovnicza's configuration and prevents unification.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-04 10:08:55 +03:00
Evgenii Stratonikov
6f45cc81fc [#1764] neofs-node: Use constants for storage types 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-27 08:57:20 +03:00
Evgenii Stratonikov
7cc68cf4d4 [#1810] blobovniczatree: Add a test for blobovnicva usage 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-23 10:19:26 +04:00
Evgenii Stratonikov
60b2930417 [#1778] neofs-node: Fix blobovnicza config type 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-12 17:06:00 +03:00
Evgenii Stratonikov
d8a00c365a [#1691] blobovniczatree: Fix active blobovnicza caching 
Maintain an invariant that any blobovnicza is present either
in `opened` or in `active` map. Otherwise, the logic becomes too
complicate because it is not obvious when we should close the blobovnicza.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-31 19:11:48 +03:00
Evgenii Stratonikov
dfac4e1c0b [#1691] blobovniczatree: Do not fill cache during initialization 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-31 19:11:48 +03:00
Evgenii Stratonikov
78bf17c0b0 [#1686] blobstor/*: Remove unneeded tests 
The blobovniczatree test also checks the amount of objects we can put,
so leave it here.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
b9a2055e1c [#1686] blobstor/*: Return ErrReadOnly for modifying operations 
This check should occur on the shard level, but because
blobstor components expose `Open(readOnly bool)` interface,
it is reasonable to expect an error here.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
6c2d3b020f [#1686] fstree: Return proper error for concurrent Delete 
If the file doesn't exist, return `apistatus.ObjectNotFound`.
First check is still there as a shortcut.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
0b95a21701 [#1686] blobstor: Add generic tests 
This tests check that each blobstor component behaves similarly when
same methods are being used. It is intended to serve as a specification
for all future components.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
b2d4cc556e [#1686] blobovnicza: Remove IterateObjects 
It is used only once, makes sense to inline.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
5f95498364 [#1686] blobstor: Add generic tests 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
e8c6dce466 [#1523] blobstor: Rename compression.CConfig to Config 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
0042c1d4a6 [#1584] blobovniczatree: move public methods to different files 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
26b4a258e0 [#1523] neofs-node: Refactor configuration 
1. Move compression parameters to the `shard` section.
2. Allow to use multiple sub-storage components in the blobstor.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
13cdbde2e2 [#1523] blobstor: Remove unused types 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
0c9e4e6a35 [#1523] blobstor: Unify request dispatch logic 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
266458fe5c [#1523] blobstor: Unify fstree and blobovnicza interfaces 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
4176b2a1bc [#1523] local_object_storage: Unify parameters for the Iterate operation 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
73f8bb3e5f [#1523] shard: Store generic storage ID in metabase 
Allow to extend blobstor with more storage sub-systems. Currently
objects stored in the FSTree have empty byte slice descriptor and object
from blobovnicza tree have the same id as earlier. Each such change in
the identifier formation should be accompanied with metabase version
increase.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00