9a622a750d
[ #970 ] fstree: Move temporary path handling in a separate function
...
Allow to easier test different implementations.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-02-09 16:12:11 +00:00
f526f49995
[ #874 ] engine: Check object existance concurrently
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-23 09:28:29 +03:00
a531eaf8bc
[ #661 ] blobstor: Add Rebuild implementation
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:32 +03:00
29fe8c41f3
[ #655 ] storage: Drop ErrorHandler
...
The only one usage was for logging.
Now logging performed by storage anyway.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-16 17:27:38 +03:00
137e987a4e
[ #655 ] storage: Drop LazyHandler
...
LazyHandler is implemented and used incorrectly.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-16 17:27:38 +03:00
79088baa06
[ #772 ] node: Apply gofumpt
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-10-31 17:03:03 +03:00
58c8722c81
[ #585 ] fstree: Add optional file counter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-16 14:47:44 +03:00
baad49990c
[ #585 ] fstree: Return logical error if object deleted
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-16 14:47:44 +03:00
0c52186572
[ #585 ] fstree: Remove unused method
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-08-16 14:47:44 +03:00
8f994163ee
[ #586 ] Fix writecache benchmarks and refactor hacky NeedsCompression
...
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-10 08:05:18 +00:00
5b7e4a51b7
[ #481 ] Update frostfs-sdk-go and error pointer receivers
...
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-09 10:26:53 +00:00
d8ecc69d00
[ #373 ] local storage: Pass parent ID
...
This is required to add shard ID as metric label.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
3ae3c8dfdb
[ #373 ] fstree: Add metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-21 15:13:26 +03:00
74578052f9
[ #412 ] node: Replace tracing package
...
Use observability module.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-01 13:23:11 +00:00
802168c0c6
[ #364 ] node: Stop flushing big object when termination signal received
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-05-26 16:46:58 +03:00
f07e2d4812
[ #285 ] lint: Fix revive/unused-parameter
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-26 17:35:04 +03:00
d62c6e4ce6
[ #242 ] node: Add tracing spans
...
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-14 10:25:53 +00:00
0920d848d0
[ #135 ] get-object: Add tracing spans
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-12 06:52:00 +00:00
760af6b912
[ #211 ] fstree: Consider ENOSPC a logical error
...
We already do this for file writing, however directory creation can also
fail.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-05 11:10:39 +00:00
Pavel Karpy
da8da1c63a
[ #98 ] fstree: Do not fail iteration over just removed files
...
A directory is read and files are saved to a local variable. The iteration
over such files may lead to a non-existing files reading due to a normal SN
operation cycle and, therefore, may lead to a returning the OS error to a
caller. Skip just removed (or lost) files as the golang std library does in
similar situations:
5f1a0320b9/src/os/dir_unix.go (L128-L133)
2023-03-22 07:26:27 +00:00
20de74a505
Rename package name
...
Due to source code relocation from GitHub.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 16:38:26 +03:00
e9f3c24229
[ #65 ] Use strings.Cut instead of strings.Split* where possible
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2023-02-28 13:39:14 +03:00
315141dc2c
[ #2252 ] fstree: Allow concurrent writes
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-20 13:53:27 +03:00
204cd3a11c
[ #31 ] fstree: Optimize treePath
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-10 12:49:31 +03:00
dee4498c1e
[ #31 ] fstree: Do not check for a file existence twice
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-10 12:49:31 +03:00
abbecf49d6
[ #31 ] fstree: Speedup string-to-address conversion
...
```
name old time/op new time/op delta
_addressFromString-8 1.25µs ±30% 1.02µs ± 6% -18.49% (p=0.000 n=9+9)
name old alloc/op new alloc/op delta
_addressFromString-8 352B ± 0% 256B ± 0% -27.27% (p=0.000 n=9+10)
name old allocs/op new allocs/op delta
_addressFromString-8 6.00 ± 0% 4.00 ± 0% -33.33% (p=0.000
n=10+10)
```
Also, assure compiler that `s` doesn't escape:
Before this commit:
```
./fstree.go:74:24: leaking param: s
./fstree.go:90:6: moved to heap: addr
```
After this commit:
```
./fstree.go:74:24: s does not escape
```
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-02-10 12:49:31 +03:00
Pavel Karpy
73bc1b0b68
[ #38 ] node: Fix linter warnings
...
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-02-06 17:27:54 +03:00
87f0e3ea25
[ #2208 ] fstree: Rename file after write
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
792319a044
[ #2208 ] fstree: Remove file if there was an error during write
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-01-25 15:31:47 +03:00
Pavel Karpy
923f84722a
Move to frostfs-node
...
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-28 15:04:29 +03:00
Evgenii Stratonikov
d65604ad30
[ #1985 ] blobstor: Allow to report multiple errors to caller
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
f564430b90
[ #1992 ] fstree: Allow working in SYNC mode
...
Make O_SYNC the default and allow to opt-out explicitly.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-11-01 09:42:26 +03:00
Evgenii Stratonikov
fcdbf5e509
[ #1969 ] local_object_storage: Add a type for logical errors
...
All logic errors are wrapped in `logicerr.Logical` type and do not
affect shard error counter.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-31 11:41:24 +03:00
Evgenii Stratonikov
328691c94f
[ #1840 ] blobstor: Return Path from components
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-10 11:14:55 +03:00
Evgenii Stratonikov
b89e71fa78
[ #1819 ] common: Add ErrNoSpace
...
Add a common error for this case because it is not an error
which should increase error counter. Single error simplifies checks on
the call-site.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-04 10:11:52 +03:00
Pavel Karpy
5c69e19016
[ #1770 ] fstree: Depth parameter int -> uint64
...
Negative values have no sense. On the other hand it differs from the
blobovnicza's configuration and prevents unification.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-10-04 10:08:55 +03:00
Evgenii Stratonikov
6f45cc81fc
[ #1764 ] neofs-node: Use constants for storage types
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-27 08:57:20 +03:00
Evgenii Stratonikov
b9a2055e1c
[ #1686 ] blobstor/*: Return ErrReadOnly for modifying operations
...
This check should occur on the shard level, but because
blobstor components expose `Open(readOnly bool)` interface,
it is reasonable to expect an error here.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
6c2d3b020f
[ #1686 ] fstree: Return proper error for concurrent Delete
...
If the file doesn't exist, return `apistatus.ObjectNotFound`.
First check is still there as a shortcut.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
0b95a21701
[ #1686 ] blobstor: Add generic tests
...
This tests check that each blobstor component behaves similarly when
same methods are being used. It is intended to serve as a specification
for all future components.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-08-30 10:02:43 +03:00
Evgenii Stratonikov
e8c6dce466
[ #1523 ] blobstor: Rename compression.CConfig to Config
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
26b4a258e0
[ #1523 ] neofs-node: Refactor configuration
...
1. Move compression parameters to the `shard` section.
2. Allow to use multiple sub-storage components in the blobstor.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
0c9e4e6a35
[ #1523 ] blobstor: Unify request dispatch logic
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
266458fe5c
[ #1523 ] blobstor: Unify fstree and blobovnicza interfaces
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
4176b2a1bc
[ #1523 ] local_object_storage: Unify parameters for the Iterate operation
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
9eb018672c
[ #1523 ] blobstor: Unify parameters for Put operation
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
d9a2f280c9
[ #1523 ] local_object_storage: Unify parameters for the Get operation
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
7df50297cd
[ #1520 ] shard: Ignore errors on metabase refill
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-21 17:25:28 +03:00
Evgenii Stratonikov
f2a7503964
[ #1462 ] fstree: Allow to fetch file content lazily
...
If we should process address based on some condition, there is no need
to read file content in memory.
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-06-03 14:30:00 +03:00
Pavel Karpy
281befec67
[ #1418 ] blobstor: Do not use pointers as parameters
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-03 07:35:17 +03:00
