TrueCloudLab/frostfs-node
20
1
Fork 28
Code Issues 107 Pull requests 4 Projects Releases 2 Packages 4 Activity Actions
4907 commits 9 branches 112 tags 52 MiB
Commit graph

30 commits

Author SHA1 Message Date
Dmitrii Stepanov
5b9928536d [#1335] writecache: Change DB engine to Pebble
Some checks failed
Tests and linters / Tests with -race (pull_request) Failing after 12s
Details
Tests and linters / Run gofumpt (pull_request) Successful in 1m14s
Details
DCO action / DCO (pull_request) Successful in 1m26s
Details
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m51s
Details
Vulncheck / Vulncheck (pull_request) Successful in 1m48s
Details
Build / Build Components (pull_request) Successful in 2m2s
Details
Tests and linters / Tests (pull_request) Successful in 3m7s
Details
Tests and linters / Staticcheck (pull_request) Successful in 3m9s
Details
Tests and linters / Lint (pull_request) Successful in 3m45s
Details
Tests and linters / gopls check (pull_request) Successful in 3m56s
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-09-05 13:31:16 +03:00
Anton Nikiforov
3195142d67 [#959] writecache: Avoid manipulation with cache in DEGRADED mode 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-03-11 18:35:41 +00:00
Anton Nikiforov
05b5f5ca85 [#959] writecache: Fix panic on Get when it is not initialized
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 2m41s
Details
DCO action / DCO (pull_request) Successful in 2m43s
Details
Tests and linters / Staticcheck (pull_request) Successful in 3m55s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m45s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m19s
Details
Tests and linters / Lint (pull_request) Successful in 5m30s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m57s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m22s
Details
Tests and linters / Tests with -race (pull_request) Successful in 8m39s
Details 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-02-14 16:10:33 +03:00
Dmitrii Stepanov
8180a0664f [#887] node: Drop badger writecache implementation
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 6m1s
Details
Build / Build Components (1.21) (pull_request) Successful in 7m37s
Details
Build / Build Components (1.20) (pull_request) Successful in 7m52s
Details
Tests and linters / Staticcheck (pull_request) Successful in 8m56s
Details
Tests and linters / Lint (pull_request) Successful in 9m26s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 15m5s
Details
Tests and linters / Tests with -race (pull_request) Successful in 15m7s
Details
DCO action / DCO (pull_request) Successful in 1m1s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 4m1s
Details 
Badger implementation isn't tested and works not well,
but requires human resources to maintain.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-22 13:00:54 +03:00
Alejandro Lopez
1a0cb0f34a [#421] Try using badger for the write-cache 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-07 08:16:57 +00:00
Evgenii Stratonikov
fe01781811 [#446] los: Wrap SSD errors in a separate type 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-16 14:53:31 +03:00
Dmitrii Stepanov
74578052f9 [#412] node: Replace tracing package 
Use observability module.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-01 13:23:11 +00:00
Dmitrii Stepanov
2ce43935f9 [#312] metrics: Add writecache metrcis 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-24 10:18:39 +00:00
Dmitrii Stepanov
d212d908b5 [#312] wc: Add metrics 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-24 10:18:39 +00:00
Evgenii Stratonikov
35c9b6b26d [#314] writecache: remove objects right after they are flushed
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-19 09:37:05 +00:00
Dmitrii Stepanov
0920d848d0 [#135] get-object: Add tracing spans 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-12 06:52:00 +00:00
Alex Vanin
20de74a505 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 16:38:26 +03:00
Pavel Karpy
923f84722a Move to frostfs-node 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-28 15:04:29 +03:00
Evgenii Stratonikov
fcdbf5e509 [#1969] local_object_storage: Add a type for logical errors 
All logic errors are wrapped in `logicerr.Logical` type and do not
affect shard error counter.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-31 11:41:24 +03:00
Evgenii Stratonikov
266458fe5c [#1523] blobstor: Unify fstree and blobovnicza interfaces 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
5aa3defc67 [#1523] writecache: Remove cloneBytes 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
ddaed283e9 [#1523] writecache: Simplify logic 
1. Remove in-memory cache. It doesn't persist objects and if we want
   more speed, `NoSync` option can be used for the bolt DB.
2. Put to the metabase in a synchronous fashion. This considerably
   simplifies overall logic and plays nicely with the metabase bolt DB
   batch settings.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
d9a2f280c9 [#1523] local_object_storage: Unify parameters for the Get operation 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Leonard Lyubich
1c30414a6c [#1454] Upgrade NeoFS SDK Go module with new IDs 
Core changes:
 * avoid package-colliding variable naming
 * avoid using pointers to IDs where unnecessary
 * avoid using `idSDK` import alias pattern
 * use `EncodeToString` for protocol string calculation and `String` for
  printing

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-01 17:41:45 +03:00
Elizaveta Chichindaeva
cc7a723d77 [#1320] English Check 
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
 2022-05-11 10:40:02 +03:00
Evgenii Stratonikov
48eb87d32c [#1149] writecache: Prevent corruption in Head 
Also, remove optimization comments:
1. Having to maintain an execute the same logic for headers as for
   objects is quite inefficient, as it increases memory footprint.
2. Unmarshaling object is a cheap operation if data slice is in memory.
3. For unmarshaling header-only, I think we need SDK support.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-31 15:40:25 +03:00
Leonard Lyubich
318639e5bf [#1247] *: Clarify docs about returned errors from apistatus package 
`apistatus` package provides types which implement build-in `error`
interface. Add `error of type` pattern when documenting these errors in
order to clarify how these errors should be handled (e.g. `errors.Is` is
not good).

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-17 16:34:00 +03:00
Leonard Lyubich
70ffdf3478 [#1247] object: Return NOT_FOUND and ALREADY_REMOVED statuses 
Replace `ErrNotFound`/`ErrAlreadyRemoved` error from
`pkg/core/object` package with `ObjectNotFound`/`ObjectAlreadyRemoved`
one from `apistatus` package. These errors are returned by storage
node's server as NeoFS API statuses.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-17 16:34:00 +03:00
Leonard Lyubich
7ccd1625af [#1214] *: Use single Object type in whole project 
Remove `Object` and `RawObject` types from `pkg/core/object` package.
Use `Object` type from NeoFS SDK Go library everywhere. Avoid using the
deprecated elements.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-04 17:45:00 +03:00
Evgenii Stratonikov
050a4bb2b0 [#1115] *: link TODOs to corresponding issues 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-02-11 12:58:59 +03:00
Pavel Karpy
1667ec9e6d [#1131] *: Adopt SDK changes 
`object.Address` has been moved to `object/address`
`object.ID` has been moved to `object/id`

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-02-08 09:45:38 +03:00
Evgenii Stratonikov
95893927aa *: replace neofs-api-go with neofs-sdk-go 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-11-12 17:29:09 +03:00
Evgenii Stratonikov
1a1435be3d [#791] neofs-lens: inspect objects by address 
```
> neofs-lens list --path ./blob/blobovnicza/1/1/0
6ay4GfhR9RgN28d5ufg63toPetkYHGcpcW7G3b7QWSek/9ibXu6v4uTwLEcME5vyHev6Zi8LpxqiWTe1dahKpAbb6
H3VBttoLQoknzMDgnVNyLZ8EpkDnQjnaxDr9fnAWeEHA/Hw1titdGh7BrTe2yLotiYbVh9FQaRRNhoNzXTyetpFgt

> neofs-lens inspect --path ./blob/blobovnicza/1/1/0 \
    --address 6ay4GfhR9RgN28d5ufg63toPetkYHGcpcW7G3b7QWSek/9ibXu6v4uTwLEcME5vyHev6Zi8LpxqiWTe1dahKpAbb6 \
    --header --out payload
Version: v2.1
Type: REGULAR
CID: 6ay4GfhR9RgN28d5ufg63toPetkYHGcpcW7G3b7QWSek
ID: 9ibXu6v4uTwLEcME5vyHev6Zi8LpxqiWTe1dahKpAbb6
Owner: 2dokPzmmcLnnR21jQB3qPppTQRgwMNMKEWD
CreatedAt: 0
PayloadSize: 32
Attributes:
  foo: bar

> hexdump -C payload
00000000  ff 6c d4 71 c4 83 f1 5f  b9 0b ad b3 7c 58 21 b6  |.l.q..._....|X!.|
00000010  d9 55 26 a4 1a 95 04 68  0b 4e 7c 8b 76 3a 1b 1d  |.U&....h.N|.v:..|
00000020
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-10-18 11:55:02 +03:00
Leonard Lyubich
aa2151fbeb [#762] write-cache: Implement Head operation 
Write cache should be able to execute HEAD operations according to spec.

Add simple implementation of `Head` method through the `Get` one. Leave
notes for future optimization.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2021-09-02 11:33:17 +03:00
Evgenii Stratonikov
59de521fd1 [#472] blobstor: implement write-cache 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-04-29 17:36:36 +03:00