TrueCloudLab/frostfs-node
20
1
Fork 27
Code Issues 101 Pull requests 4 Projects Releases 1 Activity Actions
5127 commits 9 branches 89 tags 48 MiB
Commit graph

58 commits

Author SHA1 Message Date
Dmitrii Stepanov
6db46257c0
[#1437] node: Use ctx for logging 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-11-13 10:36:07 +03:00
Dmitrii Stepanov
18182e578e [#1261] shard: Fix delete objects from FSTree
All checks were successful
DCO action / DCO (pull_request) Successful in 6m56s
Details
Vulncheck / Vulncheck (pull_request) Successful in 7m38s
Details
Build / Build Components (1.22) (pull_request) Successful in 9m57s
Details
Build / Build Components (1.21) (pull_request) Successful in 10m38s
Details
Tests and linters / gopls check (pull_request) Successful in 10m43s
Details
Tests and linters / Staticcheck (pull_request) Successful in 11m1s
Details
Tests and linters / Lint (pull_request) Successful in 13m35s
Details
Pre-commit hooks / Pre-commit (pull_request) Successful in 14m53s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 15m55s
Details
Tests and linters / Tests with -race (pull_request) Successful in 15m49s
Details
Tests and linters / Tests (1.22) (pull_request) Successful in 16m10s
Details 
Replace nil storageID with empty like by shard.Get.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-07-22 14:38:09 +03:00
Dmitrii Stepanov
dfd62ca6b1 [#864] metabase: Refactor delete/inhume 
Available -> Logic, Raw -> Phy for delete/inhume results.
Use single counter instead of vectors.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-01-09 09:59:42 +03:00
Dmitrii Stepanov
3b7c0362a8 [#861] shard: Fix Delete object
All checks were successful
DCO action / DCO (pull_request) Successful in 12m22s
Details
Build / Build Components (1.20) (pull_request) Successful in 12m44s
Details
Build / Build Components (1.21) (pull_request) Successful in 13m2s
Details
Vulncheck / Vulncheck (pull_request) Successful in 13m3s
Details
Tests and linters / Staticcheck (pull_request) Successful in 15m32s
Details
Tests and linters / Lint (pull_request) Successful in 16m26s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 26m11s
Details
Tests and linters / Tests with -race (pull_request) Successful in 28m29s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 3m28s
Details 
It is possible that object doesn't exist in metabase.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 14:25:40 +03:00
Dmitrii Stepanov
d30ab5f29e [#838] metabase: Count user objects 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-06 15:44:21 +03:00
Dmitrii Stepanov
70ab1ebd54 [#763] metrics: Add container_objects_total metric
All checks were successful
DCO action / DCO (pull_request) Successful in 3m54s
Details
Build / Build Components (1.20) (pull_request) Successful in 4m58s
Details
Build / Build Components (1.21) (pull_request) Successful in 5m16s
Details
Vulncheck / Vulncheck (pull_request) Successful in 9m54s
Details
Tests and linters / Lint (pull_request) Successful in 10m57s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 12m40s
Details
Tests and linters / Staticcheck (pull_request) Successful in 12m34s
Details
Tests and linters / Tests with -race (pull_request) Successful in 12m48s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 13m19s
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-11-08 12:30:57 +03:00
Dmitrii Stepanov
f2437f7ae9 [#734] shard: Fix Delete method
All checks were successful
DCO action / DCO (pull_request) Successful in 1m36s
Details
Build / Build Components (1.20) (pull_request) Successful in 3m46s
Details
Vulncheck / Vulncheck (pull_request) Successful in 3m16s
Details
Tests and linters / Staticcheck (pull_request) Successful in 4m42s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 6m27s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m35s
Details
Tests and linters / Tests with -race (pull_request) Successful in 6m33s
Details
Build / Build Components (1.21) (pull_request) Successful in 13m7s
Details
Tests and linters / Lint (pull_request) Successful in 19m14s
Details 
Due to the flushing data from the writecache to the storage
and simultaneous deletion, a partial deletion situation is possible.
So as a solution, deletion is allowed only when the object is in storage,
because object will be deleted from writecache by flush goroutine.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-10-16 17:00:18 +03:00
Alexander Chuprov
c1e4130020 [#146] node: Add trace_id to logs
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 3m7s
Details
DCO action / DCO (pull_request) Successful in 3m36s
Details
Build / Build Components (1.21) (pull_request) Successful in 3m29s
Details
Build / Build Components (1.20) (pull_request) Successful in 3m37s
Details
Tests and linters / Staticcheck (pull_request) Successful in 4m39s
Details
Tests and linters / Lint (pull_request) Successful in 5m2s
Details
Tests and linters / Tests (1.21) (pull_request) Successful in 6m27s
Details
Tests and linters / Tests with -race (pull_request) Successful in 6m29s
Details
Tests and linters / Tests (1.20) (pull_request) Successful in 9m19s
Details 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-09-27 11:05:27 +03:00
Alexander Chuprov
21800e9fcc [#162] core: Move literals to constants 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-08-11 15:48:42 +03:00
Alejandro Lopez
5b7e4a51b7 [#481] Update frostfs-sdk-go and error pointer receivers 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-08-09 10:26:53 +00:00
Dmitrii Stepanov
74578052f9 [#412] node: Replace tracing package 
Use observability module.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-01 13:23:11 +00:00
Dmitrii Stepanov
3220c4df9f [#376] metrics: Add GC metrics
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-31 10:22:12 +00:00
Dmitrii Stepanov
2613351008 [#387] gc: Cancel GC is change mode requested
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-05-25 09:38:16 +03:00
Dmitrii Stepanov
d62c6e4ce6 [#242] node: Add tracing spans 
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-14 10:25:53 +00:00
Evgenii Stratonikov
0e31c12e63 [#240] logs: Move log messages to constants 
Drop duplicate entities.
Format entities.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-14 05:06:09 +00:00
Alex Vanin
20de74a505 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 16:38:26 +03:00
Artem Tataurov
362f24953a [#47] shard: Switch container size metric from physical to logical capacity 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-02-17 12:03:42 +03:00
Artem Tataurov
ab21d90cfb [#1794] shard: Add increasing case for the payload size metric 
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
 2023-02-09 13:30:23 +03:00
Pavel Karpy
89a0266f5e [#1794] metrics: Track physical object capacity per shard 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-01-26 20:06:28 +03:00
Evgenii Stratonikov
9513f163aa [#2116] metrics: Track physical object capacity in the container 
Currently we track based on `PayloadSize`, because it is already stored
in the metabase and it is easier to calculate without slowing down the
whole system.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-01-26 20:06:28 +03:00
Pavel Karpy
eea2892109 [#1956] node: Lock shard's mode on its methods switch 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-30 11:07:35 +03:00
Pavel Karpy
923f84722a Move to frostfs-node 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-28 15:04:29 +03:00
Leonard Lyubich
e8c5f03c30 [#1905] shard: Don't log read-only errors of write-cache 
There is no need to log `writecache.ErrReadOnly` errors in `Delete`
method of the `Shard`.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 18:30:45 +03:00
Leonard Lyubich
b1fa084756 [#1905] shard: Decrease severity level of write-cache failure logs 
In previous implementation `Shard.Delete` logged writecache's removal
failures in `error` level. There is a need to decrease severity of these
log records since they aren't critical and don't require individual
review.

Change level of the message to `info`.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
 2022-10-28 18:30:45 +03:00
Pavel Karpy
0371d15b2f [#1953] shard: Fix debug log messages 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-10-24 21:52:18 +03:00
Pavel Karpy
431e331373 [#1658] shard: Update metric counters 
Use meta's operation results to change the metrics. Support typed object
counters.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-09-13 21:32:37 +04:00
Evgenii Stratonikov
c04126f35e [#1745] writecache: Remove IsErrNotFound 
We specify the error in the doc-comment, and it is the same for all our
components.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-09-02 11:59:53 +03:00
Pavel Karpy
745f72fff0 [#1658] node: Support object counter metrics 
Increment shard's object counter on successful `Put` calls and decrement on
`Delete`.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-08-25 19:20:33 +03:00
Evgenii Stratonikov
73f8bb3e5f [#1523] shard: Store generic storage ID in metabase 
Allow to extend blobstor with more storage sub-systems. Currently
objects stored in the FSTree have empty byte slice descriptor and object
from blobovnicza tree have the same id as earlier. Each such change in
the identifier formation should be accompanied with metabase version
increase.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
7d6df543d7 [#1523] blobstor: Export single Get, GetRange and Delete methods 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
d75d030a90 [#1523] local_object_storage: Unify parameters for the Delete operation 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
b621f5983a [#1523] local_object_storage: Move blobovnicza tree to a separate package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-08-22 13:14:19 +03:00
Evgenii Stratonikov
c8911d72d0 [#1559] shard: Do not consult metabase in a degraded mode 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
339864b720 [#1559] local_object_storage: Move shard.Mode to a separate package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
d8ba954aff [#1559] shard: Use Set prefix for parameter setting 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
34d319fed2 [#1559] metabase: Use Set prefix for parameter setting 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Evgenii Stratonikov
f58234aa2f [#1559] metabase: Remove public functions 
Reduce public interface of this package. Later each result will contain
an additional status, so it makes more sense to use the same functions
and result processing everywhere.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-07-21 17:56:06 +03:00
Pavel Karpy
7b6363f4c6 [#1460] shard: Do not use pointers as the results 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-06 18:03:12 +03:00
Pavel Karpy
281befec67 [#1418] blobstor: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Pavel Karpy
6e752f36dc [#1418] shard: Do not use pointers as parameters 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-06-03 07:35:17 +03:00
Leonard Lyubich
1c30414a6c [#1454] Upgrade NeoFS SDK Go module with new IDs 
Core changes:
 * avoid package-colliding variable naming
 * avoid using pointers to IDs where unnecessary
 * avoid using `idSDK` import alias pattern
 * use `EncodeToString` for protocol string calculation and `String` for
  printing

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-06-01 17:41:45 +03:00
Elizaveta Chichindaeva
cc7a723d77 [#1320] English Check 
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
 2022-05-11 10:40:02 +03:00
Evgenii Stratonikov
6472a170eb [#1143] shard: Introduce explicit Degraded mode 
`Degraded` mode is set automatically after error counter is over the
threshold. `ReadOnly` mode can still be set by an administrator.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2022-03-31 15:33:22 +03:00
Leonard Lyubich
70ffdf3478 [#1247] object: Return NOT_FOUND and ALREADY_REMOVED statuses 
Replace `ErrNotFound`/`ErrAlreadyRemoved` error from
`pkg/core/object` package with `ObjectNotFound`/`ObjectAlreadyRemoved`
one from `apistatus` package. These errors are returned by storage
node's server as NeoFS API statuses.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
 2022-03-17 16:34:00 +03:00
Pavel Karpy
1667ec9e6d [#1131] *: Adopt SDK changes 
`object.Address` has been moved to `object/address`
`object.ID` has been moved to `object/id`

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-02-08 09:45:38 +03:00
Pavel Karpy
375394dc99 [#1059] shard: Add shard mode to shard Info 
Provide shard mode information via `DumpInfo()`. Delete atomic field from
Shard structure since it duplicates new field.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-30 14:14:48 +03:00
Pavel Karpy
93bd6be743 [#1057] shard: Do not allow memory change operations in "read-only" 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2021-12-29 18:06:15 +03:00
Evgenii Stratonikov
95893927aa *: replace neofs-api-go with neofs-sdk-go 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-11-12 17:29:09 +03:00
Evgenii Stratonikov
71b87155ef [#521] *: use stdlib errors package 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-05-19 16:42:54 +03:00
Evgenii Stratonikov
59de521fd1 [#472] blobstor: implement write-cache 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
 2021-04-29 17:36:36 +03:00