3821645085
[ #1555 ] engine: Refactor (*StorageEngine).GetLocks
...
Tests and linters / Run gofumpt (pull_request) Successful in 3m25s
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m41s
DCO action / DCO (pull_request) Successful in 4m57s
Vulncheck / Vulncheck (pull_request) Successful in 5m28s
Build / Build Components (pull_request) Successful in 5m46s
Tests and linters / Staticcheck (pull_request) Successful in 6m0s
Tests and linters / Tests (pull_request) Successful in 6m34s
Tests and linters / Lint (pull_request) Successful in 6m51s
Tests and linters / Tests with -race (pull_request) Successful in 7m50s
Tests and linters / gopls check (pull_request) Successful in 2m51s
Tests and linters / Run gofumpt (push) Successful in 1m16s
Tests and linters / Staticcheck (push) Successful in 3m13s
Vulncheck / Vulncheck (push) Successful in 3m46s
Build / Build Components (push) Successful in 4m11s
Tests and linters / Lint (push) Successful in 4m26s
Pre-commit hooks / Pre-commit (push) Successful in 4m47s
Tests and linters / Tests with -race (push) Successful in 6m8s
Tests and linters / gopls check (push) Successful in 6m13s
Tests and linters / Tests (push) Successful in 2m36s
Refactored after renaming the method to replace the confusing `locked`
variable with `locks`.
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-12-11 15:06:38 +03:00
72470d6b48
[ #1555 ] local_object_storage: Rename method GetLocked -> GetLocks
...
Renamed to better reflect the method's purpose of returning locks
for the specified object.
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-12-11 15:06:37 +03:00
281d65435e
[ #1450 ] engine: Group object by shard before Inhume
...
Tests and linters / Run gofumpt (pull_request) Successful in 2m1s
DCO action / DCO (pull_request) Successful in 2m20s
Vulncheck / Vulncheck (pull_request) Successful in 2m23s
Build / Build Components (pull_request) Successful in 3m15s
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m21s
Tests and linters / gopls check (pull_request) Successful in 3m21s
Tests and linters / Staticcheck (pull_request) Successful in 3m49s
Tests and linters / Lint (pull_request) Successful in 4m37s
Tests and linters / Tests (pull_request) Successful in 5m0s
Tests and linters / Tests with -race (pull_request) Successful in 6m12s
Tests and linters / Run gofumpt (push) Successful in 1m46s
Vulncheck / Vulncheck (push) Successful in 2m36s
Pre-commit hooks / Pre-commit (push) Successful in 2m55s
Build / Build Components (push) Successful in 3m8s
Tests and linters / gopls check (push) Successful in 3m30s
Tests and linters / Staticcheck (push) Successful in 3m36s
Tests and linters / Lint (push) Successful in 4m31s
Tests and linters / Tests (push) Successful in 4m56s
Tests and linters / Tests with -race (push) Successful in 6m7s
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/engine
cpu: 12th Gen Intel(R) Core(TM) i5-1235U
│ old.txt │ new.txt │
│ sec/op │ sec/op vs base │
InhumeMultipart/objects=1-12 11.42m ± 1% 10.71m ± 0% -6.27% (p=0.000 n=10)
InhumeMultipart/objects=10-12 113.5m ± 0% 100.9m ± 3% -11.08% (p=0.000 n=10)
InhumeMultipart/objects=100-12 1135.4m ± 1% 681.3m ± 2% -40.00% (p=0.000 n=10)
InhumeMultipart/objects=1000-12 11.358 ± 0% 1.089 ± 1% -90.41% (p=0.000 n=10)
InhumeMultipart/objects=10000-12 113.251 ± 0% 1.645 ± 1% -98.55% (p=0.000 n=10)
geomean 1.136 265.5m -76.63%
```
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-12-04 10:09:00 +03:00
748edd1999
[ #1450 ] engine: Return shard-level error if object is expired on inhume
...
Since we have errors defined on the shard-level, it looks strage that we
check an error againt the shard-level error `ErrLockObjectRemoval`, but
then return the metabase-level error. Let's return the same shard-level
error instead.
Since we have errors defined on the shard-level
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-12-04 10:06:57 +03:00
7429553266
[ #1437 ] node: Fix contextcheck linter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:10 +03:00
6db46257c0
[ #1437 ] node: Use ctx for logging
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-13 10:36:07 +03:00
8a57c78f5f
[ #1484 ] engine: Fix engine metrics
...
Tests and linters / Run gofumpt (pull_request) Successful in 2m38s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m51s
DCO action / DCO (pull_request) Successful in 2m53s
Vulncheck / Vulncheck (pull_request) Successful in 3m44s
Tests and linters / gopls check (pull_request) Successful in 3m53s
Build / Build Components (pull_request) Successful in 4m15s
Tests and linters / Staticcheck (pull_request) Successful in 4m56s
Tests and linters / Tests with -race (pull_request) Successful in 5m5s
Tests and linters / Lint (pull_request) Successful in 5m47s
Tests and linters / Tests (pull_request) Successful in 5m51s
1. Add forgotten metrics for client requests
2. Include execIfNotBlocked into metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-11-11 12:59:20 +03:00
9a87acb87a
[ #1410 ] engine: Provide the default implementation to MetricsRegister
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-03 08:23:06 +00:00
eeca796d2e
[ #1295 ] engine: Log object address in case of error
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-06 09:45:43 +03:00
92e19feb57
[ #1147 ] node: Use public fields for shard.ExistsPrm
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
6130650bb6
[ #1147 ] node: Implement Lock\Delete requests for EC object
...
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
c681354afd
[ #938 ] engine: Fix container count removal
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-30 12:37:48 +03:00
4b8b4da681
[ #864 ] engine: Drop container count metric if container removed
...
DCO action / DCO (pull_request) Successful in 1m30s
Build / Build Components (1.21) (pull_request) Successful in 3m29s
Build / Build Components (1.20) (pull_request) Successful in 3m53s
Tests and linters / Lint (pull_request) Successful in 4m31s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m1s
Tests and linters / Staticcheck (pull_request) Successful in 4m51s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m13s
Tests and linters / Tests with -race (pull_request) Successful in 8m34s
Vulncheck / Vulncheck (pull_request) Successful in 58s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-10 10:45:32 +03:00
d75e7e9a21
[ #864 ] engine: Drop container size metric if container deleted
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-01-10 10:44:54 +03:00
c1e4130020
[ #146 ] node: Add trace_id to logs
...
Vulncheck / Vulncheck (pull_request) Successful in 3m7s
DCO action / DCO (pull_request) Successful in 3m36s
Build / Build Components (1.21) (pull_request) Successful in 3m29s
Build / Build Components (1.20) (pull_request) Successful in 3m37s
Tests and linters / Staticcheck (pull_request) Successful in 4m39s
Tests and linters / Lint (pull_request) Successful in 5m2s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m27s
Tests and linters / Tests with -race (pull_request) Successful in 6m29s
Tests and linters / Tests (1.20) (pull_request) Successful in 9m19s
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-09-27 11:05:27 +03:00
5b7e4a51b7
[ #481 ] Update frostfs-sdk-go and error pointer receivers
...
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
2023-08-09 10:26:53 +00:00
1b364d8cf4
[ #424 ] metrics: Refactor engine metrics
...
Use histogram vector to measure request duration.
Fix naming like in Prometheus best practice.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-14 14:53:32 +03:00
74578052f9
[ #412 ] node: Replace tracing package
...
Use observability module.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-06-01 13:23:11 +00:00
d62c6e4ce6
[ #242 ] node: Add tracing spans
...
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-14 10:25:53 +00:00
0e31c12e63
[ #240 ] logs: Move log messages to constants
...
Drop duplicate entities.
Format entities.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-04-14 05:06:09 +00:00
0920d848d0
[ #135 ] get-object: Add tracing spans
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-12 06:52:00 +00:00
23575e1ac0
[ #210 ] policier: Resolve contextcheck linter
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-04-05 14:55:52 +03:00
9f0bce5c15
[ #183 ] gc: Fix drop expired locked simple objects
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-03-30 15:33:42 +03:00
Pavel Karpy
97e201993b
[ #184 ] *: Unify done contexts handling
...
If `ctx.Done()`, return `ctx.Err()` in every function that returns an error.
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-03-29 10:34:54 +00:00
Pavel Karpy
f006f3b342
[ #67 ] node: Make engine's IsLocked public
...
It will allow reusing that method in expiration checks.
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2023-03-16 16:20:45 +03:00
20de74a505
Rename package name
...
Due to source code relocation from GitHub.
Signed-off-by: Alex Vanin <a.vanin@yadro.com>
2023-03-07 16:38:26 +03:00
Pavel Karpy
923f84722a
Move to frostfs-node
...
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
2022-12-28 15:04:29 +03:00
Pavel Karpy
3b61cb4f49
[ #1502 ] engine: Check all shards for LOCK'ing before inhuming
...
It allows keeping all the locked objects safe after metabase
resynchronization. Currently, all `LOCK` objects are broadcast to all nodes
in a container, it guarantees `LOCK` object presence in a regular situation.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-11-19 11:01:04 +03:00
Evgenii Stratonikov
6557f5d249
[ #1839 ] engine: Handle Inhume errors properly
...
If shard is in read-only or degraded mode, there is no need to increase
error counter.
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-07 14:25:52 +03:00
Evgenii Stratonikov
af56574849
[ #1819 ] engine: Fix error counter in Inhume
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-10-04 10:11:52 +03:00
Evgenii Stratonikov
5321f8ef9c
[ #1786 ] engine: Unify parameter setters
...
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
2022-09-15 10:28:48 +03:00
Pavel Karpy
30341f2192
[ #1687 ] *: Perform go fmt using go v1.19
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-22 18:59:57 +03:00
Pavel Karpy
156ba85326
[ #1634 ] node: Do not return expired objects
...
If an object has not been marked for removal by the GC in the current epoch
yet but has already expired, respond with `ErrObjectNotFound` api status.
Also, optimize shard iteration: a node must stop any iteration if the object
is found but gonna be removed soon.
All the checks are performed by the Metabase.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-08-04 16:31:49 +03:00
Evgenii Stratonikov
d8ba954aff
[ #1559 ] shard: Use Set prefix for parameter setting
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-07-21 17:56:06 +03:00
Pavel Karpy
cb0bb7207c
[ #1461 ] shard: Add a separate ErrLockObjectRemoval
...
Do not return `meta.ErrLockObjectRemoval` from shard's methods, add shard's
own error for that.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
Pavel Karpy
558cc1193a
[ #1461 ] engine: Clarify force removal
...
Document force removal behaviour in all the Storage engine parts.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
Pavel Karpy
51afcc1182
[ #1461 ] engine, policer: Force remove objects w/o container
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
Pavel Karpy
18ec5d7c8e
[ #1461 ] meta: Return error on lock object removal
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
Pavel Karpy
fed9e6679d
[ #1461 ] node: Unlock locked object on its lock removal
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-07-18 11:42:25 +03:00
Pavel Karpy
08bf8a68f1
[ #1460 ] engine: Do not use pointers as the results
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-06 18:03:12 +03:00
Pavel Karpy
babd382ba5
[ #1418 ] engine: Do not use pointers as parameters
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-03 07:35:17 +03:00
Pavel Karpy
6e752f36dc
[ #1418 ] shard: Do not use pointers as parameters
...
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-06-03 07:35:17 +03:00
Leonard Lyubich
1c30414a6c
[ #1454 ] Upgrade NeoFS SDK Go module with new IDs
...
Core changes:
* avoid package-colliding variable naming
* avoid using pointers to IDs where unnecessary
* avoid using `idSDK` import alias pattern
* use `EncodeToString` for protocol string calculation and `String` for
printing
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-06-01 17:41:45 +03:00
Elizaveta Chichindaeva
cc7a723d77
[ #1320 ] English Check
...
Signed-off-by: Elizaveta Chichindaeva <elizaveta@nspcc.ru>
2022-05-11 10:40:02 +03:00
Pavel Karpy
7799f8e4cf
[ #1318 ] engine: Change tombstone clear process
...
- Delete objects physically on tombstone's arrival;
- Store information about tombstones in the Graveyard;
- Clear Graveyard every epoch based on the information about TS in the
network.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-04-29 16:38:52 +03:00
Leonard Lyubich
70ffdf3478
[ #1247 ] object: Return NOT_FOUND and ALREADY_REMOVED statuses
...
Replace `ErrNotFound`/`ErrAlreadyRemoved` error from
`pkg/core/object` package with `ObjectNotFound`/`ObjectAlreadyRemoved`
one from `apistatus` package. These errors are returned by storage
node's server as NeoFS API statuses.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-17 16:34:00 +03:00
Leonard Lyubich
9dff07200c
[ #1181 ] local storage: Process expired locks similar to tombstones
...
There is a need to process expired `LOCK` objects similar to `TOMBSTONE`
ones: we collect them on `Shard`, notify all other shards about
expiration so they could unlock the objects, and only after that mark
lockers as garbage.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-15 13:03:23 +03:00
Leonard Lyubich
97b6ec8150
[ #1175 ] storage engine: Return status error on Inhume of locked object
...
Make `StorageEngine.Inhume` to forward first encountered
`apistatus.ObjectLocked` error during shard processing.
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-15 13:03:23 +03:00
Evgenii Stratonikov
c1530dec5e
[ #1115 ] shard/gc: provide parsed addresses in HandleExpiredTombstones
...
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2022-02-11 12:58:59 +03:00
Pavel Karpy
1667ec9e6d
[ #1131 ] *: Adopt SDK changes
...
`object.Address` has been moved to `object/address`
`object.ID` has been moved to `object/id`
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
2022-02-08 09:45:38 +03:00
