3cd7d23f10
[ #1439 ] node: Reduce usage of netmapAPI.NodeInfo
...
Remove outdated code from `netmap` service.
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-10-29 08:05:09 +00:00
012af5cc38
[ #1406 ] tree: Add unit-tests for ape check
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-10-29 08:04:23 +00:00
eb5336d5ff
[ #1406 ] tree: Use delete verb instead put for Remove
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-10-29 08:04:23 +00:00
bc8d79ddf9
[ #1447 ] services/tree: Move relaying code to a separate function
...
Tests and linters / Run gofumpt (pull_request) Successful in 3m7s
DCO action / DCO (pull_request) Successful in 3m18s
Vulncheck / Vulncheck (pull_request) Successful in 3m22s
Tests and linters / Staticcheck (pull_request) Successful in 4m25s
Pre-commit hooks / Pre-commit (pull_request) Successful in 5m11s
Tests and linters / Lint (pull_request) Successful in 5m13s
Build / Build Components (pull_request) Successful in 5m19s
Tests and linters / gopls check (pull_request) Successful in 6m12s
Tests and linters / Tests (pull_request) Successful in 6m58s
Tests and linters / Tests with -race (pull_request) Successful in 6m59s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-24 10:01:03 +03:00
65a4320c75
[ #1441 ] services/tree: Use grpc.WaitForReady
option when creating client
...
DCO action / DCO (pull_request) Successful in 1m10s
Tests and linters / Run gofumpt (pull_request) Successful in 1m28s
Vulncheck / Vulncheck (pull_request) Successful in 2m11s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m21s
Build / Build Components (pull_request) Successful in 2m33s
Tests and linters / gopls check (pull_request) Successful in 2m48s
Tests and linters / Staticcheck (pull_request) Successful in 3m7s
Tests and linters / Lint (pull_request) Successful in 3m30s
Tests and linters / Tests (pull_request) Successful in 4m25s
Tests and linters / Tests with -race (pull_request) Successful in 6m4s
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-10-23 11:45:44 +00:00
9a260c2e64
[ #1441 ] network/cache: Use grpc.WaitForReady
option when creating client
...
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-10-23 11:45:44 +00:00
8b6ec57c61
[ #1440 ] sdnotify: Fix status for RELOADING
...
DCO action / DCO (pull_request) Successful in 1m36s
Vulncheck / Vulncheck (pull_request) Successful in 2m16s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m28s
Tests and linters / Run gofumpt (pull_request) Successful in 2m31s
Build / Build Components (pull_request) Successful in 2m43s
Tests and linters / Staticcheck (pull_request) Successful in 2m56s
Tests and linters / gopls check (pull_request) Successful in 3m7s
Tests and linters / Lint (pull_request) Successful in 3m43s
Tests and linters / Tests (pull_request) Successful in 4m38s
Tests and linters / Tests with -race (pull_request) Successful in 5m51s
Before:
```
RELOADING=1
MONOTONIC_USEC=17951246687
STATUS=RELOADING=1
MONOTONIC_USEC=17951246687
```
After:
```
RELOADING=1
MONOTONIC_USEC=17951246687
STATUS=RELOADING
```
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-10-21 14:25:08 +03:00
c0a2f20eee
[ #1422 ] multinet: Add metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-17 13:15:08 +00:00
ef38420623
[ #1422 ] ir: Add dialer source
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-17 13:15:08 +00:00
fbdfd503e4
[ #1422 ] morph: Add dialer source support
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-17 13:15:08 +00:00
5b653aa65f
[ #1422 ] morph: Drop single client as not used
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-17 13:15:08 +00:00
e314f328c4
[ #1422 ] tree: Use dialer source for tree service connections
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-17 13:15:08 +00:00
6c96cc2af6
[ #1422 ] node: Use dialer source for SDK cache
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-17 13:15:08 +00:00
b42bcdc6fa
[ #1433 ] services/object: Put object before auxiliary info
...
DCO action / DCO (pull_request) Successful in 1m24s
Tests and linters / Run gofumpt (pull_request) Successful in 1m18s
Vulncheck / Vulncheck (pull_request) Successful in 1m30s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m20s
Build / Build Components (pull_request) Successful in 2m30s
Tests and linters / gopls check (pull_request) Successful in 2m50s
Tests and linters / Staticcheck (pull_request) Successful in 3m22s
Tests and linters / Lint (pull_request) Successful in 3m55s
Tests and linters / Tests (pull_request) Successful in 6m3s
Tests and linters / Tests with -race (pull_request) Successful in 6m2s
Consider the following operations ordering:
1. Inhume(with tombstone A) --> add tombstone mark for an object
2. --> new epoch arives
3. --> GCMark is added for a tombstone A, because it is unavailable
4. Put(A) --> return error, because the object already has a GCMark
It is possible, and I have successfully reproduced it with a test on the
shard level. However, the error is related to the specific
_ordering_ of operations with engine. And triggering race-conditions like
this is only possible on a shard level currently, so no tests are
written.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-17 14:43:13 +03:00
b0c5def2d9
[ #1433 ] shard/test: Use WithDisabledGC() option where possible
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-17 14:38:07 +03:00
90f3669399
[ #1342 ] network/cache: Add node address to error multiClient
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-10-15 14:05:33 +00:00
41038b2ec0
[ #1431 ] node: Fix 'empty slice declaration using a literal'
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-10-15 08:53:59 +00:00
d83879d4b8
[ #1431 ] node: Fix comment format
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-10-15 08:53:59 +00:00
f6582081a4
[ #1431 ] obj_storage/metabase: Delete unused variable
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-10-15 08:53:59 +00:00
00b1cecfb7
[ #1431 ] obj_storage/shard: Fix visibility of 'newMetricStore'
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-10-15 08:53:59 +00:00
63466d71b2
[ #1431 ] engine: Delete unused constants
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-10-15 08:53:59 +00:00
d53732f663
[ #1431 ] engine: Delete always false condition
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-10-15 08:53:59 +00:00
3012286452
[ #1431 ] metabase: Fix unreachable code
...
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-10-15 08:53:59 +00:00
714ff784fa
[ #1431 ] objsvc: Use specific values in message about address mismatch
...
Tests and linters / Run gofumpt (pull_request) Successful in 1m30s
DCO action / DCO (pull_request) Successful in 1m42s
Vulncheck / Vulncheck (pull_request) Successful in 2m3s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m19s
Build / Build Components (pull_request) Successful in 2m34s
Tests and linters / gopls check (pull_request) Successful in 2m49s
Tests and linters / Staticcheck (pull_request) Successful in 3m15s
Tests and linters / Lint (pull_request) Successful in 3m40s
Tests and linters / Tests (pull_request) Successful in 4m31s
Tests and linters / Tests with -race (pull_request) Successful in 5m53s
This makes troubleshooting failed operations much easier
Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
2024-10-15 07:34:03 +00:00
acd6eb1815
[ #1427 ] object: Fix Put
for EC
object when node unavailable
...
DCO action / DCO (pull_request) Successful in 34s
Tests and linters / Run gofumpt (pull_request) Successful in 43s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m6s
Tests and linters / Staticcheck (pull_request) Successful in 2m13s
Vulncheck / Vulncheck (pull_request) Successful in 2m9s
Build / Build Components (pull_request) Successful in 2m23s
Tests and linters / Lint (pull_request) Successful in 3m24s
Tests and linters / Tests (pull_request) Successful in 3m36s
Tests and linters / gopls check (pull_request) Successful in 3m46s
Tests and linters / Tests with -race (pull_request) Successful in 4m19s
There might be situation when context canceled earlier than traverser move to another part of the nodes.
To avoid this, need to wait for the result from concurrent put at each traverser iteration.
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-10-11 16:53:30 +03:00
1134760271
[ #1425 ] services/tree: Remove eACL mentions from bearer token parsing errors
...
DCO action / DCO (pull_request) Successful in 4m45s
Tests and linters / Run gofumpt (pull_request) Successful in 4m47s
Tests and linters / Staticcheck (pull_request) Successful in 6m5s
Tests and linters / Lint (pull_request) Successful in 6m43s
Vulncheck / Vulncheck (pull_request) Successful in 6m47s
Build / Build Components (pull_request) Successful in 6m59s
Tests and linters / gopls check (pull_request) Successful in 8m32s
Tests and linters / Tests with -race (pull_request) Successful in 9m5s
Tests and linters / Tests (pull_request) Successful in 9m24s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m28s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-09 10:56:49 +03:00
02bb7159a5
[ #1425 ] services/tree: Remove eACL processing
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-09 10:56:48 +03:00
cc5360a578
[ #1425 ] morph/event: Rename eacl_test.go
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-09 10:56:48 +03:00
4190fba86d
[ #1425 ] Remove SetEACL-related code
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-09 10:56:48 +03:00
936ebbb8e5
[ #1423 ] metabase: Hide BucketName
form upper levels
...
Tests and linters / Run gofumpt (pull_request) Successful in 2m11s
DCO action / DCO (pull_request) Successful in 2m28s
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m17s
Vulncheck / Vulncheck (pull_request) Successful in 4m25s
Tests and linters / Lint (pull_request) Successful in 4m37s
Tests and linters / gopls check (pull_request) Successful in 4m34s
Tests and linters / Tests with -race (pull_request) Successful in 4m37s
Build / Build Components (pull_request) Successful in 4m49s
Tests and linters / Staticcheck (pull_request) Successful in 6m1s
Tests and linters / Tests (pull_request) Successful in 7m12s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-10-08 18:41:16 +03:00
c065d55ca3
[ #1412 ] metabase: Drop logging inside transaction
...
DCO action / DCO (pull_request) Successful in 1m13s
Tests and linters / Run gofumpt (pull_request) Successful in 1m29s
Vulncheck / Vulncheck (pull_request) Successful in 2m0s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m10s
Build / Build Components (pull_request) Successful in 2m22s
Tests and linters / gopls check (pull_request) Successful in 2m36s
Tests and linters / Staticcheck (pull_request) Successful in 3m14s
Tests and linters / Lint (pull_request) Successful in 3m31s
Tests and linters / Tests (pull_request) Successful in 4m16s
Tests and linters / Tests with -race (pull_request) Successful in 5m55s
This could lead to hang the db.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-08 11:41:52 +03:00
fe9f664b57
[ #1412 ] metabase: Drop empty user attribute buckets on upgrade
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-08 11:41:52 +03:00
87f4b934d1
[ #1412 ] metabase: Run bucket drop steps on upgrade concurrently
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-08 11:41:52 +03:00
8093e145b3
[ #1412 ] adm: Resolve container type by metabase upgrade
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-08 11:41:51 +03:00
3da168f8cf
[ #1412 ] shard: Resolve container is indexed on metabase resync
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-08 11:41:51 +03:00
4572fa4874
[ #1412 ] searchSvc: Check container is indexed
...
For non S3 containers it is expected to use attributes index for some
attributes.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-08 11:41:50 +03:00
1efa64ee72
[ #1412 ] metabase: Add search by indexed attributes
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-08 11:41:50 +03:00
be744ae3e6
[ #1412 ] metabase: Index attributes for indexed containers
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-08 11:41:49 +03:00
1b520f7973
[ #1412 ] engine: Add IsIndexedContainer
flag
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-08 11:41:49 +03:00
899cd55c27
[ #1412 ] engine: PutPrm refactoring
...
Use fields instead of methods.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-08 11:41:48 +03:00
fc032838c0
[ #1215 ] blobstor/test: Cover iteration behaviour
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-07 12:44:27 +00:00
2f710d8f94
[ #1414 ] metabase: Check parameter for CountAliveObjectsInBucket
...
DCO action / DCO (pull_request) Successful in 1m33s
Tests and linters / Run gofumpt (pull_request) Successful in 1m28s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m10s
Vulncheck / Vulncheck (pull_request) Successful in 2m6s
Build / Build Components (pull_request) Successful in 2m32s
Tests and linters / gopls check (pull_request) Successful in 2m44s
Tests and linters / Staticcheck (pull_request) Successful in 3m6s
Tests and linters / Lint (pull_request) Successful in 3m35s
Tests and linters / Tests (pull_request) Successful in 4m21s
Tests and linters / Tests with -race (pull_request) Successful in 5m44s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-10-04 17:11:15 +03:00
4dc9a1b300
[ #1413 ] engine: Remove error counting methods from Shard
...
Tests and linters / Run gofumpt (pull_request) Successful in 2m4s
DCO action / DCO (pull_request) Successful in 2m22s
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m10s
Vulncheck / Vulncheck (pull_request) Successful in 4m5s
Build / Build Components (pull_request) Successful in 4m31s
Tests and linters / Staticcheck (pull_request) Successful in 4m21s
Tests and linters / gopls check (pull_request) Successful in 4m43s
Tests and linters / Lint (pull_request) Successful in 4m58s
Tests and linters / Tests (pull_request) Successful in 6m36s
Tests and linters / Tests with -race (pull_request) Successful in 7m41s
All error counting and hangling logic is present on the engine level.
Currently, we pass engine metrics with shard ID metric to shard, then
export 3 methods to manipulate these metrics.
In this commits all methods are removed and error counter is tracked on
the engine level exlusively.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-04 15:10:17 +03:00
963faa615a
[ #1413 ] engine: Cleanup shard error reporting
...
- `reportShardErrorBackground()` no longer differs from
`reportShardError()`, reflect this in its name;
- reuse common pieces of code to make it simpler.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-04 15:10:17 +03:00
9a87acb87a
[ #1410 ] engine: Provide the default implementation to MetricsRegister
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-03 08:23:06 +00:00
9206ce5cd2
[ #1410 ] shard: Provide the default implementation for MetricsWriter
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-03 08:23:06 +00:00
6c46044c9c
[ #1410 ] shard: Move MetricsWriter interface to a separate file
...
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-10-03 08:23:06 +00:00
01e3944b31
[ #1408 ] metabase: Fix tests
...
DCO action / DCO (pull_request) Successful in 1m3s
Tests and linters / Run gofumpt (pull_request) Successful in 1m21s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m27s
Vulncheck / Vulncheck (pull_request) Successful in 2m20s
Build / Build Components (pull_request) Successful in 2m41s
Tests and linters / gopls check (pull_request) Successful in 2m57s
Tests and linters / Staticcheck (pull_request) Successful in 3m5s
Tests and linters / Lint (pull_request) Successful in 3m37s
Tests and linters / Tests (pull_request) Successful in 4m31s
Tests and linters / Tests with -race (pull_request) Successful in 6m22s
No need to specify container ID for objects created with `testutil.GenerateObjectWithCID`.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-02 14:41:40 +03:00
434048e8d9
[ #1408 ] metabase: Fix EC search with slow and fast filters
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-10-02 14:36:59 +03:00
9c5ddc4dfe
[ #1407 ] tree: Set ContainerOwner
in parameter for CheckAPE
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-10-02 07:21:02 +00:00
d0ed29b3c7
[ #1350 ] node: Add ability to evacuate objects from REP 1
only
...
Tests and linters / Run gofumpt (pull_request) Successful in 50s
DCO action / DCO (pull_request) Successful in 1m41s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m53s
Vulncheck / Vulncheck (pull_request) Successful in 2m10s
Build / Build Components (pull_request) Successful in 2m43s
Tests and linters / gopls check (pull_request) Successful in 3m0s
Tests and linters / Staticcheck (pull_request) Successful in 3m5s
Tests and linters / Lint (pull_request) Successful in 3m52s
Tests and linters / Tests (pull_request) Successful in 4m40s
Tests and linters / Tests with -race (pull_request) Successful in 6m17s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-09-27 15:41:17 +03:00
5f22ba6f38
[ #1397 ] object: Correctly set namespace before APE check
...
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-27 11:43:29 +00:00
a5e1aa22c9
[ #1394 ] putSvc: Fix relay
...
Tests and linters / Run gofumpt (pull_request) Successful in 1m44s
DCO action / DCO (pull_request) Successful in 1m58s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m37s
Vulncheck / Vulncheck (pull_request) Successful in 3m11s
Tests and linters / Staticcheck (pull_request) Successful in 3m26s
Tests and linters / Lint (pull_request) Successful in 3m33s
Build / Build Components (pull_request) Successful in 3m42s
Tests and linters / gopls check (pull_request) Successful in 4m8s
Tests and linters / Tests (pull_request) Successful in 5m46s
Tests and linters / Tests with -race (pull_request) Successful in 7m31s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 17:28:03 +03:00
4fbfffd44c
[ #1388 ] putSvc: Drop unused
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
29e4cf7ba1
[ #1388 ] ir: Annotate cmode as nolint
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
bdd57c8b6b
[ #1388 ] sessionSvc: Add nolint annotations
...
Used as map key.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
d1d6e3471c
[ #1388 ] signSvc: Drop unused
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
401c398704
[ #1388 ] metabase: Drop unused
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
004ff9e9bf
[ #1388 ] blobstor: Drop unused
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
63a567a1de
[ #1388 ] engine: Drop unused
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
580cd55180
[ #1388 ] getSvc: Drop unused
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
e319bf403e
[ #1388 ] apeSvc: Drop unused and make annotations
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
aedb55f913
[ #1388 ] governance: Drop unused
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-25 08:55:38 +00:00
fd18aa363b
[ #1385 ] metabase: Optimize isTomb
check
...
DCO action / DCO (pull_request) Successful in 52s
Tests and linters / Run gofumpt (pull_request) Successful in 46s
Vulncheck / Vulncheck (pull_request) Successful in 1m44s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m59s
Build / Build Components (pull_request) Successful in 2m9s
Tests and linters / Staticcheck (pull_request) Successful in 2m15s
Tests and linters / gopls check (pull_request) Successful in 3m3s
Tests and linters / Lint (pull_request) Successful in 3m20s
Tests and linters / Tests (pull_request) Successful in 3m30s
Tests and linters / Tests with -race (pull_request) Successful in 5m50s
As tombstone and target must have the same containerID, do not iterate
other containers.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-24 17:09:06 +03:00
76268e3ea2
[ #1385 ] metabase: Validate that tombstone and target have the same container ID
...
Target container ID is taken from tombstone: cmd/frostfs-node/object.go:507
Also object of type `TOMBSTONE` contains objectID, so tombstone and
tombstoned object must have the same containerID.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-24 17:09:06 +03:00
8434f3dbfc
[ #1385 ] metabase: Use Batch
for delete-related operations
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-24 17:09:06 +03:00
34e6a309c6
[ #1356 ] engine: Evacuate object from shards concurrently
...
Tests and linters / Run gofumpt (pull_request) Successful in 1m34s
DCO action / DCO (pull_request) Successful in 1m45s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m43s
Vulncheck / Vulncheck (pull_request) Successful in 2m35s
Tests and linters / Staticcheck (pull_request) Successful in 2m44s
Build / Build Components (pull_request) Successful in 3m19s
Tests and linters / gopls check (pull_request) Successful in 3m26s
Tests and linters / Lint (pull_request) Successful in 3m37s
Tests and linters / Tests with -race (pull_request) Successful in 4m44s
Tests and linters / Tests (pull_request) Successful in 5m13s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-09-24 12:50:19 +03:00
839dead226
[ #1297 ] getSvc: Return AccessDenied instead of ObjectNotFound
...
Do not replace the access denied error if it was received earlier.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-24 12:05:09 +03:00
3bb65ba820
[ #1392 ] object: Fix target initialization within put streamer
...
* Remove `relay` field from put streamer as it's no longer used;
* Fix initialization of `Relay` object writer parameter.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-24 09:04:13 +00:00
d4493a6d08
[ #1390 ] getSvc: Fix Head EC1.1
...
Tests and linters / Run gofumpt (pull_request) Successful in 2m16s
DCO action / DCO (pull_request) Successful in 2m32s
Tests and linters / Staticcheck (pull_request) Successful in 3m37s
Vulncheck / Vulncheck (pull_request) Successful in 3m44s
Build / Build Components (pull_request) Successful in 3m58s
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m56s
Tests and linters / gopls check (pull_request) Successful in 4m26s
Tests and linters / Lint (pull_request) Successful in 5m8s
Tests and linters / Tests (pull_request) Successful in 5m48s
Tests and linters / Tests with -race (pull_request) Successful in 6m22s
If local EC chunk found, but remote node is off, then `HEAD --raw` request
returns object not found.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-23 15:12:06 +03:00
0b87be804a
[ #1381 ] engine: Fix tests
...
DCO action / DCO (pull_request) Successful in 1m35s
Tests and linters / Run gofumpt (pull_request) Successful in 1m37s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m40s
Vulncheck / Vulncheck (pull_request) Successful in 2m35s
Tests and linters / Staticcheck (pull_request) Successful in 2m52s
Build / Build Components (pull_request) Successful in 3m33s
Tests and linters / gopls check (pull_request) Successful in 3m35s
Tests and linters / Lint (pull_request) Successful in 4m44s
Tests and linters / Tests (pull_request) Successful in 5m14s
Tests and linters / Tests with -race (pull_request) Successful in 6m1s
Drop not required `Eventually` calls.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-23 11:10:29 +03:00
c34b8acedd
[ #1312 ] Drop handling of system attributes with NeoFS prefix
...
DCO action / DCO (pull_request) Successful in 1m2s
Tests and linters / Run gofumpt (pull_request) Successful in 1m14s
Vulncheck / Vulncheck (pull_request) Successful in 1m46s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m16s
Build / Build Components (pull_request) Successful in 2m29s
Tests and linters / Staticcheck (pull_request) Successful in 2m42s
Tests and linters / gopls check (pull_request) Successful in 3m5s
Tests and linters / Lint (pull_request) Successful in 3m32s
Tests and linters / Tests (pull_request) Successful in 4m14s
Tests and linters / Tests with -race (pull_request) Successful in 5m8s
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-20 11:37:18 +00:00
61d5e140e0
[ #1383 ] object: Add restrictions for Patch
method
...
DCO action / DCO (pull_request) Successful in 58s
Tests and linters / Run gofumpt (pull_request) Successful in 1m15s
Vulncheck / Vulncheck (pull_request) Successful in 1m26s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m6s
Build / Build Components (pull_request) Successful in 2m20s
Tests and linters / gopls check (pull_request) Successful in 2m34s
Tests and linters / Staticcheck (pull_request) Successful in 2m49s
Tests and linters / Lint (pull_request) Successful in 3m41s
Tests and linters / Tests (pull_request) Successful in 4m46s
Tests and linters / Tests with -race (pull_request) Successful in 4m58s
* `Patch` can't be applied for non-regular type object (tombstones,
locks etc.)
* Complex object parts can't be patched. So, if an object has EC/Split
header, it won't be patched.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-19 13:54:49 +03:00
a603d14d08
[ #1382 ] ir: Replace deprecated methods
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-18 14:29:31 +03:00
d4be2f20d4
[ #1382 ] morph: Replace deprecated methods
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-18 14:29:27 +03:00
e5c8f7ff9f
[ #1382 ] controlSvc: Replace deprecated methods
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-18 14:29:24 +03:00
1e7f9909da
[ #1382 ] policer: Replace deprecated methods
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-18 14:29:21 +03:00
ea48e928c8
[ #1366 ] logger: Make timestamp prepending optional
...
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-17 10:49:51 +03:00
89d0435b1d
[ #1374 ] tree: Use NewClient to create grpc connection in cache
...
DCO action / DCO (pull_request) Successful in 1m2s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m53s
Vulncheck / Vulncheck (pull_request) Successful in 2m4s
Build / Build Components (pull_request) Successful in 2m39s
Tests and linters / Run gofumpt (pull_request) Successful in 2m30s
Tests and linters / Staticcheck (pull_request) Successful in 2m38s
Tests and linters / gopls check (pull_request) Successful in 3m2s
Tests and linters / Lint (pull_request) Successful in 3m14s
Tests and linters / Tests (pull_request) Successful in 6m4s
Tests and linters / Tests with -race (pull_request) Successful in 6m7s
Created grpc connection should be established, so perform Healthcheck request
to check connection is ok.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-13 15:59:26 +03:00
54fe8383a4
[ #1374 ] tree: Use NewClient to create grpc connection for sync
...
Created connection will be used to sync trees, so it is ok to defer
dial to the first RPC call.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-13 15:59:26 +03:00
546d09660f
[ #1283 ] Clear systemd-notify status on exit
...
DCO action / DCO (pull_request) Successful in 3m9s
Vulncheck / Vulncheck (pull_request) Successful in 3m13s
Tests and linters / Run gofumpt (pull_request) Successful in 3m26s
Tests and linters / Staticcheck (pull_request) Successful in 4m9s
Tests and linters / Tests with -race (pull_request) Successful in 4m23s
Tests and linters / Lint (pull_request) Successful in 4m49s
Pre-commit hooks / Pre-commit (pull_request) Successful in 5m33s
Tests and linters / gopls check (pull_request) Successful in 5m59s
Build / Build Components (pull_request) Successful in 6m26s
Tests and linters / Tests (pull_request) Successful in 10m23s
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-12 14:45:06 +00:00
e3764c51df
[ #1347 ] metabase: Fix EC search
...
For EC chunks need to return EC parent object ID as
EC chunks don't have own attributes but inherit parent's.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 13:23:29 +00:00
b33559754d
[ #1367 ] fstree: Add size hint for Delete
...
DCO action / DCO (pull_request) Successful in 1m2s
Vulncheck / Vulncheck (pull_request) Successful in 1m20s
Tests and linters / Run gofumpt (pull_request) Successful in 1m44s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m6s
Build / Build Components (pull_request) Successful in 2m19s
Tests and linters / gopls check (pull_request) Successful in 2m34s
Tests and linters / Staticcheck (pull_request) Successful in 2m36s
Tests and linters / Lint (pull_request) Successful in 3m29s
Tests and linters / Tests (pull_request) Successful in 4m0s
Tests and linters / Tests with -race (pull_request) Successful in 5m48s
This allow to not to call `os.Stat` if caller already knows data size.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 15:06:33 +03:00
f345fe9a58
[ #1367 ] writecache: Move DB related code to upgrade.go
...
This is done to drop this file in the future.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 15:06:33 +03:00
3b236160a6
[ #1367 ] writecache: Drop DB label from metrics
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 15:06:33 +03:00
25d2ae8aaf
[ #1367 ] writecache: Drop BBolt related config variables
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 15:06:33 +03:00
e39378b1c3
[ #1367 ] writecache: Add background flushing objects limiter
...
To limit memory usage by background flush.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 15:06:33 +03:00
8a6e3025a0
[ #1367 ] writecache: Flush from FSTree concurrently
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 15:06:33 +03:00
2dd3a6f7a8
[ #1367 ] fstree: Add IterateInfo method
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 15:06:33 +03:00
b142b6f48e
[ #1367 ] fstree: Add size to file counter
...
FSTree file counter used by writecache. As writecache has now only one
storage, so it is required to use real object size to get writecache
size more accurate than `count * max_object_size`.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 15:06:33 +03:00
5f6c7cbdb1
[ #1367 ] writecache: Drop bbolt DB
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 14:22:29 +03:00
99be4c83a7
[ #1368 ] *: Run gofumpt
...
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-12 10:00:28 +03:00
5fac4058e8
[ #1364 ] cmd/common: Add tests for CreateViper and ReloadViper
...
DCO action / DCO (pull_request) Successful in 51s
Tests and linters / Run gofumpt (pull_request) Successful in 1m41s
Vulncheck / Vulncheck (pull_request) Successful in 1m54s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m13s
Build / Build Components (pull_request) Successful in 2m21s
Tests and linters / gopls check (pull_request) Successful in 2m31s
Tests and linters / Staticcheck (pull_request) Successful in 2m34s
Tests and linters / Lint (pull_request) Successful in 3m30s
Tests and linters / Tests (pull_request) Successful in 3m35s
Tests and linters / Tests with -race (pull_request) Successful in 3m37s
Add tests for `CreateViper` and `ReloadViper` to ensure that no extra
files, except *.yaml, *.yml, *.json, are loaded from config directory.
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-11 08:03:05 +00:00
a812932984
[ #1362 ] ape: Move common APE check logic to separate package
...
* Tree and object service have the same log for checking APE. So,
this check should be moved to common package.
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-10 12:40:34 +00:00
92fe5d90f5
[ #1359 ] writecache: Drop unused
...
DCO action / DCO (pull_request) Successful in 2m45s
Vulncheck / Vulncheck (pull_request) Successful in 3m13s
Tests and linters / Run gofumpt (pull_request) Successful in 3m48s
Tests and linters / Staticcheck (pull_request) Successful in 3m57s
Tests and linters / Tests with -race (pull_request) Successful in 4m15s
Tests and linters / Lint (pull_request) Successful in 4m38s
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m40s
Tests and linters / gopls check (pull_request) Successful in 4m41s
Build / Build Components (pull_request) Successful in 5m10s
Tests and linters / Tests (pull_request) Successful in 6m20s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-09 18:45:21 +03:00
4668efc0bf
[ #1355 ] metabase: Upgrade improvements
...
Vulncheck / Vulncheck (pull_request) Successful in 1m16s
DCO action / DCO (pull_request) Successful in 1m31s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m5s
Tests and linters / Run gofumpt (pull_request) Successful in 1m59s
Build / Build Components (pull_request) Successful in 2m13s
Tests and linters / gopls check (pull_request) Successful in 2m31s
Tests and linters / Staticcheck (pull_request) Successful in 2m38s
Tests and linters / Lint (pull_request) Successful in 3m15s
Tests and linters / Tests (pull_request) Successful in 3m32s
Tests and linters / Tests with -race (pull_request) Successful in 3m39s
Do not fail on same latest version to run compact on upgraded metabase.
Use NoSync on compact.
Log every batch on bucket delete stage.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-09 11:37:36 +03:00
d3b209c8e1
[ #1337 ] shard: Disable background rebuild
...
Vulncheck / Vulncheck (pull_request) Successful in 1m3s
DCO action / DCO (pull_request) Successful in 1m28s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m2s
Build / Build Components (pull_request) Successful in 2m15s
Tests and linters / Run gofumpt (pull_request) Successful in 2m10s
Tests and linters / gopls check (pull_request) Successful in 2m21s
Tests and linters / Staticcheck (pull_request) Successful in 2m55s
Tests and linters / Lint (pull_request) Successful in 3m29s
Tests and linters / Tests (pull_request) Successful in 3m47s
Tests and linters / Tests with -race (pull_request) Successful in 3m58s
Since `frostfs-cli control shards rebuild` command was added,
there is no need for background rebuild now.
For failover tests used used value 1 to rebuild only schema change.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-06 15:19:55 +03:00
edb1747af7
[ #1337 ] blobovniczatree: Add rebuild by overflow
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-06 15:19:54 +03:00
a61201a987
[ #1337 ] config: Move rebuild_worker_count
to shard section
...
This makes it simple to limit performance degradation for every shard
because of rebuild.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-06 13:57:27 +03:00
6b6eabe41c
[ #1337 ] cli: Add control shards rebuild
command
...
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-06 13:57:27 +03:00