Commit graph

4523 commits

Author SHA1 Message Date
0f45e3d344 [#804] ape: Implement boltdb storage for local overrides
All checks were successful
DCO action / DCO (pull_request) Successful in 2m10s
Vulncheck / Vulncheck (pull_request) Successful in 3m26s
Build / Build Components (1.20) (pull_request) Successful in 5m41s
Build / Build Components (1.21) (pull_request) Successful in 5m44s
Tests and linters / Staticcheck (pull_request) Successful in 7m10s
Tests and linters / Lint (pull_request) Successful in 8m14s
Tests and linters / Tests (1.21) (pull_request) Successful in 14m24s
Tests and linters / Tests (1.20) (pull_request) Successful in 14m41s
Tests and linters / Tests with -race (pull_request) Successful in 14m38s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 19:08:41 +03:00
e361e017f3 [#842] control: Pass target instead resource name
* Update policy-engine package version in go.mod, go.sum.
* Refactor CheckIfRequestPermitted: pass container target
  instead container ID.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 14:21:55 +00:00
39060382a1 [#842] control: Recieve target in gRPC methods for APE managing
* Introduce Target type and pass it to all gRPC methods
  for APE chain managing instead CID.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-12-07 14:21:55 +00:00
db49ad16cc [#826] blobovniczatree: Do not create DB's on init
All checks were successful
DCO action / DCO (pull_request) Successful in 4m10s
Build / Build Components (1.21) (pull_request) Successful in 7m30s
Vulncheck / Vulncheck (pull_request) Successful in 7m5s
Tests and linters / Staticcheck (pull_request) Successful in 10m6s
Tests and linters / Lint (pull_request) Successful in 10m26s
Build / Build Components (1.20) (pull_request) Successful in 12m39s
Tests and linters / Tests (1.20) (pull_request) Successful in 16m55s
Tests and linters / Tests (1.21) (pull_request) Successful in 17m13s
Tests and linters / Tests with -race (pull_request) Successful in 17m14s
Blobovniczas will be created on write requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
ad0697adc4 [#661] blobovnicza: Compute size with record size
To get more accurate size of blobovnicza use record
size (lenght of key + lenght of data).

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
e54dc3dc7c [#698] blobovnicza: Store counter values
Blobovnicza initialization take a long time because of bucket
Stat() call. So now blobovnicza stores counters in META bucket.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
5e8c08da3e [#661] blobstore: Add address to error logs
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
8911656b1a [#661] metrcis: Add rebuild percent metric
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
8bbfb2df43 [#661] blobovniczatree: Pass object size limit from config
If actual small object size value lower than default
object size limit, then unnecessary buckets created.
If actual small object size value greated than default
object size limit, then error happens.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
2407e5f5ff [#661] blobovniczatree: Do not sort DB's and indicies
Put stores object to next active DB, so there is no need to sort DBs.
In addition, it adds unnecessary DB openings.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
c6a739e746 [#661] blobovniczatree: Make Rebuild concurrent for objects
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
f1c7905263 [#661] blobovniczatree: Make Rebuild concurrent
Different DBs can be rebuild concurrently.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
d4d905ecc6 [#661] metrics: Add blobovniczatree rebuild metrics
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
b2769ca3de [#661] blobovniczatree: Make Rebuild failover safe
Now move info stores in blobovnicza, so in case of failover
rebuild completes previous operation first.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
da4fee2d0b [#698] blobovniczatree: Init blobovniczas concurrently
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:33 +03:00
422226da18 [#661] blobovniczatree: Add Rebuild implementation
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:32 +03:00
a531eaf8bc [#661] blobstor: Add Rebuild implementation
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:32 +03:00
c1667a11d2 [#661] blobovniczatree: Allow to change depth or width
Now it is possible to change depth or with of blobovniczatree.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:32 +03:00
484eb59893 [#661] blobovniczatree: Use .db extension for db files
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:32 +03:00
44552a849b [#661] shard: Add blobstor rebuilder
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-07 15:37:32 +03:00
a478050639 [#838] metabase: Resolve funlen linter
All checks were successful
DCO action / DCO (pull_request) Successful in 1m59s
Vulncheck / Vulncheck (pull_request) Successful in 3m0s
Build / Build Components (1.21) (pull_request) Successful in 3m40s
Tests and linters / Lint (pull_request) Successful in 4m57s
Tests and linters / Staticcheck (pull_request) Successful in 4m38s
Build / Build Components (1.20) (pull_request) Successful in 5m17s
Tests and linters / Tests (1.20) (pull_request) Successful in 7m1s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m39s
Tests and linters / Tests with -race (pull_request) Successful in 7m46s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-06 15:44:21 +03:00
d30ab5f29e [#838] metabase: Count user objects
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-06 15:44:21 +03:00
f314da4af3 [#838] metabase: Add user object type counter
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-06 15:44:21 +03:00
29550fe600 [#838] shard: Refactor updateMetrics method
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-12-06 15:44:21 +03:00
b892feeaf6 [#845] adm: Relax notary-enabled check
Starting from v0.104.0 `NativeActivations` config field is no longer
present and Notary activation height is always 0.

https://github.com/nspcc-dev/neo-go/pull/3212/
TrueCloudLab/frostfs-dev-env#59

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-12-06 11:08:04 +00:00
6bb27f98dd [#837] .pre-commit: Update hook versions
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-12-06 11:07:10 +00:00
44806aa9f1 [#837] go.mod: Update dependencies
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-12-06 11:07:10 +00:00
f1db468d48 [#840] adm: Update FrostFS ID deploy arguments
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 1m28s
DCO action / DCO (pull_request) Successful in 3m9s
Build / Build Components (1.21) (pull_request) Successful in 4m15s
Build / Build Components (1.20) (pull_request) Successful in 4m35s
Tests and linters / Staticcheck (pull_request) Successful in 4m58s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m48s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m6s
Tests and linters / Lint (pull_request) Successful in 7m25s
Tests and linters / Tests with -race (pull_request) Successful in 7m9s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-12-04 17:39:41 +03:00
b2c63e57ba [#651] engine/test: Speedup StorageEngine_Inhume
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-30 13:19:43 +00:00
445ebcc0e7 [#651] shard/test: Speedup Shard_Delete
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-30 13:19:43 +00:00
2302e5d342 [#651] shard/test: Refactor Shard_Delete
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-30 13:19:43 +00:00
a982c3df18 [#824] cli: Support passing chain ID in add-rule command
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-30 13:13:46 +00:00
7f6852bbd2 [#639] node: Refactor TTL cache
Migrate from internal to external TTL implementation

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2023-11-30 12:54:51 +00:00
26e4f7005c [#741] treesvc: Refactor tree sync
Fix linter issues.
Add error logging.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-30 12:45:02 +00:00
b21be1abdd [#741] treesvc: Do not update sync height if some node is unavailable
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-30 12:45:02 +00:00
b215817e14 [#741] treesvc: Remove unused height variables
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-30 12:45:02 +00:00
306f12e6c5 [#828] adm: Fix policy contract deploy
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-29 06:23:56 +00:00
5521737f0b [#808] cli: Use EnableTraverseRunHooks in cobra
Adopt EnableTraverseRunHooks to get rid of tracing boilerplate in
multiple commands. Now adding `--trace` flag is sufficient for a command
to support tracing. Finally, it looks how it _should_.

Refs #406

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-27 09:58:19 +00:00
e81a58b8da [#808] cli: Use MarkFlagsOneRequired after cobra update
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-27 09:58:19 +00:00
5048236441 [#808] go.mod: Update dependencies
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2023-11-27 09:58:19 +00:00
c516c7c5f4 [#821] node: Pass user.ID by value
All checks were successful
DCO action / DCO (pull_request) Successful in 3m45s
Build / Build Components (1.21) (pull_request) Successful in 5m18s
Build / Build Components (1.20) (pull_request) Successful in 5m28s
Tests and linters / Tests (1.20) (pull_request) Successful in 7m30s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m42s
Tests and linters / Lint (pull_request) Successful in 8m25s
Vulncheck / Vulncheck (pull_request) Successful in 9m22s
Tests and linters / Staticcheck (pull_request) Successful in 10m57s
Tests and linters / Tests with -race (pull_request) Successful in 16m53s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-23 10:21:07 +03:00
c99157f0b2 [#821] go.mod: Update SDK-Go version
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-23 10:21:06 +03:00
07390ad4e3 [#715] node: Unify config parameter names
All checks were successful
DCO action / DCO (pull_request) Successful in 2m54s
Vulncheck / Vulncheck (pull_request) Successful in 3m11s
Tests and linters / Staticcheck (pull_request) Successful in 3m55s
Build / Build Components (1.21) (pull_request) Successful in 3m51s
Build / Build Components (1.20) (pull_request) Successful in 4m6s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m10s
Tests and linters / Lint (pull_request) Successful in 5m24s
Tests and linters / Tests (1.20) (pull_request) Successful in 6m56s
Tests and linters / Tests with -race (pull_request) Successful in 7m46s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-22 17:13:50 +03:00
8d18fa159e [#667] writecache: Fix flush test
All checks were successful
DCO action / DCO (pull_request) Successful in 1m28s
Vulncheck / Vulncheck (pull_request) Successful in 3m55s
Tests and linters / Staticcheck (pull_request) Successful in 4m15s
Build / Build Components (1.21) (pull_request) Successful in 4m40s
Build / Build Components (1.20) (pull_request) Successful in 4m54s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m32s
Tests and linters / Lint (pull_request) Successful in 5m59s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m59s
Tests and linters / Tests with -race (pull_request) Successful in 6m57s
Allow to disable background flusher for testing purposes.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-17 17:45:43 +03:00
02454df14a [#809] client: Refactor PrmInit, PrmDial usage
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-17 13:37:03 +00:00
76ff26039c [#96] node: Drop neo-go's slices package
All checks were successful
DCO action / DCO (pull_request) Successful in 2m13s
Build / Build Components (1.20) (pull_request) Successful in 5m48s
Build / Build Components (1.21) (pull_request) Successful in 5m47s
Vulncheck / Vulncheck (pull_request) Successful in 5m47s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m33s
Tests and linters / Staticcheck (pull_request) Successful in 7m37s
Tests and linters / Lint (pull_request) Successful in 8m8s
Tests and linters / Tests (1.20) (pull_request) Successful in 10m38s
Tests and linters / Tests with -race (pull_request) Successful in 11m22s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-17 13:24:04 +03:00
47286ebf32 [#805] pilorama: Fix TreeDrop
* If treeID is empty then deleting buckets for cursor may get
  invalidated. So, buckets should be gathered before deleting.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2023-11-17 10:21:35 +00:00
5cfb758e4e [#806] morph: Remove container list cache
All checks were successful
DCO action / DCO (pull_request) Successful in 2m48s
Vulncheck / Vulncheck (pull_request) Successful in 3m37s
Tests and linters / Staticcheck (pull_request) Successful in 4m0s
Build / Build Components (1.21) (pull_request) Successful in 4m24s
Build / Build Components (1.20) (pull_request) Successful in 4m48s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m26s
Tests and linters / Tests (1.21) (pull_request) Successful in 5m22s
Tests and linters / Lint (pull_request) Successful in 5m37s
Tests and linters / Tests with -race (pull_request) Successful in 8m7s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-17 10:40:08 +03:00
29fe8c41f3 [#655] storage: Drop ErrorHandler
All checks were successful
DCO action / DCO (pull_request) Successful in 1m42s
Build / Build Components (1.21) (pull_request) Successful in 3m23s
Vulncheck / Vulncheck (pull_request) Successful in 3m25s
Build / Build Components (1.20) (pull_request) Successful in 4m16s
Tests and linters / Staticcheck (pull_request) Successful in 4m24s
Tests and linters / Lint (pull_request) Successful in 5m2s
Tests and linters / Tests (1.20) (pull_request) Successful in 5m36s
Tests and linters / Tests (1.21) (pull_request) Successful in 6m42s
Tests and linters / Tests with -race (pull_request) Successful in 7m18s
The only one usage was for logging.
Now logging performed by storage anyway.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-16 17:27:38 +03:00
137e987a4e [#655] storage: Drop LazyHandler
LazyHandler is implemented and used incorrectly.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2023-11-16 17:27:38 +03:00