TrueCloudLab
/
frostfs-node
18
1
Fork 21
Code Issues 89 Pull Requests 8 Actions Releases Activity
4,549 Commits
7 Branches
23 Tags
48 MiB
Commit Graph

4549 Commits (dacf580b8777250523f4999c3bb6beebd4800336)

Author SHA1 Message Date
Anton Nikiforov 7ade11922e [#834] adm: Refactor `morph` module 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-12-20 15:22:17 +03:00
Dmitrii Stepanov d69d318cb0 [#878] node: Drain internal error's channel 
This fixes shutdown panic:
1. Some morph connection gets error and passes it to internalErr channel.
2. Storage node starts to shutdow and closes internalErr channel.
3. Other morph connection gets error and tries to pass it to internalErr channel.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-19 16:38:03 +00:00
Anton Nikiforov d9cbb16bd3 [#866] Use TTL for blobovnicza tree cache 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-12-19 16:36:28 +00:00
Ekaterina Lebedeva be8f499b91 [#717] Fix mixing CID and container ID
Vulncheck / Vulncheck (pull_request) Successful in 1m13s Details
DCO action / DCO (pull_request) Successful in 2m43s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m11s Details
Tests and linters / Lint (pull_request) Successful in 5m20s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m13s Details
Build / Build Components (1.21) (pull_request) Successful in 4m22s Details
Build / Build Components (1.20) (pull_request) Successful in 4m35s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m39s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m2s Details 
All container ID outputs that may be used in scripts
were replaced with CID.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2023-12-19 12:36:29 +03:00
Airat Arifullin 7d7cf05575 [#851] ape: Initialize and use policy contract interface 
* Replace inmemory policy contract mock by initialized
  policy contract interface.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-12-18 11:06:35 +00:00
Dmitrii Stepanov 61da7dca24 [#835] node: Fix appCfg concurrent access
DCO action / DCO (pull_request) Successful in 4m37s Details
Build / Build Components (1.21) (pull_request) Successful in 6m15s Details
Build / Build Components (1.20) (pull_request) Successful in 6m44s Details
Tests and linters / Lint (pull_request) Successful in 8m46s Details
Tests and linters / Staticcheck (pull_request) Successful in 8m30s Details
Tests and linters / Tests (1.20) (pull_request) Failing after 11m2s Details
Vulncheck / Vulncheck (pull_request) Successful in 11m36s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 12m26s Details
Tests and linters / Tests with -race (pull_request) Successful in 12m51s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-14 16:38:59 +03:00
Dmitrii Stepanov f4877e7b42 [#835] grpc: Try to reconnect if endpoint listen failed 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-14 16:38:59 +03:00
Airat Arifullin bdd43f6211 [#869] object: Pass just CID to chain router 
* Do not convert CID from request to native-schema resource
  format - this step is unneccessary for APE.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-12-14 11:01:20 +00:00
Airat Arifullin 4a64b07703 [#869] cli: Pass only CID in requests for control API 
* Fix add-rule, list-rules, remove-rule, get-rule commands:
  do not convert container ID to native-schema resource format
  and pass it to control API.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-12-14 11:01:20 +00:00
Ekaterina Lebedeva 2d4c0a0f4a [#552] Add systemd notifications to ir service
DCO action / DCO (pull_request) Successful in 3m19s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m25s Details
Build / Build Components (1.21) (pull_request) Successful in 4m27s Details
Build / Build Components (1.20) (pull_request) Successful in 4m41s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m5s Details
Tests and linters / Lint (pull_request) Successful in 5m57s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 13m25s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 14m13s Details
Tests and linters / Tests with -race (pull_request) Successful in 16m15s Details 
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2023-12-13 17:51:41 +03:00
Ekaterina Lebedeva ef07c1a3c9 [#552] Add sysd notifications to storage service 
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2023-12-13 17:51:41 +03:00
Ekaterina Lebedeva eca7ac9f0d [#552] Add sdnotify package 
To avoid using third-party dependencies.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2023-12-13 17:49:26 +03:00
Ekaterina Lebedeva 9b2dce5763 [#552] Add status notification to systemd 
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2023-12-13 15:02:39 +03:00
Ekaterina Lebedeva 05f8f49289 [#552] gofumpt changes 
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2023-12-13 15:02:25 +03:00
Dmitrii Stepanov 7eb46404a1 [#863] blobovnicza: Fix counters
Vulncheck / Vulncheck (pull_request) Successful in 2m27s Details
DCO action / DCO (pull_request) Successful in 2m39s Details
Build / Build Components (1.21) (pull_request) Successful in 3m38s Details
Build / Build Components (1.20) (pull_request) Successful in 3m45s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m15s Details
Tests and linters / Lint (pull_request) Successful in 5m47s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m49s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 12m16s Details
Tests and linters / Tests with -race (pull_request) Successful in 12m31s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-13 13:34:29 +03:00
Dmitrii Stepanov 11add38e87 [#857] golangci: Add protogetter linter
DCO action / DCO (pull_request) Successful in 1m38s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m2s Details
Build / Build Components (1.21) (pull_request) Successful in 3m51s Details
Build / Build Components (1.20) (pull_request) Successful in 4m3s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m17s Details
Tests and linters / Lint (pull_request) Successful in 6m11s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 11m7s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 11m32s Details
Tests and linters / Tests with -race (pull_request) Successful in 12m22s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 16:27:02 +03:00
Dmitrii Stepanov 94ffe8bb45 [#857] golangci: Add testifylint linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 16:27:02 +03:00
Dmitrii Stepanov 5d7833c89b [#857] golangci: Add perfsprint linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 16:27:02 +03:00
Dmitrii Stepanov d2746a7d67 [#857] Makefile: Update linter version 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 16:27:02 +03:00
Dmitrii Stepanov 3b7c0362a8 [#861] shard: Fix Delete object
DCO action / DCO (pull_request) Successful in 12m22s Details
Build / Build Components (1.20) (pull_request) Successful in 12m44s Details
Build / Build Components (1.21) (pull_request) Successful in 13m2s Details
Vulncheck / Vulncheck (pull_request) Successful in 13m3s Details
Tests and linters / Staticcheck (pull_request) Successful in 15m32s Details
Tests and linters / Lint (pull_request) Successful in 16m26s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 26m11s Details
Tests and linters / Tests with -race (pull_request) Successful in 28m29s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 3m28s Details 
It is possible that object doesn't exist in metabase.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 14:25:40 +03:00
Dmitrii Stepanov 681b2c5fd4 [#825] policer: Do not drop required linking objects 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 11:04:03 +00:00
Dmitrii Stepanov a3ef7b58b4 [#755] innerring: Check container owner namespace
Vulncheck / Vulncheck (pull_request) Successful in 1m12s Details
DCO action / DCO (pull_request) Successful in 2m43s Details
Build / Build Components (1.21) (pull_request) Successful in 4m8s Details
Build / Build Components (1.20) (pull_request) Successful in 4m18s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m19s Details
Tests and linters / Lint (pull_request) Successful in 5m30s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 12m2s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 12m24s Details
Tests and linters / Tests with -race (pull_request) Successful in 13m4s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 12:36:34 +03:00
Dmitrii Stepanov 1cd2bfe51a [#755] morph: Drop FrostFSID contract usage 
Unused.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-12 12:36:34 +03:00
Dmitrii Stepanov d5c9dd3c83 [#852] ape: Use first match for eACL->APE converter
DCO action / DCO (pull_request) Successful in 8m50s Details
Vulncheck / Vulncheck (pull_request) Successful in 9m19s Details
Build / Build Components (1.21) (pull_request) Successful in 10m43s Details
Build / Build Components (1.20) (pull_request) Successful in 10m51s Details
Tests and linters / Staticcheck (pull_request) Successful in 10m48s Details
Tests and linters / Lint (pull_request) Successful in 12m4s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 13m58s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 14m21s Details
Tests and linters / Tests with -race (pull_request) Successful in 15m18s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-11 16:55:32 +03:00
Anton Nikiforov 46532fb9ce [#841] doc: Describe `epoch` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-12-11 13:14:41 +00:00
Anton Nikiforov 70e0c1e082 [#841] ir: Execute `netmap.addPeerIR` only for state `online` 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-12-11 13:14:41 +00:00
Airat Arifullin 0f45e3d344 [#804] ape: Implement boltdb storage for local overrides
DCO action / DCO (pull_request) Successful in 2m10s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m26s Details
Build / Build Components (1.20) (pull_request) Successful in 5m41s Details
Build / Build Components (1.21) (pull_request) Successful in 5m44s Details
Tests and linters / Staticcheck (pull_request) Successful in 7m10s Details
Tests and linters / Lint (pull_request) Successful in 8m14s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 14m24s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 14m41s Details
Tests and linters / Tests with -race (pull_request) Successful in 14m38s Details 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-12-07 19:08:41 +03:00
Airat Arifullin e361e017f3 [#842] control: Pass target instead resource name 
* Update policy-engine package version in go.mod, go.sum.
* Refactor CheckIfRequestPermitted: pass container target
  instead container ID.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-12-07 14:21:55 +00:00
Airat Arifullin 39060382a1 [#842] control: Recieve target in gRPC methods for APE managing 
* Introduce Target type and pass it to all gRPC methods
  for APE chain managing instead CID.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-12-07 14:21:55 +00:00
Dmitrii Stepanov db49ad16cc [#826] blobovniczatree: Do not create DB's on init
DCO action / DCO (pull_request) Successful in 4m10s Details
Build / Build Components (1.21) (pull_request) Successful in 7m30s Details
Vulncheck / Vulncheck (pull_request) Successful in 7m5s Details
Tests and linters / Staticcheck (pull_request) Successful in 10m6s Details
Tests and linters / Lint (pull_request) Successful in 10m26s Details
Build / Build Components (1.20) (pull_request) Successful in 12m39s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 16m55s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 17m13s Details
Tests and linters / Tests with -race (pull_request) Successful in 17m14s Details 
Blobovniczas will be created on write requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov ad0697adc4 [#661] blobovnicza: Compute size with record size 
To get more accurate size of blobovnicza use record
size (lenght of key + lenght of data).

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov e54dc3dc7c [#698] blobovnicza: Store counter values 
Blobovnicza initialization take a long time because of bucket
Stat() call. So now blobovnicza stores counters in META bucket.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov 5e8c08da3e [#661] blobstore: Add address to error logs 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov 8911656b1a [#661] metrcis: Add rebuild percent metric 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov 8bbfb2df43 [#661] blobovniczatree: Pass object size limit from config 
If actual small object size value lower than default
object size limit, then unnecessary buckets created.
If actual small object size value greated than default
object size limit, then error happens.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov 2407e5f5ff [#661] blobovniczatree: Do not sort DB's and indicies 
Put stores object to next active DB, so there is no need to sort DBs.
In addition, it adds unnecessary DB openings.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov c6a739e746 [#661] blobovniczatree: Make Rebuild concurrent for objects 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov f1c7905263 [#661] blobovniczatree: Make Rebuild concurrent 
Different DBs can be rebuild concurrently.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov d4d905ecc6 [#661] metrics: Add blobovniczatree rebuild metrics 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov b2769ca3de [#661] blobovniczatree: Make Rebuild failover safe 
Now move info stores in blobovnicza, so in case of failover
rebuild completes previous operation first.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov da4fee2d0b [#698] blobovniczatree: Init blobovniczas concurrently 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:33 +03:00
Dmitrii Stepanov 422226da18 [#661] blobovniczatree: Add Rebuild implementation 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:32 +03:00
Dmitrii Stepanov a531eaf8bc [#661] blobstor: Add Rebuild implementation 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:32 +03:00
Dmitrii Stepanov c1667a11d2 [#661] blobovniczatree: Allow to change depth or width 
Now it is possible to change depth or with of blobovniczatree.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:32 +03:00
Dmitrii Stepanov 484eb59893 [#661] blobovniczatree: Use .db extension for db files 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:32 +03:00
Dmitrii Stepanov 44552a849b [#661] shard: Add blobstor rebuilder 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-07 15:37:32 +03:00
Dmitrii Stepanov a478050639 [#838] metabase: Resolve funlen linter
DCO action / DCO (pull_request) Successful in 1m59s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m0s Details
Build / Build Components (1.21) (pull_request) Successful in 3m40s Details
Tests and linters / Lint (pull_request) Successful in 4m57s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m38s Details
Build / Build Components (1.20) (pull_request) Successful in 5m17s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 7m1s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m39s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m46s Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-06 15:44:21 +03:00
Dmitrii Stepanov d30ab5f29e [#838] metabase: Count user objects 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-06 15:44:21 +03:00
Dmitrii Stepanov f314da4af3 [#838] metabase: Add user object type counter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-06 15:44:21 +03:00
Dmitrii Stepanov 29550fe600 [#838] shard: Refactor `updateMetrics` method 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-12-06 15:44:21 +03:00