Commit graph

1791 commits

Author SHA1 Message Date
e39378b1c3 [#1367] writecache: Add background flushing objects limiter
To limit memory usage by background flush.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-12 15:06:33 +03:00
66e17f4b8e
[#1368] cli/container: Use dedicated method to list user attributes
All checks were successful
DCO action / DCO (pull_request) Successful in 1m45s
Vulncheck / Vulncheck (pull_request) Successful in 2m11s
Tests and linters / Run gofumpt (pull_request) Successful in 2m37s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m50s
Build / Build Components (pull_request) Successful in 3m6s
Tests and linters / Staticcheck (pull_request) Successful in 3m1s
Tests and linters / Lint (pull_request) Successful in 3m57s
Tests and linters / gopls check (pull_request) Successful in 4m44s
Tests and linters / Tests (pull_request) Successful in 5m56s
Tests and linters / Tests with -race (pull_request) Successful in 6m0s
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-12 10:01:27 +03:00
ec8da40567 [#1369] Update obsolete URLs
Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
2024-09-11 11:31:10 +00:00
dea6f031f9 [#1331] cli/tree: Add order flag to tree get-subtree
Added `--ordered` flag to sort output by ascending FileName.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-09-11 11:30:28 +00:00
5fac4058e8 [#1364] cmd/common: Add tests for CreateViper and ReloadViper
All checks were successful
DCO action / DCO (pull_request) Successful in 51s
Tests and linters / Run gofumpt (pull_request) Successful in 1m41s
Vulncheck / Vulncheck (pull_request) Successful in 1m54s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m13s
Build / Build Components (pull_request) Successful in 2m21s
Tests and linters / gopls check (pull_request) Successful in 2m31s
Tests and linters / Staticcheck (pull_request) Successful in 2m34s
Tests and linters / Lint (pull_request) Successful in 3m30s
Tests and linters / Tests (pull_request) Successful in 3m35s
Tests and linters / Tests with -race (pull_request) Successful in 3m37s
Add tests for `CreateViper` and `ReloadViper` to ensure that no extra
files, except *.yaml, *.yml, *.json, are loaded from config directory.

Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-11 08:03:05 +00:00
654d970fad [#1355] adm: Run metabase upgrade concurrently
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-06 15:54:40 +03:00
d3b209c8e1 [#1337] shard: Disable background rebuild
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 1m3s
DCO action / DCO (pull_request) Successful in 1m28s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m2s
Build / Build Components (pull_request) Successful in 2m15s
Tests and linters / Run gofumpt (pull_request) Successful in 2m10s
Tests and linters / gopls check (pull_request) Successful in 2m21s
Tests and linters / Staticcheck (pull_request) Successful in 2m55s
Tests and linters / Lint (pull_request) Successful in 3m29s
Tests and linters / Tests (pull_request) Successful in 3m47s
Tests and linters / Tests with -race (pull_request) Successful in 3m58s
Since `frostfs-cli control shards rebuild` command was added,
there is no need for background rebuild now.
For failover tests used used value 1 to rebuild only schema change.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-06 15:19:55 +03:00
a61201a987 [#1337] config: Move rebuild_worker_count to shard section
This makes it simple to limit performance degradation for every shard
because of rebuild.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-06 13:57:27 +03:00
6b6eabe41c [#1337] cli: Add control shards rebuild command
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-06 13:57:27 +03:00
108e4e07be [#1349] node: Evacuate objects without setting mode to MAINTENANCE
All checks were successful
DCO action / DCO (pull_request) Successful in 1m31s
Vulncheck / Vulncheck (pull_request) Successful in 1m32s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m12s
Tests and linters / Run gofumpt (pull_request) Successful in 2m21s
Build / Build Components (pull_request) Successful in 2m32s
Tests and linters / gopls check (pull_request) Successful in 2m36s
Tests and linters / Staticcheck (pull_request) Successful in 2m55s
Tests and linters / Tests with -race (pull_request) Successful in 3m26s
Tests and linters / Lint (pull_request) Successful in 3m31s
Tests and linters / Tests (pull_request) Successful in 3m40s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-09-05 16:08:27 +03:00
b3deb893ba [#1310] object: Move target initialization to separate package
* Split the logic of write target initialization to different packages;
* Refactor patch and put services: since both service initialize the target
  themselves.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-05 13:03:58 +00:00
7768a482b5 [#1223] lens/tui: Add TUI app for blobovnicza
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-05 08:03:52 +00:00
371d97f61a [#1223] lens/tui: Add TUI app for write cache
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-05 08:03:52 +00:00
e655336390 [#1223] lens/tui: Add app help
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-05 08:03:52 +00:00
ed396448ac [#1223] lens/tui: Add TUI app to explore metabase
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-05 08:03:52 +00:00
9cbd32bce8 [#1223] lens/tui: Add writecache schema
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-05 08:03:52 +00:00
1ae86f35a8 [#1223] lens/tui: Add metabase schema
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-05 08:03:52 +00:00
a4fb7f085b
[#1348] go.mod: Update api-go and sdk-go
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 2m36s
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m3s
Vulncheck / Vulncheck (pull_request) Successful in 2m50s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m3s
DCO action / DCO (pull_request) Successful in 2m47s
Tests and linters / Lint (pull_request) Successful in 3m39s
Tests and linters / Staticcheck (pull_request) Successful in 3m37s
Tests and linters / Tests (1.23) (pull_request) Successful in 3m43s
Build / Build Components (1.22) (pull_request) Successful in 3m32s
Build / Build Components (1.23) (pull_request) Successful in 3m32s
Tests and linters / gopls check (pull_request) Successful in 4m10s
Tests and linters / Tests with -race (pull_request) Successful in 4m16s
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-04 10:47:26 +03:00
a685fcdc96 [#1317] go.mod: Use range over int
All checks were successful
DCO action / DCO (pull_request) Successful in 2m41s
Tests and linters / Run gofumpt (pull_request) Successful in 2m32s
Vulncheck / Vulncheck (pull_request) Successful in 2m38s
Build / Build Components (1.23) (pull_request) Successful in 3m0s
Build / Build Components (1.22) (pull_request) Successful in 3m3s
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m33s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m34s
Tests and linters / Tests (1.23) (pull_request) Successful in 3m36s
Tests and linters / Staticcheck (pull_request) Successful in 3m35s
Tests and linters / Lint (pull_request) Successful in 4m18s
Tests and linters / Tests with -race (pull_request) Successful in 4m20s
Tests and linters / gopls check (pull_request) Successful in 4m25s
Since Go 1.22 a "for" statement with a "range" clause is able
to iterate through integer values from zero to an upper limit.

gopatch script:
@@
var i, e expression
@@
-for i := 0; i <= e - 1; i++ {
+for i := range e {
    ...
}

@@
var i, e expression
@@
-for i := 0; i <= e; i++ {
+for i := range e + 1 {
    ...
}

@@
var i, e expression
@@
-for i := 0; i < e; i++ {
+for i := range e {
    ...
}

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-09-03 13:00:54 +03:00
98fe24cdb7 [#1343] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-30 08:45:24 +00:00
6c2146bbc1 [#1334] metabase: Add upgrade from v2 to v3
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-30 09:06:20 +03:00
03976c6ed5 [#1341] .golangci.yml: Replace exportloopref with copyloopvar
exportloopref is deprecated.
gopatch:
```
@@
var index, value identifier
var slice expression
@@
for index, value := range slice {
...
-value := value
...
}

@@
var index, value identifier
var slice expression
@@
for index, value := range slice {
...
-index := index
...
}

@@
var value identifier
var channel expression
@@
for value := range channel {
...
-value := value
...
}
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-28 15:44:41 +00:00
a059a7dcf0 [#1329] cli: Skip linking objects in complex object processing
All checks were successful
DCO action / DCO (pull_request) Successful in 2m41s
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m42s
Tests and linters / Run gofumpt (pull_request) Successful in 3m50s
Vulncheck / Vulncheck (pull_request) Successful in 3m55s
Tests and linters / Tests (1.23) (pull_request) Successful in 4m8s
Tests and linters / Tests (1.22) (pull_request) Successful in 4m11s
Build / Build Components (1.22) (pull_request) Successful in 4m32s
Tests and linters / Tests with -race (pull_request) Successful in 4m20s
Build / Build Components (1.23) (pull_request) Successful in 4m33s
Tests and linters / Staticcheck (pull_request) Successful in 5m9s
Tests and linters / Lint (pull_request) Successful in 5m34s
Tests and linters / gopls check (pull_request) Successful in 5m24s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-23 13:51:20 +03:00
dfe825b81b [#1309] test: Clean up config after tests
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-08-22 08:49:10 +00:00
7bca428db0 [#1322] Use new protobuf marshaler
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-22 07:17:41 +00:00
a345c972bf [#1316] lint: Fix warnings
All checks were successful
DCO action / DCO (pull_request) Successful in 1m43s
Build / Build Components (1.22) (pull_request) Successful in 2m24s
Vulncheck / Vulncheck (pull_request) Successful in 2m12s
Build / Build Components (1.23) (pull_request) Successful in 2m36s
Tests and linters / Run gofumpt (pull_request) Successful in 2m21s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m48s
Tests and linters / Tests (1.23) (pull_request) Successful in 2m56s
Tests and linters / Staticcheck (pull_request) Successful in 3m8s
Tests and linters / Lint (pull_request) Successful in 3m26s
Tests and linters / Tests with -race (pull_request) Successful in 3m38s
Tests and linters / gopls check (pull_request) Successful in 3m54s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m28s
Renamed parameters `min/max` to avoid conflicts with
predeclared identifiers.

Replaced background context with parent context without
cancellation in closer functions in frostfs-node.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-08-21 18:18:25 +03:00
41104f2383 [#1307] cli: Make cli process object.patch
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-16 14:13:09 +00:00
5ed317e24c [#1307] cli: Introduce object patch command
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-16 14:13:09 +00:00
e890f1b4b1 [#1307] object: Implement Patch method
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-16 14:13:09 +00:00
a4a1c3f18b [#1307] go.mod: Bump frostfs-sdk-go/frostfs-api-go/v2 versions
* Also, resolve dependencies and conflicts for object service
  by creating stub for `Patch` method.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-16 14:13:09 +00:00
d055168e2a [#1135] ir: Add healthstatus RECONFIGURING
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-08-15 16:54:45 +03:00
93d63e1632 [#1284] writecache: Allow to seal writecache async
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-09 13:23:35 +03:00
68029d756e [#1302] writecache: Allow to specify custom page size
All checks were successful
DCO action / DCO (pull_request) Successful in 1m57s
Vulncheck / Vulncheck (pull_request) Successful in 2m6s
Tests and linters / Run gofumpt (pull_request) Successful in 2m9s
Build / Build Components (1.22) (pull_request) Successful in 2m37s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m49s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m1s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m4s
Tests and linters / Staticcheck (pull_request) Successful in 3m0s
Tests and linters / Tests with -race (pull_request) Successful in 3m9s
Tests and linters / Lint (pull_request) Successful in 3m30s
Tests and linters / gopls check (pull_request) Successful in 3m40s
Build / Build Components (1.21) (pull_request) Successful in 1m12s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-09 12:12:07 +03:00
08b1f18bca [#1296] writecache: Add count limit
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-09 06:30:32 +00:00
36efccd862 [#1298] writecache: Add shrink flag for Seal command
All checks were successful
DCO action / DCO (pull_request) Successful in 1m23s
Vulncheck / Vulncheck (pull_request) Successful in 1m48s
Tests and linters / Run gofumpt (pull_request) Successful in 1m54s
Build / Build Components (1.22) (pull_request) Successful in 2m23s
Build / Build Components (1.21) (pull_request) Successful in 2m28s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m53s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m53s
Tests and linters / Staticcheck (pull_request) Successful in 2m48s
Tests and linters / Tests (1.22) (pull_request) Successful in 2m54s
Tests and linters / Tests with -race (pull_request) Successful in 2m58s
Tests and linters / Lint (pull_request) Successful in 3m24s
Tests and linters / gopls check (pull_request) Successful in 3m36s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-08 16:32:29 +03:00
5c01bd5be8 [#1298] writecache: Add restore-mode flag for Seal command
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-07 11:20:14 +03:00
c49982d22a [#1282] cli: Allow to external addresses first for object nodes
Some checks failed
DCO action / DCO (pull_request) Successful in 1m18s
Vulncheck / Vulncheck (pull_request) Successful in 2m15s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m39s
Build / Build Components (1.22) (pull_request) Successful in 3m20s
Tests and linters / Staticcheck (pull_request) Successful in 3m34s
Tests and linters / Lint (pull_request) Successful in 3m42s
Build / Build Components (1.21) (pull_request) Successful in 3m32s
Tests and linters / Tests with -race (pull_request) Successful in 3m54s
Tests and linters / gopls check (pull_request) Successful in 3m57s
Tests and linters / Tests (1.22) (pull_request) Successful in 1m26s
Tests and linters / Tests (1.21) (pull_request) Failing after 50s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-01 09:16:19 +03:00
85a77b7c21 [#1279] adm: Interpret "root" name as empty for namespace target type
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-29 12:57:40 +00:00
8377372a40 [#1276] go.mod: Update api-go
All checks were successful
Tests and linters / Staticcheck (pull_request) Successful in 2m54s
Build / Build Components (1.22) (pull_request) Successful in 3m28s
Build / Build Components (1.21) (pull_request) Successful in 3m16s
Tests and linters / Lint (pull_request) Successful in 4m14s
Tests and linters / Tests with -race (pull_request) Successful in 8m2s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m41s
Tests and linters / Tests (1.22) (pull_request) Successful in 8m54s
Tests and linters / gopls check (pull_request) Successful in 2m53s
DCO action / DCO (pull_request) Successful in 46s
Vulncheck / Vulncheck (pull_request) Successful in 1m4s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m47s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-26 16:44:19 +03:00
1032075a21
[#1259] cli: Remove locode subcommand
Removed `frostfs-cli util locode` subcommand.
Alternative command could be found in
`git.frostfs.info/TrueCloudLab/frostfs-locode-db`.

Signed-off-by: George Bartolomey <george@bh4.ru>
2024-07-23 14:59:14 +03:00
3940bc17c1 [#1251] pilorama: Allow traversing multiple branches in parallel
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-17 11:25:07 +03:00
08953a2f94 [#1239] adm/morph: Fix transfer GAS to alphabet nodes
All checks were successful
DCO action / DCO (pull_request) Successful in 7m55s
Vulncheck / Vulncheck (pull_request) Successful in 8m4s
Build / Build Components (1.21) (pull_request) Successful in 10m21s
Build / Build Components (1.22) (pull_request) Successful in 10m21s
Tests and linters / Staticcheck (pull_request) Successful in 11m45s
Tests and linters / Lint (pull_request) Successful in 12m23s
Tests and linters / gopls check (pull_request) Successful in 14m17s
Pre-commit hooks / Pre-commit (pull_request) Successful in 16m26s
Tests and linters / Tests (1.21) (pull_request) Successful in 17m5s
Tests and linters / Tests with -race (pull_request) Successful in 17m17s
Tests and linters / Tests (1.22) (pull_request) Successful in 17m26s
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-07-11 18:28:32 +03:00
36956db123 [#1159] adm/frostfsid: Remove wallet requirement for list-*
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-07-11 14:04:37 +00:00
784e8ef857 [#1209] cli: Add --quiet flag to healthcheck command
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-07-09 12:40:32 +00:00
ca974b8b4c [#1233] cli: Drop debugee from object nodes
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-09 07:54:29 +00:00
6ef38c07bd [#1235] cli: Fix parse action
All checks were successful
DCO action / DCO (pull_request) Successful in 2m52s
Vulncheck / Vulncheck (pull_request) Successful in 3m25s
Build / Build Components (1.22) (pull_request) Successful in 4m48s
Build / Build Components (1.21) (pull_request) Successful in 4m57s
Pre-commit hooks / Pre-commit (pull_request) Successful in 5m41s
Tests and linters / gopls check (pull_request) Successful in 6m21s
Tests and linters / Staticcheck (pull_request) Successful in 7m22s
Tests and linters / Lint (pull_request) Successful in 7m56s
Tests and linters / Tests (1.21) (pull_request) Successful in 10m53s
Tests and linters / Tests with -race (pull_request) Successful in 10m54s
Tests and linters / Tests (1.22) (pull_request) Successful in 11m9s
* Parsed `object.*` lexeme should also include
  `MethodRangeObject` constant.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-08 18:35:44 +03:00
0c2b6f3dac [#1216] ape: Make services use bearer chains fed router
* Refactor object and tree service - they should instantiate
  chain router cheking the bearer token. If there are no bearer
  token rules, then defaul chain router is used.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-05 18:26:48 +00:00
78b1d9b18d [#1226] blobovniczatree: Drop leaf width limitation
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-04 12:22:06 +03:00
40c9ddb6ba [#1226] blobovniczatree: Drop init in advance option
To make blobovniczatree unlimited.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-04 12:22:06 +03:00
91bed3b0ba [#1219] Remove Container.SetEACL method
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-02 13:05:40 +00:00