Commit graph

1255 commits

Author SHA1 Message Date
e5c8f7ff9f [#1382] controlSvc: Replace deprecated methods
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-18 14:29:24 +03:00
1e7f9909da [#1382] policer: Replace deprecated methods
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-18 14:29:21 +03:00
89d0435b1d [#1374] tree: Use NewClient to create grpc connection in cache
All checks were successful
DCO action / DCO (pull_request) Successful in 1m2s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m53s
Vulncheck / Vulncheck (pull_request) Successful in 2m4s
Build / Build Components (pull_request) Successful in 2m39s
Tests and linters / Run gofumpt (pull_request) Successful in 2m30s
Tests and linters / Staticcheck (pull_request) Successful in 2m38s
Tests and linters / gopls check (pull_request) Successful in 3m2s
Tests and linters / Lint (pull_request) Successful in 3m14s
Tests and linters / Tests (pull_request) Successful in 6m4s
Tests and linters / Tests with -race (pull_request) Successful in 6m7s
Created grpc connection should be established, so perform Healthcheck request
to check connection is ok.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-13 15:59:26 +03:00
54fe8383a4 [#1374] tree: Use NewClient to create grpc connection for sync
Created connection will be used to sync trees, so it is ok to defer
dial to the first RPC call.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-13 15:59:26 +03:00
99be4c83a7
[#1368] *: Run gofumpt
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-12 10:00:28 +03:00
a812932984 [#1362] ape: Move common APE check logic to separate package
* Tree and object service have the same log for checking APE. So,
  this check should be moved to common package.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-10 12:40:34 +00:00
6b6eabe41c [#1337] cli: Add control shards rebuild command
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-09-06 13:57:27 +03:00
f652518c24 [#1357] go: Fix panic caused by using range over slice len
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 1m12s
DCO action / DCO (pull_request) Successful in 1m28s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m53s
Tests and linters / Run gofumpt (pull_request) Successful in 1m55s
Build / Build Components (pull_request) Successful in 2m14s
Tests and linters / gopls check (pull_request) Successful in 2m22s
Tests and linters / Staticcheck (pull_request) Successful in 2m30s
Tests and linters / Lint (pull_request) Successful in 3m15s
Tests and linters / Tests (pull_request) Successful in 3m25s
Tests and linters / Tests with -race (pull_request) Successful in 3m31s
If slice is altered in `for` loop, we cannot use range over its
length: it may cause panic if slice gets shorter.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-09-06 13:09:58 +03:00
273980cfb9 [#1310] object: Remove irrelevant comments
Some checks failed
DCO action / DCO (pull_request) Successful in 51s
Vulncheck / Vulncheck (pull_request) Successful in 1m49s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m9s
Build / Build Components (pull_request) Successful in 2m23s
Tests and linters / Run gofumpt (pull_request) Successful in 2m15s
Tests and linters / gopls check (pull_request) Successful in 2m37s
Tests and linters / Staticcheck (pull_request) Successful in 3m6s
Tests and linters / Lint (pull_request) Successful in 3m45s
Tests and linters / Tests with -race (pull_request) Failing after 4m37s
Tests and linters / Tests (pull_request) Failing after 2m3s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-05 16:40:32 +03:00
108e4e07be [#1349] node: Evacuate objects without setting mode to MAINTENANCE
All checks were successful
DCO action / DCO (pull_request) Successful in 1m31s
Vulncheck / Vulncheck (pull_request) Successful in 1m32s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m12s
Tests and linters / Run gofumpt (pull_request) Successful in 2m21s
Build / Build Components (pull_request) Successful in 2m32s
Tests and linters / gopls check (pull_request) Successful in 2m36s
Tests and linters / Staticcheck (pull_request) Successful in 2m55s
Tests and linters / Tests with -race (pull_request) Successful in 3m26s
Tests and linters / Lint (pull_request) Successful in 3m31s
Tests and linters / Tests (pull_request) Successful in 3m40s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-09-05 16:08:27 +03:00
b3deb893ba [#1310] object: Move target initialization to separate package
* Split the logic of write target initialization to different packages;
* Refactor patch and put services: since both service initialize the target
  themselves.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-09-05 13:03:58 +00:00
a4fb7f085b
[#1348] go.mod: Update api-go and sdk-go
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 2m36s
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m3s
Vulncheck / Vulncheck (pull_request) Successful in 2m50s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m3s
DCO action / DCO (pull_request) Successful in 2m47s
Tests and linters / Lint (pull_request) Successful in 3m39s
Tests and linters / Staticcheck (pull_request) Successful in 3m37s
Tests and linters / Tests (1.23) (pull_request) Successful in 3m43s
Build / Build Components (1.22) (pull_request) Successful in 3m32s
Build / Build Components (1.23) (pull_request) Successful in 3m32s
Tests and linters / gopls check (pull_request) Successful in 4m10s
Tests and linters / Tests with -race (pull_request) Successful in 4m16s
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2024-09-04 10:47:26 +03:00
a685fcdc96 [#1317] go.mod: Use range over int
All checks were successful
DCO action / DCO (pull_request) Successful in 2m41s
Tests and linters / Run gofumpt (pull_request) Successful in 2m32s
Vulncheck / Vulncheck (pull_request) Successful in 2m38s
Build / Build Components (1.23) (pull_request) Successful in 3m0s
Build / Build Components (1.22) (pull_request) Successful in 3m3s
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m33s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m34s
Tests and linters / Tests (1.23) (pull_request) Successful in 3m36s
Tests and linters / Staticcheck (pull_request) Successful in 3m35s
Tests and linters / Lint (pull_request) Successful in 4m18s
Tests and linters / Tests with -race (pull_request) Successful in 4m20s
Tests and linters / gopls check (pull_request) Successful in 4m25s
Since Go 1.22 a "for" statement with a "range" clause is able
to iterate through integer values from zero to an upper limit.

gopatch script:
@@
var i, e expression
@@
-for i := 0; i <= e - 1; i++ {
+for i := range e {
    ...
}

@@
var i, e expression
@@
-for i := 0; i <= e; i++ {
+for i := range e + 1 {
    ...
}

@@
var i, e expression
@@
-for i := 0; i < e; i++ {
+for i := range e {
    ...
}

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-09-03 13:00:54 +03:00
98fe24cdb7 [#1343] go.mod: Update api-go
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-30 08:45:24 +00:00
03976c6ed5 [#1341] .golangci.yml: Replace exportloopref with copyloopvar
exportloopref is deprecated.
gopatch:
```
@@
var index, value identifier
var slice expression
@@
for index, value := range slice {
...
-value := value
...
}

@@
var index, value identifier
var slice expression
@@
for index, value := range slice {
...
-index := index
...
}

@@
var value identifier
var channel expression
@@
for value := range channel {
...
-value := value
...
}
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-28 15:44:41 +00:00
7abbdca064 [#1340] getSvc: Fix access denied error handling
All checks were successful
DCO action / DCO (pull_request) Successful in 4m23s
Tests and linters / Run gofumpt (pull_request) Successful in 4m14s
Vulncheck / Vulncheck (pull_request) Successful in 4m33s
Build / Build Components (1.22) (pull_request) Successful in 4m56s
Build / Build Components (1.23) (pull_request) Successful in 4m59s
Tests and linters / Tests (1.22) (pull_request) Successful in 5m2s
Pre-commit hooks / Pre-commit (pull_request) Successful in 5m6s
Tests and linters / Tests (1.23) (pull_request) Successful in 5m8s
Tests and linters / Lint (pull_request) Successful in 5m51s
Tests and linters / gopls check (pull_request) Successful in 6m13s
Tests and linters / Tests with -race (pull_request) Successful in 6m19s
Tests and linters / Staticcheck (pull_request) Successful in 1m48s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-28 14:02:28 +03:00
6488ddee88 [#1338] object: Fix range provider in Patch handler
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 1m50s
DCO action / DCO (pull_request) Successful in 1m54s
Vulncheck / Vulncheck (pull_request) Successful in 2m29s
Tests and linters / Tests (1.23) (pull_request) Successful in 2m58s
Tests and linters / Tests (1.22) (pull_request) Successful in 2m59s
Tests and linters / Lint (pull_request) Successful in 3m17s
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m3s
Build / Build Components (1.23) (pull_request) Successful in 3m12s
Build / Build Components (1.22) (pull_request) Successful in 3m15s
Tests and linters / Staticcheck (pull_request) Successful in 3m35s
Tests and linters / gopls check (pull_request) Successful in 3m52s
Tests and linters / Tests with -race (pull_request) Successful in 4m11s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-27 16:51:12 +03:00
d6b42972a8 [#1338] object: Fix audit patch stream
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-27 16:18:29 +03:00
5e9a97fd3e [#1336] go.mod: Update api-go and sdk-go
All checks were successful
DCO action / DCO (pull_request) Successful in 5m13s
Build / Build Components (1.22) (pull_request) Successful in 6m30s
Build / Build Components (1.23) (pull_request) Successful in 6m31s
Pre-commit hooks / Pre-commit (pull_request) Successful in 6m59s
Tests and linters / Run gofumpt (pull_request) Successful in 57s
Tests and linters / Tests (1.22) (pull_request) Successful in 2m21s
Tests and linters / Tests (1.23) (pull_request) Successful in 2m19s
Tests and linters / Staticcheck (pull_request) Successful in 2m26s
Tests and linters / Lint (pull_request) Successful in 2m49s
Tests and linters / gopls check (pull_request) Successful in 3m6s
Tests and linters / Tests with -race (pull_request) Successful in 3m16s
Vulncheck / Vulncheck (pull_request) Successful in 45s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-27 14:48:43 +03:00
bd24beecf8 [#1329] putSvc: Reset SuccessAfter for non-EC objects in EC container broadcasting
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-23 13:51:16 +03:00
7bca428db0 [#1322] Use new protobuf marshaler
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-08-22 07:17:41 +00:00
8319b59238 [#1318] Fix gofumpt issue
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-08-20 14:34:54 +03:00
b7acb34fa4 [#1319] treeSvc: Do not wrap error from APE
All checks were successful
Tests and linters / Run gofumpt (pull_request) Successful in 2m43s
DCO action / DCO (pull_request) Successful in 2m56s
Tests and linters / Tests (1.21) (pull_request) Successful in 4m24s
Vulncheck / Vulncheck (pull_request) Successful in 4m12s
Tests and linters / Tests (1.22) (pull_request) Successful in 4m23s
Tests and linters / Staticcheck (pull_request) Successful in 4m18s
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m32s
Tests and linters / Tests with -race (pull_request) Successful in 4m22s
Build / Build Components (1.21) (pull_request) Successful in 4m52s
Build / Build Components (1.22) (pull_request) Successful in 4m50s
Tests and linters / Lint (pull_request) Successful in 4m55s
Tests and linters / gopls check (pull_request) Successful in 4m59s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-19 18:38:27 +03:00
eeab417dcf [#1307] object: Add APE check for Patch handler
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-16 14:13:09 +00:00
e890f1b4b1 [#1307] object: Implement Patch method
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-16 14:13:09 +00:00
a4a1c3f18b [#1307] go.mod: Bump frostfs-sdk-go/frostfs-api-go/v2 versions
* Also, resolve dependencies and conflicts for object service
  by creating stub for `Patch` method.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-08-16 14:13:09 +00:00
d055168e2a [#1135] ir: Add healthstatus RECONFIGURING
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-08-15 16:54:45 +03:00
93d63e1632 [#1284] writecache: Allow to seal writecache async
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-09 13:23:35 +03:00
c985b1198f [#1302] putSvc: Override SuccessAfter for non-regular objects in EC containers
All checks were successful
DCO action / DCO (pull_request) Successful in 3m16s
Vulncheck / Vulncheck (pull_request) Successful in 3m21s
Tests and linters / Run gofumpt (pull_request) Successful in 3m35s
Build / Build Components (1.21) (pull_request) Successful in 4m6s
Build / Build Components (1.22) (pull_request) Successful in 4m4s
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m5s
Tests and linters / Tests (1.22) (pull_request) Successful in 4m1s
Tests and linters / Tests (1.21) (pull_request) Successful in 4m6s
Tests and linters / Staticcheck (pull_request) Successful in 4m15s
Tests and linters / Tests with -race (pull_request) Successful in 4m37s
Tests and linters / Lint (pull_request) Successful in 4m48s
Tests and linters / gopls check (pull_request) Successful in 4m55s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-09 10:42:55 +03:00
36efccd862 [#1298] writecache: Add shrink flag for Seal command
All checks were successful
DCO action / DCO (pull_request) Successful in 1m23s
Vulncheck / Vulncheck (pull_request) Successful in 1m48s
Tests and linters / Run gofumpt (pull_request) Successful in 1m54s
Build / Build Components (1.22) (pull_request) Successful in 2m23s
Build / Build Components (1.21) (pull_request) Successful in 2m28s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m53s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m53s
Tests and linters / Staticcheck (pull_request) Successful in 2m48s
Tests and linters / Tests (1.22) (pull_request) Successful in 2m54s
Tests and linters / Tests with -race (pull_request) Successful in 2m58s
Tests and linters / Lint (pull_request) Successful in 3m24s
Tests and linters / gopls check (pull_request) Successful in 3m36s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-08 16:32:29 +03:00
5c01bd5be8 [#1298] writecache: Add restore-mode flag for Seal command
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-07 11:20:14 +03:00
8e51d7849a [#1295] getSvc: Assemble complex EC object headers without linking object
All checks were successful
DCO action / DCO (pull_request) Successful in 1m36s
Build / Build Components (1.21) (pull_request) Successful in 2m23s
Vulncheck / Vulncheck (pull_request) Successful in 2m7s
Tests and linters / Run gofumpt (pull_request) Successful in 2m11s
Build / Build Components (1.22) (pull_request) Successful in 2m40s
Pre-commit hooks / Pre-commit (pull_request) Successful in 2m49s
Tests and linters / Staticcheck (pull_request) Successful in 2m56s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m5s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m5s
Tests and linters / Tests with -race (pull_request) Successful in 3m8s
Tests and linters / Lint (pull_request) Successful in 3m35s
Tests and linters / gopls check (pull_request) Successful in 4m4s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-06 16:48:12 +03:00
8021bacc43 [#1288] putSvc: Respect TTL for EC put
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-08-02 13:01:42 +00:00
7e04083c27 [#1278] containerSvc: Validate FrostFSID subject exitence on Put
All checks were successful
DCO action / DCO (pull_request) Successful in 1m52s
Vulncheck / Vulncheck (pull_request) Successful in 2m18s
Build / Build Components (1.21) (pull_request) Successful in 2m52s
Tests and linters / Lint (pull_request) Successful in 2m56s
Build / Build Components (1.22) (pull_request) Successful in 2m32s
Tests and linters / Staticcheck (pull_request) Successful in 3m2s
Tests and linters / gopls check (pull_request) Successful in 3m25s
Tests and linters / Tests (1.22) (pull_request) Successful in 5m16s
Pre-commit hooks / Pre-commit (pull_request) Successful in 5m3s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m5s
Tests and linters / Tests with -race (pull_request) Successful in 8m53s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-29 16:01:37 +03:00
8377372a40 [#1276] go.mod: Update api-go
All checks were successful
Tests and linters / Staticcheck (pull_request) Successful in 2m54s
Build / Build Components (1.22) (pull_request) Successful in 3m28s
Build / Build Components (1.21) (pull_request) Successful in 3m16s
Tests and linters / Lint (pull_request) Successful in 4m14s
Tests and linters / Tests with -race (pull_request) Successful in 8m2s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m41s
Tests and linters / Tests (1.22) (pull_request) Successful in 8m54s
Tests and linters / gopls check (pull_request) Successful in 2m53s
DCO action / DCO (pull_request) Successful in 46s
Vulncheck / Vulncheck (pull_request) Successful in 1m4s
Pre-commit hooks / Pre-commit (pull_request) Successful in 1m47s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-26 16:44:19 +03:00
7fd7961dfa [#1271] getSvc: Fix local EC chunk get
All checks were successful
Build / Build Components (1.21) (pull_request) Successful in 4m22s
Tests and linters / Staticcheck (pull_request) Successful in 4m56s
Tests and linters / gopls check (pull_request) Successful in 5m59s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m20s
DCO action / DCO (pull_request) Successful in 35s
Build / Build Components (1.22) (pull_request) Successful in 1m15s
Vulncheck / Vulncheck (pull_request) Successful in 59s
Tests and linters / Lint (pull_request) Successful in 2m24s
Pre-commit hooks / Pre-commit (pull_request) Successful in 3m1s
Tests and linters / Tests (1.22) (pull_request) Successful in 6m59s
Tests and linters / Tests with -race (pull_request) Successful in 7m32s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-24 08:52:16 +03:00
8398a8b4b3 [#1271] getSvc: Fix head --raw assemble for EC
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-24 08:52:08 +03:00
eadcea8df0 [#1249] object: Remove all APE pre-checks in handlers
* Methods `Head`, `Get`, `GetRangeHash` should no longer use APE pre-checks
  as that leads only to incorrect rule chain processing for requests:
  1. Immediate return with `NoRuleFound` may be unexpected as some `Allow`
     rule is actually defined but can't be matched yet as it gets no object
     attributes;
  2. Immdediate return with `Allow` may be incorrect as some `Deny` rule
     is actually defined but can't bet matched yet as it gets no object
     attirbutes;
  3. Pre-check breaks compatibility for converted EACL-tables.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-18 13:52:43 +00:00
5e5ee545b6 [#1254] policer: Fix svacer warning
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-18 11:30:39 +00:00
e83d39e33f [#1253] deleteSvc: Use copy of common parameters
All checks were successful
DCO action / DCO (pull_request) Successful in 3m29s
Vulncheck / Vulncheck (pull_request) Successful in 3m31s
Build / Build Components (1.21) (pull_request) Successful in 4m56s
Build / Build Components (1.22) (pull_request) Successful in 4m57s
Tests and linters / Staticcheck (pull_request) Successful in 5m14s
Tests and linters / gopls check (pull_request) Successful in 5m11s
Tests and linters / Lint (pull_request) Successful in 6m23s
Tests and linters / Tests (1.22) (pull_request) Successful in 12m42s
Tests and linters / Tests (1.21) (pull_request) Successful in 12m54s
Tests and linters / Tests with -race (pull_request) Successful in 16m31s
Pre-commit hooks / Pre-commit (pull_request) Successful in 21m57s
getSvc may change the values of some fields, so Head will affect Delete
or Put. In this case, the change is necessary so that the session token
is stored in the tombstone object (EC assemble calls `ForgetTokens`).

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-17 14:24:27 +03:00
fc383ea6ae [#1253] getSvc: Fix EC objects get
Now EC objects assembling is performed concurrently.
Also fixed issue with an error in case of getting
EC object via non-container node.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-17 14:24:27 +03:00
00a88b9936 [#1251] *: Run gofumpt
All checks were successful
DCO action / DCO (pull_request) Successful in 3m50s
Vulncheck / Vulncheck (pull_request) Successful in 3m37s
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m45s
Build / Build Components (1.21) (pull_request) Successful in 5m7s
Build / Build Components (1.22) (pull_request) Successful in 5m1s
Tests and linters / Staticcheck (pull_request) Successful in 5m13s
Tests and linters / gopls check (pull_request) Successful in 5m30s
Tests and linters / Lint (pull_request) Successful in 6m31s
Tests and linters / Tests (1.21) (pull_request) Successful in 11m39s
Tests and linters / Tests with -race (pull_request) Successful in 11m37s
Tests and linters / Tests (1.22) (pull_request) Successful in 13m11s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-17 11:25:07 +03:00
3940bc17c1 [#1251] pilorama: Allow traversing multiple branches in parallel
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-17 11:25:07 +03:00
e5767c9002 [#1250] *: Reformat proto filets with clang-format
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-16 15:13:08 +00:00
286df198c9 [#1248] placement: Decouple ContainerNodes() cache from the placement builder.
Also, write tests.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-16 12:21:30 +00:00
3a48b282b6 [#1248] placement: Use epoch to track netmap versions
Previously we used pointer, this could have worked,
because most of the time, the netmap is cached.
This didn't work, however, because `lastNm` field was always nil.

Rework the mechanism completely:
1. Use epoch to track netmap versions, as it it simpler and
   is unrelated to the TTL of an underlying cache.
2. Fix a bug where the epoch could change while mutex was unlocked.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-16 12:21:30 +00:00
21431f22c0 [#1248] placement: Use cid.ID as key in the cache
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-16 12:21:30 +00:00
d5dc14c639 [#1243] object: Make APE checker set x-headers to request properties
* Update go.mod, go.sum;
* Add x-headers to request properties;
* Add a unit-test.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-16 07:28:42 +00:00
84ecd61dfd [#1233] putSvc: Try to put EC chunk to any node
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-09 07:54:29 +00:00
d90aab5454 [#1229] util: Fix session token expiration check
* Make session token expired at `current_epoch + 1` but
  not at `current_epoch` when it's still valid.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-08 08:15:56 +00:00
0c2b6f3dac [#1216] ape: Make services use bearer chains fed router
* Refactor object and tree service - they should instantiate
  chain router cheking the bearer token. If there are no bearer
  token rules, then defaul chain router is used.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-05 18:26:48 +00:00
8eb591d668 [#1231] policer: Add EC node-off unit test
Some checks failed
Build / Build Components (1.21) (pull_request) Successful in 10m29s
Build / Build Components (1.22) (pull_request) Successful in 10m24s
DCO action / DCO (pull_request) Successful in 10m29s
Vulncheck / Vulncheck (pull_request) Successful in 12m18s
Tests and linters / gopls check (pull_request) Successful in 21m40s
Tests and linters / Tests (1.21) (pull_request) Failing after 22m2s
Pre-commit hooks / Pre-commit (pull_request) Successful in 23m40s
Tests and linters / Staticcheck (pull_request) Successful in 24m10s
Tests and linters / Tests (1.22) (pull_request) Successful in 24m19s
Tests and linters / Lint (pull_request) Successful in 25m30s
Tests and linters / Tests with -race (pull_request) Successful in 25m15s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-05 12:11:36 +03:00
3a797e4682 [#1222] engine: Fix tree evacuation
All checks were successful
DCO action / DCO (pull_request) Successful in 13m4s
Vulncheck / Vulncheck (pull_request) Successful in 15m32s
Build / Build Components (1.22) (pull_request) Successful in 16m30s
Build / Build Components (1.21) (pull_request) Successful in 16m37s
Tests and linters / gopls check (pull_request) Successful in 20m33s
Tests and linters / Staticcheck (pull_request) Successful in 21m17s
Tests and linters / Lint (pull_request) Successful in 21m37s
Tests and linters / Tests with -race (pull_request) Successful in 26m40s
Tests and linters / Tests (1.21) (pull_request) Successful in 26m54s
Tests and linters / Tests (1.22) (pull_request) Successful in 27m59s
Pre-commit hooks / Pre-commit (pull_request) Successful in 39m39s
Do not fail if it is unable to evacuate tree to other node.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-04 10:38:10 +03:00
2bac82cd6f [#1222] engine: Fix object evacuation
Do not fail evacuation if it unable to evacuate object to other node.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-04 10:38:10 +03:00
4bd4667791 [#1218] tree: Fix bearer token validation
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-03 07:22:11 +00:00
f3a861806e [#1218] object: Fix bearer token validation
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-03 07:22:11 +00:00
a378ff9cf6 [#1218] object: Pass container owner for backward get method check
* `getStreamBasicChecker` must define `containerOwner` for backward checks,
  otherwise bearer token cannot be validated for the token issuer.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-07-03 07:22:11 +00:00
91bed3b0ba [#1219] Remove Container.SetEACL method
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-07-02 13:05:40 +00:00
dc2867682f [#1213] deleteSvc: Do not allow to delete EC chunks
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-07-01 06:49:35 +00:00
7085723c6b [#1074] pilorama: Allow empty filenames in SortedByFilename()
All checks were successful
DCO action / DCO (pull_request) Successful in 5m38s
Vulncheck / Vulncheck (pull_request) Successful in 5m57s
Tests and linters / gopls check (pull_request) Successful in 6m57s
Build / Build Components (1.22) (pull_request) Successful in 6m48s
Build / Build Components (1.21) (pull_request) Successful in 6m52s
Tests and linters / Lint (pull_request) Successful in 8m26s
Tests and linters / Staticcheck (pull_request) Successful in 8m48s
Tests and linters / Tests (1.22) (pull_request) Successful in 11m39s
Tests and linters / Tests (1.21) (pull_request) Successful in 11m47s
Tests and linters / Tests with -race (pull_request) Successful in 13m49s
Pre-commit hooks / Pre-commit (pull_request) Successful in 17m24s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-28 17:46:24 +03:00
4f7d76c9ef [#1206] audit: Drop not required events
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-27 10:54:31 +00:00
11a38a0a84 [#1190] tree: GroupIDs must also be target of APE checks
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-06-25 08:49:20 +00:00
0b87388c18 [#1190] object: GroupIDs must also be target of APE checks
* Also add new test case for ape middleware in container service.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-06-25 08:49:20 +00:00
621dbf58ab [#1190] container: GroupIDs must also be target of APE checks
* Also add new test case for ape middleware in container service.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-06-25 08:49:20 +00:00
a83eeddb1d [#60] control: Add GetNetmapStatus method
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-20 16:28:42 +03:00
fd28461def [#1184] ir: Add grpc middleware for control service
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 3m14s
DCO action / DCO (pull_request) Successful in 3m46s
Build / Build Components (1.21) (pull_request) Successful in 4m20s
Build / Build Components (1.22) (pull_request) Successful in 4m21s
Tests and linters / gopls check (pull_request) Successful in 4m45s
Tests and linters / Lint (pull_request) Successful in 5m16s
Tests and linters / Staticcheck (pull_request) Successful in 6m24s
Pre-commit hooks / Pre-commit (pull_request) Successful in 8m54s
Tests and linters / Tests (1.21) (pull_request) Successful in 10m57s
Tests and linters / Tests with -race (pull_request) Successful in 10m52s
Tests and linters / Tests (1.22) (pull_request) Successful in 11m20s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-19 16:05:53 +03:00
ecd1ed7a5e [#1184] node: Add audit middleware for grpc services
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-06-19 16:05:53 +03:00
5b100699d7 [#566] policer: Move isClientErrMaintenance to frostfs-sdk-go
All checks were successful
DCO action / DCO (pull_request) Successful in 1m5s
Vulncheck / Vulncheck (pull_request) Successful in 4m8s
Pre-commit hooks / Pre-commit (pull_request) Successful in 5m5s
Build / Build Components (1.21) (pull_request) Successful in 4m47s
Build / Build Components (1.22) (pull_request) Successful in 4m44s
Tests and linters / Lint (pull_request) Successful in 6m13s
Tests and linters / Staticcheck (pull_request) Successful in 6m3s
Tests and linters / gopls check (pull_request) Successful in 6m16s
Tests and linters / Tests (1.21) (pull_request) Successful in 9m19s
Tests and linters / Tests with -race (pull_request) Successful in 9m15s
Tests and linters / Tests (1.22) (pull_request) Successful in 9m22s
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-06-18 10:20:45 +03:00
239323eeef [#1157] tree: Make tree service use Bearer token's APE overrides
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-06-07 12:11:11 +00:00
04a3f891fd [#1157] object: Make APE checker use Bearer-token's APE overrides
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-06-07 12:11:11 +00:00
a90310335d [#1156] ape: Return not found when removing local overrides
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-06-07 12:10:57 +00:00
a849236b68 [#1161] node: Remove notification functionality
It is unused and will be reworked in future.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-06-07 12:10:51 +00:00
5aacb8fc86 [#1144] metabase: Save parent attributes for ec-chunks
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 4m15s
DCO action / DCO (pull_request) Successful in 4m4s
Build / Build Components (1.22) (pull_request) Successful in 5m17s
Build / Build Components (1.21) (pull_request) Successful in 5m22s
Tests and linters / Lint (pull_request) Successful in 7m6s
Tests and linters / Staticcheck (pull_request) Successful in 7m25s
Tests and linters / gopls check (pull_request) Successful in 7m19s
Pre-commit hooks / Pre-commit (pull_request) Successful in 9m39s
Tests and linters / Tests (1.22) (pull_request) Successful in 10m36s
Tests and linters / Tests (1.21) (pull_request) Successful in 10m50s
Tests and linters / Tests with -race (pull_request) Successful in 10m51s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 19:55:32 +03:00
0b367007fc [#1152] go.mod: Update api-go and sdk versions
All checks were successful
DCO action / DCO (pull_request) Successful in 5m23s
Build / Build Components (1.22) (pull_request) Successful in 7m33s
Build / Build Components (1.21) (pull_request) Successful in 7m43s
Tests and linters / Lint (pull_request) Successful in 8m51s
Tests and linters / Tests (1.22) (pull_request) Successful in 14m33s
Pre-commit hooks / Pre-commit (pull_request) Successful in 15m48s
Vulncheck / Vulncheck (pull_request) Successful in 15m12s
Tests and linters / Tests (1.21) (pull_request) Successful in 16m38s
Tests and linters / gopls check (pull_request) Successful in 19m24s
Tests and linters / Staticcheck (pull_request) Successful in 21m29s
Tests and linters / Tests with -race (pull_request) Successful in 23m16s
* Resolve conflicts for apemanager since api-go
  contains ape and apemanager packages and SDK only
  ape package.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-31 15:39:09 +03:00
c1af13b47e [#1147] node: Fix issue from gopls
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
6130650bb6 [#1147] node: Implement Lock\Delete requests for EC object
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
d355274cd0 [#1147] object: Use methods on pointer for searchsvc.execCtx
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
3555c73225 [#1147] object: Use methods on pointer for deletesvc.execCtx
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
e43e7bec3a [#1147] log: Remove redundant address field from log
Filled when logger created for `request` object from package `getsvc`.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
50923ed81c [#1147] Fix gofumpt issue
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-30 08:13:04 +00:00
3627b44e92 [#1142] tree: Fill APE-request with source IP property
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-27 10:17:17 +00:00
482c5129ac [#1142] object: Fill APE-request with source IP property
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-27 10:17:17 +00:00
43625e7536 [#1142] container: Fill APE-request property with source IP
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-27 10:17:17 +00:00
542d3adcb2 [#1105] apemanager: Implement apemanager service
* Introduce grpc server for apemanager service and
  its implementation in `pkg/services/apemanager`.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-27 09:34:21 +00:00
8fd678e269 [#1141] go.mod: Update frostfs-sdk-go and frostfs-api-go/v2
* Also fix unit-test.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-20 13:22:48 +03:00
436c9f5558 [#1129] policer: Restore EC object
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 1m43s
DCO action / DCO (pull_request) Successful in 2m53s
Build / Build Components (1.21) (pull_request) Successful in 4m6s
Build / Build Components (1.22) (pull_request) Successful in 4m31s
Tests and linters / gopls check (pull_request) Successful in 4m57s
Tests and linters / Staticcheck (pull_request) Successful in 6m14s
Tests and linters / Lint (pull_request) Successful in 6m42s
Pre-commit hooks / Pre-commit (pull_request) Successful in 9m19s
Tests and linters / Tests (1.21) (pull_request) Successful in 10m15s
Tests and linters / Tests (1.22) (pull_request) Successful in 10m36s
Tests and linters / Tests with -race (pull_request) Successful in 10m36s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-17 14:36:18 +03:00
44f2e8f27f [#1129] putSvc: Allow to put single unprepared object to EC container
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-16 16:28:49 +03:00
1cd8562db8 [#1129] policer: Refactor shortage
Drop override inside method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-16 16:28:49 +03:00
4ab6c404f7 [#1129] policer: Drop unused
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-16 16:28:49 +03:00
cbe9757490 [#1129] policer: Pull required EC chunks
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-16 16:28:49 +03:00
d45d086acd [#1129] policer: Add EC chunk replication
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-16 16:28:48 +03:00
b078fe5ba1 [#1092] control: Move SignMessage to separate package
All checks were successful
DCO action / DCO (pull_request) Successful in 6m11s
Build / Build Components (1.22) (pull_request) Successful in 7m50s
Build / Build Components (1.21) (pull_request) Successful in 8m3s
Tests and linters / Lint (pull_request) Successful in 9m45s
Tests and linters / gopls check (pull_request) Successful in 12m40s
Vulncheck / Vulncheck (pull_request) Successful in 12m35s
Tests and linters / Staticcheck (pull_request) Successful in 15m34s
Pre-commit hooks / Pre-commit (pull_request) Successful in 19m38s
Tests and linters / Tests with -race (pull_request) Successful in 21m40s
Tests and linters / Tests (1.22) (pull_request) Successful in 3m21s
Tests and linters / Tests (1.21) (pull_request) Successful in 3m36s
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-05-16 12:14:01 +03:00
b3eaa8a9bc [#1083] objsvc/v2: Check response status in RANGE_HASH forwarder
All checks were successful
DCO action / DCO (pull_request) Successful in 1m45s
Vulncheck / Vulncheck (pull_request) Successful in 4m11s
Pre-commit hooks / Pre-commit (pull_request) Successful in 5m44s
Build / Build Components (1.22) (pull_request) Successful in 5m25s
Build / Build Components (1.21) (pull_request) Successful in 5m30s
Tests and linters / Staticcheck (pull_request) Successful in 5m50s
Tests and linters / gopls check (pull_request) Successful in 5m51s
Tests and linters / Lint (pull_request) Successful in 7m44s
Tests and linters / Tests (1.22) (pull_request) Successful in 11m36s
Tests and linters / Tests (1.21) (pull_request) Successful in 12m13s
Tests and linters / Tests with -race (pull_request) Successful in 12m25s
Fixes #1083

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-15 12:04:21 +03:00
0924b62a95 [#1083] objsvc/v2: Unify response verification after forwarding
1. Use the same routine for HEAD/GET_RANGE methods.
2. Make error message similar.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-15 12:04:06 +03:00
300654b045 [#1083] objsvc/v2: Properly check response status after forwarding
Previously we had cryptic error:
```
debug   get/remote.go:38        remote call failed      {"component": "Object.Get service", "request": "HEAD", "address": "9sTxoVrhJ7WBtXQfK2NJ7zDV5yCF7BPLKK1XTxYPdGsP/BbHV4KZZ8y2BPqAT5kyjdHRLkfbtY2xf5uYoMVqxACn1", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "unexpected header type <nil>"}
```
Now we have and expected error:
```
debug   get/remote.go:38        remote call failed      {"component": "Object.Get service", "request": "HEAD", "address": "D2rqaMG4D2VHdv3HKky8UYSYmwQFH2v9oXXqtyRZPTMy/BbHV4KZZ8y2BPqAT5kyjdHRLkfbtY2xf5uYoMVqxACn1", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"}
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-15 12:04:06 +03:00
952d13cd2b [#1124] cli: Improve APE rule parsing
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 1m25s
DCO action / DCO (pull_request) Successful in 1m59s
Build / Build Components (1.21) (pull_request) Successful in 2m27s
Build / Build Components (1.22) (pull_request) Successful in 4m25s
Pre-commit hooks / Pre-commit (pull_request) Successful in 4m57s
Tests and linters / Staticcheck (pull_request) Successful in 5m38s
Tests and linters / gopls check (pull_request) Successful in 5m57s
Tests and linters / Lint (pull_request) Successful in 6m26s
Tests and linters / Tests (1.22) (pull_request) Successful in 9m5s
Tests and linters / Tests (1.21) (pull_request) Successful in 9m11s
Tests and linters / Tests with -race (pull_request) Successful in 9m4s
* Make APE rule parser to read condition's kind in unambiguous using lexemes
`ResourceCondition`, `RequestCondition` instead confusing `Object.Request`, `Object.Resource`.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-14 12:23:26 +03:00
0144117cc9 [#1125] objectSvc: Add EC header APE check
All checks were successful
Build / Build Components (1.21) (pull_request) Successful in 6m27s
DCO action / DCO (pull_request) Successful in 6m38s
Build / Build Components (1.22) (pull_request) Successful in 8m54s
Vulncheck / Vulncheck (pull_request) Successful in 8m37s
Tests and linters / gopls check (pull_request) Successful in 10m32s
Tests and linters / Staticcheck (pull_request) Successful in 11m3s
Tests and linters / Lint (pull_request) Successful in 11m27s
Pre-commit hooks / Pre-commit (pull_request) Successful in 14m16s
Tests and linters / Tests (1.21) (pull_request) Successful in 14m26s
Tests and linters / Tests (1.22) (pull_request) Successful in 15m14s
Tests and linters / Tests with -race (pull_request) Successful in 15m45s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-08 16:25:55 +03:00
ada1b9f737 [#1120] objectSvc: Fix EC put placement
Use parent object ID to compute placement.
Fix too many copies saving.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-05-08 15:23:57 +03:00
fe2c1c926f [#1112] node: Fix race warning for GetObjectAndWritePayload
All checks were successful
DCO action / DCO (pull_request) Successful in 2m10s
Vulncheck / Vulncheck (pull_request) Successful in 2m2s
Build / Build Components (1.22) (pull_request) Successful in 3m44s
Build / Build Components (1.21) (pull_request) Successful in 3m52s
Pre-commit hooks / Pre-commit (pull_request) Successful in 5m40s
Tests and linters / Staticcheck (pull_request) Successful in 6m40s
Tests and linters / Lint (pull_request) Successful in 7m11s
Tests and linters / gopls check (pull_request) Successful in 9m16s
Tests and linters / Tests (1.21) (pull_request) Successful in 10m58s
Tests and linters / Tests (1.22) (pull_request) Successful in 11m2s
Tests and linters / Tests with -race (pull_request) Successful in 11m35s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-07 14:47:21 +03:00
3e782527b8 [#1112] node: Add test for Range request for EC object
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-07 14:47:21 +03:00