Commit graph

4855 commits

Author SHA1 Message Date
b60a51b862 [#1117] ape: Introduce FormFrostfsIDRequestProperties method
* `FormFrostfsIDRequestProperties` gets user claim tags and group id and sets them
  as ape request properties.
* Make tree, container and object service use the method.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-07 10:01:21 +00:00
6c76c9b457 [#1117] core: Introduce SubjectProvider interface for FrostfsID
* Make tree, object and container services use SubjectProvider interface.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-07 10:01:21 +00:00
45f4e6939d [#1117] morph: Make frostfsid client provide GetSubjectExtended method
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-07 10:01:21 +00:00
e07869a8cf [#1100] Remove unused fields
All checks were successful
DCO action / DCO (pull_request) Successful in 2m47s
Build / Build Components (1.21) (pull_request) Successful in 3m21s
Vulncheck / Vulncheck (pull_request) Successful in 3m33s
Build / Build Components (1.22) (pull_request) Successful in 5m25s
Tests and linters / gopls check (pull_request) Successful in 5m13s
Tests and linters / Staticcheck (pull_request) Successful in 6m13s
Tests and linters / Lint (pull_request) Successful in 7m2s
Tests and linters / Tests (1.21) (pull_request) Successful in 10m2s
Tests and linters / Tests with -race (pull_request) Successful in 9m55s
Tests and linters / Tests (1.22) (pull_request) Successful in 10m11s
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-05-06 10:14:36 +03:00
ec2873caa7 [#1116] node: Fix writecache metrics
All checks were successful
DCO action / DCO (pull_request) Successful in 1m35s
Vulncheck / Vulncheck (pull_request) Successful in 3m30s
Build / Build Components (1.22) (pull_request) Successful in 4m40s
Build / Build Components (1.21) (pull_request) Successful in 4m46s
Tests and linters / gopls check (pull_request) Successful in 5m22s
Tests and linters / Staticcheck (pull_request) Successful in 6m19s
Tests and linters / Lint (pull_request) Successful in 7m7s
Tests and linters / Tests (1.21) (pull_request) Successful in 9m36s
Tests and linters / Tests with -race (pull_request) Successful in 9m30s
Tests and linters / Tests (1.22) (pull_request) Successful in 9m46s
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-05-02 20:03:33 +03:00
71789676d5 [#1114] aclsvc: Add tests for request ownership
All checks were successful
DCO action / DCO (pull_request) Successful in 5m13s
Build / Build Components (1.21) (pull_request) Successful in 9m51s
Build / Build Components (1.22) (pull_request) Successful in 10m21s
Vulncheck / Vulncheck (pull_request) Successful in 12m9s
Tests and linters / Lint (pull_request) Successful in 16m50s
Tests and linters / gopls check (pull_request) Successful in 18m8s
Tests and linters / Staticcheck (pull_request) Successful in 19m1s
Tests and linters / Tests (1.22) (pull_request) Successful in 19m59s
Tests and linters / Tests with -race (pull_request) Successful in 20m5s
Tests and linters / Tests (1.21) (pull_request) Successful in 2m52s
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-02 11:57:39 +03:00
c9efaa5819 [#966] node: Add path of the write_cache to metric labels
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-05-02 06:46:46 +00:00
4730ecfdb8 [#966] node: Refactor WriteCacheMetrics interface
Grouping common fields of methods will enhance the readability of the interface.

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-05-02 06:46:46 +00:00
52bebe9452 [#1110] node: Use single handler for new epoch event
All checks were successful
DCO action / DCO (pull_request) Successful in 5m7s
Build / Build Components (1.21) (pull_request) Successful in 6m54s
Build / Build Components (1.22) (pull_request) Successful in 6m53s
Tests and linters / Staticcheck (pull_request) Successful in 7m45s
Tests and linters / Lint (pull_request) Successful in 8m29s
Vulncheck / Vulncheck (pull_request) Successful in 8m4s
Tests and linters / gopls check (pull_request) Successful in 8m43s
Tests and linters / Tests (1.22) (pull_request) Successful in 11m48s
Tests and linters / Tests (1.21) (pull_request) Successful in 11m54s
Tests and linters / Tests with -race (pull_request) Successful in 12m48s
Bootstrap logic depends on the netmap status, which in turn depends on
the node info. Updating them in a single thread makes things more
predictable.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-27 15:17:45 +03:00
4b514f5ba0 [#1110] node: Log maintenance stop only if it was enabled
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-27 15:17:45 +03:00
c8e2ca2ab4 [#1110] node: Rename handleLocalNodeInfo()
It is used in "handler" only once, what we really do is set the
variable. And we have another "local" node info in `cfgNodeInfo`, this
one is not really local (node info), more like (local node) info, so use
setContractNodeInfo to distinguish it from the local view on the node
info.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-27 15:17:45 +03:00
3b9d12b11c [#1110] node: Fix comment about nodeInfo type
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-27 15:17:45 +03:00
411a8d0245 [#1004] blobovnicza: Use TTL for blobovnicza tree cache
All checks were successful
DCO action / DCO (pull_request) Successful in 12m46s
Vulncheck / Vulncheck (pull_request) Successful in 14m10s
Build / Build Components (1.22) (pull_request) Successful in 15m23s
Build / Build Components (1.21) (pull_request) Successful in 15m38s
Tests and linters / Staticcheck (pull_request) Successful in 3m36s
Tests and linters / gopls check (pull_request) Successful in 4m0s
Tests and linters / Lint (pull_request) Successful in 5m8s
Tests and linters / Tests (1.21) (pull_request) Successful in 7m41s
Tests and linters / Tests with -race (pull_request) Successful in 7m46s
Tests and linters / Tests (1.22) (pull_request) Successful in 7m58s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-26 19:54:29 +03:00
112a7c690f [#1103] node: Implement Get\Head requests for EC object
All checks were successful
DCO action / DCO (pull_request) Successful in 1m44s
Vulncheck / Vulncheck (pull_request) Successful in 3m3s
Build / Build Components (1.21) (pull_request) Successful in 4m0s
Build / Build Components (1.22) (pull_request) Successful in 3m57s
Tests and linters / Staticcheck (pull_request) Successful in 4m46s
Tests and linters / gopls check (pull_request) Successful in 4m48s
Tests and linters / Lint (pull_request) Successful in 5m45s
Tests and linters / Tests (1.21) (pull_request) Successful in 8m57s
Tests and linters / Tests with -race (pull_request) Successful in 9m10s
Tests and linters / Tests (1.22) (pull_request) Successful in 9m20s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 18:15:53 +03:00
167c52a1a9 [#1103] node: Reduce amount of lines for method StorageEngine.head
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 16:31:04 +03:00
700e891b85 [#1103] Fix end of file and trim trailing whitespace
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 16:31:04 +03:00
1f02ac2566 [#1103] pre-commit: Exclude *.svg
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 16:31:04 +03:00
7bc3003803 [#1104] docs: Add bearer token description to auth doc
All checks were successful
Build / Build Components (1.21) (pull_request) Successful in 8m37s
DCO action / DCO (pull_request) Successful in 8m26s
Vulncheck / Vulncheck (pull_request) Successful in 8m24s
Build / Build Components (1.22) (pull_request) Successful in 13m3s
Tests and linters / gopls check (pull_request) Successful in 13m3s
Tests and linters / Staticcheck (pull_request) Successful in 14m26s
Tests and linters / Lint (pull_request) Successful in 15m9s
Tests and linters / Tests with -race (pull_request) Successful in 16m50s
Tests and linters / Tests (1.21) (pull_request) Successful in 17m5s
Tests and linters / Tests (1.22) (pull_request) Successful in 17m8s
This is about authentication only and eACL is deprecated, so only
mention `allow_impersonate` flag.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-23 14:14:17 +03:00
6d4583f5de [#1097] docs: Describe authentication mechanisms
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-16 15:56:06 +00:00
10ee865e98 [#1096] tree: Make verifyClient fill ape request with user claim tags
All checks were successful
Vulncheck / Vulncheck (pull_request) Successful in 2m52s
DCO action / DCO (pull_request) Successful in 2m51s
Build / Build Components (1.21) (pull_request) Successful in 3m44s
Build / Build Components (1.22) (pull_request) Successful in 3m48s
Tests and linters / Staticcheck (pull_request) Successful in 4m34s
Tests and linters / gopls check (pull_request) Successful in 4m48s
Tests and linters / Lint (pull_request) Successful in 5m42s
Tests and linters / Tests with -race (pull_request) Successful in 8m56s
Tests and linters / Tests (1.21) (pull_request) Successful in 9m7s
Tests and linters / Tests (1.22) (pull_request) Successful in 9m15s
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-16 15:12:46 +03:00
c21d72ac23 [#1096] object: Make ape middleware fill request with user claim tags
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-16 15:12:44 +03:00
6772976657 [#1096] container: Make ape middleware fill request with user claim tags
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-16 15:10:20 +03:00
97e54066d0 [#1095] adm: Support user/group target for APE
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-16 11:03:50 +00:00
46bc6a7930 [#1095] cli: Support user/group target for local overrides
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-16 11:03:50 +00:00
3ea1d7b729 [#1089] control: Add USER and GROUP targets for local override storage
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-16 11:03:50 +00:00
0094186299 [#1089] control: Format proto files with clang-format
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-16 11:03:50 +00:00
91e79c98ba [#1089] ape: Provide request actor as an additional target
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-16 11:03:50 +00:00
e5e0542482 [#1085] log: Move storage log message to constants package
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-15 07:57:00 +00:00
5be36924e3 [#41] log: Log storage operations in only in Debug
They are mostly useless unless we need to _debug_ a specific issue.
The amount of logs we produce is too big.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-15 07:57:00 +00:00
6a46c6d229 [#1090] tree: Make workaround for APE checks
* Make `verifyClient` method perform APE check if a container
  was created with zero-filled basic ACL.
* Object verbs are used in APE, until tree verbs are introduced.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-15 07:45:45 +00:00
f4dcb418f2 [#1090] ape: Move ape request and resource implementations to common package
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-15 07:45:45 +00:00
661faa639e [#1090] go.mod: Update policy-engine version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-15 07:45:45 +00:00
cdae227f82 [#1083] go.mod: Bump grpc version
All checks were successful
DCO action / DCO (pull_request) Successful in 5m52s
Vulncheck / Vulncheck (pull_request) Successful in 6m19s
Build / Build Components (1.22) (pull_request) Successful in 7m43s
Build / Build Components (1.21) (pull_request) Successful in 7m49s
Tests and linters / Staticcheck (pull_request) Successful in 8m2s
Tests and linters / Lint (pull_request) Successful in 8m47s
Tests and linters / gopls check (pull_request) Successful in 8m29s
Tests and linters / Tests (1.21) (pull_request) Successful in 13m20s
Tests and linters / Tests (1.22) (pull_request) Successful in 13m26s
Tests and linters / Tests with -race (pull_request) Successful in 13m32s
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-10 16:07:57 +03:00
40781b3a20 [#1086] engine: Change mode in case of errors async
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 12:29:43 +00:00
5ef5734c4e Reapply "[#972] Drop x/exp/slices dependency"
This reverts commit 946f2ec2bf.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
669103a33e Reapply "[#972] Use slices.Sort* when useful"
This reverts commit 3359349acb.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
2c4b50a71e Reapply "[#972] Use require.ElementsMatch() where possible"
This reverts commit 7627d08914.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
2ca5dfc2f6 Reapply "[#972] Adopt slices.BinarySearch()"
This reverts commit 4bfc6d29b9.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
3dc81cb4fc Reapply "[#972] Use min/max builtins"
This reverts commit dad56d2e98.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
d864945961 Reapply "[#972] pilorama: Remove removeDuplicatesInPlace()"
This reverts commit 9f68305c2e.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
43fdb1da45 Reapply "[#972] go.mod: Bump go version to go1.21"
This reverts commit 9adcb253be.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
1b17258c04 [#1029] metabase: Add refill metrics
All checks were successful
DCO action / DCO (pull_request) Successful in 1m22s
Vulncheck / Vulncheck (pull_request) Successful in 3m11s
Build / Build Components (1.21) (pull_request) Successful in 3m56s
Build / Build Components (1.20) (pull_request) Successful in 3m59s
Tests and linters / Staticcheck (pull_request) Successful in 5m31s
Tests and linters / gopls check (pull_request) Successful in 5m26s
Tests and linters / Lint (pull_request) Successful in 6m13s
Tests and linters / Tests (1.20) (pull_request) Successful in 8m54s
Tests and linters / Tests (1.21) (pull_request) Successful in 9m13s
Tests and linters / Tests with -race (pull_request) Successful in 9m30s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 13:05:44 +03:00
e3d9dd6ee8 [#1024] blobovnicza: Copy data on iterate
DB value is only valid while the tx is alive.
But handler may to run something in other goroutine.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:21:11 +03:00
57466594fb [#1024] shard: Resync metabase concurrently
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:21:10 +03:00
1005bf4f56 [#1024] shard: Add refill metabase benchmark
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:21:10 +03:00
76398c06b0 [#1080] metabase: Add StorageID metric
All checks were successful
DCO action / DCO (pull_request) Successful in 1m19s
Build / Build Components (1.21) (pull_request) Successful in 4m6s
Vulncheck / Vulncheck (pull_request) Successful in 4m35s
Build / Build Components (1.20) (pull_request) Successful in 7m32s
Tests and linters / gopls check (pull_request) Successful in 7m43s
Tests and linters / Lint (pull_request) Successful in 9m51s
Tests and linters / Staticcheck (pull_request) Successful in 9m53s
Tests and linters / Tests (1.20) (pull_request) Successful in 12m48s
Tests and linters / Tests (1.21) (pull_request) Successful in 13m11s
Tests and linters / Tests with -race (pull_request) Successful in 14m4s
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:00:08 +03:00
7b1adfed3e [#1080] metabase: Open bucket for container counter once
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:00:08 +03:00
e74bdaa5d5 [#1080] ape: Use value for APE request
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-09 18:42:03 +03:00
338d8cbebd [#1080] ape: Do not read object headers before Head/Get
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-09 15:27:40 +03:00
5b8200de88 [#984] blobovnicza: Do not fail rebuild on big objects
If blobovnicza contains objects larger than object size parameter
value, then rebuild fails with an error, because there is no such
bucket in database. This commit forces to create bucket on rebuild.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-09 11:51:18 +00:00