Commit Graph

4660 Commits (93c0ccad4ffbda4b5b4a9a99af8ac88c97a9df28)

Author SHA1 Message Date
Evgenii Stratonikov 93c0ccad4f [#1077] objectsvc: Fix possible panic in GetRange()
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-07 14:47:21 +03:00
Anton Nikiforov 00b2b77b26 [#1112] node: Implement `Range\RangeHash` requests for EC object
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-05-07 14:47:21 +03:00
Evgenii Stratonikov a23d53b2d4 [#1118] .forgejo: Add pre-commit action
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-07 11:36:11 +00:00
Evgenii Stratonikov fc7b07f314 [#1118] docs: Fix pre-commit warnings
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-07 11:36:11 +00:00
Airat Arifullin 9cc51f86b7 [#1117] node: Introduce cache for frostfsid contract client
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-07 10:01:21 +00:00
Airat Arifullin b60a51b862 [#1117] ape: Introduce `FormFrostfsIDRequestProperties` method
* `FormFrostfsIDRequestProperties` gets user claim tags and group id and sets them
  as ape request properties.
* Make tree, container and object service use the method.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-07 10:01:21 +00:00
Airat Arifullin 6c76c9b457 [#1117] core: Introduce SubjectProvider interface for FrostfsID
* Make tree, object and container services use SubjectProvider interface.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-07 10:01:21 +00:00
Airat Arifullin 45f4e6939d [#1117] morph: Make frostfsid client provide GetSubjectExtended method
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-05-07 10:01:21 +00:00
Ekaterina Lebedeva e07869a8cf [#1100] Remove unused fields
DCO action / DCO (pull_request) Successful in 2m47s Details
Build / Build Components (1.21) (pull_request) Successful in 3m21s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m33s Details
Build / Build Components (1.22) (pull_request) Successful in 5m25s Details
Tests and linters / gopls check (pull_request) Successful in 5m13s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m13s Details
Tests and linters / Lint (pull_request) Successful in 7m2s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 10m2s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m55s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 10m11s Details
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2024-05-06 10:14:36 +03:00
Alexander Chuprov ec2873caa7 [#1116] node: Fix writecache metrics
DCO action / DCO (pull_request) Successful in 1m35s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m30s Details
Build / Build Components (1.22) (pull_request) Successful in 4m40s Details
Build / Build Components (1.21) (pull_request) Successful in 4m46s Details
Tests and linters / gopls check (pull_request) Successful in 5m22s Details
Tests and linters / Staticcheck (pull_request) Successful in 6m19s Details
Tests and linters / Lint (pull_request) Successful in 7m7s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 9m36s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m30s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 9m46s Details
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-05-02 20:03:33 +03:00
Evgenii Stratonikov 71789676d5 [#1114] aclsvc: Add tests for request ownership
DCO action / DCO (pull_request) Successful in 5m13s Details
Build / Build Components (1.21) (pull_request) Successful in 9m51s Details
Build / Build Components (1.22) (pull_request) Successful in 10m21s Details
Vulncheck / Vulncheck (pull_request) Successful in 12m9s Details
Tests and linters / Lint (pull_request) Successful in 16m50s Details
Tests and linters / gopls check (pull_request) Successful in 18m8s Details
Tests and linters / Staticcheck (pull_request) Successful in 19m1s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 19m59s Details
Tests and linters / Tests with -race (pull_request) Successful in 20m5s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 2m52s Details
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-05-02 11:57:39 +03:00
Alexander Chuprov c9efaa5819 [#966] node: Add path of the write_cache to metric labels
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-05-02 06:46:46 +00:00
Alexander Chuprov 4730ecfdb8 [#966] node: Refactor WriteCacheMetrics interface
Grouping common fields of methods will enhance the readability of the interface.

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2024-05-02 06:46:46 +00:00
Evgenii Stratonikov 52bebe9452 [#1110] node: Use single handler for new epoch event
DCO action / DCO (pull_request) Successful in 5m7s Details
Build / Build Components (1.21) (pull_request) Successful in 6m54s Details
Build / Build Components (1.22) (pull_request) Successful in 6m53s Details
Tests and linters / Staticcheck (pull_request) Successful in 7m45s Details
Tests and linters / Lint (pull_request) Successful in 8m29s Details
Vulncheck / Vulncheck (pull_request) Successful in 8m4s Details
Tests and linters / gopls check (pull_request) Successful in 8m43s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 11m48s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 11m54s Details
Tests and linters / Tests with -race (pull_request) Successful in 12m48s Details
Bootstrap logic depends on the netmap status, which in turn depends on
the node info. Updating them in a single thread makes things more
predictable.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-27 15:17:45 +03:00
Evgenii Stratonikov 4b514f5ba0 [#1110] node: Log maintenance stop only if it was enabled
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-27 15:17:45 +03:00
Evgenii Stratonikov c8e2ca2ab4 [#1110] node: Rename handleLocalNodeInfo()
It is used in "handler" only once, what we really do is set the
variable. And we have another "local" node info in `cfgNodeInfo`, this
one is not really local (node info), more like (local node) info, so use
setContractNodeInfo to distinguish it from the local view on the node
info.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-27 15:17:45 +03:00
Evgenii Stratonikov 3b9d12b11c [#1110] node: Fix comment about nodeInfo type
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-27 15:17:45 +03:00
Anton Nikiforov 411a8d0245 [#1004] blobovnicza: Use TTL for blobovnicza tree cache
DCO action / DCO (pull_request) Successful in 12m46s Details
Vulncheck / Vulncheck (pull_request) Successful in 14m10s Details
Build / Build Components (1.22) (pull_request) Successful in 15m23s Details
Build / Build Components (1.21) (pull_request) Successful in 15m38s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m36s Details
Tests and linters / gopls check (pull_request) Successful in 4m0s Details
Tests and linters / Lint (pull_request) Successful in 5m8s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 7m41s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m46s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 7m58s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-26 19:54:29 +03:00
Anton Nikiforov 112a7c690f [#1103] node: Implement `Get\Head` requests for EC object
DCO action / DCO (pull_request) Successful in 1m44s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m3s Details
Build / Build Components (1.21) (pull_request) Successful in 4m0s Details
Build / Build Components (1.22) (pull_request) Successful in 3m57s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m46s Details
Tests and linters / gopls check (pull_request) Successful in 4m48s Details
Tests and linters / Lint (pull_request) Successful in 5m45s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 8m57s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m10s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 9m20s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 18:15:53 +03:00
Anton Nikiforov 167c52a1a9 [#1103] node: Reduce amount of lines for method `StorageEngine.head`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 16:31:04 +03:00
Anton Nikiforov 700e891b85 [#1103] Fix end of file and trim trailing whitespace
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 16:31:04 +03:00
Anton Nikiforov 1f02ac2566 [#1103] pre-commit: Exclude `*.svg`
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-24 16:31:04 +03:00
Evgenii Stratonikov 7bc3003803 [#1104] docs: Add bearer token description to auth doc
Build / Build Components (1.21) (pull_request) Successful in 8m37s Details
DCO action / DCO (pull_request) Successful in 8m26s Details
Vulncheck / Vulncheck (pull_request) Successful in 8m24s Details
Build / Build Components (1.22) (pull_request) Successful in 13m3s Details
Tests and linters / gopls check (pull_request) Successful in 13m3s Details
Tests and linters / Staticcheck (pull_request) Successful in 14m26s Details
Tests and linters / Lint (pull_request) Successful in 15m9s Details
Tests and linters / Tests with -race (pull_request) Successful in 16m50s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 17m5s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 17m8s Details
This is about authentication only and eACL is deprecated, so only
mention `allow_impersonate` flag.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-23 14:14:17 +03:00
Evgenii Stratonikov 6d4583f5de [#1097] docs: Describe authentication mechanisms
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-16 15:56:06 +00:00
Airat Arifullin 10ee865e98 [#1096] tree: Make `verifyClient` fill ape request with user claim tags
Vulncheck / Vulncheck (pull_request) Successful in 2m52s Details
DCO action / DCO (pull_request) Successful in 2m51s Details
Build / Build Components (1.21) (pull_request) Successful in 3m44s Details
Build / Build Components (1.22) (pull_request) Successful in 3m48s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m34s Details
Tests and linters / gopls check (pull_request) Successful in 4m48s Details
Tests and linters / Lint (pull_request) Successful in 5m42s Details
Tests and linters / Tests with -race (pull_request) Successful in 8m56s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 9m7s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 9m15s Details
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-16 15:12:46 +03:00
Airat Arifullin c21d72ac23 [#1096] object: Make ape middleware fill request with user claim tags
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-16 15:12:44 +03:00
Airat Arifullin 6772976657 [#1096] container: Make ape middleware fill request with user claim tags
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-16 15:10:20 +03:00
Evgenii Stratonikov 97e54066d0 [#1095] adm: Support user/group target for APE
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-16 11:03:50 +00:00
Evgenii Stratonikov 46bc6a7930 [#1095] cli: Support user/group target for local overrides
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-16 11:03:50 +00:00
Evgenii Stratonikov 3ea1d7b729 [#1089] control: Add USER and GROUP targets for local override storage
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-16 11:03:50 +00:00
Evgenii Stratonikov 0094186299 [#1089] control: Format proto files with clang-format
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-16 11:03:50 +00:00
Evgenii Stratonikov 91e79c98ba [#1089] ape: Provide request actor as an additional target
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-16 11:03:50 +00:00
Evgenii Stratonikov e5e0542482 [#1085] log: Move storage log message to constants package
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-15 07:57:00 +00:00
Evgenii Stratonikov 5be36924e3 [#41] log: Log storage operations in only in Debug
They are mostly useless unless we need to _debug_ a specific issue.
The amount of logs we produce is too big.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-15 07:57:00 +00:00
Airat Arifullin 6a46c6d229 [#1090] tree: Make workaround for APE checks
* Make `verifyClient` method perform APE check if a container
  was created with zero-filled basic ACL.
* Object verbs are used in APE, until tree verbs are introduced.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-15 07:45:45 +00:00
Airat Arifullin f4dcb418f2 [#1090] ape: Move ape request and resource implementations to common package
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-15 07:45:45 +00:00
Airat Arifullin 661faa639e [#1090] go.mod: Update policy-engine version
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2024-04-15 07:45:45 +00:00
Anton Nikiforov cdae227f82 [#1083] go.mod: Bump grpc version
DCO action / DCO (pull_request) Successful in 5m52s Details
Vulncheck / Vulncheck (pull_request) Successful in 6m19s Details
Build / Build Components (1.22) (pull_request) Successful in 7m43s Details
Build / Build Components (1.21) (pull_request) Successful in 7m49s Details
Tests and linters / Staticcheck (pull_request) Successful in 8m2s Details
Tests and linters / Lint (pull_request) Successful in 8m47s Details
Tests and linters / gopls check (pull_request) Successful in 8m29s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 13m20s Details
Tests and linters / Tests (1.22) (pull_request) Successful in 13m26s Details
Tests and linters / Tests with -race (pull_request) Successful in 13m32s Details
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2024-04-10 16:07:57 +03:00
Dmitrii Stepanov 40781b3a20 [#1086] engine: Change mode in case of errors async
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 12:29:43 +00:00
Evgenii Stratonikov 5ef5734c4e Reapply "[#972] Drop x/exp/slices dependency"
This reverts commit 946f2ec2bf.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
Evgenii Stratonikov 669103a33e Reapply "[#972] Use slices.Sort* when useful"
This reverts commit 3359349acb.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
Evgenii Stratonikov 2c4b50a71e Reapply "[#972] Use require.ElementsMatch() where possible"
This reverts commit 7627d08914.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
Evgenii Stratonikov 2ca5dfc2f6 Reapply "[#972] Adopt slices.BinarySearch()"
This reverts commit 4bfc6d29b9.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
Evgenii Stratonikov 3dc81cb4fc Reapply "[#972] Use min/max builtins"
This reverts commit dad56d2e98.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
Evgenii Stratonikov d864945961 Reapply "[#972] pilorama: Remove removeDuplicatesInPlace()"
This reverts commit 9f68305c2e.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
Evgenii Stratonikov 43fdb1da45 Reapply "[#972] go.mod: Bump go version to go1.21"
This reverts commit 9adcb253be.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2024-04-10 12:09:34 +00:00
Dmitrii Stepanov 1b17258c04 [#1029] metabase: Add refill metrics
DCO action / DCO (pull_request) Successful in 1m22s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m11s Details
Build / Build Components (1.21) (pull_request) Successful in 3m56s Details
Build / Build Components (1.20) (pull_request) Successful in 3m59s Details
Tests and linters / Staticcheck (pull_request) Successful in 5m31s Details
Tests and linters / gopls check (pull_request) Successful in 5m26s Details
Tests and linters / Lint (pull_request) Successful in 6m13s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 8m54s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 9m13s Details
Tests and linters / Tests with -race (pull_request) Successful in 9m30s Details
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 13:05:44 +03:00
Dmitrii Stepanov e3d9dd6ee8 [#1024] blobovnicza: Copy data on iterate
DB value is only valid while the tx is alive.
But handler may to run something in other goroutine.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:21:11 +03:00
Dmitrii Stepanov 57466594fb [#1024] shard: Resync metabase concurrently
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:21:10 +03:00
Dmitrii Stepanov 1005bf4f56 [#1024] shard: Add refill metabase benchmark
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2024-04-10 10:21:10 +03:00