george.bartolomey/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Pull requests Activity
4698 commits 7 branches 53 tags 45 MiB
Commit graph

3086 commits

Author SHA1 Message Date
Airat Arifullin
542d3adcb2 [#1105] apemanager: Implement apemanager service 
* Introduce grpc server for apemanager service and
  its implementation in `pkg/services/apemanager`.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-27 09:34:21 +00:00
Airat Arifullin
51ade979e8 [#1105] ape: Introduce contract storage with proxy contract verification 
* `ProxyVerificationContractStorage` uses Proxy contract as a cosigner.
* `ProxyVerificationContractStorage` recreates a contract storage for each handler
  invocation because of an issue: rpc-actor from morph client may be expired. This
  way won't create a bottlenecks because it is expected that this contract storage
  implementation will be used not so often.
* Make morph client return `RPCActor` (that is websocket client in fact).
* Make `SwitchRPCGuardedActor` return `RPCActor` as it will be used for
  `ProxyVerificationContractStorage`.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-27 09:34:21 +00:00
Airat Arifullin
40b04c00ef [#1141] metabase: Fix IsUserObject method 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-20 13:24:15 +03:00
Airat Arifullin
89a80e9a0f [#1141] metabase: Fix putUniqueIndexItem 
* `GetECHeader` is not correct way to determine if an object's got
  EC-header: `ECHeader` must be used for that.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-20 13:24:08 +03:00
Airat Arifullin
8fd678e269 [#1141] go.mod: Update frostfs-sdk-go and frostfs-api-go/v2 
* Also fix unit-test.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-20 13:22:48 +03:00
Dmitrii Stepanov
436c9f5558 [#1129] policer: Restore EC object 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-17 14:36:18 +03:00
Dmitrii Stepanov
44f2e8f27f [#1129] putSvc: Allow to put single unprepared object to EC container 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:49 +03:00
Dmitrii Stepanov
0e42126ddc [#1129] object: Fix check owner for EC part 
Do not validate EC part owner if request from container node.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:49 +03:00
Dmitrii Stepanov
1cd8562db8 [#1129] policer: Refactor shortage 
Drop override inside method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:49 +03:00
Dmitrii Stepanov
4ab6c404f7 [#1129] policer: Drop unused 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:49 +03:00
Dmitrii Stepanov
cbe9757490 [#1129] policer: Pull required EC chunks 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:49 +03:00
Dmitrii Stepanov
d45d086acd [#1129] policer: Add EC chunk replication 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:48 +03:00
Alexander Chuprov
b078fe5ba1 [#1092] control: Move SignMessage to separate package 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-05-16 12:14:01 +03:00
Evgenii Stratonikov
f3e09cb09b [#1135] sdnotify: Send MONOTONIC_USEC on reload 
Fixes #1135

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 12:52:52 +00:00
Anton Nikiforov
5c582e96fd [#1136] metabase: Fix creation of ECInfoError 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-15 11:04:27 +00:00
Evgenii Stratonikov
b3eaa8a9bc [#1083] objsvc/v2: Check response status in RANGE_HASH forwarder 
Fixes #1083

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 12:04:21 +03:00
Evgenii Stratonikov
0924b62a95 [#1083] objsvc/v2: Unify response verification after forwarding 
1. Use the same routine for HEAD/GET_RANGE methods.
2. Make error message similar.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 12:04:06 +03:00
Evgenii Stratonikov
300654b045 [#1083] objsvc/v2: Properly check response status after forwarding 
Previously we had cryptic error:
```
debug   get/remote.go:38        remote call failed      {"component": "Object.Get service", "request": "HEAD", "address": "9sTxoVrhJ7WBtXQfK2NJ7zDV5yCF7BPLKK1XTxYPdGsP/BbHV4KZZ8y2BPqAT5kyjdHRLkfbtY2xf5uYoMVqxACn1", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "unexpected header type <nil>"}
```
Now we have and expected error:
```
debug   get/remote.go:38        remote call failed      {"component": "Object.Get service", "request": "HEAD", "address": "D2rqaMG4D2VHdv3HKky8UYSYmwQFH2v9oXXqtyRZPTMy/BbHV4KZZ8y2BPqAT5kyjdHRLkfbtY2xf5uYoMVqxACn1", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"}
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 12:04:06 +03:00
Evgenii Stratonikov
6e71ae3bda [#1130] fstree: Remove useless Stat() call 
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/blobstor
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                           │     old     │                new                 │
                                           │   sec/op    │   sec/op     vs base               │
SubstorageReadPerf/fstree_nosync-seq100-8    2.689µ ± 2%   2.428µ ± 4%  -9.72% (p=0.000 n=10)
SubstorageReadPerf/fstree_nosync-rand100-8   2.727µ ± 1%   2.497µ ± 2%  -8.42% (p=0.000 n=10)
geomean                                      2.708µ        2.462µ       -9.07%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-14 16:05:45 +03:00
Evgenii Stratonikov
bf9bdde8ea [#1128] util/test: Remove unused package 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-14 12:48:13 +00:00
Airat Arifullin
952d13cd2b [#1124] cli: Improve APE rule parsing 
* Make APE rule parser to read condition's kind in unambiguous using lexemes
`ResourceCondition`, `RequestCondition` instead confusing `Object.Request`, `Object.Resource`.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-14 12:23:26 +03:00
aarifullin
20baf6e112 [#1108] ape: Update policy-engine version for listing by iteration 
* Update go.mod with a new version of policy-engine pacakge.
* Adapt SwitchRPCGuardedActor to ContractStorage interface.
* Fix `frostfs-adm` util.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-14 12:17:56 +03:00
Dmitrii Stepanov
0144117cc9 [#1125] objectSvc: Add EC header APE check 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-08 16:25:55 +03:00
Dmitrii Stepanov
ada1b9f737 [#1120] objectSvc: Fix EC put placement 
Use parent object ID to compute placement.
Fix too many copies saving.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-08 15:23:57 +03:00
Anton Nikiforov
fe2c1c926f [#1112] node: Fix race warning for GetObjectAndWritePayload 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-07 14:47:21 +03:00
Anton Nikiforov
3e782527b8 [#1112] node: Add test for Range request for EC object 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-07 14:47:21 +03:00
Anton Nikiforov
21a490da8f [#1112] Fix issue from gofumpt 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-07 14:47:21 +03:00
Evgenii Stratonikov
93c0ccad4f [#1077] objectsvc: Fix possible panic in GetRange() 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-07 14:47:21 +03:00
Anton Nikiforov
00b2b77b26 [#1112] node: Implement Range\RangeHash requests for EC object 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-07 14:47:21 +03:00
aarifullin
b60a51b862 [#1117] ape: Introduce FormFrostfsIDRequestProperties method 
* `FormFrostfsIDRequestProperties` gets user claim tags and group id and sets them
  as ape request properties.
* Make tree, container and object service use the method.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-07 10:01:21 +00:00
aarifullin
6c76c9b457 [#1117] core: Introduce SubjectProvider interface for FrostfsID 
* Make tree, object and container services use SubjectProvider interface.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-07 10:01:21 +00:00
aarifullin
45f4e6939d [#1117] morph: Make frostfsid client provide GetSubjectExtended method 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-07 10:01:21 +00:00
Ekaterina Lebedeva
e07869a8cf [#1100] Remove unused fields 
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-05-06 10:14:36 +03:00
Alexander Chuprov
ec2873caa7 [#1116] node: Fix writecache metrics 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-05-02 20:03:33 +03:00
Evgenii Stratonikov
71789676d5 [#1114] aclsvc: Add tests for request ownership 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-02 11:57:39 +03:00
Alexander Chuprov
c9efaa5819 [#966] node: Add path of the write_cache to metric labels 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-05-02 06:46:46 +00:00
Alexander Chuprov
4730ecfdb8 [#966] node: Refactor WriteCacheMetrics interface 
Grouping common fields of methods will enhance the readability of the interface.

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-05-02 06:46:46 +00:00
Anton Nikiforov
411a8d0245 [#1004] blobovnicza: Use TTL for blobovnicza tree cache 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-26 19:54:29 +03:00
Anton Nikiforov
112a7c690f [#1103] node: Implement Get\Head requests for EC object 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-24 18:15:53 +03:00
Anton Nikiforov
167c52a1a9 [#1103] node: Reduce amount of lines for method StorageEngine.head 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-24 16:31:04 +03:00
Anton Nikiforov
700e891b85 [#1103] Fix end of file and trim trailing whitespace 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-24 16:31:04 +03:00
Airat Arifullin
10ee865e98 [#1096] tree: Make verifyClient fill ape request with user claim tags 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-04-16 15:12:46 +03:00
Airat Arifullin
c21d72ac23 [#1096] object: Make ape middleware fill request with user claim tags 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-04-16 15:12:44 +03:00
Airat Arifullin
6772976657 [#1096] container: Make ape middleware fill request with user claim tags 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-04-16 15:10:20 +03:00
Evgenii Stratonikov
3ea1d7b729 [#1089] control: Add USER and GROUP targets for local override storage 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-16 11:03:50 +00:00
Evgenii Stratonikov
0094186299 [#1089] control: Format proto files with clang-format 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-16 11:03:50 +00:00
Evgenii Stratonikov
91e79c98ba [#1089] ape: Provide request actor as an additional target 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-16 11:03:50 +00:00
Evgenii Stratonikov
e5e0542482 [#1085] log: Move storage log message to constants package 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-15 07:57:00 +00:00
Evgenii Stratonikov
5be36924e3 [#41] log: Log storage operations in only in Debug 
They are mostly useless unless we need to _debug_ a specific issue.
The amount of logs we produce is too big.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-15 07:57:00 +00:00
aarifullin
6a46c6d229 [#1090] tree: Make workaround for APE checks 
* Make `verifyClient` method perform APE check if a container
  was created with zero-filled basic ACL.
* Object verbs are used in APE, until tree verbs are introduced.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-04-15 07:45:45 +00:00