achuprov/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Pull requests Activity
4771 commits 43 branches 27 tags 24 MiB
Commit graph

4771 commits

This branch
This branch
All branches
Author SHA1 Message Date
Dmitrii Stepanov
cc2449beaf [#1158] metabase: Fix EC storage schema 
Do not store EC info twice.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-06-04 17:24:40 +03:00
Alexander Chuprov
643480d6fa [#1146] adm: Make --group-name flag required 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-06-03 13:54:59 +03:00
Evgenii Stratonikov
f2d2908745 Release v0.40.0 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-31 21:58:09 +03:00
Airat Arifullin
5aacb8fc86 [#1144] metabase: Save parent attributes for ec-chunks 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-31 19:55:32 +03:00
Airat Arifullin
f8e33f8e3a [#1144] metabase: Proprely choose root OID for EC-splitted objects 
* If EC-parent is a part of Split itself, then save to root bucket
  its parent;
* If EC-parent is not a part of Split itself, then save to root bucket
  OID of this EC-parent.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-31 19:53:32 +03:00
Airat Arifullin
f0edebea18 [#1144] metabase: Support ec parent filter for Search 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-31 19:53:32 +03:00
Airat Arifullin
0b367007fc [#1152] go.mod: Update api-go and sdk versions 
* Resolve conflicts for apemanager since api-go
  contains ape and apemanager packages and SDK only
  ape package.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-31 15:39:09 +03:00
Anton Nikiforov
92e19feb57 [#1147] node: Use public fields for shard.ExistsPrm 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
c1af13b47e [#1147] node: Fix issue from gopls 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
9f80d6d9a2 [#1147] Makefile: Fix gopls-run target 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
6130650bb6 [#1147] node: Implement Lock\Delete requests for EC object 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
88b8ddd902 [#1147] cli: Fix output when print EC info with flags json & proto 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
a82c8cc5b8 [#1147] gc: Execute callback for expired tombstones when they exists 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
d355274cd0 [#1147] object: Use methods on pointer for searchsvc.execCtx 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
3555c73225 [#1147] object: Use methods on pointer for deletesvc.execCtx 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
e43e7bec3a [#1147] log: Remove redundant address field from log 
Filled when logger created for `request` object from package `getsvc`.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
50923ed81c [#1147] Fix gofumpt issue 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Alex Vanin
4a34d0d40e [#1149] go.mod: Bump neo-go up to v0.106.0 
Required to work with neo-go v0.106.0 node
with default hardfork configuration. Without
neo-go client version bump, it throws error.

  failed to get network magic: unexpected hardfork: Cockatrice

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-05-30 08:11:58 +00:00
Ekaterina Lebedeva
cc3f762cf2 [#12] cli: clarify -g usage in container list 
Add usage replacement for `container list -g` and verbose
warning when using `-g` without `--owner`.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-05-28 15:18:30 +03:00
Airat Arifullin
3627b44e92 [#1142] tree: Fill APE-request with source IP property 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-27 10:17:17 +00:00
Airat Arifullin
482c5129ac [#1142] object: Fill APE-request with source IP property 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-27 10:17:17 +00:00
Airat Arifullin
43625e7536 [#1142] container: Fill APE-request property with source IP 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-27 10:17:17 +00:00
Airat Arifullin
2b02f52cd9 [#1105] cli: Add apemanager commands 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-27 09:34:21 +00:00
Airat Arifullin
542d3adcb2 [#1105] apemanager: Implement apemanager service 
* Introduce grpc server for apemanager service and
  its implementation in `pkg/services/apemanager`.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-27 09:34:21 +00:00
Airat Arifullin
51ade979e8 [#1105] ape: Introduce contract storage with proxy contract verification 
* `ProxyVerificationContractStorage` uses Proxy contract as a cosigner.
* `ProxyVerificationContractStorage` recreates a contract storage for each handler
  invocation because of an issue: rpc-actor from morph client may be expired. This
  way won't create a bottlenecks because it is expected that this contract storage
  implementation will be used not so often.
* Make morph client return `RPCActor` (that is websocket client in fact).
* Make `SwitchRPCGuardedActor` return `RPCActor` as it will be used for
  `ProxyVerificationContractStorage`.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-27 09:34:21 +00:00
Airat Arifullin
2697d4d1fe [#1143] node: Fix frostfsid cache 
* Fix `subjectFromSubjectExtended` converter.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-20 18:49:52 +03:00
Airat Arifullin
40b04c00ef [#1141] metabase: Fix IsUserObject method 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-20 13:24:15 +03:00
Airat Arifullin
89a80e9a0f [#1141] metabase: Fix putUniqueIndexItem 
* `GetECHeader` is not correct way to determine if an object's got
  EC-header: `ECHeader` must be used for that.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-20 13:24:08 +03:00
Airat Arifullin
8fd678e269 [#1141] go.mod: Update frostfs-sdk-go and frostfs-api-go/v2 
* Also fix unit-test.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-20 13:22:48 +03:00
Dmitrii Stepanov
436c9f5558 [#1129] policer: Restore EC object 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-17 14:36:18 +03:00
Dmitrii Stepanov
44f2e8f27f [#1129] putSvc: Allow to put single unprepared object to EC container 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:49 +03:00
Dmitrii Stepanov
0e42126ddc [#1129] object: Fix check owner for EC part 
Do not validate EC part owner if request from container node.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:49 +03:00
Dmitrii Stepanov
1cd8562db8 [#1129] policer: Refactor shortage 
Drop override inside method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:49 +03:00
Dmitrii Stepanov
4ab6c404f7 [#1129] policer: Drop unused 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:49 +03:00
Dmitrii Stepanov
cbe9757490 [#1129] policer: Pull required EC chunks 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:49 +03:00
Dmitrii Stepanov
d45d086acd [#1129] policer: Add EC chunk replication 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:48 +03:00
Evgenii Stratonikov
af57d5a6a1 [#1138] adm/test: Use --size=1 for negative tests 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-16 10:40:59 +00:00
Evgenii Stratonikov
2f04ce2f79 [#1138] adm: Parallelize generate-alphabet command 
This is the longest test in our suite, try to help CI a bit.

Before:
```
$ go test -run=TestGenerateAlphabet -count=10 .
ok      git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/modules/morph/generate      45.400s
```

After:
```
$ go test -run=TestGenerateAlphabet -count=10 .
ok      git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/modules/morph/generate      33.267s
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-16 10:40:59 +00:00
Alexander Chuprov
b078fe5ba1 [#1092] control: Move SignMessage to separate package 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-05-16 12:14:01 +03:00
Evgenii Stratonikov
f3e09cb09b [#1135] sdnotify: Send MONOTONIC_USEC on reload 
Fixes #1135

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 12:52:52 +00:00
Anton Nikiforov
5c582e96fd [#1136] metabase: Fix creation of ECInfoError 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-15 11:04:27 +00:00
Evgenii Stratonikov
b3eaa8a9bc [#1083] objsvc/v2: Check response status in RANGE_HASH forwarder 
Fixes #1083

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 12:04:21 +03:00
Evgenii Stratonikov
0924b62a95 [#1083] objsvc/v2: Unify response verification after forwarding 
1. Use the same routine for HEAD/GET_RANGE methods.
2. Make error message similar.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 12:04:06 +03:00
Evgenii Stratonikov
300654b045 [#1083] objsvc/v2: Properly check response status after forwarding 
Previously we had cryptic error:
```
debug   get/remote.go:38        remote call failed      {"component": "Object.Get service", "request": "HEAD", "address": "9sTxoVrhJ7WBtXQfK2NJ7zDV5yCF7BPLKK1XTxYPdGsP/BbHV4KZZ8y2BPqAT5kyjdHRLkfbtY2xf5uYoMVqxACn1", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "unexpected header type <nil>"}
```
Now we have and expected error:
```
debug   get/remote.go:38        remote call failed      {"component": "Object.Get service", "request": "HEAD", "address": "D2rqaMG4D2VHdv3HKky8UYSYmwQFH2v9oXXqtyRZPTMy/BbHV4KZZ8y2BPqAT5kyjdHRLkfbtY2xf5uYoMVqxACn1", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"}
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 12:04:06 +03:00
Evgenii Stratonikov
b4cfc80579 [#1133] node: Improve tests for shards.default config section 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 09:48:21 +03:00
Evgenii Stratonikov
d0f64c23a5 Release v0.39.0 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-14 17:02:00 +03:00
Evgenii Stratonikov
6e71ae3bda [#1130] fstree: Remove useless Stat() call 
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/blobstor
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                           │     old     │                new                 │
                                           │   sec/op    │   sec/op     vs base               │
SubstorageReadPerf/fstree_nosync-seq100-8    2.689µ ± 2%   2.428µ ± 4%  -9.72% (p=0.000 n=10)
SubstorageReadPerf/fstree_nosync-rand100-8   2.727µ ± 1%   2.497µ ± 2%  -8.42% (p=0.000 n=10)
geomean                                      2.708µ        2.462µ       -9.07%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-14 16:05:45 +03:00
Evgenii Stratonikov
bf9bdde8ea [#1128] util/test: Remove unused package 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-14 12:48:13 +00:00
Airat Arifullin
952d13cd2b [#1124] cli: Improve APE rule parsing 
* Make APE rule parser to read condition's kind in unambiguous using lexemes
`ResourceCondition`, `RequestCondition` instead confusing `Object.Request`, `Object.Resource`.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-14 12:23:26 +03:00
aarifullin
20baf6e112 [#1108] ape: Update policy-engine version for listing by iteration 
* Update go.mod with a new version of policy-engine pacakge.
* Adapt SwitchRPCGuardedActor to ContractStorage interface.
* Fix `frostfs-adm` util.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-14 12:17:56 +03:00