fyrchik/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Issues Pull requests Projects Releases Packages Wiki Activity
4871 commits 23 branches 26 tags 45 MiB
Commit graph

4871 commits

This branch
This branch
All branches
Author SHA1 Message Date
Dmitrii Stepanov
cc2449beaf [#1158] metabase: Fix EC storage schema 
Do not store EC info twice.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-06-04 17:24:40 +03:00
Alexander Chuprov
643480d6fa [#1146] adm: Make --group-name flag required 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-06-03 13:54:59 +03:00
Evgenii Stratonikov
f2d2908745 Release v0.40.0 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-31 21:58:09 +03:00
Airat Arifullin
5aacb8fc86 [#1144] metabase: Save parent attributes for ec-chunks 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-31 19:55:32 +03:00
Airat Arifullin
f8e33f8e3a [#1144] metabase: Proprely choose root OID for EC-splitted objects 
* If EC-parent is a part of Split itself, then save to root bucket
  its parent;
* If EC-parent is not a part of Split itself, then save to root bucket
  OID of this EC-parent.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-31 19:53:32 +03:00
Airat Arifullin
f0edebea18 [#1144] metabase: Support ec parent filter for Search 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-31 19:53:32 +03:00
Airat Arifullin
0b367007fc [#1152] go.mod: Update api-go and sdk versions 
* Resolve conflicts for apemanager since api-go
  contains ape and apemanager packages and SDK only
  ape package.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-31 15:39:09 +03:00
Anton Nikiforov
92e19feb57 [#1147] node: Use public fields for shard.ExistsPrm 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
c1af13b47e [#1147] node: Fix issue from gopls 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
9f80d6d9a2 [#1147] Makefile: Fix gopls-run target 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
6130650bb6 [#1147] node: Implement Lock\Delete requests for EC object 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
88b8ddd902 [#1147] cli: Fix output when print EC info with flags json & proto 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
a82c8cc5b8 [#1147] gc: Execute callback for expired tombstones when they exists 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
d355274cd0 [#1147] object: Use methods on pointer for searchsvc.execCtx 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
3555c73225 [#1147] object: Use methods on pointer for deletesvc.execCtx 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
e43e7bec3a [#1147] log: Remove redundant address field from log 
Filled when logger created for `request` object from package `getsvc`.

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Anton Nikiforov
50923ed81c [#1147] Fix gofumpt issue 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-30 08:13:04 +00:00
Alex Vanin
4a34d0d40e [#1149] go.mod: Bump neo-go up to v0.106.0 
Required to work with neo-go v0.106.0 node
with default hardfork configuration. Without
neo-go client version bump, it throws error.

  failed to get network magic: unexpected hardfork: Cockatrice

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2024-05-30 08:11:58 +00:00
Ekaterina Lebedeva
cc3f762cf2 [#12] cli: clarify -g usage in container list 
Add usage replacement for `container list -g` and verbose
warning when using `-g` without `--owner`.

Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-05-28 15:18:30 +03:00
Airat Arifullin
3627b44e92 [#1142] tree: Fill APE-request with source IP property 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-27 10:17:17 +00:00
Airat Arifullin
482c5129ac [#1142] object: Fill APE-request with source IP property 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-27 10:17:17 +00:00
Airat Arifullin
43625e7536 [#1142] container: Fill APE-request property with source IP 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-27 10:17:17 +00:00
Airat Arifullin
2b02f52cd9 [#1105] cli: Add apemanager commands 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-27 09:34:21 +00:00
Airat Arifullin
542d3adcb2 [#1105] apemanager: Implement apemanager service 
* Introduce grpc server for apemanager service and
  its implementation in `pkg/services/apemanager`.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-27 09:34:21 +00:00
Airat Arifullin
51ade979e8 [#1105] ape: Introduce contract storage with proxy contract verification 
* `ProxyVerificationContractStorage` uses Proxy contract as a cosigner.
* `ProxyVerificationContractStorage` recreates a contract storage for each handler
  invocation because of an issue: rpc-actor from morph client may be expired. This
  way won't create a bottlenecks because it is expected that this contract storage
  implementation will be used not so often.
* Make morph client return `RPCActor` (that is websocket client in fact).
* Make `SwitchRPCGuardedActor` return `RPCActor` as it will be used for
  `ProxyVerificationContractStorage`.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-27 09:34:21 +00:00
Airat Arifullin
2697d4d1fe [#1143] node: Fix frostfsid cache 
* Fix `subjectFromSubjectExtended` converter.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-20 18:49:52 +03:00
Airat Arifullin
40b04c00ef [#1141] metabase: Fix IsUserObject method 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-20 13:24:15 +03:00
Airat Arifullin
89a80e9a0f [#1141] metabase: Fix putUniqueIndexItem 
* `GetECHeader` is not correct way to determine if an object's got
  EC-header: `ECHeader` must be used for that.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-20 13:24:08 +03:00
Airat Arifullin
8fd678e269 [#1141] go.mod: Update frostfs-sdk-go and frostfs-api-go/v2 
* Also fix unit-test.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-20 13:22:48 +03:00
Dmitrii Stepanov
436c9f5558 [#1129] policer: Restore EC object 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-17 14:36:18 +03:00
Dmitrii Stepanov
44f2e8f27f [#1129] putSvc: Allow to put single unprepared object to EC container 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:49 +03:00
Dmitrii Stepanov
0e42126ddc [#1129] object: Fix check owner for EC part 
Do not validate EC part owner if request from container node.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:49 +03:00
Dmitrii Stepanov
1cd8562db8 [#1129] policer: Refactor shortage 
Drop override inside method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:49 +03:00
Dmitrii Stepanov
4ab6c404f7 [#1129] policer: Drop unused 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:49 +03:00
Dmitrii Stepanov
cbe9757490 [#1129] policer: Pull required EC chunks 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:49 +03:00
Dmitrii Stepanov
d45d086acd [#1129] policer: Add EC chunk replication 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:48 +03:00
Evgenii Stratonikov
af57d5a6a1 [#1138] adm/test: Use --size=1 for negative tests 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-16 10:40:59 +00:00
Evgenii Stratonikov
2f04ce2f79 [#1138] adm: Parallelize generate-alphabet command 
This is the longest test in our suite, try to help CI a bit.

Before:
```
$ go test -run=TestGenerateAlphabet -count=10 .
ok      git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/modules/morph/generate      45.400s
```

After:
```
$ go test -run=TestGenerateAlphabet -count=10 .
ok      git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/modules/morph/generate      33.267s
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-16 10:40:59 +00:00
Alexander Chuprov
b078fe5ba1 [#1092] control: Move SignMessage to separate package 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-05-16 12:14:01 +03:00
Evgenii Stratonikov
f3e09cb09b [#1135] sdnotify: Send MONOTONIC_USEC on reload 
Fixes #1135

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 12:52:52 +00:00
Anton Nikiforov
5c582e96fd [#1136] metabase: Fix creation of ECInfoError 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-15 11:04:27 +00:00
Evgenii Stratonikov
b3eaa8a9bc [#1083] objsvc/v2: Check response status in RANGE_HASH forwarder 
Fixes #1083

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 12:04:21 +03:00
Evgenii Stratonikov
0924b62a95 [#1083] objsvc/v2: Unify response verification after forwarding 
1. Use the same routine for HEAD/GET_RANGE methods.
2. Make error message similar.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 12:04:06 +03:00
Evgenii Stratonikov
300654b045 [#1083] objsvc/v2: Properly check response status after forwarding 
Previously we had cryptic error:
```
debug   get/remote.go:38        remote call failed      {"component": "Object.Get service", "request": "HEAD", "address": "9sTxoVrhJ7WBtXQfK2NJ7zDV5yCF7BPLKK1XTxYPdGsP/BbHV4KZZ8y2BPqAT5kyjdHRLkfbtY2xf5uYoMVqxACn1", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "unexpected header type <nil>"}
```
Now we have and expected error:
```
debug   get/remote.go:38        remote call failed      {"component": "Object.Get service", "request": "HEAD", "address": "D2rqaMG4D2VHdv3HKky8UYSYmwQFH2v9oXXqtyRZPTMy/BbHV4KZZ8y2BPqAT5kyjdHRLkfbtY2xf5uYoMVqxACn1", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"}
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 12:04:06 +03:00
Evgenii Stratonikov
b4cfc80579 [#1133] node: Improve tests for shards.default config section 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 09:48:21 +03:00
Evgenii Stratonikov
d0f64c23a5 Release v0.39.0 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-14 17:02:00 +03:00
Evgenii Stratonikov
6e71ae3bda [#1130] fstree: Remove useless Stat() call 
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/blobstor
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                           │     old     │                new                 │
                                           │   sec/op    │   sec/op     vs base               │
SubstorageReadPerf/fstree_nosync-seq100-8    2.689µ ± 2%   2.428µ ± 4%  -9.72% (p=0.000 n=10)
SubstorageReadPerf/fstree_nosync-rand100-8   2.727µ ± 1%   2.497µ ± 2%  -8.42% (p=0.000 n=10)
geomean                                      2.708µ        2.462µ       -9.07%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-14 16:05:45 +03:00
Evgenii Stratonikov
bf9bdde8ea [#1128] util/test: Remove unused package 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-14 12:48:13 +00:00
Airat Arifullin
952d13cd2b [#1124] cli: Improve APE rule parsing 
* Make APE rule parser to read condition's kind in unambiguous using lexemes
`ResourceCondition`, `RequestCondition` instead confusing `Object.Request`, `Object.Resource`.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-14 12:23:26 +03:00
aarifullin
20baf6e112 [#1108] ape: Update policy-engine version for listing by iteration 
* Update go.mod with a new version of policy-engine pacakge.
* Adapt SwitchRPCGuardedActor to ContractStorage interface.
* Fix `frostfs-adm` util.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-14 12:17:56 +03:00