potyarkin/frostfs-node
1
0
Fork 0
forked from TrueCloudLab/frostfs-node
Code Pull requests Activity
4886 commits 2 branches 70 tags 46 MiB
Commit graph

4886 commits

This branch
This branch
All branches
Author SHA1 Message Date
Dmitrii Stepanov
d45d086acd [#1129] policer: Add EC chunk replication 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-16 16:28:48 +03:00
Evgenii Stratonikov
af57d5a6a1 [#1138] adm/test: Use --size=1 for negative tests 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-16 10:40:59 +00:00
Evgenii Stratonikov
2f04ce2f79 [#1138] adm: Parallelize generate-alphabet command 
This is the longest test in our suite, try to help CI a bit.

Before:
```
$ go test -run=TestGenerateAlphabet -count=10 .
ok      git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/modules/morph/generate      45.400s
```

After:
```
$ go test -run=TestGenerateAlphabet -count=10 .
ok      git.frostfs.info/TrueCloudLab/frostfs-node/cmd/frostfs-adm/internal/modules/morph/generate      33.267s
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-16 10:40:59 +00:00
Alexander Chuprov
b078fe5ba1 [#1092] control: Move SignMessage to separate package 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-05-16 12:14:01 +03:00
Evgenii Stratonikov
f3e09cb09b [#1135] sdnotify: Send MONOTONIC_USEC on reload 
Fixes #1135

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 12:52:52 +00:00
Anton Nikiforov
5c582e96fd [#1136] metabase: Fix creation of ECInfoError 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-15 11:04:27 +00:00
Evgenii Stratonikov
b3eaa8a9bc [#1083] objsvc/v2: Check response status in RANGE_HASH forwarder 
Fixes #1083

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 12:04:21 +03:00
Evgenii Stratonikov
0924b62a95 [#1083] objsvc/v2: Unify response verification after forwarding 
1. Use the same routine for HEAD/GET_RANGE methods.
2. Make error message similar.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 12:04:06 +03:00
Evgenii Stratonikov
300654b045 [#1083] objsvc/v2: Properly check response status after forwarding 
Previously we had cryptic error:
```
debug   get/remote.go:38        remote call failed      {"component": "Object.Get service", "request": "HEAD", "address": "9sTxoVrhJ7WBtXQfK2NJ7zDV5yCF7BPLKK1XTxYPdGsP/BbHV4KZZ8y2BPqAT5kyjdHRLkfbtY2xf5uYoMVqxACn1", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "unexpected header type <nil>"}
```
Now we have and expected error:
```
debug   get/remote.go:38        remote call failed      {"component": "Object.Get service", "request": "HEAD", "address": "D2rqaMG4D2VHdv3HKky8UYSYmwQFH2v9oXXqtyRZPTMy/BbHV4KZZ8y2BPqAT5kyjdHRLkfbtY2xf5uYoMVqxACn1", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"}
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 12:04:06 +03:00
Evgenii Stratonikov
b4cfc80579 [#1133] node: Improve tests for shards.default config section 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-15 09:48:21 +03:00
Evgenii Stratonikov
d0f64c23a5 Release v0.39.0 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-14 17:02:00 +03:00
Evgenii Stratonikov
6e71ae3bda [#1130] fstree: Remove useless Stat() call 
```
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/blobstor
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
                                           │     old     │                new                 │
                                           │   sec/op    │   sec/op     vs base               │
SubstorageReadPerf/fstree_nosync-seq100-8    2.689µ ± 2%   2.428µ ± 4%  -9.72% (p=0.000 n=10)
SubstorageReadPerf/fstree_nosync-rand100-8   2.727µ ± 1%   2.497µ ± 2%  -8.42% (p=0.000 n=10)
geomean                                      2.708µ        2.462µ       -9.07%
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-14 16:05:45 +03:00
Evgenii Stratonikov
bf9bdde8ea [#1128] util/test: Remove unused package 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-14 12:48:13 +00:00
Airat Arifullin
952d13cd2b [#1124] cli: Improve APE rule parsing 
* Make APE rule parser to read condition's kind in unambiguous using lexemes
`ResourceCondition`, `RequestCondition` instead confusing `Object.Request`, `Object.Resource`.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-14 12:23:26 +03:00
aarifullin
20baf6e112 [#1108] ape: Update policy-engine version for listing by iteration 
* Update go.mod with a new version of policy-engine pacakge.
* Adapt SwitchRPCGuardedActor to ContractStorage interface.
* Fix `frostfs-adm` util.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-14 12:17:56 +03:00
Anoke
74135776c7 [#1067] adm: Fix panic on negative value 
Signed-off-by: Anoke <rustamgta1011@gmail.com>
 2024-05-13 16:44:04 +00:00
Dmitrii Stepanov
0144117cc9 [#1125] objectSvc: Add EC header APE check 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-08 16:25:55 +03:00
Dmitrii Stepanov
368218f0cc [#1120] cli: Edit object nodes output 
Print detailed information only.
Allow to output to JSON.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-08 15:23:58 +03:00
Dmitrii Stepanov
a45b548a6f [#1120] cli: Add explain to object nodes 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-08 15:23:58 +03:00
Dmitrii Stepanov
654384990c [#1120] cli: Fix object nodes for linking objects 
Do not use linking objects to get placement for complex object.
Linking objects should be stored on all container nodes, also they are not required.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-08 15:23:57 +03:00
Dmitrii Stepanov
ada1b9f737 [#1120] objectSvc: Fix EC put placement 
Use parent object ID to compute placement.
Fix too many copies saving.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-08 15:23:57 +03:00
Dmitrii Stepanov
5c730de96e [#1120] cli: Add EC support to object nodes command 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2024-05-08 15:23:57 +03:00
Alexander Chuprov
854200a874 [#1115] node: Remove unused const 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-05-07 19:11:21 +03:00
Anton Nikiforov
fe2c1c926f [#1112] node: Fix race warning for GetObjectAndWritePayload 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-07 14:47:21 +03:00
Anton Nikiforov
3e782527b8 [#1112] node: Add test for Range request for EC object 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-07 14:47:21 +03:00
Anton Nikiforov
21a490da8f [#1112] Fix issue from gofumpt 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-07 14:47:21 +03:00
Evgenii Stratonikov
93c0ccad4f [#1077] objectsvc: Fix possible panic in GetRange() 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-07 14:47:21 +03:00
Anton Nikiforov
00b2b77b26 [#1112] node: Implement Range\RangeHash requests for EC object 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-05-07 14:47:21 +03:00
Evgenii Stratonikov
a23d53b2d4 [#1118] .forgejo: Add pre-commit action 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-07 11:36:11 +00:00
Evgenii Stratonikov
fc7b07f314 [#1118] docs: Fix pre-commit warnings 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-07 11:36:11 +00:00
Airat Arifullin
9cc51f86b7 [#1117] node: Introduce cache for frostfsid contract client 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-07 10:01:21 +00:00
aarifullin
b60a51b862 [#1117] ape: Introduce FormFrostfsIDRequestProperties method 
* `FormFrostfsIDRequestProperties` gets user claim tags and group id and sets them
  as ape request properties.
* Make tree, container and object service use the method.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-07 10:01:21 +00:00
aarifullin
6c76c9b457 [#1117] core: Introduce SubjectProvider interface for FrostfsID 
* Make tree, object and container services use SubjectProvider interface.
* Fix unit-tests.

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-07 10:01:21 +00:00
aarifullin
45f4e6939d [#1117] morph: Make frostfsid client provide GetSubjectExtended method 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2024-05-07 10:01:21 +00:00
Ekaterina Lebedeva
e07869a8cf [#1100] Remove unused fields 
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
 2024-05-06 10:14:36 +03:00
Alexander Chuprov
ec2873caa7 [#1116] node: Fix writecache metrics 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-05-02 20:03:33 +03:00
Evgenii Stratonikov
71789676d5 [#1114] aclsvc: Add tests for request ownership 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-05-02 11:57:39 +03:00
Alexander Chuprov
c9efaa5819 [#966] node: Add path of the write_cache to metric labels 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-05-02 06:46:46 +00:00
Alexander Chuprov
4730ecfdb8 [#966] node: Refactor WriteCacheMetrics interface 
Grouping common fields of methods will enhance the readability of the interface.

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2024-05-02 06:46:46 +00:00
Evgenii Stratonikov
52bebe9452 [#1110] node: Use single handler for new epoch event 
Bootstrap logic depends on the netmap status, which in turn depends on
the node info. Updating them in a single thread makes things more
predictable.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-27 15:17:45 +03:00
Evgenii Stratonikov
4b514f5ba0 [#1110] node: Log maintenance stop only if it was enabled 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-27 15:17:45 +03:00
Evgenii Stratonikov
c8e2ca2ab4 [#1110] node: Rename handleLocalNodeInfo() 
It is used in "handler" only once, what we really do is set the
variable. And we have another "local" node info in `cfgNodeInfo`, this
one is not really local (node info), more like (local node) info, so use
setContractNodeInfo to distinguish it from the local view on the node
info.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-27 15:17:45 +03:00
Evgenii Stratonikov
3b9d12b11c [#1110] node: Fix comment about nodeInfo type 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-27 15:17:45 +03:00
Anton Nikiforov
411a8d0245 [#1004] blobovnicza: Use TTL for blobovnicza tree cache 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-26 19:54:29 +03:00
Anton Nikiforov
112a7c690f [#1103] node: Implement Get\Head requests for EC object 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-24 18:15:53 +03:00
Anton Nikiforov
167c52a1a9 [#1103] node: Reduce amount of lines for method StorageEngine.head 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-24 16:31:04 +03:00
Anton Nikiforov
700e891b85 [#1103] Fix end of file and trim trailing whitespace 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-24 16:31:04 +03:00
Anton Nikiforov
1f02ac2566 [#1103] pre-commit: Exclude *.svg 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2024-04-24 16:31:04 +03:00
Evgenii Stratonikov
7bc3003803 [#1104] docs: Add bearer token description to auth doc 
This is about authentication only and eACL is deprecated, so only
mention `allow_impersonate` flag.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-23 14:14:17 +03:00
Evgenii Stratonikov
6d4583f5de [#1097] docs: Describe authentication mechanisms 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2024-04-16 15:56:06 +00:00