TrueCloudLab/frostfs-node
18
1
Fork 27
Code Issues 93 Pull requests 6 Projects Releases 1 Activity Actions 1
3629 commits 8 branches 70 tags 46 MiB
Commit graph

429 commits

Author SHA1 Message Date
Dmitrii Stepanov
23575e1ac0 [#210] policier: Resolve contextcheck linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:55:52 +03:00
Dmitrii Stepanov
d85703a963 [#208] searchsvc: Refactor request forwarding
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Resolve funlen & gocognit linters for toPrm method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 16:24:55 +03:00
Dmitrii Stepanov
0b38419fbf [#208] searchsvc: Resolve context linters 
Resolve containedctx and contextcheck linters.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 16:24:50 +03:00
Dmitrii Stepanov
5f2a1531fe [#208] deletesvc: Resolve containedctx linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 16:24:40 +03:00
Dmitrii Stepanov
4941926c9d [#207] aclsvc: Drop outdated tag
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 13:22:14 +00:00
Dmitrii Stepanov
585415fa92 [#207] aclsvc: Refactor send checker 
Resolve funlen linter for putStreamBasicChecker.Send method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 13:22:14 +00:00
Dmitrii Stepanov
9ef790f782 [#207] aclsvc: Refactor object headers read 
Resolve funlen linter for readObjectHeaders method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 13:22:13 +00:00
Dmitrii Stepanov
cd33a57f44 [#207] aclsvc: Refactor EACL check 
Resolve funlen linter for CheckEACL method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-04 13:22:13 +00:00
Dmitrii Stepanov
ece6c820e7 [#199] putsvc: Refactor streamer initialization
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Resolve funlen linter for initTarget method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-03 15:58:11 +00:00
Dmitrii Stepanov
27bdddc48f [#199] putsvc: Refactor put object 
Resolve containedctx linter for streamer and remote target

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-03 15:58:11 +00:00
Dmitrii Stepanov
cecea8053a [#199] putsvc: Refactor streamer pool 
Resolve staticcheck linter for putBytesPool

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-03 15:58:11 +00:00
Dmitrii Stepanov
14d894178e [#199] putsvc: Refactor placement iterator 
Resolve funlen linter for iteratePlacement method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-03 15:58:11 +00:00
Airat Arifullin
221203beeb [#180] node: Refactor panics in unit test
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
* Replace panics in unit tests by require.NoError and t.Fatalf

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
 2023-03-29 12:39:07 +03:00
Dmitrii Stepanov
49234b915e [#155] search-service: Fix search with ST
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Search should return only objects allowed in static session

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-22 07:21:09 +00:00
Dmitrii Stepanov
1637a3edce [#155] search-service: Add search with ST test 
In case of session token (ST) with object IDs search should
return only objects allowed in static session

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-22 07:21:09 +00:00
Evgenii Stratonikov
cbc2efb1d6 [#156] object/get: Make toHeadPrm() pass gocognit
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-22 07:14:18 +00:00
Dmitrii Stepanov
44b86bac5a [#148] linter: Add contextcheck linter
Some checks failed
ci/woodpecker/push/full-pre-commit Pipeline failed
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:54:41 +03:00
Dmitrii Stepanov
481a1ca6f3 [#148] linter: Add gocognit linter 
Code with high cognitive complexity is hard intuitively to understand

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:54:41 +03:00
Dmitrii Stepanov
97c36ed3ec [#148] linter: Add funlen linter 
Long functions are hard to understand and source of errors

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:54:41 +03:00
Dmitrii Stepanov
cc8ff015b4 [#148] linter: Add containedctx linter 
Context has to be passed as an argument: https://pkg.go.dev/context

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:52:39 +03:00
Pavel Karpy
abd21f8099 [#136] Revert "[#2260] services/object: Do not assemble object with TTL=1" 
This reverts commit 2567f8020e. It assumes
that assembling logic could break some failover scenarios if request
forwarding is done. However, it also breaks requesting big objects via a
non-container node with TTL=2. Failover has been rechecked without that
commit and no problems were found. Any (if found) other bugs related to
the forwarding and object assembling must be solved more carefully.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-20 10:52:18 +03:00
Pavel Karpy
64bde68fb9 [#67] node: Accept expired locked objects 
Allow replication of any (expired too) locked object. Information about
object locking is considered to be presented on the _container nodes_.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-16 16:22:19 +03:00
Evgenii Stratonikov
22be532cbd object/put: Persist session token till the end of a session 
Previously a token could've expired in the middle of an object.PUT
stream, leading to upload being interrupted. This is bad, because user
doesn't always now what is the right values for the session token
lifetime. More than that, setting it to a very high value will
eventually blow up the session token database.

In this commit we read the session token once and reuse it for the whole
stream duration.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-16 06:45:50 +00:00
Dmitrii Stepanov
ac0a278a05 [#85] get-service: Drop unused assemble flag 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-15 09:19:45 +03:00
Dmitrii Stepanov
b8e93d4c08 [#85] get-service: Use assembler to assemble LOB 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-15 09:19:45 +03:00
Dmitrii Stepanov
07de839f18 [#85] get-service: Fix corrupted chain logic 
Should return an error in case of a broken LOB reference chain.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-15 09:19:45 +03:00
Dmitrii Stepanov
2886b1581b [#85] get-service: Add unit tests 
Add unit tests to cover all assemble statements

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-15 09:19:45 +03:00
Dmitrii Stepanov
8b9e40a848 [#85] get-service: Add assembler 
Extract assemble logic to assembler

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-15 09:19:45 +03:00
Alex Vanin
20de74a505 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 16:38:26 +03:00
Alejandro Lopez
73bb590cb1 [#64] node: Use pool_size_local and separate pool for local puts 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-02-22 13:43:19 +03:00
Alejandro Lopez
cb5468abb8 [#66] node: Replace interface{} with any 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-02-21 16:47:07 +03:00
Anton Nikiforov
22f3c7d080 [#1868] Reload config for pprof and metrics on SIGHUP 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-02-20 13:53:27 +03:00
Evgenii Stratonikov
2567f8020e [#2260] services/object: Do not assemble object with TTL=1 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-02-20 13:53:27 +03:00
Pavel Karpy
901d62567d [#57] node: Broadcast link objects 
It boosts object assembling by an _average_ container node.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-02-17 11:58:27 +03:00
Stanislav Bogatyrev
cb016d53a6 [#1] Fix comments and error messages 
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
 2023-02-06 17:41:14 +03:00
Evgenii Stratonikov
0d8366f475 [#2207] object/acl: Return status error for expired session token 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov
554b85411f [#2190] services/object: Log service error with INFO level 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-01-25 15:31:47 +03:00
Evgenii Stratonikov
6f5edac730 [#2164] network/cache: Do not reconnect to failed clients immediately 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Pavel Karpy
6a4e5e6f0a [#2144] node: Try node's private key if dynamic token fetching failed 
`GETRANGEHASH` request spawns `GETRANGE` requests if an object could not be
found locally. If the original request contains session, it can be static
and, therefore, fetching session key can not be performed successfully.
As the best effort a node could request object's range with its own key.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-30 11:07:35 +03:00
Pavel Karpy
86a4fba571 [#2144] node: Clarify KeyStorage.GetKey method 
Actualize the doc, fix API status error return.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov
04b5ec759b [#2139] object/put: Use sync.Pool for temporary payloads 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Pavel Karpy
b413094704 [#2095] node: Fix collecting child objects 
Stop child objects collection if the last returned object (the most "left"
object in the collected chain) starts exactly from the `GETRANGE`'s `from`
value.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-12-30 11:07:35 +03:00
Pavel Karpy
350eecfa13 [#2095] node: Do not allow GETRANGE requests with zero length 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-12-30 11:07:35 +03:00
Pavel Karpy
923f84722a Move to frostfs-node 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-28 15:04:29 +03:00
Pavel Karpy
d54022eacc [#2047] node: Do not send chunk twice on request forwarding 
That could happen if a node forwards request to a node that closed the
connection during the original object stream.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-12-02 11:27:48 +03:00
Evgenii Stratonikov
bd25db5d4a [#1984] metrics: Use separate metrics for success/failed requests 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-12-01 14:18:10 +03:00
Evgenii Stratonikov
e21c472dc7 [#1984] services/object: Increase put_req_count after the request is processed 
As it is specified in metrics description.

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-12-01 14:18:10 +03:00
Pavel Karpy
51963abce7 [#1972] node: Fix errors comments in the Put service 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-30 16:58:52 +03:00
Pavel Karpy
d5a14041e0 [#2040] node: Do not attach tokens in the assembly process 
A container node is expected to have full "get" access to assemble the
object.
A non-container node is expected to forward any request to a container node.
Any token is expected to be issued for an original request sender not for a
node so any new request is invalid by design with that token.

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00
Pavel Karpy
fd61bdadcb [#2040] node: Attach original meta to the spawned requests 
Do not lose meta information of the original requests: cache session and
bearer tokens of the original request b/w a new generated ones. Middle
request wrappers should not contain any meta information, since it is
useless (e.g. ACL service checks only the original tokens).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00