TrueCloudLab
/
frostfs-node
18
1
Fork 20
Code Issues 88 Pull Requests 7 Actions Releases Activity
4,178 Commits
7 Branches
20 Tags
47 MiB
Commit Graph

145 Commits (v0.37.0-rc.1)

Author SHA1 Message Date
Dmitrii Stepanov 55b82e744b [#529] objectcore: Use common sender classifier
DCO action / DCO (pull_request) Successful in 2m19s Details
Vulncheck / Vulncheck (pull_request) Successful in 3m5s Details
Build / Build Components (1.21) (pull_request) Successful in 4m8s Details
Build / Build Components (1.20) (pull_request) Successful in 4m24s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 4m57s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m43s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m2s Details
Tests and linters / Lint (pull_request) Successful in 5m21s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m17s Details 
Use common sender classifier for ACL service and format validator.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-29 10:33:06 +03:00
Dmitrii Stepanov ae81d6660a [#529] objectcore: Fix object content validation 
There are old objects where the owner of the object
may not match the one who issued the token.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-08-29 10:33:06 +03:00
Evgenii Stratonikov c9e3c9956e [#643] objsvc/put: Unify extraBroadcastEnabled usage
DCO action / DCO (pull_request) Successful in 1m26s Details
Vulncheck / Vulncheck (pull_request) Successful in 2m44s Details
Build / Build Components (1.20) (pull_request) Successful in 3m26s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m51s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m4s Details
Tests and linters / Tests (1.21) (pull_request) Successful in 5m46s Details
Tests and linters / Tests with -race (pull_request) Successful in 6m23s Details
Build / Build Components (1.21) (pull_request) Successful in 13m1s Details
Tests and linters / Lint (pull_request) Successful in 19m46s Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-24 11:03:17 +03:00
Evgenii Stratonikov facd3b2c4b [#643] objsvc/put: Unify placement iterators 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-24 11:03:17 +03:00
Evgenii Stratonikov 3fcf56f2fb [#643] objsvc/put: Copy config to `distributedTarget` 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-08-24 11:03:17 +03:00
Alexander Chuprov 21800e9fcc [#162] core: Move literals to constants 
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-08-11 15:48:42 +03:00
Anton Nikiforov 8d589314b5 [#560] node: Fix `Put` in multi REP with intersecting sets of nodes 
Once the node was processed it skipped, at the step of forming
result in case when all nodes skipped, because processed for
previous REP, service mark the whole request as incomplete.

Example of policies which are unblocked:
- REP 1 REP 1 CBF 1
- REP 4 IN X REP 4 IN Y
  CBF 4
  SELECT 2 FROM FX AS X SELECT 2 FROM FY AS Y
  FILTER Country EQ Russia OR Country EQ Sweden OR Country EQ Finland AS FY
  FILTER Price GE 0 AS FX

Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
 2023-08-08 10:22:53 +00:00
Evgenii Stratonikov 05ac9e3637 [#547] objectsvc: Work with `traversal` struct from a single thread 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-31 15:35:15 +00:00
Dmitrii Stepanov ea32913430 [#543] putsvc: Fix PutSingle implementation 
Add Lock and Delete handlers to local PutSingle.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-28 12:09:41 +00:00
Evgenii Stratonikov 5ff82ff04f [#6] services/object: Simplify local/remote targets
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Tests (1.19) (pull_request) Successful in 14m18s Details
Vulncheck / Vulncheck (pull_request) Failing after 2m23s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m8s Details
Build / Build Components (1.19) (pull_request) Successful in 3m26s Details
Build / Build Components (1.20) (pull_request) Successful in 2m56s Details
Tests and linters / Tests with -race (pull_request) Successful in 21m39s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 22m36s Details
Tests and linters / Lint (pull_request) Successful in 1m39s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
We do not use the return result from Close() and we always execute both
methods in succession. It makes sense to unite them.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-21 18:39:12 +03:00
Dmitrii Stepanov 8966dd8e35 [#463] putsvc: Use PutSingle RPC for remote target 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-18 10:52:12 +00:00
Dmitrii Stepanov ec9b738465 [#294] putsvcv2: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Dmitrii Stepanov 800a685e84 [#294] putsvc: Refactor service constructor 
Pass required deps as args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-12 07:42:10 +00:00
Evgenii Stratonikov 7da284f3e8 [#509] go.mod: Update sdk-go
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Lint (pull_request) Successful in 2m15s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 5m7s Details
Tests and linters / Tests with -race (pull_request) Successful in 4m50s Details
Tests and linters / Staticcheck (pull_request) Successful in 4m10s Details
Build / Build Components (1.19) (pull_request) Successful in 3m47s Details
Build / Build Components (1.20) (pull_request) Successful in 3m30s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 20m39s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-11 17:38:00 +03:00
Evgenii Stratonikov 80481c015c [#509] putsvc: Omit AccessIdentifiers from iteratePlacement() 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-07-11 17:38:00 +03:00
Dmitrii Stepanov a65e26878b [#486] put service: Fix error typo
ci/woodpecker/pr/pre-commit Pipeline failed Details
Build / Build Components (1.19) (pull_request) Successful in 4m12s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 2m52s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 3m49s Details
Tests and linters / Staticcheck (pull_request) Successful in 3m13s Details
Tests and linters / Lint (pull_request) Successful in 10m50s Details
Build / Build Components (1.20) (pull_request) Successful in 11m28s Details
Tests and linters / Tests with -race (pull_request) Successful in 7m54s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-10 15:49:21 +03:00
Dmitrii Stepanov fcbf90d31b [#486] node: Add PutSingle implemetation 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-10 15:49:21 +03:00
Dmitrii Stepanov 7b76527759 [#486] node: Add PutSingle wrappers 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-07-10 15:49:21 +03:00
Alexander Chuprov 033eaf77e1 [#496] node: Fix linter importas
Build / Build Components (1.20) (pull_request) Successful in 3m52s Details
Build / Build Components (1.19) (pull_request) Successful in 4m1s Details
ci/woodpecker/pr/pre-commit Pipeline was successful Details
Tests and linters / Tests with -race (pull_request) Successful in 5m36s Details
Tests and linters / Tests (1.20) (pull_request) Successful in 5m55s Details
Tests and linters / Lint (pull_request) Successful in 14m40s Details
Tests and linters / Tests (1.19) (pull_request) Successful in 15m29s Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Standardize the alias of the
import frostfs-sdk-go/object as objectSDK.

Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
 2023-07-06 15:36:41 +03:00
Evgenii Stratonikov 785d81a68a [#460] services/object: Reduce `distibutedTarget` memory footprint 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-06-21 17:07:50 +03:00
Dmitrii Stepanov 0400153b7d [#424] object: Do not store large slices in pool 
Dynamically growing an unbounded buffers can cause a large
amount of memory to be pinned and never be freed.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-13 07:19:55 +00:00
Dmitrii Stepanov 74578052f9 [#412] node: Replace tracing package 
Use observability module.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-06-01 13:23:11 +00:00
Evgenii Stratonikov b2ffd7df53 [#291] object: Use PayloadSizeLimiter from SDK
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-05 19:07:06 +03:00
Evgenii Stratonikov 35ea207df6 [#291] object: Split validating target in two 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-05-05 19:07:06 +03:00
Pavel Karpy ee58b390bb [#221] node: Allow using vector `copies_number`
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Also, take into account that value in general (it was not used before at
all).

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-05-05 16:07:13 +00:00
Dmitrii Stepanov 3d43b0f7f9 [#265] node: Fix after SDK & API-Go version up
ci/woodpecker/pr/pre-commit Pipeline was successful Details
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-18 12:09:19 +03:00
Dmitrii Stepanov d62c6e4ce6 [#242] node: Add tracing spans 
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-14 10:25:53 +00:00
Dmitrii Stepanov 200fc8b882 [#242] put: Pass context to relay function 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-14 10:25:53 +00:00
Evgenii Stratonikov 0e31c12e63 [#240] logs: Move log messages to constants 
Drop duplicate entities.
Format entities.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-04-14 05:06:09 +00:00
Dmitrii Stepanov 2ed9fd3f94 [#223] objectsvc: Refactor request parameters 
Resolve containedctx linter for commonPrm.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-10 14:16:34 +03:00
Dmitrii Stepanov 23575e1ac0 [#210] policier: Resolve contextcheck linter 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-05 14:55:52 +03:00
Dmitrii Stepanov ece6c820e7 [#199] putsvc: Refactor streamer initialization
ci/woodpecker/push/pre-commit Pipeline was successful Details 
Resolve funlen linter for initTarget method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-03 15:58:11 +00:00
Dmitrii Stepanov 27bdddc48f [#199] putsvc: Refactor put object 
Resolve containedctx linter for streamer and remote target

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-03 15:58:11 +00:00
Dmitrii Stepanov cecea8053a [#199] putsvc: Refactor streamer pool 
Resolve staticcheck linter for putBytesPool

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-03 15:58:11 +00:00
Dmitrii Stepanov 14d894178e [#199] putsvc: Refactor placement iterator 
Resolve funlen linter for iteratePlacement method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-04-03 15:58:11 +00:00
Dmitrii Stepanov 97c36ed3ec [#148] linter: Add funlen linter 
Long functions are hard to understand and source of errors

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:54:41 +03:00
Dmitrii Stepanov cc8ff015b4 [#148] linter: Add containedctx linter 
Context has to be passed as an argument: https://pkg.go.dev/context

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
 2023-03-21 09:52:39 +03:00
Pavel Karpy 64bde68fb9 [#67] node: Accept expired locked objects 
Allow replication of any (expired too) locked object. Information about
object locking is considered to be presented on the _container nodes_.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-03-16 16:22:19 +03:00
Evgenii Stratonikov 22be532cbd object/put: Persist session token till the end of a session 
Previously a token could've expired in the middle of an object.PUT
stream, leading to upload being interrupted. This is bad, because user
doesn't always now what is the right values for the session token
lifetime. More than that, setting it to a very high value will
eventually blow up the session token database.

In this commit we read the session token once and reuse it for the whole
stream duration.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2023-03-16 06:45:50 +00:00
Alexey Vanin 20de74a505 Rename package name 
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
 2023-03-07 16:38:26 +03:00
Alejandro Lopez 73bb590cb1 [#64] node: Use pool_size_local and separate pool for local puts 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-02-22 13:43:19 +03:00
Alejandro Lopez cb5468abb8 [#66] node: Replace interface{} with any 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
 2023-02-21 16:47:07 +03:00
Pavel Karpy 901d62567d [#57] node: Broadcast link objects 
It boosts object assembling by an _average_ container node.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2023-02-17 11:58:27 +03:00
Evgenii Stratonikov 6f5edac730 [#2164] network/cache: Do not reconnect to failed clients immediately 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Evgenii Stratonikov 04b5ec759b [#2139] object/put: Use `sync.Pool` for temporary payloads 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
 2022-12-30 11:07:35 +03:00
Pavel Karpy 923f84722a Move to frostfs-node 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
 2022-12-28 15:04:29 +03:00
Pavel Karpy 51963abce7 [#1972] node: Fix errors comments in the Put service 
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-30 16:58:52 +03:00
Evgenii Stratonikov 2522d924b9 [#2037] services/object: Fix concurrent map writes in traverser 
```
fatal error: concurrent map writes

goroutine 4337 [running]:
github.com/nspcc-dev/neofs-node/pkg/services/object/put.(*traversal).submitProcessed(...)
        github.com/nspcc-dev/neofs-node/pkg/services/object/put/distributed.go:78
github.com/nspcc-dev/neofs-node/pkg/services/object/put.(*distributedTarget).iteratePlacement.func1()
        github.com/nspcc-dev/neofs-node/pkg/services/object/put/distributed.go:198 +0x265
github.com/panjf2000/ants/v2.(*goWorker).run.func1()
        github.com/panjf2000/ants/v2@v2.4.0/worker.go:68 +0x97
created by github.com/panjf2000/ants/v2.(*goWorker).run
        github.com/panjf2000/ants/v2@v2.4.0/worker.go:48 +0x65
```

Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-11-19 11:01:04 +03:00
Pavel Karpy aab398f4f5 [#1972] node: Do not save objects if node not in a container 
Do not use node's local storage if it is clear that an object will be
removed anyway as a redundant. It requires moving the changing local storage
logic from the validation step to the local target implementation.
It allows performing any relations checks (e.g. object locking) only if a
node is considered as a valid container member and is expected to store
(stored previously) all the helper objects (e.g. `LOCK`, `TOMBSTONE`, etc).

Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
 2022-11-19 11:01:04 +03:00
Evgenii Stratonikov d772e35aba [#1910] .golangci.yml: Add `godot` linker 
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
 2022-10-18 15:08:26 +03:00