We already provide the pool and this argument is used only for
preallocation. No functional changes.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
DCO action / DCO (pull_request) Successful in 2m55sDetails
Vulncheck / Vulncheck (pull_request) Successful in 3m22sDetails
Tests and linters / Staticcheck (pull_request) Successful in 4m10sDetails
Build / Build Components (1.20) (pull_request) Successful in 4m58sDetails
Build / Build Components (1.21) (pull_request) Successful in 4m54sDetails
Tests and linters / Lint (pull_request) Successful in 5m47sDetails
Tests and linters / Tests (1.20) (pull_request) Successful in 6m45sDetails
Tests and linters / Tests (1.21) (pull_request) Successful in 6m58sDetails
Tests and linters / Tests with -race (pull_request) Successful in 7m42sDetails
processObject() returns 3 types of errors: container not found errors,
could not get container error and placement vector building error. Every
error will occur for all objects in container simultaneously, so we can
log each error once and safely ignore the rest.
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
* Introduce common method EverExisted
* Define DeletionInfo for struct that must implement Source
* Refactor tree srv
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
ci/woodpecker/push/pre-commit Pipeline was successfulDetails
Current implementation has some quirks. For example,
using only half of object.put.pool_size_remote threads
tells replicator that is node is 50% loaded,
but in reality we could be putting lot's of big objects.
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Previously, we can continue to return `EndOfListing` infinitely.
Reflect iterator reuse via Rewind() method.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
We use cache to avoid policing the same object multiple times in a short
time span (< 30 seconds). If we have 200_000 objects in a blobstor, it is a bit useless
-- if it takes 1 second to process an object and we have `replicator.pool_size: 20`
in config, the next iteration will happen in 10_000 second which is much
larger than 30 second. However we still consume a lot of memory, so it
makes sense to use saner default.
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
```
2022/11/15 08:40:56 worker exits from a panic: runtime error: index out of range [0] with length 0
2022/11/15 08:40:56 worker exits from panic: goroutine 1188 [running]:
github.com/panjf2000/ants/v2.(*goWorker).run.func1.1()
github.com/panjf2000/ants/v2@v2.4.0/worker.go:58 +0x10c
panic({0x1042b60, 0xc0015ae018})
runtime/panic.go:1038 +0x215
github.com/nspcc-dev/neofs-node/pkg/services/policer.(*Policer).shardPolicyWorker.func1()
github.com/nspcc-dev/neofs-node/pkg/services/policer/process.go:65 +0x366
github.com/panjf2000/ants/v2.(*goWorker).run.func1()
github.com/panjf2000/ants/v2@v2.4.0/worker.go:68 +0x97
created by github.com/panjf2000/ants/v2.(*goWorker).run
github.com/panjf2000/ants/v2@v2.4.0/worker.go:48 +0x68
```
Signed-off-by: Evgenii Stratonikov <evgeniy@morphbits.ru>
Includes extending listing methods in the Storage Engine with object types.
It allows tuning replication/policer algorithms: container nodes do
not remove `LOCK` objects as redundant and try to fulfill `LOCK` placement
on the ohter container nodes.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Make it store its internal `zap.Logger`'s level. Also, make all the
components to accept internal `logger.Logger` instead of `zap.Logger`; it
will simplify future refactor.
Signed-off-by: Pavel Karpy <carpawell@nspcc.ru>
Node response with `NODE_UNDER_MAINTENANCE` status signals that the node
was switched to maintenance mode. There is a delay between the actual
switch and the reflection in the network map of up to one epoch. To
speed up the reaction to the maintenance, it is required to recognize
such node responses in the Policer.
Make `Policer.processNodes` to exclude elements with shortage decreasing
on `NODE_UNDER_MAINTENANCE` status response.
Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
Evgenii Stratonikov
Ekaterina Lebedeva
Anton Nikiforov
Dmitrii Stepanov
Airat Arifullin
Alejandro Lopez
Alexander Chuprov
Alexey Vanin
Stanislav Bogatyrev
Pavel Karpy
Evgenii Stratonikov
Pavel Karpy
Leonard Lyubich