TrueCloudLab/frostfs-node
18
1
Fork 27
Code Issues 94 Pull requests 3 Projects Releases 1 Activity Actions

Add block and mutex profiles #39

Merged
carpawell merged 1 commit from carpawell/optional-profiles into master 2023-07-26 21:07:56 +00:00
Conversation 3 Commits 1 Files changed 1 +1 -1
carpawell commented 2023-02-01 07:24:19 +00:00 (Migrated from github.com)
Copy link

Closes #22.

@alexvanin, @KirillovDenis, @fyrchik, @acid-ant, lets discuss config format, values (int or bool?) and if the IR shoud have it too?

Also, i think that should be merged after some adoption of the https://github.com/nspcc-dev/neofs-node/pull/2094.

Closes #22. @alexvanin, @KirillovDenis, @fyrchik, @acid-ant, lets discuss config format, values (`int` or `bool`?) and if the IR shoud have it too? Also, i think that should be merged after some adoption of the https://github.com/nspcc-dev/neofs-node/pull/2094.
alexvanin (Migrated from github.com) reviewed 2023-02-01 07:24:19 +00:00
KirillovDenis (Migrated from github.com) reviewed 2023-02-01 07:24:19 +00:00
fyrchik (Migrated from github.com) reviewed 2023-02-01 07:24:19 +00:00
acid-ant (Migrated from github.com) reviewed 2023-02-01 07:24:19 +00:00
alexvanin commented 2023-02-03 14:05:53 +00:00 (Migrated from github.com)
Copy link

if the IR shoud have it too

I would like to adopt the same scheme across all apps, including IR, gateways, etc.

I am perfectly fine with proposed format.

> if the IR shoud have it too I would like to adopt the same scheme across all apps, including IR, gateways, etc. I am perfectly fine with proposed format.
fyrchik commented 2023-02-06 18:12:31 +00:00 (Migrated from github.com)
Copy link

I believe IR should have it too, it blocks a lot.
I would make a section profile.debug for this (with values block_rate, mutex_rate). This way it is obvious, that this is not intended to be used in production.

I believe IR should have it too, it blocks a lot. I would make a section `profile.debug` for this (with values `block_rate`, `mutex_rate`). This way it is obvious, that this is not intended to be used in production.
fyrchik changed title from Feat/Add optional profiles to WIP: Feat/Add optional profiles 2023-03-09 15:00:12 +00:00
carpawell referenced this pull request from a commit 2023-04-19 17:02:23 +00:00
[#39] node: Add optional profilers
carpawell referenced this pull request from a commit 2023-04-19 17:02:23 +00:00
[#39] ir: Add optional profilers
carpawell changed title from WIP: Feat/Add optional profiles to Feat/Add optional profiles 2023-04-19 17:02:25 +00:00
carpawell added 311 commits 2023-04-19 17:02:46 +00:00 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Changing env prefix and corresponding example config files.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
- Update sample configs to match unit tests
- Remove Docker container for N3 testnet
  Will return with updates when FrostFS enters N3

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
```
name                  old time/op    new time/op    delta
_addressFromString-8    1.25µs ±30%    1.02µs ± 6%  -18.49%  (p=0.000 n=9+9)

name                  old alloc/op   new alloc/op   delta
_addressFromString-8      352B ± 0%      256B ± 0%  -27.27%  (p=0.000 n=9+10)

name                  old allocs/op  new allocs/op  delta
_addressFromString-8      6.00 ± 0%      4.00 ± 0%  -33.33%  (p=0.000
n=10+10)
```

Also, assure compiler that `s` doesn't escape:
Before this commit:
```
./fstree.go:74:24: leaking param: s
./fstree.go:90:6: moved to heap: addr
```

After this commit:
```
./fstree.go:74:24: s does not escape
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
neo-go actor API uses `getVersion` call which returned incorrect magic.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Single node is used in dev-env, worth testing.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Aleksey Pastukhov <a.pastukhov@yadro.com>
It boosts object assembling by an _average_ container node.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Roman Khimov <roman@nspcc.ru>
How could you forget adding it?

Signed-off-by: Roman Khimov <roman@nspcc.ru>
"Object is expired" means that object is presented in `meta` but it is not
`ObjectNotFound` error. Previous implementation made `shard` search for an
object without `meta` which was an error.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
If we had lots of domains in one zone, `dump-hashes` for all others
can miss some domains, because we need to restrict ourselves with _some_
number.
In this commit we use neo-go sessions by default, with a proper
failback to in-script iterator unwrapping.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
To achieve high performance we must choose proper values for both
batch size and delay. For user operations we want to set low delay.
However it would prevent tree synchronization operations to form big
enough batches. For these operations, batching gives the most benefit
not only in terms of on-CPU execution cost, but also by speeding up
transaction persist (`fsync`).
In this commit we try merging batches that are already
_triggered_, but not yet _started to execute_. This way we can still
query batches for execution after the provided delay while also allowing
multiple formed batches to execute faster.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
LRU `Peek`/`Contains` take LRU mutex _inside_ of a `View` transaction.
`View` transaction itself takes `mmapLock` [1], which is lifted after tx
finishes (in `tx.Commit()` -> `tx.close()` -> `tx.db.removeTx`)

When we evict items from LRU cache mutex order is different:
first we take LRU mutex and then execute `Batch` which _does_ take
`mmapLock` in case we need to remap. Thus the deadlock.

[1] 8f4a7e1f92/db.go (L708)

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
1. Both could initialize shards in parallel.
2. Both should close shards after an error.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Currently, DELETE service sets tombstone expiration epoch to
`current epoch + 5`. This works less than ideal in private networks
where an epoch can be e.g. 10 minutes. In this case, after a node is
unavailable for more than 1 hour, already deleted objects have a chance
to reappear.

After this commit tombstone lifetime can be configured.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Previously, node could get an "infinite" small object: it could be expired
and thus could not be flushed (update its storage ID) to metabase => could
not be marked as flushed => node never removes such object and repeat all
the cycle one more time. If object exists and is not marked with GC (meta
returns `ErrObjectIsExpired`, not `ObjectNotFound` and not
`ObjectAlreadyRemoved`), its ID is safe to update _in the same_ bbolt
transaction.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Subscribing without async listening could lead to a dead-lock in the
`neo-go` client.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Missing `ReportError` method did not allow casing multi-client interface to
`errorReporter` interface and dropping broken connections.
`replicationClient` embeds that interface, and it is widely used across
node's code. Embedded interface does not allow casting its parent structure
to `errorReporter` and breaks multi client error reporting logic.
Multi-client scheme is extremely hard to maintain, it makes unpredictable
casts and does not allow tracking code flow, so it will be refactored in the
future anyway.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
In case we have many small objects in the write-cache, `indices` should
not be reused between iterations.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Timeouts on client side should node affect inter-node communication.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
The problem is that accidental timeout errors can make us to ignore
other nodes for some time. The primary purpose of the whole ignore
mechanism is not to degrade in case of failover. For this case,
closing connection and limiting the amount of dials is enough.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Currently, under a mixed load one failed PUT can lead to closing
connection for all concurrent GETs. For PUT it does no harm: we have
many other nodes to choose from. For GET we are limited by `REP N`
factor, so in case of failover we can close the connection with the only
node posessing an object, which leads to failing the whole operation.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
If an object is missing in a `meta`, shard should not look for it in
a `blobstor`.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
Create contract wallet only by init and update-config command.

Close #63

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Compute shard hash only once

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Update HRW lib and use typed HRW methods to sort shards and nodes

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Due to source code relocation from GitHub.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
It will prevent test fails with `-race` flag on components that have
background processes and make some actions on test framework.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
If `processNodes` exits earlier for some reason, `needLocalCopy` could
be false.
See https://github.com/nspcc-dev/neofs-node/issues/2267

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
In previous implementation pretty-printer of basic ACL in NeoFS CLI had
mistakes:
 * F-bit was set to `Extendable()` property instead of its inversion
 * B-bits were set to `acl.RoleInnerRing` rights

Make `PrettyPrintTableBACL` to correctly render mentioned bits.

Signed-off-by: Leonard Lyubich <ctulhurider@gmail.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Issue templates are still supported by Gitea:
https://docs.gitea.io/en-us/issue-pull-request-templates/ .

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Skip deprecated warning for now, adopting new neo-go API will be done in
another task.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Remember the last synchronized height and use it after service restart.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Add make targets to simplify pre-commit setup for individual developers.

Signed-off-by: Stanislav Bogatyrev <realloc@realloc.spb.ru>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Extract assemble logic to assembler

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Add unit tests to cover all assemble statements

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Should return an error in case of a broken LOB reference chain.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Original shellcheck action requires Docker to run and it's not always
available, especially inside Docker containers. Replacing it with python
wrapper to simplify usage with Docker-based CI systems like Drone and
WoodpeckerCI.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
Previously a token could've expired in the middle of an object.PUT
stream, leading to upload being interrupted. This is bad, because user
doesn't always now what is the right values for the session token
lifetime. More than that, setting it to a very high value will
eventually blow up the session token database.

In this commit we read the session token once and reuse it for the whole
stream duration.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
It will allow reusing that method in expiration checks.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Allow replication of any (expired too) locked object. Information about
object locking is considered to be presented on the _container nodes_.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
May make docker builds not so clean.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
This reverts commit 2567f8020e. It assumes
that assembling logic could break some failover scenarios if request
forwarding is done. However, it also breaks requesting big objects via a
non-container node with TTL=2. Failover has been rechecked without that
commit and no problems were found. Any (if found) other bugs related to
the forwarding and object assembling must be solved more carefully.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Artem Tataurov <a.tataurov@yadro.com>
In our policy we mark commits not having a PR/Issue yet with a `[#XX]`
reference to be replaced after PR creation.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Minor changes to see what fails first

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Until #139 is fixed, we can't use root inside Docker container running
CI, but Woodpecker CI can't run non-root containers until they fix
https://github.com/woodpecker-ci/woodpecker/issues/1077, hence we use
temporary kludges with custom image and manual permissions in pipelines.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Context has to be passed as an argument: https://pkg.go.dev/context

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Long functions are hard to understand and source of errors

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Code with high cognitive complexity is hard intuitively to understand

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#148] linter: Add contextcheck linter
Some checks failed
ci/woodpecker/push/full-pre-commit Pipeline failed
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
44b86bac5a 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#152] IR: Process empty basic incomes
Some checks failed
ci/woodpecker/push/full-pre-commit Pipeline failed
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
9cd8f7cea0 
If network is not configured for basic income earnings, do not distribute
GAS by the Alphabet nodes.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Added unit test that verifies that GC deletes expired
locked objects in one epoch.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
GC deletes expired locks and objects sequentially. Expired locks and
objects are now being deleted concurrently in batches. Added a config
parameter that controls the number of concurrent workers and batch size.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#145] docs: Add expired object collector params
Some checks failed
ci/woodpecker/pr/full-pre-commit Pipeline failed
Details
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/full-pre-commit Pipeline failed
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
7a31988a36 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#128] IR: Do not try to emit GAS to nobody
Some checks failed
ci/woodpecker/push/full-pre-commit Pipeline failed
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
9a4f40626c 
Fix sending GAS to an empty extra wallets receivers list. Also, send GAS to
extra wallets even if netmap is empty.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#153] ci: Minor pipeline fixes
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
9aeea0b974 
- We can skip full pre-commit run
- On a very slow agent golangci run may take up to 10 minutes

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
Signed-off-by: Liza <e.chichindaeva@yadro.com>
Add to Possible Solutions
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
ec2c5d45b4 
Signed-off-by: Liza <e.chichindaeva@yadro.com>
Initially it was there to check whether an update is being initiated by
a proper node. It is now obsolete for 2 reasons:
1. Background synchronization fetches all operations from a single node.
2. There are a lot more problems with trust in the tree service, it is
   only used in controlled environments.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#156] object/get: Make toHeadPrm() pass gocognit
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
cbc2efb1d6 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
In case of session token (ST) with object IDs search should
return only objects allowed in static session

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#155] search-service: Fix search with ST
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
49234b915e 
Search should return only objects allowed in static session

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#98] fstree: Do not fail iteration over just removed files
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
da8da1c63a 
A directory is read and files are saved to a local variable. The iteration
over such files may lead to a non-existing files reading due to a normal SN
operation cycle and, therefore, may lead to a returning the OS error to a
caller. Skip just removed (or lost) files as the golang std library does in
similar situations:
5f1a0320b9/src/os/dir_unix.go (L128-L133).

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#158] Fix gitlint run in pre-commit under CI
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
f111704ceb 
Wrong stage was used.
It still may run gitlint against only latest commit in PR.

Signed-off-by: Stanislav Bogatyrev <s.bogatyrev@yadro.com>
[#159] Add handle __SYSTEM__ sys attributes
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
342e571d89 
Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>
[#86] node: Move testing utils to one package
Some checks failed
ci/woodpecker/push/pre-commit Pipeline failed
Details
9808dec591 
Move testing utils from tests in local_object_storage package to
unified testutil package

Signed-off-by: Airat Arifullin <aarifullin@yadro.com>
[#86] node: Fix unit test and linter errors
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
34329d67ff 
Signed-off-by: Airat Arifullin <aarifullin@yadro.com>
This change allows to use `[#xx]` placeholders for issue number.
This change adds go unit test run if .go files were changed.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
[#171] Remove unit tests from pipeline
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
9ffa0d8fea 
Unit tests are included as pre-commit hook. They are
triggered by the change of .go files.

Signed-off-by: Alex Vanin <a.vanin@yadro.com>
It does not use deprecated methods anymore but also adds more code that
removes. Future refactor that will affect more components will optimize
usage of the updated API.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#73] morph: Rename vars that collide with package names
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
f3ff9fd251 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#174] go.mod: Update dependencies
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
a5ece7889d 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#141] Add big object's parts to a lock object's body
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
3c7ed21f74 
That will prevent part/link object from being removed by both an external
`DELETE` call and the object expiration procedure.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Resolve funlen linter for getObject method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for putObject method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#165] ir: Refactor config default init
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
f2e880465e 
Resolve funlen linter for defaultConfiguration function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for restoreContainers function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for deployContractCmd function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for updateContracts method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for newInitializeContext function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for depositNotary function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for init function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for manageSubnetAdmins function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for init method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#161] adm: Refactor storage-config command
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
5bf1ec348f 
Resolve funlen linter for storageConfig function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#172] Use ContainersOf() for container list fetching
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
382ecae96a 
Previously we were limited by ~2048 containers because of
neo-go VM limits.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#176] morph: Resolve funlen linter
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
91717d4b98 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#180] node: Refactor panics in unit test
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
221203beeb 
* Replace panics in unit tests by require.NoError and t.Fatalf

Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
If `ctx.Done()`, return `ctx.Err()` in every function that returns an error.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
We have already had and solved plenty of deposit issues and notary balance
is a really important thing. Deserves to be INFO even before the huge logs
severity refactor, happens on an app start only.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#184] ir: Delete unused config reader param
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
e843e7f090 
Not used since notary environments do endless and non-configurable
deposits.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#139] test: Add test storage implementation
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
341fe1688f 
This aims to reduce the usage of chmod hackery to induce or simulate
OS-related failures.

Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#183] gc: Fix drop expired locked complex objects
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
ab32067152 
Do not delete bucket keys during iteration

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#189] node: Fix bug with sync/atomic package
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
7be5a0fd79 
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
Resolve funlen linter for GlagoliticLetter.String method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for Server.Start method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for New function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for processEmit method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for processStartAudit method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for processAlphabetSync method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for verifySignature method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#185] ir: Resolve containedctx linter
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
8426d25f4b 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx linter for cfg

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for readConfig method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for initCfg function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for shardOpts method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for initContainerService function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx for remoteLoadAnnounceWriter struct

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for initObjectService function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for initNetmapService function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#168] node: Refactor reputation service
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
ed28ce24cd 
Resolve funlen linter for initReputationService function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#181] Update CHANGELOG
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
2bdf7126b8 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#2272] morph: Do not subscribe to events without listening
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
a69c6d1ec9 
It led to a neo-go dead-lock in the `subscriber` component. Subscribing to
notifications is the same RPC as any others, so it could also be blocked
forever if no async listening (reading the notification channel) routine
exists. If a number of subscriptions is big enough (or a caller is lucky
enough) subscribing loop might have not finished subscribing before the
first notification is received and then: subscribing RPC is blocked by
received notification (non)handling and listening notifications routine is
blocked by not finished subscription loop.
That commit starts listening notification channel _before_ any subscription
actions.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Resolve funlen linter for iteratePlacement method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve staticcheck linter for putBytesPool

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx linter for streamer and remote target

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#199] putsvc: Refactor streamer initialization
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
ece6c820e7 
Resolve funlen linter for initTarget method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#195] morph: use blocking unlimited pool for notifications
Some checks failed
ci/woodpecker/push/pre-commit Pipeline failed
Details
0948a280fa 
With non-blocking pool restricted by 10 in capacity, the probability of
dropping events is unexpectedly big. Notifications are an essential part of the FrostFS,
we should not drop anything, especially new epochs.
```
Mar 31 07:07:03 vedi neofs-ir[19164]: 2023-03-31T07:07:03.901Z        debug        subscriber/subscriber.go:154        new notification event from sidechain        {"name": "NewEpoch"}
Mar 31 07:07:03 vedi neofs-ir[19164]: 2023-03-31T07:07:03.901Z        warn        event/listener.go:248        listener worker pool drained        {"chain": "morph", "capacity": 10}
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#204] gc: Fix GC handlers start
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
8e5a0dcf27 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#175] adm: pipeline container iteration
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
49cc23e03c 
Do not accumulate everything in memory.
Also, CLI should be responsive.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#205] innerring: Provide alphabetState param to epochTimer
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
ee7468daa7 
Fix NPE, introduced in f09ee27a.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Resolve funlen linter for Blobovniczas.Put method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for TestIterate function

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for db.put method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for StorageEngine.get method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for StorageEngine.getRange method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen and gocognit linter for StorageEngine.Evacuate method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#188] metabase: Refactor object inhume
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
1f1aed87be 
Resolve funlen linter for db.Inhume method

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for CheckEACL method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for readObjectHeaders method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for putStreamBasicChecker.Send method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#207] aclsvc: Drop outdated tag
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
4941926c9d 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx and contextcheck linters.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#208] searchsvc: Refactor request forwarding
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
d85703a963 
Resolve funlen & gocognit linters for toPrm method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
We already do this for file writing, however directory creation can also
fail.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#211] engine: Unify shard mode checks for tree operations
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
9098d0eec0 
All operations must ensure the shard is not in a degraded mode.
Write operations must also ensure the shard is not in a read-only mode.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx and contextcheck linters.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#210] policier: Refactor nodes processing
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
d6486d172e 
Resolve funlen linter for processNodes method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#116] node: Improve shard/engine construction in tests
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
6f7b6a8813 
* Introduce testEngineWrapper that can be constructed with different options

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
Resolve containedctx for routeCtx.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#217] containersvc: Resolve containedctx linter
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
206458c841 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx and contextcheck linters.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for toHeadPrm method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for toPrm function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for toRangePrm function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Pass context to forwarder direct, without closure.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#193] getsvc: Reduce private key requests
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
c58ab0c369 
Get private key only once for request forwaring.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#116] node: Fix bug with extra generated files in TestReload
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
ab891517de 
* Create testNewEngine in engineWithShards without default opts

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Skip staticcheck for depracated methods. Will be fixed soon.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Do not pass 0 as channel capacity.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Fix shard compare loop.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#203] node: Add staticcheck target and pre-commit
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
9027695371 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#166] node: Parallelize background tree service sync
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
56282edf02 
* Run sync task for nodes in parallel within errgroup worker pool

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for Client.notaryInvoke method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen and gocognit linters for listener.listenLoop method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#219] morph: Refactor notary preparator
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
fe87735073 
Resolve funlen linter for Preparator.Prepare method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx linter for commonCommunicatorPrm type.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx linter. Cancel task by listen cancel.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#222] auditsvc: Refactor PoR audit
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
e2f13d03d7 
Resolve funlen linter for Context.checkStorageGroupPoR method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolved containedctx linters.
Renamed context structs and interfaces to more understandble names.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#212] reputationsvc: Resolve funlen linter
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
c236b54a65 
Resolve funlen linter for Calculator.iterateDaughter method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#228] node: Use uber atomic package instead standard
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
cb172e73a6 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
RemoveDuplicates() removes all duplicate object copies stored on
multiple shards. All shards are processed and the command tries to leave
a copy on the best shard according to HRW.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Doctor RPC performs complex operations on the storage engine.
Currently only duplicate removal is supported.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#191] cli: Add control shards doctor command
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
b689027d57 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Resolve funlen linter for newEngineMetrics function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#213] metrics: Refactor object metrics
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
02831d427b 
Resolve funlen linter for newObjectServiceMetrics function.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen linter for FormatValidator.ValidateContent method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx linter for SearchSGPrm and GetSGPrm  structs.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve containedctx linter for commonPrm.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve funlen & gocognit linters for traverseSplitChain method.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#223] node: Refactor cache usage
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
2c07f831c7 
Drop excess type args.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#230] CHANGELOG.md: Remove older entries
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
6bf11f7cca 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
I tried to add 4 more tests and suddenly, it became harder to navigate in
code. Move directory creation in a common function.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Before this commit it was like this:
```
BenchmarkSubstorageWritePerf/memstore-rand10-8            227425              4859 ns/op
BenchmarkSubstorageWritePerf/fstree_nosync-rand10-8     --- FAIL: BenchmarkSubstorageWritePerf/fstree_nosync-rand10-8
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
BenchmarkSubstorageWritePerf/fstree-rand10-8            --- FAIL: BenchmarkSubstorageWritePerf/fstree-rand10-8
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
    perf_test.go:165: writing entry: file exists
```

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#236] blobstor/test: Prefill storage in parallel in read benchmark
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
c4865783fc 
`blobovniczatree` takes a really long time to prefill, because each
batch takes at least 10ms, so for 10k iterations we have at least 100s of
prefill.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#135] signature: Add tracing
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
5778980252 
Add tracing to verify request and sign response.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
This reverts commit b2ca730547.
This reverts commit 5778980252.
Release v0.36.0
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
7d39fecc6a 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#113] cli: add "name" option for "get container" command
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
01c0c90a86 
* Make get container command filter out the container by attribute name

Signed-off-by: Airat Arifullin a.arifullin@yadro.com
`VerifyBlocks` is now `SkipBlockVerification` and is false by default.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Fixes #2230, fixes #2263. CustomGroups are nice while we're only calling NeoFS
contracts, but it doesn't work at all for standard ones like GAS or Notary.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
One signer in the cfg is enough.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
That's the reason #2230 and #2263 were not detected earlier, we actually had
Global scope being used before reconnection to RPC node.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#239] morph/client: Simplify code interacting with magic numbers
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
d29b13454f 
It can't be uint64 in fact, but this error is buried deeply in the NetworkInfo
API structure, so we're not touching MagicNumber() for now.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Drop duplicate entities.
Format entities.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#240] logs: Factor out common service log messages
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
cffcc7745e 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
[#100] adm: Use netmap constants from pkg
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
299b6a6938 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
[#246] .gitattributes: Do not show diff for go.sum
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
adcfce39cf 
When we update dependencies it can be rather big. However it is
generated automatically with `go mod tidy`, no need to review.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#139] Refactor blobovnicza exist test to not use chmod
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
41eb3129ae 
Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
[#238] node: Read cfg from dir even if cfg file not set
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
995db117d0 
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Add tracing spans for PUT requests.
Add tracing spans for DELETE requests.
Add tracing spans for SELECT requests.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#242] treesvc: Add tracing spans
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
6121b541b5 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#247] logs: Drop reputation log messages
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
b453bb754c 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Drop the code that was expected to work with global reputation network
parameters.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#247] client: Drop reputation related RPCs
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
070154d506 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#250] control: remove DumpShard and RestoreShard RPC
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
8466894fdf 
We have `Evacuate` with a cleaner interface.
Also, remove them from CLI and engine.

Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#254] innerring: Remove unused TimersHandlers() method from processors
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
0b42a00a60 
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
[#256] blobovniczaTree: Make Exists test stable
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
262c9c2b93 
Corrupt and request _the same_ file.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#249] logs: Drop unused consts
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
7b981bfe97 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Fix tracing panic.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
Resolve unused linter.

Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#265] node: Fix after SDK & API-Go version up
All checks were successful
ci/woodpecker/pr/pre-commit Pipeline was successful
Details
ci/woodpecker/push/pre-commit Pipeline was successful
Details
3d43b0f7f9 
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
[#255] write-cache: Fix init race condition
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
20cd080323 
Do not use WC's internals in the initialization routines without mode
protection. WC should be able to change its mode even if the initialization
is not finished yet.

Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
[#118] node: add unit concurrent tests for blobstor
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
13c8afcb02 
Signed-off-by: Airat Arifullin a.arifullin@yadro.com
[#262] meta: Do not return old expired objects
All checks were successful
ci/woodpecker/push/pre-commit Pipeline was successful
Details
3d23b08773 
Signed-off-by: Pavel Karpy <p.karpy@yadro.com>
carpawell referenced this pull request from a commit 2023-04-19 17:03:06 +00:00
[#39] node: Add optional profilers
carpawell referenced this pull request from a commit 2023-04-19 17:03:06 +00:00
[#39] ir: Add optional profilers
carpawell force-pushed carpawell/optional-profiles from 32350ed1e6 to b287a8a290 2023-04-19 17:03:10 +00:00 Compare
carpawell referenced this pull request from a commit 2023-04-19 17:05:02 +00:00
[#39] ir: Add optional profilers
carpawell force-pushed carpawell/optional-profiles from b287a8a290 to 1250dd23d7 2023-04-19 17:05:05 +00:00 Compare
carpawell referenced this pull request from a commit 2023-04-19 17:11:18 +00:00
[#39] ir: Add optional profilers
carpawell force-pushed carpawell/optional-profiles from 1250dd23d7 to 7f41586137 2023-04-19 17:11:20 +00:00 Compare
fyrchik approved these changes 2023-04-20 08:22:37 +00:00
cmd/frostfs-node/pprof.go Outdated
@ -8,0 +19,4 @@
mutexRate = -1
}
runtime.SetBlockProfileRate(blockRate)
fyrchik commented 2023-04-20 08:21:28 +00:00
Owner
Copy link

Where do we have .enabled check?

Where do we have `.enabled` check?
carpawell commented 2023-04-20 12:20:08 +00:00
Member
Copy link

hm. reorganized a little, also, added support for SIGHUP

hm. reorganized a little, also, added support for `SIGHUP`
fyrchik marked this conversation as resolved
docs/storage-node-configuration.md Outdated
@ -83,1 +89,4 @@
| Parameter | Type | Default value | Description |
|--------------|-------|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `block_rate` | `int` | `0` | Controls the fraction of goroutine blocking events that are reported in the blocking profile. The profiler aims to sample an average of one blocking event per rate nanoseconds spent blocked. Non-positive values disable profiler reports. |
fyrchik commented 2023-04-20 08:22:34 +00:00
Owner
Copy link

Can we just give a brief description refer to a Go-doc instead of copying?

Can we just give a brief description refer to a Go-doc instead of copying?
carpawell commented 2023-04-20 12:20:39 +00:00
Member
Copy link

check the updated version, please

check the updated version, please
fyrchik marked this conversation as resolved
carpawell referenced this pull request from a commit 2023-04-20 12:14:19 +00:00
[#39] node: Add optional profilers
carpawell referenced this pull request from a commit 2023-04-20 12:14:19 +00:00
[#39] ir: Add optional profilers
carpawell force-pushed carpawell/optional-profiles from 7f41586137 to 0c7040bdf2 2023-04-20 12:14:22 +00:00 Compare
carpawell referenced this pull request from a commit 2023-04-20 12:22:28 +00:00
[#39] node: Add optional profilers
carpawell referenced this pull request from a commit 2023-04-20 12:22:28 +00:00
[#39] ir: Add optional profilers
carpawell force-pushed carpawell/optional-profiles from 0c7040bdf2 to dd56abf020 2023-04-20 12:22:32 +00:00 Compare
carpawell requested review from fyrchik 2023-04-20 12:22:41 +00:00
carpawell requested review from storage-core-developers 2023-04-20 12:22:48 +00:00
carpawell requested review from storage-core-committers 2023-04-20 12:22:48 +00:00
carpawell referenced this pull request from a commit 2023-04-20 12:27:22 +00:00
[#39] node: Add optional profilers
carpawell referenced this pull request from a commit 2023-04-20 12:27:22 +00:00
[#39] ir: Add optional profilers
carpawell force-pushed carpawell/optional-profiles from dd56abf020 to a491b875fa 2023-04-20 12:27:24 +00:00 Compare
acid-ant reviewed 2023-04-20 12:37:54 +00:00
cmd/frostfs-node/main.go Outdated
@ -88,3 +87,3 @@
initAndLog(c, pprof.name, pprof.init)
initAndLog(c, metrics.name, metrics.init)
initAndLog(c, "profiler", initProfilerService)
acid-ant commented 2023-04-20 12:37:54 +00:00
Member
Copy link

pprof was on top of init process to give us the ability to debug shutdown process. Now it is possible to stuck at the metrics when shutdown node without instrument for profiling routines.

pprof was on top of init process to give us the ability to debug shutdown process. Now it is possible to stuck at the metrics when shutdown node without instrument for profiling routines.
👍 1
fyrchik commented 2023-05-10 07:34:03 +00:00
Owner
Copy link

Fixed

Fixed
dstepanov-yadro reviewed 2023-04-20 14:58:06 +00:00
cmd/frostfs-ir/main.go Outdated
@ -111,2 +112,4 @@
func initHTTPServers(cfg *viper.Viper, log *logger.Logger) []*httputil.Server {
pprof := "pprof"
pprofDebug := pprof + ".debug"
dstepanov-yadro commented 2023-04-20 14:58:06 +00:00
Member
Copy link

Do we really need this debug section? Is it about pprof overhead?

Do we really need this debug section? Is it about pprof overhead?
fyrchik commented 2023-05-10 07:59:16 +00:00
Owner
Copy link

They can have noticeable overhead for small values, e.g. reasonable default is 10k, not 1. Let's disable it by default (0 value) and remove from the debug section.

They can have noticeable overhead for small values, e.g. reasonable default is 10k, not 1. Let's disable it by default (0 value) and remove from the `debug` section.
dstepanov-yadro marked this conversation as resolved
fyrchik referenced this pull request from a commit 2023-05-10 07:33:17 +00:00
[#39] node: Add optional profilers
fyrchik referenced this pull request from a commit 2023-05-10 07:33:17 +00:00
[#39] ir: Add optional profilers
fyrchik force-pushed carpawell/optional-profiles from a491b875fa to b05cb54681 2023-05-10 07:33:19 +00:00 Compare
fyrchik commented 2023-05-10 08:00:06 +00:00
Owner
Copy link

Updated, also, please, check the values we pass to runtime.Set..., there was a bug here in the previous iteration.

Updated, also, please, check the values we pass to `runtime.Set...`, there was a bug here in the previous iteration.
fyrchik changed title from Feat/Add optional profiles to Add block and mutex profiles 2023-05-10 08:02:23 +00:00
dstepanov-yadro approved these changes 2023-05-10 11:14:14 +00:00
fyrchik referenced this pull request from a commit 2023-05-12 08:42:19 +00:00
[#39] node: Add optional profilers
fyrchik referenced this pull request from a commit 2023-05-12 08:42:19 +00:00
[#39] ir: Add optional profilers
fyrchik referenced this pull request from a commit 2023-05-12 08:42:19 +00:00
[#39] ir: Use defer for wg.Done()
fyrchik force-pushed carpawell/optional-profiles from b05cb54681 to 701e6c5323 2023-05-12 08:42:21 +00:00 Compare
fyrchik requested review from acid-ant 2023-05-12 08:42:27 +00:00
acid-ant approved these changes 2023-05-12 12:02:32 +00:00
fyrchik requested review from dstepanov-yadro 2023-05-12 12:50:05 +00:00
dstepanov-yadro requested changes 2023-05-12 14:06:17 +00:00
cmd/frostfs-ir/httpcomponent.go Outdated
@ -27,3 +30,1 @@
c.enabled = cfg.GetBool(c.enabledKey)
c.address = cfg.GetString(c.addressKey)
c.shutdownDur = cfg.GetDuration(c.shutdownTimeoutKey)
c.enabled = cfg.GetBool(c.name + ".enabled")
dstepanov-yadro commented 2023-05-12 14:03:27 +00:00
Member
Copy link

enabledKeyPostfix ?

enabledKeyPostfix ?
fyrchik commented 2023-05-12 14:50:45 +00:00
Owner
Copy link

fixed

fixed
dstepanov-yadro marked this conversation as resolved
fyrchik referenced this pull request from a commit 2023-05-12 14:50:39 +00:00
[#39] node: Add optional profilers
fyrchik referenced this pull request from a commit 2023-05-12 14:50:39 +00:00
[#39] ir: Add optional profilers
fyrchik referenced this pull request from a commit 2023-05-12 14:50:39 +00:00
[#39] ir: Use defer for wg.Done()
fyrchik force-pushed carpawell/optional-profiles from 701e6c5323 to 429a87e83b 2023-05-12 14:50:41 +00:00 Compare
dstepanov-yadro approved these changes 2023-05-12 15:00:42 +00:00
aarifullin approved these changes 2023-05-15 08:22:50 +00:00
fyrchik merged commit 429a87e83b into master 2023-05-15 08:36:49 +00:00
fyrchik deleted branch carpawell/optional-profiles 2023-05-15 08:36:50 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
fyrchik
acid-ant
dstepanov-yadro
aarifullin
alexvanin
KirillovDenis
fyrchik
acid-ant
Labels
Clear labels   
P0

Highest

  
P1

High

  
P2

Medium

  
P3

Low

  
badger

Badger DB related issues

  
frostfs-adm

   
frostfs-cli

   
frostfs-ir

   
frostfs-lens

   
frostfs-node

   
good first issue

Good for newcomers

  
triage
  
Infrastructure

Infrastructure, Automation, CI/CD or DevOps related stuff

  
blocked

The issue or PR is blocked by something

  
bug

Something is not working

  
config

Configuration format update or breaking change

  
discussion

Talking about something in order to reach a decision or to exchange ideas

  
documentation

Documentation related

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
go

Language features adoption

  
help wanted

Extra attention is needed

  
internal

   
invalid

Something is wrong

  
kludge

This is a kludge and we know it

  
observability

Related to metrics, tracing and logs

  
perfomance

Perfomance related

  
question

More information is needed

  
refactoring

Refactoring

  
wontfix

This won't be fixed

No labels
P0
P1
P2
P3
badger
frostfs-adm
frostfs-cli
frostfs-ir
frostfs-lens
frostfs-node
good first issue
triage
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
6 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-node#39
No description provided.
Delete branch "carpawell/optional-profiles"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?