Commit graph

5530 commits

Author SHA1 Message Date
8e2f919df0 [#1689] go.mod: Bump SDK version
* Fix `APEOverride` method usage in ape checker.
* Fix linter errors: factor out deprecated methods and packages.

Change-Id: I8c939f4c58c2a4e3c4e795c7224d935d40ce6f24
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2025-04-29 16:00:23 +03:00
652237500f
[#1689] tree: Dial tree service after create connection on sync
Background trees sync creates grpc connection with
`grpc.WithDefaultCallOptions(grpc.WaitForReady(true))` option. When grpc
connection created with this option, client will wait until a connection
becomes available or the RPC's deadline is reached. As background sync
has no timeout in context, so in case of client is in TRANSIENT_FAILURE
RPC call will hang forever.

Change-Id: I17c8c1d2779bb81c541f47dd0e558e0b8ed2e7c1
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-24 17:05:43 +03:00
5b6cba04cb
[#1689] node: Drop node.relay config
It is not used and not tested properly, so drop it.

Change-Id: I7c90c7391ecb4be17459415d209811ba1a693f7a
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-23 12:43:52 +03:00
b0f39dca16 [#1721] object: Make CheckAPE always validate bearer token
* The bearer token must always be validated, regardless of whether it has been impersonated;
* Fix unit-tests for tree service which check verification with bearer token.

Close #1721

Change-Id: I5f715c498ae10b2e758244e60b8f21849328a04f
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2025-04-22 19:36:42 +03:00
6bdbe6a18b [#1689] adm: Fix printSubjectInfo method
* Fix print format for primary and additional subject keys
  from frostfsid contract. Since the format corresponds to
  `neo-go wallet dump-keys` output format.

Change-Id: I9ae9fd43bfb378970786b97bd3d9d7f739466ae6
Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>
2025-04-22 17:41:27 +03:00
77b8545601
[#1689] engine: Fix removal of objects not found on node
Ensured correct object status if the object is not found on a node.
Fixed regression introduced in #1450.

Besides an object not being found on any shard, it also important to
remove it anyway in order to populate the metabase indexes because
they are responsible for the correct object status, i.e., the status
will be `object not found` without the indexes, the status will be
`object is already removed` with the indexes.

Change-Id: I6237fbc0f8bb0c4f2a51ada3a68f52950050e660
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2025-04-22 10:04:42 +03:00
487cb34c5d
[#1689] engine/test: Refactor TestInhumeIfObjectDoesntExist test
- Use the same storage engine in multiple parallel tests
- Move `Lock`, `Inhume`, `Head` calls to separate functions

Change-Id: I00849c1f068f0ab8d92061719d67d6fe786200db
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2025-04-21 15:32:04 +03:00
908b08108d
[#1689] engine/test: Fix TestInhumeIfObjectDoesntExist test
Removed an invalid test case which used exclusive options, added
object status check after removal.

Change-Id: I4551c0e4532fb669ee6c72871dc4bd34707d8469
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2025-04-21 15:31:48 +03:00
3a441f072f
[#1709] shard: Check if context canceled for shard iteration
If context has already been canceled, then there is no need to check other shards.
At the same time, it is necessary to avoid handling context cancellation
in each handler. Therefore, the context check has been moved to the shard
iteration method, which now returns an error.

Change-Id: I70030ace36593ce7d2b8376bee39fe82e9dbf88f
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-21 15:20:50 +03:00
a27e003508
[#1709] qos: Add context.Done check before schedule request
Do not push request to schedule queue, if context is already cancelled.

Change-Id: Ieef837a7d423158e3dbb0c3b4efecaa20744c845
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-21 10:53:51 +03:00
4bcb672630
[#1707] tree: Pass tracing context in forEachNode
Change-Id: I884dcd215bd3934f9b4ea43dcc77e38f9dadcf10
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-18 18:28:36 +03:00
6567ceaf13
[#1702] tree: Support reloading 'tree.authorized_keys' on SIGHUP
Allows updating the list of public keys authorized to invoke
'Tree service' operations without restarting the node.

Change-Id: I01b6e05875b7ae3f3218062eb12bf9755e87f2a3
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-04-18 10:41:46 +03:00
1a738792c0 [#1722] node, ir: Fix documentation for logger tags
Close #1722.

Change-Id: Iee9b3d96a31353622e1680d2d596ca6a9ee104b4
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2025-04-18 10:14:52 +03:00
c84013a854
[#1719] cli: Fix TTL description
Change-Id: I051a27af57a74304713c1f832dc31dbaeb10cbc6
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-16 17:04:30 +03:00
3cbff57535
[#1718] linter: Enable gocritic linter
See https://go-critic.com/overview#checkers-from-the-diagnostic-group for
list of default enabled checkers.
`ifElseChain` disabled as it generates doubtful issues.

Change-Id: I5937b116d9af8b3cdf8b06451c4904d0b3f67f68
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-16 15:22:54 +03:00
ca8b01667f
[#1718] linter: Resolve gocritic's unlambda linter
See https://go-critic.com/overview#unlambda for details.

Change-Id: Iccb2d293ce31a302fcbb2c3f9c55c9b3fa554db5
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-16 15:22:54 +03:00
b88fe8c4a7
[#1718] linter: Resolve gocritic's assignOp linter
See https://go-critic.com/overview#assignop for details.

Change-Id: I839446846437c8c74c119d8b5669f5b866c247dc
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-16 15:22:53 +03:00
64900f87e1
[#1718] linter: Resolve gocritic's singleCaseSwitch linter
See https://go-critic.com/overview#singlecaseswitch for details.

Change-Id: Ied7885f83b4116969771de6f91bc5e1e3b2a4f1e
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-16 15:22:53 +03:00
8d499f03fe
[#1718] linter: Resolve gocritic's elseif linter
See https://go-critic.com/overview#elseif for details.

Change-Id: I8fd3edfacaeea2b0a83917575d545af7e7ab4d13
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-16 15:22:52 +03:00
d2114759aa
[#1718] linter: Resolve gocritic's typeSwitchVar linter
See https://go-critic.com/overview#typeswitchvar for details

Change-Id: Ic29db32c9b080576ab51dd484b4376114e9e775c
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-16 15:22:52 +03:00
2075e09ced
[#1718] linter: Resolve gocritic's unslice linter
See https://go-critic.com/overview#unslice for details.

Change-Id: I6d21e8ce1c9bae56099dc203f5080b0e3ea0c1ef
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-16 15:22:51 +03:00
cf48069fd8
[#1718] linter: Resolve gocritic's appendAssign linter
See https://go-critic.com/overview#appendassign for details.

Change-Id: I991979ea680af25e2cec9097fa12b1c4eebc6c1d
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-16 15:22:51 +03:00
86264e4e20 [#1619] logger: Add benchmark
Change-Id: I49e90e8a3689a755755afd0638b327a6b1884795
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2025-04-16 13:07:51 +03:00
100eb8b654 [#1619] logger: Set tags for node components
Change-Id: I55ffcce9d2a74fdd47621674739b07f2e20199e3
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2025-04-16 13:07:51 +03:00
36fb15b9a4
[#1689] engine: Return error if object is locked during inhume
Return `object is locked` error if object doesn't exists but is
locked, since the locked index may be populated even when the object
itself doesn't exist.

Change-Id: If1a145c6efead9873acd33bb4fd22cf6175cbabd
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2025-04-16 12:21:13 +03:00
e45382b0c1 [#1693] util: Replace conditional panics with asserts
Change-Id: I13b566cde3e6d43d8a75aa2e9b28e63b597adff9
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2025-04-15 16:54:45 +00:00
5dd8d7e87a [#1693] network: Replace conditional panics with asserts
Change-Id: Icba39aa2ed0048d63c6efed398273627e1e4fbbe
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2025-04-15 16:54:41 +00:00
bc045b29e2 [#1693] services: Replace conditional panics with asserts
Change-Id: Ic79609e6ad867caa88ad245b3014aa7fc32e05a8
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2025-04-15 16:54:36 +00:00
fc6abe30b8 [#1693] storage: Replace conditional panics with asserts
Change-Id: I9d8ccde3c71fca716856c7bfc53da20ee0542f20
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2025-04-15 16:54:31 +00:00
a285d8924f [#1693] node: Replace conditional panics with asserts
Change-Id: I5024705fd1693d00cb9241235030a73984c2a7e1
Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>
2025-04-15 16:28:43 +00:00
410b6f70ba
[#1716] cli: Return trace ID on operation failure
Close #1716

Change-Id: I293d0cc6b7331517e8cde42eae07d65384976da5
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2025-04-15 18:38:55 +03:00
0ee7467da5 [#1715] config: Add compression config section
To group all `compression_*` parameters together.

Change-Id: I11ad9600f731903753fef1adfbc0328ef75bbf87
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-15 15:05:50 +00:00
8c746a914a [#1715] compression: Decouple Config and Compressor
Refactoring.

Change-Id: Ide2e1378f30c39045d4bacd13a902331bd4f764f
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-15 15:05:45 +00:00
98308d0cad [#1715] blobstor: Allow to specify custom compression level
Change-Id: I140c39b9dceaaeb58767061b131777af22242b19
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-15 15:05:39 +00:00
2d1232ce6d [#1689] network,core/netmap: Replace Iterate*() functions with iterators
Change-Id: I4842a3160d74c56d99ea9465d4be2f0662080605
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-04-15 14:47:32 +00:00
e65d578ba9 [#1689] Remove deprecated NodeInfo.IterateAttributes()
Change-Id: Ibd07302079efe148903aa6177759232a28616736
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-04-15 14:47:28 +00:00
bf06c4fb4b [#1689] Remove deprecated NodeInfo.IterateNetworkEndpoints()
Change-Id: Ic78f18aed11fab34ee3147ceea657296b89fe60c
Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>
2025-04-15 14:47:22 +00:00
56d09a9957 [#1640] object: Add priority metric based on geo distance
Change-Id: I3a7ea4fc4807392bf50e6ff1389c61367c953074
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2025-04-15 13:35:28 +00:00
0712c113de
[#1700] gc: Fix deadlock
`HandleExpiredLocks` gets read lock, then `shard.Close` tries to acquire
write lock, but `HandleExpiredLocks` calls `inhumeUnlockedIfExpired` or
`selectExpired`, that try to acquire read lock again.

Change-Id: Ib2ed015e859328045b5a542a4f569e5e0ff8b05b
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-15 10:06:05 +03:00
48930ec452 [#1703] cli: Allow reading RPC endpoint from config file
Allowed reading an RPC endpoint from a configuration file when
getting current epoch in the `object lock` and `bearer create`
commands.

Close #1703

Change-Id: Iea8509dff2893a02cb63f695d7f532eecd743ed8
Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>
2025-04-14 15:40:27 +00:00
f37babdc54
[#1700] shard: Lock shard's mode mutex on close
To prevent race between GC handlers and close.

Change-Id: I06219230964f000f666a56158d3563c760518c3b
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-14 14:35:16 +03:00
fd37cea443
[#1700] engine: Drop unused block execution methods
`BlockExecution` and `ResumeExecution` were used only by unit test.
So drop them and simplify code.

Change-Id: Ib3de324617e8a27fc1f015542ac5e94df5c60a6e
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-14 14:35:15 +03:00
e80632884a
[#1700] config: Drop redundant check
Target config created on level above, so limiter is always nil.

Change-Id: I1896baae5b9ddeed339a7d2b022a9a886589d362
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-14 14:35:15 +03:00
5aaa3df533
[#1700] config: Move config struct to qos package
Change-Id: Ie642fff5cd1702cda00425628e11f3fd8c514798
Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>
2025-04-14 14:35:14 +03:00
3be33b7117
[#1706] cli/playground: Mention 'help' in error message for invalid commands
Change-Id: Ica1112b907919a6d19fa1bf683f2a952c4c638e4
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-04-14 13:10:34 +03:00
29b4fbe451 [#1332] cli/playground: Add 'netmap-config' flag
Change-Id: I4342fb9a6da2a05c18ae4e0ad9f0c71550efc5ef
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-04-14 07:21:08 +00:00
0d36e93169 [#1332] cli/playground: Move command handler selection to separate function
Change-Id: I2dcbd85e61960c3cf141b815edab174e308ef858
Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>
2025-04-13 11:49:45 +00:00
8e87cbee17 [#1689] ci: Move commit checker out of Jenkinsfile
Commit checker is now configured globally for all Gerrit repositories:
  TrueCloudLab/jenkins#16

This allows us to execute commit-checker independently from the rest of
CI suite and re-check commit message format without rerunning other
tests.

Change-Id: Ib8f899b856482a5dc5d03861171585415ff6b452
Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>
2025-04-12 15:42:45 +00:00
12fc7850dd [#1619] logger: Set tags for ir components
Change-Id: Ifab575bc2a3cd83c9001cd68fffaf94c91494043
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2025-04-11 17:27:27 +03:00
dfe2f9956a [#1619] logger: Filter entries by tags provided in config
Change-Id: Ia2a79d6cb2a5eb263fb2e6db3f9cf9f2a7d57118
Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>
2025-04-11 17:27:27 +03:00