5432 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Dmitrii Stepanov	bc6cc9ae2a	[#1700] engine: Print stacks on test request limiter ... Change-Id: I4952769ca431d1049955823b41b99b0984b385fc Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-04-01 14:23:02 +03:00
Ekaterina Lebedeva	6e1576cfdb	[#1656] qos: Add tests for AdjustOutgoingIOTag Interceptors ... Change-Id: If534e756b26cf7f202039d48ecdf554b4283728b Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2025-04-01 11:55:15 +03:00
Dmitrii Stepanov	a5bae6c5af	[#1699] qos: Allow to prohibit operations for IO tag ... Change-Id: I2bee26885244e241d224860978b6de3526527e96 Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-04-01 10:08:03 +03:00
Dmitrii Stepanov	5a13830a94	[#1699] mod: Bump frostfs-qos version ... Change-Id: Ie5e708c0ca653596c6e3346aa286618868a5aee8 Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-04-01 10:08:03 +03:00
Ekaterina Lebedeva	dcb2b23a7d	[#1656] qos: Add test for SetCriticalIOTag Interceptor ... Change-Id: I4a55fcb84e6f65408a1c0120ac917e49e23354a1 Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2025-03-31 18:21:48 +03:00
Ekaterina Lebedeva	115aae7c34	[#1656] qos: Add tests for MaxActiveRPCLimiter Interceptors ... Change-Id: Ib65890ae5aec34c34e15d4ec1f05952f74f1ad26 Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2025-03-31 18:21:46 +03:00
Vitaliy Potyarkin	12a0537a7a	[#1689] ci: Add commit checker to Jenkinsfile ... - Commit checker image is built from dco-go: TrueCloudLab/dco-go#14 - 'pull_request_target' branch is defined in Jenkins job: TrueCloudLab/jenkins#10 TrueCloudLab/jenkins#11 Change-Id: Ib86c5749f9e084d736b868240c4b47014b02ba8d Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>	2025-03-31 15:08:59 +00:00
Anton Nikiforov	30d4692c3e	[#1640] go.mod: Bump version for frostfs-locode-db ... Change-Id: Ic45ae77d6209c0097575fc8f89b076b22d50d149 Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2025-03-31 10:29:41 +00:00
Airat Arifullin	2254c8aff5	[#1689] go.mod: Bump SDK version ... Change-Id: Ic946aa68c3d6da9e7d54363f8e9141c6547707d6 Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2025-03-31 11:55:29 +03:00
Dmitrii Stepanov	d432bebef4	[#1689] go.mod: Bump frostfs-qos version ... Change-Id: Iaa28da1a1e7b2f4ab7fd8ed661939eb38f4c7782 Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-28 12:40:05 +00:00
Evgenii Stratonikov	d144abc977	[#1692] metabase: Remove useless count variable ... It is always equal to `len(to)`. Change-Id: Id7a4c26e9711216b78f96e6b2511efa0773e3471 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-28 12:16:01 +00:00
Evgenii Stratonikov	a2053870e2	[#1692] metabase: Use bucket cache in ListWithCursor() ... No changes in speed, but unified approach: ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ master │ new │ │ sec/op │ sec/op vs base │ ListWithCursor/1_item-8 6.067µ ± 8% 5.731µ ± 10% ~ (p=0.052 n=10) ListWithCursor/10_items-8 25.40µ ± 11% 26.12µ ± 13% ~ (p=0.971 n=10) ListWithCursor/100_items-8 210.7µ ± 9% 203.2µ ± 6% ~ (p=0.280 n=10) geomean 31.90µ 31.22µ -2.16% │ master │ new │ │ B/op │ B/op vs base │ ListWithCursor/1_item-8 3.287Ki ± 0% 3.287Ki ± 0% ~ (p=1.000 n=10) ¹ ListWithCursor/10_items-8 15.63Ki ± 0% 15.62Ki ± 0% ~ (p=0.328 n=10) ListWithCursor/100_items-8 138.1Ki ± 0% 138.1Ki ± 0% ~ (p=0.340 n=10) geomean 19.21Ki 19.21Ki -0.00% ¹ all samples are equal │ master │ new │ │ allocs/op │ allocs/op vs base │ ListWithCursor/1_item-8 109.0 ± 0% 109.0 ± 0% ~ (p=1.000 n=10) ¹ ListWithCursor/10_items-8 380.0 ± 0% 380.0 ± 0% ~ (p=1.000 n=10) ¹ ListWithCursor/100_items-8 3.082k ± 0% 3.082k ± 0% ~ (p=1.000 n=10) ¹ geomean 503.5 503.5 +0.00% ¹ all samples are equal ``` Change-Id: Ic11673427615053656b2a60068a6d4dbd27af2cb Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-28 12:16:01 +00:00
Evgenii Stratonikov	d00c606fee	[#652] adm: Group independent stages in batches ... Each stage waits until transaction persists. This is needed to ensure the next stage will see the result of the previous one. However, some of the stages do not depend one on another, so we may execute them in parallel. `AwaitDisabled` flag is used to localize this batching on the code level. We could've removed `AwaitTx()` from respective stages, but it seems more error prone. Close #652. Change-Id: Ib9c6f6cd5e0db0f31aa1cda8e127b1fad5166336 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-28 12:15:21 +00:00
Evgenii Stratonikov	60446bb668	[#1689] adm/helper: Use proper nns bindings import ... The one in `neo-go` is for another contract. Change-Id: Ia1ac2da5e419b48801afdb26df72892d77344e0d Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-28 12:15:21 +00:00
Evgenii Stratonikov	bd8ab2d84a	[#1689] adm: Remove useless switch in NNSIsAvailable() ... After all the refactorings, there is no need to have custom behaviour for local client. Change-Id: I99e297cdeffff979524b3f89d3580ab5780e7681 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-28 12:15:21 +00:00
Evgenii Stratonikov	bce2f7bef0	[#1689] adm: Reuse neo.NewReader helper in transferNEOFinished() ... Change-Id: I27980ed87436958cb4d27278e30e05da021d1506 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-28 12:10:37 +00:00
Evgenii Stratonikov	c2c05e2228	[#1689] adm: Reuse ReadOnlyInvoker in registerCandidateRange() ... Change-Id: I544d10340825494b45a62700fa247404c18f746a Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-28 12:10:37 +00:00
Evgenii Stratonikov	0a38571a10	[#1689] adm: Simplify getCandidateRegisterPrice() ... After all the refactoring, there is no more need to have custom branch for the local client. Change-Id: I274305b0c390578fb4583759135d3e7ce58873dc Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-28 12:10:31 +00:00
Dmitrii Stepanov	632bd8e38d	[#1696] qos: Fix internal tag adjust ... If request has no tag, but request's public key is netmap node's key or one of allowed internal tag keys from config, then request must use internal IO tag. Change-Id: Iff93b626941a81b088d8999b3f2947f9501dcdf8 Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-28 07:47:12 +00:00
Anton Nikiforov	3bbee1b554	[#1619] logger: Allow to set options for zap.Logger via logger.Prm ... Change-Id: I8eed951c25d1ecf18b0aea62c6825be65a450085 Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2025-03-27 15:41:37 +03:00
Anton Nikiforov	9358938222	[#1633] go.mod: Bump frostfs-sdk-go ... Change-Id: I50c1a0d5b88e307402a5b1b2883bb9b9a357a2c7 Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2025-03-26 15:03:17 +00:00
Anton Nikiforov	5470b205fd	[#1619] gc: Fix metric frostfs_node.garbage_collector.marking_duration_seconds ... Change-Id: I957f930d1babf179d0fb6de624a90f4fe9977862 Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2025-03-26 14:14:26 +03:00
Vitaliy Potyarkin	163e2e9f83	ci: Cache pre-commit installations on Jenkins Agent ... This change introduces a custom helper from our shared library [0] that defines ad-hoc container environment to execute CI steps in. Pre-commit installation will now be cached on Jenkins Agent: builds will tolerate network hiccups better and we will also save some run time (although on non-critical path of a parallel process). [0]: TrueCloudLab/jenkins#8 Change-Id: I93b01f169c457aa35f4d8bc5b90f31b31e2bd8b2 Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>	2025-03-24 17:04:24 +03:00
Dmitrii Stepanov	0c664fa804	[#1689] qos: Fix metric description ... Change-Id: I460fdd3713e765d57ef3ff2945b9b3776f46c164 Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-24 09:42:45 +00:00
Evgenii Stratonikov	0a9d139e20	[#1689] morph/client: Remove notary hash field from notaryInfo ... Notary contract hash is constant. Change-Id: I7935580acbced5c9d567875ea75daa57cc259a3c Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-24 09:20:19 +00:00
Evgenii Stratonikov	3bb1fb744a	[#1689] morph/client: Reuse auto-generated wrappers for NNS ... Make code simpler, remove unused methods. Change-Id: I18807f2c14b5a96e533e5e3fc153e23c742c66c1 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-24 09:20:19 +00:00
Airat Arifullin	ccdd6cb767	[#1052] object: Nuke out acl middleware ... * Remove `acl` package as it's no longer used; * Remove `RequestContext`; * Fix `cmd/frostfs-node`. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2025-03-23 06:39:32 +00:00
Airat Arifullin	73e35bc885	[#1052] object: Make ape middleware form request info ... * Move some helpers from `acl/v2` package to `ape`. Also move errors; * Introduce `Metadata`, `RequestInfo` types; * Introduce `RequestInfoExtractor` interface and its implementation. The extractor's purpose is to extract request info based on request metadata. It also validates session token; * Refactor ape service - each handler forms request info and pass necessary fields to checker. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2025-03-23 06:39:32 +00:00
Dmitrii Stepanov	eed0824590	go.mod: Bump frostfs-qos version ... Change-Id: I8bc045b509ee1259cfad288477a0b7d045683f10 Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-21 16:36:03 +00:00
Evgenii Stratonikov	a4da1da767	[#905] morph/client: Fetch NNS hash once on init ... NNS contract hash is taken from the contract with ID=1. Because morph client is expected to work with the same chain, and because contract hash doesn't change on update, there is no need to fetch it from each new endpoint. Change-Id: Ic6dc18283789da076d6a0b3701139b97037714cc Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-21 15:13:54 +00:00
Evgenii Stratonikov	30099194ba	[#1689] config: Remove testnet and mainnet configs ... They are invalid and unsupported. There is neither mainnet nor testnet currently. Change-Id: I520363e2de0c22a584238accc253248be3eefea5 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-21 15:07:02 +00:00
Evgenii Stratonikov	e7e91ef634	[#1689] adm: Remove storagecfg subcommand ... It is unused and unsupported for a long time. Change-Id: I570567db4e8cb202e41286064406ad85cd0e7a39 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-21 15:05:02 +00:00
Aleksey Savchuk	4919b6a206	[#1689] node/config: Allow zero max_ops in RPC limits config ... The limiter allows zeros for limits, meaning "this operation is disabled". However, the config didn't allow zero due to the lack of distinction between "no value" and "zero" - cast functions read both `nil` and zero as zero. Now, the config allows a zero limit. Added tests. Managing such cases should be easier after #1610. Change-Id: Ifc840732390b2feb975f230573b34bf479406e05 Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2025-03-21 17:00:40 +03:00
Alexander Chuprov	d951289131	[#1294] docs: Fix description of shard switching mode ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2025-03-21 12:13:03 +00:00
Evgenii Stratonikov	016f2e11e3	[#1689] Makefile: Add more restricted .SHELLFLAGS ... Catch more errors immediately. Change-Id: I576f1b394a2b167c78c693a794ab8cca3ac1013b Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-21 11:49:49 +00:00
Evgenii Stratonikov	9aa486c9d8	[#1689] Makefile: Create dirs with -p flag ... On CI there is no `bin` directory initially, so an error occurs ``` mkdir: cannot create directory '/var/cache/jenkins-agent/workspace/gerrit/frostfs-node#55-488b12-8ac3c/bin/gofumpt': No such file or directory ``` Change-Id: I43895c8f5ed7cc5c71c8025228710279f9e75e9c Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-21 11:49:42 +00:00
Dmitrii Stepanov	af76350bfb	[#1695] qos: Sort tags by asc ... Change-Id: Ia23e392bb49d2536096de2ba07fc6f8fb7ac0489 Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-21 11:25:33 +00:00
Evgenii Stratonikov	3fa5c22ddf	[#1689] Makefile: Add default reviewers via --push-option ... Gerrit doesn't provide an easy way to have default reviewers assigned to new change requests. However, we can use `--push-option` and mention all people from storage-core-developers group. Change-Id: Ia01f8a3c5c8eb8a1dca6efb66fbe07018f6a42c9 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-21 11:25:21 +00:00
Dmitrii Stepanov	5385f9994f	[#1695] mod: Bump frostfs-observability version ... Change-Id: Id362b71f743ff70c8cd374030c9fa67e2566022f Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-21 13:28:02 +03:00
Dmitrii Stepanov	eea46a599d	[#1695] qos: Add treesync tag ... Tree sync is too much different from GC and rebuild to use the same tag for GC and tree sync. Change-Id: Ib44d5fa9a88daff507d759d0b0410cc9272e236f Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-21 13:28:02 +03:00
Anton Nikiforov	049a650b89	[#1619] logger: Simplify logger config reloading ... Change-Id: Ide892b250304b8cdb6c279f5f728c3b35f05df54 Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2025-03-21 09:03:54 +00:00
Evgenii Stratonikov	3f4717a37f	[#1692] metabase: Do not allocate map in cache unless needed ... Change-Id: I8b1015a8c7c3df4153a08fdb788117d9f0d6c333 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-21 08:56:32 +00:00
Evgenii Stratonikov	60cea8c714	[#1692] metabase/test: Fix end of iteration error check ... This is not good: ``` BenchmarkListWithCursor/1_item-8 --- FAIL: BenchmarkListWithCursor/1_item-8 list_test.go:63: error: end of object listing ``` Change-Id: I61b70937ce30fefaf16ebeb0cdb51bdd39096061 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-21 08:56:18 +00:00
Evgenii Stratonikov	7df2912a83	[#1689] Makefile: Create prepare-commit-msg hook too ... `commit-msg` is ignored when `--no-verify` option is used, so there is no way to ignore `pre-commit` while retaining `commit-msg` hook. Ignoring pre-commit is useful, though, so we might add Change-Id in `prepare-commit-msg` hook instead. It accepts more parameters, but the first one is a file with the commit message, so we may reuse `commit-msg` hook. Change-Id: I4edb79810bbe38a5dcf7f4f07535f34c6bda0da3 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-20 18:01:53 +03:00
Evgenii Stratonikov	affab25512	Makefile: Add Gerrit-related targets ... This commit adds helper targets to easily setup an existing repo for work with Gerrit. Change-Id: I0696eb8ea84cc16a9482be6a2fb0382fe624bb96 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-20 16:48:04 +03:00
Vitaliy Potyarkin	45b7796151	[#1689] ci: Reimplement CI tasks in Jenkinsfile ... This commit introduces Jenkins pipeline that duplicates the features of existing Forgejo Actions workflows. Change-Id: I657a6c27373a1ed4736ae27b4fb660e0ac86012d Signed-off-by: Vitaliy Potyarkin <v.potyarkin@yadro.com>	2025-03-20 12:55:30 +00:00
Evgenii Stratonikov	e8801dbf49	[#1691] metabase: Move cheaper conditions to the front in ListWithCursor() ... `objectLocked` call is expensive, it does IO. We may omit it if object is not expired. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-20 12:52:36 +00:00
Evgenii Stratonikov	eb9df85b98	[#1685] metabase: Cache primary bucket ... ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ expired │ primary │ │ sec/op │ sec/op vs base │ Select/string_equal-8 3.529m ± 11% 3.689m ± 7% +4.55% (p=0.023 n=10) Select/string_not_equal-8 3.440m ± 7% 3.543m ± 13% ~ (p=0.190 n=10) Select/common_prefix-8 3.240m ± 6% 3.050m ± 5% -5.85% (p=0.005 n=10) Select/unknown-8 3.198m ± 6% 2.928m ± 8% -8.44% (p=0.003 n=10) geomean 3.349m 3.287m -1.84% │ expired │ primary │ │ B/op │ B/op vs base │ Select/string_equal-8 1.885Mi ± 0% 1.786Mi ± 0% -5.23% (p=0.000 n=10) Select/string_not_equal-8 1.885Mi ± 0% 1.786Mi ± 0% -5.23% (p=0.000 n=10) Select/common_prefix-8 1.885Mi ± 0% 1.786Mi ± 0% -5.23% (p=0.000 n=10) Select/unknown-8 1.877Mi ± 0% 1.779Mi ± 0% -5.26% (p=0.000 n=10) geomean 1.883Mi 1.784Mi -5.24% │ expired │ primary │ │ allocs/op │ allocs/op vs base │ Select/string_equal-8 46.04k ± 0% 43.04k ± 0% -6.50% (p=0.000 n=10) Select/string_not_equal-8 46.04k ± 0% 43.04k ± 0% -6.50% (p=0.000 n=10) Select/common_prefix-8 46.04k ± 0% 43.04k ± 0% -6.50% (p=0.000 n=10) Select/unknown-8 45.05k ± 0% 42.05k ± 0% -6.65% (p=0.000 n=10) geomean 45.79k 42.79k -6.54% ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-20 12:52:01 +00:00
Evgenii Stratonikov	21bed3362c	[#1685] metabase: Cache expired bucket ... ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ master │ expired │ │ sec/op │ sec/op vs base │ Select/string_equal-8 4.007m ± 10% 3.529m ± 11% -11.94% (p=0.000 n=10) Select/string_not_equal-8 3.834m ± 12% 3.440m ± 7% -10.29% (p=0.029 n=10) Select/common_prefix-8 3.470m ± 9% 3.240m ± 6% ~ (p=0.105 n=10) Select/unknown-8 3.156m ± 3% 3.198m ± 6% ~ (p=0.631 n=10) geomean 3.602m 3.349m -7.03% │ master │ expired │ │ B/op │ B/op vs base │ Select/string_equal-8 1.907Mi ± 0% 1.885Mi ± 0% -1.18% (p=0.000 n=10) Select/string_not_equal-8 1.907Mi ± 0% 1.885Mi ± 0% -1.18% (p=0.000 n=10) Select/common_prefix-8 1.907Mi ± 0% 1.885Mi ± 0% -1.18% (p=0.000 n=10) Select/unknown-8 1.900Mi ± 0% 1.877Mi ± 0% -1.18% (p=0.000 n=10) geomean 1.905Mi 1.883Mi -1.18% │ master │ expired │ │ allocs/op │ allocs/op vs base │ Select/string_equal-8 47.03k ± 0% 46.04k ± 0% -2.12% (p=0.000 n=10) Select/string_not_equal-8 47.03k ± 0% 46.04k ± 0% -2.12% (p=0.000 n=10) Select/common_prefix-8 47.03k ± 0% 46.04k ± 0% -2.12% (p=0.000 n=10) Select/unknown-8 46.04k ± 0% 45.05k ± 0% -2.16% (p=0.000 n=10) geomean 46.78k 45.79k -2.13% ``` Change-Id: I9c7a5e1f5c8b9eb3f25a563fd74c6ad2a9d1b92e Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2025-03-20 12:52:01 +00:00
Dmitrii Stepanov	af5b3575d0	[#1690] qos: Do not export zero metrics counters ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2025-03-20 14:42:35 +03:00