TrueCloudLab/frostfs-node

Author	SHA1	Message	Date
Airat Arifullin	6772976657	[#1096 ] container: Make ape middleware fill request with user claim tags Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-04-16 15:10:20 +03:00
Evgenii Stratonikov	3ea1d7b729	[#1089 ] control: Add USER and GROUP targets for local override storage Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-16 11:03:50 +00:00
Evgenii Stratonikov	0094186299	[#1089 ] control: Format proto files with clang-format Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-16 11:03:50 +00:00
Evgenii Stratonikov	91e79c98ba	[#1089 ] ape: Provide request actor as an additional target Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-16 11:03:50 +00:00
Evgenii Stratonikov	e5e0542482	[#1085 ] log: Move storage log message to constants package Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-15 07:57:00 +00:00
Evgenii Stratonikov	5be36924e3	[#41 ] log: Log storage operations in only in Debug They are mostly useless unless we need to _debug_ a specific issue. The amount of logs we produce is too big. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-15 07:57:00 +00:00
aarifullin	6a46c6d229	[#1090 ] tree: Make workaround for APE checks * Make `verifyClient` method perform APE check if a container was created with zero-filled basic ACL. * Object verbs are used in APE, until tree verbs are introduced. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-04-15 07:45:45 +00:00
aarifullin	f4dcb418f2	[#1090 ] ape: Move ape request and resource implementations to common package Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-04-15 07:45:45 +00:00
Dmitrii Stepanov	40781b3a20	[#1086 ] engine: Change mode in case of errors async Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 12:29:43 +00:00
Evgenii Stratonikov	5ef5734c4e	Reapply "[#972 ] Drop x/exp/slices dependency" This reverts commit `946f2ec2bf`. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-10 12:09:34 +00:00
Evgenii Stratonikov	669103a33e	Reapply "[#972 ] Use slices.Sort* when useful" This reverts commit `3359349acb`. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-10 12:09:34 +00:00
Evgenii Stratonikov	2c4b50a71e	Reapply "[#972 ] Use require.ElementsMatch() where possible" This reverts commit `7627d08914`. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-10 12:09:34 +00:00
Evgenii Stratonikov	3dc81cb4fc	Reapply "[#972 ] Use min/max builtins" This reverts commit `dad56d2e98`. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-10 12:09:34 +00:00
Evgenii Stratonikov	d864945961	Reapply "[#972 ] pilorama: Remove removeDuplicatesInPlace()" This reverts commit `9f68305c2e`. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-10 12:09:34 +00:00
Dmitrii Stepanov	1b17258c04	[#1029 ] metabase: Add refill metrics All checks were successful DCO action / DCO (pull_request) Successful in 1m22s Details Vulncheck / Vulncheck (pull_request) Successful in 3m11s Details Build / Build Components (1.21) (pull_request) Successful in 3m56s Details Build / Build Components (1.20) (pull_request) Successful in 3m59s Details Tests and linters / Staticcheck (pull_request) Successful in 5m31s Details Tests and linters / gopls check (pull_request) Successful in 5m26s Details Tests and linters / Lint (pull_request) Successful in 6m13s Details Tests and linters / Tests (1.20) (pull_request) Successful in 8m54s Details Tests and linters / Tests (1.21) (pull_request) Successful in 9m13s Details Tests and linters / Tests with -race (pull_request) Successful in 9m30s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 13:05:44 +03:00
Dmitrii Stepanov	e3d9dd6ee8	[#1024 ] blobovnicza: Copy data on iterate DB value is only valid while the tx is alive. But handler may to run something in other goroutine. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 10:21:11 +03:00
Dmitrii Stepanov	57466594fb	[#1024 ] shard: Resync metabase concurrently Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 10:21:10 +03:00
Dmitrii Stepanov	1005bf4f56	[#1024 ] shard: Add refill metabase benchmark Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 10:21:10 +03:00
Dmitrii Stepanov	76398c06b0	[#1080 ] metabase: Add StorageID metric All checks were successful DCO action / DCO (pull_request) Successful in 1m19s Details Build / Build Components (1.21) (pull_request) Successful in 4m6s Details Vulncheck / Vulncheck (pull_request) Successful in 4m35s Details Build / Build Components (1.20) (pull_request) Successful in 7m32s Details Tests and linters / gopls check (pull_request) Successful in 7m43s Details Tests and linters / Lint (pull_request) Successful in 9m51s Details Tests and linters / Staticcheck (pull_request) Successful in 9m53s Details Tests and linters / Tests (1.20) (pull_request) Successful in 12m48s Details Tests and linters / Tests (1.21) (pull_request) Successful in 13m11s Details Tests and linters / Tests with -race (pull_request) Successful in 14m4s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 10:00:08 +03:00
Dmitrii Stepanov	7b1adfed3e	[#1080 ] metabase: Open bucket for container counter once Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-10 10:00:08 +03:00
Dmitrii Stepanov	e74bdaa5d5	[#1080 ] ape: Use value for APE request Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 18:42:03 +03:00
Dmitrii Stepanov	338d8cbebd	[#1080 ] ape: Do not read object headers before Head/Get Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 15:27:40 +03:00
Dmitrii Stepanov	5b8200de88	[#984 ] blobovnicza: Do not fail rebuild on big objects If blobovnicza contains objects larger than object size parameter value, then rebuild fails with an error, because there is no such bucket in database. This commit forces to create bucket on rebuild. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 11:51:18 +00:00
Anton Nikiforov	2b88361849	[#1062 ] object: Fix buffer allocation for `PayloadRange` All checks were successful DCO action / DCO (pull_request) Successful in 5m34s Details Vulncheck / Vulncheck (pull_request) Successful in 5m14s Details Build / Build Components (1.20) (pull_request) Successful in 8m56s Details Build / Build Components (1.21) (pull_request) Successful in 8m56s Details Tests and linters / gopls check (pull_request) Successful in 9m3s Details Tests and linters / Staticcheck (pull_request) Successful in 9m39s Details Tests and linters / Lint (pull_request) Successful in 10m9s Details Tests and linters / Tests (1.20) (pull_request) Successful in 13m43s Details Tests and linters / Tests (1.21) (pull_request) Successful in 14m4s Details Tests and linters / Tests with -race (pull_request) Successful in 14m40s Details Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-09 11:59:07 +03:00
Dmitrii Stepanov	f5b67c6735	[#1064 ] policer: Disable EC processing Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 07:08:53 +00:00
Dmitrii Stepanov	1c5e0f90aa	[#1064 ] putsvc: Add EC put Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 07:08:53 +00:00
Dmitrii Stepanov	39da643354	[#1064 ] putsvc: Refactor distributed target Extract object builder. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-09 07:08:53 +00:00
Alexander Chuprov	17f7adb640	[#1065 ] adm: Add support EC parameters Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-04-08 12:27:30 +03:00
Anton Nikiforov	9aa533e59a	[#1072 ] node, ir: Add new config option `kludge_compatibility_mode` Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-04 11:05:25 +03:00
Anton Nikiforov	d614f04a0a	[#1072 ] Fix gofumpt issues Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-04-03 22:21:14 +03:00
aarifullin	4738508ce2	[#1063 ] go.mod: Update SDK version * Update frostfs-sdk and frostfs-api-go versions. * Refactor depreacted method ReplicaNumberByIndex. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-04-03 10:33:51 +00:00
Evgenii Stratonikov	ff4c23f59a	[#1070 ] services/tree: Fix fast listing depth processing For unsorted `GetSubTree()` we return a single node for depth=1. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-02 14:41:31 +00:00
Evgenii Stratonikov	17af91619a	[#1070 ] pilorama: Fix cycling behaviour for sorted listing In case there are no items left, return empty slice. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-02 14:41:31 +00:00
Ekaterina Lebedeva	d5194ab2a6	[#949 ] metabase: fix shard.UpdateID() All checks were successful Vulncheck / Vulncheck (pull_request) Successful in 1m20s Details DCO action / DCO (pull_request) Successful in 1m59s Details Build / Build Components (1.21) (pull_request) Successful in 3m25s Details Build / Build Components (1.20) (pull_request) Successful in 4m46s Details Tests and linters / Staticcheck (pull_request) Successful in 6m5s Details Tests and linters / gopls check (pull_request) Successful in 6m17s Details Tests and linters / Lint (pull_request) Successful in 7m7s Details Tests and linters / Tests (1.20) (pull_request) Successful in 8m38s Details Tests and linters / Tests with -race (pull_request) Successful in 8m51s Details Tests and linters / Tests (1.21) (pull_request) Successful in 8m56s Details metabase.Open() now reports metabase mode metric. shard.UpdateID() needs to read shard ID from metabase => needs to open metabase. It caused reporting 'shard undefined' metrics. To avoid reporting wrong metrics metabase.GetShardID() was added which also opens metabase and does not report metrics. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-04-01 17:27:34 +03:00
Ekaterina Lebedeva	81a0346a96	[#949 ] metabase: fix metabase mode metric It used to always show CLOSED regardless of actual mode. Now metric represents actual metabase mode of operations. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-04-01 17:27:34 +03:00
Evgenii Stratonikov	e12fcc041d	[#1059 ] services/tree: Fast sorted listing Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-01 12:37:34 +00:00
Evgenii Stratonikov	f23e38c285	Revert "[#446 ] engine: Move to read-only on blobstor errors" All checks were successful DCO action / DCO (pull_request) Successful in 2m14s Details Build / Build Components (1.20) (pull_request) Successful in 4m7s Details Vulncheck / Vulncheck (pull_request) Successful in 3m30s Details Build / Build Components (1.21) (pull_request) Successful in 4m15s Details Tests and linters / Staticcheck (pull_request) Successful in 5m41s Details Tests and linters / Lint (pull_request) Successful in 6m6s Details Tests and linters / gopls check (pull_request) Successful in 6m42s Details Tests and linters / Tests (1.20) (pull_request) Successful in 7m47s Details Tests and linters / Tests (1.21) (pull_request) Successful in 8m21s Details Tests and linters / Tests with -race (pull_request) Successful in 8m20s Details This reverts commit `69df0d21c2`. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-04-01 12:48:30 +03:00
Dmitrii Stepanov	942d83611b	[#874 ] engine: Revert Check object existance concurrently This reverts commit `f526f49995`. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-04-01 08:42:34 +00:00
Evgenii Stratonikov	fd8cdb9671	[#1057 ] netmap: Do not iterate over external addresses in Node Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-03-29 20:46:24 +00:00
Ekaterina Lebedeva	0990a9b0bd	[#1055 ] blobstor: fix mode metric It used to always show CLOSED after setting shard mode to read-only regardless of actual mode. Now metric represents actual blobstor mode of operations. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-03-29 20:44:47 +00:00
Evgenii Stratonikov	c7a12ca3d8	[#1054 ] network: Optimize IsTLSEnabled() All checks were successful DCO action / DCO (pull_request) Successful in 5m33s Details Vulncheck / Vulncheck (pull_request) Successful in 5m54s Details Build / Build Components (1.21) (pull_request) Successful in 10m3s Details Tests and linters / gopls check (pull_request) Successful in 11m49s Details Build / Build Components (1.20) (pull_request) Successful in 12m47s Details Tests and linters / Staticcheck (pull_request) Successful in 13m21s Details Tests and linters / Lint (pull_request) Successful in 14m21s Details Tests and linters / Tests (1.20) (pull_request) Successful in 16m39s Details Tests and linters / Tests with -race (pull_request) Successful in 16m46s Details Tests and linters / Tests (1.21) (pull_request) Successful in 16m59s Details No big deal, but it is called multiple times in sorting routine, this easily results in 20 allocations per group traversal. ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/network cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ new │ │ sec/op │ sec/op vs base │ AddressTLSEnabled-8 184.6n ± 1% 103.3n ± 6% -44.04% (p=0.000 n=10) │ old │ new │ │ B/op │ B/op vs base │ AddressTLSEnabled-8 704.0 ± 0% 0.0 ± 0% -100.00% (p=0.000 n=10) │ old │ new │ │ allocs/op │ allocs/op vs base │ AddressTLSEnabled-8 1.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-03-21 18:48:35 +03:00
Dmitrii Stepanov	c09c701613	[#1048 ] metabase: Fix drop buckets during resync All checks were successful Vulncheck / Vulncheck (pull_request) Successful in 1m35s Details DCO action / DCO (pull_request) Successful in 2m12s Details Build / Build Components (1.21) (pull_request) Successful in 3m57s Details Build / Build Components (1.20) (pull_request) Successful in 4m45s Details Tests and linters / Staticcheck (pull_request) Successful in 5m32s Details Tests and linters / gopls check (pull_request) Successful in 5m33s Details Tests and linters / Lint (pull_request) Successful in 6m18s Details Tests and linters / Tests (1.20) (pull_request) Successful in 8m8s Details Tests and linters / Tests with -race (pull_request) Successful in 8m14s Details Tests and linters / Tests (1.21) (pull_request) Successful in 8m36s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-19 14:28:31 +03:00
aarifullin	6959e617c4	[#1047 ] object: Set container owner ID property to ape request * Introduce ContainerOwner field in RequestContext. * Set ContainerOwner in aclv2 middleware. * Set PropertyKeyContainerOwnerID for object ape request. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-03-18 15:39:50 +00:00
aarifullin	d7be70e93f	[#1040 ] object: Wrap CheckAPE errors to status errors * All methods should wrap CheckAPE error, if it occurs, to status error. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-03-14 07:34:03 +00:00
Anton Nikiforov	5ee5f1df42	[#976 ] control: Introduce new method `RemoveChainLocalOverridesByTarget` Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-13 15:33:19 +03:00
Anton Nikiforov	2d595ec15f	[#976 ] ape: Update dependency Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-13 14:54:19 +03:00
Anton Nikiforov	7ed07d2dfd	[#976 ] morph: Implement missing methods for `SwitchRPCGuardedActor` Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-13 14:54:19 +03:00
Dmitrii Stepanov	17f5463389	[#1043 ] cli: Add reset evacuation status command Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-13 10:29:45 +00:00
Dmitrii Stepanov	31e2396a5f	[#1043 ] control: Add ResetEvacuationStatus implementation Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-13 10:29:45 +00:00
Dmitrii Stepanov	926cdeb072	[#1043 ] services: Regenerate proto Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-13 10:29:45 +00:00
Airat Arifullin	5c252c9193	[#1039 ] object: Skip APE check for certain request roles All checks were successful DCO action / DCO (pull_request) Successful in 1m31s Details Vulncheck / Vulncheck (pull_request) Successful in 2m52s Details Build / Build Components (1.21) (pull_request) Successful in 3m52s Details Build / Build Components (1.20) (pull_request) Successful in 4m16s Details Tests and linters / gopls check (pull_request) Successful in 11m54s Details Tests and linters / Staticcheck (pull_request) Successful in 12m31s Details Tests and linters / Tests (1.21) (pull_request) Successful in 12m49s Details Tests and linters / Tests (1.20) (pull_request) Successful in 13m8s Details Tests and linters / Tests with -race (pull_request) Successful in 13m14s Details Tests and linters / Lint (pull_request) Successful in 13m31s Details * Skip APE check if a role is Container. * Skip APE check if a role is IR and methods are get-like. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-03-12 16:15:20 +03:00
Anton Nikiforov	3195142d67	[#959 ] writecache: Avoid manipulation with cache in `DEGRADED` mode Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-11 18:35:41 +00:00
Dmitrii Stepanov	d433b49265	[#973 ] node: Resolve perfsprint linter All checks were successful DCO action / DCO (pull_request) Successful in 2m40s Details Vulncheck / Vulncheck (pull_request) Successful in 3m41s Details Build / Build Components (1.20) (pull_request) Successful in 4m27s Details Build / Build Components (1.21) (pull_request) Successful in 5m6s Details Tests and linters / Staticcheck (pull_request) Successful in 6m16s Details Tests and linters / gopls check (pull_request) Successful in 6m23s Details Tests and linters / Lint (pull_request) Successful in 6m48s Details Tests and linters / Tests (1.20) (pull_request) Successful in 9m4s Details Tests and linters / Tests with -race (pull_request) Successful in 9m9s Details Tests and linters / Tests (1.21) (pull_request) Successful in 9m23s Details `fmt.Errorf can be replaced with errors.New` and `fmt.Sprintf can be replaced with string addition` Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-11 17:55:50 +03:00
Dmitrii Stepanov	66a26b7775	[#973 ] node: Resolve revive: unused-parameter linter Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-11 17:11:49 +03:00
Dmitrii Stepanov	0882840bf5	[#634 ] shard: Add writecache inhume tests Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-06 13:12:34 +03:00
Anton Nikiforov	ae5bb87e70	Revert "[#866 ] Use TTL for blobovnicza tree cache" All checks were successful DCO action / DCO (pull_request) Successful in 2m7s Details Vulncheck / Vulncheck (pull_request) Successful in 3m7s Details Tests and linters / Staticcheck (pull_request) Successful in 5m12s Details Tests and linters / Lint (pull_request) Successful in 6m23s Details Tests and linters / Tests with -race (pull_request) Successful in 7m11s Details Tests and linters / gopls check (pull_request) Successful in 7m27s Details Tests and linters / Tests (1.20) (pull_request) Successful in 8m5s Details Tests and linters / Tests (1.21) (pull_request) Successful in 8m22s Details Build / Build Components (1.21) (pull_request) Successful in 1m41s Details Build / Build Components (1.20) (pull_request) Successful in 1m56s Details This reverts commit `d9cbb16bd3`. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-03-01 19:29:33 +03:00
Dmitrii Stepanov	d6534fd755	[#1016 ] frostfs-node: Fix gopls issues Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-03-01 12:13:43 +03:00
Airat Arifullin	93bf9acbc2	[#898 ] control: Remove removed flag from RemoveChainLocalOverrideResponse * Remove removed flag in service.proto for RemoveChainLocalOverrideResponse. * Regenerate control API. * Return error only if RemoveOverride returns non-NotFound code. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-28 19:07:07 +00:00
Airat Arifullin	75a1a95c2c	[#986 ] tree: Skip ACL checks if basicACL mask is unset Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-28 19:05:57 +00:00
Airat Arifullin	b1d171c261	[#986 ] container: Interpret APE NoRuleFound as request deny * If APE check returns NoRuleFound, then it is taken for request deny. * Add more unit-test for ape container middleware. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-28 19:05:57 +00:00
Airat Arifullin	7cc368e188	[#986 ] object: Introduce soft ape checks * Soft APE check means that APE should allow request even it gets status NoRuleFound for a request. Otherwise, it is interpreted as Deny. * Soft APE check is performed if basic ACL mask is not set. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-28 19:05:57 +00:00
Dmitrii Stepanov	918613546f	[#1008 ] metabase: Do not update storageID on put All checks were successful DCO action / DCO (pull_request) Successful in 4m56s Details Build / Build Components (1.21) (pull_request) Successful in 7m52s Details Build / Build Components (1.20) (pull_request) Successful in 8m11s Details Vulncheck / Vulncheck (pull_request) Successful in 8m9s Details Tests and linters / Staticcheck (pull_request) Successful in 9m58s Details Tests and linters / Lint (pull_request) Successful in 10m42s Details Tests and linters / Tests (1.20) (pull_request) Successful in 12m14s Details Tests and linters / Tests with -race (pull_request) Successful in 12m17s Details Tests and linters / Tests (1.21) (pull_request) Successful in 12m34s Details There may be a race condition between put an object and flushing the writecache: 1. Put object to the writecache 2. Writecache flushes object to the blobstore and sets blobstore's storageID 3. Put object to the metabase, set writecache's storageID Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-28 11:01:50 +03:00
Dmitrii Stepanov	2ad433dbcb	[#1005 ] engine: Drop shards weights All checks were successful DCO action / DCO (pull_request) Successful in 4m7s Details Vulncheck / Vulncheck (pull_request) Successful in 4m53s Details Build / Build Components (1.21) (pull_request) Successful in 5m46s Details Build / Build Components (1.20) (pull_request) Successful in 6m21s Details Tests and linters / Staticcheck (pull_request) Successful in 7m45s Details Tests and linters / Lint (pull_request) Successful in 8m44s Details Tests and linters / Tests (1.21) (pull_request) Successful in 13m1s Details Tests and linters / Tests (1.20) (pull_request) Successful in 15m42s Details Tests and linters / Tests with -race (pull_request) Successful in 16m10s Details Unused. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-26 17:25:05 +03:00
Evgenii Stratonikov	4b13b85173	[#1000 ] morph: Fix batch size in TraverseIterator() Initial prefetch size can be arbitrary an restricted only by VM/RPC limits. For TraverseIterator() there is an explicit check on the server-side, though. Introduced in `df055fead5`. Refs #931. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-23 06:24:40 +00:00
Evgenii Stratonikov	e18f0f5178	[#999 ] morph: Use Global scope for proxy contract All checks were successful DCO action / DCO (pull_request) Successful in 2m28s Details Vulncheck / Vulncheck (pull_request) Successful in 2m52s Details Build / Build Components (1.21) (pull_request) Successful in 4m5s Details Build / Build Components (1.20) (pull_request) Successful in 4m12s Details Tests and linters / Lint (pull_request) Successful in 6m10s Details Tests and linters / Staticcheck (pull_request) Successful in 5m59s Details Tests and linters / Tests (1.20) (pull_request) Successful in 8m24s Details Tests and linters / Tests with -race (pull_request) Successful in 8m38s Details Tests and linters / Tests (1.21) (pull_request) Successful in 8m58s Details Proxy contract can now be used as an owner of NNS domains, thus we need it not only to pay for the transaction but also to check domain ownership. CalledByEntry is not enough, because we may register NNS domains owned by proxy indirectly from the container contract. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-21 14:28:42 +03:00
Dmitrii Stepanov	7470c383dd	[#997 ] metabase: Drop toMoveIt bucket All checks were successful DCO action / DCO (pull_request) Successful in 2m9s Details Vulncheck / Vulncheck (pull_request) Successful in 3m3s Details Build / Build Components (1.20) (pull_request) Successful in 3m56s Details Build / Build Components (1.21) (pull_request) Successful in 3m54s Details Tests and linters / Staticcheck (pull_request) Successful in 5m26s Details Tests and linters / Lint (pull_request) Successful in 6m19s Details Tests and linters / Tests (1.20) (pull_request) Successful in 8m6s Details Tests and linters / Tests with -race (pull_request) Successful in 8m20s Details Tests and linters / Tests (1.21) (pull_request) Successful in 8m39s Details It is not used. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-21 10:06:05 +03:00
Evgenii Stratonikov	adf7ebab5b	[#996 ] metabase: Speed up bucket creation Most of the time it exits, e.g. when it is per-container and use on each object PUT. Bbolt implementation first tries to create bucket and then returns it if it exists. Create operation uses cursor and thus is not very lightweight, we can avoid it. ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/metabase cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ new │ │ sec/op │ sec/op vs base │ Put/parallel-8 174.4µ ± 3% 163.3µ ± 3% -6.39% (p=0.000 n=10) Put/sequential-8 263.3µ ± 2% 259.0µ ± 1% -1.64% (p=0.000 n=10) geomean 214.3µ 205.6µ -4.05% │ old │ new │ │ B/op │ B/op vs base │ Put/parallel-8 275.3Ki ± 3% 281.1Ki ± 4% ~ (p=0.063 n=10) Put/sequential-8 413.0Ki ± 2% 426.6Ki ± 2% +3.29% (p=0.003 n=10) geomean 337.2Ki 346.3Ki +2.70% │ old │ new │ │ allocs/op │ allocs/op vs base │ Put/parallel-8 678.0 ± 1% 524.5 ± 2% -22.64% (p=0.000 n=10) Put/sequential-8 1.329k ± 0% 1.183k ± 0% -10.91% (p=0.000 n=10) geomean 949.1 787.9 -16.98% ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-20 15:42:58 +00:00
Evgenii Stratonikov	9f68305c2e	Revert "[#972 ] pilorama: Remove removeDuplicatesInPlace()" This reverts commit `45fd4e4ff1`. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 15:36:01 +00:00
Evgenii Stratonikov	dad56d2e98	Revert "[#972 ] Use min/max builtins" This reverts commit `89784b2e0a`. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 15:36:01 +00:00
Evgenii Stratonikov	7627d08914	Revert "[#972 ] Use require.ElementsMatch() where possible" This reverts commit `6d9707ff1f`. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 15:36:01 +00:00
Evgenii Stratonikov	3359349acb	Revert "[#972 ] Use slices.Sort* when useful" This reverts commit `b871d7a5e8`. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 15:36:01 +00:00
Evgenii Stratonikov	946f2ec2bf	Revert "[#972 ] Drop x/exp/slices dependency" This reverts commit `f3e50772fd`. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 15:36:01 +00:00
Dmitrii Stepanov	13d5cd3e21	[#991 ] logger: Fix journald logger All checks were successful DCO action / DCO (pull_request) Successful in 1m7s Details Vulncheck / Vulncheck (pull_request) Successful in 3m32s Details Build / Build Components (1.22) (pull_request) Successful in 3m52s Details Build / Build Components (1.21) (pull_request) Successful in 3m58s Details Tests and linters / Lint (pull_request) Successful in 5m13s Details Tests and linters / Tests (1.22) (pull_request) Successful in 7m22s Details Tests and linters / Tests (1.21) (pull_request) Successful in 7m34s Details Tests and linters / Tests with -race (pull_request) Successful in 7m32s Details Tests and linters / Staticcheck (pull_request) Successful in 2m22s Details Allow to change logger level. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-19 16:18:13 +03:00
Evgenii Stratonikov	f3e50772fd	[#972 ] Drop x/exp/slices dependency Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 13:13:09 +00:00
Evgenii Stratonikov	b871d7a5e8	[#972 ] Use slices.Sort* when useful Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 13:13:09 +00:00
Evgenii Stratonikov	6d9707ff1f	[#972 ] Use require.ElementsMatch() where possible Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 13:13:09 +00:00
Evgenii Stratonikov	89784b2e0a	[#972 ] Use min/max builtins Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 13:13:09 +00:00
Evgenii Stratonikov	45fd4e4ff1	[#972 ] pilorama: Remove removeDuplicatesInPlace() Also, check that slices.CompareFunc() indeed passes all the tests before removal. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 13:13:09 +00:00
Evgenii Stratonikov	15fc5bac26	[#972 ] keyer: Use UncompressedBytes() for marshaling elliptic.Marshal() becomes deprecated in go1.21 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-19 13:13:09 +00:00
Dmitrii Stepanov	2680192ba0	[#988 ] objectSvc: Fix `SetMarshalData` for PutSingle All checks were successful DCO action / DCO (pull_request) Successful in 5m10s Details Vulncheck / Vulncheck (pull_request) Successful in 5m24s Details Build / Build Components (1.21) (pull_request) Successful in 7m40s Details Build / Build Components (1.20) (pull_request) Successful in 7m50s Details Tests and linters / Staticcheck (pull_request) Successful in 7m52s Details Tests and linters / Lint (pull_request) Successful in 9m8s Details Tests and linters / Tests with -race (pull_request) Successful in 10m54s Details Tests and linters / Tests (1.20) (pull_request) Successful in 11m12s Details Tests and linters / Tests (1.21) (pull_request) Successful in 11m7s Details After api-go update it is required to pass marshal data to `SetMarshalData`. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-15 17:21:08 +03:00
Anton Nikiforov	05b5f5ca85	[#959 ] writecache: Fix panic on `Get` when it is not initialized All checks were successful Vulncheck / Vulncheck (pull_request) Successful in 2m41s Details DCO action / DCO (pull_request) Successful in 2m43s Details Tests and linters / Staticcheck (pull_request) Successful in 3m55s Details Build / Build Components (1.21) (pull_request) Successful in 3m45s Details Build / Build Components (1.20) (pull_request) Successful in 4m19s Details Tests and linters / Lint (pull_request) Successful in 5m30s Details Tests and linters / Tests (1.20) (pull_request) Successful in 7m57s Details Tests and linters / Tests (1.21) (pull_request) Successful in 8m22s Details Tests and linters / Tests with -race (pull_request) Successful in 8m39s Details Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-14 16:10:33 +03:00
Anton Nikiforov	2429508ac5	[#959 ] shard: Skip rebuild in `DEGRADED` mode Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-14 15:39:28 +03:00
Anton Nikiforov	0bd030507e	[#948 ] metrics: Set actual value for `shard_id` after restart All checks were successful DCO action / DCO (pull_request) Successful in 2m26s Details Vulncheck / Vulncheck (pull_request) Successful in 3m56s Details Tests and linters / Staticcheck (pull_request) Successful in 4m18s Details Build / Build Components (1.21) (pull_request) Successful in 4m11s Details Build / Build Components (1.20) (pull_request) Successful in 4m19s Details Tests and linters / Lint (pull_request) Successful in 6m3s Details Tests and linters / Tests (1.20) (pull_request) Successful in 8m7s Details Tests and linters / Tests (1.21) (pull_request) Successful in 8m26s Details Tests and linters / Tests with -race (pull_request) Successful in 9m20s Details Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-13 09:43:21 +03:00
Anton Nikiforov	6a5769d1da	[#948 ] Fix `gofumpt` issue Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-13 09:40:46 +03:00
Dmitrii Stepanov	962e5a9c19	[#736 ] logger: Add journald support Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-12 06:17:22 +00:00
Evgenii Stratonikov	b36a453238	[#970 ] fstree: Add build tag to enable generic version on linux Unless tested, generic version can start gaining bugs. With a separate build tag we can have the best of both worlds: 1. Use optimized implementation for linux by default. 2. Run tests or benchmarks for both. Note that they are not actually run automatically now, but this is at leas possible. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-09 16:12:11 +00:00
Evgenii Stratonikov	abd502215f	[#970 ] fstree: Move file locking to the generic writer It is not a part of FSTree itself, but rather a way to solve concurrent counter update on non-linux implementations. New linux implementations is pretty simple: link fails when the file exists, unlink fails when the file doesn't exist. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-09 16:12:11 +00:00
Evgenii Stratonikov	fb74524ac7	[#970 ] fstree: Move delete implementation to a separate file Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-09 16:12:11 +00:00
Evgenii Stratonikov	7f692409cf	[#970 ] fstree: Handle unsupported O_TMPFILE Metabase test relied on this behaviour, so fix the test too. Cherry-picking was hard and did too many conflicts, here is an original PR: https://github.com/nspcc-dev/neofs-node/pull/2624 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-09 16:12:11 +00:00
Roman Khimov	fc31b9c947	[#970 ] fstree: Add linux-specific file writer using O_TMPFILE O_TMPFILE is implemented for all modern FSes and it's much easier and safer to use. If application crashes in the middle of writing this file would be gone and won't leave any garbage. Notice that this implementation makes a different choice wrt EEXIST handling, generic one always overwrites, while this one keeps the old data. There is no real performance difference. SSD (slow&old), XFS, Core i7-8565U: Sync ``` name old time/op new time/op delta Put/size=1024,thread=1/fstree-8 1.74ms ± 3% 0.06ms ± 7% -96.31% (p=0.000 n=10+10) Put/size=1024,thread=20/fstree-8 10.0ms ±41% 1.1ms ±18% -88.95% (p=0.000 n=9+10) Put/size=1024,thread=100/fstree-8 32.3ms ±60% 6.5ms ±14% -79.97% (p=0.000 n=10+10) Put/size=1048576,thread=1/fstree-8 17.8ms ±90% 3.4ms ±70% -81.08% (p=0.000 n=10+10) Put/size=1048576,thread=20/fstree-8 103ms ±174% 112ms ±158% ~ (p=0.971 n=10+10) Put/size=1048576,thread=100/fstree-8 949ms ±78% 583ms ±132% ~ (p=0.089 n=10+10) name old alloc/op new alloc/op delta Put/size=1024,thread=1/fstree-8 3.17kB ± 1% 1.96kB ± 0% -38.09% (p=0.000 n=10+10) Put/size=1024,thread=20/fstree-8 59.6kB ± 1% 39.2kB ± 1% -34.30% (p=0.000 n=8+10) Put/size=1024,thread=100/fstree-8 299kB ± 0% 198kB ± 0% -33.90% (p=0.000 n=7+9) Put/size=1048576,thread=1/fstree-8 3.38kB ± 1% 2.36kB ± 1% -30.22% (p=0.000 n=10+10) Put/size=1048576,thread=20/fstree-8 65.7kB ± 4% 47.7kB ± 6% -27.27% (p=0.000 n=10+10) Put/size=1048576,thread=100/fstree-8 351kB ± 8% 245kB ± 8% -30.22% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Put/size=1024,thread=1/fstree-8 30.3 ± 2% 21.0 ± 0% -30.69% (p=0.000 n=10+10) Put/size=1024,thread=20/fstree-8 554 ± 1% 413 ± 0% -25.35% (p=0.000 n=8+10) Put/size=1024,thread=100/fstree-8 2.77k ± 0% 2.07k ± 0% -25.27% (p=0.000 n=7+10) Put/size=1048576,thread=1/fstree-8 32.0 ± 0% 25.0 ± 0% -21.88% (p=0.000 n=9+8) Put/size=1048576,thread=20/fstree-8 609 ± 5% 494 ± 6% -18.93% (p=0.000 n=10+10) Put/size=1048576,thread=100/fstree-8 3.25k ± 9% 2.50k ± 8% -23.21% (p=0.000 n=10+10) ``` No sync ``` name old time/op new time/op delta Put/size=1024,thread=1/fstree-8 71.3µs ±10% 59.8µs ±10% -16.21% (p=0.000 n=10+10) Put/size=1024,thread=20/fstree-8 1.43ms ± 6% 1.22ms ±13% -14.53% (p=0.000 n=10+10) Put/size=1024,thread=100/fstree-8 8.12ms ± 3% 6.36ms ± 2% -21.67% (p=0.000 n=8+9) Put/size=1048576,thread=1/fstree-8 1.88ms ±70% 1.61ms ±78% ~ (p=0.393 n=10+10) Put/size=1048576,thread=20/fstree-8 32.7ms ±28% 34.2ms ±112% ~ (p=0.968 n=9+10) Put/size=1048576,thread=100/fstree-8 262ms ±56% 226ms ±34% ~ (p=0.447 n=10+9) name old alloc/op new alloc/op delta Put/size=1024,thread=1/fstree-8 2.89kB ± 0% 1.96kB ± 0% -32.28% (p=0.000 n=10+10) Put/size=1024,thread=20/fstree-8 58.2kB ± 0% 39.5kB ± 0% -32.09% (p=0.000 n=8+8) Put/size=1024,thread=100/fstree-8 291kB ± 0% 198kB ± 0% -32.19% (p=0.000 n=9+9) Put/size=1048576,thread=1/fstree-8 3.05kB ± 1% 2.13kB ± 1% -30.16% (p=0.000 n=10+9) Put/size=1048576,thread=20/fstree-8 62.6kB ± 0% 44.3kB ± 0% -29.23% (p=0.000 n=9+9) Put/size=1048576,thread=100/fstree-8 302kB ± 0% 210kB ± 1% -30.39% (p=0.000 n=9+9) name old allocs/op new allocs/op delta Put/size=1024,thread=1/fstree-8 27.0 ± 0% 21.0 ± 0% -22.22% (p=0.000 n=10+10) Put/size=1024,thread=20/fstree-8 539 ± 0% 415 ± 0% -22.98% (p=0.000 n=10+10) Put/size=1024,thread=100/fstree-8 2.69k ± 0% 2.07k ± 0% -23.09% (p=0.000 n=9+9) Put/size=1048576,thread=1/fstree-8 28.0 ± 0% 22.3 ± 3% -20.36% (p=0.000 n=8+10) Put/size=1048576,thread=20/fstree-8 577 ± 0% 458 ± 0% -20.72% (p=0.000 n=9+9) Put/size=1048576,thread=100/fstree-8 2.76k ± 0% 2.15k ± 0% -22.05% (p=0.000 n=9+8) ``` HDD (LVM), ext4, Ryzen 5 1600: Sync ``` │ fs.sync-generic │ fs.sync-linux │ │ sec/op │ sec/op vs base │ Put/size=1024,thread=1/fstree-12 34.70m ± 19% 33.59m ± 16% ~ (p=0.529 n=10) Put/size=1024,thread=20/fstree-12 188.8m ± 8% 189.2m ± 16% ~ (p=0.739 n=10) Put/size=1024,thread=100/fstree-12 264.8m ± 22% 273.6m ± 28% ~ (p=0.353 n=10) Put/size=1048576,thread=1/fstree-12 54.90m ± 14% 47.08m ± 18% ~ (p=0.063 n=10) Put/size=1048576,thread=20/fstree-12 244.1m ± 14% 220.4m ± 22% ~ (p=0.579 n=10) Put/size=1048576,thread=100/fstree-12 847.2m ± 5% 893.6m ± 3% +5.48% (p=0.000 n=10) geomean 164.3m 158.9m -3.29% │ fs.sync-generic │ fs.sync-linux │ │ B/op │ B/op vs base │ Put/size=1024,thread=1/fstree-12 3.375Ki ± 1% 2.471Ki ± 1% -26.80% (p=0.000 n=10) Put/size=1024,thread=20/fstree-12 66.62Ki ± 6% 49.21Ki ± 6% -26.15% (p=0.000 n=10) Put/size=1024,thread=100/fstree-12 319.2Ki ± 1% 230.9Ki ± 2% -27.64% (p=0.000 n=10) Put/size=1048576,thread=1/fstree-12 3.457Ki ± 1% 2.559Ki ± 1% -25.97% (p=0.000 n=10) Put/size=1048576,thread=20/fstree-12 66.91Ki ± 1% 49.16Ki ± 1% -26.52% (p=0.000 n=10) Put/size=1048576,thread=100/fstree-12 338.8Ki ± 2% 252.3Ki ± 3% -25.54% (p=0.000 n=10) geomean 42.17Ki 31.02Ki -26.44% │ fs.sync-generic │ fs.sync-linux │ │ allocs/op │ allocs/op vs base │ Put/size=1024,thread=1/fstree-12 33.00 ± 0% 27.00 ± 0% -18.18% (p=0.000 n=10) Put/size=1024,thread=20/fstree-12 639.5 ± 1% 519.0 ± 2% -18.84% (p=0.000 n=10) Put/size=1024,thread=100/fstree-12 3.059k ± 1% 2.478k ± 2% -18.99% (p=0.000 n=10) Put/size=1048576,thread=1/fstree-12 33.50 ± 1% 28.00 ± 4% -16.42% (p=0.000 n=10) Put/size=1048576,thread=20/fstree-12 638.5 ± 1% 520.0 ± 1% -18.56% (p=0.000 n=10) Put/size=1048576,thread=100/fstree-12 3.209k ± 2% 2.655k ± 2% -17.28% (p=0.000 n=10) geomean 405.3 332.1 -18.05% ``` No sync ``` │ fs.nosync-generic │ fs.nosync-linux │ │ sec/op │ sec/op vs base │ Put/size=1024,thread=1/fstree-12 148.2µ ± 20% 136.6µ ± 19% -7.89% (p=0.029 n=10) Put/size=1024,thread=20/fstree-12 1.140m ± 26% 1.364m ± 16% ~ (p=0.143 n=10) Put/size=1024,thread=100/fstree-12 11.93m ± 68% 26.89m ± 62% ~ (p=0.123 n=10) Put/size=1048576,thread=1/fstree-12 1.302m ± 3% 1.287m ± 5% ~ (p=0.481 n=10) Put/size=1048576,thread=20/fstree-12 77.52m ± 8% 74.07m ± 7% ~ (p=0.278 n=10+9) Put/size=1048576,thread=100/fstree-12 226.1m ± ∞ ¹ geomean 5.986m 3.434m +18.60% ² ¹ need >= 6 samples for confidence interval at level 0.95 ² benchmark set differs from baseline; geomeans may not be comparable │ fs.nosync-generic │ fs.nosync-linux │ │ B/op │ B/op vs base │ Put/size=1024,thread=1/fstree-12 2.879Ki ± 0% 1.972Ki ± 0% -31.51% (p=0.000 n=10) Put/size=1024,thread=20/fstree-12 55.94Ki ± 1% 37.90Ki ± 1% -32.25% (p=0.000 n=10) Put/size=1024,thread=100/fstree-12 272.6Ki ± 0% 182.1Ki ± 9% -33.21% (p=0.000 n=10) Put/size=1048576,thread=1/fstree-12 3.158Ki ± 0% 2.259Ki ± 0% -28.46% (p=0.000 n=10) Put/size=1048576,thread=20/fstree-12 58.87Ki ± 0% 41.03Ki ± 0% -30.30% (p=0.000 n=10+9) Put/size=1048576,thread=100/fstree-12 299.8Ki ± ∞ ¹ geomean 36.71Ki 16.60Ki -31.17% ² ¹ need >= 6 samples for confidence interval at level 0.95 ² benchmark set differs from baseline; geomeans may not be comparable │ fs.nosync-generic │ fs.nosync-linux │ │ allocs/op │ allocs/op vs base │ Put/size=1024,thread=1/fstree-12 28.00 ± 0% 22.00 ± 0% -21.43% (p=0.000 n=10) Put/size=1024,thread=20/fstree-12 530.0 ± 0% 407.5 ± 1% -23.11% (p=0.000 n=10) Put/size=1024,thread=100/fstree-12 2.567k ± 0% 1.956k ± 9% -23.77% (p=0.000 n=10) Put/size=1048576,thread=1/fstree-12 30.00 ± 0% 24.00 ± 0% -20.00% (p=0.000 n=10) Put/size=1048576,thread=20/fstree-12 553.5 ± 0% 434.0 ± 0% -21.59% (n=10+9) Put/size=1048576,thread=100/fstree-12 2.803k ± ∞ ¹ geomean 347.9 178.8 -21.99% ² ¹ need >= 6 samples for confidence interval at level 0.95 ² benchmark set differs from baseline; geomeans may not be comparable ``` Signed-off-by: Roman Khimov <roman@nspcc.ru> Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-09 16:12:11 +00:00
Evgenii Stratonikov	ff488b53a1	[#970 ] fstree: Move write functions to a separate file Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-09 16:12:11 +00:00
Evgenii Stratonikov	9a622a750d	[#970 ] fstree: Move temporary path handling in a separate function Allow to easier test different implementations. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-09 16:12:11 +00:00
Anton Nikiforov	d19ade23c8	[#959 ] node: Set mode to shard's components when open it Avoid opening database for `metabase` and `cache` in `Degraded` mode. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-09 14:04:01 +00:00
Dmitrii Stepanov	db67c21d55	[#947 ] engine: Evacuate trees to remote nodes Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-09 11:33:15 +03:00
Dmitrii Stepanov	728150d1d2	[#947 ] engine: Evacuate trees to local shards Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-09 11:33:15 +03:00
Dmitrii Stepanov	15d853ea22	[#947 ] controlSvc: Return tree evacuation stat Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-09 11:20:39 +03:00
Dmitrii Stepanov	b3f3505ada	[#947 ] cli: Allow to specify evacuation scope It may be required to evacuate only objects or only tree or all, so now it spossible to specify. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-09 11:20:38 +03:00
Dmitrii Stepanov	a6eb66bf9c	[#947 ] evacuate: Refactor evacuate parameters Drop methods to make it easier to extend. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-09 11:20:38 +03:00
Dmitrii Stepanov	8e2a0611f4	[#947 ] tree: Add method to list all trees Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-09 11:20:38 +03:00
Evgenii Stratonikov	edbe06e07e	[#956 ] policer/test: Reuse testPool helper Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-08 15:49:43 +00:00
Evgenii Stratonikov	cbfeb72466	[#956 ] policer: Remove WithMaxCapacity option We already provide the pool and this argument is used only for preallocation. No functional changes. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-08 15:49:43 +00:00
Dmitrii Stepanov	cfc5ce7853	[#964 ] metabase: Drop GC marks if object not found GC inhumes expired locks and tombstones on all the shards. So it could be GC mark without object. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-08 07:54:39 +00:00
Evgenii Stratonikov	c3fa902780	[#969 ] policer: Restrict the number of remembered errors All checks were successful DCO action / DCO (pull_request) Successful in 4m23s Details Vulncheck / Vulncheck (pull_request) Successful in 4m57s Details Build / Build Components (1.21) (pull_request) Successful in 6m3s Details Build / Build Components (1.20) (pull_request) Successful in 9m2s Details Tests and linters / Staticcheck (pull_request) Successful in 10m4s Details Tests and linters / Lint (pull_request) Successful in 10m51s Details Tests and linters / Tests (1.21) (pull_request) Successful in 10m44s Details Tests and linters / Tests (1.20) (pull_request) Successful in 10m56s Details Tests and linters / Tests with -race (pull_request) Successful in 10m57s Details Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-08 10:10:41 +03:00
Evgenii Stratonikov	6010dfdf3d	[#969 ] policer: Make error skip thread-safe Introduces in `afd2ba9a66`. Refs #914 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-08 10:10:41 +03:00
Evgenii Stratonikov	a6c9a337cd	[#965 ] morph: Get rid of container.List invocations ContainersOf() is better in almost every aspect, besides creating a session when the containers number is between 1024 and 2048 (prefetch script does limited unwrapping). Making List() private helps to ensure it is no longer used and can be safely removed in future. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-07 08:56:27 +00:00
Dmitrii Stepanov	9ba48c582d	[#917 ] engine: Allow to detach shards Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-06 14:49:47 +03:00
Dmitrii Stepanov	4358d3c423	[#917 ] controlSvc: Add DetachShards handler Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-02-06 14:47:52 +03:00
Ekaterina Lebedeva	afd2ba9a66	[#110 ] Add check for repeated error log in policer All checks were successful DCO action / DCO (pull_request) Successful in 2m55s Details Vulncheck / Vulncheck (pull_request) Successful in 3m22s Details Tests and linters / Staticcheck (pull_request) Successful in 4m10s Details Build / Build Components (1.20) (pull_request) Successful in 4m58s Details Build / Build Components (1.21) (pull_request) Successful in 4m54s Details Tests and linters / Lint (pull_request) Successful in 5m47s Details Tests and linters / Tests (1.20) (pull_request) Successful in 6m45s Details Tests and linters / Tests (1.21) (pull_request) Successful in 6m58s Details Tests and linters / Tests with -race (pull_request) Successful in 7m42s Details processObject() returns 3 types of errors: container not found errors, could not get container error and placement vector building error. Every error will occur for all objects in container simultaneously, so we can log each error once and safely ignore the rest. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-02-06 00:56:41 +03:00
Evgenii Stratonikov	602ee11123	[#934 ] containersvc: Marhal public key in short format for APE Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-02-02 17:51:38 +00:00
Airat Arifullin	c1a5b831b6	[#955 ] chainbase: Fix rule chain unmarshalling All checks were successful DCO action / DCO (pull_request) Successful in 4m41s Details Vulncheck / Vulncheck (pull_request) Successful in 4m46s Details Tests and linters / Staticcheck (pull_request) Successful in 6m30s Details Tests and linters / Lint (pull_request) Successful in 7m36s Details Build / Build Components (1.21) (pull_request) Successful in 8m57s Details Build / Build Components (1.20) (pull_request) Successful in 9m8s Details Tests and linters / Tests (1.21) (pull_request) Successful in 10m32s Details Tests and linters / Tests (1.20) (pull_request) Successful in 10m42s Details Tests and linters / Tests with -race (pull_request) Successful in 11m23s Details * Use correct way DecodeBytes instead unmarshalling by json. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-02 20:28:04 +03:00
Anton Nikiforov	9916598dfb	[#922 ] control: Extend api with `ListOverrideDefinedTargets` Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-02 12:09:51 +00:00
Anton Nikiforov	95e15f499f	[#922 ] Update files generated by protoc Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-02 12:09:51 +00:00
Anton Nikiforov	2cb04379a4	[#922 ] go.mod: Update APE Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-02 12:09:51 +00:00
Airat Arifullin	a5446bc17d	[#952 ] object: Pass namespace within context in ACL service All checks were successful DCO action / DCO (pull_request) Successful in 6m23s Details Vulncheck / Vulncheck (pull_request) Successful in 7m3s Details Build / Build Components (1.21) (pull_request) Successful in 8m21s Details Build / Build Components (1.20) (pull_request) Successful in 8m31s Details Tests and linters / Staticcheck (pull_request) Successful in 11m1s Details Tests and linters / Lint (pull_request) Successful in 11m26s Details Tests and linters / Tests (1.20) (pull_request) Successful in 12m51s Details Tests and linters / Tests (1.21) (pull_request) Successful in 13m14s Details Tests and linters / Tests with -race (pull_request) Successful in 13m31s Details Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-02 14:48:11 +03:00
Anton Nikiforov	d0eadf7ea2	[#799 ] engine: Skip put when object removed from shard Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-01 17:49:22 +00:00
Anton Nikiforov	6534252c22	[#799 ] policer: Refactor method `processNodes` Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-02-01 17:49:22 +00:00
Airat Arifullin	5be2af881a	[#934 ] container: Make container APE middleware read namespaces * Those methods that can access already existing containers and thus can get container properties should read namespace from Zone property. If Zone is not set, take a namespace for root. * Otherwise, define namespaces by owner ID via frostfs-id contract. * Improve unit-tests, consider more cases. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-01 17:38:24 +00:00
Airat Arifullin	4352bd0e8e	[#934 ] ape: Transform empty namespace within chainbase Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-02-01 17:38:24 +00:00
Anton Nikiforov	483a67b170	[#937 ] ape: Validate chain resource name All checks were successful DCO action / DCO (pull_request) Successful in 2m13s Details Build / Build Components (1.20) (pull_request) Successful in 3m38s Details Vulncheck / Vulncheck (pull_request) Successful in 3m13s Details Tests and linters / Staticcheck (pull_request) Successful in 6m15s Details Build / Build Components (1.21) (pull_request) Successful in 7m11s Details Tests and linters / Lint (pull_request) Successful in 9m35s Details Tests and linters / Tests (1.21) (pull_request) Successful in 13m56s Details Tests and linters / Tests (1.20) (pull_request) Successful in 14m14s Details Tests and linters / Tests with -race (pull_request) Successful in 6m2s Details Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-01-31 11:34:35 +03:00
Anton Nikiforov	e3573de6db	[#930 ] gc: Stop internal activity by context Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-01-31 08:30:34 +00:00
Anton Nikiforov	c441296592	[#930 ] policer: Release task pool when context cancelled Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-01-31 08:30:34 +00:00
Dmitrii Stepanov	675eec91f3	[#938 ] shard: Update only changed counters All checks were successful DCO action / DCO (pull_request) Successful in 5m40s Details Vulncheck / Vulncheck (pull_request) Successful in 5m33s Details Tests and linters / Staticcheck (pull_request) Successful in 7m12s Details Build / Build Components (1.21) (pull_request) Successful in 7m25s Details Build / Build Components (1.20) (pull_request) Successful in 7m29s Details Tests and linters / Lint (pull_request) Successful in 8m26s Details Tests and linters / Tests (1.20) (pull_request) Successful in 9m48s Details Tests and linters / Tests with -race (pull_request) Successful in 9m56s Details Tests and linters / Tests (1.21) (pull_request) Successful in 10m10s Details If metric value hasn't changed, but we update metric, then non existed metric will apear with zero value. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-30 12:37:48 +03:00
Dmitrii Stepanov	c681354afd	[#938 ] engine: Fix container count removal Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-30 12:37:48 +03:00
Evgenii Stratonikov	df055fead5	[#931 ] morph: Provide batch size for container listing explicitly Besides VM stack item limit we also have restrictions on the size of JSON for a stackitem, which is 128k. This limit is much harder to calculate, because JSON representation includes type and the encoding is different for different items. Thus is makes no sense to invent our own default, so use the one provided by neo-go. But for container listing we know exactly what we process, so use big enough value, which is tested. Introduced in `be8607a1f6`. Refs #902 Refs https://github.com/nspcc-dev/neo-go/blob/v0.105.0/pkg/vm/stackitem/json.go#L353 Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-01-29 14:00:11 +00:00
Evgenii Stratonikov	6e2cc32768	[#681 ] objsvc: Validate session token owner for local sessions Previously, the check was in place only when session token was missing. Format validator checks are applied only to fully-prepared object, so this lead to the following situation: 1. Object is put locally with malformed token, because there are no checks. 2. Object cannot be replicated, because the token is malformed. This is now fixed and token check is done before any payload receival. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-01-26 08:52:29 +00:00
Anton Nikiforov	b6fc3321c5	[#876 ] Fix linters Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-01-25 20:26:13 +03:00
Airat Arifullin	1fe7736d92	[#925 ] morph: Introduce switch rpc guarded rpc actor * Introduce switch rpc guarded rpc actor in morph client to prevent using invalidated rpc actor when RPC switch happens. * Initialize NewContractStorage with SwitchRPCGuardedActor. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-25 13:24:13 +00:00
Dmitrii Stepanov	5ed330e436	[#927 ] metabase: Delete GC marks All checks were successful DCO action / DCO (pull_request) Successful in 6m34s Details Vulncheck / Vulncheck (pull_request) Successful in 7m19s Details Build / Build Components (1.20) (pull_request) Successful in 7m48s Details Build / Build Components (1.21) (pull_request) Successful in 9m39s Details Tests and linters / Staticcheck (pull_request) Successful in 10m36s Details Tests and linters / Lint (pull_request) Successful in 11m26s Details Tests and linters / Tests (1.20) (pull_request) Successful in 12m57s Details Tests and linters / Tests (1.21) (pull_request) Successful in 13m8s Details Tests and linters / Tests with -race (pull_request) Successful in 13m7s Details `key` is changed inside `db.get`, so encode address again after get. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-24 18:51:16 +03:00
Dmitrii Stepanov	931a5e9aaf	[#918 ] engine: Move shard to degraded mode if metabase open failed All checks were successful DCO action / DCO (pull_request) Successful in 1m55s Details Build / Build Components (1.20) (pull_request) Successful in 2m14s Details Vulncheck / Vulncheck (pull_request) Successful in 2m15s Details Build / Build Components (1.21) (pull_request) Successful in 4m6s Details Tests and linters / Tests (1.21) (pull_request) Successful in 5m58s Details Tests and linters / Tests (1.20) (pull_request) Successful in 6m24s Details Tests and linters / Staticcheck (pull_request) Successful in 6m11s Details Tests and linters / Lint (pull_request) Successful in 6m37s Details Tests and linters / Tests with -race (pull_request) Successful in 6m35s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-23 11:16:40 +03:00
Airat Arifullin	f2f3294fc3	[#919 ] ape: Improve error messages in ape service * Wrap all APE middleware errors in apeErr that makes errors more explicit with status AccessDenied. * Use denyingRuleErr for denying status from chain router. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-23 08:11:24 +00:00
Dmitrii Stepanov	f526f49995	[#874 ] engine: Check object existance concurrently All checks were successful DCO action / DCO (pull_request) Successful in 1m36s Details Vulncheck / Vulncheck (pull_request) Successful in 2m33s Details Build / Build Components (1.21) (pull_request) Successful in 3m9s Details Build / Build Components (1.20) (pull_request) Successful in 4m49s Details Tests and linters / Tests (1.21) (pull_request) Successful in 6m5s Details Tests and linters / Staticcheck (pull_request) Successful in 6m6s Details Tests and linters / Tests (1.20) (pull_request) Successful in 6m39s Details Tests and linters / Lint (pull_request) Successful in 7m0s Details Tests and linters / Tests with -race (pull_request) Successful in 6m46s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-23 09:28:29 +03:00
Dmitrii Stepanov	f5160b27fc	[#920 ] tests: Fix data races All checks were successful DCO action / DCO (pull_request) Successful in 2m2s Details Build / Build Components (1.21) (pull_request) Successful in 3m35s Details Vulncheck / Vulncheck (pull_request) Successful in 3m3s Details Tests and linters / Staticcheck (pull_request) Successful in 5m23s Details Tests and linters / Tests (1.20) (pull_request) Successful in 7m33s Details Tests and linters / Tests (1.21) (pull_request) Successful in 7m41s Details Tests and linters / Tests with -race (pull_request) Successful in 7m47s Details Tests and linters / Lint (pull_request) Successful in 8m12s Details Build / Build Components (1.20) (pull_request) Successful in 1m32s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-19 14:06:05 +03:00
Airat Arifullin	be33070550	[#910 ] control: Take empty namespace for 'root' Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-19 06:22:10 +00:00
Dmitrii Stepanov	63d3ed1ad8	[#904 ] tests: Close test engine after test All checks were successful DCO action / DCO (pull_request) Successful in 2m7s Details Vulncheck / Vulncheck (pull_request) Successful in 2m37s Details Build / Build Components (1.20) (pull_request) Successful in 4m0s Details Build / Build Components (1.21) (pull_request) Successful in 3m56s Details Tests and linters / Tests (1.21) (pull_request) Successful in 6m25s Details Tests and linters / Staticcheck (pull_request) Successful in 6m22s Details Tests and linters / Tests (1.20) (pull_request) Successful in 6m43s Details Tests and linters / Tests with -race (pull_request) Successful in 6m47s Details Tests and linters / Lint (pull_request) Successful in 7m5s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-17 19:04:39 +03:00
Dmitrii Stepanov	57171907e3	[#904 ] metabase: Return if object was actuall inserted This requires to count metrics properly. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-17 19:04:39 +03:00
Dmitrii Stepanov	c1a80235db	[#904 ] metabase: Log Inhume operation It will be very useful for troubleshooting. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-17 18:42:52 +03:00
Airat Arifullin	96b020626f	[#915 ] ape: Fix method name in getStreamBasicChecker All checks were successful DCO action / DCO (pull_request) Successful in 2m2s Details Build / Build Components (1.21) (pull_request) Successful in 2m17s Details Build / Build Components (1.20) (pull_request) Successful in 3m2s Details Vulncheck / Vulncheck (pull_request) Successful in 2m39s Details Tests and linters / Tests (1.21) (pull_request) Successful in 5m54s Details Tests and linters / Staticcheck (pull_request) Successful in 5m49s Details Tests and linters / Tests (1.20) (pull_request) Successful in 6m11s Details Tests and linters / Lint (pull_request) Successful in 6m44s Details Tests and linters / Tests with -race (pull_request) Successful in 6m32s Details * Replace incorrect MethodGetContainer by MethodGetObject constant. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-16 23:52:37 +03:00
Airat Arifullin	c8baf76fae	[#872 ] object: Introduce APE middlewar for object service All checks were successful DCO action / DCO (pull_request) Successful in 2m4s Details Vulncheck / Vulncheck (pull_request) Successful in 3m12s Details Build / Build Components (1.21) (pull_request) Successful in 4m1s Details Build / Build Components (1.20) (pull_request) Successful in 4m13s Details Tests and linters / Staticcheck (pull_request) Successful in 4m3s Details Tests and linters / Lint (pull_request) Successful in 8m7s Details Tests and linters / Tests (1.20) (pull_request) Successful in 8m14s Details Tests and linters / Tests (1.21) (pull_request) Successful in 8m18s Details Tests and linters / Tests with -race (pull_request) Successful in 8m24s Details Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-12 18:41:35 +03:00
Dmitrii Stepanov	52ffa9f164	[#891 ] getSvc: Refactor Get service V2 creation All checks were successful DCO action / DCO (pull_request) Successful in 2m46s Details Vulncheck / Vulncheck (pull_request) Successful in 2m53s Details Build / Build Components (1.20) (pull_request) Successful in 4m31s Details Build / Build Components (1.21) (pull_request) Successful in 4m26s Details Tests and linters / Lint (pull_request) Successful in 6m9s Details Tests and linters / Staticcheck (pull_request) Successful in 6m7s Details Tests and linters / Tests (1.20) (pull_request) Successful in 8m27s Details Tests and linters / Tests (1.21) (pull_request) Successful in 9m4s Details Tests and linters / Tests with -race (pull_request) Successful in 9m48s Details Use arguments for mandatory fields. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-12 13:35:38 +03:00
Dmitrii Stepanov	394f086fe2	[#891 ] getSvc: Fix get range hash implementation Get range can perform GET request, so this request must be done from container node to not to get access denied error. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-12 13:35:38 +03:00
Evgenii Stratonikov	be8607a1f6	[#902 ] morph: Avoid creating session in TestInvokeIterator When the number of items to iterate over is less than 2048, there is no need to create a session and consume resources. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-01-12 10:25:56 +00:00
Dmitrii Stepanov	a2ab373a0a	[#895 ] metabase: Do not delete GC mark for virtual objects Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Dmitrii Stepanov	7166e77c2b	[#895 ] test: Add logger to test shard Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Dmitrii Stepanov	47dcfa20f3	[#895 ] test: Use t.Cleanup only for external resources Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Dmitrii Stepanov	836818fb75	[#895 ] test: Use zaptest logger Previous implementation mixes logs from different tests, now logs are separeted by test. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Dmitrii Stepanov	f1b2b8bffa	[#895 ] test: Fix NewLogger arguments list `debug` is always true. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-11 12:32:09 +00:00
Airat Arifullin	a8e52ef7aa	[#898 ] control: Fix codes for returning APE errors Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-11 12:31:31 +00:00
Airat Arifullin	5d982976fd	[#898 ] ape: Fix bug with creating type bucket in chainbase Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-01-11 12:31:31 +00:00
Denis Kirillov	c19396d203	[#885 ] control: Make chain id bytes in grpc Signed-off-by: Denis Kirillov <d.kirillov@yadro.com>	2024-01-11 07:24:22 +00:00
Dmitrii Stepanov	5c0a736a25	[#899 ] containerSvc: Fix invalid session token type Some checks failed DCO action / DCO (pull_request) Successful in 1m23s Details Vulncheck / Vulncheck (pull_request) Successful in 3m29s Details Tests and linters / Tests (1.21) (pull_request) Failing after 3m58s Details Build / Build Components (1.21) (pull_request) Successful in 3m46s Details Build / Build Components (1.20) (pull_request) Successful in 3m52s Details Tests and linters / Lint (pull_request) Successful in 4m48s Details Tests and linters / Staticcheck (pull_request) Successful in 5m5s Details Tests and linters / Tests (1.20) (pull_request) Successful in 7m4s Details Tests and linters / Tests with -race (pull_request) Successful in 8m36s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-10 18:37:54 +03:00
Alexander Chuprov	79bebe4a68	[#884 ] cli: Fix error message for undefined endpoint Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2024-01-10 10:02:16 +00:00
Dmitrii Stepanov	4b8b4da681	[#864 ] engine: Drop container count metric if container removed All checks were successful DCO action / DCO (pull_request) Successful in 1m30s Details Build / Build Components (1.21) (pull_request) Successful in 3m29s Details Build / Build Components (1.20) (pull_request) Successful in 3m53s Details Tests and linters / Lint (pull_request) Successful in 4m31s Details Tests and linters / Tests (1.20) (pull_request) Successful in 5m1s Details Tests and linters / Staticcheck (pull_request) Successful in 4m51s Details Tests and linters / Tests (1.21) (pull_request) Successful in 5m13s Details Tests and linters / Tests with -race (pull_request) Successful in 8m34s Details Vulncheck / Vulncheck (pull_request) Successful in 58s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-10 10:45:32 +03:00
Dmitrii Stepanov	d75e7e9a21	[#864 ] engine: Drop container size metric if container deleted Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-10 10:44:54 +03:00
Dmitrii Stepanov	dfd62ca6b1	[#864 ] metabase: Refactor delete/inhume Available -> Logic, Raw -> Phy for delete/inhume results. Use single counter instead of vectors. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-01-09 09:59:42 +03:00
Evgenii Stratonikov	225fe2d4d5	[#894 ] blobovniczatree: Speedup rebuild test Down from 3s to 300ms. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-12-29 16:28:54 +00:00
Evgenii Stratonikov	530249e3bd	[#893 ] go.mod: Update neo-go Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-12-29 15:03:47 +00:00
Dmitrii Stepanov	581887148a	[#569 ] cli: Add `control shards writecache seal` command All checks were successful DCO action / DCO (pull_request) Successful in 2m46s Details Vulncheck / Vulncheck (pull_request) Successful in 3m3s Details Build / Build Components (1.21) (pull_request) Successful in 3m53s Details Build / Build Components (1.20) (pull_request) Successful in 4m0s Details Tests and linters / Staticcheck (pull_request) Successful in 5m31s Details Tests and linters / Lint (pull_request) Successful in 6m17s Details Tests and linters / Tests (1.20) (pull_request) Successful in 12m47s Details Tests and linters / Tests (1.21) (pull_request) Successful in 13m14s Details Tests and linters / Tests with -race (pull_request) Successful in 13m12s Details It does the same as `control shards flush-writecache --seal`, but has better name. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 16:05:37 +03:00
Dmitrii Stepanov	7a9db5bcdd	[#569 ] writecache: Do not wait modeMtx if mode changes Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 16:05:37 +03:00
Dmitrii Stepanov	32c282ca10	[#569 ] writecache: Refactor flush Make single RUnlock call instead of two. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 16:05:37 +03:00
Dmitrii Stepanov	0cb0fc1735	[#569 ] writecache: Allow to seal writecache after flush Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 16:05:37 +03:00
Dmitrii Stepanov	b118734909	[#890 ] getsvc: Log node PK Some checks failed DCO action / DCO (pull_request) Successful in 13m55s Details Vulncheck / Vulncheck (pull_request) Successful in 16m51s Details Build / Build Components (1.20) (pull_request) Successful in 17m52s Details Build / Build Components (1.21) (pull_request) Successful in 18m9s Details Tests and linters / Tests (1.20) (pull_request) Failing after 24m34s Details Tests and linters / Tests (1.21) (pull_request) Failing after 25m56s Details Tests and linters / Staticcheck (pull_request) Successful in 26m46s Details Tests and linters / Lint (pull_request) Successful in 28m6s Details Tests and linters / Tests with -race (pull_request) Failing after 33m35s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-29 14:24:13 +03:00
Dmitrii Stepanov	764f70634d	[#881 ] containerSvc: Add APE validation All checks were successful Vulncheck / Vulncheck (pull_request) Successful in 2m40s Details DCO action / DCO (pull_request) Successful in 2m27s Details Build / Build Components (1.21) (pull_request) Successful in 3m22s Details Tests and linters / Lint (pull_request) Successful in 4m52s Details Tests and linters / Staticcheck (pull_request) Successful in 4m46s Details Build / Build Components (1.20) (pull_request) Successful in 4m54s Details Tests and linters / Tests (1.20) (pull_request) Successful in 11m59s Details Tests and linters / Tests (1.21) (pull_request) Successful in 12m38s Details Tests and linters / Tests with -race (pull_request) Successful in 13m10s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-27 11:05:34 +03:00
Dmitrii Stepanov	8180a0664f	[#887 ] node: Drop badger writecache implementation All checks were successful Vulncheck / Vulncheck (pull_request) Successful in 6m1s Details Build / Build Components (1.21) (pull_request) Successful in 7m37s Details Build / Build Components (1.20) (pull_request) Successful in 7m52s Details Tests and linters / Staticcheck (pull_request) Successful in 8m56s Details Tests and linters / Lint (pull_request) Successful in 9m26s Details Tests and linters / Tests (1.21) (pull_request) Successful in 15m5s Details Tests and linters / Tests with -race (pull_request) Successful in 15m7s Details DCO action / DCO (pull_request) Successful in 1m1s Details Tests and linters / Tests (1.20) (pull_request) Successful in 4m1s Details Badger implementation isn't tested and works not well, but requires human resources to maintain. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-22 13:00:54 +03:00
Anton Nikiforov	eab981bf1a	[#876 ] Fix linter error Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-12-21 18:21:37 +03:00
Anton Nikiforov	d9cbb16bd3	[#866 ] Use TTL for blobovnicza tree cache Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-12-19 16:36:28 +00:00
Airat Arifullin	7d7cf05575	[#851 ] ape: Initialize and use policy contract interface * Replace inmemory policy contract mock by initialized policy contract interface. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-12-18 11:06:35 +00:00
Airat Arifullin	bdd43f6211	[#869 ] object: Pass just CID to chain router * Do not convert CID from request to native-schema resource format - this step is unneccessary for APE. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-12-14 11:01:20 +00:00
Ekaterina Lebedeva	2d4c0a0f4a	[#552 ] Add systemd notifications to ir service All checks were successful DCO action / DCO (pull_request) Successful in 3m19s Details Vulncheck / Vulncheck (pull_request) Successful in 3m25s Details Build / Build Components (1.21) (pull_request) Successful in 4m27s Details Build / Build Components (1.20) (pull_request) Successful in 4m41s Details Tests and linters / Staticcheck (pull_request) Successful in 5m5s Details Tests and linters / Lint (pull_request) Successful in 5m57s Details Tests and linters / Tests (1.20) (pull_request) Successful in 13m25s Details Tests and linters / Tests (1.21) (pull_request) Successful in 14m13s Details Tests and linters / Tests with -race (pull_request) Successful in 16m15s Details Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2023-12-13 17:51:41 +03:00
Ekaterina Lebedeva	eca7ac9f0d	[#552 ] Add sdnotify package To avoid using third-party dependencies. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2023-12-13 17:49:26 +03:00
Ekaterina Lebedeva	05f8f49289	[#552 ] gofumpt changes Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2023-12-13 15:02:25 +03:00
Dmitrii Stepanov	7eb46404a1	[#863 ] blobovnicza: Fix counters All checks were successful Vulncheck / Vulncheck (pull_request) Successful in 2m27s Details DCO action / DCO (pull_request) Successful in 2m39s Details Build / Build Components (1.21) (pull_request) Successful in 3m38s Details Build / Build Components (1.20) (pull_request) Successful in 3m45s Details Tests and linters / Staticcheck (pull_request) Successful in 4m15s Details Tests and linters / Lint (pull_request) Successful in 5m47s Details Tests and linters / Tests (1.20) (pull_request) Successful in 11m49s Details Tests and linters / Tests (1.21) (pull_request) Successful in 12m16s Details Tests and linters / Tests with -race (pull_request) Successful in 12m31s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-13 13:34:29 +03:00
Dmitrii Stepanov	11add38e87	[#857 ] golangci: Add protogetter linter All checks were successful DCO action / DCO (pull_request) Successful in 1m38s Details Vulncheck / Vulncheck (pull_request) Successful in 3m2s Details Build / Build Components (1.21) (pull_request) Successful in 3m51s Details Build / Build Components (1.20) (pull_request) Successful in 4m3s Details Tests and linters / Staticcheck (pull_request) Successful in 5m17s Details Tests and linters / Lint (pull_request) Successful in 6m11s Details Tests and linters / Tests (1.20) (pull_request) Successful in 11m7s Details Tests and linters / Tests (1.21) (pull_request) Successful in 11m32s Details Tests and linters / Tests with -race (pull_request) Successful in 12m22s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 16:27:02 +03:00
Dmitrii Stepanov	94ffe8bb45	[#857 ] golangci: Add testifylint linter Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 16:27:02 +03:00
Dmitrii Stepanov	5d7833c89b	[#857 ] golangci: Add perfsprint linter Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 16:27:02 +03:00
Dmitrii Stepanov	3b7c0362a8	[#861 ] shard: Fix Delete object All checks were successful DCO action / DCO (pull_request) Successful in 12m22s Details Build / Build Components (1.20) (pull_request) Successful in 12m44s Details Build / Build Components (1.21) (pull_request) Successful in 13m2s Details Vulncheck / Vulncheck (pull_request) Successful in 13m3s Details Tests and linters / Staticcheck (pull_request) Successful in 15m32s Details Tests and linters / Lint (pull_request) Successful in 16m26s Details Tests and linters / Tests (1.21) (pull_request) Successful in 26m11s Details Tests and linters / Tests with -race (pull_request) Successful in 28m29s Details Tests and linters / Tests (1.20) (pull_request) Successful in 3m28s Details It is possible that object doesn't exist in metabase. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 14:25:40 +03:00
Dmitrii Stepanov	681b2c5fd4	[#825 ] policer: Do not drop required linking objects Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 11:04:03 +00:00
Dmitrii Stepanov	a3ef7b58b4	[#755 ] innerring: Check container owner namespace All checks were successful Vulncheck / Vulncheck (pull_request) Successful in 1m12s Details DCO action / DCO (pull_request) Successful in 2m43s Details Build / Build Components (1.21) (pull_request) Successful in 4m8s Details Build / Build Components (1.20) (pull_request) Successful in 4m18s Details Tests and linters / Staticcheck (pull_request) Successful in 4m19s Details Tests and linters / Lint (pull_request) Successful in 5m30s Details Tests and linters / Tests (1.20) (pull_request) Successful in 12m2s Details Tests and linters / Tests (1.21) (pull_request) Successful in 12m24s Details Tests and linters / Tests with -race (pull_request) Successful in 13m4s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 12:36:34 +03:00
Dmitrii Stepanov	1cd2bfe51a	[#755 ] morph: Drop FrostFSID contract usage Unused. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-12 12:36:34 +03:00
Anton Nikiforov	70e0c1e082	[#841 ] ir: Execute `netmap.addPeerIR` only for state `online` Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-12-11 13:14:41 +00:00
aarifullin	0f45e3d344	[#804 ] ape: Implement boltdb storage for local overrides All checks were successful DCO action / DCO (pull_request) Successful in 2m10s Details Vulncheck / Vulncheck (pull_request) Successful in 3m26s Details Build / Build Components (1.20) (pull_request) Successful in 5m41s Details Build / Build Components (1.21) (pull_request) Successful in 5m44s Details Tests and linters / Staticcheck (pull_request) Successful in 7m10s Details Tests and linters / Lint (pull_request) Successful in 8m14s Details Tests and linters / Tests (1.21) (pull_request) Successful in 14m24s Details Tests and linters / Tests (1.20) (pull_request) Successful in 14m41s Details Tests and linters / Tests with -race (pull_request) Successful in 14m38s Details Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-12-07 19:08:41 +03:00
aarifullin	e361e017f3	[#842 ] control: Pass target instead resource name * Update policy-engine package version in go.mod, go.sum. * Refactor CheckIfRequestPermitted: pass container target instead container ID. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-12-07 14:21:55 +00:00
aarifullin	39060382a1	[#842 ] control: Recieve target in gRPC methods for APE managing * Introduce Target type and pass it to all gRPC methods for APE chain managing instead CID. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-12-07 14:21:55 +00:00
Dmitrii Stepanov	db49ad16cc	[#826 ] blobovniczatree: Do not create DB's on init All checks were successful DCO action / DCO (pull_request) Successful in 4m10s Details Build / Build Components (1.21) (pull_request) Successful in 7m30s Details Vulncheck / Vulncheck (pull_request) Successful in 7m5s Details Tests and linters / Staticcheck (pull_request) Successful in 10m6s Details Tests and linters / Lint (pull_request) Successful in 10m26s Details Build / Build Components (1.20) (pull_request) Successful in 12m39s Details Tests and linters / Tests (1.20) (pull_request) Successful in 16m55s Details Tests and linters / Tests (1.21) (pull_request) Successful in 17m13s Details Tests and linters / Tests with -race (pull_request) Successful in 17m14s Details Blobovniczas will be created on write requests. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	ad0697adc4	[#661 ] blobovnicza: Compute size with record size To get more accurate size of blobovnicza use record size (lenght of key + lenght of data). Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	e54dc3dc7c	[#698 ] blobovnicza: Store counter values Blobovnicza initialization take a long time because of bucket Stat() call. So now blobovnicza stores counters in META bucket. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	5e8c08da3e	[#661 ] blobstore: Add address to error logs Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	8911656b1a	[#661 ] metrcis: Add rebuild percent metric Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	2407e5f5ff	[#661 ] blobovniczatree: Do not sort DB's and indicies Put stores object to next active DB, so there is no need to sort DBs. In addition, it adds unnecessary DB openings. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	c6a739e746	[#661 ] blobovniczatree: Make Rebuild concurrent for objects Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	f1c7905263	[#661 ] blobovniczatree: Make Rebuild concurrent Different DBs can be rebuild concurrently. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	d4d905ecc6	[#661 ] metrics: Add blobovniczatree rebuild metrics Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	b2769ca3de	[#661 ] blobovniczatree: Make Rebuild failover safe Now move info stores in blobovnicza, so in case of failover rebuild completes previous operation first. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	da4fee2d0b	[#698 ] blobovniczatree: Init blobovniczas concurrently Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:33 +03:00
Dmitrii Stepanov	422226da18	[#661 ] blobovniczatree: Add Rebuild implementation Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:32 +03:00
Dmitrii Stepanov	a531eaf8bc	[#661 ] blobstor: Add Rebuild implementation Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:32 +03:00
Dmitrii Stepanov	c1667a11d2	[#661 ] blobovniczatree: Allow to change depth or width Now it is possible to change depth or with of blobovniczatree. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:32 +03:00
Dmitrii Stepanov	484eb59893	[#661 ] blobovniczatree: Use .db extension for db files Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:32 +03:00
Dmitrii Stepanov	44552a849b	[#661 ] shard: Add blobstor rebuilder Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-07 15:37:32 +03:00
Dmitrii Stepanov	a478050639	[#838 ] metabase: Resolve funlen linter All checks were successful DCO action / DCO (pull_request) Successful in 1m59s Details Vulncheck / Vulncheck (pull_request) Successful in 3m0s Details Build / Build Components (1.21) (pull_request) Successful in 3m40s Details Tests and linters / Lint (pull_request) Successful in 4m57s Details Tests and linters / Staticcheck (pull_request) Successful in 4m38s Details Build / Build Components (1.20) (pull_request) Successful in 5m17s Details Tests and linters / Tests (1.20) (pull_request) Successful in 7m1s Details Tests and linters / Tests (1.21) (pull_request) Successful in 7m39s Details Tests and linters / Tests with -race (pull_request) Successful in 7m46s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-06 15:44:21 +03:00
Dmitrii Stepanov	d30ab5f29e	[#838 ] metabase: Count user objects Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-06 15:44:21 +03:00
Dmitrii Stepanov	f314da4af3	[#838 ] metabase: Add user object type counter Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-06 15:44:21 +03:00
Dmitrii Stepanov	29550fe600	[#838 ] shard: Refactor `updateMetrics` method Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-12-06 15:44:21 +03:00
Alexander Chuprov	b2c63e57ba	[#651 ] engine/test: Speedup StorageEngine_Inhume Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-11-30 13:19:43 +00:00
Alexander Chuprov	445ebcc0e7	[#651 ] shard/test: Speedup Shard_Delete Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-11-30 13:19:43 +00:00
Alexander Chuprov	2302e5d342	[#651 ] shard/test: Refactor Shard_Delete Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-11-30 13:19:43 +00:00
Dmitrii Stepanov	26e4f7005c	[#741 ] treesvc: Refactor tree sync Fix linter issues. Add error logging. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-30 12:45:02 +00:00
Dmitrii Stepanov	b21be1abdd	[#741 ] treesvc: Do not update sync height if some node is unavailable Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-30 12:45:02 +00:00
Dmitrii Stepanov	b215817e14	[#741 ] treesvc: Remove unused height variables Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-30 12:45:02 +00:00
Dmitrii Stepanov	c516c7c5f4	[#821 ] node: Pass user.ID by value All checks were successful DCO action / DCO (pull_request) Successful in 3m45s Details Build / Build Components (1.21) (pull_request) Successful in 5m18s Details Build / Build Components (1.20) (pull_request) Successful in 5m28s Details Tests and linters / Tests (1.20) (pull_request) Successful in 7m30s Details Tests and linters / Tests (1.21) (pull_request) Successful in 7m42s Details Tests and linters / Lint (pull_request) Successful in 8m25s Details Vulncheck / Vulncheck (pull_request) Successful in 9m22s Details Tests and linters / Staticcheck (pull_request) Successful in 10m57s Details Tests and linters / Tests with -race (pull_request) Successful in 16m53s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-23 10:21:07 +03:00
Dmitrii Stepanov	07390ad4e3	[#715 ] node: Unify config parameter names All checks were successful DCO action / DCO (pull_request) Successful in 2m54s Details Vulncheck / Vulncheck (pull_request) Successful in 3m11s Details Tests and linters / Staticcheck (pull_request) Successful in 3m55s Details Build / Build Components (1.21) (pull_request) Successful in 3m51s Details Build / Build Components (1.20) (pull_request) Successful in 4m6s Details Tests and linters / Tests (1.21) (pull_request) Successful in 5m10s Details Tests and linters / Lint (pull_request) Successful in 5m24s Details Tests and linters / Tests (1.20) (pull_request) Successful in 6m56s Details Tests and linters / Tests with -race (pull_request) Successful in 7m46s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-22 17:13:50 +03:00
Dmitrii Stepanov	8d18fa159e	[#667 ] writecache: Fix flush test All checks were successful DCO action / DCO (pull_request) Successful in 1m28s Details Vulncheck / Vulncheck (pull_request) Successful in 3m55s Details Tests and linters / Staticcheck (pull_request) Successful in 4m15s Details Build / Build Components (1.21) (pull_request) Successful in 4m40s Details Build / Build Components (1.20) (pull_request) Successful in 4m54s Details Tests and linters / Tests (1.20) (pull_request) Successful in 5m32s Details Tests and linters / Lint (pull_request) Successful in 5m59s Details Tests and linters / Tests (1.21) (pull_request) Successful in 6m59s Details Tests and linters / Tests with -race (pull_request) Successful in 6m57s Details Allow to disable background flusher for testing purposes. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-17 17:45:43 +03:00
aarifullin	02454df14a	[#809 ] client: Refactor PrmInit, PrmDial usage Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-17 13:37:03 +00:00
Dmitrii Stepanov	76ff26039c	[#96 ] node: Drop neo-go's `slices` package All checks were successful DCO action / DCO (pull_request) Successful in 2m13s Details Build / Build Components (1.20) (pull_request) Successful in 5m48s Details Build / Build Components (1.21) (pull_request) Successful in 5m47s Details Vulncheck / Vulncheck (pull_request) Successful in 5m47s Details Tests and linters / Tests (1.21) (pull_request) Successful in 7m33s Details Tests and linters / Staticcheck (pull_request) Successful in 7m37s Details Tests and linters / Lint (pull_request) Successful in 8m8s Details Tests and linters / Tests (1.20) (pull_request) Successful in 10m38s Details Tests and linters / Tests with -race (pull_request) Successful in 11m22s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-17 13:24:04 +03:00
aarifullin	47286ebf32	[#805 ] pilorama: Fix TreeDrop * If treeID is empty then deleting buckets for cursor may get invalidated. So, buckets should be gathered before deleting. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-17 10:21:35 +00:00
Dmitrii Stepanov	29fe8c41f3	[#655 ] storage: Drop `ErrorHandler` All checks were successful DCO action / DCO (pull_request) Successful in 1m42s Details Build / Build Components (1.21) (pull_request) Successful in 3m23s Details Vulncheck / Vulncheck (pull_request) Successful in 3m25s Details Build / Build Components (1.20) (pull_request) Successful in 4m16s Details Tests and linters / Staticcheck (pull_request) Successful in 4m24s Details Tests and linters / Lint (pull_request) Successful in 5m2s Details Tests and linters / Tests (1.20) (pull_request) Successful in 5m36s Details Tests and linters / Tests (1.21) (pull_request) Successful in 6m42s Details Tests and linters / Tests with -race (pull_request) Successful in 7m18s Details The only one usage was for logging. Now logging performed by storage anyway. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-16 17:27:38 +03:00
Dmitrii Stepanov	137e987a4e	[#655 ] storage: Drop `LazyHandler` LazyHandler is implemented and used incorrectly. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-16 17:27:38 +03:00
aarifullin	4d5be5ccb5	[#811 ] ape: Update policy-engine module version and rebase All checks were successful DCO action / DCO (pull_request) Successful in 4m23s Details Vulncheck / Vulncheck (pull_request) Successful in 5m31s Details Build / Build Components (1.21) (pull_request) Successful in 7m33s Details Build / Build Components (1.20) (pull_request) Successful in 7m40s Details Tests and linters / Staticcheck (pull_request) Successful in 8m22s Details Tests and linters / Lint (pull_request) Successful in 9m23s Details Tests and linters / Tests with -race (pull_request) Successful in 11m20s Details Tests and linters / Tests (1.21) (pull_request) Successful in 11m32s Details Tests and linters / Tests (1.20) (pull_request) Successful in 11m41s Details Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-16 11:31:37 +03:00
Evgenii Stratonikov	a952a406a2	[#506 ] container: Use uint64 for epoch type It is `uint64` in netmap source interfaces and other code. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-11-14 19:00:11 +00:00
Evgenii Stratonikov	f04806ccd3	[#506 ] container: Use user.ID in DeletionInfo response Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-11-14 19:00:11 +00:00
Dmitrii Stepanov	8088063195	[#787 ] netmap: Refactor `NewEpoch` method All checks were successful DCO action / DCO (pull_request) Successful in 3m58s Details Vulncheck / Vulncheck (pull_request) Successful in 4m51s Details Build / Build Components (1.21) (pull_request) Successful in 5m47s Details Build / Build Components (1.20) (pull_request) Successful in 5m57s Details Tests and linters / Tests (1.20) (pull_request) Successful in 6m25s Details Tests and linters / Staticcheck (pull_request) Successful in 6m31s Details Tests and linters / Lint (pull_request) Successful in 7m15s Details Tests and linters / Tests (1.21) (pull_request) Successful in 7m16s Details Tests and linters / Tests with -race (pull_request) Successful in 8m30s Details Split for user and control methods. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-13 17:22:31 +03:00
Dmitrii Stepanov	c8a62ffedd	[#787 ] morph: Calculate VUB and nonce when hash is nil All checks were successful DCO action / DCO (pull_request) Successful in 6m41s Details Vulncheck / Vulncheck (pull_request) Successful in 7m54s Details Build / Build Components (1.21) (pull_request) Successful in 9m29s Details Build / Build Components (1.20) (pull_request) Successful in 9m46s Details Tests and linters / Staticcheck (pull_request) Successful in 12m42s Details Tests and linters / Tests (1.20) (pull_request) Successful in 13m51s Details Tests and linters / Tests (1.21) (pull_request) Successful in 13m50s Details Tests and linters / Tests with -race (pull_request) Successful in 14m26s Details Tests and linters / Lint (pull_request) Successful in 15m21s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-13 17:13:03 +03:00
Dmitrii Stepanov	2393d13e4d	[#787 ] morph: Return VUB for IR service calls Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-13 17:13:03 +03:00
Dmitrii Stepanov	518f3baf41	[#787 ] morph: Return VUB from `Invoke` method Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-13 17:13:03 +03:00
Dmitrii Stepanov	bdfa523487	[#787 ] proto: Add VUB field for IR service Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-13 17:13:03 +03:00
Dmitrii Stepanov	0f75e48138	[#796 ] policer: Fix tombstone objects replication Tombstone objects must be replicated to all container nodes. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-09 13:39:33 +03:00
Dmitrii Stepanov	7cdae4f660	[#792 ] proto: Regenerate with fixed version Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-09 10:09:13 +00:00
Dmitrii Stepanov	9133b4389e	[#788 ] objectsvc: Fix formatting (gofumpt) All checks were successful DCO action / DCO (pull_request) Successful in 3m19s Details Vulncheck / Vulncheck (pull_request) Successful in 3m40s Details Build / Build Components (1.21) (pull_request) Successful in 4m17s Details Build / Build Components (1.20) (pull_request) Successful in 4m32s Details Tests and linters / Staticcheck (pull_request) Successful in 4m46s Details Tests and linters / Tests (1.21) (pull_request) Successful in 5m9s Details Tests and linters / Lint (pull_request) Successful in 5m28s Details Tests and linters / Tests (1.20) (pull_request) Successful in 5m24s Details Tests and linters / Tests with -race (pull_request) Successful in 7m38s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-09 10:27:32 +03:00
Dmitrii Stepanov	1b22801eed	[#788 ] engine: Fix flaky tests Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-09 10:25:46 +03:00
aarifullin	3534d6d05b	[#794 ] objectsvc: Return accidentally removed acl checks for Head All checks were successful DCO action / DCO (pull_request) Successful in 1m42s Details Vulncheck / Vulncheck (pull_request) Successful in 3m23s Details Build / Build Components (1.21) (pull_request) Successful in 4m22s Details Build / Build Components (1.20) (pull_request) Successful in 5m44s Details Tests and linters / Staticcheck (pull_request) Successful in 6m3s Details Tests and linters / Lint (pull_request) Successful in 6m35s Details Tests and linters / Tests (1.20) (pull_request) Successful in 8m32s Details Tests and linters / Tests with -race (pull_request) Successful in 8m47s Details Tests and linters / Tests (1.21) (pull_request) Successful in 8m54s Details Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-08 17:13:58 +03:00
Airat Arifullin	66848d3288	[#770 ] cli: Add methods to work with APE rules via control svc * Add methods to frostfs-cli * Implement rpc in control service Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-08 13:34:03 +00:00
Airat Arifullin	8e11ef46b8	[#770 ] object: Introduce ape chain checker for object svc * Introduce Request type converted from RequestInfo type to implement policy-engine's Request interface * Implement basic ape checker to check if a request is permitted to be performed * Make put handlers use APE checker instead EACL Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-08 13:34:03 +00:00
Airat Arifullin	5ec73fe8a0	[#770 ] node: Introduce ape chain source * Provide methods to access rule chains with access policy engine (APE) chain source * Initialize apeChainSource within object service initialization * Share apeChainSource with control service * Implement dummy apeChainSource instance based on in-memory implementation Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-08 13:34:03 +00:00
Airat Arifullin	3a2c319b87	[#770 ] control: Generate gRPC methods to manipulate APE chains * Define new types and gRPC methods to manipulate APE chains in control service. * Stub gRPC handlers for the generated methods. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-08 13:34:03 +00:00
Dmitrii Stepanov	70ab1ebd54	[#763 ] metrics: Add container_objects_total metric All checks were successful DCO action / DCO (pull_request) Successful in 3m54s Details Build / Build Components (1.20) (pull_request) Successful in 4m58s Details Build / Build Components (1.21) (pull_request) Successful in 5m16s Details Vulncheck / Vulncheck (pull_request) Successful in 9m54s Details Tests and linters / Lint (pull_request) Successful in 10m57s Details Tests and linters / Tests (1.21) (pull_request) Successful in 12m40s Details Tests and linters / Staticcheck (pull_request) Successful in 12m34s Details Tests and linters / Tests with -race (pull_request) Successful in 12m48s Details Tests and linters / Tests (1.20) (pull_request) Successful in 13m19s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-08 12:30:57 +03:00
Dmitrii Stepanov	9c98fa6152	[#763 ] metabase: Add container objects counter Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-08 12:30:57 +03:00
Dmitrii Stepanov	226e84d782	[#684 ] node: Add skipped objects count to evacuation result Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-07 12:17:11 +00:00
Dmitrii Stepanov	523fb3ca51	[#684 ] proto: Add skipped count to evacuation status response Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-07 12:17:11 +00:00
aarifullin	74c91eeef5	[#777 ] client: Refactor PrmContainerList, PrmObjectSearch usage Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2023-11-06 06:50:11 +00:00
Dmitrii Stepanov	20d6132f31	[#531 ] signSvc: Add SetMarshaledData method call All checks were successful Vulncheck / Vulncheck (pull_request) Successful in 1m25s Details DCO action / DCO (pull_request) Successful in 1m36s Details Build / Build Components (1.21) (pull_request) Successful in 2m40s Details Build / Build Components (1.20) (pull_request) Successful in 3m2s Details Tests and linters / Tests (1.21) (pull_request) Successful in 5m41s Details Tests and linters / Staticcheck (pull_request) Successful in 5m32s Details Tests and linters / Lint (pull_request) Successful in 5m59s Details Tests and linters / Tests (1.20) (pull_request) Successful in 6m3s Details Tests and linters / Tests with -race (pull_request) Successful in 6m6s Details To reduce memory allocations add `SetMarshaledData` method call to return already marshalled data in next `StableMarshal` calls. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-02 17:34:33 +03:00
Anton Nikiforov	a26483fc30	[#749 ] morph: Fix panic when closing morph client Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-11-01 10:48:10 +00:00
Dmitrii Stepanov	c80b46fad3	[#754 ] blobstor: Estimate compressability All checks were successful DCO action / DCO (pull_request) Successful in 1m59s Details Vulncheck / Vulncheck (pull_request) Successful in 3m31s Details Build / Build Components (1.20) (pull_request) Successful in 4m37s Details Build / Build Components (1.21) (pull_request) Successful in 4m33s Details Tests and linters / Tests (1.20) (pull_request) Successful in 4m54s Details Tests and linters / Staticcheck (pull_request) Successful in 4m49s Details Tests and linters / Tests with -race (pull_request) Successful in 5m9s Details Tests and linters / Lint (pull_request) Successful in 6m4s Details Tests and linters / Tests (1.21) (pull_request) Successful in 6m9s Details Now it is possible to enable compressability estimation. If data is likely uncompressable, it should reduce CPU time and memory. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-11-01 11:24:32 +03:00
Dmitrii Stepanov	05b508f79a	[#772 ] proto: Fix file ending All checks were successful DCO action / DCO (pull_request) Successful in 3m47s Details Vulncheck / Vulncheck (pull_request) Successful in 4m47s Details Build / Build Components (1.21) (pull_request) Successful in 6m5s Details Build / Build Components (1.20) (pull_request) Successful in 6m13s Details Tests and linters / Tests (1.20) (pull_request) Successful in 13m9s Details Tests and linters / Staticcheck (pull_request) Successful in 13m17s Details Tests and linters / Lint (pull_request) Successful in 13m49s Details Tests and linters / Tests (1.21) (pull_request) Successful in 14m12s Details Tests and linters / Tests with -race (pull_request) Successful in 4m30s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-31 17:03:04 +03:00
Dmitrii Stepanov	79088baa06	[#772 ] node: Apply gofumpt Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-31 17:03:03 +03:00
Evgenii Stratonikov	00aa6d9749	[#633 ] shard/test: Fix TestCounters() Introduced in `362f24953a`, forgotten to be changed because test generator didn't provide payload size. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-31 12:53:28 +00:00
Evgenii Stratonikov	b8f79f4227	[#633 ] shard/test: Fix race conditions in TestCounters() Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-31 12:53:28 +00:00
Evgenii Stratonikov	261d281154	[#762 ] go.mod: Update sdk-go Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-31 11:22:29 +00:00
Dmitrii Stepanov	869518be0a	[#728 ] writecache: Fix Badger writecache race. All checks were successful Build / Build Components (1.21) (pull_request) Successful in 1m47s Details DCO action / DCO (pull_request) Successful in 2m1s Details Vulncheck / Vulncheck (pull_request) Successful in 3m5s Details Build / Build Components (1.20) (pull_request) Successful in 3m58s Details Tests and linters / Tests (1.20) (pull_request) Successful in 5m43s Details Tests and linters / Staticcheck (pull_request) Successful in 5m49s Details Tests and linters / Lint (pull_request) Successful in 6m20s Details Tests and linters / Tests with -race (pull_request) Successful in 6m8s Details Tests and linters / Tests (1.21) (pull_request) Successful in 3m6s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-30 18:36:41 +03:00
Dmitrii Stepanov	d4b6ebe7e7	[#725 ] writecache: Fix metric values All checks were successful DCO action / DCO (pull_request) Successful in 1m33s Details Build / Build Components (1.21) (pull_request) Successful in 3m11s Details Build / Build Components (1.20) (pull_request) Successful in 3m22s Details Vulncheck / Vulncheck (pull_request) Successful in 3m34s Details Tests and linters / Tests (1.20) (pull_request) Successful in 6m0s Details Tests and linters / Tests (1.21) (pull_request) Successful in 6m1s Details Tests and linters / Staticcheck (pull_request) Successful in 6m4s Details Tests and linters / Tests with -race (pull_request) Successful in 3m58s Details Tests and linters / Lint (pull_request) Successful in 11m43s Details Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-10-27 12:22:29 +03:00
Anton Nikiforov	121f5c4dd8	[#757 ] ir: Do not exclude node in maintenance mode from netmap All checks were successful Build / Build Components (1.20) (pull_request) Successful in 6m20s Details DCO action / DCO (pull_request) Successful in 6m45s Details Vulncheck / Vulncheck (pull_request) Successful in 6m20s Details Build / Build Components (1.21) (pull_request) Successful in 8m34s Details Tests and linters / Tests (1.20) (pull_request) Successful in 9m21s Details Tests and linters / Tests (1.21) (pull_request) Successful in 9m17s Details Tests and linters / Staticcheck (pull_request) Successful in 9m15s Details Tests and linters / Lint (pull_request) Successful in 9m42s Details Tests and linters / Tests with -race (pull_request) Successful in 11m34s Details Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2023-10-26 10:50:32 +03:00
Evgenii Stratonikov	9f7c2d8810	[#752 ] innerring: Simplify keyPosition() All checks were successful DCO action / DCO (pull_request) Successful in 3m0s Details Vulncheck / Vulncheck (pull_request) Successful in 3m10s Details Build / Build Components (1.20) (pull_request) Successful in 4m3s Details Build / Build Components (1.21) (pull_request) Successful in 4m8s Details Tests and linters / Staticcheck (pull_request) Successful in 5m13s Details Tests and linters / Lint (pull_request) Successful in 5m44s Details Tests and linters / Tests (1.20) (pull_request) Successful in 7m0s Details Tests and linters / Tests (1.21) (pull_request) Successful in 7m7s Details Tests and linters / Tests with -race (pull_request) Successful in 7m50s Details Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-10-25 16:06:44 +03:00

... 3 4 5 6 7 ...

3243 commits