3247 commits

Author	SHA1	Message	Date
Dmitrii Stepanov	5f6c7cbdb1	[#1367] writecache: Drop bbolt DB ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-12 14:22:29 +03:00
Aleksey Savchuk	99be4c83a7	[#1368] *: Run gofumpt ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-09-12 10:00:28 +03:00
Aleksey Savchuk	5fac4058e8	[#1364] cmd/common: Add tests for CreateViper and ReloadViper ... Add tests for `CreateViper` and `ReloadViper` to ensure that no extra files, except *.yaml, *.yml, *.json, are loaded from config directory. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-09-11 08:03:05 +00:00
Airat Arifullin	a812932984	[#1362] ape: Move common APE check logic to separate package ... * Tree and object service have the same log for checking APE. So, this check should be moved to common package. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-09-10 12:40:34 +00:00
Dmitrii Stepanov	92fe5d90f5	[#1359] writecache: Drop unused ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-09 18:45:21 +03:00
Dmitrii Stepanov	4668efc0bf	[#1355] metabase: Upgrade improvements ... Do not fail on same latest version to run compact on upgraded metabase. Use NoSync on compact. Log every batch on bucket delete stage. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-09 11:37:36 +03:00
Dmitrii Stepanov	d3b209c8e1	[#1337] shard: Disable background rebuild ... Since `frostfs-cli control shards rebuild` command was added, there is no need for background rebuild now. For failover tests used used value 1 to rebuild only schema change. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 15:19:55 +03:00
Dmitrii Stepanov	edb1747af7	[#1337] blobovniczatree: Add rebuild by overflow ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 15:19:54 +03:00
Dmitrii Stepanov	a61201a987	[#1337] config: Move rebuild_worker_count to shard section ... This makes it simple to limit performance degradation for every shard because of rebuild. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 13:57:27 +03:00
Dmitrii Stepanov	6b6eabe41c	[#1337] cli: Add control shards rebuild command ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 13:57:27 +03:00
Dmitrii Stepanov	d508da8397	[#1337] blobovniczatree: Add rebuild by fill percent ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 13:57:27 +03:00
Dmitrii Stepanov	007827255e	[#1337] blobovniczatree: Add .rebuild temp files ... This allows to reduce open/close DBs to check incompleted rebuilds. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 13:57:27 +03:00
Ekaterina Lebedeva	f652518c24	[#1357] go: Fix panic caused by using range over slice len ... If slice is altered in `for` loop, we cannot use range over its length: it may cause panic if slice gets shorter. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-09-06 13:09:58 +03:00
Airat Arifullin	273980cfb9	[#1310] object: Remove irrelevant comments ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-09-05 16:40:32 +03:00
Anton Nikiforov	108e4e07be	[#1349] node: Evacuate objects without setting mode to MAINTENANCE ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-09-05 16:08:27 +03:00
Airat Arifullin	b3deb893ba	[#1310] object: Move target initialization to separate package ... * Split the logic of write target initialization to different packages; * Refactor patch and put services: since both service initialize the target themselves. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-09-05 13:03:58 +00:00
Aleksey Savchuk	a4fb7f085b	[#1348] go.mod: Update api-go and sdk-go ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-09-04 10:47:26 +03:00
Ekaterina Lebedeva	a685fcdc96	[#1317] go.mod: Use range over int ... Since Go 1.22 a "for" statement with a "range" clause is able to iterate through integer values from zero to an upper limit. gopatch script: @@ var i, e expression @@ -for i := 0; i <= e - 1; i++ { +for i := range e { ... } @@ var i, e expression @@ -for i := 0; i <= e; i++ { +for i := range e + 1 { ... } @@ var i, e expression @@ -for i := 0; i < e; i++ { +for i := range e { ... } Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-09-03 13:00:54 +03:00
Aleksey Savchuk	2b3fc50681	[#1320] shard: Fix TestGCDropsObjectInhumedFromWritecache flaky test ... The `TestGCDropsObjectInhumedFromWritecache` test was flaky because a running asynchronous rebuild operation prevented GC from deleting the object. A test-only shard option `WithDisabledRebuild` has been added to fix this. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-09-02 10:26:53 +03:00
Evgenii Stratonikov	98fe24cdb7	[#1343] go.mod: Update api-go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-30 08:45:24 +00:00
Dmitrii Stepanov	882c068410	[#1334] metabase: Store upgrade flag ... This allows to check if metabase upgrade was not completed. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-30 09:06:20 +03:00
Dmitrii Stepanov	6c2146bbc1	[#1334] metabase: Add upgrade from v2 to v3 ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-30 09:06:20 +03:00
Evgenii Stratonikov	03976c6ed5	[#1341] .golangci.yml: Replace exportloopref with copyloopvar ... exportloopref is deprecated. gopatch: ``` @@ var index, value identifier var slice expression @@ for index, value := range slice { ... -value := value ... } @@ var index, value identifier var slice expression @@ for index, value := range slice { ... -index := index ... } @@ var value identifier var channel expression @@ for value := range channel { ... -value := value ... } ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-28 15:44:41 +00:00
Dmitrii Stepanov	7abbdca064	[#1340] getSvc: Fix access denied error handling ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-28 14:02:28 +03:00
Airat Arifullin	6488ddee88	[#1338] object: Fix range provider in Patch handler ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-08-27 16:51:12 +03:00
Airat Arifullin	d6b42972a8	[#1338] object: Fix audit patch stream ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-08-27 16:18:29 +03:00
Evgenii Stratonikov	5e9a97fd3e	[#1336] go.mod: Update api-go and sdk-go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-27 14:48:43 +03:00
Evgenii Stratonikov	806ea37101	[#1328] pilorama: Do not skip items in SortedByFilename ... Benchmark results: ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/pilorama cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ new │ │ sec/op │ sec/op vs base │ ForestSortedIteration/bbolt,root-8 207.2µ ± 6% 173.6µ ± 6% -16.23% (p=0.000 n=10) ForestSortedIteration/bbolt,leaf-8 3.910µ ± 5% 3.928µ ± 7% ~ (p=0.529 n=10) geomean 28.46µ 26.11µ -8.27% ``` They are not representative, as the worst case is when we have multiple items of different lengths. However, `FileName` is usually less than 100 in practice, so the asymptotics is the same. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-26 06:11:32 +00:00
Evgenii Stratonikov	80099d9a2f	[#1328] pilorama: Add tricky test for SortedByFilename ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-26 06:11:32 +00:00
Dmitrii Stepanov	bd24beecf8	[#1329] putSvc: Reset SuccessAfter for non-EC objects in EC container broadcasting ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-23 13:51:16 +03:00
Dmitrii Stepanov	76f67ea34e	[#1323] metabase: Bump version ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Dmitrii Stepanov	7d0d781db1	[#1323] metabase: Drop user attribute index ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Dmitrii Stepanov	0f08a2efba	[#1323] metabase: Resolve funlen linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Dmitrii Stepanov	7bf20c9f1f	[#1323] metabase: Add expiration epoch buckets ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Dmitrii Stepanov	2542d4f5df	[#1323] metabase: Drop payload checksum index ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Dmitrii Stepanov	15dae8685e	[#1323] metabase: Drop ownerID index ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Evgenii Stratonikov	7bca428db0	[#1322] Use new protobuf marshaler ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-22 07:17:41 +00:00
Ekaterina Lebedeva	a345c972bf	[#1316] lint: Fix warnings ... Renamed parameters `min/max` to avoid conflicts with predeclared identifiers. Replaced background context with parent context without cancellation in closer functions in frostfs-node. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-08-21 18:18:25 +03:00
Anton Nikiforov	cfda9003a7	[#1318] meta: Add test TestInhumeECObject ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-08-20 16:21:27 +03:00
Anton Nikiforov	6ff0b0996b	[#1318] metrics: Fix container_size_bytes for EC ... When node put chunk into EC container, `policer` may remove it as redundant. This chunk marked as removed. When parent object removed and `gc` start iterating over chunk, node count removing chunk twice. Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-08-20 15:33:28 +03:00
Anton Nikiforov	8319b59238	[#1318] Fix gofumpt issue ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-08-20 14:34:54 +03:00
Dmitrii Stepanov	b7acb34fa4	[#1319] treeSvc: Do not wrap error from APE ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-19 18:38:27 +03:00
Airat Arifullin	eeab417dcf	[#1307] object: Add APE check for Patch handler ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-08-16 14:13:09 +00:00
Airat Arifullin	e890f1b4b1	[#1307] object: Implement Patch method ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-08-16 14:13:09 +00:00
Airat Arifullin	a4a1c3f18b	[#1307] go.mod: Bump frostfs-sdk-go/frostfs-api-go/v2 versions ... * Also, resolve dependencies and conflicts for object service by creating stub for `Patch` method. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-08-16 14:13:09 +00:00
Ekaterina Lebedeva	ec1509de4e	[#1262] sdnotify: Send system monotonic time on reload ... The synchronized service reload protocol added in systemd version 253 requires that the service provides a MONOTONIC_USEC field alongside the RELOADING=1 notification message for synchronization purposes. The value carried in this field must be the system CLOCK_MONOTONIC timestamp at the time the notification message was generated as systemd compares it to other CLOCK_MONOTONIC timestamps taken by pid1. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-08-16 12:05:31 +03:00
Ekaterina Lebedeva	5da41f1fe5	Revert "[#1262] sdnotify: Get rid of go:linkname for nanotime" ... This reverts commit 327d364f34. Reverted due to the problem with reload signal sent by systemd. `frostfs-ir` service reconfigures correctly and service's statuses are being reported to systemd. However, since we replaced `go:linkname` & `nanotime()` with `time.Since()`, systemd refuses to accept reload signal response from `frostfs-ir`. To maintain correct behaviour it was decided to revevrt systemd-related changes until a better solution is found. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-08-15 16:54:45 +03:00
Ekaterina Lebedeva	d055168e2a	[#1135] ir: Add healthstatus RECONFIGURING ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-08-15 16:54:45 +03:00
Dmitrii Stepanov	80ce7c3a00	[#1284] shard: Resolve funlen linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-09 13:23:35 +03:00
Dmitrii Stepanov	93d63e1632	[#1284] writecache: Allow to seal writecache async ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-09 13:23:35 +03:00