5120 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Airat Arifullin	a812932984	[#1362] ape: Move common APE check logic to separate package ... * Tree and object service have the same log for checking APE. So, this check should be moved to common package. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-09-10 12:40:34 +00:00
Dmitrii Stepanov	92fe5d90f5	[#1359] writecache: Drop unused ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-09 18:45:21 +03:00
Dmitrii Stepanov	4668efc0bf	[#1355] metabase: Upgrade improvements ... Do not fail on same latest version to run compact on upgraded metabase. Use NoSync on compact. Log every batch on bucket delete stage. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-09 11:37:36 +03:00
Dmitrii Stepanov	654d970fad	[#1355] adm: Run metabase upgrade concurrently ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 15:54:40 +03:00
Dmitrii Stepanov	d3b209c8e1	[#1337] shard: Disable background rebuild ... Since `frostfs-cli control shards rebuild` command was added, there is no need for background rebuild now. For failover tests used used value 1 to rebuild only schema change. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 15:19:55 +03:00
Dmitrii Stepanov	edb1747af7	[#1337] blobovniczatree: Add rebuild by overflow ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 15:19:54 +03:00
Dmitrii Stepanov	a61201a987	[#1337] config: Move rebuild_worker_count to shard section ... This makes it simple to limit performance degradation for every shard because of rebuild. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 13:57:27 +03:00
Dmitrii Stepanov	6b6eabe41c	[#1337] cli: Add control shards rebuild command ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 13:57:27 +03:00
Dmitrii Stepanov	d508da8397	[#1337] blobovniczatree: Add rebuild by fill percent ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 13:57:27 +03:00
Dmitrii Stepanov	007827255e	[#1337] blobovniczatree: Add .rebuild temp files ... This allows to reduce open/close DBs to check incompleted rebuilds. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-09-06 13:57:27 +03:00
Ekaterina Lebedeva	f652518c24	[#1357] go: Fix panic caused by using range over slice len ... If slice is altered in `for` loop, we cannot use range over its length: it may cause panic if slice gets shorter. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-09-06 13:09:58 +03:00
Airat Arifullin	273980cfb9	[#1310] object: Remove irrelevant comments ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-09-05 16:40:32 +03:00
Anton Nikiforov	108e4e07be	[#1349] node: Evacuate objects without setting mode to MAINTENANCE ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-09-05 16:08:27 +03:00
Airat Arifullin	b3deb893ba	[#1310] object: Move target initialization to separate package ... * Split the logic of write target initialization to different packages; * Refactor patch and put services: since both service initialize the target themselves. Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-09-05 13:03:58 +00:00
Aleksey Savchuk	7768a482b5	[#1223] lens/tui: Add TUI app for blobovnicza ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-09-05 08:03:52 +00:00
Aleksey Savchuk	371d97f61a	[#1223] lens/tui: Add TUI app for write cache ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-09-05 08:03:52 +00:00
Aleksey Savchuk	e655336390	[#1223] lens/tui: Add app help ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-09-05 08:03:52 +00:00
Aleksey Savchuk	ed396448ac	[#1223] lens/tui: Add TUI app to explore metabase ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-09-05 08:03:52 +00:00
Aleksey Savchuk	9cbd32bce8	[#1223] lens/tui: Add writecache schema ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-09-05 08:03:52 +00:00
Aleksey Savchuk	1ae86f35a8	[#1223] lens/tui: Add metabase schema ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-09-05 08:03:52 +00:00
Aleksey Savchuk	b9043433a0	[#1223] scripts: Add script to populate metabase ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-09-05 08:03:52 +00:00
Aleksey Savchuk	a4fb7f085b	[#1348] go.mod: Update api-go and sdk-go ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-09-04 10:47:26 +03:00
Ekaterina Lebedeva	a685fcdc96	[#1317] go.mod: Use range over int ... Since Go 1.22 a "for" statement with a "range" clause is able to iterate through integer values from zero to an upper limit. gopatch script: @@ var i, e expression @@ -for i := 0; i <= e - 1; i++ { +for i := range e { ... } @@ var i, e expression @@ -for i := 0; i <= e; i++ { +for i := range e + 1 { ... } @@ var i, e expression @@ -for i := 0; i < e; i++ { +for i := range e { ... } Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-09-03 13:00:54 +03:00
Aleksey Savchuk	2b3fc50681	[#1320] shard: Fix TestGCDropsObjectInhumedFromWritecache flaky test ... The `TestGCDropsObjectInhumedFromWritecache` test was flaky because a running asynchronous rebuild operation prevented GC from deleting the object. A test-only shard option `WithDisabledRebuild` has been added to fix this. Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-09-02 10:26:53 +03:00
Evgenii Stratonikov	98fe24cdb7	[#1343] go.mod: Update api-go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-30 08:45:24 +00:00
Dmitrii Stepanov	882c068410	[#1334] metabase: Store upgrade flag ... This allows to check if metabase upgrade was not completed. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-30 09:06:20 +03:00
Dmitrii Stepanov	6c2146bbc1	[#1334] metabase: Add upgrade from v2 to v3 ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-30 09:06:20 +03:00
Evgenii Stratonikov	03976c6ed5	[#1341] .golangci.yml: Replace exportloopref with copyloopvar ... exportloopref is deprecated. gopatch: ``` @@ var index, value identifier var slice expression @@ for index, value := range slice { ... -value := value ... } @@ var index, value identifier var slice expression @@ for index, value := range slice { ... -index := index ... } @@ var value identifier var channel expression @@ for value := range channel { ... -value := value ... } ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-28 15:44:41 +00:00
Evgenii Stratonikov	7e97df4878	[#1341] Makefile: Update golangci-lint ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-28 15:44:41 +00:00
Evgenii Stratonikov	01b6f1733c	[#1341] Makefile: Build linter with -trimpath ... Fix error with go1.23: ``` Error: build linters: unable to load custom analyzer "truecloudlab-linters": ../linters/bin/external_linters.so, plugin.Open("/repo/frostfs/linters/bin/external_linters"): plugin was built with a different version of package cmp Failed executing command with error: build linters: unable to load custom analyzer "truecloudlab-linters": ../linters/bin/external_linters.so, plugin.Open("/repo/frostfs/linters/bin/external_linters"): plugin was built with a different version of package cmp ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-28 15:44:41 +00:00
Dmitrii Stepanov	7abbdca064	[#1340] getSvc: Fix access denied error handling ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-28 14:02:28 +03:00
Airat Arifullin	6488ddee88	[#1338] object: Fix range provider in Patch handler ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-08-27 16:51:12 +03:00
Airat Arifullin	d6b42972a8	[#1338] object: Fix audit patch stream ... Signed-off-by: Airat Arifullin <a.arifullin@yadro.com>	2024-08-27 16:18:29 +03:00
Evgenii Stratonikov	5e9a97fd3e	[#1336] go.mod: Update api-go and sdk-go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-27 14:48:43 +03:00
Evgenii Stratonikov	fa7f9fbce2	[#1333] go.mod: Update api-go ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-26 15:37:43 +03:00
Evgenii Stratonikov	806ea37101	[#1328] pilorama: Do not skip items in SortedByFilename ... Benchmark results: ``` goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/pilorama cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ new │ │ sec/op │ sec/op vs base │ ForestSortedIteration/bbolt,root-8 207.2µ ± 6% 173.6µ ± 6% -16.23% (p=0.000 n=10) ForestSortedIteration/bbolt,leaf-8 3.910µ ± 5% 3.928µ ± 7% ~ (p=0.529 n=10) geomean 28.46µ 26.11µ -8.27% ``` They are not representative, as the worst case is when we have multiple items of different lengths. However, `FileName` is usually less than 100 in practice, so the asymptotics is the same. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-26 06:11:32 +00:00
Evgenii Stratonikov	80099d9a2f	[#1328] pilorama: Add tricky test for SortedByFilename ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-26 06:11:32 +00:00
Dmitrii Stepanov	a059a7dcf0	[#1329] cli: Skip linking objects in complex object processing ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-23 13:51:20 +03:00
Dmitrii Stepanov	bd24beecf8	[#1329] putSvc: Reset SuccessAfter for non-EC objects in EC container broadcasting ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-23 13:51:16 +03:00
Aleksey Savchuk	dfe825b81b	[#1309] test: Clean up config after tests ... Signed-off-by: Aleksey Savchuk <a.savchuk@yadro.com>	2024-08-22 08:49:10 +00:00
Dmitrii Stepanov	76f67ea34e	[#1323] metabase: Bump version ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Dmitrii Stepanov	7d0d781db1	[#1323] metabase: Drop user attribute index ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Dmitrii Stepanov	0f08a2efba	[#1323] metabase: Resolve funlen linter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Dmitrii Stepanov	7bf20c9f1f	[#1323] metabase: Add expiration epoch buckets ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Dmitrii Stepanov	2542d4f5df	[#1323] metabase: Drop payload checksum index ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Dmitrii Stepanov	15dae8685e	[#1323] metabase: Drop ownerID index ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2024-08-22 08:21:40 +00:00
Evgenii Stratonikov	7bca428db0	[#1322] Use new protobuf marshaler ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2024-08-22 07:17:41 +00:00
Ekaterina Lebedeva	a345c972bf	[#1316] lint: Fix warnings ... Renamed parameters `min/max` to avoid conflicts with predeclared identifiers. Replaced background context with parent context without cancellation in closer functions in frostfs-node. Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-08-21 18:18:25 +03:00
Ekaterina Lebedeva	8c1082b31a	[#1316] go.mod: Bump go version to 1.22 ... Signed-off-by: Ekaterina Lebedeva <ekaterina.lebedeva@yadro.com>	2024-08-21 15:09:29 +03:00
Anton Nikiforov	cfda9003a7	[#1318] meta: Add test TestInhumeECObject ... Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>	2024-08-20 16:21:27 +03:00