905 commits

Author	SHA1	Message	Date
Evgenii Stratonikov	96e690883f	[#638] Unify test loggers ... In some places we have debug=false, in others debug=true. Let's be consistent. Semantic patch: ``` @@ @@ -test.NewLogger(..., false) +test.NewLogger(..., true) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-23 11:21:05 +00:00
Evgenii Stratonikov	322c1dc273	[#638] Use test.NewLogger() in tests ... Semantic patch (restricted to **/*_test.go): ``` @@ @@ +import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger/test" -import "go.uber.org/zap" -import "git.frostfs.info/TrueCloudLab/frostfs-node/pkg/util/logger" -&logger.Logger{Logger: zap.L()} +test.NewLogger(t, false) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-23 11:21:05 +00:00
Evgenii Stratonikov	82cc453be9	[#xx] shard: Fix data race in metrics tests ... Protect test metric store fields with a mutex. Probably, not every field should be protected, but better safe than sorry. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-23 10:26:12 +00:00
Alejandro Lopez	345a1a69a2	[#635] Use internal key type when deleting from badger wc ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-22 10:53:19 +03:00
Dmitrii Stepanov	354a92ea2c	[#602] blobovnicza: Add leaf width implementation ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-21 10:27:32 +03:00
Alexander Chuprov	4d9a6c07fb	[#618] core: Replace fmt.Sprintf with strconv.FormatUint ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-08-21 07:14:50 +00:00
Dmitrii Stepanov	f2811f8585	[#602] metrics: Add blobovnicza items counter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-18 13:01:27 +03:00
Dmitrii Stepanov	c4e1d8eb07	[#602] node: Fix blobovnicza typos ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-18 11:14:10 +03:00
Dmitrii Stepanov	10e63537b2	[#602] metrics: Rename blobovnicza size metric ... `Size` is not size, but open db size. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-17 19:17:35 +00:00
Dmitrii Stepanov	809e97626b	[#602] blobovnicza: Fix size counter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-17 19:17:35 +00:00
Dmitrii Stepanov	2e49d7ea7e	[#602] blobovnicza: Init before using ... Fix blobovnicza size: after restart size metric resets. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-17 19:17:35 +00:00
Evgenii Stratonikov	f7042c5a6f	[#609] Replace zaptest.NewLogger() with zap.L() ... Semantic patch: ``` @@ @@ -import "go.uber.org/zap/zaptest" +import "go.uber.org/zap" -zaptest.NewLogger(t) +zap.L() ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-17 16:51:10 +00:00
Evgenii Stratonikov	e604a3d749	[#607] *: Use zap.Stringer() where possible ... Semantic patch: ``` @@ var f expression var t expression var a expression @@ f( ..., - zap.String(t, a.String()), + zap.Stringer(t, a), ..., ) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-17 16:50:02 +00:00
Evgenii Stratonikov	a8de37c8a2	[#607] *: Remove redundant if on error returns ... Semantic patch: ``` @@ @@ -if err != nil { return err } -return nil +return err ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-17 16:50:02 +00:00
Evgenii Stratonikov	5335e7089e	[#615] pilorama: Speedup TestForest_ApplyRandom() ... Some of our pilorama tests fail on CI. The reasons are not obvious, but one possible improvement is using `WithNoSync` option for these. It should have much effect, because we are writing on the tmpfs, but doesn't hurt anyway. If I replace `t.TempDir()` with a local directory, test execution time goes down from 5s (sync) to 0.4s (nosync), which is the same time as with `t.TempDir()`. Maybe we have some strange CI configuration. ``` panic: test timed out after 10m0s running tests: TestForest_ApplyRandom (8m22s) TestForest_ApplyRandom/bbolt (8m21s) ... goroutine 170 [syscall]: syscall.Syscall(0xc000100000?, 0xc00047b758?, 0x6aff9a?, 0xc00041c1b0?) /opt/hostedtoolcache/go/1.20.7/x64/src/syscall/syscall_linux.go:69 +0x27 syscall.Fdatasync(0x9e35c0?) /opt/hostedtoolcache/go/1.20.7/x64/src/syscall/zsyscall_linux_amd64.go:418 +0x2a go.etcd.io/bbolt.fdatasync(0xc000189000?) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-16 19:09:08 +00:00
Dmitrii Stepanov	2efe9cc1be	[#585] writecache: Fix DB counter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-16 14:47:44 +03:00
Dmitrii Stepanov	58c8722c81	[#585] fstree: Add optional file counter ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-16 14:47:44 +03:00
Dmitrii Stepanov	baad49990c	[#585] fstree: Return logical error if object deleted ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-16 14:47:44 +03:00
Dmitrii Stepanov	0c52186572	[#585] fstree: Remove unused method ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-16 14:47:44 +03:00
Dmitrii Stepanov	eec97d177e	[#585] writecache: Count items periodically ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-16 14:47:44 +03:00
Evgenii Stratonikov	d15199c5d8	[#596] engine: Consider context errors as logical ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-16 10:39:41 +03:00
Alejandro Lopez	88b6755c5e	[#598] Fix use-after-close bug in badger writecache ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-15 07:04:06 +00:00
Alejandro Lopez	ae8be495c8	[#xx] Avoid manual management of files in tests ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-14 14:01:39 +03:00
Alejandro Lopez	376f03a445	[#598] Hold mode mutex when setting mode ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-14 07:12:27 +00:00
Alexander Chuprov	21800e9fcc	[#162] core: Move literals to constants ... Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-08-11 15:48:42 +03:00
Evgenii Stratonikov	b44a8dd46c	[#597] *: Fix linter warnings ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-11 09:55:51 +03:00
Alejandro Lopez	8f994163ee	[#586] Fix writecache benchmarks and refactor hacky NeedsCompression ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-10 08:05:18 +00:00
Alejandro Lopez	023b90342c	[#584] Disable compression in badger writecache ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-10 10:14:06 +03:00
Evgenii Stratonikov	d641cba2fc	[#587] Do not use math/rand.Read ... Fix staticcheck warnings after go1.20 update. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-08-09 16:02:44 +03:00
Alejandro Lopez	5b7e4a51b7	[#481] Update frostfs-sdk-go and error pointer receivers ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-09 10:26:53 +00:00
Alejandro Lopez	ae322e9f73	[#576] Set SyncWrites for badger writecache by default ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-09 11:28:26 +03:00
Alejandro Lopez	7da4306e38	[#575] writecache: Fix log level for badger writecache ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-08 08:16:36 +00:00
Dmitrii Stepanov	0e697266c3	[#563] writecache: Fix metrics and bolt delete ... Estimate cache size after delete objects to update metric. Update counters on small object deletion. Do not count bbolt DB file as FSTree object. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-08-07 12:53:28 +00:00
Alejandro Lopez	1a0cb0f34a	[#421] Try using badger for the write-cache ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-07 08:16:57 +00:00
Alejandro Lopez	65c72f3e0b	[#559] Remove manual path handling in fstree tests ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-08-03 10:03:41 +03:00
Evgenii Stratonikov	35dc64bd7b	[#547] metabase: Fix datarace in tests ... Quite an old one bf9e938a3b. Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-31 15:35:15 +00:00
Evgenii Stratonikov	ec8b4fdc48	[#541] writecache/test: Close writecache on exit ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-29 10:57:25 +00:00
Evgenii Stratonikov	ad5f527bd3	[#541] writecache/test: Remove initWC() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-07-29 10:57:25 +00:00
Dmitrii Stepanov	af82c2865e	[#335] treesvc: Fix inmemory unit tests and nil meta items ... Bolt forest saves empty slice of items. Now inmemory forest does it the same way. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-20 10:14:10 +03:00
Dmitrii Stepanov	b4e72a2dfd	[#335] treesvc: Sort nodes by Filename in GetSubTree ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-20 10:14:10 +03:00
Dmitrii Stepanov	3e8de14e7d	[#382] evacuate: Fix unit tests ... Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-07-17 10:24:05 +00:00
Alejandro Lopez	11027945d8	[#479] writecache: Fix writecache fstree flush premature ctx cancel ... Signed-off-by: Alejandro Lopez <a.lopez@yadro.com>	2023-07-14 10:25:52 +03:00
Alexander Chuprov	033eaf77e1	[#496] node: Fix linter importas ... Standardize the alias of the import frostfs-sdk-go/object as objectSDK. Signed-off-by: Alexander Chuprov <a.chuprov@yadro.com>	2023-07-06 15:36:41 +03:00
Evgenii Stratonikov	cab51c8cbe	[#1] metabase: Rename blindlyProcess() ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-28 12:13:20 +00:00
Evgenii Stratonikov	f4c71cea65	[#1] *: Replace outdated FIXME/TODO links ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-28 12:13:20 +00:00
Dmitrii Stepanov	43d263c3d5	[#428] linter: Fix unkeyed assignment ... Thanks to gopls. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-26 13:29:39 +00:00
Dmitrii Stepanov	cac4ed93d6	[#428] engine: Add low_mem config parameter ... Concurrent initialization in case of the metabase resync leads to high memory consumption and potential OOM. Signed-off-by: Dmitrii Stepanov <d.stepanov@yadro.com>	2023-06-26 13:29:39 +00:00
Evgenii Stratonikov	4bf345225c	[#447] pilorama: Use named constant for the key size ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-26 07:42:05 +00:00
Evgenii Stratonikov	b4ce0b0412	[#447] pilorama: Do not undo log for create ops ... ``` goos: linux goarch: amd64 cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz │ old │ new │ │ sec/op │ sec/op vs base │ Create-8 36.48µ ± 11% 30.34µ ± 14% -16.84% (p=0.000 n=10) │ old │ new │ │ B/op │ B/op vs base │ Create-8 43.01Ki ± 4% 37.78Ki ± 5% -12.15% (p=0.000 n=10) │ old │ new │ │ allocs/op │ allocs/op vs base │ Create-8 166.0 ± 3% 146.0 ± 3% -12.05% (p=0.000 n=10) ``` Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-26 07:42:05 +00:00
Evgenii Stratonikov	dd3874eff1	[#447] pilorama: Add benchmark for create ops ... Signed-off-by: Evgenii Stratonikov <e.stratonikov@yadro.com>	2023-06-26 07:42:05 +00:00