TrueCloudLab/frostfs-node
20
1
Fork 27
Code Issues 101 Pull requests 5 Projects Releases 1 Activity Actions

Add page_size config parameter for writecache #1302

Merged
fyrchik merged 1 commit from dstepanov-yadro/frostfs-node:fix/writecache_page_size into master 2024-09-04 19:51:11 +00:00
Conversation 0 Commits 1 Files changed 12 +35 -4
dstepanov-yadro commented 2024-08-08 10:38:30 +00:00
Member
Copy link

Page size impacts on put->flush->put performance:

go test -benchmem -run=^$ -tags integration -bench ^BenchmarkWriteAfterDelete$ git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/writecache/benchmark -count=1 -benchtime=100000x
goos: linux
goarch: amd64
pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/writecache/benchmark
cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz

page size = 4KB

BenchmarkWriteAfterDelete/8192B_before-8                  100000            171722 ns/op           69292 B/op        110 allocs/op
BenchmarkWriteAfterDelete/8192B_after-8                   100000            284256 ns/op           76790 B/op        111 allocs/op

BenchmarkWriteAfterDelete/16384B_before-8                 100000            200868 ns/op          126701 B/op        111 allocs/op
BenchmarkWriteAfterDelete/16384B_after-8                  100000            518799 ns/op          151644 B/op        113 allocs/op

BenchmarkWriteAfterDelete/32768B_before-8                  50000            296476 ns/op          237870 B/op        110 allocs/op
BenchmarkWriteAfterDelete/32768B_after-8                   50000            702433 ns/op          279851 B/op        113 allocs/op

BenchmarkWriteAfterDelete/65536B_before-8                  50000            814187 ns/op          457027 B/op        117 allocs/op
BenchmarkWriteAfterDelete/65536B_after-8                   50000           1289879 ns/op          532543 B/op        114 allocs/op     111 allocs/op


page size = 16KB

BenchmarkWriteAfterDelete/8192B_before-8                  100000            147907 ns/op           84450 B/op        104 allocs/op
BenchmarkWriteAfterDelete/8192B_after-8                   100000            180532 ns/op           88176 B/op        106 allocs/op

BenchmarkWriteAfterDelete/16384B_before-8                 100000            239426 ns/op          142686 B/op        105 allocs/op
BenchmarkWriteAfterDelete/16384B_after-8                  100000            270541 ns/op          149202 B/op        107 allocs/op

BenchmarkWriteAfterDelete/32768B_before-8                  50000            331565 ns/op          252709 B/op        104 allocs/op
BenchmarkWriteAfterDelete/32768B_after-8                   50000            493900 ns/op          266252 B/op        106 allocs/op

BenchmarkWriteAfterDelete/65536B_before-8                  50000            797049 ns/op          467213 B/op        105 allocs/op
BenchmarkWriteAfterDelete/65536B_after-8                   50000            941480 ns/op          498645 B/op        108 allocs/op


page size = 32KB

BenchmarkWriteAfterDelete/8192B_before-8                  100000            162660 ns/op           65424 B/op        101 allocs/op
BenchmarkWriteAfterDelete/8192B_after-8                   100000            186344 ns/op           64717 B/op        100 allocs/op

BenchmarkWriteAfterDelete/16384B_before-8                 100000            268184 ns/op          161061 B/op        103 allocs/op
BenchmarkWriteAfterDelete/16384B_after-8                  100000            301734 ns/op          167633 B/op        105 allocs/op

BenchmarkWriteAfterDelete/32768B_before-8                  50000            408917 ns/op          282038 B/op        103 allocs/op
BenchmarkWriteAfterDelete/32768B_after-8                   50000            568146 ns/op          290141 B/op        104 allocs/op

BenchmarkWriteAfterDelete/65536B_before-8                  50000            834409 ns/op          494080 B/op        104 allocs/op
BenchmarkWriteAfterDelete/65536B_after-8                   50000            934131 ns/op          516328 B/op        105 allocs/op

So allow to specify page size via config.

Page size impacts on `put->flush->put` performance: ``` go test -benchmem -run=^$ -tags integration -bench ^BenchmarkWriteAfterDelete$ git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/writecache/benchmark -count=1 -benchtime=100000x goos: linux goarch: amd64 pkg: git.frostfs.info/TrueCloudLab/frostfs-node/pkg/local_object_storage/writecache/benchmark cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz page size = 4KB BenchmarkWriteAfterDelete/8192B_before-8 100000 171722 ns/op 69292 B/op 110 allocs/op BenchmarkWriteAfterDelete/8192B_after-8 100000 284256 ns/op 76790 B/op 111 allocs/op BenchmarkWriteAfterDelete/16384B_before-8 100000 200868 ns/op 126701 B/op 111 allocs/op BenchmarkWriteAfterDelete/16384B_after-8 100000 518799 ns/op 151644 B/op 113 allocs/op BenchmarkWriteAfterDelete/32768B_before-8 50000 296476 ns/op 237870 B/op 110 allocs/op BenchmarkWriteAfterDelete/32768B_after-8 50000 702433 ns/op 279851 B/op 113 allocs/op BenchmarkWriteAfterDelete/65536B_before-8 50000 814187 ns/op 457027 B/op 117 allocs/op BenchmarkWriteAfterDelete/65536B_after-8 50000 1289879 ns/op 532543 B/op 114 allocs/op 111 allocs/op page size = 16KB BenchmarkWriteAfterDelete/8192B_before-8 100000 147907 ns/op 84450 B/op 104 allocs/op BenchmarkWriteAfterDelete/8192B_after-8 100000 180532 ns/op 88176 B/op 106 allocs/op BenchmarkWriteAfterDelete/16384B_before-8 100000 239426 ns/op 142686 B/op 105 allocs/op BenchmarkWriteAfterDelete/16384B_after-8 100000 270541 ns/op 149202 B/op 107 allocs/op BenchmarkWriteAfterDelete/32768B_before-8 50000 331565 ns/op 252709 B/op 104 allocs/op BenchmarkWriteAfterDelete/32768B_after-8 50000 493900 ns/op 266252 B/op 106 allocs/op BenchmarkWriteAfterDelete/65536B_before-8 50000 797049 ns/op 467213 B/op 105 allocs/op BenchmarkWriteAfterDelete/65536B_after-8 50000 941480 ns/op 498645 B/op 108 allocs/op page size = 32KB BenchmarkWriteAfterDelete/8192B_before-8 100000 162660 ns/op 65424 B/op 101 allocs/op BenchmarkWriteAfterDelete/8192B_after-8 100000 186344 ns/op 64717 B/op 100 allocs/op BenchmarkWriteAfterDelete/16384B_before-8 100000 268184 ns/op 161061 B/op 103 allocs/op BenchmarkWriteAfterDelete/16384B_after-8 100000 301734 ns/op 167633 B/op 105 allocs/op BenchmarkWriteAfterDelete/32768B_before-8 50000 408917 ns/op 282038 B/op 103 allocs/op BenchmarkWriteAfterDelete/32768B_after-8 50000 568146 ns/op 290141 B/op 104 allocs/op BenchmarkWriteAfterDelete/65536B_before-8 50000 834409 ns/op 494080 B/op 104 allocs/op BenchmarkWriteAfterDelete/65536B_after-8 50000 934131 ns/op 516328 B/op 105 allocs/op ``` So allow to specify page size via config.
dstepanov-yadro force-pushed fix/writecache_page_size from c84f071b44 to d6fa39cdf9 2024-08-08 10:39:59 +00:00 Compare
fyrchik approved these changes 2024-08-08 10:46:51 +00:00
Dismissed
fyrchik reviewed 2024-08-08 10:47:28 +00:00
pkg/local_object_storage/writecache/benchmark/writecache_test.go Outdated
@ -30,0 +42,4 @@
})
require.NoError(b, cache.Flush(context.Background(), false, false))
b.Run(fmt.Sprintf("%dB_after", payloadSize), func(b *testing.B) {
b.N = objectsCount
fyrchik commented 2024-08-08 10:47:28 +00:00
Owner
Copy link

Please, check staticcheck warnings.

Please, check staticcheck warnings.
fyrchik commented 2024-08-08 10:48:03 +00:00
Owner
Copy link

And we need new benchmark results after the fix.

And we need new benchmark results after the fix.
dstepanov-yadro commented 2024-08-08 12:59:37 +00:00
Author
Member
Copy link

done

done
fyrchik dismissed fyrchik's review 2024-08-08 10:47:44 +00:00
Reason:

assigning to b.N

dstepanov-yadro force-pushed fix/writecache_page_size from d6fa39cdf9 to a7e7a8c8bc 2024-08-08 12:57:50 +00:00 Compare
fyrchik reviewed 2024-08-08 14:15:11 +00:00
pkg/local_object_storage/writecache/util.go Outdated
@ -11,3 +11,3 @@
// OpenDB opens BoltDB instance for write-cache. Opens in read-only mode if ro is true.
func OpenDB(p string, ro bool, openFile func(string, int, fs.FileMode) (*os.File, error)) (*bbolt.DB, error) {
func OpenDB(p string, ro bool, openFile func(string, int, fs.FileMode) (*os.File, error), pageSize int) (*bbolt.DB, error) {
fyrchik commented 2024-08-08 14:15:11 +00:00
Owner
Copy link

It should be noted, that this parameter does nothing if the DB already exists.

It should be noted, that this parameter does nothing if the DB already exists.
dstepanov-yadro commented 2024-08-08 14:21:14 +00:00
Author
Member
Copy link

Done in storage-node-configuration.md

Done in storage-node-configuration.md
fyrchik marked this conversation as resolved
dstepanov-yadro force-pushed fix/writecache_page_size from a7e7a8c8bc to 4c6d082529 2024-08-08 14:20:43 +00:00 Compare
fyrchik reviewed 2024-08-08 14:29:28 +00:00
pkg/local_object_storage/writecache/benchmark/writecache_test.go Outdated
@ -28,2 +29,4 @@
}
func BenchmarkWriteAfterDelete(b *testing.B) {
const payloadSize = 32 << 10
fyrchik commented 2024-08-08 14:29:28 +00:00
Owner
Copy link

It would be nice to have different payload sizes, at least to estimate possible degradations.

It would be nice to have different payload sizes, at least to estimate possible degradations.
fyrchik approved these changes 2024-08-08 14:29:30 +00:00
dstepanov-yadro requested review from storage-core-committers 2024-08-09 06:17:06 +00:00
dstepanov-yadro requested review from storage-core-developers 2024-08-09 06:17:06 +00:00
acid-ant approved these changes 2024-08-09 08:02:32 +00:00
dstepanov-yadro force-pushed fix/writecache_page_size from 4c6d082529 to 3fd03040b8 2024-08-09 08:18:54 +00:00 Compare
aarifullin approved these changes 2024-08-09 09:11:01 +00:00
dstepanov-yadro force-pushed fix/writecache_page_size from 3fd03040b8 to 68029d756e 2024-08-09 09:12:29 +00:00 Compare
fyrchik approved these changes 2024-08-09 09:25:19 +00:00
fyrchik merged commit 68029d756e into master 2024-08-09 09:25:42 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
acid-ant
aarifullin
fyrchik
Labels
Clear labels   
P0

Highest

  
P1

High

  
P2

Medium

  
P3

Low

  
badger

Badger DB related issues

  
frostfs-adm

   
frostfs-cli

   
frostfs-ir

   
frostfs-lens

   
frostfs-node

   
good first issue

Good for newcomers

  
triage
  
Infrastructure

Infrastructure, Automation, CI/CD or DevOps related stuff

  
blocked

The issue or PR is blocked by something

  
bug

Something is not working

  
config

Configuration format update or breaking change

  
discussion

Talking about something in order to reach a decision or to exchange ideas

  
documentation

Documentation related

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
go

Language features adoption

  
help wanted

Extra attention is needed

  
internal

   
invalid

Something is wrong

  
kludge

This is a kludge and we know it

  
observability

Related to metrics, tracing and logs

  
perfomance

Perfomance related

  
question

More information is needed

  
refactoring

Refactoring

  
wontfix

This won't be fixed

No labels
P0
P1
P2
P3
badger
frostfs-adm
frostfs-cli
frostfs-ir
frostfs-lens
frostfs-node
good first issue
triage
Infrastructure
blocked
bug
config
discussion
documentation
duplicate
enhancement
go
help wanted
internal
invalid
kludge
observability
perfomance
question
refactoring
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-node#1302
No description provided.
Delete branch "dstepanov-yadro/frostfs-node:fix/writecache_page_size"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?