* replacer: evaluate format once and improve perf by ~3x
This improves the performance of logging by almost 3x and reduces memory
usage by ~8x.
Benchmark results:
benchmark old ns/op new ns/op delta
BenchmarkReplacer-12 644 324 -49.69%
BenchmarkReplacer_CommonLogFormat-12 4228 1471 -65.21%
benchmark old allocs new allocs delta
BenchmarkReplacer-12 8 2 -75.00%
BenchmarkReplacer_CommonLogFormat-12 51 17 -66.67%
benchmark old bytes new bytes delta
BenchmarkReplacer-12 240 48 -80.00%
BenchmarkReplacer_CommonLogFormat-12 3723 446 -88.02%
* replacer: code review comments
* bufPool: document why we use a pointer to a slice
* parseFormat: fix confusing comment
* TestParseFormat_Nodes: rename to TestParseFormatNodes
* replacer: use a value for bufPool instead of a pointer
* replacer: remove comment
* replacer: replace labels with knownLabels
The previous slice of label names is no longer needed.
* pkg/replace: make it more efficient.
Remove the map that is allocated on every write and make it more static,
but just defining a function that gets called for a label and returns
its value.
Remove the interface definition and just implement what is needed in our
case. Add benchmark test for replace as well.
Extend metadata test to test multiple values (pretty sure this didn't
work, but there wasn't a test for it, so can't be sure).
Update all callers to use it - concurrent use should be fine as we pass
everything by value.
Benchmarks in replacer:
new: BenchmarkReplacer-4 300000 4717 ns/op 240 B/op 8 allocs/op
old: BenchmarkReplacer-4 300000 4368 ns/op 384 B/op 11 allocs/op
Added benchmark function to the old code to test it.
~~~
func BenchmarkReplacer(b *testing.B) {
w := dnstest.NewRecorder(&test.ResponseWriter{})
r := new(dns.Msg)
r.SetQuestion("example.org.", dns.TypeHINFO)
r.MsgHdr.AuthenticatedData = true
b.ResetTimer()
b.ReportAllocs()
repl := New(context.TODO(), r, w, "")
for i := 0; i < b.N; i++ {
repl.Replace("{type} {name} {size}")
}
}
~~~
New code contains (of course a different one). The amount of ops is
more, which might be good to look at some more. For all the allocations
is seems it was quite performant.
This looks to be 50% faster, and there is less allocations in log
plugin:
old: BenchmarkLogged-4 20000 70526 ns/op
new: BenchmarkLogged-4 30000 57558 ns/op
Signed-off-by: Miek Gieben <miek@miek.nl>
* Stickler bot
Signed-off-by: Miek Gieben <miek@miek.nl>
* Improve test coverage
Signed-off-by: Miek Gieben <miek@miek.nl>
* typo
Signed-off-by: Miek Gieben <miek@miek.nl>
* Add test for malformed log lines
Signed-off-by: Miek Gieben <miek@miek.nl>
Default to using pkg/log for all logging and use a fixed time prefix
which is RFC3339Millli (doesn't exist in time, so we just extended
RFC3339), i.e. Nano might be pushing it.
Logs go from:
2018/10/30 19:14:55 [INFO] CoreDNS-1.2.5
2018/10/30 19:14:55 [INFO] linux/amd64, go1.11,
to:
2018-10-30T19:10:07.547Z [INFO] CoreDNS-1.2.5
2018-10-30T19:10:07.547Z [INFO] linux/amd64, go1.11,
Which includes the timezone - which oddly the std log package doesn't
natively do.
Signed-off-by: Miek Gieben <miek@miek.nl>
* Current stage of the log files. Test need to be done as well as formatting of times.
* Finished testing. All altered classes test pass along with my additions
* Updated the replacer package to print the units as well. May take out.
* Changed the time units to be within the rules. Fixed the test as well.
* Fixed some tests, updated the readme, fixed the replacer class.
* Updates of standardizing only to seconds in response duration. Need to revert README.
* Reverted readme.
* Added a small test in new replacer.
* Changed replacer to inline the strconv for duration.
Add a full test server impl in this new package + tests. Move
dnsrecorder into this package as well and finish up the commented out
tests that were left in the old dnsrecorder package.
Update all callers and tests.
* Rename middleware to plugin
first pass; mostly used 'sed', few spots where I manually changed
text.
This still builds a coredns binary.
* fmt error
* Rename AddMiddleware to AddPlugin
* Readd AddMiddleware to remain backwards compat
2017-09-14 09:36:06 +01:00
Renamed from middleware/pkg/replacer/replacer.go (Browse further)
