* Added identity and version support to dnstap plugin
Signed-off-by: Daniel Jolly <code@danieljolly.com>
* Added missing commas
Signed-off-by: Daniel Jolly <code@danieljolly.com>
* Moved byte slice conversions to setup rather than handler.
Fixed indentation issue.
Signed-off-by: Daniel Jolly <code@danieljolly.com>
* Improved setup config parsing and added tests to detect various configurations
Signed-off-by: Daniel Jolly <code@danieljolly.com>
Signed-off-by: Daniel Jolly <code@danieljolly.com>
Co-authored-by: Daniel Jolly <code@danieljolly.com>
This PR cldean up remaining left over of github.com/golang/protobuf in coredns,
as github.com/golang/protobuf has been deprecated.
Note the github.com/golang/protobuf is now only indirectly
used by external libraries. Manually updating coredns' dependency to futher clean up has been tried.
However, it was not possible as external library usages of github.com/golang/protobuf are too interleaved.
This PR fixes 4939.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* plugin/dnstap: remove config struct
this struct is an uneeded intermidiate to get a dnstap it can be
removed. Remove the dnstapio subpkg: it's also not needed. Make *many*
functions and structs private now that we can.
Signed-off-by: Miek Gieben <miek@miek.nl>
* correct logging
Signed-off-by: Miek Gieben <miek@miek.nl>
* plugin/dnstap: remove encoder*.go
Those files reimplemented parts of the dnstap spec, we can just use the
dnstap functions for that. This leaves all the queuing that is enabled
and drops messages if the dnstap reader can't keep up. In the new code
flush() would never return an error (at least I couldn't make it do so),
so the reconnect functionally is moved to kick off when we get write
errors.
Some smaller cosmetic changes as well, `d.socket` is now `proto`, which
makes the dial() function smaller.
Total testing time is now <1s (which was the impetus to look into this
plugin *again*).
See #4238
The buffered channel needs to be sized correctly, as we may need to do
some queing if the dnstap reader can't keep up.
Signed-off-by: Miek Gieben <miek@miek.nl>
* add missing file
Signed-off-by: Miek Gieben <miek@miek.nl>
* update doc on queing
Signed-off-by: Miek Gieben <miek@miek.nl>
* plugin/dnstap: various cleanups
A recent issue made me look into this plugin, I suspect various other
cleanups (hopefully deletion of code) can be made as well
Remove identical functions ToClientQuery etc, and just use tap.Message
as the base type in plugin. Keep msg/ for a few helper functions that
may proof useful.
This remove the whole test directory as we will just check the things we
are interested in which gives much better feedback and keeps that code
closer together.
tapwr dir is also not needed, writer_test.go was just duplicating the
tests already done. This moves writer.go to the top directory.
Make the only user of dnstap, the forward plugin, use the newer code
also remove the test, a better test there would be a full e2e test to
see the correct thing happens.
Cleanup the Tapper interface and move it to dnstapio where it belongs,
remove higher level interfaces that are not used. This remove
dnstap.Tapper and dnstap.IORoutines.
Use the standard mechanism for getting access to a plugin and remove
shuffling the plugin into the context.
Signed-off-by: Miek Gieben <miek@miek.nl>
* use opts to get the correct proto
Signed-off-by: Miek Gieben <miek@miek.nl>
* Various fixes
Signed-off-by: Miek Gieben <miek@miek.nl>
* Remove bad addr test, as dnstap is only called from within coredns where these fields have been preparsed
Signed-off-by: Miek Gieben <miek@miek.nl>
* dnstap: remove saving the error
all these fields have been preparsed, no need for dnstap to be pedantic
and check (and save!) this error again.
Simplifies it a bit more.
Signed-off-by: Miek Gieben <miek@miek.nl>
* Update plugin/forward/dnstap.go
Co-authored-by: Ruslan Drozhdzh <30860269+rdrozhdzh@users.noreply.github.com>
* Code review
Signed-off-by: Miek Gieben <miek@miek.nl>
* add back in preferUDP
Signed-off-by: Miek Gieben <miek@miek.nl>
* nit
Signed-off-by: Miek Gieben <miek@miek.nl>
Co-authored-by: Ruslan Drozhdzh <30860269+rdrozhdzh@users.noreply.github.com>
* For caddy v1 in our org
This RP changes all imports for caddyserver/caddy to coredns/caddy. This
is the v1 code of caddy.
For the coredns/caddy repo the following changes have been made:
* anything not needed by us is deleted
* all `telemetry` stuff is deleted
* all its import paths are also changed to point to coredns/caddy
* the v1 branch has been moved to the master branch
* a v1.1.0 tag has been added to signal the latest release
Signed-off-by: Miek Gieben <miek@miek.nl>
* Fix imports
Signed-off-by: Miek Gieben <miek@miek.nl>
* Group coredns/caddy with out plugins
Signed-off-by: Miek Gieben <miek@miek.nl>
* remove this file
Signed-off-by: Miek Gieben <miek@miek.nl>
* Relax import ordering
github.com/coredns is now also a coredns dep, this makes
github.com/coredns/caddy fit more natural in the list.
Signed-off-by: Miek Gieben <miek@miek.nl>
* Fix final import
Signed-off-by: Miek Gieben <miek@miek.nl>
* Move to CODEOWNERS
No change in who own what; just a move to CODEOWNERS. This allows
dreck cleanups.
Added .dreck.yaml for alias and exec.
Fixes: #3486
Signed-off-by: Miek Gieben <miek@miek.nl>
* stickler bot
Signed-off-by: Miek Gieben <miek@miek.nl>
* sort the file
Signed-off-by: Miek Gieben <miek@miek.nl>
* Run gostaticheck
Run gostaticcheck on the codebase and fix almost all flagged items.
Only keep
* coremain/run.go:192:2: var appVersion is unused (U1000)
* plugin/chaos/setup.go:54:3: the surrounding loop is unconditionally terminated (SA4004)
* plugin/etcd/setup.go:103:3: the surrounding loop is unconditionally terminated (SA4004)
* plugin/pkg/replacer/replacer.go:274:13: argument should be pointer-like to avoid allocations (SA6002)
* plugin/route53/setup.go:124:28: session.New is deprecated: Use NewSession functions to create sessions instead. NewSession has the same functionality as New except an error can be returned when the func is called instead of waiting to receive an error until a request is made. (SA1019)
* test/grpc_test.go:25:69: grpc.WithTimeout is deprecated: use DialContext and context.WithTimeout instead. Will be supported throughout 1.x. (SA1019)
The first one isn't true, as this is set via ldflags. The rest is
minor. The deprecation should be fixed at some point; I'll file some
issues.
Signed-off-by: Miek Gieben <miek@miek.nl>
* Make sure to plug in the plugins
import the plugins, that file that did this was removed, put it in the
reload test as this requires an almost complete coredns server.
Signed-off-by: Miek Gieben <miek@miek.nl>
Remove all these uses and just make them work on caddy.Controller. Also
don't export parsing functions as their should be private to the plugin.
Signed-off-by: Miek Gieben <miek@miek.nl>
Abstract the caddy call and make it simpler.
See #3261 for some part of the discussion.
Go from:
~~~ go
func init() {
caddy.RegisterPlugin("any", caddy.Plugin{
ServerType: "dns",
Action: setup,
})
}
~~~
To:
~~~ go
func init() { plugin.Register("any", setup) }
~~~
This requires some external documents in coredns.io to be updated as
well; the old way still works, so it's backwards compatible.
Signed-off-by: Miek Gieben <miek@miek.nl>
* Update Caddy to 1.0.1, and update import path
This fix updates caddy to 1.0.1 and also
updates the import path to github.com/caddyserver/caddy
This fix fixes 2959
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Also update plugin.cfg
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Update and bump zplugin.go
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Stop importing testing in the main binary
Stop importing "testing" into the main binary:
* test/helpers.go imported it; remote that and change function signature
* update all tests that use this
Signed-off-by: Miek Gieben <miek@miek.nl>
* Drop import testing from metrics plugin
Signed-off-by: Miek Gieben <miek@miek.nl>
* more fiddling
Signed-off-by: Miek Gieben <miek@miek.nl>
* Move functions from pkg/transport to pkg/parse
Although "parse" is a fairly generic name I believe this is somewhat
better named. pkg/transport keeps a few constants that are uses
throughout for the rest is is renaming a bunch (and the fallout from
there to make things compile again).
Signed-off-by: Miek Gieben <miek@miek.nl>
* Fix tests
Signed-off-by: Miek Gieben <miek@miek.nl>
* Clean up tests logging
This cleans up the travis logs so you can see the failures better.
Older tests in tests/ would call log.SetOutput(ioutil.Discard) in
a haphazard way. This add log.Discard and put an `init` function in each
package's dir (no way to do this globally). The cleanup in tests/ is
clear.
All plugins also got this init function to have some uniformity and kill
any (future) logging there in the tests as well.
There is a one-off in pkg/healthcheck because that does log.
Signed-off-by: Miek Gieben <miek@miek.nl>
* bring back original log_test.go
Signed-off-by: Miek Gieben <miek@miek.nl>
* suppress logging here as well
Signed-off-by: Miek Gieben <miek@miek.nl>
Uppercase all these test errors as well. And extend the presubmit to
check for these in the future. Also do a slightly smarter grep to only
get t.<something>. as (because dump regexp) this also grep over non test
files.
* update docs
* plugins: use plugin specific logging
Hooking up pkg/log also changed NewWithPlugin to just take a string
instead of a plugin.Handler as that is more flexible and for instance
the Root "plugin" doesn't implement it fully.
Same logging from the reload plugin:
.:1043
2018/04/22 08:56:37 [INFO] CoreDNS-1.1.1
2018/04/22 08:56:37 [INFO] linux/amd64, go1.10.1,
CoreDNS-1.1.1
linux/amd64, go1.10.1,
2018/04/22 08:56:37 [INFO] plugin/reload: Running configuration MD5 = ec4c9c55cd19759ea1c46b8c45742b06
2018/04/22 08:56:54 [INFO] Reloading
2018/04/22 08:56:54 [INFO] plugin/reload: Running configuration MD5 = 9e2bfdd85bdc9cceb740ba9c80f34c1a
2018/04/22 08:56:54 [INFO] Reloading complete
* update docs
* better doc
* global: move to context
Move from golang.org/x/net/context to std lib's context.
Change done with:
for i in $(grep -l '/context' **/*.go); do sed -e 's|golang.org/x/net/context|context|' -i $i; echo $i; done
for i in **/*.go; do goimports -w $i; done
* drop from dns.pb.go as well
* Update all plugins to use plugin/pkg/log
I wish this could have been done with sed. Alas manually changed all
callers to use the new plugin/pkg/log package.
* Error -> Info
* Add docs to debug plugin as well
* Easier way to dnstap?
* Remove unnecessary function parameter from Tapper
* golint
* golint 2
* Proxy dnstap tests
* README.md & doc
* net.IP
* Proxy test was incorrect
* Small changes
* Update README.md
* Was not reporting dnstap errors + test
* Wasn't working at all, now it's ok
* Thanks Travis
This fix enables goimports check and fixes several imports format
so that the import sections are prettier, e.g.:
```
import (
- "github.com/miekg/dns"
"regexp"
"strconv"
"strings"
+
+ "github.com/miekg/dns"
)
```
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This should have everyone, but the process was quite manual. The rename
from middleware -> plugin also meant I had to do some extra digging on
who actually submitted the PR. I also double checked the current list of
people with commit access.
Every plugin now has an OWNERS, except *reverse*. I'll file a bug for
that.
* Add manual pages
Generate manual pages from the README and extend README with Name and
Description sections.
The generation requires 'ronn' which may not be available. Just check in
all generated manual pages.
- added dnstapEncoder object which incapsulates marshalling of dnstap
messages to protobuf and writing data to connection
- dnstapEncoder writes data directly to connection object. It doesn't
use the framestream's "write" method, because it writes data to
intermediate buffer (bufio.Writer) which leads to unnecessary
data copying and drops the performance
- dnstapEncoder reuses a preallocated buffer for marshalling dnstap
messages. Many messages are added to the same buffer. They are
separated with a "frame length" 4-byte values, so the buffer content
is writen to connection object in the format compatible with
framestream library
- added test which guarantees that dnstapEncoder output is the same
as framestream Encoder output
- the performance increase is about 50% in (dio *dnstapIO) serve() method
of dnstap plugin. The overall coredns performance increase is about 10%
in the following configuration:
.:1053 {
erratic {
drop 0
truncate 0
delay 0
}
dnstap tcp://127.0.0.1:6000 full
errors stdout
}
tested with dnsperf tool
