A connection may be opened for each `--transfers` and `--checkers`
now. Connections are checked when putting them in the pool and
getting them out the pool so it should recover from network errors
much better.
This fixes#1561, fixes#1541, fixes#1381, fixes#1158, fixes#1538
This simplifies the implementation of remotes. The only required
interface is now `List` which is a simple one level directory list.
Optionally remotes may implement `ListR` if they have an efficient way
of doing a recursive list.
* Add options to Put, PutUnchecked and Update for all Fses
* Use these to create HashOption
* Implement this in local
* Pass the option in fs.Copy
This has the effect that we only calculate hashes we need to in the
local Fs which speeds up transfers significantly.
Because there is a period of time between checking a directory needs
creating and creating it, the leads to errors where directories are
attempting to be created twice.
Add locking on a per directory basis to fix while doing mkdir.
