Commit graph

7397 commits

Author SHA1 Message Date
Nick Craig-Wood
d4df3f2154 acd: Download files >= 9GB with their tempLink direct from s3
This files the problem downloading files > 10GB.

Fixes #204 Fixes #313
2016-01-30 18:08:44 +00:00
Nick Craig-Wood
8ae424c5a3 Emphasize testing sync with --dry-run and -v 2016-01-29 07:59:33 +00:00
Nick Craig-Wood
cae19df058 s3: URL escape CopySource
This fixes metadata update and copy for files with `+` in

Fixes #315
2016-01-27 17:39:33 +00:00
Nick Craig-Wood
8c211fc8df Warn the user about files with same name but different case
Relates to #107 & #119.
2016-01-26 16:57:09 +00:00
Nick Craig-Wood
74a71f7824 Add tests for --delete-before, --delete-during and --delete-after 2016-01-26 16:57:09 +00:00
Nick Craig-Wood
12b51c5eb8 Remove duplicate check for filter IncludeObject 2016-01-26 16:57:09 +00:00
klauspost
14069fd8e6 Implement --delete-before, --delete-during, --delete-after - fixes #252. 2016-01-26 16:57:09 +00:00
Nick Craig-Wood
cd62f41606 Reduce number of logs and show hash type where appropriate 2016-01-24 18:06:57 +00:00
Nick Craig-Wood
109d4ee490 Prefix all test remotes with rclone-test- and make names more pronouncable 2016-01-24 12:37:46 +00:00
Nick Craig-Wood
18ebec8276 Check remote is empty between integration tests 2016-01-24 12:37:19 +00:00
Nick Craig-Wood
c47b4f828f acd: Fix deadlock in directory traversal code 2016-01-24 11:20:55 +00:00
Nick Craig-Wood
c3a0c0c451 swift: Fix upload from unprivileged user - fixes #273 2016-01-23 20:32:53 +00:00
Nick Craig-Wood
6cb0de43ce Deprecate compiling with go1.3 2016-01-23 17:27:00 +00:00
Nick Craig-Wood
83f0d3e03d acd: remove 409 conflict from error codes we will retry
This should fix the very long pauses or getting stuck people have seen
in uploads.
2016-01-23 17:02:09 +00:00
Nick Craig-Wood
eda4130703 Fix integration tests so they can be run independently and out of order - fixes #291
* Make all integration tests start with an empty remote
  * Add an -individual flag so this can be a different bucket/container/directory
  * Fix up tests after changing the hashers
  * Add sha1sum test
  * Make directory checking in tests sleep more to fix acd inconsistencies
  * Factor integration tests to make more maintainable
  * Ensure remote writes have a fstest.CheckItems() before use
    * this fixes eventual consistency on the directory listings later
  * Call fs.Stats.ResetCounters() before every fs.Sync()

Note that the tests shouldn't be run concurrently as fs.Config is global state.
2016-01-23 17:02:09 +00:00
Nick Craig-Wood
ccba859812 Test all available hashes for each remote 2016-01-23 09:10:36 +00:00
Nick Craig-Wood
de3cf5e8d7 Add -verbose flag to unit tests and add some more eventual consistency retries 2016-01-20 20:06:05 +00:00
Nick Craig-Wood
ce305321b6 amazon cloud drive: Fix "Next token is expired" - Fixes #289 Fixes #263
This should also fix the consequent "409 Conflict" name already exists errors.
2016-01-20 20:05:52 +00:00
Nick Craig-Wood
e6117e978e Add Werner Beroux to contributors 2016-01-20 16:33:28 +00:00
Werner Beroux
4b40898743 Update filtering.md
Clarify by removing the extension which makes it confusing if not careful.
2016-01-20 16:16:24 +01:00
Nick Craig-Wood
ae3a0ec27e b2: Don't re-read the SHA1 if we already have it 2016-01-19 08:21:20 +00:00
Nick Craig-Wood
d9458fb4ee b2: return error in Hash from readFileMetadata operation 2016-01-19 08:21:10 +00:00
Nick Craig-Wood
27f67edb1a Fix formatting problem in sha1sum 2016-01-17 13:56:42 +00:00
Nick Craig-Wood
3ffea738e6 Make hash constants start from 1 not 2 2016-01-17 10:47:24 +00:00
Nick Craig-Wood
a63dd6020c onedrive: fix incorrectly decoded SHA-1 2016-01-17 10:46:36 +00:00
Nick Craig-Wood
d0678bc3e5 local: report error on stat in Hash in case file disappeared 2016-01-17 10:46:19 +00:00
klauspost
ce04a073ef Update templates to changes in the latest hugo version
Fixes #295
2016-01-16 14:11:52 +00:00
Nick Craig-Wood
c337a367f3 Make make serve fail if make website would fail 2016-01-16 14:10:57 +00:00
klauspost
7ae40cb352 Update information on revised hash functionality. 2016-01-16 10:17:11 +00:00
Nick Craig-Wood
e8daab7971 Fix integration tests for remotes with unsupported hash schemes 2016-01-16 09:45:15 +00:00
klauspost
78c3a5ccfa Add support for multiple hash types.
Add support for multiple hash types with negotiation of common hash types for comparison.

Manually rebased version of #277 (see discussion there)
2016-01-11 13:39:33 +01:00
Nick Craig-Wood
2142c75846 Add missing docs for options - fixes #278 2016-01-10 12:04:20 +00:00
Nick Craig-Wood
c724d8f614 dropbox: Make file exclusion error controllable with -q #287 2016-01-10 11:49:04 +00:00
Nick Craig-Wood
af5f4ee724 Make --include rules add their implict exclude * at the end of the filter list
This means you can mix `--include` and `--include-from` with the
other filters (eg `--exclude`) but you must include all the files you
want in the include statement.

Fixes #280
2016-01-10 11:42:53 +00:00
Nick Craig-Wood
01aa4394a6 Explain that errored sync doesn't delete files - fixes #285 2016-01-10 10:44:33 +00:00
Nick Craig-Wood
2646519712 Add --memprofile flag 2016-01-09 15:25:48 +00:00
Nick Craig-Wood
5b2efd563a Add Xavier Lucas to contributors 2016-01-08 08:32:52 +00:00
xlucas
e7b7432079 OVH Swift authentication enpoint 2016-01-08 08:30:13 +00:00
Nick Craig-Wood
ea2ef4443b Remove -verbose from errcheck 2016-01-08 08:20:04 +00:00
klauspost
25f22ec561 Add "--ignore-existing" flag.
Add option to completely ignore existing files and not consider them for transfer.

Fixes #274
2016-01-08 08:20:04 +00:00
Nick Craig-Wood
5189231a34 Tweaks to rclone authorize
* Document the headless / remote setup procedure
  * Move Config constants into fs
  * Parse arguments in main for Authorize
2016-01-07 20:31:23 +00:00
klauspost
bcbd30bb8a Add easier headless configuration.
This will allow setting up a remote with copy&paste of values to a headless machine. It will allow copy+pasting a token into the configuration.

This requires rclone to be on a machine with a proper browser. Custom client id and secrets are supported.

To test token generation, use `rclone auth "fs type"`.
2016-01-07 20:31:23 +00:00
Nick Craig-Wood
c245183101 Stop errcheck running for go < 1.5 2016-01-07 16:37:51 +00:00
klauspost
4ce2a84df0 Document workaround for ACD maximum file size.
Document workaround for ACD maximum file size and display a warning in verbose mode before upload starts.

Fixes #215.
2016-01-05 17:12:16 +00:00
klauspost
3c31d711b3 Add local file system option to disable UNC on Windows.
This will add an option to disable UNC conversion on Windows to deal with buggy file system implementations like EncFS.

Fixes #261
2016-01-05 17:08:11 +00:00
Nick Craig-Wood
3f5d8390ba Add Björn Harrtell to contributors 2016-01-05 17:05:31 +00:00
Björn Harrtell
78edafcaac drive: add --drive-auth-owner-only to only consider files owned by the user. 2016-01-05 17:02:04 +00:00
Nick Craig-Wood
1ce3673006 Add -clean flag to test_all.go to clean left over test directories 2016-01-03 21:49:26 +00:00
Nick Craig-Wood
3423de65fa Make canonical place for all fs in fs/all/all.go 2016-01-03 14:12:45 +00:00
Nick Craig-Wood
0c81439bc3 Fix upload_github target 2016-01-02 12:18:32 +00:00