Nick Craig-Wood
798502b204
fs: add more errors to be considered temporary errors
...
This makes a framework for adding temporary errors identified by
syscall number or by error string.
Fixes #1660
2017-09-14 18:01:43 +01:00
Stefan Breunig
56dedc49e3
rcat: properly report if the upload fails
2017-09-13 20:21:52 +02:00
Girish Ramakrishnan
4c45cbea18
copy: error out if dst could not be listed
2017-09-12 09:29:44 +01:00
Stefan Breunig
a91448c83a
rcat: honor --dry-run even for small files
2017-09-11 22:28:16 +02:00
Stefan Breunig
80b1f2a494
rcat: configurable small files cutoff and implement proper upload verification
2017-09-11 08:26:53 +02:00
Stefan Breunig
57817397a0
rcat: directly upload small files without streaming them
2017-09-11 08:25:34 +02:00
Nick Craig-Wood
cbea06026a
Make check obey --ignore-size - fixes #1643
2017-09-01 11:20:41 +01:00
Nick Craig-Wood
2c8d6e86cc
fs: fix gofmt
2017-08-31 16:01:19 +01:00
cbruegg
bb6300b032
Fix bwlimit toggle in conjunction with schedules ( Fixes #1607 )
2017-08-31 15:33:29 +01:00
Stefan Breunig
a122b9fa7a
yandex: implement streaming uploads (see #1614 )
2017-08-19 14:07:23 +02:00
Alex McGrath Kraak
2cc9071791
http: add --user-agent option. close #1557
2017-08-17 06:49:27 +02:00
Nick Craig-Wood
265fb8a5e2
fs: Manage empty directories - fixes #100
...
During the sync we collect a list of directories which should be empty
and attempt to rmdir them at the end of the sync. If the directories
are not empty then the rmdir will fail, logging a message but not
erroring the sync.
2017-08-09 21:07:00 +01:00
Nick Craig-Wood
8a1a900733
fstest: use Feature.CanHaveEmptyDirectories to sharpen tests
...
Now we actually test whether the directories are present or not,
filtering out empty directories in the test using the
CanHaveEmptyDirectories flag.
2017-08-09 20:55:08 +01:00
Nick Craig-Wood
20ae7d562b
fs: Add CanHaveEmptyDirectories and BucketBased feature flags to all remotes
2017-08-09 20:55:08 +01:00
Nick Craig-Wood
ec2ea37ad2
fs: Add --disable flag to disable optional features - fixes #1551
...
Eg to disable server side copy use `--disable copy`, to see a list of
what you can disable, `--disable help`.
2017-08-07 21:34:45 +01:00
Nick Craig-Wood
bfe812ea6b
dedupe: implement merging of duplicate directories - fixes #1243
2017-08-07 15:36:41 +01:00
Nick Craig-Wood
db1995e63a
Add MergeDirs optional interface and implement it for drive
2017-08-07 15:32:47 +01:00
Nick Craig-Wood
81a2ab599f
fs: add optional ID to fs.Directory and set it in the remotes which care
2017-08-07 15:31:22 +01:00
Stefan Breunig
40f24e0ea3
config: use absolute ConfigPath to ensure newly written config is on the same mount - fixes #1569
2017-08-05 12:13:25 +02:00
Nick Craig-Wood
92d2e1f8d7
azureblob: rework and complete #801
...
* Fixup bitrot (rclone and Azure library)
* Implement Copy
* Add modtime to metadata under mtime key as RFC3339Nano
* Make multipart upload work
* Make it pass the integration tests
* Fix uploading of zero length blobs
* Rename to azureblob as it seems likely we will do azurefile
* Add docs
2017-08-04 22:56:16 +01:00
Andrei Dragomir
036fd61a50
Added Azure Blob storage support #801
2017-08-04 22:54:27 +01:00
wuyu
ec5b72f8d5
Add new QingStor remote
...
Add new package qingstor to support QingStor API.
Add new unit test for its and tested through; But I commented
on some tests case because of some of the features of QingStor.
Add new docs for it.
2017-08-04 17:25:47 +01:00
Nick Craig-Wood
f682002b84
fs: Make tests create a new bucket rather than purging the old one
...
This enables QingStor to pass the tests as it has a 2 minute lockout
on deleting the old bucket then creating it again.
2017-08-04 17:09:28 +01:00
Stefan Breunig
28a18303f3
implement rcat – fixes #230 , fixes #1001
2017-08-03 21:42:35 +02:00
Nick Craig-Wood
3e3a59768e
fs/test_all: fix after fstest factorisation
2017-08-03 20:01:05 +01:00
Nick Craig-Wood
e01741b557
fs: Cleaning up directories in test is no longer needed
...
..as it is done in the finalise method.
2017-08-03 20:01:05 +01:00
Nick Craig-Wood
71fe046937
fs: Add Find method to DirTree
2017-07-26 16:38:53 +01:00
Nick Craig-Wood
d5ff7104e5
fs: Implement NewDirTree for non --fast-list
2017-07-26 16:38:44 +01:00
Nick Craig-Wood
cd4895690a
fstest: Factor test initialisation into Initialise()
2017-07-26 16:38:33 +01:00
Nick Craig-Wood
1ecf2bcbd5
fs: fix typo in --bind description
2017-07-23 23:08:33 +01:00
Nick Craig-Wood
c3d6cc91ec
Fix --bind flag changes under go1.6
...
Correcting 9f24639568
2017-07-23 22:36:32 +01:00
Nick Craig-Wood
9f24639568
Add --bind flag for choosing the local addr on outgoing connections - fixes #1087
...
Supported by all remotes except FTP.
2017-07-23 16:27:39 +01:00
Nick Craig-Wood
7b81f12dad
box: add docs
...
* reorder remotes so they are in alphabetical order by full name everywhere
* update CONTRIBUTING doc
2017-07-23 11:32:34 +01:00
Nick Craig-Wood
d279161cee
Implement box storage remote - #97
2017-07-23 11:32:34 +01:00
Nick Craig-Wood
db6009126d
Fix test failure with new stretchr/testify - fixes #1550
2017-07-23 08:59:07 +01:00
Nick Craig-Wood
c8b5ee1e54
Start v1.37-DEV development
2017-07-22 20:43:06 +01:00
Nick Craig-Wood
a73ecec11f
Version v1.37
2017-07-22 20:04:29 +01:00
Nick Craig-Wood
0cf19ef66a
Make ListDirSorted check for subdirectories and write test
2017-07-19 09:36:27 +01:00
Nick Craig-Wood
655891170f
Check in ListDirSorted that the directory entries all belong
2017-07-18 23:39:42 +01:00
Nick Craig-Wood
6f71260acf
Add --tpslimit and --tpslimit-burst to limit transactions per second for HTTP
...
This is useful if you are being rate limited or banned by your cloud
storage provider.
2017-07-16 17:25:39 +01:00
Nick Craig-Wood
62e28d0a72
Replace token bucket limiter github.com/tsenart/tb with golang.org/x/time/rate
...
In tests tsenart/tb has proved inaccurate at low rates.
2017-07-16 16:14:44 +01:00
Nick Craig-Wood
72add5ab27
sync: state whether duplicates are objects are directories
2017-07-08 15:42:18 +01:00
Nick Craig-Wood
69ff009264
Use a stable sort for sorting directory entries
...
This is useful if there are duplicates. Assuming the remote delivers
the entries in a consistent order, this will give the best user
experience in syncing as it will consistently use the first entry for
the sync comparison.
2017-07-06 14:07:26 +01:00
Nick Craig-Wood
27b157580e
Move make_test_files.go into bin
2017-07-06 11:54:57 +01:00
Nick Craig-Wood
f0512d1a52
Fix missing fs.Dir -> fs.Directory
2017-07-06 11:31:36 +01:00
Nick Craig-Wood
575e779b55
Warn about duplicate files when syncing - fixes #1506
...
Error about unsorted directories and test thoroughly
2017-06-30 21:24:13 +01:00
Nick Craig-Wood
e7d04fc103
Create fs.Directory interface and use it everywhere
2017-06-30 14:26:59 +01:00
Nick Craig-Wood
e2d7d413ef
fs: rename BasicInfo to DirEntry
2017-06-30 14:26:58 +01:00
Nick Craig-Wood
e7e9aa0dfa
fs: Remove unused ListFser interface
2017-06-30 14:26:58 +01:00
Nick Craig-Wood
f88300a153
Don't Mkdir at the start of sync - fixes #1131
...
This is possible now that the bucket based remotes will create the
buckets on demand (9c1e703777
2017-06-29 12:31:53 +01:00
