TrueCloudLab/rclone
16
0
Fork 2
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions 1
1461 commits 181 branches 114 tags 335 MiB
Commit graph

94 commits

Author SHA1 Message Date
Nick Craig-Wood
c3a0c0c451 swift: Fix upload from unprivileged user - fixes #273 2016-01-23 20:32:53 +00:00
Nick Craig-Wood
ccba859812 Test all available hashes for each remote 2016-01-23 09:10:36 +00:00
klauspost
78c3a5ccfa Add support for multiple hash types. 
Add support for multiple hash types with negotiation of common hash types for comparison.

Manually rebased version of #277 (see discussion there)
 2016-01-11 13:39:33 +01:00
xlucas
e7b7432079 OVH Swift authentication enpoint 2016-01-08 08:30:13 +00:00
Nick Craig-Wood
8369b5209f swift: Make sure we read the size for 0 length files - Fixes #237 
This was causing a problem with sync for chunked files.  The directory
listing would read their size back as 0 and see that the size had
changed and immediately resync it.
 2015-12-17 13:30:58 +00:00
Nick Craig-Wood
fcea3777c0 Implement Hubic storage system - fixes #200 2015-11-14 08:08:52 +00:00
Nick Craig-Wood
e8ba43c479 swift: Use ContentType from Object to avoid lookups in listings - fixes #208 2015-11-11 17:19:57 +00:00
Nick Craig-Wood
5bb76cc35c Stop SetModTime losing metadata (eg X-Object-Manifest) - fixes #203 2015-11-11 17:00:08 +00:00
Nick Craig-Wood
a795d93bc3 swift, s3, googlecloudstorage: Don't delete the container/bucket if fs wasn't at root - fixes #172 2015-11-07 15:32:40 +00:00
Nick Craig-Wood
5df04cb763 swift: ignore directory marker objects where appropriate - fixes #190 
* When creating a LimitedFs
  * When calling List() to list files
  * In the Storable() method
  * Add a Purge() method to delete the directory marker objects too

This is a partial fix for #172
 2015-11-07 15:32:11 +00:00
Nick Craig-Wood
b257de4aba Be more constistent with naming in remotes 
* External objects are called Fs and Object
  * Object.fs always points to the Fs
 2015-11-07 11:14:46 +00:00
Nick Craig-Wood
95fd79faf9 swift: use Content-Length on uploads - fixes #125 2015-10-03 16:04:51 +01:00
Nick Craig-Wood
b79dc01016 swift: stop chunked operations logging "Failed to read info: Object Not Found" 2015-10-03 16:04:51 +01:00
Nick Craig-Wood
4a0a42c2f1 swift: large file upload fixes 
* Read metadata in file listing for 0 length files to fix syncs
  * Ignore non-existent files in isManifestFile to fix errors on copy
  * remove nsToSwiftFloatString - experiments with the swift program
    indicate that it puts a variable number of points after the
    decimal, so might as well use the one in the swift library.
  * Make sure segments get deleted properly when move from segmented
    to non segmented and vice versa
  * Use internal list routine to detect errors on listing
  * Remove the _segments container if possible
  * Remove manifest first when deleting
 2015-09-26 17:58:04 +01:00
Sergey Tolmachev
cc7b9af50e swift: support files > 5GB - fixes #46 
* Write segments to ..._segments container
  * Choice of container and segment names compatible with swift tool
    * See http://docs.openstack.org/developer/swift/overview_large_objects.html
  * Controlled by command line flag --swift-chunk-size
  * Segments removed on delete
 2015-09-26 13:03:58 +01:00
Nick Craig-Wood
68fef49c55 swift: fetch headers as the only source of metadata 2015-09-26 13:03:58 +01:00
Nick Craig-Wood
e9c915e6fe Fix golint warnings 2015-09-22 18:47:16 +01:00
Nick Craig-Wood
e2717a031e Implement Mover and DirMover interfaces fixes #115 
* unit tests
  * local
  * drive
  * dropbox
 2015-09-01 21:49:13 +01:00
Nick Craig-Wood
92745aa950 Add Root() to Fs interface 2015-09-01 21:49:13 +01:00
Nick Craig-Wood
a96b522958 Implement server side copies if possible - fixes #99 
Add optional fs.Copier interface

Implemented for
  * swift
  * s3
  * drive
  * dropbox
  * google cloud storage
 2015-08-23 21:18:38 +01:00
Nick Craig-Wood
fedf81c2b7 Add Name() to Fs interface to return name as passed to NewFs 2015-08-23 13:36:38 +01:00
Leonid Shalupov
3fcff32524 do not print stats in quiet mode - fixes #70 
...unless had some errors or stats interval requested.

Add fs.ErrorLog to differentiate between Logs which should be
suppressed and errors which shouldn't.
 2015-08-10 11:17:54 +01:00
Nick Craig-Wood
1d254a3674 Implement --timeout and --conntimeout - fixes #54 
NB dropbox still to do
 2015-05-10 11:29:55 +01:00
Nick Craig-Wood
c47c9cd440 swift: add region parameter - fixes #38 2015-03-04 17:09:53 +00:00
Nick Craig-Wood
ee6b39aa6c Add tenant parameter for swift - fixes #13 2014-12-12 15:26:08 +00:00
Nick Craig-Wood
d3c16608e4 Test Listing the Root of each Fs 2014-07-31 23:20:39 +01:00
Nick Craig-Wood
3e27ff1b95 Add Root List test and fs.Limited tests for single files 2014-07-31 21:35:29 +01:00
Nick Craig-Wood
c389616657 all: make private functions / variables / constant which shouldn't be public 2014-07-29 17:50:07 +01:00
Nick Craig-Wood
457ef2c190 Automatically generate the tests files for each Fs 2014-07-29 17:18:22 +01:00
Nick Craig-Wood
b013c58537 swift: return directories without / in ListDir 2014-07-24 23:13:33 +01:00
Nick Craig-Wood
a5b0d88608 Make tests for each Fs 
Factor tests out of rclonetest
 2014-07-24 23:13:32 +01:00
Nick Craig-Wood
282cba20a0 swift, s3, dropbox: fix metadata read on Update() 
This was causing changed files to be marked as corrupted on upload
 2014-07-20 11:23:05 +01:00
Nick Craig-Wood
5a8bce6353 swift: Read metadata after Update or Put 2014-07-19 13:05:06 +01:00
Nick Craig-Wood
c1a245d1c8 Factor UserAgent to fs and move Version to fs 2014-07-13 19:19:58 +01:00
Nick Craig-Wood
99695d57ab Implement single file operations for all file systems 2014-05-05 22:17:57 +01:00
Nick Craig-Wood
d0ca58bbb1 swift: Support sub container paths 2014-05-05 18:26:37 +01:00
Nick Craig-Wood
02afcb00e9 Factor Fs.Put into Object.Update and call Update rather than Put if possible 2014-04-18 17:49:01 +01:00
Nick Craig-Wood
92ec29fe3f Factor the generic code into fs and add some more intefaces 2014-03-28 22:26:42 +00:00
Nick Craig-Wood
2e5f0ef258 Rename filing systems modules (so I can tell them apart in emacs buffers!) 2014-03-16 10:09:55 +00:00
Nick Craig-Wood
0a108832e2 Remove filesystem flags and put in config file with editor 2014-03-15 16:06:11 +00:00
Nick Craig-Wood
a34bf2ae5e Make imports absolute and use own copy of goamz pending patch inclusion 2013-07-11 17:37:33 +01:00
Nick Craig-Wood
f1a08ad6bf Namespace the swift options 2013-06-28 18:24:06 +01:00
Nick Craig-Wood
b1f681b912 Rename fs.FsX to fs.X 2013-06-28 08:57:32 +01:00
Nick Craig-Wood
6085dc1b5a Refactor into sub filesystems 2013-06-27 22:17:51 +01:00