TrueCloudLab/rclone
16
0
Fork 2
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions 1
1866 commits 181 branches 114 tags 335 MiB
Commit graph

241 commits

Author SHA1 Message Date
Ernest Borowski
9fbff7bcab mountlib: check if directory is not empty before mounting - fixes #1386 
Signed-off-by: Ernest Borowski <er.borowski@gmail.com>
 2017-11-14 21:24:31 +00:00
Nick Craig-Wood
68d0b5adbb serve webdav: this implements a webdav server for any rclone remote. 2017-11-04 10:24:11 +00:00
Nick Craig-Wood
16e16bc220 serve http: use vfs to cache the directories and support Range header 2017-11-04 10:24:11 +00:00
Nick Craig-Wood
2083ac6e2a vfs: add ECLOSED and tidy errors 2017-11-04 10:24:10 +00:00
Nick Craig-Wood
22ee839d05 cmount,vfs: unify Read and Write handles and File and Dir where possible 2017-11-04 10:24:10 +00:00
Nick Craig-Wood
5634659ea3 mount,vfs: unify Read and Write handles in preparation for ReadWrite handles 2017-11-04 10:24:10 +00:00
Nick Craig-Wood
e18122e88b vfs: add tests and subsequent fixes 
* Tests for VFS layer
  * Small fixes found during testing
  * Fix Close, Flush and Release behaviour for ReadFileHandle and WriteFileHandle
  * Fix nil object bugs on File
 2017-11-04 10:24:10 +00:00
Nick Craig-Wood
07ec8073fe mount: remove unused DirEntry struct 2017-11-03 13:00:00 +00:00
Nick Craig-Wood
8184ec4b70 vfs: add EPERM to errors 2017-11-03 13:00:00 +00:00
Nick Craig-Wood
190367d917 vfs: factor duplicated Open code into vfs from mount/cmount 2017-11-03 13:00:00 +00:00
Nick Craig-Wood
7e065440fb vfs: rename Lookup to Stat to be more in keeping with os 2017-11-03 12:59:59 +00:00
Nick Craig-Wood
e8883e9fdb vfs: factor flags into vfsflags and remove global variables 2017-11-03 12:59:59 +00:00
Nick Craig-Wood
1a8f824bad vfs: use os package errors where possible 2017-11-03 12:59:59 +00:00
Nick Craig-Wood
c1aaff220d Factor new vfs module out of cmd/mountlib 
This is an OS style file system abstraction with directory caching
used in mount, cmount, serve webdav and serve http.
 2017-11-03 12:59:59 +00:00
Nick Craig-Wood
6da6b2556b mountlib: make directory entries be returned in sorted order 2017-11-03 12:59:59 +00:00
Nick Craig-Wood
ca19fd2d7e mountlib: Make read/write file handles support more standard interfaces 
Including Read, ReadAt, Seek, Close for read handles and Write,
WriteAt, Close for read handles.
 2017-11-03 12:59:59 +00:00
Nick Craig-Wood
2fac74b517 mountlib: store only Node in *Dir removing DirEntry struct 2017-11-03 12:59:59 +00:00
Nick Craig-Wood
8b6daaa877 mountlib: add DirEntry() to Node interface 2017-11-03 12:59:59 +00:00
Nick Craig-Wood
3af9d63261 mountlib: add Remove and RemoveAll methods to Node 2017-11-03 12:59:59 +00:00
Nick Craig-Wood
c6cd2a5280 mountlib: add parent and entry to Dir 2017-11-03 12:59:59 +00:00
Nick Craig-Wood
0bb84efe75 mountlib: Rename Remove to RemoveName 2017-11-03 12:59:59 +00:00
Nick Craig-Wood
3ec15ac2bd mountlib: make sure Node is always set in DirEntry 
This simplifies the code and makes using the DirEntry.Node usable when
using ReadDir.
 2017-11-03 12:59:58 +00:00
Nick Craig-Wood
750690503e mountlib: make Node satisfy os.FileInfo interface 2017-11-03 12:59:58 +00:00
Nick Craig-Wood
54950d3423 mountlib: make more useful as a general purpose file system adaptor 2017-11-03 12:59:58 +00:00
Nick Craig-Wood
96665c16cb serve http: make it compile on go1.6 and go1.7 2017-10-26 21:52:29 +01:00
Nick Craig-Wood
cd146415d1 serve http: error if Range supplied (not supported yet) 
Also add Server header
 2017-10-24 23:18:36 +01:00
Nick Craig-Wood
2740c965c0 serve http: Fix timeouts 2017-10-24 23:07:46 +01:00
Nick Craig-Wood
6669165b6b serve http command to serve a remote over HTTP 
This implements a basic webserver to serve an rclone remote over HTTP.

It also sets up the framework for adding more types of server later.
 2017-10-24 13:25:49 +01:00
Dan Dascalescu
9417194751 Fix dedupe description typo 2017-10-16 21:51:31 +01:00
Nick Craig-Wood
edfab09eb9 config: add sub commands for full config file management 
Previously config sub commands were manually parsed rather than using
cobra.

Make config command have the following sub commands:

 * create    Create a new remote with name, type and options.
 * delete    Delete an existing remote <name>.
 * dump      Dump the config file as JSON.
 * edit      Enter an interactive configuration session.
 * file      Show path of configuration file in use.
 * providers List in JSON format all the providers and options.
 * show      Print (decrypted) config file, or the config for a single remote.
 * update    Update options in an existing remote.

The following changes were made to existing commands

 * listproviders was renamed to providers
 * listoptions was removed in favour of providing the output in providers
 * jsonconfig was renamed to create
 * an optional parameter was added to the show command
 2017-10-14 11:50:41 +01:00
thierry
0575623dff Add config listproviders, listoptions, jsonconfig for automated config 
Addition of a method listing the providers, a method listing the
options of a provider and method of manual configuration.
 2017-10-13 17:17:36 +01:00
Nick Craig-Wood
7f3dc9b5c4 Implement WebDAV remote #580 
This has special knowledge of Owncloud and Nextcloud to enable more
functionality such as mod times.
 2017-10-09 16:19:37 +01:00
Nick Craig-Wood
9e9297838f Implement pcloud remote - #418 2017-10-01 11:37:35 +01:00
Nick Craig-Wood
63d0734c71 tree: remove workaround for tree library bug now it is fixed 2017-09-30 15:51:14 +01:00
ishuah
7195e44dce crypt: added cryptdecode command - #1129 2017-09-24 20:02:59 +01:00
Stefan
85877f3adc config: add show/file subcommands which print the config/its path (fixes #1086) 2017-09-19 17:59:19 +02:00
Stefan Breunig
12405f9f41 fuse: re-use rcat to support uploads for all remotes (fixes #1672) 2017-09-16 22:49:08 +02:00
Stefan Breunig
80b1f2a494 rcat: configurable small files cutoff and implement proper upload verification 2017-09-11 08:26:53 +02:00
bpicode
f1f7e0e6f9 support for zsh auto-completion - #983 2017-08-31 16:21:28 +01:00
Stefan Breunig
0158ab6926 info: add check to stream files with unknown size 2017-08-22 08:00:10 +02:00
Nick Craig-Wood
bfe812ea6b dedupe: implement merging of duplicate directories - fixes #1243 2017-08-07 15:36:41 +01:00
Nick Craig-Wood
92d2e1f8d7 azureblob: rework and complete #801 
* Fixup bitrot (rclone and Azure library)
  * Implement Copy
  * Add modtime to metadata under mtime key as RFC3339Nano
  * Make multipart upload work
  * Make it pass the integration tests
  * Fix uploading of zero length blobs
  * Rename to azureblob as it seems likely we will do azurefile
  * Add docs
 2017-08-04 22:56:16 +01:00
wuyu
ec5b72f8d5 Add new QingStor remote 
Add new package qingstor to support QingStor API.

Add new unit test for its and tested through; But I commented
on some tests case because of some of the features of QingStor.

Add new docs for it.
 2017-08-04 17:25:47 +01:00
Nick Craig-Wood
7d34caac83 cmd: add os and go version to rclone version output 2017-08-04 14:25:55 +01:00
Stefan Breunig
28a18303f3 implement rcat – fixes #230, fixes #1001 2017-08-03 21:42:35 +02:00
Nick Craig-Wood
21aca68680 tree: fix when running under Windows 2017-08-01 14:46:21 +01:00
Andy Pilate
ef328c5497 Fixes typo in command dedupe definition 2017-07-27 21:17:57 +01:00
Nick Craig-Wood
f3060caf04 Implement tree command - fixes #1528 2017-07-26 23:06:48 +01:00
Nick Craig-Wood
ebc8361933 mount: Add notes on Windows limitations from Bill Zissimopoulos 2017-07-26 21:08:24 +01:00
Nick Craig-Wood
cd4895690a fstest: Factor test initialisation into Initialise() 2017-07-26 16:38:33 +01:00
Nick Craig-Wood
de54fd4c64 mount: add docs for windows install 2017-07-23 13:05:02 +01:00
Nick Craig-Wood
7b81f12dad box: add docs 
* reorder remotes so they are in alphabetical order by full name everywhere
  * update CONTRIBUTING doc
 2017-07-23 11:32:34 +01:00
Nick Craig-Wood
c223464cd0 mount: fix panic on renames - fixes #1533 
Make sure d.items is not nil and improve locking
 2017-07-22 11:00:51 +01:00
Yaroslav Halchenko
3ddcbce989 DOC: any empty directoryies -> empty directories (fixes #1546) 2017-07-22 10:24:41 +02:00
Nick Craig-Wood
e86f62c3e8 Add rclone info internal command for testing out limits of remotes 2017-07-03 15:05:27 +01:00
Nick Craig-Wood
e7d04fc103 Create fs.Directory interface and use it everywhere 2017-06-30 14:26:59 +01:00
Nick Craig-Wood
e2d7d413ef fs: rename BasicInfo to DirEntry 2017-06-30 14:26:58 +01:00
Nick Craig-Wood
aa20486485 Add --stats-log-level so can see --stats without -v - fixes #1180 
The most common use for this flag is likely to be showing the stats
without using -v by using `--stats-log-level NOTICE`.
 2017-06-26 22:50:37 +01:00
sainaen
1817d8f631 crypt: Fix typo in cryptcheck's short description 2017-06-26 12:35:20 +01:00
Nick Craig-Wood
b360527931 mount: fix hang on errored upload 
In certain circumstances if an upload failed then the mount could hang
indefinitely. This was fixed by closing the read pipe after the Put
completed.  This will cause the write side to return a pipe closed
error fixing the hang.

Fixes #1498
 2017-06-26 12:08:51 +01:00
Nick Craig-Wood
b22c4c4307 http: fix, tidy and rework ready for release 
* Fix remaining problems
  * Refactor to make testing easier and add a test suite
  * Make path parsing more robust.
  * Add single file operations
  * Add MimeType reading for objects
  * Add documentation
  * Note go1.7+ is required to build
 2017-06-20 21:27:14 +01:00
Nick Craig-Wood
dcce65b2b3 mount/cmount: factor duplicated code into mountlib 2017-06-19 14:36:51 +01:00
Nick Craig-Wood
97364fd0b6 ncdu: disable on plan9 and solaris as termbox isn't supported there 2017-06-15 20:10:54 +01:00
Nick Craig-Wood
c34f11a92f rclone ncdu for exploring a remote with a text based user interface. 2017-06-15 17:44:17 +01:00
Nick Craig-Wood
7be49249d3 Add lsjson command - fixes #1063 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
7e20e16cff core: Implement Walk directory listing and use in place of Lister 
This is in preparation for removing the Lister code and replacing the
fundamental operation in the Fs with listing a single directory.
 2017-06-14 16:49:40 +01:00
Nick Craig-Wood
68333d34a1 dropbox: make setting mod time on existing files work properly 
This is a fix left over from the v2 conversion.  Dropbox ignores the
client modification on an incoming file if it was identical to the
existing file.  This change deletes the existing file first before
re-uploading the new one.
 2017-06-13 13:58:39 +01:00
Nick Craig-Wood
740b3f6ae2 Fix problems found with ineffassign 2017-06-13 11:52:36 +01:00
Nick Craig-Wood
28fcc53e45 mount test: retry umount as it fails occasionally 
This is because of the background releasing of files which happens
after all the files are closed.
 2017-06-13 10:52:10 +01:00
Nick Craig-Wood
d30cc1e119 Factor RemoteSplit into fs 2017-06-07 12:27:33 +01:00
Stefan Breunig
9782c264e9 hand dirCacheTime through again 2017-06-01 09:02:22 +01:00
Nick Craig-Wood
52e96bc0e2 dropbox: add missing dbhashsum command 
This was missed from 6381959850
 2017-05-30 19:26:06 +01:00
Nick Craig-Wood
6381959850 dropbox: support Dropbox content hashing scheme - fixes #1302 
* add support to hashing module
  * add dbhashsum to list the hashes
  * add support to dropbox module

This means objects up and downloaded to/from Dropbox will have their
hashes checked.

Note after this change local objects are calculating MD5, SHA1 and
DBHASH which is excessive and needs to be fixed.
 2017-05-29 12:04:44 +01:00
Stefan Breunig
a2e3af0523 poll for Google Drive changes when mounted 2017-05-28 17:54:52 +01:00
Sjur Fredriksen
ade61fa756 Updated FTP to follow SFTP standards, updated documentation 2017-05-25 09:30:15 +01:00
Nick Craig-Wood
8f54dc06a2 Use build tags to control when and where cmount is built 2017-05-19 17:08:04 +01:00
Nick Craig-Wood
2cae017738 mountlib: fix race condition in cache clear 2017-05-19 15:47:52 +01:00
Nick Craig-Wood
ecedcd0e7f cmount: stop failing tests on Windows 2017-05-15 17:40:44 +01:00
Nick Craig-Wood
3dff91d691 mount: add missing build constraint to fix Windows build 2017-05-15 17:40:15 +01:00
Nick Craig-Wood
b259f8b752 cmount, mount, mountlib: make --read-only reject modify operations 
Normally mount/cmount use `-o ro` to get the kernel to mark the fs as
read only.  However this is ignored by WinFsp, so in addition if
`--read-only` is in effect then return EROFS ("Read only File System")
from all methods which attempt to modify something.
 2017-05-15 16:56:47 +01:00
Nick Craig-Wood
8be8a8e41b mountlib: on read only open of file, make open pending until first read 
This fixes a problem with Windows which seems fond of opening files
just to read their attributes and closing them again.
 2017-05-15 16:56:47 +01:00
Nick Craig-Wood
f9500729b7 mountlib: fix cross platform tests 2017-05-15 16:56:46 +01:00
Nick Craig-Wood
204a19e67f cmount: Wait for mountpoint to appear on Windows before declaring mounted 2017-05-15 16:56:46 +01:00
Nick Craig-Wood
e6ffe3464c cmount: check for filesystem blowing up before Init is called 2017-05-15 16:56:46 +01:00
Nick Craig-Wood
0384364c3e cmount: pass --FileSystemName under windows 2017-05-15 16:56:46 +01:00
Nick Craig-Wood
763facfd78 cmount: implement --fuse-flag to pass commands to fuse library directly 
Useful for `--fuse-flag -h` to see exactly which options the library supports.
 2017-05-15 16:56:46 +01:00
Nick Craig-Wood
bc88f1dafa cmount: fix openFile leak 2017-05-15 16:56:46 +01:00
Nick Craig-Wood
0c055a1215 cmount: Statfs: reduce max size of volume for Windows 2017-05-15 16:56:46 +01:00
Nick Craig-Wood
938d7951ab cmount: allow extra options to pass to fuse with -o 2017-05-15 16:56:45 +01:00
Nick Craig-Wood
b4466bd9b1 Add -o uid=-1 -o gid=-1 for Windows/WinFsp 2017-05-15 16:56:45 +01:00
Nick Craig-Wood
31f76aa464 cmount: implement no-ops for Fsync, Chmod, Chown, Access, Fsyncdir and stop using fuse.FileSystemBase 2017-05-15 16:56:45 +01:00
Nick Craig-Wood
c887c164dc cmount: add function tracing 2017-05-15 16:56:45 +01:00
Nick Craig-Wood
115ac00222 mount, mountlib: move function tracing into mount 2017-05-15 16:56:45 +01:00
Nick Craig-Wood
abda616f84 mountlib: make Nodes also be fmt.Stringer so they debug nicely 2017-05-15 16:56:45 +01:00
Nick Craig-Wood
9c3048580a cmount: fix code quality warnings 2017-05-15 16:56:45 +01:00
Nick Craig-Wood
c1d5faa32a mountlib: fix code quality warnings 2017-05-15 16:56:45 +01:00
Nick Craig-Wood
d127d8686a mountlib: pass options in fsys not as args 2017-05-15 16:56:44 +01:00
Nick Craig-Wood
bc9856b570 Forward port 930ff266f2 to cmount branch 
compare checksums on upload/download via FUSE
 2017-05-15 16:56:44 +01:00
Nick Craig-Wood
855071cc19 cmount: name the command mount under windows and cmount under linux 2017-05-15 16:56:44 +01:00
Nick Craig-Wood
b179540e80 cmount: fix Getattr to work on directories 2017-05-15 16:56:44 +01:00