KurlesHS/rclone
1
0
Fork 0
forked from TrueCloudLab/rclone
Code Pull requests Activity
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
1029 commits 182 branches 6 tags 335 MiB
Go 98.6%
HTML 0.4%
Shell 0.3%
Python 0.3%
JavaScript 0.2%
Other 0.1%
Find a file
Nick Craig-Wood 48cdedc97b Re-implement sync routine to work a directory at a time 
Multiple directories (up to --checkers worth) are scanned at once.

This uses much less memory than the previous scheme - only the amount
of memory needed to hold an entire directory listing of objects.

For directory based remotes the speed is unchanged.

For bucket based remotes, instead of doing one API call to list the
whole bucket, it does multiple calls, one for each pseudo directory.
However these are done in parallel so in practice this seems to speed
up directory listings.

This replaces the existing sync method as it performs faster and uses
less memory.

The old sync method is available with the temporary --old-sync-method
flag.

Fixes #517
Fixes #439
Fixes #236
Fixes #1067
2017-02-04 10:30:25 +00:00
amazonclouddrive Factor token renewer from amazonclouddrive to oauthutil 2017-01-29 20:45:44 +00:00
b2 b2: constrain memory usage when doing multipart uploads #439 2017-02-03 08:03:04 +00:00
bin Complete SFTP remote #521 2017-02-04 10:29:46 +00:00
cmd Make fallback purge delete empty directories too. 2017-02-04 10:29:46 +00:00
crypt crypt: Add --crypt-show-mapping to show encrypted file mapping 2017-01-29 10:14:17 +00:00
dircache Add directory parameter to Rmdir and Mkdir #100 #831 2016-11-26 12:02:53 +00:00
docs Make --delete-after the default and refactor --delete-{before,during,after} parsing 2017-02-04 10:30:25 +00:00
drive drive: detect files using file size as well as md5 - fixes 980 2017-02-03 08:00:03 +00:00
dropbox Define a new Features() method for Fs 2017-01-16 17:33:25 +00:00
fs Re-implement sync routine to work a directory at a time 2017-02-04 10:30:25 +00:00
fstest Complete SFTP remote #521 2017-02-04 10:29:46 +00:00
Godeps sftp: add required packages to vendor 2017-02-04 10:29:18 +00:00
googlecloudstorage Define a new Features() method for Fs 2017-01-16 17:33:25 +00:00
graphics New graphics used by forum.rclone.org 2016-10-04 11:31:42 +01:00
hubic Define a new Features() method for Fs 2017-01-16 17:33:25 +00:00
local Implement -L, --copy-links flag to allow rclone to follow symlinks 2017-01-29 13:43:20 +00:00
oauthutil Factor token renewer from amazonclouddrive to oauthutil 2017-01-29 20:45:44 +00:00
onedrive onedrive: use token renewer to stop auth errors on long uploads 2017-01-29 20:45:45 +00:00
pacer Make exponential backoff work exactly as per google specification - fixes #583 2016-10-17 17:57:09 +01:00
rest acd: Reset the headers on tempurl redirect #802 2016-10-26 18:42:41 +01:00
s3 s3: Added ca-central-1 region 2017-02-02 21:14:10 +00:00
sftp sftp: Fix remote race on creating directories 2017-02-04 10:29:46 +00:00
swift Define a new Features() method for Fs 2017-01-16 17:33:25 +00:00
vendor sftp: add required packages to vendor 2017-02-04 10:29:18 +00:00
yandex Define a new Features() method for Fs 2017-01-16 17:33:25 +00:00
.gitignore Remove obsolete entry 2016-10-04 11:37:18 +01:00
.travis.yml Update go to 1.7.4 and 1.6.4 in CI 2016-12-09 17:12:11 +00:00
appveyor.yml Use a vendor directory for repeatable builds - fixes #816 2016-11-05 18:18:08 +00:00
CONTRIBUTING.md Add a writing documentation section and update document 2017-01-29 09:47:47 +00:00
COPYING Initial commit - some small parts working 2012-11-18 17:32:31 +00:00
ISSUE_TEMPLATE.md Add a link to the forum in the issue template 2016-10-24 12:34:18 +01:00
Makefile Make "make quicktest" ignore a config file if present for local running 2017-01-16 17:54:18 +00:00
MANUAL.html Typo: the the -> the in docs and comments 2017-01-03 20:48:26 +00:00
MANUAL.md Typo: the the -> the in docs and comments 2017-01-03 20:48:26 +00:00
MANUAL.txt Typo: the the -> the in docs and comments 2017-01-03 20:48:26 +00:00
notes.txt Replace test_all.sh with test_all.go which is cross platform and parallel 2015-12-30 09:26:34 +00:00
rclone.1 Typo: the the -> the in docs and comments 2017-01-03 20:48:26 +00:00
rclone.go Make sure high level retries show with -q - fixes #648 2016-09-12 17:50:19 +01:00
README.md Complete SFTP remote #521 2017-02-04 10:29:46 +00:00
RELEASE.md Version v1.35 2017-01-02 15:33:06 +00:00

README.md

Logo

Website | Documentation | Contributing | Changelog | Installation | Forum G+

Build Status Windows Build Status GoDoc

Rclone is a command line program to sync files and directories to and from

  • Google Drive
  • Amazon S3
  • Openstack Swift / Rackspace cloud files / Memset Memstore
  • Dropbox
  • Google Cloud Storage
  • Amazon Drive
  • Microsoft One Drive
  • Hubic
  • Backblaze B2
  • Yandex Disk
  • SFTP
  • The local filesystem

Features

  • MD5/SHA1 hashes checked at all times for file integrity
  • Timestamps preserved on files
  • Partial syncs supported on a whole file basis
  • Copy mode to just copy new/changed files
  • Sync (one way) mode to make a directory identical
  • Check mode to check for file hash equality
  • Can sync to and from network, eg two different cloud accounts
  • Optional encryption (Crypt)
  • Optional FUSE mount

See the home page for installation, usage, documentation, changelog and configuration walkthroughs.

License

This is free software under the terms of MIT the license (check the COPYING file included in this package).