rclone/docs/content/changelog.md

1001 lines
48 KiB
Markdown
Raw Normal View History

---
title: "Documentation"
description: "Rclone Changelog"
2018-03-19 10:06:13 +00:00
date: "2018-03-19"
---
Changelog
---------
2018-03-19 10:06:13 +00:00
* v1.40 - 2018-03-19
* New backends
* Alias backend to create aliases for existing remote names (Fabian Möller)
* New commands
* `lsf`: list for parsing purposes (Jakub Tasiemski)
* by default this is a simple non recursive list of files and directories
* it can be configured to add more info in an easy to parse way
* `serve restic`: for serving a remote as a Restic REST endpoint
* This enables restic to use any backends that rclone can access
* Thanks Alexander Neumann for help, patches and review
* `rc`: enable the remote control of a running rclone
* The running rclone must be started with --rc and related flags.
* Currently there is support for bwlimit, and flushing for mount and cache.
* New Features
* `--max-delete` flag to add a delete threshold (Bjørn Erik Pedersen)
* All backends now support RangeOption for ranged Open
* `cat`: Use RangeOption for limited fetches to make more efficient
* `cryptcheck`: make reading of nonce more efficient with RangeOption
* serve http/webdav/restic
* support SSL/TLS
* add `--user` `--pass` and `--htpasswd` for authentication
* `copy`/`move`: detect file size change during copy/move and abort transfer (ishuah)
* `cryptdecode`: added option to return encrypted file names. (ishuah)
* `lsjson`: add `--encrypted` to show encrypted name (Jakub Tasiemski)
* Add `--stats-file-name-length` to specify the printed file name length for stats (Will Gunn)
* Compile
* Code base was shuffled and factored
* backends moved into a backend directory
* large packages split up
* See the CONTRIBUTING.md doc for info as to what lives where now
* Update to using go1.10 as the default go version
* Implement daily [full integration tests](https://pub.rclone.org/integration-tests/)
* Release
* Include a source tarball and sign it and the binaries
* Sign the git tags as part of the release process
* Add .deb and .rpm packages as part of the build
* Make a beta release for all branches on the main repo (but not pull requests)
* Bug Fixes
* config: fixes errors on non existing config by loading config file only on first access
* config: retry saving the config after failure (Mateusz)
* sync: when using `--backup-dir` don't delete files if we can't set their modtime
* this fixes odd behaviour with Dropbox and `--backup-dir`
* fshttp: fix idle timeouts for HTTP connections
* `serve http`: fix serving files with : in - fixes
* Fix `--exclude-if-present` to ignore directories which it doesn't have permission for (Iakov Davydov)
* Make accounting work properly with crypt and b2
* remove `--no-traverse` flag because it is obsolete
* Mount
* Add `--attr-timeout` flag to control attribute caching in kernel
* this now defaults to 0 which is correct but less efficient
* see [the mount docs](/commands/rclone_mount/#attribute-caching) for more info
* Add `--daemon` flag to allow mount to run in the background (ishuah)
* Fix: Return ENOSYS rather than EIO on attempted link
* This fixes FileZilla accessing an rclone mount served over sftp.
* Fix setting modtime twice
* Mount tests now run on CI for Linux (mount & cmount)/Mac/Windows
* Many bugs fixed in the VFS layer - see below
* VFS
* Many fixes for `--vfs-cache-mode` writes and above
* Update cached copy if we know it has changed (fixes stale data)
* Clean path names before using them in the cache
* Disable cache cleaner if `--vfs-cache-poll-interval=0`
* Fill and clean the cache immediately on startup
* Fix Windows opening every file when it stats the file
* Fix applying modtime for an open Write Handle
* Fix creation of files when truncating
* Write 0 bytes when flushing unwritten handles to avoid race conditions in FUSE
* Downgrade "poll-interval is not supported" message to Info
* Make OpenFile and friends return EINVAL if O_RDONLY and O_TRUNC
* Local
* Downgrade "invalid cross-device link: trying copy" to debug
* Make DirMove return fs.ErrorCantDirMove to allow fallback to Copy for cross device
* Fix race conditions updating the hashes
* Cache
* Add support for polling - cache will update when remote changes on supported backends
* Reduce log level for Plex api
* Fix dir cache issue
* Implement `--cache-db-wait-time` flag
* Improve efficiency with RangeOption and RangeSeek
* Fix dirmove with temp fs enabled
* Notify vfs when using temp fs
* Offline uploading
* Remote control support for path flushing
* Amazon cloud drive
* Rclone no longer has any working keys - disable integration tests
* Implement DirChangeNotify to notify cache/vfs/mount of changes
* Azureblob
* Don't check for bucket/container presense if listing was OK
* this makes rclone do one less request per invocation
* Improve accounting for chunked uploads
* Backblaze B2
* Don't check for bucket/container presense if listing was OK
* this makes rclone do one less request per invocation
* Box
* Improve accounting for chunked uploads
* Dropbox
* Fix custom oauth client parameters
* Google Cloud Storage
* Don't check for bucket/container presense if listing was OK
* this makes rclone do one less request per invocation
* Google Drive
* Migrate to api v3 (Fabian Möller)
* Add scope configuration and root folder selection
* Add `--drive-impersonate` for service accounts
* thanks to everyone who tested, explored and contributed docs
* Add `--drive-use-created-date` to use created date as modified date (nbuchanan)
* Request the export formats only when required
* This makes rclone quicker when there are no google docs
* Fix finding paths with latin1 chars (a workaround for a drive bug)
* Fix copying of a single Google doc file
* Fix `--drive-auth-owner-only` to look in all directories
* HTTP
* Fix handling of directories with & in
* Onedrive
* Removed upload cutoff and always do session uploads
* this stops the creation of multiple versions on business onedrive
* Overwrite object size value with real size when reading file. (Victor)
* this fixes oddities when onedrive misreports the size of images
* Pcloud
* Remove unused chunked upload flag and code
* Qingstor
* Don't check for bucket/container presense if listing was OK
* this makes rclone do one less request per invocation
* S3
* Support hashes for multipart files (Chris Redekop)
* Initial support for IBM COS (S3) (Giri Badanahatti)
* Update docs to discourage use of v2 auth with CEPH and others
* Don't check for bucket/container presense if listing was OK
* this makes rclone do one less request per invocation
* Fix server side copy and set modtime on files with + in
* SFTP
* Add option to disable remote hash check command execution (Jon Fautley)
* Add `--sftp-ask-password` flag to prompt for password when needed (Leo R. Lundgren)
* Add `set_modtime` configuration option
* Fix following of symlinks
* Fix reading config file outside of Fs setup
* Fix reading $USER in username fallback not $HOME
* Fix running under crontab - Use correct OS way of reading username
* Swift
* Fix refresh of authentication token
* in v1.39 a bug was introduced which ignored new tokens - this fixes it
* Fix extra HEAD transaction when uploading a new file
* Don't check for bucket/container presense if listing was OK
* this makes rclone do one less request per invocation
* Webdav
* Add new time formats to support mydrive.ch and others
2017-12-23 13:07:45 +00:00
* v1.39 - 2017-12-23
* New backends
* WebDAV
* tested with nextcloud, owncloud, put.io and others!
* Pcloud
* cache - wraps a cache around other backends (Remus Bunduc)
* useful in combination with mount
* NB this feature is in beta so use with care
* New commands
* serve command with subcommands:
* serve webdav: this implements a webdav server for any rclone remote.
* serve http: command to serve a remote over HTTP
* config: add sub commands for full config file management
* create/delete/dump/edit/file/password/providers/show/update
* touch: to create or update the timestamp of a file (Jakub Tasiemski)
* New Features
* curl install for rclone (Filip Bartodziej)
* --stats now shows percentage, size, rate and ETA in condensed form (Ishuah Kariuki)
* --exclude-if-present to exclude a directory if a file is present (Iakov Davydov)
* rmdirs: add --leave-root flag (lewpam)
* move: add --delete-empty-src-dirs flag to remove dirs after move (Ishuah Kariuki)
* Add --dump flag, introduce --dump requests, responses and remove --dump-auth, --dump-filters
* Obscure X-Auth-Token: from headers when dumping too
* Document and implement exit codes for different failure modes (Ishuah Kariuki)
* Compile
* Bug Fixes
* Retry lots more different types of errors to make multipart transfers more reliable
* Save the config before asking for a token, fixes disappearing oauth config
* Warn the user if --include and --exclude are used together (Ernest Borowski)
* Fix duplicate files (eg on Google drive) causing spurious copies
* Allow trailing and leading whitespace for passwords (Jason Rose)
* ncdu: fix crashes on empty directories
* rcat: fix goroutine leak
* moveto/copyto: Fix to allow copying to the same name
* Mount
* --vfs-cache mode to make writes into mounts more reliable.
* this requires caching files on the disk (see --cache-dir)
* As this is a new feature, use with care
* Use sdnotify to signal systemd the mount is ready (Fabian Möller)
* Check if directory is not empty before mounting (Ernest Borowski)
* Local
* Add error message for cross file system moves
* Fix equality check for times
* Dropbox
* Rework multipart upload
* buffer the chunks when uploading large files so they can be retried
* change default chunk size to 48MB now we are buffering them in memory
* retry every error after the first chunk is done successfully
* Fix error when renaming directories
* Swift
* Fix crash on bad authentication
* Google Drive
* Add service account support (Tim Cooijmans)
* S3
* Make it work properly with Digital Ocean Spaces (Andrew Starr-Bochicchio)
* Fix crash if a bad listing is received
* Add support for ECS task IAM roles (David Minor)
* Backblaze B2
* Fix multipart upload retries
* Fix --hard-delete to make it work 100% of the time
* Swift
* Allow authentication with storage URL and auth key (Giovanni Pizzi)
* Add new fields for swift configuration to support IBM Bluemix Swift (Pierre Carlson)
* Add OS_TENANT_ID and OS_USER_ID to config
* Allow configs with user id instead of user name
* Check if swift segments container exists before creating (John Leach)
* Fix memory leak in swift transfers (upstream fix)
* SFTP
* Add option to enable the use of aes128-cbc cipher (Jon Fautley)
* Amazon cloud drive
* Fix download of large files failing with "Only one auth mechanism allowed"
* crypt
* Option to encrypt directory names or leave them intact
* Implement DirChangeNotify (Fabian Möller)
* onedrive
* Add option to choose resourceURL during setup of OneDrive Business account if more than one is available for user
2017-09-30 13:19:47 +00:00
* v1.38 - 2017-09-30
* New backends
* Azure Blob Storage (thanks Andrei Dragomir)
* Box
* Onedrive for Business (thanks Oliver Heyme)
* QingStor from QingCloud (thanks wuyu)
* New commands
* `rcat` - read from standard input and stream upload
* `tree` - shows a nicely formatted recursive listing
* `cryptdecode` - decode crypted file names (thanks ishuah)
* `config show` - print the config file
* `config file` - print the config file location
* New Features
* Empty directories are deleted on `sync`
* `dedupe` - implement merging of duplicate directories
* `check` and `cryptcheck` made more consistent and use less memory
* `cleanup` for remaining remotes (thanks ishuah)
* `--immutable` for ensuring that files don't change (thanks Jacob McNamee)
* `--user-agent` option (thanks Alex McGrath Kraak)
* `--disable` flag to disable optional features
* `--bind` flag for choosing the local addr on outgoing connections
* Support for zsh auto-completion (thanks bpicode)
* Stop normalizing file names but do a normalized compare in `sync`
* Compile
* Update to using go1.9 as the default go version
* Remove snapd build due to maintenance problems
* Bug Fixes
* Improve retriable error detection which makes multipart uploads better
* Make `check` obey `--ignore-size`
* Fix bwlimit toggle in conjunction with schedules (thanks cbruegg)
* `config` ensures newly written config is on the same mount
* Local
* Revert to copy when moving file across file system boundaries
* `--skip-links` to suppress symlink warnings (thanks Zhiming Wang)
* Mount
* Re-use `rcat` internals to support uploads from all remotes
* Dropbox
* Fix "entry doesn't belong in directory" error
* Stop using deprecated API methods
* Swift
* Fix server side copy to empty container with `--fast-list`
* Google Drive
* Change the default for `--drive-use-trash` to `true`
* S3
* Set session token when using STS (thanks Girish Ramakrishnan)
* Glacier docs and error messages (thanks Jan Varho)
* Read 1000 (not 1024) items in dir listings to fix Wasabi
* Backblaze B2
* Fix SHA1 mismatch when downloading files with no SHA1
* Calculate missing hashes on the fly instead of spooling
* `--b2-hard-delete` to permanently delete (not hide) files (thanks John Papandriopoulos)
* Hubic
* Fix creating containers - no longer have to use the `default` container
* Swift
* Optionally configure from a standard set of OpenStack environment vars
* Add `endpoint_type` config
* Google Cloud Storage
* Fix bucket creation to work with limited permission users
* SFTP
* Implement connection pooling for multiple ssh connections
* Limit new connections per second
* Add support for MD5 and SHA1 hashes where available (thanks Christian Brüggemann)
* HTTP
* Fix URL encoding issues
* Fix directories with `:` in
* Fix panic with URL encoded content
2017-07-22 17:16:46 +00:00
* v1.37 - 2017-07-22
* New backends
* FTP - thanks to Antonio Messina
* HTTP - thanks to Vasiliy Tolstov
* New commands
* rclone ncdu - for exploring a remote with a text based user interface.
* rclone lsjson - for listing with a machine readable output
* rclone dbhashsum - to show Dropbox style hashes of files (local or Dropbox)
* New Features
* Implement --fast-list flag
* This allows remotes to list recursively if they can
* This uses less transactions (important if you pay for them)
* This may or may not be quicker
2017-09-02 15:42:03 +00:00
* This will use more memory as it has to hold the listing in memory
2017-07-22 17:16:46 +00:00
* --old-sync-method deprecated - the remaining uses are covered by --fast-list
* This involved a major re-write of all the listing code
* Add --tpslimit and --tpslimit-burst to limit transactions per second
* this is useful in conjuction with `rclone mount` to limit external apps
* Add --stats-log-level so can see --stats without -v
* Print password prompts to stderr - Hraban Luyat
* Warn about duplicate files when syncing
* Oauth improvements
* allow auth_url and token_url to be set in the config file
* Print redirection URI if using own credentials.
* Don't Mkdir at the start of sync to save transactions
* Compile
* Update build to go1.8.3
* Require go1.6 for building rclone
* Compile 386 builds with "GO386=387" for maximum compatibility
* Bug Fixes
* Fix menu selection when no remotes
* Config saving reworked to not kill the file if disk gets full
* Don't delete remote if name does not change while renaming
* moveto, copyto: report transfers and checks as per move and copy
* Local
* Add --local-no-unicode-normalization flag - Bob Potter
* Mount
* Now supported on Windows using cgofuse and WinFsp - thanks to Bill Zissimopoulos for much help
* Compare checksums on upload/download via FUSE
* Unmount when program ends with SIGINT (Ctrl+C) or SIGTERM - Jérôme Vizcaino
* On read only open of file, make open pending until first read
* Make --read-only reject modify operations
* Implement ModTime via FUSE for remotes that support it
* Allow modTime to be changed even before all writers are closed
* Fix panic on renames
* Fix hang on errored upload
* Crypt
* Report the name:root as specified by the user
* Add an "obfuscate" option for filename encryption - Stephen Harris
* Amazon Drive
* Fix initialization order for token renewer
* Remove revoked credentials, allow oauth proxy config and update docs
* B2
* Reduce minimum chunk size to 5MB
* Drive
* Add team drive support
* Reduce bandwidth by adding fields for partial responses - Martin Kristensen
* Implement --drive-shared-with-me flag to view shared with me files - Danny Tsai
* Add --drive-trashed-only to read only the files in the trash
* Remove obsolete --drive-full-list
* Add missing seek to start on retries of chunked uploads
* Fix stats accounting for upload
* Convert / in names to a unicode equivalent (/)
* Poll for Google Drive changes when mounted
* OneDrive
* Fix the uploading of files with spaces
* Fix initialization order for token renewer
* Display speeds accurately when uploading - Yoni Jah
* Swap to using http://localhost:53682/ as redirect URL - Michael Ledin
* Retry on token expired error, reset upload body on retry - Yoni Jah
* Google Cloud Storage
* Add ability to specify location and storage class via config and command line - thanks gdm85
* Create container if necessary on server side copy
* Increase directory listing chunk to 1000 to increase performance
* Obtain a refresh token for GCS - Steven Lu
* Yandex
* Fix the name reported in log messages (was empty)
* Correct error return for listing empty directory
* Dropbox
* Rewritten to use the v2 API
* Now supports ModTime
* Can only set by uploading the file again
* If you uploaded with an old rclone, rclone may upload everything again
* Use `--size-only` or `--checksum` to avoid this
* Now supports the Dropbox content hashing scheme
* Now supports low level retries
* S3
* Work around eventual consistency in bucket creation
* Create container if necessary on server side copy
* Add us-east-2 (Ohio) and eu-west-2 (London) S3 regions - Zahiar Ahmed
* Swift, Hubic
* Fix zero length directory markers showing in the subdirectory listing
* this caused lots of duplicate transfers
* Fix paged directory listings
* this caused duplicate directory errors
* Create container if necessary on server side copy
* Increase directory listing chunk to 1000 to increase performance
* Make sensible error if the user forgets the container
* SFTP
* Add support for using ssh key files
* Fix under Windows
* Fix ssh agent on Windows
* Adapt to latest version of library - Igor Kharin
2017-03-18 11:16:43 +00:00
* v1.36 - 2017-03-18
* New Features
* SFTP remote (Jack Schmidt)
* Re-implement sync routine to work a directory at a time reducing memory usage
* Logging revamped to be more inline with rsync - now much quieter
* -v only shows transfers
* -vv is for full debug
* --syslog to log to syslog on capable platforms
* Implement --backup-dir and --suffix
* Implement --track-renames (initial implementation by Bjørn Erik Pedersen)
* Add time-based bandwidth limits (Lukas Loesche)
* rclone cryptcheck: checks integrity of crypt remotes
* Allow all config file variables and options to be set from environment variables
* Add --buffer-size parameter to control buffer size for copy
* Make --delete-after the default
* Add --ignore-checksum flag (fixed by Hisham Zarka)
* rclone check: Add --download flag to check all the data, not just hashes
* rclone cat: add --head, --tail, --offset, --count and --discard
* rclone config: when choosing from a list, allow the value to be entered too
* rclone config: allow rename and copy of remotes
* rclone obscure: for generating encrypted passwords for rclone's config (T.C. Ferguson)
* Comply with XDG Base Directory specification (Dario Giovannetti)
* this moves the default location of the config file in a backwards compatible way
* Release changes
* Ubuntu snap support (Dedsec1)
* Compile with go 1.8
* MIPS/Linux big and little endian support
* Bug Fixes
* Fix copyto copying things to the wrong place if the destination dir didn't exist
* Fix parsing of remotes in moveto and copyto
* Fix --delete-before deleting files on copy
* Fix --files-from with an empty file copying everything
* Fix sync: don't update mod times if --dry-run set
* Fix MimeType propagation
* Fix filters to add ** rules to directory rules
* Local
* Implement -L, --copy-links flag to allow rclone to follow symlinks
* Open files in write only mode so rclone can write to an rclone mount
* Fix unnormalised unicode causing problems reading directories
* Fix interaction between -x flag and --max-depth
* Mount
* Implement proper directory handling (mkdir, rmdir, renaming)
* Make include and exclude filters apply to mount
* Implement read and write async buffers - control with --buffer-size
* Fix fsync on for directories
* Fix retry on network failure when reading off crypt
* Crypt
* Add --crypt-show-mapping to show encrypted file mapping
* Fix crypt writer getting stuck in a loop
* **IMPORTANT** this bug had the potential to cause data corruption when
* reading data from a network based remote and
* writing to a crypt on Google Drive
* Use the cryptcheck command to validate your data if you are concerned
* If syncing two crypt remotes, sync the unencrypted remote
* Amazon Drive
* Fix panics on Move (rename)
* Fix panic on token expiry
* B2
* Fix inconsistent listings and rclone check
* Fix uploading empty files with go1.8
* Constrain memory usage when doing multipart uploads
* Fix upload url not being refreshed properly
* Drive
* Fix Rmdir on directories with trashed files
* Fix "Ignoring unknown object" when downloading
* Add --drive-list-chunk
* Add --drive-skip-gdocs (Károly Oláh)
* OneDrive
* Implement Move
* Fix Copy
* Fix overwrite detection in Copy
* Fix waitForJob to parse errors correctly
* Use token renewer to stop auth errors on long uploads
* Fix uploading empty files with go1.8
* Google Cloud Storage
* Fix depth 1 directory listings
* Yandex
* Fix single level directory listing
* Dropbox
* Normalise the case for single level directory listings
* Fix depth 1 listing
* S3
* Added ca-central-1 region (Jon Yergatian)
2017-01-02 15:30:34 +00:00
* v1.35 - 2017-01-02
* New Features
* moveto and copyto commands for choosing a destination name on copy/move
* rmdirs command to recursively delete empty directories
* Allow repeated --include/--exclude/--filter options
* Only show transfer stats on commands which transfer stuff
* show stats on any command using the `--stats` flag
* Allow overlapping directories in move when server side dir move is supported
* Add --stats-unit option - thanks Scott McGillivray
* Bug Fixes
* Fix the config file being overwritten when two rclones are running
* Make rclone lsd obey the filters properly
* Fix compilation on mips
* Fix not transferring files that don't differ in size
* Fix panic on nil retry/fatal error
* Mount
* Retry reads on error - should help with reliability a lot
* Report the modification times for directories from the remote
* Add bandwidth accounting and limiting (fixes --bwlimit)
* If --stats provided will show stats and which files are transferring
* Support R/W files if truncate is set.
* Implement statfs interface so df works
* Note that write is now supported on Amazon Drive
* Report number of blocks in a file - thanks Stefan Breunig
* Crypt
* Prevent the user pointing crypt at itself
* Fix failed to authenticate decrypted block errors
* these will now return the underlying unexpected EOF instead
* Amazon Drive
* Add support for server side move and directory move - thanks Stefan Breunig
* Fix nil pointer deref on size attribute
* B2
* Use new prefix and delimiter parameters in directory listings
* This makes --max-depth 1 dir listings as used in mount much faster
* Reauth the account while doing uploads too - should help with token expiry
* Drive
* Make DirMove more efficient and complain about moving the root
* Create destination directory on Move()
2016-11-06 10:17:52 +00:00
* v1.34 - 2016-11-06
* New Features
* Stop single file and `--files-from` operations iterating through the source bucket.
* Stop removing failed upload to cloud storage remotes
* Make ContentType be preserved for cloud to cloud copies
* Add support to toggle bandwidth limits via SIGUSR2 - thanks Marco Paganini
* `rclone check` shows count of hashes that couldn't be checked
* `rclone listremotes` command
* Support linux/arm64 build - thanks Fredrik Fornwall
* Remove `Authorization:` lines from `--dump-headers` output
* Bug Fixes
* Ignore files with control characters in the names
* Fix `rclone move` command
* Delete src files which already existed in dst
* Fix deletion of src file when dst file older
* Fix `rclone check` on crypted file systems
* Make failed uploads not count as "Transferred"
* Make sure high level retries show with `-q`
* Use a vendor directory with godep for repeatable builds
* `rclone mount` - FUSE
* Implement FUSE mount options
* `--no-modtime`, `--debug-fuse`, `--read-only`, `--allow-non-empty`, `--allow-root`, `--allow-other`
* `--default-permissions`, `--write-back-cache`, `--max-read-ahead`, `--umask`, `--uid`, `--gid`
* Add `--dir-cache-time` to control caching of directory entries
* Implement seek for files opened for read (useful for video players)
* with `-no-seek` flag to disable
* Fix crash on 32 bit ARM (alignment of 64 bit counter)
* ...and many more internal fixes and improvements!
* Crypt
* Don't show encrypted password in configurator to stop confusion
* Amazon Drive
* New wait for upload option `--acd-upload-wait-per-gb`
* upload timeouts scale by file size and can be disabled
* Add 502 Bad Gateway to list of errors we retry
* Fix overwriting a file with a zero length file
* Fix ACD file size warning limit - thanks Felix Bünemann
* Local
* Unix: implement `-x`/`--one-file-system` to stay on a single file system
* thanks Durval Menezes and Luiz Carlos Rumbelsperger Viana
* Windows: ignore the symlink bit on files
* Windows: Ignore directory based junction points
* B2
* Make sure each upload has at least one upload slot - fixes strange upload stats
* Fix uploads when using crypt
* Fix download of large files (sha1 mismatch)
* Return error when we try to create a bucket which someone else owns
* Update B2 docs with Data usage, and Crypt section - thanks Tomasz Mazur
* S3
* Command line and config file support for
* Setting/overriding ACL - thanks Radek Senfeld
* Setting storage class - thanks Asko Tamm
* Drive
* Make exponential backoff work exactly as per Google specification
* add `.epub`, `.odp` and `.tsv` as export formats.
* Swift
* Don't read metadata for directory marker objects
2016-08-24 21:58:24 +00:00
* v1.33 - 2016-08-24
* New Features
* Implement encryption
* data encrypted in NACL secretbox format
* with optional file name encryption
* New commands
* rclone mount - implements FUSE mounting of remotes (EXPERIMENTAL)
* works on Linux, FreeBSD and OS X (need testers for the last 2!)
* rclone cat - outputs remote file or files to the terminal
* rclone genautocomplete - command to make a bash completion script for rclone
* Editing a remote using `rclone config` now goes through the wizard
* Compile with go 1.7 - this fixes rclone on macOS Sierra and on 386 processors
* Use cobra for sub commands and docs generation
* drive
* Document how to make your own client_id
* s3
* User-configurable Amazon S3 ACL (thanks Radek Šenfeld)
* b2
* Fix stats accounting for upload - no more jumping to 100% done
* On cleanup delete hide marker if it is the current file
* New B2 API endpoint (thanks Per Cederberg)
* Set maximum backoff to 5 Minutes
* onedrive
* Fix URL escaping in file names - eg uploading files with `+` in them.
* amazon cloud drive
* Fix token expiry during large uploads
* Work around 408 REQUEST_TIMEOUT and 504 GATEWAY_TIMEOUT errors
* local
* Fix filenames with invalid UTF-8 not being uploaded
* Fix problem with some UTF-8 characters on OS X
2016-07-13 16:32:39 +00:00
* v1.32 - 2016-07-13
* Backblaze B2
* Fix upload of files large files not in root
2016-07-13 11:26:22 +00:00
* v1.31 - 2016-07-13
* New Features
* Reduce memory on sync by about 50%
* Implement --no-traverse flag to stop copy traversing the destination remote.
* This can be used to reduce memory usage down to the smallest possible.
* Useful to copy a small number of files into a large destination folder.
* Implement cleanup command for emptying trash / removing old versions of files
* Currently B2 only
* Single file handling improved
* Now copied with --files-from
* Automatically sets --no-traverse when copying a single file
* Info on using installing with ansible - thanks Stefan Weichinger
* Implement --no-update-modtime flag to stop rclone fixing the remote modified times.
* Bug Fixes
* Fix move command - stop it running for overlapping Fses - this was causing data loss.
* Local
* Fix incomplete hashes - this was causing problems for B2.
* Amazon Drive
* Rename Amazon Cloud Drive to Amazon Drive - no changes to config file needed.
* Swift
* Add support for non-default project domain - thanks Antonio Messina.
* S3
* Add instructions on how to use rclone with minio.
* Add ap-northeast-2 (Seoul) and ap-south-1 (Mumbai) regions.
* Skip setting the modified time for objects > 5GB as it isn't possible.
* Backblaze B2
* Add --b2-versions flag so old versions can be listed and retreived.
* Treat 403 errors (eg cap exceeded) as fatal.
* Implement cleanup command for deleting old file versions.
* Make error handling compliant with B2 integrations notes.
* Fix handling of token expiry.
* Implement --b2-test-mode to set `X-Bz-Test-Mode` header.
* Set cutoff for chunked upload to 200MB as per B2 guidelines.
* Make upload multi-threaded.
* Dropbox
* Don't retry 461 errors.
2016-06-18 15:58:00 +00:00
* v1.30 - 2016-06-18
2016-06-18 15:29:53 +00:00
* New Features
* Directory listing code reworked for more features and better error reporting (thanks to Klaus Post for help). This enables
* Directory include filtering for efficiency
* --max-depth parameter
* Better error reporting
* More to come
* Retry more errors
* Add --ignore-size flag - for uploading images to onedrive
* Log -v output to stdout by default
* Display the transfer stats in more human readable form
* Make 0 size files specifiable with `--max-size 0b`
* Add `b` suffix so we can specify bytes in --bwlimit, --min-size etc
* Use "password:" instead of "password>" prompt - thanks Klaus Post and Leigh Klotz
* Bug Fixes
* Fix retry doing one too many retries
* Local
* Fix problems with OS X and UTF-8 characters
* Amazon Drive
2016-06-18 15:29:53 +00:00
* Check a file exists before uploading to help with 408 Conflict errors
* Reauth on 401 errors - this has been causing a lot of problems
* Work around spurious 403 errors
* Restart directory listings on error
* Google Drive
* Check a file exists before uploading to help with duplicates
* Fix retry of multipart uploads
* Backblaze B2
* Implement large file uploading
* S3
* Add AES256 server-side encryption for - thanks Justin R. Wilson
* Google Cloud Storage
* Make sure we don't use conflicting content types on upload
* Add service account support - thanks Michal Witkowski
* Swift
* Add auth version parameter
* Add domain option for openstack (v3 auth) - thanks Fabian Ruff
2016-04-18 17:30:29 +00:00
* v1.29 - 2016-04-18
* New Features
* Implement `-I, --ignore-times` for unconditional upload
* Improve `dedupe`command
* Now removes identical copies without asking
* Now obeys `--dry-run`
* Implement `--dedupe-mode` for non interactive running
* `--dedupe-mode interactive` - interactive the default.
* `--dedupe-mode skip` - removes identical files then skips anything left.
* `--dedupe-mode first` - removes identical files then keeps the first one.
* `--dedupe-mode newest` - removes identical files then keeps the newest one.
* `--dedupe-mode oldest` - removes identical files then keeps the oldest one.
* `--dedupe-mode rename` - removes identical files then renames the rest to be different.
* Bug fixes
* Make rclone check obey the `--size-only` flag.
* Use "application/octet-stream" if discovered mime type is invalid.
* Fix missing "quit" option when there are no remotes.
* Google Drive
* Increase default chunk size to 8 MB - increases upload speed of big files
* Speed up directory listings and make more reliable
* Add missing retries for Move and DirMove - increases reliability
* Preserve mime type on file update
* Backblaze B2
* Enable mod time syncing
* This means that B2 will now check modification times
* It will upload new files to update the modification times
* (there isn't an API to just set the mod time.)
* If you want the old behaviour use `--size-only`.
* Update API to new version
* Fix parsing of mod time when not in metadata
* Swift/Hubic
* Don't return an MD5SUM for static large objects
* S3
* Fix uploading files bigger than 50GB
2016-03-01 09:00:01 +00:00
* v1.28 - 2016-03-01
* New Features
* Configuration file encryption - thanks Klaus Post
* Improve `rclone config` adding more help and making it easier to understand
* Implement `-u`/`--update` so creation times can be used on all remotes
* Implement `--low-level-retries` flag
* Optionally disable gzip compression on downloads with `--no-gzip-encoding`
* Bug fixes
* Don't make directories if `--dry-run` set
* Fix and document the `move` command
* Fix redirecting stderr on unix-like OSes when using `--log-file`
* Fix `delete` command to wait until all finished - fixes missing deletes.
* Backblaze B2
* Use one upload URL per go routine fixes `more than one upload using auth token`
* Add pacing, retries and reauthentication - fixes token expiry problems
* Upload without using a temporary file from local (and remotes which support SHA1)
* Fix reading metadata for all files when it shouldn't have been
* Drive
* Fix listing drive documents at root
* Disable copy and move for Google docs
* Swift
* Fix uploading of chunked files with non ASCII characters
* Allow setting of `storage_url` in the config - thanks Xavier Lucas
* S3
* Allow IAM role and credentials from environment variables - thanks Brian Stengaard
* Allow low privilege users to use S3 (check if directory exists during Mkdir) - thanks Jakub Gedeon
* Amazon Drive
2016-03-01 09:00:01 +00:00
* Retry on more things to make directory listings more reliable
2016-01-31 17:50:13 +00:00
* v1.27 - 2016-01-31
* New Features
* Easier headless configuration with `rclone authorize`
* Add support for multiple hash types - we now check SHA1 as well as MD5 hashes.
* `delete` command which does obey the filters (unlike `purge`)
* `dedupe` command to deduplicate a remote. Useful with Google Drive.
* Add `--ignore-existing` flag to skip all files that exist on destination.
* Add `--delete-before`, `--delete-during`, `--delete-after` flags.
* Add `--memprofile` flag to debug memory use.
* Warn the user about files with same name but different case
* Make `--include` rules add their implict exclude * at the end of the filter list
* Deprecate compiling with go1.3
* Amazon Drive
2016-01-31 17:50:13 +00:00
* Fix download of files > 10 GB
* Fix directory traversal ("Next token is expired") for large directory listings
* Remove 409 conflict from error codes we will retry - stops very long pauses
* Backblaze B2
* SHA1 hashes now checked by rclone core
* Drive
* Add `--drive-auth-owner-only` to only consider files owned by the user - thanks Björn Harrtell
* Export Google documents
* Dropbox
* Make file exclusion error controllable with -q
* Swift
* Fix upload from unprivileged user.
* S3
* Fix updating of mod times of files with `+` in.
* Local
* Add local file system option to disable UNC on Windows.
2016-01-02 12:03:38 +00:00
* v1.26 - 2016-01-02
* New Features
* Yandex storage backend - thank you Dmitry Burdeev ("dibu")
* Implement Backblaze B2 storage backend
* Add --min-age and --max-age flags - thank you Adriano Aurélio Meirelles
* Make ls/lsl/md5sum/size/check obey includes and excludes
* Fixes
* Fix crash in http logging
* Upload releases to github too
* Swift
* Fix sync for chunked files
* OneDrive
2016-01-02 12:03:38 +00:00
* Re-enable server side copy
* Don't mask HTTP error codes with JSON decode error
* S3
* Fix corrupting Content-Type on mod time update (thanks Joseph Spurrier)
2015-11-14 13:06:39 +00:00
* v1.25 - 2015-11-14
* New features
* Implement Hubic storage system
* Fixes
* Fix deletion of some excluded files without --delete-excluded
* This could have deleted files unexpectedly on sync
* Always check first with `--dry-run`!
* Swift
* Stop SetModTime losing metadata (eg X-Object-Manifest)
* This could have caused data loss for files > 5GB in size
* Use ContentType from Object to avoid lookups in listings
* OneDrive
2015-11-14 13:06:39 +00:00
* disable server side copy as it seems to be broken at Microsoft
2015-11-07 16:21:39 +00:00
* v1.24 - 2015-11-07
* New features
* Add support for Microsoft OneDrive
2015-11-07 16:21:39 +00:00
* Add `--no-check-certificate` option to disable server certificate verification
* Add async readahead buffer for faster transfer of big files
* Fixes
* Allow spaces in remotes and check remote names for validity at creation time
* Allow '&' and disallow ':' in Windows filenames.
* Swift
* Ignore directory marker objects where appropriate - allows working with Hubic
* Don't delete the container if fs wasn't at root
* S3
* Don't delete the bucket if fs wasn't at root
* Google Cloud Storage
* Don't delete the bucket if fs wasn't at root
2015-10-03 15:24:07 +00:00
* v1.23 - 2015-10-03
* New features
* Implement `rclone size` for measuring remotes
* Fixes
* Fix headless config for drive and gcs
* Tell the user they should try again if the webserver method failed
* Improve output of `--dump-headers`
* S3
* Allow anonymous access to public buckets
* Swift
* Stop chunked operations logging "Failed to read info: Object Not Found"
* Use Content-Length on uploads for extra reliability
2015-09-28 18:38:20 +00:00
* v1.22 - 2015-09-28
* Implement rsync like include and exclude flags
* swift
* Support files > 5GB - thanks Sergey Tolmachev
2015-09-22 20:05:11 +00:00
* v1.21 - 2015-09-22
* New features
* Display individual transfer progress
* Make lsl output times in localtime
* Fixes
* Fix allowing user to override credentials again in Drive, GCS and ACD
* Amazon Drive
2015-09-22 20:05:11 +00:00
* Implement compliant pacing scheme
* Google Drive
* Make directory reads concurrent for increased speed.
2015-09-15 06:42:31 +00:00
* v1.20 - 2015-09-15
* New features
* Amazon Drive support
2015-09-15 06:42:31 +00:00
* Oauth support redone - fix many bugs and improve usability
* Use "golang.org/x/oauth2" as oauth libary of choice
* Improve oauth usability for smoother initial signup
* drive, googlecloudstorage: optionally use auto config for the oauth token
* Implement --dump-headers and --dump-bodies debug flags
* Show multiple matched commands if abbreviation too short
* Implement server side move where possible
* local
* Always use UNC paths internally on Windows - fixes a lot of bugs
* dropbox
* force use of our custom transport which makes timeouts work
* Thanks to Klaus Post for lots of help with this release
2015-08-28 08:47:13 +00:00
* v1.19 - 2015-08-28
* New features
* Server side copies for s3/swift/drive/dropbox/gcs
* Move command - uses server side copies if it can
* Implement --retries flag - tries 3 times by default
* Build for plan9/amd64 and solaris/amd64 too
* Fixes
* Make a current version download with a fixed URL for scripting
* Ignore rmdir in limited fs rather than throwing error
* dropbox
* Increase chunk size to improve upload speeds massively
* Issue an error message when trying to upload bad file name
2015-08-17 16:59:37 +00:00
* v1.18 - 2015-08-17
* drive
* Add `--drive-use-trash` flag so rclone trashes instead of deletes
* Add "Forbidden to download" message for files with no downloadURL
* dropbox
* Remove datastore
* This was deprecated and it caused a lot of problems
* Modification times and MD5SUMs no longer stored
* Fix uploading files > 2GB
* s3
* use official AWS SDK from github.com/aws/aws-sdk-go
* **NB** will most likely require you to delete and recreate remote
* enable multipart upload which enables files > 5GB
* tested with Ceph / RadosGW / S3 emulation
* many thanks to Sam Liston and Brian Haymore at the [Utah
Center for High Performance Computing](https://www.chpc.utah.edu/) for a Ceph test account
* misc
* Show errors when reading the config file
* Do not print stats in quiet mode - thanks Leonid Shalupov
* Add FAQ
* Fix created directories not obeying umask
* Linux installation instructions - thanks Shimon Doodkin
2015-06-14 14:36:16 +00:00
* v1.17 - 2015-06-14
* dropbox: fix case insensitivity issues - thanks Leonid Shalupov
2015-06-09 17:00:33 +00:00
* v1.16 - 2015-06-09
* Fix uploading big files which was causing timeouts or panics
* Don't check md5sum after download with --size-only
2015-06-06 14:45:00 +00:00
* v1.15 - 2015-06-06
* Add --checksum flag to only discard transfers by MD5SUM - thanks Alex Couper
* Implement --size-only flag to sync on size not checksum & modtime
* Expand docs and remove duplicated information
* Document rclone's limitations with directories
* dropbox: update docs about case insensitivity
* v1.14 - 2015-05-21
* local: fix encoding of non utf-8 file names - fixes a duplicate file problem
* drive: docs about rate limiting
* google cloud storage: Fix compile after API change in "google.golang.org/api/storage/v1"
* v1.13 - 2015-05-10
* Revise documentation (especially sync)
* Implement --timeout and --conntimeout
* s3: ignore etags from multipart uploads which aren't md5sums
* v1.12 - 2015-03-15
* drive: Use chunked upload for files above a certain size
* drive: add --drive-chunk-size and --drive-upload-cutoff parameters
* drive: switch to insert from update when a failed copy deletes the upload
* core: Log duplicate files if they are detected
* v1.11 - 2015-03-04
* swift: add region parameter
* drive: fix crash on failed to update remote mtime
* In remote paths, change native directory separators to /
* Add synchronization to ls/lsl/lsd output to stop corruptions
* Ensure all stats/log messages to go stderr
* Add --log-file flag to log everything (including panics) to file
* Make it possible to disable stats printing with --stats=0
* Implement --bwlimit to limit data transfer bandwidth
* v1.10 - 2015-02-12
* s3: list an unlimited number of items
* Fix getting stuck in the configurator
* v1.09 - 2015-02-07
* windows: Stop drive letters (eg C:) getting mixed up with remotes (eg drive:)
* local: Fix directory separators on Windows
* drive: fix rate limit exceeded errors
* v1.08 - 2015-02-04
* drive: fix subdirectory listing to not list entire drive
* drive: Fix SetModTime
* dropbox: adapt code to recent library changes
* v1.07 - 2014-12-23
* google cloud storage: fix memory leak
* v1.06 - 2014-12-12
* Fix "Couldn't find home directory" on OSX
* swift: Add tenant parameter
* Use new location of Google API packages
* v1.05 - 2014-08-09
* Improved tests and consequently lots of minor fixes
* core: Fix race detected by go race detector
* core: Fixes after running errcheck
* drive: reset root directory on Rmdir and Purge
* fs: Document that Purger returns error on empty directory, test and fix
* google cloud storage: fix ListDir on subdirectory
* google cloud storage: re-read metadata in SetModTime
* s3: make reading metadata more reliable to work around eventual consistency problems
* s3: strip trailing / from ListDir()
* swift: return directories without / in ListDir
* v1.04 - 2014-07-21
* google cloud storage: Fix crash on Update
* v1.03 - 2014-07-20
* swift, s3, dropbox: fix updated files being marked as corrupted
* Make compile with go 1.1 again
* v1.02 - 2014-07-19
* Implement Dropbox remote
* Implement Google Cloud Storage remote
* Verify Md5sums and Sizes after copies
* Remove times from "ls" command - lists sizes only
* Add add "lsl" - lists times and sizes
* Add "md5sum" command
* v1.01 - 2014-07-04
* drive: fix transfer of big files using up lots of memory
* v1.00 - 2014-07-03
* drive: fix whole second dates
* v0.99 - 2014-06-26
* Fix --dry-run not working
* Make compatible with go 1.1
* v0.98 - 2014-05-30
* s3: Treat missing Content-Length as 0 for some ceph installations
* rclonetest: add file with a space in
* v0.97 - 2014-05-05
* Implement copying of single files
* s3 & swift: support paths inside containers/buckets
* v0.96 - 2014-04-24
* drive: Fix multiple files of same name being created
* drive: Use o.Update and fs.Put to optimise transfers
* Add version number, -V and --version
* v0.95 - 2014-03-28
* rclone.org: website, docs and graphics
* drive: fix path parsing
* v0.94 - 2014-03-27
* Change remote format one last time
* GNU style flags
* v0.93 - 2014-03-16
* drive: store token in config file
* cross compile other versions
* set strict permissions on config file
* v0.92 - 2014-03-15
* Config fixes and --config option
* v0.91 - 2014-03-15
* Make config file
* v0.90 - 2013-06-27
* Project named rclone
* v0.00 - 2012-11-18
* Project started