alexvanin/rclone
1
0
Fork 0
forked from TrueCloudLab/rclone
Code Pull requests Activity
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
4180 commits 180 branches 114 tags 335 MiB
Go 98.6%
HTML 0.4%
Shell 0.3%
Python 0.3%
JavaScript 0.2%
Other 0.1%
Find a file
Nick Craig-Wood b1d8da484b azureblob: retry InvalidBlobOrBlock error as it may indicate block concurrency problems 
According to Microsoft support this error can be caused by

> A timing/concurrency issue where the PUT operations are happening
> about the same time for a single blob. The Put Block List operation
> writes a blob by specifying the list of block IDs that make up the
> blob. In order to be written as part of a blob, a block must have
> been successfully written to the server in a prior Put Block
> operation.
>
> Documentation reference:
>
> https://docs.microsoft.com/en-us/rest/api/storageservices/put-block
>
> This error can happen when doing concurrent upload commits after you
> have started the upload but before you commit. In that case, the
> upload fails. The application can retry this error or attempt some
> other recovery action based on the required scenario.

See: https://forum.rclone.org/t/error-while-syncing-with-azure-blob-storage-x-ms-error-code-invalidbloborblock/15561
2020-04-28 17:47:10 +01:00
.github build: update to use go1.14 for the build 2020-02-26 21:26:47 +00:00
backend azureblob: retry InvalidBlobOrBlock error as it may indicate block concurrency problems 2020-04-28 17:47:10 +01:00
bin doc: fix links in generated documentation 2020-03-01 09:14:37 +00:00
cmd dbhashsum: deprecate: use rclone hashsum DropboxHash instead 2020-04-23 11:13:13 +01:00
contrib/docker Contrib: Add sample WebDAV server Docker Compose manifest. 2019-09-15 16:06:54 +01:00
docs Add Adam Stroud to contributors 2020-04-28 17:47:10 +01:00
fs accounting: fix race condition in tests 2020-04-24 12:32:09 +01:00
fstest fstest: create AssertTimeEqualWithPrecision from CheckTimeEqualWithPrecision 2020-04-22 11:33:00 +01:00
graphics graphics: add more differently sized logos 2020-01-15 16:25:20 +00:00
lib jwtutil: Fix error handling 2020-04-23 17:52:14 +01:00
vendor vendor: add github.com/aalpar/deheap 2020-03-16 15:50:04 +00:00
vfs vfs: fix tests for Statfs when running on backends with unknowns 2020-04-22 18:14:03 +01:00
.gitattributes build: add azure pipelines build 2019-08-06 10:31:32 +01:00
.gitignore s3: Add StackPath Object Storage Support 2020-01-31 16:05:44 +00:00
.golangci.yml build: move linter build tags into Makefile to fix golangci-lint 2019-04-12 15:48:36 +01:00
CONTRIBUTING.md gendocs: add autogenerated header to all docs 2020-02-10 15:29:39 +00:00
COPYING Initial commit - some small parts working 2012-11-18 17:32:31 +00:00
Dockerfile Dockerfile: remove GOOS and GOARCH 2020-04-15 17:09:51 +01:00
go.mod vendor: add github.com/aalpar/deheap 2020-03-16 15:50:04 +00:00
go.sum vendor: add github.com/aalpar/deheap 2020-03-16 15:50:04 +00:00
MAINTAINERS.md Add @Max-Sum as the union backend maintainer 2020-03-21 18:16:42 +00:00
Makefile build: compress the test builds 2020-01-17 11:47:40 +00:00
MANUAL.html Version v1.51.0 2020-02-01 10:40:01 +00:00
MANUAL.md Version v1.51.0 2020-02-01 10:40:01 +00:00
MANUAL.txt Version v1.51.0 2020-02-01 10:40:01 +00:00
notes.txt Replace test_all.sh with test_all.go which is cross platform and parallel 2015-12-30 09:26:34 +00:00
rclone.1 Version v1.51.0 2020-02-01 10:40:01 +00:00
rclone.go lib: add plugin support 2019-09-29 11:05:10 +01:00
README.md docs: Update README.md shields for changed CI 2020-02-01 20:17:40 +00:00
RELEASE.md Start v1.51.0-DEV development 2020-02-01 12:32:21 +00:00
VERSION Version v1.51.0 2020-02-01 10:40:01 +00:00

README.md

rclone logo

Website | Documentation | Download | Contributing | Changelog | Installation | Forum

Build Status Go Report Card GoDoc Docker Pulls

Rclone

Rclone ("rsync for cloud storage") is a command line program to sync files and directories to and from different cloud storage providers.

Storage providers

Please see the full list of all storage providers and their features

Features

  • MD5/SHA-1 hashes checked at all times for file integrity
  • Timestamps preserved on files
  • Partial syncs supported on a whole file basis
  • Copy mode to just copy new/changed files
  • Sync (one way) mode to make a directory identical
  • Check mode to check for file hash equality
  • Can sync to and from network, e.g. two different cloud accounts
  • Optional large file chunking (Chunker)
  • Optional encryption (Crypt)
  • Optional cache (Cache)
  • Optional FUSE mount (rclone mount)
  • Multi-threaded downloads to local disk
  • Can serve local or remote files over HTTP/WebDav/FTP/SFTP/dlna

Installation & documentation

Please see the rclone website for:

Downloads

License

This is free software under the terms of MIT the license (check the COPYING file included in this package).