potyarkin/rclone
1
0
Fork 0
forked from TrueCloudLab/rclone
Code Pull requests Activity
1852 commits 3 branches 114 tags 338 MiB
Commit graph

48 commits

Author SHA1 Message Date
Nick Craig-Wood
11da2a6c9b Break the fs package up into smaller parts. 
The purpose of this is to make it easier to maintain and eventually to
allow the rclone backends to be re-used in other projects without
having to use the rclone configuration system.

The new code layout is documented in CONTRIBUTING.
 2018-01-15 17:51:14 +00:00
Will Gunn
5c7e8d5a2b fs: Add --stats-file-name-length to specify the printed file name length for stats 
Fixes #1206
 2018-01-10 20:32:36 +00:00
ishuah
b05e472d2e stats: condensed transfer output to fit 80x25 terminals 2017-12-09 10:48:36 +03:00
ishuah
b4aa920a3d stats: show the amount of data transferred in kb/mb - fixes #1167 2017-11-21 12:40:02 +03:00
ishuah
aec2265be0 rclone: implement exit codes - #1136 2017-11-15 17:48:37 +03:00
Nick Craig-Wood
2c8d6e86cc fs: fix gofmt 2017-08-31 16:01:19 +01:00
cbruegg
bb6300b032 Fix bwlimit toggle in conjunction with schedules (Fixes #1607) 2017-08-31 15:33:29 +01:00
Stefan Breunig
28a18303f3 implement rcat – fixes #230, fixes #1001 2017-08-03 21:42:35 +02:00
Nick Craig-Wood
62e28d0a72 Replace token bucket limiter github.com/tsenart/tb with golang.org/x/time/rate 
In tests tsenart/tb has proved inaccurate at low rates.
 2017-07-16 16:14:44 +01:00
Nick Craig-Wood
aa20486485 Add --stats-log-level so can see --stats without -v - fixes #1180 
The most common use for this flag is likely to be showing the stats
without using -v by using `--stats-log-level NOTICE`.
 2017-06-26 22:50:37 +01:00
Nick Craig-Wood
740b3f6ae2 Fix problems found with ineffassign 2017-06-13 11:52:36 +01:00
Nick Craig-Wood
033d1eb7af Refactor Account interface 2017-02-17 09:15:24 +00:00
Nick Craig-Wood
928be0f1fd mount: fix seek with buffering to use correct interface 
Stop pre-cache before seeking which stops lots of excess data transfer
 2017-02-17 08:55:24 +00:00
Nick Craig-Wood
f15c6b68b6 Re-add the async buffer on seek - fixes #1137 2017-02-15 22:54:21 +00:00
Nick Craig-Wood
3f778d70f7 Add sync.Pool to async reader 2017-02-15 22:37:58 +00:00
Nick Craig-Wood
7fa687b3e1 fs: Async buffer: use ReadFill to fill the chunks and increase to 1MB 2017-02-14 22:36:37 +00:00
Nick Craig-Wood
493da54113 Add --buffer-size parameter to control buffer size for copy 2017-02-14 22:36:37 +00:00
Nick Craig-Wood
0366ea39c5 Reassign some logging levels 2017-02-11 17:56:05 +00:00
Nick Craig-Wood
80f53176d9 Rename log functions and factor into own file 2017-02-11 17:54:50 +00:00
Marco Paganini
6611d92e21 Only start bandwidth ticker when necessary. 
- Only start the token ticker when the timetable entry has more than one
  entry.
- This fixes the "Scheduled bandwidth change" log message when no
  bwlimit is specified.
- Fixes #987
 2017-01-04 19:03:49 -08:00
Marco Paganini
3b0f944e23 Add time-based bandwidth limits. 
- Change the --bwlimit command line parameter to accept both a limit (as
  before) or a full timetable (formatted as "hh:mm,limit
  hh:mm,limit...")
- The timetable is checked once a minute by a ticker function. A new
  tokenBucket is created every time a bandwidth change is necessary.
- This change is compatible with the SIGUSR2 change to toggle bandwidth
  limits.

This resolves #221.
 2017-01-03 21:00:38 +00:00
Nick Craig-Wood
4482e75f38 Fix golint 2016-12-15 21:02:41 +00:00
Nick Craig-Wood
0117aeafbf mount: this removes the async buffering as it was killing seek performance 2016-12-15 17:08:52 +00:00
Nick Craig-Wood
be4fd51289 fuse: Add bandwidth accounting and buffering 
This fixes rclone mount ignoring bwlimit and increases buffering which
should speed up transfers greatly.

Fixes #796
Fixes #690
 2016-12-04 16:57:47 +00:00
Scott McGillivray
f9df545e3c add --stats-unit option and improve alignment for --stats output 2016-11-28 12:18:30 +00:00
Nick Craig-Wood
e65059e431 Fix non-windows/non-unix builds for bwlimit/SIGUSR2 feature and add a mutex 
The race detector complained whenever SIGUSR2 was sent to rclone so
this adds a mutex to prevent concurrent access.
 2016-10-30 19:20:16 +00:00
Marco Paganini
cc4f5ba7ba Add support to toggle bandwidth limits via SIGUSR2. 
Sending rclone a SIGUSR2 signal will toggle the limiter between off and
the limit set with the --bwlimit command-line option.
 2016-10-30 17:46:59 +00:00
Nick Craig-Wood
83ba59749f Make failed uploads not count as "Transferred" - fixes #708 2016-09-12 18:15:58 +01:00
Nick Craig-Wood
2eaac80c86 b2 with crypt: fix crash when uploading large files - fixes #673 2016-09-05 18:10:01 +01:00
Nick Craig-Wood
037a000cc8 b2: fix stats accounting for upload - fixes #602 2016-08-22 21:19:38 +01:00
Nick Craig-Wood
a20d80565b Tidy stats output - fixes #541 2016-07-11 13:04:30 +01:00
Nick Craig-Wood
0a43ff9c13 Modify interface for accounting to take a string not an fs.Object 2016-07-02 16:58:50 +01:00
Nick Craig-Wood
f438f1e9ef Fix stats print 2016-06-18 16:41:46 +01:00
Nick Craig-Wood
67d0375b98 Audit use of log.Print and change to Debug, Log, or ErrorLog as appropriate 2016-06-06 21:23:54 +01:00
Nick Craig-Wood
e2788aa729 Display the transfer stats in more human readable form - fixes #428 2016-06-03 22:49:50 +01:00
Nick Craig-Wood
ed72c678f8 Protect accounting from being closed twice 2015-11-06 15:26:58 +00:00
Nick Craig-Wood
e9c915e6fe Fix golint warnings 2015-09-22 18:47:16 +01:00
Nick Craig-Wood
e391311512 gofmt 2015-09-17 18:42:39 +01:00
klauspost
788ef76f1c Show more of the filename and align output. 
Print more of the file name, and make the output aligned, so it is nicer on frequent updates.
 2015-09-17 15:53:45 +02:00
klauspost
0a5870208e Display individual transfer progress 
Improve progress printing by displaying individual file progress, as well
as a moving average speed with ETA. Example output:

2015/09/15 16:38:21
Transferred:    183599104 Bytes (4646.49 kByte/s)
Errors:                 0
Checks:                 1
Transferred:            0
Elapsed time:       38.5s
Transferring:
 * 01_06_14.mp3: 33% done. avg: 1280.5, cur: 1288.8 kByte/s. ETA: 1m12s
 * 01_12_15.mp3: 33% done. avg: 1002.2, cur:  943.4 kByte/s. ETA: 1m17s
 * 01_13_14.mp3: 48% done. avg: 1456.8, cur: 1425.2 kByte/s. ETA: 39s
 * 01_19_15.mp3: 28% done. avg: 1226.9, cur: 1114.4 kByte/s. ETA: 1m37s
 2015-09-16 19:42:46 +01:00
Nick Craig-Wood
0c6f816a49 Implement --retries flag - fixes #109 2015-08-20 21:07:00 +01:00
Nick Craig-Wood
74994a2ec1 Fix uploading big files which was causing timeouts or panics 
The symtom was one of these two on upload of files only
  * panic: d.nx != 0 in crypto/md5.(*digest).checkSum
  * read tcp: i/o timeout

It turned out to be a combination of two upstream bugs

  * 5a2187309e
  * https://groups.google.com/forum/#!topic/golang-dev/0Nl6k5Sj6UU

This commit contains a work-around for the second problem, I've fixed
the first and had the change accepted upstream.
 2015-06-09 17:32:45 +01:00
Nick Craig-Wood
e22b445cff Implement --size-only flag to sync on size not checksum & modtime - fixes #75 2015-06-06 08:49:01 +01:00
Alex Couper
7af1282375 Add --checksum flag to only discard transfers by MD5SUM - fixes #61 
Useful for copying between backends where checksum fetching is fast,
ie any of s3, swift, drive or googlecloudstorage.
 2015-06-05 19:46:03 +01:00
Nick Craig-Wood
a287e3ced7 Implement --bwlimit to limit data transfer bandwidth 2015-02-27 15:03:47 +00:00
Nick Craig-Wood
754c54b64f Fix differences print out on check 2013-07-02 08:33:17 +01:00
Nick Craig-Wood
b1f681b912 Rename fs.FsX to fs.X 2013-06-28 08:57:32 +01:00
Nick Craig-Wood
6085dc1b5a Refactor into sub filesystems 2013-06-27 22:17:51 +01:00
Renamed from accounting.go (Browse further)