TrueCloudLab/rclone
16
0
Fork 2
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions 1
1423 commits 181 branches 114 tags 335 MiB
Commit graph

40 commits

Author SHA1 Message Date
Nick Craig-Wood
62e28d0a72 Replace token bucket limiter github.com/tsenart/tb with golang.org/x/time/rate 
In tests tsenart/tb has proved inaccurate at low rates.
 2017-07-16 16:14:44 +01:00
Nick Craig-Wood
aa20486485 Add --stats-log-level so can see --stats without -v - fixes #1180 
The most common use for this flag is likely to be showing the stats
without using -v by using `--stats-log-level NOTICE`.
 2017-06-26 22:50:37 +01:00
Nick Craig-Wood
740b3f6ae2 Fix problems found with ineffassign 2017-06-13 11:52:36 +01:00
Nick Craig-Wood
033d1eb7af Refactor Account interface 2017-02-17 09:15:24 +00:00
Nick Craig-Wood
928be0f1fd mount: fix seek with buffering to use correct interface 
Stop pre-cache before seeking which stops lots of excess data transfer
 2017-02-17 08:55:24 +00:00
Nick Craig-Wood
f15c6b68b6 Re-add the async buffer on seek - fixes #1137 2017-02-15 22:54:21 +00:00
Nick Craig-Wood
3f778d70f7 Add sync.Pool to async reader 2017-02-15 22:37:58 +00:00
Nick Craig-Wood
7fa687b3e1 fs: Async buffer: use ReadFill to fill the chunks and increase to 1MB 2017-02-14 22:36:37 +00:00
Nick Craig-Wood
493da54113 Add --buffer-size parameter to control buffer size for copy 2017-02-14 22:36:37 +00:00
Nick Craig-Wood
0366ea39c5 Reassign some logging levels 2017-02-11 17:56:05 +00:00
Nick Craig-Wood
80f53176d9 Rename log functions and factor into own file 2017-02-11 17:54:50 +00:00
Marco Paganini
6611d92e21 Only start bandwidth ticker when necessary. 
- Only start the token ticker when the timetable entry has more than one
  entry.
- This fixes the "Scheduled bandwidth change" log message when no
  bwlimit is specified.
- Fixes #987
 2017-01-04 19:03:49 -08:00
Marco Paganini
3b0f944e23 Add time-based bandwidth limits. 
- Change the --bwlimit command line parameter to accept both a limit (as
  before) or a full timetable (formatted as "hh:mm,limit
  hh:mm,limit...")
- The timetable is checked once a minute by a ticker function. A new
  tokenBucket is created every time a bandwidth change is necessary.
- This change is compatible with the SIGUSR2 change to toggle bandwidth
  limits.

This resolves #221.
 2017-01-03 21:00:38 +00:00
Nick Craig-Wood
4482e75f38 Fix golint 2016-12-15 21:02:41 +00:00
Nick Craig-Wood
0117aeafbf mount: this removes the async buffering as it was killing seek performance 2016-12-15 17:08:52 +00:00
Nick Craig-Wood
be4fd51289 fuse: Add bandwidth accounting and buffering 
This fixes rclone mount ignoring bwlimit and increases buffering which
should speed up transfers greatly.

Fixes #796
Fixes #690
 2016-12-04 16:57:47 +00:00
Scott McGillivray
f9df545e3c add --stats-unit option and improve alignment for --stats output 2016-11-28 12:18:30 +00:00
Nick Craig-Wood
e65059e431 Fix non-windows/non-unix builds for bwlimit/SIGUSR2 feature and add a mutex 
The race detector complained whenever SIGUSR2 was sent to rclone so
this adds a mutex to prevent concurrent access.
 2016-10-30 19:20:16 +00:00
Marco Paganini
cc4f5ba7ba Add support to toggle bandwidth limits via SIGUSR2. 
Sending rclone a SIGUSR2 signal will toggle the limiter between off and
the limit set with the --bwlimit command-line option.
 2016-10-30 17:46:59 +00:00
Nick Craig-Wood
83ba59749f Make failed uploads not count as "Transferred" - fixes #708 2016-09-12 18:15:58 +01:00
Nick Craig-Wood
2eaac80c86 b2 with crypt: fix crash when uploading large files - fixes #673 2016-09-05 18:10:01 +01:00
Nick Craig-Wood
037a000cc8 b2: fix stats accounting for upload - fixes #602 2016-08-22 21:19:38 +01:00
Nick Craig-Wood
a20d80565b Tidy stats output - fixes #541 2016-07-11 13:04:30 +01:00
Nick Craig-Wood
0a43ff9c13 Modify interface for accounting to take a string not an fs.Object 2016-07-02 16:58:50 +01:00
Nick Craig-Wood
f438f1e9ef Fix stats print 2016-06-18 16:41:46 +01:00
Nick Craig-Wood
67d0375b98 Audit use of log.Print and change to Debug, Log, or ErrorLog as appropriate 2016-06-06 21:23:54 +01:00
Nick Craig-Wood
e2788aa729 Display the transfer stats in more human readable form - fixes #428 2016-06-03 22:49:50 +01:00
Nick Craig-Wood
ed72c678f8 Protect accounting from being closed twice 2015-11-06 15:26:58 +00:00
Nick Craig-Wood
e9c915e6fe Fix golint warnings 2015-09-22 18:47:16 +01:00
Nick Craig-Wood
e391311512 gofmt 2015-09-17 18:42:39 +01:00
klauspost
788ef76f1c Show more of the filename and align output. 
Print more of the file name, and make the output aligned, so it is nicer on frequent updates.
 2015-09-17 15:53:45 +02:00
klauspost
0a5870208e Display individual transfer progress 
Improve progress printing by displaying individual file progress, as well
as a moving average speed with ETA. Example output:

2015/09/15 16:38:21
Transferred:    183599104 Bytes (4646.49 kByte/s)
Errors:                 0
Checks:                 1
Transferred:            0
Elapsed time:       38.5s
Transferring:
 * 01_06_14.mp3: 33% done. avg: 1280.5, cur: 1288.8 kByte/s. ETA: 1m12s
 * 01_12_15.mp3: 33% done. avg: 1002.2, cur:  943.4 kByte/s. ETA: 1m17s
 * 01_13_14.mp3: 48% done. avg: 1456.8, cur: 1425.2 kByte/s. ETA: 39s
 * 01_19_15.mp3: 28% done. avg: 1226.9, cur: 1114.4 kByte/s. ETA: 1m37s
 2015-09-16 19:42:46 +01:00
Nick Craig-Wood
0c6f816a49 Implement --retries flag - fixes #109 2015-08-20 21:07:00 +01:00
Nick Craig-Wood
74994a2ec1 Fix uploading big files which was causing timeouts or panics 
The symtom was one of these two on upload of files only
  * panic: d.nx != 0 in crypto/md5.(*digest).checkSum
  * read tcp: i/o timeout

It turned out to be a combination of two upstream bugs

  * 5a2187309e
  * https://groups.google.com/forum/#!topic/golang-dev/0Nl6k5Sj6UU

This commit contains a work-around for the second problem, I've fixed
the first and had the change accepted upstream.
 2015-06-09 17:32:45 +01:00
Nick Craig-Wood
e22b445cff Implement --size-only flag to sync on size not checksum & modtime - fixes #75 2015-06-06 08:49:01 +01:00
Alex Couper
7af1282375 Add --checksum flag to only discard transfers by MD5SUM - fixes #61 
Useful for copying between backends where checksum fetching is fast,
ie any of s3, swift, drive or googlecloudstorage.
 2015-06-05 19:46:03 +01:00
Nick Craig-Wood
a287e3ced7 Implement --bwlimit to limit data transfer bandwidth 2015-02-27 15:03:47 +00:00
Nick Craig-Wood
754c54b64f Fix differences print out on check 2013-07-02 08:33:17 +01:00
Nick Craig-Wood
b1f681b912 Rename fs.FsX to fs.X 2013-06-28 08:57:32 +01:00
Nick Craig-Wood
6085dc1b5a Refactor into sub filesystems 2013-06-27 22:17:51 +01:00
Renamed from accounting.go (Browse further)