KurlesHS/rclone
1
0
Fork 0
forked from TrueCloudLab/rclone
Code Pull requests Activity

onedrive: warn on gateway timeout errors

Browse source 
It seems that when doing chunked uploads to onedrive, if the chunks
take more than 3 minutes or so to upload then they may timeout with
error 504 Gateway Timeout.

This change produces an error (just once) suggesting lowering
`--onedrive-chunk-size` or decreasing `--transfers`.

This is easy to replicate with:

    rclone copy -Pvv --bwlimit 0.05M 20M onedrive:20M

See: https://forum.rclone.org/t/default-onedrive-chunk-size-does-not-work/20010/
This commit is contained in:
Nick Craig-Wood 2020-11-02 16:49:27 +00:00
parent e8c84d8b53
commit bedf6e90d2
1 changed files with 6 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
backend/onedrive/onedrive.go
View file

@ -427,6 +427,8 @@ var retryErrorCodes = []int{
509, // Bandwidth Limit Exceeded 509, // Bandwidth Limit Exceeded
} }
var gatewayTimeoutError sync.Once
// shouldRetry returns a boolean as to whether this resp and err // shouldRetry returns a boolean as to whether this resp and err
// deserve to be retried. It returns the err as a convenience // deserve to be retried. It returns the err as a convenience
func shouldRetry(resp *http.Response, err error) (bool, error) { func shouldRetry(resp *http.Response, err error) (bool, error) {
@ -451,6 +453,10 @@ func shouldRetry(resp *http.Response, err error) (bool, error) {
fs.Debugf(nil, "Too many requests. Trying again in %d seconds.", retryAfter) fs.Debugf(nil, "Too many requests. Trying again in %d seconds.", retryAfter)
} }
} }
case 504: // Gateway timeout
gatewayTimeoutError.Do(func() {
fs.Errorf(nil, "%v: upload chunks may be taking too long - try reducing --onedrive-chunk-size or decreasing --transfers", err)
})
case 507: // Insufficient Storage case 507: // Insufficient Storage
return false, fserrors.FatalError(err) return false, fserrors.FatalError(err)
} }