qingstor: add upload chunk size/concurrency/cutoff control #2851

* --upload-chunk-size
* --upload-concurrency
* --upload-cutoff
This commit is contained in:
Nick Craig-Wood 2019-01-04 11:24:20 +00:00
parent 9685be64cd
commit a2341cc412
4 changed files with 166 additions and 21 deletions

View file

@ -234,4 +234,50 @@ Number of connection retries.
- Type: int
- Default: 3
#### --qingstor-upload-cutoff
Cutoff for switching to chunked upload
Any files larger than this will be uploaded in chunks of chunk_size.
The minimum is 0 and the maximum is 5GB.
- Config: upload_cutoff
- Env Var: RCLONE_QINGSTOR_UPLOAD_CUTOFF
- Type: SizeSuffix
- Default: 200M
#### --qingstor-chunk-size
Chunk size to use for uploading.
When uploading files larger than upload_cutoff they will be uploaded
as multipart uploads using this chunk size.
Note that "--qingstor-upload-concurrency" chunks of this size are buffered
in memory per transfer.
If you are transferring large files over high speed links and you have
enough memory, then increasing this will speed up the transfers.
- Config: chunk_size
- Env Var: RCLONE_QINGSTOR_CHUNK_SIZE
- Type: SizeSuffix
- Default: 4M
#### --qingstor-upload-concurrency
Concurrency for multipart uploads.
This is the number of chunks of the same file that are uploaded
concurrently.
If you are uploading small numbers of large file over high speed link
and these uploads do not fully utilize your bandwidth, then increasing
this may help to speed up the transfers.
- Config: upload_concurrency
- Env Var: RCLONE_QINGSTOR_UPLOAD_CONCURRENCY
- Type: int
- Default: 4
<!--- autogenerated options stop -->