drive: add docs for --fast-list and add to integration tests
This commit is contained in:
parent
dc5a734522
commit
f7c0b2407d
3 changed files with 42 additions and 3 deletions
|
@ -312,6 +312,45 @@ d) Delete this remote
|
|||
y/e/d> y
|
||||
```
|
||||
|
||||
### --fast-list ###
|
||||
|
||||
This remote supports `--fast-list` which allows you to use fewer
|
||||
transactions in exchange for more memory. See the [rclone
|
||||
docs](/docs/#fast-list) for more details.
|
||||
|
||||
It does this by combining multiple `list` calls into a single API request.
|
||||
|
||||
This works by combining many `'%s' in parents` filters into one expression.
|
||||
To list the contents of directories a, b and c, the the following requests will be send by the regular `List` function:
|
||||
```
|
||||
trashed=false and 'a' in parents
|
||||
trashed=false and 'b' in parents
|
||||
trashed=false and 'c' in parents
|
||||
```
|
||||
These can now be combined into a single request:
|
||||
```
|
||||
trashed=false and ('a' in parents or 'b' in parents or 'c' in parents)
|
||||
```
|
||||
|
||||
The implementation of `ListR` will put up to 50 `parents` filters into one request.
|
||||
It will use the `--checkers` value to specify the number of requests to run in parallel.
|
||||
|
||||
In tests, these batch requests were up to 20x faster than the regular method.
|
||||
Running the following command against different sized folders gives:
|
||||
```
|
||||
rclone lsjson -vv -R --checkers=6 gdrive:folder
|
||||
```
|
||||
|
||||
small folder (220 directories, 700 files):
|
||||
|
||||
- without `--fast-list`: 38s
|
||||
- with `--fast-list`: 10s
|
||||
|
||||
large folder (10600 directories, 39000 files):
|
||||
|
||||
- without `--fast-list`: 22:05 min
|
||||
- with `--fast-list`: 58s
|
||||
|
||||
### Modified time ###
|
||||
|
||||
Google drive stores modification times accurate to 1 ms.
|
||||
|
|
|
@ -134,7 +134,7 @@ operations more efficient.
|
|||
| Dropbox | Yes | Yes | Yes | Yes | No [#575](https://github.com/ncw/rclone/issues/575) | No | Yes | Yes | Yes |
|
||||
| FTP | No | No | Yes | Yes | No | No | Yes | No [#2178](https://github.com/ncw/rclone/issues/2178) | No |
|
||||
| Google Cloud Storage | Yes | Yes | No | No | No | Yes | Yes | No [#2178](https://github.com/ncw/rclone/issues/2178) | No |
|
||||
| Google Drive | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes |
|
||||
| Google Drive | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
|
||||
| HTTP | No | No | No | No | No | No | No | No [#2178](https://github.com/ncw/rclone/issues/2178) | No |
|
||||
| Hubic | Yes † | Yes | No | No | No | Yes | Yes | No [#2178](https://github.com/ncw/rclone/issues/2178) | Yes |
|
||||
| Mega | Yes | No | Yes | Yes | No | No | No | No [#2178](https://github.com/ncw/rclone/issues/2178) | Yes |
|
||||
|
|
|
@ -45,7 +45,7 @@ var (
|
|||
{
|
||||
Name: "TestCryptDrive:",
|
||||
SubDir: false,
|
||||
FastList: false,
|
||||
FastList: true,
|
||||
},
|
||||
{
|
||||
Name: "TestCryptSwift:",
|
||||
|
@ -55,7 +55,7 @@ var (
|
|||
{
|
||||
Name: "TestDrive:",
|
||||
SubDir: false,
|
||||
FastList: false,
|
||||
FastList: true,
|
||||
},
|
||||
{
|
||||
Name: "TestDropbox:",
|
||||
|
|
Loading…
Reference in a new issue