2014-03-28 22:34:13 +00:00
|
|
|
---
|
|
|
|
title: "Google drive"
|
|
|
|
description: "Rclone docs for Google drive"
|
2016-04-12 20:33:55 +00:00
|
|
|
date: "2016-04-12"
|
2014-03-28 22:34:13 +00:00
|
|
|
---
|
|
|
|
|
2014-07-17 19:03:11 +00:00
|
|
|
<i class="fa fa-google"></i> Google Drive
|
|
|
|
-----------------------------------------
|
|
|
|
|
2014-03-28 22:34:13 +00:00
|
|
|
Paths are specified as `drive:path`
|
|
|
|
|
2014-07-17 19:03:11 +00:00
|
|
|
Drive paths may be as deep as required, eg `drive:directory/subdirectory`.
|
2014-03-28 22:34:13 +00:00
|
|
|
|
|
|
|
The initial setup for drive involves getting a token from Google drive
|
|
|
|
which you need to do in your browser. `rclone config` walks you
|
|
|
|
through it.
|
|
|
|
|
|
|
|
Here is an example of how to make a remote called `remote`. First run:
|
|
|
|
|
|
|
|
rclone config
|
|
|
|
|
|
|
|
This will guide you through an interactive setup process:
|
|
|
|
|
|
|
|
```
|
2017-06-01 19:12:11 +00:00
|
|
|
No remotes found - make a new one
|
2014-03-28 22:34:13 +00:00
|
|
|
n) New remote
|
2017-06-01 19:12:11 +00:00
|
|
|
r) Rename remote
|
|
|
|
c) Copy remote
|
|
|
|
s) Set configuration password
|
2014-03-28 22:34:13 +00:00
|
|
|
q) Quit config
|
2017-06-01 19:12:11 +00:00
|
|
|
n/r/c/s/q> n
|
2014-03-28 22:34:13 +00:00
|
|
|
name> remote
|
2016-02-21 13:39:04 +00:00
|
|
|
Type of storage to configure.
|
|
|
|
Choose a number from below, or type in your own value
|
2016-07-11 11:42:44 +00:00
|
|
|
1 / Amazon Drive
|
2016-02-21 13:39:04 +00:00
|
|
|
\ "amazon cloud drive"
|
2017-01-09 05:09:19 +00:00
|
|
|
2 / Amazon S3 (also Dreamhost, Ceph, Minio)
|
2016-02-21 13:39:04 +00:00
|
|
|
\ "s3"
|
|
|
|
3 / Backblaze B2
|
|
|
|
\ "b2"
|
|
|
|
4 / Dropbox
|
|
|
|
\ "dropbox"
|
2017-01-09 05:09:19 +00:00
|
|
|
5 / Encrypt/Decrypt a remote
|
|
|
|
\ "crypt"
|
2017-06-01 19:12:11 +00:00
|
|
|
6 / FTP Connection
|
|
|
|
\ "ftp"
|
|
|
|
7 / Google Cloud Storage (this is not Google Drive)
|
2016-02-21 13:39:04 +00:00
|
|
|
\ "google cloud storage"
|
2017-06-01 19:12:11 +00:00
|
|
|
8 / Google Drive
|
2016-02-21 13:39:04 +00:00
|
|
|
\ "drive"
|
2017-06-01 19:12:11 +00:00
|
|
|
9 / Hubic
|
2016-02-21 13:39:04 +00:00
|
|
|
\ "hubic"
|
2017-06-01 19:12:11 +00:00
|
|
|
10 / Local Disk
|
2016-02-21 13:39:04 +00:00
|
|
|
\ "local"
|
2017-06-01 19:12:11 +00:00
|
|
|
11 / Microsoft OneDrive
|
2016-02-21 13:39:04 +00:00
|
|
|
\ "onedrive"
|
2017-06-01 19:12:11 +00:00
|
|
|
12 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
|
2016-02-21 13:39:04 +00:00
|
|
|
\ "swift"
|
2017-06-01 19:12:11 +00:00
|
|
|
13 / SSH/SFTP Connection
|
2017-03-05 10:14:57 +00:00
|
|
|
\ "sftp"
|
2017-06-01 19:12:11 +00:00
|
|
|
14 / Yandex Disk
|
2016-02-21 13:39:04 +00:00
|
|
|
\ "yandex"
|
2017-06-01 19:12:11 +00:00
|
|
|
Storage> 8
|
2015-10-03 13:23:12 +00:00
|
|
|
Google Application Client Id - leave blank normally.
|
2017-09-01 17:54:14 +00:00
|
|
|
client_id>
|
2015-10-03 13:23:12 +00:00
|
|
|
Google Application Client Secret - leave blank normally.
|
2017-09-01 17:54:14 +00:00
|
|
|
client_secret>
|
2017-11-29 21:34:19 +00:00
|
|
|
Service Account Credentials JSON file path - needed only if you want use SA instead of interactive login.
|
|
|
|
service_account_file>
|
2014-03-28 22:34:13 +00:00
|
|
|
Remote config
|
2015-09-12 13:17:39 +00:00
|
|
|
Use auto config?
|
|
|
|
* Say Y if not sure
|
2015-09-30 14:04:02 +00:00
|
|
|
* Say N if you are working on a remote or headless machine or Y didn't work
|
2015-09-12 13:17:39 +00:00
|
|
|
y) Yes
|
|
|
|
n) No
|
|
|
|
y/n> y
|
|
|
|
If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
|
|
|
|
Log in and authorize rclone for access
|
|
|
|
Waiting for code...
|
|
|
|
Got code
|
2017-06-01 19:12:11 +00:00
|
|
|
Configure this as a team drive?
|
|
|
|
y) Yes
|
|
|
|
n) No
|
|
|
|
y/n> n
|
2014-03-28 22:34:13 +00:00
|
|
|
--------------------
|
|
|
|
[remote]
|
2017-09-01 17:54:14 +00:00
|
|
|
client_id =
|
|
|
|
client_secret =
|
2014-03-28 22:34:13 +00:00
|
|
|
token = {"AccessToken":"xxxx.x.xxxxx_xxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","RefreshToken":"1/xxxxxxxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxx","Expiry":"2014-03-16T13:57:58.955387075Z","Extra":null}
|
|
|
|
--------------------
|
|
|
|
y) Yes this is OK
|
|
|
|
e) Edit this remote
|
|
|
|
d) Delete this remote
|
|
|
|
y/e/d> y
|
|
|
|
```
|
|
|
|
|
2015-09-12 13:17:39 +00:00
|
|
|
Note that rclone runs a webserver on your local machine to collect the
|
|
|
|
token as returned from Google if you use auto config mode. This only
|
|
|
|
runs from the moment it opens your browser to the moment you get back
|
|
|
|
the verification code. This is on `http://127.0.0.1:53682/` and this
|
|
|
|
it may require you to unblock it temporarily if you are running a host
|
|
|
|
firewall, or use manual mode.
|
|
|
|
|
2014-03-28 22:34:13 +00:00
|
|
|
You can then use it like this,
|
|
|
|
|
|
|
|
List directories in top level of your drive
|
|
|
|
|
|
|
|
rclone lsd remote:
|
|
|
|
|
|
|
|
List all the files in your drive
|
|
|
|
|
|
|
|
rclone ls remote:
|
|
|
|
|
|
|
|
To copy a local directory to a drive directory called backup
|
|
|
|
|
|
|
|
rclone copy /home/source remote:backup
|
|
|
|
|
2017-11-29 21:34:19 +00:00
|
|
|
### Service Account support ###
|
|
|
|
|
|
|
|
You can set up rclone with Google Drive in an unattended mode,
|
|
|
|
i.e. not tied to a specific end-user Google account. This is useful
|
|
|
|
when you want to synchronise files onto machines that don't have
|
|
|
|
actively logged-in users, for example build machines.
|
|
|
|
|
|
|
|
To create a service account and obtain its credentials, go to the
|
|
|
|
[Google Developer Console](https://console.developers.google.com) and
|
|
|
|
use the "Create Credentials" button. After creating an account, a JSON
|
|
|
|
file containing the Service Account's credentials will be downloaded
|
|
|
|
onto your machine. These credentials are what rclone will use for
|
|
|
|
authentication.
|
|
|
|
|
|
|
|
To use a Service Account instead of OAuth2 token flow, enter the path
|
|
|
|
to your Service Account credentials at the `service_account_file`
|
|
|
|
prompt and rclone won't use the browser based authentication
|
|
|
|
flow.
|
|
|
|
|
2017-06-01 19:12:11 +00:00
|
|
|
### Team drives ###
|
|
|
|
|
|
|
|
If you want to configure the remote to point to a Google Team Drive
|
|
|
|
then answer `y` to the question `Configure this as a team drive?`.
|
|
|
|
|
|
|
|
This will fetch the list of Team Drives from google and allow you to
|
|
|
|
configure which one you want to use. You can also type in a team
|
|
|
|
drive ID if you prefer.
|
|
|
|
|
|
|
|
For example:
|
|
|
|
|
|
|
|
```
|
|
|
|
Configure this as a team drive?
|
|
|
|
y) Yes
|
|
|
|
n) No
|
|
|
|
y/n> y
|
|
|
|
Fetching team drive list...
|
|
|
|
Choose a number from below, or type in your own value
|
|
|
|
1 / Rclone Test
|
|
|
|
\ "xxxxxxxxxxxxxxxxxxxx"
|
|
|
|
2 / Rclone Test 2
|
|
|
|
\ "yyyyyyyyyyyyyyyyyyyy"
|
|
|
|
3 / Rclone Test 3
|
|
|
|
\ "zzzzzzzzzzzzzzzzzzzz"
|
|
|
|
Enter a Team Drive ID> 1
|
|
|
|
--------------------
|
|
|
|
[remote]
|
2017-09-01 17:54:14 +00:00
|
|
|
client_id =
|
|
|
|
client_secret =
|
2017-06-01 19:12:11 +00:00
|
|
|
token = {"AccessToken":"xxxx.x.xxxxx_xxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","RefreshToken":"1/xxxxxxxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxx","Expiry":"2014-03-16T13:57:58.955387075Z","Extra":null}
|
|
|
|
team_drive = xxxxxxxxxxxxxxxxxxxx
|
|
|
|
--------------------
|
|
|
|
y) Yes this is OK
|
|
|
|
e) Edit this remote
|
|
|
|
d) Delete this remote
|
|
|
|
y/e/d> y
|
|
|
|
```
|
|
|
|
|
2015-06-06 09:05:21 +00:00
|
|
|
### Modified time ###
|
2014-03-28 22:34:13 +00:00
|
|
|
|
|
|
|
Google drive stores modification times accurate to 1 ms.
|
2015-05-10 11:17:24 +00:00
|
|
|
|
2015-06-06 09:05:21 +00:00
|
|
|
### Revisions ###
|
2015-05-10 11:17:24 +00:00
|
|
|
|
|
|
|
Google drive stores revisions of files. When you upload a change to
|
|
|
|
an existing file to google drive using rclone it will create a new
|
|
|
|
revision of that file.
|
|
|
|
|
|
|
|
Revisions follow the standard google policy which at time of writing
|
|
|
|
was
|
|
|
|
|
|
|
|
* They are deleted after 30 days or 100 revisions (whatever comes first).
|
|
|
|
* They do not count towards a user storage quota.
|
2015-05-21 17:39:46 +00:00
|
|
|
|
2015-08-16 13:49:58 +00:00
|
|
|
### Deleting files ###
|
|
|
|
|
2017-09-15 10:58:50 +00:00
|
|
|
By default rclone will send all files to the trash when deleting
|
|
|
|
files. If deleting them permanently is required then use the
|
|
|
|
`--drive-use-trash=false` flag, or set the equivalent environment
|
|
|
|
variable.
|
2015-08-16 13:49:58 +00:00
|
|
|
|
2017-09-01 17:54:14 +00:00
|
|
|
### Emptying trash ###
|
|
|
|
|
|
|
|
If you wish to empty your trash you can use the `rclone cleanup remote:`
|
|
|
|
command which will permanently delete all your trashed files. This command
|
|
|
|
does not take any path arguments.
|
|
|
|
|
2015-11-14 11:38:35 +00:00
|
|
|
### Specific options ###
|
|
|
|
|
|
|
|
Here are the command line options specific to this cloud storage
|
|
|
|
system.
|
|
|
|
|
2017-07-06 14:32:57 +00:00
|
|
|
#### --drive-auth-owner-only ####
|
|
|
|
|
|
|
|
Only consider files owned by the authenticated user.
|
|
|
|
|
2015-11-14 11:38:35 +00:00
|
|
|
#### --drive-chunk-size=SIZE ####
|
|
|
|
|
2016-04-12 20:33:55 +00:00
|
|
|
Upload chunk size. Must a power of 2 >= 256k. Default value is 8 MB.
|
|
|
|
|
|
|
|
Making this larger will improve performance, but note that each chunk
|
|
|
|
is buffered in memory one per transfer.
|
|
|
|
|
|
|
|
Reducing this will reduce memory usage but decrease performance.
|
2015-11-14 11:38:35 +00:00
|
|
|
|
2016-01-26 16:52:53 +00:00
|
|
|
#### --drive-formats ####
|
|
|
|
|
|
|
|
Google documents can only be exported from Google drive. When rclone
|
|
|
|
downloads a Google doc it chooses a format to download depending upon
|
|
|
|
this setting.
|
|
|
|
|
|
|
|
By default the formats are `docx,xlsx,pptx,svg` which are a sensible
|
|
|
|
default for an editable document.
|
|
|
|
|
|
|
|
When choosing a format, rclone runs down the list provided in order
|
|
|
|
and chooses the first file format the doc can be exported as from the
|
|
|
|
list. If the file can't be exported to a format on the formats list,
|
|
|
|
then rclone will choose a format from the default list.
|
|
|
|
|
|
|
|
If you prefer an archive copy then you might use `--drive-formats
|
|
|
|
pdf`, or if you prefer openoffice/libreoffice formats you might use
|
2016-11-26 11:05:40 +00:00
|
|
|
`--drive-formats ods,odt,odp`.
|
2016-01-26 16:52:53 +00:00
|
|
|
|
|
|
|
Note that rclone adds the extension to the google doc, so if it is
|
|
|
|
calles `My Spreadsheet` on google docs, it will be exported as `My
|
|
|
|
Spreadsheet.xlsx` or `My Spreadsheet.pdf` etc.
|
|
|
|
|
|
|
|
Here are the possible extensions with their corresponding mime types.
|
|
|
|
|
|
|
|
| Extension | Mime Type | Description |
|
|
|
|
| --------- |-----------| ------------|
|
|
|
|
| csv | text/csv | Standard CSV format for Spreadsheets |
|
|
|
|
| doc | application/msword | Micosoft Office Document |
|
|
|
|
| docx | application/vnd.openxmlformats-officedocument.wordprocessingml.document | Microsoft Office Document |
|
2016-09-19 17:08:10 +00:00
|
|
|
| epub | application/epub+zip | E-book format |
|
2016-01-26 16:52:53 +00:00
|
|
|
| html | text/html | An HTML Document |
|
|
|
|
| jpg | image/jpeg | A JPEG Image File |
|
2017-01-09 05:09:19 +00:00
|
|
|
| odp | application/vnd.oasis.opendocument.presentation | Openoffice Presentation |
|
2016-01-26 16:52:53 +00:00
|
|
|
| ods | application/vnd.oasis.opendocument.spreadsheet | Openoffice Spreadsheet |
|
|
|
|
| ods | application/x-vnd.oasis.opendocument.spreadsheet | Openoffice Spreadsheet |
|
|
|
|
| odt | application/vnd.oasis.opendocument.text | Openoffice Document |
|
|
|
|
| pdf | application/pdf | Adobe PDF Format |
|
|
|
|
| png | image/png | PNG Image Format|
|
|
|
|
| pptx | application/vnd.openxmlformats-officedocument.presentationml.presentation | Microsoft Office Powerpoint |
|
|
|
|
| rtf | application/rtf | Rich Text Format |
|
|
|
|
| svg | image/svg+xml | Scalable Vector Graphics Format |
|
2016-09-19 17:08:10 +00:00
|
|
|
| tsv | text/tab-separated-values | Standard TSV format for spreadsheets |
|
2016-01-26 16:52:53 +00:00
|
|
|
| txt | text/plain | Plain Text |
|
|
|
|
| xls | application/vnd.ms-excel | Microsoft Office Spreadsheet |
|
|
|
|
| xlsx | application/vnd.openxmlformats-officedocument.spreadsheetml.sheet | Microsoft Office Spreadsheet |
|
|
|
|
| zip | application/zip | A ZIP file of HTML, Images CSS |
|
|
|
|
|
2017-07-06 14:32:57 +00:00
|
|
|
#### --drive-list-chunk int ####
|
|
|
|
|
|
|
|
Size of listing chunk 100-1000. 0 to disable. (default 1000)
|
|
|
|
|
|
|
|
#### --drive-shared-with-me ####
|
|
|
|
|
|
|
|
Only show files that are shared with me
|
|
|
|
|
2017-01-29 17:06:26 +00:00
|
|
|
#### --drive-skip-gdocs ####
|
|
|
|
|
|
|
|
Skip google documents in all listings. If given, gdocs practically become invisible to rclone.
|
|
|
|
|
2017-07-06 14:32:57 +00:00
|
|
|
#### --drive-trashed-only ####
|
|
|
|
|
2017-07-19 13:35:58 +00:00
|
|
|
Only show files that are in the trash. This will show trashed files
|
|
|
|
in their original directory structure.
|
2017-07-06 14:32:57 +00:00
|
|
|
|
|
|
|
#### --drive-upload-cutoff=SIZE ####
|
|
|
|
|
|
|
|
File size cutoff for switching to chunked upload. Default is 8 MB.
|
|
|
|
|
|
|
|
#### --drive-use-trash ####
|
|
|
|
|
2017-09-15 10:58:50 +00:00
|
|
|
Controls whether files are sent to the trash or deleted
|
|
|
|
permanently. Defaults to true, namely sending files to the trash. Use
|
|
|
|
`--drive-use-trash=false` to delete files permanently instead.
|
2017-07-06 14:32:57 +00:00
|
|
|
|
2015-06-06 09:05:21 +00:00
|
|
|
### Limitations ###
|
2015-05-21 17:39:46 +00:00
|
|
|
|
|
|
|
Drive has quite a lot of rate limiting. This causes rclone to be
|
|
|
|
limited to transferring about 2 files per second only. Individual
|
|
|
|
files may be transferred much faster at 100s of MBytes/s but lots of
|
|
|
|
small files can take a long time.
|
2016-08-24 21:06:41 +00:00
|
|
|
|
2017-08-07 20:10:03 +00:00
|
|
|
Server side copies are also subject to a separate rate limit. If you
|
|
|
|
see User rate limit exceeded errors, wait at least 24 hours and retry.
|
|
|
|
You can disable server side copies with `--disable copy` to download
|
|
|
|
and upload the files if you prefer.
|
2017-07-24 10:06:10 +00:00
|
|
|
|
2017-07-09 22:23:53 +00:00
|
|
|
### Duplicated files ###
|
|
|
|
|
|
|
|
Sometimes, for no reason I've been able to track down, drive will
|
|
|
|
duplicate a file that rclone uploads. Drive unlike all the other
|
|
|
|
remotes can have duplicated files.
|
|
|
|
|
|
|
|
Duplicated files cause problems with the syncing and you will see
|
|
|
|
messages in the log about duplicates.
|
|
|
|
|
|
|
|
Use `rclone dedupe` to fix duplicated files.
|
|
|
|
|
|
|
|
Note that this isn't just a problem with rclone, even Google Photos on
|
|
|
|
Android duplicates files on drive sometimes.
|
|
|
|
|
|
|
|
### Rclone appears to be re-copying files it shouldn't ###
|
|
|
|
|
|
|
|
There are two possible reasons for rclone to recopy files which
|
|
|
|
haven't changed to Google Drive.
|
|
|
|
|
|
|
|
The first is the duplicated file issue above - run `rclone dedupe` and
|
|
|
|
check your logs for duplicate object or directory messages.
|
|
|
|
|
|
|
|
The second is that sometimes Google reports different sizes for the
|
|
|
|
Google Docs exports which will cause rclone to re-download Google Docs
|
|
|
|
for no apparent reason. `--ignore-size` is a not very satisfactory
|
|
|
|
work-around for this if it is causing you a lot of problems.
|
|
|
|
|
2017-07-10 22:06:06 +00:00
|
|
|
### Google docs downloads sometimes fail with "Failed to copy: read X bytes expecting Y" ###
|
|
|
|
|
|
|
|
This is the same problem as above. Google reports the google doc is
|
|
|
|
one size, but rclone downloads a different size. Work-around with the
|
|
|
|
`--ignore-size` flag or wait for rclone to retry the download which it
|
|
|
|
will.
|
|
|
|
|
2016-08-24 21:06:41 +00:00
|
|
|
### Making your own client_id ###
|
|
|
|
|
|
|
|
When you use rclone with Google drive in its default configuration you
|
|
|
|
are using rclone's client_id. This is shared between all the rclone
|
|
|
|
users. There is a global rate limit on the number of queries per
|
|
|
|
second that each client_id can do set by Google. rclone already has a
|
|
|
|
high quota and I will continue to make sure it is high enough by
|
|
|
|
contacting Google.
|
|
|
|
|
|
|
|
However you might find you get better performance making your own
|
|
|
|
client_id if you are a heavy user. Or you may not depending on exactly
|
|
|
|
how Google have been raising rclone's rate limit.
|
|
|
|
|
|
|
|
Here is how to create your own Google Drive client ID for rclone:
|
|
|
|
|
|
|
|
1. Log into the [Google API
|
|
|
|
Console](https://console.developers.google.com/) with your Google
|
|
|
|
account. It doesn't matter what Google account you use. (It need not
|
|
|
|
be the same account as the Google Drive you want to access)
|
|
|
|
|
|
|
|
2. Select a project or create a new project.
|
|
|
|
|
2017-10-16 02:46:20 +00:00
|
|
|
3. Under "ENABLE APIS AND SERVICES" search for "Drive", and enable the
|
|
|
|
then "Google Drive API".
|
2016-08-24 21:06:41 +00:00
|
|
|
|
2017-10-16 02:46:20 +00:00
|
|
|
4. Click "Credentials" in the left-side panel (not "Create
|
2016-08-24 21:06:41 +00:00
|
|
|
credentials", which opens the wizard), then "Create credentials", then
|
|
|
|
"OAuth client ID". It will prompt you to set the OAuth consent screen
|
|
|
|
product name, if you haven't set one already.
|
|
|
|
|
|
|
|
5. Choose an application type of "other", and click "Create". (the
|
|
|
|
default name is fine)
|
|
|
|
|
|
|
|
6. It will show you a client ID and client secret. Use these values
|
|
|
|
in rclone config to add a new remote or edit an existing remote.
|
|
|
|
|
|
|
|
(Thanks to @balazer on github for these instructions.)
|