TrueCloudLab/rclone
16
0
Fork 2
Code Issues 1 Pull requests 1 Projects Releases Packages Wiki Activity Actions 1
5197 commits 181 branches 114 tags 335 MiB
Commit graph

1466 commits

Author SHA1 Message Date
Martin Michlmayr
7a5b531bd0 docs: fix cosmetics issues 2020-05-26 13:49:09 +01:00
Martin Michlmayr
f204d95a8b docs: fix range 
This is a range so there should be two dots (as in the other ranges).
 2020-05-25 11:25:23 +01:00
Martin Michlmayr
7a5b814e59 docs: fix cosmetic issues 2020-05-25 11:25:23 +01:00
Martin Michlmayr
aa6c06d751 docs: use code elements more consistently 2020-05-25 11:25:23 +01:00
Martin Michlmayr
ffb031a71b docs: link to usage section of docs 
Since the page refers to the "Usage" section, let's actually link
to that specific section.
 2020-05-25 11:25:23 +01:00
Martin Michlmayr
041b201abd doc: fix typos throughout docs and code 2020-05-25 11:23:58 +01:00
Nick Craig-Wood
518d39815c sync,copy,move: add --check-first to do all checking before starting transfers 
See: https://forum.rclone.org/t/rclone-sync-doing-transfer-and-checking-in-paralel/16352/
 2020-05-22 17:50:07 +01:00
Nick Craig-Wood
147f97d1f7 sync: allow --max-backlog to be -ve meaning as large as possible 2020-05-22 17:50:07 +01:00
Nick Craig-Wood
7f44735709 Add Daniel Slyman to contributors 2020-05-22 17:50:07 +01:00
Martin Michlmayr
db9d94f5de doc: improve wording 2020-05-20 15:54:51 +01:00
Martin Michlmayr
a36ef8582f doc: use consistent capitalization 2020-05-20 15:54:51 +01:00
Martin Michlmayr
4aee962233 doc: fix typos throughout docs and code 2020-05-20 15:54:51 +01:00
Fred
5f71d186b2 seafile: implement 2FA 2020-05-20 15:46:35 +01:00
Daniel Slyman
56c9fdb53c
docs: updates filesize limitations for OneDrive Personal 2020-05-20 08:11:25 +01:00
Animosity022
47474687eb
Update drive.md 
Changing the and to or..
 2020-05-19 21:03:08 -04:00
Animosity022
abe753ca86
Update drive.md 
Just adding in for the different scenarios if a Google Suite user or a vanilla Google account.
 2020-05-19 21:02:11 -04:00
Nick Craig-Wood
ecdfd80459 Add Brandon McNama to contributors 2020-05-19 12:22:59 +01:00
Nick Craig-Wood
a268eedb1d Add Martin Michlmayr to contributors 2020-05-19 12:22:59 +01:00
Martin Michlmayr
fb169a8b54
doc: fix typos throughout docs 2020-05-19 12:02:44 +01:00
calisro
bcbfad1482
sft[: added --sftp-pem-key to support inline key files 2020-05-19 11:55:38 +01:00
Thibault Molleman
8c37ae8f5c
docs: drive: google console doesn't list 'other' anymore 
The option of 'other' seems to be gone from the

https://console.developers.google.com/apis/credentials/oauthclient

page. It only lists these now:

- Web application
- Android
- Chrome app
- iOS
- TVs and Limited Input devices
- Desktop app
- Universal Windows Platform (UWP)
 2020-05-19 11:52:23 +01:00
Nick Craig-Wood
610f40f700 local: implement --local-no-sparse flag for disabling sparse files #2469 
This also introduces a one time warning for sparse files and updates
the docs to warn about them.
 2020-05-19 10:16:43 +01:00
Brandon Philips
633f50cd3e
googlephotos: create feature/favorites directory - Fixes #4189 
Enable access “Favorite” images on Google Photos backend.

This adds a “feature/favorites” folder in the Google Photos backend
and uses the Feature Filter API:

https://developers.google.com/photos/library/reference/rest/v1/mediaItems/search#Filters
 2020-05-18 17:55:16 +01:00
calisro
d04d4edc40
Update ftp.md (#4241) 2020-05-16 15:28:15 -04:00
Nick Craig-Wood
c4bc249b66 Add Ben Zenker to contributors 2020-05-15 16:18:25 +01:00
Ben Zenker
899c8e0697 march: added flag to allow Unicode filenames to remain unique 
If your filenames contain two near-identical Unicode characters,
rclone will normalize these, making them identical. This flag
gives you the ability to keep them unique. This might
create unintended side effects, such as duplicating files that
contain certain Unicode characters, when downloading them from
certain cloud providers to a macOS filesystem.

Fixes #4228
 2020-05-15 12:28:01 +01:00
Nick Craig-Wood
4a1b644bfb azureblob: implement streaming of unknown sized files 
See: https://forum.rclone.org/t/rclone-rcat-azure-blob-container-sas-token-403-error/16286/3
 2020-05-14 11:56:15 +01:00
Nick Craig-Wood
e91b509578 fs: allow --min-age/--max-age to take a date as well as a duration 
Fixes #4211
 2020-05-12 17:49:33 +01:00
Nick Craig-Wood
32507774de Add Caleb Case to contributors 2020-05-12 17:01:26 +01:00
Caleb Case
0ce662faad Tardigrade Backend 2020-05-12 15:56:50 +00:00
albertony
962fbc8257 jottacloud: update docs regarding cleanup, removed remains from old auth, and added warning about special mountpoints. 2020-05-11 11:41:17 +02:00
Nick Craig-Wood
e103c4c26a Add Maxime Suret to contributors 2020-05-09 15:16:15 +01:00
Nick Craig-Wood
437f9e2cef Add Sébastien Gross to contributors 2020-05-07 11:25:09 +01:00
Nick Craig-Wood
b7c21310b4 Add Fred to contributors 2020-05-06 18:33:44 +01:00
Fred
c754e89906 seafile: New backend for seafile server 2020-05-06 17:33:22 +00:00
Nick Craig-Wood
afde340c9e gcs: fix --header-upload - #59 
Before this code we were settig the headers on the PUT request. However this isn't where GCS needs them.

After this fix we set the headers in the object upload request itself.

This means that we only support a limited range of headers

- Cache-Control
- Content-Disposition
- Content-Encoding
- Content-Language
- Content-Type
- X-Goog-Meta-

Note for the last of those are for setting custom metadata in the form
"X-Goog-Meta-Key: value".
 2020-05-06 17:34:23 +01:00
Nick Craig-Wood
d0ad83de4b Add ElonH to contributors 2020-05-06 17:34:23 +01:00
Nick Craig-Wood
dcf945ed58 docs: add bin/.ignored-emails for removing email addresses from authors.md 
Remove an email as requested for Anagh Kumar Baranwal
 2020-05-04 17:38:25 +01:00
Nick Craig-Wood
7f8d74e903 docs: refresh rclone authorize help 2020-05-04 10:40:26 +01:00
Nick Craig-Wood
f2b1fedc4f drive: follow shortcuts by default, skip with --drive-skip-shortcuts 
Before this change rclone would skip all shortcuts with a message

    Ignoring unknown document type "application/vnd.google-apps.shortcut"

After this message rclone resolves the shortcuts by default to the
actual files that they point to. See the docs for more info.

The --drive-skip-shortcuts flag can be used to skip shortcuts.
 2020-05-02 18:28:38 +01:00
Nick Craig-Wood
a97261c54f Add gitch1 to contributors 2020-05-01 13:02:32 +01:00
gitch1
0f5579c0ba docs : add --password-command Powershell Wiki link 
add Windows Powershell example wiki link to  --password-command doc
 2020-04-30 17:31:58 +01:00
Nick Craig-Wood
54edf38d0e Add Matan Rosenberg to contributors 2020-04-29 17:08:21 +01:00
Nick Craig-Wood
a3f0992a22 Add Kush to contributors 2020-04-29 16:09:45 +01:00
Nick Craig-Wood
e2916f3a55 local: implement backend command "noop" for testing purposes 2020-04-29 10:10:57 +01:00
Nick Craig-Wood
7238ae18f9 Add Adam Stroud to contributors 2020-04-28 17:47:10 +01:00
Adam Stroud
8e91f83174 googlecloudstorage: Add ARCHIVE storage class to help 2020-04-27 11:40:21 +01:00
Nick Craig-Wood
f6346a4d29 fs: add --header flag to add options to every HTTP transaction #59 2020-04-23 15:24:21 +01:00
Tim Gallant
b705ead3fd docs: adds --header-download and --header-upload 2020-04-23 11:07:21 +01:00
Tim Gallant
93caa459e3 fs/config: add header-download and header-upload flags 2020-04-23 11:07:21 +01:00
Nick Craig-Wood
8176202e6d Add Sunil Patra to contributors 2020-04-22 18:14:03 +01:00
Nick Craig-Wood
1b74879b8b Add David Bramwell to contributors 2020-04-22 18:14:03 +01:00
Sunil Patra
39319b4858 @Sunil-P 
box: Added support for interchangeable root folder for Box backend - #3422
 2020-04-22 17:00:13 +01:00
Sunil Patra
4af5c9aed7 pCloud: Added support for interchangeable root folder for pCloud backend - #3957 2020-04-22 16:58:01 +01:00
Nick Craig-Wood
1e4589db18 Add Martin Stone to contributors 2020-04-19 12:41:38 +01:00
Denis
31a1cc46b7
copyurl: add no-clobber flag - fixes #3950 2020-04-19 12:40:17 +01:00
Martin Stone
9d4b3580a5 docs: fix description of --quiet - fixes #4132 2020-04-16 18:12:17 +01:00
Nick Craig-Wood
b07bef2a6b Add Daven to contributors 2020-04-15 18:13:35 +01:00
Nick Craig-Wood
705e60d0ad Add Brandon Philips to contributors 2020-04-15 18:13:35 +01:00
Jon Fautley
53874bd8ee cmd: add --error-on-no-transfer option 
This allows rclone to exit with a non-zero return code if no files are
transferred. This is useful when calling rclone as part of a workflow/script
pipeline as it allows the end user to stop processing if no files have been
transferred.

NB: Enabling this option will return in rclone exiting non-zero if there are no
transfers. Depending on how your're currently using rclone in your scripts,
this may break existing setups!
 2020-04-15 17:06:40 +01:00
Nick Craig-Wood
2c4aadb588 Revert "install.sh: create ~/.config/rclone directory" 
This reverts commit d694bb30e5.

If it creates a config directory then it leaves it owned by root which
is very confusing for new users.
 2020-04-11 18:40:46 +01:00
buengese
2c2f4a6a05 jottacloud: implement --jottacloud-trashed-only 2020-04-11 16:42:25 +02:00
Nick Craig-Wood
9f3449d944 Add Michael G to contributors 2020-04-11 13:29:13 +01:00
Nick Craig-Wood
fc663d98d1 New email for Ankur Gupta in contributors 2020-04-03 11:51:08 +01:00
Ankur Gupta
08c2cb784f
filter: Added --files-from-raw flag 
--files-from parses input files by ignoring comments starting with # and ;
and stripping whitespace from start and end of strings.

The --files-from-raw flag was added that reads every line from the file ignoring
comment characters and not stripping whitespace while maintaining
backwards compatibility.

Fixes #3762
 2020-04-03 10:36:24 +01:00
Nick Craig-Wood
2a62471e4c Add Jack Anderson to contributors 2020-03-31 18:17:36 +01:00
Nick Craig-Wood
1af9fcbbfa Add Samantha McVey to contributors 2020-03-28 18:33:34 +00:00
Samantha McVey
6765303de4 docs: unmystify how crypt stores encryption password in config 
Without explaining exactly how this is generated, it can be confusing
and worrying to not know how the password that encrypts your data is
stored.

This also brings peace of mind to the user that even though
the same password is obscured differently each time, all the data to
get back to the original password remains. Explaining how it works
is much better than the reader of the documentation having to trust
a blackboxy/magical mechanism.
 2020-03-26 17:14:45 +00:00
Nick Craig-Wood
304ee97944 Add harry to contributors 2020-03-26 16:23:46 +00:00
Nick Craig-Wood
6c351c15f8 Add Mark Spieth to contributors 2020-03-23 17:12:56 +00:00
Nick Craig-Wood
b49ab9f9c7 Add Max Sum to contributors 2020-03-21 18:13:05 +00:00
Max Sum
78a9e7440a union: Implement multiple writable remotes 2020-03-21 18:11:24 +00:00
Nick Craig-Wood
410e17a2ec Add Elan Ruusamäe to contributors 2020-03-21 17:39:51 +00:00
Elan Ruusamäe
23f7943645 Dropbox: Add info about required redirect url 2020-03-20 14:51:53 +00:00
Nick Craig-Wood
1108895180 Add Bernd Schoolmann to contributors 2020-03-20 14:01:02 +00:00
Bernd Schoolmann
158870bcdb
fs: Add --track-renames-strategy for configurable matching criteria for --track-renames 
This commit adds the `--track-renames-strategy` flag which allows the
user to choose the strategy for tracking renames when using the
`--track-renames` flag.

This can be "hash" or "modtime" or both currently.

This, when used with `--track-renames-strategy modtime` enables
support for tracking renames in encrypted remotes.

Fixes #3696
Fixes #2721
 2020-03-20 13:04:56 +00:00
Fernando
1d3987bbbd
docs: Fix typo and reword contact.md 2020-03-18 14:21:58 +00:00
Nick Craig-Wood
472d4799d1 qingstor: make rclone cleanup remove pending multipart uploads older than 24h 2020-03-18 12:49:21 +00:00
Nick Craig-Wood
72eba4dbb6 Add greatroar to contributors 2020-03-18 12:49:21 +00:00
Nick Craig-Wood
1e9b8e043a Add fishbullet to contributors 2020-03-16 17:17:08 +00:00
fishbullet
eb0fc21533 fs: filter flags ability to read from stdin - fixes #4034 2020-03-16 17:16:50 +00:00
Nick Craig-Wood
c227a90b52 sync: implement --order-by xxx,mixed 2020-03-16 15:50:04 +00:00
Nick Craig-Wood
b03462ab04 Add Patryk Jakuszew to contributors 2020-03-13 21:45:09 +00:00
Nick Craig-Wood
a433698b00 Add Joachim Brandon LeBlanc to contributors 2020-03-10 12:00:30 +00:00
Nick Craig-Wood
747edf42c1 azureblob: document container level SAS URL from root now needs container 
In 8a0775ce3c which was released in v1.49.0 we inadvertently
stopped SAS URLs working from the root without a container name.

Previously to this change you could use `rclone mount azsas:` and it
would actually be equivalent to `rclone mount azsas:container`. After
this change, only `rclone mount azsas:container` will work, `rclone
mount azsas:` will have a directory in the root called "container".

After some discussion it was decided not to revert this change as the
current behaviour is more logical and in line with the similar
behaviour for the b2 backend.

Instead the documentation was updated to show exactly how container
level SAS URLs behave.

Fixes #4028
 2020-03-05 14:56:36 +00:00
Nick Craig-Wood
ce23cb2093 Add evileye to contributors 2020-03-05 14:07:32 +00:00
Nick Craig-Wood
1c800efbac Add Robert-André Mauchin to contributors 2020-03-03 12:41:08 +00:00
Nick Craig-Wood
452fdbf1c1 Add Franklyn Tackitt to contributors 2020-03-02 17:31:23 +00:00
Nick Craig-Wood
51686bd1ef Add Shing Kit Chan to contributors 2020-03-02 17:31:23 +00:00
Gary Kim
38a4d50e73 rcd: Add Prometheus metrics support - fixes #3858 
Signed-off-by: Gary Kim <gary@garykim.dev>
 2020-03-01 09:58:34 +00:00
Franklyn Tackitt
2b3d13a841 fs: Use --cutoff-mode hard,soft,catious instead of 3 --max-transfer-mode flags 
Fixes #2672
 2020-03-01 09:49:55 +00:00
Shing Kit Chan
6f1766dd9e fs: Add support for --max-transfer-cutoff modes #2672 
This also adds max transfer cut off check for server side copies too
 2020-03-01 09:49:55 +00:00
Nick Craig-Wood
bae2644667 Add Yves G to contributors 2020-03-01 09:17:51 +00:00
Valeriy.Vyrva
f6f95822c1 doc: fix links in generated documentation 2020-03-01 09:14:37 +00:00
Nick Craig-Wood
54a0c6b8ad Add valery1707 to contributors 2020-02-28 16:26:45 +00:00
valery1707
1ad23c4dc8
mailru: Describe 2FA requirements (#4015) 
Fair enough
 2020-02-28 16:54:09 +03:00
Nick Craig-Wood
d6e10dba33 docs: fix confusion over processor names in download table 
See: https://forum.rclone.org/t/intel-processor-download-help/14558
 2020-02-26 16:39:46 +00:00
Nick Craig-Wood
55b5eded23 Add Frederick Zhang to contributors 2020-02-12 11:33:56 +00:00
Frederick Zhang
b71e1a16b1 docs: update account type during OneDrive app registration 2020-02-12 11:04:49 +00:00
Nick Craig-Wood
dc31212c3d Add Tim Gallant to contributors 2020-02-11 14:28:13 +00:00
Nick Craig-Wood
a54210a2e4 docs: restore missing mount --daemon docs 
This was done as part of ebfeec9fb4 which unfortunately patched
the auto generated files.
 2020-02-10 15:29:39 +00:00
Nick Craig-Wood
040d226028 docs: restore missing QingStor doc fix 
This was done as part of 64fce8438b which unfortunately patched
the auto generated files.
 2020-02-10 15:29:39 +00:00
Nick Craig-Wood
8b664c3ec5 docs: restore lost spelling fixes 
These came from 3d424c6e08 which unfortunately got added the
docs to the auto generated files.
 2020-02-10 15:29:39 +00:00
Nick Craig-Wood
102a38bb95 docs: restore lost VFS poll interval docs 
These came from 3d475dc0ee which unfortunately got added the
docs to the auto generated files.
 2020-02-10 15:29:39 +00:00
Nick Craig-Wood
7a54e13110 docs: restore lost VFS case insensitive docs 
These came from 1c4e33d4ad which unfortunately
added the docs to the auto generated files.
 2020-02-10 15:29:39 +00:00
Nick Craig-Wood
feee92c790 docs: restore lost mount share docs 
These came from 162fdfe455 which unfortunately added the docs to
the auto generated files.
 2020-02-10 15:29:39 +00:00
Nick Craig-Wood
de93852512 docs: restore lost auth proxy logs 
These came from f2a789ea98 which unfortunately added the docs to
the auto generated files.
 2020-02-10 15:29:39 +00:00
Nick Craig-Wood
dfb710eab7 gendocs: add autogenerated header to all docs 2020-02-10 15:29:39 +00:00
Nick Craig-Wood
7c55fafe33 Add Durval Menezes to contributors 2020-02-01 13:41:47 +00:00
Nick Craig-Wood
2e7fe06beb Add Dave Koston to contributors 2020-02-01 13:41:47 +00:00
Durval Menezes
8ff91ff31b
docs: Update the "Making your own client_id" section for drive 
...so it accurately describes the new "Enhanced Security" Google process to get your own Client ID and Client Secret to use with rclone.
 2020-02-01 13:41:18 +00:00
Nick Craig-Wood
43daecd89b Version v1.51.0 2020-02-01 10:40:01 +00:00
Dave Koston
9f99c20232 s3: Add StackPath Object Storage Support 2020-01-31 16:05:44 +00:00
Nick Craig-Wood
b3e7a9d01c Add Benjapol Worakan to contributors 2020-01-29 13:38:21 +00:00
Benjapol Worakan
01fc063128 docs: fix spacing error under -P in docs.md 2020-01-29 13:38:03 +00:00
Nick Craig-Wood
27a34dd183 Add Motonori IWAMURO to contributors 2020-01-29 12:16:37 +00:00
Nick Craig-Wood
0e5537cd25 Add unbelauscht to contributors 2020-01-26 09:23:03 +00:00
unbelauscht
151d0a274e swift: Update OVH API endpoint - Fixes #3890 2020-01-24 17:43:00 +00:00
Nick Craig-Wood
dc77ec4ba1 Add boosh to contributors 2020-01-24 13:29:36 +00:00
boosh
0d7573dd81 fs: Add --max-duration flag to control the maximum duration of a transfer session 
This gives you more control over how long rclone will run for, making
it easier to script backups, e.g. via cron. Once the `--max-duration`
time limit is reached, no new transfers will be initiated, but those
already in-flight will be allowed to complete.

Fixes #985
 2020-01-24 13:28:56 +00:00
Nick Craig-Wood
ede36b001b Add Damon Permezel to contributors 2020-01-23 15:40:23 +00:00
Nick Craig-Wood
3afb2a4798 config: use SpaceSepList for argument to --password-command 
This is to enable arguments with spaces in.
 2020-01-23 15:39:15 +00:00
Nick Craig-Wood
62dbdcdbcc config: use the environment variable which goes with --password-command 2020-01-23 15:39:15 +00:00
Damon Permezel
06df133159 config: add --password-command to allow dynamic config password - fixes #3694 2020-01-23 15:39:15 +00:00
Xiaoxing Ye
0ab2693da6 doc: add desc about gzip and http dump 
Fix #3872
 2020-01-23 12:42:44 +00:00
Nick Craig-Wood
4b1cb1be43 Add jtagcat to contributors 2020-01-23 12:38:56 +00:00
Nick Craig-Wood
9d96680329 Add thestigma to contributors 2020-01-23 12:38:56 +00:00
jtagcat
d694bb30e5 install.sh: create ~/.config/rclone directory 
This allows people with their config already made copy their config to the new server(s), without having to create the directories.
 2020-01-23 12:37:46 +00:00
thestigma
7125cb10f5 doc: Add note to filtering.md about --files-from and rclone lsf 2020-01-20 17:37:24 +00:00
Nick Craig-Wood
ba421fd069 Add landall to contributors 2020-01-20 17:30:27 +00:00
Nick Craig-Wood
f338a2d907 Add Benjamin Richter to contributors 2020-01-20 12:46:46 +00:00
Xiaoxing Ye
ccaca04a5d rcd: move webgui apart; option to disable browser 
Fix #3601, #3785
 2020-01-20 12:27:55 +00:00
Nick Craig-Wood
c789436580 The memory backend 
This is a bucket based remote
 2020-01-18 10:41:08 +00:00
Nick Craig-Wood
277d94feac fshttp: add --expect-continue-timeout default 1s - fixes #3835 
Before this change the expect/continue timeout was set to
--conntimeout which was 60s by default which is too long to wait.

This was noticed when using s3 with a proxy which apparently didn't
support expect / continue properly.

Set --expect-continue-timeout 0 to disable expect/continue.
 2020-01-18 09:49:22 +00:00
Nick Craig-Wood
bedeaf23af sugarsync: new backend - fixes #622 2020-01-17 17:39:34 +00:00
Nick Craig-Wood
5697caf20b Add Felix Hungenberg to contributors 2020-01-17 15:00:01 +00:00
Felix Hungenberg
68056f08ab docs: Use correct link for sftp wikipedia article 2020-01-17 13:20:42 +00:00
Thomas Eales
3a1b41ac22 docs: fixed small typo 
is -> it
 2020-01-16 15:24:36 +00:00
Nick Craig-Wood
375d25f158 sync: implement --order-by flag to order transfers - fixes #1205 2020-01-16 15:24:36 +00:00
Nick Craig-Wood
0e57335396 docs: document new backend encoder parameter 2020-01-16 14:40:36 +00:00
Nick Craig-Wood
89634795b0 Add Paul Tinsley to contributors 2020-01-15 16:25:04 +00:00
Paul Tinsley
f2a789ea98 serve sftp: Add support for public key with auth proxy - fixes #3572 2020-01-15 16:23:57 +00:00
Nick Craig-Wood
ba01d5e8ab Add Thomas Eales to contributors 2020-01-12 14:23:57 +00:00
Nick Craig-Wood
e510d460c2 Add Kuang-che Wu to contributors 2020-01-12 14:23:57 +00:00
Nick Craig-Wood
87c844bce1 onedrive: clarify docs around making your own client ID 
See: https://forum.rclone.org/t/onedrive-token-configuration-failure/13634
 2020-01-11 16:30:13 +00:00
Nick Craig-Wood
6b3021209a Add Ole Schütt to contributors 2020-01-09 10:36:44 +00:00
Nick Craig-Wood
1af4bb0c84 Add Tennix to contributors 2020-01-05 19:50:00 +00:00
Tennix
15d19131bd s3: use aws web identity role provider 2020-01-05 19:49:31 +00:00
Nick Craig-Wood
2e0774f3cf Add Thomas Kriechbaumer to contributors 2020-01-03 22:18:23 +00:00
buengese
69ac04fec9 docs: add GetSky to list of supported providers 2020-01-02 15:37:33 +01:00
Thomas Kriechbaumer
584e705c0c s3: introduce list_chunk option for bucket listing 
The S3 ListObject API returns paginated bucket listings, with
"MaxKeys" items for each GET call.

The default value is 1000 entries, but for buckets with millions of
objects it might make sense to request more elements per request, if
the backend supports it. This commit adds a "list_chunk" option for
the user to specify a lower or higher value.

This commit does not add safe guards around this value - if a user
decides to request a too large list, it might result in connection
timeouts (on the server or client).

In AWS S3, there is a fixed limit of 1000, some other services might
have one too.  In Ceph, this can be configured in RadosGW.
 2020-01-02 12:15:01 +00:00
Nick Craig-Wood
32a3ba9e3f Add Outvi V to contributors 2020-01-02 11:52:43 +00:00
Outvi V
db1c7f9ca8 s3: Add new region Asia Patific (Hong Kong) 2020-01-02 11:10:48 +00:00
Nick Craig-Wood
207474abab sync: add --no-check-dest flag - fixes #3616 2019-12-29 16:47:57 +00:00
Nick Craig-Wood
f754d897e5 Add Wei He to contributors 2019-12-28 13:29:08 +00:00
Ivan Andreev
41ba1bba2b chunker: reduce length of temporary suffix 2019-12-09 16:56:32 +00:00
Nick Craig-Wood
e48145f959 Add David Cole to contributors 2019-12-04 12:14:30 +00:00
Nick Craig-Wood
2150cf7362 Add email for Aleksandar Janković 2019-12-04 12:14:21 +00:00
David Cole
707e51eac7 docs: correct typo in gui docs 2019-12-04 12:08:52 +00:00
Nick Craig-Wood
33c80bbb96 jottacloud: add URL to generate Login Token to config wizard 2019-11-28 10:03:48 +00:00
Nick Craig-Wood
ed39adc65b Add Fernando to contributors 2019-11-27 11:40:44 +00:00
Fernando
162fdfe455 mount: document remotes as network shares on Windows 
Provided instructions for mounting remotes as network shares/network drives in a Windows environment
 2019-11-27 11:40:24 +00:00
buengese
8f33c932f2 jottacloud: update docs for new auth method 2019-11-26 13:49:49 +00:00
Marco Molteni
d72f3e31c0 docs/install: explain how to workaround macOS Gatekeeper requiring notarization 
Fix #3689
 2019-11-26 12:33:30 +00:00
Nick Craig-Wood
4641bd5116 Add anuar45 to contributors 2019-11-21 11:16:04 +00:00
Nick Craig-Wood
e14d968f8d Start v1.50.2-DEV development 2019-11-19 16:51:32 +00:00
Nick Craig-Wood
1e6278556c Add Maciej Zimnoch to contributors 2019-11-18 16:28:19 +00:00
Nick Craig-Wood
303f4ee152 Add Ankur Gupta to contributors 2019-11-18 16:28:19 +00:00
Maciej Zimnoch
f5443ac939 accounting: clear finished transfer in stats-reset 
In order to reduce memory usage `stats-reset` also
clears finished transfers.

Fixes #3734
 2019-11-18 14:25:32 +00:00
Nick Craig-Wood
3da1cbfc81 Add Marco Molteni to contributors 2019-11-14 21:51:34 +00:00
Marco Molteni
0c9a8cf776 doc: add Scaleway to the S3 table of contents 
Hello, documentation for Scaleway was already there, but the TOC was missing it.
 2019-11-14 21:49:43 +00:00
Nick Craig-Wood
f3871377c3 Add Sebastian Brandt to contributors 2019-11-14 12:54:42 +00:00
Nick Craig-Wood
cc9a7dc073 Add Barry Muldrey to contributors 2019-11-14 12:54:42 +00:00
Nick Craig-Wood
b61dd809ee Add new email for Anagh Kumar Baranwal 2019-11-14 12:54:38 +00:00
Nick Craig-Wood
0eaf5475ef Start v1.50.1-DEV development 2019-11-02 15:26:01 +00:00
Xiaoxing Ye
520ddbcceb config: do not open browser on headless if google fs 
On google fs (drive, google photos, and google cloud storage), if
headless is selected, do not open browser.

This also supplies a new option "auth-no-open-browser" for authorize
if the user does not want it.

This should fix #3323.
 2019-10-30 14:12:42 +00:00
Nick Craig-Wood
7ff95c6250 Add Xiaoxing Ye to contributors 2019-10-30 12:20:10 +00:00
Nick Craig-Wood
199ac61bde rc: add methods to turn on blocking and mutex profiling 2019-10-28 22:11:38 +00:00
Nick Craig-Wood
a40cc1167d Add zero-24 to contributors 2019-10-28 16:49:33 +00:00
zero-24
c57ea8d867 docs: add instructions to create your own dropbox app ID 2019-10-28 16:49:16 +00:00
Nick Craig-Wood
1868c77e16 rc: fix formatting of docs 2019-10-27 10:43:40 +00:00
Nick Craig-Wood
6fabf476cf Version v1.50.0 2019-10-26 11:04:54 +01:00
Nick Craig-Wood
18c029e0f0 Add dausruddin to contributors 2019-10-21 22:28:44 +01:00
Nick Craig-Wood
22bf8589cd Add Saksham Khanna to contributors 2019-10-17 15:05:46 +01:00
Nick Craig-Wood
0871c57f1b Add Carlos Ferreyra to contributors 2019-10-17 15:05:46 +01:00
Nick Craig-Wood
90a23ae01b Add Bryce Larson to contributors 2019-10-16 10:27:07 +01:00
Bryce Larson
dd150efdd7 docs: fix --use-server-modtime spelling in docs 2019-10-15 19:54:42 +01:00
Nick Craig-Wood
7d3b67f6cc Add AlexandrBoltris to contributors 2019-10-15 19:46:44 +01:00
AlexandrBoltris
c526bdb579 docs: typo fix in faq.md 2019-10-14 17:07:29 +01:00
Chaitanya
b63e9befe8 rc docs: fix code section not rendering properly due to missing quotes 2019-10-13 12:26:37 +01:00
Ivan Andreev
910c80bd02 chunker: option to hash all files 2019-10-13 10:43:46 +01:00
Ivan Andreev
9049bb62ca chunker: prevent chunk corruption, survive meta-like input 2019-10-13 10:43:46 +01:00
Ivan Andreev
7aa2b4191c chunker: reservations for future extensions 2019-10-13 10:43:46 +01:00
Alex Chen
41ed33b08e docs: update onedrive/sharepoint docs on some known issues 2019-10-12 12:08:22 +01:00
Nick Craig-Wood
f3b0f8a9f0 sync: --update/-u not transfer files that haven't changed - fixes #3232 
Before this change --update would transfer any file which was newer
than the destination regardless of whether it had changed or not.
This is needlessly wasteful of bandwidth.

After this change --update will only transfer files if they are newer
**and** they are different (checked with checksum and size).
 2019-10-12 11:54:56 +01:00
Nick Craig-Wood
c892a6f8ef Add Michele Caci to contributors 2019-10-11 16:17:24 +01:00
Nick Craig-Wood
bc45f6f952 Add Arijit Biswas to contributors 2019-10-11 15:25:20 +01:00
Arijit Biswas
3d807ab449 docs: update onedrive docs on creating own client ID 
Updated the creating "own Client ID and Key" based on new portal (portal.azure.com).
 2019-10-11 15:24:54 +01:00
Nick Craig-Wood
a4d572d004 Add Vighnesh SK to contributors 2019-10-10 16:05:29 +01:00
Nick Craig-Wood
3f40849343 Add Brett Dutro to contributors 2019-10-09 10:07:50 +01:00
Nick Craig-Wood
7271a404db Add Tyler to contributors 2019-10-09 10:07:50 +01:00
Nick Craig-Wood
2a82aca225 Add Sezal Agrawal to contributors 2019-10-08 19:59:47 +01:00
Nick Craig-Wood
d210fecf3b Add Raphael to contributors 2019-10-05 17:07:02 +01:00
Nick Craig-Wood
f962fb9499 Add SwitchJS to contributors 2019-10-05 17:07:02 +01:00
Raphael
7f378ca8e3 documentation: add sharepoint required flags fixes #3564 
Enhance the WebDAV documentation with information regarding the flags that are required to make Rclone work correctly with SharePoint.
fixes #3564
 2019-10-05 17:06:44 +01:00
SwitchJS
9a5ea9c8a8 docs: fix spelling 2019-10-05 16:00:39 +01:00
Nick Craig-Wood
d15425e8c8 Start v1.49.5-DEV development 2019-10-05 12:42:28 +01:00
Nick Craig-Wood
5271fe3b3f yandex: use lib/encoder 2019-10-05 10:22:43 +01:00
Nick Craig-Wood
cbdab14057 Add 庄天翼 to contributors 2019-10-04 16:53:51 +01:00
Nick Craig-Wood
135717e12b mailru: use lib/encoder 2019-10-02 20:10:01 +01:00
Aleksandar Jankovic
6b55b8b133 s3: add option for multipart failiure behaviour 
This is needed for resuming uploads across different sessions.
 2019-10-02 16:49:16 +01:00
Nick Craig-Wood
fd51f24906 putio: use lib/encoder 
And in the process
- fix a bug with + and & in file name
- fix NewObject returning directories as files
 2019-10-02 11:34:08 +01:00
Nick Craig-Wood
4615343b73 premiumizeme: use lib/encoder 2019-10-02 11:34:08 +01:00
Fabian Möller
b9bd15a8c9 koofr: use lib/encoder 
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
 2019-09-30 22:00:25 +01:00
Nick Craig-Wood
8d8fad724b ficher: use lib/encoder 2019-09-30 22:00:25 +01:00
Nick Craig-Wood
d122d1d191 qingstor: use lib/encoder 2019-09-30 22:00:25 +01:00
Nick Craig-Wood
35d6ff89bf azureblob: use lib/encoder 2019-09-30 22:00:24 +01:00
Nick Craig-Wood
53bec33027 swift: use lib/encoder 2019-09-30 22:00:24 +01:00
Fabian Möller
3304bb7a56 googlecloudstorage: use lib/encoder 
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
 2019-09-30 22:00:24 +01:00
Fabian Möller
7689bd7e21 amazonclouddrive: use lib/encoder 
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
 2019-09-30 22:00:24 +01:00
Fabian Möller
33f129fbbc s3: use lib/encoder 
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
 2019-09-30 22:00:24 +01:00
Nick Craig-Wood
32af4cd6f3 ftp: use lib/encoder 2019-09-30 22:00:24 +01:00
Fabian Möller
00b2c02bf4 pcloud: use lib/encoder 
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
 2019-09-30 22:00:24 +01:00
Fabian Möller
33aea5d43f mega: use lib/encoder 
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
 2019-09-30 22:00:24 +01:00
Fabian Möller
13d8b7979d b2: use lib/encoder 
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
 2019-09-30 22:00:24 +01:00
Fabian Möller
c8d3e57418 encodings: add . and .. to all backends, except Drive 2019-09-30 14:05:49 +01:00
Fabian Möller
4b9fdb8475 opendrive: use lib/encoder 2019-09-30 14:05:49 +01:00
Fabian Möller
dac20093c5 onedrive: use lib/encoder 2019-09-30 14:05:49 +01:00
Fabian Möller
d211347d46 dropbox: use lib/encoder 2019-09-30 14:05:49 +01:00
Fabian Möller
4837bc3546 jottacloud: use lib/encoder 2019-09-30 14:05:49 +01:00
Fabian Möller
69c51325bb drive: use lib/encoder 2019-09-30 14:05:49 +01:00
Fabian Möller
05e4f10436 box: use lib/encoder 2019-09-30 14:05:49 +01:00
Fabian Möller
a98a750fc9 local: use lib/encoder 2019-09-30 14:05:49 +01:00
Fabian Möller
a56c9ab61d docs: add section for restricted filenames 2019-09-30 14:05:48 +01:00
Nick Craig-Wood
4627ac5709 New backend for Citrix Sharefile - Fixes #1543 
Many thanks to Bob Droog for organizing a test account and extensive
testing.
 2019-09-30 12:28:33 +01:00
Nick Craig-Wood
74f6300875 Start v1.49.4-DEV development 2019-09-29 19:47:59 +01:00
Nick Craig-Wood
25a0703b45 Add Richard Patel to contributors 2019-09-29 11:14:33 +01:00
Ivan Andreev
ccecfa9cb1 chunker: finish meta-format before release 
changes:
- chunker: remove GetTier and SetTier
- remove wdmrcompat metaformat
- remove fastopen strategy
- make hash_type option non-advanced
- adverise hash support when possible
- add metadata field "ver", run strict checks
- describe internal behavior in comments
- improve documentation

note:
wdmrcompat used to write file name in the metadata, so maximum metadata
size was 1K; removing it allows to cap size by 200 bytes now.
 2019-09-25 11:03:33 +01:00
Ivan Andreev
c68c919cea docs: chunker documentation 2019-09-24 12:45:12 +01:00
Nick Craig-Wood
28cc2009d4 Add Anthony Rusdi to contributors 2019-09-21 14:39:03 +01:00
Nick Craig-Wood
dd4fe9ff60 Add David to contributors 2019-09-21 14:39:03 +01:00
David
4788545b05 box: add options to get access token via JWT auth 2019-09-20 17:15:16 +01:00
Matei David
70e043e641 Fixed typo in Docker image doc. 2019-09-19 16:17:57 +01:00
Matei David
ab89e93968 Add Matei David to contributors 2019-09-17 10:12:32 +01:00
Matei David
070a8bfcd8 Dockerfile fixes 
- ref: https://forum.rclone.org/t/run-docker-container-in-userspace/11734/7
- enable userspace operation
- enable Docker userspace mount exposed to the host
- add more Docker image usage documentation
 2019-09-17 10:12:32 +01:00
Nick Craig-Wood
b8b12a4000 Start v1.49.3-DEV development 2019-09-15 18:10:08 +01:00
Nick Craig-Wood
0baafb158f mount: allow files of unkown size to be read properly 
Before this change, files of unknown size (eg Google Docs) would
appear in file listings with 0 size and would only allow 0 bytes to be
read.

This change sets the direct_io flag in the FUSE return which bypasses
the cache for these files.  This means that they can be read properly.

This is compatible with some, but not all applications.
 2019-09-14 13:22:33 +01:00
Nick Craig-Wood
7b29ed8ec1 Add Lars Lehtonen to contributors 2019-09-13 23:50:54 +01:00
Nick Craig-Wood
3b91fb6a2f Add David Baumgold to contributors 2019-09-13 18:39:36 +01:00
David Baumgold
7d8c15c030 docs: GitHub has a capital H 2019-09-13 18:39:23 +01:00
Ivan Andreev
ba1daea072 mailru: backend for mail.ru 2019-09-09 21:56:16 +01:00
Nick Craig-Wood
94eb9a4014 Start v1.49.2-DEV development 2019-09-08 17:36:21 +01:00
calisro
2298834e83 docs: spell out google photos download limitations 2019-09-07 11:47:53 +01:00
Nick Craig-Wood
27a730ef8f docs: Update changelog and RELEASE.md from v1.49.1 release 2019-09-06 12:07:09 +01:00
Nick Craig-Wood
7b0966880e Add Ivan Andreev to contributors 2019-09-04 21:32:16 +01:00
Ivan Andreev
1c4e33d4ad vfs: add flag --vfs-case-insensitive for windows/macOS mounts 
rclone mount when run on Windows & macOS will now default to `--vfs-case-insensitive`.
This means that
 2019-09-04 21:30:48 +01:00
Nick Craig-Wood
f97a3e853e Add Alfonso Montero to contributors 2019-09-03 17:26:13 +01:00
Denis
b71ac141cc copyurl: add --auto-filename flag for using file name from url in destination path (#3451) 2019-09-03 17:25:19 +01:00
Nick Craig-Wood
5932acfee3 rc: fix docs for config/create /update /password 2019-09-03 08:34:15 +01:00
Nick Craig-Wood
a3fb460c6b docs: add info on how to build and use the docker images 2019-09-02 14:30:11 +01:00
Cnly
e2b5ed6c7a docs: fix template argument for mktemp in install.sh 2019-09-02 13:01:45 +08:00
Nick Craig-Wood
a6387e1f81 Version v1.49.0 2019-08-26 15:25:20 +01:00
Nick Craig-Wood
73e010aff9 docs: make the config walkthroughs consistent for each backend 2019-08-26 10:47:17 +01:00
Nick Craig-Wood
a3faf98aa0 docs: add docs about GUI 2019-08-25 20:32:41 +01:00
Nick Craig-Wood
93810a739d docs: update fontawesome free to 5.10.2 and fixup broken images 2019-08-25 11:09:19 +01:00
Nick Craig-Wood
5d4d5d2b07 docs: update logo on website 2019-08-25 11:09:19 +01:00
Nick Craig-Wood
f02fc5d5b5 Add Andreas Chlupka to contributors 2019-08-25 11:09:19 +01:00
Nick Craig-Wood
f67798d73e Add Cenk Alti to contributors 2019-08-23 12:11:51 +01:00
Cenk Alti
a1ca65bd80 putio: add new backend 2019-08-23 12:11:36 +01:00
Nick Craig-Wood
82c6c77e07 Add Patrick Wang to contributors 2019-08-20 17:46:13 +01:00
yparitcher
ba0e1ea6ae Docs: add emptydir support to table 2019-08-17 09:45:20 +01:00
buengese
4e43fa746a jottacloud: update config docs 2019-08-15 11:46:56 +00:00
Nick Craig-Wood
eb087a3b04 operations: disable multi thread copy for local to local copies #3419 
...unless --multi-thread-streams has been set explicitly
 2019-08-14 15:07:38 +01:00
Nick Craig-Wood
106cf1852d Add ginvine to contributors 2019-08-14 13:40:15 +01:00
Nick Craig-Wood
50b8f15b5d Add another email for Laura Hausmann to contributors 2019-08-14 13:40:07 +01:00
Nick Craig-Wood
8247c8a6af rc: add anchor tags to the docs so links are consistent 2019-08-13 11:57:01 +01:00
Chaitanya
bbb8d43716 rc: (docs) Add new parameters --rc-web-gui and --rc-allow-origin, --rc-web-fetch-url and rc-web-gui-update to documentation. 2019-08-12 19:04:12 +01:00
Nick Craig-Wood
a1263e70cf premiumizeme: new backend for premiumize.me - Fixes #3063 2019-08-10 19:17:51 +01:00
Nick Craig-Wood
f47e5220a2 Add Abhinav Sharma to contributors 2019-08-10 17:31:25 +01:00
Nick Craig-Wood
99eaa76dc8 Add Macavirus to contributors 2019-08-10 14:13:24 +01:00
Macavirus
ba3b0a175e docs: Add rsync.net stub link to SFTP page 2019-08-10 14:13:15 +01:00
Macavirus
01c0c0b009 docs: Add C14 Cold Storage to homepage and SFTP backend 2019-08-10 14:13:15 +01:00
Nick Craig-Wood
5195075677 Add Michał Matczuk to contributors 2019-08-08 23:42:03 +01:00
Nick Craig-Wood
78150e82a2 docs: update bugs and limitations document 2019-08-04 12:33:39 +01:00
Nick Craig-Wood
6f0cc51eeb Add Chaitanya Bankanhal to contributors 2019-08-04 12:33:39 +01:00
Aleksandar Jankovic
6a3e301303 accounting: add call to clear stats 
- Make calls more consistent by changing path to kebab case.
- Add stacktrace information to job panics
 2019-08-02 16:56:19 +01:00
Nick Craig-Wood
f1a687c540 Add justina777 to contributors 2019-08-02 15:57:25 +01:00
Nick Craig-Wood
57d5de6fba build: fix up package paths after repo move 
git grep -l github.com/ncw/rclone | xargs -d'\n' perl -i~ -lpe 's|github.com/ncw/rclone|github.com/rclone/rclone|g'
goimports -w `find . -name \*.go`
 2019-07-28 18:47:38 +01:00
Aleksandar Jankovic
ff235e4e56 docs: update documentation for stats 2019-07-28 14:48:19 +01:00
Nick Craig-Wood
2d561b51db Add EliEron to contributors 2019-07-28 12:33:21 +01:00
Nick Craig-Wood
9241a93c2d Add justinalin to contributors 2019-07-28 12:33:21 +01:00
EliEron
fb32f77bac Docs: Fix typos in filtering demonstrations 2019-07-28 12:32:53 +01:00
justinalin
520fb03bfd log: add --use-json-log for JSON logging 2019-07-28 12:05:50 +01:00
yparitcher
ccc416e62b b2: Fix link sharing #3314 2019-07-28 11:47:31 +01:00
Nick Craig-Wood
9af0a704af Add Paul Millar to contributors 2019-07-24 20:34:39 +01:00
Nick Craig-Wood
691e5ae5f0 Add Yi FU to contributors 2019-07-24 20:34:39 +01:00
yparitcher
8e8b78d7e5 Implement --compare-dest & --copy-dest Fixes #3278 2019-07-22 19:42:29 +01:00
Paul Millar
e4f6ccbff2 webdav: add docs for using bearer_token_command with oidc-agent 2019-07-22 16:01:55 +01:00
Yi FU
0a1169e659 ssh: opt-in support for diffie-hellman-group-exchange-sha256 diffie-hellman-group-exchange-sha1 - fixes #1810 2019-07-13 12:21:56 +02:00
Nick Craig-Wood
3cd63a00be googlephotos: fix configuration walkthrough 2019-07-04 15:19:59 +01:00