Commit graph

163 commits

Author SHA1 Message Date
Nick Craig-Wood
525433e6dd build: fix "Illegal instruction" error for ARMv6 builds - fixes #4553
Before this change we used `go build -i` to build the releases in parallel.

However this causes the ARMv6 and ARMv7 build to get mixed up somehow,
causing an illegal instruction when running rclone binaries on ARMv6.

See go bug: https://github.com/golang/go/issues/41223

This removes the -i which should have no effect on build times on the
CI and appears to fix the problem.
2020-09-04 16:30:50 +01:00
Nick Craig-Wood
f71f6c57d7 build: explicitly set ARM version to fix build #4553 2020-09-04 16:30:50 +01:00
Nick Craig-Wood
94a0991584 vfs: set the modtime of the cache file immediately
Before this change we set the modtime of the cache file when all
writers had finished.

This has the unfortunate effect that the file is uploaded with the
wrong modtime which means on backends which can't set modtimes except
when uploading files it is wrong.

This change sets the modtime of the cache file immediately in the
cache and in turn sets the modtime in the file info.
2020-08-20 16:24:04 +01:00
Nick Craig-Wood
d2ad293fae vfs: fix rename tests by waiting for writes to complete
Before this change the background writing of the file was racing with
the test of the object on the remote.

This meant that the tests passed locally but failed on a lot of the
remotes.
2020-08-19 17:04:17 +01:00
Nick Craig-Wood
3a14b1d5a9 build: make rclone build with wasm
Needed to drop
- azureblob backend
- cache backend
- qingstor backend
- cachestats command
- ncdu command
2020-08-10 17:32:21 +01:00
Nick Craig-Wood
01280798e9 build: drop macOS 386 build as it is no longer supported by go1.15
The go team made the decision to drop support for 32 bit macOS as 32
bit apps are no longer supported by macOS and 32 bit hardware hasn't
been produced by Apple for over 10 years.
2020-08-09 12:59:17 +01:00
Nick Craig-Wood
177d2f2f79 build: add script for torturing the VFS 2020-08-06 17:26:18 +01:00
Morten Linderud
67098511db make_manual: Support SOURCE_DATE_EPOCH
The documentation contains an embedded datetime which do not read
SOURCE_DATE_EPOCH. This makes the documentation unreproducible when
distribution tries to recreate previous build of the package.

This patch ensures we attempt to read the environment variable before
default to the current build time.

Signed-off-by: Morten Linderud <morten@linderud.pw>
2020-07-15 13:23:08 +01:00
Nick Craig-Wood
e2201689cf build: add ARMv7 to the supported builds - fixes #748 2020-07-07 12:13:20 +01:00
Garrett Squire
4f7f5404ce
build: fix file handle leak in GitHub release tool
This is a small patch to remove a defer statement found in a for loop.
It instead closes the file after it is done copying the bytes from the
tar file reader.
2020-07-04 10:51:37 +01:00
Nick Craig-Wood
4bb103ef43 build: speed up tidy-beta script by doing fewer directory traversals 2020-06-24 10:01:24 +01:00
Nick Craig-Wood
62a1a561cf build: test-repeat.sh add -tag to buildflags 2020-06-20 14:52:04 +01:00
Nick Craig-Wood
d8eea0e397 build: run gofmt -s to simplify the code: suggested by Go Report Card 2020-06-18 18:45:39 +01:00
Nick Craig-Wood
33c8709439 build: fix windows/amd64 build and icon/version embedding #4304
The parameters were being passed to goversioninfo in the wrong order
so that the 64 bit .syso was actually a 32 bit .syso thus calling the
linker to fail.
2020-06-18 16:08:38 +01:00
albertony
6a2b7b97d7
build: Add file properties and icon to Windows executable (fixes #4304) 2020-06-11 09:26:14 +01:00
Nick Craig-Wood
5866b1b017 bin: test-repeat.sh - script to run tests many times with individual logs 2020-06-10 17:08:31 +01:00
Nick Craig-Wood
8493f3939c bin: not-in-stable.go - script to help with merging fixes to the stable branch 2020-06-10 17:07:43 +01:00
Nick Craig-Wood
6882aeff97 Version v1.52.0 2020-05-27 17:31:10 +01:00
Nick Craig-Wood
a0922643e6 build: fix build after nfpm change to drop bindir 2020-05-27 17:31:09 +01:00
Nick Craig-Wood
74b8cbfb84 docs: set unsafe HTML parsing to false and fix raw HTML insertion
This means that markdown files can't contain <thing> any more.
2020-05-27 17:31:09 +01:00
Nick Craig-Wood
80dab10ec9 docs: Fix generated docs
In distributed docs
- Make img tags absolute
- Add donate link in docs
- Fix provider table
2020-05-27 17:31:08 +01:00
Nick Craig-Wood
038648aaca docs: rename about.md to _index.md after Hugo upgrade
Use the official way of including markdown content into index.html
2020-05-27 17:31:08 +01:00
Martin Michlmayr
041b201abd doc: fix typos throughout docs and code 2020-05-25 11:23:58 +01:00
Nick Craig-Wood
d685e7b4b5 bin: add check-merged.go to find local branches which might have been merged 2020-05-13 12:42:45 +01:00
Caleb Case
0ce662faad Tardigrade Backend 2020-05-12 15:56:50 +00:00
Fred
c754e89906 seafile: New backend for seafile server 2020-05-06 17:33:22 +00:00
Nick Craig-Wood
dcf945ed58 docs: add bin/.ignored-emails for removing email addresses from authors.md
Remove an email as requested for Anagh Kumar Baranwal
2020-05-04 17:38:25 +01:00
Nick Craig-Wood
ba7f7c8319 build: add -trimpath to release build for reproduceable builds 2020-04-30 12:43:40 +01:00
Nick Craig-Wood
1aa1a2c174 backend: add new backend command for backend specific commands
These commands are for implementing backend specific
functionality. They have documentation which is placed automatically
into the backend doc.

There is a simple test for the feature in the backend tests.
2020-04-29 10:10:57 +01:00
Valeriy.Vyrva
f6f95822c1 doc: fix links in generated documentation 2020-03-01 09:14:37 +00:00
Nick Craig-Wood
c789436580 The memory backend
This is a bucket based remote
2020-01-18 10:41:08 +00:00
Nick Craig-Wood
bedeaf23af sugarsync: new backend - fixes #622 2020-01-17 17:39:34 +00:00
Nick Craig-Wood
51dca8c8d4 bin: update windows test paths for new setup 2020-01-09 10:55:18 +00:00
Nick Craig-Wood
f73dae1e77 bin/get-github-release: support tar.bz2 files 2019-10-01 16:23:36 +01:00
Nick Craig-Wood
4627ac5709 New backend for Citrix Sharefile - Fixes #1543
Many thanks to Bob Droog for organizing a test account and extensive
testing.
2019-09-30 12:28:33 +01:00
Ivan Andreev
c68c919cea docs: chunker documentation 2019-09-24 12:45:12 +01:00
Ivan Andreev
ba1daea072 mailru: backend for mail.ru 2019-09-09 21:56:16 +01:00
Nick Craig-Wood
07dfb3aa11 bin: convert python scripts to python3 2019-09-06 22:55:28 +01:00
Nick Craig-Wood
a6387e1f81 Version v1.49.0 2019-08-26 15:25:20 +01:00
Nick Craig-Wood
a3faf98aa0 docs: add docs about GUI 2019-08-25 20:32:41 +01:00
Cenk Alti
a1ca65bd80 putio: add new backend 2019-08-23 12:11:36 +01:00
Nick Craig-Wood
a1263e70cf premiumizeme: new backend for premiumize.me - Fixes #3063 2019-08-10 19:17:51 +01:00
Nick Craig-Wood
e6ab237fcd serve: add auth proxy infrastructure 2019-08-06 11:43:42 +01:00
Nick Craig-Wood
5ef3c988eb bin: add script to test all commits compile for git bisect 2019-08-04 13:29:59 +01:00
Nick Craig-Wood
57d5de6fba build: fix up package paths after repo move
git grep -l github.com/ncw/rclone | xargs -d'\n' perl -i~ -lpe 's|github.com/ncw/rclone|github.com/rclone/rclone|g'
goimports -w `find . -name \*.go`
2019-07-28 18:47:38 +01:00
Nick Craig-Wood
a1cfe61ffd googlephotos: Backend for accessing Google Photos #369 2019-07-02 15:26:55 +01:00
Laura
dde4dd0198 fichier: 1fichier support - fixes #2908
This was started by Fionera, finished off by Laura with fixes and more
docs from Nick.

Co-authored-by: Fionera <fionera@fionera.de>
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
2019-06-30 18:35:01 +01:00
Nick Craig-Wood
4ee6de5c3e docs: add a new page with global flags and link to it from the command docs
In f544234 we removed the global flags from each command as it was
making each page very big and causing 1000s of lines of duplication in
the man page.

This change adds a new flags page with all the global flags on and
links each command page to it.

Fixes #3273
2019-06-20 16:45:44 +01:00
Gary Kim
44f6491731 bin: update make_changelog.py to support semver 2019-06-13 14:51:42 +01:00
Nick Craig-Wood
bb80586473 bin/get-github-release: fetch the most recent not the least recent 2019-03-18 11:29:37 +00:00
jaKa
118a8b949e koofr: implemented a backend for Koofr cloud storage service.
Implemented a Koofr REST API backend.
Added said backend to tests.
Added documentation for said backend.
2019-03-06 13:41:43 +00:00
Nick Craig-Wood
0a56a168ff bin/get-github-release.go: scrape the downloads page to avoid the API limit
This should fix pull requests build failures which can't use the
github token.
2019-02-25 21:34:59 +00:00
Nick Craig-Wood
71d1890316 build: ignore testbuilds when uploading to github 2019-02-09 12:22:06 +00:00
Nick Craig-Wood
fffdbb31f5 bin/get-github-release.go: Use GOPATH/bin by preference to place binary 2019-02-02 11:45:07 +00:00
Nick Craig-Wood
cacefb9a82 bin/get-github-release.go: automatically choose the right os/arch
This fixes the install of golangci-lint on non Linux platforms
2019-02-02 11:45:07 +00:00
Nick Craig-Wood
98120bb864 bin/get-github-release.go: enable extraction of binary not in root of tar
Also fix project name regexp to allow -
2019-02-02 11:34:51 +00:00
Nick Craig-Wood
0c4ed35b9b build: improve beta tidy script 2019-02-01 16:40:55 +00:00
Nick Craig-Wood
8e107b9657 build: update the build container to use latest go version for circleci 2019-01-18 13:26:27 +00:00
Nick Craig-Wood
31e2ce03c3 fstests: re-arrange backend integration tests so they can be retried
Before this change backend integration tests depended on each other,
so tests could not be retried.

After this change we nest tests to ensure that tests are provided with
the starting state they expect.

Tell the integration test runner that it can retry backend tests also.

This also includes bin/test_independence.go which runs each test
individually for a backend to prove that they are independent.
2018-11-22 20:12:12 +00:00
Scott Edlund
f139e07380 enable softfloat on MIPS arch
MIPS does not have a floating point unit.  Enable softfloat to build binaries run on devices that do not have MIPS_FPU enabled in their kernel.
2018-11-07 14:26:48 +00:00
Anagh Kumar Baranwal
6531126eb2 Fixes the rc docs creation
Signed-off-by: Anagh Kumar Baranwal <anaghk.dos@gmail.com>
2018-10-25 11:29:59 +01:00
Anagh Kumar Baranwal
a3c55462a8 Set python version explicitly to 2 to avoid issues on systems where
the default python version is `3`

Signed-off-by: Anagh Kumar Baranwal <anaghk.dos@gmail.com>
2018-10-23 12:14:52 +01:00
Nick Craig-Wood
f96ce5674b Version v1.44 2018-10-15 11:03:08 +01:00
Nick Craig-Wood
6b1f915ebc fs: Implement RegInfo.FileName to return the on disk filename for a backend
Use it in make_backend_docs.py
2018-10-06 11:47:46 +01:00
Nick Craig-Wood
78b9bd77f5 docs: auto generate backend options documentation
This inserts the output of "rclone help backend xxx" into the help
pages for each backend.
2018-10-06 11:47:46 +01:00
Nick Craig-Wood
e44fa5db8e build: update git bisect scripts 2018-09-19 16:04:57 +01:00
Nick Craig-Wood
b3d8bc61ac build: add example scripts for bisecting rclone and go 2018-09-14 11:17:51 +01:00
Nick Craig-Wood
9fe39f25e1 union: add missing docs 2018-09-07 15:14:08 +01:00
Nick Craig-Wood
b0dd218fea build: make tidy-beta for removing old beta releases 2018-09-01 18:21:54 +01:00
Nick Craig-Wood
8e2b3268be build: Automatically compile the changelog to make editing easier 2018-09-01 12:50:35 +01:00
Sebastian Bünger
007c7757d4 Add docs for Jottacloud 2018-08-10 11:14:34 +01:00
Nick Craig-Wood
97beff5370 build: keep track of compile failures better in cross-compile 2018-07-02 10:09:18 +01:00
Jakub Karlicek
53292527bb opendrive: fill out the functionality #1026
* Add Mkdir, Rmdir, Purge, Delete, SetModTime, Copy, Move, DirMove
 * Update file size after upload
 * Add Open seek
 * Set private permission for new folder and uploaded file
 * Add docs
 * Update List function
 * Fix UserSessionInfo struct
 * Fix socket leaks
 * Don’t close resp.Body in Open method
 * Get hash when listing files
2018-05-12 10:07:25 +01:00
Nick Craig-Wood
5eecbd83ee bin: make make_test_files.go work properly on Windows 2018-05-09 16:59:29 +01:00
Nick Craig-Wood
cb5bd47e61 build: fix errors spotted by ineffassign linter
These were mostly caused by shadowing err and a good fraction of them
will have caused errors not to be propagated properly.
2018-05-05 17:32:41 +01:00
Nick Craig-Wood
9ab2521ef2 rc: autogenerate and tidy the docs and commands
* Rename rc/pid -> core/pid
  * Sort the output of `rc list`
  * Make a script to autogenerate the docs
  * Tidy docs
2018-04-23 20:57:17 +01:00
Nick Craig-Wood
1f255a8567 Add a mega.nz remote #163
Not supported yet:
  * Hash
  * ModTime
  * Server Side Copy

Otherwise fully functional and passing all the tests.
2018-04-18 21:09:54 +01:00
Nick Craig-Wood
86e5a35491 Implement Remote Control for rclone #2111
This implements a remote control protocol activated with the --rc flag
and a new command `rclone rc` to use that interface.

Still to do
  * docs - need finishing
  * tests
2018-03-14 22:58:20 +02:00
Nick Craig-Wood
ee3c45676f bin/get-github-release.go: use GITHUB_USER/GITHUB_TOKEN when available
This should help with rate limiting problems when running under
travis.
2018-03-07 10:18:09 +00:00
Nick Craig-Wood
2e7e15461b bin/get-github-release.go: report body of HTTP responses with errors 2018-03-07 10:18:06 +00:00
Nick Craig-Wood
85e0b87c99 build: add .deb and .rpm output for the build
This uses https://github.com/goreleaser/nfpm to create the .deb and
.rpm packages from the standard build output.
2018-03-06 12:37:44 +00:00
Nick Craig-Wood
d41017a277 A script to download and install the latest release of a github package 2018-03-06 12:37:44 +00:00
Fabian Möller
b183bd7f00 alias: add new backend to create aliases for remote names #1049
The alias backend is a wrapper for an existing remote.
It allows you to name a "remote:path" as an "alias:".
2018-02-06 18:23:47 +00:00
Fabian Möller
1f5e23aedb scripts: make absolute paths consistent
Change absolute binary paths in scripts to /usr/bin/env or make them
relative.
This allows the scripts to be used on linux distributions
like NixOS, where binaries are not located in /usr/ or /bin/.
2018-01-26 09:39:05 +00:00
Nick Craig-Wood
e00616b016 Write version.txt on building into root of downloads 2017-12-07 21:49:32 +00:00
Nick Craig-Wood
085c690798 build: add in 64bit path for WinFSP headers 2017-11-20 12:46:24 +00:00
Remus Bunduc
016abf825e cache: first version 2017-11-15 15:23:21 +00:00
Nick Craig-Wood
7f3dc9b5c4 Implement WebDAV remote #580
This has special knowledge of Owncloud and Nextcloud to enable more
functionality such as mod times.
2017-10-09 16:19:37 +01:00
Nick Craig-Wood
9e9297838f Implement pcloud remote - #418 2017-10-01 11:37:35 +01:00
Nick Craig-Wood
3e9861eecf docs: improve links to cloud providers 2017-09-30 09:28:49 +01:00
Nick Craig-Wood
ee6a35d750 Test compilation of all arches
* Add compile_all step to Makefile
  * Add this to travis
  * Add -compile-only flag to cross-compile.go to save time making the zips
2017-08-04 23:20:26 +01:00
Nick Craig-Wood
92d2e1f8d7 azureblob: rework and complete #801
* Fixup bitrot (rclone and Azure library)
  * Implement Copy
  * Add modtime to metadata under mtime key as RFC3339Nano
  * Make multipart upload work
  * Make it pass the integration tests
  * Fix uploading of zero length blobs
  * Rename to azureblob as it seems likely we will do azurefile
  * Add docs
2017-08-04 22:56:16 +01:00
wuyu
ec5b72f8d5 Add new QingStor remote
Add new package qingstor to support QingStor API.

Add new unit test for its and tested through; But I commented
on some tests case because of some of the features of QingStor.

Add new docs for it.
2017-08-04 17:25:47 +01:00
Nick Craig-Wood
7b81f12dad box: add docs
* reorder remotes so they are in alphabetical order by full name everywhere
  * update CONTRIBUTING doc
2017-07-23 11:32:34 +01:00
Nick Craig-Wood
27b157580e Move make_test_files.go into bin 2017-07-06 11:54:57 +01:00
Nick Craig-Wood
3f288bc9ea Added decrypt_names.py to help decoding encrypted logs 2017-07-06 11:53:39 +01:00
Nick Craig-Wood
b22c4c4307 http: fix, tidy and rework ready for release
* Fix remaining problems
  * Refactor to make testing easier and add a test suite
  * Make path parsing more robust.
  * Add single file operations
  * Add MimeType reading for objects
  * Add documentation
  * Note go1.7+ is required to build
2017-06-20 21:27:14 +01:00
Nick Craig-Wood
bbbc202ee6 Add ftp.md to docs builder and update docs 2017-06-15 20:12:26 +01:00
Nick Craig-Wood
8f54dc06a2 Use build tags to control when and where cmount is built 2017-05-19 17:08:04 +01:00
Nick Craig-Wood
df0cfa9735 Add -no-clean flag to cross-compile.go 2017-05-15 17:41:16 +01:00