Commit graph

6245 commits

Author SHA1 Message Date
albertony
0e77072dcc vfs: error strings should not be capitalized 2022-05-16 12:43:43 +02:00
albertony
2437eb3cce vfs: fix incorrect detection of root in parent directory utility function
When using filepath.Dir, a difference to path.Dir is that it returns os PathSeparator
instead of slash when the path consists entirely of separators.

Also fixed casing of the function name, use OS in all caps instead of Os
as recommended here: https://github.com/golang/go/wiki/CodeReviewComments#initialisms
2022-05-16 12:43:43 +02:00
Michael C Tiernan - MIT-Research Computing Project
a12c94caff
docs: replaced extended ascii smart quotes (#6171) 2022-05-16 12:34:54 +02:00
Kaspian
542c1616b8
docs: small style fixes 2022-05-13 23:06:45 +01:00
albertony
8697f0bd26 about: improved error message 2022-05-13 12:08:10 +01:00
albertony
a9f18f8093 Set proper exit code for errors that are not low-level retried (e.g. size/timestamp changing)
Fixes #5785
2022-05-13 12:02:55 +01:00
Derek Battams
8e5e230b81 b2: use chunksize lib to determine chunksize dynamically
Fixes #4643
2022-05-13 09:25:48 +01:00
Derek Battams
c0985e93b7 azureblob: use chunksize lib to determine chunksize dynamically 2022-05-13 09:25:48 +01:00
Derek Battams
fb4f7555c7 s3: use chunksize lib to determine chunksize dynamically 2022-05-13 09:25:48 +01:00
Derek Battams
f2e7a2e794 chunksize: initial implementation of chunksize helper lib 2022-05-13 09:25:48 +01:00
Erik van Velzen
9e4854955c storj: fix put
The "relative" argument was missing when Put'ing a file. This
sets an incorrect object entry in the cache, leading to the file being
unreadable when using mount functionality.

Fixes #6151
2022-05-12 20:43:54 +01:00
Vincent Murphy
319ac225e4
s3: backend restore command to skip non-GLACIER objects 2022-05-12 20:42:37 +01:00
albertony
a9d3283d97 jottacloud: fix listing output of remote with special characters
This fixes the failing integration test: TestIntegration/FsMkdir/FsPutFiles/FsIsFile
2022-05-12 20:41:07 +01:00
Nick Craig-Wood
edf0412464 test_all: correct path for Internet Archive test
This is to make it use the rclone project's account and to remove the
/ in the remote name which was crashing the integration tester!
2022-05-12 20:38:05 +01:00
Nick Craig-Wood
e6194a4b83 Add Hugal31 to contributors 2022-05-12 20:38:05 +01:00
Nick Craig-Wood
7f05990623 Add Werner to contributors 2022-05-12 20:38:05 +01:00
Nick Craig-Wood
e16f2a566f Add Kaspian to contributors 2022-05-12 20:38:05 +01:00
Hugal31
a36fef8a66
rclone.mount: ignore _netdev mount argument - FIxes #5808
Do not trigger an error upon parsing argument starting with underscores.

_netdev was already ignored after parsing.
2022-05-12 20:27:13 +01:00
Werner
6500e1d205
docs: bisync: minor grammar fix 2022-05-12 20:24:52 +01:00
Kaspian
9f7484e4e9
docs: faq: small grammar fix 2022-05-12 20:23:48 +01:00
Michael C Tiernan - MIT-Research Computing Project
0ba702ccf4
install: Pre verify sudo authorization "-v" before calling curl.
The way this was, the curl interferes with the bash invocation. Here it will authenticate first.
2022-05-12 20:19:33 +01:00
Nick Craig-Wood
6f91198b57 s3: Support Cloudflare R2 - fixes #5642 2022-05-12 08:49:20 +01:00
Nick Craig-Wood
cf0a72aecd fs: fix FixRangeOption to do nothing on unknown sized objects
FixRangeOption shouldn't be called on an object of unknown size, but
if it is, make sure it does nothing.

See: #5642
2022-05-12 08:49:20 +01:00
Nick Craig-Wood
f6fd6ee777 fs: fix FixRangeOption make SeekOptions into absolute RangeOptions
Cloudflare R2 doesn't support range options like `Range: bytes=21-`.

This patch makes FixRangeOption turn a SeekOption into an absolute
RangeOption like this `Range: bytes=21-25` to interoperate with R2.

See: #5642
2022-05-12 08:49:20 +01:00
Nick Craig-Wood
1e66d052fd fs: fix FixRangeOption to make fetch to end Range options absolute
Before this change FixRangeOption was leaving `Range: bytes=21-`
alone, thus not fulfilling its contract of making Range requests
absolute.

As it happens this form isn't supported by Cloudflare R2.

After this change the request is normalised to `Range: bytes=21-25`.

See: #5642
2022-05-12 08:49:20 +01:00
Nick Craig-Wood
e5974ac4b0 s3: use PutObject from the aws SDK to upload single part objects
Before this change rclone used presigned requests to upload single
part objects. This was because of a limitation in the SDK which didn't
allow non seekable io.Readers to be passed in.

This is incompatible with some S3 backends, and rclone wasn't adding
the `X-Amz-Content-Sha256: UNSIGNED-PAYLOAD` header which was
incompatible with other S3 backends.

The SDK now allows for this so rclone can use PutObject directly.

This sets the `X-Amz-Content-Sha256: UNSIGNED-PAYLOAD` flag on the PUT
request. However rclone will add a `Content-Md5` header if at all
possible so the body data is still protected.

Note that the old behaviour can still be configured if required with
the `use_presigned_request` config parameter.

Fixes #5422
2022-05-12 08:49:20 +01:00
Nick Craig-Wood
50a0c3482d lib/readers: add FakeSeeker to adapt io.Reader to io.ReadSeeker #5422 2022-05-12 08:49:20 +01:00
Nick Craig-Wood
389a29b017 Add Michael C Tiernan - MIT-Research Computing Project to contributors 2022-05-12 08:49:20 +01:00
Nick Craig-Wood
9dcf9375e8 Add Mateusz Puczyński to contributors 2022-05-12 08:49:20 +01:00
Lesmiscore
1d6d41fb91
backend/internetarchive: fix uploads can take very long time
* fill in empty values for non-wait mode
* add tracking metadata to observe file change
* completely remove getHashes
* remove unreliable update tests

Closes #6150
2022-05-07 21:54:14 +01:00
Michael C Tiernan - MIT-Research Computing Project
a3d4307892
install: set the modes on the files and/or directories on macOS
Changes made for macOS specific for that style of system.
Paths are established/defined singularly and modes are set automatically
when created. (Platform specific.)
2022-05-05 17:50:22 +01:00
ehsantdy
a446106041
s3: update Arvancloud default values and correct docs 2022-05-02 16:04:01 +01:00
Mateusz Puczyński
607172b6ec go: run go mod tidy and set version to minimum supported go - go1.16
This fixes `go mod tidy` needing manual intervention to build with
previous go versions.
2022-05-01 13:28:20 +01:00
Nick Craig-Wood
94757277bc build: fix go mod tidy removing golang.org/x/mobile dependency
Before this running `go mod tidy` caused the build to break because it
removed the dependency on golang.org/x/mobile and a command line tool
from this package is needed for the build.

This adds an explicit dependency which will mean the tool is always present.
2022-05-01 13:28:20 +01:00
Nick Craig-Wood
deab86867c build: support mount on windows/arm64 - all windows binaries now not cgo
This builds all windows binaries without CGO but with cmount.

cgofuse has a compile mode which works without CGO on Windows for
amd64/x86/arm64 architectures so switch to using that.
2022-04-29 18:04:21 +01:00
Nick Craig-Wood
c0c5b3bc6b build: add rclone version step 2022-04-29 18:04:21 +01:00
Nick Craig-Wood
a947f298e6 cmount: remove cgo from windows build requirements since it builds without 2022-04-29 18:04:21 +01:00
Nick Craig-Wood
1b0128ecb2 cmount: update winfsp/cgofuse dependency for windows/arm64 build 2022-04-29 18:04:21 +01:00
Nick Craig-Wood
c5395db1f1 Changelog updates from Version v1.58.1 2022-04-29 13:11:23 +01:00
Nick Craig-Wood
6e5382fc99 Add SwazRGB to contributors 2022-04-29 13:11:23 +01:00
Nick Craig-Wood
134592adaa Add ehsantdy to contributors 2022-04-29 13:11:23 +01:00
Nick Craig-Wood
36e614f550 Add Lesmiscore to contributors 2022-04-29 13:11:23 +01:00
Nick Craig-Wood
7bfed98b48 Add Zsolt Ero to contributors 2022-04-29 13:11:23 +01:00
Nick Craig-Wood
f471096fd0 Add Leroy van Logchem to contributors 2022-04-29 13:11:23 +01:00
SwazRGB
4cebade95d b2: Add b2-version-at flag to show file versions at time
Uses b2_list_file_versions to retrieve all file versions, and returns
the one that was active at the specified time

This is especially useful in combination with other backup tools, such
as restic, which may use rclone as a backend.
2022-04-28 16:29:13 +01:00
SwazRGB
a8cd18faf3 fs: Implement fs.Time
Similar to fs.Duration but parses into a timestamp instead

Supports parsing from:

* Any of the date formats in parseTimeDates
* A time.Duration offset from now
* parseDurationSuffixes offset from now
2022-04-28 16:29:13 +01:00
ehsantdy
e34c543660
s3: Add ArvanCloud AOS to provider list 2022-04-28 10:42:30 +01:00
Lesmiscore
598364ad0f backend/internetarchive: add support for Internet Archive
This adds support for Internet Archive (archive.org) Items.
2022-04-28 10:25:38 +01:00
Zsolt Ero
211dbe9aee
docs: Add --multi-thread-streams note to --transfers. 2022-04-27 18:40:39 +01:00
albertony
4829527dac jottacloud: refactor timestamp handling
Jottacloud have several different apis and endpoints using a mix of different
timestamp formats. In existing code the list operation (after the recent liststream
implementation) uses format from golang's time.RFC3339 constant. Uploads (using the
allocate api) uses format from a hard coded constant with value identical to golang's
time.RFC3339. And then we have the classic JFS time format, which is similar to RFC3339
but not identical, using a different constant. Also the naming is a bit confusing,
since the term api is used both as a generic term and also as a reference to the
newer format used in the api subdomain where the allocate endpoint is located.
This commit refactors these things a bit.
2022-04-27 13:41:39 +02:00