Commit graph

7949 commits

Author SHA1 Message Date
Dan McArdle
ad605ee356 cmd/gitannex: Drop chdir from e2e tests
Now that e2e tests are running in parallel, undoing the chdir to the
temp dir was causing flaky failures on cleanup. We don't need it anyway
because the worrisome subcommands have their working directory
controlled by `runInRepo()`.

Issue #7625
2024-05-13 18:44:31 +01:00
Dan McArdle
4ab235c06c cmd/gitannex: Repeat TestEndToEnd for all layout modes
I'm hopeful that running these in parallel will not impact CI runtime
very much, but that likely depends on the number of CPU cores and
whether the tmp filesystem is backed by memory vs a physical disk.

Issue #7625
2024-05-13 18:44:31 +01:00
Dan McArdle
9a2b85d71c cmd/gitannex: Refactor e2e tests, add layout compat tests
TestEndToEndRepoLayoutCompat exercises git-annex-remote-rclone-builtin
and git-annex-remote-rclone on the same rclone remote to ensure they are
compatible. It repeats the same test for all known layout modes.

Issue #7625
2024-05-13 18:44:31 +01:00
Dan McArdle
29b58dd4c5 cmd/gitannex: Add support for different layouts
This commit adds support for the same repo layouts supported by
git-annex-remote-rclone. This should enable git-annex users with remotes
of type "rclone" to switch to a "rclone-builtin" without needing to
retransfer content.

Issue #7625
2024-05-13 18:44:31 +01:00
Dan McArdle
36ad4eb145 cmd/gitannex: Simplify messageParser's finalParameter() func
Issue #7625
2024-05-13 18:44:31 +01:00
nielash
61ab519791 chunker: fix finalizer already set error
Before this change, cache.PinUntilFinalized was called twice if the root pointed
to a composite multi-chunk file without metadata, resulting in a fatal "finalizer
already set" error. This change fixes the issue.
2024-05-13 18:33:54 +01:00
nielash
678941afc1 mailru: use --tpslimit 10 on bisync tests
see https://github.com/rclone/rclone/issues/7768#issuecomment-2060888980
2024-05-13 18:33:11 +01:00
nielash
b153254b3a bisync: ignore "Implicitly create directory" messages on tests 2024-05-13 18:32:55 +01:00
nielash
17cd7a9496 quatrix: fix f.String() not including subpath 2024-05-13 18:32:41 +01:00
Nick Craig-Wood
0735f44f91 operations: fix lsjson --encrypted when using --crypt-XXX parameters
Before this change an `rclone lsjson --encrypted` command where
additional `--crypt-` parameters were supplied on the command line:

    rclone lsjson --crypt-description XXX --encrypted secret:

Produced an error like this:

    Failed to lsjson: ListJSON failed to load config for crypt remote: config name contains invalid characters...

This was due to an incorrect lookup of the crypt config to create the
encrypted mapping.

Fixes #7833
2024-05-13 17:59:58 +01:00
Nick Craig-Wood
04c69959b8 Add Sunny to contributors 2024-05-13 17:59:58 +01:00
Nick Craig-Wood
25cc8c927a Add Michael Terry to contributors 2024-05-13 17:59:58 +01:00
Sunny
6356b51b33
serve http: added content-length header when html directory is served 2024-05-13 17:24:54 +01:00
albertony
1890608f55 docs: minor formatting improvement 2024-05-13 12:50:22 +02:00
Michael Terry
cd76fd9219 oauthutil: clear client secret if client ID is set
When an external OAuth flow is being used (i.e. a client ID and an
OAuth token are set in the config), a client secret should not be set.
If one is, the server may reject a token refresh attempt.

But there's no way to clear out a backend's default client secret via
configuration, since empty-string config values are ignored.

So instead, when a client ID is set, we should clear out any default
client secret, since it wouldn't apply anyway.
2024-05-11 16:03:32 +01:00
Nick Craig-Wood
5b8cdaff39 drive: fix description being overwritten on server side moves
Before this the description for files got overwritten on server side
moves.

This change stops rclone setting the description to the leaf name
completely.

See: https://forum.rclone.org/t/file-descriptions-in-google-drive-not-shown-after-dedupe/45510
Fixes: #7770
2024-05-11 15:59:40 +01:00
albertony
f2f559230c bump golangci/golangci-lint-action from 4 to 6
Version 5 removed go cache management, and therefore also options skip-pkg-cache and
skip-build-cache, because the cache related to go itself is already handled by
actions/setup-go, and now it only caches golangci-lint analysis. Since we run multiple
golangci-lint-action steps for different goos, we want to cache package and build cache
and golangci-lint results from all of them, and therefore this commit now changes the
approach by disabling all built-in caching and introducing a separate cache step to
handle it properly.
2024-05-11 14:43:29 +02:00
nielash
e0b38cc9ac onedrive: add support for group permissions
This change adds support for "group" identities, and SharePoint variants
"siteUser" and "siteGroup". It also adds support for using any identity type
(including "application" and "device") as a recipient source when adding
permissions.
2024-05-10 16:25:08 +01:00
nielash
68dc79eddd onedrive: fix references to deprecated permissions properties
Before this change, metadata permissions used the `grantedTo` and
`grantedToIdentities` properties, which are deprecated on OneDrive Business in
favor of `grantedToV2` and `grantedToIdentitiesV2`. After this change, OneDrive
Business uses the new V2 versions, while OneDrive Personal still uses the
originals, as the V2 versions are not available for OneDrive Personal. (see
https://learn.microsoft.com/en-us/answers/questions/1079737/inconsistency-between-grantedtov2-and-grantedto-re)
2024-05-10 16:25:08 +01:00
nielash
76cea0c704 onedrive: skip writing permissions with 'owner' role
The 'owner' role is an implicit role that can't be removed, so don't try to.
2024-05-10 16:25:08 +01:00
Nick Craig-Wood
41d5d8b88a build: add issue label notification machinery 2024-05-10 16:15:28 +01:00
Nick Craig-Wood
aa2746d0de union: fix deleting dirs when all remotes can't have empty dirs 2024-05-10 11:15:12 +01:00
wiserain
b2f6aac754
pikpak: improve getFile() usage
Previously, `getFile()` was called indiscriminately during uploads, moves, 
and download link generation. This could lead to users with download limit 
causing subsequent operations like uploads and moves to fail. 
This PR optimizes the use of getFile(), by only calling it 
when it's strictly necessary.
2024-05-08 09:09:56 +09:00
Eric Wolf
a0dacf4930
docs: exit code 9 requires --error-on-no-transfer
Updated exit code 9 definition to include that it requires the use of the "--error-on-no-transfer" flag with a link to that section.
2024-05-07 09:18:05 +02:00
IoT Maestro
c5ff5afc21 ulozto: Fix handling of root paths with leading / trailing slashes.
This fixes #7796
2024-05-04 20:04:30 +01:00
Nick Craig-Wood
bd8523f208 fstest: reduce precision of directory time checks on CI
For unknown reasons the precision of modification times of directories
on the CI is > 15mS compared to files which are 100nS. The tests
work fine when run in Virtualbox though so I conjecture this is
something to do with the file system used there.
2024-05-03 12:29:18 +01:00
nielash
0bfd70c405 sync: remove now superfluous copyEmptyDirectories function 2024-05-03 12:29:18 +01:00
Nick Craig-Wood
47735d8fe1 sync: fix failed to update directory timestamp or metadata: directory not found
See: https://forum.rclone.org/t/empty-dirs-not-wanted/45059/14
Co-authored-by: nielash <nielronash@gmail.com>
2024-05-03 12:29:18 +01:00
Nick Craig-Wood
617534112b sync: fix directory modification times not being set
Co-authored-by: nielash <nielronash@gmail.com>
2024-05-03 12:29:18 +01:00
Nick Craig-Wood
271ec43189 sync: don't need to sync directories if they haven't been modified
Before this change we synced directories regardless if the source
directory existed. It is irrelevant whether the source directory
exists or not, what we need to know is has the directory been
modified.

Co-authored-by: nielash <nielronash@gmail.com>
2024-05-03 12:29:18 +01:00
Nick Craig-Wood
10eb4742dd sync: fix creation of empty directories when --create-empty-src-dirs=false
In v1.66.0 the changes to enable metadata preservation on directories
introduced a regression, namely that empty directories were created
despite the state of the --create-empty-src-dirs flag.

This patch fixes the problem by letting the normal rclone directory
creation create the directories and fixing up their timestamps and
metadata afterwards if --create-empty-src-dirs=false.

Fixes #7689
See: https://forum.rclone.org/t/empty-dirs-not-wanted/45059/
See: https://forum.rclone.org/t/how-to-ignore-empty-directories-when-uploading-from-windows/45057/
2024-05-03 12:29:18 +01:00
Nick Craig-Wood
2a2ec06ec1 sync: fix management of empty directories to make it more accurate
Before this change we used the same datastructure for managing empty
directories for both --create-empty-src-dirs in sync/copy/move and for
the --delete-empty-src-dirs flag in move.

These two uses are subtly incompatible and this change uses a separate
datastructure for both uses. This makes it more accurate and easier to
understand.
2024-05-03 12:29:18 +01:00
Nick Craig-Wood
7237b142fa drive: be more explicit in debug when setting permissions fail 2024-05-02 18:10:16 +01:00
Nick Craig-Wood
254e514330 onedrive,drive: make errors setting permissions into no retry errors 2024-04-30 09:34:33 +01:00
Nick Craig-Wood
9fa610088f docs: add Backblaze as a sponsor 2024-04-29 12:43:13 +01:00
Nick Craig-Wood
d2fa45acf3 storj: update bio on request 2024-04-29 11:49:13 +01:00
albertony
a86eb7ad50 docs: note that newer linux kernel version is required for ARMv5 2024-04-27 22:21:40 +02:00
Nick Craig-Wood
1fef8e667c build: migrate bucket storage for the project to new provider
This changes

- beta.rclone.org
- www.rclone.org
- pub.rclone.org
- downloads.rclone.org
2024-04-25 17:04:18 +01:00
Nick Craig-Wood
a5daef3892 Add hidewrong to contributors 2024-04-25 17:04:18 +01:00
Nick Craig-Wood
5bf70c68f1 swift: implement --swift-use-segments-container to allow >5G files on Blomp
This switches between storing chunks in a separate container suffixed
with `_segments` (the default) and a directory in the root
`.file-segments`)

By default the `.file-segments` mode will be auto selected if
`auth_url`s that require it are detected.

If the `.file-segments` mode is in use then rclone will omit that
directory from listings.

See: https://forum.rclone.org/t/blomp-unable-to-upload-5gb-files/42498/
2024-04-25 11:14:14 +01:00
Nick Craig-Wood
8a18c29835 random: update Password docs 2024-04-25 11:14:14 +01:00
albertony
29ed17d19c build: add linting for different values of GOOS 2024-04-22 19:29:12 +02:00
albertony
7ee22fcdf9 build: fix linting issues reported by running golangci-lint with different GOOS 2024-04-22 19:29:12 +02:00
albertony
159e274921 build: fix linting issues reported by golangci-lint on windows 2024-04-22 19:29:12 +02:00
albertony
fdc56b21c1 log: fix lint issue SA1019: syscall.Syscall has been deprecated since Go 1.18: Use SyscallN instead. 2024-04-22 19:29:12 +02:00
albertony
1ca825b6f0 build: run go mod tidy 2024-04-22 19:29:12 +02:00
Kyle Reynolds
d36bc8833c backend http: Adding no-escape flag for option to not escape URL metacharacters in path names - fixes issue #7637 2024-04-22 17:57:09 +01:00
nielash
8977655869 bisync: avoid starting tests we don't have time to finish
To prevent all-or-nothing retries, for tests that take longer (in total) than the
-timeout but less than the -timeout * -maxtries

https://github.com/rclone/rclone/pull/7743#issuecomment-2057250848
2024-04-19 22:29:41 +01:00
nielash
58e09e1cd4 bisync: skip test if config string contains a space 2024-04-19 22:29:41 +01:00
Kyle Reynolds
64734dfe41 fs accounting: Add deleted files total size to status summary line - fixes issue #7190 2024-04-18 22:09:23 +01:00