TrueCloudLab/rclone
16
0
Fork 2
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

local: always use readlink to read symlink size

Browse source 
It was discovered on some Android systems, the stat size of a symlink
is different to the size that readlink returns.

This was giving errors like this

    transport connection broken: http: ContentLength=30 with Body length 28

There are enough exceptions to the size of readlink being different to
the size of stat that this patch now always does readlink to work out
the size of a symlink.

Since symlinks are relatively uncommon this shouldn't affect
performance too much and will mean that the size is always correct.

This deprecates the --local-zero-size-links flag which is now
effectively always enabled.

See: https://forum.rclone.org/t/problem-with-symlinks-and-links/23840/
This commit is contained in:
Nick Craig-Wood 2021-05-01 12:39:29 +01:00
parent 9ff6f48d74
commit 6ef7178ee4
1 changed files with 14 additions and 11 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

25
backend/local/local.go
View file

@ -73,16 +73,16 @@ points, as you explicitly acknowledge that they should be skipped.`,
Advanced: true,
}, {
Name: "zero_size_links",
Help: `Assume the Stat size of links is zero (and read them instead)
Help: `Assume the Stat size of links is zero (and read them instead) (Deprecated)
On some virtual filesystems (such ash LucidLink), reading a link size via a Stat call always returns 0.
However, on unix it reads as the length of the text in the link. This may cause errors like this when
syncing:
Rclone used to use the Stat size of links as the link size, but this fails in quite a few places
Failed to copy: corrupted on transfer: sizes differ 0 vs 13
- Windows
- On some virtual filesystems (such ash LucidLink)
- Android
Setting this flag causes rclone to read the link and use that as the size of the link
instead of 0 which in most cases fixes the problem.`,
So rclone now always reads the link
`,
Default: false,
Advanced: true,
}, {
@ -196,7 +196,6 @@ type Options struct {
FollowSymlinks bool `config:"copy_links"`
TranslateSymlinks bool `config:"links"`
SkipSymlinks bool `config:"skip_links"`
ZeroSizeLinks bool `config:"zero_size_links"`
NoUTFNorm bool `config:"no_unicode_normalization"`
NoCheckUpdated bool `config:"no_check_updated"`
NoUNC bool `config:"nounc"`
@ -1267,9 +1266,13 @@ func (o *Object) setMetadata(info os.FileInfo) {
o.modTime = info.ModTime()
o.mode = info.Mode()
o.fs.objectMetaMu.Unlock()
// On Windows links read as 0 size so set the correct size here
// Optionally, users can turn this feature on with the zero_size_links flag
if (runtime.GOOS == "windows" || o.fs.opt.ZeroSizeLinks) && o.translatedLink {
// Read the size of the link.
//
// The value in info.Size() is not always correct
// - Windows links read as 0 size
// - Some virtual filesystems (such ash LucidLink) links read as 0 size
// - Android - some versions the links are larger than readlink suggests
if o.translatedLink {
linkdst, err := os.Readlink(o.path)
if err != nil {
fs.Errorf(o, "Failed to read link size: %v", err)