local: fix copying from Windows Volume Shadows
For some files the Windows Volume Shadow Service (VSS) advertises the file size as X in the directory listing but returns a different number Y on stat-ing the file. If the file is opened and read there are Y bytes available for reading. Existing copy tools copy Y bytes rather than X so for consistency rclone should do the same. This fixes the problem by stat-ing the file immediately before opening it. This will also reduce the unnecessary occurrence of "can't copy - source file is being updated" errors; if the file has finished changing by the time we come to copy it then we now can copy it successfully. See: https://forum.rclone.org/t/consistently-getting-corrupted-on-transfer-sizes-differ-syncing-to-an-smb-share/42218/
This commit is contained in:
parent
b0a87d7cf1
commit
5b0f9dc4e3
1 changed files with 11 additions and 0 deletions
|
@ -146,6 +146,11 @@ time we:
|
||||||
- Only checksum the size that stat gave
|
- Only checksum the size that stat gave
|
||||||
- Don't update the stat info for the file
|
- Don't update the stat info for the file
|
||||||
|
|
||||||
|
**NB** do not use this flag on a Windows Volume Shadow (VSS). For some
|
||||||
|
unknown reason, files in a VSS sometimes show different sizes from the
|
||||||
|
directory listing (where the initial stat value comes from on Windows)
|
||||||
|
and when stat is called on them directly. Other copy tools always use
|
||||||
|
the direct stat value and setting this flag will disable that.
|
||||||
`,
|
`,
|
||||||
Default: false,
|
Default: false,
|
||||||
Advanced: true,
|
Advanced: true,
|
||||||
|
@ -1123,6 +1128,12 @@ func (o *Object) Open(ctx context.Context, options ...fs.OpenOption) (in io.Read
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Update the file info before we start reading
|
||||||
|
err = o.lstat()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
// If not checking updated then limit to current size. This means if
|
// If not checking updated then limit to current size. This means if
|
||||||
// file is being extended, readers will read a o.Size() bytes rather
|
// file is being extended, readers will read a o.Size() bytes rather
|
||||||
// than the new size making for a consistent upload.
|
// than the new size making for a consistent upload.
|
||||||
|
|
Loading…
Reference in a new issue