local: fix copying from Windows Volume Shadows
For some files the Windows Volume Shadow Service (VSS) advertises the file size as X in the directory listing but returns a different number Y on stat-ing the file. If the file is opened and read there are Y bytes available for reading. Existing copy tools copy Y bytes rather than X so for consistency rclone should do the same. This fixes the problem by stat-ing the file immediately before opening it. This will also reduce the unnecessary occurrence of "can't copy - source file is being updated" errors; if the file has finished changing by the time we come to copy it then we now can copy it successfully. See: https://forum.rclone.org/t/consistently-getting-corrupted-on-transfer-sizes-differ-syncing-to-an-smb-share/42218/
This commit is contained in:
parent
b0a87d7cf1
commit
5b0f9dc4e3
1 changed files with 11 additions and 0 deletions
|
@ -146,6 +146,11 @@ time we:
|
|||
- Only checksum the size that stat gave
|
||||
- Don't update the stat info for the file
|
||||
|
||||
**NB** do not use this flag on a Windows Volume Shadow (VSS). For some
|
||||
unknown reason, files in a VSS sometimes show different sizes from the
|
||||
directory listing (where the initial stat value comes from on Windows)
|
||||
and when stat is called on them directly. Other copy tools always use
|
||||
the direct stat value and setting this flag will disable that.
|
||||
`,
|
||||
Default: false,
|
||||
Advanced: true,
|
||||
|
@ -1123,6 +1128,12 @@ func (o *Object) Open(ctx context.Context, options ...fs.OpenOption) (in io.Read
|
|||
}
|
||||
}
|
||||
|
||||
// Update the file info before we start reading
|
||||
err = o.lstat()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// If not checking updated then limit to current size. This means if
|
||||
// file is being extended, readers will read a o.Size() bytes rather
|
||||
// than the new size making for a consistent upload.
|
||||
|
|
Loading…
Reference in a new issue