Commit graph

7 commits

Author SHA1 Message Date
Nick Craig-Wood
ee04732cbb vfs: factor writeback and downloaders into their own packages 2020-06-30 12:03:39 +01:00
Nick Craig-Wood
79455cc71e vfs: downloaders: remove unused osPath 2020-06-30 12:03:39 +01:00
Nick Craig-Wood
042e5fe097 vfs: downloader: limit the reader to 10 errors before giving up 2020-06-30 12:03:39 +01:00
Nick Craig-Wood
279a516c53 vfs: add tests and fix bugs for vfscache.Item
Item
- Remove unused method getName
- Fix Truncate on unopened file
- Fix bug when downloading segments to fill out file on close
- Fix bug when WriteAt extends the file and we don't mark space as used

downloader
- Retry failed waiters every 5 seconds
2020-06-30 12:01:36 +01:00
Nick Craig-Wood
9ac5c6de14 vfs: cache: rework file downloader
- Download to multiple places at once in the stream
- Restart as necessary
- Timeout unused downloaders
- Close reader if too much data skipped
- Only use one file handle as use item for writing
- Implement --vfs-read-chunk-size and --vfs-read-chunk-limit
- fix deadlock between asyncbuffer and vfs cache downloader
- fix display of stream abandoned error which should be ignored
2020-06-30 12:01:36 +01:00
Nick Craig-Wood
e4e53a2e61 vfs: add --vfs-writeback option to delay writes back to cloud storage
This is enabled by default and can be disabled with --vfs-writeback 0
2020-06-30 12:01:36 +01:00
Nick Craig-Wood
917cb4acb3 vfs: implement partial reads in --vfs-cache-mode full
This allows reads to only read part of the file and it keeps on disk a
cache of what parts of each file have been loaded.

File data itself is kept in sparse files.
2020-06-30 12:01:36 +01:00