Commit graph

9 commits

Author SHA1 Message Date
Nick Craig-Wood
ee04732cbb vfs: factor writeback and downloaders into their own packages 2020-06-30 12:03:39 +01:00
Nick Craig-Wood
ef2d036884 vfs: make writeback heap sort in insertion order if expiry times equal
This makes the tests 100% consistent on platforms which have a lower
resolution timer like Windows.
2020-06-30 12:01:36 +01:00
Nick Craig-Wood
b9ff495483 vfs: writeback - stop the timer explicitly on transfers exceeded 2020-06-30 12:01:36 +01:00
Nick Craig-Wood
8506066926 vfs: use call after functions in writeback to simplify code
This also fixes a bug in the uploader which didn't restart the timer
when the queue was empty.
2020-06-30 12:01:36 +01:00
Nick Craig-Wood
43018973ac vfs: decouple writeback from Item so it can be tested 2020-06-30 12:01:36 +01:00
Nick Craig-Wood
fa472a340e vfs: fix writeback deadlocks and other bugs
- fix deadlock when cancelling upload
- fix double upload and panic after cancelled upload
- fix cancelation strategy of uploading files
    - don't cancel uploads if we don't modify the file
    - cancel uploads if we do modify the file
- fix deadlock between Item and writeback
- fix confusion about whether writeback item was being uploaded
- fix cornercases in cancelling uploads and removing files
2020-06-30 12:01:36 +01:00
Nick Craig-Wood
58a7faa281 vfs: Make tests run reliably
On file Remove
- cancel any writebacks in progress
- ignore error message deleting non existent file if file was in the
  process of being uploaded

Writeback
- Don't transfer the file if it has disappeared in the meantime
- Take our own copy of the file name to avoid deadlocks
- Fix delayed retry logic
- Wait for upload to finish when cancelling upload

Fix race condition in item saving

Fix race condition in vfscache test

Make sure we delete the file on the error path - this makes cascading
failures much less likely
2020-06-30 12:01:36 +01:00
Nick Craig-Wood
496a87a665 vfs: restart pending uploads on restart of the cache 2020-06-30 12:01:36 +01:00
Nick Craig-Wood
e4e53a2e61 vfs: add --vfs-writeback option to delay writes back to cloud storage
This is enabled by default and can be disabled with --vfs-writeback 0
2020-06-30 12:01:36 +01:00