alexvanin/restic
1
0
Fork 0
forked from TrueCloudLab/restic
Code Pull requests Activity
1270 commits 8 branches 49 tags 79 MiB
Commit graph

1270 commits

This branch
This branch
All branches
Author SHA1 Message Date
Christian Kemper
8f5ff379b7 Introduced a configurable object path prefix for s3 repositories. 
Prepends the object path prefix to all s3 paths and allows to have multiple independent
restic backup repositories in a single s3 bucket.

Removed the hardcoded "restic" prefix from s3 paths.

Use "restic" as the default object path prefix for s3 if no other prefix gets specified.
This will retain backward compatibility with existing s3 repository configurations.

Simplified the parse flow to have a single point where we parse the bucket name and the prefix within the bucket.

Added tests for s3 object path prefix and the new default prefix to config_test and location_test.
 2016-02-14 06:05:38 -08:00
Alexander Neumann
eccbcb73a1 Merge pull request #425 from restic/fix-loadall 
backend.LoadAll: return nil on expected error
 2016-02-08 00:37:04 +01:00
Alexander Neumann
e781e1cf1d Merge pull request #424 from restic/fix-backup 
Fix backup of root directory
 2016-02-08 00:06:19 +01:00
Alexander Neumann
e9a21c1dc6 backend.LoadAll: return nil on expected error 
The current code returns io.ErrUnexpectedEOF, but it is the normal,
expected behaviour of the function LoadAll() to load until the item is
completely loaded. Therefore, the io.ErrUnexpectedEOF is not returned to
the caller.
 2016-02-07 23:48:54 +01:00
Alexander Neumann
a37ed45534 Add test for LoadAll with too large buffer 
LoadAll() should not pass on io.ErrUnexpectedEOF, since the occurrence
of this error is normal.
 2016-02-07 23:48:03 +01:00
Alexander Neumann
26484d0c7b pipe: Ignore excluded directories 
When top-level directories are ignored, they were still added to the
top-level Dir{} object. This commit ignores them.
 2016-02-07 23:23:16 +01:00
Alexander Neumann
68db75b4e3 pipe/archiver: Add more debug messages 2016-02-07 23:22:06 +01:00
Alexander Neumann
9048eb676b pipe: join replaced paths with original path 
When saving `/`, it was replaced with the contents, but without the
proper path. So `/` was replaced by [`boot`, `bin`, `home`, ...], but
without prefixing the entry name with the proper path.
 2016-02-07 22:18:37 +01:00
Alexander Neumann
6a5b022939 archiver: Add error reporting for directories 
When an error occurred while walking a directory, this error wasn't
reported to the user before.
 2016-02-07 22:18:00 +01:00
Alexander Neumann
57a24b2cdf Merge pull request #421 from restic/fix-fuse-with-special-names 
fuse: Replace special node names with their content
 2016-02-07 20:43:29 +01:00
Alexander Neumann
da47389483 Merge pull request #420 from restic/archiver-unique-paths 
archiver: deduplicate list of paths to save
 2016-02-07 20:31:18 +01:00
Alexander Neumann
4c329110c5 Merge pull request #419 from restic/fix-backup-root 
Fix backup of "/"
 2016-02-07 20:31:08 +01:00
Alexander Neumann
46fbae0d71 Merge pull request #418 from benmur/show-pack-name-on-rebuild-index-error 
Handle pack loading errors in rebuild-index
 2016-02-07 19:59:01 +01:00
Alexander Neumann
1835e988cf fuse: Replace special node names with their content 
This is the companion fix for #419 and allows mounting repositories with
special directory names directly below the snapshot.

Closes #403
 2016-02-07 19:51:29 +01:00
Alexander Neumann
537347d9b5 archiver: deduplicate list of paths to save 2016-02-07 19:35:35 +01:00
Alexander Neumann
811dbfa52d Make TestWalkerPath absolute before walking 2016-02-07 19:34:02 +01:00
Rached Ben Mustapha
ba35ca522a Handle pack loading errors in rebuild-index 
Errors returned from backend.LoadAll() were not handled, leading to
these fatal errors from the unpacker trying to read the size from the end of
an empty buffer:

`seeking to read header length failed: bytes.Reader.Seek: negative position`

This change takes care of returning on error, as well as showing which pack
failed to load and validating pack integrity.
 2016-02-07 18:30:47 +00:00
Alexander Neumann
c6a1f2e2f3 pipe: Handle special paths gracefully 
This fixes handling special paths "." and "/". When such a path name is
found, it is replaced by the contents of the path before walking.
 2016-02-07 19:30:00 +01:00
Alexander Neumann
604c27f001 Test pipe walker for invalid paths 
It was discovered that when restic is instructed to save `/`, the tree
structures in the repository contain an invalid node.

When saving the dir `/home/user`, the following structure is created:

    snapshot
         -> tree
             nodes: ["user"]
             [...]

When the root directory `/` is saved, the structure is as follows:

    snapshot
         -> tree
             nodes: ["/"]
             [...]

This behavior is caused by the walker in pipe.go sending a node with the
name "." to the archiver, so this commit adds a test for invalid node
names.
 2016-02-07 19:29:44 +01:00
Alexander Neumann
0535490618 Merge pull request #413 from restic/reduce-travis-matrix 
Reduce jobs run on Travis
 2016-02-06 14:10:54 +01:00
Alexander Neumann
9c0fc4930b Merge pull request #412 from restic/activate-hound 
Enable HoundCI checking for Go
 2016-02-06 13:37:13 +01:00
Alexander Neumann
a45f2cb205 Reduce jobs run on Travis 
Skip building and testing restic with Go 1.3, 1.4 and 1.6 on osx.
 2016-02-06 13:04:49 +01:00
Alexander Neumann
3fd5b5975a Merge pull request #411 from restic/fix-appveyor-sourceforge 
Always use NetCologne SourceForge mirror
 2016-02-06 13:01:55 +01:00
Alexander Neumann
9175f0b6af Enable HoundCI checking for Go 2016-02-05 21:15:46 +01:00
Alexander Neumann
1160d03279 Always use NetCologne SourceForge mirror 
The one automatically selected by the SourceForge CDN fails currently:

    appveyor DownloadFile http://downloads.sourceforge.net/project/gnuwin32/tar/1.13-1/tar-1.13-1-bin.zip -FileName tar.zip
    Error downloading file: Unable to connect to the remote server Command exited with code 2
 2016-02-05 21:13:14 +01:00
Alexander Neumann
789e0df49e Merge pull request #408 from benmur/validate-pack-checksums 
checker: Validate pack checksums before unpacking
 2016-02-05 19:45:21 +01:00
Rached Ben Mustapha
83bbf21f1a checker: Validate pack checksums before unpacking 
This avoids reading a possibly invalid size at the end of a corrupted pack
 2016-02-04 22:55:39 +01:00
Alexander Neumann
7a8054d678 Add link to record of talk at C4 Cologne 2016-02-04 19:56:41 +01:00
Alexander Neumann
c0bbb7254d Merge pull request #406 from restic/fix-405 
Test and fix for #405
 2016-02-02 20:53:52 +01:00
Alexander Neumann
4f1f03cdb9 Move testing for known blobs to Archiver 
This removes the list of in-flight blobs from the master index and
instead keeps a list of "known" blobs in the Archiver. "known" here
means: either already processed, or included in an index. This property
is tested atomically, when the blob is not in the list of "known" blobs,
it is added to the list and the caller is responsible to make this
happen (i.e. save the blob).
 2016-02-01 23:50:56 +01:00
Alexander Neumann
382c766983 Move test for #405: Test Archiver instead of Repo 2016-02-01 23:50:41 +01:00
Alexander Neumann
f5f6e9cf37 Add test to reproduce #405 2016-02-01 23:35:01 +01:00
Alexander Neumann
cf88b33383 Merge pull request #398 from restic/update-minio-go 
Update minio-library
 2016-01-29 13:14:37 +01:00
Alexander Neumann
57615edd3a Merge pull request #400 from restic/update-go-appveyor 
Update Go version for CI tests
 2016-01-29 13:13:42 +01:00
Alexander Neumann
49cb88d158 Add note about lightning talk at FOSDEM 2016-01-28 22:40:52 +01:00
Alexander Neumann
1464d84cf5 Update minio-go 
This should work with Go 1.3/1.4 again
 2016-01-28 22:38:29 +01:00
Alexander Neumann
74ce027924 Merge pull request #397 from restic/remove-readcloser 
Remove backend.ReadCloser, this is not used any more
 2016-01-28 22:31:30 +01:00
Alexander Neumann
39f698886a Update Travis Go version 2016-01-28 22:28:17 +01:00
Alexander Neumann
9f8f2bc874 Update Go version for appveyor 2016-01-28 00:00:28 +01:00
Alexander Neumann
f8daadc5ef Update minio-library 
This addresses #388
 2016-01-27 23:23:47 +01:00
Alexander Neumann
f2371db2a9 repository: remove decryptReadCloser 2016-01-27 22:47:09 +01:00
Alexander Neumann
7d1775e000 Remove backend.ReadCloser 2016-01-27 22:35:18 +01:00
Alexander Neumann
ce4a7f16ca Merge pull request #395 from restic/rework-backend-interface 
WIP: Rework backend interface
 2016-01-27 22:11:20 +01:00
Alexander Neumann
322eca86bc mem backend: remove unused code 2016-01-27 21:33:48 +01:00
Alexander Neumann
3d06e6083a CI: download minio for the correct os and arch 2016-01-26 23:52:39 +01:00
Alexander Neumann
b64006221c CI: Download minio server, do not compile latest master 2016-01-26 23:50:27 +01:00
Alexander Neumann
1fde872016 CI: only build minio on Go 1.5.1 and above 2016-01-26 22:36:06 +01:00
Alexander Neumann
2701eabe39 Remove ContinuousReader 2016-01-26 22:35:51 +01:00
Alexander Neumann
c388101217 s3: Unexport structure 2016-01-26 22:19:44 +01:00
Alexander Neumann
1528d1ca83 sftp: Reduce duplicate code, add error check 2016-01-26 22:16:24 +01:00