TrueCloudLab/restic
16
0
Fork 2
Code Issues 1 Pull requests 1 Projects Releases Packages Wiki Activity Actions
6107 commits 9 branches 49 tags 79 MiB
Commit graph

1214 commits

Author SHA1 Message Date
Alexander Neumann
9310cd0cd6 Merge pull request #2056 from restic/recover-data 
Add new command 'recover'
 2018-10-28 14:26:48 +01:00
Alexander Neumann
19fd0f101f Merge pull request 2053 from andyjack/correct-doc-for-self-update 
Fix incorrect command name in long doc for self-update
 2018-10-20 11:51:13 +02:00
Alexander Neumann
8c91c51d1b Fix timestamps 2018-10-20 11:08:03 +02:00
Alexander Neumann
7e28bf7e97 Add 'recover' command 2018-10-20 11:08:03 +02:00
Gábor Lipták
d2ee58f2e9
Correct typo 2018-10-18 21:30:14 -04:00
Andy Jack
3f25537a06 Fix incorrect command name in long doc for self-update 2018-10-18 18:45:31 -04:00
Andreas Nüßlein
6eedd66c1a
tiny formatting fix 
I noticed this discrepancy when trying to parse the output of restic.
 2018-10-17 09:41:11 +02:00
Alexander Neumann
4c2072d875 Merge pull request 1719 from ifedorenko/mt-restorer 
new optimized multithreaded restore implementation
 2018-10-14 20:29:16 +02:00
Igor Fedorenko
9e24154ec9 restore: significantly reduce memory footprint 
reworked restore error callback to use file location
path instead of much heavier Node. this reduced restore
memory usage by as much as 50% in some of my tests.

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2018-10-14 17:39:42 +02:00
Igor Fedorenko
da57302fca restore: Removed legacy restore implementation 
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2018-10-14 17:37:30 +02:00
Igor Fedorenko
1869930d95 restore: New optimized multithreaded implementation 
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2018-10-14 17:37:30 +02:00
Alexander Neumann
a432b42c81 self-update: Check current version before download 2018-10-14 17:29:46 +02:00
Alexander Neumann
7d0f2eaf24 self-update: Use correct path to restic binary 
Closes #2041
 2018-10-14 17:29:19 +02:00
Alexander Neumann
41a4d67d93 Merge pull request 1962 from mholt/lsjson 
ls: Stream output when using --json option
 2018-10-14 15:51:38 +02:00
Alexander Neumann
d7baa67acb snapshots: Don't print empty line for --compact 
Closes #2038
 2018-10-13 20:17:34 +02:00
Alexander Neumann
167397c18c Set development version for 0.9.3 2018-10-13 13:48:31 +02:00
Alexander Neumann
be36c5f150 Add version for 0.9.3 2018-10-13 13:48:31 +02:00
Johannes Hertenstein
277cba4b32 Catch errors when walking cache directories 2018-10-08 15:47:34 +02:00
Johannes Hertenstein
ed651df19b Use correct method for joining paths 2018-10-08 15:47:08 +02:00
Johannes Hertenstein
641dc65e6e Output directory size in cache command 2018-10-07 14:37:51 +02:00
Daniel Bretoi
966e5a5575 use latest instead of newest in forget language 2018-10-06 12:02:22 -07:00
Alexander Neumann
5dbef3712e Merge pull request #2027 from j6s/feature.cache-dir-information 
Print base directory with cache command
 2018-10-06 12:40:48 +02:00
Johannes Hertenstein
9b8deb51ba Print base directory with cache command 2018-10-05 21:23:57 +02:00
Johannes Hertenstein
0916ff71bd Add defaults to command line help of --cache-dir 2018-10-04 19:50:05 +02:00
Alexander Neumann
585a5e3416 Use --host for all commands, deprecate --hostname 2018-10-03 14:12:35 +02:00
Alexander Neumann
fb31d66951 backup: Improve error message for invalid pattern 2018-10-03 11:12:51 +02:00
Stefano Fancello
e2da0a416c Expand Glob (wildcards character) in paths in file in --files-from 2018-10-03 10:37:35 +02:00
Alexander Neumann
6d53e767d5 cache: Print message when new cache is created 
Sometimes, users run restic without retaining the local cache
directories. This was reported several times in the past.

Restic will now print a message whenever a new cache directory is
created from scratch (i.e. it did not exist before), so users have a
chance to recognize when the cache is not kept between different runs of
restic.
 2018-08-29 22:01:10 +02:00
Quentin Bourgeois
a065ada46a
Make generated man pages more easy to read with items 
Allow Cobra to output a more list items friendly format, this make
reading of the documentation more easily using information from
manpages.
 2018-08-27 17:42:54 +02:00
Alexander Neumann
896089976a Fix comment 2018-08-26 19:54:17 +02:00
Alexander Neumann
de307ea2ab Merge pull request #1876 from restic/forget-explain 
forget: Add --explain
 2018-08-25 21:48:44 +02:00
Alexander Neumann
76387b6cd0 Remove old text table implementation 2018-08-20 22:00:23 +02:00
Alexander Neumann
9fd3796d93 forget: Display reasons why snapshots are kept 
This change displays the reasons for keeping a snapshot in the table,
unless `--compact` is specified.
 2018-08-20 21:47:51 +02:00
Alexander Neumann
15ad0e5bc7 walk: Pass parent tree ID to WalkFunc 2018-08-19 23:28:04 +02:00
Mikael Berthe
1f27d17c0d walker.Walk: Pass parent tree-id to WalkFunc 2018-08-19 23:28:04 +02:00
Mikael Berthe
8af918a1e4 find: Add support for multiple patterns or objects 2018-08-19 23:28:04 +02:00
Mikael Berthe
bb5425a1d8 find: Add support for blobs, IDs and packs lookups 
With --blob, --tree and --pack, the find command now lists the snapshots
that contain a specific tree or blob, or the snapshots that contain
blobs belonging to a given pack.
It also displays the pack ID a blob belongs to.

A list of IDs can be given, as long as the IDs are all of the same type.
 2018-08-19 22:32:30 +02:00
Matthew Holt
22475729ce Select specific Node fields for listing 2018-08-19 09:03:47 -06:00
Matthew Holt
04c67d700d ls: Stream output when using --json option 2018-08-19 00:18:43 -06:00
kitone
48cc2f2188 fix: switch struct_type value to lower case 2018-08-18 14:57:50 +02:00
kitone
7925217e25 ls: Add JSON output support for restic ls cmd 2018-08-18 14:57:50 +02:00
Alexander Neumann
401a564486 Merge pull request #1961 from mholt/statsoutput 
stats: Show what was scanned and scanning mode used
 2018-08-18 14:29:14 +02:00
Matthew Holt
2d89311d49 stats: Show what was scanned and scanning mode used 2018-08-17 17:15:30 -06:00
Alexander Neumann
5a25ad1972 Update version in build.go 2018-08-17 21:17:26 +02:00
Alexander Neumann
1ab5703404 prune: Fix calculation for removed bytes 2018-08-14 22:06:05 +02:00
Alexander Neumann
49d95e9a50 Merge pull request #1949 from restic/add-self-update 
Add command self-update
 2018-08-13 22:02:34 +02:00
Alexander Neumann
5fee36fa84 Merge pull request #1941 from mholt/lsfilter 
ls: Implement directory filter, optionally subfolders
 2018-08-13 21:58:32 +02:00
Alexander Neumann
6e1a3987b7 Add 'self-update' command 
This commit adds a command called `self-update` which downloads the
latest released version of restic from GitHub and replacing the current
binary with it. It does not rely on any external program (so it'll work
everywhere), but still verifies the GPG signature using the embedded GPG
public key.

By default, the `self-update` command is hidden behind the `selfupdate`
built tag, which is only set when restic is built using `build.go`. The
reason for this is that downstream distributions will then not include
the command by default, so users are encouraged to use the
platform-specific distribution mechanism.
 2018-08-12 23:34:47 +02:00
Alexander Neumann
9630398e3b ls: Rework and simplify logic 
This commit introduces two functions: withinDir() and
approachingMatchingTree()

Both bind the list of directories with a closure, so we don't need to
iterate over the list in the function passed to Walk(). This reduces the
indentation level and since we can just use return, we don't need the
breaks any more.

The case that len(dirs) == 0 can also be handled by the functions with a
return, which saves another indentation level.

The main function body of the function passed to Walk() was reduced to
three cases:

 * Within one of the dirs: Print the node, and if recursive operation is
   requested, directly return, so the walker continues recursive
   traversal

 * Approaching one of the dirs: don't print anything, but continue
   recursive traversal.

 * Nothing of the two: abort walking this branch of the tree.
 2018-08-12 23:13:34 +02:00
Alexander Neumann
7e34de4c29 ls: Add comments 2018-08-12 22:18:44 +02:00
Alexander Neumann
ace5cc4ed3 ls: Only skip directory nodes 
Special case for Walk(): When SkipDir is returned for a non-dir node,
the remaining nodes for the current tree are skipped. We don't want
that.
 2018-08-12 22:02:59 +02:00
Alexander Neumann
7f617cfd7f ls: Use nodepath for filter 2018-08-12 22:01:38 +02:00
Alexander Neumann
0deb4e5994 ls: Check dirs before opening the repository 
Users get feedback instantly, and before any expensive network calls
have been made.
 2018-08-12 21:59:57 +02:00
Andreas Skielboe
b07bb3d8c3 Reject files excluded by name before calling lstat to improve scan speed 
Adds a SelectByName method to the archive and scanner which only require
the filename as input, and can thus be run before calling lstat on the
file. Can speed up scanning significantly if a lot of filename excludes
are used.
 2018-08-12 17:51:12 +02:00
Matthew Holt
11ce572894 Fix bug where some folder listings were empty 2018-08-11 17:17:43 -06:00
Matthew Holt
7a468d1226 Speed up nonrecursive queries; include exact filter match 2018-08-11 16:18:09 -06:00
Matthew Holt
00e2fd8b5f Apply feedback and use SkipNode 2018-08-11 15:25:22 -06:00
Matthew Holt
ac3bd6b2eb Replace Exitf with errors.Fatalf 2018-08-10 22:15:33 -06:00
Matthew Holt
156d85a29b Minor fixes/tweaks; add docs 2018-08-10 22:10:02 -06:00
Matthew Holt
8c146eac4b ls: Implement directory filter, optionally subfolders 2018-08-10 21:41:38 -06:00
Matthew Holt
6f5b0f3622 stat: Improve error message for bad snapshot ID (fixes #1933) 2018-08-08 16:49:36 -06:00
Alexander Neumann
3422c1ca83 Merge pull request #1729 from mholt/stats 
Implement `restic stats` command to get more info about a repository
 2018-07-31 23:24:36 +02:00
Matthew Holt
f6b2731aa5 stats: Add manual doc, improve -h doc 
Also rename files-by-content to files-by-contents, once and for all
 2018-07-31 22:54:10 +02:00
Alexander Neumann
3678ec9ad8 Print message for password being read from stdin 
Closes #1911
 2018-07-31 20:21:18 +02:00
Alexander Neumann
12c797700e make statsWalkSnapshot return a function 2018-07-27 21:44:59 +02:00
Matthew Holt
daca9d6815 Consolidate mode flags; use new Walk function 2018-07-27 21:27:40 +02:00
Matthew Holt
930602a444 Update comment now that question was answered 2018-07-27 21:27:39 +02:00
Matthew Holt
acb05e7855 Fix filepath uniqueness bug for blobs-per-file mode 2018-07-27 21:27:39 +02:00
Matthew Holt
a7b95d716a Implement four counting modes 2018-07-27 21:27:39 +02:00
Matthew Holt
925b542eb0 Count unique files by blob sequence rather than tree ID 2018-07-27 21:27:39 +02:00
Matthew Holt
f7659bd8b0 stats: Initial implementation of stats command 2018-07-27 21:27:39 +02:00
Alexander Neumann
f715bef82f Merge pull request #1899 from garrmcnu/check-cache-dir 
check: Use --cache-dir argument
 2018-07-22 21:03:52 +02:00
Garry McNulty
7603ab7ac1 check: Update --cache-dir argument handling based on code review comments 
The temporary cache directory is created in the specified directory, or
if not specified in the default temporary directory.
 2018-07-22 18:24:11 +01:00
Garry McNulty
f1f69bc648 check: Use --cache-dir argument 
Closes #1880
 2018-07-20 20:51:20 +01:00
Alexander Neumann
fb74de6360 Return an error when exclude files cannot be read 2018-07-18 21:39:07 +02:00
Alexander Neumann
b511f4dce2 Improve help message for check 2018-07-05 22:19:08 +02:00
Mikael Berthe
fbf8073dfc Fix find -i (case-insensitive search) 2018-06-24 19:35:11 +02:00
Alexander Neumann
7ddf91b65c Merge pull request #1772 from ifedorenko/restore-verify 
Restore verify
 2018-06-23 14:50:29 +02:00
Alexander Neumann
8dae2de2ce Merge pull request #1858 from restic/fix-1857 
Allow excluding files with $ via --exclude-file
 2018-06-22 21:22:23 +02:00
Alexander Neumann
025ec9dff5 Allow excluding files with $ via --exclude-file 
Previously it wasn't possbile to exclude files with a literal dollar
sign (`$`) via exclude files, now users can write `$$` for that.
 2018-06-22 20:46:04 +02:00
David Roman
2384c1cee7
Ignore ExcludeOtherFs if Stdin is true 
Closes: #1807
 2018-06-21 23:12:38 +02:00
Steve Kriss
5088905502 add JSON output support for restic key list cmd 
Signed-off-by: Steve Kriss <steve@heptio.com>
 2018-06-18 15:26:26 -07:00
Iku Iwasa
f080142137 Support NetBSD without fuse 2018-06-16 15:55:04 +09:00
Igor Fedorenko
e206680947 restore: New --verify flag to verify restored files content 
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2018-06-11 07:57:22 -04:00
Igor Fedorenko
5fa6dc53cb Refactor: introduced restorer tree visitor 
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2018-06-11 07:57:22 -04:00
Igor Fedorenko
26be094f28 Refactor: moved restorer to separate package 
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2018-06-11 07:57:22 -04:00
Alexander Neumann
321efec60c Fix integration tests on Windows 2018-06-10 00:00:22 +02:00
Alexander Neumann
8f26fe271c ls: Use walker for ls 2018-06-09 23:35:20 +02:00
Alexander Neumann
081743d0a5 find: Use walker.Walk 2018-06-09 23:35:20 +02:00
Alexander Neumann
ce01ca30d6 find: Correct tree pruning optimization 
The `find` command will now take care to only mark trees as "not found"
when the pattern couldn't be found within any subtree.

Closes #1825, #1823
 2018-06-09 18:59:13 +02:00
Alexander Neumann
e2d347a698 find: Use OS independent slash-based format 2018-06-09 18:58:13 +02:00
Alexander Neumann
42ebb0a0a6 backup: Parse timestamp earlier 2018-06-09 18:21:12 +02:00
Alexander Neumann
bfd923e81e rclone: Respect bandwith limits 2018-05-22 20:48:17 +02:00
Alexander Neumann
8d9d218d1c list: Improve error message 
Before:

    $ restic list
    Fatal: type not specified

After:

    $ restic list
    Fatal: type not specified, usage: list [blobs|packs|index|snapshots|keys|locks]

Closes #1783
 2018-05-17 19:41:56 +02:00
Mikael Berthe
a868a30f4d Fix restic backup --one-file-system / 
This patch should fix the following panic when trying to backup the
root filesystem with thre --one-file-system flag:

    % restic backup --one-file-system /
    (...)
    panic: item /, device id 2082 not found, allowedDevs: map[/:2082]
 2018-05-16 13:44:14 +02:00
Alexander Neumann
cc627e832b Add custom Duration type 2018-05-13 12:48:42 +02:00
Alexander Neumann
5a0f0e3faa Add support for keeping a range of snapshots 2018-05-13 12:48:42 +02:00
Matthew Holt
b52f2aa9a4 forget: Add policy to keep snapshots before a date 2018-05-13 12:48:10 +02:00
Alexander Neumann
159badf5ba Merge pull request #1778 from restic/fix-1771 
archiver: Improve error handling
 2018-05-13 00:13:54 +02:00
Alexander Neumann
903a3a31dc Merge pull request #1776 from restic/fix-1775 
Always use cleaned path for excludes
 2018-05-13 00:13:52 +02:00
Alexander Neumann
581c62ee72 archiver: Improve error handling 
This commit changes how the worker goroutines for saving e.g. blobs
interact. Before, it was possible to get stuck sending an instruction to
archive a file or dir when no worker goroutines were available any more.
This commit introduces a `done` channel for each of the worker pools,
which is set to the channel returned by `tomb.Dying()`, so it is closed
when the first worker returned an error.
 2018-05-12 23:55:54 +02:00
Alexander Neumann
ef7747313d backup: Use absolute paths for allowed devices 2018-05-11 21:32:44 +02:00
Alexander Neumann
18d4ac2fd9 backup: Always use cleaned path for excludes 2018-05-11 21:32:41 +02:00
Alexander Neumann
9180e2c48a Remove unneeded file excludes 2018-05-11 21:26:10 +02:00
Gaige Lama
d3c0bd6d0e
Fix small typo 
"explicitely" should be "explicitly"
 2018-05-11 10:55:12 -04:00
Mikael Berthe
ffc276a603 cat: Do not display a stack trace 
Don't show a stack trace when "restic cat snapshot" is invoked with
invalid/nonexistent IDs.
 2018-05-10 19:22:46 +02:00
Alexander Neumann
fca4fe4459 backup: Disable status output for --quiet 2018-05-02 21:24:18 +02:00
Alexander Neumann
d0974c155d backup: Tune user interface a bit 
Make non-verbose mode a bit more verbose.
 2018-05-01 22:18:32 +02:00
Alexander Neumann
8026e6fdfb Merge pull request #1749 from restic/add-cache-cmd 
Add 'cache' command to list and cleanup cache dirs
 2018-05-01 21:25:52 +02:00
Alexander Neumann
f928aeec34 Add 'cache' command 2018-05-01 16:27:44 +02:00
Alexander Neumann
f77bc0fae8 Use textfile.Read() to read files 
This converts the text to UTF-8 and removes a Byte Order Mark.
 2018-05-01 16:25:53 +02:00
Alexander Neumann
d8bbe5dc84 Print repository ID after opening 2018-04-30 14:19:07 +02:00
Alexander Neumann
d926b9fd80 Add profile build flag 2018-04-30 14:19:04 +02:00
Alexander Neumann
4ba8d40282 Add block profile option 2018-04-30 14:18:45 +02:00
Alexander Neumann
4fb1401266 Fix --cacert help text 2018-04-29 13:17:33 +02:00
Alexander Neumann
1449d7dc29 Remove background checking code 2018-04-28 22:08:11 +02:00
Alexander Neumann
0e78ac92d8 Use new archiver code for backup 2018-04-28 22:08:11 +02:00
Alexander Neumann
9fac2ca832 Add flags to set verbosity 2018-04-25 14:42:45 +02:00
Alexander Neumann
a5c0cf2324 Add workaround for symlinked temp dir on darwin 
Chdir to the tempdir, then use os.Getwd() to get the name that
filepath.Abs() uses (and stores in the Snapshot).
 2018-04-25 14:42:45 +02:00
Alexander Neumann
38926d8576 Use new archiver code in tests 2018-04-25 14:42:45 +02:00
Alexander Neumann
b804279fe8 integration tests: Don't print anything to stdout 2018-04-22 11:37:05 +02:00
Alexander Neumann
a069467e72 ls: Improve output 2018-04-22 11:37:05 +02:00
Alexander Neumann
09365cc4ea Add --trace-profile 2018-04-22 11:37:05 +02:00
Eri Bastos
2aa6b49651 Return exit code 130 when SIGINT is received 2018-04-20 21:09:50 +02:00
Matthew Holt
c5829e9ffc key: Add flag for non-interactive password changes 2018-04-20 14:29:39 +02:00
Alexander Neumann
1c1fede399 Improve error message for orphaned pack files 2018-04-07 10:07:54 +02:00
Eri Bastos
901e1b129c Fixed issue #1608 - Use --time argument properly 
Backups via stdin will now handle --time argument and pass it down as
expected
 2018-04-03 14:40:42 -03:00
Alexander Neumann
01fe719aff check: Make sure temp cache dir is removed 2018-04-01 18:09:53 +02:00
Alexander Neumann
2c964df3e2 Merge pull request #1699 from restic/fix-incremental-backup-test 
Slightly increment size for TestIncrementalBackup
 2018-04-01 14:33:12 +02:00
Alexander Neumann
a95eb33616 check: Use cache in temporary directory if possible 
Closes #1694
 2018-04-01 13:59:27 +02:00
Alexander Neumann
4544a77172 Slightly increment size for TestIncrementalBackup 
This should make the test more reliable, it should hit the accidental
"repo is has grown too much" way less often.
 2018-04-01 13:49:42 +02:00
Alexander Neumann
fe99340e40 Add rclone backend 2018-04-01 10:16:31 +02:00
Alexander Neumann
d3f9c8b362 forget: Print policy 2018-03-30 10:24:26 +02:00
Leo R. Lundgren
923be90906 Make version output one line. 2018-03-21 21:49:03 +01:00
Bowen Song
34671d7c9b Skip archiver.Scan before backup when --quiet is set 2018-03-20 20:44:10 +00:00
Alexander Neumann
8206f85d2e Merge pull request #1552 from lawrencejones/use-auto-auth 
Automatically load Google auth
 2018-03-18 19:53:30 +01:00
Alexander Neumann
dfb6d0fced Reduce data for TestIncrementalBackup 2018-03-11 21:17:27 +01:00
Alexander Neumann
c6c1dccc53 Reduce data set for TestRestore 2018-03-11 21:10:37 +01:00
Alexander Neumann
279566bafe Reduce dataset for integration tests 2018-03-11 21:07:47 +01:00
Alexander Neumann
c67a8452f7 Disable polynomial check for chunker for tests 2018-03-11 20:59:40 +01:00
Lawrence Jones
0dfdc11ed9
Automatically load Google auth 
This change removes the hardcoded Google auth mechanism for the GCS
backend, instead using Google's provided client library to discover and
generate credential material.

Google recommend that client libraries use their common auth mechanism
in order to authorise requests against Google services. Doing so means
you automatically support various types of authentication, from the
standard GOOGLE_APPLICATION_CREDENTIALS environment variable to making
use of Google's metadata API if running within Google Container Engine.
 2018-03-11 17:11:25 +00:00
Eri Bastos
b9d643358a Skip checking for password file existence if command is 'version' - Issue #1632 2018-03-08 17:55:03 -04:00
Jasper Lievisse Adriaanse
96311d1a2b Add support for illumos/Solaris 
This does come without xattr/fuse support at this point.

NB: not hooking up the integration tests as restic won't compile without
    cgo with Go < 1.10.
 2018-03-04 20:11:29 +00:00
Alexander Neumann
7d59df1ab8 mount: Ignore non-existing locks 
Closes #1637
 2018-02-25 13:11:03 +01:00
Igor Fedorenko
07d080830e Add --read-data-subset flag to check command 
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2018-02-18 23:31:27 -05:00
Alexander Neumann
c5575c7ed9 Strip spaces from items read via --files-from 
In #1590, it was mentioned that while lines read from exclude files via
`--exclude-file` have leading and trailing spaces stripped, this is not
the case for lines read via `--files-from`. This commit fixes that,
spaces are always stripped.
 2018-02-11 20:56:11 +01:00
Alexander Neumann
cbfa516575 Remove archiver progress "data processed" bandwith 
This commit removes the bandwidth displayed during backup process. It is
misleading and seldomly correct, because it's neither the "read
bandwidth" (only for the very first backup) nor the "upload bandwidth".
Many users are confused about (and rightly so), c.f. #1581, #1033, #1591

We'll eventually replace this display with something more relevant when
 #1494 is done.
 2018-01-28 09:30:26 +01:00
Alexander Neumann
5dc8d3588d GS: Use generic http transport 
During the development of #1524 I discovered that the Google Cloud
Storage backend did not yet use the HTTP transport, so things such as
bandwidth limiting did not work. This commit does the necessary magic to
make the GS library use our HTTP transport.
 2018-01-27 20:12:34 +01:00
Alexander Neumann
c34db983d8 Read TLS client cert and key from the same file 2018-01-27 14:02:01 +01:00
Bryce Chidester
e805b968b1 Support for TLS client certificate authentication 
This adds --tls-client-cert and --tls-client-key parameters and enables use
of that certificate/key pair when connecting to https servers.
 2018-01-27 13:18:22 +01:00
Alexander Neumann
ff3de66ddf Merge pull request #1582 from restic/optimize-debug-log 
Optimize debug logs
 2018-01-26 21:57:18 +01:00
Alexander Neumann
f63c7048c7 index: Limit index files to 3000 pack files 2018-01-26 21:07:16 +01:00
Alexander Neumann
663c57ab4d debug: Remove manual Str() call Log() 2018-01-25 20:49:41 +01:00
Alexander Neumann
9c55e8d69c Merge pull request #1549 from MJDSys/more_index_lookup_avoids 
More optimizations to avoid calling Index.Lookup()
 2018-01-24 20:53:30 +01:00
Matthew Dawson
df2c03a6a4
repository/master_index: Optimize Index.Lookup() 
When looking up a blob in the master index, with several
indexes present in the master index, a significant amount of time
is spent generating errors for each failed lookup.  However, these
errors are often used to check if a blob is present, but the contents
are not inspected making the overhead of the error not useful.

Instead, change Index.Lookup (and Index.LookupSize) to instead return
a boolean denoting if the blob was found instead of an error.  Also change
all the calls to these functions to handle the new function signature.

benchmark                                            old ns/op     new ns/op     delta
BenchmarkMasterIndexLookupSingleIndex-6              820           897           +9.39%
BenchmarkMasterIndexLookupMultipleIndex-6            12821         2001          -84.39%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       5378          492           -90.85%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     17026         1649          -90.31%

benchmark                                            old allocs     new allocs     delta
BenchmarkMasterIndexLookupSingleIndex-6              9              9              +0.00%
BenchmarkMasterIndexLookupMultipleIndex-6            59             19             -67.80%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       22             6              -72.73%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     72             16             -77.78%

benchmark                                            old bytes     new bytes     delta
BenchmarkMasterIndexLookupSingleIndex-6              160           160           +0.00%
BenchmarkMasterIndexLookupMultipleIndex-6            3200          240           -92.50%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       1232          48            -96.10%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     4272          128           -97.00%
 2018-01-23 22:25:56 -05:00
Alexander Neumann
b0c6e53241 Fix calls to repo/backend.List() everywhere 2018-01-21 21:15:09 +01:00
Alexander Neumann
663e8284b2 init: Correct text (backend vs. repository) 
Closes #1565
 2018-01-20 09:51:49 +01:00
Alexander Neumann
0bdb131521 Remove SuspendSignalHandler 2018-01-17 23:14:47 +01:00
Alexander Neumann
97565d68a2 fuse/mount: Add option for snapshot template 2018-01-15 14:47:17 +01:00
Alexander Neumann
92316a9853 restore: Improve error message for more than one ID 2018-01-06 22:44:18 +01:00
Alexander Neumann
c912b38bf0 forget: Also run prune when only IDs are forgotten 2018-01-01 21:27:40 +01:00
Alexander Neumann
18ecd9df30 Improve limiting HTTP based backends 2017-12-29 12:43:49 +01:00
Gábor Lipták
32505c3916
Correct golint warning on return 2017-12-26 17:09:41 -05:00
Alexander Neumann
84e493beba b2: Warn when account ID or key is empty 
Closes #1490
 2017-12-19 21:12:38 +01:00
Alexander Neumann
2f26fb8834 Only print cache warning for terminals 2017-12-13 19:58:10 +01:00
Alexander Neumann
d3ebe1311f backup: Reject filenames with paths read from stdin 2017-12-13 19:55:53 +01:00
Alexander Neumann
3a5e040b7e Merge pull request #1439 from armhold/propagate-context 
replace ad-hoc context.TODO() with gopts.ctx
 2017-12-08 20:27:36 +01:00
George Armhold
1695c8ed55 use global context for check, debug, dump, find, forget, init, key, 
list, mount, tag, unlock commands

gh-1434
 2017-12-06 07:02:55 -05:00
George Armhold
366622f09a use global context for cat and ls 
gh-1434
 2017-12-06 05:48:39 -05:00
Alexander Neumann
0405e67f8b Change diff output slightly 2017-12-05 22:26:56 +01:00
Alexander Neumann
df350e1f6e Improve diff 2017-12-05 22:05:22 +01:00
Alexander Neumann
2424012d75 Add 'diff' command 2017-12-03 18:33:35 +01:00
Alexander Neumann
88607fc625 Correctly format all node types 2017-12-03 17:38:55 +01:00
Alexander Neumann
ad82781743 Reenable cache 
In 6341c7d72c, the cache was accidentally
disabled due to a bug, this commit reenables the cache.
 2017-12-03 15:52:57 +01:00
George Armhold
d886cb5c27 replace ad-hoc context.TODO() with gopts.ctx, so that cancellation 
can properly trickle down from cmd_*.

gh-1434
 2017-12-03 07:22:14 -05:00
Alexander Neumann
2579fe6b7b Ignore SIGPIPE 
Handling SIGPIPE made restic abort when a TCP connection was reset by a
server. This happened on DigitalOcean Spaces, which uses the s3 backend.
 2017-11-30 21:23:43 +01:00
Alexander Neumann
eddb8549ef backup: By default, do not save the access time 
This can be re-enabled with `--with-atime`.
 2017-11-28 21:31:35 +01:00
Alexander Neumann
2567026ccb Merge pull request #1447 from fawick/fixMultipleRejectIfPresent 
Fix multiple rejectIfPresent cancelling each other
 2017-11-28 19:20:29 +01:00
Alexander Neumann
0cc8fc6f18 Merge pull request #1431 from n0npax/master 
fix #1411
 2017-11-27 21:54:06 +01:00
Fabian Wickborn
27fadd2c6e Document approach for multiple reject-if-present test 2017-11-27 21:38:15 +01:00
Alexander Neumann
dc38265b54 Merge pull request #1436 from restic/remove-old-cache 
Remove old cache directories
 2017-11-27 21:37:05 +01:00
Fabian Wickborn
1ea518d5ef cmd/restic: Use a dedicated cache for each rejectIfPresent 2017-11-27 17:33:53 +01:00
Fabian Wickborn
901cd5edef cmd/restic: Add test for rejectIfPresent bug 
All RejectFuncs returned by rejectIfPresent share the same rejection
cache and hence might cancel each other out.
 2017-11-27 17:26:19 +01:00
Alexander Neumann
35a5307db3 Merge pull request #1415 from armhold/signals 
also handle SIGPIPE in cleanup routines
 2017-11-24 21:28:08 +01:00
Alexander Neumann
6341c7d72c cache: Add option to remove old cache dirs 2017-11-24 20:53:28 +01:00
Alexander Neumann
fa893ee477 cache: Add detection code for old cache dirs 2017-11-24 20:53:26 +01:00
n0npax
b12bba4e2a fix #1411 
replace panic during index save with fatal error
 2017-11-19 11:40:47 +01:00
George Armhold
0ed2401711 exit 1 if received signal is other than SIGINT 
send cleanup msg to stderr, not stdout
gh-1413
 2017-11-09 07:16:01 -05:00
George Armhold
c347431907 also handle SIGPIPE in cleanup routines 
fixes gh-1413: restic fails to cleanup locks when bash pipeline fails
 2017-11-05 07:14:27 -05:00
Björn Ketelaars
f63d7048f9 Fix test failing on OpenBSD #1307 
Is seems that #1307 is similar to #1087, which describes a comparable
observation on Apple's new filesystem. #1389 Has been committed and
fixes the problem on Darwin.

Although I'm not sure if the root cause of the issue is the same the
solution is similar for OpenBSD, and leverages #1389.
 2017-11-05 07:27:58 +01:00
Alexander Neumann
526aaca6f5 Merge pull request #1389 from wojas/fix-modtime-tests-on-apfs 
Fix modtime tests on APFS: allow 1us difference
 2017-10-29 14:20:40 +01:00
Konrad Wojas
f3016a9096 Darwin test fix: allow 1μs timestamp difference 
On Darwin, allow a 1μs difference in restored timestamps, because
macOS <10.13 cannot restore with nanosecond precision and the current
version of Go (1.9.2) does not yet support the new syscall required
for this. (#1087 #1389)
 2017-10-29 20:48:07 +08:00
Zlatko Čalušić
134f834c60 Use lowercase in messages from check/tag commands, too 2017-10-27 21:06:34 +02:00
Konrad Wojas
c5da90a5b7 Add --last flag to snapshots command 
Add --last flag to snapshots command to only show the last entry for any
(hostname, paths) combination.

This makes it easier to check when various paths were last backed up.
 2017-10-26 14:02:29 +08:00
Alexander Neumann
8d37b723ca Upadte vendored dependencies 2017-10-22 10:07:36 +02:00
TobyLL
9e3dde8ec7 Fix whitespace 2017-10-19 15:52:06 +01:00
TobyLL
9dba182e51 Ignore comments (lines starting with #) in the --files-from file 2017-10-19 15:48:22 +01:00
Alexander Neumann
beaf55f1fc dump: Allow absolute paths 2017-10-16 20:22:01 +02:00
Alexander Neumann
0e7281eb71 Merge pull request #1346 from fawick/cmdCatFile 
Add dump sub-command for dumping files to stdout
 2017-10-16 20:13:44 +02:00
Alexander Neumann
897c923cc9 Retry failed backend requests 2017-10-14 15:56:25 +02:00
Alexander Neumann
67193e3deb Improve error message when creating lock failed 2017-10-14 15:56:25 +02:00
Fabian Wickborn
d1d9c3f9d7 Renamed cmd_catfile.go to cmd_dump.go 2017-10-14 13:55:21 +02:00
Fabian Wickborn
cd5cbe0910 Rename debug dump related variable and run function 2017-10-14 13:55:00 +02:00
Fabian Wickborn
814e992c0b Rename subcommand catfile to dump 2017-10-14 11:34:04 +02:00
Fabian Wickborn
87d084e18c Add subcommand dump 2017-10-12 20:24:34 +02:00
Alexander Neumann
f2314b26ba Move 'dump' to 'debug dump' 2017-10-12 20:18:45 +02:00
Alexander Neumann
74dcf41f25 Disable 'dump' for non-debug builds 2017-10-12 20:14:48 +02:00
rmdashrf
32637a0328 Basic rate limiting implementation. 
Added `--limit-upload` and `--limit-download` flags to rate limit
backups and restores.
 2017-10-11 20:01:20 -07:00
Alexander Neumann
c5553ec855 Merge pull request #1276 from fawick/supply_ca_cert 
Add REST backend option to use CA root certificate
 2017-10-08 09:47:23 +02:00
Alexander Neumann
bb3ed54291 Merge pull request #1337 from mungomat/fuse_updateRepo 
fuse: mount and backup in parallel (#1330)
 2017-10-08 09:45:28 +02:00
Tobias Klein
02200acad0 fuse: mount and backup in parallel (#1330) 2017-10-07 11:48:48 +02:00
Alexander Neumann
1a2d190bdb Merge pull request #1334 from felix9/backup_tags 
backup should use latest parent regardless of tags
 2017-10-07 10:45:02 +02:00
Fabian Wickborn
f0f17db847 Cache evaluated directories in isExcludedByFile 
Fixes #1271.
 2017-10-06 22:11:22 +02:00
Felix Lee
7b44fd0f9d fix #1143 
Backup was choosing a parent snapshot that had the same tags, which
makes backup unnecessarily slow when there are newer snapshots with
different tags.

There's no reason parent has to have the same tags.

This change makes backup choose the newest snapshot instead.
 2017-10-05 11:48:26 -07:00
Fabian Wickborn
69a6e622d0 Add REST backend option to use CA root certificate 
Closes #1114.
 2017-10-04 22:14:10 +02:00
Alexander Neumann
5d1c1f721e Merge pull request #1324 from antonlindstrom/password-feedback 
Add password successful feedback
 2017-10-04 21:31:24 +02:00
Alexander Neumann
cdef55bb88 Merge pull request #1312 from hgfischer/issue-1119 
nit: please remove dot imports #1119
 2017-10-04 21:01:40 +02:00
Anton Lindstrom
a7baea0522 Output password successful on terminal stdout 
This removes the conditions that checks if the password is supplied
through environment variable or file and outputs password is successful
on terminal and when --quiet is not supplied.
 2017-10-04 14:55:04 +02:00
Anton Lindstrom
55e6003749 Add password successful feedback 
This adds some feedback when entering the password on the command line.
When the password is entered and supplied through stdin (and stdout is a
terminal) then the a message saying `password is correct` if correct is
printed.
 2017-10-04 13:45:05 +02:00
Alexander Neumann
657a1d75af snapshots: Add short ID to JSON output 
We're using the short ID in all output to users, so it should also be
included in the JSON output of the `snapshots` command.
 2017-10-03 15:24:09 +02:00
Anton Lindstrom
d46314648e Add explicit OK output to check command 
This adds additional output to the check command when no errors were
found. It means that when all checks have been completed, the following
output is displayed:

	No errors were found

The output is added to make sure that it is easier to understand that no
errors were found.

Full example output:

	Create exclusive lock for repository
	Load indexes
	Check all packs
	Check snapshots, trees and blobs
	No errors were found
 2017-10-03 12:57:51 +02:00
Alexander Neumann
e45011af57 Merge pull request #1304 from jannickfahlbusch/showAmountOfSnapshots 
Show the number of snapshots
 2017-10-03 12:41:47 +02:00
Alexander Neumann
fb09884893 Merge pull request #1311 from felix9/dump-no-trees 
"dump" command no longer has a "trees" option
 2017-10-03 12:40:24 +02:00
Alexander Neumann
d1eecafa63 Merge pull request #1314 from jannickfahlbusch/fixTypo 
Fix typo in comment
 2017-10-03 12:39:24 +02:00
Alexander Neumann
034b0b8040 forget: Run prune for manually forgotten snapshots 2017-10-03 11:56:13 +02:00
Jannick Fahlbusch
3dcacb3730
Fix typo in comment 2017-10-02 18:19:22 +02:00
Herbert
033589a66b Use rtest on these as well to keep codebase consistent 2017-10-02 17:48:08 +02:00
Jannick Fahlbusch
e46a647c45
Just print the number of snapshots 2017-10-02 17:05:59 +02:00
Herbert
3473c3f7b6 Remove all dot-imports 2017-10-02 15:06:39 +02:00
Jannick Fahlbusch
2f10e25738
Show the amount of snapshots 2017-10-01 17:14:54 +02:00
Felix Lee
555bd257bd dump command no longer has 'trees' option 2017-10-01 08:04:52 -07:00
Alexander Neumann
dad1c87afe Rework generation of manpages and completion files 
This commit removes the `manpages` and `autocomplet` commands and
replaces them with the more generic `generate` command. Also, zsh
completion file support was added.
 2017-09-26 13:12:12 +02:00
Alexander Neumann
56fccecd06 prune: Repack mixed pack files 2017-09-24 21:54:53 +02:00
Alexander Neumann
62ba9f1950 check: Disable cache by default 2017-09-24 21:54:53 +02:00
Alexander Neumann
610b676444 Automatically exclude current restic cache 2017-09-24 21:54:53 +02:00
Alexander Neumann
9be24a1c9f Add cache 
This commits adds rudimentary support for a cache directory, enabled by
default. The cache directory is created if it does not exist. The cache
is used if there's anything in it, newly created snapshot and index
files are written to the cache automatically.
 2017-09-24 21:54:53 +02:00
Fabian Wickborn
ea1ab96749 Always sort snapshots lists ascending by timestamp 
Fixes #1219.
 2017-09-24 13:01:13 +02:00
Alexander Neumann
9c6b7f688e Merge pull request #1270 from restic/sftp-allow-password-prompt 
sftp: Allow password entry
 2017-09-23 22:13:04 +02:00
Alexander Neumann
fb9729fdb9 sftp: Allow password entry 
This was a bit tricky: We start the ssh binary, but we want it to ignore
SIGINT. In contrast, restic itself should process SIGINT and clean up
properly. Before, we used `setsid()` to give the ssh process its own
process group, but that means it cannot prompt the user for a password
because the tty is gone.

So, now we're passing in two functions that ignore SIGINT just before
the ssh process is started and re-install it after start.
 2017-09-23 11:43:33 +02:00
Alexander Neumann
45a09c76ff Allow suspending SIGINT handler 2017-09-23 11:12:56 +02:00
Dmitriy Morozov
ae60188eb9 Add --compact option to forget 2017-09-22 16:32:59 -07:00
Alexander Neumann
e7de3b5f9d Merge pull request #1266 from JaCoB1123/document_hostname_option 
Add note about rescan to hostname flag (fixes #1221)
 2017-09-22 15:40:24 +02:00
Jan Bader
727fb6eabe Add note about rescan to hostname flag (fixes #1221) 2017-09-22 14:29:04 +02:00
Alexander Neumann
a4577769ae Merge pull request #1262 from mafgh/bytes 
fix duplicate bytes in prune output
 2017-09-22 11:59:10 +02:00
Stefan Völkel
7f927d4774 fix duplicate bytes in prune output 2017-09-22 10:07:24 +02:00
Tobias Klein
1a83a739dc fuse: added symlink 'latest' to snapshots-dir 2017-09-21 16:41:20 +02:00
Alexander Neumann
d7e644272f prune: Add plausibility check 2017-09-19 10:50:07 +02:00
Alexander Neumann
fca9a523e9 Merge pull request #1241 from restic/fix-timestamp-check 
Use .Equal() instead of == for time.Time
 2017-09-16 14:54:26 +02:00
Alexander Neumann
a60e751217 Use .Equal() instead of == for time.Time 
Closes #1238
 2017-09-15 20:57:35 +02:00
Alexander Neumann
81c5d8a968 Correct bash completion file path 2017-09-15 20:45:16 +02:00
Alexander Neumann
c8c01a5cae Merge pull request #1223 from mrzv/snapshots-compact 
Add --compact option to snapshots
 2017-09-13 14:02:31 +02:00
Alexander Neumann
0f25ef9498 Merge pull request #1230 from shayneholmes/update-short-help 
Update style in short help commands
 2017-09-11 22:25:31 +02:00
Alexander Neumann
36eaa22ed0 Merge pull request #1205 from mungomat/backup_time 
Backup time
 2017-09-11 19:01:41 +02:00
Shayne Holmes
9eb39cef05 Capitalize short help commands 
Unify existing Cobra help command, and git-help's style.
 2017-09-11 09:32:44 -07:00
Shayne Holmes
ee6150f67c Change short help messages to imperative voice 
Unify the output of `restic help`.
 2017-09-11 09:26:13 -07:00
Alexander Neumann
b1af544b1d Merge pull request #1224 from restic/improve-exclude-caches 
Allow multiple exclude-if-present
 2017-09-11 17:31:24 +02:00
Dmitriy Morozov
7a221f2473 Run changes through gofmt 2017-09-10 15:09:28 -07:00
Dmitriy Morozov
bdbe956c5c Add --compact option to snapshots 
With --compact, snapshots doesn't list directories and puts all tags on a single
line. This way each snapshot takes up exactly one line.
 2017-09-10 13:06:43 -07:00
Alexander Neumann
b6aeea425b Merge pull request #1196 from mungomat/forget_groupByTagsOnly 
forget: group-by-tags-only
 2017-09-10 20:52:15 +02:00
Alexander Neumann
a8aa4eb06c Rename parameter filename -> path 2017-09-10 20:28:21 +02:00
Alexander Neumann
0b2947dedb Add test for rejectByPattern 2017-09-10 15:31:58 +02:00
Alexander Neumann
47ddd34266 Improve test 2017-09-10 15:20:41 +02:00
Alexander Neumann
2fdca5d310 Improve debug message 2017-09-10 15:16:05 +02:00
Alexander Neumann
e5d4e33509 Improve error message if no targets specified 2017-09-10 15:14:11 +02:00
Alexander Neumann
e117f613af Move device test into new RejectFunc 2017-09-10 15:13:40 +02:00
Alexander Neumann
0dfdf02885 Rework pattern excludes 2017-09-10 14:34:28 +02:00
Alexander Neumann
4a0129fc2b Rename excludeByFile -> rejectIfPresent 2017-09-10 14:25:58 +02:00
Alexander Neumann
a9c705009c Move reject functions to new file 2017-09-10 14:25:25 +02:00
Alexander Neumann
d937ad8cf6 Rename FilenameCheck to RejectFunc 
We already have the opposite: pipe.SelectFunc(item string, fi
os.FileInfo) bool, so RejectFunc is a good name.
 2017-09-10 14:21:51 +02:00
Alexander Neumann
36e70228f2 Handle invalid key file 2017-09-10 10:55:01 +02:00
Alexander Neumann
c22c582546 Allow multiple exclude-if-present 2017-09-09 21:24:29 +02:00
Alexander Neumann
ea75509d6e Print warning for non-existing items 2017-09-09 21:12:41 +02:00
Tobias Klein
ed30bd7b76 gofmt 2017-09-09 18:19:19 +02:00
Tobias Klein
8f9ef4402b error in case of unknown grouping option 2017-09-09 15:55:37 +02:00
Tobias Klein
43ff971dfd new sub-option for backup: time 
New option to specify the timestamp for a backup
 2017-09-09 13:26:35 +02:00
Fabian Wickborn
dbda892542 Add option to exclude directories with a tagfile 
The option is named --exclude-if-present and accepts a parameter
filename[:content]. Directories are excluded and their contents is not
backed up if they contain a file with the specified name and,
optionally, that starts with the specified content. The tagfile itself
is never excluded.

There is also a shortcut --exclude-caches that works in the same way as
the likewise-named option of tar(1): Directories are recognized as cache
if they contain a file named "CACHEDIR.TAG.

Closes #317.
 2017-09-09 09:57:42 +02:00
Tobias Klein
1073bfba37 flexible grouping option for the forget-command 2017-09-06 20:14:18 +02:00
Alexander Neumann
fa2ee78a5c Merge pull request #1044 from lloeki/982-improve-restore 
Improve restore
 2017-09-04 21:51:12 +02:00
Tobias Klein
e4a5cdc5bc forget: group-by-tags-only 2017-09-03 17:11:25 +02:00
Alexander Neumann
0ee1650f82 Merge pull request #1191 from prattmic/profile 
debug: properly handle interrupted profiles
 2017-09-03 09:49:46 +02:00
Alexander Neumann
de8c64e767 Use deterministic date for man pages 2017-09-02 09:27:11 +02:00
Michael Pratt
e4c469c149 debug: properly handle interrupted profiles 
By default (i.e., without profile.NoShutdownHook), profile.Start listens
for SIGINT and will stop the profile and call os.Exit(0).

restic already listens for SIGINT and runs its own cleanup handlers
before calling os.Exit(0).

As is, these handlers are racing when an interrupt occurs, and in my
experience, restic tends to win the race, resulting in an unusable
profile.

Eliminate the race and properly stop profiles on interrupt by disabling
package profile's signal handler and instead stop the profile in a
restic cleanup handler.
 2017-08-28 22:03:26 -07:00
Ricardo Seriani
62ed776a8c Change key rm command to key remove 
Change key rm command to key remove, to follow manual and other commands
 2017-08-17 11:03:26 -03:00
Loic Nageleisen
f880ff21aa Fixing restore with excluded 
An exclude filter is basically a 'wildcard but foo', so even if a
childMayMatch, other children of a dir may not, therefore childMayMatch
does not matter, but we should not go down unless the dir is selected
for restore.
 2017-08-16 15:25:02 +02:00
Loic Nageleisen
4a36993c19 Smarter filter when children won't match 
This improves restore performance by several orders of magniture by not
going through the whole tree recursively when we can anticipate that no
match will ever occur.
 2017-08-16 15:25:02 +02:00
Alexander Neumann
b67c178672 Merge pull request #1149 from restic/azure-support 
Add Azure blob storage as backend
 2017-08-09 21:30:35 +02:00
Dipta Das
ba75a3884c Add Google Cloud Storage as backend 
Environment variables:
GOOGLE_PROJECT_ID=gcp-project-id
GOOGLE_APPLICATION_CREDENTIALS=path-to-json-file

Environment variables for test:
RESTIC_TEST_GS_PROJECT_ID=gcp-project-id
RESTIC_TEST_GS_APPLICATION_CREDENTIALS=path-to-json-file
RESTIC_TEST_GS_REPOSITORY=gs:us-central1/test-bucket

Init repository:
$ restic -r gs🪣/[prefix] init
 2017-08-06 21:47:55 +02:00
Alexander Neumann
d91d89eef6 azure: Create container if it does not exist 2017-08-06 21:47:04 +02:00
Dipta Das
3a85b6b7c6 Add Azure Blob Storage as backend 
Environment variables:
AZURE_ACCOUNT_NAME=storage-account-name
AZURE_ACCOUNT_KEY=storage-account-key

Environment variables for test:
RESTIC_TEST_AZURE_ACCOUNT_NAME=storage-account-name
RESTIC_TEST_AZURE_ACCOUNT_KEY=storage-account-key
RESTIC_TEST_AZURE_REPOSITORY=azure:restic-test-container

Init repository:
$ restic -r azure:container-name:/prefix/dir init
 2017-08-06 21:47:04 +02:00
Alexander Neumann
6bc43a4198 manpage: Remove auto gen tag from man page 2017-08-06 21:31:01 +02:00
Alexander Neumann
e348b3deeb manpage: Do not panic when no command is given 2017-08-06 21:01:49 +02:00
Alexander Neumann
739350fd8e backup: Do not print hostname in help text 
This is necessary so that the manpage generation is deterministic and we
can test if the man pages are up to date when the CI tests run.
 2017-08-05 12:05:53 +02:00
Alexander Neumann
a3ab17b470 Add 'manpage' command to generate manual pages 2017-08-05 10:57:01 +02:00
Johannes Kohnen
37aad2e3aa Close exclude files and check errors 2017-08-01 17:34:27 +00:00
donat
893bc9f777 Error message in case PasswordFile is missing 2017-07-27 14:23:08 +03:00
Michael Stapelberg
b89d3cc4d0 fix typo: explicitely → explicitly 2017-07-27 08:24:53 +02:00
Alexander Neumann
2e804511ca Merge pull request #1133 from middelink/fix-1132 
Force restic to ask the password when adding a key.
 2017-07-26 22:11:37 +02:00
Pauline Middelink
c95e2b009e Simpify cmd_backup and cmd_init now we have the password in gopts. 2017-07-24 23:32:55 +02:00
Pauline Middelink
d5615a67c8 Refactor password resolving. 
Instead of determining the password lazily during ReadPassword(), do so now in
cobra.PersistentPreRunE() so we can store the result in the globalOptions and
reuse/override when applicable without having to worry about the environment
or flag options interfering.
 2017-07-24 23:05:37 +02:00
Pauline Middelink
d9b9bbd4a8 Force restic to ask the password when adding a key. 
As `restic key add` uses the same `ReadPasswordTwice()` as the
rest of restic, it is sensitive to the environment variable
RESTIC_PASSWORD or --password-file= override.

When asking for the new key, temporary remove these 2 overrides, forcing
the password to be asked.
 2017-07-24 22:00:44 +02:00
Alexander Neumann
d780b1eede Rework withTestEnvironment 
Switch from a function passed as a parameter to a cleanup function,
which is also executed when the test function panics, so no temporary
directories are left behind.
 2017-07-24 21:32:34 +02:00
Pauline Middelink
1717391f6c Fuse testing leaves test mountpoint around. Move it under the testing tree which is removed after each test. 2017-07-24 20:33:39 +02:00
Alexander Neumann
23c903074c Move restic package to internal/restic 2017-07-24 17:43:32 +02:00
Alexander Neumann
94b27e8933 Fix paths for tests 2017-07-23 14:39:57 +02:00
Alexander Neumann
6caeff2408 Run goimports 2017-07-23 14:21:03 +02:00
Alexander Neumann
83d1a46526 Moves files 2017-07-23 14:19:13 +02:00
Alexander Neumann
b63399d606 Move things around for gb 
This moves all restic source files to src/, and all vendored
dependencies to vendor/src.
 2016-02-20 17:31:20 +01:00
Alexander Neumann
1287b307ac Add cleanup handler to restore terminal state 
Closes #402
 2016-02-13 18:29:26 +01:00
Alexander Neumann
4ec5050cbb backup: Hide status output for narrow terminals 
closes #428
 2016-02-10 17:28:48 +01:00
Rached Ben Mustapha
ba35ca522a Handle pack loading errors in rebuild-index 
Errors returned from backend.LoadAll() were not handled, leading to
these fatal errors from the unpacker trying to read the size from the end of
an empty buffer:

`seeking to read header length failed: bytes.Reader.Seek: negative position`

This change takes care of returning on error, as well as showing which pack
failed to load and validating pack integrity.
 2016-02-07 18:30:47 +00:00
Alexander Neumann
2c3a6a6fa9 cmd_rebuild_index: Remove calls to GetReader() 2016-01-24 00:42:04 +01:00
Alexander Neumann
61551b0591 cmd_cat: Remove calls to GetReader() 2016-01-24 00:42:04 +01:00
Alexander Neumann
f05a32509e Add "Test" prefix to backend test functions 2016-01-23 19:12:02 +01:00
Alexander Neumann
6ba56befad Abort fuse integration test on error 
Before, the fuse integration test was run and the tests were never
finished, because the testing code did not detect any errors when the
fusermount binary returned an error. This commit fixes it.
 2016-01-23 19:10:43 +01:00
Alexander Neumann
8b7bf8691d backend: Remove Get() 
This is the first commit that removes the (redundant) Get() method of
the backend interface. Get(x, y) is equivalent to GetReader(x, y, 0, 0).
 2016-01-23 13:13:05 +01:00
Alexander Neumann
1dd4c52a8b Add comments, configure flag library 2016-01-17 18:48:05 +01:00
Alexander Neumann
1922a4272c s3: fix usage 
Ignore error response for existing bucket, add more debug code.
 2016-01-17 18:48:05 +01:00
Alexander Neumann
7d5f8214cf use new backend open with config 2016-01-17 18:48:05 +01:00
Alexander Neumann
3ac1d0e4d1 add progress 2015-12-06 17:29:31 +01:00
Alexander Neumann
0e66a66bce read packs concurrently 2015-12-06 17:09:06 +01:00
Alexander Neumann
a1440c819b Make NoLock a global option 2015-11-13 12:33:59 +01:00
Alexander Neumann
6edb7e02d0 Allow restoring and listing without locking 2015-11-10 22:13:53 +01:00
Alexander Neumann
c79dcbd7c4 Allow checking a read-only repo with --no-lock 2015-11-10 21:41:22 +01:00
Alexander Neumann
acba82c8f7 Merge pull request #252 from restic/repack-blobs 
WIP: Repack blobs
 2015-11-09 20:57:57 +01:00
Alexander Neumann
1f9aea9905 fix test on windows, reset read-only flag 2015-11-08 22:41:45 +01:00
Alexander Neumann
742d69bf4d Add another test for optimizing unused blobs 2015-11-08 22:38:17 +01:00
Alexander Neumann
5776b8f01c remove ConvertIndex 2015-11-08 22:27:13 +01:00
Alexander Neumann
2e6eee991d Add test for optimize command with old indexes 2015-11-08 22:21:08 +01:00
Alexander Neumann
43e2c9837e check: removing orphaned packs is handled in 'optimize' 2015-11-08 21:24:51 +01:00
Alexander Neumann
c4fc7b52ae Add 'optimize' command that repacks blobs 2015-11-08 21:10:03 +01:00
Alexander Neumann
cd948b56ac cmd_check: Don't display unused blobs by default 2015-11-08 20:46:52 +01:00
Alexander Neumann
ea41a1045f Add integration test for error on readdirnames 2015-11-06 23:19:56 +01:00
Alexander Neumann
18d7f7f835 Read password from stdin if terminal is not a tty 2015-11-04 22:20:58 +01:00
Alexander Neumann
1fc0d78913 Refactor Index.Store() to take a PackedBlob 2015-11-02 19:07:03 +01:00
Alexander Neumann
50c2f2e87f cmd_cat: allow dumping raw tree blobs 2015-11-02 17:36:04 +01:00
Alexander Neumann
fccde030d5 Refactor Index.Lookup() to return struct PackedBlob 2015-10-31 14:47:42 +01:00
Alexander Neumann
b5976474dd backup: add debug output for excluded files/dirs 2015-10-27 22:34:30 +01:00
Alexander Neumann
00e05ae3c9 bugfix: close pack files after reading the header 2015-10-27 20:39:52 +01:00
Alexander Neumann
74cd134b54 rebuild index: remember already stored blobs 2015-10-25 22:34:22 +01:00
Alexander Neumann
734ae7fcb8 Add test for corner case 
It was observed that a restic repository still contained overlapping
indexes after `rebuild-index` has been called. This is caused by
instantly forgetting that blobs have already been saved once a full
index has been written during index rebuilding.

This commit adds a (failing) test that shows the behaviour.
 2015-10-25 21:51:57 +01:00
Alexander Neumann
7b8e42a763 Silence rebuild-index tests 2015-10-25 21:51:46 +01:00
Alexander Neumann
efbce9f0fa rebuild-index: handle not yet indexed packs 2015-10-25 18:07:51 +01:00
Alexander Neumann
88849c06a6 rebuild-index: Refactor a bit 2015-10-25 17:53:02 +01:00
Alexander Neumann
6aed9f268b Add command rebuild-index 2015-10-25 17:24:52 +01:00
Alexander Neumann
96ecc26507 Let the checker return a list of hints along with errors 2015-10-25 16:26:50 +01:00
Alexander Neumann
650eab6a0e Fix typo in dump usage 2015-10-25 13:19:35 +01:00
Alexander Neumann
7ab9915859 Fix 'dump' command 2015-10-12 22:42:31 +02:00
Alexander Neumann
86fcd170f6 Add and use MasterIndex 2015-10-12 22:34:12 +02:00
Alexander Neumann
de2c20be84 Dump individual indexes 2015-10-12 20:53:07 +02:00
Alexander Neumann
db85ab8aa0 Use the correct channel for sending errors 2015-10-11 19:13:45 +02:00
Chapuis Bertil
c765688779 find command integration tests 2015-08-28 19:31:05 +02:00
Chapuis Bertil
d4686ebcc5 Load the index and search subtree 2015-08-27 23:21:44 +02:00
Alexander Neumann
fcb769fa3b Merge pull request #262 from restic/fix-tests 
Fix tests
 2015-08-18 22:45:50 +02:00
Alexander Neumann
1a47ea4ab8 test helpers: add RemoveAll and ResetReadOnly 
This is mainly needed in Windows, where files and dirs cannot be
removed unless they are writeable.
 2015-08-18 21:40:40 +02:00
klauspost
a3570af500 Create separate disEntry.equals for Windows. 
Windows does not have UID/GID the same way as unix,
so we don't attempt to compare them.
 2015-08-17 11:10:12 +02:00
klauspost
d5dab39a4a Disable FUSE test on Windows. 2015-08-17 11:02:04 +02:00
Klaus Post
4dc746dac2 Change repository type detection to first check if path is a directory that exists. 
The method of determining if a repository exists doesn't work on Windows, since
the "url.Scheme" will contain the drive letter - "c" in "c:\backup",
so as a first step we check if the URL can be opened as a file,
and if so, we assume it is a 'local' type repository.
 2015-08-14 15:39:16 +02:00
Klaus Post
fb3778abb6 Disable mount command on Windows. 
FUSE does not support Windows, so we disable it on Windows.
 2015-08-14 15:33:11 +02:00
Alexander Neumann
dde0fd8421 Add option to use root as owner of files/dirs 2015-07-26 20:56:27 +02:00
Alexander Neumann
05e2afba0b Merge pull request #249 from restic/reduce-fuse-memory-usage 
Reduce memory usage for fuse mount
 2015-07-26 19:20:13 +02:00
Alexander Neumann
b85927576b Address code review comments 2015-07-26 18:01:20 +02:00
Alexander Neumann
d1629e1e4e fuse: move to top level 2015-07-26 17:20:26 +02:00
Alexander Neumann
bd746a0425 fuse: refactor and add tests for fuse.file 2015-07-26 16:43:42 +02:00
Alexander Neumann
9753c37e31 Remove dead code 
This removes dead code (functions that aren't called) detected with
`deadcode`.
 2015-07-26 14:56:34 +02:00
Alexander Neumann
55ddd5317d Reduce memory usage for fuse mount 
This changes `repository.LoadBlob()` so that a destination buffer must
be provided, which enables the fuse code to use a buffer from a
`sync.Pool`. In addition, release the buffers when the file is closed.
At the moment, the max memory usage is defined by the max file size that
is read in one go (e.g. with `cat`). It could be further optimized by
implementing a LRU caching scheme.
 2015-07-26 14:25:59 +02:00
Alexander Neumann
054256468b Fix message display for parent snapshot id 2015-07-25 18:10:23 +02:00
Alexander Neumann
5cdcc99eba Use array instead of hash for backend.ID 
Since backend.ID is always a slice of constant length, use an array
instead of a slice. Mostly, arrays behave as slices, except that an
array cannot be nil, so use `*backend.ID` insteaf of `backend.ID` in
places where the absence of an ID is possible (e.g. for the Subtree of a
Node, which may not present when the node is a file node).

This change allows to directly use backend.ID as the the key for a map,
so that arbitrary data structures (e.g. a Set implemented as a
map[backend.ID]struct{}) can easily be formed.
 2015-07-25 18:01:57 +02:00
Alexander Neumann
deaca157fe restore: ignore errors 
Also add a test for restoring a file that is owned by root. Closes #244.
 2015-07-25 12:59:59 +02:00
Alexander Neumann
cc34401152 Record exclude patterns in snapshot 
This adds the exclude patterns used to create a backup in the snapshot,
so we can later compute statistics (like git does) on the data
structure, e.g. added/removed files etc. For that, we need the exclude
pattern.
 2015-07-22 22:43:32 +02:00
Alexander Neumann
9911d46996 fuse: rename child/children -> node/items 2015-07-21 21:34:59 +02:00
Alexander Neumann
3f4b5b8d48 fuse/mount: display symlinks properly 2015-07-21 21:25:05 +02:00
Alexander Neumann
1ac72b8813 Fix style issue 2015-07-21 21:24:06 +02:00
Alexander Neumann
99dae57b4f fuse: use node.Type instead of node.Mode 2015-07-21 21:23:40 +02:00
Alexander Neumann
aeb5a694d3 Merge pull request #238 from restic/fix-checks-on-bsd 
Refactor skipping symlink timestamp checks on *bsd and darwin
 2015-07-21 19:47:33 +02:00
Alexander Neumann
258b6a77ee Refactor skipping symlink ModTime checks, add OpenBSD 2015-07-20 21:29:21 +02:00
Alexander Neumann
bd3ce5d4a3 Fix tests on freebsd 2015-07-20 21:08:29 +02:00
Alexander Neumann
1da89253cf Add include filter for restore 
Include and exclude filter are mutually exclusive.
 2015-07-20 19:21:01 +02:00
Alexander Neumann
c0337a2675 Add exclude filter to restorer and 'restore' command 2015-07-20 19:21:01 +02:00
Alexander Neumann
7fd52f9f57 Add exclude filter to archiver and 'backup' command 2015-07-20 19:21:01 +02:00
Matthieu Rakotojaona
3767eb2675 Unmount and remove directory for mount in tests 2015-07-19 23:04:18 +02:00
Matthieu Rakotojaona
e44716381c Unmount when closing application from cli 2015-07-19 23:04:18 +02:00
Matthieu Rakotojaona
eadfcd3f9e Add waitForMount for OSX 2015-07-19 23:04:18 +02:00
Matthieu Rakotojaona
fe6f1c01f3 Make inodeFromBackendId more explicit 2015-07-19 23:04:18 +02:00
Matthieu Rakotojaona
a4d122e5ae Cleanup mount after test 2015-07-19 23:04:18 +02:00
Matthieu Rakotojaona
d7888d4dd5 Fix checks in fuse tests 2015-07-19 23:04:18 +02:00
Matthieu Rakotojaona
ca6b7ec533 Add Index.LookupSize 2015-07-19 23:04:18 +02:00
Matthieu Rakotojaona
b1426826cc Extract fuse structs 2015-07-19 23:03:33 +02:00
Matthieu Rakotojaona
0606b9884e Make file.Read more intelligible 2015-07-19 23:03:33 +02:00
Matthieu Rakotojaona
3731a94367 Use functions to create names 2015-07-19 23:03:32 +02:00
Matthieu Rakotojaona
a016f82051 Fix coding style 2015-07-19 23:03:32 +02:00
Matthieu Rakotojaona
e654a9659c Protect the snapshots cache with a RWMutex 2015-07-19 23:03:32 +02:00
Matthieu Rakotojaona
0e7c1668d5 Make mountpoint readable by owner only 2015-07-19 23:03:32 +02:00
Matthieu Rakotojaona
9ff98d34ef Add build and test instructions 
* Don't build on openbsd
* Don't test fuse on travis
 2015-07-19 23:03:32 +02:00
Matthieu Rakotojaona
1f79a19293 Comment the go trick about static verification of a struct implementing an interface 2015-07-19 23:03:32 +02:00
Matthieu Rakotojaona
a8cd74ba7e Cache known snapshots instead of re-traversing the repository every time 2015-07-19 23:03:32 +02:00
Matthieu Rakotojaona
c9b3eebc09 Check that mountpoints are created 2015-07-19 23:03:32 +02:00
Matthieu Rakotojaona
25b090a074 Add integration tests for mount command 2015-07-19 23:03:32 +02:00
Matthieu Rakotojaona
414ade5b3f Signal readiness of mount to potential callers 2015-07-19 23:03:32 +02:00
Matthieu Rakotojaona
a6ce7d9845 Use more restic-y idioms 2015-07-19 23:03:32 +02:00
Matthieu Rakotojaona
3a82612244 Update bazil.org/fuse version 2015-07-19 23:03:32 +02:00
Matthieu Rakotojaona
32a321e51e Add simple mount command 2015-07-19 23:03:32 +02:00
Alexander Neumann
76817da922 Run cleanup handlers in main function 2015-07-19 17:57:18 +02:00
Alexander Neumann
dac89bf544 Allow more than one cleanup handler 2015-07-19 17:50:55 +02:00
Alexander Neumann
159b9e80c4 cmd/restic: implement lock refresh 2015-07-12 22:10:01 +02:00
Alexander Neumann
d36f07c6eb checker: add option to remove orphaned packs 2015-07-12 17:14:10 +02:00
Alexander Neumann
af02c323cd checker: use channel of error instead of slice 2015-07-12 01:46:10 +02:00
Alexander Neumann
04f172b38d checker: Add checker and tests 2015-07-11 16:27:41 +02:00
Alexander Neumann
54c4c29a89 Add checker and command 'check' to replace 'fsck' 2015-07-11 16:27:38 +02:00
Alexander Neumann
389ec9b101 Add tests for restore filter 2015-07-09 20:12:31 +02:00
Alexander Neumann
3e0a97fb13 Fix restore filter 
Internally rename restorer.Filter -> restorer.SelectForRestore to make
semantic clear.

In addition, swap parameters to filepath.Match() so that the pattern can
really be matched.

Limitation: The filter only works on the filename, not on any path
component, e.g. '*.go' selects all go files, 'subdir/foo*' doesn't
select anything.

Fixes #202.
 2015-07-09 20:12:31 +02:00
Florian Weingarten
c2d5a9209e fix darwin integration tests 2015-07-09 11:13:40 -04:00
Alexander Neumann
57eb1a4eea cmd_backup: filter non-existing target files/dirs 2015-07-06 23:02:16 +02:00
Alexander Neumann
c8bb688e26 Command snapshots: do not list first dir twice 2015-07-06 22:26:31 +02:00
Alexander Neumann
03983f0907 repository: Rename Load() -> LoadAndDecrypt() 2015-07-05 11:07:10 +02:00
Alexander Neumann
281eea9c05 sftp: Use own process group for ssh subprocess 2015-07-05 11:07:10 +02:00
Alexander Neumann
e657287eac cmd/restic: Add command unlock, improve error message 2015-06-28 13:52:23 +02:00
Alexander Neumann
47212dde8c cmd/restic: Do not require exclusive lock for listing keys 2015-06-28 13:52:22 +02:00
Alexander Neumann
13e9a35f96 cmd/restic: Add lock handling, interrupt cleanup 2015-06-28 13:52:22 +02:00
Alexander Neumann
65a0def949 cmd/restic: Add locks to commands 2015-06-28 13:52:22 +02:00
Alexander Neumann
7d2699b429 cmd/restic: Rename variable, no functional changes 2015-06-28 13:52:22 +02:00
Alexander Neumann
d51fd436b5 Add locking functions 2015-06-28 13:52:22 +02:00
Florian Weingarten
f8c30556d3 Ask for password after create repo succeeded 2015-06-26 13:58:43 -04:00
Alexander Neumann
9e9177ab73 Add build.go 2015-06-24 20:29:58 +02:00
Alexander Neumann
d5020ac109 Fix debug functions 2015-06-24 20:00:01 +02:00
Alexander Neumann
6c2d6bfd11 Merge pull request #184 from howeyc/backend-s3 
Add Amazon S3 backend.
 2015-06-23 20:14:47 +02:00
Alexander Neumann
0005191d71 Remove dirdiff and gentestdata 2015-06-21 17:40:22 +02:00
Alexander Neumann
5ae04b6834 Add last integration tests, remove testsuite 2015-06-21 17:40:22 +02:00
Alexander Neumann
43d4558a90 Add test for backing up non-existing directories 2015-06-21 17:40:22 +02:00
Alexander Neumann
675f341b6d Output warnings/errors to configurable writer 2015-06-21 17:40:22 +02:00
Alexander Neumann
cfaf8ab8a6 Add integration test for key handling 2015-06-21 17:40:22 +02:00
Alexander Neumann
a99a460b32 Fix integration tests 2015-06-21 17:40:22 +02:00
Alexander Neumann
a43733d552 Introduce Verbosef 2015-06-21 17:40:22 +02:00
Alexander Neumann
4388474cdc Restructure cmd/restic, no functional changes 2015-06-21 17:40:22 +02:00
Alexander Neumann
2fa259816b rename opts to mainOpts 2015-06-21 17:40:22 +02:00
Alexander Neumann
1216ded14b Add integration test for key command 2015-06-21 17:40:22 +02:00
Alexander Neumann
e2563b3eca Fix comments 2015-06-21 17:40:22 +02:00
Alexander Neumann
a176b1b5a6 Add more integration tests 2015-06-21 17:40:22 +02:00
Alexander Neumann
9853fbcf48 Remove more flags from tests 2015-06-21 17:40:21 +02:00
Alexander Neumann
3a65f27c3f Automatically find fixtures for integration test 2015-06-21 17:40:21 +02:00
Alexander Neumann
030f08a410 Remove flags from tests 2015-06-21 17:40:21 +02:00
Alexander Neumann
d8d09b6d69 Fix restic configuration for integration tests 2015-06-21 17:40:21 +02:00
Alexander Neumann
12677b4f8a Use flag instead of build tag to run integration tests 2015-06-21 17:40:21 +02:00
Alexander Neumann
7c107acf0b More integration tests 2015-06-21 17:40:21 +02:00
Alexander Neumann
954025e399 Add integration test with the go testing framework 2015-06-21 17:40:21 +02:00
Chris Howey
47186806cf Add Amazon S3 backend. 2015-06-14 08:28:50 -05:00
Alexander Neumann
9c375ea382 Refactor backup a bit 2015-06-07 19:08:37 +02:00
Alexander Neumann
fd80499954 Refactor terminal recognition, add --quiet parameter 2015-06-07 18:30:11 +02:00
Alexander Neumann
f5537e7a0e Refactor configuration of cache dir and repository 2015-06-07 18:15:03 +02:00
Alexander Neumann
848793df84 Remove unused file 2015-06-07 18:13:22 +02:00
Alexander Neumann
0314d2b0c3 fsck: Fix check for orphaned blobs 2015-05-17 23:41:53 +02:00
Alexander Neumann
df1e3e03e9 Add 'dump' command 
'dump' writes internal data structes pretty-printed as JSON to stdout.
This was done to debug fsck error messages.

In contrast to the 'cat' command, this one prints the data structures as
there are interpreted by restic, not as they are stored in the
repository. This means that only the merged index from all the index
files is printed out.

This is meant for debugging only, it's compiled only when the "debug"
tag is active.
 2015-05-17 23:15:08 +02:00
Alexander Neumann
af381c2ab0 Remove Repository.Remove() 2015-05-17 20:58:22 +02:00
Alexander Neumann
eb6dfcf58c Remove Repository.Test() 2015-05-17 20:58:22 +02:00
Alexander Neumann
6e38a8a033 Move FindSnapshot, make Repository.List() return IDs 2015-05-17 20:58:22 +02:00
Florian Weingarten
bace4607bf Fix ModTime for directories 2015-05-13 23:11:31 -04:00
Alexander Neumann
95536e8a21 Rename 'Repo' -> 'Repository' 2015-05-09 23:59:58 +02:00
Alexander Neumann
232c472836 Move package 'repo' to package 'repository' 2015-05-09 23:52:03 +02:00
Alexander Neumann
b2dcdf00e3 Merge pull request #178 from restic/refactor-server 
Rename Server -> Repository
 2015-05-09 22:59:02 +02:00
Alexander Neumann
ae21938f3e Rename 'Repository' -> Repo 2015-05-09 17:41:28 +02:00
Alexander Neumann
cd4c56b927 wip 2015-05-09 13:54:47 +02:00
Alexander Neumann
d9b5832034 Rename variables 2015-05-09 13:47:21 +02:00
Alexander Neumann
87ebf12945 Rename 'Server' to 'Repository' 2015-05-09 13:25:52 +02:00
Alexander Neumann
8be9e95d20 Rename package 'server' to 'repo' 2015-05-09 13:24:15 +02:00
Alexander Neumann
861e1b9910 Add doc stubs to all the packages 2015-05-09 13:00:01 +02:00
Chris Howey
e8615eb593 Fix ls command 
Added missing loadindex call. Also fixed the recursive call to
use the subtree to stop infinte recursion.
 2015-05-05 20:50:10 -05:00
Alexander Neumann
bdbb3ab329 Remove pools.go 2015-05-05 00:14:07 +02:00
Alexander Neumann
35af933f24 server: Rename CreateMasterKey() to Init() 2015-05-04 20:40:17 +02:00
Alexander Neumann
1213d87b1a server: Only save crypto.Key 
At the moment, the server doesn't need the full server.Key (master and
user key), just the master key.
 2015-05-03 18:56:20 +02:00
Alexander Neumann
765e3dc66f restic: Add 'cat config' command 2015-05-03 18:56:20 +02:00
Alexander Neumann
d4bf5bb279 server: Add config 2015-05-03 18:56:20 +02:00
Alexander Neumann
bebb08ee7e server/key: Rename CreateKey -> CreateMasterKey 2015-05-03 18:56:20 +02:00
Alexander Neumann
9b54fd7bdb server: rename LoadJSONEncrypted -> LoadJSONUnpacked 2015-05-03 18:56:20 +02:00
Alexander Neumann
b3a75ecc1c Correct usage for backup cmd 2015-05-02 02:05:09 +02:00
Florian Weingarten
0d9360a815 more cleanup 2015-04-29 22:33:34 -04:00
Alexander Neumann
24fc5d343f Remove leftover comment 2015-04-30 00:25:00 +02:00
Alexander Neumann
29ead95c96 Adapt fsck command to packed blobs (unfinished) 2015-04-29 23:27:36 +02:00
Alexander Neumann
60a0fe8349 Major restructure, bundle blobs 
This commit also breaks the repository format.
 2015-04-29 23:27:32 +02:00
Alexander Neumann
d19b23d4f1 Move Server and Key to new sub-package 2015-04-26 14:46:15 +02:00
Alexander Neumann
6871e1bb58 Merge remote-tracking branch 'fw42/fix_nan_and_inf' 2015-04-26 12:09:14 +02:00
Florian Weingarten
61b1a5b1de Count errors separately and not as files that are "done" 2015-04-26 01:54:35 +00:00
Florian Weingarten
6c776468ab Fix floating point presentation 2015-04-25 13:20:41 -04:00
Alexander Neumann
87a1946417 Merge remote-tracking branch 'fw42/rename_change_to_passwd' 2015-04-25 11:11:27 +02:00
Alexander Neumann
fd472cb59d Use short ID for init 2015-04-25 11:07:51 +02:00
Alexander Neumann
19486987b0 Merge remote-tracking branch 'fw42/mixed_caps' 2015-04-25 11:06:13 +02:00
Florian Weingarten
dead2b7e93 rename "change" to "passwd" 2015-04-25 02:44:26 -04:00
Florian Weingarten
170d147427 Consistent writing 2015-04-24 19:43:27 -04:00
Florian Weingarten
ebae478631 Prefer mixedCaps over underscore 2015-04-24 19:39:32 -04:00
Florian Weingarten
441f4deb66 remember your password 2015-04-24 19:08:05 -04:00
Matthieu Rakotojaona
1d0f8fa80e Take last snapshot as parent by default and allow force re-reading 2015-04-07 22:03:50 +02:00
Alexander Neumann
4d24f2eab3 Correct plen for snapshots 2015-03-28 16:16:19 +01:00
Alexander Neumann
223108c6ae Remove unused vars and const 2015-03-28 15:28:08 +01:00
Alexander Neumann
f157f775da Add checks for Server.Load(), use Blob for load 2015-03-28 15:07:08 +01:00
Alexander Neumann
5e69788eac Refactor backends 2015-03-28 14:36:50 +01:00
Alexander Neumann
53ad706c6d Remove unused 'ram' tool 2015-03-22 21:03:25 +01:00
Alexander Neumann
0ed2a066a0 Document crypto and master key JSON struct 2015-03-22 21:03:01 +01:00
Alexander Neumann
5257c54585 Correct string for catting data blobs 2015-03-22 15:42:46 +01:00
Alexander Neumann
1c0e76ccd6 Bugfix: Backup changed data in incremental mode 2015-03-22 15:09:54 +01:00
Alexander Neumann
cfa2229bc0 Refactor cache refresh for blobs, add progress 2015-03-22 15:09:50 +01:00
Alexander Neumann
865d315ad6 Shorten status output if terminal is small 
Closes #107
 2015-03-16 20:48:29 +01:00
Alexander Neumann
489cc654e8 Fix ETA display 
Closes #111
 2015-03-16 20:20:53 +01:00
Alexander Neumann
b17840c6ee Refactor cache and 'cache' command 2015-03-14 12:57:29 +01:00
Alexander Neumann
e1fc17aeb1 Implement per-repository cache 2015-03-14 12:16:28 +01:00
Alexander Neumann
14298fe232 Cache blobs for each snapshot 2015-03-09 23:50:32 +01:00
Alexander Neumann
798ac97a52 Re-enable dedup, add test (closes #95) 2015-03-07 12:05:33 +01:00
Alexander Neumann
f167366be5 Add walk for trees, restructure 2015-03-02 19:58:06 +01:00
Alexander Neumann
c8be54564f Add local cache 2015-03-02 10:04:01 +01:00
Alexander Neumann
a59b0ec1f6 Add progress report for loading blobs 2015-02-21 15:32:48 +01:00
Alexander Neumann
58cded6b75 Refactor progress reporting 2015-02-21 14:25:06 +01:00
Alexander Neumann
a0fea201d9 Reenable Preloading 2015-02-18 23:10:59 +01:00
Alexander Neumann
3bb2aba141 Remove test files 2015-02-18 21:50:59 +01:00
Alexander Neumann
65aae5f8c1 Disable preloading for now 2015-02-17 23:38:15 +01:00
Alexander Neumann
b6f25aa690 Remove Each(), add basic stats 2015-02-17 23:11:56 +01:00
Alexander Neumann
4bb7f2f2ed Add Preloader for blobs 2015-02-17 22:39:44 +01:00
Alexander Neumann
4d1e7b9f24 Change parameters of LoadTree() 2015-02-17 22:37:34 +01:00
Alexander Neumann
d4cfa7b797 Remove scanner (not needed any more) 2015-02-16 23:47:14 +01:00
Alexander Neumann
f8f8107d55 wip 2015-02-15 15:17:24 +01:00
Alexander Neumann
c884704bce Refactor pool stats 2015-02-08 13:46:51 +01:00
Alexander Neumann
57b8373bc9 Progress: Rename functions, fix documentation 2015-02-07 15:27:09 +01:00
Alexander Neumann
316e761f27 fsck: Check minimal set of blobs 2015-02-04 22:33:48 +01:00
Alexander Neumann
84ad6cf4d2 Return error on backup 2015-02-03 22:05:46 +01:00
Alexander Neumann
fb95f02af6 Refactor testsuite 2015-01-17 16:32:24 +01:00
Alexander Neumann
25a214809b Refactor debug into debug module 2015-01-17 16:32:13 +01:00
Alexander Neumann
bcb19c811b Add 'version' command 2015-01-16 21:26:33 +01:00
Alexander Neumann
7af11eaaae Refactor tests, integrate debug break points 2015-01-14 17:32:37 +01:00
Alexander Neumann
203a911de9 Refactor repository structure 
Merge Map data type into Tree.
 2015-01-14 16:34:30 +01:00
Alexander Neumann
33bcf31bae Fix incremental backup 
Copying blobs (for unchanged files and subtrees) from old BlobList to
new BlobList was missing
 2015-01-05 21:40:43 +01:00
Alexander Neumann
4b070358ef fsck: more verbose error messages 2015-01-05 20:38:49 +01:00
Alexander Neumann
6eb969a492 Update modified files, store error message 2015-01-04 23:25:13 +01:00
Alexander Neumann
e543f5926c Refactor Archiver and backup command 
Improve incremental backup by first scanning the tree, loading the old
tree and afterwards comparing both trees in memory.
 2015-01-04 22:39:30 +01:00
Alexander Neumann
fe231af7fc Remove error from return value of NewContentHandler 2015-01-04 20:07:30 +01:00
Alexander Neumann
4b70bba588 Split Scanner from Archiver, refactor Progress 2015-01-04 18:23:00 +01:00
Alexander Neumann
a93bc3c991 Rename functions, remove code for compare 
* Archiver: Rename loadTree -> scan and LoadTree -> Scan
 * Archiver: Remove code to compare against an old snapshot/tree,
   the current implementation was really slow
 2015-01-04 16:13:50 +01:00
Alexander Neumann
5b82475d74 Merge branch 'restore-pattern', closes #69 2015-01-02 23:04:29 +01:00
Sebastian Schmidt
b1dbc6f062 Support for pattern in restore command 
This just matches the passed pattern against the full source path with
filepath.Match which, in contrast go filepath.Glob, doesn't match the
directory separator with '*' and is not terribly useful that way.
Someone should replace that by a more sophisticated matcher.
 2015-01-02 23:03:07 +01:00
Alexander Neumann
2e28b88e3a Write prompt to stderr instead of stdout 2015-01-02 22:36:53 +01:00
Sebastian Schmidt
190b1cdcc7 Fix 'ls' command 
CmdLs needs to implement flags.Commander, so change (CmdLs) Execute to
only take a []string to get actually called.
 2015-01-01 11:30:30 +01:00
Alexander Neumann
1944ab13d4 Add dangling blob detection and removal to 'fsck' 2014-12-22 14:46:54 +01:00
Alexander Neumann
a981141e49 Rename Table.Print -> Table.Write 2014-12-22 11:21:14 +01:00
Alexander Neumann
5431b025a3 Reduce code duplication in key handling 2014-12-21 18:16:22 +01:00
Alexander Neumann
ef41a77aff Remove explicit key handling 2014-12-21 18:10:19 +01:00
Alexander Neumann
cc147c002e Introduce type Server 2014-12-21 17:37:29 +01:00
Alexander Neumann
0e1045301a Optimize 'cat' and 'snapshots' commands 2014-12-21 17:29:37 +01:00
Alexander Neumann
879abd0d12 Add time to command 'find' 2014-12-07 17:23:00 +01:00
Alexander Neumann
28bb061ad3 Refactor commands 2014-12-07 16:32:23 +01:00
Alexander Neumann
b3deca33a9 Add command 'find' 2014-12-07 14:44:01 +01:00
Alexander Neumann
988d18bb1d Improve output 2014-12-07 14:26:32 +01:00
Alexander Neumann
202984a93f Rename snapshot.Content -> snapshot.Tree 2014-12-07 14:14:50 +01:00
Alexander Neumann
45e40eb27a Correctly encode non utf8 node names 2014-12-07 13:06:35 +01:00
Alexander Neumann
fa94d408f3 Remove old khepri binary 2014-12-06 17:59:21 +01:00
Alexander Neumann
e2fea0d088 Rename khepri -> restic 2014-12-05 21:45:49 +01:00
Alexander Neumann
0ac1716463 Fix error message for invalid snapshot id in fsck 2014-11-30 22:49:41 +01:00
Alexander Neumann
172b7bf123 Move command init into cmd_* files 2014-11-30 22:39:58 +01:00
Alexander Neumann
bb8c52a974 Add 'fsck' command 2014-11-30 22:35:42 +01:00
Alexander Neumann
3325fa07ea Check timestamps for files 2014-11-30 22:34:21 +01:00
Alexander Neumann
65c9a78e95 Remove extra linefeed from table header in 'snapshots' 2014-11-28 00:01:56 +01:00
Alexander Neumann
3e2c9dc267 Show currently used key in 'key list' 2014-11-27 23:26:19 +01:00
Alexander Neumann
3b243ed1dd Add command "key change" to change repository password 2014-11-25 23:23:09 +01:00
Alexander Neumann
43ccee3b08 Add command "key rm" 2014-11-25 23:18:02 +01:00
Alexander Neumann
f7fcd7c04c Add command "key add" 2014-11-25 23:07:00 +01:00
Alexander Neumann
d0a63cd49f Add command "key list" 2014-11-25 22:52:53 +01:00
Alexander Neumann
ad7e1d043f Add simple ascii table implementation 2014-11-25 22:39:35 +01:00
Alexander Neumann
bb62218743 Add functions to compute unique prefix length 2014-11-24 22:09:22 +01:00
Alexander Neumann
2c5d07a571 List snapshots, accept snapshot id prefix 
Example:

    $ ./khepri snapshots
    ID        Date                 Source      Directory
    --------------------------------------------------------------------------------
    fa31d65b  2014-11-24 19:45:11  kasimir     /home/user/testdata
    20bdc140  2014-11-24 20:00:47  kasimir     /home/user/testdata
    326cb59d  2014-11-24 20:01:40  kasimir     /home/user/testdata
    20ff988b  2014-11-24 20:35:35  kasimir     /home/user
 2014-11-24 21:12:32 +01:00
Alexander Neumann
339c63befc Add "list maps" command 2014-11-23 23:29:29 +01:00
Alexander Neumann
725e5bfca5 Require repository 2014-11-23 23:19:47 +01:00
Alexander Neumann
33ce4e33aa Store maps in new subdir "maps" 2014-11-23 22:26:01 +01:00
Alexander Neumann
f95788ed90 Limit memory usage, add several sync.Pool 2014-11-23 21:26:20 +01:00
Alexander Neumann
575635753e Correct wildcard in Makefile 2014-11-23 15:15:11 +01:00
Alexander Neumann
a780462b80 Change bps and eta calculation to prevent overflow 2014-11-23 14:56:41 +01:00
Alexander Neumann
6d0a6844e2 Add space between number and unit 2014-11-23 14:54:37 +01:00
Alexander Neumann
3f0ea78b86 Clear line with ANSI escape on update 2014-11-23 14:34:18 +01:00
Alexander Neumann
7e26567b8b Pretty status for backup 2014-11-23 13:51:15 +01:00
Alexander Neumann
21df0e50e5 Refactor stats 
* channel instead of callback func
 * cumulate Stats for slow receivers
 2014-11-23 13:51:02 +01:00
Alexander Neumann
7d1ba8ab65 Ignore permission errors for backup 2014-11-23 12:15:19 +01:00
Alexander Neumann
94d1482888 Save multiple files in parallel 2014-11-18 21:46:44 +01:00
Alexander Neumann
4a3a6861e2 Scan directory first, then backup. Add stats. 2014-11-17 23:52:31 +01:00
Alexander Neumann
616a2c749d Use different tags for debug log 2014-11-17 23:52:31 +01:00
Alexander Neumann
cf33b01582 Load config.mk in Makefile 2014-11-17 23:52:31 +01:00
Alexander Neumann
7006e13ca9 Add dependencies 2014-11-17 23:52:31 +01:00
Alexander Neumann
bd105b69a8 Add debug() function to khepri cmd and lib 2014-11-17 23:52:31 +01:00
Alexander Neumann
804cebde67 Rename backend subdir 'blobs' to 'data' 2014-11-16 13:22:19 +01:00
Alexander Neumann
ff3d0840d8 Fix integration tests 
Todo for later:
 * fix Futimes() syscall (will be done in #24)
 * return error in dirdiff when ModTime doesn't match
 2014-11-16 12:05:54 +01:00
Alexander Neumann
d20c4ff316 Correctly output blob after decryption with cat 2014-11-15 20:19:34 +01:00
Alexander Neumann
8277daa9e1 Add Makefile and version.sh, add 'version' command 2014-11-15 19:08:15 +01:00
Alexander Neumann
ebd4f97350 Update import path 2014-11-15 19:04:32 +01:00
Alexander Neumann
bb5bfe1ba0 Cleanup, remove unused commands 2014-10-05 16:17:40 +02:00
Alexander Neumann
09702c1c61 Add commands 'cat','list' and 'ls' 2014-10-05 14:44:59 +02:00
Alexander Neumann
f848afed27 Add SFTP backend 2014-10-04 19:20:15 +02:00
Alexander Neumann
30ab03b7b7 Add decrypt, refactor 2014-10-03 21:44:55 +02:00
Alexander Neumann
03ca69407d Add method to create repository 
Also disables automatic creation on open
 2014-09-23 21:16:54 +02:00
Alexander Neumann
f0287b2c9a Restructure, adapt for max 16MB chunk size 2014-09-21 16:33:20 +02:00
Alexander Neumann
f1f96c4206 Remove old cmd/ experiments 2014-09-21 16:02:57 +02:00
Alexander Neumann
0a4c9aab36 Add error to messages 2014-09-21 15:58:52 +02:00
Alexander Neumann
16dfd73ecb Extend cmd/split 2014-09-18 22:40:59 +02:00
Alexander Neumann
18131f88bf Add code for content defined chunking (CDC) 2014-09-18 21:10:20 +02:00
Alexander Neumann
40aea3d612 Remove temp dir 2014-08-12 21:39:14 +02:00
Alexander Neumann
d66996e648 Move restore functionality into khepri package 2014-08-11 23:14:40 +02:00
Alexander Neumann
2428843faa Refactor 2014-08-11 22:47:24 +02:00
Alexander Neumann
d60828fc15 Add dirdiff to tests 2014-08-06 20:09:51 +02:00
Alexander Neumann
017426467b Refactor Makefiles 2014-08-06 20:09:41 +02:00
Alexander Neumann
16d7974b3f Add dirdiff command 2014-08-06 20:09:24 +02:00
Alexander Neumann
ab602b88de Add Makefiles and simple test suite 2014-08-05 23:13:19 +02:00
Alexander Neumann
f45a9c8e9f Fix logging 2014-08-05 23:13:07 +02:00
Alexander Neumann
ad8125d83c Add command 'fsck' 2014-08-04 23:25:58 +02:00
Alexander Neumann
b923a2065a Rename field in Snapshot: Tree -> TreeID 2014-08-04 23:25:32 +02:00
Alexander Neumann
500f4f9997 Add command 'snapshots' 2014-08-04 22:55:54 +02:00
Alexander Neumann
e8b83e460f Refactor 2014-08-04 22:46:14 +02:00
Alexander Neumann
022f514b09 Rename ListIDs -> List 2014-08-04 22:15:04 +02:00
Alexander Neumann
e525655dcb Fix backup command 2014-08-04 21:53:25 +02:00
Alexander Neumann
b3c2d82331 Refactor, add Object and Snapshot 2014-08-04 20:51:09 +02:00
Alexander Neumann
fbd33636f0 Rename types and errmsg function 2014-08-03 16:36:52 +02:00
Alexander Neumann
5cbd1d0090 Add command 'list' 2014-08-01 22:20:28 +02:00
Alexander Neumann
11d6aa5328 Fix commands 2014-08-01 22:20:15 +02:00
Alexander Neumann
13bb557cdc Restructured repository 2014-07-28 20:21:19 +02:00