Compare commits

..

3 commits

Author SHA1 Message Date
Michael Eischer
e1bb2129ad report result of delete operation after failed upload 2021-03-08 21:54:46 +01:00
Michael Eischer
95b7f8dd81 s3: Fix sanity check
The sanity check shouldn't replace the error message if there is already
one.
2021-03-08 21:54:46 +01:00
Michael Eischer
29e39e247a stat file after retrying an upload 2021-03-08 21:54:46 +01:00
918 changed files with 27145 additions and 69661 deletions

View file

@ -1,45 +0,0 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: community, triage, bug
assignees: ''
---
<!--- Provide a general summary of the issue in the Title above -->
## Expected Behavior
<!--- If you're describing a bug, tell us what should happen -->
<!--- If you're suggesting a change/improvement, tell us how it should work -->
## Current Behavior
<!--- If describing a bug, tell us what happens instead of the expected behavior -->
<!--- If suggesting a change/improvement, explain the difference from current behavior -->
## Possible Solution
<!--- Not obligatory -->
<!--- If no reason/fix/additions for the bug can be suggested, -->
<!--- uncomment the following phrase: -->
<!--- No fix can be suggested by a QA engineer. Further solutions shall be up to developers. -->
## Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to -->
<!--- reproduce this bug. -->
1.
## Context
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context helps us come up with a solution that is most useful in the real world -->
## Regression
<!-- Is this issue a regression? (Yes / No) -->
<!-- If Yes, optionally please include version or commit id or PR# that caused this regression, if you have these details. -->
## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used:
* Server setup and configuration:
* Operating System and version (`uname -a`):

View file

@ -1 +0,0 @@
blank_issues_enabled: false

View file

@ -1,20 +0,0 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: community, triage
assignees: ''
---
## Is your feature request related to a problem? Please describe.
<!--- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
## Describe the solution you'd like
<!--- A clear and concise description of what you want to happen. -->
## Describe alternatives you've considered
<!--- A clear and concise description of any alternative solutions or features you've considered. -->
## Additional context
<!--- Add any other context or screenshots about the feature request here. -->

View file

@ -1,24 +0,0 @@
on:
pull_request:
push:
branches:
- tcl/master
jobs:
builds:
name: Builds
runs-on: ubuntu-latest
strategy:
matrix:
go_versions: [ '1.22', '1.23' ]
fail-fast: false
steps:
- uses: actions/checkout@v3
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: '${{ matrix.go_versions }}'
- name: Build binary
run: make

View file

@ -1,20 +0,0 @@
on: [pull_request]
jobs:
dco:
name: DCO
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Setup Go
uses: actions/setup-go@v3
with:
go-version: '1.23'
- name: Run commit format checker
uses: https://git.frostfs.info/TrueCloudLab/dco-go@v3
with:
from: 'origin/${{ github.event.pull_request.base.ref }}'

View file

@ -1,50 +0,0 @@
on:
pull_request:
push:
branches:
- tcl/master
jobs:
lint:
name: Lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: '1.23'
cache: true
- name: Install linters
run: go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest
- name: Run linters
# we run the linter only through the FrostFS backend code
run: golangci-lint --timeout=5m run internal/backend/frostfs/
tests:
name: Tests
runs-on: oci-runner
strategy:
matrix:
go_versions: [ '1.23' ]
fail-fast: false
steps:
- uses: actions/checkout@v3
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: '${{ matrix.go_versions }}'
- name: Tests for the FrostFS backend
env:
RESTIC_TEST_FUSE: false
# run only tests for the FrostFS backend
run: |-
podman-service.sh
podman info
go test -v github.com/restic/restic/internal/backend/frostfs

2
.gitattributes vendored
View file

@ -1,2 +0,0 @@
# Workaround for https://github.com/golang/go/issues/52268.
**/testdata/fuzz/*/* eol=lf

View file

@ -32,30 +32,23 @@ Output of `restic version`
--------------------------
What backend/service did you use to store the repository?
---------------------------------------------------------
Problem description / Steps to reproduce
----------------------------------------
How did you run restic exactly?
-------------------------------
<!--
This section should include at least:
* A description of the problem you are having with restic.
* The complete command line and any environment variables you used to
configure restic's backend access. Make sure to replace sensitive values!
* The output of the commands, what restic prints gives may give us much
information to diagnose the problem!
* The more time you spend describing an easy way to reproduce the behavior (if
this is possible), the easier it is for the project developers to fix it!
-->
What backend/server/service did you use to store the repository?
----------------------------------------------------------------
Expected behavior
-----------------
@ -72,12 +65,22 @@ In this section, please try to concentrate on observations, so only describe
what you observed directly.
-->
Steps to reproduce the behavior
-------------------------------
<!--
The more time you spend describing an easy way to reproduce the behavior (if
this is possible), the easier it is for the project developers to fix it!
-->
Do you have any idea what may have caused this?
-----------------------------------------------
<!--
Did something noteworthy happen on your system, Internet connection, backend services, etc?
-->
Do you have an idea how to solve the issue?
-------------------------------------------
Did restic help you today? Did it make you happy in any way?

View file

@ -1,7 +1,13 @@
<!--
Thank you very much for contributing code or documentation to restic! Please
fill out the following questions to make it easier for us to review your
changes.
You do not need to check all the boxes below all at once, feel free to take
your time and add more commits. If you're done and ready for review, please
check the last box.
-->
What does this PR change? What problem does it solve?
@ -11,8 +17,8 @@ What does this PR change? What problem does it solve?
Describe the changes and their purpose here, as detailed as needed.
-->
Was the change previously discussed in an issue or on the forum?
----------------------------------------------------------------
Was the change discussed in an issue or in the forum before?
------------------------------------------------------------
<!--
Link issues and relevant forum posts here.
@ -24,17 +30,11 @@ is closed automatically when this PR is merged.
Checklist
---------
<!--
You do not need to check all the boxes below all at once. Feel free to take
your time and add more commits. If you're done and ready for review, please
check the last box. Enable a checkbox by replacing [ ] with [x].
-->
- [ ] I have read the [contribution guidelines](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#providing-patches).
- [ ] I have [enabled maintainer edits](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/allowing-changes-to-a-pull-request-branch-created-from-a-fork).
- [ ] I have added tests for all code changes.
- [ ] I have added documentation for relevant changes (in the manual).
- [ ] There's a new file in `changelog/unreleased/` that describes the changes for our users (see [template](https://github.com/restic/restic/blob/master/changelog/TEMPLATE)).
- [ ] I have run `gofmt` on the code in all commits.
- [ ] All commit messages are formatted in the same style as [the other commits in the repo](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#git-commits).
- [ ] I'm done! This pull request is ready for review.
- [ ] I have read the [Contribution Guidelines](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#providing-patches)
- [ ] I have enabled [maintainer edits for this PR](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/allowing-changes-to-a-pull-request-branch-created-from-a-fork)
- [ ] I have added tests for all changes in this PR
- [ ] I have added documentation for the changes (in the manual)
- [ ] There's a new file in `changelog/unreleased/` that describes the changes for our users (template [here](https://github.com/restic/restic/blob/master/changelog/TEMPLATE))
- [ ] I have run `gofmt` on the code in all commits
- [ ] All commit messages are formatted in the same style as [the other commits in the repo](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#git-commits)
- [ ] I'm done, this Pull Request is ready for review

View file

@ -1,13 +0,0 @@
version: 2
updates:
# Dependencies listed in go.mod
- package-ecosystem: "gomod"
directory: "/" # Location of package manifests
schedule:
interval: "monthly"
# Dependencies listed in .github/workflows/*.yml
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "monthly"

View file

@ -1,66 +0,0 @@
name: Create and publish a Docker image
on:
push:
tags:
- 'v*'
branches:
- 'master'
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
build-and-push-image:
if: github.repository == 'restic/restic'
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Set up QEMU
uses: docker/setup-qemu-action@68827325e0b33c7199eb31dd4e31fbe9023e06e3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@2b51285047da1547ffb1b2203d8be4c0af6b1f20
- name: Ensure consistent binaries
run: |
echo "removing git directory for consistency with release binaries"
rm -rf .git
# remove VCS information from release builds, keep VCS for nightly builds on master
if: github.ref != 'refs/heads/master'
- name: Build and push Docker image
uses: docker/build-push-action@15560696de535e4014efeff63c48f16952e52dd1
with:
push: true
context: .
file: docker/Dockerfile.release
platforms: linux/386,linux/amd64,linux/arm,linux/arm64
pull: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

View file

@ -7,14 +7,6 @@ on:
# run tests for all pull requests
pull_request:
merge_group:
permissions:
contents: read
env:
latest_go: "1.22.x"
GO111MODULE: on
jobs:
test:
@ -23,39 +15,28 @@ jobs:
# list of jobs to run:
include:
- job_name: Windows
go: 1.22.x
go: 1.15.x
os: windows-latest
- job_name: macOS
go: 1.22.x
go: 1.15.x
os: macOS-latest
test_fuse: false
- job_name: Linux
go: 1.22.x
go: 1.15.x
os: ubuntu-latest
test_cloud_backends: true
test_fuse: true
check_changelog: true
- job_name: Linux (race)
go: 1.22.x
os: ubuntu-latest
test_fuse: true
test_opts: "-race"
- job_name: Linux
go: 1.21.x
go: 1.14.x
os: ubuntu-latest
test_fuse: true
- job_name: Linux
go: 1.20.x
os: ubuntu-latest
test_fuse: true
- job_name: Linux
go: 1.19.x
go: 1.13.x
os: ubuntu-latest
test_fuse: true
@ -66,18 +47,15 @@ jobs:
GOPROXY: https://proxy.golang.org
steps:
- name: Check out code
uses: actions/checkout@v4
- name: Set up Go ${{ matrix.go }}
uses: actions/setup-go@v5
uses: actions/setup-go@v2
with:
go-version: ${{ matrix.go }}
- name: Get programs (Linux/macOS)
run: |
echo "build Go tools"
go install github.com/restic/rest-server/cmd/rest-server@master
go get github.com/restic/rest-server/...
echo "install minio server"
mkdir $HOME/bin
@ -99,7 +77,7 @@ jobs:
chmod 755 $HOME/bin/rclone
rm -rf rclone*
# add $HOME/bin to path ($GOBIN was already added to the path by setup-go@v3)
# add $HOME/bin to path ($GOBIN was already added to the path by setup-go@v2)
echo $HOME/bin >> $GITHUB_PATH
if: matrix.os == 'ubuntu-latest' || matrix.os == 'macOS-latest'
@ -109,7 +87,7 @@ jobs:
$ProgressPreference = 'SilentlyContinue'
echo "build Go tools"
go install github.com/restic/rest-server/cmd/rest-server@master
go get github.com/restic/rest-server/...
echo "install minio server"
mkdir $Env:USERPROFILE/bin
@ -121,7 +99,7 @@ jobs:
unzip rclone.zip
copy rclone*/rclone.exe $Env:USERPROFILE/bin
# add $USERPROFILE/bin to path ($GOBIN was already added to the path by setup-go@v3)
# add $USERPROFILE/bin to path ($GOBIN was already added to the path by setup-go@v2)
echo $Env:USERPROFILE\bin >> $Env:GITHUB_PATH
echo "install tar"
@ -142,23 +120,18 @@ jobs:
echo $Env:USERPROFILE\tar\bin >> $Env:GITHUB_PATH
if: matrix.os == 'windows-latest'
- name: Check out code
uses: actions/checkout@v2
- name: Build with build.go
run: |
go run build.go
- name: Minimal test
run: |
./restic init
./restic backup .
env:
RESTIC_REPOSITORY: ../testrepo
RESTIC_PASSWORD: password
- name: Run local Tests
env:
RESTIC_TEST_FUSE: ${{ matrix.test_fuse }}
run: |
go test -cover ${{matrix.test_opts}} ./...
go test -cover ./...
- name: Test cloud backends
env:
@ -199,14 +172,12 @@ jobs:
# only run cloud backend tests for pull requests from and pushes to our
# own repo, otherwise the secrets are not available
# Skip for Dependabot pull requests as these are run without secrets
# https://docs.github.com/en/code-security/dependabot/working-with-dependabot/automating-dependabot-with-github-actions#responding-to-events
if: ((github.repository == 'restic/restic' && github.event_name == 'push') || github.event.pull_request.head.repo.full_name == github.repository) && (github.actor != 'dependabot[bot]') && matrix.test_cloud_backends
if: (github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository) && matrix.test_cloud_backends
- name: Check changelog files with calens
run: |
echo "install calens"
go install github.com/restic/calens@latest
go get github.com/restic/calens
echo "check changelog files"
calens
@ -215,56 +186,64 @@ jobs:
cross_compile:
strategy:
# ATTENTION: the list of architectures must be in sync with helpers/build-release-binaries/main.go!
matrix:
# run cross-compile in three batches parallel so the overall tests run faster
subset:
- "0/3"
- "1/3"
- "2/3"
# run cross-compile in two batches parallel so the overall tests run faster
targets:
- "linux/386 linux/amd64 linux/arm linux/arm64 linux/ppc64le linux/mips linux/mipsle linux/mips64 linux/mips64le \
openbsd/386 openbsd/amd64"
- "freebsd/386 freebsd/amd64 freebsd/arm \
aix/ppc64 \
darwin/amd64 \
netbsd/386 netbsd/amd64 \
windows/386 windows/amd64 \
solaris/amd64"
env:
go: 1.15.x
GOPROXY: https://proxy.golang.org
runs-on: ubuntu-latest
name: Cross Compile for subset ${{ matrix.subset }}
name: Cross Compile for ${{ matrix.targets }}
steps:
- name: Check out code
uses: actions/checkout@v4
- name: Set up Go ${{ env.latest_go }}
uses: actions/setup-go@v5
- name: Set up Go ${{ env.go }}
uses: actions/setup-go@v2
with:
go-version: ${{ env.latest_go }}
go-version: ${{ env.go }}
- name: Cross-compile for subset ${{ matrix.subset }}
- name: Check out code
uses: actions/checkout@v2
- name: Install gox
run: |
mkdir build-output build-output-debug
go run ./helpers/build-release-binaries/main.go -o build-output -s . --platform-subset ${{ matrix.subset }}
go run ./helpers/build-release-binaries/main.go -o build-output-debug -s . --platform-subset ${{ matrix.subset }} --tags debug
go get github.com/mitchellh/gox
- name: Cross-compile with gox for ${{ matrix.targets }}
env:
GOFLAGS: "-trimpath"
GOX_ARCHS: "${{ matrix.targets }}"
run: |
mkdir build-output
gox -parallel 2 -verbose -osarch "$GOX_ARCHS" -output "build-output/{{.Dir}}_{{.OS}}_{{.Arch}}" ./cmd/restic
gox -parallel 2 -verbose -osarch "$GOX_ARCHS" -tags debug -output "build-output/{{.Dir}}_{{.OS}}_{{.Arch}}_debug" ./cmd/restic
lint:
name: lint
runs-on: ubuntu-latest
permissions:
contents: read
# allow annotating code in the PR
checks: write
steps:
- name: Check out code
uses: actions/checkout@v4
- name: Set up Go ${{ env.latest_go }}
uses: actions/setup-go@v5
with:
go-version: ${{ env.latest_go }}
uses: actions/checkout@v2
- name: golangci-lint
uses: golangci/golangci-lint-action@v6
uses: golangci/golangci-lint-action@v2
with:
# Required: the version of golangci-lint is required and must be specified without patch version: we always use the latest patch version.
version: v1.57.1
version: v1.36
# Optional: show only new issues if it's a pull request. The default value is `false`.
only-new-issues: true
args: --verbose --timeout 5m
# only run golangci-lint for pull requests, otherwise ALL hints get
@ -277,59 +256,3 @@ jobs:
echo "check if go.mod and go.sum are up to date"
go mod tidy
git diff --exit-code go.mod go.sum
analyze:
name: Analyze results
needs: [test, cross_compile, lint]
if: always()
permissions: # no need to access code
contents: none
runs-on: ubuntu-latest
steps:
- name: Decide whether the needed jobs succeeded or failed
uses: re-actors/alls-green@05ac9388f0aebcb5727afa17fcccfecd6f8ec5fe
with:
jobs: ${{ toJSON(needs) }}
docker:
name: docker
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v4
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
# list of Docker images to use as base name for tags
images: |
restic/restic
# generate Docker tags based on the following events/attributes
tags: |
type=schedule
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
type=sha
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build and push
id: docker_build
uses: docker/build-push-action@v6
with:
push: false
context: .
file: docker/Dockerfile
pull: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

2
.gitignore vendored
View file

@ -1,5 +1,3 @@
/.idea
/restic
/restic.exe
/.vagrant
/.vscode

View file

@ -10,10 +10,13 @@ linters:
# make sure all errors returned by functions are handled
- errcheck
# find unused code
- deadcode
# show how code can be simplified
- gosimple
# make sure code is formatted
# # make sure code is formatted
- gofmt
# examine code and report suspicious constructs, such as Printf calls whose
@ -21,7 +24,7 @@ linters:
- govet
# make sure names and comments are used according to the conventions
- revive
- golint
# detect when assignments to existing variables are not used
- ineffassign
@ -32,14 +35,15 @@ linters:
# find unused variables, functions, structs, types, etc.
- unused
# find unused struct fields
- structcheck
# find unused global variables
- varcheck
# parse and typecheck code
- typecheck
# ensure that http response bodies are closed
- bodyclose
- importas
issues:
# don't use the default exclude rules, this hides (among others) ignored
# errors from Close() calls
@ -47,23 +51,7 @@ issues:
# list of things to not warn about
exclude:
# revive: do not warn about missing comments for exported stuff
- exported (function|method|var|type|const) .* should have comment or be unexported
# revive: ignore constants in all caps
# golint: do not warn about missing comments for exported stuff
- exported (function|method|var|type|const) `.*` should have comment or be unexported
# golint: ignore constants in all caps
- don't use ALL_CAPS in Go names; use CamelCase
# revive: lots of packages don't have such a comment
- "package-comments: should have a package comment"
# staticcheck: there's no easy way to replace these packages
- "SA1019: \"golang.org/x/crypto/poly1305\" is deprecated"
- "SA1019: \"golang.org/x/crypto/openpgp\" is deprecated"
exclude-rules:
# revive: ignore unused parameters in tests
- path: (_test\.go|testing\.go|backend/.*/tests\.go)
text: "unused-parameter:"
linters-settings:
importas:
alias:
- pkg: github.com/restic/restic/internal/test
alias: rtest

View file

@ -1,22 +0,0 @@
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
version: 2
build:
os: ubuntu-22.04
tools:
python: "3.11"
# Build HTMLZip
formats:
- htmlzip
# Build documentation in the docs/ directory with Sphinx
sphinx:
configuration: doc/conf.py
# https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html
python:
install:
- requirements: doc/requirements.txt

File diff suppressed because it is too large Load diff

View file

@ -6,18 +6,18 @@ Ways to Help Out
Thank you for your contribution! Please **open an issue first** (or add a
comment to an existing issue) if you plan to work on any code or add a new
feature. This way, duplicate work is prevented and we can discuss your ideas
and design first. Small bugfixes are an exception to this rule, just open a
pull request in this case.
and design first.
There are several ways you can help us out. First of all code contributions and
bug fixes are most welcome. However even "minor" details as fixing spelling
errors, improving documentation or pointing out usability issues are a great
help also.
The restic project uses the GitHub infrastructure (see the
[project page](https://github.com/restic/restic)) for all related discussions
as well as the [forum](https://forum.restic.net/) and the `#restic` channel
on [irc.libera.chat](https://kiwiirc.com/nextclient/#ircs://irc.libera.chat:6697/#restic).
on [irc.freenode.net](https://kiwiirc.com/nextclient/irc.freenode.net/restic).
If you want to find an area that currently needs improving have a look at the
open issues listed at the
@ -49,8 +49,9 @@ environment was used and so on. Please tell us at least the following things:
Remember, the easier it is for us to reproduce the bug, the earlier it will be
corrected!
In addition, you can instruct restic to create a debug log by setting the
environment variable `DEBUG_LOG` to a file, e.g. like this:
In addition, you can compile restic with debug support by running
`go run build.go -tags debug` and instructing it to create a debug
log by setting the environment variable `DEBUG_LOG` to a file, e.g. like this:
$ export DEBUG_LOG=/tmp/restic-debug.log
$ restic backup ~/work
@ -59,19 +60,6 @@ Please be aware that the debug log file will contain potentially sensitive
things like file and directory names, so please either redact it before
uploading it somewhere or post only the parts that are really relevant.
If restic gets stuck, please also include a stacktrace in the description.
On non-Windows systems, you can send a SIGQUIT signal to restic or press
`Ctrl-\` to achieve the same result. This causes restic to print a stacktrace
and then exit immediately. This will not damage your repository, however,
it might be necessary to manually clean up stale lock files using
`restic unlock`.
On Windows, please set the environment variable `RESTIC_DEBUG_STACKTRACE_SIGINT`
to `true` and press `Ctrl-C` to create a stacktrace.
If you think restic uses too much memory or a too large cache directory, then
please include the output of `restic stats --mode debug`.
Development Environment
=======================
@ -79,8 +67,8 @@ Development Environment
The repository contains the code written for restic in the directories
`cmd/` and `internal/`.
Make sure you have the minimum required Go version installed. Clone the repo
(without having `$GOPATH` set) and `cd` into the directory:
Restic requires Go version 1.13 or later for compiling. Clone the repo (without
having `$GOPATH` set) and `cd` into the directory:
$ unset GOPATH
$ git clone https://github.com/restic/restic
@ -90,42 +78,12 @@ Then use the `go` tool to build restic:
$ go build ./cmd/restic
$ ./restic version
restic 0.14.0-dev (compiled manually) compiled with go1.19 on linux/amd64
To create a debug build use:
$ go build -tags debug ./cmd/restic
restic 0.10.0-dev (compiled manually) compiled with go1.15.2 on linux/amd64
You can run all tests with the following command:
$ go test ./...
Performance and Memory Usage Issues
===================================
Debug builds of restic support the `--block-profile`, `--cpu-profile`,
`--mem-profile`, and `--trace-profile` options which collect performance data
that later on can be analyzed using the go tools:
$ restic --cpu-profile . [...]
$ go tool pprof -http localhost:12345 cpu.pprof
To analyze a trace profile use `go tool trace -http=localhost:12345 trace.out`.
As the memory usage of restic changes over time, it may be useful to capture a
snapshot of the current heap. This is possible using then `--listen-profile`
option. Then while restic runs you can query and afterwards analyze the heap statistics.
$ restic --listen-profile localhost:12345 [...]
$ curl http://localhost:12345/debug/pprof/heap -o heap.pprof
$ go tool pprof -http localhost:12345 heap.pprof
Further useful tools are setting the environment variable `GODEBUG=gctrace=1`,
which provides information about garbage collector runs. For a graphical variant
combine this with gcvis.
Providing Patches
=================
@ -166,10 +124,7 @@ down to the following steps:
writing, ask yourself: If I were the user, what would I need to be aware
of with this change?
8. Do not edit the man pages under `doc/man` or `doc/manual_rest.rst` -
these are autogenerated before new releases.
9. Once your code looks good and passes all the tests, we'll merge it. Thanks
8. Once your code looks good and passes all the tests, we'll merge it. Thanks
a lot for your contribution!
Please provide the patches for each bug or feature in a separate branch and

View file

@ -10,7 +10,8 @@ For detailed usage and installation instructions check out the [documentation](h
You can ask questions in our [Discourse forum](https://forum.restic.net).
## Quick start
Quick start
-----------
Once you've [installed](https://restic.readthedocs.io/en/latest/020_installation.html) restic, start
off with creating a repository for your backups:
@ -45,8 +46,8 @@ Therefore, restic supports the following backends for storing backups natively:
- [Local directory](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#local)
- [sftp server (via SSH)](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#sftp)
- [HTTP REST server](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#rest-server) ([protocol](https://restic.readthedocs.io/en/latest/100_references.html#rest-backend), [rest-server](https://github.com/restic/rest-server))
- [Amazon S3](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#amazon-s3) (either from Amazon or using the [Minio](https://minio.io) server)
- [HTTP REST server](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#rest-server) ([protocol](doc/100_references.rst#rest-backend), [rest-server](https://github.com/restic/rest-server))
- [AWS S3](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#amazon-s3) (either from Amazon or using the [Minio](https://minio.io) server)
- [OpenStack Swift](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#openstack-swift)
- [BackBlaze B2](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#backblaze-b2)
- [Microsoft Azure Blob Storage](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#microsoft-azure-blob-storage)
@ -58,7 +59,7 @@ Therefore, restic supports the following backends for storing backups natively:
Restic is a program that does backups right and was designed with the
following principles in mind:
- **Easy**: Doing backups should be a frictionless process, otherwise
- **Easy:** Doing backups should be a frictionless process, otherwise
you might be tempted to skip it. Restic should be easy to configure
and use, so that, in the event of a data loss, you can just restore
it. Likewise, restoring data should not be complicated.
@ -91,17 +92,20 @@ reproduce a byte identical version from the source code for that
release. Instructions on how to do that are contained in the
[builder repository](https://github.com/restic/builder).
## News
News
----
You can follow the restic project on Mastodon [@resticbackup](https://fosstodon.org/@restic) or subscribe to
You can follow the restic project on Twitter [@resticbackup](https://twitter.com/resticbackup) or by subscribing to
the [project blog](https://restic.net/blog/).
## License
License
-------
Restic is licensed under [BSD 2-Clause License](https://opensource.org/licenses/BSD-2-Clause). You can find the
complete text in [`LICENSE`](LICENSE).
complete text in [``LICENSE``](LICENSE).
## Sponsorship
Sponsorship
-----------
Backend integration tests for Google Cloud Storage and Microsoft Azure Blob
Storage are sponsored by [AppsCode](https://appscode.com)!

View file

@ -1 +1 @@
0.17.3
0.12.0

View file

@ -3,8 +3,8 @@
// This program aims to make building Go programs for end users easier by just
// calling it with `go run`, without having to setup a GOPATH.
//
// This program checks for a minimum Go version. It will use Go modules for
// compilation. It builds the package configured as Main in the Config struct.
// This program needs Go >= 1.12. It'll use Go modules for compilation. It
// builds the package configured as Main in the Config struct.
// BSD 2-Clause License
//
@ -35,7 +35,6 @@
// OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//go:build ignore_build_go
// +build ignore_build_go
package main
@ -43,6 +42,7 @@ package main
import (
"fmt"
"io"
"io/ioutil"
"os"
"os/exec"
"path/filepath"
@ -58,7 +58,7 @@ var config = Config{
Main: "./cmd/restic", // package name for the main package
DefaultBuildTags: []string{"selfupdate"}, // specify build tags which are always used
Tests: []string{"./..."}, // tests to run
MinVersion: GoVersion{Major: 1, Minor: 18, Patch: 0}, // minimum Go version supported
MinVersion: GoVersion{Major: 1, Minor: 11, Patch: 0}, // minimum Go version supported
}
// Config configures the build.
@ -123,8 +123,17 @@ func printEnv(env []string) {
// build runs "go build args..." with GOPATH set to gopath.
func build(cwd string, env map[string]string, args ...string) error {
// -trimpath removes all absolute paths from the binary.
a := []string{"build", "-trimpath"}
a := []string{"build"}
// try to remove all absolute paths from resulting binary
if goVersion.AtLeast(GoVersion{1, 13, 0}) {
// use the new flag introduced by Go 1.13
a = append(a, "-trimpath")
} else {
// otherwise try to trim as many paths as possible
a = append(a, "-asmflags", fmt.Sprintf("all=-trimpath=%s", cwd))
a = append(a, "-gcflags", fmt.Sprintf("all=-trimpath=%s", cwd))
}
if enablePIE {
a = append(a, "-buildmode=pie")
@ -178,7 +187,7 @@ func test(cwd string, env map[string]string, args ...string) error {
// getVersion returns the version string from the file VERSION in the current
// directory.
func getVersionFromFile() string {
buf, err := os.ReadFile("VERSION")
buf, err := ioutil.ReadFile("VERSION")
if err != nil {
verbosePrintf("error reading file VERSION: %v\n", err)
return ""
@ -318,8 +327,12 @@ func (v GoVersion) String() string {
}
func main() {
if !goVersion.AtLeast(GoVersion{1, 12, 0}) {
die("Go version (%v) is too old, restic requires Go >= 1.12\n", goVersion)
}
if !goVersion.AtLeast(config.MinVersion) {
fmt.Fprintf(os.Stderr, "Detected version %s is too old, restic requires at least %s\n", goVersion, config.MinVersion)
fmt.Fprintf(os.Stderr, "%s detected, this program requires at least %s\n", goVersion, config.MinVersion)
os.Exit(1)
}
@ -380,12 +393,6 @@ func main() {
}
}
solarisMinVersion := GoVersion{Major: 1, Minor: 20, Patch: 0}
if env["GOARCH"] == "solaris" && !goVersion.AtLeast(solarisMinVersion) {
fmt.Fprintf(os.Stderr, "Detected version %s is too old, restic requires at least %s for Solaris\n", goVersion, solarisMinVersion)
os.Exit(1)
}
verbosePrintf("detected Go version %v\n", goVersion)
preserveSymbols := false

View file

@ -10,7 +10,7 @@ https://github.com/restic/restic/issues/2244
NOTE: This new implementation does not guarantee order in which blobs
are written to the target files and, for example, the last blob of a
file can be written to the file before any of the preceding file blobs.
file can be written to the file before any of the preceeding file blobs.
It is therefore possible to have gaps in the data written to the target
files if restore fails or interrupted by the user.

View file

@ -1,6 +1,6 @@
Bugfix: Don't abort the stats command when data blobs are missing
Running the stats command in the blobs-per-file mode on a repository with
Runing the stats command in the blobs-per-file mode on a repository with
missing data blobs previously resulted in a crash.
https://github.com/restic/restic/pull/2668

View file

@ -1,6 +1,6 @@
Bugfix: Mark repository files as read-only when using the local backend
Files stored in a local repository were marked as writable on the
Files stored in a local repository were marked as writeable on the
filesystem for non-Windows systems, which did not prevent accidental file
modifications outside of restic. In addition, the local backend did not work
with certain filesystems and network mounts which do not permit modifications

View file

@ -5,7 +5,7 @@ another process using an exclusive lock through a filesystem snapshot. Restic
was unable to backup those files before. This update enables backing up these
files.
This needs to be enabled explicitly using the --use-fs-snapshot option of the
This needs to be enabled explicitely using the --use-fs-snapshot option of the
backup command.
https://github.com/restic/restic/issues/340

View file

@ -1,7 +1,7 @@
Bugfix: Correct statistics for overlapping backup sources
Bugfix: Correct statistics for overlapping targets
A user reported that restic's statistics and progress information during backup
was not correctly calculated when the backup sources (files/dirs to save)
was not correctly calculated when the backup targets (files/dirs to save)
overlap. For example, consider a directory `foo` which contains (among others)
a file `foo/bar`. When `restic backup foo foo/bar` was run, restic counted the
size of the file `foo/bar` twice, so the completeness percentage as well as the

View file

@ -2,7 +2,7 @@ Enhancement: Parallelize scan of snapshot content in `copy` and `prune`
The `copy` and `prune` commands used to traverse the directories of
snapshots one by one to find used data. This snapshot traversal is
now parallelized which can speed up this step several times.
now parallized which can speed up this step several times.
In addition the `check` command now reports how many snapshots have
already been processed.

View file

@ -1,13 +0,0 @@
Bugfix: Improve error handling for rclone and REST backend over HTTP2
When retrieving data from the rclone / REST backend while also using HTTP2
restic did not detect when no data was returned at all. This could cause
for example the `check` command to report the following error:
Pack ID does not match, want [...], got e3b0c442
This has been fixed by correctly detecting and retrying the incomplete download.
https://github.com/restic/restic/issues/2742
https://github.com/restic/restic/pull/3453
https://forum.restic.net/t/http2-stream-closed-connection-reset-context-canceled/3743/10

View file

@ -1,6 +0,0 @@
Enhancement: Add release binaries for s390x architecture on Linux
We've added release binaries for Linux using the s390x architecture.
https://github.com/restic/restic/issues/2780
https://github.com/restic/restic/pull/3452

View file

@ -1,11 +0,0 @@
Bugfix: Fix terminal output redirection for PowerShell
When redirecting the output of restic using PowerShell on Windows, the
output contained terminal escape characters. This has been fixed by
properly detecting the terminal type.
In addition, the mintty terminal now shows progress output for the backup
command.
https://github.com/restic/restic/issues/3111
https://github.com/restic/restic/pull/3325

View file

@ -1,9 +0,0 @@
Bugfix: Treat an empty password as a fatal error for repository init
When attempting to initialize a new repository, if an empty password was
supplied, the repository would be created but the init command would return
an error with a stack trace. Now, if an empty password is provided, it is
treated as a fatal error, and no repository is created.
https://github.com/restic/restic/issues/3214
https://github.com/restic/restic/pull/3283

View file

@ -1,8 +0,0 @@
Change: Empty files now have size of 0 in `ls --json` output
The `ls --json` command used to omit the sizes of empty files in its
output. It now reports a size of zero explicitly for regular files,
while omitting the size field for all other types.
https://github.com/restic/restic/issues/3247
https://github.com/restic/restic/pull/3257

View file

@ -1,11 +0,0 @@
Bugfix: `copy` failed to copy snapshots in rare cases
The `copy` command could in rare cases fail with the error message `SaveTree(...)
returned unexpected id ...`. This has been fixed.
On Linux/BSDs, the error could be caused by backing up symlinks with non-UTF-8
target paths. Note that, due to limitations in the repository format, these are
not stored properly and should be avoided if possible.
https://github.com/restic/restic/issues/3267
https://github.com/restic/restic/pull/3310

View file

@ -1,11 +0,0 @@
Bugfix: `backup --quiet` no longer prints status information
A regression in the latest restic version caused the output of `backup --quiet`
to contain large amounts of backup progress information when run using an
interactive terminal. This is fixed now.
A workaround for this bug is to run restic as follows:
`restic backup --quiet [..] | cat -`.
https://github.com/restic/restic/issues/3184
https://github.com/restic/restic/pull/3186

View file

@ -1,14 +0,0 @@
Enhancement: Add `--repository-file2` option to `init` and `copy` command
The `init` and `copy` command can now be used with the `--repository-file2`
option or the `$RESTIC_REPOSITORY_FILE2` environment variable.
These to options are in addition to the `--repo2` flag and allow you to read
the destination repository from a file.
Using both `--repository-file` and `--repo2` options resulted in an error for
the `copy` or `init` command. The handling of this combination of options has
been fixed. A workaround for this issue is to only use `--repo` or `-r` and
`--repo2` for `init` or `copy`.
https://github.com/restic/restic/issues/3293
https://github.com/restic/restic/pull/3294

View file

@ -1,7 +0,0 @@
Bugfix: Fix crash of `check --read-data-subset=x%` run for an empty repository
The command `restic check --read-data-subset=x%` crashed when run for an empty
repository. This has been fixed.
https://github.com/restic/restic/issues/3296
https://github.com/restic/restic/pull/3309

View file

@ -1,8 +0,0 @@
Bugfix: Fix `fdopendir: not a directory` error for local backend
The `check`, `list packs`, `prune` and `rebuild-index` commands failed
for the local backend when the `data` folder in the repository contained
files. This has been fixed.
https://github.com/restic/restic/issues/3302
https://github.com/restic/restic/pull/3308

View file

@ -1,5 +0,0 @@
Enhancement: Add auto-completion support for fish
The `generate` command now supports fish auto completion.
https://github.com/restic/restic/pull/3312

View file

@ -1,8 +0,0 @@
Bugfix: Print `created new cache` message only on a terminal
The message `created new cache` was printed even when the output wasn't a
terminal. That broke piping `restic dump` output to tar or zip if cache
directory didn't exist. The message is now only printed on a terminal.
https://github.com/restic/restic/issues/3334
https://github.com/restic/restic/pull/3343

View file

@ -1,8 +0,0 @@
Enhancement: SFTP backend now checks for disk space
Backing up over SFTP previously spewed multiple generic "failure" messages
when the remote disk was full. It now checks for disk space before writing
a file and fails immediately with a "no space left on device" message.
https://github.com/restic/restic/issues/3336
https://github.com/restic/restic/pull/3345

View file

@ -1,6 +0,0 @@
Enhancement: Add release binaries for Apple Silicon
We've added release binaries for macOS on Apple Silicon (M1).
https://github.com/restic/restic/issues/3377
https://github.com/restic/restic/pull/3394

View file

@ -1,7 +0,0 @@
Bugfix: Fix crash of `backup --exclude='**'`
The exclude filter `**`, which excludes all files, caused restic to crash. This
has been corrected.
https://github.com/restic/restic/issues/3380
https://github.com/restic/restic/pull/3393

View file

@ -1,20 +0,0 @@
Enhancement: Add `--keep-within-hourly` option to restic forget
The `forget` command allowed keeping a given number of hourly
backups or to keep all backups within a given interval, but it
was not possible to specify keeping hourly backups within a given
interval.
The new `--keep-within-hourly` option now offers this functionality.
Similar options for daily/weekly/monthly/yearly are also implemented,
the new options are:
--keep-within-hourly <1y2m3d4h>
--keep-within-daily <1y2m3d4h>
--keep-within-weekly <1y2m3d4h>
--keep-within-monthly <1y2m3d4h>
--keep-within-yearly <1y2m3d4h>
https://github.com/restic/restic/issues/3414
https://github.com/restic/restic/pull/3416
https://forum.restic.net/t/forget-policy/4014/11

View file

@ -1,9 +0,0 @@
Enhancement: Support filtering and specifying untagged snapshots
It was previously not possible to specify an empty tag with the `--tag` and
`--keep-tag` options. This has now been fixed, such that `--tag ''` and
`--keep-tag ''` now matches snapshots without tags. This allows e.g. the
`snapshots` and `forget` commands to only operate on untagged snapshots.
https://github.com/restic/restic/issues/3456
https://github.com/restic/restic/pull/3457

View file

@ -1,9 +0,0 @@
Enhancement: Allow specifying limit of `snapshots` list
The `--last` option allowed limiting the output of the `snapshots`
command to the latest snapshot for each host. The new `--latest n`
option allows limiting the output to the latest `n` snapshots.
This change deprecates the option `--last` in favour of `--latest 1`.
https://github.com/restic/restic/pull/3167

View file

@ -1,6 +0,0 @@
Bugfix: Fix possibly missing backup summary of JSON output in case of error
When using `--json` output it happened from time to time that the summary
output was missing in case an error occurred. This has been fixed.
https://github.com/restic/restic/pull/3305

View file

@ -1,7 +0,0 @@
Enhancement: Optimize read performance of mount command
Reading large files in a mounted repository may be up to five times faster.
This improvement primarily applies to repositories stored at a backend that can
be accessed with low latency, like e.g. the local backend.
https://github.com/restic/restic/pull/3426

View file

@ -1,13 +0,0 @@
Enhancement: `find --pack` fallback to index if data file is missing
When investigating a repository with missing data files, it might be useful to
determine affected snapshots before running `rebuild-index`. Previously,
`find --pack pack-id` returned no data as it required accessing the data file.
Now, if the necessary data is still available in the repository index, it gets
retrieved from there.
The command now also supports looking up multiple pack files in a single `find`
run.
https://github.com/restic/restic/pull/3427
https://forum.restic.net/t/missing-packs-not-found/2600

View file

@ -1,8 +0,0 @@
Bugfix: Correctly handle download errors during `restore`
Due to a regression in restic 0.12.0, the `restore` command in some cases did
not retry download errors and only printed a warning. This has been fixed by
retrying incomplete data downloads.
https://github.com/restic/restic/issues/3439
https://github.com/restic/restic/pull/3449

View file

@ -1,10 +0,0 @@
Bugfix: Never lock repository for `list locks`
The `list locks` command previously locked to the repository by default. This
had the problem that it wouldn't work for an exclusively locked repository and
that the command would also display its own lock file which can be confusing.
Now, the `list locks` command never locks the repository.
https://github.com/restic/restic/issues/1106
https://github.com/restic/restic/pull/3665

View file

@ -1,16 +0,0 @@
Enhancement: Add `--dry-run`/`-n` option to `backup` command
Testing exclude filters and other configuration options was error prone as
wrong filters could cause files to be uploaded unintentionally. It was also
not possible to estimate beforehand how much data would be uploaded.
The `backup` command now has a `--dry-run`/`-n` option, which performs all the
normal steps of a backup without actually writing anything to the repository.
Passing -vv will log information about files that would be added, allowing for
verification of source and exclusion options before running the real backup.
https://github.com/restic/restic/issues/1542
https://github.com/restic/restic/pull/2308
https://github.com/restic/restic/pull/3210
https://github.com/restic/restic/pull/3300

View file

@ -1,15 +0,0 @@
Enhancement: Add upload checksum for Azure, GS, S3 and Swift backends
Previously only the B2 and partially the Swift backends verified the integrity
of uploaded (encrypted) files. The verification works by informing the backend
about the expected hash of the uploaded file. The backend then verifies the
upload and thereby rules out any data corruption during upload.
We have now added upload checksums for the Azure, GS, S3 and Swift backends,
which besides integrity checking for uploads also means that restic can now be
used to store backups in S3 buckets which have Object Lock enabled.
https://github.com/restic/restic/issues/2202
https://github.com/restic/restic/issues/2700
https://github.com/restic/restic/issues/3023
https://github.com/restic/restic/pull/3246

View file

@ -1,29 +0,0 @@
Enhancement: Support negative include/exclude patterns
If a pattern starts with an exclamation mark and it matches a file that was
previously matched by a regular pattern, the match is cancelled. Notably,
this can be used with `--exclude-file` to cancel the exclusion of some files.
It works similarly to `.gitignore`, with the same limitation; Once a directory
is excluded, it is not possible to include files inside the directory.
Example of use as an exclude pattern for the `backup` command:
$HOME/**/*
!$HOME/Documents
!$HOME/code
!$HOME/.emacs.d
!$HOME/games
# [...]
node_modules
*~
*.o
*.lo
*.pyc
# [...]
$HOME/code/linux/*
!$HOME/code/linux/.git
# [...]
https://github.com/restic/restic/issues/233
https://github.com/restic/restic/pull/2311

View file

@ -1,13 +0,0 @@
Bugfix: Make cache crash-resistant and usable by multiple concurrent processes
The restic cache directory (`RESTIC_CACHE_DIR`) could end up in a broken state
in the event of restic (or the OS) crashing. This is now less likely to occur
as files are downloaded to a temporary location before being moved to their
proper location.
This also allows multiple concurrent restic processes to operate on a single
repository without conflicts. Previously, concurrent operations could cause
segfaults because the processes saw each other's partially downloaded files.
https://github.com/restic/restic/issues/2345
https://github.com/restic/restic/pull/2838

View file

@ -1,10 +0,0 @@
Enhancement: Add warning for S3 if partial credentials are provided
Previously restic did not notify about incomplete credentials when using the
S3 backend, instead just reporting access denied.
Restic now checks that both the AWS key ID and secret environment variables are
set before connecting to the remote server, and reports an error if not.
https://github.com/restic/restic/issues/2388
https://github.com/restic/restic/pull/3532

View file

@ -1,11 +0,0 @@
Bugfix: Improve error handling of repository locking
Previously, when the lock refresh failed to delete the old lock file, it forgot
about the newly created one. Instead it continued trying to delete the old
(usually no longer existing) lock file and thus over time lots of lock files
accumulated. This has now been fixed.
https://github.com/restic/restic/issues/2452
https://github.com/restic/restic/issues/2473
https://github.com/restic/restic/issues/2562
https://github.com/restic/restic/pull/3512

View file

@ -1,8 +0,0 @@
Enhancement: Support JSON output and quiet mode for the `diff` command
The `diff` command now supports outputting machine-readable output in JSON
format. To enable this, pass the `--json` option to the command. To only print
the summary and suppress detailed output, pass the `--quiet` option.
https://github.com/restic/restic/issues/2508
https://github.com/restic/restic/pull/3592

View file

@ -1,8 +0,0 @@
Enhancement: Add flag to disable TLS verification for self-signed certificates
There is now an `--insecure-tls` global option in restic, which disables TLS
verification for self-signed certificates in order to support some development
workflows.
https://github.com/restic/restic/issues/2656
https://github.com/restic/restic/pull/2657

View file

@ -1,8 +0,0 @@
Bugfix: Don't print progress for `backup --json --quiet`
Unlike the text output, the `--json` output format still printed progress
information even in `--quiet` mode. This has now been fixed by always
disabling the progress output in quiet mode.
https://github.com/restic/restic/issues/2738
https://github.com/restic/restic/pull/3264

View file

@ -1,9 +0,0 @@
Enhancement: Atomic uploads for the SFTP backend
The SFTP backend did not upload files atomically. An interrupted upload could
leave an incomplete file behind which could prevent restic from accessing the
repository. This has now been fixed and uploads in the SFTP backend are done
atomically.
https://github.com/restic/restic/issues/3003
https://github.com/restic/restic/pull/3524

View file

@ -1,6 +0,0 @@
Enhancement: Add xattr (extended attributes) support for Solaris
Restic now supports xattr for the Solaris operating system.
https://github.com/restic/restic/issues/3127
https://github.com/restic/restic/pull/3628

View file

@ -1,8 +0,0 @@
Bugfix: Make `check` command honor `RESTIC_CACHE_DIR` environment variable
Previously, the `check` command didn't honor the `RESTIC_CACHE_DIR` environment
variable, which caused problems in certain system/usage configurations. This
has now been fixed.
https://github.com/restic/restic/issues/3382
https://github.com/restic/restic/pull/3474

View file

@ -1,10 +0,0 @@
Enhancement: Skip lock creation on `forget` if `--no-lock` and `--dry-run`
Restic used to silently ignore the `--no-lock` option of the `forget` command.
It now skips creation of lock file in case both `--dry-run` and `--no-lock`
are specified. If `--no-lock` option is specified without `--dry-run`, restic
prints a warning message to stderr.
https://github.com/restic/restic/issues/3464
https://github.com/restic/restic/pull/3623

View file

@ -1,8 +0,0 @@
Enhancement: Support random subset by size in `check --read-data-subset`
The `--read-data-subset` option of the `check` command now supports a third way
of specifying the subset to check, namely `nS` where `n` is a size in bytes with
suffix `S` as k/K, m/M, g/G or t/T.
https://github.com/restic/restic/issues/3490
https://github.com/restic/restic/pull/3548

View file

@ -1,8 +0,0 @@
Bugfix: Make `copy` command honor `--no-lock` for source repository
The `copy` command previously did not respect the `--no-lock` option for the
source repository, causing failures with read-only storage backends. This has
now been fixed such that the option is now respected.
https://github.com/restic/restic/issues/3518
https://github.com/restic/restic/pull/3589

View file

@ -1,9 +0,0 @@
Enhancement: Improve handling of temporary B2 delete errors
Deleting files on B2 could sometimes fail temporarily, which required restic to
retry the delete operation. In some cases the file was deleted nevertheless,
causing the retries and ultimately the restic command to fail. This has now been
fixed.
https://github.com/restic/restic/issues/3541
https://github.com/restic/restic/pull/3544

View file

@ -1,8 +0,0 @@
Enhancement: Add file mode in symbolic notation to `ls --json`
The `ls --json` command now provides the file mode in symbolic notation (using
the `permissions` key), aligned with `find --json`.
https://github.com/restic/restic/issues/3542
https://github.com/restic/restic/pull/3573
https://forum.restic.net/t/restic-ls-understanding-file-mode-with-json/4371

View file

@ -1,10 +0,0 @@
Bugfix: Fix hang with Backblaze B2 on SSL certificate authority error
Previously, if a request failed with an SSL unknown certificate authority
error, the B2 backend retried indefinitely and restic would appear to hang.
This has now been fixed and restic instead fails with an error message.
https://github.com/restic/restic/issues/3556
https://github.com/restic/restic/issues/2355
https://github.com/restic/restic/pull/3571

View file

@ -1,15 +0,0 @@
Bugfix: Fix rclone backend prematurely exiting when receiving SIGINT on Windows
Previously, pressing Ctrl+C in a Windows console where restic was running with
rclone as the backend would cause rclone to exit prematurely due to getting a
`SIGINT` signal at the same time as restic. Restic would then wait for a long
time for time with "unexpected EOF" and "rclone stdio connection already closed"
errors.
This has now been fixed by restic starting the rclone process detached from the
console restic runs in (similar to starting processes in a new process group on
Linux), which enables restic to gracefully clean up rclone (which now never gets
the `SIGINT`).
https://github.com/restic/restic/issues/3601
https://github.com/restic/restic/pull/3602

View file

@ -1,14 +0,0 @@
Change: Ignore parent snapshot for `backup --stdin`
Restic uses a parent snapshot to speed up directory scanning when performing
backups, but this only wasted time and memory when the backup source is stdin
(using the `--stdin` option of the `backup` command), since no directory scanning
is performed in this case.
Snapshots made with `backup --stdin` no longer have a parent snapshot, which allows
restic to skip some startup operations and saves a bit of resources.
The `--parent` option is still available for `backup --stdin`, but is now ignored.
https://github.com/restic/restic/issues/3641
https://github.com/restic/restic/pull/3645

View file

@ -1,8 +0,0 @@
Bugfix: The `mount` command now reports symlinks sizes
Symlinks used to have size zero in restic mountpoints, confusing some
third-party tools. They now have a size equal to the byte length of their
target path, as required by POSIX.
https://github.com/restic/restic/issues/3667
https://github.com/restic/restic/pull/3668

View file

@ -1,7 +0,0 @@
Enhancement: Speed up the `restore --verify` command
The `--verify` option lets the `restore` command verify the file content
after it has restored a snapshot. The performance of this operation has
now been improved by up to a factor of two.
https://github.com/restic/restic/pull/2594

View file

@ -1,10 +0,0 @@
Enhancement: The `backup` command no longer updates file access times on Linux
When reading files during backup, restic used to cause the operating system to
update the files' access times. Note that this did not apply to filesystems with
disabled file access times.
Restic now instructs the operating system not to update the file access time,
if the user running restic is the file owner or has root permissions.
https://github.com/restic/restic/pull/2816

View file

@ -1,9 +0,0 @@
Enhancement: Make `recover` collect only unreferenced trees
Previously, the `recover` command used to generate a snapshot containing *all*
root trees, even those which were already referenced by a snapshot.
This has been improved such that it now only processes trees not already
referenced by any snapshot.
https://github.com/restic/restic/pull/2880

View file

@ -1,12 +0,0 @@
Enhancement: Verify that new or modified keys are stored correctly
When adding a new key or changing the password of a key, restic used to just
create the new key (and remove the old one, when changing the password). There
was no verification that the new key was stored correctly and works properly.
As the repository cannot be decrypted without a valid key file, this could in
rare cases cause the repository to become inaccessible.
Restic now checks that new key files actually work before continuing. This
can protect against some (rare) cases of hardware or storage problems.
https://github.com/restic/restic/pull/3429

View file

@ -1,12 +0,0 @@
Enhancement: Improve local backend's resilience to (system) crashes
Restic now ensures that files stored using the `local` backend are created
atomically (that is, files are either stored completely or not at all). This
ensures that no incomplete files are left behind even if restic is terminated
while writing a file.
In addition, restic now tries to ensure that the directory in the repository
which contains a newly uploaded file is also written to disk. This can prevent
missing files if the system crashes or the disk is not properly unmounted.
https://github.com/restic/restic/pull/3436

View file

@ -1,9 +0,0 @@
Bugfix: `rebuild-index` failed if an index file was damaged
Previously, the `rebuild-index` command would fail with an error if an index
file was damaged or truncated. This has now been fixed.
On older restic versions, a (slow) workaround is to use
`rebuild-index --read-all-packs` or to manually delete the damaged index.
https://github.com/restic/restic/pull/3488

View file

@ -1,10 +0,0 @@
Enhancement: Cache blobs read by the `dump` command
When dumping a file using the `dump` command, restic did not cache blobs in any
way, so even consecutive runs of the same blob were loaded from the repository
again and again, slowing down the dump.
Now, the caching mechanism already used by the `fuse` command is also used by
the `dump` command. This makes dumping much faster, especially for sparse files.
https://github.com/restic/restic/pull/3508

View file

@ -1,8 +0,0 @@
Enhancement: Support configurable timeout for the rclone backend
A slow rclone backend could cause restic to time out while waiting for the
repository to open. Restic now offers an `-o rclone.timeout` option to make
this timeout configurable.
https://github.com/restic/restic/issues/3511
https://github.com/restic/restic/pull/3514

View file

@ -1,6 +0,0 @@
Change: Require Go 1.14 or newer
Restic now requires Go 1.14 to build. This allows it to use new
standard library features instead of an external dependency.
https://github.com/restic/restic/issues/3519

View file

@ -1,8 +0,0 @@
Bugfix: Fix handling of `prune --max-repack-size=0`
Restic ignored the `--max-repack-size` option when passing a value of 0. This
has now been fixed.
As a workaround, `--max-repack-size=1` can be used with older versions of restic.
https://github.com/restic/restic/pull/3591

View file

@ -1,10 +0,0 @@
Enhancement: Improve `copy` performance by parallelizing IO
Restic copy previously only used a single thread for copying blobs between
repositories, which resulted in limited performance when copying small blobs
to/from a high latency backend (i.e. any remote backend, especially b2).
Copying will now use 8 parallel threads to increase the throughput of the copy
operation.
https://github.com/restic/restic/pull/3593

View file

@ -1,11 +0,0 @@
Bugfix: Avoid choosing parent snapshots newer than time of new snapshot
The `backup` command, when a `--parent` was not provided, previously chose the
most recent matching snapshot as the parent snapshot. However, this didn't make
sense when the user passed `--time` to create a new snapshot older than the most
recent snapshot.
Instead, `backup` now chooses the most recent snapshot which is not newer than
the snapshot-being-created's timestamp, to avoid any time travel.
https://github.com/restic/restic/pull/3619

View file

@ -1,9 +0,0 @@
Enhancement: Support pruning even when the disk is full
When running out of disk space it was no longer possible to add or remove
data from a repository. To help with recovering from such a deadlock, the
prune command now supports an `--unsafe-recover-no-free-space` option to
recover from these situations. Make sure to read the documentation first!
https://github.com/restic/restic/issues/1153
https://github.com/restic/restic/pull/3481

View file

@ -1,8 +0,0 @@
Change: Support debug log creation in release builds
Creating a debug log was only possible in debug builds which required users to
manually build restic. We changed the release builds to allow creating debug
logs by simply setting the environment variable `DEBUG_LOG=logname.log`.
https://github.com/restic/restic/issues/1842
https://github.com/restic/restic/pull/3826

View file

@ -1,28 +0,0 @@
Enhancement: Add compression support
We've added compression support to the restic repository format. To create a
repository using the new format run `init --repository-version 2`. Please note
that the repository cannot be read by restic versions prior to 0.14.0.
You can configure whether data is compressed with the option `--compression`. It
can be set to `auto` (the default, which will compress very fast), `max` (which
will trade backup speed and CPU usage for better compression), or `off` (which
disables compression). Each setting is only applied for the current run of restic
and does *not* apply to future runs. The option can also be set via the
environment variable `RESTIC_COMPRESSION`.
To upgrade in place run `migrate upgrade_repo_v2` followed by `prune`. See the
documentation for more details. The migration checks the repository integrity
and upgrades the repository format, but will not change any data. Afterwards,
prune will rewrite the metadata to make use of compression.
As an alternative you can use the `copy` command to migrate snapshots; First
create a new repository using
`init --repository-version 2 --copy-chunker-params --repo2 path/to/old/repo`,
and then use the `copy` command to copy all snapshots to the new repository.
https://github.com/restic/restic/issues/21
https://github.com/restic/restic/issues/3779
https://github.com/restic/restic/pull/3666
https://github.com/restic/restic/pull/3704
https://github.com/restic/restic/pull/3733

View file

@ -1,18 +0,0 @@
Enhancement: Adaptive IO concurrency based on backend connections
Many commands used hard-coded limits for the number of concurrent operations.
This prevented speed improvements by increasing the number of connections used
by a backend.
These limits have now been replaced by using the configured number of backend
connections instead, which can be controlled using the
`-o <backend-name>.connections=5` option. Commands will then automatically
scale their parallelism accordingly.
To limit the number of CPU cores used by restic, you can set the environment
variable `GOMAXPROCS` accordingly. For example to use a single CPU core, use
`GOMAXPROCS=1`.
https://github.com/restic/restic/issues/2162
https://github.com/restic/restic/issues/1467
https://github.com/restic/restic/pull/3611

View file

@ -1,8 +0,0 @@
Bugfix: Support `self-update` on Windows
Restic `self-update` would fail in situations where the operating system
locks running binaries, including Windows. The new behavior works around
this by renaming the running file and swapping the updated file in place.
https://github.com/restic/restic/issues/2248
https://github.com/restic/restic/pull/3675

View file

@ -1,12 +0,0 @@
Enhancement: Allow pack size customization
Restic now uses a target pack size of 16 MiB by default. This can be customized
using the `--pack-size size` option. Supported pack sizes range between 4 and
128 MiB.
It is possible to migrate an existing repository to _larger_ pack files using
`prune --repack-small`. This will rewrite every pack file which is
significantly smaller than the target size.
https://github.com/restic/restic/issues/2291
https://github.com/restic/restic/pull/3731

View file

@ -1,14 +0,0 @@
Enhancement: Allow use of SAS token to authenticate to Azure
Previously restic only supported AccountKeys to authenticate to Azure
storage accounts, which necessitates giving a significant amount of
access.
We added support for Azure SAS tokens which are a more fine-grained
and time-limited manner of granting access. Set the `AZURE_ACCOUNT_NAME`
and `AZURE_ACCOUNT_SAS` environment variables to use a SAS token for
authentication. Note that if `AZURE_ACCOUNT_KEY` is set, it will take
precedence.
https://github.com/restic/restic/issues/2295
https://github.com/restic/restic/pull/3661

View file

@ -1,13 +0,0 @@
Enhancement: Improve backup speed with many small files
We have restructured the backup pipeline to continue reading files while all
upload connections are busy. This allows the backup to already prepare the next
data file such that the upload can continue as soon as a connection becomes
available. This can especially improve the backup performance for high latency
backends.
The upload concurrency is now controlled using the `-o <backend-name>.connections=5`
option.
https://github.com/restic/restic/issues/2696
https://github.com/restic/restic/pull/3489

View file

@ -1,15 +0,0 @@
Enhancement: Make snapshot directory structure of `mount` command customizable
We've added the possibility to customize the snapshot directory structure of
the `mount` command using templates passed to the `--snapshot-template` option.
The formatting of snapshots' timestamps is now controlled using `--time-template`
and supports subdirectories to for example group snapshots by year. Please
see `restic help mount` for further details.
Characters in tag names which are not allowed in a filename are replaced by
underscores `_`. For example a tag `foo/bar` will result in a directory name
of `foo_bar`.
https://github.com/restic/restic/issues/2907
https://github.com/restic/restic/pull/2913
https://github.com/restic/restic/pull/3691

View file

@ -1,12 +0,0 @@
Enhancement: Optimize handling of duplicate blobs in `prune`
Restic `prune` always used to repack all data files containing duplicate
blobs. This effectively removed all duplicates during prune. However, as a
consequence all these data files were repacked even if the unused repository
space threshold could be reached with less work.
This is now changed and `prune` works nice and fast even when there are lots
of duplicate blobs.
https://github.com/restic/restic/issues/3114
https://github.com/restic/restic/pull/3290

View file

@ -1,13 +0,0 @@
Change: Deprecate `check --check-unused` and add further checks
Since restic 0.12.0, it is expected to still have unused blobs after running
`prune`. This made the `--check-unused` option of the `check` command rather
useless and tended to confuse users. This option has been deprecated and is
now ignored.
The `check` command now also warns if a repository is using either the legacy
S3 layout or mixed pack files with both tree and data blobs. The latter is
known to cause performance problems.
https://github.com/restic/restic/issues/3295
https://github.com/restic/restic/pull/3730

View file

@ -1,14 +0,0 @@
Bugfix: List snapshots in backend at most once to resolve snapshot IDs
Many commands support specifying a list of snapshot IDs which are then used to
determine the snapshots to be processed by the command. To resolve snapshot IDs
or `latest`, and check that these exist, restic previously listed all snapshots
stored in the repository. Depending on the backend this could be a slow and/or
expensive operation.
Restic now lists the snapshots only once and remembers the result in order to
resolve all further snapshot IDs swiftly.
https://github.com/restic/restic/issues/3428
https://github.com/restic/restic/pull/3570
https://github.com/restic/restic/pull/3395

View file

@ -1,14 +0,0 @@
Bugfix: Fix rare 'not found in repository' error for `copy` command
In rare cases `copy` (and other commands) would report that `LoadTree(...)`
returned an `id [...] not found in repository` error. This could be caused by
a backup or copy command running concurrently. The error was only temporary;
running the failed restic command a second time as a workaround did resolve the
error.
This issue has now been fixed by correcting the order in which restic reads data
from the repository. It is now guaranteed that restic only loads snapshots for
which all necessary data is already available.
https://github.com/restic/restic/issues/3432
https://github.com/restic/restic/pull/3570

View file

@ -1,10 +0,0 @@
Enhancement: Improve handling of temporary files on Windows
In some cases restic failed to delete temporary files, causing the current
command to fail. This has now been fixed by ensuring that Windows automatically
deletes the file. In addition, temporary files are only written to disk when
necessary, reducing disk writes.
https://github.com/restic/restic/issues/3465
https://github.com/restic/restic/issues/1551
https://github.com/restic/restic/pull/3610

View file

@ -1,7 +0,0 @@
Bugfix: The `diff` command incorrectly listed some files as added
There was a bug in the `diff` command, causing it to always show files in a
removed directory as added. This has now been fixed.
https://github.com/restic/restic/issues/3685
https://github.com/restic/restic/pull/3686

View file

@ -1,13 +0,0 @@
Bugfix: Fix rclone (shimmed by Scoop) and sftp not working on Windows
In #3602 a fix was introduced to address the problem of `rclone` prematurely
exiting when Ctrl+C is pressed on Windows. The solution was to create the
subprocess with its console detached from the restic console.
However, this solution failed when using `rclone` installed by Scoop or using
`sftp` with a passphrase-protected private key. We've now fixed this by using
a different approach to prevent Ctrl-C from passing down too early.
https://github.com/restic/restic/issues/3681
https://github.com/restic/restic/issues/3692
https://github.com/restic/restic/pull/3696

View file

@ -1,11 +0,0 @@
Enhancement: Validate exclude patterns before backing up
Exclude patterns provided via `--exclude`, `--iexclude`, `--exclude-file` or
`--iexclude-file` previously weren't validated. As a consequence, invalid
patterns resulted in files that were meant to be excluded being backed up.
Restic now validates all patterns before running the backup and aborts with
a fatal error if an invalid pattern is detected.
https://github.com/restic/restic/issues/3709
https://github.com/restic/restic/pull/3734

View file

@ -1,13 +0,0 @@
Bugfix: Directory sync errors for repositories accessed via SMB
On Linux and macOS, accessing a repository via a SMB/CIFS mount resulted in
restic failing to save the lock file, yielding the following errors:
Save(<lock/071fe833f0>) returned error, retrying after 552.330144ms: sync /repo/locks: no such file or directory
Save(<lock/bf789d7343>) returned error, retrying after 552.330144ms: sync /repo/locks: invalid argument
This has now been fixed by ignoring the relevant error codes.
https://github.com/restic/restic/issues/3720
https://github.com/restic/restic/issues/3751
https://github.com/restic/restic/pull/3752

Some files were not shown because too many files have changed in this diff Show more