KurlesHS/restic
1
0
Fork 0
forked from TrueCloudLab/restic
Code Pull requests Activity

Merge pull request #5017 from MichaelEischer/rewrite-data-loss

Browse source 
rewrite: Document handling of "cannot encode tree" errors
This commit is contained in:
Michael Eischer 2024-08-29 16:57:13 +02:00 committed by GitHub
commit dab3e549af
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 20 additions and 9 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
doc/045_working_with_repos.rst
View file

@ -305,6 +305,13 @@ In order to preview the changes which ``rewrite`` would make, you can use the
modifying the repository. Instead restic will only print the actions it would modifying the repository. Instead restic will only print the actions it would
perform. perform.
.. note:: The ``rewrite`` command verifies that it does not modify snapshots in
unexpected ways and fails with an ``cannot encode tree at "[...]" without loosing information``
error otherwise. This can occur when rewriting a snapshot created by a newer
version of restic or some third-party implementation.
To convert a snapshot into the format expected by the ``rewrite`` command
use ``restic repair snapshots <snapshotID>``.
Modifying metadata of snapshots Modifying metadata of snapshots
=============================== ===============================

22
doc/design.rst
View file

@ -126,8 +126,8 @@ the option ``-o local.layout=default``, valid values are ``default`` and
``s3legacy``. The option for the sftp backend is named ``sftp.layout``, for the ``s3legacy``. The option for the sftp backend is named ``sftp.layout``, for the
s3 backend ``s3.layout``. s3 backend ``s3.layout``.
S3 Legacy Layout S3 Legacy Layout (deprecated)
---------------- -----------------------------
Unfortunately during development the Amazon S3 backend uses slightly different Unfortunately during development the Amazon S3 backend uses slightly different
paths (directory names use singular instead of plural for ``key``, paths (directory names use singular instead of plural for ``key``,
@ -152,8 +152,7 @@ the ``data`` directory. The S3 Legacy repository layout looks like this:
/snapshot /snapshot
└── 22a5af1bdc6e616f8a29579458c49627e01b32210d09adb288d1ecda7c5711ec └── 22a5af1bdc6e616f8a29579458c49627e01b32210d09adb288d1ecda7c5711ec
The S3 backend understands and accepts both forms, new backends are Restic 0.17 is the last version that supports the legacy layout.
always created with the default layout for compatibility reasons.
Pack Format Pack Format
=========== ===========
@ -234,7 +233,9 @@ Individual files for the index, locks or snapshots are encrypted
and authenticated like Data and Tree Blobs, so the outer structure is and authenticated like Data and Tree Blobs, so the outer structure is
``IV || Ciphertext || MAC`` again. In repository format version 1 the ``IV || Ciphertext || MAC`` again. In repository format version 1 the
plaintext always consists of a JSON document which must either be an plaintext always consists of a JSON document which must either be an
object or an array. object or an array. The JSON encoder must deterministically encode the
document and should match the behavior of the Go standard library implementation
in ``encoding/json``.
Repository format version 2 adds support for compression. The plaintext Repository format version 2 adds support for compression. The plaintext
now starts with a header to indicate the encoding version to distinguish now starts with a header to indicate the encoding version to distinguish
@ -473,6 +474,10 @@ A snapshot references a tree by the SHA-256 hash of the JSON string
representation of its contents. Trees and data are saved in pack files representation of its contents. Trees and data are saved in pack files
in a subdirectory of the directory ``data``. in a subdirectory of the directory ``data``.
The JSON encoder must deterministically encode the document and should
match the behavior of the Go standard library implementation in ``encoding/json``.
This ensures that trees can be properly deduplicated.
The command ``restic cat blob`` can be used to inspect the tree The command ``restic cat blob`` can be used to inspect the tree
referenced above (piping the output of the command to ``jq .`` so that referenced above (piping the output of the command to ``jq .`` so that
the JSON is indented): the JSON is indented):
@ -507,12 +512,11 @@ this metadata is generated:
- The name is quoted using `strconv.Quote <https://pkg.go.dev/strconv#Quote>`__ - The name is quoted using `strconv.Quote <https://pkg.go.dev/strconv#Quote>`__
before being saved. This handles non-unicode names, but also changes the before being saved. This handles non-unicode names, but also changes the
representation of names containing ``"`` or ``\``. representation of names containing ``"`` or ``\``.
- The filemode saved is the mode defined by `fs.FileMode <https://pkg.go.dev/io/fs#FileMode>`__ - The filemode saved is the mode defined by `fs.FileMode <https://pkg.go.dev/io/fs#FileMode>`__
masked by ``os.ModePerm | os.ModeType | os.ModeSetuid | os.ModeSetgid | os.ModeSticky`` masked by ``os.ModePerm | os.ModeType | os.ModeSetuid | os.ModeSetgid | os.ModeSticky``
- When the entry references a directory, the field ``subtree`` contains the plain text
When the entry references a directory, the field ``subtree`` contains the plain text ID of another tree object.
ID of another tree object. - Check the implementation for a full struct definition.
When the command ``restic cat blob`` is used, the plaintext ID is needed When the command ``restic cat blob`` is used, the plaintext ID is needed
to print a tree. The tree referenced above can be dumped as follows: to print a tree. The tree referenced above can be dumped as follows: