Commit graph

8708 commits

Author SHA1 Message Date
Srigovind Nayak
db686592a1
debug: azure add debug log to show access-tier 2024-10-20 20:24:49 +05:30
Srigovind Nayak
bff3341d10
azure: add support for hot, cool, or cool access tiers 2024-10-20 15:27:21 +05:30
Michael Eischer
5fe6607127
Merge pull request #5084 from greatroar/utimesnano
Simplify and refactor restoring of timestamps
2024-10-19 12:47:13 +00:00
greatroar
8f20d5dcd5 fs: Refactor UtimesNano replacements
Previously, nodeRestoreTimestamps would do something like

	if node.Type == restic.NodeTypeSymlink {
	    return nodeRestoreSymlinkTimestamps(...)
	}
	return syscall.UtimesNano(...)

where nodeRestoreSymlinkTimestamps was either a no-op or a
reimplementation of syscall.UtimesNano that handles symlinks, with some
repeated converting between timestamp types. The Linux implementation
was a bit clumsy, requiring three syscalls to set the timestamps.

In this new setup, there is a function utimesNano that has three
implementations:

* on Linux, it's a modified syscall.UtimesNano that uses
  AT_SYMLINK_NOFOLLOW and AT_FDCWD so it can handle any type in a single
  call;
* on other Unix platforms, it just calls the syscall function after
  skipping symlinks;
* on Windows, it's the modified UtimesNano that was previously called
  nodeRestoreSymlinkTimestamps, except with different arguments.
2024-10-19 12:04:09 +02:00
greatroar
f967a33ccc fs: Use AT_FDCWD in Linux nodeRestoreSymlinkTimestamps
There's no need to open the containing directory. This is exactly what
syscall.UtimesNano does, except for the AT_SYMLINK_NOFOLLOW flag.
2024-10-19 11:29:35 +02:00
Michael Eischer
ee9a5cdf70 add vss metadata changelog 2024-10-18 22:51:55 +02:00
Michael Eischer
46dce1f4fa backup: work around file deletion error in test 2024-10-18 22:51:55 +02:00
Michael Eischer
841f8bfef0 redirect test log output to t.Log() 2024-10-18 22:51:55 +02:00
Michael Eischer
1f5791222a backup: test that vss backups work if underlying data was removed 2024-10-18 22:51:55 +02:00
Michael Eischer
ec43594003 add vss metadata changelog 2024-10-18 22:36:03 +02:00
Michael Eischer
a7b13bd603 fs: remove file.Name() from interface
The only user was archiver.fileSaver.
2024-10-18 22:29:03 +02:00
Michael Eischer
0c711f5605 archiver: use correct filepath in fileSaver for vss
When using the VSS FS, then `f.Name()` contained the filename in the
snapshot. This caused a double mapping when calling NodeFromFileInfo.
2024-10-18 22:29:03 +02:00
Michael Eischer
4df2e33568 archiver: properly create node for vss backups
Previously, NodeFromFileInfo used the original file path to create the
node, which also meant that extended metadata was read from there
instead of within the vss snapshot.

This change is a temporary solution for restic 0.17.2 and will be
replaced with a clean fix in restic 0.18.0.
2024-10-18 22:26:18 +02:00
Michael Eischer
11c1fbce20
Merge pull request #5098 from MichaelEischer/prepare-patch-release
Prepare patch release
2024-10-18 22:20:27 +02:00
Michael Eischer
e1faf7b18c backup: work around file deletion error in test 2024-10-18 22:08:10 +02:00
Connor Findlay
9553d873ff backend/azure: Add tests for both token types
Add two new test cases, TestBackendAzureAccountToken and
TestBackendAzureContainerToken, that ensure that the authorization using
both types of token works.

This introduces two new environment variables,
RESTIC_TEST_AZURE_ACCOUNT_SAS and RESTIC_TEST_AZURE_CONTAINER_SAS, that
contain the tokens to use when testing restic. If an environment
variable is missing, the related test is skipped.
2024-10-18 21:59:03 +02:00
Connor Findlay
048c3bb240 changelog: Add changes in issue-4004
Add changelog entry in the 'unreleased' sub-folder for changes
introduced when fixing issue #4004.
2024-10-18 21:59:03 +02:00
Connor Findlay
d6e76a22a8 backend/azure: Handle Container SAS/SAT
Ignore AuthorizationFailure caused by using a container level SAS/SAT
token when calling GetProperties during the Create() call. This is because the
GetProperties call expects an Account Level token, and the container
level token simply lacks the appropriate permissions. Supressing the
Authorization Failure is OK, because if the token is actually invalid,
this is caught elsewhere when we try to actually use the token to do
work.
2024-10-18 21:59:03 +02:00
Michael Eischer
e3a022f9b5 add irregular files bug changelog 2024-10-18 21:58:04 +02:00
Michael Eischer
fe269c752a repair snapshots: remove irregular files 2024-10-18 21:57:52 +02:00
Michael Eischer
fc1fc00aa4 backup: exclude irregular files from backup
restic cannot backup irregular files as those don't behave like normal
files. Thus skip them with an error.
2024-10-18 21:56:41 +02:00
greatroar
3c82fe6ef5 fs: Include filename in mknod errors 2024-10-18 21:53:15 +02:00
Michael Eischer
986d981bf6 tag: fix swallowed error if repository cannot be opened 2024-10-18 21:50:29 +02:00
Michael Eischer
0df2fa8135 fs: retry preallocate on Linux if interrupted by signal 2024-10-18 21:47:59 +02:00
Roman Inflianskas
49ccb7734c list: validate subcommand 2024-10-18 21:47:59 +02:00
Roman Inflianskas
491cc65e3a list: add subcommand completion 2024-10-18 21:47:59 +02:00
Damien Clark
8c1d6a50c1 cache: fix race condition in cache cleanup
Fix multiple restic processes executing concurrently and racing to remove obsolete snapshots.

Co-authored-by: Michael Eischer <michael.eischer@fau.de>
2024-10-18 21:47:59 +02:00
Michael Eischer
9386acc4a6 Fix indentation of blockquotes in github release notes 2024-10-18 21:47:59 +02:00
Git'Fellow
5b60d49654 fix: shorten sentence 2024-10-18 21:47:59 +02:00
Git'Fellow
8056181301 docs: Recommend to setup B2 versions lifecycle rules 2024-10-18 21:46:58 +02:00
Michael Eischer
fc6f1b4b06 redirect test log output to t.Log() 2024-10-18 21:43:46 +02:00
Michael Eischer
9f206601af backup: test that vss backups work if underlying data was removed 2024-10-18 21:43:46 +02:00
Michael Eischer
ca79cb92e3 fs/vss: test that vss functions actually read from snapshot 2024-10-18 21:43:46 +02:00
Michael Eischer
352605d9f0 fs: remove file.Name() from interface
The only user was archiver.fileSaver.
2024-10-18 21:43:23 +02:00
Michael Eischer
26b77a543d archiver: use correct filepath in fileSaver for vss
When using the VSS FS, then `f.Name()` contained the filename in the
snapshot. This caused a double mapping when calling NodeFromFileInfo.
2024-10-18 21:41:02 +02:00
Michael Eischer
b988754a6d fs/vss: reuse functions from underlying FS
OpenFile, Stat and Lstat should reuse the underlying FS implementation
to avoid diverging behavior.
2024-10-18 19:30:05 +02:00
Michael Eischer
60960d2405 fs/vss: properly create node from vss path
Previously, NodeFromFileInfo used the original file path to create the
node, which also meant that extended metadata was read from there
instead of within the vss snapshot.
2024-10-18 19:27:44 +02:00
Michael Eischer
7c02141548
Merge pull request #5093 from Seefin/fix-containerSAS
Fix Azure Container Token Auth
2024-10-17 18:45:06 +00:00
Connor Findlay
b434f560cc backend/azure: Add tests for both token types
Add two new test cases, TestBackendAzureAccountToken and
TestBackendAzureContainerToken, that ensure that the authorization using
both types of token works.

This introduces two new environment variables,
RESTIC_TEST_AZURE_ACCOUNT_SAS and RESTIC_TEST_AZURE_CONTAINER_SAS, that
contain the tokens to use when testing restic. If an environment
variable is missing, the related test is skipped.
2024-10-17 20:38:03 +02:00
Connor Findlay
7bdfcf13fb changelog: Add changes in issue-4004
Add changelog entry in the 'unreleased' sub-folder for changes
introduced when fixing issue #4004.
2024-10-17 20:38:03 +02:00
Connor Findlay
2e704c69ac backend/azure: Handle Container SAS/SAT
Ignore AuthorizationFailure caused by using a container level SAS/SAT
token when calling GetProperties during the Create() call. This is because the
GetProperties call expects an Account Level token, and the container
level token simply lacks the appropriate permissions. Supressing the
Authorization Failure is OK, because if the token is actually invalid,
this is caught elsewhere when we try to actually use the token to do
work.
2024-10-17 20:38:03 +02:00
Michael Eischer
5838896962 doc: document safety feature for --target / --delete 2024-10-17 19:45:03 +02:00
Michael Eischer
bcd5ac34bb
Merge pull request #5060 from MichaelEischer/proper-nodefromfileinfo
fs: move NodeFromFileInfo into FS interface
2024-10-16 21:34:37 +02:00
Michael Eischer
618f306f13
Merge pull request #5054 from phillipp/dump-compress-zip
dump: add --compress flag to compress archives
2024-10-16 19:17:47 +00:00
Michael Eischer
75711446e1 fs: move NodeFromFileInfo into FS interface 2024-10-16 21:17:21 +02:00
Michael Eischer
c3b3120e10
Merge pull request #5057 from MichaelEischer/fix-backup-irregular
backup: fix handling of files with type irregular
2024-10-16 21:13:08 +02:00
Michael Eischer
e29d38f8bf dump/zip: test that files are compressed 2024-10-16 21:11:24 +02:00
Michael Eischer
da3c02405b dump/zip: only compress regular files 2024-10-16 21:09:05 +02:00
Michael Eischer
55c150054d add irregular files bug changelog 2024-10-16 20:54:08 +02:00
Michael Eischer
012cb06fe9 repair snapshots: remove irregular files 2024-10-16 20:54:08 +02:00