From aaa8b7738a78a3de5a6b1dba86ba1fe09db8a6cd Mon Sep 17 00:00:00 2001 From: buengese Date: Wed, 11 Nov 2020 19:56:16 +0100 Subject: [PATCH] compress: initial documentation --- docs/content/_index.md | 1 + docs/content/compress.md | 136 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 137 insertions(+) create mode 100644 docs/content/compress.md diff --git a/docs/content/_index.md b/docs/content/_index.md index 639120397..2ed8383cb 100644 --- a/docs/content/_index.md +++ b/docs/content/_index.md @@ -44,6 +44,7 @@ using local disk. Virtual backends wrap local and cloud file systems to apply [encryption](/crypt/), [caching](/cache/), +[compression](/compress/) [chunking](/chunker/) and [joining](/union/). diff --git a/docs/content/compress.md b/docs/content/compress.md new file mode 100644 index 000000000..992f9f455 --- /dev/null +++ b/docs/content/compress.md @@ -0,0 +1,136 @@ +--- +title: "Compress" +description: "Compression Remote" +date: "2020-09-14" +--- + +Compress (Experimental) +----------------------------------------- + +The `Compress` remote adds compression to another remote. It is best used with remotes containing +many large compressible files. + +Please read the [warnings](#warnings) before using this remote. + +To use this remote, all you need to do is specify another remote and a compression mode to use: + +``` +Current remotes: + +Name Type +==== ==== +remote_to_press sometype + +e) Edit existing remote +$ rclone config +n) New remote +d) Delete remote +r) Rename remote +c) Copy remote +s) Set configuration password +q) Quit config +e/n/d/r/c/s/q> n +name> compress +... + 8 / Compress a remote + \ "compress" +... +Storage> compress +** See help for compress backend at: https://rclone.org/compress/ ** + +Remote to compress. +Enter a string value. Press Enter for the default (""). +remote> remote_to_press:subdir +Compression mode. +Enter a string value. Press Enter for the default ("gzip"). +Choose a number from below, or type in your own value + 1 / Gzip compression balanced for speed and compression strength. + \ "gzip" +compression_mode> gzip +Edit advanced config? (y/n) +y) Yes +n) No (default) +y/n> n +Remote config +-------------------- +[compress] +type = compress +remote = remote_to_press:subdir +compression_mode = gzip +-------------------- +y) Yes this is OK (default) +e) Edit this remote +d) Delete this remote +y/e/d> y +``` + +### Compression Modes +Currently only gzip compression is supported, it provides a decent balance between speed and strength and is well +supported by other application. Compression strength can further be configured via an advanced setting where 0 is no +compression and 9 is strongest compression. + +#### Filetype +If you open a remote wrapped by press, you will see that there are many files with an extension corresponding to +the compression algorithm you chose. These files are standard files that can be opened by various archive programs, +but they have some hidden metadata that allows them to be used by rclone. +While you may download and decompress these files at will, do **not** manually delete or rename files. Files without +correct metadata files will not be recognized by rclone. + +### File names + +The compressed files will be named `*.###########.gz` where `*` is the base file and the `#` part is base64 encoded +size of the uncompressed file. The file names should not be changed by anything other than the rclone compression backend. + +#### Experimental +This remote is currently **experimental**. Things may break and data may be lost. Anything you do with this remote is +at your own risk. Please understand the risks associated with using experimental code and don't use this remote in +critical applications. + +{{< rem autogenerated options start" - DO NOT EDIT - instead edit fs.RegInfo in backend/compress/compress.go then run make backenddocs" >}} +### Standard Options + +Here are the standard options specific to compress (Compress a remote). + +#### --compress-remote + +Remote to compress. + +- Config: remote +- Env Var: RCLONE_COMPRESS_REMOTE +- Type: string +- Default: "" + +#### --compress-mode + +Compression mode. + +- Config: mode +- Env Var: RCLONE_COMPRESS_MODE +- Type: string +- Default: "gzip" +- Examples: + - "gzip" + - Standard gzip compression with fastest parameters. + +### Advanced Options + +Here are the advanced options specific to compress (Compress a remote). + +#### --compress-level + +GZIP compression level (-2 to 9). + + Generally -1 (default, equivalent to 5) is recommended. + Levels 1 to 9 increase compressiong at the cost of speed.. Going past 6 + generally offers very little return. + + Level -2 uses Huffmann encoding only. Only use if you now what you + are doing + Level 0 turns off compression. + +- Config: level +- Env Var: RCLONE_COMPRESS_LEVEL +- Type: int +- Default: -1 + +{{< rem autogenerated options stop >}}