KurlesHS/rclone
1
0
Fork 0
forked from TrueCloudLab/rclone
Code Pull requests Activity

docs: note use of regexp filtering prevents directory optimisation

Browse source 
Discussed in the forum:
https://forum.rclone.org/t/rclone-regex-in-filter-causes-spurious-directory-catch-all-filter/30985

Signed-off-by: Matthew Vernon <mvernon@wikimedia.org>
This commit is contained in:
Matthew Vernon 2022-06-07 15:05:24 +01:00 committed by Nick Craig-Wood
parent a9c49c50a0
commit e11bfacfcf
1 changed files with 10 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
docs/content/filtering.md
View file

@ -131,7 +131,11 @@ The regular expressions used are as defined in the [Go regular
expression reference](https://golang.org/pkg/regexp/syntax/). Regular expression reference](https://golang.org/pkg/regexp/syntax/). Regular
expressions should be enclosed in `{{` `}}`. They will match only the expressions should be enclosed in `{{` `}}`. They will match only the
last path segment if the glob doesn't start with `/` or the whole path last path segment if the glob doesn't start with `/` or the whole path
name if it does. name if it does. Note that rclone does not attempt to parse the
supplied regular expression, meaning that using any regular expression
filter will prevent rclone from using [directory filter rules](#directory_filter),
as it will instead check every path against
the supplied regular expression(s).
Here is how the `{{regexp}}` is transformed into an full regular Here is how the `{{regexp}}` is transformed into an full regular
expression to match the entire path: expression to match the entire path:
@ -247,7 +251,7 @@ currently a means provided to pass regular expression filter options into
rclone directly though character class filter rules contain character rclone directly though character class filter rules contain character
classes. [Go regular expression reference](https://golang.org/pkg/regexp/syntax/) classes. [Go regular expression reference](https://golang.org/pkg/regexp/syntax/)
### How filter rules are applied to directories ### How filter rules are applied to directories {#directory_filter}
Rclone commands are applied to path/file names not Rclone commands are applied to path/file names not
directories. The entire contents of a directory can be matched directories. The entire contents of a directory can be matched
@ -263,6 +267,10 @@ recurse into subdirectories. This potentially optimises access to a remote
by avoiding listing unnecessary directories. Whether optimisation is by avoiding listing unnecessary directories. Whether optimisation is
desirable depends on the specific filter rules and source remote content. desirable depends on the specific filter rules and source remote content.
If any [regular expression filters](#regexp) are in use, then no
directory recursion optimisation is possible, as rclone must check
every path against the supplied regular expression(s).
Directory recursion optimisation occurs if either: Directory recursion optimisation occurs if either:
* A source remote does not support the rclone `ListR` primitive. local, * A source remote does not support the rclone `ListR` primitive. local,