KurlesHS/rclone
1
0
Fork 0
forked from TrueCloudLab/rclone
Code Pull requests Activity
rclone/docs/content/frostfs.md
Aleksey Kravchenko 0255757831 [#1] Add frostfs backend docs 
Signed-off-by: Aleksey Kravchenko <al.kravchenko@yadro.com>
2025-01-16 11:11:33 +03:00

466 lines
12 KiB
Markdown
Raw Blame History

---
title: "FrostFS"
description: "Rclone docs for FrostFS backend"
versionIntroduced: "---"
---
# {{< icon "fa fa-file" >}} FrostFS
Rclone FrostFS support is provided using the
[git.frostfs.info/TrueCloudLab/frostfs-sdk-go](https://git.frostfs.info/TrueCloudLab/frostfs-sdk-go) package
## Configuration
To create an FrostFS configuration named `remote`, run
rclone config
Rclone config guides you through an interactive setup process. A minimal
rclone FrostFS remote definition only requires endpoint and path to FrostFS user wallet.
```
No remotes found, make a new one?
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
Enter name for new remote.
name> remote
Option Storage.
Type of storage to configure.
Choose a number from below, or type in your own value.
1 / 1Fichier
\ (fichier)
2 / Akamai NetStorage
\ (netstorage)
3 / Alias for an existing remote
\ (alias)
4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, ArvanCloud, Ceph, ChinaMobile, Cloudflare, DigitalOcean, Dreamhost, GCS, HuaweiOBS, IBMCOS, IDrive, IONOS, LyveCloud, Leviia, Liara, Linode, Magalu, Minio, Netease, Petabox, RackCorp, Rclone, Scaleway, SeaweedFS, StackPath, Storj, Synology, TencentCOS, Wasabi, Qiniu and others
\ (s3)
5 / Backblaze B2
\ (b2)
6 / Better checksums for other remotes
\ (hasher)
7 / Box
\ (box)
8 / Cache a remote
\ (cache)
9 / Citrix Sharefile
\ (sharefile)
10 / Combine several remotes into one
\ (combine)
11 / Compress a remote
\ (compress)
12 / Distributed, decentralized object storage FrostFS
\ (frostfs)
13 / Dropbox
\ (dropbox)
14 / Encrypt/Decrypt a remote
\ (crypt)
15 / Enterprise File Fabric
\ (filefabric)
16 / FTP
\ (ftp)
17 / Files.com
\ (filescom)
18 / Gofile
\ (gofile)
19 / Google Cloud Storage (this is not Google Drive)
\ (google cloud storage)
20 / Google Drive
\ (drive)
21 / Google Photos
\ (google photos)
22 / HTTP
\ (http)
23 / Hadoop distributed file system
\ (hdfs)
24 / HiDrive
\ (hidrive)
25 / ImageKit.io
\ (imagekit)
26 / In memory object storage system.
\ (memory)
27 / Internet Archive
\ (internetarchive)
28 / Jottacloud
\ (jottacloud)
29 / Koofr, Digi Storage and other Koofr-compatible storage providers
\ (koofr)
30 / Linkbox
\ (linkbox)
31 / Local Disk
\ (local)
32 / Mail.ru Cloud
\ (mailru)
33 / Mega
\ (mega)
34 / Microsoft Azure Blob Storage
\ (azureblob)
35 / Microsoft Azure Files
\ (azurefiles)
36 / Microsoft OneDrive
\ (onedrive)
37 / OpenDrive
\ (opendrive)
38 / OpenStack Swift (Rackspace Cloud Files, Blomp Cloud Storage, Memset Memstore, OVH)
\ (swift)
39 / Oracle Cloud Infrastructure Object Storage
\ (oracleobjectstorage)
40 / Pcloud
\ (pcloud)
41 / PikPak
\ (pikpak)
42 / Pixeldrain Filesystem
\ (pixeldrain)
43 / Proton Drive
\ (protondrive)
44 / Put.io
\ (putio)
45 / QingCloud Object Storage
\ (qingstor)
46 / Quatrix by Maytech
\ (quatrix)
47 / SMB / CIFS
\ (smb)
48 / SSH/SFTP
\ (sftp)
49 / Sia Decentralized Cloud
\ (sia)
50 / Storj Decentralized Cloud Storage
\ (storj)
51 / Sugarsync
\ (sugarsync)
52 / Transparently chunk/split large files
\ (chunker)
53 / Uloz.to
\ (ulozto)
54 / Union merges the contents of several upstream fs
\ (union)
55 / Uptobox
\ (uptobox)
56 / WebDAV
\ (webdav)
57 / Yandex Disk
\ (yandex)
58 / Zoho
\ (zoho)
59 / premiumize.me
\ (premiumizeme)
60 / seafile
\ (seafile)
Storage> frostfs
Option endpoint.
Endpoints to connect to FrostFS node
Choose a number from below, or type in your own value.
1 / One endpoint.
\ (s01.frostfs.devenv:8080)
2 / Multiple endpoints to form pool.
\ (s01.frostfs.devenv:8080 s02.frostfs.devenv:8080)
3 / Multiple endpoints with priority (less value is higher priority). Until s01 is healthy all request will be send to it.
\ (s01.frostfs.devenv:8080,1 s02.frostfs.devenv:8080,2)
4 / Multiple endpoints with priority and weights. After s01 is unhealthy requests will be send to s02 and s03 in proportions 10% and 90% respectively.
\ (s01.frostfs.devenv:8080,1,1 s02.frostfs.devenv:8080,2,1 s03.frostfs.devenv:8080,2,9)
endpoint> s01.frostfs.devenv:8080,1 s02.frostfs.devenv:8080,2
Option connection_timeout.
FrostFS connection timeout
Enter a value of type Duration. Press Enter for the default (4s).
connection_timeout>
Option request_timeout.
FrostFS request timeout
Enter a value of type Duration. Press Enter for the default (12s).
request_timeout>
Option rebalance_interval.
FrostFS rebalance connections interval
Enter a value of type Duration. Press Enter for the default (15s).
rebalance_interval>
Option session_expiration.
FrostFS session expiration epoch
Enter a signed integer. Press Enter for the default (4294967295).
session_expiration>
Option ape_cache_invalidation_duration.
APE cache invalidation duration
Enter a value of type Duration. Press Enter for the default (8s).
ape_cache_invalidation_duration>
Option ape_cache_invalidation_timeout.
APE cache invalidation timeout
Enter a value of type Duration. Press Enter for the default (24s).
ape_cache_invalidation_timeout>
Option ape_chain_check_interval.
The interval for verifying that the APE chain is saved in FrostFS.
Enter a value of type Duration. Press Enter for the default (500ms).
ape_chain_check_interval>
Option rpc_endpoint.
Endpoint to connect to Neo rpc node
Enter a value. Press Enter to leave empty.
rpc_endpoint>
Option wallet.
Path to wallet
Enter a value.
wallet> /wallets/wallet.conf
Option address.
Address of account
Enter a value. Press Enter to leave empty.
address>
Option password.
Password to decrypt wallet
Enter a value. Press Enter to leave empty.
password>
Option placement_policy.
Placement policy for new containers
Choose a number from below, or type in your own value of type string.
Press Enter for the default (REP 3).
1 / Container will have 3 replicas
\ (REP 3)
placement_policy> REP 1
Option container_creation_policy.
Container creation policy for new containers
Choose a number from below, or type in your own value of type string.
Press Enter for the default (private).
1 / Public container, anyone can read and write
\ (public-read-write)
2 / Public container, owner can read and write, others can only read
\ (public-read)
3 / Private container, only owner has access to it
\ (private)
container_creation_policy>
Edit advanced config?
y) Yes
n) No (default)
y/n> n
Configuration complete.
Options:
- type: frostfs
- endpoint: s01.frostfs.devenv:8080,1 s02.frostfs.devenv:8080,2
- password: M@zPGpWDravchenko/develop/go/playground/play_with_ape/cfg/wallet_akrav.js
- placement_policy: REP 1
Keep this "remote" remote?
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y
```
As a remote root directory, you can use either the container identifier or its user-friendly name.
If you choose to use a container name, rclone will create a new container with that name when
it's used for the first time.
For example, both of the following commands would be valid if there was a `~/test-copy` directory and a container with
the identifier `23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf`:
rclone copy ~/test-copy remote:23fk3Bcw5mPZ4YtYkTLJbQebtM2WXHz4HL8FgsrTJkSf/test-copy
rclone copy ~/test-copy remote:container-name/test-copy
{{< rem autogenerated options start" - DO NOT EDIT - instead edit fs.RegInfo in backend/frostfs/frostfs.go then run make backenddocs" >}}
### Standard options
Here are the Standard options specific to frostfs (Distributed, decentralized object storage FrostFS).
#### --frostfs-endpoint
Endpoints to connect to FrostFS node
Properties:
- Config: endpoint
- Env Var: RCLONE_FROSTFS_ENDPOINT
- Type: string
- Required: true
- Examples:
- "s01.frostfs.devenv:8080"
- One endpoint.
- "s01.frostfs.devenv:8080 s02.frostfs.devenv:8080"
- Multiple endpoints to form pool.
- "s01.frostfs.devenv:8080,1 s02.frostfs.devenv:8080,2"
- Multiple endpoints with priority (lower value has higher priority). Until s01 is healthy, all requests will be sent to it.
- "s01.frostfs.devenv:8080,1,1 s02.frostfs.devenv:8080,2,1 s03.frostfs.devenv:8080,2,9"
- Multiple endpoints with priority and weights. After the s01 endpoint is unhealthy, requests will be sent to the s02 and s03 endpoints in proportions of 10% and 90%, respectively.
#### --frostfs-connection-timeout
FrostFS connection timeout
Properties:
- Config: connection_timeout
- Env Var: RCLONE_FROSTFS_CONNECTION_TIMEOUT
- Type: Duration
- Default: 4s
#### --frostfs-request-timeout
FrostFS request timeout
Properties:
- Config: request_timeout
- Env Var: RCLONE_FROSTFS_REQUEST_TIMEOUT
- Type: Duration
- Default: 12s
#### --frostfs-rebalance-interval
FrostFS rebalance connections interval
Properties:
- Config: rebalance_interval
- Env Var: RCLONE_FROSTFS_REBALANCE_INTERVAL
- Type: Duration
- Default: 15s
#### --frostfs-session-expiration
FrostFS session expiration epoch
Properties:
- Config: session_expiration
- Env Var: RCLONE_FROSTFS_SESSION_EXPIRATION
- Type: int
- Default: 4294967295
#### --frostfs-ape-cache-invalidation-duration
APE cache invalidation duration
Properties:
- Config: ape_cache_invalidation_duration
- Env Var: RCLONE_FROSTFS_APE_CACHE_INVALIDATION_DURATION
- Type: Duration
- Default: 8s
#### --frostfs-ape-cache-invalidation-timeout
APE cache invalidation timeout
Properties:
- Config: ape_cache_invalidation_timeout
- Env Var: RCLONE_FROSTFS_APE_CACHE_INVALIDATION_TIMEOUT
- Type: Duration
- Default: 24s
#### --frostfs-ape-chain-check-interval
The interval for verifying that the APE chain is saved in FrostFS.
Properties:
- Config: ape_chain_check_interval
- Env Var: RCLONE_FROSTFS_APE_CHAIN_CHECK_INTERVAL
- Type: Duration
- Default: 500ms
#### --frostfs-rpc-endpoint
Endpoint to connect to Neo rpc node
Properties:
- Config: rpc_endpoint
- Env Var: RCLONE_FROSTFS_RPC_ENDPOINT
- Type: string
- Required: false
#### --frostfs-wallet
Path to wallet
Properties:
- Config: wallet
- Env Var: RCLONE_FROSTFS_WALLET
- Type: string
- Required: true
#### --frostfs-address
Address of account
Properties:
- Config: address
- Env Var: RCLONE_FROSTFS_ADDRESS
- Type: string
- Required: false
#### --frostfs-password
Password to decrypt wallet
Properties:
- Config: password
- Env Var: RCLONE_FROSTFS_PASSWORD
- Type: string
- Required: false
#### --frostfs-placement-policy
Placement policy for new containers
Properties:
- Config: placement_policy
- Env Var: RCLONE_FROSTFS_PLACEMENT_POLICY
- Type: string
- Default: "REP 3"
- Examples:
- "REP 3"
- Container will have 3 replicas
#### --frostfs-container-creation-policy
Container creation policy for new containers
Properties:
- Config: container_creation_policy
- Env Var: RCLONE_FROSTFS_CONTAINER_CREATION_POLICY
- Type: string
- Default: "private"
- Examples:
- "public-read-write"
- Public container, anyone can read and write
- "public-read"
- Public container, owner can read and write, others can only read
- "private"
- Private container, only owner has access to it
### Advanced options
Here are the Advanced options specific to frostfs (Distributed, decentralized object storage FrostFS).
#### --frostfs-description
Description of the remote.
Properties:
- Config: description
- Env Var: RCLONE_FROSTFS_DESCRIPTION
- Type: string
- Required: false
{{< rem autogenerated options stop >}}
View git blame Copy permalink