6.3 KiB
FrostFS local Development and Testing environment
Overview
Tools to set up local FrostFS network and N3 privnets. Devenv, for short.
Prerequisites
Make sure you have installed all of the following prerequisites on your machine:
- docker
- docker-compose
- make (
3.82+) - expect
- openssl
- jq
- base64 (coreutils)
Quick Start
Clone repo:
$ git clone https://git.frostfs.info/TrueCloudLab/frostfs-dev-env.git
Run next commands from project's root:
$ make get
This command should be executed for the first run only to execute
make hosts. It is part of the make up and, if the hosts have
been added already, there is no need to run it separately.
$ make hosts
192.168.130.10 bastion.frostfs.devenv
192.168.130.50 main-chain.frostfs.devenv
192.168.130.61 ir01.frostfs.devenv
...
192.168.130.74 s04.frostfs.devenv
This command shows addresses and hostnames of components. Add make hosts
output to your local /etc/hosts file.
Run all services with command:
$ make up
Also, you should add self-signed node (s04.frostfs.devenv) certificate to trusted
store (default location might be changed using CA_CERTS_TRUSTED_STORE
variable). This step is required for client services (frostfs-http-gw,
frostfs-s3-gw) to interact with the node:
$ sudo make prepare.storage
Change FrostFS global configuration values with make update.* commands. The
password of inner ring wallet is one. See examples in make help.
$ make update.epoch_duration val=30
Waiting for transactions to persist...
For instructions on how to set up DevEnv on macOS, please refer the
guide in docs directory.
How it's organized
.
├── Makefile # Commands to manage devenv
├── .services # List of services to work with
├── services # Services definitions and files
│ ├── basenet
│ ├── chain
│ ├── ir
│ ├── morph_chain
│ └── storage
├── vendor # Temporary files and artifacts
└── wallets # Wallet files to manage GAS assets
Main commands and targets to manage devenv's services are in Makefile.
Each service is defined in it's own directory under services/ with all
required files to run and scripts to get external artifacts or dependencies.
The list of services and the starting order is defined in .services file. You
can comment out services you don't want to start or add your own new services.
You can find more information on each service in docs directory.
Maybe you will find the answer for your question in F.A.Q.
Using FrostFS Admin Tool in dev-env
Devenv supports FrostFS network management via frostfs-adm.
services/ir contains the Alphabet wallet in a proper format, specify it
with --alphabet-wallets flag.
Notable make targets
make help will print the brief description of available targets. Here we
describe some of them in a more detailed way.
up
Start all Devenv services.
This target call pull to get container images, get to download required
artifacts, vendor/hosts to generate hosts file and then starts all services in
the order defined in .services file.
down
Shutdowns all services. This will destroy all containers and networks. All changes made inside containers will be lost.
hosts
Display addresses and host names for each running service, if available.
clean
Clean up vendor directory.
s3cred
Registers user wallet and issues s3 credentials.
Usage and default parameter values:
make s3cred [password=""] [contract_password=s3] [wallet=/user_wallet.json] [gate_public_key=0313b1ac3a8076e155a7e797b24f0b650cccad5941ea59d7cfd51a024a8b2a06bf]
As soon as the storage node is in the network map (see above) you can generate S3 credentials:
$ make s3cred
{
"access_key_id": "EXArWh8x1zeHG3851s1RtoCo7dowxF6rhLGA15nbMffT0AKRSjJ5fmcqf3Ht2VCAkfmPQUVARghRB77xHCA1BoN2p",
"secret_access_key": "d70c1dba83f0f90bb231f06f1ce0e0dfbcfb122f4b4345a3c18d3869c359b79f",
"owner_private_key": "140947599afd9ca89af4b358c3176eb046e554d942a0dc99a8e06f3e43c8f4ad",
"wallet_public_key": "0324e76288fcb900100d01802a14ef977cca45ad073561230446df14b344c858b6",
"container_id": "EXArWh8x1zeHG3851s1RtoCo7dowxF6rhLGA15nbMffT"
}
Running without any parameters will result in defaults which are based on the private key from
/user-wallet.json file and /wallet.json contract wallet.
Now let's configure an S3 client (AWS CLI will be used as example):
$ aws configure
AWS Access Key ID []: EXArWh8x1zeHG3851s1RtoCo7dowxF6rhLGA15nbMffT0AKRSjJ5fmcqf3Ht2VCAkfmPQUVARghRB77xHCA1BoN2p
AWS Secret Access Key []: d70c1dba83f0f90bb231f06f1ce0e0dfbcfb122f4b4345a3c18d3869c359b79f
Default region name []: us-east-1
Default output format []: json
If you need to create credentials for different users, put user wallets to wallets dir and specify them via wallet parameter.
Pass wallet password in password parameter if it's not default. The same is for contract_wallet and gate_public_key params.
$ make s3cred wallet=custom_wallet.json password=test
{
"access_key_id": "jHhL5B33o16R4jQsb8wm9A3RRdS6KrTB5N4bja9Jys904W7xXFNKqem2ACvTRWRYJsZMCUikYFSokN7pPJziWyDi",
"secret_access_key": "21bb64fafa32c82417fd8b97ac56cc8a085998a3852632d52fe7042453daa440",
"owner_private_key": "10f6f9d7a47bb0bf68363ad8a99fe69f1493f8b6e1665b3e4e83feb2d5c7ee39",
"wallet_public_key": "03e38759973a6bb722baabc2dd84036a39f0b2f53d32fec45a4dacde8a50fe4b70",
"container_id": "jHhL5B33o16R4jQsb8wm9A3RRdS6KrTB5N4bja9Jys9"
}
To get credentials from custom wallet, place it in wallets dir before start.
cred
Usage and default parameter values:
make cred [password=""] [contract_password=s3] [wallet=/user_wallet.json]
The same as s3cred, but it doesn't issues s3 credentials.
Contributing
Feel free to contribute to this project after reading the contributing guidelines.
Before starting to work on a certain topic, create an new issue first, describing the feature/topic you are going to implement.