FrostFS local Development and Testing environment

---

Overview

Tools to set up local FrostFS network and N3 privnets. Devenv, for short.

Prerequisites

Make sure you have installed all of the following prerequisites on your machine:

• docker
• docker-compose
• make (3.82+)
• expect
• openssl
• jq
• base64 (coreutils)

Quick Start

Clone repo:

$ git clone https://git.frostfs.info/TrueCloudLab/frostfs-dev-env.git

Run next commands from project's root:

$ make get

This command should be executed for the first run only to execute make hosts. It is part of the make up and, if the hosts have been added already, there is no need to run it separately.

$ make hosts
192.168.130.10 bastion.frostfs.devenv
192.168.130.50 main-chain.frostfs.devenv
192.168.130.61 ir01.frostfs.devenv
...
192.168.130.74 s04.frostfs.devenv

This command shows addresses and hostnames of components. Add make hosts output to your local /etc/hosts file.

Run all services with command:

$ make up

Also, you should add self-signed node (s04.frostfs.devenv) certificate to trusted store (default location might be changed using CA_CERTS_TRUSTED_STORE variable). This step is required for client services (frostfs-http-gw, frostfs-s3-gw) to interact with the node:

$ sudo make prepare.storage

Change FrostFS global configuration values with make update.* commands. The password of inner ring wallet is one. See examples in make help.

$ make update.epoch_duration val=30
Waiting for transactions to persist...

For instructions on how to set up DevEnv on macOS, please refer the guide in docs directory.

How it's organized

.
├── Makefile         # Commands to manage devenv
├── .services        # List of services to work with
├── services         # Services definitions and files
│   ├── basenet
│   ├── chain
│   ├── ir
│   ├── morph_chain
│   └── storage
├── vendor           # Temporary files and artifacts
└── wallets          # Wallet files to manage GAS assets

Main commands and targets to manage devenv's services are in Makefile.

Each service is defined in it's own directory under services/ with all required files to run and scripts to get external artifacts or dependencies.

The list of services and the starting order is defined in .services file. You can comment out services you don't want to start or add your own new services.

You can find more information on each service in docs directory.

Maybe you will find the answer for your question in F.A.Q.

Using FrostFS Admin Tool in dev-env

Devenv supports FrostFS network management via frostfs-adm. services/ir contains the Alphabet wallet in a proper format, specify it with --alphabet-wallets flag.

Notable make targets

make help will print the brief description of available targets. Here we describe some of them in a more detailed way.

up

Start all Devenv services.

This target call pull to get container images, get to download required artifacts, vendor/hosts to generate hosts file and then starts all services in the order defined in .services file.

down

Shutdowns all services. This will destroy all containers and networks. All changes made inside containers will be lost.

hosts

Display addresses and host names for each running service, if available.

clean

Clean up vendor directory.

s3cred

Registers user wallet and issues s3 credentials.

Usage and default parameter values:

make s3cred [password=""] [contract_password=s3] [wallet=/user_wallet.json] [gate_public_key=0313b1ac3a8076e155a7e797b24f0b650cccad5941ea59d7cfd51a024a8b2a06bf]

As soon as the storage node is in the network map (see above) you can generate S3 credentials:

$ make s3cred
{
  "access_key_id": "EXArWh8x1zeHG3851s1RtoCo7dowxF6rhLGA15nbMffT0AKRSjJ5fmcqf3Ht2VCAkfmPQUVARghRB77xHCA1BoN2p",
  "secret_access_key": "d70c1dba83f0f90bb231f06f1ce0e0dfbcfb122f4b4345a3c18d3869c359b79f",
  "owner_private_key": "140947599afd9ca89af4b358c3176eb046e554d942a0dc99a8e06f3e43c8f4ad",
  "wallet_public_key": "0324e76288fcb900100d01802a14ef977cca45ad073561230446df14b344c858b6",
  "container_id": "EXArWh8x1zeHG3851s1RtoCo7dowxF6rhLGA15nbMffT"
}                   

Running without any parameters will result in defaults which are based on the private key from /user-wallet.json file and /wallet.json contract wallet.

Now let's configure an S3 client (AWS CLI will be used as example):

$ aws configure
AWS Access Key ID []: EXArWh8x1zeHG3851s1RtoCo7dowxF6rhLGA15nbMffT0AKRSjJ5fmcqf3Ht2VCAkfmPQUVARghRB77xHCA1BoN2p
AWS Secret Access Key []: d70c1dba83f0f90bb231f06f1ce0e0dfbcfb122f4b4345a3c18d3869c359b79f
Default region name []: us-east-1
Default output format []: json

If you need to create credentials for different users, put user wallets to wallets dir and specify them via wallet parameter. Pass wallet password in password parameter if it's not default. The same is for contract_wallet and gate_public_key params.

$ make s3cred wallet=custom_wallet.json password=test
{
  "access_key_id": "jHhL5B33o16R4jQsb8wm9A3RRdS6KrTB5N4bja9Jys904W7xXFNKqem2ACvTRWRYJsZMCUikYFSokN7pPJziWyDi",
  "secret_access_key": "21bb64fafa32c82417fd8b97ac56cc8a085998a3852632d52fe7042453daa440",
  "owner_private_key": "10f6f9d7a47bb0bf68363ad8a99fe69f1493f8b6e1665b3e4e83feb2d5c7ee39",
  "wallet_public_key": "03e38759973a6bb722baabc2dd84036a39f0b2f53d32fec45a4dacde8a50fe4b70",
  "container_id": "jHhL5B33o16R4jQsb8wm9A3RRdS6KrTB5N4bja9Jys9"
}

To get credentials from custom wallet, place it in wallets dir before start.

cred

Usage and default parameter values:

make cred [password=""] [contract_password=s3] [wallet=/user_wallet.json]

The same as s3cred, but it doesn't issues s3 credentials.

Contributing

Feel free to contribute to this project after reading the contributing guidelines.

Before starting to work on a certain topic, create an new issue first, describing the feature/topic you are going to implement.

License

• GNU General Public License v3.0