Make targets for issuing credentials #86

Merged
alexvanin merged 1 commit from nzinkevich/frostfs-dev-env:make_creds into master 2024-10-17 10:08:49 +00:00
6 changed files with 124 additions and 0 deletions

View file

@ -60,6 +60,7 @@ get: $(foreach SVC, $(GET_SVCS), get.$(SVC))
.PHONY: up .PHONY: up
up: up/basic up: up/basic
@$(foreach SVC, $(START_SVCS), $(shell docker-compose -f services/$(SVC)/docker-compose.yml up -d)) @$(foreach SVC, $(START_SVCS), $(shell docker-compose -f services/$(SVC)/docker-compose.yml up -d))
./vendor/frostfs-adm morph proxy-add-account --config frostfs-adm.yml --account=`docker container exec morph_chain neo-go wallet dump-keys -w /wallets/s3-wallet.json | head -1 | awk '{print $1}'` || die "Couldn't set s3-gw wallet as proxy wallet"
potyarkin marked this conversation as resolved Outdated

What wallet is this hardcoded account from? A quick grep through my copy of the repo did not come up with any results

What wallet is this hardcoded account from? A quick grep through my copy of the repo did not come up with any results

Fixed. It's a contract wallet address

Fixed. It's a contract wallet address

Could we, please, calculate --account value using:

account=`docker container exec -it morph_chain neo-go wallet dump-keys -w services/s3_gate/wallet.json | head -1 | awk '{print $1}'`

?

Could we, please, calculate `--account` value using: ```bash account=`docker container exec -it morph_chain neo-go wallet dump-keys -w services/s3_gate/wallet.json | head -1 | awk '{print $1}'` ``` ?
@echo "Full FrostFS Developer Environment is ready" @echo "Full FrostFS Developer Environment is ready"
# Build up FrostFS # Build up FrostFS

View file

@ -137,6 +137,65 @@ Display addresses and host names for each running service, if available.
Clean up `vendor` directory. Clean up `vendor` directory.
### s3cred
Registers user wallet and issues s3 credentials.
Usage and default parameter values:
```sh
make s3cred [password=""] [contract_password=s3] [wallet=/user_wallet.json] [gate_public_key=0313b1ac3a8076e155a7e797b24f0b650cccad5941ea59d7cfd51a024a8b2a06bf]
```
As soon as the storage node is in the network map (see above) you can generate S3
credentials:
``` sh
$ make s3cred
{
"access_key_id": "EXArWh8x1zeHG3851s1RtoCo7dowxF6rhLGA15nbMffT0AKRSjJ5fmcqf3Ht2VCAkfmPQUVARghRB77xHCA1BoN2p",
"secret_access_key": "d70c1dba83f0f90bb231f06f1ce0e0dfbcfb122f4b4345a3c18d3869c359b79f",
"owner_private_key": "140947599afd9ca89af4b358c3176eb046e554d942a0dc99a8e06f3e43c8f4ad",
"wallet_public_key": "0324e76288fcb900100d01802a14ef977cca45ad073561230446df14b344c858b6",
"container_id": "EXArWh8x1zeHG3851s1RtoCo7dowxF6rhLGA15nbMffT"
}
```
Running without any parameters will result in defaults which are based on the private key from
`/user-wallet.json` file and `/wallet.json` contract wallet.
Now let's configure an S3 client (AWS CLI will be used as example):
``` sh
$ aws configure
AWS Access Key ID []: EXArWh8x1zeHG3851s1RtoCo7dowxF6rhLGA15nbMffT0AKRSjJ5fmcqf3Ht2VCAkfmPQUVARghRB77xHCA1BoN2p
AWS Secret Access Key []: d70c1dba83f0f90bb231f06f1ce0e0dfbcfb122f4b4345a3c18d3869c359b79f
Default region name []: us-east-1
Default output format []: json
```
If you need to create credentials for different users, put user wallets to `wallets` dir and specify them via `wallet` parameter.
Pass wallet password in `password` parameter if it's not default. The same is for `contract_wallet` and `gate_public_key` params.
```sh
$ make s3cred wallet=custom_wallet.json password=test
{
"access_key_id": "jHhL5B33o16R4jQsb8wm9A3RRdS6KrTB5N4bja9Jys904W7xXFNKqem2ACvTRWRYJsZMCUikYFSokN7pPJziWyDi",
"secret_access_key": "21bb64fafa32c82417fd8b97ac56cc8a085998a3852632d52fe7042453daa440",
"owner_private_key": "10f6f9d7a47bb0bf68363ad8a99fe69f1493f8b6e1665b3e4e83feb2d5c7ee39",
"wallet_public_key": "03e38759973a6bb722baabc2dd84036a39f0b2f53d32fec45a4dacde8a50fe4b70",
"container_id": "jHhL5B33o16R4jQsb8wm9A3RRdS6KrTB5N4bja9Jys9"
}
```
To get credentials from custom wallet, place it in `wallets` dir before start.
### cred
Usage and default parameter values:
```sh
make cred [password=""] [contract_password=s3] [wallet=/user_wallet.json]
```
The same as `s3cred`, but it doesn't issues s3 credentials.
## Contributing ## Contributing
Feel free to contribute to this project after reading the [contributing Feel free to contribute to this project after reading the [contributing

View file

@ -19,6 +19,7 @@ services:
- ./config.yml:/wallets/config.yml - ./config.yml:/wallets/config.yml
- ./../../vendor/hosts:/etc/hosts - ./../../vendor/hosts:/etc/hosts
- ./../../wallets/wallet.json:/wallets/wallet.json - ./../../wallets/wallet.json:/wallets/wallet.json
- ./../s3_gate/wallet.json:/wallets/s3-wallet.json
- chains:/chains - chains:/chains
networks: networks:

View file

@ -12,11 +12,17 @@ services:
internet: internet:
ipv4_address: ${IPV4_PREFIX}.82 ipv4_address: ${IPV4_PREFIX}.82
volumes: volumes:
# Gate wallet

There are 3 different wallet volumes with the same name.
Do we need them all?
From the name alone, it is not obvious what are the use-cases for each.

There are 3 different wallet volumes with the same name. Do we need them all? From the name alone, it is not obvious what are the use-cases for each.

Now these volumes have corresponding comments. One is a service wallet, another one is for user operation, and for more there are separate dir of custom wallets to use.

Now these volumes have corresponding comments. One is a service wallet, another one is for user operation, and for more there are separate dir of custom wallets to use.
- ./wallet.json:/wallet.json - ./wallet.json:/wallet.json
dkirillov marked this conversation as resolved Outdated

Let's use frostfs-dev-env/wallets/wallet.json wallet by default

Let's use `frostfs-dev-env/wallets/wallet.json` wallet by default
# Custom user wallets
- ./wallets:/wallets
# Default user wallet
- ./../../wallets/wallet.json:/wallets/wallet.json
- ./tls.key:/tls.key - ./tls.key:/tls.key
- ./tls.crt:/tls.crt - ./tls.crt:/tls.crt
- ./../../vendor/hosts:/etc/hosts - ./../../vendor/hosts:/etc/hosts
- ./cfg:/etc/frostfs/s3 - ./cfg:/etc/frostfs/s3
- ./issue-creds.sh:/usr/bin/issue-creds.sh
stop_signal: SIGTERM stop_signal: SIGTERM
stop_grace_period: 15s stop_grace_period: 15s
env_file: [ ".env", ".s3.env", ".int_test.env" ] env_file: [ ".env", ".s3.env", ".int_test.env" ]
@ -34,6 +40,8 @@ services:
- S3_GW_PEERS_2_WEIGHT=0.2 - S3_GW_PEERS_2_WEIGHT=0.2
- S3_GW_PEERS_3_ADDRESS=s04.${LOCAL_DOMAIN}:8080 - S3_GW_PEERS_3_ADDRESS=s04.${LOCAL_DOMAIN}:8080
- S3_GW_PEERS_3_WEIGHT=0.2 - S3_GW_PEERS_3_WEIGHT=0.2
- AUTHMATE_WALLET_PASSPHRASE=
- AUTHMATE_WALLET_CONTRACT_PASSPHRASE=s3
networks: networks:
s3_gate_int: s3_gate_int:

41
services/s3_gate/issue-creds.sh Executable file
View file

@ -0,0 +1,41 @@
#!/bin/bash
initUser() {
/bin/frostfs-s3-authmate register-user \
--wallet $WALLET_PATH \
--rpc-endpoint http://morph-chain.frostfs.devenv:30333 \
--username $USERNAME \
--contract-wallet /wallet.json 1> /dev/null && touch $WALLET_CACHE/$USERNAME
}
issueCreds() {
Outdated
Review

issueCreds

issueCreds
/bin/frostfs-s3-authmate issue-secret \
--wallet $WALLET_PATH \
--peer s01.frostfs.devenv:8080 \
--gate-public-key $S3_GATE_PUBLIC_KEY \
--container-placement-policy "REP 3"
dkirillov marked this conversation as resolved Outdated

Probably we should use different policy. REP 3 for example

Probably we should use different policy. `REP 3` for example
dstepanov-yadro marked this conversation as resolved Outdated

What's happen if my container has other policy?

What's happen if my container has other policy?

This policy is used to create container to store 'accessbox' object with tokens for s3-gateway to use for authorization. So any valid policy can be used here.

For example, I often use REP 4 policy when I test some split-brain issues in dev-env. This way all nodes contain accessbox and authorization does not affect the case I want to check.

This policy is used to create container to store 'accessbox' object with tokens for s3-gateway to use for authorization. So any valid policy can be used here. For example, I often use `REP 4` policy when I test some split-brain issues in dev-env. This way all nodes contain accessbox and authorization does not affect the case I want to check.
}
set -e
WALLET_PATH=/wallets/$2
if [[ -z "$2" ]]; then
WALLET_PATH=/wallets/wallet.json
fi
S3_GATE_PUBLIC_KEY=$3
if [[ -z "$3" ]]; then
S3_GATE_PUBLIC_KEY=0313b1ac3a8076e155a7e797b24f0b650cccad5941ea59d7cfd51a024a8b2a06bf
fi
WALLET_CACHE=/data/wallets
mkdir -p $WALLET_CACHE
USERNAME=$(echo $WALLET_PATH | md5sum | cut -d' ' -f1)
if [ ! -e $WALLET_CACHE/$USERNAME ]; then
initUser
fi
if [ $1 == "s3" ]; then
issueCreds
fi

View file

@ -0,0 +1,14 @@
.PHONY: s3cred register
password?=
potyarkin marked this conversation as resolved Outdated

Empty strings in Makefile are just empty. This should be password?=

Quotes work for now, but that's more of an accident:

  • You build the command line below: AUTHMATE_WALLET_PASSPHRASE="$(password)"
  • Make substitutes that for AUTHMATE_WALLET_PASSPHRASE=""""
  • It works because shell treats it as a concatenation of two empty strings: first+second quotes produce an empty string, third+fourth quotes produce an empty string

A few accidentsedits later it might cost someone a long debug session.

Empty strings in Makefile are just empty. This should be `password?=` Quotes work for now, but that's more of an accident: - You build the command line below: `AUTHMATE_WALLET_PASSPHRASE="$(password)"` - Make substitutes that for `AUTHMATE_WALLET_PASSPHRASE=""""` - It works because shell treats it as a concatenation of two empty strings: first+second quotes produce an empty string, third+fourth quotes produce an empty string A few ~~accidents~~edits later it might cost someone a long debug session.
contract_password?=s3
gate_public_key?=
potyarkin marked this conversation as resolved Outdated

This recipe asks for /user_wallet.json password on my machine, and s3 is not accepted as valid. Maybe we can avoid interactive prompt here?

This recipe asks for `/user_wallet.json` password on my machine, and `s3` is not accepted as valid. Maybe we can avoid interactive prompt here?

Fixed, now if wallet has non-empty password, you can pass it via password parameter. Now it also has optional params contract_password and gate_public_key if contract wallet is not default. Example given in README.md

Fixed, now if wallet has non-empty password, you can pass it via `password` parameter. Now it also has optional params `contract_password` and `gate_public_key` if contract wallet is not default. Example given in README.md
wallet?=
# Register wallet & generate S3 credentials
s3cred:
potyarkin marked this conversation as resolved Outdated

Did you test this? I'm not sure that $(wallet) is visible from inside the s3_gate container.

Did you test this? I'm not sure that `$(wallet)` is visible from inside the `s3_gate` container.

Added wallets directory for storing custom wallets

Added `wallets` directory for storing custom wallets
@docker exec -e AUTHMATE_WALLET_PASSPHRASE="$(password)" -e AUTHMATE_WALLET_CONTRACT_PASSPHRASE="$(contract_password)" s3_gate /usr/bin/issue-creds.sh s3 "$(wallet)" "$(gate_public_key)"
potyarkin marked this conversation as resolved Outdated

This will not work if password contains whitespace

This will not work if password contains whitespace
# Only registers user wallet
dkirillov marked this conversation as resolved Outdated

Incorrect comment

Incorrect comment
register:
dkirillov marked this conversation as resolved Outdated

Should we rename this target? To be more clear that this command just register user and don't create any credentials

Should we rename this target? To be more clear that this command just register user and don't create any credentials
@docker exec -e AUTHMATE_WALLET_PASSPHRASE="$(password)" -e AUTHMATE_WALLET_CONTRACT_PASSPHRASE="$(contract_password)" s3_gate /usr/bin/issue-creds.sh native "$(wallet)"