Make targets for issuing credentials #86

Merged
alexvanin merged 1 commit from nzinkevich/frostfs-dev-env:make_creds into master 2024-10-17 10:08:49 +00:00
Member

Close #84

Signed-off-by: Nikita Zinkevich n.zinkevich@yadro.com

Close #84 Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
nzinkevich added 1 commit 2024-09-30 14:07:39 +00:00
[#84] Port targets for issuing credentials
All checks were successful
DCO action / DCO (pull_request) Successful in 46s
fc39619f6b
Signed-off-by: Nikita Zinkevich <n.zinkevich@yadro.com>
nzinkevich requested review from storage-services-committers 2024-09-30 14:08:12 +00:00
nzinkevich requested review from storage-services-developers 2024-09-30 14:08:13 +00:00
nzinkevich requested review from storage-core-committers 2024-09-30 14:08:13 +00:00
potyarkin reviewed 2024-09-30 14:44:46 +00:00
Makefile Outdated
@ -60,6 +60,7 @@ get: $(foreach SVC, $(GET_SVCS), get.$(SVC))
.PHONY: up
up: up/basic
@$(foreach SVC, $(START_SVCS), $(shell docker-compose -f services/$(SVC)/docker-compose.yml up -d))
./vendor/frostfs-adm morph proxy-add-account --config frostfs-adm.yml --account NYpbUf3vVDfaL1UyiUxwEfR6dtrZSBztis || die "Couldn't set s3-gw wallet as proxy wallet"
Member

What wallet is this hardcoded account from? A quick grep through my copy of the repo did not come up with any results

What wallet is this hardcoded account from? A quick grep through my copy of the repo did not come up with any results
Author
Member

Fixed. It's a contract wallet address

Fixed. It's a contract wallet address
potyarkin marked this conversation as resolved
@ -0,0 +2,4 @@
# Generate S3 credentials
s3cred:
@docker exec -it s3_gate /usr/bin/issue-creds.sh s3
Member

This recipe asks for /user_wallet.json password on my machine, and s3 is not accepted as valid. Maybe we can avoid interactive prompt here?

This recipe asks for `/user_wallet.json` password on my machine, and `s3` is not accepted as valid. Maybe we can avoid interactive prompt here?
Author
Member

Fixed, now if wallet has non-empty password, you can pass it via password parameter. Now it also has optional params contract_password and gate_public_key if contract wallet is not default. Example given in README.md

Fixed, now if wallet has non-empty password, you can pass it via `password` parameter. Now it also has optional params `contract_password` and `gate_public_key` if contract wallet is not default. Example given in README.md
potyarkin marked this conversation as resolved
@ -0,0 +6,4 @@
# Generate S3 credentials based on imported wallets
s3cred-custom:
@docker exec -it s3_gate /usr/bin/issue-creds.sh s3 $(wallet)
Member

Did you test this? I'm not sure that $(wallet) is visible from inside the s3_gate container.

Did you test this? I'm not sure that `$(wallet)` is visible from inside the `s3_gate` container.
Author
Member

Added wallets directory for storing custom wallets

Added `wallets` directory for storing custom wallets
potyarkin marked this conversation as resolved
@ -0,0 +1 @@
{"version":"1.0","accounts":[{"address":"NQjbiXuAoZHCifBJ9H1TQ7SPQA3EzdA1Mr","key":"6PYVwvn4kpcHD2VedzwcKcFgGYooeiBrWaJdXg1WEag3fzNWMwPKnKDKV4","label":"nikita","contract":{"script":"DCEDzN6yW9sasGL4sGpyAq9I47Ly2b2JjRz0WSqLoW1sIU9BVuezJw==","parameters":[{"name":"parameter0","type":"Signature"}],"deployed":false},"lock":false,"isDefault":false}],"scrypt":{"n":16384,"r":8,"p":8},"extra":{"Tokens":null}}
Member

"label":"nikita"

We like you very much, but this is not really appropriate for a public repo :)

> `"label":"nikita"` We like you very much, but this is not really appropriate for a public repo :)
potyarkin marked this conversation as resolved
Member

xk6-frostfs also uses a separate wallet. What about adding it as well?

`xk6-frostfs` also uses a separate [wallet](https://git.frostfs.info/TrueCloudLab/xk6-frostfs/src/branch/master/scenarios/files/wallet.json). What about adding it as well?
nzinkevich force-pushed make_creds from fc39619f6b to 1d97f4bd88 2024-10-01 06:51:19 +00:00 Compare
Author
Member

xk6-frostfs also uses a separate wallet. What about adding it as well?

Do you mean user wallet? If so, you can place wallet in service/s3_gate/wallets directory and call make s3cred wallet=wallet.json to issue creds

> `xk6-frostfs` also uses a separate [wallet](https://git.frostfs.info/TrueCloudLab/xk6-frostfs/src/branch/master/scenarios/files/wallet.json). What about adding it as well? Do you mean user wallet? If so, you can place wallet in `service/s3_gate/wallets` directory and call `make s3cred wallet=wallet.json` to issue creds
nzinkevich changed title from [#84] Port targets for issuing credentials to [#84] Port Make targets for issuing credentials 2024-10-01 07:49:32 +00:00
nzinkevich changed title from [#84] Port Make targets for issuing credentials to [#84] Make targets for issuing credentials 2024-10-01 07:49:39 +00:00
aarifullin reviewed 2024-10-01 08:51:20 +00:00
Makefile Outdated
@ -60,6 +60,7 @@ get: $(foreach SVC, $(GET_SVCS), get.$(SVC))
.PHONY: up
up: up/basic
@$(foreach SVC, $(START_SVCS), $(shell docker-compose -f services/$(SVC)/docker-compose.yml up -d))
./vendor/frostfs-adm morph proxy-add-account --config frostfs-adm.yml --account NUUb82KR2JrVByHs2YSKgtK29gKnF5q6Vt || die "Couldn't set s3-gw wallet as proxy wallet"
Member

Could we, please, calculate --account value using:

account=`docker container exec -it morph_chain neo-go wallet dump-keys -w services/s3_gate/wallet.json | head -1 | awk '{print $1}'`

?

Could we, please, calculate `--account` value using: ```bash account=`docker container exec -it morph_chain neo-go wallet dump-keys -w services/s3_gate/wallet.json | head -1 | awk '{print $1}'` ``` ?
pogpp approved these changes 2024-10-01 09:12:28 +00:00
Dismissed
pogpp reviewed 2024-10-01 09:14:07 +00:00
@ -0,0 +8,4 @@
--contract-wallet /wallet.json 1> /dev/null && touch $WALLET_CACHE/$USERNAME
}
issueAWS() {
Member

issueCreds

issueCreds
dkirillov reviewed 2024-10-01 12:45:31 +00:00
@ -0,0 +9,4 @@
s3cred:
@docker exec -e AUTHMATE_WALLET_PASSPHRASE=$(password) -e AUTHMATE_WALLET_CONTRACT_PASSPHRASE=$(contract_password) s3_gate /usr/bin/issue-creds.sh s3 $(wallet) $(gate_public_key)
# Generate S3 credentials
Member

Incorrect comment

Incorrect comment
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-01 12:48:21 +00:00
@ -13,10 +13,13 @@ services:
ipv4_address: ${IPV4_PREFIX}.82
volumes:
- ./wallet.json:/wallet.json
- ./user_wallet.json:/user_wallet.json
Member

Let's use frostfs-dev-env/wallets/wallet.json wallet by default

Let's use `frostfs-dev-env/wallets/wallet.json` wallet by default
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-01 12:52:12 +00:00
@ -0,0 +13,4 @@
--wallet $WALLET_PATH \
--peer s01.frostfs.devenv:8080 \
--gate-public-key $S3_GATE_PUBLIC_KEY \
--container-placement-policy "REP 1"
Member

Probably we should use different policy. REP 3 for example

Probably we should use different policy. `REP 3` for example
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-01 12:54:09 +00:00
@ -0,0 +10,4 @@
@docker exec -e AUTHMATE_WALLET_PASSPHRASE=$(password) -e AUTHMATE_WALLET_CONTRACT_PASSPHRASE=$(contract_password) s3_gate /usr/bin/issue-creds.sh s3 $(wallet) $(gate_public_key)
# Generate S3 credentials
cred:
Member

Should we rename this target? To be more clear that this command just register user and don't create any credentials

Should we rename this target? To be more clear that this command just register user and don't create any credentials
dkirillov marked this conversation as resolved
dkirillov reviewed 2024-10-01 12:55:11 +00:00
@ -0,0 +1 @@
{"version":"1.0","accounts":[{"address":"NQ7XnUVh5Uf5Vi9WxFRWHoShP8jHcHx6cd","key":"6PYT8dNaPWmhuiF6LovPyxbg7b3yrBic4gyk57JC7Fed5kkdX7bPn7Y6Pe","label":"custom","contract":{"script":"DCECyvyY2A+nacS1BGtLY63A9RzHTi4P1urM4LjLVrkroLNBVuezJw==","parameters":[{"name":"parameter0","type":"Signature"}],"deployed":false},"lock":false,"isDefault":false}],"scrypt":{"n":16384,"r":8,"p":8},"extra":{"Tokens":null}}
Member

Why do we need this new wallet?

Why do we need this new wallet?
dkirillov marked this conversation as resolved
potyarkin reviewed 2024-10-01 15:21:05 +00:00
@ -0,0 +7,4 @@
# Generate S3 credentials
s3cred:
@docker exec -e AUTHMATE_WALLET_PASSPHRASE=$(password) -e AUTHMATE_WALLET_CONTRACT_PASSPHRASE=$(contract_password) s3_gate /usr/bin/issue-creds.sh s3 $(wallet) $(gate_public_key)
Member

This will not work if password contains whitespace

This will not work if password contains whitespace
potyarkin marked this conversation as resolved
nzinkevich force-pushed make_creds from 1d97f4bd88 to 1918f2aa02 2024-10-03 06:59:04 +00:00 Compare
nzinkevich dismissed pogpp's review 2024-10-03 06:59:04 +00:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

dkirillov approved these changes 2024-10-03 12:26:17 +00:00
Dismissed
dkirillov left a comment
Member

LGTM

LGTM
potyarkin reviewed 2024-10-03 13:34:18 +00:00
@ -0,0 +1,14 @@
.PHONY: s3cred register
password?=""
Member

Empty strings in Makefile are just empty. This should be password?=

Quotes work for now, but that's more of an accident:

  • You build the command line below: AUTHMATE_WALLET_PASSPHRASE="$(password)"
  • Make substitutes that for AUTHMATE_WALLET_PASSPHRASE=""""
  • It works because shell treats it as a concatenation of two empty strings: first+second quotes produce an empty string, third+fourth quotes produce an empty string

A few accidentsedits later it might cost someone a long debug session.

Empty strings in Makefile are just empty. This should be `password?=` Quotes work for now, but that's more of an accident: - You build the command line below: `AUTHMATE_WALLET_PASSPHRASE="$(password)"` - Make substitutes that for `AUTHMATE_WALLET_PASSPHRASE=""""` - It works because shell treats it as a concatenation of two empty strings: first+second quotes produce an empty string, third+fourth quotes produce an empty string A few ~~accidents~~edits later it might cost someone a long debug session.
potyarkin marked this conversation as resolved
nzinkevich force-pushed make_creds from 1918f2aa02 to 54af37899a 2024-10-04 13:16:57 +00:00 Compare
nzinkevich dismissed dkirillov's review 2024-10-04 13:16:57 +00:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

dkirillov approved these changes 2024-10-08 09:28:58 +00:00
Dismissed
nzinkevich requested review from storage-services-committers 2024-10-10 09:53:00 +00:00
nzinkevich requested review from storage-services-developers 2024-10-10 09:53:00 +00:00
alexvanin approved these changes 2024-10-11 09:58:02 +00:00
Dismissed
nzinkevich requested review from storage-core-developers 2024-10-14 08:43:44 +00:00
dstepanov-yadro reviewed 2024-10-14 15:05:35 +00:00
@ -0,0 +13,4 @@
--wallet $WALLET_PATH \
--peer s01.frostfs.devenv:8080 \
--gate-public-key $S3_GATE_PUBLIC_KEY \
--container-placement-policy "REP 3"

What's happen if my container has other policy?

What's happen if my container has other policy?
Owner

This policy is used to create container to store 'accessbox' object with tokens for s3-gateway to use for authorization. So any valid policy can be used here.

For example, I often use REP 4 policy when I test some split-brain issues in dev-env. This way all nodes contain accessbox and authorization does not affect the case I want to check.

This policy is used to create container to store 'accessbox' object with tokens for s3-gateway to use for authorization. So any valid policy can be used here. For example, I often use `REP 4` policy when I test some split-brain issues in dev-env. This way all nodes contain accessbox and authorization does not affect the case I want to check.
dstepanov-yadro marked this conversation as resolved
fyrchik reviewed 2024-10-15 10:21:56 +00:00
@ -13,10 +13,13 @@ services:
ipv4_address: ${IPV4_PREFIX}.82
volumes:
- ./wallet.json:/wallet.json
Owner

There are 3 different wallet volumes with the same name.
Do we need them all?
From the name alone, it is not obvious what are the use-cases for each.

There are 3 different wallet volumes with the same name. Do we need them all? From the name alone, it is not obvious what are the use-cases for each.
Owner

Now these volumes have corresponding comments. One is a service wallet, another one is for user operation, and for more there are separate dir of custom wallets to use.

Now these volumes have corresponding comments. One is a service wallet, another one is for user operation, and for more there are separate dir of custom wallets to use.
dstepanov-yadro approved these changes 2024-10-17 07:09:52 +00:00
Dismissed
acid-ant changed title from [#84] Make targets for issuing credentials to Make targets for issuing credentials 2024-10-17 07:23:08 +00:00
nzinkevich force-pushed make_creds from 54af37899a to 20e6f09106 2024-10-17 09:36:23 +00:00 Compare
nzinkevich dismissed dkirillov's review 2024-10-17 09:36:23 +00:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

nzinkevich dismissed alexvanin's review 2024-10-17 09:36:23 +00:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

nzinkevich dismissed dstepanov-yadro's review 2024-10-17 09:36:23 +00:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

nzinkevich force-pushed make_creds from 20e6f09106 to 636be7352e 2024-10-17 09:37:36 +00:00 Compare
alexvanin approved these changes 2024-10-17 10:05:04 +00:00
Owner

Merging it, because all required approved were collected before last force change for #86 (comment)

Merging it, because all required approved were collected before last force change for https://git.frostfs.info/TrueCloudLab/frostfs-dev-env/pulls/86#issuecomment-54688
alexvanin merged commit 636be7352e into master 2024-10-17 10:08:49 +00:00
alexvanin deleted branch make_creds 2024-10-17 10:08:55 +00:00
Sign in to join this conversation.
No reviewers
TrueCloudLab/storage-services-developers
TrueCloudLab/storage-core-developers
No milestone
No project
No assignees
9 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: TrueCloudLab/frostfs-dev-env#86
No description provided.