Wallet balance always zero during container creation

potyarkin commented

2024-09-18 07:57:05 +00:00

Member

Expected Behavior

We should be able to create containers when ContainerFee is configured to a non-zero value (given that subject's wallet has enough GAS).

Current Behavior

We can not create containers if a fee is required. This happens because balance seen by container_contract is always zero - regardless of actual wallet's balance.

Possible Solution

No fix can be suggested by reporter. Further solutions shall be up to developers.

Steps to Reproduce (for bugs)

Create a container with default frostfs-dev-env config (ContainerFee = 0): everything is OK

$ ~/bringup/bin/frostfs-cli-v0.42.9 container create --wallet wallets/wallet.json --rpc-endpoint s01.frostfs.devenv:8080 --basic-acl 0 --await --name mycontainer --policy 'REP 1'                             
Enter password > 
CID: 8qbkyV7ye1227XpCJyrC96mnSC3kV9cU2WaJdHSMGgRF
awaiting...
container has been persisted on sidechain

Set non-zero container fee:

$ make update.container_fee val=1
Waiting for transactions to persist...

Can't create containers anymore:

$ ~/bringup/bin/frostfs-cli-v0.42.9 container create --wallet wallets/wallet.json --rpc-endpoint s01.frostfs.devenv:8080 --basic-acl 0 --await --name mycontainer --policy 'REP 1'
Enter password > 
put container rpc error: status: code = 1024 message = frostfs error: chain/client: contract execution finished with state FAULT; exception: at instruction 1500 (THROW): unhandled exception: "insufficient balance to create container"
Exit code: 1

Even though we have enough GAS:

$ ~/bringup/bin/neo-go-v0.104.0 wallet nep17 balance  --wallet wallets/wallet.json --rpc-endpoint http://morph-chain.frostfs.devenv:30333                                                  
Account NbUgTSFvPmsRxmGeWpuuGeJUoRoi6PErcM
GAS: GasToken (d2a4cff31913016155e38e474a2c06d08be276cf)
        Amount : 100
        Updated: 148669

Context

Many thanks to @aarifullin for help with debugging! (private chat)

We have confirmed that container_contract always sees zero value instead of wallet balance by modifying the contract and checking neo-go logs: ownerID confirms that the contract operates on the wallet we expect it to, but logged balance is always 0.

diff --git a/container/container_contract.go b/container/container_contract.go
index 7946923..5374d89 100644
--- a/container/container_contract.go
+++ b/container/container_contract.go
@@ -202,6 +202,11 @@ func PutNamed(container []byte, signature interop.Signature,
         aliasFee := contract.Call(netmapContractAddr, "config", contract.ReadOnly, AliasFeeKey).(int)
         containerFee += aliasFee
     }
+    runtime.Log("len(alphabet) = " + std.Itoa10(len(alphabet)))
+    runtime.Log("containerFee = " + std.Itoa10(containerFee))
+    runtime.Log("balance = " + std.Itoa10(balance))
+    runtime.Log("ownerID.base58 = " + std.Base58Encode(ownerID))
 
     if balance < containerFee*len(alphabet) {
         panic("insufficient balance to create container")

Ayrat suggested that this bug may be caused by permissions mismatch:

balance_contract must comply with NEP-17 standard and therefore balanceOf must be safe
PutNamed method (where this error occurs) is not marked as safe - and data in the following snippet is always nil even though the corresponding key exists:

 func getAccount(ctx storage.Context, key any) Account {
 	data := storage.Get(ctx, key)
 	if data != nil {
 		acc := std.Deserialize(data.([]byte)).(account)
 		return Account{
 			Balance: common.FromFixedWidth64(acc.Balance),
 			Until:   common.FromFixedWidth64(acc.Until),
 			Parent:  acc.Parent,
 		}
 	}
 	return Account{}
 }

Regression

Can't be assessed by reporter.

Your Environment

Version used:
- frostfs-dev-env (commit 7538bd9b171409fc5eeb5c08d074efe5792db133)
- frostfs-contract v0.19.2 (defined by dev-env)
- Same behavior reproduced with frostfs-contract v0.19.4 outside of dev-env
Server setup and configuration:
- Initially encountered in a custom sidechain/ir deployment
- Reproduced in frostfs-dev-env
Operating System and version (uname -a): Linux hostname 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) x86_64 GNU/Linux

## Expected Behavior We should be able to create containers when `ContainerFee` is configured to a non-zero value (given that subject's wallet has enough GAS). ## Current Behavior We can not create containers if a fee is required. This happens because `balance` [seen by container_contract](https://git.frostfs.info/TrueCloudLab/frostfs-contract/src/commit/3e221b973a3cfe387113990bab56b33f78bc0c18/container/container_contract.go#L206-L208) is always zero - regardless of actual wallet's balance. ## Possible Solution No fix can be suggested by reporter. Further solutions shall be up to developers. ## Steps to Reproduce (for bugs) Create a container with default frostfs-dev-env config (`ContainerFee = 0`): everything is OK ```console $ ~/bringup/bin/frostfs-cli-v0.42.9 container create --wallet wallets/wallet.json --rpc-endpoint s01.frostfs.devenv:8080 --basic-acl 0 --await --name mycontainer --policy 'REP 1' Enter password > CID: 8qbkyV7ye1227XpCJyrC96mnSC3kV9cU2WaJdHSMGgRF awaiting... container has been persisted on sidechain ``` Set non-zero container fee: ```console $ make update.container_fee val=1 Waiting for transactions to persist... ``` Can't create containers anymore: ```console $ ~/bringup/bin/frostfs-cli-v0.42.9 container create --wallet wallets/wallet.json --rpc-endpoint s01.frostfs.devenv:8080 --basic-acl 0 --await --name mycontainer --policy 'REP 1' Enter password > put container rpc error: status: code = 1024 message = frostfs error: chain/client: contract execution finished with state FAULT; exception: at instruction 1500 (THROW): unhandled exception: "insufficient balance to create container" Exit code: 1 ``` Even though we have enough GAS: ```console $ ~/bringup/bin/neo-go-v0.104.0 wallet nep17 balance --wallet wallets/wallet.json --rpc-endpoint http://morph-chain.frostfs.devenv:30333 Account NbUgTSFvPmsRxmGeWpuuGeJUoRoi6PErcM GAS: GasToken (d2a4cff31913016155e38e474a2c06d08be276cf) Amount : 100 Updated: 148669 ``` ## Context Many thanks to @aarifullin for help with debugging! ([private chat](https://chat.yadro.com/yadro/pl/sgcjpf4g3p8n9yfdfaoc6ohddw)) We have confirmed that container_contract always sees zero value instead of wallet balance by modifying the contract and checking neo-go logs: `ownerID` confirms that the contract operates on the wallet we expect it to, but logged `balance` is always 0. ```diff diff --git a/container/container_contract.go b/container/container_contract.go index 7946923..5374d89 100644 --- a/container/container_contract.go +++ b/container/container_contract.go @@ -202,6 +202,11 @@ func PutNamed(container []byte, signature interop.Signature, aliasFee := contract.Call(netmapContractAddr, "config", contract.ReadOnly, AliasFeeKey).(int) containerFee += aliasFee } + runtime.Log("len(alphabet) = " + std.Itoa10(len(alphabet))) + runtime.Log("containerFee = " + std.Itoa10(containerFee)) + runtime.Log("balance = " + std.Itoa10(balance)) + runtime.Log("ownerID.base58 = " + std.Base58Encode(ownerID)) if balance < containerFee*len(alphabet) { panic("insufficient balance to create container") ``` Ayrat [suggested](https://chat.yadro.com/yadro/pl/n5jijfk65jydpcj68ac146t3uo) that this bug may be caused by permissions mismatch: - balance_contract must comply with NEP-17 standard and therefore `balanceOf` must be safe - `PutNamed` method (where this error occurs) is not marked as safe - and `data` in the following snippet is always nil even though the corresponding `key` exists: https://git.frostfs.info/TrueCloudLab/frostfs-contract/src/commit/3e221b973a3cfe387113990bab56b33f78bc0c18/balance/balance_contract.go#L371-L383 ## Regression Can't be assessed by reporter. ## Your Environment  * Version used: * frostfs-dev-env (commit [7538bd9b171409fc5eeb5c08d074efe5792db133](https://git.frostfs.info/TrueCloudLab/frostfs-dev-env/src/commit/7538bd9b171409fc5eeb5c08d074efe5792db133)) * frostfs-contract v0.19.2 (defined by dev-env) * Same behavior reproduced with frostfs-contract v0.19.4 outside of dev-env * Server setup and configuration: * Initially encountered in a custom sidechain/ir deployment * Reproduced in frostfs-dev-env * Operating System and version (`uname -a`): `Linux hostname 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) x86_64 GNU/Linux`

👍 1

potyarkin added the

bug

label 2024-09-18 07:57:05 +00:00

potyarkin referenced this issue from potyarkin/bringup

2024-09-18 08:14:38 +00:00

Как создавать контейнеры, если ContainerFee > 0? #2

potyarkin referenced this issue from a commit

2024-09-18 09:17:38 +00:00

Set morph config to match frostfs-dev-env

fyrchik commented

2024-10-08 14:41:19 +00:00

Owner

This fee should be deposited to the balance contract.
During container creation the GAS is transferred to the alphabet nodes from the balance contract via transferX method
https://git.frostfs.info/TrueCloudLab/frostfs-contract/src/branch/master/container/container_contract.go#L219

This fee should be deposited to the `balance` contract. During container creation the GAS is transferred to the alphabet nodes from the balance contract via `transferX` method https://git.frostfs.info/TrueCloudLab/frostfs-contract/src/branch/master/container/container_contract.go#L219

potyarkin commented

2024-10-09 07:34:06 +00:00

Author

Member

I'm not sure if I understand your comment. Was it addressed to me or is it some kind of note to self during debug?

Balance check (which panics) happens before transferX call you linked, and it checks the balance of the wallet that called container create. Were you trying to point to some misconfiguration on my end?

I'm not sure if I understand your comment. Was it addressed to me or is it some kind of note to self during debug? Balance check (which panics) happens before `transferX` call you linked, and it checks the balance of the wallet that called `container create`. Were you trying to point to some misconfiguration on my end?

fyrchik commented

2024-10-09 07:58:31 +00:00

Owner

What I am trying to say is that wallet balance is different from its deposit in the balance contract, they are unrelated to each other.
Are we on the same note here?

What I am trying to say is that wallet balance is different from its deposit in the `balance` contract, they are unrelated to each other. Are we on the same note here?

fyrchik commented

2024-10-09 07:59:23 +00:00

Owner

What I am trying to say is that wallet balance is different from its deposit in the balance contract, they are unrelated to each other.
Are we on the same note here?

What I am trying to say is that wallet balance is different from its deposit in the `balance` contract, they are unrelated to each other. Are we on the same note here?

fyrchik commented

2024-10-09 07:59:47 +00:00

Owner

What I am trying to say is that wallet balance is different from its deposit in the balance contract, they are unrelated to each other.
Are we on the same note here?

What I am trying to say is that wallet balance is different from its deposit in the `balance` contract, they are unrelated to each other. Are we on the same note here?

potyarkin commented

2024-10-09 08:39:06 +00:00

Author

Member

wallet balance is different from its deposit in the balance contract, they are unrelated to each other

This was unexpected. I guess I need to do some reading on the balance contract.

In the end, whose responsibility is it to make that deposit? Should it be done automatically by frostfs-cli container create or should the user do it beforehand in some other way?

> wallet balance is different from its deposit in the balance contract, they are unrelated to each other This was unexpected. I guess I need to do some reading on the `balance` contract. In the end, whose responsibility is it to make that deposit? Should it be done automatically by `frostfs-cli container create` or should the user do it beforehand in some other way?

fyrchik commented

2024-10-09 09:04:56 +00:00

Owner

They are similar, actually.
Wallet balance is its GAS balance, that is some number in the native GAS contract.
Balance contract is similar, but it is our contract. Balance contract has its own hash, and thus has its own balance in the GAS contract!
When we transfer our GAS to the balance contract it starts to own them, but, being a contract, it remembers whom it has received this funds from.

The important part here is that balance contract governs how this GAS is being distributed: node can make a deposit, alphabet nodes can take the reward. This would not be possible with the native contract in the presence of malicious actors.

They are similar, actually. Wallet balance is its GAS balance, that is some number in the **native** GAS contract. Balance contract is similar, but it is **our** contract. Balance contract has its own hash, and thus has its own balance in the GAS contract! When we transfer our GAS to the balance contract it starts to own them, but, being a contract, it remembers whom it has received this funds from. The important part here is that `balance` contract governs how this GAS is being distributed: node can make a deposit, alphabet nodes can take the reward. This would not be possible with the native contract in the presence of malicious actors.

fyrchik commented

2024-10-09 09:08:24 +00:00

Owner

In the end, whose responsibility is it to make that deposit?

A client is reponsible.
frostfs-adm morph dump-hashes command can give you the hash of the balance contract.
neo-go wallet nep17 transfer command can allow you to transfer funds from your wallet to the balance contract.

>In the end, whose responsibility is it to make that deposit? A client is reponsible. `frostfs-adm morph dump-hashes` command can give you the hash of the `balance` contract. `neo-go wallet nep17 transfer` command can allow you to transfer funds from your wallet to the `balance` contract.

potyarkin commented

2024-10-09 09:17:12 +00:00

Author

Member

Notes from IRL chat:

Depositing tokens automatically during frostfs-cli container create is not possible because by design frostfs-cli should interact only with FrostFS storage API and is not aware of sidechain RPC endpoints
Adding relevant helper to frostfs-adm may be useful (if this functionality is not already covered by existing ones)

Notes from IRL chat: - Depositing tokens automatically during `frostfs-cli container create` is not possible because by design `frostfs-cli` should interact only with FrostFS storage API and is not aware of sidechain RPC endpoints - Adding relevant helper to `frostfs-adm` may be useful (if this functionality is not already covered by existing ones)