TrueCloudLab/frostfs-node

Fork 27

Unbounded bootstrap fee for new storage nodes #1395

New issue

Open

opened 2024-09-26 13:01:59 +00:00 by potyarkin · 4 comments

potyarkin commented

2024-09-26 13:01:59 +00:00

Member

Expected Behavior

Transaction fees collected from storage node wallet during bootstrap should be similar for all storage nodes, and should not grow based on current balance of storage node wallet.

Current Behavior

It appears that storage node wallet loses 50% (plus some change) of its balance during bootstrap. That does not seem sensible.

Balance of storage node wallet

Before bootstrap, GAS	After bootstrap, GAS
10	4.8767892
100	49.8767492
1000	499.8767492
10000	4999.8767492
100000	49999.8767492
1000000	499999.8767492

In examples above ≈0.124 GAS appears to be in the ballpark of normal transaction fee. Where do the other 50% of the balance go?

Possible Solution

No fix can be suggested by reporter. Further solutions shall be up to developers.

Steps to Reproduce (for bugs)

I've prepared a reproducible example based on frostfs-dev-env:

Clone my branch
Start frostfs-dev-env as usual: make up/basic
Launch my Makefile: cd extra; make GAS=123. This will:
- Generate a new wallet for storage node
- Transfer some GAS to this wallet
- Start a new storage node and join it into -dev-env network
- Wait 10 seconds
- Kill the node and print storage wallet balance

Example output (truncated):

$ make GAS=1000000
...
./bin/neo-go wallet init --account --wallet-config new_storage_wallet.template

{
        "version": "1.0",
        "accounts": [
         ...
 }

wallet successfully created, file location is new_storage_wallet.json
./bin/neo-go wallet nep17 balance -r http://10.200.97.90:30333 -w new_storage_wallet.json
../vendor/frostfs-adm morph refill-gas --storage-wallet new_storage_wallet.json --gas 1000000 \
        --rpc-endpoint http://10.200.97.90:30333 --alphabet-wallets ../services/ir \
        --config ../frostfs-adm.yml
./bin/neo-go wallet nep17 balance -r http://10.200.97.90:30333 -w new_storage_wallet.json
./bin/frostfs-node --config-dir config &
NODE_PID=$!
sleep 10
kill $NODE_PID
./bin/neo-go wallet nep17 balance -r http://10.200.97.90:30333 -w new_storage_wallet.json
Account NfrRKbL5QmS6JyaeW3FTooTxNza44JR4vJ
Waiting for transactions to persist...
Account NfrRKbL5QmS6JyaeW3FTooTxNza44JR4vJ
GAS: GasToken (d2a4cff31913016155e38e474a2c06d08be276cf)
        Amount : 1000000
        Updated: 8636
2024-09-26T12:39:34.108Z        info    frostfs-node/main.go:76 initializing profiler service...
...
2024-09-26T12:39:38.402Z        debug   frostfs-node/morph.go:236       new block       {"index": 8641}
2024-09-26T12:39:39.403Z        debug   frostfs-node/morph.go:236       new block       {"index": 8642}
2024-09-26T12:39:40.402Z        debug   frostfs-node/morph.go:236       new block       {"index": 8643}
2024-09-26T12:39:41.404Z        debug   frostfs-node/morph.go:236       new block       {"index": 8644}
2024-09-26T12:39:42.405Z        debug   frostfs-node/morph.go:236       new block       {"index": 8645}
2024-09-26T12:39:43.408Z        debug   frostfs-node/morph.go:236       new block       {"index": 8646}
2024-09-26T12:39:43.494Z        info    frostfs-node/config.go:1254     termination signal has been received, stopping...
...
Account NfrRKbL5QmS6JyaeW3FTooTxNza44JR4vJ
GAS: GasToken (d2a4cff31913016155e38e474a2c06d08be276cf)
        Amount : 499999.8767492
        Updated: 8637

Context

I've noticed this while watching wallet balance when I was debugging another unrelated issue.

Regression

Your Environment

Version used:
- frostfs-dev-env @ d0c32731f2c468dafd979337ad04af1075d2bca2
- frostfs-node 0.42.9
Server setup and configuration: frostfs-dev-env + custom storage node
Operating System and version (uname -a): Linux hostname 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) x86_64 GNU/Linux

## Expected Behavior Transaction fees collected from storage node wallet during bootstrap should be similar for all storage nodes, and should not grow based on current balance of storage node wallet. ## Current Behavior It appears that storage node wallet loses 50% (plus some change) of its balance during bootstrap. That does not seem sensible. ### Balance of storage node wallet | Before bootstrap, GAS | After bootstrap, GAS | |-----------------:|----------------:| | 10 | 4.8767892 | | 100 | 49.8767492 | | 1000 | 499.8767492 | | 10000 | 4999.8767492 | | 100000 | 49999.8767492 | | 1000000 | 499999.8767492 | In examples above ≈0.124 GAS appears to be in the ballpark of normal transaction fee. Where do the other 50% of the balance go? ## Possible Solution No fix can be suggested by reporter. Further solutions shall be up to developers. ## Steps to Reproduce (for bugs) I've prepared a reproducible example based on frostfs-dev-env: - Clone [my branch](https://git.frostfs.info/potyarkin/frostfs-dev-env/src/branch/bug/storage-node-bootstrap-cost) - Start frostfs-dev-env as usual: `make up/basic` - Launch my Makefile: `cd extra; make GAS=123`. This will: - Generate a new wallet for storage node - Transfer some GAS to this wallet - Start a new storage node and join it into -dev-env network - Wait 10 seconds - Kill the node and print storage wallet balance Example output (truncated): ``` $ make GAS=1000000 ... ./bin/neo-go wallet init --account --wallet-config new_storage_wallet.template { "version": "1.0", "accounts": [ ... } wallet successfully created, file location is new_storage_wallet.json ./bin/neo-go wallet nep17 balance -r http://10.200.97.90:30333 -w new_storage_wallet.json ../vendor/frostfs-adm morph refill-gas --storage-wallet new_storage_wallet.json --gas 1000000 \ --rpc-endpoint http://10.200.97.90:30333 --alphabet-wallets ../services/ir \ --config ../frostfs-adm.yml ./bin/neo-go wallet nep17 balance -r http://10.200.97.90:30333 -w new_storage_wallet.json ./bin/frostfs-node --config-dir config & NODE_PID=$! sleep 10 kill $NODE_PID ./bin/neo-go wallet nep17 balance -r http://10.200.97.90:30333 -w new_storage_wallet.json Account NfrRKbL5QmS6JyaeW3FTooTxNza44JR4vJ Waiting for transactions to persist... Account NfrRKbL5QmS6JyaeW3FTooTxNza44JR4vJ GAS: GasToken (d2a4cff31913016155e38e474a2c06d08be276cf) Amount : 1000000 Updated: 8636 2024-09-26T12:39:34.108Z info frostfs-node/main.go:76 initializing profiler service... ... 2024-09-26T12:39:38.402Z debug frostfs-node/morph.go:236 new block {"index": 8641} 2024-09-26T12:39:39.403Z debug frostfs-node/morph.go:236 new block {"index": 8642} 2024-09-26T12:39:40.402Z debug frostfs-node/morph.go:236 new block {"index": 8643} 2024-09-26T12:39:41.404Z debug frostfs-node/morph.go:236 new block {"index": 8644} 2024-09-26T12:39:42.405Z debug frostfs-node/morph.go:236 new block {"index": 8645} 2024-09-26T12:39:43.408Z debug frostfs-node/morph.go:236 new block {"index": 8646} 2024-09-26T12:39:43.494Z info frostfs-node/config.go:1254 termination signal has been received, stopping... ... Account NfrRKbL5QmS6JyaeW3FTooTxNza44JR4vJ GAS: GasToken (d2a4cff31913016155e38e474a2c06d08be276cf) Amount : 499999.8767492 Updated: 8637 ``` ## Context I've noticed this while watching wallet balance when I was debugging another unrelated issue. ## Regression No ## Your Environment * Version used: * frostfs-dev-env @ d0c32731f2c468dafd979337ad04af1075d2bca2 * frostfs-node 0.42.9 * Server setup and configuration: frostfs-dev-env + custom storage node * Operating System and version (`uname -a`): `Linux hostname 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) x86_64 GNU/Linux`

potyarkin added the

bug

triage

frostfs-node

labels 2024-09-26 13:02:21 +00:00

potyarkin commented

2024-09-26 13:56:04 +00:00

Author

Member

Related discussion in private chat

Related discussion in [private chat](https://chat.yadro.com/yadro/pl/a58qajqg57dbpkyn593tdhogoh)

fyrchik commented

2024-10-01 12:09:47 +00:00

Owner

// gasDivisor defines what part of GAS balance (1/gasDivisor)

It seems legit, because node doesn't need GAS for other things.
Do you have any reason to change the amout of GAS we deposit?
If so, let's discuss, if not, let's close this issue.

We give GAS to the notary service as a deposit https://github.com/nspcc-dev/neo-go/blob/master/docs/notary.md (it boils down to notary service having to be able to use that GAS at some point _in future_, to pay for the work being done _now_). Currently, it is hard-coded to be exactly half of the node balance https://git.frostfs.info/TrueCloudLab/frostfs-node/src/commit/7f8a1dcf8e238a08af84a1ef9e180541f783b71f/cmd/frostfs-node/morph.go#L141 It seems legit, because node doesn't need GAS for other things. Do you have any reason to change the amout of GAS we deposit? If so, let's discuss, if not, let's close this issue.

potyarkin commented

2024-10-01 13:06:48 +00:00

Author

Member

Reply reposted from instant messenger:

Current behavior may surprise third party enthusiasts attempting to join their storage nodes into public FrostFS network.

As far as I understand the fee is not easily reversible, so we should at least notify the user we're about to spend half of what they own. It would be even better to limit the upper bound in case some bored rich kid happens to come by 😄

_Reply reposted from instant messenger:_ > Current behavior may surprise third party enthusiasts attempting to join their storage nodes into public FrostFS network. > > As far as I understand the fee is not easily reversible, so we should at least notify the user we're about to spend half of what they own. It would be even better to limit the upper bound in case some bored rich kid happens to come by :smile:

fyrchik added this to the vNext milestone 2024-10-02 06:22:04 +00:00

fyrchik commented

2024-10-02 11:57:53 +00:00

Owner

A constant target seems OK too.
Currently, we have multiplicative hard-coded bound, depending on th enode balance, this could be configured too.
Something like

morph:
  notary:
    deposit_target: 100.000
    deposit_refill_threshold: 50.000

morph:
  notary:
    deposit_refill_threshold_divisor: 3
    deposit_target_divisor: 2

The example above is too complex, I would like to have 2 parameters at best and make then non-conflicting.
The thing I dont like is that flat values mean absolutely nothing, they depend on what the node does in the network and this may change rapidly between releases.

A constant target seems OK too. Currently, we have multiplicative hard-coded bound, depending on th enode balance, this could be configured too. Something like ``` morph: notary: deposit_target: 100.000 deposit_refill_threshold: 50.000 ``` or ``` morph: notary: deposit_refill_threshold_divisor: 3 deposit_target_divisor: 2 ``` The example above is too complex, I would like to have 2 parameters at best and make then non-conflicting. The thing I dont like is that flat values mean absolutely nothing, they depend on what the node does in the network and this may change rapidly between releases.

👀 1