Add netmap-aware dialer

fyrchik commented

2023-10-05 12:34:08 +00:00

Owner

DIaling by IP has problems:

Client must implement failure handling themselves in case of IP redundancy.
External client must fetch netmap to connect to the node anyway. And then pick a node. In reality he wants to just put an object, not think about which node and address to choose.

The suggestion is to implement generic dialer (or resolver) in some form:

c.Dial("frostfs://public-key")

Benefits:

All protocols are handled transparently.
We can choose external (from ExternalAddr attribute) or internal IP, based on the context.
We can even choose which address to pick, based on some statistics we accumulate.
We can transparently balance queries with a single client.

After this we can avoid providing an address completely! Client works with the storage network, not with a specific node, so something like this should be feasible:

c := client.New(netmap, ...)
c.PutObject(obj)

Here in addition to (3) we can choose not only address, but a node too.

Related #34

DIaling by IP has problems: 1. Client must implement failure handling themselves in case of IP redundancy. 2. External client must fetch netmap to connect to the node anyway. And then pick a node. In reality he wants to just put an object, not think about which node and address to choose. The suggestion is to implement generic dialer (or resolver) in some form: ``` c.Dial("frostfs://public-key") ``` Benefits: 1. All protocols are handled transparently. 2. We can choose external (from `ExternalAddr` attribute) or internal IP, based on the context. 3. We can even choose which address to pick, based on some statistics we accumulate. 4. We can transparently balance queries with a single client. After this we can avoid providing an address completely! Client works with the storage network, not with a specific node, so something like this should be feasible: ``` c := client.New(netmap, ...) c.PutObject(obj) ``` Here in addition to (3) we can choose not only address, but a node too. Related #34

👍 1

fyrchik commented

2023-10-05 12:35:16 +00:00

Author

Owner

My suggestion is to first try adding gRPC resolver. If it isn't possible, then move this to the client.

fyrchik added the

enhancement

discussion

labels 2023-10-05 12:40:28 +00:00

alexvanin commented

2023-10-19 06:44:29 +00:00

Owner

While avoiding IP-addresses is a great idea, there are some cases when it might be useful for client applications. For example, when client wants to access storage network through custom light-weight frostfs-node. This node won't participate in network map, so client can't use frostfs://public-key scheme to connect.

It would be nice to keep both IP and public-key dials in the library for different approaches.

While avoiding IP-addresses is a great idea, there are some cases when it might be useful for client applications. For example, when client wants to access storage network through custom light-weight frostfs-node. This node won't participate in network map, so client can't use `frostfs://public-key` scheme to connect. It would be nice to keep both IP and public-key dials in the library for different approaches.

fyrchik commented

2023-10-19 08:39:38 +00:00

Author

Owner

Yeah, this task is not about replacing, but augmenting the client or make it a wrapper on top of a raw-ip one.