node: Add metrics for current GRPC endpoint status #580

acid-ant · 2023-08-09T06:25:36Z

acid-ant commented

2023-08-09 06:25:36 +00:00

Close #565
Metrics example:

# HELP frostfs_node_grpc_health GRPC Endpoint health
# TYPE frostfs_node_grpc_health gauge
frostfs_node_grpc_health{endpoint="s01.frostfs.devenv:8080"} 1
frostfs_node_grpc_health{endpoint="s01ssss.frostfs.devenv:8080"} 0

Signed-off-by: Anton Nikiforov an.nikiforov@yadro.com

Close #565 Metrics example: ``` # HELP frostfs_node_grpc_health GRPC Endpoint health # TYPE frostfs_node_grpc_health gauge frostfs_node_grpc_health{endpoint="s01.frostfs.devenv:8080"} 1 frostfs_node_grpc_health{endpoint="s01ssss.frostfs.devenv:8080"} 0 ``` Signed-off-by: Anton Nikiforov <an.nikiforov@yadro.com>

acid-ant requested review from storage-core-committers 2023-08-09 06:25:48 +00:00

acid-ant requested review from storage-core-developers 2023-08-09 06:25:49 +00:00

fyrchik reviewed 2023-08-09 10:24:32 +00:00

cmd/frostfs-node/grpc.go Outdated

					
				@ -69,9 +69,11 @@ func initGRPC(c *cfg) {

						lis, err := net.Listen("tcp", sc.Endpoint())

						if err != nil {

							c.metricsCollector.GrpcMetrics().MarkUnhealthy(sc.Endpoint())

fyrchik commented

2023-08-09 10:21:59 +00:00

srv.Serve can fail too, what about setting status there?

`srv.Serve` can fail too, what about setting status there?

acid-ant commented

2023-08-09 12:03:51 +00:00

My fault, updated. Introduced new slice because listener return string representation of address based on implementation.

fyrchik commented

2023-08-09 14:20:39 +00:00

What are the differences between representations?

acid-ant commented

2023-08-09 14:59:20 +00:00

According to doc it is up to implementation the exact form of the string. Looks like there are no guarantees that String() will return the value which equal to arguments of net.Listen(...)
https://github.com/golang/go/blob/master/src/net/net.go#L96

According to doc it is up to implementation the exact form of the string. Looks like there are no guarantees that String() will return the value which equal to arguments of `net.Listen(...)` https://github.com/golang/go/blob/master/src/net/net.go#L96

pkg/metrics/grpc.go Outdated

					
				@ -0,0 +20,4 @@

							prometheus.GaugeOpts{

								Namespace: namespace,

								Subsystem: grpcSubsystem,

								Name:      "health",

fyrchik commented

2023-08-09 10:20:57 +00:00

Maybe frostfs_grpc_server_health?

Maybe `frostfs_grpc_server_health`?

acid-ant commented

2023-08-09 12:01:32 +00:00

Agree, updated. Now it looks like this:

# TYPE frostfs_node_grpc_server_health gauge
frostfs_node_grpc_server_health{endpoint="s01.frostfs.devenv:8080"} 1
frostfs_node_grpc_server_health{endpoint="s01ssss.frostfs.devenv:8080"} 0

Agree, updated. Now it looks like this: ``` # TYPE frostfs_node_grpc_server_health gauge frostfs_node_grpc_server_health{endpoint="s01.frostfs.devenv:8080"} 1 frostfs_node_grpc_server_health{endpoint="s01ssss.frostfs.devenv:8080"} 0 ```

fyrchik marked this conversation as resolved

acid-ant force-pushed bugfix/565-metrics-for-grpc-endpnt from aca9cfba17 to 1d65a1addb

2023-08-09 12:01:14 +00:00

Compare

dstepanov-yadro approved these changes 2023-08-10 07:02:28 +00:00

aarifullin approved these changes 2023-08-10 09:02:22 +00:00

aarifullin left a comment

LGTM

fyrchik merged commit 4ad0ebb32f into master

2023-08-10 16:37:52 +00:00

Sign in to join this conversation.

No reviewers