Need to logging on which interfaces we were successfully created listener #709

New issue

Closed

opened 2023-09-28 09:13:04 +00:00 by anikeev-yadro · 1 comment

anikeev-yadro commented

2023-09-28 09:13:04 +00:00

Member

For more simple problem triaging by logs.
Because now we don't understand on which interfaces we were successfully created listener.

Sep 27 15:57:58 az frostfs-node[8681]: 2023-09-27T15:57:58.063Z        info        frostfs-node/main.go:77        storage engine service has been successfully initialized
Sep 27 15:57:58 az frostfs-node[8681]: 2023-09-27T15:57:58.063Z        info        frostfs-node/main.go:75        initializing gRPC service...
Sep 27 15:57:58 az frostfs-node[8681]: 2023-09-27T15:57:58.064Z        error        frostfs-node/grpc.go:73        can't listen gRPC endpoint        {"error": "listen tcp 192.168.201.152:8080: bind
: cannot assign requested address"}
Sep 27 15:57:58 az frostfs-node[8681]: 2023-09-27T15:57:58.064Z        error        frostfs-node/grpc.go:73        can't listen gRPC endpoint        {"error": "listen tcp 10.78.70.152:8080: bind: cannot assign requested address"}
Sep 27 15:57:58 az frostfs-node[8681]: 2023-09-27T15:57:58.064Z        error        frostfs-node/grpc.go:73        can't listen gRPC endpoint        {"error": "listen tcp 10.78.71.152:8080: bind: cannot assign requested address"}
Sep 27 15:57:58 az frostfs-node[8681]: 2023-09-27T15:57:58.064Z        info        frostfs-node/main.go:77        gRPC service has been successfully initialized
Sep 27 15:57:58 az frostfs-node[8681]: 2023-09-27T15:57:58.064Z        info        frostfs-node/main.go:75        initializing netmap service...
Sep 27 15:57:58 az frostfs-node[8681]: 2023-09-27T15:57:58.064Z        warn        client/constructor.go:148        could not create RPC client for endpoint        {"error": "WS client creation: dial tcp 127.0.0.1:40332: connect: connection refused", "endpoint": "ws://127.0.0.1:40332/ws"}
Sep 27 15:57:59 az frostfs-node[8681]: 2023-09-27T15:57:59.096Z        info        client/constructor.go:151        created RPC client for endpoint        {"endpoint": "ws://192.168.201.154:40332/ws"}
Sep 27 15:57:59 az frostfs-node[8681]: 2023-09-27T15:57:59.099Z        info        frostfs-node/morph.go:68        failed to set group signer scope, continue with Global        {"error": "record has not been found in NNS contract"}

We understand only with additional step

service@az:~$ sudo netstat -lp |grep node
tcp        0      0 localhost:6671          0.0.0.0:*               LISTEN      8681/frostfs-node
tcp        0      0 localhost:6672          0.0.0.0:*               LISTEN      8681/frostfs-node
tcp        0      0 192.168.201.52:http-alt 0.0.0.0:*               LISTEN      8681/frostfs-node
tcp        0      0 localhost:http-alt      0.0.0.0:*               LISTEN      8681/frostfs-node
tcp        0      0 localhost:8091          0.0.0.0:*               LISTEN      8681/frostfs-node

Describe the solution you'd like

Need to logging on which interfaces we were successfully created listener

## Is your feature request related to a problem? Please describe. For more simple problem triaging by logs. Because now we don't understand on which interfaces we were successfully created listener. ``` Sep 27 15:57:58 az frostfs-node[8681]: 2023-09-27T15:57:58.063Z info frostfs-node/main.go:77 storage engine service has been successfully initialized Sep 27 15:57:58 az frostfs-node[8681]: 2023-09-27T15:57:58.063Z info frostfs-node/main.go:75 initializing gRPC service... Sep 27 15:57:58 az frostfs-node[8681]: 2023-09-27T15:57:58.064Z error frostfs-node/grpc.go:73 can't listen gRPC endpoint {"error": "listen tcp 192.168.201.152:8080: bind : cannot assign requested address"} Sep 27 15:57:58 az frostfs-node[8681]: 2023-09-27T15:57:58.064Z error frostfs-node/grpc.go:73 can't listen gRPC endpoint {"error": "listen tcp 10.78.70.152:8080: bind: cannot assign requested address"} Sep 27 15:57:58 az frostfs-node[8681]: 2023-09-27T15:57:58.064Z error frostfs-node/grpc.go:73 can't listen gRPC endpoint {"error": "listen tcp 10.78.71.152:8080: bind: cannot assign requested address"} Sep 27 15:57:58 az frostfs-node[8681]: 2023-09-27T15:57:58.064Z info frostfs-node/main.go:77 gRPC service has been successfully initialized Sep 27 15:57:58 az frostfs-node[8681]: 2023-09-27T15:57:58.064Z info frostfs-node/main.go:75 initializing netmap service... Sep 27 15:57:58 az frostfs-node[8681]: 2023-09-27T15:57:58.064Z warn client/constructor.go:148 could not create RPC client for endpoint {"error": "WS client creation: dial tcp 127.0.0.1:40332: connect: connection refused", "endpoint": "ws://127.0.0.1:40332/ws"} Sep 27 15:57:59 az frostfs-node[8681]: 2023-09-27T15:57:59.096Z info client/constructor.go:151 created RPC client for endpoint {"endpoint": "ws://192.168.201.154:40332/ws"} Sep 27 15:57:59 az frostfs-node[8681]: 2023-09-27T15:57:59.099Z info frostfs-node/morph.go:68 failed to set group signer scope, continue with Global {"error": "record has not been found in NNS contract"} ``` We understand only with additional step ``` service@az:~$ sudo netstat -lp |grep node tcp 0 0 localhost:6671 0.0.0.0:* LISTEN 8681/frostfs-node tcp 0 0 localhost:6672 0.0.0.0:* LISTEN 8681/frostfs-node tcp 0 0 192.168.201.52:http-alt 0.0.0.0:* LISTEN 8681/frostfs-node tcp 0 0 localhost:http-alt 0.0.0.0:* LISTEN 8681/frostfs-node tcp 0 0 localhost:8091 0.0.0.0:* LISTEN 8681/frostfs-node ``` ## Describe the solution you'd like Need to logging on which interfaces we were successfully created listener

anikeev-yadro added the

triage

label 2023-09-28 09:13:04 +00:00

acid-ant self-assigned this 2023-09-29 06:10:26 +00:00

acid-ant commented

2023-10-02 07:40:34 +00:00

Member

Test hints for pprof, metrics, grpc, control:

annikifa@annikiforov:~/workspace$ docker logs s02 2>&1 | grep listening
2023-10-02T09:12:54.899Z	info	frostfs-node/grpc.go:113	start listening endpoint	{"service": "gRPC", "endpoint": "192.168.130.72:8080"}
2023-10-02T09:12:55.905Z	info	frostfs-node/control.go:72	start listening endpoint	{"service": "control", "endpoint": "s02.frostfs.devenv:8081"}
2023-10-02T09:12:55.905Z	info	frostfs-node/httpcomponent.go:47	start listening endpoint	{"service": "pprof", "endpoint": ":6060"}
2023-10-02T09:12:55.905Z	info	frostfs-node/httpcomponent.go:47	start listening endpoint	{"service": "metrics", "endpoint": ":9090"}
annikifa@annikiforov:~/workspace$ 
annikifa@annikiforov:~/workspace$

Test hints for nats:

annikifa@annikiforov:~/workspace$ 
annikifa@annikiforov:~/workspace$ docker logs s02 2>&1 | grep nats
2023-10-02T09:12:54.899Z	info	frostfs-node/notificator.go:171	nats: successfully connected to endpoint	{"endpoint": "tls://nats.frostfs.devenv:4222"}
annikifa@annikiforov:~/workspace$

Test hints for `pprof`, `metrics`, `grpc`, `control`: ``` annikifa@annikiforov:~/workspace$ docker logs s02 2>&1 | grep listening 2023-10-02T09:12:54.899Z info frostfs-node/grpc.go:113 start listening endpoint {"service": "gRPC", "endpoint": "192.168.130.72:8080"} 2023-10-02T09:12:55.905Z info frostfs-node/control.go:72 start listening endpoint {"service": "control", "endpoint": "s02.frostfs.devenv:8081"} 2023-10-02T09:12:55.905Z info frostfs-node/httpcomponent.go:47 start listening endpoint {"service": "pprof", "endpoint": ":6060"} 2023-10-02T09:12:55.905Z info frostfs-node/httpcomponent.go:47 start listening endpoint {"service": "metrics", "endpoint": ":9090"} annikifa@annikiforov:~/workspace$ annikifa@annikiforov:~/workspace$ ``` Test hints for `nats`: ``` annikifa@annikiforov:~/workspace$ annikifa@annikiforov:~/workspace$ docker logs s02 2>&1 | grep nats 2023-10-02T09:12:54.899Z info frostfs-node/notificator.go:171 nats: successfully connected to endpoint {"endpoint": "tls://nats.frostfs.devenv:4222"} annikifa@annikiforov:~/workspace$ ```

acid-ant referenced this issue from a pull request that will close it,

2023-10-02 07:42:03 +00:00

node: Put in log info about listening endpoints #714