After node and ir services was stopped status was not changed and kept `STOPPING` #1283

New issue

Closed

opened 2024-07-31 07:30:54 +00:00 by anikeev-yadro · 0 comments

anikeev-yadro commented

2024-07-31 07:30:54 +00:00

Member

Related to #552

Expected Behavior

After node and ir services was stopped the status changed to STOPPED

Current Behavior

After node and ir services was stopped the status kept STOPPING

Steps to Reproduce (for bugs)

Stop the node (ir) and see the Status field

root@aanikeev-node1:/etc/frostfs/storage# systemctl status frostfs-storage
○ frostfs-storage.service - FrostFS Storage node
     Loaded: loaded (/usr/lib/systemd/system/frostfs-storage.service; enabled; preset: disabled)
     Active: inactive (dead) since Wed 2024-07-31 06:55:10 UTC; 4min 19s ago
   Duration: 3min 30.145s
    Process: 1071039 ExecStart=/usr/bin/frostfs-node --config /etc/frostfs/storage/config.yml --config-dir /etc/frostfs/storage/conf.d (code=exited, status=0/SUCCESS)
   Main PID: 1071039 (code=exited, status=0/SUCCESS)
     Status: "STOPPING"
        CPU: 3min 26.023s

июл 31 06:55:10 aanikeev-node1 frostfs-node[1071039]: 2024-07-31T06:55:10.257Z        info        shard/gc.go:266        waiting for GC workers to stop...
июл 31 06:55:10 aanikeev-node1 frostfs-node[1071039]: 2024-07-31T06:55:10.257Z        info        shard/rebuilder.go:73        blobstore rebuild stopped        {"shard_id": "KEczo87648zk4fjTTbp2ki"}
июл 31 06:55:10 aanikeev-node1 frostfs-node[1071039]: 2024-07-31T06:55:10.258Z        info        shard/gc.go:266        waiting for GC workers to stop...
июл 31 06:55:10 aanikeev-node1 frostfs-node[1071039]: 2024-07-31T06:55:10.258Z        info        shard/rebuilder.go:73        blobstore rebuild stopped        {"shard_id": "M35CgaGTAaduhfkLcdb7AV"}
июл 31 06:55:10 aanikeev-node1 frostfs-node[1071039]: 2024-07-31T06:55:10.260Z        info        shard/gc.go:266        waiting for GC workers to stop...
июл 31 06:55:10 aanikeev-node1 frostfs-node[1071039]: 2024-07-31T06:55:10.260Z        info        frostfs-node/config.go:1076        all components of the storage engine closed successfully
июл 31 06:55:10 aanikeev-node1 frostfs-node[1071039]: 2024-07-31T06:55:10.261Z        info        frostfs-node/config.go:1255        termination signal processing is complete
июл 31 06:55:10 aanikeev-node1 systemd[1]: frostfs-storage.service: Deactivated successfully.
июл 31 06:55:10 aanikeev-node1 systemd[1]: Stopped FrostFS Storage node.
июл 31 06:55:10 aanikeev-node1 systemd[1]: frostfs-storage.service: Consumed 3min 26.023s CPU time.
root@aanikeev-node1:/etc/frostfs/storage# systemctl start frostfs-storage

root@aanikeev-node1[alone_datacenter]:~# systemctl status frostfs-ir
○ frostfs-ir.service - FrostFS InnerRing node
     Loaded: loaded (/usr/lib/systemd/system/frostfs-ir.service; enabled; preset: disabled)
     Active: inactive (dead) since Wed 2024-07-31 07:34:13 UTC; 3s ago
   Duration: 18h 4min 9.292s
    Process: 61597 ExecStart=/usr/bin/frostfs-ir --config /etc/frostfs/ir/config.yml -config-dir /etc/frostfs/ir/conf.d (code=exited, status=0/SUCCESS)
   Main PID: 61597 (code=exited, status=0/SUCCESS)
     Status: "STOPPING"
        CPU: 3min 21.966s

июл 31 07:34:13 aanikeev-node1 systemd[1]: Stopping FrostFS InnerRing node...
июл 31 07:34:13 aanikeev-node1 frostfs-ir[61597]: 2024-07-31T07:34:13.052Z        info        frostfs-ir/config.go:72        termination signal has been received, stopping...
июл 31 07:34:13 aanikeev-node1 frostfs-ir[61597]: 2024-07-31T07:34:13.053Z        info        event/listener.go:258        stop event listener by context        {"chain": "morph", "reason": "context canceled"}
июл 31 07:34:13 aanikeev-node1 frostfs-ir[61597]: 2024-07-31T07:34:13.060Z        info        frostfs-ir/httpcomponent.go:60        shutdown prometheus
июл 31 07:34:13 aanikeev-node1 frostfs-ir[61597]: 2024-07-31T07:34:13.060Z        info        frostfs-ir/httpcomponent.go:60        shutdown pprof
июл 31 07:34:13 aanikeev-node1 frostfs-ir[61597]: 2024-07-31T07:34:13.060Z        info        frostfs-ir/config.go:75        termination signal processing is complete
июл 31 07:34:13 aanikeev-node1 frostfs-ir[61597]: 2024-07-31T07:34:13.060Z        info        frostfs-ir/main.go:112        application stopped
июл 31 07:34:13 aanikeev-node1 systemd[1]: frostfs-ir.service: Deactivated successfully.
июл 31 07:34:13 aanikeev-node1 systemd[1]: Stopped FrostFS InnerRing node.
июл 31 07:34:13 aanikeev-node1 systemd[1]: frostfs-ir.service: Consumed 3min 21.966s CPU time.

Regression

Your Environment

Virtual

##Version

FrostFS Storage node
Version: v0.42.2
GoVersion: go1.22.5

FrostFS Inner Ring node
Version: v0.42.2
GoVersion: go1.22.5

Related to https://git.frostfs.info/TrueCloudLab/frostfs-node/issues/552 ## Expected Behavior After node and ir services was stopped the status changed to `STOPPED` ## Current Behavior After node and ir services was stopped the status kept `STOPPING` ## Steps to Reproduce (for bugs) 1. Stop the node (ir) and see the `Status` field ``` root@aanikeev-node1:/etc/frostfs/storage# systemctl status frostfs-storage ○ frostfs-storage.service - FrostFS Storage node Loaded: loaded (/usr/lib/systemd/system/frostfs-storage.service; enabled; preset: disabled) Active: inactive (dead) since Wed 2024-07-31 06:55:10 UTC; 4min 19s ago Duration: 3min 30.145s Process: 1071039 ExecStart=/usr/bin/frostfs-node --config /etc/frostfs/storage/config.yml --config-dir /etc/frostfs/storage/conf.d (code=exited, status=0/SUCCESS) Main PID: 1071039 (code=exited, status=0/SUCCESS) Status: "STOPPING" CPU: 3min 26.023s июл 31 06:55:10 aanikeev-node1 frostfs-node[1071039]: 2024-07-31T06:55:10.257Z info shard/gc.go:266 waiting for GC workers to stop... июл 31 06:55:10 aanikeev-node1 frostfs-node[1071039]: 2024-07-31T06:55:10.257Z info shard/rebuilder.go:73 blobstore rebuild stopped {"shard_id": "KEczo87648zk4fjTTbp2ki"} июл 31 06:55:10 aanikeev-node1 frostfs-node[1071039]: 2024-07-31T06:55:10.258Z info shard/gc.go:266 waiting for GC workers to stop... июл 31 06:55:10 aanikeev-node1 frostfs-node[1071039]: 2024-07-31T06:55:10.258Z info shard/rebuilder.go:73 blobstore rebuild stopped {"shard_id": "M35CgaGTAaduhfkLcdb7AV"} июл 31 06:55:10 aanikeev-node1 frostfs-node[1071039]: 2024-07-31T06:55:10.260Z info shard/gc.go:266 waiting for GC workers to stop... июл 31 06:55:10 aanikeev-node1 frostfs-node[1071039]: 2024-07-31T06:55:10.260Z info frostfs-node/config.go:1076 all components of the storage engine closed successfully июл 31 06:55:10 aanikeev-node1 frostfs-node[1071039]: 2024-07-31T06:55:10.261Z info frostfs-node/config.go:1255 termination signal processing is complete июл 31 06:55:10 aanikeev-node1 systemd[1]: frostfs-storage.service: Deactivated successfully. июл 31 06:55:10 aanikeev-node1 systemd[1]: Stopped FrostFS Storage node. июл 31 06:55:10 aanikeev-node1 systemd[1]: frostfs-storage.service: Consumed 3min 26.023s CPU time. root@aanikeev-node1:/etc/frostfs/storage# systemctl start frostfs-storage ``` ``` root@aanikeev-node1[alone_datacenter]:~# systemctl status frostfs-ir ○ frostfs-ir.service - FrostFS InnerRing node Loaded: loaded (/usr/lib/systemd/system/frostfs-ir.service; enabled; preset: disabled) Active: inactive (dead) since Wed 2024-07-31 07:34:13 UTC; 3s ago Duration: 18h 4min 9.292s Process: 61597 ExecStart=/usr/bin/frostfs-ir --config /etc/frostfs/ir/config.yml -config-dir /etc/frostfs/ir/conf.d (code=exited, status=0/SUCCESS) Main PID: 61597 (code=exited, status=0/SUCCESS) Status: "STOPPING" CPU: 3min 21.966s июл 31 07:34:13 aanikeev-node1 systemd[1]: Stopping FrostFS InnerRing node... июл 31 07:34:13 aanikeev-node1 frostfs-ir[61597]: 2024-07-31T07:34:13.052Z info frostfs-ir/config.go:72 termination signal has been received, stopping... июл 31 07:34:13 aanikeev-node1 frostfs-ir[61597]: 2024-07-31T07:34:13.053Z info event/listener.go:258 stop event listener by context {"chain": "morph", "reason": "context canceled"} июл 31 07:34:13 aanikeev-node1 frostfs-ir[61597]: 2024-07-31T07:34:13.060Z info frostfs-ir/httpcomponent.go:60 shutdown prometheus июл 31 07:34:13 aanikeev-node1 frostfs-ir[61597]: 2024-07-31T07:34:13.060Z info frostfs-ir/httpcomponent.go:60 shutdown pprof июл 31 07:34:13 aanikeev-node1 frostfs-ir[61597]: 2024-07-31T07:34:13.060Z info frostfs-ir/config.go:75 termination signal processing is complete июл 31 07:34:13 aanikeev-node1 frostfs-ir[61597]: 2024-07-31T07:34:13.060Z info frostfs-ir/main.go:112 application stopped июл 31 07:34:13 aanikeev-node1 systemd[1]: frostfs-ir.service: Deactivated successfully. июл 31 07:34:13 aanikeev-node1 systemd[1]: Stopped FrostFS InnerRing node. июл 31 07:34:13 aanikeev-node1 systemd[1]: frostfs-ir.service: Consumed 3min 21.966s CPU time. ``` ## Regression No ## Your Environment Virtual ##Version ``` FrostFS Storage node Version: v0.42.2 GoVersion: go1.22.5 ``` ``` FrostFS Inner Ring node Version: v0.42.2 GoVersion: go1.22.5 ```

anikeev-yadro added the

bug

triage

labels 2024-07-31 07:30:54 +00:00

anikeev-yadro changed title from ~~After node was stopped status was not changed and kept STOPPING~~ to After node and ir services was stopped status was not changed and kept STOPPING

2024-07-31 07:34:58 +00:00