一、问题背景
存在某个服务此前开机后运行正常,今天服务器重启后该服务未能正常启动,一定程度上影响了业务系统的运行,随即进行排查。
二、排查过程
首先查看服务状态:systemctl status YLMonitor
[root@cheliangweb sh]# systemctl status YLMonitor
● YLMonitor.service - YLMonitor service
Loaded: loaded (/etc/systemd/system/YLMonitor.service; enabled; vendor preset: disabled)
Active: active (running) since 二 2023-09-12 09:35:23 CST; 28s ago
Main PID: 8078 (YLMonitor)
Tasks: 8
CGroup: /system.slice/YLMonitor.service
└─8078 /temp/YLMonitor/YLMonitor -c /temp/YLMonitor/YLMonitor.ini
9月 12 09:35:23 cheliangweb systemd[1]: Started YLMonitor service.
9月 12 09:35:23 cheliangweb YLMonitor[8078]: 2023/09/12 09:35:23 [I] [root.go:220] start frpc serv...ni]
9月 12 09:35:23 cheliangweb YLMonitor[8078]: 2023/09/12 09:35:23 [I] [service.go:301] [cbf8b715e98...9f]
9月 12 09:35:23 cheliangweb YLMonitor[8078]: 2023/09/12 09:35:23 [I] [proxy_manager.go:150] [cbf8b...pc]
9月 12 09:35:23 cheliangweb YLMonitor[8078]: 2023/09/12 09:35:23 [I] [control.go:172] [cbf8b715e98...ess
9月 12 09:35:24 cheliangweb YLMonitor[8078]: 2023/09/12 09:35:24 [I] [control.go:172] [cbf8b715e98...ess
9月 12 09:35:24 cheliangweb YLMonitor[8078]: 2023/09/12 09:35:24 [I] [control.go:172] [cbf8b715e98...ess
Hint: Some lines were ellipsized, use -l to show in full.
journalctl -u YLMonitor
继续查看服务日志:
根据以上信息判断可能是DNS服务的问题,开机后通过该DNS连接目标域名不可达,随即服务启动失败。
vim /etc/resolv.conf
添加一条114的DNS。
# Generated by NetworkManager
nameserver 114.114.114.114
再次执行重启,发现服务正常:
[root@cheliangweb sh]# systemctl status YLMonitor
● YLMonitor.service - YLMonitor service
Loaded: loaded (/etc/systemd/system/YLMonitor.service; enabled; vendor preset: disabled)
Active: active (running) since 二 2023-09-12 09:35:23 CST; 28s ago
Main PID: 8078 (YLMonitor)
Tasks: 8
CGroup: /system.slice/YLMonitor.service
└─8078 /temp/YLMonitor/YLMonitor -c /temp/YLMonitor/YLMonitor.ini
9月 12 09:35:23 cheliangweb systemd[1]: Started YLMonitor service.
9月 12 09:35:23 cheliangweb YLMonitor[8078]: 2023/09/12 09:35:23 [I] [root.go:220] start frpc serv...ni]
9月 12 09:35:23 cheliangweb YLMonitor[8078]: 2023/09/12 09:35:23 [I] [service.go:301] [cbf8b715e98...9f]
9月 12 09:35:23 cheliangweb YLMonitor[8078]: 2023/09/12 09:35:23 [I] [proxy_manager.go:150] [cbf8b...pc]
9月 12 09:35:23 cheliangweb YLMonitor[8078]: 2023/09/12 09:35:23 [I] [control.go:172] [cbf8b715e98...ess
9月 12 09:35:24 cheliangweb YLMonitor[8078]: 2023/09/12 09:35:24 [I] [control.go:172] [cbf8b715e98...ess
9月 12 09:35:24 cheliangweb YLMonitor[8078]: 2023/09/12 09:35:24 [I] [control.go:172] [cbf8b715e98...ess
Hint: Some lines were ellipsized, use -l to show in full.