nvme 硬盘I/O读写能力下降的问题
发表于 : 2021-12-10 9:25
root@ghnode:/var/log# cd /var/log
root@ghnode:/var/log# cat syslog
Dec 10 06:48:39 ghnode kernel: [7222436.263648] nvme nvme1: I/O 691 QID 32 timeout, completion polled
Dec 10 07:00:53 ghnode snapd[2715773]: storehelpers.go:710: cannot refresh: snap has no updates available: "core18", "core20", "lxd"
Dec 10 07:00:53 ghnode snapd[2715773]: autorefresh.go:536: auto-refresh: all snaps are up-to-date
Dec 10 07:17:01 ghnode CRON[2253567]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Dec 10 07:21:31 ghnode kernel: [7224408.470712] nvme nvme3: I/O 256 QID 19 timeout, completion polled
Dec 10 07:56:00 ghnode kernel: [7226476.933010] nvme nvme1: I/O 516 QID 68 timeout, completion polled
Dec 10 07:56:33 ghnode kernel: [7226509.700739] nvme nvme4: I/O 399 QID 92 timeout, completion polled
Dec 10 07:57:05 ghnode kernel: [7226541.700429] nvme nvme1: I/O 780 QID 56 timeout, completion polled
Dec 10 07:57:46 ghnode kernel: [7226583.428105] nvme nvme0: I/O 130 QID 93 timeout, completion polled
Dec 10 08:14:32 ghnode kernel: [7227589.499523] nvme nvme0: I/O 425 QID 98 timeout, completion polled
Dec 10 08:17:01 ghnode CRON[2254017]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Dec 10 08:41:11 ghnode kernel: [7229188.462013] nvme nvme0: I/O 375 QID 119 timeout, completion polled
Dec 10 08:44:06 ghnode kernel: [7229362.796498] nvme nvme0: I/O 827 QID 99 timeout, completion polled
Dec 10 08:48:34 ghnode kernel: [7229630.826114] nvme nvme1: I/O 497 QID 8 timeout, completion polled
Dec 10 08:54:56 ghnode systemd[1]: Started Session 2559 of user milan.
Dec 10 08:54:56 ghnode systemd[1]: Started Session 2560 of user milan.
Dec 10 09:13:39 ghnode kernel: [7231136.093326] nvme nvme3: I/O 170 QID 112 timeout, completion polled
Dec 10 09:13:44 ghnode systemd[1]: Started Session 2561 of user milan.
Dec 10 09:13:45 ghnode systemd[1]: Started Session 2562 of user milan.
Dec 10 09:14:47 ghnode systemd[1]: session-2541.scope: Succeeded.
Dec 10 09:17:01 ghnode CRON[2272322]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
nvme nvme1: I/O 497 QID 8 timeout, completion polled 这个是什么问题, 该怎么解决
root@ghnode:/var/log# cat syslog
Dec 10 06:48:39 ghnode kernel: [7222436.263648] nvme nvme1: I/O 691 QID 32 timeout, completion polled
Dec 10 07:00:53 ghnode snapd[2715773]: storehelpers.go:710: cannot refresh: snap has no updates available: "core18", "core20", "lxd"
Dec 10 07:00:53 ghnode snapd[2715773]: autorefresh.go:536: auto-refresh: all snaps are up-to-date
Dec 10 07:17:01 ghnode CRON[2253567]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Dec 10 07:21:31 ghnode kernel: [7224408.470712] nvme nvme3: I/O 256 QID 19 timeout, completion polled
Dec 10 07:56:00 ghnode kernel: [7226476.933010] nvme nvme1: I/O 516 QID 68 timeout, completion polled
Dec 10 07:56:33 ghnode kernel: [7226509.700739] nvme nvme4: I/O 399 QID 92 timeout, completion polled
Dec 10 07:57:05 ghnode kernel: [7226541.700429] nvme nvme1: I/O 780 QID 56 timeout, completion polled
Dec 10 07:57:46 ghnode kernel: [7226583.428105] nvme nvme0: I/O 130 QID 93 timeout, completion polled
Dec 10 08:14:32 ghnode kernel: [7227589.499523] nvme nvme0: I/O 425 QID 98 timeout, completion polled
Dec 10 08:17:01 ghnode CRON[2254017]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Dec 10 08:41:11 ghnode kernel: [7229188.462013] nvme nvme0: I/O 375 QID 119 timeout, completion polled
Dec 10 08:44:06 ghnode kernel: [7229362.796498] nvme nvme0: I/O 827 QID 99 timeout, completion polled
Dec 10 08:48:34 ghnode kernel: [7229630.826114] nvme nvme1: I/O 497 QID 8 timeout, completion polled
Dec 10 08:54:56 ghnode systemd[1]: Started Session 2559 of user milan.
Dec 10 08:54:56 ghnode systemd[1]: Started Session 2560 of user milan.
Dec 10 09:13:39 ghnode kernel: [7231136.093326] nvme nvme3: I/O 170 QID 112 timeout, completion polled
Dec 10 09:13:44 ghnode systemd[1]: Started Session 2561 of user milan.
Dec 10 09:13:45 ghnode systemd[1]: Started Session 2562 of user milan.
Dec 10 09:14:47 ghnode systemd[1]: session-2541.scope: Succeeded.
Dec 10 09:17:01 ghnode CRON[2272322]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
nvme nvme1: I/O 497 QID 8 timeout, completion polled 这个是什么问题, 该怎么解决