IB/srp: Fix deadlock between host removal and multipathd
If scsi_remove_host() is invoked after a SCSI device has been blocked,
if the fast_io_fail_tmo or dev_loss_tmo work gets scheduled on the
workqueue executing srp_remove_work() and if an I/O request is
scheduled after the SCSI device had been blocked by e.g. multipathd
then the following deadlock can occur:
kworker/6:1 D
ffff880831f3c460 0 195 2 0x00000000
Call Trace:
[<
ffffffff814aafd9>] schedule+0x29/0x70
[<
ffffffff814aa0ef>] schedule_timeout+0x10f/0x2a0
[<
ffffffff8105af6f>] msleep+0x2f/0x40
[<
ffffffff8123b0ae>] __blk_drain_queue+0x4e/0x180
[<
ffffffff8123d2d5>] blk_cleanup_queue+0x225/0x230
[<
ffffffffa0010732>] __scsi_remove_device+0x62/0xe0 [scsi_mod]
[<
ffffffffa000ed2f>] scsi_forget_host+0x6f/0x80 [scsi_mod]
[<
ffffffffa0002eba>] scsi_remove_host+0x7a/0x130 [scsi_mod]
[<
ffffffffa07cf5c5>] srp_remove_work+0x95/0x180 [ib_srp]
[<
ffffffff8106d7aa>] process_one_work+0x1ea/0x6c0
[<
ffffffff8106dd9b>] worker_thread+0x11b/0x3a0
[<
ffffffff810758bd>] kthread+0xed/0x110
[<
ffffffff814b972c>] ret_from_fork+0x7c/0xb0
multipathd D
ffff880096acc460 0 5340 1 0x00000000
Call Trace:
[<
ffffffff814aafd9>] schedule+0x29/0x70
[<
ffffffff814aa0ef>] schedule_timeout+0x10f/0x2a0
[<
ffffffff814ab79b>] io_schedule_timeout+0x9b/0xf0
[<
ffffffff814abe1c>] wait_for_completion_io_timeout+0xdc/0x110
[<
ffffffff81244b9b>] blk_execute_rq+0x9b/0x100
[<
ffffffff8124f665>] sg_io+0x1a5/0x450
[<
ffffffff8124fd21>] scsi_cmd_ioctl+0x2a1/0x430
[<
ffffffff8124fef2>] scsi_cmd_blk_ioctl+0x42/0x50
[<
ffffffffa00ec97e>] sd_ioctl+0xbe/0x140 [sd_mod]
[<
ffffffff8124bd04>] blkdev_ioctl+0x234/0x840
[<
ffffffff811cb491>] block_ioctl+0x41/0x50
[<
ffffffff811a0df0>] do_vfs_ioctl+0x300/0x520
[<
ffffffff811a1051>] SyS_ioctl+0x41/0x80
[<
ffffffff814b9962>] tracesys+0xd0/0xd5
Fix this by scheduling removal work on another workqueue than the
transport layer timers.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Sagi Grimberg <sagig@mellanox.com>
Reviewed-by: David Dillow <dave@thedillows.org>
Cc: Sebastian Parschauer <sebastian.riemer@profitbricks.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Roland Dreier <roland@purestorage.com>