From: Jiri Pirko <jpirko@redhat.com>
Date: Mon, 9 Aug 2010 18:38:33 +0200
Subject: [pci] msi: add option for lockless interrupt mode
Message-id: <20100809163833.GB3060@psychotron.redhat.com>
O-Subject: [RHEL5.3.z patch] BZ621938 [pci] msi: add option for lockless interrupt mode
Bugzilla: 621938
RH-Acked-by: Jarod Wilson <jarod@redhat.com>
RH-Acked-by: Jiri Olsa <jolsa@redhat.com>

BZ621938
https://bugzilla.redhat.com/show_bug.cgi?id=621938

====================== taken from the original 5.6 patch ======================
wmealing  suggested a patch for MSI.  He has an excellent write up in
the BZ of why the patch is neccesary for RHEL5 and I agree with his
assessment.

Simply the problem is this: On every MSI interrupt, we mask and unmask the
MSI irq on the PCI dev.  This masking and unmasking is an expensive PCI
read and write which are serialized via a pci lock.

Obviously, that is a significant performance hit.  We only need to mask when
we're changing the affinity of the interrupt, not for every interrupt.

After the initial submit, revers pointed out that blindly changing this could
potentially introduce regressions in MSI capable drivers.  Instead of making
the msi-no-pci-lock access the default, I really should be keeping the original
code, and making no pci lock msi access optional.  I've therefore introduced
a kernel parameter, msi_nolock (name suggested by jwilson), to enable the
"faster" MSI.

Successfully tested on a few systems (a mix of large and small AMD and Intel
systems) by me.
===============================================================================

Fixed to be applicable on 5.3.z.

Brew:
https://brewweb.devel.redhat.com/taskinfo?taskID=2666598

Please review and ack.

Jirka

Signed-off-by: Jiri Pirko <jpirko@redhat.com>

diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index a58b01c..ae370d2 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -1054,6 +1054,10 @@ running once the system is up.
 	mpu401=		[HW,OSS]
 			Format: <io>,<irq>
 
+	msi_nolock	[MSI] Don't acquire pci_lock on msi interrupts,
+			improves throughput but may cause issues for some
+			devices
+
 	MTD_Partition=	[MTD]
 			Format: <name>,<region-number>,<size>,<offset>
 
diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c
index 96bec55..94c20af 100644
--- a/drivers/pci/msi.c
+++ b/drivers/pci/msi.c
@@ -203,7 +203,7 @@ static void shutdown_msi_irq(unsigned int vector)
 	spin_unlock_irqrestore(&msi_lock, flags);
 }
 
-static void end_msi_irq_wo_maskbit(unsigned int vector)
+static void end_msi_irq(unsigned int vector)
 {
 	move_native_irq(vector);
 	ack_APIC_irq();
@@ -263,7 +263,7 @@ static struct hw_interrupt_type msi_irq_wo_maskbit_type = {
 	.enable		= do_nothing,
 	.disable	= do_nothing,
 	.ack		= do_nothing,
-	.end		= end_msi_irq_wo_maskbit,
+	.end		= end_msi_irq,
 	.set_affinity	= set_msi_affinity
 };
 
@@ -1266,6 +1266,18 @@ void pci_no_msi(void)
 	pci_msi_enable = 0;
 }
 
+static int __init enable_msi_nolock(char *str)
+{
+	printk(KERN_INFO "Enabling MSI without pci_lock on interrupts\n");
+	msix_irq_type.ack = do_nothing;
+	msix_irq_type.end = end_msi_irq;
+	msi_irq_w_maskbit_type.ack = do_nothing;
+	msi_irq_w_maskbit_type.end = end_msi_irq;
+
+	return 0;
+}
+__setup("msi_nolock", enable_msi_nolock);
+
 EXPORT_SYMBOL(pci_enable_msi);
 EXPORT_SYMBOL(pci_disable_msi);
 EXPORT_SYMBOL(pci_enable_msix);