From patchwork Fri Apr 8 12:49:25 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Suthikulpanit, Suravee" X-Patchwork-Id: 65404 Delivered-To: patch@linaro.org Received: by 10.112.43.237 with SMTP id z13csp50553lbl; Fri, 8 Apr 2016 05:52:56 -0700 (PDT) X-Received: by 10.66.136.41 with SMTP id px9mr12596877pab.80.1460119976773; Fri, 08 Apr 2016 05:52:56 -0700 (PDT) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id p10si759523pfj.229.2016.04.08.05.52.56; Fri, 08 Apr 2016 05:52:56 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@amdcloud.onmicrosoft.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758379AbcDHMwQ (ORCPT + 29 others); Fri, 8 Apr 2016 08:52:16 -0400 Received: from mail-bn1on0061.outbound.protection.outlook.com ([157.56.110.61]:21856 "EHLO na01-bn1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754469AbcDHMuR (ORCPT ); Fri, 8 Apr 2016 08:50:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=9eD8yPS7M6qpIBtH0M5pxAZ6Uq5W3X6a9Lb83ePlT0s=; b=TeZ0Y8sOXE6gPlPF2kpY3OHNW6CwjWh5Mx7WIxJKVBhYLMFcH3L9NAeiTBpGu0uAgAkYiyOwWUTN8d62Fh3V3qTlhzipRbjdX8XKop12ds4XNVOx6cstMofuNc6EJ3jkGi+5B6nAxVWk0fOzFqua8TFmMIKYZnY8w5nas81j90I= Authentication-Results: redhat.com; dkim=none (message not signed) header.d=none; redhat.com; dmarc=none action=none header.from=amd.com; Received: from localhost.localdomain (124.121.8.20) by SN1PR12MB0446.namprd12.prod.outlook.com (10.162.105.14) with Microsoft SMTP Server (TLS) id 15.1.453.26; Fri, 8 Apr 2016 12:50:07 +0000 From: Suravee Suthikulpanit To: , , , , , CC: , , , , Suravee Suthikulpanit Subject: [PART2 RFC v1 4/9] iommu/amd: Adding GALOG interrupt handler Date: Fri, 8 Apr 2016 07:49:25 -0500 Message-ID: <1460119770-2896-5-git-send-email-Suravee.Suthikulpanit@amd.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1460119770-2896-1-git-send-email-Suravee.Suthikulpanit@amd.com> References: <1460119770-2896-1-git-send-email-Suravee.Suthikulpanit@amd.com> MIME-Version: 1.0 X-Originating-IP: [124.121.8.20] X-ClientProxiedBy: HKNPR06CA0022.apcprd06.prod.outlook.com (10.141.16.12) To SN1PR12MB0446.namprd12.prod.outlook.com (10.162.105.14) X-MS-Office365-Filtering-Correlation-Id: ce425cbb-383c-4a46-0d58-08d35fac524b X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0446; 2:UHGoIw+llV+zOD4EHmlprj2crLO/vu5ev2F+h15yxivFBsBpWF1XEZ4765DmpiU/jsFZTkGNfll2PjbwEf83OlWrPyZ174e1djzx2T5dm5X4U0WbdesklsphvnZKjnp05dWMi/eSD3CJ/xYyld+MQer+GNLco2YcaztFDYbuwvHdNCuHmG2aFi4cDlDwTJHo; 3:ZLEA4P7WgfgYmIC6ac6P7ulVgID3bJaBMhu9yA/n3uo+Yz3zlVBQUupZX2UAQYcietw60IR1KGy2ZPKbjIFOqm8KdSWxgo/tgbNvt20xs8e/hSF+S9gXVUwRJPM2zXxx X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:SN1PR12MB0446; X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0446; 25:aptPmmjQ3m/wvHgodQqFqwzbJf07PhAEJK3xDUrlQQ8CLelvi917wWszk0LX030OprnN1L/q8LYQKElo4JnPKVVuikGfWVzTaGdLT9ggMlh0D3GefF/tjO4KrK2JhNraMDtpXGqIySJ4tyfQZOQXAkmyjYAwQvUmw9EFQ//BKaVBNrXmbj6C1990/1VZRAiDS5sB5y2fkNSGEpd3xuYPwYCLiDy5I2R08hzK5TuEQFDc0fTHwY2XXPiJnecUbgdA0p3MGh3B5dvWijQtpHg5zx1FQQ2OjVMXDWH37I2/AxydIEEASlh+kP2t4E2w1dKrMQSWyMgK7d+8ffhu/mq1J/02PAOftRNRfnyPoprlRcIuiFNlYtPKqfZm2H5vg9xKdZth313P5g1atkDGwf5ko0b8CTf2V+11xh+fLxdK7yy/AHU17K7kSxbCO6wKkL+7nZ/TROv7T1eYyXZTNDlt/i8YfcWrM3ZZBQJJFULLFWZGX/XlEQr9IbpXdB7uTmEx359KFN9srVXafH3G3qt1kjGgqkNyVhCLjh8o+LEb9HDOFcy3+LT94oTLUZWIZgp9BPzFftLO8ZQ6RcyCJAYdOzGb2/U7NJTeZ8o9Qz51tHWpmzauFrojuwX+b1MoANjTq3xQGrVwVVYXZeqx8c1HMgpWxh+hGxZjMHOb7F77VvFuGSahDrQCJ6c38xE55wZNSmJOriplObk4HEmM6bsZHQ== X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0446; 20:e0iYsgR5phCD33ES3OZXFIy9DWMm2voe6jCdGkQwW4rjCiqKRqxYbm6HRwx3eLZQEeOblLPZ4taAox52zp8NTQJzp8dl2iMXtY7OKtqUsSfzEn+KKv+0qYSrFzUVKpYh9c1xe2QinXne38yvQA0llTcAcYft5udSdNu3FtrMXZh3k+1wIjKYiBoruYlHEmYb0d7C/jZmW/e28v3xEK7Ii5wh8ohAa44u3D0lbSP9YC5I3ZgLv+dK4KYuZ8vAsq93n2CHovQhSo/9VVW6bfH2DYM3U1zzCG/rrkLWNQmEFIDKiu4rmtn4pCKicFv8ZMth+1sGkBSHRHIt0fmQL+hNe3io1RBPrmaJEmg3hp2lnZwjvTxCVQvvGm6QL0o39VCcWy5lkD1PPRh6pLrz2iEm46TJfytJT1780A9BDai75oKIRie+ZmCDfRZpVa5Yr1uYzfiAbPB0cXdRO3L7usdtBkdP1RcKa0RBsg5dtRq5r58zXqv099+RfsQBv5CuB4CZ; 4:BZAWTTWDc1YnbV26E7JsTJpiIKJ9n3JXPmau53tK+52UvkkTqbwcbFcES19Ix933PZx6wrWoe26CzMRhNj35AZGOM+hDeDJJNplkSuIC8HVhi9q9iank4PTmwhdyLcvKOGnIA4c3uj9wpiDIho1z7MCgzq18sCP+qA4ikbrF+s0awVNlBBKYMW7igSZW29smMr91GafGxUTQ/CgyOKg6mUn2d9L66XZO00DR2ha0XkWMbvFZ12Uww5xSWDkYgBRouMRKx1eTRdWfd1D05JSvgnjpLu9xyKUugE9PPoF2MtfcptlT8JQGuycB5tEJdiXRbQzUL/SYTt+ftAtt8zzxCWfAF5HD69E6Cc3NpsZRwy1TrdiYKslhxl+OIMOfwPXt X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046); SRVR:SN1PR12MB0446; BCL:0; PCL:0; RULEID:; SRVR:SN1PR12MB0446; X-Forefront-PRVS: 0906E83A25 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6069001)(6009001)(50466002)(48376002)(77096005)(47776003)(66066001)(229853001)(36756003)(86362001)(1096002)(42186005)(19580405001)(19580395003)(92566002)(6116002)(3846002)(586003)(5004730100002)(50986999)(76176999)(4326007)(189998001)(5003940100001)(2950100001)(2201001)(2906002)(5008740100001)(5001770100001)(81166005)(50226001); DIR:OUT; SFP:1101; SCL:1; SRVR:SN1PR12MB0446; H:localhost.localdomain; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0446; 23:Oj+6PzTNXVmr6DUW4elZTDuWQ5/1MszRWMz4T5dZ0bHv/hkcD21U1kIwGmN6V9WT+1xyzuScUQcX/tyqSjKrVCZ0mCBwRLEn2d0Noth+k0NRy9b8BRQeg6H/IeDxSPBnZ3rx9pXy51pD34sNxKp+UeB8MF2FhhpyT/2Q71cugz9YpoXnlXGr7c8COUGWBTVS3Vv5LNe/qyMRmGg9HbwW6sRSROqbWC9jAGmELfKdjxnNBdatidFDQ5j/DUcs5jAaUnZw7EUdKmgyXb+n3DBKcDcObFwm2l3/N5/bZZ1rkNYSF/PJ56iZESTn4WcvhuLD7mM8ppkeoncUP1BdJERfusBvP/xbPk5CXVF1VmoQlM/fqO/V5oS8upZsmls/kTeSDWjY4uGGFZWNC46SfHK8jgedDjXMkNipendjaR/sIuCSMyRnN2Oz6V+KUoT6MTNVTX12DQ2pF5f+GwNmR6l9AlvoZLfod6z95Xo/Qv0JKkE+JwnRacb3uVdEtBFDhltaGzTwrzv4J4S97zdWHhaWc3aMrqO0gS1a+jX+923eCXYUfRiEi3jq991pPiVR9AF8wXXEdZiXY1yFRIYZE+bWOEczova/PVFrxOJww/IDP0+vTScKsER++rMz5Vo4KfozVh7akU9c0a5biob9JrnJvGavtCdtJbz4fcSnPlo0Oy9owNNtaYEcq7cZtNFSK5E5KntStZQu2GNvLhtSc3p9cHtaoZtnJMSVcjcGU+rElpPInCzUMP2ckk3sv79b+rXaaAWruXehyT6UvFe7RccdP9HPlEIM2/kp+IamebB3K4MI96L/GvIOKumu2fvFynbUPmFMSe6hjHwYhRvecURYvK0mIH2QuKT4AVyXl7hdUbKzdIlL2ejwjydDb37Fp5KawaOcJTCgOBLFawux3OY2UQ== X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0446; 5:yCpNu+qKhk0LBrU4WLGWtcrosYvUmUFqc8BWoFGx8JvQcYBD1rv4niDO4ft6J0SzMPQAhM1lhOVkreOSiideJy3sGMm+beBeOO/6SJsFT08tHehSg0H+jDMFTXDu+gTTJfGDxrxua8n+aWWTQezbIA==; 24:6Qb/tO3xA7ZYNDoIMAof54wiYqFtgbjHLI7f28JRIcs4MSz/zbaJRey8fwvZSTmt/wO/zX3JRC/w+eJX3CnqvKkQ78B13GNPBieWuBO0WE0=; 20:S96UaruY2zQ9qlYEpxZhP290Wb8ajmgIi7Og62q2ntzSMUarF3meei4EU0H7AmGUXgq5RN+ZMj0/JZtwC4nEW4RURJxQ48bFkbjFdlZJlEFM5DDnfAF4tjmo+A+vDjcKnwRcJ/7pjf9P/rLAzbH0HT+8DqF0foZNNOhHRF9g5xUs/XPX9ZaZzJ33hZSRLPC4jmf9PfKuDqPkrB/KBOW4E2bzzcktDM0t9Zqrp9VgPGGsUYBfvB5VgbbrHZGz7kg+ X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2016 12:50:07.4659 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB0446 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Suravee Suthikulpanit This patch adds AMD IOMMU guest virtual APIC log (GALOG) handler. When IOMMU hardware receives an interrupt targeting a blocking vcpu, it creates an entry in the GALOG, and generates an interrupt to notify the AMD IOMMU driver. At this point, the driver processes the log entry, and notify the SVM driver via the registered iommu_ga_log_notifier function. Signed-off-by: Suravee Suthikulpanit --- drivers/iommu/amd_iommu.c | 103 ++++++++++++++++++++++++++++++++++++++++++++-- include/linux/amd-iommu.h | 10 +++++ 2 files changed, 110 insertions(+), 3 deletions(-) -- 1.9.1 diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c index af6079a..8cdde339 100644 --- a/drivers/iommu/amd_iommu.c +++ b/drivers/iommu/amd_iommu.c @@ -424,6 +424,8 @@ DECLARE_STATS_COUNTER(complete_ppr); DECLARE_STATS_COUNTER(invalidate_iotlb); DECLARE_STATS_COUNTER(invalidate_iotlb_all); DECLARE_STATS_COUNTER(pri_requests); +DECLARE_STATS_COUNTER(galog_max); +DECLARE_STATS_COUNTER(galog_total); static struct dentry *stats_dir; static struct dentry *de_fflush; @@ -462,6 +464,8 @@ static void amd_iommu_stats_init(void) amd_iommu_stats_add(&invalidate_iotlb); amd_iommu_stats_add(&invalidate_iotlb_all); amd_iommu_stats_add(&pri_requests); + amd_iommu_stats_add(&galog_max); + amd_iommu_stats_add(&galog_total); } #endif @@ -655,14 +659,102 @@ static void iommu_poll_ppr_log(struct amd_iommu *iommu) } } +static int (*iommu_ga_log_notifier)(int, int, int); + +int amd_iommu_register_ga_log_notifier(int (*notifier)(int, int, int)) +{ + iommu_ga_log_notifier = notifier; + + return 0; +} +EXPORT_SYMBOL(amd_iommu_register_ga_log_notifier); + +static void iommu_handle_ga_guest_nr_entry(struct amd_iommu *iommu, + u16 devid, u32 ga_tag) +{ + struct amd_ir_data *ir_data; + unsigned long flags; + int vec = 0; + + if (!iommu_ga_log_notifier) + return; + + spin_lock_irqsave(&iommu->ga_hash_lock, flags); + hash_for_each_possible(iommu->ga_hash, ir_data, hnode, ga_tag) { + vec = ir_data->irte_ga_entry.hi.fields.vector; + break; + } + spin_unlock_irqrestore(&iommu->ga_hash_lock, flags); + + if (vec) { + pr_debug("AMD-Vi: %s: devid=%#x, ga_tag=%#x\n", + __func__, devid, ga_tag); + + if (iommu_ga_log_notifier(GATAG_TO_AVICTAG(ga_tag), + GATAG_TO_VCPUID(ga_tag), vec) != 0) + pr_err("AMD-Vi: GA log notifier failed.\n"); + } +} + +static void iommu_poll_ga_log(struct amd_iommu *iommu) +{ + u32 head, tail, cnt = 0; + + if (iommu->ga_log == NULL) + return; + + head = readl(iommu->mmio_base + MMIO_GA_HEAD_OFFSET); + tail = readl(iommu->mmio_base + MMIO_GA_TAIL_OFFSET); + + while (head != tail) { + volatile u64 *raw; + u64 entry; + + raw = (u64 *)(iommu->ga_log + head); + cnt++; + + /* Avoid memcpy function-call overhead */ + entry = *raw; + + /* Update head pointer of hardware ring-buffer */ + head = (head + GA_ENTRY_SIZE) % GA_LOG_SIZE; + writel(head, iommu->mmio_base + MMIO_GA_HEAD_OFFSET); + + /* Handle GA entry */ + switch (GA_REQ_TYPE(entry)) { + case GA_GUEST_NR: + iommu_handle_ga_guest_nr_entry(iommu, + GA_DEVID(entry), + GA_TAG(entry)); + break; + default: + break; + } + + /* Refresh ring-buffer information */ + head = readl(iommu->mmio_base + MMIO_GA_HEAD_OFFSET); + tail = readl(iommu->mmio_base + MMIO_GA_TAIL_OFFSET); + } + + ADD_STATS_COUNTER(galog_total, cnt); + + if (STATS_COUNTER(galog_max) < cnt) + SET_STATS_COUNTER(galog_max, cnt); +} + +#define AMD_IOMMU_INT_MASK \ + (MMIO_STATUS_EVT_INT_MASK | \ + MMIO_STATUS_PPR_INT_MASK | \ + MMIO_STATUS_GALOG_INT_MASK) + irqreturn_t amd_iommu_int_thread(int irq, void *data) { struct amd_iommu *iommu = (struct amd_iommu *) data; u32 status = readl(iommu->mmio_base + MMIO_STATUS_OFFSET); - while (status & (MMIO_STATUS_EVT_INT_MASK | MMIO_STATUS_PPR_INT_MASK)) { - /* Enable EVT and PPR interrupts again */ - writel((MMIO_STATUS_EVT_INT_MASK | MMIO_STATUS_PPR_INT_MASK), + while (status & AMD_IOMMU_INT_MASK) { + /* Enable EVT and PPR and GA interrupts again */ + writel(AMD_IOMMU_INT_MASK, iommu->mmio_base + MMIO_STATUS_OFFSET); if (status & MMIO_STATUS_EVT_INT_MASK) { @@ -675,6 +767,11 @@ irqreturn_t amd_iommu_int_thread(int irq, void *data) iommu_poll_ppr_log(iommu); } + if (status & MMIO_STATUS_GALOG_INT_MASK) { + pr_devel("AMD-Vi: Processing IOMMU GA Log\n"); + iommu_poll_ga_log(iommu); + } + /* * Hardware bug: ERBT1312 * When re-enabling interrupt (by writing 1 diff --git a/include/linux/amd-iommu.h b/include/linux/amd-iommu.h index 2b08e79..36648fe 100644 --- a/include/linux/amd-iommu.h +++ b/include/linux/amd-iommu.h @@ -169,10 +169,20 @@ typedef void (*amd_iommu_invalidate_ctx)(struct pci_dev *pdev, int pasid); extern int amd_iommu_set_invalidate_ctx_cb(struct pci_dev *pdev, amd_iommu_invalidate_ctx cb); +/* IOMMU AVIC Function */ +extern int +amd_iommu_register_ga_log_notifier(int (*notifier)(int, int, int)); + #else static inline int amd_iommu_detect(void) { return -ENODEV; } +static inline int +amd_iommu_register_ga_log_notifier(int (*notifier)(int, int, int)) +{ + return 0; +} + #endif #endif /* _ASM_X86_AMD_IOMMU_H */