From patchwork Wed Aug 16 16:18:16 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 110260 Delivered-To: patch@linaro.org Received: by 10.140.95.78 with SMTP id h72csp995465qge; Wed, 16 Aug 2017 09:18:39 -0700 (PDT) X-Received: by 10.84.128.73 with SMTP id 67mr2405178pla.155.1502900319061; Wed, 16 Aug 2017 09:18:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1502900319; cv=none; d=google.com; s=arc-20160816; b=N11RaBQrEQVFirOq0x4m41fpMb6pTE4mKj7LjyFXOmHKBwnXXidcBjcRoHzLol6fZ0 b5IkzN60fIhhQeLXGNGZaTOcg2W3O67RfK4XYTG73TacVH9Vf8TDzt3W00GbKu6aciys e3ZQmhenPKAFF8Bo2FH/WKWm8pVTawZ0OH0arw71LcI0LYxqVHVNpxxoGac8S/y75W1j IOs7m/rR6Ds2R+YPymQNvgv/moHJ5rdZMFSxrU1NMH7WVKGYTyLNZ4TlukrHWr0r7wqT FlXkmoUaF7DXzBo2OH1RzgcVJybC81KQEGL/V9sV4fRRkJJpO+nlb+fSiPCBEQPFh61D CKlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :arc-authentication-results; bh=1x/Q795z7czEZ42s2jXb4EYvqJIFMoMWpnmstKZmfYQ=; b=ETspz0lER2lQw1ZuH888DARgZT+oZvsUrO06ZHiCFNcfdHqljlfA9miSUwczVGnVXa 2RupIahtoMe5m7EX4qT1TuW860H6JGViF+foZWpvMOU4ie01yY2Yh94fcYseXicl+xjv rK+akSbtlpK/vKTRWs1mTdxcccKOPMxl02T2b1J1JPuj2EB+yiYpKf0iuntC3tqPyYsW A9ZWd+jG0FKgjFSNbLjtMc8V/yiuXXSjOr5wVKrLJvW6UUC6wJz6zALUrK7a5Cn+ed8y RwREpsC83kNhST1VYFQ8TdKhK3ONCEkW2TMsfphOAiE3qVkPStI3ekdzn/doeCrMsmuy uaEQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m23si681292pgc.461.2017.08.16.09.18.38; Wed, 16 Aug 2017 09:18:39 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752166AbdHPQSX (ORCPT + 26 others); Wed, 16 Aug 2017 12:18:23 -0400 Received: from foss.arm.com ([217.140.101.70]:37532 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751461AbdHPQSW (ORCPT ); Wed, 16 Aug 2017 12:18:22 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E2D582B; Wed, 16 Aug 2017 09:18:21 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B3B213F577; Wed, 16 Aug 2017 09:18:21 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id BE4A01AE17BD; Wed, 16 Aug 2017 17:18:21 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, Will Deacon , Peter Zijlstra Subject: [PATCH v2 1/2] perf/aux: Make aux_{head, wakeup} ring_buffer members long Date: Wed, 16 Aug 2017 17:18:16 +0100 Message-Id: <1502900297-21839-1-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The aux_head and aux_wakeup members of struct ring_buffer are defined using the local_t type, despite the fact that they are only accessed via the perf_aux_output_* functions, which cannot race with each other for a given ring buffer. This patch changes the type of the members to long, so we can avoid using the local_* API where it isn't needed. Cc: Mark Rutland Cc: Alexander Shishkin Cc: Peter Zijlstra Signed-off-by: Will Deacon --- kernel/events/internal.h | 4 ++-- kernel/events/ring_buffer.c | 31 ++++++++++++++----------------- 2 files changed, 16 insertions(+), 19 deletions(-) -- 2.1.4 diff --git a/kernel/events/internal.h b/kernel/events/internal.h index 486fd78eb8d5..2941b868353c 100644 --- a/kernel/events/internal.h +++ b/kernel/events/internal.h @@ -38,9 +38,9 @@ struct ring_buffer { struct user_struct *mmap_user; /* AUX area */ - local_t aux_head; + long aux_head; local_t aux_nest; - local_t aux_wakeup; + long aux_wakeup; unsigned long aux_pgoff; int aux_nr_pages; int aux_overwrite; diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c index 14a240ff439d..87c28faca8e4 100644 --- a/kernel/events/ring_buffer.c +++ b/kernel/events/ring_buffer.c @@ -367,7 +367,7 @@ void *perf_aux_output_begin(struct perf_output_handle *handle, if (WARN_ON_ONCE(local_xchg(&rb->aux_nest, 1))) goto err_put; - aux_head = local_read(&rb->aux_head); + aux_head = rb->aux_head; handle->rb = rb; handle->event = event; @@ -382,7 +382,7 @@ void *perf_aux_output_begin(struct perf_output_handle *handle, */ if (!rb->aux_overwrite) { aux_tail = ACCESS_ONCE(rb->user_page->aux_tail); - handle->wakeup = local_read(&rb->aux_wakeup) + rb->aux_watermark; + handle->wakeup = rb->aux_wakeup + rb->aux_watermark; if (aux_head - aux_tail < perf_aux_size(rb)) handle->size = CIRC_SPACE(aux_head, aux_tail, perf_aux_size(rb)); @@ -434,12 +434,12 @@ void perf_aux_output_end(struct perf_output_handle *handle, unsigned long size) handle->aux_flags |= PERF_AUX_FLAG_OVERWRITE; aux_head = handle->head; - local_set(&rb->aux_head, aux_head); + rb->aux_head = aux_head; } else { handle->aux_flags &= ~PERF_AUX_FLAG_OVERWRITE; - aux_head = local_read(&rb->aux_head); - local_add(size, &rb->aux_head); + aux_head = rb->aux_head; + rb->aux_head += size; } if (size || handle->aux_flags) { @@ -451,11 +451,10 @@ void perf_aux_output_end(struct perf_output_handle *handle, unsigned long size) handle->aux_flags); } - aux_head = rb->user_page->aux_head = local_read(&rb->aux_head); - - if (aux_head - local_read(&rb->aux_wakeup) >= rb->aux_watermark) { + rb->user_page->aux_head = rb->aux_head; + if (rb->aux_head - rb->aux_wakeup >= rb->aux_watermark) { wakeup = true; - local_add(rb->aux_watermark, &rb->aux_wakeup); + rb->aux_wakeup += rb->aux_watermark; } if (wakeup) { @@ -480,22 +479,20 @@ EXPORT_SYMBOL_GPL(perf_aux_output_end); int perf_aux_output_skip(struct perf_output_handle *handle, unsigned long size) { struct ring_buffer *rb = handle->rb; - unsigned long aux_head; if (size > handle->size) return -ENOSPC; - local_add(size, &rb->aux_head); + rb->aux_head += size; - aux_head = rb->user_page->aux_head = local_read(&rb->aux_head); - if (aux_head - local_read(&rb->aux_wakeup) >= rb->aux_watermark) { + rb->user_page->aux_head = rb->aux_head; + if (rb->aux_head - rb->aux_wakeup >= rb->aux_watermark) { perf_output_wakeup(handle); - local_add(rb->aux_watermark, &rb->aux_wakeup); - handle->wakeup = local_read(&rb->aux_wakeup) + - rb->aux_watermark; + rb->aux_wakeup += rb->aux_watermark; + handle->wakeup = rb->aux_wakeup + rb->aux_watermark; } - handle->head = aux_head; + handle->head = rb->aux_head; handle->size -= size; return 0; From patchwork Wed Aug 16 16:18:17 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 110259 Delivered-To: patch@linaro.org Received: by 10.140.95.78 with SMTP id h72csp995262qge; Wed, 16 Aug 2017 09:18:27 -0700 (PDT) X-Received: by 10.101.77.76 with SMTP id j12mr2051824pgt.326.1502900307035; Wed, 16 Aug 2017 09:18:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1502900307; cv=none; d=google.com; s=arc-20160816; b=ObacL9zUe4PbRKDVRb5WqkwIpWL8rxdYWkEARzhkDNnbGqi0Ha6Qs/pJuNPGNQBd37 kt0dtCYQcLbwoyB8mBowhnoE40N/NrlYtFCmjZKeoWbunKFP5k9hJWkQsf8KHGouN+19 7hUx62vCOntRitCHCY3u71XzzCrYzidTFz6XozFAEwxhLLZiWegpJrIITcXEbeaJ9YwK YvkyQ8vsnnrq0esl8rO/cddqccoZnQwvbE41+BEHQLtDOswEcrw4quxe6GBqsuVsOxAb 73Z3jq1DOyEgpojKovRXLHTmifjysa166MmQ68lQWrKOWxypEc1rwHMlDZr6XwzM4H1y 4yPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=NR8BXwZC7/DvU3koipIwOJKWcheZazljAUtAR/borpw=; b=YzEY3cnhf2Z9rnBOcYg8Bj+r7oUKYY30sOjZEpi6JiFhWXWO8qfVVMeB32SUcAGaIt CRJ+y+HuZKDgrJTqBWQAe0I+Qn5GTenti27MOUOSKfwxpsRYNIya/wIgd+Uhq1ZSdH43 4MkZ3+HBtn29fc248VYc2+t7fwwz0h1GysnNFxE7QafrytW6A/nMYw4Rh7r3e8RYfpJy orzUkH9xTBRyO47DLUYOc8DXD0Ol6rmtf8fTctw+6ymJLYvIaICWeu7WGEdOwv7QIP9x j2nna0VfsY/4HN6exmwezP1oIoJL4HcTNvKWlpr2a8QBkCLHUz4p1lYI+6FmY6AjDgrX a2gw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id o65si702222pfo.562.2017.08.16.09.18.26; Wed, 16 Aug 2017 09:18:27 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752413AbdHPQSY (ORCPT + 26 others); Wed, 16 Aug 2017 12:18:24 -0400 Received: from foss.arm.com ([217.140.101.70]:37534 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751517AbdHPQSW (ORCPT ); Wed, 16 Aug 2017 12:18:22 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EE8DE13D5; Wed, 16 Aug 2017 09:18:21 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C06563F59E; Wed, 16 Aug 2017 09:18:21 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id CAFC21AE0928; Wed, 16 Aug 2017 17:18:21 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, Will Deacon , Peter Zijlstra Subject: [PATCH v2 2/2] perf/aux: Ensure aux_wakeup represents most recent wakeup index Date: Wed, 16 Aug 2017 17:18:17 +0100 Message-Id: <1502900297-21839-2-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1502900297-21839-1-git-send-email-will.deacon@arm.com> References: <1502900297-21839-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The aux_watermark member of struct ring_buffer represents the period (in terms of bytes) at which wakeup events should be generated when data is written to the aux buffer in non-snapshot mode. On hardware that cannot generate an interrupt when the aux_head reaches an arbitrary wakeup index (such as ARM SPE), the aux_head sampled from handle->head in perf_aux_output_{skip,end} may in fact be past the wakeup index. This can lead to wakeup slowly falling behind the head. For example, consider the case where hardware can only generate an interrupt on a page-boundary and the aux buffer is initialised as follows: // Buffer size is 2 * PAGE_SIZE rb->aux_head = rb->aux_wakeup = 0 rb->aux_watermark = PAGE_SIZE / 2 following the first perf_aux_output_begin call, the handle is initialised with: handle->head = 0 handle->size = 2 * PAGE_SIZE handle->wakeup = PAGE_SIZE / 2 and the hardware will be programmed to generate an interrupt at PAGE_SIZE. When the interrupt is raised, the hardware head will be at PAGE_SIZE, so calling perf_aux_output_end(handle, PAGE_SIZE) puts the ring buffer into the following state: rb->aux_head = PAGE_SIZE rb->aux_wakeup = PAGE_SIZE / 2 rb->aux_watermark = PAGE_SIZE / 2 and then the next call to perf_aux_output_begin will result in: handle->head = handle->wakeup = PAGE_SIZE for which the semantics are unclear and, for a smaller aux_watermark (e.g. PAGE_SIZE / 4), then the wakeup would in fact be behind head at this point. This patch fixes the problem by rounding down the aux_head (as sampled from the handle) to the nearest aux_watermark boundary when updating rb->aux_wakeup, therefore taking into account any overruns by the hardware. Cc: Peter Zijlstra Acked-by: Alexander Shishkin Reported-by: Mark Rutland Signed-off-by: Will Deacon --- kernel/events/internal.h | 2 +- kernel/events/ring_buffer.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) -- 2.1.4 diff --git a/kernel/events/internal.h b/kernel/events/internal.h index 2941b868353c..5377c591c57a 100644 --- a/kernel/events/internal.h +++ b/kernel/events/internal.h @@ -40,7 +40,7 @@ struct ring_buffer { /* AUX area */ long aux_head; local_t aux_nest; - long aux_wakeup; + long aux_wakeup; /* last aux_watermark boundary crossed by aux_head */ unsigned long aux_pgoff; int aux_nr_pages; int aux_overwrite; diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c index 87c28faca8e4..dcb2e8c4ad54 100644 --- a/kernel/events/ring_buffer.c +++ b/kernel/events/ring_buffer.c @@ -454,7 +454,7 @@ void perf_aux_output_end(struct perf_output_handle *handle, unsigned long size) rb->user_page->aux_head = rb->aux_head; if (rb->aux_head - rb->aux_wakeup >= rb->aux_watermark) { wakeup = true; - rb->aux_wakeup += rb->aux_watermark; + rb->aux_wakeup = rounddown(rb->aux_head, rb->aux_watermark); } if (wakeup) { @@ -488,7 +488,7 @@ int perf_aux_output_skip(struct perf_output_handle *handle, unsigned long size) rb->user_page->aux_head = rb->aux_head; if (rb->aux_head - rb->aux_wakeup >= rb->aux_watermark) { perf_output_wakeup(handle); - rb->aux_wakeup += rb->aux_watermark; + rb->aux_wakeup = rounddown(rb->aux_head, rb->aux_watermark); handle->wakeup = rb->aux_wakeup + rb->aux_watermark; }