From patchwork Fri Jul 28 16:54:22 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 108895 Delivered-To: patch@linaro.org Received: by 10.182.45.195 with SMTP id p3csp676933obm; Fri, 28 Jul 2017 09:54:22 -0700 (PDT) X-Received: by 10.98.27.206 with SMTP id b197mr8032286pfb.149.1501260862865; Fri, 28 Jul 2017 09:54:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501260862; cv=none; d=google.com; s=arc-20160816; b=JjSgopdWNU/XwpIxgeW0qRrfPSJxTAixAeEooNQ85XB6YzFovTpSE1D57gNmJywVJ1 Pms7hlyWPoIpyD+lHTw8XWsGvvU1w9o0KBeytFJrY4yqdnh1QO9GENR6ZtUaI5rDCpIM gSFakhsGgsukM0fjGFgR80QBgJf5dvdvmOeOjD8DKD05FZiDR64ZB+6mvVDofaTIZvNx pde7Q87BMeh6IYZ4TsHcXVoDwHGAboUUJrsh29W1dA912V/gMtdxmD0ElKTG85/mMv7V 0Bvnm3pS+v8hJ5NBUHN/5yzIVfbIF8UHdE/BAlVz8HVZIyjxYMUmZjkGICrCkbUVabbI CreQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :arc-authentication-results; bh=MEj5XPxm8Pn4MdeXPV6WtfuYzdemrdqMwcYPLj3Gpnc=; b=V6oW8X7dr+fz0sF/mMgszeNJH5Zj05/M9SUICmNTx5iCSA77rpn77XSJtM++2t4zd0 9CtGme1kkjr31+4ywEdNMlf4YdONS6ARi+acqx7dDpDxWVdlRr2j7KBZWg+3VFzEMnfg Q0ENRxEOnP+UWMMkBGmLpUsUICS74NxaYzggwXL+hKgm0TXS+bYfagbnoNkuudZtsxaa qeMb0qCD2dVZw5Ql/XAb7izy5Fc5VVrnPGdfCxqcF8Z+7iuKjiGUt6+oztN/VlgGJSte KE7S+AyXkGVF3qKMSpO20IQ/i7BMSIxvVW4+rkp1vbEZE/M54kVRRGXnFNuUvbBHa2/V u7ig== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id p127si8095121pfg.452.2017.07.28.09.54.22; Fri, 28 Jul 2017 09:54:22 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752565AbdG1QyT (ORCPT + 26 others); Fri, 28 Jul 2017 12:54:19 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:33328 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752235AbdG1QyR (ORCPT ); Fri, 28 Jul 2017 12:54:17 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DA5D215AD; Fri, 28 Jul 2017 09:54:16 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A58E43F483; Fri, 28 Jul 2017 09:54:16 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id D0C651AE30C0; Fri, 28 Jul 2017 17:54:24 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org Cc: mark.rutland@arm.com, linux-arm-kernel@lists.infradead.org, Will Deacon , Alexander Shishkin , Peter Zijlstra Subject: [PATCH 1/2] perf/aux: Make aux_{head, wakeup} ring_buffer members long Date: Fri, 28 Jul 2017 17:54:22 +0100 Message-Id: <1501260863-14687-1-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The aux_head and aux_wakeup members of struct ring_buffer are defined using the local_t type, despite the fact that they are only accessed via the perf_aux_output_* functions, which cannot race with each other for a given ring buffer. This patch changes the type of the members to long, so we can avoid using the local_* API where it isn't needed. Cc: Mark Rutland Cc: Alexander Shishkin Cc: Peter Zijlstra Signed-off-by: Will Deacon --- kernel/events/internal.h | 4 ++-- kernel/events/ring_buffer.c | 27 +++++++++++++-------------- 2 files changed, 15 insertions(+), 16 deletions(-) -- 2.1.4 diff --git a/kernel/events/internal.h b/kernel/events/internal.h index 486fd78eb8d5..2941b868353c 100644 --- a/kernel/events/internal.h +++ b/kernel/events/internal.h @@ -38,9 +38,9 @@ struct ring_buffer { struct user_struct *mmap_user; /* AUX area */ - local_t aux_head; + long aux_head; local_t aux_nest; - local_t aux_wakeup; + long aux_wakeup; unsigned long aux_pgoff; int aux_nr_pages; int aux_overwrite; diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c index 14a240ff439d..330df5a7f762 100644 --- a/kernel/events/ring_buffer.c +++ b/kernel/events/ring_buffer.c @@ -367,7 +367,7 @@ void *perf_aux_output_begin(struct perf_output_handle *handle, if (WARN_ON_ONCE(local_xchg(&rb->aux_nest, 1))) goto err_put; - aux_head = local_read(&rb->aux_head); + aux_head = rb->aux_head; handle->rb = rb; handle->event = event; @@ -382,7 +382,7 @@ void *perf_aux_output_begin(struct perf_output_handle *handle, */ if (!rb->aux_overwrite) { aux_tail = ACCESS_ONCE(rb->user_page->aux_tail); - handle->wakeup = local_read(&rb->aux_wakeup) + rb->aux_watermark; + handle->wakeup = rb->aux_wakeup + rb->aux_watermark; if (aux_head - aux_tail < perf_aux_size(rb)) handle->size = CIRC_SPACE(aux_head, aux_tail, perf_aux_size(rb)); @@ -434,12 +434,12 @@ void perf_aux_output_end(struct perf_output_handle *handle, unsigned long size) handle->aux_flags |= PERF_AUX_FLAG_OVERWRITE; aux_head = handle->head; - local_set(&rb->aux_head, aux_head); + rb->aux_head = aux_head; } else { handle->aux_flags &= ~PERF_AUX_FLAG_OVERWRITE; - aux_head = local_read(&rb->aux_head); - local_add(size, &rb->aux_head); + aux_head = rb->aux_head; + rb->aux_head += size; } if (size || handle->aux_flags) { @@ -451,11 +451,11 @@ void perf_aux_output_end(struct perf_output_handle *handle, unsigned long size) handle->aux_flags); } - aux_head = rb->user_page->aux_head = local_read(&rb->aux_head); + aux_head = rb->user_page->aux_head = rb->aux_head; - if (aux_head - local_read(&rb->aux_wakeup) >= rb->aux_watermark) { + if (aux_head - rb->aux_wakeup >= rb->aux_watermark) { wakeup = true; - local_add(rb->aux_watermark, &rb->aux_wakeup); + rb->aux_wakeup += rb->aux_watermark; } if (wakeup) { @@ -485,14 +485,13 @@ int perf_aux_output_skip(struct perf_output_handle *handle, unsigned long size) if (size > handle->size) return -ENOSPC; - local_add(size, &rb->aux_head); + rb->aux_head += size; - aux_head = rb->user_page->aux_head = local_read(&rb->aux_head); - if (aux_head - local_read(&rb->aux_wakeup) >= rb->aux_watermark) { + aux_head = rb->user_page->aux_head = rb->aux_head; + if (aux_head - rb->aux_wakeup >= rb->aux_watermark) { perf_output_wakeup(handle); - local_add(rb->aux_watermark, &rb->aux_wakeup); - handle->wakeup = local_read(&rb->aux_wakeup) + - rb->aux_watermark; + rb->aux_wakeup += rb->aux_watermark; + handle->wakeup = rb->aux_wakeup + rb->aux_watermark; } handle->head = aux_head; From patchwork Fri Jul 28 16:54:23 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 108894 Delivered-To: patch@linaro.org Received: by 10.182.45.195 with SMTP id p3csp676904obm; Fri, 28 Jul 2017 09:54:21 -0700 (PDT) X-Received: by 10.84.236.70 with SMTP id h6mr8442183pln.348.1501260861075; Fri, 28 Jul 2017 09:54:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501260861; cv=none; d=google.com; s=arc-20160816; b=eaC527TwyLZQyV6xgW/84ocyYkHUi9RRlH7fYsjlv32p2A+Toz/gROysgZyPlFYhtt NYEC66fjl0q+LZIJxDC5Uq7nXlEBk4472xF1ZbqbI27IXtlbBKyYm+/2oeVPFPhm7ymv bH/od9VuVhd2/E9sVzzLobo3LiRbUtk+7G0zdmbOp/Z3jksKDR0W/gFyMh1Sfc3WOv22 L2TeMr/SSKtG3kYEbmY+5W5mWCtiFTInHwn3gVXRqXKf/Zs+mAXABOlzNjjc1+AdkW81 FKFF39XnrdxMwcbqYG3rzzRnJ0M/F7OoT3+KRqTxdmw5RJWa61NfcAdZNRaLU9gaOjSN B24g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=Gu2QZU/9mF5Ze/utuIHVdsDhQOhkPmDfK98jk3XJpuo=; b=tOHHOhPVix6fIdNs4csShyCwwR9x4QtjA58UatOaDV2gOD7qetrjpLjUGYzVaymreB HDqON4SH6ER9RGaidkTWexzxrWqwvP8R4nxnuXWJqxMJt24kx1wqaTcsQVwNTXr0qUFv grJbtreeV7y5D3IJfgK7SaUTmojVEPspm9YlIJpDyTiFFklPnKb9ff4pBOkgUIkW9JHY GuqttZZmx07bxNgF0SaxbJmQTICRNqfWt6ic5NfXuvo/ouejYC9gTq5jhQWBMmhkzDqH rArEcXiSbNnnDE9yNshq5+NLHn68v1JZ4PE5NHug/aqZb8dRCm8BOKv2yv92q/vzm4Ke UWkA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 39si4061342plc.386.2017.07.28.09.54.20; Fri, 28 Jul 2017 09:54:21 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752544AbdG1QyT (ORCPT + 26 others); Fri, 28 Jul 2017 12:54:19 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:33318 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751882AbdG1QyR (ORCPT ); Fri, 28 Jul 2017 12:54:17 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E446D15BE; Fri, 28 Jul 2017 09:54:16 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B206A3F59D; Fri, 28 Jul 2017 09:54:16 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id D8A501AE06E0; Fri, 28 Jul 2017 17:54:24 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org Cc: mark.rutland@arm.com, linux-arm-kernel@lists.infradead.org, Will Deacon , Alexander Shishkin , Peter Zijlstra Subject: [PATCH 2/2] perf/aux: Ensure aux_wakeup represents most recent wakeup index Date: Fri, 28 Jul 2017 17:54:23 +0100 Message-Id: <1501260863-14687-2-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1501260863-14687-1-git-send-email-will.deacon@arm.com> References: <1501260863-14687-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The aux_watermark member of struct ring_buffer represents the period (in terms of bytes) at which wakeup events should be generated when data is written to the aux buffer in non-snapshot mode. On hardware that cannot generate an interrupt when the aux_head reaches an arbitrary wakeup index (such as ARM SPE), the aux_head sampled from handle->head in perf_aux_output_{skip,end} may in fact be past the wakeup index. This can lead to wakeup slowly falling behind the head. For example, consider the case where hardware can only generate an interrupt on a page-boundary and the aux buffer is initialised as follows: // Buffer size is 2 * PAGE_SIZE rb->aux_head = rb->aux_wakeup = 0 rb->aux_watermark = PAGE_SIZE / 2 following the first perf_aux_output_begin call, the handle is initialised with: handle->head = 0 handle->size = 2 * PAGE_SIZE handle->wakeup = PAGE_SIZE / 2 and the hardware will be programmed to generate an interrupt at PAGE_SIZE. When the interrupt is raised, the hardware head will be at PAGE_SIZE, so calling perf_aux_output_end(handle, PAGE_SIZE) puts the ring buffer into the following state: rb->aux_head = PAGE_SIZE rb->aux_wakeup = PAGE_SIZE / 2 rb->aux_watermark = PAGE_SIZE / 2 and then the next call to perf_aux_output_begin will result in: handle->head = handle->wakeup = PAGE_SIZE for which the semantics are unclear and, for a smaller aux_watermark (e.g. PAGE_SIZE / 4), then the wakeup would in fact be behind head at this point. This patch fixes the problem by rounding down the aux_head (as sampled from the handle) to the nearest aux_watermark boundary when updating rb->aux_wakeup, therefore taking into account any overruns by the hardware. Cc: Alexander Shishkin Cc: Peter Zijlstra Reported-by: Mark Rutland Signed-off-by: Will Deacon --- kernel/events/ring_buffer.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- 2.1.4 Acked-by: Alexander Shishkin diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c index 330df5a7f762..8e511e52fc1b 100644 --- a/kernel/events/ring_buffer.c +++ b/kernel/events/ring_buffer.c @@ -455,7 +455,7 @@ void perf_aux_output_end(struct perf_output_handle *handle, unsigned long size) if (aux_head - rb->aux_wakeup >= rb->aux_watermark) { wakeup = true; - rb->aux_wakeup += rb->aux_watermark; + rb->aux_wakeup = rounddown(aux_head, rb->aux_watermark); } if (wakeup) { @@ -490,7 +490,7 @@ int perf_aux_output_skip(struct perf_output_handle *handle, unsigned long size) aux_head = rb->user_page->aux_head = rb->aux_head; if (aux_head - rb->aux_wakeup >= rb->aux_watermark) { perf_output_wakeup(handle); - rb->aux_wakeup += rb->aux_watermark; + rb->aux_wakeup = rounddown(aux_head, rb->aux_watermark); handle->wakeup = rb->aux_wakeup + rb->aux_watermark; }