From patchwork Fri Apr 22 10:15:46 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 66420 Delivered-To: patch@linaro.org Received: by 10.140.93.198 with SMTP id d64csp630662qge; Fri, 22 Apr 2016 03:16:19 -0700 (PDT) X-Received: by 10.55.155.22 with SMTP id d22mr7509075qke.26.1461320179876; Fri, 22 Apr 2016 03:16:19 -0700 (PDT) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id 16si3326929qgg.44.2016.04.22.03.16.19 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 22 Apr 2016 03:16:19 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:58552 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atY8J-0004Lv-Eb for patch@linaro.org; Fri, 22 Apr 2016 06:16:19 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42934) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atY7i-0003YR-Up for qemu-devel@nongnu.org; Fri, 22 Apr 2016 06:15:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1atY7e-0003Eo-NY for qemu-devel@nongnu.org; Fri, 22 Apr 2016 06:15:42 -0400 Received: from mail-wm0-x22b.google.com ([2a00:1450:400c:c09::22b]:38737) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atY7e-0003BW-0U for qemu-devel@nongnu.org; Fri, 22 Apr 2016 06:15:38 -0400 Received: by mail-wm0-x22b.google.com with SMTP id u206so19259339wme.1 for ; Fri, 22 Apr 2016 03:15:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=z1NCGCi2QKYjbdl43eVBsXvpzHNpjcVZhrdWnPIFamE=; b=d+Ti+IQz0dTTq0YPcGSmkCh2O47pJSzmLTNrx4yGkR+znna+5I9d1/lSGxaLWWieel N/VjGuRO/V3mPMgVcECpMWjGRX+etV4k7IxPU7T+e56ufCs10xE1f+qAu+lYUs7qkuCT s7TEe4JPjQdQqlU0L7/mjJlBoPZ2q6EInu4BM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=z1NCGCi2QKYjbdl43eVBsXvpzHNpjcVZhrdWnPIFamE=; b=H/1g/DscA2IyPhIQCccOgKOXyM8icoK+gWZ7gJkJUHCwG/JSBy2mBScZllTOS1xRDb E2GIL+VvPOMo2EmSMQ+zalkQhPYV/I52g5qE04JHERsLc73NB+T4dxHgsuJ5XgV20DE4 Udwtof7cdHA1NQRGxF3tCOQX0wc2rqkt1mAfedgR+O/UbCKaB4MRxh4nFXqU7Ftb3sow NGrHfHE1azPeRKCGlRmxqV9VnXFT4G98UeSns1Plm/ArWRtgUrS5lKOlAA1bVi3EaMdp zXmm9TH0KoGkEOBu5Zt3JRDo5pBcWXY5wj9bsYxy2CL+MLnmw38cuhmxYAq42QeISNX1 N2Dw== X-Gm-Message-State: AOPr4FVvCqL60ysE2jyBQkbBn/wbmnOd88b9NNKW/62hj8XJ4Z+7arRVs3tAfvGbcybArdJh X-Received: by 10.194.95.198 with SMTP id dm6mr20276534wjb.136.1461320137364; Fri, 22 Apr 2016 03:15:37 -0700 (PDT) Received: from localhost ([94.18.191.146]) by smtp.gmail.com with ESMTPSA id iv1sm7332929wjb.34.2016.04.22.03.15.36 (version=TLS1_2 cipher=AES128-SHA bits=128/128); Fri, 22 Apr 2016 03:15:36 -0700 (PDT) Date: Fri, 22 Apr 2016 12:15:46 +0200 From: Christoffer Dall To: Alexander Graf Message-ID: <20160422101546.GB30824@cbox> References: <20160421162348.GA24178@cbox> <57192EED.2040501@suse.de> <20160422100118.GD25288@cbox> <5719F7BC.4090703@suse.de> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <5719F7BC.4090703@suse.de> User-Agent: Mutt/1.5.21 (2010-09-15) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::22b Subject: Re: [Qemu-devel] Performance regression using KVM/ARM X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marc Zyngier , Peter Maydell , Paolo Bonzini , qemu-devel@nongnu.org, "Michael S. Tsirkin" Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" On Fri, Apr 22, 2016 at 12:06:52PM +0200, Alexander Graf wrote: > On 04/22/2016 12:01 PM, Christoffer Dall wrote: > >On Thu, Apr 21, 2016 at 09:50:05PM +0200, Alexander Graf wrote: > >> > >>On 21.04.16 18:23, Christoffer Dall wrote: > >>>Hi, > >>> > >>>Commit 9fac18f (oslib: allocate PROT_NONE pages on top of RAM, > >>>2015-09-10) had the unfortunate side effect that memory slots registered > >>>with KVM no longer contain a userspace address that is aligned to a 2M > >>>boundary, causing the use of THP to fail in the kernel. > >>> > >>>I fail to see where in the QEMU code we should be asking for a 2M > >>>alignment of our memory region. Can someone help pointing me to the > >>>right place to fix this or suggest a patch? > >>> > >>>This causes a performance regssion of hackbench on KVM/ARM of about 62% > >>>compared to the workload running with THP. > >>> > >>>We have verified that this is indeed the cause of the failure by adding > >>>various prints to QEMU and the kernel, but unfortunatley my QEMU > >>>knowledge is not sufficient for me to fix it myself. > >>> > >>>Any help would be much appreciated! > >>The code changed quite heavily since I last looked at it, but could you > >>please try whether the (untested) patch below makes a difference? > >> > >> > >Unfortunately this doesn't make any difference. It feels to me like > >we're missing specifying a 2M alignemnt in QEMU somewhere, but I can't > >properly understand the links between the actual allocation, registering > >mem slots with the KVM part of QEMU, and actually setting up KVM user > >memory regions. > > > >What has to happen is that the resulting struct > >kvm_userspace_memory_region() has the same alignment offset from 2M (the > >huge page size) of the ->guest_phys_addr and ->userspace-addr fields. > > Well, I would expect that the guest address space is always very big > aligned - and definitely at least 2MB - so we're safe there. > > That means we only need to align the qemu virtual address. There > used to be a memalign() call for that, but it got replaced with > direct mmap() and then a lot of code changed on top. Looking at the > logs, I'm sure Paolo knows the answer though :) > Peter just pointed me to a change I remember doing for ARM, so perhaps this fix is the right one? Thanks, -Christoffer diff --git a/util/oslib-posix.c b/util/oslib-posix.c index d25f671..a36e734 100644 --- a/util/oslib-posix.c +++ b/util/oslib-posix.c @@ -35,7 +35,7 @@ extern int daemon(int, int); #endif -#if defined(__linux__) && (defined(__x86_64__) || defined(__arm__)) +#if defined(__linux__) && (defined(__x86_64__) || defined(__arm__)) || defined(__aarch64__) /* Use 2 MiB alignment so transparent hugepages can be used by KVM. Valgrind does not support alignments larger than 1 MiB, therefore we need special code which handles running on Valgrind. */