From patchwork Wed May 7 11:47:43 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Stabellini X-Patchwork-Id: 29758 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-pd0-f199.google.com (mail-pd0-f199.google.com [209.85.192.199]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 6FA0720534 for ; Wed, 7 May 2014 11:49:48 +0000 (UTC) Received: by mail-pd0-f199.google.com with SMTP id r10sf3593436pdi.10 for ; Wed, 07 May 2014 04:49:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:date:from:to:subject:in-reply-to :message-id:references:user-agent:mime-version:cc:precedence:list-id :list-unsubscribe:list-archive:list-post:list-help:list-subscribe :sender:errors-to:x-original-sender :x-original-authentication-results:mailing-list:content-type :content-transfer-encoding; bh=UP3vKXYf1SCKlpEhQmbf4D+CUdAcSOyEsfDU6zlsEXo=; b=ezQOTwi7FIyOgBRf0PNNxU5c3VeMrASUaI2LICfWcHuv7gVvBpYt7Zv99dFZFhw+FD JQpMw1iCf1sXOyT3ZcoWk/4dKcVbJlxHcgr1cTuEbuA6n5UcJ18mS6N+2WaCbAh9t5EC Xe3Ncr6U7/mO4bk8MzJpQk28JsSVinJT2R/RTnL2GhXJx58PAkFu0ASoA3CLM8N5cgIJ FvfoVs8tRyWWeiRHd8w9fKc1TC6rhlVeGa7UCjmHvXeJlOBI3HN7hFIWjaTWqJVLfSIS yrCEhl+xmeH2u0V0kp3/gAuf3OzbynabHTcTIqpECDzLJ8f8PSNn23YoJiNowXUI0Shd QLWQ== X-Gm-Message-State: ALoCoQms9ITAax0iPFybQzYZ431p/RVp+RBKUJK+ItIKHVqSdWwBjr9L3gLLMn/5/eT8XO1/SyRB X-Received: by 10.67.4.170 with SMTP id cf10mr4068887pad.31.1399463387492; Wed, 07 May 2014 04:49:47 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.96.72 with SMTP id j66ls1614671qge.99.gmail; Wed, 07 May 2014 04:49:47 -0700 (PDT) X-Received: by 10.52.169.67 with SMTP id ac3mr92210vdc.92.1399463387349; Wed, 07 May 2014 04:49:47 -0700 (PDT) Received: from mail-vc0-f175.google.com (mail-vc0-f175.google.com [209.85.220.175]) by mx.google.com with ESMTPS id yq2si2849244vcb.199.2014.05.07.04.49.47 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 07 May 2014 04:49:47 -0700 (PDT) Received-SPF: none (google.com: patch+caf_=patchwork-forward=linaro.org@linaro.org does not designate permitted sender hosts) client-ip=209.85.220.175; Received: by mail-vc0-f175.google.com with SMTP id lh4so1078564vcb.34 for ; Wed, 07 May 2014 04:49:47 -0700 (PDT) X-Received: by 10.53.1.69 with SMTP id be5mr11036410vdd.27.1399463387242; Wed, 07 May 2014 04:49:47 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.221.72 with SMTP id ib8csp301699vcb; Wed, 7 May 2014 04:49:46 -0700 (PDT) X-Received: by 10.229.79.2 with SMTP id n2mr34142686qck.11.1399463386647; Wed, 07 May 2014 04:49:46 -0700 (PDT) Received: from bombadil.infradead.org (bombadil.infradead.org. [2001:1868:205::9]) by mx.google.com with ESMTPS id v10si6556322qat.231.2014.05.07.04.49.46 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 07 May 2014 04:49:46 -0700 (PDT) Received-SPF: none (google.com: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org does not designate permitted sender hosts) client-ip=2001:1868:205::9; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1Wi0Ks-0002gx-NL; Wed, 07 May 2014 11:48:30 +0000 Received: from smtp.citrix.com ([66.165.176.89]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1Wi0Kp-0002bk-AU for linux-arm-kernel@lists.infradead.org; Wed, 07 May 2014 11:48:28 +0000 X-IronPort-AV: E=Sophos; i="4.97,1002,1389744000"; d="scan'208"; a="128567716" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 07 May 2014 11:48:06 +0000 Received: from ukmail1.uk.xensource.com (10.80.16.128) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.3.181.6; Wed, 7 May 2014 07:48:05 -0400 Received: from kaball.uk.xensource.com ([10.80.2.59]) by ukmail1.uk.xensource.com with esmtp (Exim 4.69) (envelope-from ) id 1Wi0KS-000175-Mo; Wed, 07 May 2014 12:48:04 +0100 Date: Wed, 7 May 2014 12:47:43 +0100 From: Stefano Stabellini X-X-Sender: sstabellini@kaball.uk.xensource.com To: David Vrabel Subject: Re: [Xen-devel] [PATCH] Xen: hibernation is x86-only at the moment In-Reply-To: <536A1198.8030406@citrix.com> Message-ID: References: <5165529.jneGVBC5b5@wuerfel> <5360C1A7.8060401@citrix.com> <536905B5.6040900@citrix.com> <536A1198.8030406@citrix.com> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 X-DLP: MIA2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20140507_044827_939673_85DBB902 X-CRM114-Status: GOOD ( 37.44 ) X-Spam-Score: -5.7 (-----) X-Spam-Report: SpamAssassin version 3.3.2 on bombadil.infradead.org summary: Content analysis details: (-5.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high trust [66.165.176.89 listed in list.dnswl.org] -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.7 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -0.0 SPF_PASS SPF: sender matches SPF record Cc: Ian Campbell , Arnd Bergmann , Stefano Stabellini , Russ Dill , Sebastian Capella , Pavel Machek , xen-devel@lists.xenproject.org, Russell King , linux-arm-kernel@lists.infradead.org X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: stefano.stabellini@eu.citrix.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: patch+caf_=patchwork-forward=linaro.org@linaro.org does not designate permitted sender hosts) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 On Wed, 7 May 2014, David Vrabel wrote: > On 07/05/14 11:45, Stefano Stabellini wrote: > > On Tue, 6 May 2014, David Vrabel wrote: > >> On 06/05/14 14:35, Stefano Stabellini wrote: > >>> On Wed, 30 Apr 2014, David Vrabel wrote: > >>>> On 29/04/14 21:00, Arnd Bergmann wrote: > >>>>> In commit 603fb42a66499ab "ARM: 8011/1: ARM hibernation / suspend-to-disk", > >>>>> currently in linux-next, the ARM architecture gains support for > >>>>> hibernation (suspend-to-disk). Xen supports this in principle, but only > >>>>> has an architecture specific hypercall defined on x86, which leads > >>>>> to a build error when both hibernation and Xen support are enabled: > >>>>> > >>>>> drivers/xen/manage.c:105:2: error: implicit declaration of function 'HYPERVISOR_suspend' [-Werror=implicit-function-declaration] > >>>>> si->cancelled = HYPERVISOR_suspend(si->arg); > >>>>> > >>>>> It is probably a good idea to define this hypercall on ARM as well > >>>>> and provide an implementation in the host, but until that is done, > >>>>> this patch helps disable the broken code in the Xen guest by making > >>>>> it depend on CONFIG_X86. > >>> > >>> Thanks Arnd. > >>> > >>> > >>>> Since this isn't a regression in 3.15-rcX can you provide the > >>>> appropriate hypercall on ARM. This does not require that there is an > >>>> implementation on Xen. > >>>> > >>>> When the Xen implementation exists it will be easier to use if the > >>>> support already exists in Linux. > >>> > >>> > >>> The sched_on hypercall is already implemented on ARM. > >>> However SCHEDOP_shutdown, defined as it is now, is unusable on ARM: > >>> > >>> /* > >>> * Halt execution of this domain (all VCPUs) and notify the system controller. > >>> * @arg == pointer to sched_shutdown_t structure. > >>> * > >>> * If the sched_shutdown_t reason is SHUTDOWN_suspend then this > >>> * hypercall takes an additional extra argument which should be the > >>> * MFN of the guest's start_info_t. > >>> * > >>> * In addition, which reason is SHUTDOWN_suspend this hypercall > >>> * returns 1 if suspend was cancelled or the domain was merely > >>> * checkpointed, and 0 if it is resuming in a new domain. > >>> */ > >>> #define SCHEDOP_shutdown 2 > >>> > >>> > >>> We don't have a start_info, and even if we had, we wouldn't know the > >>> MFN. > >>> I think we should make it available on ARM only if we change the > >>> interface making the third argument x86 only. Of course that would > >>> impact the linux side implementation too. > >> > >> We do not want a different number of arguments to this hypercall. Just > >> pass 0 for the MFN. This hypercall docs should be updated to say this. > >> > >> This doesn't seem any different to x86 HVM where the start_info_mfn > >> parameter is also not relevant. > >> > >> if (xen_hvm_domain()) { > >> si.arg = 0UL; > >> si.pre = NULL; > >> si.post = &xen_hvm_post_suspend; > >> } else { > >> si.arg = virt_to_mfn(xen_start_info); > >> si.pre = &xen_pre_suspend; > >> si.post = &xen_post_suspend; > >> } > >> > >> Changing this to be !xen_pv_domain() would then do the right thing for arm. > > > > xen_hvm_domain returns true on arm so that's not needed. > > Unfortunately the patch turns out a bit ugly because of the way > > hypercalls are implemented in assembly on arm and arm64. > > Also I needed to provide quite a few empty stubs for other xen arch > > specific functions. > > Thanks. If you want to add a commit message and apply to > devel/for-linus-3.16: > > Reviewed-by: David Vrabel > > > --- a/arch/arm/include/asm/xen/hypercall.h > > +++ b/arch/arm/include/asm/xen/hypercall.h > > @@ -34,6 +34,7 @@ > > #define _ASM_ARM_XEN_HYPERCALL_H > > > > #include > > +#include > > > > long privcmd_call(unsigned call, unsigned long a1, > > unsigned long a2, unsigned long a3, > > @@ -48,6 +49,16 @@ int HYPERVISOR_memory_op(unsigned int cmd, void *arg); > > int HYPERVISOR_physdev_op(int cmd, void *arg); > > int HYPERVISOR_vcpu_op(int cmd, int vcpuid, void *extra_args); > > int HYPERVISOR_tmem_op(void *arg); > > +int HYPERVISOR_sched_op_shutdown(int cmd, void *arg, unsigned long unused); > > + > > +static inline int > > +HYPERVISOR_suspend(unsigned long start_info_mfn) > > +{ > > + struct sched_shutdown r = { .reason = SHUTDOWN_suspend }; > > + > > + /* start_info_mfn is unused on ARM, pass 0 instead */ > > + return HYPERVISOR_sched_op_shutdown(SCHEDOP_shutdown, &r, 0); > > +} > > > > static inline void > > MULTI_update_va_mapping(struct multicall_entry *mcl, unsigned long va, > > diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c > > index b96723e..6d88c18 100644 > > --- a/arch/arm/xen/enlighten.c > > +++ b/arch/arm/xen/enlighten.c > > @@ -339,6 +339,17 @@ static int __init xen_pm_init(void) > > } > > late_initcall(xen_pm_init); > > > > + > > +/* empty stubs */ > > +void xen_arch_pre_suspend(void) { } > > +void xen_arch_post_suspend(int suspend_cancelled) { } > > +void xen_arch_hvm_post_suspend(int suspend_cancelled) { } > > +void xen_mm_pin_all(void) { } > > +void xen_mm_unpin_all(void) { } > > I wonder if the calls to xen_mm_pin_all() and xen_mm_unpin_all() could > be moved into the arch_pre_suspend() and arch_post_suspend() calls > instead. So there wouldn't need to be stubs provided for arm. This is how it looks, including the arm64 change: --- arm,arm64/xen: introduce HYPERVISOR_suspend Introduce HYPERVISOR_suspend: it is a call to sched_op with an additional argument that is always 0 on arm and arm64. Introduce a few additional empty stubs for Xen arch specific functions called by drivers/xen/manage.c. Signed-off-by: Stefano Stabellini diff --git a/arch/arm/include/asm/xen/hypercall.h b/arch/arm/include/asm/xen/hypercall.h index 7658150..b5827e6 100644 --- a/arch/arm/include/asm/xen/hypercall.h +++ b/arch/arm/include/asm/xen/hypercall.h @@ -34,6 +34,7 @@ #define _ASM_ARM_XEN_HYPERCALL_H #include +#include long privcmd_call(unsigned call, unsigned long a1, unsigned long a2, unsigned long a3, @@ -49,6 +50,16 @@ int HYPERVISOR_physdev_op(int cmd, void *arg); int HYPERVISOR_vcpu_op(int cmd, int vcpuid, void *extra_args); int HYPERVISOR_tmem_op(void *arg); int HYPERVISOR_multicall(struct multicall_entry *calls, uint32_t nr); +int HYPERVISOR_sched_op_shutdown(int cmd, void *arg, unsigned long unused); + +static inline int +HYPERVISOR_suspend(unsigned long start_info_mfn) +{ + struct sched_shutdown r = { .reason = SHUTDOWN_suspend }; + + /* start_info_mfn is unused on ARM, pass 0 instead */ + return HYPERVISOR_sched_op_shutdown(SCHEDOP_shutdown, &r, 0); +} static inline void MULTI_update_va_mapping(struct multicall_entry *mcl, unsigned long va, diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c index b96723e..6d88c18 100644 --- a/arch/arm/xen/enlighten.c +++ b/arch/arm/xen/enlighten.c @@ -339,6 +339,17 @@ static int __init xen_pm_init(void) } late_initcall(xen_pm_init); + +/* empty stubs */ +void xen_arch_pre_suspend(void) { } +void xen_arch_post_suspend(int suspend_cancelled) { } +void xen_arch_hvm_post_suspend(int suspend_cancelled) { } +void xen_mm_pin_all(void) { } +void xen_mm_unpin_all(void) { } +void xen_timer_resume(void) { } +void xen_arch_resume(void) { } + + /* In the hypervisor.S file. */ EXPORT_SYMBOL_GPL(HYPERVISOR_event_channel_op); EXPORT_SYMBOL_GPL(HYPERVISOR_grant_table_op); @@ -351,3 +362,4 @@ EXPORT_SYMBOL_GPL(HYPERVISOR_physdev_op); EXPORT_SYMBOL_GPL(HYPERVISOR_vcpu_op); EXPORT_SYMBOL_GPL(HYPERVISOR_tmem_op); EXPORT_SYMBOL_GPL(privcmd_call); +EXPORT_SYMBOL_GPL(HYPERVISOR_sched_op_shutdown); diff --git a/arch/arm/xen/hypercall.S b/arch/arm/xen/hypercall.S index 44e3a5f..2bec3fc 100644 --- a/arch/arm/xen/hypercall.S +++ b/arch/arm/xen/hypercall.S @@ -103,3 +103,9 @@ ENTRY(privcmd_call) ldm sp!, {r4} mov pc, lr ENDPROC(privcmd_call); + +ENTRY(HYPERVISOR_sched_op_shutdown) + mov r12, #__HYPERVISOR_sched_op + __HVC(XEN_IMM) + mov pc, lr +ENDPROC(HYPERVISOR_sched_op_shutdown) diff --git a/arch/arm64/xen/hypercall.S b/arch/arm64/xen/hypercall.S index 8bbe940..1fb8cab 100644 --- a/arch/arm64/xen/hypercall.S +++ b/arch/arm64/xen/hypercall.S @@ -92,3 +92,9 @@ ENTRY(privcmd_call) hvc XEN_IMM ret ENDPROC(privcmd_call); + +ENTRY(HYPERVISOR_sched_op_shutdown) + mov x16, #__HYPERVISOR_sched_op + hvc XEN_IMM + ret +ENDPROC(HYPERVISOR_sched_op_shutdown)