From patchwork Thu Apr 24 14:13:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884948 Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D1A81A0B08 for ; Thu, 24 Apr 2025 14:13:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504030; cv=none; b=rsZ1O3XU1d/QX+M6TYopDkNznDFy2Di3FLPBoPtKLw4KkQ1ktH9ynGTBQcfssnxeptPuG5nvbx/S51RAXYKlwwpmJbxWpuFwW1EfUihjCTgBvdJInVCx3AYUa8SWciIH+zhMg/4nY/QvHSl5aJTAlPWFKw32h/dDZVjSieOHIoo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504030; c=relaxed/simple; bh=g59SsG0N84wb4f6piJJJcigRQ9wXVyh8p4R6BkHSxiU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=s5kFjecHTJr3xP2COsbnIBAu68og4eusBJ+P4DU53LVfwEHYvd0Ciuu3DXoVu+xtlHtlYrsPyqqjuC6PKQkJh2wvTDbQk/0Uw2RUvEt30YwTNhknMFzsSVFfg8KDEcIF9XC31/yVQa3KCjhFN/xA2bUSG0vMREClPbINEK8o36Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=d4ub933F; arc=none smtp.client-ip=209.85.221.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="d4ub933F" Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-39c266c1389so744582f8f.1 for ; Thu, 24 Apr 2025 07:13:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504026; x=1746108826; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9pJfw58KOVJlcvsi2YwD0da6h5VnpsCyDZ/he/rc1kk=; b=d4ub933FvrTaDPxCrebBqFETW/r4aHP5Zgr0uloJ5fW2q1BgsSLYE9WfvREpVpK5Yz rFlJg17xycIpGpNDwGK1zzI7gb73tqT1kOVRMlGs7gMFeSid/hTjvGdK6tahBOKODrA9 MIxU7d7+HD7laNzNMfGSRvGavpy/f6Zy7eHGHmV0O8Ii55ik8j1NGSeVDdWQlkcSaI6z le/xGuUuVHwERSloTou3x5Ocpx3XILcqiE2eyLPMMyRuEnmmiAM+J8jK63LuX3uz/rpZ y4zVZGqKyx6CMRrPLhtw2EZmfxMNZSgCyjBgTPAQ6bb1QZRnylv5EaQbwNSG0ZYHJqTG uOng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504027; x=1746108827; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9pJfw58KOVJlcvsi2YwD0da6h5VnpsCyDZ/he/rc1kk=; b=jH9cEj+/AJ9xI4FMxfUOOG766CuM9Zaflzm+hxlFSJXSY2Fa2DSEmfrNWIAniqN9YF 6Z4+lFr+Pw82rAY13jrSBIn3D30Oq5dKf6YJ4U7/nEhjEf9lXs4plghbUy0ngJjUeq0e HlSQ48jTiOSlSERHMIiQgd87X41WSFZ5bex4Ht6BVtpkT4X90HAgP+Ric4fvoXBXVbs4 9C38CmTcZnasvHOzk97+lxjTVugGnDsXYKm/VKhTYiGmhN/NB9B2hYyCkaPGssgztd+D Rs4QHEcWMnRp0Ckki0I2MmG0dLQG7zQaVwqT2nYBMkXPplpBpfKgPPr47EHdnGu29Qki VYuA== X-Forwarded-Encrypted: i=1; AJvYcCV/RDx7q48UDeWjJCiGvlNYh+btNCn4HHQ/RRB3CIDw54HZKEfdqKf2hvxb/tDlcwbW/RO44Ekl5UTTjpI8@vger.kernel.org X-Gm-Message-State: AOJu0Yzjvf1UbGiXGwX5Pj+SEQnbd7/CCgE0Mxl9SvdwFzoeO7ngFOwB cT8lGYE9+rPFLkhI1rtGVdRE//6uZIub1HHp7+4nIphuffH/Kr+NVuPG7bRLnro= X-Gm-Gg: ASbGncsv8/47Qr6cYOJPrDXqKLZyp92DF8zy+msUOgda5rJezvSDtGDxxUJn/VZmTTf GXRYHnSk+kwFkFG3AC0EMbX/iBwaWl05E8C+N8i246630+L0ODxn2rVO/TfzyKju2ny0vhlOmyv dyFKsin1mDwwD8XmjVCnAvAkj1HDRpXJbK2giRpi4dIx8jp/MC6o8kU6dTI14tckwtRARcXfG3I EouO0QSlCRw3Kt6R16ttvIrCjVG1PXypt83FcMjgPRZrj4I+zRcrq8l9vkWowGn4fg2mf2P2hLH 35gaEIP68xN4Fn884+YgQzsGdntvvDyucWJnJRIsVVbLwopWAxVek9OYADXFRfXuBa4kgeFNOyk nUEYMJVkd4hPUqGcN X-Google-Smtp-Source: AGHT+IGfRrIJRrexUH30juVxzQxuzQYHkS7hGTbMmJLV2Lp0+vmOpc/tkASkqUk2GbTXQepziYNH6A== X-Received: by 2002:a05:6000:2510:b0:391:304f:34e7 with SMTP id ffacd0b85a97d-3a06cfab9abmr2560733f8f.44.1745504026564; Thu, 24 Apr 2025 07:13:46 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:46 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 01/34] KVM: Allow arch-specific vCPU allocation and freeing Date: Thu, 24 Apr 2025 15:13:08 +0100 Message-Id: <20250424141341.841734-2-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Gunyah KVM backend [1] requires custom vCPU allocation to associate architecture-specific state with each virtual CPU. The generic KVM core currently allocates vCPUs directly using the kvm_vcpu_cache slab, which does not allow architecture code to intervene in the allocation process. Introduce two weakly-defined functions, kvm_arch_vcpu_alloc() and kvm_arch_vcpu_free(), which default to using kmem_cache_zalloc() and kmem_cache_free() respectively. Architectures can override these functions to implement custom vCPU allocation behavior. Replace all direct allocations and frees of vCPUs in kvm_main.c with calls to these helper functions to allow arch-specific substitution. This change is required to support architectures such as Gunyah that must allocate architecture-private state along with the vCPU. [1] https://github.com/quic/gunyah-hypervisor Signed-off-by: Karim Manaouil --- include/linux/kvm_host.h | 2 ++ virt/kvm/kvm_main.c | 16 +++++++++++++--- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 1dedc421b3e3..3461346b37e0 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1581,6 +1581,8 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id); int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu); void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu); void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu); +struct kvm_vcpu *kvm_arch_vcpu_alloc(void); +void kvm_arch_vcpu_free(struct kvm_vcpu *vcpu); #ifdef CONFIG_HAVE_KVM_PM_NOTIFIER int kvm_arch_pm_notifier(struct kvm *kvm, unsigned long state); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 69782df3617f..dbb7ed95523f 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -476,7 +476,7 @@ static void kvm_vcpu_destroy(struct kvm_vcpu *vcpu) put_pid(vcpu->pid); free_page((unsigned long)vcpu->run); - kmem_cache_free(kvm_vcpu_cache, vcpu); + kvm_arch_vcpu_free(vcpu); } void kvm_destroy_vcpus(struct kvm *kvm) @@ -4067,6 +4067,16 @@ static void kvm_create_vcpu_debugfs(struct kvm_vcpu *vcpu) } #endif +struct kvm_vcpu __attribute__((weak)) *kvm_arch_vcpu_alloc(void) +{ + return kmem_cache_zalloc(kvm_vcpu_cache, GFP_KERNEL_ACCOUNT); +} + +void __attribute__((weak)) kvm_arch_vcpu_free(struct kvm_vcpu *vcpu) +{ + return kmem_cache_free(kvm_vcpu_cache, vcpu); +} + /* * Creates some virtual cpus. Good luck creating more than one. */ @@ -4103,7 +4113,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) kvm->created_vcpus++; mutex_unlock(&kvm->lock); - vcpu = kmem_cache_zalloc(kvm_vcpu_cache, GFP_KERNEL_ACCOUNT); + vcpu = kvm_arch_vcpu_alloc(); if (!vcpu) { r = -ENOMEM; goto vcpu_decrement; @@ -4182,7 +4192,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) vcpu_free_run_page: free_page((unsigned long)vcpu->run); vcpu_free: - kmem_cache_free(kvm_vcpu_cache, vcpu); + kvm_arch_vcpu_free(vcpu); vcpu_decrement: mutex_lock(&kvm->lock); kvm->created_vcpus--; From patchwork Thu Apr 24 14:13:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884046 Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F8FB1A5BBD for ; Thu, 24 Apr 2025 14:13:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504031; cv=none; b=hJfORtUFmiaccYrGSg44Ljek3unh8HR5/wgHJE4ZYmWFpL4GTZqkBeXhDRy6ziGIGJJJ1+dYwSBWqXSrWtGoq7E3lU82EvY1dAbfGnLKZep86re1xSsQqZPp4azenHyTZD6R10z/I2CM3ykk0t9g4GB8Fd3Hz6BV76F0ePHVKpE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504031; c=relaxed/simple; bh=CvFb1okbnEK9wiiZwwdcGSqHE02tQmbzZOFs1CBcYGk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kpeVBIU/nJh1lHegoXCqG2Em+sZbnii6HTgkWIt0deM9ccNhj08CK4fgCfI/eYFzT40bpO4j4jeviQnmfNXONry1ygERhBewig4dvPH/zUMmFP32EMzEtC0HFcZleMcTAYuD9nsFCXvnhqhAe6JfAHSJXbfnya33uY4SIOsfYDk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=iLOwI++f; arc=none smtp.client-ip=209.85.221.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="iLOwI++f" Received: by mail-wr1-f42.google.com with SMTP id ffacd0b85a97d-3912fdddf8fso1551218f8f.1 for ; Thu, 24 Apr 2025 07:13:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504028; x=1746108828; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hvXrACbOQaGlhym8epnVf2FRavXrOILN2RCY7mrPgRw=; b=iLOwI++fMXomw84Au6sHZmvf4XEzvfBtSvOEbGXBg1IqUMdLW0+v/JK3OSvPQBiK07 G7DAOlkh3Dn7nSPFwYfcmRBHN4JhU6Z7fOWSzh4GJgLLSKO1hmkm6NG5WZH0AfoEC9Bo NaVFDFNV39iAFQcXUNIXRLyqVzXYxsfBRnVLgKEH3URDsPi8jYpP5eiVPiklRMRJO+fD KUIEEE9ixPIFXkDrY/7Qt3c6/QbTS76DZoKYstIM7n24LZL98RND02LEMC+RNX+PpEON PTHZTJSA7cSemMaHjeV4uQBh4Xu/4+vnuIYtqe3zD61Dyl0L28stdByuBTQVTBs9JVfw TXHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504028; x=1746108828; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hvXrACbOQaGlhym8epnVf2FRavXrOILN2RCY7mrPgRw=; b=wB0468jp2vSZm2eiK+xXakThBV7qJ1wiG5oWb86qYZhrfzxODQOjcLrQBnu+w/0OTO AwEWgpxjAHuK627nuxWX7bnojdQhAorBfy06dM5KW4CfTYZ2lVBWgHHAbdWLhbtddGAn 7zPnZV50lyWFhmWpBIEf8KxCSJOKKcGQMPkTxDTggmQqa5tbBSEMcZ6fjzDqyV8KC5sf GBHRQHzhj+hf9weY+kX6RwQodZRjJ9jVmQFY/S34YWO37HwdXESIgJ0xVsanFMAxCGcp 8THud6FtpNtz4zPrb4eXuqI9oagbOOetZtO/gesHV87ShlNG2Z0DyBkko5GXu7u+zj07 /CyQ== X-Forwarded-Encrypted: i=1; AJvYcCWNFu7PZocSwNwZsfsclmCiMyKQMA8vddbI8geqZ0xvHVxpUlwXaQCgnDvObIu63qsAiWF1BaxK8iOzo1rI@vger.kernel.org X-Gm-Message-State: AOJu0Yy0kpucxj3k2KU8SNn5s2+srWTEGYNt20/Cw2vl/W21MkQZEMx5 5txrwjdJu6RAcJmxx5RJkoU6l07CzKAc8wImR0utd1AQWTaIXW7d818D6fTZ988= X-Gm-Gg: ASbGncvhuznOZCFRJNns5QZMzLwpk0E5ij0TWSOz8OlstL276boDESaikSS+KtkSIoo NwBhPXzaLwHQd+n9IW3UTsLom+MkwwHAVuwGnVSxe3WCIg/DUIIOW1sbQiIfC9Hk8QmeNtfEHNU o6e+7+nkbQZ4dJ2k23QwbyZrTKZUn8bvFaPmMAmchn32GO0UpFQhYpGjox1kpPG5Y9D8+X7t7TA TIXfrdcCMK3+y1sk5S7igpHtD0zee4oCJAqO1B3dRXLActU94SW8lf/crt2PaWrTQG/IRAELQKY S4S/gnGT+RtVrw7/Qn98QpkyV03Qoarf+Of3Fr1enRRC7M8noX8tRiBvty/WPFfBSTPy/ixNGqX s8Vi+t7FwTHRLClWMncVZ/s203MI= X-Google-Smtp-Source: AGHT+IEBkZ1IBJCII/ywyTdMC0fLKprAp65skAqA7s+zC8i8AN1EGINZeECW9VcRyNwMhKCFTJTxcg== X-Received: by 2002:adf:fb4c:0:b0:391:1218:d5f4 with SMTP id ffacd0b85a97d-3a06d698d62mr2225510f8f.23.1745504027803; Thu, 24 Apr 2025 07:13:47 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:47 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 02/34] KVM: irqfd: Add architecture hooks for irqfd allocation and initialization Date: Thu, 24 Apr 2025 15:13:09 +0100 Message-Id: <20250424141341.841734-3-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Some KVM backends, such as Gunyah, require irqfd structures to carry platform-specific state or setup logic. To support these use cases, introduce three weakly-defined functions: - kvm_arch_irqfd_alloc() - kvm_arch_irqfd_free() - kvm_arch_irqfd_init() These allow KVM backends to override irqfd allocation, teardown, and initialization logic. The default implementations simply allocate/finalize a standard `struct kvm_kernel_irqfd`, maintaining existing behaviour. This change is required by the Gunyah backend, which uses these hooks to associate irqfd objects with Gunyah-specific bell resource handles for IRQ injection via hypercalls. Signed-off-by: Karim Manaouil --- include/linux/kvm_irqfd.h | 4 ++++ virt/kvm/eventfd.c | 31 ++++++++++++++++++++++++++++--- 2 files changed, 32 insertions(+), 3 deletions(-) diff --git a/include/linux/kvm_irqfd.h b/include/linux/kvm_irqfd.h index 8ad43692e3bb..e8d21d443c58 100644 --- a/include/linux/kvm_irqfd.h +++ b/include/linux/kvm_irqfd.h @@ -61,4 +61,8 @@ struct kvm_kernel_irqfd { struct irq_bypass_producer *producer; }; +struct kvm_kernel_irqfd *kvm_arch_irqfd_alloc(void); +void kvm_arch_irqfd_free(struct kvm_kernel_irqfd *irqfd); +int kvm_arch_irqfd_init(struct kvm_kernel_irqfd *irqfd); + #endif /* __LINUX_KVM_IRQFD_H */ diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c index 11e5d1e3f12e..5f3776a1b960 100644 --- a/virt/kvm/eventfd.c +++ b/virt/kvm/eventfd.c @@ -32,6 +32,24 @@ static struct workqueue_struct *irqfd_cleanup_wq; +struct kvm_kernel_irqfd __attribute__((weak)) +*kvm_arch_irqfd_alloc(void) +{ + return kzalloc(sizeof(struct kvm_kernel_irqfd), GFP_KERNEL_ACCOUNT); +} + +void __attribute__((weak)) +kvm_arch_irqfd_free(struct kvm_kernel_irqfd *irqfd) +{ + kfree(irqfd); +} + +int __attribute__((weak)) +kvm_arch_irqfd_init(struct kvm_kernel_irqfd *irqfd) +{ + return 0; +} + bool __attribute__((weak)) kvm_arch_irqfd_allowed(struct kvm *kvm, struct kvm_irqfd *args) { @@ -153,7 +171,7 @@ irqfd_shutdown(struct work_struct *work) irq_bypass_unregister_consumer(&irqfd->consumer); #endif eventfd_ctx_put(irqfd->eventfd); - kfree(irqfd); + kvm_arch_irqfd_free(irqfd); } @@ -315,7 +333,7 @@ kvm_irqfd_assign(struct kvm *kvm, struct kvm_irqfd *args) if (!kvm_arch_irqfd_allowed(kvm, args)) return -EINVAL; - irqfd = kzalloc(sizeof(*irqfd), GFP_KERNEL_ACCOUNT); + irqfd = kvm_arch_irqfd_alloc(); if (!irqfd) return -ENOMEM; @@ -396,6 +414,13 @@ kvm_irqfd_assign(struct kvm *kvm, struct kvm_irqfd *args) init_waitqueue_func_entry(&irqfd->wait, irqfd_wakeup); init_poll_funcptr(&irqfd->pt, irqfd_ptable_queue_proc); + /* + * Give a chance to archiectures to finish initilization. + * E.g. Gunyah needs to register a resource ticket for this irq. + */ + if (kvm_arch_irqfd_init(irqfd)) + goto fail; + spin_lock_irq(&kvm->irqfds.lock); ret = 0; @@ -452,7 +477,7 @@ kvm_irqfd_assign(struct kvm *kvm, struct kvm_irqfd *args) eventfd_ctx_put(eventfd); out: - kfree(irqfd); + kvm_arch_irqfd_free(irqfd); return ret; } From patchwork Thu Apr 24 14:13:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884947 Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDAB31A072C for ; Thu, 24 Apr 2025 14:13:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504032; cv=none; b=uVKL/Ms/fzrTDFqiLC4jCMci0YI2BtJJJaHnP4Az4UlDep5VoiOROiG5ailCdB8T9wEShjckBhDqSFOvtigWGDIwZeJz3nhUj9BzjP6osF3D+M+zV2TRJmAUEZHIwu9pc/4ZtaNMKRoGdXHG/Uoen2ajJYYuWK66Casa+xZ+0og= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504032; c=relaxed/simple; bh=wzaNO/FV98NvOhVwcurDeWV9o2lzkiDEMgzO9vOQkfc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cglXseCeQv31UJ2WRFoykt9TPoErkci5tYixzvUyoEeAtCGdLpOCyVj+nZ0x5VjMfaFaRDlVBu/xBVElq2g62LkQytILUoxwXDBEyXe8AL7qEUNjTwQ63E8OY8CIv/HefpHSbvSMoLZaVZEu7ULgK9MKVP83KG/IK0igdU+unSQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=AJZId15e; arc=none smtp.client-ip=209.85.221.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="AJZId15e" Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-39ee651e419so653747f8f.3 for ; Thu, 24 Apr 2025 07:13:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504029; x=1746108829; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9Hk5UBQTvuxPsQW0619Tu3I4qsqkftwTtmBE5Ybb85g=; b=AJZId15e7pdgeqWr+FMAv4lLCiYL8lS9X4n246zeYuB5IaR/DGRrpcovPDeJif1ult pCCP+lQCy1PBuRRsue4enbNhuItViUP50zrTPxmT0nYVJuKw/mn8cCZ5OQlhg6eDHW9M a05wR8Ubv3dLoceKbyhzZ/riusaGcx+r9eKtqhXZP/YlnDIR6V5OqRyMhOR7PXfzMpu9 bd39oniRh5fAZXnOSJ6Smcyyg/jnOY7Ep8YHqXr5Ac8Lg6Ki8jPdYS6b+DDfyDjPiHA+ KmVtSGfJ5KzOQuk2U26Ugg9Q5sqD/HC/WlFq2brgPR9T1soyDON1tKSxraJ1r6NZZ3nr p44Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504029; x=1746108829; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9Hk5UBQTvuxPsQW0619Tu3I4qsqkftwTtmBE5Ybb85g=; b=FSNkpKcZ2ZmCvc7zZbEU7Y3or8RvC+1htiCWVY9os/pZh54Oc79di3xdaqQXHvupLw SPxobCF5h+tZs8r+u2odC8e0pH2+jbbbwJdLI5QeJW2THXo5ACQlaemblGuXUSpSHpNf ixxYpanvfTtodXOjD7BGABW90RZI1CRm562oUFog8+on26+5y+dUJad17ouGf8RhcOXw UeGulNe8HIi+VVe/vYdY3yjxPZER7a/0U8jj9LzIwwk1htpW3i7Ob/1Vv4jrjgylGgM8 BjVcS41YV5OckD3t/K7DrqxNDDqvxtdpBZ0muQ525rDGwd5Y9aGJfDaFIaEG7wJPTbJC LS5g== X-Forwarded-Encrypted: i=1; AJvYcCXbO+myVrXNQZ9hLpseTm/EOgkl4DeBEBL90h8vj/93JWk2BsC/hyo2zYZVZnwWWWUrsPIG3a7Rlj0s9cMB@vger.kernel.org X-Gm-Message-State: AOJu0YwpZO/xy7wC+IhUIwAJNI+h2KVb1oaQpeXQbGIxw37W6xFvo8nw 0eSWCbUnUK414vT+RorsUtaFPhrinN4Eph3kyxxdg+rkIWaJ/GYASHxcvmVSj7E= X-Gm-Gg: ASbGncuHLmQyj/DOZnBOIIG839AC8yi1hXuq+QE1/Ld1KxNPn3zusyajcrORWYM7+xS DUNPQzipdSX5LqqwHz/J2aQdaVEnu6gyKVmJnA65SqKcYtNoBb7c0Utt6m1muozNSdAVwklMZYe RAlTwMsqjSOVlA0Pv8st3pihxSrQ4Ms8xcXf7AdpmtTNF9KsGIXUHDJS6toFAVZh2IuPni4oRSh CTtZx1EkRsX8bpvC72nzZu+ZO3mZuXMCuTa1Pfmf++7mrh14xj/Qs1NdTDC2HCIN7rC6p37QO98 K24LlgXMcPy/1DLkLZ+QytGtt0DNEVe/J4myTxVXAnJrs+TmgdwIwIjCYaq8knfGcTQW6BerNYe ekPnPFrISU5f/sZz8BHl8YfhZEAY= X-Google-Smtp-Source: AGHT+IEX65fw+AuwIbm8DfCqVPIwMLBPu7X4mEmOAObvmlxgMqJT5rSe8ZZGCXBt4CgJFkEQIJbeNw== X-Received: by 2002:a05:6000:1a85:b0:391:39fb:59c8 with SMTP id ffacd0b85a97d-3a06cf61418mr2487543f8f.25.1745504029007; Thu, 24 Apr 2025 07:13:49 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:48 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 03/34] KVM: irqfd: Allow KVM backends to override IRQ injection via set_irq callback Date: Thu, 24 Apr 2025 15:13:10 +0100 Message-Id: <20250424141341.841734-4-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Some KVM backends, such as Gunyah, require custom mechanisms to inject interrupts into the guest. For example, Gunyah performs IRQ injection through a hypercall to the underlying hypervisor. To support such use case, this patch introduces a new optional callback field `set_irq` in `struct kvm_kernel_irqfd`. If this callback is set, irqfd injection will use the provided function instead of calling kvm_set_irq() directly. The default behavior is unchanged for existing users that do not override the `set_irq` field. Signed-off-by: Karim Manaouil --- include/linux/kvm_irqfd.h | 1 + virt/kvm/eventfd.c | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/include/linux/kvm_irqfd.h b/include/linux/kvm_irqfd.h index e8d21d443c58..7d54bc12c4bf 100644 --- a/include/linux/kvm_irqfd.h +++ b/include/linux/kvm_irqfd.h @@ -46,6 +46,7 @@ struct kvm_kernel_irqfd { /* Used for level IRQ fast-path */ int gsi; struct work_struct inject; + int (*set_irq)(struct kvm_kernel_irqfd *); /* The resampler used by this irqfd (resampler-only) */ struct kvm_kernel_irqfd_resampler *resampler; /* Eventfd notified on resample (resampler-only) */ diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c index 5f3776a1b960..d6702225e7f2 100644 --- a/virt/kvm/eventfd.c +++ b/virt/kvm/eventfd.c @@ -63,6 +63,11 @@ irqfd_inject(struct work_struct *work) container_of(work, struct kvm_kernel_irqfd, inject); struct kvm *kvm = irqfd->kvm; + if (irqfd->set_irq) { + irqfd->set_irq(irqfd); + return; + } + if (!irqfd->resampler) { kvm_set_irq(kvm, KVM_USERSPACE_IRQ_SOURCE_ID, irqfd->gsi, 1, false); From patchwork Thu Apr 24 14:13:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884045 Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 03D431D63E4 for ; Thu, 24 Apr 2025 14:13:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504034; cv=none; b=naj990Z5lhbD7vvmE5XCP8sR5hJUqUBeWEXgSPtRs/Bl6IegKp+hpABGDeb7yEIXav2ILs/leju9UFuVQ/VkqRiYgV2E0wyD4B1dQmGP5t2Hzg+KMVsOTVrBT5JhX69VwTILwNZ1K8zidhcyVXHPqNorSv6CzU6xCIiVJwb0ilc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504034; c=relaxed/simple; bh=yHQVbNXozVjVHt5pOWEf6CFk11nfYNM4wHopdbI5gbI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=r+lOvLYC7PSVNfBL2pSu3FXXwc6412t2X+396qiNGNaH96bXrlqPXwlQ6QmQl39xzFGIcLi1Rho7x97z6XqqlTDAX/nsgFnNSv1U/WQA8evdw/Zd6pBEN3mDIXlqyIOQyqTvD8tSBbRsH3wn7nF7exQ52z5Yzr4fq4QAhPdPRSY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=TMpYcwLk; arc=none smtp.client-ip=209.85.128.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="TMpYcwLk" Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-440685d6afcso10746705e9.0 for ; Thu, 24 Apr 2025 07:13:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504030; x=1746108830; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AlljNambtbwwvkzAI4LjAz2eo7Ti1KUJrgs8M2UB6Bg=; b=TMpYcwLkfLTj/ATVziMuksWaNsV3xs053pw6b7Tre2eq7ME2fqLNfMFtBGsCfJ0eLv YaiXKTnIikVyMFLz6c8RFczV2dxEuTPlqUn0VbE8hvXxjTAi2mOelNzXjdxPAmnkSCfd 3LifOZSqAm46lqRHFyEjsJsmzn/kOOCCpL0PCLxtWs3mOsRWSy+t7/E8RaWNKpD4LyWW QEM8MkS1ukD7WBR9ph7HvsZQWjM1KNmig+Vr6IRoe9cSARPdbP8NtrVV38KehoUVsO5h vE9r4xl6/rZ+05p1xaciTkPpWpp6a1HaUXeVGrmcWltf1wygu6Br9SLBLSbQuwtYg8jY 0m/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504030; x=1746108830; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AlljNambtbwwvkzAI4LjAz2eo7Ti1KUJrgs8M2UB6Bg=; b=l/7N1/fhBGXxXzP7cYBg42aedUdrT3jswCoAgelTRm65HcftNToFYz9qK8U54C1UAT Fs33YwvJF9y8O1sRAayozCkxHw790f6WDjpTd/uxYd5sHtPI0r2OLzl6g50dFVIwZxKF es/ivKXZ/jGwQ8K7j6M6f4tEG/00DVao3lsVgaj+nFxa7JLjoUCGCDv3umaGa4L9wNs4 vGmUYFj9VJ0R8hn7b1OqpXETzjkbQOG4WA+ahe7hlw/ao6q408cGmmLxUgrggo9IW7Uk IKyJHCSFLYAqMtEv2kkcGU3uoqLQNOexT3Sz9+An+aBmNIGV4oaMkI1VqYKRDa7RzVZl g6XQ== X-Forwarded-Encrypted: i=1; AJvYcCVHaRdcxHHI1NkNaYRq+f8GGqj9f6UNVKFkKu09ZXGe2HkMaXCLtFrxx3OpOiQhU/ztcMceAORXSTnEXKc8@vger.kernel.org X-Gm-Message-State: AOJu0Yy9ZuTsdnvwmfVCOLCcOXxwJebKqLb5n8EgTBGjqrnJgjGkTkKJ flhTa6+C2Ut1hmt7iZZvcPE24EtQ0bRqvrv3JPWOClNiji70rs/JII3rXtoOZKIqn4lpYGm2pvm AddI= X-Gm-Gg: ASbGncv/bIU+xyVzi5zxzE2oXR39zv1uvv1LlGbZ9ZcHXDA8dqxqxwzzi11++iS9Baa Hv6mmi6+Axj3EgI6+NC2EHmacAKEfyxplSTFggAB6qQKQhV6y1bpSiCS8WjShlu6u3+MJvNKmr2 l3z0sakWfa0afJGUVFWuE9tromz96Lzp30HQzHSK0W9rWv1yO4AOIAaIY3QgEMeafC4HToaOwnE 7SUEHcnJW4Lb+cqEZDthVUIfknJfanxMcvdsBzWPTykEVtmRxUbGnSRoc1oh6j6iDg1gBq+hWnc tmYQhkfS15DraE80Lu2mrWU0WJMk6P9X+rSqIDOeQRRJS1ZAYX+wmy+VXzULyYS90+pRXhE2bmI Hq/v+pKrgAc2VDT6L X-Google-Smtp-Source: AGHT+IGjTSYHUF3tyGvvpZJy9ttFB+SjTk/OBEoax4K5qs9LZf6N8lXayXTA0lInN+EeJy++KbQTNQ== X-Received: by 2002:a05:6000:18a2:b0:391:47d8:de2d with SMTP id ffacd0b85a97d-3a06cf5ed4cmr2481988f8f.23.1745504030257; Thu, 24 Apr 2025 07:13:50 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:49 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 04/34] KVM: Add weak stubs for irqchip-related functions for Gunyah builds Date: Thu, 24 Apr 2025 15:13:11 +0100 Message-Id: <20250424141341.841734-5-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The generic KVM core code (e.g., kvm_main.c and eventfd.c) calls into irqchip-specific helpers such as kvm_set_irq(), kvm_irq_map_gsi(), and kvm_irq_map_chip_pin(). These functions are defined in kvm_irqchip.c, which is not required or compiled when porting KVM to run on top of the Gunyah hypervisor. To allow building the KVM core code without linking errors in such configurations, provide weak stub implementations of these functions in eventfd.c. These stubs return appropriate default values (e.g., -ENXIO or -1) to indicate that the functionality is not available. This allows the KVM core to build successfully for platforms that do not use the in-kernel irqchip support, such as Gunyah. Signed-off-by: Karim Manaouil --- virt/kvm/eventfd.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c index d6702225e7f2..2a658d8277ed 100644 --- a/virt/kvm/eventfd.c +++ b/virt/kvm/eventfd.c @@ -56,6 +56,26 @@ kvm_arch_irqfd_allowed(struct kvm *kvm, struct kvm_irqfd *args) return true; } +int __attribute__((weak)) +kvm_set_irq(struct kvm *kvm, int irq_source_id, u32 irq, int level, + bool line_status) +{ + return -ENXIO; +} + +int __attribute__((weak)) +kvm_irq_map_gsi(struct kvm *kvm, + struct kvm_kernel_irq_routing_entry *entries, int gsi) +{ + return 0; +} + +int __attribute__((weak)) +kvm_irq_map_chip_pin(struct kvm *kvm, unsigned irqchip, unsigned pin) +{ + return -1; +} + static void irqfd_inject(struct work_struct *work) { From patchwork Thu Apr 24 14:13:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884946 Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 68A8325A65F for ; Thu, 24 Apr 2025 14:13:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504037; cv=none; b=Xf4gmako7QbcwVpNrAvm1HeZ3R0J+DsP6jOEbRMSpsqCO7bTJYOlmoShWbIpVtF8HZ63mtoveeSwlfUOsQWtXzpDdYOGuvhPsrjuq7Hi+ssmZS2q0J9mc/CwmmtmpbDXSIV7q+R1cLy5JpOmBHxihwgLjuSw8pfwpB81BzPYLLc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504037; c=relaxed/simple; bh=62G1u7Mmcj0LoLg/GsZOxRpsdU6hf0LcgEiDACKmXjQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=rpm8l0EDvFeWRDvxROo9+TkOJ+lV5DLc1Seu0Phcy5gtClJdpBi/HiiqkSIdrWgIcDls1lgIOuDRr+0/pSqBljH0I2LQsq2fO84wr/hkZestEKLt+a98GssOXETSSIYFlAGWTibh+BnufwykfZdK3VBJLNqMpDkMl5/zdV0e5q0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=YyLEy0uC; arc=none smtp.client-ip=209.85.221.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="YyLEy0uC" Received: by mail-wr1-f53.google.com with SMTP id ffacd0b85a97d-39d83782ef6so1647886f8f.0 for ; Thu, 24 Apr 2025 07:13:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504031; x=1746108831; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BvSif0ZiFcbo45REElIUdI7xtRbISNuAr1Y97HbCqdI=; b=YyLEy0uC+aKHIkM4oaNNOOvTEZxl/5B4cXMJWSiA5CH0VKETdRs1XxMP9euG6CgE2e Z23rgG1SUAKMC/rdVHWjNbC+XWmyfueGcOirkOcsGMCQPLCujAKCkPge1SHWgctMolRg VsK7AbhITHdDJZ/mZTkqKU2opmMA5akZbn1I1JWbXeLTDgeuLCR+bqweC7o3r8fqG0JP rvxIZ6AW6JAFEQIIudhQnHCQgdgEg6zG4+kSRQJN8FbIPXzyGwglOHS25g10OTkEfunL lbyuyS6pqq7X1Q8tQ/K+fZGkXN0vkHcZtBXrR60SkIIevTg4zC9iJj8ScVY3BHkXRwUW mzag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504031; x=1746108831; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BvSif0ZiFcbo45REElIUdI7xtRbISNuAr1Y97HbCqdI=; b=hX8n9gr/7pQ2NPvF3U2yVlX8zs3TjfV2ivfOo+qXEg1gJGl6EaXRR4NnjtG20U5GHB 9U+92vJMH8VOORA1XnqfnmzWoh7VbaTrgk23lz6V3FEcx/Qczb0V5sTqscPS6yoLUZTf 4ruDVfDF5IMmWWF8PZrKxnXX1JlTzBh+D9PAg44nSgDjuQkJ67B4TIJxjJGeaAYDCku8 XVPUTOO4sS161m+LD4KsDYFJFmsImVpZKMMr/RD0DUJ1mSMin4wz5YSFpW4Wo3UCtmRg ANsCLLdSLLqH800m8T9IY2jSfvwxUEacNyrrJaSgVe5Nlm0aL6ir/oRJ/bQOjxcxcqOf PI+g== X-Forwarded-Encrypted: i=1; AJvYcCUEAcgS9K/3dCZmvVCDdea2ta7FkpMN9/QbgjI5JYt9+FFsZVUzYDsFGjQdZuFIGv6rgJJ82uBf/ZRto3uq@vger.kernel.org X-Gm-Message-State: AOJu0Yw5VtP+QlfnutJGg0f8+R6zmAyreziccJE+pKk2oWwAk5rogF2n DOPcbGslO2bBg1YpjktQyCnHZ50ECPQGAcbEA/JSdDtLYG/gPC356ZOoQq/0Dd8= X-Gm-Gg: ASbGncting8RsHUb3Gzox+GGxYrGAGmvv74K4+/PfOZccdrIQNuYqjL5hlJ6LYN+K1q esAmqGCDtMIapfSfXCSCm0fz37u65UmkLHdBCVebX9TnU1AQIiO4OOHJvHeb2tGYwvM1ljH4uAq 3yQXgugVZCsDx0IQY3F8nLupgC1G10+X7C9FWBxe+pTPfO7HfXFzyvu9ZF6lGf4nSQ8vgMhpPKo 8FN1OV9OtGW+E1gmenwavjWSZQ93G6CcKjBts7nYSYJF4a6LJan8fsaNQenpcHbZvfxe/IlxW3l Cb2/IVoiVY76mfxg27bUpgqKEaEX7EdSU0GPjG6QKP6HVBmgny4AoecaewXAlC9BIvh9sfQU+Qp 3QnVnF2euL4ziISWy X-Google-Smtp-Source: AGHT+IGs6TOHidyOdLjO/aiH4v937+mffmDnW0sidZAyNfcPHTexYT+C0CxkEqdPf2DwJcaHqEXYnw== X-Received: by 2002:adf:ebcc:0:b0:39c:1efb:eec9 with SMTP id ffacd0b85a97d-3a06d673778mr1956533f8f.13.1745504031472; Thu, 24 Apr 2025 07:13:51 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:51 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 05/34] KVM: Add KVM_SET_DTB_ADDRESS ioctl to pass guest DTB address from userspace Date: Thu, 24 Apr 2025 15:13:12 +0100 Message-Id: <20250424141341.841734-6-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Some hypervisors, such as Gunyah, require access to the guest's device tree blob (DTB) in order to inspect and/or modify it before starting the guest. The userspace virtual machine monitor (e.g. QEMU) is responsible for loading the guest's DTB into memory at a guest physical address, but the hypervisor backend must be informed of that address and size. To support this use case, introduce a new ioctl: KVM_SET_DTB_ADDRESS. This allows userspace to provide the guest physical address and size of the DTB via a `struct kvm_dtb`, which is now stored in `struct kvm`. The ioctl allows platform-specific backends like Gunyah to retrieve the DTB location when configuring the VM. This patch also increments the KVM API version to 13 to reflect the addition of this new ioctl. Signed-off-by: Karim Manaouil --- include/linux/kvm_host.h | 1 + include/uapi/linux/kvm.h | 14 ++++++++++---- virt/kvm/kvm_main.c | 8 ++++++++ 3 files changed, 19 insertions(+), 4 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 3461346b37e0..4e98c7cad2bd 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -862,6 +862,7 @@ struct kvm { /* Protected by slots_locks (for writes) and RCU (for reads) */ struct xarray mem_attr_array; #endif + struct kvm_dtb dtb; char stats_id[KVM_STATS_NAME_SIZE]; }; diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index c6988e2c68d5..8f8161cd61a7 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -14,7 +14,7 @@ #include #include -#define KVM_API_VERSION 12 +#define KVM_API_VERSION 13 /* * Backwards-compatible definitions. @@ -43,6 +43,11 @@ struct kvm_userspace_memory_region2 { __u64 pad2[14]; }; +struct kvm_dtb { + __u64 guest_phys_addr; + __u64 size; +}; + /* * The bit 0 ~ bit 15 of kvm_userspace_memory_region::flags are visible for * userspace, other bits are reserved for kvm internal use which are defined @@ -1190,11 +1195,12 @@ struct kvm_vfio_spapr_tce { #define KVM_SET_IDENTITY_MAP_ADDR _IOW(KVMIO, 0x48, __u64) #define KVM_SET_USER_MEMORY_REGION2 _IOW(KVMIO, 0x49, \ struct kvm_userspace_memory_region2) +#define KVM_SET_DTB_ADDRESS _IOW(KVMIO, 0x50, struct kvm_dtb) /* enable ucontrol for s390 */ -#define KVM_S390_UCAS_MAP _IOW(KVMIO, 0x50, struct kvm_s390_ucas_mapping) -#define KVM_S390_UCAS_UNMAP _IOW(KVMIO, 0x51, struct kvm_s390_ucas_mapping) -#define KVM_S390_VCPU_FAULT _IOW(KVMIO, 0x52, unsigned long) +#define KVM_S390_UCAS_MAP _IOW(KVMIO, 0x55, struct kvm_s390_ucas_mapping) +#define KVM_S390_UCAS_UNMAP _IOW(KVMIO, 0x56, struct kvm_s390_ucas_mapping) +#define KVM_S390_VCPU_FAULT _IOW(KVMIO, 0x57, unsigned long) /* Device model IOC */ #define KVM_CREATE_IRQCHIP _IO(KVMIO, 0x60) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index dbb7ed95523f..a984051e2470 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -5121,6 +5121,14 @@ static long kvm_vm_ioctl(struct file *filp, r = kvm_vm_ioctl_set_memory_region(kvm, &mem); break; } + case KVM_SET_DTB_ADDRESS: { + r = 0; + if (copy_from_user(&kvm->dtb, argp, sizeof(struct kvm_dtb))) { + r = -EFAULT; + goto out; + } + break; + } case KVM_GET_DIRTY_LOG: { struct kvm_dirty_log log; From patchwork Thu Apr 24 14:13:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884945 Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4F3027D78C for ; Thu, 24 Apr 2025 14:13:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504038; cv=none; b=BQMzQkWowipzdtW28eiSRfOo+wIIAraCSISlpGLCLWjCsGD6J3wXjXbZ2NODq9mI+9B8pCbpQDzDENBog+QWGUBrnH4X3SVBtKpvc3SYpsULEGkD98kZ4+iN0rObNhdThQaJzRxTHaVNaFhbxQnaWW0NzjTJEVBRVZByaCeUauA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504038; c=relaxed/simple; bh=mRPK+89hbVEkRoZvoKf3k11XmtiTEyAylGP5LEAOcB4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=d++hSWMHPl9qkEWX7p4BW86bwew8jxGeiepTqmbKCKoWImhc/IZU8De1VFnaKm3QBnJB3dN50yhlCTXYg7xNsVZxQ0KFr7niK6jn7VSvrZHdqaHWDmUk+ADaI2JOZYqwIv3f0yz2c6fko5G+acddGY77PMEky3mMdgMUVPV+/a0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=NHz03KRf; arc=none smtp.client-ip=209.85.221.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="NHz03KRf" Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-3913d129c1aso753236f8f.0 for ; Thu, 24 Apr 2025 07:13:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504033; x=1746108833; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8jfERcd+/8hI2lWUkL6Z3PV6Xql42InuHmm1RJaBik0=; b=NHz03KRfKRU1ydrBTpF950pSDo/fHHj/qFQsZBAH0nAr4J9XrBebUanz4RG0Ixizpb zizFOX7Vu6tkDrj2qPw3CD+XTvSRf647mi6yu15W/sUo85Ma/WYJ03VBSbAyl3QveuSj y+rNVChsDvDU1Owq9C37MZKVnLhZcFMRtLzhu9lxwqU4fxn2gQzaTp0qmCCwCCdkyUob GKkWZ9icDjECa2TnEhe8KvnOI/31ftMaK1j05MmE06f1oYp23/LsxAb0xjpcv5N66sz+ xsWF4R4435th0RzZhvo28yWhWepgCqYIkW7Is/iTl37ZQJa/SEUSfIzKb2Tm1RJkuXbn BUjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504033; x=1746108833; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8jfERcd+/8hI2lWUkL6Z3PV6Xql42InuHmm1RJaBik0=; b=oRURP7eUelyinXq38lCFsFEnynFcQNpeOmk27bGkQreyPAwu0dg/Y3yKgmejZI2955 6WXdzlh90dk7ZfwF53w/9OCuCkX3+Ins9i/cVesQudjQZEhXkYHn/mP0KpCUpSJambDq vLl3QHK9AZzuUNLH6bbGhDU9sRqpbeMbKzYAYrhOW4/N7zb9RVhSP+e3NhUpzRB1MXW3 F0kabfN6NI4FXgqRyFGArmA5cbI2MsOqv2+vtn+P4DdrcOA5RAqlfAZQTwu25A/N3ye4 DyggRQGJIY+0Hk7dDSfv07ieK0fWmxxhn0Wp5o9tmDySQ1ACr4b1sUdQzylf+HYhxbK8 PYUQ== X-Forwarded-Encrypted: i=1; AJvYcCUTFKnixr1yPgine46OEddPuqg5Ya989PvhheGMyKqYySpxdfjy1MMx20oJ7vPwuY6b13rPmjKua/Z+j6KV@vger.kernel.org X-Gm-Message-State: AOJu0YwdYHVI0GW0cDEty8XuR+uVt0ERYu2VBGWsuoWoa+hEgH/dmmJ5 IZAcrlq1iPbvzpfsHfqEICH6uB107nVTYJhlax2EQpPTD7wC7M71zmBi7vrZpkQ= X-Gm-Gg: ASbGncsx4zE9dhc/9ssUZ8JNSSY+m020uZ/y+IbTPjiAHiFKAf4SkNWKVRE1xDS3Wkl BRwuJMs5sQGjlf+OClny1X5ODlcRTYgvZu3D//COOxvgXjJnM9yZ1kKWTdXxPGg50tZ3rmVPiIE MDhDc/b+7zTa7KLTh4mYWnhbNFY30BtofI5/B8N4IZodywx1a1/wjJ52AVeImJuHkC/URdv2M75 k6LK9/XLUOt+NIEHat5jfNOE2VpmKMlhu09LvEzVVEgt9EZM04IO+li3VGGTRrcyg6NQKAcKOwW Cv4lEawomopgx1cc3iQ6S+uen7JkMS+35ubO7sPvWe8W72917yy/cF2cZpBm8Ka5W6cDfGONEM0 yRmd0FhQc8o9ubXH4 X-Google-Smtp-Source: AGHT+IGx0xNiXWRY0y1pEy2kIqf26zFRwFHT6eClNX/GVzcfRlYZlyiUPQIhI5t0dInRmkb4qoBxLw== X-Received: by 2002:adf:e2d0:0:b0:39c:1f02:44d8 with SMTP id ffacd0b85a97d-3a06d647344mr1972721f8f.4.1745504032796; Thu, 24 Apr 2025 07:13:52 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:52 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 06/34] KVM: gunyah: Add initial Gunyah backend support Date: Thu, 24 Apr 2025 15:13:13 +0100 Message-Id: <20250424141341.841734-7-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This patch introduces the initial skeleton for supporting the Gunyah hypervisor [1] as a KVM backend on arm64. The Gunyah backend implements a different KVM architecture backend under `arch/arm64/kvm/` alongside the existing support for AArch64's native virtualization. Please, note that the two are mutually exclusive at build time. Key highlights of this patch: - Introduces a new Kconfig split: `CONFIG_KVM_ARM` for native support, and a variant for Gunyah-backed virtualization. - Adds `gunyah.c`, a new arch backend file that implements the minimal KVM architecture callbacks and stub interfaces required by the KVM core to build and boot. - Refactors Makefile and build rules to support mutually exclusive builds of `CONFIG_KVM_ARM` and `CONFIG_GUNYAH`. - Introduces a dummy implementation of required KVM stubs such as: `kvm_arch_init_vm()`, `kvm_arch_vcpu_create()`, `kvm_age_gfn()`, etc. This serves as a starting point for developing virtualization support for guests running under the Gunyah hypervisor. Subsequent patches in the series will add support for memory mapping, virtual CPUs, IRQ injection, and other guest lifecycle mechanisms. CONFIG_GUNYAH is going to be introduced in the next patch imlpementing the Gunyah driver. [1] https://www.qualcomm.com/developer/blog/2024/01/gunyah-hypervisor-software-supporting-protected-vms-android-virtualization-framework Signed-off-by: Karim Manaouil --- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/include/asm/virt.h | 7 + arch/arm64/kernel/cpufeature.c | 4 + arch/arm64/kernel/image-vars.h | 2 +- arch/arm64/kvm/Kconfig | 22 +- arch/arm64/kvm/Makefile | 14 +- arch/arm64/kvm/gunyah.c | 736 ++++++++++++++++++++++++++ include/kvm/arm_pmu.h | 2 +- include/linux/irqchip/arm-vgic-info.h | 2 +- include/linux/perf/arm_pmu.h | 2 +- 10 files changed, 781 insertions(+), 12 deletions(-) create mode 100644 arch/arm64/kvm/gunyah.c diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index e98cfe7855a6..efbfe31d262d 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1417,7 +1417,7 @@ static inline bool kvm_pmu_counter_deferred(struct perf_event_attr *attr) return (!has_vhe() && attr->exclude_host); } -#ifdef CONFIG_KVM +#ifdef CONFIG_KVM_ARM void kvm_set_pmu_events(u64 set, struct perf_event_attr *attr); void kvm_clr_pmu_events(u64 clr); bool kvm_set_pmuserenr(u64 val); diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index ebf4a9f943ed..80ddb409e0cb 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -82,11 +82,18 @@ bool is_kvm_arm_initialised(void); DECLARE_STATIC_KEY_FALSE(kvm_protected_mode_initialized); +#ifndef CONFIG_KVM_ARM +static inline bool is_pkvm_initialized(void) +{ + return false; +} +#else static inline bool is_pkvm_initialized(void) { return IS_ENABLED(CONFIG_KVM) && static_branch_likely(&kvm_protected_mode_initialized); } +#endif /* Reports the availability of HYP mode */ static inline bool is_hyp_mode_available(void) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 9c4d6d552b25..50a251c28a48 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -3548,6 +3548,9 @@ static void verify_sme_features(void) cpacr_restore(cpacr); } +#ifndef CONFIG_KVM_ARM +static void verify_hyp_capabilities(void) { } +#else static void verify_hyp_capabilities(void) { u64 safe_mmfr1, mmfr0, mmfr1; @@ -3578,6 +3581,7 @@ static void verify_hyp_capabilities(void) cpu_die_early(); } } +#endif static void verify_mpam_capabilities(void) { diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 5e3c4b58f279..7688f53b55bd 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -72,7 +72,7 @@ PROVIDE(__pi__data = _data); PROVIDE(__pi___bss_start = __bss_start); PROVIDE(__pi__end = _end); -#ifdef CONFIG_KVM +#ifdef CONFIG_KVM_ARM /* * KVM nVHE code has its own symbol namespace prefixed with __kvm_nvhe_, to diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index 096e45acadb2..eb43eabcf61b 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -17,9 +17,25 @@ menuconfig VIRTUALIZATION if VIRTUALIZATION +###################### Gunyah-based KVM ###################### +menuconfig KVM + bool "Kernel-based Virtual Machine (KVM) under Gunyah" + depends on GUNYAH + select KVM_COMMON + select KVM_GENERIC_MMU_NOTIFIER + select KVM_MMIO + select HAVE_KVM_IRQCHIP + select HAVE_KVM_READONLY_MEM + +###################### Native ARM KVM ###################### +config KVM_ARM + bool + depends on !GUNYAH + menuconfig KVM bool "Kernel-based Virtual Machine (KVM) support" - depends on AS_HAS_ARMV8_4 + depends on !GUNYAH && AS_HAS_ARMV8_4 + select KVM_ARM select KVM_COMMON select KVM_GENERIC_HARDWARE_ENABLING select KVM_GENERIC_MMU_NOTIFIER @@ -43,6 +59,8 @@ menuconfig KVM If unsure, say N. +if KVM_ARM + config NVHE_EL2_DEBUG bool "Debug mode for non-VHE EL2 object" depends on KVM @@ -82,5 +100,5 @@ config PTDUMP_STAGE2_DEBUGFS kernel. If in doubt, say N. - +endif # KVM_ARM endif # VIRTUALIZATION diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile index 209bc76263f1..5b54eb329ce2 100644 --- a/arch/arm64/kvm/Makefile +++ b/arch/arm64/kvm/Makefile @@ -8,12 +8,14 @@ ccflags-y += -I $(src) include $(srctree)/virt/kvm/Makefile.kvm obj-$(CONFIG_KVM) += kvm.o -obj-$(CONFIG_KVM) += hyp/ +obj-$(CONFIG_KVM_ARM) += hyp/ CFLAGS_sys_regs.o += -Wno-override-init CFLAGS_handle_exit.o += -Wno-override-init -kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \ +kvm-$(CONFIG_GUNYAH) += gunyah.o + +kvm-$(CONFIG_KVM_ARM) += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \ inject_fault.o va_layout.o handle_exit.o \ guest.o debug.o reset.o sys_regs.o stacktrace.o \ vgic-sys-reg-v3.o fpsimd.o pkvm.o \ @@ -25,9 +27,11 @@ kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \ vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \ vgic/vgic-its.o vgic/vgic-debug.o vgic/vgic-v3-nested.o -kvm-$(CONFIG_HW_PERF_EVENTS) += pmu-emul.o pmu.o -kvm-$(CONFIG_ARM64_PTR_AUTH) += pauth.o -kvm-$(CONFIG_PTDUMP_STAGE2_DEBUGFS) += ptdump.o +ifeq ($(CONFIG_KVM_ARM),y) + kvm-$(CONFIG_HW_PERF_EVENTS) += pmu-emul.o pmu.o + kvm-$(CONFIG_ARM64_PTR_AUTH) += pauth.o + kvm-$(CONFIG_PTDUMP_STAGE2_DEBUGFS) += ptdump.o +endif always-y := hyp_constants.h hyp-constants.s diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c new file mode 100644 index 000000000000..0095610166ad --- /dev/null +++ b/arch/arm64/kvm/gunyah.c @@ -0,0 +1,736 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * KVM port to Qualcomm's Gunyah Hypervisor + * + * Copyright (C) 2024-2025 Linaro Ltd. + * + * Author: Karim Manaouil + * + */ +#include +#include +#include +#include +#include + +static enum kvm_mode kvm_mode = KVM_MODE_DEFAULT; + +enum kvm_mode kvm_get_mode(void) +{ + return kvm_mode; +} + +const struct _kvm_stats_desc kvm_vm_stats_desc[] = { + KVM_GENERIC_VM_STATS() +}; + +const struct kvm_stats_header kvm_vm_stats_header = { + .name_size = KVM_STATS_NAME_SIZE, + .num_desc = ARRAY_SIZE(kvm_vm_stats_desc), + .id_offset = sizeof(struct kvm_stats_header), + .desc_offset = sizeof(struct kvm_stats_header) + KVM_STATS_NAME_SIZE, + .data_offset = sizeof(struct kvm_stats_header) + KVM_STATS_NAME_SIZE + + sizeof(kvm_vm_stats_desc), +}; + +const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { + KVM_GENERIC_VCPU_STATS(), + STATS_DESC_COUNTER(VCPU, hvc_exit_stat), + STATS_DESC_COUNTER(VCPU, wfe_exit_stat), + STATS_DESC_COUNTER(VCPU, wfi_exit_stat), + STATS_DESC_COUNTER(VCPU, mmio_exit_user), + STATS_DESC_COUNTER(VCPU, mmio_exit_kernel), + STATS_DESC_COUNTER(VCPU, signal_exits), + STATS_DESC_COUNTER(VCPU, exits) +}; + +const struct kvm_stats_header kvm_vcpu_stats_header = { + .name_size = KVM_STATS_NAME_SIZE, + .num_desc = ARRAY_SIZE(kvm_vcpu_stats_desc), + .id_offset = sizeof(struct kvm_stats_header), + .desc_offset = sizeof(struct kvm_stats_header) + KVM_STATS_NAME_SIZE, + .data_offset = sizeof(struct kvm_stats_header) + KVM_STATS_NAME_SIZE + + sizeof(kvm_vcpu_stats_desc), +}; + +static bool core_reg_offset_is_vreg(u64 off) +{ + return off >= KVM_REG_ARM_CORE_REG(fp_regs.vregs) && + off < KVM_REG_ARM_CORE_REG(fp_regs.fpsr); +} + +static u64 core_reg_offset_from_id(u64 id) +{ + return id & ~(KVM_REG_ARCH_MASK | KVM_REG_SIZE_MASK | KVM_REG_ARM_CORE); +} + +static int core_reg_size_from_offset(const struct kvm_vcpu *vcpu, u64 off) +{ + int size; + + switch (off) { + case KVM_REG_ARM_CORE_REG(regs.regs[0]) ... + KVM_REG_ARM_CORE_REG(regs.regs[30]): + case KVM_REG_ARM_CORE_REG(regs.sp): + case KVM_REG_ARM_CORE_REG(regs.pc): + case KVM_REG_ARM_CORE_REG(regs.pstate): + case KVM_REG_ARM_CORE_REG(sp_el1): + case KVM_REG_ARM_CORE_REG(elr_el1): + case KVM_REG_ARM_CORE_REG(spsr[0]) ... + KVM_REG_ARM_CORE_REG(spsr[KVM_NR_SPSR - 1]): + size = sizeof(__u64); + break; + + case KVM_REG_ARM_CORE_REG(fp_regs.vregs[0]) ... + KVM_REG_ARM_CORE_REG(fp_regs.vregs[31]): + size = sizeof(__uint128_t); + break; + + case KVM_REG_ARM_CORE_REG(fp_regs.fpsr): + case KVM_REG_ARM_CORE_REG(fp_regs.fpcr): + size = sizeof(__u32); + break; + + default: + return -EINVAL; + } + + if (!IS_ALIGNED(off, size / sizeof(__u32))) + return -EINVAL; + + /* + * The KVM_REG_ARM64_SVE regs must be used instead of + * KVM_REG_ARM_CORE for accessing the FPSIMD V-registers on + * SVE-enabled vcpus: + */ + if (vcpu_has_sve(vcpu) && core_reg_offset_is_vreg(off)) + return -EINVAL; + + return size; +} + +static void *core_reg_addr(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) +{ + u64 off = core_reg_offset_from_id(reg->id); + int size = core_reg_size_from_offset(vcpu, off); + + if (size < 0) + return NULL; + + if (KVM_REG_SIZE(reg->id) != size) + return NULL; + + switch (off) { + case KVM_REG_ARM_CORE_REG(regs.regs[0]) ... + KVM_REG_ARM_CORE_REG(regs.regs[30]): + off -= KVM_REG_ARM_CORE_REG(regs.regs[0]); + off /= 2; + return &vcpu->arch.ctxt.regs.regs[off]; + + case KVM_REG_ARM_CORE_REG(regs.sp): + return &vcpu->arch.ctxt.regs.sp; + + case KVM_REG_ARM_CORE_REG(regs.pc): + return &vcpu->arch.ctxt.regs.pc; + + case KVM_REG_ARM_CORE_REG(regs.pstate): + return &vcpu->arch.ctxt.regs.pstate; + + case KVM_REG_ARM_CORE_REG(sp_el1): + return __ctxt_sys_reg(&vcpu->arch.ctxt, SP_EL1); + + case KVM_REG_ARM_CORE_REG(elr_el1): + return __ctxt_sys_reg(&vcpu->arch.ctxt, ELR_EL1); + + case KVM_REG_ARM_CORE_REG(spsr[KVM_SPSR_EL1]): + return __ctxt_sys_reg(&vcpu->arch.ctxt, SPSR_EL1); + + case KVM_REG_ARM_CORE_REG(spsr[KVM_SPSR_ABT]): + return &vcpu->arch.ctxt.spsr_abt; + + case KVM_REG_ARM_CORE_REG(spsr[KVM_SPSR_UND]): + return &vcpu->arch.ctxt.spsr_und; + + case KVM_REG_ARM_CORE_REG(spsr[KVM_SPSR_IRQ]): + return &vcpu->arch.ctxt.spsr_irq; + + case KVM_REG_ARM_CORE_REG(spsr[KVM_SPSR_FIQ]): + return &vcpu->arch.ctxt.spsr_fiq; + + case KVM_REG_ARM_CORE_REG(fp_regs.vregs[0]) ... + KVM_REG_ARM_CORE_REG(fp_regs.vregs[31]): + off -= KVM_REG_ARM_CORE_REG(fp_regs.vregs[0]); + off /= 4; + return &vcpu->arch.ctxt.fp_regs.vregs[off]; + + case KVM_REG_ARM_CORE_REG(fp_regs.fpsr): + return &vcpu->arch.ctxt.fp_regs.fpsr; + + case KVM_REG_ARM_CORE_REG(fp_regs.fpcr): + return &vcpu->arch.ctxt.fp_regs.fpcr; + + default: + return NULL; + } +} + +static int get_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) +{ + /* + * Because the kvm_regs structure is a mix of 32, 64 and + * 128bit fields, we index it as if it was a 32bit + * array. Hence below, nr_regs is the number of entries, and + * off the index in the "array". + */ + __u32 __user *uaddr = (__u32 __user *)(unsigned long)reg->addr; + int nr_regs = sizeof(struct kvm_regs) / sizeof(__u32); + void *addr; + u32 off; + + /* Our ID is an index into the kvm_regs struct. */ + off = core_reg_offset_from_id(reg->id); + if (off >= nr_regs || + (off + (KVM_REG_SIZE(reg->id) / sizeof(__u32))) >= nr_regs) + return -ENOENT; + + addr = core_reg_addr(vcpu, reg); + if (!addr) + return -EINVAL; + + if (copy_to_user(uaddr, addr, KVM_REG_SIZE(reg->id))) + return -EFAULT; + + return 0; +} + +static int set_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) +{ + __u32 __user *uaddr = (__u32 __user *)(unsigned long)reg->addr; + int nr_regs = sizeof(struct kvm_regs) / sizeof(__u32); + __uint128_t tmp; + void *valp = &tmp, *addr; + u64 off; + int err = 0; + + /* Our ID is an index into the kvm_regs struct. */ + off = core_reg_offset_from_id(reg->id); + if (off >= nr_regs || + (off + (KVM_REG_SIZE(reg->id) / sizeof(__u32))) >= nr_regs) + return -ENOENT; + + addr = core_reg_addr(vcpu, reg); + if (!addr) + return -EINVAL; + + if (KVM_REG_SIZE(reg->id) > sizeof(tmp)) + return -EINVAL; + + if (copy_from_user(valp, uaddr, KVM_REG_SIZE(reg->id))) { + err = -EFAULT; + goto out; + } + + memcpy(addr, valp, KVM_REG_SIZE(reg->id)); +out: + return err; +} + +static int get_sys_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) +{ + __u32 __user *uaddr = (__u32 __user *)(unsigned long)reg->addr; + u64 dummy_val = 0; + + if (copy_to_user(uaddr, &dummy_val, KVM_REG_SIZE(reg->id))) + return -EFAULT; + + return 0; +} + +static int copy_core_reg_indices(const struct kvm_vcpu *vcpu, + u64 __user *uindices) +{ + unsigned int i; + int n = 0; + + for (i = 0; i < sizeof(struct kvm_regs) / sizeof(__u32); i++) { + u64 reg = KVM_REG_ARM64 | KVM_REG_ARM_CORE | i; + int size = core_reg_size_from_offset(vcpu, i); + + if (size < 0) + continue; + + switch (size) { + case sizeof(__u32): + reg |= KVM_REG_SIZE_U32; + break; + + case sizeof(__u64): + reg |= KVM_REG_SIZE_U64; + break; + + case sizeof(__uint128_t): + reg |= KVM_REG_SIZE_U128; + break; + + default: + WARN_ON(1); + continue; + } + + if (uindices) { + if (put_user(reg, uindices)) + return -EFAULT; + uindices++; + } + + n++; + } + + return n; +} + +static unsigned long num_core_regs(const struct kvm_vcpu *vcpu) +{ + return copy_core_reg_indices(vcpu, NULL); +} + +int kvm_arm_get_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) +{ + /* We currently use nothing arch-specific in upper 32 bits */ + if ((reg->id & ~KVM_REG_SIZE_MASK) >> 32 != KVM_REG_ARM64 >> 32) + return -EINVAL; + + switch (reg->id & KVM_REG_ARM_COPROC_MASK) { + case KVM_REG_ARM_CORE: + return get_core_reg(vcpu, reg); + case KVM_REG_ARM64_SYSREG: + return get_sys_reg(vcpu, reg); + default: + return -ENOENT; + } +} + +int kvm_arm_set_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) +{ + /* We currently use nothing arch-specific in upper 32 bits */ + if ((reg->id & ~KVM_REG_SIZE_MASK) >> 32 != KVM_REG_ARM64 >> 32) + return -EINVAL; + + switch (reg->id & KVM_REG_ARM_COPROC_MASK) { + case KVM_REG_ARM_CORE: + return set_core_reg(vcpu, reg); + default: + return -ENOENT; + } +} + +int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) +{ + return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE; +} + +vm_fault_t kvm_arch_vcpu_fault(struct kvm_vcpu *vcpu, struct vm_fault *vmf) +{ + return VM_FAULT_SIGBUS; +} + +void kvm_arch_create_vm_debugfs(struct kvm *kvm) +{ +} + +void kvm_arch_destroy_vm(struct kvm *kvm) +{ + kvm_destroy_vcpus(kvm); + return; +} + +long kvm_arch_dev_ioctl(struct file *filp, + unsigned int ioctl, unsigned long arg) +{ + return -EINVAL; +} + +bool kvm_arch_irqchip_in_kernel(struct kvm *kvm) +{ + return false; +} + +bool kvm_arch_intc_initialized(struct kvm *kvm) +{ + return true; +} + +struct kvm_vcpu *kvm_arch_vcpu_alloc(void) +{ + return NULL; +} + +int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id) +{ + return 0; +} + +int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) +{ + return -EINVAL; +} + +void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu) +{ +} + +void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu) +{ +} + +void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu) +{ +} + +void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) +{ +} + +void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) +{ +} + +void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) +{ +} + +int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu, + struct kvm_mp_state *mp_state) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, + struct kvm_mp_state *mp_state) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_runnable(struct kvm_vcpu *v) +{ + return 0; +} + +bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu) +{ + return false; +} + +int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) +{ + return -EINVAL; +} + +long kvm_arch_vcpu_ioctl(struct file *filp, + unsigned int ioctl, unsigned long arg) +{ + struct kvm_vcpu *vcpu = filp->private_data; + void __user *argp = (void __user *)arg; + long r; + + switch (ioctl) { + case KVM_ARM_VCPU_INIT: { + struct kvm_vcpu_init init; + + r = -EFAULT; + if (copy_from_user(&init, argp, sizeof(init))) + break; + + vcpu_set_flag(vcpu, VCPU_INITIALIZED); + r = 0; + break; + } + case KVM_SET_ONE_REG: + case KVM_GET_ONE_REG: { + struct kvm_one_reg reg; + + r = -ENOEXEC; + if (unlikely(!kvm_vcpu_initialized(vcpu))) + break; + + r = -EFAULT; + if (copy_from_user(®, argp, sizeof(reg))) + break; + + if (ioctl == KVM_SET_ONE_REG) + r = kvm_arm_set_reg(vcpu, ®); + else + r = kvm_arm_get_reg(vcpu, ®); + break; + } + case KVM_GET_REG_LIST: { + struct kvm_reg_list __user *user_list = argp; + struct kvm_reg_list reg_list; + unsigned n; + + r = -ENOEXEC; + if (unlikely(!kvm_vcpu_initialized(vcpu))) + break; + + r = -EFAULT; + if (copy_from_user(®_list, user_list, sizeof(reg_list))) + break; + + n = reg_list.n; + reg_list.n = num_core_regs(vcpu); + if (copy_to_user(user_list, ®_list, sizeof(reg_list))) + break; + r = -E2BIG; + if (n < reg_list.n) + break; + + r = 0; + copy_core_reg_indices(vcpu, user_list->reg); + break; + } + case KVM_ARM_VCPU_FINALIZE: { + return 0; + } + default: + pr_info("gunyah: %s: unrecognised vcpu ioctl %u\n", __func__, ioctl); + r = -EINVAL; + } + + return r; +} + +int kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_ioctl_set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu, + struct kvm_sregs *sregs) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu, + struct kvm_sregs *sregs) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_ioctl_get_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_ioctl_set_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_ioctl_translate(struct kvm_vcpu *vcpu, + struct kvm_translation *tr) +{ + return -EINVAL; +} + +int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu, + struct kvm_guest_debug *dbg) +{ + return -EINVAL; +} + +void kvm_arch_vcpu_put_debug_state_flags(struct kvm_vcpu *vcpu) +{ +} + +int kvm_vm_ioctl_enable_cap(struct kvm *kvm, + struct kvm_enable_cap *cap) +{ + return -EINVAL; +} + +int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) +{ + void __user *argp = (void __user *)arg; + + switch (ioctl) { + case KVM_ARM_PREFERRED_TARGET: { + struct kvm_vcpu_init init = { + .target = KVM_ARM_TARGET_GENERIC_V8, + }; + + if (copy_to_user(argp, &init, sizeof(init))) + return -EFAULT; + + return 0; + } + case KVM_ARM_SET_COUNTER_OFFSET: { + return -ENXIO; + } + case KVM_ARM_SET_DEVICE_ADDR: { + struct kvm_arm_device_addr dev_addr; + + if (copy_from_user(&dev_addr, argp, sizeof(dev_addr))) + return -EFAULT; + + return -ENODEV; + } + case KVM_HAS_DEVICE_ATTR: { + return -ENXIO; + } + case KVM_SET_DEVICE_ATTR: { + return -ENXIO; + } + case KVM_ARM_GET_REG_WRITABLE_MASKS: { + return -ENXIO; + } + default: + return -EINVAL; + } +} + +int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) +{ + int r; + + switch (ext) { + case KVM_CAP_IOEVENTFD: + case KVM_CAP_USER_MEMORY: + case KVM_CAP_SYNC_MMU: + case KVM_CAP_ONE_REG: + case KVM_CAP_READONLY_MEM: + case KVM_CAP_VCPU_ATTRIBUTES: + case KVM_CAP_ARM_USER_IRQ: + case KVM_CAP_ARM_SET_DEVICE_ADDR: + r = 1; + break; + case KVM_CAP_NR_VCPUS: + /* + * ARM64 treats KVM_CAP_NR_CPUS differently from all other + * architectures, as it does not always bound it to + * KVM_CAP_MAX_VCPUS. It should not matter much because + * this is just an advisory value. + */ + r = min_t(unsigned int, num_online_cpus(), KVM_MAX_VCPUS); + break; + case KVM_CAP_MAX_VCPUS: + case KVM_CAP_MAX_VCPU_ID: + r = KVM_MAX_VCPUS; + break; + default: + r = 0; + } + + return r; +} + +int kvm_vm_ioctl_irq_line(struct kvm *kvm, struct kvm_irq_level *irq_level, + bool line_status) +{ + return -ENXIO; +} + +int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) +{ + return 0; +} + +void kvm_arch_flush_shadow_all(struct kvm *kvm) +{ +} + +int kvm_arch_flush_remote_tlbs(struct kvm *kvm) +{ + return -EINVAL; +} + +int kvm_arch_flush_remote_tlbs_range(struct kvm *kvm, + gfn_t gfn, u64 nr_pages) +{ + return -EINVAL; +} + +void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm, + struct kvm_memory_slot *slot, + gfn_t gfn_offset, unsigned long mask) +{ +} + +void kvm_arch_commit_memory_region(struct kvm *kvm, + struct kvm_memory_slot *old, + const struct kvm_memory_slot *new, + enum kvm_mr_change change) +{ +} + +int kvm_arch_prepare_memory_region(struct kvm *kvm, + const struct kvm_memory_slot *old, + struct kvm_memory_slot *new, + enum kvm_mr_change change) +{ + return 0; +} + +void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *slot) +{ +} + +void kvm_arch_memslots_updated(struct kvm *kvm, u64 gen) +{ +} + +void kvm_arch_flush_shadow_memslot(struct kvm *kvm, + struct kvm_memory_slot *slot) +{ +} + +bool kvm_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range) +{ + return false; +} + + +bool kvm_test_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range) +{ + return false; +} + +bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range) +{ + return false; +} + +int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log) +{ + return -EINVAL; +} + +__init void kvm_compute_layout(void) +{ +} + +__init void kvm_apply_hyp_relocations(void) +{ +} + +void __init kvm_hyp_reserve(void) +{ +} + +void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) +{ +} + +int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu) +{ + return -EINVAL; +} + +struct kvm *kvm_arch_alloc_vm(void) +{ + return NULL; +} diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h index 96754b51b411..575864e93f79 100644 --- a/include/kvm/arm_pmu.h +++ b/include/kvm/arm_pmu.h @@ -12,7 +12,7 @@ #define KVM_ARMV8_PMU_MAX_COUNTERS 32 -#if IS_ENABLED(CONFIG_HW_PERF_EVENTS) && IS_ENABLED(CONFIG_KVM) +#if IS_ENABLED(CONFIG_HW_PERF_EVENTS) && IS_ENABLED(CONFIG_KVM_ARM) struct kvm_pmc { u8 idx; /* index into the pmu->pmc array */ struct perf_event *perf_event; diff --git a/include/linux/irqchip/arm-vgic-info.h b/include/linux/irqchip/arm-vgic-info.h index a75b2c7de69d..7a4a6051ffa6 100644 --- a/include/linux/irqchip/arm-vgic-info.h +++ b/include/linux/irqchip/arm-vgic-info.h @@ -36,7 +36,7 @@ struct gic_kvm_info { bool no_hw_deactivation; }; -#ifdef CONFIG_KVM +#ifdef CONFIG_KVM_ARM void vgic_set_kvm_info(const struct gic_kvm_info *info); #else static inline void vgic_set_kvm_info(const struct gic_kvm_info *info) {} diff --git a/include/linux/perf/arm_pmu.h b/include/linux/perf/arm_pmu.h index 6dc5e0cd76ca..c6cb2db9402a 100644 --- a/include/linux/perf/arm_pmu.h +++ b/include/linux/perf/arm_pmu.h @@ -170,7 +170,7 @@ int arm_pmu_acpi_probe(armpmu_init_fn init_fn); static inline int arm_pmu_acpi_probe(armpmu_init_fn init_fn) { return 0; } #endif -#ifdef CONFIG_KVM +#ifdef CONFIG_KVM_ARM void kvm_host_pmu_init(struct arm_pmu *pmu); #else #define kvm_host_pmu_init(x) do { } while(0) From patchwork Thu Apr 24 14:13:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884044 Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 00CE427F74B for ; Thu, 24 Apr 2025 14:13:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504038; cv=none; b=BMR7L8khbMlKqDoTZD/lNZfmp8N7jFmsNTwokJlAO6tq3QlYor/Dra3ysC6Z+znedvbFsCYMKnSTHKQH9xZ+V4xDDVnrdwsoMr5bbiXUC+SY0yrXV8Eu09ca4xwWnDNXl7DL45nOh1eS566dVvqUYJKl8cVmMktpXBH/2TUHeGc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504038; c=relaxed/simple; bh=pMsi9MMq97g7CjVTvZZzMUKV739tetlHWJOghEinxps=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=d5qbRZ1AqjLu8oTLp9oRFVoGLYhHANJY5zUjdbHFOhXNkPHg+KQZNOXNOnJpExrv0OAqSXchmkuzjJ8FFEZsymhZHe69Bm4VK/DmLZ/Wu7ci2lDYhy2wDpcMqHZSVPtW5779BSY/jffE0QAv4QqnYxmhaYS5MRscSycmQk6ZxIo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=sTk4m9/r; arc=none smtp.client-ip=209.85.221.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="sTk4m9/r" Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-39c0dfba946so793440f8f.3 for ; Thu, 24 Apr 2025 07:13:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504034; x=1746108834; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=S74NKpGFaVYOESsCH+rK4kh5DkFdas6Gq4YwtRp9OLI=; b=sTk4m9/r/YqM6gAkyB9+f+Cf9v2YjMXe9/V7HSO7QbIMep386kIqnkxl+8b5ZgmaMi vq11UPzDeTnTokkrMaJkq1om7bX7cplUE72LWmmRp55xsdwa8xhIOHfkaFCBWupg7bWu r7rCnHfm8rpJMFtqLaSRxYaEy7UlfKA0KEbr52xdM6miVQnel9dDetTl4FMpxMkuR0dM QFswRunRyDAjTnUH1ma3M3jX4DJL1/2E4yj7mgGDCP0h6+LhtvAXMqxx6zZjfdR+TDB3 ymPhFpTwlgSj4rRMQ7FTLfCHMZ2gFbLb8agEcLvCUWlIUGhnsRVx/KzpLAIKXO+7aBGD mElA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504034; x=1746108834; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S74NKpGFaVYOESsCH+rK4kh5DkFdas6Gq4YwtRp9OLI=; b=kdRfHu1gXmtO84B2O+RUxkKKD2LDsF49lvTwGEODLmZSFefpZ50LZQdm3jWeVBQzko CXEOAwQem1h+gM/mS51p0M+F3Rt/hhNroWen3h97W3/5J2vV6lsShOayoiBBzuxnQOLB ZHvtvD4rFYGY06kwSwN4jU4mPClKzjSJxv5oiG6eiaDogcH7sDlIBaIHcZT+Mx612/jk 7+S2O6q4oTXhJKnwnIzkjQAFXuSKqu9+v6xxXksSFKN6UmZtrUMBEn1RCs5Q9Nql1++u 9yf7wlumL+vrxBntc7jBMNgHnMylecqt9M+ez1xE11MRpiNXSMEBNwZAPrn5SWDbmQsI c8YQ== X-Forwarded-Encrypted: i=1; AJvYcCXQ+AKZ6W6ZzOS8nVAEBHq7z34dusW1hgRV0CHguuBDLk6UANIWw4Y8lm10E9+XhhU29vlfLbM84bwRmVgh@vger.kernel.org X-Gm-Message-State: AOJu0YwxhkxOYddYIVYKgbh721t3vn1W8BRlZeblsHpPFD4iiOtbgz0f k0rDu6T9F/h+HQElMOAOLIiMIqW5IRs8SpSWGFENzyoG0mqDmDsPUmJRevJ1a8Q= X-Gm-Gg: ASbGncuYpPQeARljvANnKGVNJWDdNX7fn9L26cYaPy+kyIHq2izRednvBuSev3aBUgh d9GR+4zE0bXIOtdyU5+wvebNf80/X3BJ+8S1HtuUNfgxjWSIm/KKs50+aC42TJV6CDqjym7x5Gy bipGNE3uNUFMfoCmWb6UUchhMkiwNrkJ6ERnIshVWO01qE2zNI1hliiu60C1aEVVWh2Cz2MxItw EuVYa5NgscMtV2rfcAUA9cYgh6an/JBUK5893KSZQs06gYllesUH5lMEPT3+ER/10heJef38k+R ED8uz0zQ6UhEfPdsf276L9/nrMfmCRuPr8bPWrqbPLhvRRtlbrppdmxmS52Oi48RqhAf0YMHSHb GmpdvwuQMrdgoZKHP X-Google-Smtp-Source: AGHT+IHx9bXGbc/XwzdBX2sEvhGOWsNMsxK7elP9ZHfuoFqJa71fFuvenAr2zCrjXuMcvE5wd1GoWA== X-Received: by 2002:a05:6000:290d:b0:391:3b11:d604 with SMTP id ffacd0b85a97d-3a06cfa9d47mr2143588f8f.54.1745504034133; Thu, 24 Apr 2025 07:13:54 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:53 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 07/34] KVM: gunyah: Pin guest memory Date: Thu, 24 Apr 2025 15:13:14 +0100 Message-Id: <20250424141341.841734-8-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Qualcomm's Gunyah hypervisor allows to implement protected VMs, in which private memory given to the guest is no more accessible to the host (an access violation will be raised if the host tries to read/write that memory). In the context of protected VMs (aka confidential computing), the consensus to manage this memory is via guest_memfd. We would like this port to be be based on guest_memfd. However, for this RFC, the port allocates anonymous pages, which are subject to migration and swap out. That will trigger a violation if the memory is private, which is the case for most of the guest's main memory. Since the memory is allocated and given to the guest for possibly an unbounded amount of time, we longterm pin the pages to prevent the kernel from touching and possibly swapping or migrating those pages. In upcoming versions of this port, we intend to move to guest_memfd. Signed-off-by: Karim Manaouil --- arch/arm64/include/asm/kvm_host.h | 3 ++ arch/arm64/kvm/gunyah.c | 68 ++++++++++++++++++++++++++++--- 2 files changed, 66 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index efbfe31d262d..9c8e173fc9c1 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -229,6 +229,9 @@ struct kvm_s2_mmu { }; struct kvm_arch_memory_slot { +#ifdef CONFIG_GUNYAH + struct page **pages; +#endif }; /** diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index 0095610166ad..9c37ab20d7e2 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -660,11 +660,47 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm, { } -void kvm_arch_commit_memory_region(struct kvm *kvm, - struct kvm_memory_slot *old, - const struct kvm_memory_slot *new, - enum kvm_mr_change change) +static int gunyah_pin_user_memory(struct kvm *kvm, struct kvm_memory_slot *memslot) { + unsigned int gup_flags = FOLL_WRITE | FOLL_LONGTERM; + unsigned long start = memslot->userspace_addr; + struct vm_area_struct *vma; + struct page **pages; + int ret; + + if (!memslot->npages) + return 0; + + /* It needs to be a valid VMA-backed region */ + mmap_read_lock(current->mm); + vma = find_vma(current->mm, start); + if (!vma || start < vma->vm_start) { + mmap_read_unlock(current->mm); + return 0; + } + if (!(vma->vm_flags & VM_READ) || !(vma->vm_flags & VM_WRITE)) { + mmap_read_unlock(current->mm); + return 0; + } + mmap_read_unlock(current->mm); + + pages = kvcalloc(memslot->npages, sizeof(*pages), GFP_KERNEL); + if (!pages) + return -ENOMEM; + + ret = pin_user_pages_fast(start, memslot->npages, gup_flags, pages); + if (ret < 0) { + goto err; + } else if (ret != memslot->npages) { + ret = -EIO; + goto err; + } else { + memslot->arch.pages = pages; + return 0; + } +err: + kvfree(pages); + return ret; } int kvm_arch_prepare_memory_region(struct kvm *kvm, @@ -672,11 +708,33 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, struct kvm_memory_slot *new, enum kvm_mr_change change) { - return 0; + int ret; + + switch (change) { + case KVM_MR_CREATE: + ret = gunyah_pin_user_memory(kvm, new); + break; + default: + return 0; + } + return ret; +} + +void kvm_arch_commit_memory_region(struct kvm *kvm, + struct kvm_memory_slot *old, + const struct kvm_memory_slot *new, + enum kvm_mr_change change) +{ } void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *slot) { + if (!slot->arch.pages) + return; + + unpin_user_pages(slot->arch.pages, slot->npages); + + kvfree(slot->arch.pages); } void kvm_arch_memslots_updated(struct kvm *kvm, u64 gen) From patchwork Thu Apr 24 14:13:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884043 Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C767027FD6A for ; Thu, 24 Apr 2025 14:13:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504040; cv=none; b=tgMQEzFzlv7E4ovX1WOPP32FgBSFr0obM5faO8i9UVAoqMW5t2EfyHU7RnZNNivISHVNZtVHXD3jKvh7Bm2t4wXm3uur5zL8qJ097BdSvhVjsOHy5fLuhSFxoRsK7xLOWjoNYpw66uKpRmEcZizf9yqMjXEIOu6VP7TMHa3UekQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504040; c=relaxed/simple; bh=DSNuQ7AXSuhIK9F2t1tEvg2Y4H20TKEQaYmf+C1AL10=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cbVE8GHxhL+QqCBxph0S/8R6fyR9Ue/GHEn7Pndn6Q1vOhMtAgtM6ZPCKj1S70RfZWltp/PJRlfqDlgxTh8Kv1A7UurkmFMkkF9seK3H0nItruyynndZPddMYHo0rc7UUtRsexoXTY2Ijav9jXBdxFWM9Cqq9g4hSgVT8ekcT6g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=C6+9rZe2; arc=none smtp.client-ip=209.85.221.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="C6+9rZe2" Received: by mail-wr1-f41.google.com with SMTP id ffacd0b85a97d-3912d2c89ecso912232f8f.2 for ; Thu, 24 Apr 2025 07:13:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504036; x=1746108836; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ys+r4b/bN1N9MasjKmdqEss1ssV3C7RnEHLMtxbroxU=; b=C6+9rZe2ytaKfy31PFB1wcxiH/emj9kOFZMA/GobiGttgsHV8DN2J7qHDUtLWyOGce BZ6rSf+K57Qgh7g99IxoF1d8LhSoFkNNOwOv0KHH6IBHd36eNHKmENIzmmCPxmxuy2Qm 8OKsThyJvgNswpgkfF+BqG9t1F6VdQcsCVOx5vbJ/FMHIlKtHrQDQFjmeHz4HK2zXwDY 0XAgA9WCObJbYTrt4fxANbZARkUmbaSfoSkV8qAT7wuBwtu7n8YStylxfe2wreoFuOHv oX+JtpLTDBp4PZnRUTtZghzd6v3+YbuUWRhbkZWrxBiwO6bgKLTMLXzfehZrz72T+edk ZiJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504036; x=1746108836; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ys+r4b/bN1N9MasjKmdqEss1ssV3C7RnEHLMtxbroxU=; b=ghy/uG4cDE/NbT0befCm4QxUQZ0ilUliUm0PEGxTYKTBr+yOEua8WkCd9kJM1psiPQ 4zl0xdlXLwVQqMUOSHcewFsMY80ce1FzR7v/NYKRpUEt3N7kZOU35QJe2qntbBdoD5xd jc4goKKi+pZCF8hKzSXS4zesUWjj2ELv2U3WuyYGcbiVWWkGgzIGo6HqfRMyjKd3QYHz lzoVhjfbHhS9g8n3R1hT0mPnRDf6UGFUEHCLhDvSYFcFKqfCMHA4uLPfqlToJ4eni2nz G40ToUhRIUk2zjnEvT1QKECauiaYDVt61U6vSEBQOys4ObNZRInbKF4e6Do5JNq5vBsO wP+g== X-Forwarded-Encrypted: i=1; AJvYcCXGqqd8tvD3QsaawcHWGs42XSNP7w6/UXM0GDq07nl9RP36d+1MpHXuOL2gm+Vop97djQ5T4+LdPy8QszaM@vger.kernel.org X-Gm-Message-State: AOJu0YwnAMo+9t0gOdIey1n9rZZ1Q0FW5r5c2cjm5oJQ7E1VMni1ewxr JddSlJjCz9t2HfNeMuqNK5Yva8UY5fwg/y33Kr/DeoDkVyciycTXP65Q0g5NFoo= X-Gm-Gg: ASbGncu3pX2m8j6T2JZt3SYfc7gvxc2ZP0DXfNZVhex6Pqqzv1rYdJRWqxCtyhqwQME yKcCwsLbaeuTrYzmkZG9tWe/gckou9JdaHZEtM5lgcjOQVZCL+bKc2IQYQV14Ur2pMnMtB6CnIF pFbTgB/QY+cllxi5i7T9Eq7Y7oeioUPdI5fXu4EUpEzXbVx/q4NXyci/HM34nIaQbL0qJM6RFMj GKR2MnzNKv8UaAZ0W35T5nJN5aaoW5Jz2O7KCEdSOxDjBW6zotvBuvBn6qUl5QdW/m1uhUH2Jy7 yo4JnOmd+saVgai/tKN54SOBUzfmix2f2S9ft0FaxL71sk51snGclqIfi+5sT3xtNcDsEL4XMfO 3+K+gHx8VRoQ/HbjQ X-Google-Smtp-Source: AGHT+IEV5wfX7DpJOSMs66ZIGfzUTJLqW6iFySId3MuH6gASyxb/C/SH9jqJ4yi7vysyGZwOnlgesg== X-Received: by 2002:a5d:47cb:0:b0:391:3fa7:bf77 with SMTP id ffacd0b85a97d-3a06cf63697mr2421146f8f.31.1745504035418; Thu, 24 Apr 2025 07:13:55 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:54 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 08/34] docs: gunyah: Introduce Gunyah Hypervisor Date: Thu, 24 Apr 2025 15:13:15 +0100 Message-Id: <20250424141341.841734-9-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman Gunyah is an open-source Type-1 hypervisor developed by Qualcomm. It does not depend on any lower-privileged OS/kernel code for its core functionality. This increases its security and can support a smaller trusted computing based when compared to Type-2 hypervisors. Add documentation describing the Gunyah hypervisor and the main components of the Gunyah hypervisor which are of interest to Linux virtualization development. Signed-off-by: Elliot Berman Reviewed-by: Srivatsa Vaddagiri Signed-off-by: Karim Manaouil --- Documentation/virt/gunyah/index.rst | 135 ++++++++++++++++++++ Documentation/virt/gunyah/message-queue.rst | 68 ++++++++++ Documentation/virt/index.rst | 1 + 3 files changed, 204 insertions(+) create mode 100644 Documentation/virt/gunyah/index.rst create mode 100644 Documentation/virt/gunyah/message-queue.rst diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst new file mode 100644 index 000000000000..fba2c7a11d0f --- /dev/null +++ b/Documentation/virt/gunyah/index.rst @@ -0,0 +1,135 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================= +Gunyah Hypervisor +================= + +.. toctree:: + :maxdepth: 1 + + message-queue + +Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and runs in +a more privileged CPU level (EL2 on Aarch64). It does not depend on a less +privileged operating system for its core functionality. This increases its +security and can support a much smaller trusted computing base than a Type-2 +hypervisor. + +Gunyah is an open source hypervisor. The source repository is available at +https://github.com/quic/gunyah-hypervisor. + +Gunyah provides these following features. + +- Scheduling: + + A scheduler for virtual CPUs (vCPUs) on physical CPUs enables time-sharing + of the CPUs. Gunyah supports two models of scheduling which can coexist on + a running system: + + 1. Hypervisor vCPU scheduling in which Gunyah hypervisor schedules vCPUS on + its own. The default is a real-time priority with round-robin scheduler. + 2. "Proxy" scheduling in which an owner-VM can donate the remainder of its + own vCPU's time slice to an owned-VM's vCPU via a hypercall. + +- Memory Management: + + APIs handling memory, abstracted as objects, limiting direct use of physical + addresses. Memory ownership and usage tracking of all memory under its control. + Memory partitioning between VMs is a fundamental security feature. + +- Interrupt Virtualization: + + Interrupt ownership is tracked and interrupt delivery is directly to the + assigned VM. Gunyah makes use of hardware interrupt virtualization where + possible. + +- Inter-VM Communication: + + There are several different mechanisms provided for communicating between VMs. + + 1. Message queues + 2. Doorbells + 3. Virtio MMIO transport + 4. Shared memory + +- Virtual platform: + + Architectural devices such as interrupt controllers and CPU timers are + directly provided by the hypervisor as well as core virtual platform devices + and system APIs such as ARM PSCI. + +- Device Virtualization: + + Para-virtualization of devices is supported using inter-VM communication and + virtio transport support. Select stage 2 faults by virtual machines that use + proxy-scheduled vCPUs can be handled directly by Linux to provide Type-2 + hypervisor style on-demand paging and/or device emulation. + +Architectures supported +======================= +AArch64 with a GICv3 or GICv4.1 + +Resources and Capabilities +========================== + +Services/resources provided by the Gunyah hypervisor are accessible to a +virtual machine through capabilities. A capability is an access control +token granting the holder a set of permissions to operate on a specific +hypervisor object (conceptually similar to a file-descriptor). +For example, inter-VM communication using Gunyah doorbells and message queues +is performed using hypercalls taking Capability ID arguments for the required +IPC objects. These resources are described in Linux as a struct gunyah_resource. + +Unlike UNIX file descriptors, there is no path-based or similar lookup of +an object to create a new Capability, meaning simpler security analysis. +Creation of a new Capability requires the holding of a set of privileged +Capabilities which are typically never given out by the Resource Manager (RM). + +Gunyah itself provides no APIs for Capability ID discovery. Enumeration of +Capability IDs is provided by RM as a higher level service to VMs. + +Resource Manager +================ + +The Gunyah Resource Manager (RM) is a privileged application VM supporting the +Gunyah Hypervisor. It provides policy enforcement aspects of the virtualization +system. The resource manager can be treated as an extension of the Hypervisor +but is separated to its own partition to ensure that the hypervisor layer itself +remains small and secure and to maintain a separation of policy and mechanism in +the platform. The resource manager runs at arm64 NS-EL1, similar to other +virtual machines. + +Communication with the resource manager from other virtual machines happens as +described in message-queue.rst. Details about the specific messages can be found +in drivers/virt/gunyah/rsc_mgr.c + +:: + + +-------+ +--------+ +--------+ + | RM | | VM_A | | VM_B | + +-.-.-.-+ +---.----+ +---.----+ + | | | | + +-.-.-----------.------------.----+ + | | \==========/ | | + | \========================/ | + | Gunyah | + +---------------------------------+ + +The source for the resource manager is available at +https://github.com/quic/gunyah-resource-manager. + +The resource manager provides the following features: + +- VM lifecycle management: allocating a VM, starting VMs, destruction of VMs +- VM access control policy, including memory sharing and lending +- Interrupt routing configuration +- Forwarding of system-level events (e.g. VM shutdown) to owner VM +- Resource (capability) discovery + +A VM requires boot configuration to establish communication with the resource +manager. This is provided to VMs via a 'hypervisor' device tree node which is +overlaid to the VMs DT by the RM. This node lets guests know they are running +as a Gunyah guest VM, how to communicate with resource manager, and basic +description and capabilities of this VM. See +Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml for a +description of this node. diff --git a/Documentation/virt/gunyah/message-queue.rst b/Documentation/virt/gunyah/message-queue.rst new file mode 100644 index 000000000000..96864708f442 --- /dev/null +++ b/Documentation/virt/gunyah/message-queue.rst @@ -0,0 +1,68 @@ +.. SPDX-License-Identifier: GPL-2.0 + +Message Queues +============== +Message queue is a simple low-capacity IPC channel between two virtual machines. +It is intended for sending small control and configuration messages. Each +message queue is unidirectional and buffered in the hypervisor. A full-duplex +IPC channel requires a pair of queues. + +The size of the queue and the maximum size of the message that can be passed is +fixed at creation of the message queue. Resource manager is presently the only +use case for message queues, and creates messages queues between itself and VMs +with a fixed maximum message size of 240 bytes. Longer messages require a +further protocol on top of the message queue messages themselves. For instance, +communication with the resource manager adds a header field for sending longer +messages which are split into smaller fragments. + +The diagram below shows how message queue works. A typical configuration +involves 2 message queues. Message queue 1 allows VM_A to send messages to VM_B. +Message queue 2 allows VM_B to send messages to VM_A. + +1. VM_A sends a message of up to 240 bytes in length. It makes a hypercall + with the message to request the hypervisor to add the message to + message queue 1's queue. The hypervisor copies memory into the internal + message queue buffer; the memory doesn't need to be shared between + VM_A and VM_B. + +2. Gunyah raises the corresponding interrupt for VM_B (Rx vIRQ) when any of + these happens: + + a. gunyah_msgq_send() has PUSH flag. This is a typical case when the message + queue is being used to implement an RPC-like interface. + b. Explicitly with gunyah_msgq_push hypercall from VM_A. + c. Message queue has reached a threshold depth. Typically, this threshold + depth is the size of the queue (in other words: when queue is full, Rx + vIRQ is raised). + +3. VM_B calls gunyah_msgq_recv() and Gunyah copies message to requested buffer. + +4. Gunyah raises the corresponding interrupt for VM_A (Tx vIRQ) when the message + queue falls below a watermark depth. Typically, this is when the queue is + drained. Note the watermark depth and the threshold depth for the Rx vIRQ are + independent values. Coincidentally, this signal is conceptually similar to + Clear-to-Send. + +For VM_B to send a message to VM_A, the process is identical, except that +hypercalls reference message queue 2's capability ID. The IRQ will be different +for the second message queue. + +:: + + +-------------------+ +-----------------+ +-------------------+ + | VM_A | |Gunyah hypervisor| | VM_B | + | | | | | | + | | | | | | + | | Tx | | | | + | |-------->| | Rx vIRQ | | + |gunyah_msgq_send() | Tx vIRQ |Message queue 1 |-------->|gunyah_msgq_recv() | + | |<------- | | | | + | | | | | | + | | | | | | + | | | | Tx | | + | | Rx vIRQ | |<--------| | + |gunyah_msgq_recv() |<--------|Message queue 2 | Tx vIRQ |gunyah_msgq_send() | + | | | |-------->| | + | | | | | | + | | | | | | + +-------------------+ +-----------------+ +---------------+ diff --git a/Documentation/virt/index.rst b/Documentation/virt/index.rst index 7fb55ae08598..15869ee059b3 100644 --- a/Documentation/virt/index.rst +++ b/Documentation/virt/index.rst @@ -16,6 +16,7 @@ Virtualization Support coco/sev-guest coco/tdx-guest hyperv/index + gunyah/index .. only:: html and subproject From patchwork Thu Apr 24 14:13:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884944 Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3678280A5A for ; Thu, 24 Apr 2025 14:13:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504041; cv=none; b=L2Uyq1IBdriGF1Fi4shfkvkYIlZRrdYI0U3j6Z/yaf+PnFPCwf6OMNH/fCsuuj3JA16uxIRVsHwsxDmPEfy/yocjBFF+cKaYIch7XhJ1aCeRIsxYX0sz5YjwWeeatsgbUz+uT4T/nyJxHVw+295XWaa6ncpGp9spIoPSRfJLCq8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504041; c=relaxed/simple; bh=nHdCN2XoC/vofWX1JvKbFFkxCi8mZFPmuv+AC6Hf5jo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZQpL8TcJqIiCD2J0ZDBmweEl3El0VPeexRYo2U4B/igGbIACsEKrSLHN8/sjPou/0ab6nIpltQSfupTq/IbpjKXKHcE+/NwzL4Vo40SVOt5Bt2SvWmhAjqJNO/40d62Dxxf5Jh0scsW0uAYu6J0WYsRwWVw1bIJHzmon0lxij9c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=ntOOgl9n; arc=none smtp.client-ip=209.85.221.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="ntOOgl9n" Received: by mail-wr1-f54.google.com with SMTP id ffacd0b85a97d-3912fdddf8fso1551362f8f.1 for ; Thu, 24 Apr 2025 07:13:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504037; x=1746108837; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3UEu5ZzgQoIbj2nEACS31q52k6ad3AHbKAZTQ1reR+A=; b=ntOOgl9n/47oroZlPn6W85vBTzu5j6rwgBhEPREb4qe+N2tqnT4YnQz2DuoRLWKqKE OBXxavFaZDwHbMUBy6/cWhwHQ5WvLaNgJ25npQJjfp/ft17XqPt9NoiwflfuD8Zh8kVp rlKDeiU34oH6MboBQ2L0v6yiz5rpUdrX6d4/L7lsJxBJwjw3Go1zmnt2zDqnPO4bE+Ja 3rFMR1TjmrEwKb7z5vPaZ4/B9B8CB5b++pxgnG7+tJ1rSk38I32tp19nAfOLM8peApt2 krnIwGIp5rlXpsAmiWONQf5GB/0WDGINuZKsk39kuKePSkTXs789DL2q9Y66Sl6J2IQW WiEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504037; x=1746108837; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3UEu5ZzgQoIbj2nEACS31q52k6ad3AHbKAZTQ1reR+A=; b=dz6Ogwv4WOCuHSJ4sx2saZVnEikv4uaJCBm6wEz8J5R8t01YvtJ7IPJWvV4upjen6l V1A8xQ1QtUK8cSgeRH3v7Vw7JU8EudlNi8P+VUPTEJs7omCpSttuDGDx+uj9owDj4K3a TUJILlH1ulaO+831CjA/jIX7tJ1CSTY7wF2/2CXUIgZuuYsOpLta1LKmNocO0J1JBAjN dJceMjE7IhNT4i1Fe5VCWtN5DVSKC2dX744fv9g6EGF+4DGNpD1zdLMZ0B3/xmNdzgWx fz7HmGsqs1tEFDVfl1lzAiBF5Jg8IPhKkbUnWTKBstzO3DeVbdlTNAeJdijd+JqSxxcA /PPA== X-Forwarded-Encrypted: i=1; AJvYcCVbu+1cIn86E8OCzkB2H9Bd49/0wFrlyOX8NN9qk+gkc06ynFlqB5QWY+qbWtK0qrJ3O2Ap/TcpZmckqh9e@vger.kernel.org X-Gm-Message-State: AOJu0YwjX3iTocn8iAx9q9ma21LYOmjxjvwem/gpSWsoiak/RamcSgAx I3wj8ddtU9MxCVKHr15n80yCMMVxx+hStDziQEbCcHbxgoWmpYpbj+c88KOgu+A= X-Gm-Gg: ASbGnctjh7sqe95vGcD14TDSY/tIqoeCrlWvuT0DlYB9RXopEwdzCD/y+GW428ZOFbA OHuNWS1wVcgOz23J04ymzl8JzwZK+kxljaev2MNOwwudk4SPzs/pyJvvf0WB2n60z3g6yS9IWVa SVfJY5Jbv2NTZ/eIjyWyIXpRQVQrl98F99/0jEj8EnKVHIEdw+ZAyGJhn7SlWzQIEWSOlfjwpG+ DhP/2vxwnpF8shnBb4M4XfQrfXEIxGxQRysUVP8wcyUYxQwFu1ugmT1JNYDI5hP9ycsVDMKxOQP L4O9bXoEu4uConNTYm1DZHt/1vNCXOXTk7Da8im5CvAVuUauMeC0d7807jCmVqMzvQzZ8HkFVTU Sp6+y+TgqTpIKL2jK X-Google-Smtp-Source: AGHT+IGbR1o2FMJriTxMh0pqGyENTmZYeFEOhlcuDYlWi636WYjhDM2clZgLdfLT+7JHm7l12d/JYg== X-Received: by 2002:a05:6000:22c4:b0:3a0:3d18:285 with SMTP id ffacd0b85a97d-3a06d6dd54emr2218203f8f.25.1745504036794; Thu, 24 Apr 2025 07:13:56 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:56 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 09/34] dt-bindings: Add binding for gunyah hypervisor Date: Thu, 24 Apr 2025 15:13:16 +0100 Message-Id: <20250424141341.841734-10-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman The Gunyah Resource Manager applies a devicetree overlay describing the virtual platform configuration of the guest VM, such as the message queue capability IDs for communicating with the Resource Manager. This information is not otherwise discoverable by a VM: the Gunyah hypervisor core does not provide a direct interface to discover capability IDs nor a way to communicate with RM without having already known the corresponding message queue capability ID. Add the DT bindings that Gunyah adheres for the hypervisor node and message queues. Reviewed-by: Rob Herring Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- .../bindings/firmware/gunyah-hypervisor.yaml | 82 +++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml diff --git a/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml new file mode 100644 index 000000000000..cdeb4885a807 --- /dev/null +++ b/Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml @@ -0,0 +1,82 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/firmware/gunyah-hypervisor.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Gunyah Hypervisor + +maintainers: + - Prakruthi Deepak Heragu + - Elliot Berman + +description: |+ + Gunyah virtual machines use this information to determine the capability IDs + of the message queues used to communicate with the Gunyah Resource Manager. + See also: https://github.com/quic/gunyah-resource-manager/blob/develop/src/vm_creation/dto_construct.c + +properties: + compatible: + const: gunyah-hypervisor + + "#address-cells": + description: Number of cells needed to represent 64-bit capability IDs. + const: 2 + + "#size-cells": + description: must be 0, because capability IDs are not memory address + ranges and do not have a size. + const: 0 + +patternProperties: + "^gunyah-resource-mgr(@.*)?": + type: object + description: + Resource Manager node which is required to communicate to Resource + Manager VM using Gunyah Message Queues. + + properties: + compatible: + const: gunyah-resource-manager + + reg: + items: + - description: Gunyah capability ID of the TX message queue + - description: Gunyah capability ID of the RX message queue + + interrupts: + items: + - description: Interrupt for the TX message queue + - description: Interrupt for the RX message queue + + additionalProperties: false + + required: + - compatible + - reg + - interrupts + +additionalProperties: false + +required: + - compatible + - "#address-cells" + - "#size-cells" + +examples: + - | + #include + + hypervisor { + #address-cells = <2>; + #size-cells = <0>; + compatible = "gunyah-hypervisor"; + + gunyah-resource-mgr@0 { + compatible = "gunyah-resource-manager"; + interrupts = , /* TX allowed IRQ */ + ; /* RX requested IRQ */ + reg = <0x00000000 0x00000000>, /* TX capability ID */ + <0x00000000 0x00000001>; /* RX capability ID */ + }; + }; From patchwork Thu Apr 24 14:13:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884042 Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 043312820B5 for ; Thu, 24 Apr 2025 14:13:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504042; cv=none; b=lIp4gxY14+bGGKcKke8x6IiUf/HD4rfUNSNUSH1NYdX/aXGcRo6xt1zpSXEFwAdkqGrOjzNELfFKHKPQkQ58zYt6JzYFvOjltvrKv8/3CJE3JgQSFZHmPPjluWoGQzEvIGA5RC1oLDh4UavR9gpKD/zu6PymZ5t3OtIItgslG4A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504042; c=relaxed/simple; bh=zpH5RWwn66pK/eMc3LSMNEkV/3yaevWoUA1/kvmrIcs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AiEg1T9W95ZjjjIpmHj8yrdEDLTbxA5voMAWHdq0U+GZmSFx37f8B+M0xNh0p2VE4IVvSHBdL9ku7CHCdWL+yviWXewtfEI3jvPvypcchVWD63OZc/gRd8aZl7oSmDQe71jiKoNi2S8mpJXtTu+xXkmAOmjQaN22Gz1g7Phtvbs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=j9+iKa4W; arc=none smtp.client-ip=209.85.221.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="j9+iKa4W" Received: by mail-wr1-f54.google.com with SMTP id ffacd0b85a97d-39ee623fe64so1026444f8f.1 for ; Thu, 24 Apr 2025 07:13:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504038; x=1746108838; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RusfJk/Wwgs+hfa3WlHRZBdqt7Zg7qYVP8LFIn+weQg=; b=j9+iKa4WCT/mz4U6jOuEjxqBAIM17fNBuPra7vwuEWL0QXoKe6DkDeChGdfSfN2jqZ uCaRbiExMKGx6vkJmOf6uDYYMezLzUdvO3PzNablg6nTHG7sKC5ygO7dxRWP12DQb/Lo ncNWWVX00LNc9y6vT9WTBjPdHkjhCEe1ppVdr+2tyhjPTJf7tIR0lHWwMgINoRHhtrzz KBZh7idT029V7GaVEdLIlUlagIvjMV72pL5JYshgff1oMFthqajbTK4j6rmb1Xy/vP62 O6qovaWNGZ14gRNMMC0hn7jK7GW9RnF132Bax0dN4BK/6ziNhLy/woVUD4piL85of4bM P50g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504038; x=1746108838; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RusfJk/Wwgs+hfa3WlHRZBdqt7Zg7qYVP8LFIn+weQg=; b=OYI4wLPNe1gzSXc7ELUH47LYPRwPGrbhdeUpOFvRPkiahsrRYWulfXYquG7aQhpwqk i+5dQ8TY3/wIjUd3wDzcV8+rxWxYgxqB5CDtTvfxtzbJ6yNuN4SWFhxPpd5ccqFGigcG zzcYegnCJ2xqEDWvKKrDFUS/pKpjcuBb6IxyI1268Qd5XAJijR1y1l5jil8Thi669u3T ZVhK3xuYHnyg4jjP7YkNWQGzC38yDhk6hJ40SB6kditAONPCZnaQd3WkzpB60lyDR1+d coQdGCIbKbeYLhlGoMpzHpu/Q9RYwcA76KYyo8XlIZK728HqsY0TuzCm3VV3fnQHEc0+ jLFg== X-Forwarded-Encrypted: i=1; AJvYcCUa/9D0n017XwTM3TyatHlSBpVAlsCRAxKc0Siy2DQrax4UMa4u55dJ29Nd+k5TZkq/npJLyWLUaOvYNs80@vger.kernel.org X-Gm-Message-State: AOJu0YwXV/ioGBYnkmaQRYgfU4O3sIPRCMiqYiTqN/kJJx/KE5pHLbz9 bxTTeT5xv4WDyC6OFWCQBfCijAHumdfMf6//vKcgmw8rlJoQYEwYQvMC50OdIeg= X-Gm-Gg: ASbGncsDwRPP/dGdQL5ANaYbDLnx5Smjob4BrwLHYf3Bj6ZcTPrsicOT8lGUS1aUQ02 oJONssyP8l9ExqcS/rFTrVkTI4ykVWttE1YhquViVBxfhUZ4ZXdK4qz895p2ofFlOaCqESjkVNI R4Ugsg51OOYLc4hkJFdE6DX4nZM+Kv8DSnZi/6yTl0Eg5pqlew2JoD/BZh3c5PpqoCyMB2Cvjdi DvyoXCwnGPU/V4IpjE3HM6XLxgLh7cdyTgCxcSel/whfDIW4m/ekTMfxqxkxGFv00ACbm2KvNxP PBjcLDSK361xpu0B4EF0RxEVWECluDS8hdlYzQwVitnY+szrJpXOxXC0pTxKayHZ9rwdnv6j744 DUuYAqiSvh+H2XtPgoM+Wbq7ejT8= X-Google-Smtp-Source: AGHT+IFmkqrLYBA6pxT3bgHP40xSeHaZYwIrDUExxYU5PjMxiNNE1p1s+eiHaAutjdes9ZQldE7V0g== X-Received: by 2002:a05:6000:188d:b0:39e:dbee:f644 with SMTP id ffacd0b85a97d-3a06cfa5a93mr2715557f8f.46.1745504038249; Thu, 24 Apr 2025 07:13:58 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:57 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman , Dmitry Baryshkov , Srinivas Kandagatla , Alex Elder Subject: [RFC PATCH 10/34] gunyah: Common types and error codes for Gunyah hypercalls Date: Thu, 24 Apr 2025 15:13:17 +0100 Message-Id: <20250424141341.841734-11-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman Add architecture-independent standard error codes, types, and macros for Gunyah hypercalls. Reviewed-by: Dmitry Baryshkov Reviewed-by: Srinivas Kandagatla Reviewed-by: Alex Elder Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- include/linux/gunyah.h | 106 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100644 include/linux/gunyah.h diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h new file mode 100644 index 000000000000..1eab631a49b6 --- /dev/null +++ b/include/linux/gunyah.h @@ -0,0 +1,106 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _LINUX_GUNYAH_H +#define _LINUX_GUNYAH_H + +#include +#include +#include + +/* Matches resource manager's resource types for VM_GET_HYP_RESOURCES RPC */ +enum gunyah_resource_type { + /* clang-format off */ + GUNYAH_RESOURCE_TYPE_BELL_TX = 0, + GUNYAH_RESOURCE_TYPE_BELL_RX = 1, + GUNYAH_RESOURCE_TYPE_MSGQ_TX = 2, + GUNYAH_RESOURCE_TYPE_MSGQ_RX = 3, + GUNYAH_RESOURCE_TYPE_VCPU = 4, + GUNYAH_RESOURCE_TYPE_MEM_EXTENT = 9, + GUNYAH_RESOURCE_TYPE_ADDR_SPACE = 10, + /* clang-format on */ +}; + +struct gunyah_resource { + enum gunyah_resource_type type; + u64 capid; + unsigned int irq; +}; + +/******************************************************************************/ +/* Common arch-independent definitions for Gunyah hypercalls */ +#define GUNYAH_CAPID_INVAL U64_MAX +#define GUNYAH_VMID_ROOT_VM 0xff + +enum gunyah_error { + /* clang-format off */ + GUNYAH_ERROR_OK = 0, + GUNYAH_ERROR_UNIMPLEMENTED = -1, + GUNYAH_ERROR_RETRY = -2, + + GUNYAH_ERROR_ARG_INVAL = 1, + GUNYAH_ERROR_ARG_SIZE = 2, + GUNYAH_ERROR_ARG_ALIGN = 3, + + GUNYAH_ERROR_NOMEM = 10, + + GUNYAH_ERROR_ADDR_OVFL = 20, + GUNYAH_ERROR_ADDR_UNFL = 21, + GUNYAH_ERROR_ADDR_INVAL = 22, + + GUNYAH_ERROR_DENIED = 30, + GUNYAH_ERROR_BUSY = 31, + GUNYAH_ERROR_IDLE = 32, + + GUNYAH_ERROR_IRQ_BOUND = 40, + GUNYAH_ERROR_IRQ_UNBOUND = 41, + + GUNYAH_ERROR_CSPACE_CAP_NULL = 50, + GUNYAH_ERROR_CSPACE_CAP_REVOKED = 51, + GUNYAH_ERROR_CSPACE_WRONG_OBJ_TYPE = 52, + GUNYAH_ERROR_CSPACE_INSUF_RIGHTS = 53, + GUNYAH_ERROR_CSPACE_FULL = 54, + + GUNYAH_ERROR_MSGQUEUE_EMPTY = 60, + GUNYAH_ERROR_MSGQUEUE_FULL = 61, + /* clang-format on */ +}; + +/** + * gunyah_error_remap() - Remap Gunyah hypervisor errors into a Linux error code + * @gunyah_error: Gunyah hypercall return value + */ +static inline int gunyah_error_remap(enum gunyah_error gunyah_error) +{ + switch (gunyah_error) { + case GUNYAH_ERROR_OK: + return 0; + case GUNYAH_ERROR_NOMEM: + return -ENOMEM; + case GUNYAH_ERROR_DENIED: + case GUNYAH_ERROR_CSPACE_CAP_NULL: + case GUNYAH_ERROR_CSPACE_CAP_REVOKED: + case GUNYAH_ERROR_CSPACE_WRONG_OBJ_TYPE: + case GUNYAH_ERROR_CSPACE_INSUF_RIGHTS: + return -EACCES; + case GUNYAH_ERROR_CSPACE_FULL: + case GUNYAH_ERROR_BUSY: + case GUNYAH_ERROR_IDLE: + return -EBUSY; + case GUNYAH_ERROR_IRQ_BOUND: + case GUNYAH_ERROR_IRQ_UNBOUND: + case GUNYAH_ERROR_MSGQUEUE_FULL: + case GUNYAH_ERROR_MSGQUEUE_EMPTY: + return -EIO; + case GUNYAH_ERROR_UNIMPLEMENTED: + return -EOPNOTSUPP; + case GUNYAH_ERROR_RETRY: + return -EAGAIN; + default: + return -EINVAL; + } +} + +#endif From patchwork Thu Apr 24 14:13:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884943 Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A050128466D for ; Thu, 24 Apr 2025 14:14:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504044; cv=none; b=kzwEam3pVTGyXF9Y8Kl0FAGmnlZZtXjioHHNFlSnNR6m9Ifik/VMbnPApCWixQIET4WIeUn3ItI3MBQ+YX6XsT95CeAJfW3Oa/p35eCsT+m2F6XETsFYyC0DXJCkEMXy9QW1GysEP0/xeUVc5ImGgl5CDQADAvZexVrhEsKn99A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504044; c=relaxed/simple; bh=sW44/p8fc9q82gGRUYbPMkfXFBjgbu9rU1+e99sjorc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ER/Ij5/sTnDMaAnAELyIjeUke/3VyFmUlNN0lAOiC0TQZyFCsPD7xl3NC6uu37eaOILqjAzFj7aggCieducdcn7SENEYxPtUt/DuYtPOWj9FOIVQblDVKXYkRYhQxASsHQ6vGiByWAUo56xkeIaLNOjcQVCazXgCavclYNzvvjQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=qfStsnQM; arc=none smtp.client-ip=209.85.128.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="qfStsnQM" Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-43cf0d787eeso12170435e9.3 for ; Thu, 24 Apr 2025 07:14:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504040; x=1746108840; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hnFOyyVscJj2zQjUVyQj//yqO74ALl7OUDNbU7iCFM0=; b=qfStsnQMn2V6Gkyza+2neBnGTryb06KyiNTsIkI4+wXFmPDOSW7W7vx20DwQvLwbKL MAGsoZPCYp4+XWPKiNC2sYM+V3wbJ/ruiWAHn5ImY61QgFJT8D0aLHhaFTdeem+3xlIj /Ztr/j4cAQOj68cMsA1UXOX+PUIr0MCQqNDJ5Pi14nB/ltSND35KsRchGPiZG7moxYRf 7KOKiXjB3sO7E4AKmSMyszIq9mZOmv7mdM4mLAIk0dBcXf/1k8WUPzvTqsz++p73EL5k OUnzS1reM7AjX55Rd478N8i2ttIRFmcjyZR6d6P2Sft4zq4h+DapfkoqkgMH2B2nnCtt voYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504040; x=1746108840; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hnFOyyVscJj2zQjUVyQj//yqO74ALl7OUDNbU7iCFM0=; b=X6/RXWUB4BbEObshK1AaP0sO88gODDz1X2lIdUsw3qFULg8+qi0uG6Jnz2Z3m1O1Tb 7R9F3R1SA76B4AwcT5LIbbkdrOYirnUaPPuvY45I+y8uYL5/hHMWP/XOw3Y2jo1lROT+ K3D0gKyerqZBKiZWHrAoLjPIOj1hGWI/saAcAcyIp8ad5OwzDyjm2YqZUpftO2HgjoyS 1Y9O6tluEcVaZueEhUSq1t7Rro5zUXlsjc2JJsN5rG3VgyHMOJ9lYc6Lr7GzkfxucagS lqd8H76lq90JN/r2HI5PpE7H3DFIL/1Vb3gtS2qetTW6XeUnR7A1kD3NrmPyYo5D2bCW mbCw== X-Forwarded-Encrypted: i=1; AJvYcCWesoJg3Vcwn8E+LzsA0JtKAgBxu0r2zFKKS/nJkF5wGe90YlDOB8ae5KlDQbmnWpcfD5UFnD5OXxjaVja4@vger.kernel.org X-Gm-Message-State: AOJu0Yx18JXGR61wWGPEeyJJPAXve/Kpk4XHSAQNZMEYHEWsWCjp9u5f 7fCqGxpvzzhyxnhFhm+S9RDZg7AbgFfsgNfWjai7ZclICfCdXtkBQr0T4LSa+Cc= X-Gm-Gg: ASbGncupatThXOY0vN4vFoy9oDL3CTJnz9InZ+4XyC+vdOEiEmWCP2opNQKJ16F9Z6s +1Vevnsk+5Z/m+5P94acV5MHwotYiKBCcOUImphfXEVCUL0IfA3/KksXY8mH+8qRsEEwYngk47K drikQO5RDIbFwiobaMmt7DCK/QsFXQdtxP6HPbnrntnIs4GDWMUqM8O+SuaEdHHh9j8+7YZBZCs HqJniatv2JgXOjB4PaeOBW3e+KYMQij2wexHT163LwZF+aJjwEKd90V3gwG3kXBjGFqD0PwuFlI IE/NKBYRyXgjHljFILjBeNBIImh9sONGdF5bJRTgJ/bJyMVE8ZNmpIVYL51lWfACW/d/M+o3NuH E7Q0Mz5ptrw9O82g1 X-Google-Smtp-Source: AGHT+IFe20I1qXpRONr6or0FTVbxh/3TZQo8fNSyYvn8TOfhWs9pYDOoLZ+ij1siBFtRALlOMdvrUQ== X-Received: by 2002:a5d:648c:0:b0:39c:2669:d7f4 with SMTP id ffacd0b85a97d-3a06cfa3406mr2184828f8f.43.1745504039514; Thu, 24 Apr 2025 07:13:59 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:13:59 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman , Srinivas Kandagatla , Alex Elder Subject: [RFC PATCH 11/34] gunyah: Add hypercalls to identify Gunyah Date: Thu, 24 Apr 2025 15:13:18 +0100 Message-Id: <20250424141341.841734-12-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman Add hypercalls to identify when Linux is running in a virtual machine under Gunyah. There are two calls to help identify Gunyah: 1. gh_hypercall_get_uid() returns a UID when running under a Gunyah hypervisor. 2. gh_hypercall_hyp_identify() returns build information and a set of feature flags that are supported by Gunyah. Reviewed-by: Srinivas Kandagatla Reviewed-by: Alex Elder Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- arch/arm64/Kbuild | 1 + arch/arm64/gunyah/Makefile | 3 ++ arch/arm64/gunyah/gunyah_hypercall.c | 62 ++++++++++++++++++++++++++++ drivers/virt/Kconfig | 2 + drivers/virt/gunyah/Kconfig | 12 ++++++ include/linux/gunyah.h | 38 +++++++++++++++++ 6 files changed, 118 insertions(+) create mode 100644 arch/arm64/gunyah/Makefile create mode 100644 arch/arm64/gunyah/gunyah_hypercall.c create mode 100644 drivers/virt/gunyah/Kconfig diff --git a/arch/arm64/Kbuild b/arch/arm64/Kbuild index 5bfbf7d79c99..e4847ba0e3c9 100644 --- a/arch/arm64/Kbuild +++ b/arch/arm64/Kbuild @@ -3,6 +3,7 @@ obj-y += kernel/ mm/ net/ obj-$(CONFIG_KVM) += kvm/ obj-$(CONFIG_XEN) += xen/ obj-$(subst m,y,$(CONFIG_HYPERV)) += hyperv/ +obj-$(CONFIG_GUNYAH) += gunyah/ obj-$(CONFIG_CRYPTO) += crypto/ # for cleaning diff --git a/arch/arm64/gunyah/Makefile b/arch/arm64/gunyah/Makefile new file mode 100644 index 000000000000..84f1e38cafb1 --- /dev/null +++ b/arch/arm64/gunyah/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_GUNYAH) += gunyah_hypercall.o diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunyah_hypercall.c new file mode 100644 index 000000000000..d44663334f38 --- /dev/null +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -0,0 +1,62 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include + +/* {c1d58fcd-a453-5fdb-9265-ce36673d5f14} */ +static const uuid_t GUNYAH_UUID = UUID_INIT(0xc1d58fcd, 0xa453, 0x5fdb, 0x92, + 0x65, 0xce, 0x36, 0x67, 0x3d, 0x5f, + 0x14); + +bool arch_is_gunyah_guest(void) +{ + struct arm_smccc_res res; + uuid_t uuid; + u32 *up; + + arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID, &res); + + up = (u32 *)&uuid.b[0]; + up[0] = lower_32_bits(res.a0); + up[1] = lower_32_bits(res.a1); + up[2] = lower_32_bits(res.a2); + up[3] = lower_32_bits(res.a3); + + return uuid_equal(&uuid, &GUNYAH_UUID); +} +EXPORT_SYMBOL_GPL(arch_is_gunyah_guest); + +#define GUNYAH_HYPERCALL(fn) \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64, \ + ARM_SMCCC_OWNER_VENDOR_HYP, fn) + +/* clang-format off */ +#define GUNYAH_HYPERCALL_HYP_IDENTIFY GUNYAH_HYPERCALL(0x8000) +/* clang-format on */ + +/** + * gunyah_hypercall_hyp_identify() - Returns build information and feature flags + * supported by Gunyah. + * @hyp_identity: filled by the hypercall with the API info and feature flags. + */ +void gunyah_hypercall_hyp_identify( + struct gunyah_hypercall_hyp_identify_resp *hyp_identity) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_HYP_IDENTIFY, &res); + + hyp_identity->api_info = res.a0; + hyp_identity->flags[0] = res.a1; + hyp_identity->flags[1] = res.a2; + hyp_identity->flags[2] = res.a3; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_hyp_identify); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Hypervisor Hypercalls"); diff --git a/drivers/virt/Kconfig b/drivers/virt/Kconfig index d8c848cf09a6..5a6781405a60 100644 --- a/drivers/virt/Kconfig +++ b/drivers/virt/Kconfig @@ -49,4 +49,6 @@ source "drivers/virt/acrn/Kconfig" source "drivers/virt/coco/Kconfig" +source "drivers/virt/gunyah/Kconfig" + endif diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig new file mode 100644 index 000000000000..6f4c85db80b5 --- /dev/null +++ b/drivers/virt/gunyah/Kconfig @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config GUNYAH + tristate "Gunyah Virtualization drivers" + depends on ARM64 + help + The Gunyah drivers are the helper interfaces that run in a guest VM + such as basic inter-VM IPC and signaling mechanisms, and higher level + services such as memory/device sharing, IRQ sharing, and so on. + + Say Y/M here to enable the drivers needed to interact in a Gunyah + virtual environment. diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 1eab631a49b6..33bcbd22d39f 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -6,9 +6,11 @@ #ifndef _LINUX_GUNYAH_H #define _LINUX_GUNYAH_H +#include #include #include #include +#include /* Matches resource manager's resource types for VM_GET_HYP_RESOURCES RPC */ enum gunyah_resource_type { @@ -103,4 +105,40 @@ static inline int gunyah_error_remap(enum gunyah_error gunyah_error) } } +enum gunyah_api_feature { + /* clang-format off */ + GUNYAH_FEATURE_DOORBELL = 1, + GUNYAH_FEATURE_MSGQUEUE = 2, + GUNYAH_FEATURE_VCPU = 5, + GUNYAH_FEATURE_MEMEXTENT = 6, + /* clang-format on */ +}; + +bool arch_is_gunyah_guest(void); + +#define GUNYAH_API_V1 1 + +/* Other bits reserved for future use and will be zero */ +/* clang-format off */ +#define GUNYAH_API_INFO_API_VERSION_MASK GENMASK_ULL(13, 0) +#define GUNYAH_API_INFO_BIG_ENDIAN BIT_ULL(14) +#define GUNYAH_API_INFO_IS_64BIT BIT_ULL(15) +#define GUNYAH_API_INFO_VARIANT_MASK GENMASK_ULL(63, 56) +/* clang-format on */ + +struct gunyah_hypercall_hyp_identify_resp { + u64 api_info; + u64 flags[3]; +}; + +static inline u16 +gunyah_api_version(const struct gunyah_hypercall_hyp_identify_resp *gunyah_api) +{ + return FIELD_GET(GUNYAH_API_INFO_API_VERSION_MASK, + gunyah_api->api_info); +} + +void gunyah_hypercall_hyp_identify( + struct gunyah_hypercall_hyp_identify_resp *hyp_identity); + #endif From patchwork Thu Apr 24 14:13:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884041 Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CAFE7284B53 for ; Thu, 24 Apr 2025 14:14:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504045; cv=none; b=IscAGrAGviNUYJ2yNkVvo/PSOccFrA/W5NQv0Y9JD3dePlU1hG5qA6R/rHjZlCiEcvXFEuEl8jkbZVtluXtrnZS8wUP9V2QduWPo6jV6aJZZ9UiViOxTbh2Pq/T5DqJPE4jMqTOS/CvZYooguqm9+1TlJd8RuE077yvuYZFYxb0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504045; c=relaxed/simple; bh=sOc+WgCBKAU0O5bB7WH2YBqY9uNwSXL1/wb8/ZYbbhE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EyEH7+4gifGenPY/Yd6oYlQkNJLC7AMm6H5QziGoT+UeQEpPHNkCYN+eDavr5Pe03oyldAK00f9gWian3OuNXbMKeXdeADEl6D9vHWqMfYZWlMe2cKhedCYil/gR6PW4D5wK2Qj/TPBW3uyMjuDSRII0Hs6+K8FzNCz5v82AhC4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=aH33Dmtg; arc=none smtp.client-ip=209.85.128.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="aH33Dmtg" Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-43cfe574976so7691715e9.1 for ; Thu, 24 Apr 2025 07:14:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504041; x=1746108841; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OmKgb2P7k7ka3054x1IngydlF8noVj1FtGkMjW5J68k=; b=aH33DmtgkOHqP2hwtI+u07WChhA2Fulbri6fmZWxNnrjT9saYqe/QqBtRtVXhmy4mf Z5EXAZ1d8kblBZtjPBBu6of5eGkEaIx6RxIM9ZAoY2QreLBNLypaWtkE8SgCw+Wwy+Lq beDcf6VKK7dfLO+SfEVGqoj1kMDTW32GRytELD1r9HRvEyNagr0FTcQEYNenN8TqHsUG LFjBtao9OX7HlFxQGZdHxETlqI+oCLtPuZARLhzLK1ZCNKjga90rDDb1brJbUzGiqGbm T7SnEpgHWa6qYBz/iWUCC88dfhzIHxczzTlYdnXNwhGuJRDXB2Kp3M9orQuiHaRDkKCl QaZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504041; x=1746108841; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OmKgb2P7k7ka3054x1IngydlF8noVj1FtGkMjW5J68k=; b=IL9cDbHCiCtB1N/EmBD+xTnUVUEM7dfn5wtLh830xgjNuPr3NeezmN2yvDLYQ2HsEX 2dYzY34K/cGIjr6eSjqObLXW5nRRc98Jp0UwucxgWmpnpEmoUsv1dLG+4nCzfdl4iJPi /6bhfwBEywJnrZkvhNyc0jQpF0c+YIAlMw4onc2FLH+DnkxNzfmVn/dKu6PGcigkq41/ POQE+KmpcIor4VPYTQxn5erKTROI2bWe7Hv5XlkiU/ETQsem5tHsXVykslXPDboL4nlZ SYkU3JONZmkg/7vyHJg3leUyedrhhCd6yaLLTe1wjJnI/1KVKbAtwFkACFSSWa3zAiUS cwWQ== X-Forwarded-Encrypted: i=1; AJvYcCXBou4wB8oU/mQxEe5UoLRkMXt2iTmteS/pIjOPEYa/K5U/GKhL6IcgAr9NDeKQVkl4OvwCB9J1/CBwqrLi@vger.kernel.org X-Gm-Message-State: AOJu0YwuWdoPgIp4ret3Q+2p925aCaSyda//CApTF1iNrR5tmZ+0rA65 biJNiFtWBBGpPX92p6Lb1lLHMaLBUNYOd8OA8d2nq1QrybqulmV8Rt2S9lG0qns= X-Gm-Gg: ASbGncuLnqVVP+P+NaJABZxEoER83JTijFVSAUNbzWiW7w11HpTmqRVXXS3fCzxWz++ U+mNk000jymmE8+PMrRSia4SJiKneg371JuBBVIK0TguMRR/veFTom58luCGoKUKJFBiVb5GAdF o+/OQ1DVF6ryKCk0//swxJpammkQQC6xgWKBBvCCK65NA2tTIybnSGZuvN1H0Ia1Ka2ahau2LNv 9ovoveWZLrVxq7tSdWsOLFGjs0nzlWDqYtmARsSmXvJ3yIbogoFQhnKBkcM0yWx+CHMYPVSIrnT IqvtEUUvRVf1pi4g1q9DnfTlAcdNSf5nnu94QMT6iIsl2riBzbsP6DywhVRG0Ek57/hCTe570cS PW1E+c/gWcmdMnMpaqSMj3wvpq9s= X-Google-Smtp-Source: AGHT+IGfkZ98OulZ0pJYU5c1ld+aBqoKjFuSpgzRjo7Jvo/4HTkJCAjHkRJEje6F6WXTnu8CxQELEw== X-Received: by 2002:a05:600c:3b0c:b0:43c:e8ba:e166 with SMTP id 5b1f17b1804b1-4409bd83db4mr21261755e9.22.1745504040937; Thu, 24 Apr 2025 07:14:00 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.13.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:00 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 12/34] gunyah: Add hypervisor driver Date: Thu, 24 Apr 2025 15:13:19 +0100 Message-Id: <20250424141341.841734-13-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman Add driver to detect when running under Gunyah. It performs basic identification hypercall and populates the platform bus for resource manager to probe. Signed-off-by: Elliot Berman Reviewed-by: Srivatsa Vaddagiri Signed-off-by: Karim Manaouil --- drivers/virt/Makefile | 1 + drivers/virt/gunyah/Makefile | 3 +++ drivers/virt/gunyah/gunyah.c | 52 ++++++++++++++++++++++++++++++++++++ 3 files changed, 56 insertions(+) create mode 100644 drivers/virt/gunyah/Makefile create mode 100644 drivers/virt/gunyah/gunyah.c diff --git a/drivers/virt/Makefile b/drivers/virt/Makefile index f29901bd7820..ef6a3835d078 100644 --- a/drivers/virt/Makefile +++ b/drivers/virt/Makefile @@ -10,3 +10,4 @@ obj-y += vboxguest/ obj-$(CONFIG_NITRO_ENCLAVES) += nitro_enclaves/ obj-$(CONFIG_ACRN_HSM) += acrn/ obj-y += coco/ +obj-y += gunyah/ diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile new file mode 100644 index 000000000000..34f32110faf9 --- /dev/null +++ b/drivers/virt/gunyah/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_GUNYAH) += gunyah.o diff --git a/drivers/virt/gunyah/gunyah.c b/drivers/virt/gunyah/gunyah.c new file mode 100644 index 000000000000..3e795e3ba881 --- /dev/null +++ b/drivers/virt/gunyah/gunyah.c @@ -0,0 +1,52 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2023-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include + +static int gunyah_probe(struct platform_device *pdev) +{ + struct gunyah_hypercall_hyp_identify_resp gunyah_api; + + if (!arch_is_gunyah_guest()) + return -ENODEV; + + gunyah_hypercall_hyp_identify(&gunyah_api); + + pr_info("Running under Gunyah hypervisor %llx/v%u\n", + FIELD_GET(GUNYAH_API_INFO_VARIANT_MASK, gunyah_api.api_info), + gunyah_api_version(&gunyah_api)); + + /* Might move this out to individual drivers if there's ever an API version bump */ + if (gunyah_api_version(&gunyah_api) != GUNYAH_API_V1) { + pr_info("Unsupported Gunyah version: %u\n", + gunyah_api_version(&gunyah_api)); + return -ENODEV; + } + + return devm_of_platform_populate(&pdev->dev); +} + +static const struct of_device_id gunyah_of_match[] = { + { .compatible = "qcom,gunyah-hypervisor" }, + {} +}; +MODULE_DEVICE_TABLE(of, gunyah_of_match); + +/* clang-format off */ +static struct platform_driver gunyah_driver = { + .probe = gunyah_probe, + .driver = { + .name = "gunyah", + .of_match_table = gunyah_of_match, + } +}; +/* clang-format on */ +module_platform_driver(gunyah_driver); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Driver"); From patchwork Thu Apr 24 14:13:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884942 Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A336288C8A for ; Thu, 24 Apr 2025 14:14:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504046; cv=none; b=K15OB8NXPbCkCX+R2AFXiEOUBFyECzRUNMJU9TyeAREUr1RgxcELYx0TE+S1xgkQCvP+pMwQ3DXt0FPEKePQC7jqcXzFA0zl+7YCSI1qfNX533At/1/XJ4QlwLWwweTJqXLE0bimLJw9MsJ/pOjEFGiLnmXT7Uf2pZ75YQ72C50= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504046; c=relaxed/simple; bh=cRJdZ/BvpG7Nf2HfvkjONlA6iN6G9YZT/kvCa6h8Pqk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=T6yRULJQ7TsQxXKV6ODxkoEUp0ixzfs63rgB6a/X7MlfGgtbZU4Ct+to6vH+2EEFkCWmJ/0N6mYXDoFdgYafYphtbcYUdPcSZDFyt1M+wrmXQNMXIJzP/42gTkCmWjapVPnHEv1Er0Q4shmV+Yc7qW5ZSMHlIJ7pjDRpTbRhmbk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=aAsLp2wi; arc=none smtp.client-ip=209.85.128.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="aAsLp2wi" Received: by mail-wm1-f51.google.com with SMTP id 5b1f17b1804b1-43d2d952eb1so7616575e9.1 for ; Thu, 24 Apr 2025 07:14:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504042; x=1746108842; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Iugp3VZMso7R61paVQ20sor9LWthQ+vE2iyUDGP7wsk=; b=aAsLp2wiL4SnGGfuvk26Iktqh4BJnxFfKcsSxvjPEagGx7fi/GliYmXzZdgxm3FCgY R/qwu/efZXZX7uKjiHZUoXchN/W/kcp5mMR2C5anR2hcIJ3vbCm0XaoUacmQvPV2g4gC HvHEBkPZop2ckBPqpxuors2ZNP2+rwt8rpotniV0U9YiCzMm2V0lXTsv1bEPFgXatILl U/awvhfv2MzQmr0N5FntwhNSvD/lPVBu2z7/NmOrnMklRKIOceLIopPA4Y1gRvEZFHOD 9Kr+mQE/POTKy/7qng9AwhhvaG/MFOECRAnsoxCXf/aZag79pLOhLgzi7sjdzUZ9WG4K /FNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504042; x=1746108842; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Iugp3VZMso7R61paVQ20sor9LWthQ+vE2iyUDGP7wsk=; b=Gt4S4aHZ0LTE83aRMCkUt+sgr/w4wfHisuY/LNmYUu0HPZeMDWMOocB3JMNRbhoXb7 ElSJUDmV+GzJZ01fo80GDNkInfiMHSTdZ24ACFR5FWMcGdGBvxcp7EkHyf3sR3O+QrWY YXThh2nzY2PysczoFqRH+jUeHgY5emMJHtrCmgrIrqBIAqU/pnkXzgAqJ+V6iA7RZz52 HJhr1EzjX6ul9Uarj1r+D/6DfmcSgufKKf6CoY/+OQ8/NO7pTPtopLwm+3NAstzZqjm0 HLYhZ1ssVpDUjRLh1RSBToquoft8dqom7pWqAEWJSmkADv0XFfxaAF/ilM37Yk6Bi9VE dsSQ== X-Forwarded-Encrypted: i=1; AJvYcCXuLoDoG6C9bPDYyTKpry0hP7oL17JJBqFNpV/9WFm5kWSEhcEP7ejxEVtaEogAt0p+VUUdtZ9O2Z56j2Ed@vger.kernel.org X-Gm-Message-State: AOJu0YxGQdxyGoYJkaB40qE3JYg9l/1dgc6Na0GT9mF1P25MS4uGrzDY xoUPpvaj7ppg8muLLe9Ps1i/l7ioU0l0knWGESJGUAbs6N2zDpAOmi4igWMCWMc= X-Gm-Gg: ASbGnct/aIa2aC095YWjp5ZKVt/xacSz/ZxtyCkYPSiuekcD0eDTLyC+UCicMldvIRx ZJW+vaxXetlrhNA8iyiUOX6rsv6M3/FTENkuMto4oxvV93FRa9Iibpe8IGCMs/T6zNw6LUUTrVI 4PRFWVfSF49zvO8QQqqc+oOAqdWzxJNTEqSZ/qHSE/zYzIyGpXyIkvqg8YwOUaZ3q7V9RFdDhzp R1G2NyPXU77WvmwhZugyx32Cw5dFI87PPVl9bMA9TzdN7KTHP8n72qPsMay08EPW4DXdp7ZmtA0 UCHot1x6iWczndDgm9BJNJWaX0OnpZq1BKpiCEV+rjVdAymU5jw6DPgkquxfLiI5/29iwIt7wm3 1gNi0K2ue91p5BNT5 X-Google-Smtp-Source: AGHT+IFb5JrKaNG2KfYhmzfwfaDb/IjykBP33KVSW9og9BEhgmgsW+e69hlvAOQj6I5jNodxA+AwfQ== X-Received: by 2002:a05:600c:c092:b0:43c:e2dd:98ea with SMTP id 5b1f17b1804b1-4409c3b1e1amr23970305e9.22.1745504042259; Thu, 24 Apr 2025 07:14:02 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:01 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman , Alex Elder , Srinivas Kandagatla Subject: [RFC PATCH 13/34] gunyah: Add hypercalls to send and receive messages Date: Thu, 24 Apr 2025 15:13:20 +0100 Message-Id: <20250424141341.841734-14-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman Add hypercalls to send and receive messages on a Gunyah message queue. Reviewed-by: Alex Elder Reviewed-by: Srinivas Kandagatla Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- arch/arm64/gunyah/gunyah_hypercall.c | 55 ++++++++++++++++++++++++++++ include/linux/gunyah.h | 8 ++++ 2 files changed, 63 insertions(+) diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunyah_hypercall.c index d44663334f38..1302e128be6e 100644 --- a/arch/arm64/gunyah/gunyah_hypercall.c +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -37,6 +37,8 @@ EXPORT_SYMBOL_GPL(arch_is_gunyah_guest); /* clang-format off */ #define GUNYAH_HYPERCALL_HYP_IDENTIFY GUNYAH_HYPERCALL(0x8000) +#define GUNYAH_HYPERCALL_MSGQ_SEND GUNYAH_HYPERCALL(0x801B) +#define GUNYAH_HYPERCALL_MSGQ_RECV GUNYAH_HYPERCALL(0x801C) /* clang-format on */ /** @@ -58,5 +60,58 @@ void gunyah_hypercall_hyp_identify( } EXPORT_SYMBOL_GPL(gunyah_hypercall_hyp_identify); +/** + * gunyah_hypercall_msgq_send() - Send a buffer on a message queue + * @capid: capability ID of the message queue to add message + * @size: Size of @buff + * @buff: Address of buffer to send + * @tx_flags: See GUNYAH_HYPERCALL_MSGQ_TX_FLAGS_* + * @ready: If the send was successful, ready is filled with true if more + * messages can be sent on the queue. If false, then the tx IRQ will + * be raised in future when send can succeed. + */ +enum gunyah_error gunyah_hypercall_msgq_send(u64 capid, size_t size, void *buff, + u64 tx_flags, bool *ready) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_MSGQ_SEND, capid, size, + (uintptr_t)buff, tx_flags, 0, &res); + + if (res.a0 == GUNYAH_ERROR_OK) + *ready = !!res.a1; + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_msgq_send); + +/** + * gunyah_hypercall_msgq_recv() - Send a buffer on a message queue + * @capid: capability ID of the message queue to add message + * @buff: Address of buffer to copy received data into + * @size: Size of @buff + * @recv_size: If the receive was successful, recv_size is filled with the + * size of data received. Will be <= size. + * @ready: If the receive was successful, ready is filled with true if more + * messages are ready to be received on the queue. If false, then the + * rx IRQ will be raised in future when recv can succeed. + */ +enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid, void *buff, size_t size, + size_t *recv_size, bool *ready) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_MSGQ_RECV, capid, (uintptr_t)buff, + size, 0, &res); + + if (res.a0 == GUNYAH_ERROR_OK) { + *recv_size = res.a1; + *ready = !!res.a2; + } + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_msgq_recv); + MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("Gunyah Hypervisor Hypercalls"); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 33bcbd22d39f..acd70f982425 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -141,4 +141,12 @@ gunyah_api_version(const struct gunyah_hypercall_hyp_identify_resp *gunyah_api) void gunyah_hypercall_hyp_identify( struct gunyah_hypercall_hyp_identify_resp *hyp_identity); +/* Immediately raise RX vIRQ on receiver VM */ +#define GUNYAH_HYPERCALL_MSGQ_TX_FLAGS_PUSH BIT(0) + +enum gunyah_error gunyah_hypercall_msgq_send(u64 capid, size_t size, void *buff, + u64 tx_flags, bool *ready); +enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid, void *buff, size_t size, + size_t *recv_size, bool *ready); + #endif From patchwork Thu Apr 24 14:13:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884040 Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D300B289368 for ; Thu, 24 Apr 2025 14:14:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504049; cv=none; b=WkAv6T1X56xPKgCjLMoYJHTpuZ0b9AyzYsrNVv0WQGsfhIpHr49vPHA/VJqgCcb/6sb15aPOz8Y/ojamdOjAk0PBbHRH+h2t71g/kYnxiQQdU5B9QU6J3LVZ8B54oZlSmoguDeQwV/OBKCZ65YwjE4rsIIK9XzgaOvy972OvxKc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504049; c=relaxed/simple; bh=m+fPEAqeKx8MI3a+lqdiH9/ulP+VSGYzhW8y82AELSw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GU5az0BY3FLKOXauP+uJVDVyxPm09ORxyBZq11OdaYiLYzwhWNzOLoiZfoqKGDdJabDuXYwQ34lzePcBsbSwRlIthcTtbOW6CpJ+rAJ+xEheMDTnxQW0mrE+XbtfBxOWffJ6Xq8P4w5CBF1vyTfUJTUqsiS+VYYr+rnqjut6yEw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=QSVzhmLG; arc=none smtp.client-ip=209.85.221.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="QSVzhmLG" Received: by mail-wr1-f54.google.com with SMTP id ffacd0b85a97d-39c1ef4acf2so877754f8f.0 for ; Thu, 24 Apr 2025 07:14:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504044; x=1746108844; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+XE4xl5smx9gWubImYnQvAwnhzC9G4S+d/giF3voQDM=; b=QSVzhmLG80Fx0P05gKPq2TWi04hwkFYKyLuhUtoU4bhDSfHBBssAdoNA1EsFtqRqMb krd4u2GshbDYbYBaOEvxEIh/a3rYMWfdSym4kAhgTHT9hbzXRC+cWulnVsKu9l9xe7fN pwfo/P5YhTjdzA6a0BH8jCZEuKy+4NkyumE4WtXIoY6vGSZfVuyfqGJlq7+QqyWpX5aU Vq6y+p72perGggywfEZRwaHYBC3pRyljQ6h/CBeyoONmdVAkrSzBqN1cZzIC35vbYd+t ns1rQajDhaC1yX+Zuowhp4FtD8MgA2+69cQmTHE9dQzuSZ2EqeM+fLsiCI/ov9l77cdD b7lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504044; x=1746108844; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+XE4xl5smx9gWubImYnQvAwnhzC9G4S+d/giF3voQDM=; b=aD8Mw3El/5Uxo0OX35UoDStIqhePDs69zgxJktvn2qfGoH8b4zXa1nGJVYcmTQbpRQ XP5wxVT6sP30h1chfVi1fR93PK1Qy5eRC6wtDRlEsekrxjDt7ctVEmumGlzkCvU/4Jgm eVhUv4IB0i8RyDl6bytEgGW/3f6qSBM4DDdKqnaiAdrx/8pAhcyKbW9/rWn30HU88wBI cMqB4rzmvdCeMOnXyHQKVOxNTzcN6xbKj8PCFtA9/fGEHmJUFLzaSQD3vM9LFfg5IVUY 1IE1UeY/fK5YjyHajQQWtt/Td0cwxESOVus084enDgsm6+TP3iqOFcnrh2V+ZzKiragP 7+lA== X-Forwarded-Encrypted: i=1; AJvYcCX9X1QQyoAaBI2QrengW2gotWvOGvGVZrrDudQ1CT7EemvuV4Fww8edCJ65ZmXW5Ou1v7oYWrgpjDVOgwlx@vger.kernel.org X-Gm-Message-State: AOJu0YwIkxFMvCfHqn2Hjcaad669NzBb8lvrgPQR2rqbNpDixboXyK9a 7Bs49/QQdWe1ujq2n5AAi01DFig/5jWlek6LlvbXmfmVxRg9UUbV+Y/oBIO3w4o= X-Gm-Gg: ASbGncuCLJ1b4CdNqPADrL9ge+eBO548u0Z33ucr+04UrWsnn3tUVQJ+1UBT09RjITw wDw5w+VFSc9O538ewkXELRzbgNehFvvjEtr/1YN/QA03hu5HOmh4+r9CVsiVPpPX15MJiJIYhRW SU4ZwvqgYfcjtc1BeRLZndB8ugxPF8a8CuSvSSF58Bpw+KdaQduNb1RoyJMlRz8e3LVM0K8tzBc TzsgTejtus3OwOw7iRiXLcyp7W4e8l5PPFyhjJWaMP5b7+4RGbdQq/BbqnU+IFh5EBolR6wRMXG /HUuE9pAr4wNLYyYcgUCmbI93xgl7RISxyxOTViK+K0Gv/Bj9qmF0Z6yiPjjHcjnUOONQs0tVa2 CAWh8sVZKlwVQymNy X-Google-Smtp-Source: AGHT+IE3uJouYJ/7M/rrSd1eFvrf/qS5HF+61EQrttb/wpTbAdACHBohOTrf1XeLM5WbZ4ptiBZEjw== X-Received: by 2002:a5d:5985:0:b0:391:2ab1:d4c2 with SMTP id ffacd0b85a97d-3a06cfabac2mr2257224f8f.37.1745504043594; Thu, 24 Apr 2025 07:14:03 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:03 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 14/34] gunyah: Add resource manager RPC core Date: Thu, 24 Apr 2025 15:13:21 +0100 Message-Id: <20250424141341.841734-15-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman The resource manager is a special virtual machine which is always running on a Gunyah system. It provides APIs for creating and destroying VMs, secure memory management, sharing/lending of memory between VMs, and setup of inter-VM communication. Calls to the resource manager are made via message queues. This patch implements the basic probing and RPC mechanism to make those API calls. Request/response calls can be made with gh_rm_call. Drivers can also register to notifications pushed by RM via gh_rm_register_notifier Specific API calls that resource manager supports will be implemented in subsequent patches. Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- drivers/virt/gunyah/Makefile | 4 +- drivers/virt/gunyah/rsc_mgr.c | 724 +++++++++++++++++++++++++++++++++ include/linux/gunyah_rsc_mgr.h | 27 ++ 3 files changed, 754 insertions(+), 1 deletion(-) create mode 100644 drivers/virt/gunyah/rsc_mgr.c create mode 100644 include/linux/gunyah_rsc_mgr.h diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 34f32110faf9..c2308389f551 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,3 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_GUNYAH) += gunyah.o +gunyah_rsc_mgr-y += rsc_mgr.o + +obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr.o diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c new file mode 100644 index 000000000000..75fc86887868 --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr.c @@ -0,0 +1,724 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include + +#include + +/* clang-format off */ +#define RM_RPC_API_VERSION_MASK GENMASK(3, 0) +#define RM_RPC_HEADER_WORDS_MASK GENMASK(7, 4) +#define RM_RPC_API_VERSION FIELD_PREP(RM_RPC_API_VERSION_MASK, 1) +#define RM_RPC_HEADER_WORDS FIELD_PREP(RM_RPC_HEADER_WORDS_MASK, \ + (sizeof(struct gunyah_rm_rpc_hdr) / sizeof(u32))) +#define RM_RPC_API (RM_RPC_API_VERSION | RM_RPC_HEADER_WORDS) + +#define RM_RPC_TYPE_CONTINUATION 0x0 +#define RM_RPC_TYPE_REQUEST 0x1 +#define RM_RPC_TYPE_REPLY 0x2 +#define RM_RPC_TYPE_NOTIF 0x3 +#define RM_RPC_TYPE_MASK GENMASK(1, 0) + +#define GUNYAH_RM_MAX_NUM_FRAGMENTS 62 +#define RM_RPC_FRAGMENTS_MASK GENMASK(7, 2) +/* clang-format on */ + +struct gunyah_rm_rpc_hdr { + u8 api; + u8 type; + __le16 seq; + __le32 msg_id; +} __packed; + +struct gunyah_rm_rpc_reply_hdr { + struct gunyah_rm_rpc_hdr hdr; + __le32 err_code; /* GUNYAH_RM_ERROR_* */ +} __packed; + +#define GUNYAH_RM_MSGQ_MSG_SIZE 240 +#define GUNYAH_RM_PAYLOAD_SIZE \ + (GUNYAH_RM_MSGQ_MSG_SIZE - sizeof(struct gunyah_rm_rpc_hdr)) + +/* RM Error codes */ +enum gunyah_rm_error { + /* clang-format off */ + GUNYAH_RM_ERROR_OK = 0x0, + GUNYAH_RM_ERROR_UNIMPLEMENTED = 0xFFFFFFFF, + GUNYAH_RM_ERROR_NOMEM = 0x1, + GUNYAH_RM_ERROR_NORESOURCE = 0x2, + GUNYAH_RM_ERROR_DENIED = 0x3, + GUNYAH_RM_ERROR_INVALID = 0x4, + GUNYAH_RM_ERROR_BUSY = 0x5, + GUNYAH_RM_ERROR_ARGUMENT_INVALID = 0x6, + GUNYAH_RM_ERROR_HANDLE_INVALID = 0x7, + GUNYAH_RM_ERROR_VALIDATE_FAILED = 0x8, + GUNYAH_RM_ERROR_MAP_FAILED = 0x9, + GUNYAH_RM_ERROR_MEM_INVALID = 0xA, + GUNYAH_RM_ERROR_MEM_INUSE = 0xB, + GUNYAH_RM_ERROR_MEM_RELEASED = 0xC, + GUNYAH_RM_ERROR_VMID_INVALID = 0xD, + GUNYAH_RM_ERROR_LOOKUP_FAILED = 0xE, + GUNYAH_RM_ERROR_IRQ_INVALID = 0xF, + GUNYAH_RM_ERROR_IRQ_INUSE = 0x10, + GUNYAH_RM_ERROR_IRQ_RELEASED = 0x11, + /* clang-format on */ +}; + +/** + * struct gunyah_rm_message - Represents a complete message from resource manager + * @payload: Combined payload of all the fragments (msg headers stripped off). + * @size: Size of the payload received so far. + * @msg_id: Message ID from the header. + * @type: RM_RPC_TYPE_REPLY or RM_RPC_TYPE_NOTIF. + * @num_fragments: total number of fragments expected to be received. + * @fragments_received: fragments received so far. + * @reply: Fields used for request/reply sequences + */ +struct gunyah_rm_message { + void *payload; + size_t size; + u32 msg_id; + u8 type; + + u8 num_fragments; + u8 fragments_received; + + /** + * @ret: Linux return code, there was an error processing message + * @seq: Sequence ID for the main message. + * @rm_error: For request/reply sequences with standard replies + * @seq_done: Signals caller that the RM reply has been received + */ + struct { + int ret; + u16 seq; + enum gunyah_rm_error rm_error; + struct completion seq_done; + } reply; +}; + +/** + * struct gunyah_rm - private data for communicating w/Gunyah resource manager + * @dev: pointer to RM platform device + * @tx_ghrsc: message queue resource to TX to RM + * @rx_ghrsc: message queue resource to RX from RM + * @active_rx_message: ongoing gunyah_rm_message for which we're receiving fragments + * @call_xarray: xarray to allocate & lookup sequence IDs for Request/Response flows + * @next_seq: next ID to allocate (for xa_alloc_cyclic) + * @recv_msg: cached allocation for Rx messages + * @send_msg: cached allocation for Tx messages. Must hold @send_lock to manipulate. + * @send_lock: synchronization to allow only one request to be sent at a time + * @send_ready: completed when we know Tx message queue can take more messages + * @nh: notifier chain for clients interested in RM notification messages + */ +struct gunyah_rm { + struct device *dev; + struct gunyah_resource tx_ghrsc; + struct gunyah_resource rx_ghrsc; + struct gunyah_rm_message *active_rx_message; + + struct xarray call_xarray; + u32 next_seq; + + unsigned char recv_msg[GUNYAH_RM_MSGQ_MSG_SIZE]; + unsigned char send_msg[GUNYAH_RM_MSGQ_MSG_SIZE]; + struct mutex send_lock; + struct completion send_ready; + struct blocking_notifier_head nh; +}; + +/* Global resource manager instance */ +struct gunyah_rm *gunyah_rm; +EXPORT_SYMBOL_GPL(gunyah_rm); + +/** + * gunyah_rm_error_remap() - Remap Gunyah resource manager errors into a Linux error code + * @rm_error: "Standard" return value from Gunyah resource manager + */ +static inline int gunyah_rm_error_remap(enum gunyah_rm_error rm_error) +{ + switch (rm_error) { + case GUNYAH_RM_ERROR_OK: + return 0; + case GUNYAH_RM_ERROR_UNIMPLEMENTED: + return -EOPNOTSUPP; + case GUNYAH_RM_ERROR_NOMEM: + return -ENOMEM; + case GUNYAH_RM_ERROR_NORESOURCE: + return -ENODEV; + case GUNYAH_RM_ERROR_DENIED: + return -EPERM; + case GUNYAH_RM_ERROR_BUSY: + return -EBUSY; + case GUNYAH_RM_ERROR_INVALID: + case GUNYAH_RM_ERROR_ARGUMENT_INVALID: + case GUNYAH_RM_ERROR_HANDLE_INVALID: + case GUNYAH_RM_ERROR_VALIDATE_FAILED: + case GUNYAH_RM_ERROR_MAP_FAILED: + case GUNYAH_RM_ERROR_MEM_INVALID: + case GUNYAH_RM_ERROR_MEM_INUSE: + case GUNYAH_RM_ERROR_MEM_RELEASED: + case GUNYAH_RM_ERROR_VMID_INVALID: + case GUNYAH_RM_ERROR_LOOKUP_FAILED: + case GUNYAH_RM_ERROR_IRQ_INVALID: + case GUNYAH_RM_ERROR_IRQ_INUSE: + case GUNYAH_RM_ERROR_IRQ_RELEASED: + return -EINVAL; + default: + return -EBADMSG; + } +} + +static int gunyah_rm_init_message_payload(struct gunyah_rm_message *message, + const void *msg, size_t hdr_size, + size_t msg_size) +{ + const struct gunyah_rm_rpc_hdr *hdr = msg; + size_t max_buf_size, payload_size; + + if (msg_size < hdr_size) + return -EINVAL; + + payload_size = msg_size - hdr_size; + + message->num_fragments = FIELD_GET(RM_RPC_FRAGMENTS_MASK, hdr->type); + message->fragments_received = 0; + + /* There's not going to be any payload, no need to allocate buffer. */ + if (!payload_size && !message->num_fragments) + return 0; + + if (message->num_fragments > GUNYAH_RM_MAX_NUM_FRAGMENTS) + return -EINVAL; + + max_buf_size = payload_size + + (message->num_fragments * GUNYAH_RM_PAYLOAD_SIZE); + + message->payload = kzalloc(max_buf_size, GFP_KERNEL); + if (!message->payload) + return -ENOMEM; + + memcpy(message->payload, msg + hdr_size, payload_size); + message->size = payload_size; + return 0; +} + +static void gunyah_rm_abort_message(struct gunyah_rm *rm) +{ + kfree(rm->active_rx_message->payload); + + switch (rm->active_rx_message->type) { + case RM_RPC_TYPE_REPLY: + rm->active_rx_message->reply.ret = -EIO; + complete(&rm->active_rx_message->reply.seq_done); + break; + case RM_RPC_TYPE_NOTIF: + fallthrough; + default: + kfree(rm->active_rx_message); + } + + rm->active_rx_message = NULL; +} + +static inline void gunyah_rm_try_complete_message(struct gunyah_rm *rm) +{ + struct gunyah_rm_message *message = rm->active_rx_message; + + if (!message || message->fragments_received != message->num_fragments) + return; + + switch (message->type) { + case RM_RPC_TYPE_REPLY: + complete(&message->reply.seq_done); + break; + case RM_RPC_TYPE_NOTIF: + blocking_notifier_call_chain(&rm->nh, message->msg_id, + message->payload); + + kfree(message->payload); + kfree(message); + break; + default: + dev_err_ratelimited(rm->dev, + "Invalid message type (%u) received\n", + message->type); + gunyah_rm_abort_message(rm); + break; + } + + rm->active_rx_message = NULL; +} + +static void gunyah_rm_process_notif(struct gunyah_rm *rm, const void *msg, + size_t msg_size) +{ + const struct gunyah_rm_rpc_hdr *hdr = msg; + struct gunyah_rm_message *message; + int ret; + + if (rm->active_rx_message) { + dev_err(rm->dev, + "Unexpected new notification, still processing an active message"); + gunyah_rm_abort_message(rm); + } + + message = kzalloc(sizeof(*message), GFP_KERNEL); + if (!message) + return; + + message->type = RM_RPC_TYPE_NOTIF; + message->msg_id = le32_to_cpu(hdr->msg_id); + + ret = gunyah_rm_init_message_payload(message, msg, sizeof(*hdr), + msg_size); + if (ret) { + dev_err(rm->dev, + "Failed to initialize message for notification: %d\n", + ret); + kfree(message); + return; + } + + rm->active_rx_message = message; + + gunyah_rm_try_complete_message(rm); +} + +static void gunyah_rm_process_reply(struct gunyah_rm *rm, const void *msg, + size_t msg_size) +{ + const struct gunyah_rm_rpc_reply_hdr *reply_hdr = msg; + struct gunyah_rm_message *message; + u16 seq_id; + + seq_id = le16_to_cpu(reply_hdr->hdr.seq); + message = xa_load(&rm->call_xarray, seq_id); + + if (!message || message->msg_id != le32_to_cpu(reply_hdr->hdr.msg_id)) + return; + + if (rm->active_rx_message) { + dev_err(rm->dev, + "Unexpected new reply, still processing an active message"); + gunyah_rm_abort_message(rm); + } + + if (gunyah_rm_init_message_payload(message, msg, sizeof(*reply_hdr), + msg_size)) { + dev_err(rm->dev, + "Failed to alloc message buffer for sequence %d\n", + seq_id); + /* Send message complete and error the client. */ + message->reply.ret = -ENOMEM; + complete(&message->reply.seq_done); + return; + } + + message->reply.rm_error = le32_to_cpu(reply_hdr->err_code); + rm->active_rx_message = message; + + gunyah_rm_try_complete_message(rm); +} + +static void gunyah_rm_process_cont(struct gunyah_rm *rm, + struct gunyah_rm_message *message, + const void *msg, size_t msg_size) +{ + const struct gunyah_rm_rpc_hdr *hdr = msg; + size_t payload_size = msg_size - sizeof(*hdr); + + if (!rm->active_rx_message) + return; + + /* + * hdr->fragments and hdr->msg_id preserves the value from first reply + * or notif message. To detect mishandling, check it's still intact. + */ + if (message->msg_id != le32_to_cpu(hdr->msg_id) || + message->num_fragments != + FIELD_GET(RM_RPC_FRAGMENTS_MASK, hdr->type)) { + gunyah_rm_abort_message(rm); + return; + } + + memcpy(message->payload + message->size, msg + sizeof(*hdr), + payload_size); + message->size += payload_size; + message->fragments_received++; + + gunyah_rm_try_complete_message(rm); +} + +static irqreturn_t gunyah_rm_rx(int irq, void *data) +{ + enum gunyah_error gunyah_error; + struct gunyah_rm_rpc_hdr *hdr; + struct gunyah_rm *rm = data; + void *msg = &rm->recv_msg[0]; + size_t len; + bool ready; + + do { + gunyah_error = gunyah_hypercall_msgq_recv(rm->rx_ghrsc.capid, + msg, + sizeof(rm->recv_msg), + &len, &ready); + if (gunyah_error != GUNYAH_ERROR_OK) { + if (gunyah_error != GUNYAH_ERROR_MSGQUEUE_EMPTY) + dev_warn(rm->dev, + "Failed to receive data: %d\n", + gunyah_error); + return IRQ_HANDLED; + } + + if (len < sizeof(*hdr)) { + dev_err_ratelimited( + rm->dev, + "Too small message received. size=%ld\n", len); + continue; + } + + hdr = msg; + if (hdr->api != RM_RPC_API) { + dev_err(rm->dev, "Unknown RM RPC API version: %x\n", + hdr->api); + return IRQ_HANDLED; + } + + switch (FIELD_GET(RM_RPC_TYPE_MASK, hdr->type)) { + case RM_RPC_TYPE_NOTIF: + gunyah_rm_process_notif(rm, msg, len); + break; + case RM_RPC_TYPE_REPLY: + gunyah_rm_process_reply(rm, msg, len); + break; + case RM_RPC_TYPE_CONTINUATION: + gunyah_rm_process_cont(rm, rm->active_rx_message, msg, + len); + break; + default: + dev_err(rm->dev, + "Invalid message type (%lu) received\n", + FIELD_GET(RM_RPC_TYPE_MASK, hdr->type)); + return IRQ_HANDLED; + } + } while (ready); + + return IRQ_HANDLED; +} + +static irqreturn_t gunyah_rm_tx(int irq, void *data) +{ + struct gunyah_rm *rm = data; + + complete(&rm->send_ready); + + return IRQ_HANDLED; +} + +static int gunyah_rm_msgq_send(struct gunyah_rm *rm, size_t size, bool push) +{ + const u64 tx_flags = push ? GUNYAH_HYPERCALL_MSGQ_TX_FLAGS_PUSH : 0; + enum gunyah_error gunyah_error; + void *data = &rm->send_msg[0]; + bool ready; + + lockdep_assert_held(&rm->send_lock); + +again: + wait_for_completion(&rm->send_ready); + gunyah_error = gunyah_hypercall_msgq_send(rm->tx_ghrsc.capid, size, + data, tx_flags, &ready); + + /* Should never happen because Linux properly tracks the ready-state of the msgq */ + if (WARN_ON(gunyah_error == GUNYAH_ERROR_MSGQUEUE_FULL)) + goto again; + + if (ready) + complete(&rm->send_ready); + + return gunyah_error_remap(gunyah_error); +} + +static int gunyah_rm_send_request(struct gunyah_rm *rm, u32 message_id, + const void *req_buf, size_t req_buf_size, + struct gunyah_rm_message *message) +{ + size_t buf_size_remaining = req_buf_size; + const void *req_buf_curr = req_buf; + struct gunyah_rm_rpc_hdr *hdr = + (struct gunyah_rm_rpc_hdr *)&rm->send_msg[0]; + struct gunyah_rm_rpc_hdr hdr_template; + void *payload = hdr + 1; + u32 cont_fragments = 0; + size_t payload_size; + bool push; + int ret; + + if (req_buf_size > + GUNYAH_RM_MAX_NUM_FRAGMENTS * GUNYAH_RM_PAYLOAD_SIZE) { + dev_warn( + rm->dev, + "Limit (%lu bytes) exceeded for the maximum message size: %lu\n", + GUNYAH_RM_MAX_NUM_FRAGMENTS * GUNYAH_RM_PAYLOAD_SIZE, + req_buf_size); + dump_stack(); + return -E2BIG; + } + + if (req_buf_size) + cont_fragments = (req_buf_size - 1) / GUNYAH_RM_PAYLOAD_SIZE; + + hdr_template.api = RM_RPC_API; + hdr_template.type = FIELD_PREP(RM_RPC_TYPE_MASK, RM_RPC_TYPE_REQUEST) | + FIELD_PREP(RM_RPC_FRAGMENTS_MASK, cont_fragments); + hdr_template.seq = cpu_to_le16(message->reply.seq); + hdr_template.msg_id = cpu_to_le32(message_id); + + ret = mutex_lock_interruptible(&rm->send_lock); + if (ret) + return ret; + + do { + *hdr = hdr_template; + + /* Copy payload */ + payload_size = min(buf_size_remaining, GUNYAH_RM_PAYLOAD_SIZE); + memcpy(payload, req_buf_curr, payload_size); + req_buf_curr += payload_size; + buf_size_remaining -= payload_size; + + /* Only the last message should have push flag set */ + push = !buf_size_remaining; + ret = gunyah_rm_msgq_send(rm, sizeof(*hdr) + payload_size, + push); + if (ret) + break; + + hdr_template.type = + FIELD_PREP(RM_RPC_TYPE_MASK, RM_RPC_TYPE_CONTINUATION) | + FIELD_PREP(RM_RPC_FRAGMENTS_MASK, cont_fragments); + } while (buf_size_remaining); + + mutex_unlock(&rm->send_lock); + return ret; +} + +/** + * gunyah_rm_call: Achieve request-response type communication with RPC + * @rm: Pointer to Gunyah resource manager internal data + * @message_id: The RM RPC message-id + * @req_buf: Request buffer that contains the payload + * @req_buf_size: Total size of the payload + * @resp_buf: Pointer to a response buffer + * @resp_buf_size: Size of the response buffer + * + * Make a request to the Resource Manager and wait for reply back. For a successful + * response, the function returns the payload. The size of the payload is set in + * resp_buf_size. The resp_buf must be freed by the caller when 0 is returned + * and resp_buf_size != 0. + * + * req_buf should be not NULL for req_buf_size >0. If req_buf_size == 0, + * req_buf *can* be NULL and no additional payload is sent. + * + * Context: Process context. Will sleep waiting for reply. + * Return: 0 on success. <0 if error. + */ +int gunyah_rm_call(struct gunyah_rm *rm, u32 message_id, const void *req_buf, + size_t req_buf_size, void **resp_buf, size_t *resp_buf_size) +{ + struct gunyah_rm_message message = { 0 }; + u32 seq_id; + int ret; + + /* message_id 0 is reserved. req_buf_size implies req_buf is not NULL */ + if (!rm || !message_id || (!req_buf && req_buf_size)) + return -EINVAL; + + message.type = RM_RPC_TYPE_REPLY; + message.msg_id = message_id; + + message.reply.seq_done = + COMPLETION_INITIALIZER_ONSTACK(message.reply.seq_done); + + /* Allocate a new seq number for this message */ + ret = xa_alloc_cyclic(&rm->call_xarray, &seq_id, &message, xa_limit_16b, + &rm->next_seq, GFP_KERNEL); + if (ret < 0) + return ret; + message.reply.seq = lower_16_bits(seq_id); + + /* Send the request to the Resource Manager */ + ret = gunyah_rm_send_request(rm, message_id, req_buf, req_buf_size, + &message); + if (ret < 0) { + dev_warn(rm->dev, "Failed to send request. Error: %d\n", ret); + goto out; + } + + /* + * Wait for response. Uninterruptible because rollback based on what RM did to VM + * requires us to know how RM handled the call. + */ + wait_for_completion(&message.reply.seq_done); + + /* Check for internal (kernel) error waiting for the response */ + if (message.reply.ret) { + ret = message.reply.ret; + goto out; + } + + /* Got a response, did resource manager give us an error? */ + if (message.reply.rm_error != GUNYAH_RM_ERROR_OK) { + dev_warn(rm->dev, "RM rejected message %08x. Error: %d\n", + message_id, message.reply.rm_error); + ret = gunyah_rm_error_remap(message.reply.rm_error); + kfree(message.payload); + goto out; + } + + /* Everything looks good, return the payload */ + if (resp_buf_size) + *resp_buf_size = message.size; + + if (message.size && resp_buf) { + *resp_buf = message.payload; + } else { + /* kfree in case RM sent us multiple fragments but never any data in + * those fragments. We would've allocated memory for it, but message.size == 0 + */ + kfree(message.payload); + } + +out: + xa_erase(&rm->call_xarray, message.reply.seq); + return ret; +} + +int gunyah_rm_notifier_register(struct gunyah_rm *rm, struct notifier_block *nb) +{ + return blocking_notifier_chain_register(&rm->nh, nb); +} +EXPORT_SYMBOL_GPL(gunyah_rm_notifier_register); + +int gunyah_rm_notifier_unregister(struct gunyah_rm *rm, + struct notifier_block *nb) +{ + return blocking_notifier_chain_unregister(&rm->nh, nb); +} +EXPORT_SYMBOL_GPL(gunyah_rm_notifier_unregister); + +static int gunyah_platform_probe_capability(struct platform_device *pdev, + int idx, + struct gunyah_resource *ghrsc) +{ + int ret; + + ghrsc->irq = platform_get_irq(pdev, idx); + if (ghrsc->irq < 0) { + dev_err(&pdev->dev, "Failed to get %s irq: %d\n", + idx ? "rx" : "tx", ghrsc->irq); + return ghrsc->irq; + } + + ret = of_property_read_u64_index(pdev->dev.of_node, "reg", idx, + &ghrsc->capid); + if (ret) { + dev_err(&pdev->dev, "Failed to get %s capid: %d\n", + idx ? "rx" : "tx", ret); + return ret; + } + + return 0; +} + +static int gunyah_rm_probe_tx_msgq(struct gunyah_rm *rm, + struct platform_device *pdev) +{ + int ret; + + rm->tx_ghrsc.type = GUNYAH_RESOURCE_TYPE_MSGQ_TX; + ret = gunyah_platform_probe_capability(pdev, 0, &rm->tx_ghrsc); + if (ret) + return ret; + + enable_irq_wake(rm->tx_ghrsc.irq); + + return devm_request_irq(rm->dev, rm->tx_ghrsc.irq, gunyah_rm_tx, 0, + "gunyah_rm_tx", rm); +} + +static int gunyah_rm_probe_rx_msgq(struct gunyah_rm *rm, + struct platform_device *pdev) +{ + int ret; + + rm->rx_ghrsc.type = GUNYAH_RESOURCE_TYPE_MSGQ_RX; + ret = gunyah_platform_probe_capability(pdev, 1, &rm->rx_ghrsc); + if (ret) + return ret; + + enable_irq_wake(rm->rx_ghrsc.irq); + + return devm_request_threaded_irq(rm->dev, rm->rx_ghrsc.irq, NULL, + gunyah_rm_rx, IRQF_ONESHOT, + "gunyah_rm_rx", rm); +} + +static int gunyah_rm_probe(struct platform_device *pdev) +{ + int ret; + + gunyah_rm = devm_kzalloc(&pdev->dev, sizeof(*gunyah_rm), GFP_KERNEL); + if (!gunyah_rm) + return -ENOMEM; + + platform_set_drvdata(pdev, gunyah_rm); + gunyah_rm->dev = &pdev->dev; + + mutex_init(&gunyah_rm->send_lock); + init_completion(&gunyah_rm->send_ready); + BLOCKING_INIT_NOTIFIER_HEAD(&gunyah_rm->nh); + xa_init_flags(&gunyah_rm->call_xarray, XA_FLAGS_ALLOC); + + device_init_wakeup(&pdev->dev, true); + + ret = gunyah_rm_probe_tx_msgq(gunyah_rm, pdev); + if (ret) + return ret; + /* assume RM is ready to receive messages from us */ + complete(&gunyah_rm->send_ready); + + ret = gunyah_rm_probe_rx_msgq(gunyah_rm, pdev); + if (ret) + return ret; + + return 0; +} + +static const struct of_device_id gunyah_rm_of_match[] = { + { .compatible = "gunyah-resource-manager" }, + {} +}; +MODULE_DEVICE_TABLE(of, gunyah_rm_of_match); + +static struct platform_driver gunyah_rm_driver = { + .probe = gunyah_rm_probe, + .driver = { + .name = "gunyah_rsc_mgr", + .of_match_table = gunyah_rm_of_match, + }, +}; +module_platform_driver(gunyah_rm_driver); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Resource Manager Driver"); diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h new file mode 100644 index 000000000000..87e919cc1e28 --- /dev/null +++ b/include/linux/gunyah_rsc_mgr.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#ifndef __GUNYAH_RSC_MGR_H +#define __GUNYAH_RSC_MGR_H + +#include +#include + +#define GUNYAH_VMID_INVAL U16_MAX + +struct gunyah_rm; + +extern struct gunyah_rm *gunyah_rm; + +int gunyah_rm_notifier_register(struct gunyah_rm *rm, + struct notifier_block *nb); +int gunyah_rm_notifier_unregister(struct gunyah_rm *rm, + struct notifier_block *nb); + + +int gunyah_rm_call(struct gunyah_rm *rsc_mgr, u32 message_id, + const void *req_buf, size_t req_buf_size, void **resp_buf, + size_t *resp_buf_size); + +#endif From patchwork Thu Apr 24 14:13:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884941 Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E250E28A1E7 for ; Thu, 24 Apr 2025 14:14:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504050; cv=none; b=g47QXF/N93RHTWpmzMMSClUBRr2zZ5rzmsuyanktm+Szb4ceu0zWPqr4yO4fkA74BMbsaFhAPgnRuxhxLdw/XpxR3QHZKvXYxFlTJYhcjSSJ1brwh+qA1AoDyuNJGg4e8/BlzSHVVmLy5OMo4iJZ343hsBcw4qei4ONdxpz/r2U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504050; c=relaxed/simple; bh=r3nd9vghu4s4uBkV+N+7fBpy1dkBUZqwpUsMh9tj/Uw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fMxo2pE52rA9FZFRAcQYg9STwwn6j3gBGmFoVQUqECLZuCXxihl1kj4QvpKGZPbiLyUgCvGrsafYyoihGpH3/B2KVNifUPWvdOOKDTVZoDdxxnk7bW/hU9iqnij7Bzd/TOoi1iHQxq1Ldq35G3/q/Hwts/NAigTk6/57U59FE8g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=VQsAJJKu; arc=none smtp.client-ip=209.85.128.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="VQsAJJKu" Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-43ed8d32a95so9462445e9.3 for ; Thu, 24 Apr 2025 07:14:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504045; x=1746108845; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XrxJtuJG442F2RUwTFuDHGt6qjNm8WZW6h3yhfXqCrI=; b=VQsAJJKu0h2qk9A9xzNKMZK6r7K8M27J7ZmUtQbNAP1wZYFB/nBWgwA/ggE5n+Qd0I TAXhGrBlm1ThQgmwgmbNAZv+Yo9Ay/NQMbTr8OY4Nptw/Bewli5OGnagneHTc/Wjotm9 AJn7JH7o4LC3fD6XeYIMbpPwyxx8UCMxwPcHCgpQXXD+BNlzCmkt/71/U4tNSiUyG4xk Rll0CsbyHj5CQEjj6X4hZHyoFYjVUJzJBfOjvNIasG3FXHLXEcGxcewR++AvyQPTVTDP qHEIzbgBpFkjq7gi12n50ADGWVrJWD9RwThMWmj+qfGF3o2bPCldXWF5lBMrjjtVuQYm QUgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504045; x=1746108845; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XrxJtuJG442F2RUwTFuDHGt6qjNm8WZW6h3yhfXqCrI=; b=oZ7xR2DMXJsJWS4HQZx1iJ9paIa65b+14W5QPdR4Z2kUWPIktlmRbnPjIfk2LF8YmV 94KcJLqT3G8XyPRA1l7LA50SXWKJKhrWJq9ewSmrQBvAnFlpNNMj/iHBQQx0wXs8arzZ 7Km5vAP9ndT/lSJSEvSAa+XQwgIT2UZpgxzYUtYv122QNkRBiWbogFvvChSZ3Eacf+s7 LJJZNXiDVsBVQA6ZHlfWIH0R5DAcAXxoHa5IVzqn6UBdZjx+5hQD3Nsdw8b41R3T1FJ+ l/jYnwcKtdD+G6pmHKwNn0PR1Hi0rGvFEoXqNOibpKOtY+dypsHVn8/sd35+efFU09fJ jlBQ== X-Forwarded-Encrypted: i=1; AJvYcCW/RQxUsk7dXjIqBUSgKvHkYPzFmlU63wtPbIY8FIq2IVyT2NDxxdXDoHtr6FFwoI87DkD2Hhs7q1QOF0Qz@vger.kernel.org X-Gm-Message-State: AOJu0YyFsjDQyxDSVhxJOmJbCdhyylB3oJBdpB0I0FOgxh1xQqm7NkmR R3d0YWg/7NbBVjQ/iheux7ykvTatg3WXpyMntAxmKzCXM/IFJmD59Ix4bCYAe7LlEc1onjaC2WE G X-Gm-Gg: ASbGnctuwqwLNc8Id6Ghr5fG3X65spdjFvear2x3mL1082R7eBqz7uwLxj3elSf1CQK aPRnarwRWuROmnTTapTjdfHWv09pbnHTJCEDOILeW1ADCW1MeMaWUipdlO/HFQCMxbFrLewnXg5 qeJYz9XAFRonBDGiyEhG5uiBucEE2Dl+aVfPvvYGF14HBfeonJYuYgIHqW0Dg8GqDafKr0R7KBD 2URZO3fAZKXvhhSUaAMsrunhtNCyaopfMOzOfqBJRQyvpLYbTX2c2HC6zspF8N20fU5eB2ZGKRz 9sX7o9llcUrWXbYBOqC7OVB0LjTCm4j8+nR114PRHJZ4OKq1dGQYTguNSsY2CWI4meH4sGgmpOH rTa7eKSkAH09GTZGg X-Google-Smtp-Source: AGHT+IF2yQw22hjehK9elALveIYUJReJwgCD95mE9PZgbazIybojHN298a9GDoCEMK47GVkaXYd77A== X-Received: by 2002:a05:600c:4e48:b0:439:8c80:6af4 with SMTP id 5b1f17b1804b1-4409bd23f55mr24505185e9.19.1745504045045; Thu, 24 Apr 2025 07:14:05 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:04 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 15/34] gunyah: Add VM lifecycle RPC Date: Thu, 24 Apr 2025 15:13:22 +0100 Message-Id: <20250424141341.841734-16-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman Add Gunyah Resource Manager RPC interfaces to launch an unauthenticated virtual machine. Signed-off-by: Elliot Berman Reviewed-by: Srivatsa Vaddagiri Signed-off-by: Karim Manaouil --- drivers/virt/gunyah/Makefile | 2 +- drivers/virt/gunyah/rsc_mgr_rpc.c | 252 ++++++++++++++++++++++++++++++ include/linux/gunyah_rsc_mgr.h | 79 +++++++++- 3 files changed, 331 insertions(+), 2 deletions(-) create mode 100644 drivers/virt/gunyah/rsc_mgr_rpc.c diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index c2308389f551..b1bdf3e84155 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -gunyah_rsc_mgr-y += rsc_mgr.o +gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr.o diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mgr_rpc.c new file mode 100644 index 000000000000..626ad2565548 --- /dev/null +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -0,0 +1,252 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include + +#include + +/* Message IDs: VM Management */ +/* clang-format off */ +#define GUNYAH_RM_RPC_VM_ALLOC_VMID 0x56000001 +#define GUNYAH_RM_RPC_VM_DEALLOC_VMID 0x56000002 +#define GUNYAH_RM_RPC_VM_START 0x56000004 +#define GUNYAH_RM_RPC_VM_STOP 0x56000005 +#define GUNYAH_RM_RPC_VM_RESET 0x56000006 +#define GUNYAH_RM_RPC_VM_CONFIG_IMAGE 0x56000009 +#define GUNYAH_RM_RPC_VM_INIT 0x5600000B +#define GUNYAH_RM_RPC_VM_GET_HYP_RESOURCES 0x56000020 +#define GUNYAH_RM_RPC_VM_GET_VMID 0x56000024 +#define GUNYAH_RM_RPC_VM_SET_BOOT_CONTEXT 0x56000031 +#define GUNYAH_RM_RPC_VM_SET_DEMAND_PAGING 0x56000033 +#define GUNYAH_RM_RPC_VM_SET_ADDRESS_LAYOUT 0x56000034 +/* clang-format on */ + +struct gunyah_rm_vm_common_vmid_req { + __le16 vmid; + __le16 _padding; +} __packed; + +/* Call: VM_ALLOC */ +struct gunyah_rm_vm_alloc_vmid_resp { + __le16 vmid; + __le16 _padding; +} __packed; + +/* Call: VM_STOP */ +#define GUNYAH_RM_VM_STOP_FLAG_FORCE_STOP BIT(0) + +#define GUNYAH_RM_VM_STOP_REASON_FORCE_STOP 3 + +struct gunyah_rm_vm_stop_req { + __le16 vmid; + u8 flags; + u8 _padding; + __le32 stop_reason; +} __packed; + +/* Call: VM_CONFIG_IMAGE */ +struct gunyah_rm_vm_config_image_req { + __le16 vmid; + __le16 auth_mech; + __le32 mem_handle; + __le64 image_offset; + __le64 image_size; + __le64 dtb_offset; + __le64 dtb_size; +} __packed; + +/* + * Several RM calls take only a VMID as a parameter and give only standard + * response back. Deduplicate boilerplate code by using this common call. + */ +static int gunyah_rm_common_vmid_call(struct gunyah_rm *rm, u32 message_id, + u16 vmid) +{ + struct gunyah_rm_vm_common_vmid_req req_payload = { + .vmid = cpu_to_le16(vmid), + }; + + return gunyah_rm_call(rm, message_id, &req_payload, sizeof(req_payload), + NULL, NULL); +} + +/** + * gunyah_rm_alloc_vmid() - Allocate a new VM in Gunyah. Returns the VM identifier. + * @rm: Handle to a Gunyah resource manager + * @vmid: Use 0 to dynamically allocate a VM. A reserved VMID can be supplied + * to request allocation of a platform-defined VM. + * + * Return: the allocated VMID or negative value on error + */ +int gunyah_rm_alloc_vmid(struct gunyah_rm *rm, u16 vmid) +{ + struct gunyah_rm_vm_common_vmid_req req_payload = { + .vmid = cpu_to_le16(vmid), + }; + struct gunyah_rm_vm_alloc_vmid_resp *resp_payload; + size_t resp_size; + void *resp; + int ret; + + ret = gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_ALLOC_VMID, &req_payload, + sizeof(req_payload), &resp, &resp_size); + if (ret) + return ret; + + if (!vmid) { + resp_payload = resp; + ret = le16_to_cpu(resp_payload->vmid); + kfree(resp); + } + + return ret; +} +ALLOW_ERROR_INJECTION(gunyah_rm_alloc_vmid, ERRNO); + +/** + * gunyah_rm_dealloc_vmid() - Dispose of a VMID + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gunyah_rm_alloc_vmid + */ +int gunyah_rm_dealloc_vmid(struct gunyah_rm *rm, u16 vmid) +{ + return gunyah_rm_common_vmid_call(rm, GUNYAH_RM_RPC_VM_DEALLOC_VMID, + vmid); +} +ALLOW_ERROR_INJECTION(gunyah_rm_dealloc_vmid, ERRNO); + +/** + * gunyah_rm_vm_reset() - Reset a VM's resources + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gunyah_rm_alloc_vmid + * + * As part of tearing down the VM, request RM to clean up all the VM resources + * associated with the VM. Only after this, Linux can clean up all the + * references it maintains to resources. + */ +int gunyah_rm_vm_reset(struct gunyah_rm *rm, u16 vmid) +{ + return gunyah_rm_common_vmid_call(rm, GUNYAH_RM_RPC_VM_RESET, vmid); +} +ALLOW_ERROR_INJECTION(gunyah_rm_vm_reset, ERRNO); + +/** + * gunyah_rm_vm_start() - Move a VM into "ready to run" state + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gunyah_rm_alloc_vmid + * + * On VMs which use proxy scheduling, vcpu_run is needed to actually run the VM. + * On VMs which use Gunyah's scheduling, the vCPUs start executing in accordance with Gunyah + * scheduling policies. + */ +int gunyah_rm_vm_start(struct gunyah_rm *rm, u16 vmid) +{ + return gunyah_rm_common_vmid_call(rm, GUNYAH_RM_RPC_VM_START, vmid); +} +ALLOW_ERROR_INJECTION(gunyah_rm_vm_start, ERRNO); + +/** + * gunyah_rm_vm_stop() - Send a request to Resource Manager VM to forcibly stop a VM. + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gunyah_rm_alloc_vmid + */ +int gunyah_rm_vm_stop(struct gunyah_rm *rm, u16 vmid) +{ + struct gunyah_rm_vm_stop_req req_payload = { + .vmid = cpu_to_le16(vmid), + .flags = GUNYAH_RM_VM_STOP_FLAG_FORCE_STOP, + .stop_reason = cpu_to_le32(GUNYAH_RM_VM_STOP_REASON_FORCE_STOP), + }; + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_STOP, &req_payload, + sizeof(req_payload), NULL, NULL); +} +ALLOW_ERROR_INJECTION(gunyah_rm_vm_stop, ERRNO); + +/** + * gunyah_rm_vm_configure() - Prepare a VM to start and provide the common + * configuration needed by RM to configure a VM + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier allocated with gunyah_rm_alloc_vmid + * @auth_mechanism: Authentication mechanism used by resource manager to verify + * the virtual machine + * @mem_handle: Handle to a previously shared memparcel that contains all parts + * of the VM image subject to authentication. + * @image_offset: Start address of VM image, relative to the start of memparcel + * @image_size: Size of the VM image + * @dtb_offset: Start address of the devicetree binary with VM configuration, + * relative to start of memparcel. + * @dtb_size: Maximum size of devicetree binary. + */ +int gunyah_rm_vm_configure(struct gunyah_rm *rm, u16 vmid, + enum gunyah_rm_vm_auth_mechanism auth_mechanism, + u32 mem_handle, u64 image_offset, u64 image_size, + u64 dtb_offset, u64 dtb_size) +{ + struct gunyah_rm_vm_config_image_req req_payload = { + .vmid = cpu_to_le16(vmid), + .auth_mech = cpu_to_le16(auth_mechanism), + .mem_handle = cpu_to_le32(mem_handle), + .image_offset = cpu_to_le64(image_offset), + .image_size = cpu_to_le64(image_size), + .dtb_offset = cpu_to_le64(dtb_offset), + .dtb_size = cpu_to_le64(dtb_size), + }; + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_CONFIG_IMAGE, &req_payload, + sizeof(req_payload), NULL, NULL); +} +ALLOW_ERROR_INJECTION(gunyah_rm_vm_configure, ERRNO); + +/** + * gunyah_rm_vm_init() - Move the VM to initialized state. + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier + * + * RM will allocate needed resources for the VM. + */ +int gunyah_rm_vm_init(struct gunyah_rm *rm, u16 vmid) +{ + return gunyah_rm_common_vmid_call(rm, GUNYAH_RM_RPC_VM_INIT, vmid); +} +ALLOW_ERROR_INJECTION(gunyah_rm_vm_init, ERRNO); + +/** + * gunyah_rm_get_hyp_resources() - Retrieve hypervisor resources (capabilities) associated with a VM + * @rm: Handle to a Gunyah resource manager + * @vmid: VMID of the other VM to get the resources of + * @resources: Set by gunyah_rm_get_hyp_resources and contains the returned hypervisor resources. + * Caller must free the resources pointer if successful. + */ +int gunyah_rm_get_hyp_resources(struct gunyah_rm *rm, u16 vmid, + struct gunyah_rm_hyp_resources **resources) +{ + struct gunyah_rm_vm_common_vmid_req req_payload = { + .vmid = cpu_to_le16(vmid), + }; + struct gunyah_rm_hyp_resources *resp; + size_t resp_size; + int ret; + + ret = gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_GET_HYP_RESOURCES, + &req_payload, sizeof(req_payload), (void **)&resp, + &resp_size); + if (ret) + return ret; + + if (!resp_size) + return -EBADMSG; + + if (resp_size < struct_size(resp, entries, 0) || + resp_size != + struct_size(resp, entries, le32_to_cpu(resp->n_entries))) { + kfree(resp); + return -EBADMSG; + } + + *resources = resp; + return 0; +} +ALLOW_ERROR_INJECTION(gunyah_rm_get_hyp_resources, ERRNO); diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index 87e919cc1e28..294e847c27ed 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -19,9 +19,86 @@ int gunyah_rm_notifier_register(struct gunyah_rm *rm, int gunyah_rm_notifier_unregister(struct gunyah_rm *rm, struct notifier_block *nb); +struct gunyah_rm_vm_exited_payload { + __le16 vmid; + __le16 exit_type; + __le32 exit_reason_size; + u8 exit_reason[]; +} __packed; + +enum gunyah_rm_notification_id { + /* clang-format off */ + GUNYAH_RM_NOTIFICATION_VM_EXITED = 0x56100001, + GUNYAH_RM_NOTIFICATION_VM_STATUS = 0x56100008, + /* clang-format on */ +}; + +enum gunyah_rm_vm_status { + /* clang-format off */ + GUNYAH_RM_VM_STATUS_NO_STATE = 0, + GUNYAH_RM_VM_STATUS_INIT = 1, + GUNYAH_RM_VM_STATUS_READY = 2, + GUNYAH_RM_VM_STATUS_RUNNING = 3, + GUNYAH_RM_VM_STATUS_PAUSED = 4, + GUNYAH_RM_VM_STATUS_LOAD = 5, + GUNYAH_RM_VM_STATUS_AUTH = 6, + GUNYAH_RM_VM_STATUS_INIT_FAILED = 8, + GUNYAH_RM_VM_STATUS_EXITED = 9, + GUNYAH_RM_VM_STATUS_RESETTING = 10, + GUNYAH_RM_VM_STATUS_RESET = 11, + /* clang-format on */ +}; + +struct gunyah_rm_vm_status_payload { + __le16 vmid; + u16 reserved; + u8 vm_status; + u8 os_status; + __le16 app_status; +} __packed; + +int gunyah_rm_alloc_vmid(struct gunyah_rm *rm, u16 vmid); +int gunyah_rm_dealloc_vmid(struct gunyah_rm *rm, u16 vmid); +int gunyah_rm_vm_reset(struct gunyah_rm *rm, u16 vmid); +int gunyah_rm_vm_start(struct gunyah_rm *rm, u16 vmid); +int gunyah_rm_vm_stop(struct gunyah_rm *rm, u16 vmid); + +enum gunyah_rm_vm_auth_mechanism { + /* clang-format off */ + GUNYAH_RM_VM_AUTH_NONE = 0, + GUNYAH_RM_VM_AUTH_QCOM_PIL_ELF = 1, + GUNYAH_RM_VM_AUTH_QCOM_ANDROID_PVM = 2, + /* clang-format on */ +}; + +int gunyah_rm_vm_configure(struct gunyah_rm *rm, u16 vmid, + enum gunyah_rm_vm_auth_mechanism auth_mechanism, + u32 mem_handle, u64 image_offset, u64 image_size, + u64 dtb_offset, u64 dtb_size); +int gunyah_rm_vm_init(struct gunyah_rm *rm, u16 vmid); + +struct gunyah_rm_hyp_resource { + u8 type; + u8 reserved; + __le16 partner_vmid; + __le32 resource_handle; + __le32 resource_label; + __le64 cap_id; + __le32 virq_handle; + __le32 virq; + __le64 base; + __le64 size; +} __packed; + +struct gunyah_rm_hyp_resources { + __le32 n_entries; + struct gunyah_rm_hyp_resource entries[]; +} __packed; + +int gunyah_rm_get_hyp_resources(struct gunyah_rm *rm, u16 vmid, + struct gunyah_rm_hyp_resources **resources); int gunyah_rm_call(struct gunyah_rm *rsc_mgr, u32 message_id, const void *req_buf, size_t req_buf_size, void **resp_buf, size_t *resp_buf_size); - #endif From patchwork Thu Apr 24 14:13:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884039 Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 548CC281376 for ; Thu, 24 Apr 2025 14:14:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504051; cv=none; b=CTPfxAn82BtlvSXHOSWZp2kExn9XLY96H5094DDRsPPNscGcFzRj7H06ff1j7mytu2m89FTjR4tG2X34uIxph0jMMmGQaNhx5V9Rgkc/94HlWED0PTj7FyYEH7WL3kbXlMfKsPms7+MFp5VTRhaQCZ7jr5pnPPPNfQMCM2Oppvk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504051; c=relaxed/simple; bh=o5jUxv0nZ8uJl1blXMpoQdyQe0eMlKyDBmLFSdRHE2s=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=BnUfUxRKH+nZMDm8dj4A1jCCPYiklITCiR9p074+IGL3xob+q5xk57JTxffJxr/D9wCBkomWY/t/2P0qRcQayZu0HKA+zx66LDBF/R5+ETC168gyGxes6nufbS4mnaHEMAEvoEEvLFF928Y9l9r3Y5Ij9QkKF5Iy+DXGl3sgI0k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=b3AS8B4F; arc=none smtp.client-ip=209.85.128.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="b3AS8B4F" Received: by mail-wm1-f50.google.com with SMTP id 5b1f17b1804b1-43cf58eea0fso5047985e9.0 for ; Thu, 24 Apr 2025 07:14:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504046; x=1746108846; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UoVyA4T/XR+PrBla9t1/dUC6ibbVMr9f/wkTq4NWm5c=; b=b3AS8B4FhCvDA56c2cSPpxuBmsrjqdga0bD6lbyS8PvkIA8cf7zlRq6z9fzmPupRgA /YnSM9O7e/PrryqZAq6/G3a4y/TkQBZFSIAc02ai8aEvNP4AvSxYq1D4Sl5VPOHgZGYc 2LrFTQeSnpc1eMY5/9YihDnNsyuSakGuC/dtwQxrU8BZx3iK+9FKcx8IIrQM2z2l6p4O MD8VKCRcCSBpAlWE6T371f7EkZZpH/XyyM6Wv36SIm7EmyZeQweTmv0Jw0Q+kAzlnh1y eju6ObBdHz0TqpVVqjmihnPbh1sJDmyc6gvm/YBsXQr9tZC87D4B2YdvUpou/brTXAIh GK6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504046; x=1746108846; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UoVyA4T/XR+PrBla9t1/dUC6ibbVMr9f/wkTq4NWm5c=; b=jYdJo2E+NlTYFkbfPsomTCQOgSxliZ5oQACxlBUGUVPpTOO95j3sUfvcQxyN759F5d mp9+XB8l/AWi6QbmONWG0Iongwyu4RTcPlgowgAYN+oJjB1mHDDng0xINMb2rO3AEDJz R3fxv9aIrkAB0dstVfVlk9014bSN6RbdvSjVeMdovRiYpT6UVAp4I5dO29i/4JKdvxkf X/fBiGEdVZtboNOdoos75FM85mrCW5FRlpW7CxuDi3gfh6c7548/2MeZqSF2iIFuu53Y r7FuTtpi9DPwvB3PI1JUg2enWH021dH7/F/OBamb3pDcHeg19OltyXT6BHZ5ExqsVtKS OjYA== X-Forwarded-Encrypted: i=1; AJvYcCU2iZE97HEaR5o2VpUfSqlxC9QK5GGGZIsLZl1yRVXE6lBQPFvayxhgwHXF0CCo02/L8SLGSpaOCSkn5yNj@vger.kernel.org X-Gm-Message-State: AOJu0YzxpTnWIoEp8z7E9jWi31HcUy3M+Y0O65IdjRG3rQBr7QJjFsaY //wtOEg0YnHqhBuRUaFl3Mf3DvdmQkhWH+gSEn8TUE0ml4r/4c86o2Zq/NnrD54= X-Gm-Gg: ASbGncs9CBZ2CQJQhygwXojtcOCQ/AyuN2BEnWrChYIJKSr6cHzx6LCI8arMLmErcqp MHu6qOe1klhoy4VrzYfmy5Ff2yRqxOy6eWyAck/lMkyma8LEayH2LtK5nQA1GF+RSr7ZWOfemGO dSOjwOWGkcwHoLXkaPg0Mqg6mhGGug7NR5xNXP210J3HpOUaJ3xYWHkLsQIOklnRL3ueb3kSXfw m6MahXqNLpPPY7kHGA+W6YDZScBr6pIxDlRAqgAe5amHwPLdBD7VlPrZQFwlWVKlKzckwxaaTGl vAgG6R1pLUulZMcBHix2WQdkJr0flCd9GAqMnRoHREAyw3ZkNbc5OcNyXdDjrwvIzLL51gTJJu3 Hl1U2NDXMKemkA1of X-Google-Smtp-Source: AGHT+IFaUaoW58kCoH3cGCkS9884/3gN3P0U2I1fqPQYARfS16RUhRZKrV74ekkuGWYubSRTquJKGQ== X-Received: by 2002:a5d:64ab:0:b0:391:2a9f:2fcb with SMTP id ffacd0b85a97d-3a06cfa8400mr2501557f8f.36.1745504046320; Thu, 24 Apr 2025 07:14:06 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:05 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 16/34] gunyah: Add basic VM lifecycle management Date: Thu, 24 Apr 2025 15:13:23 +0100 Message-Id: <20250424141341.841734-17-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This patch hooks up Gunyah virtual machine lifecycle management with the KVM backend by implementing the kvm_arch_alloc_vm(), kvm_arch_destroy_vm(), and kvm_arch_free_vm() hooks. The Gunyah VM management logic—VMID allocation, configuration, initialization, start/stop, teardown, and notifier handling—is based on the implementation introduced in [1], authored by Elliot Berman and Prakruthi Deepak Heragu. The original code added a special ioctl interface to support userspace initialization of guest VMs. This patch reuses the same logic, but ported to KVM, allowing to use KVM's ioctl interface to create Gunyah-based guests. [1] Commit: 532788ce71c9 ("gunyah: vm_mgr: Add VM start/stop") Link: https://lore.kernel.org/lkml/20240222-gunyah-v17-10-1e9da6763d38@quicinc.com/ Co-developed-by: Elliot Berman Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Karim Manaouil --- arch/arm64/include/asm/kvm_host.h | 5 + arch/arm64/kvm/gunyah.c | 196 ++++++++++++++++++++++++++++-- drivers/virt/gunyah/rsc_mgr_rpc.c | 2 +- include/linux/gunyah.h | 32 +++++ 4 files changed, 227 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 9c8e173fc9c1..53358d3f5fa8 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -1591,4 +1591,9 @@ void kvm_set_vm_id_reg(struct kvm *kvm, u32 reg, u64 val); #define kvm_has_s1poe(k) \ (kvm_has_feat((k), ID_AA64MMFR3_EL1, S1POE, IMP)) +#ifndef CONFIG_KVM_ARM +#define __KVM_HAVE_ARCH_VM_FREE +void kvm_arch_free_vm(struct kvm *kvm); +#endif + #endif /* __ARM64_KVM_HOST_H__ */ diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index 9c37ab20d7e2..a3c29ae985c9 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -13,6 +13,12 @@ #include #include +#include +#include + +#undef pr_fmt +#define pr_fmt(fmt) "gunyah: " fmt + static enum kvm_mode kvm_mode = KVM_MODE_DEFAULT; enum kvm_mode kvm_get_mode(void) @@ -338,12 +344,6 @@ void kvm_arch_create_vm_debugfs(struct kvm *kvm) { } -void kvm_arch_destroy_vm(struct kvm *kvm) -{ - kvm_destroy_vcpus(kvm); - return; -} - long kvm_arch_dev_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) { @@ -788,7 +788,189 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu) return -EINVAL; } +static int gunyah_vm_rm_notification_status(struct gunyah_vm *ghvm, void *data) +{ + struct gunyah_rm_vm_status_payload *payload = data; + + if (le16_to_cpu(payload->vmid) != ghvm->vmid) + return NOTIFY_OK; + + /* All other state transitions are synchronous to a corresponding RM call */ + if (payload->vm_status == GUNYAH_RM_VM_STATUS_RESET) { + down_write(&ghvm->status_lock); + ghvm->vm_status = payload->vm_status; + up_write(&ghvm->status_lock); + wake_up(&ghvm->vm_status_wait); + } + + return NOTIFY_DONE; +} + +static int gunyah_vm_rm_notification_exited(struct gunyah_vm *ghvm, void *data) +{ + struct gunyah_rm_vm_exited_payload *payload = data; + + if (le16_to_cpu(payload->vmid) != ghvm->vmid) + return NOTIFY_OK; + + down_write(&ghvm->status_lock); + ghvm->vm_status = GUNYAH_RM_VM_STATUS_EXITED; + up_write(&ghvm->status_lock); + wake_up(&ghvm->vm_status_wait); + + return NOTIFY_DONE; +} + +static int gunyah_vm_rm_notification(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct gunyah_vm *ghvm = container_of(nb, struct gunyah_vm, nb); + + switch (action) { + case GUNYAH_RM_NOTIFICATION_VM_STATUS: + return gunyah_vm_rm_notification_status(ghvm, data); + case GUNYAH_RM_NOTIFICATION_VM_EXITED: + return gunyah_vm_rm_notification_exited(ghvm, data); + default: + return NOTIFY_OK; + } +} + +static void gunyah_vm_stop(struct gunyah_vm *ghvm) +{ + int ret; + + if (ghvm->vm_status == GUNYAH_RM_VM_STATUS_RUNNING) { + ret = gunyah_rm_vm_stop(ghvm->rm, ghvm->vmid); + if (ret) + pr_warn("Failed to stop VM: %d\n", ret); + } + + wait_event(ghvm->vm_status_wait, + ghvm->vm_status != GUNYAH_RM_VM_STATUS_RUNNING); +} + +static int gunyah_vm_start(struct gunyah_vm *ghvm) +{ + int ret; + + down_write(&ghvm->status_lock); + if (ghvm->vm_status != GUNYAH_RM_VM_STATUS_NO_STATE) { + up_write(&ghvm->status_lock); + return 0; + } + + ghvm->nb.notifier_call = gunyah_vm_rm_notification; + ret = gunyah_rm_notifier_register(ghvm->rm, &ghvm->nb); + if (ret) + goto err; + + ret = gunyah_rm_alloc_vmid(ghvm->rm, 0); + if (ret < 0) { + gunyah_rm_notifier_unregister(ghvm->rm, &ghvm->nb); + goto err; + } + ghvm->vmid = ret; + ghvm->vm_status = GUNYAH_RM_VM_STATUS_LOAD; + + ret = gunyah_rm_vm_configure(ghvm->rm, ghvm->vmid, ghvm->auth, 0, 0, 0, 0, 0); + if (ret) { + pr_warn("Failed to configure VM: %d\n", ret); + goto err; + } + + ret = gunyah_rm_vm_init(ghvm->rm, ghvm->vmid); + if (ret) { + ghvm->vm_status = GUNYAH_RM_VM_STATUS_INIT_FAILED; + pr_warn("Failed to initialize VM: %d\n", ret); + goto err; + } + ghvm->vm_status = GUNYAH_RM_VM_STATUS_READY; + + ret = gunyah_rm_vm_start(ghvm->rm, ghvm->vmid); + if (ret) { + pr_warn("Failed to start VM: %d\n", ret); + goto err; + } + + ghvm->vm_status = GUNYAH_RM_VM_STATUS_RUNNING; + up_write(&ghvm->status_lock); + return 0; +err: + up_write(&ghvm->status_lock); + return ret; +} + +static struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) +{ + struct gunyah_vm *ghvm; + + ghvm = kzalloc(sizeof(*ghvm), GFP_KERNEL); + if (!ghvm) + return ERR_PTR(-ENOMEM); + + ghvm->vmid = GUNYAH_VMID_INVAL; + ghvm->rm = rm; + + init_rwsem(&ghvm->status_lock); + init_waitqueue_head(&ghvm->vm_status_wait); + ghvm->vm_status = GUNYAH_RM_VM_STATUS_NO_STATE; + + return ghvm; +} + +static void gunyah_destroy_vm(struct gunyah_vm *ghvm) +{ + int ret; + + /** + * We might race with a VM exit notification, but that's ok: + * gh_rm_vm_stop() will just return right away. + */ + if (ghvm->vm_status == GUNYAH_RM_VM_STATUS_RUNNING) + gunyah_vm_stop(ghvm); + + if (ghvm->vm_status == GUNYAH_RM_VM_STATUS_EXITED || + ghvm->vm_status == GUNYAH_RM_VM_STATUS_READY || + ghvm->vm_status == GUNYAH_RM_VM_STATUS_INIT_FAILED) { + ret = gunyah_rm_vm_reset(ghvm->rm, ghvm->vmid); + if (!ret) + wait_event(ghvm->vm_status_wait, + ghvm->vm_status == GUNYAH_RM_VM_STATUS_RESET); + else + pr_warn("Failed to reset the vm: %d\n", ret); + } + + if (ghvm->vm_status > GUNYAH_RM_VM_STATUS_NO_STATE) { + gunyah_rm_notifier_unregister(ghvm->rm, &ghvm->nb); + ret = gunyah_rm_dealloc_vmid(ghvm->rm, ghvm->vmid); + if (ret) + pr_warn("Failed to deallocate vmid: %d\n", ret); + } +} + struct kvm *kvm_arch_alloc_vm(void) { - return NULL; + struct gunyah_vm *ghvm; + + ghvm = gunyah_vm_alloc(gunyah_rm); + if (IS_ERR(ghvm)) + return NULL; + + return &ghvm->kvm; +} + +void kvm_arch_destroy_vm(struct kvm *kvm) +{ + struct gunyah_vm *ghvm = kvm_to_gunyah(kvm); + + kvm_destroy_vcpus(kvm); + gunyah_destroy_vm(ghvm); +} + +void kvm_arch_free_vm(struct kvm *kvm) +{ + struct gunyah_vm *ghvm = kvm_to_gunyah(kvm); + + kfree(ghvm); } diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mgr_rpc.c index 626ad2565548..936592177ddb 100644 --- a/drivers/virt/gunyah/rsc_mgr_rpc.c +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -3,8 +3,8 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include #include - #include /* Message IDs: VM Management */ diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index acd70f982425..1f4389eb21fb 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -11,6 +11,12 @@ #include #include #include +#include + +#include + +#define kvm_to_gunyah(kvm_ptr) \ + container_of(kvm_ptr, struct gunyah_vm, kvm) /* Matches resource manager's resource types for VM_GET_HYP_RESOURCES RPC */ enum gunyah_resource_type { @@ -31,6 +37,32 @@ struct gunyah_resource { unsigned int irq; }; +/** + * struct gunyah_vm - Main representation of a Gunyah Virtual machine + memory shared with the guest. + * @vmid: Gunyah's VMID for this virtual machine + * @kvm: kvm instance for this VM + * @rm: Pointer to the resource manager struct to make RM calls + * @nb: Notifier block for RM notifications + * @vm_status: Current state of the VM, as last reported by RM + * @vm_status_wait: Wait queue for status @vm_status changes + * @status_lock: Serializing state transitions + * @auth: Authentication mechanism to be used by resource manager when + * launching the VM + */ +struct gunyah_vm { + u16 vmid; + struct kvm kvm; + struct gunyah_rm *rm; + + struct notifier_block nb; + enum gunyah_rm_vm_status vm_status; + wait_queue_head_t vm_status_wait; + struct rw_semaphore status_lock; + + enum gunyah_rm_vm_auth_mechanism auth; +}; + /******************************************************************************/ /* Common arch-independent definitions for Gunyah hypercalls */ #define GUNYAH_CAPID_INVAL U64_MAX From patchwork Thu Apr 24 14:13:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884940 Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B56228C5A4 for ; Thu, 24 Apr 2025 14:14:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504053; cv=none; b=aguxTBwopfs0BjMqHxJRRUTEwTkBij5fJZecpivlsqzCJ3Knv+AUHua/d3D8qtoI9AiCVnVyVHWeOzAj96PX9cnx8w319lSC9S7/Wyx9za+LX+mO6gqeHbCGVAarCqAB0wb6Uv0ZSh/Afli5vOFmaeUW4F/21RXNVCo5MFzP1Sc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504053; c=relaxed/simple; bh=S+aezV66gXf4D+s9yS2EKNbYmZv94XHap0Nf+ZRRUHk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Qq+oLS4Mq9AdCdZnIbtyyCTm5pwzS5HOTXx7J835U7tNewBUEn678j0mTRW4oky+IiDrTbg6+cE1c1XkdXW4B0PoaoizbhcZDlLPy8RYpUz53BvaK6jy+oKU1yBKQrMA+1RqfgqW+NOs/OtNw7N3KH8OBkHbJpuwwS/WO+yHpD4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=ZVrYkVnP; arc=none smtp.client-ip=209.85.221.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="ZVrYkVnP" Received: by mail-wr1-f46.google.com with SMTP id ffacd0b85a97d-39d83782ef6so1648172f8f.0 for ; Thu, 24 Apr 2025 07:14:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504048; x=1746108848; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=98GtKQowyHwUXJ3/oi0BeOIZGrWAP4CATPU0kL4zC8c=; b=ZVrYkVnPnl9Ys6EiWjO0nW/ZKL9GZPhxZM8Jdq4yhpWJfZHOL7zyEOxMrBZQgE4ywb S/dHcmgPS1jkO5i8DXxJ+fn8JUsYf3LS8ffoJXnN9Obvw7bF4KK95uYdIdxYSk3FOog7 4gZdjcJDvoxbNYrVrTnD6TfC3gXwFj0Q7fKu28uJZMdpJQQ9WazFoPCxhnuMNwow6cLm SwXd2GQgnZH64avWP2TPUZEJ5EEpzYL2FwLsoeVkaOXNR0FrskyMcsvHbDAlTyEBhCCH fBNLGXr2xfE/z7LpOxCBiHFnskchBvSkkplqymZ6IXk/VkZsoT4gAZf1MYFXX32dFOmo V/hA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504048; x=1746108848; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=98GtKQowyHwUXJ3/oi0BeOIZGrWAP4CATPU0kL4zC8c=; b=kIOV3LIP+OT51UvriFPlvJQlljftoUSaycPKUHGmztmwA53w0go3CWJZ88bQlTBuBx xXXx0GMlGkXduXjjTqz+HZ9pnct/Crf31SBFitfrvwfK462ZUCc5HIi3vMThUw58wVZp EiXdVhdzynTLck9LDxZcAID/DL0qtVZZlqtX9lr6Tl2HIClHgTQbX6ftlKEQkkMOSUSo sKnq55IKJgOZVaWQqTantuysoReDbYULlA5w4ZydEjy5otz1EooByrp4PC8SFBFptiN6 8aAGA+sxjXDKS6oWVY+Ayq9vxKO9a1lmHUyssZI45bvzrIj1YfaNzyxBubSwOHVXwaMF lRKw== X-Forwarded-Encrypted: i=1; AJvYcCUDK9a8kn3uvEJKI0jBwCoCREus6QP4xL/2tW14NQmANyW/mdNvS0u0rtqoN7j3I2BSO/QdvEk2Wd10vAN9@vger.kernel.org X-Gm-Message-State: AOJu0YzRyYU1AhvmJMNEHWm88UtqKmdT+ahUETxAzeQPOOpaUUetUZmS jGMIDVzStvA926ZuVxShqWEtXURxL9yoFH+fFi6L2o8R7U7AzuVtGw0alcefQ0M= X-Gm-Gg: ASbGncuvZ2gfuZWdW9mAtOfSfQo4vz1gE7MGKUst3tMt/KijCo3lVtb1MBSYQHVCKKc ZaWirPW/mi7YmXlsl7ydp8y4KCEgWa+I7gwVvqccx+ZqUgITn9ERDuTVwhL5JdAdQ2rObdKa+AR Vrhsca2M4h0MxIEkmUm8robL6lveV/MwiwODimkxV2RTmcZpjdLhzcNLsMMCBM+QvyVtAxb61kI z3Nj7LIt4LYCOQAhLRwodv/Q8hiFpoyFwCz27THNJaXi07sIw8kJhIqwiDOpOwrywOti3xy/sF9 mVSkJzWRcAYbLUR5qFITohW3oYK1qyZW0+PA/R6ihFia+XA5nWj3DFV9U6EqsEQ1Yx3A1EnP+Ek YhEgfqvIUuwkXa/QH X-Google-Smtp-Source: AGHT+IFQAg02MXbpqUbddZDz1Nf5W/i1x6IwVsYQPoOcChQDsuxBB7Db2uzIfhJZTYe0HZGVw3TbTg== X-Received: by 2002:a05:6000:18ae:b0:39f:cd2:1fa1 with SMTP id ffacd0b85a97d-3a06d647026mr2922759f8f.3.1745504048036; Thu, 24 Apr 2025 07:14:08 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:07 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 17/34] gunyah: Translate gh_rm_hyp_resource into gunyah_resource Date: Thu, 24 Apr 2025 15:13:24 +0100 Message-Id: <20250424141341.841734-18-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman When booting a Gunyah virtual machine, the host VM may gain capabilities to interact with resources for the guest virtual machine. Examples of such resources are vCPUs or message queues. To use those resources, we need to translate the RM response into a gunyah_resource structure which are useful to Linux drivers. Presently, Linux drivers need only to know the type of resource, the capability ID, and an interrupt. On ARM64 systems, the interrupt reported by Gunyah is the GIC interrupt ID number and always a SPI or extended SPI. Signed-off-by: Elliot Berman Reviewed-by: Srivatsa Vaddagiri Signed-off-by: Karim Manaouil --- arch/arm64/include/asm/gunyah.h | 35 +++++++++++++++++ drivers/virt/gunyah/rsc_mgr.c | 68 +++++++++++++++++++++++++++++++++ include/linux/gunyah.h | 2 + include/linux/gunyah_rsc_mgr.h | 4 ++ 4 files changed, 109 insertions(+) create mode 100644 arch/arm64/include/asm/gunyah.h diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunyah.h new file mode 100644 index 000000000000..29079d1a4df2 --- /dev/null +++ b/arch/arm64/include/asm/gunyah.h @@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#ifndef _ASM_GUNYAH_H +#define _ASM_GUNYAH_H + +#include +#include + +static inline int arch_gunyah_fill_irq_fwspec_params(u32 virq, + struct irq_fwspec *fwspec) +{ + /* Assume that Gunyah gave us an SPI or ESPI; defensively check it */ + if (WARN(virq < 32, "Unexpected virq: %d\n", virq)) { + return -EINVAL; + } else if (virq <= 1019) { + fwspec->param_count = 3; + fwspec->param[0] = 0; /* GIC_SPI */ + fwspec->param[1] = virq - 32; /* virq 32 -> SPI 0 */ + fwspec->param[2] = IRQ_TYPE_EDGE_RISING; + } else if (WARN(virq < 4096, "Unexpected virq: %d\n", virq)) { + return -EINVAL; + } else if (virq < 5120) { + fwspec->param_count = 3; + fwspec->param[0] = 2; /* GIC_ESPI */ + fwspec->param[1] = virq - 4096; /* virq 4096 -> ESPI 0 */ + fwspec->param[2] = IRQ_TYPE_EDGE_RISING; + } else { + WARN(1, "Unexpected virq: %d\n", virq); + return -EINVAL; + } + return 0; +} +#endif diff --git a/drivers/virt/gunyah/rsc_mgr.c b/drivers/virt/gunyah/rsc_mgr.c index 75fc86887868..2d34b0ba98b2 100644 --- a/drivers/virt/gunyah/rsc_mgr.c +++ b/drivers/virt/gunyah/rsc_mgr.c @@ -9,8 +9,10 @@ #include #include #include +#include #include +#include #include /* clang-format off */ @@ -118,6 +120,7 @@ struct gunyah_rm_message { * @send_lock: synchronization to allow only one request to be sent at a time * @send_ready: completed when we know Tx message queue can take more messages * @nh: notifier chain for clients interested in RM notification messages + * @parent_fwnode: Parent IRQ fwnode to translate Gunyah hwirqs to Linux irqs */ struct gunyah_rm { struct device *dev; @@ -133,6 +136,8 @@ struct gunyah_rm { struct mutex send_lock; struct completion send_ready; struct blocking_notifier_head nh; + + struct fwnode_handle *parent_fwnode; }; /* Global resource manager instance */ @@ -177,6 +182,53 @@ static inline int gunyah_rm_error_remap(enum gunyah_rm_error rm_error) } } +struct gunyah_resource * +gunyah_rm_alloc_resource(struct gunyah_rm *rm, + struct gunyah_rm_hyp_resource *hyp_resource) +{ + struct gunyah_resource *ghrsc; + int ret; + + ghrsc = kzalloc(sizeof(*ghrsc), GFP_KERNEL); + if (!ghrsc) + return NULL; + + ghrsc->type = hyp_resource->type; + ghrsc->capid = le64_to_cpu(hyp_resource->cap_id); + ghrsc->irq = IRQ_NOTCONNECTED; + ghrsc->rm_label = le32_to_cpu(hyp_resource->resource_label); + if (hyp_resource->virq) { + struct irq_fwspec fwspec; + + + fwspec.fwnode = rm->parent_fwnode; + ret = arch_gunyah_fill_irq_fwspec_params(le32_to_cpu(hyp_resource->virq), &fwspec); + if (ret) { + dev_err(rm->dev, + "Failed to translate interrupt for resource %d label: %d: %d\n", + ghrsc->type, ghrsc->rm_label, ret); + } + + ret = irq_create_fwspec_mapping(&fwspec); + if (ret < 0) { + dev_err(rm->dev, + "Failed to allocate interrupt for resource %d label: %d: %d\n", + ghrsc->type, ghrsc->rm_label, ret); + kfree(ghrsc); + return NULL; + } + ghrsc->irq = ret; + } + + return ghrsc; +} + +void gunyah_rm_free_resource(struct gunyah_resource *ghrsc) +{ + irq_dispose_mapping(ghrsc->irq); + kfree(ghrsc); +} + static int gunyah_rm_init_message_payload(struct gunyah_rm_message *message, const void *msg, size_t hdr_size, size_t msg_size) @@ -676,6 +728,7 @@ static int gunyah_rm_probe_rx_msgq(struct gunyah_rm *rm, static int gunyah_rm_probe(struct platform_device *pdev) { + struct device_node *parent_irq_node; int ret; gunyah_rm = devm_kzalloc(&pdev->dev, sizeof(*gunyah_rm), GFP_KERNEL); @@ -695,6 +748,21 @@ static int gunyah_rm_probe(struct platform_device *pdev) ret = gunyah_rm_probe_tx_msgq(gunyah_rm, pdev); if (ret) return ret; + + parent_irq_node = of_irq_find_parent(pdev->dev.of_node); + if (!parent_irq_node) { + dev_err(&pdev->dev, + "Failed to find interrupt parent of resource manager\n"); + return -ENODEV; + } + + gunyah_rm->parent_fwnode = of_node_to_fwnode(parent_irq_node); + if (!gunyah_rm->parent_fwnode) { + dev_err(&pdev->dev, + "Failed to find interrupt parent domain of resource manager\n"); + return -ENODEV; + } + /* assume RM is ready to receive messages from us */ complete(&gunyah_rm->send_ready); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 1f4389eb21fb..40ea21b17195 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -35,6 +35,8 @@ struct gunyah_resource { enum gunyah_resource_type type; u64 capid; unsigned int irq; + + u32 rm_label; }; /** diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index 294e847c27ed..c0fe516d54a8 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -97,6 +97,10 @@ struct gunyah_rm_hyp_resources { int gunyah_rm_get_hyp_resources(struct gunyah_rm *rm, u16 vmid, struct gunyah_rm_hyp_resources **resources); +struct gunyah_resource * +gunyah_rm_alloc_resource(struct gunyah_rm *rm, + struct gunyah_rm_hyp_resource *hyp_resource); +void gunyah_rm_free_resource(struct gunyah_resource *ghrsc); int gunyah_rm_call(struct gunyah_rm *rsc_mgr, u32 message_id, const void *req_buf, size_t req_buf_size, void **resp_buf, From patchwork Thu Apr 24 14:13:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884038 Received: from mail-wr1-f44.google.com (mail-wr1-f44.google.com [209.85.221.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 59B3C28CF6F for ; Thu, 24 Apr 2025 14:14:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504053; cv=none; b=Rf2rDgk+f8ULo4dqQh8gnkCBRooQxMrvdCvEjnIYQRSdCuNC3n+0BCkF5qafx29unlROgGrlYRIbyf/Q2E89OrJAmMXGAue0KbKQ9dm4ApMU8zISz9amkNrMaU7djIzv+ZK5S+3UNlZaX7n6Rd9C6D85gRK7AYLye72Y/w6z+Bw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504053; c=relaxed/simple; bh=DcGv+45qXx8VGU67Z7HeTl3hoEPjIE7jytPc3+geiuE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ntH9ZcbEbXSLkN19Jxmc/EQp60BQhSfwHmJD9c1LFMIeQ6lg1Jw6A3/Zy4nUDPXVtxbhxvs6lWHHoktjhjgtunMG9RwcCtzz/BJSXO99pM0RMdMEsQE5c/+VgIbBLNZnwqINJnNZs3jPrMLPD/uJyZmbqNoAvtFQ+bNq3b9HDGU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=YIRJC4+s; arc=none smtp.client-ip=209.85.221.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="YIRJC4+s" Received: by mail-wr1-f44.google.com with SMTP id ffacd0b85a97d-39c1efbefc6so802678f8f.1 for ; Thu, 24 Apr 2025 07:14:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504050; x=1746108850; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=um+352HlfSoMWZjyUPYbdpZGkULscMwPKC2qBZ9EnFo=; b=YIRJC4+sfP9K7ZxYZWGxTQMjNFQy2JI0tIeeK/BfvqfqS1GKAfbm5Fr5c0XV+tMqwW 9vVk0Ng3BuugZHjkFlsmW21qafmVfN99B3w4ye3xbHNzRJvmrEp8PVmeQQjPcgB231Kw MEisY8bi3PgsLmrLOV0dsPGFF2gOQDmDs20JrwveUJuu3vnofJlkHVCBncOYpfYTIFik TKNKBOwtDhKLWFH5cAPv06B/e4FIi8IoFcPWtqrnOcdHMeH90XTTy92j+BJCGdBQF/8v sIr/HhwK+QNWj3y2FNuztGVfZ7lwO8R4ARngY8kmLmNjLUTHLQBg/Jl1IrMqzMveSjNo uRgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504050; x=1746108850; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=um+352HlfSoMWZjyUPYbdpZGkULscMwPKC2qBZ9EnFo=; b=GsKvHOQJ5mjbyZ89HuP+Q+Xt7NWzi08fS+CjRYFQiGC7elLHFoA4aXxcBrwK6hUGFJ 1R+12In6GykEcLq+oIU0XOsnesX1SPf6cR8e5kDaIMo756MHj58qi1K87Ux8yVXmqaEP czjLMJ+4RxiX+e/z2H3dx0epYwPxqx/xIwDivBx1gVkIFUw0hHb7qVPbvxdVM4keNZXl /Zb4sIfCP67a7QVkRm+2hpifSjdZ1ZBsgs4tZ7ok0QqVhlIOzbcvaCACPNxqR6zdTugU 5yFkVL/lmDOGPKURLBKF5QLyKAZ23fIU+q5xhKUR+9rAdb6Jh++eXaMZ9MtSKBf4KjDZ 1EKw== X-Forwarded-Encrypted: i=1; AJvYcCUVl5H/Nfl3usg5/a/2rCwbm+EzHsuNVDKQVWzqvTlCQPRYxdTz8LJJ9U62BG9cqQabyixSU6XQD1jhd40t@vger.kernel.org X-Gm-Message-State: AOJu0YzHFGBmuiZeebavEpnebQjEG5y8tO8RehPXm0pMvVLh+iggkpQl Yu2FI+janQVqKb6xiNUXAnlo1zc8bCSvEkaIvsOKQ0QXZFc03Y6mW4W8GkUbsPE= X-Gm-Gg: ASbGncvnCwKwZYQqBC9wR4ahKyqSGQyLONVL9Np3tmoQ/LtBLEv6ZTGHZIFMzLy0uXe aDH537sIMlx9RfZveymbqP2ku4l+WYNobh9Jx1v+aQZcfijTpkYDXiOyCEJVPE3CmlYE+k+bJOa AI8KT4gcLTHE0z5O+14BfqnHQuKHjMHSpHw08Zz4h37z3TqgmghIMdgLDO7RzbK6m5jWl8fYsW2 L4p/19Eb0vr0aFgeBukwqDFHbZfp4zS6v5+gFdd1AXbi67vr5RLJ7toTnpP9HkyIPCzsUM4fXU5 ixVJEYDuuwtmJPRZoz3jqLGa/reuvvH1sOPzOCoRM4IftwL99qp7XgBer9gvunuydV+8FDXnP1P OSdIYibyV0Nmy15Iw X-Google-Smtp-Source: AGHT+IEvmLar5qQKwS4wz8XFX51INY2Yy6tUFq9mAIas/T9L785Up8KKwGDWzGpkxhc6ebHp9w3CBA== X-Received: by 2002:a5d:64c7:0:b0:391:4389:f36a with SMTP id ffacd0b85a97d-3a06cfaf773mr2225643f8f.48.1745504049421; Thu, 24 Apr 2025 07:14:09 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:08 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman , Alex Elder Subject: [RFC PATCH 18/34] gunyah: Add resource tickets Date: Thu, 24 Apr 2025 15:13:25 +0100 Message-Id: <20250424141341.841734-19-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman Some VM functions need to acquire Gunyah resources. For instance, Gunyah vCPUs are exposed to the host as a resource. The Gunyah vCPU function will register a resource ticket and be able to interact with the hypervisor once the resource ticket is filled. Resource tickets are the mechanism for functions to acquire ownership of Gunyah resources. Gunyah functions can be created before the VM's resources are created and made available to Linux. A resource ticket identifies a type of resource and a label of a resource which the ticket holder is interested in. Resources are created by Gunyah as configured in the VM's devicetree configuration. Gunyah doesn't process the label and that makes it possible for userspace to create multiple resources with the same label. Resource ticket owners need to be prepared for populate to be called multiple times if userspace created multiple resources with the same label. Reviewed-by: Alex Elder Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 123 +++++++++++++++++++++++++++++++++++++++- include/linux/gunyah.h | 42 +++++++++++++- 2 files changed, 161 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index a3c29ae985c9..084ee1091770 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -330,6 +330,104 @@ int kvm_arm_set_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) } } +static int gunyah_vm_add_resource_ticket(struct gunyah_vm *ghvm, + struct gunyah_vm_resource_ticket *ticket) +{ + struct gunyah_vm_resource_ticket *iter; + struct gunyah_resource *ghrsc, *rsc_iter; + int ret = 0; + + mutex_lock(&ghvm->resources_lock); + list_for_each_entry(iter, &ghvm->resource_tickets, vm_list) { + if (iter->resource_type == ticket->resource_type && + iter->label == ticket->label) { + ret = -EEXIST; + goto out; + } + } + + list_add(&ticket->vm_list, &ghvm->resource_tickets); + INIT_LIST_HEAD(&ticket->resources); + + list_for_each_entry_safe(ghrsc, rsc_iter, &ghvm->resources, list) { + if (ghrsc->type == ticket->resource_type && + ghrsc->rm_label == ticket->label) { + if (ticket->populate(ticket, ghrsc)) + list_move(&ghrsc->list, &ticket->resources); + } + } +out: + mutex_unlock(&ghvm->resources_lock); + return ret; +} + +static void __gunyah_vm_remove_resource_ticket(struct gunyah_vm *ghvm, + struct gunyah_vm_resource_ticket *ticket) +{ + struct gunyah_resource *ghrsc, *iter; + + list_for_each_entry_safe(ghrsc, iter, &ticket->resources, list) { + ticket->unpopulate(ticket, ghrsc); + list_move(&ghrsc->list, &ghvm->resources); + } + list_del(&ticket->vm_list); +} + +static void gunyah_vm_remove_resource_ticket(struct gunyah_vm *ghvm, + struct gunyah_vm_resource_ticket *ticket) +{ + + mutex_lock(&ghvm->resources_lock); + __gunyah_vm_remove_resource_ticket(ghvm, ticket); + mutex_unlock(&ghvm->resources_lock); +} + +static void gunyah_vm_add_resource(struct gunyah_vm *ghvm, + struct gunyah_resource *ghrsc) +{ + struct gunyah_vm_resource_ticket *ticket; + + mutex_lock(&ghvm->resources_lock); + list_for_each_entry(ticket, &ghvm->resource_tickets, vm_list) { + if (ghrsc->type == ticket->resource_type && + ghrsc->rm_label == ticket->label) { + if (ticket->populate(ticket, ghrsc)) + list_add(&ghrsc->list, &ticket->resources); + else + list_add(&ghrsc->list, &ghvm->resources); + /* unconditonal -- we prevent multiple identical + * resource tickets so there will not be some other + * ticket elsewhere in the list if populate() failed. + */ + goto found; + } + } + list_add(&ghrsc->list, &ghvm->resources); +found: + mutex_unlock(&ghvm->resources_lock); +} + +static void gunyah_vm_clean_resources(struct gunyah_vm *ghvm) +{ + struct gunyah_vm_resource_ticket *ticket, *titer; + struct gunyah_resource *ghrsc, *riter; + + mutex_lock(&ghvm->resources_lock); + if (!list_empty(&ghvm->resource_tickets)) { + pr_warn("Dangling resource tickets:\n"); + list_for_each_entry_safe(ticket, titer, &ghvm->resource_tickets, + vm_list) { + pr_warn(" %pS\n", ticket->populate); + __gunyah_vm_remove_resource_ticket(ghvm, ticket); + } + } + + list_for_each_entry_safe(ghrsc, riter, &ghvm->resources, list) { + gunyah_rm_free_resource(ghrsc); + } + mutex_unlock(&ghvm->resources_lock); +} + int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE; @@ -852,7 +950,9 @@ static void gunyah_vm_stop(struct gunyah_vm *ghvm) static int gunyah_vm_start(struct gunyah_vm *ghvm) { - int ret; + struct gunyah_rm_hyp_resources *resources; + struct gunyah_resource *ghrsc; + int i, n, ret; down_write(&ghvm->status_lock); if (ghvm->vm_status != GUNYAH_RM_VM_STATUS_NO_STATE) { @@ -887,6 +987,22 @@ static int gunyah_vm_start(struct gunyah_vm *ghvm) } ghvm->vm_status = GUNYAH_RM_VM_STATUS_READY; + ret = gunyah_rm_get_hyp_resources(ghvm->rm, ghvm->vmid, &resources); + if (ret) { + pr_warn("Failed to get hyp resources for VM: %d\n", ret); + goto err; + } + + for (i = 0, n = le32_to_cpu(resources->n_entries); i < n; i++) { + ghrsc = gunyah_rm_alloc_resource(ghvm->rm, + &resources->entries[i]); + if (!ghrsc) { + ret = -ENOMEM; + goto err; + } + gunyah_vm_add_resource(ghvm, ghrsc); + } + ret = gunyah_rm_vm_start(ghvm->rm, ghvm->vmid); if (ret) { pr_warn("Failed to start VM: %d\n", ret); @@ -915,6 +1031,9 @@ static struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) init_rwsem(&ghvm->status_lock); init_waitqueue_head(&ghvm->vm_status_wait); ghvm->vm_status = GUNYAH_RM_VM_STATUS_NO_STATE; + mutex_init(&ghvm->resources_lock); + INIT_LIST_HEAD(&ghvm->resources); + INIT_LIST_HEAD(&ghvm->resource_tickets); return ghvm; } @@ -930,6 +1049,8 @@ static void gunyah_destroy_vm(struct gunyah_vm *ghvm) if (ghvm->vm_status == GUNYAH_RM_VM_STATUS_RUNNING) gunyah_vm_stop(ghvm); + gunyah_vm_clean_resources(ghvm); + if (ghvm->vm_status == GUNYAH_RM_VM_STATUS_EXITED || ghvm->vm_status == GUNYAH_RM_VM_STATUS_READY || ghvm->vm_status == GUNYAH_RM_VM_STATUS_INIT_FAILED) { diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 40ea21b17195..573e3bbd4cb6 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -18,6 +19,8 @@ #define kvm_to_gunyah(kvm_ptr) \ container_of(kvm_ptr, struct gunyah_vm, kvm) +struct gunyah_vm; + /* Matches resource manager's resource types for VM_GET_HYP_RESOURCES RPC */ enum gunyah_resource_type { /* clang-format off */ @@ -35,10 +38,39 @@ struct gunyah_resource { enum gunyah_resource_type type; u64 capid; unsigned int irq; - + struct list_head list; u32 rm_label; }; +/** + * struct gunyah_vm_resource_ticket - Represents a ticket to reserve access to VM resource(s) + * @label: Label of the resource from resource manager this ticket reserves. + * @vm_list: for @gunyah_vm->resource_tickets + * @resources: List of resource(s) associated with this ticket + * (members are from @gunyah_resource->list) + * @resource_type: Type of resource this ticket reserves + * @populate: callback provided by the ticket owner and called when a resource is found that + * matches @resource_type and @label. Note that this callback could be called + * multiple times if userspace created mutliple resources with the same type/label. + * This callback may also have significant delay after gunyah_vm_add_resource_ticket() + * since gunyah_vm_add_resource_ticket() could be called before the VM starts. + * @unpopulate: callback provided by the ticket owner and called when the ticket owner should no + * longer use the resource provided in the argument. When unpopulate() returns, + * the ticket owner should not be able to use the resource any more as the resource + * might being freed. + */ +struct gunyah_vm_resource_ticket { + u32 label; + struct list_head vm_list; + struct list_head resources; + enum gunyah_resource_type resource_type; + bool (*populate)(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc); + void (*unpopulate)(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc); +}; + + /** * struct gunyah_vm - Main representation of a Gunyah Virtual machine memory shared with the guest. @@ -49,6 +81,9 @@ struct gunyah_resource { * @vm_status: Current state of the VM, as last reported by RM * @vm_status_wait: Wait queue for status @vm_status changes * @status_lock: Serializing state transitions + * @resource_lock: Serializing addition of resources and resource tickets + * @resources: List of &struct gunyah_resource that are associated with this VM + * @resource_tickets: List of &struct gunyah_vm_resource_ticket * @auth: Authentication mechanism to be used by resource manager when * launching the VM */ @@ -56,12 +91,13 @@ struct gunyah_vm { u16 vmid; struct kvm kvm; struct gunyah_rm *rm; - struct notifier_block nb; enum gunyah_rm_vm_status vm_status; wait_queue_head_t vm_status_wait; struct rw_semaphore status_lock; - + struct mutex resources_lock; + struct list_head resources; + struct list_head resource_tickets; enum gunyah_rm_vm_auth_mechanism auth; }; From patchwork Thu Apr 24 14:13:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884939 Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C6AB928DEFA for ; Thu, 24 Apr 2025 14:14:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504056; cv=none; b=gJlgI9gcC72WEc1eECzTKarbHzM1Ng0e/gCH+Fdqr3SEodmGVylvQ3fX3NDs03NTgZHeMHv67roH1jiunQ/BV2Bu+6bjDDmiXVt6jPhtGLy1dS4PVFg/yK/6IuMx5iRLrSJljUzCzTlvy7zzha2nbySEfX57CmGslkUEUQ3Tsk8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504056; c=relaxed/simple; bh=2aIlJTxyajv9w1JNq2QUWNBaFE1l1Iz4MgPOHzkWSHs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Na/nWySWP0rMl75owzx3kAaDtXqRLP5cg8+sWj2m+cP0nOxLwZIV3WC7Qr1+uAp/ll0vwTYfXnYxpF/pmS+NGinBtFg4KlqMnUvvKWveonSoeXwASc/kuT0vmPGaEGRJs6DwKIsFYwmbtdm980xK0jYoeXaZm+hducdO7VbAsBA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=PJZRrhye; arc=none smtp.client-ip=209.85.221.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="PJZRrhye" Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-39c0e0bc733so989525f8f.1 for ; Thu, 24 Apr 2025 07:14:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504051; x=1746108851; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mnnNrRFux5qVcGjuNFA0c9YQRdgVNUximYQurTlnUC4=; b=PJZRrhyeB9kFuh5fnr6Nbxc6hBWs4a64QdPEFmN6EjippJ8/wFu+CIKUVczzDUBqh3 afTa0zEhR+philf+002ackoL6VTNxbBgqwzH3tecbFIHxuzwa93EtCfPNNnTilwq7gm6 Q5ZOQ52nyt5CUsXzxFumP4ORZN0zxiWGoAx1odVSisLG4VW79iVky/ct8IuYjx3ziW0A NB1P3pxFsx/voIgr8Fn9gjhRZAk+/21vaiehhkayNW42Fqk6NyJaexztNuffnx+/BqOc qigxJdZ2AUmPsYD6qeYmRUQhvRWkn8h0xx33NFpO8sBwTFvmDtZI0e4/CPbM9C+izJzZ xVlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504051; x=1746108851; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mnnNrRFux5qVcGjuNFA0c9YQRdgVNUximYQurTlnUC4=; b=qbHviPec2RuKse61CxHUYe8JOG7Dl95BuDY5r68kr+onDKA7Av3sKQfDYlbpz9Z8kl 8ejoy+pxk07haY1Hfgeqw3uD9lhkGWDK5cGYzGbiLvTPTlqfvgKSgSL8SDs2om/OvyGO Mn1UoOshMbHPrVzJaR3FZb+3ZAvA2mbsbDcYCROm1dt5DLuLR+nhQqOqxGs7oHgt6HGw YXlDWLQW8+PEa6zgVCqzluupHrlGG/gLGHO41TbXqGjvW29EQ8Fi9XjOWCNCBg3NRE+P 7A2Z1JTJig6JE9k3wgj8whyUHGZSrfBhGL7kEkPcurbrN80+izacClVqD9i7LVAaVa64 tlBQ== X-Forwarded-Encrypted: i=1; AJvYcCXyH1GvZUN0yxi52j7xjDZe6gmw1ZCaL6OGlzij2ryo6ScX0uapd8nMOI7FF/vILmsjKudBIEf2iXtuXhj5@vger.kernel.org X-Gm-Message-State: AOJu0Yyjw6xS4Y2uq7+11BOnWxgjBUBvQnF7AAxw/gP8dvK1OamQyrpb s1zOpix2xpFcP6Lh6u8PzzFJvl8coDoR3cqNEkQ8Rdr6jgpcNn++a5wdgckOj3c= X-Gm-Gg: ASbGnctLPuD2pdrzhyllNuuoiIfmENohGK/9GmAV1WV24oVLBynAcyX5jEG0UU5spSw IeCMoXPRoqMVBSbf4Iaj7XexrRzg8aEsewnDvJbKAEewX3vbVLGS6WQuLKTo0M4B1y0pm35yR2i WhDw0bf+diyBV1aRr0Iik2vQI1AAl7m4twbGAy/DSxXCtNu3tthxJXBu7H2YfDVhHg7lvrvQmvt 6FLg7QdQG1eqmrsYJKOOXsmrB6Ne+F/KCoYp4di62dRD8lMeZ/NdiLG5XPM2fV5gNxHtDR7fvLv Oj0FPtQ+kwl04MEqyxRE0AabqhIUcnqCfXCF3wDA/UFYqrXWjKUzqWrekhAhWXs2/j5EOhLTVV6 TXY3ozUC5w7V8iFqv X-Google-Smtp-Source: AGHT+IElBFJDMQbW748Xhn6uR3Gy7UiwZwfaHD4iyiwy3AqrKyTZYkKWwIgzmc9HZX/EcWP9CEo9Lw== X-Received: by 2002:a05:6000:18a2:b0:391:47d8:de2d with SMTP id ffacd0b85a97d-3a06cf5ed4cmr2483522f8f.23.1745504050693; Thu, 24 Apr 2025 07:14:10 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:10 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 19/34] gunyah: Add hypercalls for running a vCPU Date: Thu, 24 Apr 2025 15:13:26 +0100 Message-Id: <20250424141341.841734-20-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman Add hypercall to donate CPU time to a vCPU. Signed-off-by: Elliot Berman Reviewed-by: Srivatsa Vaddagiri Signed-off-by: Karim Manaouil --- arch/arm64/gunyah/gunyah_hypercall.c | 37 ++++++++++++++++++++++++++++ include/linux/gunyah.h | 35 ++++++++++++++++++++++++++ 2 files changed, 72 insertions(+) diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunyah_hypercall.c index 1302e128be6e..fee21df42c17 100644 --- a/arch/arm64/gunyah/gunyah_hypercall.c +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -39,6 +39,7 @@ EXPORT_SYMBOL_GPL(arch_is_gunyah_guest); #define GUNYAH_HYPERCALL_HYP_IDENTIFY GUNYAH_HYPERCALL(0x8000) #define GUNYAH_HYPERCALL_MSGQ_SEND GUNYAH_HYPERCALL(0x801B) #define GUNYAH_HYPERCALL_MSGQ_RECV GUNYAH_HYPERCALL(0x801C) +#define GUNYAH_HYPERCALL_VCPU_RUN GUNYAH_HYPERCALL(0x8065) /* clang-format on */ /** @@ -113,5 +114,41 @@ enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid, void *buff, size_t size, } EXPORT_SYMBOL_GPL(gunyah_hypercall_msgq_recv); +/** + * gunyah_hypercall_vcpu_run() - Donate CPU time to a vcpu + * @capid: capability ID of the vCPU to run + * @resume_data: Array of 3 state-specific resume data + * @resp: Filled reason why vCPU exited when return value is GUNYAH_ERROR_OK + * + * See also: + * https://github.com/quic/gunyah-hypervisor/blob/develop/docs/api/gunyah_api.md#run-a-proxy-scheduled-vcpu-thread + */ +enum gunyah_error +gunyah_hypercall_vcpu_run(u64 capid, unsigned long *resume_data, + struct gunyah_hypercall_vcpu_run_resp *resp) +{ + struct arm_smccc_1_2_regs args = { + .a0 = GUNYAH_HYPERCALL_VCPU_RUN, + .a1 = capid, + .a2 = resume_data[0], + .a3 = resume_data[1], + .a4 = resume_data[2], + /* C language says this will be implictly zero. Gunyah requires 0, so be explicit */ + .a5 = 0, + }; + struct arm_smccc_1_2_regs res; + + arm_smccc_1_2_hvc(&args, &res); + if (res.a0 == GUNYAH_ERROR_OK) { + resp->sized_state = res.a1; + resp->state_data[0] = res.a2; + resp->state_data[1] = res.a3; + resp->state_data[2] = res.a4; + } + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_vcpu_run); + MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("Gunyah Hypervisor Hypercalls"); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 573e3bbd4cb6..f86f14018734 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -219,4 +219,39 @@ enum gunyah_error gunyah_hypercall_msgq_send(u64 capid, size_t size, void *buff, enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid, void *buff, size_t size, size_t *recv_size, bool *ready); +struct gunyah_hypercall_vcpu_run_resp { + union { + enum { + /* clang-format off */ + /* VCPU is ready to run */ + GUNYAH_VCPU_STATE_READY = 0, + /* VCPU is sleeping until an interrupt arrives */ + GUNYAH_VCPU_STATE_EXPECTS_WAKEUP = 1, + /* VCPU is powered off */ + GUNYAH_VCPU_STATE_POWERED_OFF = 2, + /* VCPU is blocked in EL2 for unspecified reason */ + GUNYAH_VCPU_STATE_BLOCKED = 3, + /* VCPU has returned for MMIO READ */ + GUNYAH_VCPU_ADDRSPACE_VMMIO_READ = 4, + /* VCPU has returned for MMIO WRITE */ + GUNYAH_VCPU_ADDRSPACE_VMMIO_WRITE = 5, + /* VCPU blocked on fault where we can demand page */ + GUNYAH_VCPU_ADDRSPACE_PAGE_FAULT = 7, + /* clang-format on */ + } state; + u64 sized_state; + }; + u64 state_data[3]; +}; + +enum { + GUNYAH_ADDRSPACE_VMMIO_ACTION_EMULATE = 0, + GUNYAH_ADDRSPACE_VMMIO_ACTION_RETRY = 1, + GUNYAH_ADDRSPACE_VMMIO_ACTION_FAULT = 2, +}; + +enum gunyah_error +gunyah_hypercall_vcpu_run(u64 capid, unsigned long *resume_data, + struct gunyah_hypercall_vcpu_run_resp *resp); + #endif From patchwork Thu Apr 24 14:13:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884037 Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F067E28E5EE for ; Thu, 24 Apr 2025 14:14:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504057; cv=none; b=aaXUvf+LBXSlFvfUIvuXy28n1L2sCcKPis3wZeE8ODoWv1nvVQLu0BFYBi8Zg05O9gv8cB9oOXVpQlK9j1shw9mCvHBgpN6IwLUmL690nuy2eCbno5IPZtz/BiCI6KsedGc2m026kZOIwhpur6wkKpYpbp3lL/+qd6DQe9Rc7bo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504057; c=relaxed/simple; bh=v1GZAOzGYATKFr8YmGOLT0nlJPt5FIGEotq+V+hCZbE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=XpHHew/hLA+vP+2uWIoCCeQltsbS/Lbco9RWA6i97b0F1cRQQV2KZzJvWZAnd1kbYlpQcQ16/6kDr6Eof1ZwID5GaaJR1azGtsovoDDN8fJKyuwXFo1gen98De9iF8nr7VEmTPF44qCDsYlQ5WshCadwAP3pDSC0/JWHUvLgs8g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=uiibvrls; arc=none smtp.client-ip=209.85.221.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="uiibvrls" Received: by mail-wr1-f47.google.com with SMTP id ffacd0b85a97d-39c266c1389so745069f8f.1 for ; Thu, 24 Apr 2025 07:14:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504052; x=1746108852; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Npq0NQtrgJlAV8FqtQiHtQ8TrFptNSWm5BxQWT8PEM0=; b=uiibvrlsxKeiO53O7KMvUTq6V5pxeqYhQonnUHBTJ3lNoajeqVhT/CrQeOmBs76CJJ 3dW1Mfe9W3IL29grfp4eFoqb9IEinywSh88118AlS2XUfe5/2f4q9lwj7wicMEgBTZGA A7BvFOCBgQd5oFs1xsVSt5/KgqIuphAHU1cd4to0vS8BwqGMKrZccWuxQ3VkfvT+E04s TX5puDb0Pm/QJ57M4LcXEQBgTcYN0jIE66UJi8Buxh3YB/D9zNymkWtF1k0T+EdWBRCL Hq+xoMrBRScN45umw1ADddkPHOL3SlIAma7TU2fkoE54wXBK8tgWzQrnCwWMXxNgvbK4 Ln9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504052; x=1746108852; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Npq0NQtrgJlAV8FqtQiHtQ8TrFptNSWm5BxQWT8PEM0=; b=ti93H77LsHJhxdKb2nHZt2t7+rHdgILfROLCD+9S3HCZglz4RJlr0lpayux5ZBXcHE qReQ7O8/vl1HDSD6vnT4NbBjwpPAgzIMzcukuZLLgJMCU5KYExIGbmDGv72ZMzZrVdDa JNFQzvxDJ83K74wHSEkesDSEt3WxsIp2y2bHaNvepORp7Gmj5Sk0nysRbrN+z2Nahf2D TRfHE5Ejbq7CDwF3ogF/9FZP7a1h9k7C81ySLAXsYYA5djM9Ba0w5c3U8zwRbSanZkj/ G/sb8Yd7gsdQ0jw2COePJnqqUMqeZEoqC8Bj9C/agW9koVuykSxo/2fzUSx29icV3+MV CRQg== X-Forwarded-Encrypted: i=1; AJvYcCXkAvxlkSJ+hQJ5HRhXH1bVqrwSXwoUY7XHvL9TTQUWiDpRr+07p/1HSAJnOvAZPP1Ytgvm42ylA/TWdpTt@vger.kernel.org X-Gm-Message-State: AOJu0Yx/TDd0lC/drN7CocPGZ7PSfePp3G6ky0auSwfiLtJ+CSfzzDqK i4ZVdY8DydXLTe+afoVEQaLyCdlUHAxyiNDSpIC2JwC3AZEDdKR1ShdDkPg12Nc= X-Gm-Gg: ASbGnct8pqUF7I1U8mFJ/MbTN2c+0KDXmUfBzn1QYhtYh2ypgugEJ6TblsLsCvCUZcZ 4xvtr+66D+1VGKUI7BL5b8fNN/VnguqG0bHCJmkzf9lVWwtMekt+nozzXY/bx5rxHBlTtlDxzwR EkDuHKAPs05BT/JagXHyDqiWXadFE87lLUwBY7Eg8n6jHs0I2bBtSO8mPFXg241VJSUhJIx3kz1 /fPhthtjTVEhqk1szpbhX0w9QXasHQPIaNGxq23ZFDWyOvorS+/W9IrT7INpaaEmB5Zb2+mWX0U e/pz/gY+3kiGB85vwwhTOJ4FeoSOqcLEix/qFWsRfID1WMql8XcZ4hX4UCetmf+U7KQ2av/Mmom WJqC0w32+zEqEvGjw X-Google-Smtp-Source: AGHT+IGDNQni2+JFXMwQWuIVE9mUFnHRCikRQcusA6Pgtq8l9mbgJyFlCKPQ4I6jOpeZedp+Ki0cWw== X-Received: by 2002:a5d:5f4f:0:b0:39c:1f10:c736 with SMTP id ffacd0b85a97d-3a06cfab8cbmr2032162f8f.43.1745504052080; Thu, 24 Apr 2025 07:14:12 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:11 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 20/34] gunyah: add proxy-scheduled vCPUs Date: Thu, 24 Apr 2025 15:13:27 +0100 Message-Id: <20250424141341.841734-21-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This patch is based heavily on the original Gunyah vCPU support from Elliot Berman and Prakruthi Deepak Heragu: https://lore.kernel.org/lkml/20240222-gunyah-v17-14-1e9da6763d38@quicinc.com/ The original implementation had its own character device interface. This patch ports Gunyah vCPU management to KVM (e.g., `kvm_arch_vcpu_run()` calls Gunyah hypervisor, running as firmware, via hypercalls, which then runs the vCPU). This enables Gunyah vCPUs to be driven through the standard KVM userspace interface (e.g., via QEMU), while transparently using Gunyah’s proxy-scheduled vCPU mechanisms under the hood. Co-developed-by: Elliot Berman Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 348 +++++++++++++++++++++++++++++++++++++++- include/linux/gunyah.h | 51 ++++++ 2 files changed, 395 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index 084ee1091770..e066482c2e71 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -19,6 +19,8 @@ #undef pr_fmt #define pr_fmt(fmt) "gunyah: " fmt +static int gunyah_vm_start(struct gunyah_vm *ghvm); + static enum kvm_mode kvm_mode = KVM_MODE_DEFAULT; enum kvm_mode kvm_get_mode(void) @@ -458,9 +460,311 @@ bool kvm_arch_intc_initialized(struct kvm *kvm) return true; } -struct kvm_vcpu *kvm_arch_vcpu_alloc(void) +/* + * When hypervisor allows us to schedule vCPU again, it gives us an interrupt + */ +static irqreturn_t gunyah_vcpu_irq_handler(int irq, void *data) +{ + struct gunyah_vcpu *vcpu = data; + + complete(&vcpu->ready); + return IRQ_HANDLED; +} + +static int gunyah_vcpu_rm_notification(struct notifier_block *nb, + unsigned long action, void *data) { - return NULL; + struct gunyah_vcpu *vcpu = container_of(nb, struct gunyah_vcpu, nb); + struct gunyah_rm_vm_exited_payload *exit_payload = data; + + /* Wake up userspace waiting for the vCPU to be runnable again */ + if (action == GUNYAH_RM_NOTIFICATION_VM_EXITED && + le16_to_cpu(exit_payload->vmid) == vcpu->ghvm->vmid) + complete(&vcpu->ready); + + return NOTIFY_OK; +} + +static int gunyah_handle_page_fault( + struct gunyah_vcpu *vcpu, + const struct gunyah_hypercall_vcpu_run_resp *vcpu_run_resp) +{ + return -EINVAL; +} + +static bool gunyah_kvm_handle_mmio(struct gunyah_vcpu *vcpu, + unsigned long resume_data[3], + const struct gunyah_hypercall_vcpu_run_resp *vcpu_run_resp) +{ + struct kvm_vcpu *kvm_vcpu = &vcpu->kvm_vcpu; + struct kvm_run *run = kvm_vcpu->run; + u64 addr = vcpu_run_resp->state_data[0]; + u64 len = vcpu_run_resp->state_data[1]; + u64 data = vcpu_run_resp->state_data[2]; + bool write; + + if (WARN_ON(len > sizeof(u64))) + len = sizeof(u64); + + if (vcpu_run_resp->state == GUNYAH_VCPU_ADDRSPACE_VMMIO_READ) { + write = false; + /* + * Record that we need to give vCPU user's supplied + * value next gunyah_vcpu_run() + */ + vcpu->state = GUNYAH_VCPU_RUN_STATE_MMIO_READ; + } else { + /* TODO: HANDLE IOEVENTFD !! */ + write = true; + vcpu->state = GUNYAH_VCPU_RUN_STATE_MMIO_WRITE; + } + + if (write) + memcpy(run->mmio.data, &data, len); + + run->mmio.is_write = write; + run->mmio.phys_addr = addr; + run->mmio.len = len; + kvm_vcpu->mmio_needed = 1; + + kvm_vcpu->stat.mmio_exit_user++; + run->exit_reason = KVM_EXIT_MMIO; + + return false; +} + +static int gunyah_handle_mmio_resume(struct gunyah_vcpu *vcpu, + unsigned long resume_data[3]) +{ + struct kvm_vcpu *kvm_vcpu = &vcpu->kvm_vcpu; + struct kvm_run *run = kvm_vcpu->run; + + resume_data[1] = GUNYAH_ADDRSPACE_VMMIO_ACTION_EMULATE; + if (vcpu->state == GUNYAH_VCPU_RUN_STATE_MMIO_READ) + memcpy(&resume_data[0], run->mmio.data, run->mmio.len); + return 0; +} + +/** + * gunyah_vcpu_check_system() - Check whether VM as a whole is running + * @vcpu: Pointer to gunyah_vcpu + * + * Returns true if the VM is alive. + * Returns false if the vCPU is the VM is not alive (can only be that VM is shutting down). + */ +static bool gunyah_vcpu_check_system(struct gunyah_vcpu *vcpu) + __must_hold(&vcpu->lock) +{ + bool ret = true; + + down_read(&vcpu->ghvm->status_lock); + if (likely(vcpu->ghvm->vm_status == GUNYAH_RM_VM_STATUS_RUNNING)) + goto out; + + vcpu->state = GUNYAH_VCPU_RUN_STATE_SYSTEM_DOWN; + ret = false; +out: + up_read(&vcpu->ghvm->status_lock); + return ret; +} + +static int gunyah_vcpu_run(struct gunyah_vcpu *vcpu) +{ + struct gunyah_hypercall_vcpu_run_resp vcpu_run_resp; + struct kvm_vcpu *kvm_vcpu = &vcpu->kvm_vcpu; + struct kvm_run *run = kvm_vcpu->run; + unsigned long resume_data[3] = { 0 }; + enum gunyah_error gunyah_error; + int ret = 0; + + if (mutex_lock_interruptible(&vcpu->lock)) + return -ERESTARTSYS; + + if (!vcpu->rsc) { + ret = -ENODEV; + goto out; + } + + switch (vcpu->state) { + case GUNYAH_VCPU_RUN_STATE_UNKNOWN: + if (vcpu->ghvm->vm_status != GUNYAH_RM_VM_STATUS_RUNNING) { + /** + * Check if VM is up. If VM is starting, will block + * until VM is fully up since that thread does + * down_write. + */ + if (!gunyah_vcpu_check_system(vcpu)) + goto out; + } + vcpu->state = GUNYAH_VCPU_RUN_STATE_READY; + break; + case GUNYAH_VCPU_RUN_STATE_MMIO_READ: + case GUNYAH_VCPU_RUN_STATE_MMIO_WRITE: + ret = gunyah_handle_mmio_resume(vcpu, resume_data); + if (ret) + goto out; + vcpu->state = GUNYAH_VCPU_RUN_STATE_READY; + break; + case GUNYAH_VCPU_RUN_STATE_SYSTEM_DOWN: + goto out; + default: + break; + } + + run->exit_reason = KVM_EXIT_UNKNOWN; + + while (!ret && !signal_pending(current)) { + if (vcpu->immediate_exit) { + ret = -EINTR; + goto out; + } + gunyah_error = gunyah_hypercall_vcpu_run( + vcpu->rsc->capid, resume_data, &vcpu_run_resp); + + if (gunyah_error == GUNYAH_ERROR_OK) { + memset(resume_data, 0, sizeof(resume_data)); + + switch (vcpu_run_resp.state) { + case GUNYAH_VCPU_STATE_READY: + if (need_resched()) + schedule(); + break; + case GUNYAH_VCPU_STATE_POWERED_OFF: + /** + * vcpu might be off because the VM is shut down + * If so, it won't ever run again + */ + if (!gunyah_vcpu_check_system(vcpu)) + goto out; + /** + * Otherwise, another vcpu will turn it on (e.g. + * by PSCI) and hyp sends an interrupt to wake + * Linux up. + */ + fallthrough; + case GUNYAH_VCPU_STATE_EXPECTS_WAKEUP: + ret = wait_for_completion_interruptible( + &vcpu->ready); + /** + * reinitialize completion before next + * hypercall. If we reinitialize after the + * hypercall, interrupt may have already come + * before re-initializing the completion and + * then end up waiting for event that already + * happened. + */ + reinit_completion(&vcpu->ready); + /** + * Check VM status again. Completion + * might've come from VM exiting + */ + if (!ret && !gunyah_vcpu_check_system(vcpu)) + goto out; + break; + case GUNYAH_VCPU_STATE_BLOCKED: + schedule(); + break; + case GUNYAH_VCPU_ADDRSPACE_VMMIO_READ: + case GUNYAH_VCPU_ADDRSPACE_VMMIO_WRITE: + if (!gunyah_kvm_handle_mmio(vcpu, resume_data, + &vcpu_run_resp)) + goto out; + break; + case GUNYAH_VCPU_ADDRSPACE_PAGE_FAULT: + ret = gunyah_handle_page_fault(vcpu, &vcpu_run_resp); + if (ret) + goto out; + break; + default: + pr_warn( + "Unknown vCPU state: %llx\n", + vcpu_run_resp.sized_state); + schedule(); + break; + } + } else if (gunyah_error == GUNYAH_ERROR_RETRY) { + schedule(); + } else { + ret = gunyah_error_remap(gunyah_error); + } + } + +out: + mutex_unlock(&vcpu->lock); + + if (signal_pending(current)) + return -ERESTARTSYS; + + return ret; +} + +static bool gunyah_vcpu_populate(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc) +{ + struct gunyah_vcpu *vcpu = + container_of(ticket, struct gunyah_vcpu, ticket); + int ret; + + mutex_lock(&vcpu->lock); + if (vcpu->rsc) { + pr_warn("vcpu%d already got a Gunyah resource", vcpu->ticket.label); + ret = -EEXIST; + goto out; + } + vcpu->rsc = ghrsc; + + ret = request_irq(vcpu->rsc->irq, gunyah_vcpu_irq_handler, + IRQF_TRIGGER_RISING, "gunyah_vcpu", vcpu); + if (ret) { + pr_warn("Failed to request vcpu irq %d: %d", vcpu->rsc->irq, + ret); + goto out; + } + + enable_irq_wake(vcpu->rsc->irq); +out: + mutex_unlock(&vcpu->lock); + return !ret; +} + +static void gunyah_vcpu_unpopulate(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc) +{ + struct gunyah_vcpu *vcpu = + container_of(ticket, struct gunyah_vcpu, ticket); + + vcpu->immediate_exit = true; + complete_all(&vcpu->ready); + mutex_lock(&vcpu->lock); + free_irq(vcpu->rsc->irq, vcpu); + vcpu->rsc = NULL; + mutex_unlock(&vcpu->lock); +} + +static int gunyah_vcpu_create(struct gunyah_vm *ghvm, struct gunyah_vcpu *vcpu, int id) +{ + int r; + + mutex_init(&vcpu->lock); + init_completion(&vcpu->ready); + + vcpu->ghvm = ghvm; + vcpu->nb.notifier_call = gunyah_vcpu_rm_notification; + /** + * Ensure we run after the vm_mgr handles the notification and does + * any necessary state changes. + */ + vcpu->nb.priority = -1; + r = gunyah_rm_notifier_register(ghvm->rm, &vcpu->nb); + if (r) + return r; + + vcpu->ticket.resource_type = GUNYAH_RESOURCE_TYPE_VCPU; + vcpu->ticket.label = id; + vcpu->ticket.populate = gunyah_vcpu_populate; + vcpu->ticket.unpopulate = gunyah_vcpu_unpopulate; + + return gunyah_vm_add_resource_ticket(ghvm, &vcpu->ticket); } int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id) @@ -470,7 +774,8 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id) int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) { - return -EINVAL; + GUNYAH_STATE(vcpu); + return gunyah_vcpu_create(ghvm, ghvcpu, vcpu->vcpu_id); } void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu) @@ -479,6 +784,28 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu) void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu) { + GUNYAH_STATE(vcpu); + + gunyah_rm_notifier_unregister(ghvcpu->ghvm->rm, &ghvcpu->nb); + gunyah_vm_remove_resource_ticket(ghvcpu->ghvm, &ghvcpu->ticket); + kfree(ghvcpu); +} + +struct kvm_vcpu *kvm_arch_vcpu_alloc(void) +{ + struct gunyah_vcpu *vcpu; + + vcpu = kzalloc(sizeof(*vcpu), GFP_KERNEL_ACCOUNT); + if (!vcpu) + return NULL; + return &vcpu->kvm_vcpu; +} + +void kvm_arch_vcpu_free(struct kvm_vcpu *kvm_vcpu) +{ + struct gunyah_vcpu *vcpu = gunyah_vcpu(kvm_vcpu); + + kfree(vcpu); } void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu) @@ -521,7 +848,20 @@ bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu) int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) { - return -EINVAL; + GUNYAH_STATE(vcpu); + int ret; + + if (!xchg(&ghvm->started, 1)) { + ret = gunyah_vm_start(ghvm); + if (ret) { + xchg(&ghvm->started, 0); + goto out; + } + } + ret = gunyah_vcpu_run(ghvcpu); +out: + return ret; + } long kvm_arch_vcpu_ioctl(struct file *filp, diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index f86f14018734..fa6e3fd4bee1 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -16,9 +16,16 @@ #include +#define gunyah_vcpu(kvm_vcpu_ptr) \ + container_of(kvm_vcpu_ptr, struct gunyah_vcpu, kvm_vcpu) + #define kvm_to_gunyah(kvm_ptr) \ container_of(kvm_ptr, struct gunyah_vm, kvm) +#define GUNYAH_STATE(kvm_vcpu) \ + struct gunyah_vm __maybe_unused *ghvm = kvm_to_gunyah(kvm_vcpu->kvm); \ + struct gunyah_vcpu __maybe_unused *ghvcpu = gunyah_vcpu(kvm_vcpu) + struct gunyah_vm; /* Matches resource manager's resource types for VM_GET_HYP_RESOURCES RPC */ @@ -89,6 +96,7 @@ struct gunyah_vm_resource_ticket { */ struct gunyah_vm { u16 vmid; + bool started; struct kvm kvm; struct gunyah_rm *rm; struct notifier_block nb; @@ -101,6 +109,49 @@ struct gunyah_vm { enum gunyah_rm_vm_auth_mechanism auth; }; +/** + * struct gunyah_vcpu - Track an instance of gunyah vCPU + * @kvm_vcpu: kvm instance + * @rsc: Pointer to the Gunyah vCPU resource, will be NULL until VM starts + * @lock: One userspace thread at a time should run the vCPU + * @ghvm: Pointer to the main VM struct; quicker look up than going through + * @f->ghvm + * @state: Our copy of the state of the vCPU, since userspace could trick + * kernel to behave incorrectly if we relied on @vcpu_run + * @ready: if vCPU goes to sleep, hypervisor reports to us that it's sleeping + * and will signal interrupt (from @rsc) when it's time to wake up. + * This completion signals that we can run vCPU again. + * @nb: When VM exits, the status of VM is reported via @vcpu_run->status. + * We need to track overall VM status, and the nb gives us the updates from + * Resource Manager. + * @ticket: resource ticket to claim vCPU# for the VM + */ +struct gunyah_vcpu { + struct kvm_vcpu kvm_vcpu; + struct gunyah_resource *rsc; + struct mutex lock; + struct gunyah_vm *ghvm; + + /** + * Track why the vcpu_run hypercall returned. This mirrors the vcpu_run + * structure shared with userspace, except is used internally to avoid + * trusting userspace to not modify the vcpu_run structure. + */ + enum { + GUNYAH_VCPU_RUN_STATE_UNKNOWN = 0, + GUNYAH_VCPU_RUN_STATE_READY, + GUNYAH_VCPU_RUN_STATE_MMIO_READ, + GUNYAH_VCPU_RUN_STATE_MMIO_WRITE, + GUNYAH_VCPU_RUN_STATE_SYSTEM_DOWN, + } state; + + bool immediate_exit; + struct completion ready; + + struct notifier_block nb; + struct gunyah_vm_resource_ticket ticket; +}; + /******************************************************************************/ /* Common arch-independent definitions for Gunyah hypercalls */ #define GUNYAH_CAPID_INVAL U64_MAX From patchwork Thu Apr 24 14:13:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884938 Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2D41B28E615 for ; Thu, 24 Apr 2025 14:14:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504057; cv=none; b=CHQnSF6cuQXZqxAKAWC9EmYDuoiDIfLmhgMas77d8W42jjD8mbESgTvg8pSGJ0SmevI2Ao8VHiotmNKCDr5MRPM+PYHOA1J+IY8ZvcVPpdD0Jbuibn4vLNUDAlvlZEFW3B6nFVz21o5HhLvh8Yk7FWhMAlwyzsgBSOu2SXQlD9Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504057; c=relaxed/simple; bh=stGjEAXhqZzqZDA4Zy4FUNW4LEqqvQTzHgd1ovyETPw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=e9r5NFHwI9AmS6B4K+/Uu/LgZJqi5shjJXDZPHxkKt4QuXp50AGa+xSO8PBqrU/L1tLUvFjk+naBl7wfdmKCmiM7B+6BObHvhWtR0/BCK44uGBMXFr7+KTHvzbPdUdw8kgjfTeBchBTkeafPOE86Rr9xQDcEMIQx59tP/TR3wFY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=c+72HnD5; arc=none smtp.client-ip=209.85.221.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="c+72HnD5" Received: by mail-wr1-f47.google.com with SMTP id ffacd0b85a97d-39c0dfba946so793798f8f.3 for ; Thu, 24 Apr 2025 07:14:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504053; x=1746108853; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XbjrAsGjkzHb8tCUfbOmumB9MeDC4bmSUl/RmrD2cWk=; b=c+72HnD5/D8MG9G0SFJbb1Xi+Ip2linbgO0v5+sPWmdtg1xKGVY7/JTb31iY2U6w/P aZcID0zyLXbxvVi+6sg2TtcXxCuM5XboTHBF7kUs70fVR9oaO7HME799SG71ZxFgG7ZN yR4fh4IYnHQUH7kZ+7ooo+vuG4xLT7Wf/7r8WDXkSX2j0WDMln6VE3MPhoKJDCLDjPvr YN13BfURiLm/PPd0CtVivdhZ8tGPV/YZUglcfVPiVZh2vyXBRMTopr5eY0uNbdSUsmHO Q1SmJPQsl3ozkuSBpqvOradrt/k/hxe1iiTTzCH8w2SZVTew6fGj+GPLahoyvvlqpw8F 732w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504053; x=1746108853; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XbjrAsGjkzHb8tCUfbOmumB9MeDC4bmSUl/RmrD2cWk=; b=dzIMkcK2A4l6QI/I2CwqiF6UlQx/Ki3XZqzl++aL4gT+4PXFwhn0/PR4in4A73gl3D NAk0Y8EshqjI/1FryKxMtINuGiNW4EosGayiLDpkLoVwrtU5aDKu1Q1nrngZwvAJoahJ JELvg9YPrDbtWe0c7iv3bMu9S3x2WRkPihIDo30T5re83wdc4TyBLjc13AIx6dq+KLqV cYwUKroQqe1SGokKCj+lH8hmah+ngty9SK2pk3K6LzKGW3RUTDqRAuPco+WUAQnBR2Fy ZQNppPizLfueOQP7lEFyFoN/A72Kcqlv+39k/W4t0sOLfK3Re2UiS6WkrQyBM5Kc6pVz QXOg== X-Forwarded-Encrypted: i=1; AJvYcCW6U1Z5pAL33KB5+qlt5W+IGO7WuCCLtMBiwzLLa6LnYfawSQRMs4L6PS6DuhuOX4jU5n5gkLpFYwBOCXck@vger.kernel.org X-Gm-Message-State: AOJu0YzQ6IbwIokkHJRT3E/xK1vlvf/P8iUqJLyXVgRm4Jwo3tpeOPIq zThHaPmu1YqjTMSWgnv4UQXynvrnGsOkylXEaP0w947Hc+kFXdLrGMpR3ooh25w= X-Gm-Gg: ASbGncu+lifGSBgq6tRt3N0T4xLMIFxFjiwyMzELZchXz3FjgW7Q6sfCDzLN0msB48H dUVvqCkuIQ7cKJp+e/w+sZjL4+F2HrGlula/Ng0dFPwQJEE0d6mgHthxdXMu0ZTU35ah0M84ktd YRdJfsSVVV2uTTAwV1MYqKOvWrQjwug3jMt0nugRizsbuMqPQjZUglWQlOZH+1hPvykFMJeVD2U hMHYLwylqgyT3xhzubqu5VMNOqILxs7ekr4tbWzHGDQxdVItBG2Xszj7Ucq7N1mQkN1kmcOm26J oQLKImQu5oZwyxU7+B50Zho9wNX7cVpaFinc6fqgcXKIpK7aO0Mq94ZKfoPFy+Bh9B+b5Mr37a8 CWesQ5dc0CBb7Uadg X-Google-Smtp-Source: AGHT+IGxrh9LsxSTvY+9a9cBpLnjZelejrN6GQ8nRlvgc6+sGPLvZ+HV3GV5bjEoSe2VS8MsJ3m6Aw== X-Received: by 2002:a05:6000:4202:b0:391:40bd:6222 with SMTP id ffacd0b85a97d-3a06cf5c719mr2302063f8f.22.1745504053364; Thu, 24 Apr 2025 07:14:13 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:12 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman , Srivatsa Vaddagiri Subject: [RFC PATCH 21/34] gunyah: Add hypercalls for demand paging Date: Thu, 24 Apr 2025 15:13:28 +0100 Message-Id: <20250424141341.841734-22-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman Three hypercalls are needed to support demand paging. In create page mappings for a virtual machine's address space, memory must be moved to a memory extent that is allowed to be mapped into that address space. Memory extents are Gunyah's implementation of access control. Once the memory is moved to the proper memory extent, the memory can be mapped into the VM's address space. Implement the bindings to perform those hypercalls. Signed-off-by: Elliot Berman Reviewed-by: Srivatsa Vaddagiri Signed-off-by: Karim Manaouil --- arch/arm64/gunyah/gunyah_hypercall.c | 87 ++++++++++++++++++++++++++++ arch/arm64/include/asm/gunyah.h | 22 +++++++ include/linux/gunyah.h | 56 ++++++++++++++++++ 3 files changed, 165 insertions(+) diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunyah_hypercall.c index fee21df42c17..38403dc28c66 100644 --- a/arch/arm64/gunyah/gunyah_hypercall.c +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -39,6 +39,9 @@ EXPORT_SYMBOL_GPL(arch_is_gunyah_guest); #define GUNYAH_HYPERCALL_HYP_IDENTIFY GUNYAH_HYPERCALL(0x8000) #define GUNYAH_HYPERCALL_MSGQ_SEND GUNYAH_HYPERCALL(0x801B) #define GUNYAH_HYPERCALL_MSGQ_RECV GUNYAH_HYPERCALL(0x801C) +#define GUNYAH_HYPERCALL_ADDRSPACE_MAP GUNYAH_HYPERCALL(0x802B) +#define GUNYAH_HYPERCALL_ADDRSPACE_UNMAP GUNYAH_HYPERCALL(0x802C) +#define GUNYAH_HYPERCALL_MEMEXTENT_DONATE GUNYAH_HYPERCALL(0x8061) #define GUNYAH_HYPERCALL_VCPU_RUN GUNYAH_HYPERCALL(0x8065) /* clang-format on */ @@ -114,6 +117,90 @@ enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid, void *buff, size_t size, } EXPORT_SYMBOL_GPL(gunyah_hypercall_msgq_recv); +/** + * gunyah_hypercall_addrspace_map() - Add memory to an address space from a memory extent + * @capid: Address space capability ID + * @extent_capid: Memory extent capability ID + * @vbase: location in address space + * @extent_attrs: Attributes for the memory + * @flags: Flags for address space mapping + * @offset: Offset into memory extent (physical address of memory) + * @size: Size of memory to map; must be page-aligned + */ +enum gunyah_error gunyah_hypercall_addrspace_map(u64 capid, u64 extent_capid, u64 vbase, + u32 extent_attrs, u32 flags, u64 offset, u64 size) +{ + struct arm_smccc_1_2_regs args = { + .a0 = GUNYAH_HYPERCALL_ADDRSPACE_MAP, + .a1 = capid, + .a2 = extent_capid, + .a3 = vbase, + .a4 = extent_attrs, + .a5 = flags, + .a6 = offset, + .a7 = size, + /* C language says this will be implictly zero. Gunyah requires 0, so be explicit */ + .a8 = 0, + }; + struct arm_smccc_1_2_regs res; + + arm_smccc_1_2_hvc(&args, &res); + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_addrspace_map); + +/** + * gunyah_hypercall_addrspace_unmap() - Remove memory from an address space + * @capid: Address space capability ID + * @extent_capid: Memory extent capability ID + * @vbase: location in address space + * @flags: Flags for address space mapping + * @offset: Offset into memory extent (physical address of memory) + * @size: Size of memory to map; must be page-aligned + */ +enum gunyah_error gunyah_hypercall_addrspace_unmap(u64 capid, u64 extent_capid, u64 vbase, + u32 flags, u64 offset, u64 size) +{ + struct arm_smccc_1_2_regs args = { + .a0 = GUNYAH_HYPERCALL_ADDRSPACE_UNMAP, + .a1 = capid, + .a2 = extent_capid, + .a3 = vbase, + .a4 = flags, + .a5 = offset, + .a6 = size, + /* C language says this will be implictly zero. Gunyah requires 0, so be explicit */ + .a7 = 0, + }; + struct arm_smccc_1_2_regs res; + + arm_smccc_1_2_hvc(&args, &res); + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_addrspace_unmap); + +/** + * gunyah_hypercall_memextent_donate() - Donate memory from one memory extent to another + * @options: donate options + * @from_capid: Memory extent capability ID to donate from + * @to_capid: Memory extent capability ID to donate to + * @offset: Offset into memory extent (physical address of memory) + * @size: Size of memory to donate; must be page-aligned + */ +enum gunyah_error gunyah_hypercall_memextent_donate(u32 options, u64 from_capid, u64 to_capid, + u64 offset, u64 size) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_MEMEXTENT_DONATE, options, from_capid, to_capid, + offset, size, 0, &res); + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_memextent_donate); + /** * gunyah_hypercall_vcpu_run() - Donate CPU time to a vcpu * @capid: capability ID of the vCPU to run diff --git a/arch/arm64/include/asm/gunyah.h b/arch/arm64/include/asm/gunyah.h index 29079d1a4df2..4adf24977fd1 100644 --- a/arch/arm64/include/asm/gunyah.h +++ b/arch/arm64/include/asm/gunyah.h @@ -32,4 +32,26 @@ static inline int arch_gunyah_fill_irq_fwspec_params(u32 virq, } return 0; } + +enum arch_gunyah_memtype { + /* clang-format off */ + GUNYAH_MEMTYPE_DEVICE_nGnRnE = 0, + GUNYAH_DEVICE_nGnRE = 1, + GUNYAH_DEVICE_nGRE = 2, + GUNYAH_DEVICE_GRE = 3, + + GUNYAH_NORMAL_NC = 0b0101, + GUNYAH_NORMAL_ONC_IWT = 0b0110, + GUNYAH_NORMAL_ONC_IWB = 0b0111, + GUNYAH_NORMAL_OWT_INC = 0b1001, + GUNYAH_NORMAL_WT = 0b1010, + GUNYAH_NORMAL_OWT_IWB = 0b1011, + GUNYAH_NORMAL_OWB_INC = 0b1101, + GUNYAH_NORMAL_OWB_IWT = 0b1110, + GUNYAH_NORMAL_WB = 0b1111, + /* clang-format on */ +}; + +#define ARCH_GUNYAH_DEFAULT_MEMTYPE GUNYAH_NORMAL_WB + #endif diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index fa6e3fd4bee1..2648d3a623de 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -270,6 +270,62 @@ enum gunyah_error gunyah_hypercall_msgq_send(u64 capid, size_t size, void *buff, enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid, void *buff, size_t size, size_t *recv_size, bool *ready); +#define GUNYAH_ADDRSPACE_SELF_CAP 0 + +enum gunyah_pagetable_access { + /* clang-format off */ + GUNYAH_PAGETABLE_ACCESS_NONE = 0, + GUNYAH_PAGETABLE_ACCESS_X = 1, + GUNYAH_PAGETABLE_ACCESS_W = 2, + GUNYAH_PAGETABLE_ACCESS_R = 4, + GUNYAH_PAGETABLE_ACCESS_RX = 5, + GUNYAH_PAGETABLE_ACCESS_RW = 6, + GUNYAH_PAGETABLE_ACCESS_RWX = 7, + /* clang-format on */ +}; + +/* clang-format off */ +#define GUNYAH_MEMEXTENT_MAPPING_USER_ACCESS GENMASK_ULL(2, 0) +#define GUNYAH_MEMEXTENT_MAPPING_KERNEL_ACCESS GENMASK_ULL(6, 4) +#define GUNYAH_MEMEXTENT_MAPPING_TYPE GENMASK_ULL(23, 16) +/* clang-format on */ + +enum gunyah_memextent_donate_type { + /* clang-format off */ + GUNYAH_MEMEXTENT_DONATE_TO_CHILD = 0, + GUNYAH_MEMEXTENT_DONATE_TO_PARENT = 1, + GUNYAH_MEMEXTENT_DONATE_TO_SIBLING = 2, + GUNYAH_MEMEXTENT_DONATE_TO_PROTECTED = 3, + GUNYAH_MEMEXTENT_DONATE_FROM_PROTECTED = 4, + /* clang-format on */ +}; + +enum gunyah_addrspace_map_flag_bits { + /* clang-format off */ + GUNYAH_ADDRSPACE_MAP_FLAG_PARTIAL = 0, + GUNYAH_ADDRSPACE_MAP_FLAG_PRIVATE = 1, + GUNYAH_ADDRSPACE_MAP_FLAG_VMMIO = 2, + GUNYAH_ADDRSPACE_MAP_FLAG_NOSYNC = 31, + /* clang-format on */ +}; + +enum gunyah_error gunyah_hypercall_addrspace_map(u64 capid, u64 extent_capid, + u64 vbase, u32 extent_attrs, + u32 flags, u64 offset, + u64 size); +enum gunyah_error gunyah_hypercall_addrspace_unmap(u64 capid, u64 extent_capid, + u64 vbase, u32 flags, + u64 offset, u64 size); + +/* clang-format off */ +#define GUNYAH_MEMEXTENT_OPTION_TYPE_MASK GENMASK_ULL(7, 0) +#define GUNYAH_MEMEXTENT_OPTION_NOSYNC BIT(31) +/* clang-format on */ + +enum gunyah_error gunyah_hypercall_memextent_donate(u32 options, u64 from_capid, + u64 to_capid, u64 offset, + u64 size); + struct gunyah_hypercall_vcpu_run_resp { union { enum { From patchwork Thu Apr 24 14:13:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884036 Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 909B228F517 for ; Thu, 24 Apr 2025 14:14:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504059; cv=none; b=uUZZW2Z33be001/h1IG/OP/ruWQ9GbjQuvQSa/r4t1nSIBwFkAsHhdjDNSdFqQ93UMz5YA8lGoHQ09ULdd48CVTxC0phsKYFaZJde6PxvayzYhixsgIvxBGiOtmz+KvsZZPVf6pdKM2pE5698NCCaawq9tOexZk7JSUVMGgE7/M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504059; c=relaxed/simple; bh=5x04Nc6aN+Qwp5pR10UBtd/47cBSvolTTTY6q9NX8X0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=CRbweQY0btD1YawLJb/baVKNFrgcCiMcZarPqAzeZ3FQoyWNoR3kEpPCCILqC0Q+HVKJseJ59BwXv1ISNfTgap8bmHFTgRq5lBbm86BzZAGXyt1GIm1C22gSZJU10zIR4209H72mZIVOAESpI2b2h5x1vQtYC4I8dEQoznKnBBQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=Vo+qX0tx; arc=none smtp.client-ip=209.85.221.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="Vo+qX0tx" Received: by mail-wr1-f49.google.com with SMTP id ffacd0b85a97d-39c266c2dd5so1179140f8f.3 for ; Thu, 24 Apr 2025 07:14:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504055; x=1746108855; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2+5ILmQ9VnpjXFSHOfGNmys0dMzibsKAG6xUYDKquGI=; b=Vo+qX0txD0lSEyG/0ftOH91Av7zp+QzKC4xemOoWPqXd3LVq48luY27dYzPzgaecnC ygbLJUhxOjaEmrUhZ5hp0Q9pUQ7vJQSRZAF6/BmdFSLTmSnyY39ETKz6qJlMVZD1BxJ2 DIMk9D+fvMxFe0f2TrsVSYnd2l9iLVm7OYDaZrqAxSXncjnApI+DAnIJRsx7T15xixPt 1dCmpKe/rcwUJ6zSy2S8Eo8+glDn+5e58dx1ZhRtQ7FwvzhLS0e/zkURm+LprL4vr8AV V25Hvqi0ZQ74zVBbBCUfyWBNQrJZhjS1yEove6AlxwCHDRAevfhFIcG6oIgss9SnzdLm VOcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504055; x=1746108855; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2+5ILmQ9VnpjXFSHOfGNmys0dMzibsKAG6xUYDKquGI=; b=D44pnqZc98cFNfZ+jiCWsi4ULCpfoz8bzcB2SAaAP3WCn+XWYz3HegreChM68iwgoX exQsNfOlLfRJbb2YCxulES12pNqisLejZHlnvm4hom/7TDxPkFhnwM0oQDUWYAdbh+X4 2gja2ywv/uaFlZKmq5D7IgvyhdAD5bHIqqV51nbtMXUt94Dgw8Qx0C8APgCAbgBwWuAL oeyVfCP4FgCNiMNOWZP0U8k4duYWNImXxKtooEzvLAGS2JR/iB4rNihE6u4rscPu5/Vf 570dPYDiv7ekBLhbnMRjiBcj8cBVYZg3k91vc/TEsr79KVVhQJXq4iUgLDQJTkwA72SR Wivw== X-Forwarded-Encrypted: i=1; AJvYcCWdSsXWkhZa9+CHPQ6901+n2lacKodbmVLM0nP96sYZL0FQBdFofkYanUX2e3ZQ9p46krQ99qRofUP5JNWF@vger.kernel.org X-Gm-Message-State: AOJu0YzNSfjVnyOyz1l68He67xMrjdpJ2kud28xJVnmdz3xaPdsH2EG8 ZzajWpokbeKqWlGmZQOrETrn+ZW8tpDLrk6XhpsCcVDdNRYeXtLyAbMO4ixQ0D0= X-Gm-Gg: ASbGncuL2Y2ndTnsKaFHv94UO9qk94hQxNu6ogmPtfkJiP6bS7Bg8MNpRoRXPz4xM0k VN0XtLiSVrV5q/yvs1v/8kyBmbnpFmyp/rSESqR6rvIIEEA+bYYz6/bCdHfCgpx2c415/C8cZRP uL0p2bWw4QucaXnEAGKKy+YSLfZTfWvikDkcs4P8I3c7mlwgZcBLoIP7LAUUM7jirULEVc9jmvD rO6e2EuIHSqY8mfCdqTeKH3rAslbDHhZzigWcIQal8jceWKYfIYshQ1nlyW+0eRETSwU/4EO+eh f8P8TvixODojkmw6yB3ye8RIx8vKeYWaet0je2PjvYqAoKwo1EU3ECAmNeBN6E0YT6Euv51zs0S CaLp7MkrmrOgC49SoyEsYnYpmBzg= X-Google-Smtp-Source: AGHT+IFKyLoVojvk9QX3IrdV6JaO3B9fcEr46gjxxC0HMPFKU10zHQ1xzS7LtC4gCESH366Qprg02w== X-Received: by 2002:a5d:64e6:0:b0:39c:dfa:ca71 with SMTP id ffacd0b85a97d-3a06cfab61dmr2330540f8f.49.1745504054653; Thu, 24 Apr 2025 07:14:14 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:14 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 22/34] gunyah: Add memory parcel RPC Date: Thu, 24 Apr 2025 15:13:29 +0100 Message-Id: <20250424141341.841734-23-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman In a Gunyah hypervisor system using the Gunyah Resource Manager, the "standard" unit of donating, lending and sharing memory is called a memory parcel (memparcel). A memparcel is an abstraction used by the resource manager for securely managing donating, lending and sharing memory, which may be physically and virtually fragmented, without dealing directly with physical memory addresses. Memparcels are created and managed through the RM RPC functions for lending, sharing and reclaiming memory from VMs. When creating a new VM the initial VM memory containing the VM image and the VM's device tree blob must be provided as a memparcel. The memparcel must be created using the RM RPC for lending and mapping the memory to the VM. Signed-off-by: Elliot Berman Reviewed-by: Srivatsa Vaddagiri Signed-off-by: Karim Manaouil --- drivers/virt/gunyah/rsc_mgr_rpc.c | 204 ++++++++++++++++++++++++++++++ include/linux/gunyah_rsc_mgr.h | 50 ++++++++ 2 files changed, 254 insertions(+) diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mgr_rpc.c index 936592177ddb..0266c2a8d583 100644 --- a/drivers/virt/gunyah/rsc_mgr_rpc.c +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -7,6 +7,12 @@ #include #include +/* Message IDs: Memory Management */ +#define GUNYAH_RM_RPC_MEM_LEND 0x51000012 +#define GUNYAH_RM_RPC_MEM_SHARE 0x51000013 +#define GUNYAH_RM_RPC_MEM_RECLAIM 0x51000015 +#define GUNYAH_RM_RPC_MEM_APPEND 0x51000018 + /* Message IDs: VM Management */ /* clang-format off */ #define GUNYAH_RM_RPC_VM_ALLOC_VMID 0x56000001 @@ -23,6 +29,49 @@ #define GUNYAH_RM_RPC_VM_SET_ADDRESS_LAYOUT 0x56000034 /* clang-format on */ +/* Call: MEM_LEND, MEM_SHARE */ +#define GUNYAH_RM_MAX_MEM_ENTRIES 512 + +#define GUNYAH_MEM_SHARE_REQ_FLAGS_APPEND BIT(1) + +struct gunyah_rm_mem_share_req_header { + u8 mem_type; + u8 _padding0; + u8 flags; + u8 _padding1; + __le32 label; +} __packed; + +struct gunyah_rm_mem_share_req_acl_section { + __le16 n_entries; + __le16 _padding; + struct gunyah_rm_mem_acl_entry entries[]; +} __packed; + +struct gunyah_rm_mem_share_req_mem_section { + __le16 n_entries; + __le16 _padding; + struct gunyah_rm_mem_entry entries[]; +} __packed; + +/* Call: MEM_RELEASE */ +struct gunyah_rm_mem_release_req { + __le32 mem_handle; + u8 flags; /* currently not used */ + u8 _padding0; + __le16 _padding1; +} __packed; + +/* Call: MEM_APPEND */ +#define GUNYAH_MEM_APPEND_REQ_FLAGS_END BIT(0) + +struct gunyah_rm_mem_append_req_header { + __le32 mem_handle; + u8 flags; + u8 _padding0; + __le16 _padding1; +} __packed; + struct gunyah_rm_vm_common_vmid_req { __le16 vmid; __le16 _padding; @@ -72,6 +121,161 @@ static int gunyah_rm_common_vmid_call(struct gunyah_rm *rm, u32 message_id, NULL, NULL); } +static int gunyah_rm_mem_append(struct gunyah_rm *rm, u32 mem_handle, + struct gunyah_rm_mem_entry *entries, + size_t n_entries) +{ + struct gunyah_rm_mem_append_req_header *req __free(kfree) = NULL; + struct gunyah_rm_mem_share_req_mem_section *mem; + int ret = 0; + size_t n; + + req = kzalloc(sizeof(*req) + struct_size(mem, entries, GUNYAH_RM_MAX_MEM_ENTRIES), + GFP_KERNEL); + if (!req) + return -ENOMEM; + + req->mem_handle = cpu_to_le32(mem_handle); + mem = (void *)(req + 1); + + while (n_entries) { + req->flags = 0; + if (n_entries > GUNYAH_RM_MAX_MEM_ENTRIES) { + n = GUNYAH_RM_MAX_MEM_ENTRIES; + } else { + req->flags |= GUNYAH_MEM_APPEND_REQ_FLAGS_END; + n = n_entries; + } + + mem->n_entries = cpu_to_le16(n); + memcpy(mem->entries, entries, sizeof(*entries) * n); + + ret = gunyah_rm_call(rm, GUNYAH_RM_RPC_MEM_APPEND, req, + sizeof(*req) + struct_size(mem, entries, n), + NULL, NULL); + if (ret) + break; + + entries += n; + n_entries -= n; + } + + return ret; +} + +/** + * gunyah_rm_mem_share() - Share memory with other virtual machines. + * @rm: Handle to a Gunyah resource manager + * @p: Information about the memory to be shared. + * + * Sharing keeps Linux's access to the memory while the memory parcel is shared. + */ +int gunyah_rm_mem_share(struct gunyah_rm *rm, struct gunyah_rm_mem_parcel *p) +{ + u32 message_id = p->n_acl_entries == 1 ? GUNYAH_RM_RPC_MEM_LEND : + GUNYAH_RM_RPC_MEM_SHARE; + size_t msg_size, initial_mem_entries = p->n_mem_entries, resp_size; + struct gunyah_rm_mem_share_req_acl_section *acl; + struct gunyah_rm_mem_share_req_mem_section *mem; + struct gunyah_rm_mem_share_req_header *req_header; + size_t acl_size, mem_size; + u32 *attr_section; + bool need_append = false; + __le32 *resp; + void *msg; + int ret; + + if (!p->acl_entries || !p->n_acl_entries || !p->mem_entries || + !p->n_mem_entries || p->n_acl_entries > U8_MAX || + p->mem_handle != GUNYAH_MEM_HANDLE_INVAL) + return -EINVAL; + + if (initial_mem_entries > GUNYAH_RM_MAX_MEM_ENTRIES) { + initial_mem_entries = GUNYAH_RM_MAX_MEM_ENTRIES; + need_append = true; + } + + acl_size = struct_size(acl, entries, p->n_acl_entries); + mem_size = struct_size(mem, entries, initial_mem_entries); + + /* The format of the message goes: + * request header + * ACL entries (which VMs get what kind of access to this memory parcel) + * Memory entries (list of memory regions to share) + * Memory attributes (currently unused, we'll hard-code the size to 0) + */ + msg_size = sizeof(struct gunyah_rm_mem_share_req_header) + acl_size + + mem_size + + sizeof(u32); /* for memory attributes, currently unused */ + + msg = kzalloc(msg_size, GFP_KERNEL); + if (!msg) + return -ENOMEM; + + req_header = msg; + acl = (void *)req_header + sizeof(*req_header); + mem = (void *)acl + acl_size; + attr_section = (void *)mem + mem_size; + + req_header->mem_type = p->mem_type; + if (need_append) + req_header->flags |= GUNYAH_MEM_SHARE_REQ_FLAGS_APPEND; + req_header->label = cpu_to_le32(p->label); + + acl->n_entries = cpu_to_le32(p->n_acl_entries); + memcpy(acl->entries, p->acl_entries, + flex_array_size(acl, entries, p->n_acl_entries)); + + mem->n_entries = cpu_to_le16(initial_mem_entries); + memcpy(mem->entries, p->mem_entries, + flex_array_size(mem, entries, initial_mem_entries)); + + /* Set n_entries for memory attribute section to 0 */ + *attr_section = 0; + + ret = gunyah_rm_call(rm, message_id, msg, msg_size, (void **)&resp, + &resp_size); + kfree(msg); + + if (ret) + return ret; + + p->mem_handle = le32_to_cpu(*resp); + kfree(resp); + + if (need_append) { + ret = gunyah_rm_mem_append( + rm, p->mem_handle, &p->mem_entries[initial_mem_entries], + p->n_mem_entries - initial_mem_entries); + if (ret) { + gunyah_rm_mem_reclaim(rm, p); + p->mem_handle = GUNYAH_MEM_HANDLE_INVAL; + } + } + + return ret; +} +ALLOW_ERROR_INJECTION(gunyah_rm_mem_share, ERRNO); + +/** + * gunyah_rm_mem_reclaim() - Reclaim a memory parcel + * @rm: Handle to a Gunyah resource manager + * @parcel: Information about the memory to be reclaimed. + * + * RM maps the associated memory back into the stage-2 page tables of the owner VM. + */ +int gunyah_rm_mem_reclaim(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *parcel) +{ + struct gunyah_rm_mem_release_req req = { + .mem_handle = cpu_to_le32(parcel->mem_handle), + }; + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_MEM_RECLAIM, &req, sizeof(req), + NULL, NULL); +} +ALLOW_ERROR_INJECTION(gunyah_rm_mem_reclaim, ERRNO); + /** * gunyah_rm_alloc_vmid() - Allocate a new VM in Gunyah. Returns the VM identifier. * @rm: Handle to a Gunyah resource manager diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index c0fe516d54a8..c42a0cb42ba6 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -9,6 +9,7 @@ #include #define GUNYAH_VMID_INVAL U16_MAX +#define GUNYAH_MEM_HANDLE_INVAL U32_MAX struct gunyah_rm; @@ -57,6 +58,55 @@ struct gunyah_rm_vm_status_payload { __le16 app_status; } __packed; +#define GUNYAH_RM_ACL_X BIT(0) +#define GUNYAH_RM_ACL_W BIT(1) +#define GUNYAH_RM_ACL_R BIT(2) + +struct gunyah_rm_mem_acl_entry { + __le16 vmid; + u8 perms; + u8 reserved; +} __packed; + +struct gunyah_rm_mem_entry { + __le64 phys_addr; + __le64 size; +} __packed; + +enum gunyah_rm_mem_type { + GUNYAH_RM_MEM_TYPE_NORMAL = 0, + GUNYAH_RM_MEM_TYPE_IO = 1, +}; + +/* + * struct gunyah_rm_mem_parcel - Info about memory to be lent/shared/donated/reclaimed + * @mem_type: The type of memory: normal (DDR) or IO + * @label: An client-specified identifier which can be used by the other VMs to identify the purpose + * of the memory parcel. + * @n_acl_entries: Count of the number of entries in the @acl_entries array. + * @acl_entries: An array of access control entries. Each entry specifies a VM and what access + * is allowed for the memory parcel. + * @n_mem_entries: Count of the number of entries in the @mem_entries array. + * @mem_entries: An array of regions to be associated with the memory parcel. Addresses should be + * (intermediate) physical addresses from Linux's perspective. + * @mem_handle: On success, filled with memory handle that RM allocates for this memory parcel + */ +struct gunyah_rm_mem_parcel { + enum gunyah_rm_mem_type mem_type; + u32 label; + size_t n_acl_entries; + struct gunyah_rm_mem_acl_entry *acl_entries; + size_t n_mem_entries; + struct gunyah_rm_mem_entry *mem_entries; + u32 mem_handle; +}; + +/* RPC Calls */ +int gunyah_rm_mem_share(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *parcel); +int gunyah_rm_mem_reclaim(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *parcel); + int gunyah_rm_alloc_vmid(struct gunyah_rm *rm, u16 vmid); int gunyah_rm_dealloc_vmid(struct gunyah_rm *rm, u16 vmid); int gunyah_rm_vm_reset(struct gunyah_rm *rm, u16 vmid); From patchwork Thu Apr 24 14:13:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884936 Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B15AF1B3929 for ; Thu, 24 Apr 2025 14:14:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504064; cv=none; b=J1LCx7XZLa85dPhnJ5tij+fd/E+M8VbrAbF1d9G4UQeVCggzWS8iR2w8G/m8pJjvNS6gsRFHQrESCy8kIdbVWK6RrokBaE8QMkutJb8qLZoB1z7uApW9RzHe3sGnc1wTH5fed0tiicfhktqrDTJP0/MnRGYJlh/39X3+pAg9PYI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504064; c=relaxed/simple; bh=NPLvNPRTGZlQyPSKuchNVhThznQp0bCS99QwrEGsnQg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kVMNjkn5Mk0pq0rI8Pbyk3cWKkGdTgiYPpYfKvAKgNWe8Y8GivPvG36pbFKvzJqsSqQe03tjf7+tXDHA8q3GrQaXKqwvBNrtmQ/eHs3sKS2cZhaL8zARFaHrZ42+Q4thq4AygruZpBpjqn6FIBnxRRZC7LmRshisrLUSKpCiiqw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=LGaJZa+G; arc=none smtp.client-ip=209.85.128.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="LGaJZa+G" Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-43cf848528aso7716405e9.2 for ; Thu, 24 Apr 2025 07:14:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504056; x=1746108856; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2LhbI7ata+xuMo4MQ3LDVVCBYF7iuDckOVgkFelFFVk=; b=LGaJZa+Gqb1A6S6wHwdOIGAHjxh7A1/kh4PeILy9RALRqV113f7vzYy1odu2l7vDad EGO/7UIQ+8a/vpk9fqX4WLk9mgwDX8Iqcgh5Po46E6eTHRQqJi+ePdSBD97nvLHLh2lV +ayjuxpSwfFQuHTTzrZACtNRtbsOMvO7svd1tlUGmopAuMl7qbpMEZxhTvCGdqVdatIC ZumxSbQ7yTOs0+CipPSOO+VKWmdUw3/IaxRWuKiox9SJWFDazZyqw5SujqsRLLpGx54c 5yS0QvcZM5+Ac8QSB3TW3YT/3YhvunOXPS3Php1i2WecpS6diPIMdBBEQPF2Oj71erem 0cNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504056; x=1746108856; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2LhbI7ata+xuMo4MQ3LDVVCBYF7iuDckOVgkFelFFVk=; b=EPLpdD9eX7YqXAoRAJBua+8bt9z6fuwjD9tvzIM/KjHE4lwdlH6dKZJLTT8yMmFb01 x2QPcGDilka0m2Fy5ZkSQgLb7VlcSro1foqx+xLCPivBN8rquGpBHFot7Trl/hpAGJ7C Ml1OUZKReAqaBmBIJ9mn7uh/XFEmL/o5Ofph042YnGPYxIlicU5m21bTyETa0pPH3vtw Yw7K8cGWeduVVcR7G4P7gQChF05AJHpF3T1zWNqngzzbflLzMvWqeFRaeJELjiY4ijke +oGjZrd8oTd9n5whEt7KXicEeq9Jw9Of3gUwT3h7w6vFpu1HIsWE504vRffyh7qttEr/ AXvA== X-Forwarded-Encrypted: i=1; AJvYcCVxE6vMep2e2Oup0ogc43e4YMtJ5Yx0ARcS4fdaytPN4DwuzH5HI/UH5Sw7Aj12FxXGE++4FGUq//TzlCSK@vger.kernel.org X-Gm-Message-State: AOJu0YzM3Vut3Vfz15+CEGfSBI99t9IImy5CeiZ+sfLGEAS1oUwy7G0t IpqT2MiYAMKSwqz5WKMxAEQk7UjsZkW7YXszF8kWtvcyu9GEbFeBp7mrkkutwek= X-Gm-Gg: ASbGncs8rY8TkCiKzs66uGM9xy6Iz/qaKn/fD++R4JxzH4TbXcahV6NLISDrrGWc9rC JriTF756UJokAopIwYCbtI9HumT3LjRO8Oo2XDGg3npP3AQ26AsrILnADFXQPjJ5Moz8aWWplAD k+2gQiSH8vAcPWx+7a3F3GAlAgMXZ8VnC0fbF/zanj4+1yaETHxQ3DVXKI9A8hWTCjR0Zq/bTNH u0PqCMJsi2GhbSp0r56xbXDJdZ2qS0MDHdmdKpubCWelgBfGPw+rln7i2l1/fM/AmawUjHO3jA4 Me1ZRLdgmlwHxeb/r9ibrm0T7VFmxcGKvPE9Xncx9BogTU0wpZSqJWW5XBgc3BW/cEu2LklbFs7 IItdT7r80Jh2ANbxN X-Google-Smtp-Source: AGHT+IHqcrFkop2AMczZB7InA+Ahpw3ClIzGk7Xop+jnijaBOC/Y4QzTBvOvzogG1ZcwEVc5x1XCjQ== X-Received: by 2002:a05:600c:8106:b0:440:6852:5b31 with SMTP id 5b1f17b1804b1-4409bd0d902mr31333525e9.10.1745504055954; Thu, 24 Apr 2025 07:14:15 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:15 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 23/34] gunyah: Add interfaces to map memory into guest address space Date: Thu, 24 Apr 2025 15:13:30 +0100 Message-Id: <20250424141341.841734-24-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman This patch is a subset of [1], without gunyah guest-memfd parts. I added the original commit message below. [1] https://lore.kernel.org/lkml/20240222-gunyah-v17-20-1e9da6763d38@quicinc.com/ --- arch/arm64/kvm/gunyah.c | 212 +++++++++++++++++++++++++++++++++++++++- include/linux/gunyah.h | 6 ++ 2 files changed, 217 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index e066482c2e71..5d18d133df50 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -10,8 +10,8 @@ #include #include #include -#include #include +#include #include #include @@ -19,6 +19,15 @@ #undef pr_fmt #define pr_fmt(fmt) "gunyah: " fmt +#define GUNYAH_VM_ADDRSPACE_LABEL 0 +#define GUNYAH_VM_MEM_EXTENT_GUEST_PRIVATE_LABEL 0 +#define GUNYAH_VM_MEM_EXTENT_HOST_SHARED_LABEL 1 +#define GUNYAH_VM_MEM_EXTENT_GUEST_SHARED_LABEL 3 +#define GUNYAH_VM_MEM_EXTENT_HOST_PRIVATE_LABEL 2 + +#define WRITE_TAG (1 << 0) +#define SHARE_TAG (1 << 1) + static int gunyah_vm_start(struct gunyah_vm *ghvm); static enum kvm_mode kvm_mode = KVM_MODE_DEFAULT; @@ -332,6 +341,23 @@ int kvm_arm_set_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg) } } +static bool gunyah_vm_resource_ticket_populate_noop( + struct gunyah_vm_resource_ticket *ticket, struct gunyah_resource *ghrsc) +{ + return true; +} +static void gunyah_vm_resource_ticket_unpopulate_noop( + struct gunyah_vm_resource_ticket *ticket, struct gunyah_resource *ghrsc) +{ +} + +static inline struct gunyah_resource * +__first_resource(struct gunyah_vm_resource_ticket *ticket) +{ + return list_first_entry_or_null(&ticket->resources, + struct gunyah_resource, list); +} + static int gunyah_vm_add_resource_ticket(struct gunyah_vm *ghvm, struct gunyah_vm_resource_ticket *ticket) { @@ -430,6 +456,159 @@ static void gunyah_vm_clean_resources(struct gunyah_vm *ghvm) mutex_unlock(&ghvm->resources_lock); } +static inline u32 donate_flags(bool share) +{ + if (share) + return FIELD_PREP_CONST(GUNYAH_MEMEXTENT_OPTION_TYPE_MASK, + GUNYAH_MEMEXTENT_DONATE_TO_SIBLING); + else + return FIELD_PREP_CONST(GUNYAH_MEMEXTENT_OPTION_TYPE_MASK, + GUNYAH_MEMEXTENT_DONATE_TO_PROTECTED); +} + +static inline u32 reclaim_flags(bool share) +{ + if (share) + return FIELD_PREP_CONST(GUNYAH_MEMEXTENT_OPTION_TYPE_MASK, + GUNYAH_MEMEXTENT_DONATE_TO_SIBLING); + else + return FIELD_PREP_CONST(GUNYAH_MEMEXTENT_OPTION_TYPE_MASK, + GUNYAH_MEMEXTENT_DONATE_FROM_PROTECTED); +} + +static int gunyah_memory_provide_folio(struct gunyah_vm *ghvm, + struct folio *folio, gfn_t gfn, bool share, bool write) +{ + struct gunyah_resource *guest_extent, *host_extent, *addrspace; + u32 map_flags = BIT(GUNYAH_ADDRSPACE_MAP_FLAG_PARTIAL); + u64 extent_attrs; + gfn_t gpa = gfn_to_gpa(gfn); + phys_addr_t pa = PFN_PHYS(folio_pfn(folio)); + enum gunyah_pagetable_access access; + size_t size = folio_size(folio); + enum gunyah_error gunyah_error; + unsigned long tag = 0; + int ret; + + if (share) { + guest_extent = __first_resource(&ghvm->guest_shared_extent_ticket); + host_extent = __first_resource(&ghvm->host_shared_extent_ticket); + } else { + guest_extent = __first_resource(&ghvm->guest_private_extent_ticket); + host_extent = __first_resource(&ghvm->host_private_extent_ticket); + } + addrspace = __first_resource(&ghvm->addrspace_ticket); + + if (!addrspace || !guest_extent || !host_extent) + return -ENODEV; + + if (share) { + map_flags |= BIT(GUNYAH_ADDRSPACE_MAP_FLAG_VMMIO); + tag |= SHARE_TAG; + } else { + map_flags |= BIT(GUNYAH_ADDRSPACE_MAP_FLAG_PRIVATE); + } + + if (write) + tag |= WRITE_TAG; + + if (share && write) + access = GUNYAH_PAGETABLE_ACCESS_RW; + else if (share && !write) + access = GUNYAH_PAGETABLE_ACCESS_R; + else if (!share && write) + access = GUNYAH_PAGETABLE_ACCESS_RWX; + else /* !share && !write */ + access = GUNYAH_PAGETABLE_ACCESS_RX; + + gunyah_error = gunyah_hypercall_memextent_donate(donate_flags(share), + host_extent->capid, + guest_extent->capid, + pa, size); + if (gunyah_error != GUNYAH_ERROR_OK) { + pr_err("Failed to donate memory for guest address 0x%016llx: %d\n", + gpa, gunyah_error); + return gunyah_error_remap(gunyah_error); + } + + extent_attrs = + FIELD_PREP_CONST(GUNYAH_MEMEXTENT_MAPPING_TYPE, + ARCH_GUNYAH_DEFAULT_MEMTYPE) | + FIELD_PREP(GUNYAH_MEMEXTENT_MAPPING_USER_ACCESS, access) | + FIELD_PREP(GUNYAH_MEMEXTENT_MAPPING_KERNEL_ACCESS, access); + gunyah_error = gunyah_hypercall_addrspace_map(addrspace->capid, + guest_extent->capid, gpa, + extent_attrs, map_flags, + pa, size); + if (gunyah_error != GUNYAH_ERROR_OK) { + pr_err("Failed to map guest address 0x%016llx: %d\n", gpa, + gunyah_error); + ret = gunyah_error_remap(gunyah_error); + goto memextent_reclaim; + } + + return 0; +memextent_reclaim: + gunyah_error = gunyah_hypercall_memextent_donate(reclaim_flags(share), + guest_extent->capid, + host_extent->capid, pa, + size); + if (gunyah_error != GUNYAH_ERROR_OK) + pr_err("Failed to reclaim memory donation for guest address 0x%016llx: %d\n", + gpa, gunyah_error); + return ret; +} + +static int gunyah_memory_reclaim_folio(struct gunyah_vm *ghvm, + struct folio *folio, gfn_t gfn, bool share) +{ + u32 map_flags = BIT(GUNYAH_ADDRSPACE_MAP_FLAG_PARTIAL); + struct gunyah_resource *guest_extent, *host_extent, *addrspace; + enum gunyah_error gunyah_error; + phys_addr_t pa; + size_t size; + int ret; + + addrspace = __first_resource(&ghvm->addrspace_ticket); + if (!addrspace) + return -ENODEV; + + guest_extent = __first_resource(&ghvm->guest_private_extent_ticket); + host_extent = __first_resource(&ghvm->host_private_extent_ticket); + map_flags |= BIT(GUNYAH_ADDRSPACE_MAP_FLAG_PRIVATE); + + pa = PFN_PHYS(folio_pfn(folio)); + size = folio_size(folio); + + gunyah_error = gunyah_hypercall_addrspace_unmap(addrspace->capid, + guest_extent->capid, + gfn_to_gpa(gfn), + map_flags, pa, size); + if (gunyah_error != GUNYAH_ERROR_OK) { + pr_err_ratelimited( + "Failed to unmap guest address 0x%016llx: %d\n", + gfn_to_gpa(gfn), gunyah_error); + ret = gunyah_error_remap(gunyah_error); + goto err; + } + + gunyah_error = gunyah_hypercall_memextent_donate(reclaim_flags(share), + guest_extent->capid, + host_extent->capid, pa, + size); + if (gunyah_error != GUNYAH_ERROR_OK) { + pr_err_ratelimited( + "Failed to reclaim memory donation for guest address 0x%016llx: %d\n", + gfn_to_gpa(gfn), gunyah_error); + ret = gunyah_error_remap(gunyah_error); + goto err; + } + + return 0; +err: + return ret; +} + int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE; @@ -1357,6 +1536,17 @@ static int gunyah_vm_start(struct gunyah_vm *ghvm) return ret; } +static inline void setup_extent_ticket(struct gunyah_vm *ghvm, + struct gunyah_vm_resource_ticket *ticket, + u32 label) +{ + ticket->resource_type = GUNYAH_RESOURCE_TYPE_MEM_EXTENT; + ticket->label = label; + ticket->populate = gunyah_vm_resource_ticket_populate_noop; + ticket->unpopulate = gunyah_vm_resource_ticket_unpopulate_noop; + gunyah_vm_add_resource_ticket(ghvm, ticket); +} + static struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) { struct gunyah_vm *ghvm; @@ -1375,6 +1565,20 @@ static struct gunyah_vm *gunyah_vm_alloc(struct gunyah_rm *rm) INIT_LIST_HEAD(&ghvm->resources); INIT_LIST_HEAD(&ghvm->resource_tickets); + ghvm->addrspace_ticket.resource_type = GUNYAH_RESOURCE_TYPE_ADDR_SPACE; + ghvm->addrspace_ticket.label = GUNYAH_VM_ADDRSPACE_LABEL; + ghvm->addrspace_ticket.populate = gunyah_vm_resource_ticket_populate_noop; + ghvm->addrspace_ticket.unpopulate = gunyah_vm_resource_ticket_unpopulate_noop; + gunyah_vm_add_resource_ticket(ghvm, &ghvm->addrspace_ticket); + + setup_extent_ticket(ghvm, &ghvm->host_private_extent_ticket, + GUNYAH_VM_MEM_EXTENT_HOST_PRIVATE_LABEL); + setup_extent_ticket(ghvm, &ghvm->host_shared_extent_ticket, + GUNYAH_VM_MEM_EXTENT_HOST_SHARED_LABEL); + setup_extent_ticket(ghvm, &ghvm->guest_private_extent_ticket, + GUNYAH_VM_MEM_EXTENT_GUEST_PRIVATE_LABEL); + setup_extent_ticket(ghvm, &ghvm->guest_shared_extent_ticket, + GUNYAH_VM_MEM_EXTENT_GUEST_SHARED_LABEL); return ghvm; } @@ -1389,6 +1593,12 @@ static void gunyah_destroy_vm(struct gunyah_vm *ghvm) if (ghvm->vm_status == GUNYAH_RM_VM_STATUS_RUNNING) gunyah_vm_stop(ghvm); + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->addrspace_ticket); + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->host_shared_extent_ticket); + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->host_private_extent_ticket); + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->guest_shared_extent_ticket); + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->guest_private_extent_ticket); + gunyah_vm_clean_resources(ghvm); if (ghvm->vm_status == GUNYAH_RM_VM_STATUS_EXITED || diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 2648d3a623de..d1048d0fc246 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -14,6 +14,7 @@ #include #include +#include #include #define gunyah_vcpu(kvm_vcpu_ptr) \ @@ -107,6 +108,11 @@ struct gunyah_vm { struct list_head resources; struct list_head resource_tickets; enum gunyah_rm_vm_auth_mechanism auth; + struct gunyah_vm_resource_ticket addrspace_ticket; + struct gunyah_vm_resource_ticket host_private_extent_ticket; + struct gunyah_vm_resource_ticket host_shared_extent_ticket; + struct gunyah_vm_resource_ticket guest_private_extent_ticket; + struct gunyah_vm_resource_ticket guest_shared_extent_ticket; }; /** From patchwork Thu Apr 24 14:13:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884937 Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3256C290BA4 for ; Thu, 24 Apr 2025 14:14:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504063; cv=none; b=cRwCjmFTFSOSE5i73HkcX0C9jlUQZcKPM4by9OiK+MjdWzRnuPUTq6o9tk3tZkjETQ2Cz4RBDrapdiWPZjWzEV2jyUlAf0UdCb6tlFE+/7FCw3zwkN0OLDagI4thKOyrOdskKclO0Hu2ury4d8spnUobvWPsuEbYqVaG+DoOLMs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504063; c=relaxed/simple; bh=vLbxwMoJK25jtdiinc5JboXzA9vEq1NIw2shYX4meSY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Rg77hKB/arHA4NQK/6/j5oArSd/EZ2UUzUFjbaklNckZAgNQxDsqv1aN3rsP4wLkZiMG+UeZF/HrBgSrPSzKK5kSlnAFbxDTydH7bVennwVEeYo3pwfG2FZ5NBpwZKgZkdeWPO1esR9iUc8lFtTH402mJMI+iI/OFIFWJrS2D7g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=V1AW5rAW; arc=none smtp.client-ip=209.85.221.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="V1AW5rAW" Received: by mail-wr1-f41.google.com with SMTP id ffacd0b85a97d-39c0dfad22aso840803f8f.2 for ; Thu, 24 Apr 2025 07:14:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504057; x=1746108857; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YXeykB8FkcDuL2Q2W63/7k8ahxVIXL64cFUzN9R5114=; b=V1AW5rAWrfLG1Wkq/87KzJUariKgRAJdGvyfb3BPRdNwLW0FK72L3c9c0jBBXVAWyJ 3uhlbzwZO5P44RGLoNPqy6hqRo+K6BS+8X+oJbGEDcD2u5ziLSPhQyl6froafzR5x2Pj W56JpMYuWxH8iQ5NCPwma7xAfz7VjRJ+HUjGqdBgBP7o14nB3mY3ntsB/13uvlIF0t7v PeeB6LCB5wS0t5MD6QUXJSTQ1A3UVHSnxnlyXgSvMqVW181kFj2gc3OrTjwm/qNIhhVG Ks6f5LSXc2a9yUO5eRMQBc4wXwkuVnnHcCmvMbP3O6hkh/aGjrC9ZXgu4iyndXiS5YbQ ZwRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504057; x=1746108857; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YXeykB8FkcDuL2Q2W63/7k8ahxVIXL64cFUzN9R5114=; b=i5xUzdyOPOWGqXv0QG2GV7geOVPoOPhZO8A30x+N9s/c52r5b/kXEK9Aq5Dp7u4p2C akABpPei6TvfuBO2Xe7r/exFsTCgSrW29dlLAuworGym34WgvIeQCg+srwysb1tbYtdS zmvFtERih7fH0Kbmfi5etbgwVTsPhFHSBGhrGBhDcqMfyxtckTHx/cYpFKS+9Ke1BVQ9 l1Y6lLs/pedDhmQ+q6MSF+ejHVVHpBYbxpebIz90VVly3E/f/tPLWpkQm99P2Hlmj4q3 lrOFQFolQNG6jKVOh529RKS5AQl7vVvLN0TW+Q4JmCm1akvwcJYH1qeTshbXTV8hk+rs S45w== X-Forwarded-Encrypted: i=1; AJvYcCXP4Afx0sjpw1dIyPGjOCrO+ti2cb5UdfCM1IaEakXejcZLBm34wJTviuq93roQCd0omoMKVYCejOf8YmNO@vger.kernel.org X-Gm-Message-State: AOJu0YyHrUzjxx7dhxUk+DN6y8Dda92nSWWp3lc1eLKCC/dFz51wM2IC aj/2cXGsvQLbNJ+wc6NaVEMlRAFPt/zhltRdQSmD6CYDW7H3NK5dqb9e56QX+W0= X-Gm-Gg: ASbGncsIq+FgoEUnuZYN2EulJ1JYstIBEyj6h9b3NeEeHXSruBxQJiBGVbJeCVgD+F+ wfHbP77qT2eGk0OnYmo+EAqS7zWHkU5D/zKI9FGYYUtjdOn6kqL/chNN1GurUVhviymWNrpmmDm Yorym9nyDYgTioWei+kbCOCJpImvhkPF/+UruMXEvL+uCsysaIQeZBjmfGbdgBCEh6fy2pSDT2z xOs1sZTkst4lQxnX610YtM66G2LRIdTAYzWBXri6s2GfS/uvF2Wm/i0/L2RH/n1/BIUeBFcvyZb yqd7kEYGgBlnwNODD7Nie6Xc3HBdvQdA2Lq7GmjHBsd+R1Me21R5rfrQWtE2XRTgPvRLS+xUtDD KuttPy5U7dqM4Il9MONuLyvXWDFE= X-Google-Smtp-Source: AGHT+IFg8XOlOSbfkRtBuJWxisCALodsFHPa4IUTZp3Kj17cI2U8pp461yntnlLPPaPA3d0u/jWMRw== X-Received: by 2002:a05:6000:420f:b0:399:71d4:a9 with SMTP id ffacd0b85a97d-3a06cfb2391mr2162635f8f.52.1745504057229; Thu, 24 Apr 2025 07:14:17 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:16 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 24/34] gunyah: Add platform ops on mem_lend/mem_reclaim Date: Thu, 24 Apr 2025 15:13:31 +0100 Message-Id: <20250424141341.841734-25-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman On Qualcomm platforms, there is a firmware entity which controls access to physical pages. In order to share memory with another VM, this entity needs to be informed that the guest VM should have access to the memory. Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 29 ++++- drivers/virt/gunyah/Kconfig | 4 + drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/gunyah_platform_hooks.c | 117 ++++++++++++++++++++ drivers/virt/gunyah/rsc_mgr_rpc.c | 19 +++- include/linux/gunyah.h | 63 +++++++++-- include/linux/gunyah_rsc_mgr.h | 10 ++ 7 files changed, 226 insertions(+), 17 deletions(-) create mode 100644 drivers/virt/gunyah/gunyah_platform_hooks.c diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index 5d18d133df50..7216db642174 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -488,7 +488,7 @@ static int gunyah_memory_provide_folio(struct gunyah_vm *ghvm, size_t size = folio_size(folio); enum gunyah_error gunyah_error; unsigned long tag = 0; - int ret; + int ret, tmp; if (share) { guest_extent = __first_resource(&ghvm->guest_shared_extent_ticket); @@ -521,6 +521,11 @@ static int gunyah_memory_provide_folio(struct gunyah_vm *ghvm, else /* !share && !write */ access = GUNYAH_PAGETABLE_ACCESS_RX; + ret = gunyah_rm_platform_pre_demand_page(ghvm->rm, ghvm->vmid, access, + folio); + if (ret) + return ret; + gunyah_error = gunyah_hypercall_memextent_donate(donate_flags(share), host_extent->capid, guest_extent->capid, @@ -528,7 +533,8 @@ static int gunyah_memory_provide_folio(struct gunyah_vm *ghvm, if (gunyah_error != GUNYAH_ERROR_OK) { pr_err("Failed to donate memory for guest address 0x%016llx: %d\n", gpa, gunyah_error); - return gunyah_error_remap(gunyah_error); + ret = gunyah_error_remap(gunyah_error); + goto platform_release; } extent_attrs = @@ -556,6 +562,14 @@ static int gunyah_memory_provide_folio(struct gunyah_vm *ghvm, if (gunyah_error != GUNYAH_ERROR_OK) pr_err("Failed to reclaim memory donation for guest address 0x%016llx: %d\n", gpa, gunyah_error); +platform_release: + tmp = gunyah_rm_platform_reclaim_demand_page(ghvm->rm, ghvm->vmid, + access, folio); + if (tmp) { + pr_err("Platform failed to reclaim memory for guest address 0x%016llx: %d", + gpa, tmp); + return ret; + } return ret; } @@ -565,6 +579,7 @@ static int gunyah_memory_reclaim_folio(struct gunyah_vm *ghvm, u32 map_flags = BIT(GUNYAH_ADDRSPACE_MAP_FLAG_PARTIAL); struct gunyah_resource *guest_extent, *host_extent, *addrspace; enum gunyah_error gunyah_error; + enum gunyah_pagetable_access access; phys_addr_t pa; size_t size; int ret; @@ -604,6 +619,16 @@ static int gunyah_memory_reclaim_folio(struct gunyah_vm *ghvm, goto err; } + access = GUNYAH_PAGETABLE_ACCESS_RWX; + + ret = gunyah_rm_platform_reclaim_demand_page(ghvm->rm, ghvm->vmid, access, folio); + if (ret) { + pr_err_ratelimited( + "Platform failed to reclaim memory for guest address 0x%016llx: %d", + gfn_to_gpa(gfn), ret); + goto err; + } + return 0; err: return ret; diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 6f4c85db80b5..23ba523d25dc 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -3,6 +3,7 @@ config GUNYAH tristate "Gunyah Virtualization drivers" depends on ARM64 + select GUNYAH_PLATFORM_HOOKS help The Gunyah drivers are the helper interfaces that run in a guest VM such as basic inter-VM IPC and signaling mechanisms, and higher level @@ -10,3 +11,6 @@ config GUNYAH Say Y/M here to enable the drivers needed to interact in a Gunyah virtual environment. + +config GUNYAH_PLATFORM_HOOKS + tristate diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index b1bdf3e84155..45cabba3110c 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -3,3 +3,4 @@ gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr.o +obj-$(CONFIG_GUNYAH_PLATFORM_HOOKS) += gunyah_platform_hooks.o diff --git a/drivers/virt/gunyah/gunyah_platform_hooks.c b/drivers/virt/gunyah/gunyah_platform_hooks.c new file mode 100644 index 000000000000..8a1af171e4c9 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_platform_hooks.c @@ -0,0 +1,117 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include + +#include + +static const struct gunyah_rm_platform_ops *rm_platform_ops; +static DECLARE_RWSEM(rm_platform_ops_lock); + +int gunyah_rm_platform_pre_mem_share(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel) +{ + int ret = 0; + + down_read(&rm_platform_ops_lock); + if (rm_platform_ops && rm_platform_ops->pre_mem_share) + ret = rm_platform_ops->pre_mem_share(rm, mem_parcel); + up_read(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_rm_platform_pre_mem_share); + +int gunyah_rm_platform_post_mem_reclaim(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel) +{ + int ret = 0; + + down_read(&rm_platform_ops_lock); + if (rm_platform_ops && rm_platform_ops->post_mem_reclaim) + ret = rm_platform_ops->post_mem_reclaim(rm, mem_parcel); + up_read(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_rm_platform_post_mem_reclaim); + +int gunyah_rm_platform_pre_demand_page(struct gunyah_rm *rm, u16 vmid, + enum gunyah_pagetable_access access, + struct folio *folio) +{ + int ret = 0; + + down_read(&rm_platform_ops_lock); + if (rm_platform_ops && rm_platform_ops->pre_demand_page) + ret = rm_platform_ops->pre_demand_page(rm, vmid, access, folio); + up_read(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_rm_platform_pre_demand_page); + +int gunyah_rm_platform_reclaim_demand_page(struct gunyah_rm *rm, u16 vmid, + enum gunyah_pagetable_access access, + struct folio *folio) +{ + int ret = 0; + + down_read(&rm_platform_ops_lock); + if (rm_platform_ops && rm_platform_ops->pre_demand_page) + ret = rm_platform_ops->release_demand_page(rm, vmid, access, + folio); + up_read(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_rm_platform_reclaim_demand_page); + +int gunyah_rm_register_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops) +{ + int ret = 0; + + down_write(&rm_platform_ops_lock); + if (!rm_platform_ops) + rm_platform_ops = platform_ops; + else + ret = -EEXIST; + up_write(&rm_platform_ops_lock); + return ret; +} +EXPORT_SYMBOL_GPL(gunyah_rm_register_platform_ops); + +void gunyah_rm_unregister_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops) +{ + down_write(&rm_platform_ops_lock); + if (rm_platform_ops == platform_ops) + rm_platform_ops = NULL; + up_write(&rm_platform_ops_lock); +} +EXPORT_SYMBOL_GPL(gunyah_rm_unregister_platform_ops); + +static void _devm_gunyah_rm_unregister_platform_ops(void *data) +{ + gunyah_rm_unregister_platform_ops( + (const struct gunyah_rm_platform_ops *)data); +} + +int devm_gunyah_rm_register_platform_ops( + struct device *dev, const struct gunyah_rm_platform_ops *ops) +{ + int ret; + + ret = gunyah_rm_register_platform_ops(ops); + if (ret) + return ret; + + return devm_add_action(dev, _devm_gunyah_rm_unregister_platform_ops, + (void *)ops); +} +EXPORT_SYMBOL_GPL(devm_gunyah_rm_register_platform_ops); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("Gunyah Platform Hooks"); diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mgr_rpc.c index 0266c2a8d583..ec187d116dd7 100644 --- a/drivers/virt/gunyah/rsc_mgr_rpc.c +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -212,6 +212,12 @@ int gunyah_rm_mem_share(struct gunyah_rm *rm, struct gunyah_rm_mem_parcel *p) if (!msg) return -ENOMEM; + ret = gunyah_rm_platform_pre_mem_share(rm, p); + if (ret) { + kfree(msg); + return ret; + } + req_header = msg; acl = (void *)req_header + sizeof(*req_header); mem = (void *)acl + acl_size; @@ -237,8 +243,10 @@ int gunyah_rm_mem_share(struct gunyah_rm *rm, struct gunyah_rm_mem_parcel *p) &resp_size); kfree(msg); - if (ret) + if (ret) { + gunyah_rm_platform_post_mem_reclaim(rm, p); return ret; + } p->mem_handle = le32_to_cpu(*resp); kfree(resp); @@ -270,9 +278,14 @@ int gunyah_rm_mem_reclaim(struct gunyah_rm *rm, struct gunyah_rm_mem_release_req req = { .mem_handle = cpu_to_le32(parcel->mem_handle), }; + int ret; + + ret = gunyah_rm_call(rm, GUNYAH_RM_RPC_MEM_RECLAIM, &req, sizeof(req), + NULL, NULL); + if (ret) + return ret; - return gunyah_rm_call(rm, GUNYAH_RM_RPC_MEM_RECLAIM, &req, sizeof(req), - NULL, NULL); + return gunyah_rm_platform_post_mem_reclaim(rm, parcel); } ALLOW_ERROR_INJECTION(gunyah_rm_mem_reclaim, ERRNO); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index d1048d0fc246..1d363ab8967a 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -158,6 +158,57 @@ struct gunyah_vcpu { struct gunyah_vm_resource_ticket ticket; }; +enum gunyah_pagetable_access { + /* clang-format off */ + GUNYAH_PAGETABLE_ACCESS_NONE = 0, + GUNYAH_PAGETABLE_ACCESS_X = 1, + GUNYAH_PAGETABLE_ACCESS_W = 2, + GUNYAH_PAGETABLE_ACCESS_R = 4, + GUNYAH_PAGETABLE_ACCESS_RX = 5, + GUNYAH_PAGETABLE_ACCESS_RW = 6, + GUNYAH_PAGETABLE_ACCESS_RWX = 7, + /* clang-format on */ +}; + +struct gunyah_rm_platform_ops { + int (*pre_mem_share)(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel); + int (*post_mem_reclaim)(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel); + + int (*pre_demand_page)(struct gunyah_rm *rm, u16 vmid, + enum gunyah_pagetable_access access, + struct folio *folio); + int (*release_demand_page)(struct gunyah_rm *rm, u16 vmid, + enum gunyah_pagetable_access access, + struct folio *folio); +}; + +#if IS_ENABLED(CONFIG_GUNYAH_PLATFORM_HOOKS) +int gunyah_rm_register_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops); +void gunyah_rm_unregister_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops); +int devm_gunyah_rm_register_platform_ops( + struct device *dev, const struct gunyah_rm_platform_ops *ops); +#else +static inline int gunyah_rm_register_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops) +{ + return 0; +} +static inline void gunyah_rm_unregister_platform_ops( + const struct gunyah_rm_platform_ops *platform_ops) +{ +} +static inline int +devm_gunyah_rm_register_platform_ops(struct device *dev, + const struct gunyah_rm_platform_ops *ops) +{ + return 0; +} +#endif + /******************************************************************************/ /* Common arch-independent definitions for Gunyah hypercalls */ #define GUNYAH_CAPID_INVAL U64_MAX @@ -278,18 +329,6 @@ enum gunyah_error gunyah_hypercall_msgq_recv(u64 capid, void *buff, size_t size, #define GUNYAH_ADDRSPACE_SELF_CAP 0 -enum gunyah_pagetable_access { - /* clang-format off */ - GUNYAH_PAGETABLE_ACCESS_NONE = 0, - GUNYAH_PAGETABLE_ACCESS_X = 1, - GUNYAH_PAGETABLE_ACCESS_W = 2, - GUNYAH_PAGETABLE_ACCESS_R = 4, - GUNYAH_PAGETABLE_ACCESS_RX = 5, - GUNYAH_PAGETABLE_ACCESS_RW = 6, - GUNYAH_PAGETABLE_ACCESS_RWX = 7, - /* clang-format on */ -}; - /* clang-format off */ #define GUNYAH_MEMEXTENT_MAPPING_USER_ACCESS GENMASK_ULL(2, 0) #define GUNYAH_MEMEXTENT_MAPPING_KERNEL_ACCESS GENMASK_ULL(6, 4) diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index c42a0cb42ba6..fb3feee73490 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -155,4 +155,14 @@ void gunyah_rm_free_resource(struct gunyah_resource *ghrsc); int gunyah_rm_call(struct gunyah_rm *rsc_mgr, u32 message_id, const void *req_buf, size_t req_buf_size, void **resp_buf, size_t *resp_buf_size); + +int gunyah_rm_platform_pre_mem_share(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel); +int gunyah_rm_platform_post_mem_reclaim( + struct gunyah_rm *rm, struct gunyah_rm_mem_parcel *mem_parcel); + +int gunyah_rm_platform_pre_demand_page(struct gunyah_rm *rm, u16 vmid, + u32 flags, struct folio *folio); +int gunyah_rm_platform_reclaim_demand_page(struct gunyah_rm *rm, u16 vmid, + u32 flags, struct folio *folio); #endif From patchwork Thu Apr 24 14:13:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884035 Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A7559291148 for ; Thu, 24 Apr 2025 14:14:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504063; cv=none; b=r+ebFqmnBDmdIKkJKDJZZSNx1i5EMXyVj+zBkA47DemI3eA9WmnFgS3l/46AMfTk5FWD6Tl2F+7puQjRQDxUs5hSLrrEuxIj9H7f3pKrl/cjVCQCy01tpfjEHryHEhwvZLgiGrMO4S242olV4YSI6q8+z/au3iKq6IJOxb97YK8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504063; c=relaxed/simple; bh=jJJ9JgoTw3N1VJhubYw/1Fl9AFSEgg6AWN3vhACh320=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=SKfW8/vsRN+hX4Dc2XIcnROGTrnYNBeia2qEyksEEDxo9/0FiqEBjdDFOo+345G3zOoPfnRl124PHvgseUjBGHB+8rgdTfC6xvtt/fP2RCYDwbe/MiqPetd4zEVyUyjDq8TmilDWaKpB+7QU481osnodiyIfGePO9bxLMM9/qTs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=l2sDkHvc; arc=none smtp.client-ip=209.85.221.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="l2sDkHvc" Received: by mail-wr1-f46.google.com with SMTP id ffacd0b85a97d-3913958ebf2so866742f8f.3 for ; Thu, 24 Apr 2025 07:14:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504059; x=1746108859; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JWj7uDN15UXCYW87dO7IIzOLdNiL+f7UvEUy/qTIHd4=; b=l2sDkHvcTVBFdmyOzriXRy7V+POCWjRrzn/Rwj7DOrq3S1JzIBaq3FrRtBcpA8OwMp Hir/SBUQIEXfJUrgAdjUlyNF563afxrDlJRhSr4WOfzLXQ2rK5TArXvAwDzZult/FV65 r6DDBHUY/qeAzFATXGba6ZBggzU9VoaB3DAyorTkS/mCC3BTqRiL8DiG6m53eEtZrwFJ l89nyucBkjfbwad8dYxizW4TuSDIOCmCHCx4QVWDoSWdokMSBungKMUDhFXwiwfG4Xhn CZOnmrDGcRcaau6dmfP0X/dN4oXzEgXEfA9EON6B3BMxVTgXNZST8w+6qss5uRZmoLmm 5KKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504059; x=1746108859; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JWj7uDN15UXCYW87dO7IIzOLdNiL+f7UvEUy/qTIHd4=; b=xFxN5aKiWJPKkhHdwftYtrsES4R1FPLJNNiNyHLvLsDAz8b37W69LOn6npQZKIdrw1 5p1SJ+k6h+SuHyFKPRiLdD+BMQ4mbbwKCZhFiUHzABGoP6KI10mKqZGI/H9xwxDDwReZ ldyl9PBYjMUIpyj6vTvtaUbJbajuKtxxiMnhSX/G64jgR2R4deaNajI3MfiYTV/r52Dh Smd3255EEk6s0Sy7r/KDBwLeh1vCQvGgMfJZLNFvVoV+16rc7xEM5f2L05zAZ02OtPBl E9FG3aJI2rzPd8XV7irpEfD5OjoYJf+lCS2VcMvmwVCgNNfzVv2Z621Sw8mtFo24O0Ze LHNQ== X-Forwarded-Encrypted: i=1; AJvYcCUu9cQFqOWn2tpeK5WZ4/Hcim4tTnqplWKlvYvCzYsbs/5m6V+Y0ohMnA23blnemO6LtWB3cl3wXcapcqkf@vger.kernel.org X-Gm-Message-State: AOJu0YxLRlW2rOqQSSTNbVau4E/8oBwfn2u37Y7CYN3WwnpBvgVbu0Tf asWKHZH9u+7NlraD2R0vjZs6Ut1DfFyHs9fjJjixvmEUNGee07EPNaNrPpk+EqU= X-Gm-Gg: ASbGncsqKdzHfPeYZttWaVAbav2Dh5wt2C1cK1DBUkK5X2wf5+U+0J20U6au0GvetsY ejbbu1ibAGhNlwbdhQURmgPv/oilNBBPPcHc9xlDuWKyRKd9u9dDWU3GWUlDk9FIzvWMDzaZ1Vd /k1Fy4wRR6g9JL9iSihEYs4BjQkpypkee0YbTxeY6KBef2hXSIUza1RVHN+6J7nfWll8FsAbF9Y 2lE+SE5v89jFYp1S7wNZEjGIpSkvROy88Pttg0xmm2p8Bda4JtssWi2C6kX0Rzqx0dUimQj2FfR bOuYOBpNWxMp1TPFrNk1UccOvDf6I77ZxBl9HHbkiHqlkKt/sB0BiA/cCnNfxHrALmzRX8GoaoX sNx+s/28maIqZGDQ6rEHxgM/A8KE= X-Google-Smtp-Source: AGHT+IEZ62CGRLn02mw6Sp1FYoWsMMZ/LoDmb0ZQtBfZ0JQt0jIozyM3jnvRsN2MwUnOvlF4h+jCiw== X-Received: by 2002:a05:6000:184c:b0:39f:64ff:668d with SMTP id ffacd0b85a97d-3a06cfa5a65mr2546775f8f.40.1745504058677; Thu, 24 Apr 2025 07:14:18 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:18 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman , Alex Elder Subject: [RFC PATCH 25/34] gunyah: Add Qualcomm Gunyah platform ops Date: Thu, 24 Apr 2025 15:13:32 +0100 Message-Id: <20250424141341.841734-26-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman Qualcomm platforms have a firmware entity which performs access control to physical pages. Dynamically started Gunyah virtual machines use the QCOM_SCM_RM_MANAGED_VMID for access. Linux thus needs to assign access to the memory used by guest VMs. Gunyah doesn't do this operation for us since it is the current VM (typically VMID_HLOS) delegating the access and not Gunyah itself. Use the Gunyah platform ops to achieve this so that only Qualcomm platforms attempt to make the needed SCM calls. Reviewed-by: Alex Elder Co-developed-by: Prakruthi Deepak Heragu Signed-off-by: Prakruthi Deepak Heragu Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- drivers/virt/gunyah/Kconfig | 13 ++ drivers/virt/gunyah/Makefile | 1 + drivers/virt/gunyah/gunyah_qcom.c | 220 ++++++++++++++++++++++++++++++ 3 files changed, 234 insertions(+) create mode 100644 drivers/virt/gunyah/gunyah_qcom.c diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig index 23ba523d25dc..fe2823dc48ba 100644 --- a/drivers/virt/gunyah/Kconfig +++ b/drivers/virt/gunyah/Kconfig @@ -4,6 +4,7 @@ config GUNYAH tristate "Gunyah Virtualization drivers" depends on ARM64 select GUNYAH_PLATFORM_HOOKS + imply GUNYAH_QCOM_PLATFORM if ARCH_QCOM help The Gunyah drivers are the helper interfaces that run in a guest VM such as basic inter-VM IPC and signaling mechanisms, and higher level @@ -14,3 +15,15 @@ config GUNYAH config GUNYAH_PLATFORM_HOOKS tristate + +config GUNYAH_QCOM_PLATFORM + tristate "Support for Gunyah on Qualcomm platforms" + depends on GUNYAH + select GUNYAH_PLATFORM_HOOKS + select QCOM_SCM + help + Enable support for interacting with Gunyah on Qualcomm + platforms. Interaction with Qualcomm firmware requires + extra platform-specific support. + + Say Y/M here to use Gunyah on Qualcomm platforms. diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile index 45cabba3110c..349c37e9f0ad 100644 --- a/drivers/virt/gunyah/Makefile +++ b/drivers/virt/gunyah/Makefile @@ -4,3 +4,4 @@ gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o obj-$(CONFIG_GUNYAH) += gunyah.o gunyah_rsc_mgr.o obj-$(CONFIG_GUNYAH_PLATFORM_HOOKS) += gunyah_platform_hooks.o +obj-$(CONFIG_GUNYAH_QCOM_PLATFORM) += gunyah_qcom.o diff --git a/drivers/virt/gunyah/gunyah_qcom.c b/drivers/virt/gunyah/gunyah_qcom.c new file mode 100644 index 000000000000..f2342d51a018 --- /dev/null +++ b/drivers/virt/gunyah/gunyah_qcom.c @@ -0,0 +1,220 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2023-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include + +#define QCOM_SCM_RM_MANAGED_VMID 0x3A +#define QCOM_SCM_MAX_MANAGED_VMID 0x3F + +static int +qcom_scm_gunyah_rm_pre_mem_share(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel) +{ + struct qcom_scm_vmperm *new_perms __free(kfree) = NULL; + u64 src, src_cpy; + int ret = 0, i, n; + u16 vmid; + + new_perms = kcalloc(mem_parcel->n_acl_entries, sizeof(*new_perms), + GFP_KERNEL); + if (!new_perms) + return -ENOMEM; + + for (n = 0; n < mem_parcel->n_acl_entries; n++) { + vmid = le16_to_cpu(mem_parcel->acl_entries[n].vmid); + if (vmid <= QCOM_SCM_MAX_MANAGED_VMID) + new_perms[n].vmid = vmid; + else + new_perms[n].vmid = QCOM_SCM_RM_MANAGED_VMID; + if (mem_parcel->acl_entries[n].perms & GUNYAH_RM_ACL_X) + new_perms[n].perm |= QCOM_SCM_PERM_EXEC; + if (mem_parcel->acl_entries[n].perms & GUNYAH_RM_ACL_W) + new_perms[n].perm |= QCOM_SCM_PERM_WRITE; + if (mem_parcel->acl_entries[n].perms & GUNYAH_RM_ACL_R) + new_perms[n].perm |= QCOM_SCM_PERM_READ; + } + + src = BIT_ULL(QCOM_SCM_VMID_HLOS); + + for (i = 0; i < mem_parcel->n_mem_entries; i++) { + src_cpy = src; + ret = qcom_scm_assign_mem( + le64_to_cpu(mem_parcel->mem_entries[i].phys_addr), + le64_to_cpu(mem_parcel->mem_entries[i].size), &src_cpy, + new_perms, mem_parcel->n_acl_entries); + if (ret) + break; + } + + /* Did it work ok? */ + if (!ret) + return 0; + + src = 0; + for (n = 0; n < mem_parcel->n_acl_entries; n++) { + vmid = le16_to_cpu(mem_parcel->acl_entries[n].vmid); + if (vmid <= QCOM_SCM_MAX_MANAGED_VMID) + src |= BIT_ULL(vmid); + else + src |= BIT_ULL(QCOM_SCM_RM_MANAGED_VMID); + } + + new_perms[0].vmid = QCOM_SCM_VMID_HLOS; + new_perms[0].perm = QCOM_SCM_PERM_EXEC | QCOM_SCM_PERM_WRITE | + QCOM_SCM_PERM_READ; + + for (i--; i >= 0; i--) { + src_cpy = src; + WARN_ON_ONCE(qcom_scm_assign_mem( + le64_to_cpu(mem_parcel->mem_entries[i].phys_addr), + le64_to_cpu(mem_parcel->mem_entries[i].size), &src_cpy, + new_perms, 1)); + } + + return ret; +} + +static int +qcom_scm_gunyah_rm_post_mem_reclaim(struct gunyah_rm *rm, + struct gunyah_rm_mem_parcel *mem_parcel) +{ + struct qcom_scm_vmperm new_perms; + u64 src = 0, src_cpy; + int ret = 0, i, n; + u16 vmid; + + new_perms.vmid = QCOM_SCM_VMID_HLOS; + new_perms.perm = QCOM_SCM_PERM_EXEC | QCOM_SCM_PERM_WRITE | + QCOM_SCM_PERM_READ; + + for (n = 0; n < mem_parcel->n_acl_entries; n++) { + vmid = le16_to_cpu(mem_parcel->acl_entries[n].vmid); + if (vmid <= QCOM_SCM_MAX_MANAGED_VMID) + src |= (1ull << vmid); + else + src |= (1ull << QCOM_SCM_RM_MANAGED_VMID); + } + + for (i = 0; i < mem_parcel->n_mem_entries; i++) { + src_cpy = src; + ret = qcom_scm_assign_mem( + le64_to_cpu(mem_parcel->mem_entries[i].phys_addr), + le64_to_cpu(mem_parcel->mem_entries[i].size), &src_cpy, + &new_perms, 1); + WARN_ON_ONCE(ret); + } + + return ret; +} + +static int +qcom_scm_gunyah_rm_pre_demand_page(struct gunyah_rm *rm, u16 vmid, + enum gunyah_pagetable_access access, + struct folio *folio) +{ + struct qcom_scm_vmperm new_perms[2]; + unsigned int n = 1; + u64 src; + + new_perms[0].vmid = QCOM_SCM_RM_MANAGED_VMID; + new_perms[0].perm = QCOM_SCM_PERM_EXEC | QCOM_SCM_PERM_WRITE | + QCOM_SCM_PERM_READ; + if (access != GUNYAH_PAGETABLE_ACCESS_X && + access != GUNYAH_PAGETABLE_ACCESS_RX && + access != GUNYAH_PAGETABLE_ACCESS_RWX) { + new_perms[1].vmid = QCOM_SCM_VMID_HLOS; + new_perms[1].perm = QCOM_SCM_PERM_EXEC | QCOM_SCM_PERM_WRITE | + QCOM_SCM_PERM_READ; + n++; + } + + src = BIT_ULL(QCOM_SCM_VMID_HLOS); + + return qcom_scm_assign_mem(__pfn_to_phys(folio_pfn(folio)), + folio_size(folio), &src, new_perms, n); +} + +static int +qcom_scm_gunyah_rm_release_demand_page(struct gunyah_rm *rm, u16 vmid, + enum gunyah_pagetable_access access, + struct folio *folio) +{ + struct qcom_scm_vmperm new_perms; + u64 src; + + new_perms.vmid = QCOM_SCM_VMID_HLOS; + new_perms.perm = QCOM_SCM_PERM_EXEC | QCOM_SCM_PERM_WRITE | + QCOM_SCM_PERM_READ; + + src = BIT_ULL(QCOM_SCM_RM_MANAGED_VMID); + + if (access != GUNYAH_PAGETABLE_ACCESS_X && + access != GUNYAH_PAGETABLE_ACCESS_RX && + access != GUNYAH_PAGETABLE_ACCESS_RWX) + src |= BIT_ULL(QCOM_SCM_VMID_HLOS); + + return qcom_scm_assign_mem(__pfn_to_phys(folio_pfn(folio)), + folio_size(folio), &src, &new_perms, 1); +} + +static struct gunyah_rm_platform_ops qcom_scm_gunyah_rm_platform_ops = { + .pre_mem_share = qcom_scm_gunyah_rm_pre_mem_share, + .post_mem_reclaim = qcom_scm_gunyah_rm_post_mem_reclaim, + .pre_demand_page = qcom_scm_gunyah_rm_pre_demand_page, + .release_demand_page = qcom_scm_gunyah_rm_release_demand_page, +}; + +/* {19bd54bd-0b37-571b-946f-609b54539de6} */ +static const uuid_t QCOM_EXT_UUID = UUID_INIT(0x19bd54bd, 0x0b37, 0x571b, 0x94, + 0x6f, 0x60, 0x9b, 0x54, 0x53, + 0x9d, 0xe6); + +#define GUNYAH_QCOM_EXT_CALL_UUID_ID \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32, \ + ARM_SMCCC_OWNER_VENDOR_HYP, 0x3f01) + +static bool gunyah_has_qcom_extensions(void) +{ + struct arm_smccc_res res; + uuid_t uuid; + u32 *up; + + arm_smccc_1_1_smc(GUNYAH_QCOM_EXT_CALL_UUID_ID, &res); + + up = (u32 *)&uuid.b[0]; + up[0] = lower_32_bits(res.a0); + up[1] = lower_32_bits(res.a1); + up[2] = lower_32_bits(res.a2); + up[3] = lower_32_bits(res.a3); + + return uuid_equal(&uuid, &QCOM_EXT_UUID); +} + +static int __init qcom_gunyah_platform_hooks_register(void) +{ + if (!gunyah_has_qcom_extensions()) + return -ENODEV; + + pr_info("Enabling Gunyah hooks for Qualcomm platforms.\n"); + + return gunyah_rm_register_platform_ops( + &qcom_scm_gunyah_rm_platform_ops); +} + +static void __exit qcom_gunyah_platform_hooks_unregister(void) +{ + gunyah_rm_unregister_platform_ops(&qcom_scm_gunyah_rm_platform_ops); +} + +module_init(qcom_gunyah_platform_hooks_register); +module_exit(qcom_gunyah_platform_hooks_unregister); +MODULE_DESCRIPTION("Qualcomm Technologies, Inc. Platform Hooks for Gunyah"); +MODULE_LICENSE("GPL"); From patchwork Thu Apr 24 14:13:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884034 Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE840291171 for ; Thu, 24 Apr 2025 14:14:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504064; cv=none; b=XhxHVzaMdGapTGUuKy0j3vZmklYbCJQLACaerhMlubJrOGaT2mPj33612dODlKfa46k3pnKziN87KhjoB2Sk+rmU/Yd2cNBtPMPtmY2BP1qtgPRyePTz89LxC0VpGIztMVdWL2iJEZlqKxqzsuhpc8mh+C66Rg6xFHESUQQh2uo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504064; c=relaxed/simple; bh=l8MXGFGw31uuwPrLVCURxj+dBhiUP5Eq/JDGE+Ioyyo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=aTchR76HhnFxPEQVix0P4olVbiBY6+BI7eyruJd5ShmPt5Gq3a+SfVFL1gDBt3kYQU1ZQ8gYCTb5JSHhMdO7L5CouEgSKps8gpqacU7giDkXZEyFtDCrLg15WX9rThNJRk15h8e+X/DMwHV3Cz/afmp+TqdeFG5eudjdK6/q0a0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=rQRWxIZ8; arc=none smtp.client-ip=209.85.128.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="rQRWxIZ8" Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-43d0c18e84eso5333545e9.3 for ; Thu, 24 Apr 2025 07:14:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504060; x=1746108860; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ULK/WgQrAoOKgv7yTbnqoplbgLc/ybTV2ol4z7wD4V8=; b=rQRWxIZ8fpXO6y7A7UEoBuCu58ZSGVRY0fjpUvj5yf8BrVkOihzZTjogYnJHmrD9OG 5bBkzKMlKMuW7rijJvKvn5tzGE4ax3rinG5VQMmuwWDDuQieJiCbQmPJDAMonyD/KNBe cfZbIYruffocTqvtlrq7tyjPuoXof+RxuzdU2huHJcpAqTA1ujaVCIQRTZKoTztXvzwh qZMS1XOZsce0KykvQTP+oDgnnDxzTkiBHhsH6l3NUVVKKeZ39mJEAUhN2/65eho8OO4X IHXSFmu5YI3FeaFhWrHR1akw5UVoAhlIobKVnFlNNIkIfax5YE5OOBHoU4//QWZhg5DI vofQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504060; x=1746108860; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ULK/WgQrAoOKgv7yTbnqoplbgLc/ybTV2ol4z7wD4V8=; b=WNeSDdm+6G3pFFBmoHDLI23g+ePXZSd6Ms1Za0TjhiSIVZHrR7bylyBKeQNNV8ZRI/ FaYoWWfmUx/AMDYvrj2nWtkZ4AunVGu7BUWd+vxTqtH6yGsDKPK+qpIhnY1uaBEwv9+0 4obmWhaoMY7uzUgxWj66XD/A7ScTsNVLOCI4wIgPbn9WYxl2Xz80KkOwo7dcFTPIaKUD bYMl/by1jiEzsuT+LS7swch2WHtPYzQP6fQ/rR38/l/vBIGAKA4F2XGzcGztABmaNkrc eDIPPEPHaLl/sSjM+w3+qZxefAulrOpK1xClqT2rpbrprF29TmaXEkRogB7sDGNaSYeC jp+A== X-Forwarded-Encrypted: i=1; AJvYcCVY5u/f9iK3MmmMvPgQdeT9t2fFrYnAEo8EYIgOQweu27XYaMV6JZ+jmkiuclRWbXwnoc3GuvgKBq8Al2vo@vger.kernel.org X-Gm-Message-State: AOJu0Yw6vTtwrhwM0B1T5D+L615WogVVHqAPyzOODPQLlJuE6xidT/GH IyCG0m9Zw+BIjsPmWKNanZm8UZbadncWKBczDrKmYkqGt8RchsMKiHsrP4Xwhfo2W4wuv3gqDR6 p X-Gm-Gg: ASbGncuZYy9q3QaTNebcH7jzxhSuVImvdIUffB3I03gYPpi1I3PJxxH0mjdCKxKVufl VkebxIperHOoVKFKZalyk9CvUT3W3f+dB3pza/vu0hN2Sy3Q1D6ztbwPitvTbBwErABB8GgvJFK wAPT4Pn1Sr7oDwkF5FZMXUs++vG28kD4zu1S9QfbfcFmQwTue1rZpahc6PHDZKgV5hizTk0ngOZ N2vw/2LCF3LioCO7jepwo1RtOEQum+MUOyYE3dbVW6QoSQK8EsuKCxGdmEUIOK5YWOJxyxeMDzo Y/w2YAhn3MsDJ4ASCXeQjEGg91s5v/8GmjIieGd5XEK4knYmY6bimQYGF0O5VWJLxlco22s6SW9 7V93gfDSaSHyCLw53 X-Google-Smtp-Source: AGHT+IFE5/vk9/X0Lu9aLCAdTyRnF+oi4a9Eo+hij9nbvfLjJtzlSBwWD4MqvNxQh/Fi8Qjri674AQ== X-Received: by 2002:a05:6000:220c:b0:3a0:65bc:3543 with SMTP id ffacd0b85a97d-3a06cf65a2bmr2283431f8f.35.1745504059899; Thu, 24 Apr 2025 07:14:19 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:19 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 26/34] gunyah: Share memory parcels Date: Thu, 24 Apr 2025 15:13:33 +0100 Message-Id: <20250424141341.841734-27-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Gunyah Resource Manager sets up a virtual machine based on a device tree which lives in guest memory. Resource manager requires this memory to be provided as a memory parcel for it to read and manipulate. Implement a function to construct a memory parcel from the guest's pinned memory pages. Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 80 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index 7216db642174..ef0971146b56 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -634,6 +634,86 @@ static int gunyah_memory_reclaim_folio(struct gunyah_vm *ghvm, return ret; } +static int gunyah_share_memory_parcel(struct gunyah_vm *ghvm, + struct gunyah_rm_mem_parcel *parcel, gfn_t gfn, u64 nr) +{ + struct kvm *kvm = &ghvm->kvm; + struct kvm_memory_slot *memslot; + struct page **pages; + int ret; + u64 i; + + if (!nr) + return -EINVAL; + + memslot = gfn_to_memslot(kvm, gfn); + if (!memslot) + return -ENOENT; + + parcel->mem_entries = kcalloc(nr, sizeof(*parcel->mem_entries), GFP_KERNEL); + if (!parcel->mem_entries) + return -ENOMEM; + parcel->n_mem_entries = nr; + + pages = memslot->arch.pages + (gfn - memslot->base_gfn); + + for (i = 0; i < nr; i++) { + parcel->mem_entries[i].size = cpu_to_le64(PAGE_SIZE); + parcel->mem_entries[i].phys_addr = cpu_to_le64(page_to_phys(pages[i])); + } + + parcel->n_acl_entries = 1; + parcel->acl_entries = kcalloc(parcel->n_acl_entries, + sizeof(*parcel->acl_entries), GFP_KERNEL); + if (!parcel->n_acl_entries) { + ret = -ENOMEM; + goto free_entries; + } + parcel->acl_entries[0].vmid = cpu_to_le16(ghvm->vmid); + parcel->acl_entries[0].perms |= GUNYAH_RM_ACL_R; + parcel->acl_entries[0].perms |= GUNYAH_RM_ACL_W; + parcel->acl_entries[0].perms |= GUNYAH_RM_ACL_X; + parcel->mem_handle = GUNYAH_MEM_HANDLE_INVAL; + + ret = gunyah_rm_mem_share(ghvm->rm, parcel); + if (ret) + goto free_acl; + + return ret; +free_acl: + kfree(parcel->acl_entries); + parcel->acl_entries = NULL; +free_entries: + kfree(parcel->mem_entries); + parcel->mem_entries = NULL; + parcel->n_mem_entries = 0; + + return ret; +} + +static int gunyah_reclaim_memory_parcel(struct gunyah_vm *ghvm, + struct gunyah_rm_mem_parcel *parcel, gfn_t gfn, u64 nr) +{ + int ret; + + if (parcel->mem_handle != GUNYAH_MEM_HANDLE_INVAL) { + ret = gunyah_rm_mem_reclaim(ghvm->rm, parcel); + if (ret) { + dev_err(ghvm->parent, "Failed to reclaim parcel: %d\n", + ret); + /* We can't reclaim the pages -- hold onto the pages + * forever because we don't know what state the memory + * is in + */ + return ret; + } + parcel->mem_handle = GUNYAH_MEM_HANDLE_INVAL; + kfree(parcel->mem_entries); + kfree(parcel->acl_entries); + } + return 0; +} + int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE; From patchwork Thu Apr 24 14:13:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884935 Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 795ED2918E4 for ; Thu, 24 Apr 2025 14:14:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504066; cv=none; b=PH9Yj9qHyc0eJtZ74hLbGOmPvtxbHDQiB7WbXYI6LPjZ0QO510onwfv/HHU2aT8hs4e8PspkAKE6QRfeCBk/26EOrz27LuXphGIP3SatklX4VZIz/FuJEl92hu43WQwKksmmZ2aOeiBZQTtPKLRxKb/R2enM00NJ5oV8Ff9Prt4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504066; c=relaxed/simple; bh=Yi4ZtnxtwEwwcGq5Pu3T2/cALo63bY4y5tVWXmp+Y3c=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=O6mrpRaYeD8jO7Frt3bTgyvaevbLJj410EEDk0Roa02xNSPV7zGtJ/q3lXNrJ0bsb1P0GqLw/h4l6XT5cI24OSo+kKS6M7IJHpao0ip0vHNhAskYXBCZPFNG00xIGzv0DaufkjSrNn5wb6m25Okvc0/0s/lqi0QW+IF7i6kjNYA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=P3uDnySM; arc=none smtp.client-ip=209.85.221.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="P3uDnySM" Received: by mail-wr1-f46.google.com with SMTP id ffacd0b85a97d-39c0dfba946so793954f8f.3 for ; Thu, 24 Apr 2025 07:14:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504061; x=1746108861; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KqI809F7wKcnGZMELURBH0WwNUVNARNAdRqZ1Gwku8c=; b=P3uDnySMxJTWQ7/cn0c6QZfS+2mrcsS4YkITm9Jqvkwwd5JKcVX1YIHlyx536M/Icz 4zoZfnwIj3oiZI26ksGkdThqhKWvQPNuKpAtg0fkVVqb0ht0/qc+RXXvjh9+I7NEmj82 l2vILtAz+xiQF9fd9P4TADL3bYT0NkFm03GbsGx9OF16fSnpc8NRO2c8JfG9wsP2i7AG 5RmNr/QZ9Rn4X7SQjxqek8nWQOPb2h12lqjSgAFaeCHojqBNfD7Rc22d4ZYqnlClkN05 bPkc1avSLqwWPj47rGA9wVVzjqkp1NktobmDDPe/qldAM3KePajlpvrZKB615gfDflo7 4MJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504061; x=1746108861; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KqI809F7wKcnGZMELURBH0WwNUVNARNAdRqZ1Gwku8c=; b=bnxXKXfbHxfzUcUZawpr1GbM7zrXE85v/4O032gsseH7nQyMJeEyHkhd+Vr0Qb6zd3 ZWtUM+WRAhY+nZw0Kkw9PDSQVbnlmTBd6MZfWeiMVbZhizVblWyw1tvkp7AdAp/mvmmt ITUc+CXMHTsThSUXm2T8xoQpMyF2Q6urMTBHLe7E8dQB99f0Xf2MAnkujnieeWKa5fd2 Iovvv+qbu0Gp9NCmgo4iqYjWAzOVgPPnN7xUeVFEPo3Nz5MsMIJyYTsLs4Sj3s4YzM+F nGbKwPdcMkoRNcPqaXGijmFDskRh+PoYCWRJeWlezNVi1hzqqBfuS+9tHXnrBtUXBRAk biSA== X-Forwarded-Encrypted: i=1; AJvYcCW+7UR9wUwk4AnvJixYCnFvbTUO2xPBSt57UJGnwWu04GI+JOu2/sS0McvX4Nu/G+8/RRjCszvz7taS5PJz@vger.kernel.org X-Gm-Message-State: AOJu0Yxn4VIbQXNmiysrt/Ghk5DwMpdlhU3daZtBKJXMtfXM7dodhkJE bfdQ1+LvXJGpGhfvbZ5SxBLIK2lKaiL9xIa+CX7f+TM2OrXQ7j/vwl5MP2qJ3xs= X-Gm-Gg: ASbGnctlhXpPqgChbjD8DVE5kyAUdAkL2lrW9cAz11qw+fWcFYvfWE0sXWvvqoIdqZ0 G9Lzl9LGz/gtHjhXYSUbbIWGXeIVI/vKv4TuE7tRg1k0a23uvUGYNc1PGdsVMza1/SOYMWeBIfG VGsLzVKwEhkXrvolONyXJF5SYBY6OLV4Auu31JSPlqEm3+4bmovcI30FRc4hZf1Si75WN0EgLZF 7aDJUWY4S+1QxB6M8WG9IrIqUBhk2Iljgx6X4iBGESQf+tLzyBKQS5OTyPYZ6zfowkM8l+BiKuO rleDsSrmmrr9VI57hYmwir4UE2/At9jnj+0jCwhqw/bHHc7a+Zn8deSgR+SlO94iYxxLqfkZRQ3 32JPOGTH39PgrzmSn X-Google-Smtp-Source: AGHT+IHu0xINB3AI0ORmGPEOw/RldxQp1Lf0CLnYLBWHSDl9/V/LyKw8BRQRYMCgyV0pnZHvllr1jA== X-Received: by 2002:a5d:584e:0:b0:39a:c9ac:cd3a with SMTP id ffacd0b85a97d-3a06cfa9c91mr2435184f8f.51.1745504061234; Thu, 24 Apr 2025 07:14:21 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:20 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 27/34] gunyah: Share guest VM dtb configuration to Gunyah Date: Thu, 24 Apr 2025 15:13:34 +0100 Message-Id: <20250424141341.841734-28-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman Gunyah Resource Manager sets up a virtual machine based on a device tree which lives in guest memory. Resource manager requires this memory to be provided as a memory parcel for it to read and manipulate. Construct a memory parcel, lend it to the virtual machine, and inform resource manager about the device tree location (the memory parcel ID and offset into the memory parcel). Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 27 ++++++++++++++++++++++++--- include/linux/gunyah.h | 10 ++++++++++ 2 files changed, 34 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index ef0971146b56..687f2beea4e7 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -699,8 +699,7 @@ static int gunyah_reclaim_memory_parcel(struct gunyah_vm *ghvm, if (parcel->mem_handle != GUNYAH_MEM_HANDLE_INVAL) { ret = gunyah_rm_mem_reclaim(ghvm->rm, parcel); if (ret) { - dev_err(ghvm->parent, "Failed to reclaim parcel: %d\n", - ret); + pr_err("Failed to reclaim parcel: %d\n", ret); /* We can't reclaim the pages -- hold onto the pages * forever because we don't know what state the memory * is in @@ -1574,6 +1573,7 @@ static void gunyah_vm_stop(struct gunyah_vm *ghvm) static int gunyah_vm_start(struct gunyah_vm *ghvm) { + struct kvm *kvm = &ghvm->kvm; struct gunyah_rm_hyp_resources *resources; struct gunyah_resource *ghrsc; int i, n, ret; @@ -1597,7 +1597,18 @@ static int gunyah_vm_start(struct gunyah_vm *ghvm) ghvm->vmid = ret; ghvm->vm_status = GUNYAH_RM_VM_STATUS_LOAD; - ret = gunyah_rm_vm_configure(ghvm->rm, ghvm->vmid, ghvm->auth, 0, 0, 0, 0, 0); + ghvm->dtb.parcel_start = gpa_to_gfn(kvm->dtb.guest_phys_addr); + ghvm->dtb.parcel_pages = gpa_to_gfn(kvm->dtb.size); + ret = gunyah_share_memory_parcel(ghvm, &ghvm->dtb.parcel, + ghvm->dtb.parcel_start, + ghvm->dtb.parcel_pages); + if (ret) { + pr_warn("Failed to allocate parcel for DTB: %d\n", ret); + goto err; + } + + ret = gunyah_rm_vm_configure(ghvm->rm, ghvm->vmid, ghvm->auth, + ghvm->dtb.parcel.mem_handle, 0, 0, 0, kvm->dtb.size); if (ret) { pr_warn("Failed to configure VM: %d\n", ret); goto err; @@ -1698,6 +1709,16 @@ static void gunyah_destroy_vm(struct gunyah_vm *ghvm) if (ghvm->vm_status == GUNYAH_RM_VM_STATUS_RUNNING) gunyah_vm_stop(ghvm); + if (ghvm->vm_status == GUNYAH_RM_VM_STATUS_LOAD || + ghvm->vm_status == GUNYAH_RM_VM_STATUS_READY || + ghvm->vm_status == GUNYAH_RM_VM_STATUS_INIT_FAILED) { + ret = gunyah_reclaim_memory_parcel(ghvm, &ghvm->dtb.parcel, + ghvm->dtb.parcel_start, + ghvm->dtb.parcel_pages); + if (ret) + pr_err("Failed to reclaim DTB parcel: %d\n", ret); + } + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->addrspace_ticket); gunyah_vm_remove_resource_ticket(ghvm, &ghvm->host_shared_extent_ticket); gunyah_vm_remove_resource_ticket(ghvm, &ghvm->host_private_extent_ticket); diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 1d363ab8967a..72aafc813664 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -94,6 +94,12 @@ struct gunyah_vm_resource_ticket { * @resource_tickets: List of &struct gunyah_vm_resource_ticket * @auth: Authentication mechanism to be used by resource manager when * launching the VM + * @dtb: For tracking dtb configuration when launching the VM + * @dtb.parcel_start: Guest frame number where the memory parcel that we lent to + * VM (DTB could start in middle of folio; we lend entire + * folio; parcel_start is start of the folio) + * @dtb.parcel_pages: Number of pages lent for the memory parcel + * @dtb.parcel: Data for resource manager to lend the parcel */ struct gunyah_vm { u16 vmid; @@ -113,6 +119,10 @@ struct gunyah_vm { struct gunyah_vm_resource_ticket host_shared_extent_ticket; struct gunyah_vm_resource_ticket guest_private_extent_ticket; struct gunyah_vm_resource_ticket guest_shared_extent_ticket; + struct { + gfn_t parcel_start, parcel_pages; + struct gunyah_rm_mem_parcel parcel; + } dtb; }; /** From patchwork Thu Apr 24 14:13:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884033 Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7DAF5292913 for ; Thu, 24 Apr 2025 14:14:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504067; cv=none; b=gin92XHZNe5IPw2NxQ5fAlJAUM3MA6bsGZZEOqhuoYYnGGW58JNIqrZ7AYLN2SVrhJ+QFRJC5PjaWnireKF9nquo4xcfaGYWIqOAtP4nRYGkVYAH2coRFjHgCWnnYz8V6Uz07JC5YfW7noO1rO5wKn84Uz01lc4MLqzMqK2Mwig= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504067; c=relaxed/simple; bh=TszZQ//DMZ7Wex/EWSs2RVro6IlBnBW/flMNEyx2TWU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=h03cPScIScQogBqBqvs5yFUFvcU9aNlNc2ua2xnW+C1+H1FCaOxLF2TRlRINsV2JpGmaisywrl6zOlCVN6geiUNlLElzNXQq5zH6z/YkmeQkt9aSPb00qPTG0ADKR/+zWziDz08TNbf5umTqsLv8u2rCDEC4LgEekg/aH3+Q7og= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=x9sK5gL9; arc=none smtp.client-ip=209.85.221.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="x9sK5gL9" Received: by mail-wr1-f42.google.com with SMTP id ffacd0b85a97d-39ee5a5bb66so710765f8f.3 for ; Thu, 24 Apr 2025 07:14:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504063; x=1746108863; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=88AiON8knVFtAm4gGlYsfQZYlJxkn1/VFztoyFrvZWY=; b=x9sK5gL9U0W0aBidfjPjwipqtUklIVk4o1kIne0qmrnkewk+mp+beWI7OgPH1JKmB/ e77X33ZePtK10LJpX1wgpF4Z3Dh/LIWgdKfwdSz7XOGTSrMgkNDfD9HP9pYpOk97XOjh 2yWryuObm+Pz29EgXo9Y04ybVkN8i755Yvsi4ZiZyR/5oIcg9WTSRFCy8ZlOCRwBcihU J7WLXsU5YtdUhQBh6TDZMZHJ5UeSCEzi+pL68GQgNSIh8mtoAvdiGc2tgh2cZwuEDjOa mMkJOTIMkbQb5qkfxHFt+I3J9vB0w2Zbw/VGaeBE4tJr/KilNB8IKtpgE8HnWXzRIazx qZ6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504063; x=1746108863; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=88AiON8knVFtAm4gGlYsfQZYlJxkn1/VFztoyFrvZWY=; b=To8Rq8ipwaQg9bHl0AcJqUhVLfT2QSnjpwzIPKfohQNPsNuHeuKY+XkszWxosSEOHN UIQEI+YPUGIvB7J4jLObAm6yuYapAu/BfuXqzuDYJMAEzFluvCoHzqthAYeZq67y3dHb fGL6ZYK01KDQiemFFHhb69BFVBBrpoXrmL98wZG0osIYnSRyOd8qavnHoTaamm3l+YD2 tNo7X2//2QMGw+BNf4U3khHjfF3WqzBNPD7varGYTDbgsDOZ73cFL1hmJ72Pt/3m5i/T pCKEiXBW14DjYgFv3RQCRaPNqdE0/XePJgQoo5qRosQ5y9XY4bkHyTQE+qzjhzThrTK+ 5OPw== X-Forwarded-Encrypted: i=1; AJvYcCVYqstdHzrk9kTO88hQwIr2zPhFMXcRe5YEASSbuiKfc2J6vKRURJF937GTAsOcub66ikpkDJwrWYXTC1x+@vger.kernel.org X-Gm-Message-State: AOJu0YykUVUBMQMDmK1zyAvz5/kaHTh0WjOXvQlfgEPrMFLbGHMtIpv2 WrtejRmLK7Sn9QVj9cTAR4FKacmrsYe7dcrTbgXaIKAspOi57N2aI+dF1qaP9QA= X-Gm-Gg: ASbGncvSbBhsEsGFCfpu88mj0fDCUaPsHpuMXR4GQpQ7AJLYCDAMT8wjEaTzXUpST5q 359eds0zMHJ1fiuHOlpcyie0hnVCKFT4+NaRR9ORNkYN1pygQe7RHzX4z1j0/dRUTS27Cmtqg+x gb1qCq6jr4x8cs0GKmNgcC1j7NGbIU8iedyQNzzqIe4RwcKJiwQ2dWDYSc2ghNSGSMLwbfF9kds zPReEgsgZWIGTwBMaJnszZcU03qDz3ZS5wae1ZE64AQmC0ZmT44Uv/rLgzc4Gona3+aNzJnbveK Q8Sic9SefheptjUNyydIg39eRp3b/m9UPOvWlmVCoSezy7WooL0A3VrPZxC4E++81GWlpSIb1es XackgIUknPtmqR1zj X-Google-Smtp-Source: AGHT+IE6SiBau6EY9Mh/ltJdHWfjy1fT3q7qjVfQcBmH9UUhGIZ598Z7cZVBqAGGCWYaKx9pVYAqXA== X-Received: by 2002:a05:6000:43d6:10b0:3a0:7017:61f6 with SMTP id ffacd0b85a97d-3a07017626dmr1028260f8f.14.1745504062675; Thu, 24 Apr 2025 07:14:22 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:22 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 28/34] gunyah: Add RPC to enable demand paging Date: Thu, 24 Apr 2025 15:13:35 +0100 Message-Id: <20250424141341.841734-29-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman Add Gunyah Resource Manager RPC to enable demand paging for a virtual machine. Resource manager needs to be informed of private memory regions which will be demand paged and the location where the DTB memory parcel should live in the guest's address space. Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- drivers/virt/gunyah/rsc_mgr_rpc.c | 71 +++++++++++++++++++++++++++++++ include/linux/gunyah_rsc_mgr.h | 12 ++++++ 2 files changed, 83 insertions(+) diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mgr_rpc.c index ec187d116dd7..7fccd871cc0b 100644 --- a/drivers/virt/gunyah/rsc_mgr_rpc.c +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -106,6 +106,23 @@ struct gunyah_rm_vm_config_image_req { __le64 dtb_size; } __packed; +/* Call: VM_SET_DEMAND_PAGING */ +struct gunyah_rm_vm_set_demand_paging_req { + __le16 vmid; + __le16 _padding; + __le32 range_count; + DECLARE_FLEX_ARRAY(struct gunyah_rm_mem_entry, ranges); +} __packed; + +/* Call: VM_SET_ADDRESS_LAYOUT */ +struct gunyah_rm_vm_set_address_layout_req { + __le16 vmid; + __le16 _padding; + __le32 range_id; + __le64 range_base; + __le64 range_size; +} __packed; + /* * Several RM calls take only a VMID as a parameter and give only standard * response back. Deduplicate boilerplate code by using this common call. @@ -467,3 +484,57 @@ int gunyah_rm_get_hyp_resources(struct gunyah_rm *rm, u16 vmid, return 0; } ALLOW_ERROR_INJECTION(gunyah_rm_get_hyp_resources, ERRNO); + +/** + * gunyah_rm_vm_set_demand_paging() - Enable demand paging of memory regions + * @rm: Handle to a Gunyah resource manager + * @vmid: VMID of the other VM + * @count: Number of demand paged memory regions + * @entries: Array of the regions + */ +int gunyah_rm_vm_set_demand_paging(struct gunyah_rm *rm, u16 vmid, u32 count, + struct gunyah_rm_mem_entry *entries) +{ + struct gunyah_rm_vm_set_demand_paging_req *req __free(kfree) = NULL; + size_t req_size; + + req_size = struct_size(req, ranges, count); + if (req_size == SIZE_MAX) + return -EINVAL; + + req = kzalloc(req_size, GFP_KERNEL); + if (!req) + return -ENOMEM; + + req->vmid = cpu_to_le16(vmid); + req->range_count = cpu_to_le32(count); + memcpy(req->ranges, entries, sizeof(*entries) * count); + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_SET_DEMAND_PAGING, req, + req_size, NULL, NULL); +} +ALLOW_ERROR_INJECTION(gunyah_rm_vm_set_demand_paging, ERRNO); + +/** + * gunyah_rm_vm_set_address_layout() - Set the start address of images + * @rm: Handle to a Gunyah resource manager + * @vmid: VMID of the other VM + * @range_id: Which image to set + * @base_address: Base address + * @size: Size + */ +int gunyah_rm_vm_set_address_layout(struct gunyah_rm *rm, u16 vmid, + enum gunyah_rm_range_id range_id, + u64 base_address, u64 size) +{ + struct gunyah_rm_vm_set_address_layout_req req = { + .vmid = cpu_to_le16(vmid), + .range_id = cpu_to_le32(range_id), + .range_base = cpu_to_le64(base_address), + .range_size = cpu_to_le64(size), + }; + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_SET_ADDRESS_LAYOUT, &req, + sizeof(req), NULL, NULL); +} +ALLOW_ERROR_INJECTION(gunyah_rm_vm_set_address_layout, ERRNO); diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index fb3feee73490..f16e64af9273 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -152,6 +152,18 @@ gunyah_rm_alloc_resource(struct gunyah_rm *rm, struct gunyah_rm_hyp_resource *hyp_resource); void gunyah_rm_free_resource(struct gunyah_resource *ghrsc); +int gunyah_rm_vm_set_demand_paging(struct gunyah_rm *rm, u16 vmid, u32 count, + struct gunyah_rm_mem_entry *mem_entries); +enum gunyah_rm_range_id { + GUNYAH_RM_RANGE_ID_IMAGE = 0, + GUNYAH_RM_RANGE_ID_FIRMWARE = 1, +}; + +int gunyah_rm_vm_set_address_layout(struct gunyah_rm *rm, u16 vmid, + enum gunyah_rm_range_id range_id, + u64 base_address, u64 size); + + int gunyah_rm_call(struct gunyah_rm *rsc_mgr, u32 message_id, const void *req_buf, size_t req_buf_size, void **resp_buf, size_t *resp_buf_size); From patchwork Thu Apr 24 14:13:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884934 Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D92C5293B5A for ; Thu, 24 Apr 2025 14:14:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504068; cv=none; b=VpQkSf5NrZYnEkmqUpH1Ob3N5gq6cs3AaKCmaehwJLFhQK3Wkrr9uGjqlrNWtS6n4uwrBOPSsBKZrfL6ZmjtiscjZoUBHk66V8nB44L1pfprn+/vcY0DDEqdy/HaK8ADBi3BDTbIq0yW4iTpgznRUYE9hHMeDI5kxR2JoA19Zkc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504068; c=relaxed/simple; bh=4ABx24IRjfvVXtvRhqG0T6gWoZOLeg1qYCJYADA24es=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=shJBScNBVVVy+tE3GjC7P54mciai8j+Hz6ZPB/BIVK3eNFXdJEt+bC7b4cQmVI+EO8OK28pFQgLpd9JCH2zKzYZ3z8Ze9SdJ35Y2mPNyvfXTxvlMGisJmazobZCVOGvDozr6G+ucWT6eFRn7cQIuzQVZLNULihEc3ccFfm9ty3s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=c+Nx3rUg; arc=none smtp.client-ip=209.85.128.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="c+Nx3rUg" Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-43cec5cd73bso6362145e9.3 for ; Thu, 24 Apr 2025 07:14:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504064; x=1746108864; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WC4sArz2tKH+wjPmCs+Bj3DeYtgEPbRq63zRADPtWtk=; b=c+Nx3rUg6FC30JCuD2Slbwo5+E/SGjmyOKEVkH02gOU9x0Hi+Z9SokAYrM0TCEBD94 /E8LdTFmJt47WiNqPE9Oo5ak510zj8YatEwCUGn1J39+r9X5CdwHRXD65/AmPorFjMsk KllBv+H0VgPpePwNqe8IC76HHXo9/M8QZ1Uatk2bGgwQX7X97snbbuyrkC0YxcAHkPMD FE6DAGYZSWu1B1Zn88kvQXvWxfU39uZDxxvq2CL8WDYMU07HBaznbPo1TP2KhyGY4uFg 1HzllzIWEJAJj3aP8e4tUFZ10cnNsI5v+yd0+BErIupk0hGOT/s2P6OsfIzIg5EFOID4 /HiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504064; x=1746108864; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WC4sArz2tKH+wjPmCs+Bj3DeYtgEPbRq63zRADPtWtk=; b=bMyklF6opX5+5ksVLp+rknZ5QQgzyWN5SJDQRLh88EE5Rvc5rW70IH67cj4w2hc98D YG7yO4HMDFW+BD4x0ryIEIVVLf7xMxD82WDTpp9gXwXXH9iq+DvlhAFjKhjDgb19YRvF NX86n1Shmf8CvRYwlqNcJGJ7CvXAbqmZdGqbfw9mkhqbF7/RWaF3/J8HBrcq2DcflsUw b2vMJyIb2zcolh5pKjJQskeQQNwQr1FbtXt4mioaijPtVKiYFd0RFu65YPR3HAZt1kRk NZuO0vcgo4M1DGY2Njab+D4G3Gces15ZnJS1IiumFTE3HnigGc3n+8WY/9ICddM8TaZZ 7aQg== X-Forwarded-Encrypted: i=1; AJvYcCUscfzTaCRcRomYLefavZcFK5/dyixwqoO6SWQ26lDNnQ8rSe9N0ljIMDepdz06inv8FMKQmrZZ0TKAsDps@vger.kernel.org X-Gm-Message-State: AOJu0YxeA/cyBTVTLnTfwyxcpjNnwVsSj/urX2Ele4hIMYNFg8q5m5jf pEjEx7nUFdDXRzQNfCGSZYNnjbGzHI6QhEwDuU2/GUzmNTN9qYqq3yMyQEAh3gE= X-Gm-Gg: ASbGncs1r9/ZGXwzdFzpnpXAVt0N7Ghx9Vb4dA2NDXKxxor+ZmCl2EV77PN25T88lFC JZnZAAtY6wUxPNh/1aOBVF7ZD6z0Rgv9EJcCWP6u8XW2ZSRCQ4/fyrUAflq1tksrrv8spiZBuNt NvlSgy6qk+djX2o3F7L1xqCfb+7bNnowiFVP5xXZ9QYec3+YvDO0MgzqA90cQs1/yjc58MAWshJ U4CKABmvF9H/dFT25j2HGe1W9FkEYQF1yDB8UGJoix0EEm0lqwviEyHtiEgzv2u6Z8LGLnsAB5D WE5XSFVGq1Ih5BtkpLAoWzwjVrimCGUpkShP62ySMclWjPb0WOylvG4d4tDgPTiUCAQFLV5PH7p 3/PDURQ/7q6VxCaD8 X-Google-Smtp-Source: AGHT+IGeGIUXl8vOV0aXlznupMFAAmbzGK8/bha/61z42fQMkuhHsWrpW3uTjHsZlOeGVmwfVkNdPQ== X-Received: by 2002:a05:6000:18ad:b0:39a:c9ac:cd58 with SMTP id ffacd0b85a97d-3a06cf63742mr2096999f8f.29.1745504063860; Thu, 24 Apr 2025 07:14:23 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:23 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 29/34] gunyah: Enable demand paging Date: Thu, 24 Apr 2025 15:13:36 +0100 Message-Id: <20250424141341.841734-30-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Tell resource manager to enable demand paging and wire vCPU faults to provide the backing folio. When a folio is donated to the guest, PG_private is set on the folio so that we can recognize it later when taking it back (e.g. to reclaim the folio when the VM exists). Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 182 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 179 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index 687f2beea4e7..e8037d636e8f 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -8,6 +8,7 @@ * */ #include +#include #include #include #include @@ -476,7 +477,7 @@ static inline u32 reclaim_flags(bool share) GUNYAH_MEMEXTENT_DONATE_FROM_PROTECTED); } -static int gunyah_memory_provide_folio(struct gunyah_vm *ghvm, +static int __gunyah_memory_provide_folio(struct gunyah_vm *ghvm, struct folio *folio, gfn_t gfn, bool share, bool write) { struct gunyah_resource *guest_extent, *host_extent, *addrspace; @@ -573,7 +574,7 @@ static int gunyah_memory_provide_folio(struct gunyah_vm *ghvm, return ret; } -static int gunyah_memory_reclaim_folio(struct gunyah_vm *ghvm, +static int gunyah_memory_reclaim_folio_locked(struct gunyah_vm *ghvm, struct folio *folio, gfn_t gfn, bool share) { u32 map_flags = BIT(GUNYAH_ADDRSPACE_MAP_FLAG_PARTIAL); @@ -713,6 +714,144 @@ static int gunyah_reclaim_memory_parcel(struct gunyah_vm *ghvm, return 0; } +static int gunyah_memory_provide_folio(struct gunyah_vm *ghvm, gfn_t gfn, bool write) +{ + struct kvm *kvm = &ghvm->kvm; + struct kvm_memory_slot *memslot; + struct page *page; + struct folio *folio; + int ret; + + /* Gunyah always starts guest address space at 1G */ + if (gfn < gpa_to_gfn(SZ_1G)) + return -EINVAL; + + memslot = gfn_to_memslot(kvm, gfn); + if (!memslot) + return -ENOENT; + + page = memslot->arch.pages[gfn - memslot->base_gfn]; + folio = page_folio(page); + + folio_lock(folio); + /* Did we race with another vCPU? */ + if (folio_test_private(folio)) { + folio_unlock(folio); + return 0; + } + + ret = __gunyah_memory_provide_folio(ghvm, folio, gfn, false, true); + if (ret) { + folio_unlock(folio); + return ret; + } + folio_set_private(folio); + folio_unlock(folio); + + return 0; +} + +static int gunyah_reclaim_memory_range(struct gunyah_vm *ghvm, gfn_t start, gfn_t nr) +{ + struct kvm *kvm = &ghvm->kvm; + struct kvm_memory_slot *slot; + struct kvm_memslots *slots; + struct kvm_memslot_iter iter; + gfn_t end = start + nr; + int i, ret; + + for (i = 0; i < kvm_arch_nr_memslot_as_ids(kvm); i++) { + slots = __kvm_memslots(kvm, i); + + kvm_for_each_memslot_in_gfn_range(&iter, slots, start, end) { + struct page **pages, *page; + struct folio *folio; + unsigned long offset; + unsigned long reclaimed = 0; + + slot = iter.slot; + pages = slot->arch.pages; + for (offset = 0; offset < slot->npages;) { + page = pages[offset]; + folio = page_folio(page); + folio_lock(folio); + if (folio_test_private(folio)) { + ret = gunyah_memory_reclaim_folio_locked(ghvm, folio, + slot->base_gfn + offset, false); + if (ret) { + WARN_ON_ONCE(1); + return ret; + } + folio_clear_private(folio); + reclaimed++; + } + folio_unlock(folio); + offset += folio_nr_pages(folio); + } + } + } + return 0; +} + +static int gunyah_memory_parcel_to_paged(struct gunyah_vm *ghvm, gfn_t start, gfn_t nr) +{ + struct kvm *kvm = &ghvm->kvm; + struct kvm_memory_slot *memslot; + struct page **pages, *page; + struct folio *folio; + int i; + + memslot = gfn_to_memslot(kvm, start); + if (!memslot) + return -ENOENT; + + if (start - memslot->base_gfn < nr) + return -EINVAL; + + pages = &memslot->arch.pages[start - memslot->base_gfn]; + + for (i = 0; i < nr;) { + page = pages[i]; + folio = page_folio(page); + VM_BUG_ON(folio_test_private(folio)); + folio_set_private(folio); + i += folio_nr_pages(folio); + } + + return 0; +} + +static int gunyah_start_paging(struct gunyah_vm *ghvm) +{ + struct kvm_memslots *slots; + struct kvm_memory_slot *slot; + struct gunyah_rm_mem_entry *entries, *entry; + int count = 0; + int bkt, ret; + + slots = kvm_memslots(&ghvm->kvm); + kvm_for_each_memslot(slot, bkt, slots) { + if (slot->base_gfn >= PFN_DOWN(SZ_1G)) + count++; + } + + entries = entry = kcalloc(count, sizeof(*entries), GFP_KERNEL); + if (!entries) + return -ENOMEM; + + kvm_for_each_memslot(slot, bkt, slots) { + if (slot->base_gfn < PFN_DOWN(SZ_1G)) + continue; + entry->phys_addr = cpu_to_le64(gfn_to_gpa(slot->base_gfn)); + entry->size = cpu_to_le64(gfn_to_gpa(slot->npages)); + entry++; + } + + ret = gunyah_rm_vm_set_demand_paging(ghvm->rm, ghvm->vmid, count, entries); + kfree(entries); + return ret; +} + int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE; @@ -772,7 +911,14 @@ static int gunyah_handle_page_fault( struct gunyah_vcpu *vcpu, const struct gunyah_hypercall_vcpu_run_resp *vcpu_run_resp) { - return -EINVAL; + u64 addr = vcpu_run_resp->state_data[0]; + bool write = !!vcpu_run_resp->state_data[1]; + int ret = 0; + + ret = gunyah_memory_provide_folio(vcpu->ghvm, gpa_to_gfn(addr), write); + if (!ret || ret == -EAGAIN) + return 0; + return ret; } static bool gunyah_kvm_handle_mmio(struct gunyah_vcpu *vcpu, @@ -1614,6 +1760,28 @@ static int gunyah_vm_start(struct gunyah_vm *ghvm) goto err; } + ret = gunyah_start_paging(ghvm); + if (ret) { + pr_warn("Failed to set up demand paging: %d\n", ret); + goto err; + } + + ret = gunyah_memory_parcel_to_paged(ghvm, ghvm->dtb.parcel_start, + ghvm->dtb.parcel_pages); + if (ret) { + pr_warn("Failed to set up paging for memparcel: %d\n", ret); + goto err; + } + + ret = gunyah_rm_vm_set_address_layout( + ghvm->rm, ghvm->vmid, GUNYAH_RM_RANGE_ID_IMAGE, + ghvm->dtb.parcel_start << PAGE_SHIFT, + ghvm->dtb.parcel_pages << PAGE_SHIFT); + if (ret) { + pr_warn("Failed to set location of DTB mem parcel: %d\n", ret); + goto err; + } + ret = gunyah_rm_vm_init(ghvm->rm, ghvm->vmid); if (ret) { ghvm->vm_status = GUNYAH_RM_VM_STATUS_INIT_FAILED; @@ -1719,6 +1887,14 @@ static void gunyah_destroy_vm(struct gunyah_vm *ghvm) pr_err("Failed to reclaim DTB parcel: %d\n", ret); } + /** + * If this fails, we're going to lose the memory for good and is + * BUG_ON-worthy, but not unrecoverable (we just lose memory). + * This call should always succeed though because the VM is in not + * running and RM will let us reclaim all the memory. + */ + WARN_ON(gunyah_reclaim_memory_range(ghvm, gpa_to_gfn(SZ_1G), gpa_to_gfn(U64_MAX))); + gunyah_vm_remove_resource_ticket(ghvm, &ghvm->addrspace_ticket); gunyah_vm_remove_resource_ticket(ghvm, &ghvm->host_shared_extent_ticket); gunyah_vm_remove_resource_ticket(ghvm, &ghvm->host_private_extent_ticket); From patchwork Thu Apr 24 14:13:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884032 Received: from mail-wr1-f50.google.com (mail-wr1-f50.google.com [209.85.221.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 11CF9294A03 for ; Thu, 24 Apr 2025 14:14:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504069; cv=none; b=kulY3jNokok7DDvGbpcqqmSj4+cM9zts96IjXXEc7NpzwZ6Z5ZT15//nhuw4Dcwy2NNL66YCY90xJbCcnMsHmGc5i4i0IalSnXCF9/zqWzseqe9ZDwNqg+67xfjRWyqB+ujn7Ko6dHAtIXGQNNFeEFOzKFtCRR4bc8YkZfka5H4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504069; c=relaxed/simple; bh=KiBOPw2VoeUP9jPtQen3QJ8+JIhOTaUQt1mQ67vK+Wk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GGL9LHXg9+HR5h6753XVg+drvr3VT4BYcKujWGdg8GzV2oNY+fe3H2ak+U/ED0hnsD2cD3e/snvMOOyCmBBX3sXqPJ36Tfn8bmF5OfLPW6jLE+UI4+5BvIZtZ+OEzcZalCAklZnUVdUrnNQAKHeqU5h3UnTf1iHbCGkzdj5Tn50= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=BCIEvZRv; arc=none smtp.client-ip=209.85.221.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="BCIEvZRv" Received: by mail-wr1-f50.google.com with SMTP id ffacd0b85a97d-39c1ef4acf2so878190f8f.0 for ; Thu, 24 Apr 2025 07:14:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504065; x=1746108865; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Fkw6jo/oeFh6qnWXDp6jlBdNEq901uF/wqE5/ZMg0xQ=; b=BCIEvZRv2R68O63R8brL2WfAmdo925iFx38XKdTQ0t+jxsjdLI1JKc/L5tWtXUgyok xDFWVlpPvWYx3toLYNTof9Xh1eEc4LiLVBxk/xTOK8U9tAr4RdU8PEjKTFoM+KdjiQWz pCiFnyymKrfV9BdBIiFf7Gf3o5zHiH04R18oZMKpjtiX9CWwCovgdK9tgLDSRknqUzkM pAupuCjzcAsPWd8wmZa1UTKIuC8oavN0xCcVzBzPMBzOVC6AKRg3Sx0z2ncIHuz4FWqM vFWHksGt5xY/giD2q9cuPyRcRZmD5RZoZP3Vb3qdL1mGwMuAsRjqhT70yzTBAX9WrW8k sZyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504065; x=1746108865; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Fkw6jo/oeFh6qnWXDp6jlBdNEq901uF/wqE5/ZMg0xQ=; b=vxeBZ3ApSLVc4m8kkTrXQrkPDER4o1rm+rhAersISmrShNGBMROfDdaJcZ1BWCDPJj 3I1L6OCFvQ0r7m6M2pJLDbXutGiGjqqMzsEquktY0QBH7elR0Ckogk2xi34l/yNb6Me7 VciA+Botok3CnwIGAwAks2lQ6/FlXADwU7IM1lGO4nOcnf2htUWnusXXa7obH8hBAcQP Fbuj16/Fp95Zu6q5HbAl1rceRUhZEQT+W6N/F3I7ZbvrI3rycVgDMzNQMFDmb9Y4VGFv Gh7kJ3BJ151RMfmp3FjNRKKx943B6FZFjg5AxVVMRZ3/IGFPG7airXYrEnSwMGTNoE+C zlHA== X-Forwarded-Encrypted: i=1; AJvYcCX4z7JYM4fsIlJfpTFdpLlc46XXrgtdrq/El5pSSsu1eci6PPaEE4oGRWzKKlqQnvQhAFRM26d/vxYuZ2g9@vger.kernel.org X-Gm-Message-State: AOJu0YxqVBHx6+PpgZul//kNGrOCY0vrMqiHbyaezn8AZl+qC+8eqRLj 5TdJxA/6aIn6CgU0aynbr8J584M7Y0BTCrzyausrIPkJaXhabVld6U6dFNcY80E= X-Gm-Gg: ASbGncsZlBYb8qaBSdY69lecCLEd5A9LqjzxyGz2/6vJUDDvUaeq0AgpjnK5krkX693 JhCTZjoNIBkQmd09frxqp3M57GOA0sVKNQw52kh/D0/Y+MocOFrNjTQ7wW3TcySvJLic/ymmQ/S HobPk0oRgBhbCjAG1ZHXwduM9MNjyRlPh12WopZ3qJrDFL00PJMYB5khZePPWUHca1BrpOh/Gy/ F3ESkButenzoIKAQ9JRIZrKJ+szq7zFI9xx8FIxFNJOJE8l81LjFytAHX9AIB9cMV+Kf3u7dPut IPujdPJt/eJ1cL0/G/QCqnhzTiYLGUR+l56UKT5T0pG1nzF3y6vVr3hhpEtvHllppmCxGUp4K/+ UMRXNM2mSdA4LGLBr X-Google-Smtp-Source: AGHT+IELqmSvaP6OfocDnjzMS7mUki3w51itbTq+w393mXtO1Vl290MnI9dSMs1K0cj071gNYf3OVQ== X-Received: by 2002:a5d:6da2:0:b0:39e:cc0c:9789 with SMTP id ffacd0b85a97d-3a06cf4f2f6mr2251923f8f.11.1745504065280; Thu, 24 Apr 2025 07:14:25 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:24 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman Subject: [RFC PATCH 30/34] gunyah: Add RPC to set VM boot context Date: Thu, 24 Apr 2025 15:13:37 +0100 Message-Id: <20250424141341.841734-31-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman The initial context of a the primary vCPU can be initialized by performing RM RPC calls. Signed-off-by: Elliot Berman Reviewed-by: Srivatsa Vaddagiri Signed-off-by: Karim Manaouil --- drivers/virt/gunyah/rsc_mgr_rpc.c | 32 +++++++++++++++++++++++++++++++ include/linux/gunyah_rsc_mgr.h | 2 ++ 2 files changed, 34 insertions(+) diff --git a/drivers/virt/gunyah/rsc_mgr_rpc.c b/drivers/virt/gunyah/rsc_mgr_rpc.c index 7fccd871cc0b..78c4d1d5d42a 100644 --- a/drivers/virt/gunyah/rsc_mgr_rpc.c +++ b/drivers/virt/gunyah/rsc_mgr_rpc.c @@ -106,6 +106,15 @@ struct gunyah_rm_vm_config_image_req { __le64 dtb_size; } __packed; +/* Call: VM_SET_BOOT_CONTEXT */ +struct gunyah_rm_vm_set_boot_context_req { + __le16 vmid; + u8 reg_set; + u8 reg_index; + __le32 _padding; + __le64 value; +} __packed; + /* Call: VM_SET_DEMAND_PAGING */ struct gunyah_rm_vm_set_demand_paging_req { __le16 vmid; @@ -447,6 +456,29 @@ int gunyah_rm_vm_init(struct gunyah_rm *rm, u16 vmid) } ALLOW_ERROR_INJECTION(gunyah_rm_vm_init, ERRNO); +/** + * gunyah_rm_vm_set_boot_context() - set the initial boot context of the primary vCPU + * @rm: Handle to a Gunyah resource manager + * @vmid: VM identifier + * @reg_set: See &enum gunyah_vm_boot_context_reg + * @reg_index: Which register to set; must be 0 for REG_SET_PC + * @value: Value to set in the register + */ +int gunyah_rm_vm_set_boot_context(struct gunyah_rm *rm, u16 vmid, u8 reg_set, + u8 reg_index, u64 value) +{ + struct gunyah_rm_vm_set_boot_context_req req_payload = { + .vmid = cpu_to_le16(vmid), + .reg_set = reg_set, + .reg_index = reg_index, + .value = cpu_to_le64(value), + }; + + return gunyah_rm_call(rm, GUNYAH_RM_RPC_VM_SET_BOOT_CONTEXT, + &req_payload, sizeof(req_payload), NULL, NULL); +} +ALLOW_ERROR_INJECTION(gunyah_rm_vm_set_boot_context, ERRNO); + /** * gunyah_rm_get_hyp_resources() - Retrieve hypervisor resources (capabilities) associated with a VM * @rm: Handle to a Gunyah resource manager diff --git a/include/linux/gunyah_rsc_mgr.h b/include/linux/gunyah_rsc_mgr.h index f16e64af9273..6a07fe25b0ba 100644 --- a/include/linux/gunyah_rsc_mgr.h +++ b/include/linux/gunyah_rsc_mgr.h @@ -126,6 +126,8 @@ int gunyah_rm_vm_configure(struct gunyah_rm *rm, u16 vmid, u32 mem_handle, u64 image_offset, u64 image_size, u64 dtb_offset, u64 dtb_size); int gunyah_rm_vm_init(struct gunyah_rm *rm, u16 vmid); +int gunyah_rm_vm_set_boot_context(struct gunyah_rm *rm, u16 vmid, u8 reg_set, + u8 reg_index, u64 value); struct gunyah_rm_hyp_resource { u8 type; From patchwork Thu Apr 24 14:13:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884933 Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4D54F2951B2 for ; Thu, 24 Apr 2025 14:14:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504070; cv=none; b=ojbO9nVYaDLaAZF22h6o4E+DUV6nZLOKg9rHQVAXBuvup5B4PRq7pLCuYf8gX4x4ID6QH9t7KQXwGX12D3/7HBzw7BRTh2QLlrAV8VwBkpRe1QAXZ3wObflUMQNAUbhg/y7NMblvmuIxODzRRXZ98IZ3kT3IJNcpafoMF3EqqCg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504070; c=relaxed/simple; bh=lrb7r2WDY6/wiV70trrJFyhVfjh+ttGLRTnjEamSc+I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=h/J6fzF5X8nvGHoGZxh80tTZ7uM1Qcu0XZLkjt1FGKak9bizYFWdP2EPxetXZhmEKwYdnrCuVmvdXrWEtoxqt3VrCrF35agOdvhQG9LWoHJBFjj7LEp1BWE1DP6auc1rWvYSEKmdFPESyu9Pm0DYzzpI0Pbr+tHRcXA/OdFLccc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=XbJe9QFr; arc=none smtp.client-ip=209.85.221.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="XbJe9QFr" Received: by mail-wr1-f42.google.com with SMTP id ffacd0b85a97d-3912d2c89ecso912781f8f.2 for ; Thu, 24 Apr 2025 07:14:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504066; x=1746108866; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Z+7xBf3XhJMHfpZe6B0obtpZq77mLhIssmTOLfcblvc=; b=XbJe9QFr/198bFBMe0kDnmRcwfggZDJWfbsXRfz0KwxrS2sZZlklMbpaxSGQxDkso+ 8ZXyoRzxGpnaCcazFtAGjzLP1pIclvwLO3OZ82s8nmcercWkwXZw8VibNb3/naT9Ch7t lg4UXSgMyYgXagUHDHFgvj01YQsKI9EPC3slYHdT8pudV5tBWXWWe/Tom8r43tad1QHM LhYy7YlW9awbC0iyAv+MA/R4vKbUctEshurVcj6pQcQIczcjshb1apYJ50NtfCBgRg9j xvoFgWgGCU4Rfum3+xIrinWqTd7Gsi9yE21RS8LOJsCn1TS3Iq/dvVZHkPePYPAwuMLX EB9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504066; x=1746108866; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Z+7xBf3XhJMHfpZe6B0obtpZq77mLhIssmTOLfcblvc=; b=E34Hdle9SOSjzdSMuIJ3RdONXmL0V9Ss+XbBS9FiRVs1Kfr/PK2pymNngLqXcjbpLn QMPjC2dA539zn6gNDnoOl0fzGNB6G99k9PCZPHGzHxHn0KGDd/0xWbdNH/ohx84oSM10 SlC+Dzm+pG2ht6QMcTHJdIiPiKD+am+9pSssfuLZsVpr5qyUPsGY52B6x3CoklFOygT0 LgEDigWFm+hXTjLOstqhQ5tK0dKI1ZAynQMavpc3WkMg0sYMd+j0M2voAQ/E6cVHQIWp +Eidn4EuiH2yU9MWzGjy6EjQPUGOLotP/1Kzrvh70dzzN1Hiz3r3ol1gxE9AxrcE7ZpB HIuA== X-Forwarded-Encrypted: i=1; AJvYcCWily0ItBMPBPZvAztHXSrKtj3jJ66Hf/2ZFGo8haDR5ICkK170iworok7jD7zXOeOLwNsYFkuwNOAs002F@vger.kernel.org X-Gm-Message-State: AOJu0YzTULrab+VZM3Fl2HgDAgHhPqgb0Raj0y1jLs+NVhWgSX7nJz/x 8c2KDa0kabj22ITCoGvDQR+gioqNmfMTzcV90h3tlycgqsqS/rpIcA7i5/bWVYA= X-Gm-Gg: ASbGnctVlQ//vhU8e/f0VoySaicBzd/bnh71A8Mqx/u4vJIzddbBFVrXNN9hp+3bUSK fbZW4xkN1ZCE84zvBgSyDf3/U2icc5phTg6uk/kdbFypnNYpeytQC+azT5nLZhdv4cw2B1XXsyc DcBBr9hTtcyB6/bn4zk8eoDPr+psonE3/IE8AJdG/PesUhCRRzA5n19/c7eKAIOrBqAuqXU4S8M fu+pGqHhrFuoIf/jUKRUvkmYOPVt6TPCDA8XFZ+kZC13Hh0WG0EFIkyCNlrlr++Fw+CGPNPAqTi GK7uGS1E/x6N78Jj9zPnX677xH06DoTxmNr7l/F1tVyh64HNKOgumqtW8D2YLJjJhIYoeps0tE1 4IRVOK16P29PhGpkq X-Google-Smtp-Source: AGHT+IHRjNQ9hktsGOI0gDbs88pFfwmyof1gAXg0Su1amTrqhLAxn7c+kspUyHkoAKmkYrrR9nwrhw== X-Received: by 2002:a5d:48ca:0:b0:39c:1257:cd41 with SMTP id ffacd0b85a97d-3a06cfd462dmr1866395f8f.59.1745504066490; Thu, 24 Apr 2025 07:14:26 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:26 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 31/34] gunyah: allow userspace to set boot cpu context Date: Thu, 24 Apr 2025 15:13:38 +0100 Message-Id: <20250424141341.841734-32-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Allow userspace hypervisor (e.g. Qemu) to set the context of the boot cpu. At the moment, only the program counter (PC) is needed. Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index e8037d636e8f..df922be2429e 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -1703,6 +1703,24 @@ static int gunyah_vm_rm_notification(struct notifier_block *nb, } } +/* + * We only need to set PC to start of kernel + */ +static int gunyah_vm_set_boot_ctx(struct gunyah_vm *ghvm) +{ + struct kvm_vcpu *vcpu = kvm_get_vcpu(&ghvm->kvm, 0); + u64 core_reg = KVM_REG_ARM64 | KVM_REG_SIZE_U64 | KVM_REG_ARM_CORE; + struct kvm_one_reg reg; + u64 *regaddr; + + reg.id = core_reg | KVM_REG_ARM_CORE_REG(regs.pc); + regaddr = core_reg_addr(vcpu, ®); + + /* We only need to set PC atm. regset is 1 */ + return gunyah_rm_vm_set_boot_context( + ghvm->rm, ghvm->vmid, 1, 0, *regaddr); +} + static void gunyah_vm_stop(struct gunyah_vm *ghvm) { int ret; @@ -1790,6 +1808,12 @@ static int gunyah_vm_start(struct gunyah_vm *ghvm) } ghvm->vm_status = GUNYAH_RM_VM_STATUS_READY; + ret = gunyah_vm_set_boot_ctx(ghvm); + if (ret) { + pr_warn("Failed to setup boot context: %d\n", ret); + goto err; + } + ret = gunyah_rm_get_hyp_resources(ghvm->rm, ghvm->vmid, &resources); if (ret) { pr_warn("Failed to get hyp resources for VM: %d\n", ret); From patchwork Thu Apr 24 14:13:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884031 Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB7C32918CC for ; Thu, 24 Apr 2025 14:14:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504073; cv=none; b=S7XbMgG7CENElku7u+yYu/7/1QzQZtEHlxyJqJvKbYLZ65p2bC90xjrqk81aOCdGr85ZFhQcpBPo/Ah+Wq/3W2iVKm07uEQMZoYC3qE5/SUdTzyLkvlnKi7Nxr2zpIAECigI1jcYsUxS8nxjuQgWUPUx7lG/Q1/DRnD7o/AY2DA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504073; c=relaxed/simple; bh=D+WfMC8Jh2PxaUkmB3v59ZfRR9QkXj4ch9yqlrdrhtI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=M05v2IZAPo/UZ5n9hQnT3pFFvl3/e51PTnYhSGdDoJNK6m/KOdDzbEOyAS1f0pLujMVroV81FIDY73C6M5uSL6yw9UD4mNpaVR0ID0XwA+Ka7sbPHGJOjHwywhpwoWHGiw5oekSGLFaacCJApA2bZhMPrI9OBN5QKhMl5qbN1+g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=cu5ALZQi; arc=none smtp.client-ip=209.85.128.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="cu5ALZQi" Received: by mail-wm1-f47.google.com with SMTP id 5b1f17b1804b1-43d04ea9d9aso5012885e9.3 for ; Thu, 24 Apr 2025 07:14:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504068; x=1746108868; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Oa6gfxRAjkR7wfTiIJFT8jbfGXVVwGTh4OuP6vCgukQ=; b=cu5ALZQiIvlO6N17My54QqYBjdiqAlejII9I8zQrhp2pVmzqF+hSfAfcV/5EziFzLU HmeTtCwOQjWkkrPhOQMNCO82OS8I6W+dXyRNbCFcrb0mNF7klQL+wk3qmhDVWZ9nKJxD cECOGDSYt8P9TYQ0I5tS4Vh7MWWCPiG2tjUwfDYm8YmczxC1zKKXNFsfvanK/KPrvSnE hbEuun554g2wES6pGRLaz5MU1gdlj8xo69Pa6LH9iGHhqQir7aSxe/vOHN1G1c7DhIzv 3+N4PlbC2aobhj+JT5f8pjOv7GHH4FkWZ5UcApKcq5rjRT/mUl1qL8Bmtb+XHj2aB8Vy nHlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504068; x=1746108868; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Oa6gfxRAjkR7wfTiIJFT8jbfGXVVwGTh4OuP6vCgukQ=; b=dn2t8zL0VY/Ht4CgC32Rux+UZSp3SHJOQ5j210X89cXPYrdCPGwPxedGM6pfwpDl1W Z3+W0cqrxLLvRwv8O2VJmGDPgcZX2g1pxJ+MT07KxAC2vBJTxx5YGKVJBCo8ZnfhVgqx hsEzewJNtAZJnluqMM/D1xnHvS/usWcYfEvmb1ezdOuXd5Ho0iuFl/9Z+V4in+0ex2+b Erxo/DQkXycG2Pj9eZ1b93hdbf+/6d9xSZ2oalroOOFOytx4IxmYWbP6hI9NtU9qlJx9 xsfwfdWEQ6Rm/fGp84Hq2FMxidHgaM6yk1+j+N+gj8aSvgXq5szW9b/aIwGsQn17YM5W JJUw== X-Forwarded-Encrypted: i=1; AJvYcCVTBxJsCh/NKs/UDLDMsZDVDD2VrXOwrVLeWNRlT1dEUjVt6CgDbuXWPTM/F9zrd/1XTcnymjcCdH7PSvRT@vger.kernel.org X-Gm-Message-State: AOJu0YyzBtFTjQr9xAV2aqVs8HBXF74D1oOMVUnGUzu7RIVaPseq6oO6 2G5bjuMxktg3aADOX4AubHicLQXCaPcpOLdZcEu5AU6iU50wuWwR1lYCrdtew7U= X-Gm-Gg: ASbGncttJhIDUOHhYo+x1w4bG01aCDOFZ66Bhq3dNpuj2UymEwWvvRR+R/Oy1c3Sf7j 8FvP55d/fN4p8FDMRpgw4GcJ+6XD8WAxZbMN9iNqdi73YYJn1qeR3c00D5T/VxTkPKh8IyYFRDk y81aAn5mvPCWgJJK7LrgowAmDuNr4rapjy4YE2R0U30I2oCskas0DqHv1K/C5U/SkeLEvwbrvx9 adgnMO/3pbwXSipwRMoPByLmteWQZ13V+t71Cb95/9wc4zV+ko2YEqir7DAtIAdObIJzC1zNzHV RI5V6Geg4LL8xH8AW9BHfVbBhDdDnoftBCpcky1+N5UMrxSDOXkegs0jqO7wn6NyFkY0rHN6bW1 QIenkKWz3UskaixJU X-Google-Smtp-Source: AGHT+IERVpNU2DILo0bvt8R4DAynThAP/fjqterv2/ng44U7A/g/KpyVK5SJtSkw42xdSMDhah0Vrw== X-Received: by 2002:a05:6000:1acb:b0:391:2f15:c1f4 with SMTP id ffacd0b85a97d-3a06cfad64amr2138772f8f.55.1745504067786; Thu, 24 Apr 2025 07:14:27 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:27 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt , Elliot Berman , Alex Elder Subject: [RFC PATCH 32/34] gunyah: Add hypercalls for sending doorbell Date: Thu, 24 Apr 2025 15:13:39 +0100 Message-Id: <20250424141341.841734-33-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Elliot Berman Gunyah doorbells allow a virtual machine to signal another using interrupts. Add the hypercalls needed to assert the interrupt. Reviewed-by: Alex Elder Signed-off-by: Elliot Berman Signed-off-by: Karim Manaouil --- arch/arm64/gunyah/gunyah_hypercall.c | 38 ++++++++++++++++++++++++++++ include/linux/gunyah.h | 5 ++++ 2 files changed, 43 insertions(+) diff --git a/arch/arm64/gunyah/gunyah_hypercall.c b/arch/arm64/gunyah/gunyah_hypercall.c index 38403dc28c66..3c2672d683ae 100644 --- a/arch/arm64/gunyah/gunyah_hypercall.c +++ b/arch/arm64/gunyah/gunyah_hypercall.c @@ -37,6 +37,8 @@ EXPORT_SYMBOL_GPL(arch_is_gunyah_guest); /* clang-format off */ #define GUNYAH_HYPERCALL_HYP_IDENTIFY GUNYAH_HYPERCALL(0x8000) +#define GUNYAH_HYPERCALL_BELL_SEND GUNYAH_HYPERCALL(0x8012) +#define GUNYAH_HYPERCALL_BELL_SET_MASK GUNYAH_HYPERCALL(0x8015) #define GUNYAH_HYPERCALL_MSGQ_SEND GUNYAH_HYPERCALL(0x801B) #define GUNYAH_HYPERCALL_MSGQ_RECV GUNYAH_HYPERCALL(0x801C) #define GUNYAH_HYPERCALL_ADDRSPACE_MAP GUNYAH_HYPERCALL(0x802B) @@ -64,6 +66,42 @@ void gunyah_hypercall_hyp_identify( } EXPORT_SYMBOL_GPL(gunyah_hypercall_hyp_identify); +/** + * gunyah_hypercall_bell_send() - Assert a gunyah doorbell + * @capid: capability ID of the doorbell + * @new_flags: bits to set on the doorbell + * @old_flags: Filled with the bits set before the send call if return value is GUNYAH_ERROR_OK + */ +enum gunyah_error gunyah_hypercall_bell_send(u64 capid, u64 new_flags, u64 *old_flags) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_BELL_SEND, capid, new_flags, 0, &res); + + if (res.a0 == GUNYAH_ERROR_OK && old_flags) + *old_flags = res.a1; + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_bell_send); + +/** + * gunyah_hypercall_bell_set_mask() - Set masks on a Gunyah doorbell + * @capid: capability ID of the doorbell + * @enable_mask: which bits trigger the receiver interrupt + * @ack_mask: which bits are automatically acknowledged when the receiver + * interrupt is ack'd + */ +enum gunyah_error gunyah_hypercall_bell_set_mask(u64 capid, u64 enable_mask, u64 ack_mask) +{ + struct arm_smccc_res res; + + arm_smccc_1_1_hvc(GUNYAH_HYPERCALL_BELL_SET_MASK, capid, enable_mask, ack_mask, 0, &res); + + return res.a0; +} +EXPORT_SYMBOL_GPL(gunyah_hypercall_bell_set_mask); + /** * gunyah_hypercall_msgq_send() - Send a buffer on a message queue * @capid: capability ID of the message queue to add message diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 72aafc813664..26fdfa3174da 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -329,6 +329,11 @@ gunyah_api_version(const struct gunyah_hypercall_hyp_identify_resp *gunyah_api) void gunyah_hypercall_hyp_identify( struct gunyah_hypercall_hyp_identify_resp *hyp_identity); +enum gunyah_error gunyah_hypercall_bell_send(u64 capid, u64 new_flags, + u64 *old_flags); +enum gunyah_error gunyah_hypercall_bell_set_mask(u64 capid, u64 enable_mask, + u64 ack_mask); + /* Immediately raise RX vIRQ on receiver VM */ #define GUNYAH_HYPERCALL_MSGQ_TX_FLAGS_PUSH BIT(0) From patchwork Thu Apr 24 14:13:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884932 Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CC75B2957DB for ; Thu, 24 Apr 2025 14:14:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504073; cv=none; b=C602nOkIE5agjh1CK3yKCLzsKzCMoyXUh4MiWfZY8nqOI7el7qCCo8sk94nWocwJyH8wrqKiLdnM+9mggaGVQJKtI+dL9gVQ1wTMH+QEpsRLr5x7ZPCSSDammhWOsoAJ1BD/JtA/jSUfkOx2ARsF/s/119oE7fYAP2VxakBGu60= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504073; c=relaxed/simple; bh=fw0dWabNAHpVG50PP6u8LJhYKTqtfzfYN79tONhSgqw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mf1FMgBxBPxEgJnk6wivspOjWrCJJvPwVajjSye6LBKzSdwkmQm6bI0t+jkXTVIT8HWqQCSLlv2RtqsJh0wMqGBi/BYQXHUoaBTetMe7bq+HiIW45VD5VcGjYhAf3iq2Tlo1l4nLqJoqQMUevU27AK6iIPDeEua3E0ncB8+jz5U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=atJfyX5S; arc=none smtp.client-ip=209.85.221.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="atJfyX5S" Received: by mail-wr1-f48.google.com with SMTP id ffacd0b85a97d-39141ffa9fcso1251811f8f.0 for ; Thu, 24 Apr 2025 07:14:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504069; x=1746108869; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IXvNlcutyMXVdoUVJkL/Ta8/a2mdxRJrpPZQuCyEfUg=; b=atJfyX5SL9JWC3DgDFlzekyvtegoru7FHIeAWSer2RL8B2oqSTeEBPso/EErYNOhQw GynviIERB4+BRDCddxvnf5ZCrI8grdjdTFpReSNrTo8VEm0rZiWwjGEH99jdgs5R06Lp LeHZywumUi1xcYgi6paPqbga8qvUrr1g/fVzUT81Y3L9HboF/2Tgz1mXQ4MPE/2Wmc5i Bg0nTbpM/ouQA/kVbVZv5Fqu3zPOuOWbDbYh24xIqR6POCxXJL9suBsRwyHAc6YjawCF XL99SLseyGwrOaccFlYJkNoQCSqhmkmoinrLnp2DUtF6gMUMGvfoCTSSZhaCCfro4ix0 m69g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504069; x=1746108869; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IXvNlcutyMXVdoUVJkL/Ta8/a2mdxRJrpPZQuCyEfUg=; b=OfB6eDr61sbF6LxaQdBjUGsjYXyOTXvSfJlvp84UMw2/e3Je+RijZWsRgH1s3U9LPM EU0x+euP7sBjIkstOAWtfk/wgGazoKvysMTMY2879XF4KSo0W1Ri9ahzjs4Am1qchXLw T7nep5K/WjGX5AySI0Jh4CDDEomfIW9MfAIgVfE/nugENYzl0WY78iTeMzMRlCWo0FF1 aV0D/L10vpqX/HMR9qp5ymd1NEjq5yxH/YnwWZDBmmIOrHmBiwX/0r/np1YSfw/oLLcl LfyTd/NBx8B+zLH6bZV1IfqslTTHN3wqFSy21pyknTk3u5sxn2bxzRH95eLdUqAuckP0 eLWw== X-Forwarded-Encrypted: i=1; AJvYcCWU5St5ePIKB/iLscvvI095jftE6gxEXdq4BF4KTS40U3aFyz7vIPTb72zCRIgcr7/NnKvyz2umR4sHsYHr@vger.kernel.org X-Gm-Message-State: AOJu0YwXHyOinbLPVu5cGVkoxmrPSsLQDJ7CSZrSbdL4uTDYV/SJqQjQ Pwj+ryokZPnNrJZpkTzhhiFQTBdbZp2fBHpKCHYiJneA7NM3tyCozN7JRevUU6A= X-Gm-Gg: ASbGncsTQ54LuXe1i27Yokop+ZWhIGRyW+mN4GX4BQ2chiyC4sWcQJb0kFjHiyZP4Ky YxvE7JBYasvXTbUcEo/rXjdiFAHeTKibbbChry6ZJAi+9irnaqehf4M3O55WOzLXndXNXnxd8Y7 8p8G8wXFqfhpx/kBiArMB8RCHJb3fM4+rAhjnxK8FI/ZwEP6Nu4Utvgry7nK416dlxIFEysA0cK HU++927CGvKKXxFV6X3wAbEo9tS652XvwhX9dt7qG5z6BOMWVYv5vXPLhequiK/0BS89Ra8obU6 9nlfXm5md1Cc9hixg6YLmOWTKoufctWAalW6GHJrxVfYFvi8vcycuQglMkf0KSzsJk91y/eUExK pzhhRH1uAvifOLjct X-Google-Smtp-Source: AGHT+IH0tCAHy+56/cv4ZEG1Nkj43NDfif06A0s6QNH24+K7YCAgJ/1BfT2MGE/c41oOF+y78aq+Lw== X-Received: by 2002:a5d:4092:0:b0:39a:c80b:8288 with SMTP id ffacd0b85a97d-3a06cf635f7mr2018268f8f.33.1745504069033; Thu, 24 Apr 2025 07:14:29 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:28 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 33/34] KVM: gunyah: Implement irqfd interface Date: Thu, 24 Apr 2025 15:13:40 +0100 Message-Id: <20250424141341.841734-34-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Enables support for injecting interrupts into guest vCPUs via KVM's irqfd interface. Irqfds are requested from userspace via KVM_IRQFD ioctl. kvm_arch_irqfd_init() implementation, in Gunyah, then creates a resource ticket for the irqfd. The userspace must also create a devicetree node to create a doorbell with the corresponding label. Later after VM configuration (in gunyah_vm_start()), the resource ticket will be populated and the irqfd instance will be bound to its corresponding doorbell (identified with a capability id). When userspace asserts the irq line, irqfd->set_irq() callback is called to inject an interrupt into the guest via a hypercall. Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 100 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index df922be2429e..23b9128bf5b1 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -29,6 +29,14 @@ #define WRITE_TAG (1 << 0) #define SHARE_TAG (1 << 1) +struct gunyah_irqfd { + struct gunyah_vm *ghvm; + struct gunyah_resource *ghrsc; + struct gunyah_vm_resource_ticket ticket; + struct kvm_kernel_irqfd irqfd; + bool level; +}; + static int gunyah_vm_start(struct gunyah_vm *ghvm); static enum kvm_mode kvm_mode = KVM_MODE_DEFAULT; @@ -852,6 +860,98 @@ static int gunyah_start_paging(struct gunyah_vm *ghvm) return ret; } +static bool gunyah_irqfd_populate(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc) +{ + struct gunyah_irqfd *irqfd = + container_of(ticket, struct gunyah_irqfd, ticket); + int ret; + + if (irqfd->ghrsc) { + pr_warn("irqfd%d already got a Gunyah resource", irqfd->ticket.label); + return false; + } + + irqfd->ghrsc = ghrsc; + if (irqfd->level) { + /* Configure the bell to trigger when bit 0 is asserted (see + * irq_wakeup) and for bell to automatically clear bit 0 once + * received by the VM (ack_mask). need to make sure bit 0 is cleared right away, + * otherwise the line will never be deasserted. Emulating edge + * trigger interrupt does not need to set either mask + * because irq is listed only once per gunyah_hypercall_bell_send + */ + ret = gunyah_hypercall_bell_set_mask(irqfd->ghrsc->capid, 1, 1); + if (ret) + pr_warn("irq %d couldn't be set as level triggered." + "Might cause IRQ storm if asserted\n", + irqfd->ticket.label); + } + + return true; +} + +static void gunyah_irqfd_unpopulate(struct gunyah_vm_resource_ticket *ticket, + struct gunyah_resource *ghrsc) +{ +} + +static int gunyah_set_irq(struct kvm_kernel_irqfd *kvm_irqfd) +{ + int ret; + struct gunyah_irqfd *irqfd = + container_of(kvm_irqfd, struct gunyah_irqfd, irqfd); + + if (irqfd->ghrsc) { + if (gunyah_hypercall_bell_send(irqfd->ghrsc->capid, 1, NULL)) { + pr_err_ratelimited("Failed to inject interrupt %d: %d\n", + irqfd->ticket.label, ret); + return -1; + } + } else { + pr_err_ratelimited("Premature injection of interrupt\n"); + return -1; + } + + return 1; +} + +struct kvm_kernel_irqfd *kvm_arch_irqfd_alloc(void) +{ + struct gunyah_irqfd *irqfd; + + irqfd = kzalloc(sizeof(struct gunyah_irqfd), GFP_KERNEL); + if (!irqfd) + return NULL; + + return &irqfd->irqfd; +} + +void kvm_arch_irqfd_free(struct kvm_kernel_irqfd *kvm_irqfd) +{ + struct gunyah_irqfd *irqfd = container_of(kvm_irqfd, struct gunyah_irqfd, irqfd); + + gunyah_vm_remove_resource_ticket(irqfd->ghvm, &irqfd->ticket); + kfree(irqfd); +} + +int kvm_arch_irqfd_init(struct kvm_kernel_irqfd *kvm_irqfd) +{ + struct gunyah_vm *ghvm = container_of(kvm_irqfd->kvm, struct gunyah_vm, kvm); + struct gunyah_irqfd *irqfd = container_of(kvm_irqfd, struct gunyah_irqfd, irqfd); + + kvm_irqfd->set_irq = gunyah_set_irq; + + irqfd->ghvm = ghvm; + + irqfd->ticket.resource_type = GUNYAH_RESOURCE_TYPE_BELL_TX; + irqfd->ticket.label = kvm_irqfd->gsi; + irqfd->ticket.populate = gunyah_irqfd_populate; + irqfd->ticket.unpopulate = gunyah_irqfd_unpopulate; + + return gunyah_vm_add_resource_ticket(ghvm, &irqfd->ticket); +} + int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu) { return kvm_vcpu_exiting_guest_mode(vcpu) == IN_GUEST_MODE; From patchwork Thu Apr 24 14:13:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karim Manaouil X-Patchwork-Id: 884030 Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1423B29617A for ; Thu, 24 Apr 2025 14:14:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504074; cv=none; b=Pr47TLa1KErdOWnMosDVIsAZY48Ubsf1MB53gO23Sblsa5/3rcZxAN3fIOIVb0f0Rrt4IWQaXF6/zvQNF6vtNdHbVL5Ihmldkz28Ea4gtg02UMy++rwYG1iKH/n7o6W6gG71RauHtmrq/iZh0DDJZdlRC5VcXIskDBadEi7zYWw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745504074; c=relaxed/simple; bh=N53i4bFiZ0PwECXO+3O9PEZg7KbhghQli4Mhy0f7goE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KbEsX9wB7s3VmUoKVl/JES/4gLp2OoO1+y3xK0ZNCpupiIc4SAUc/XeF8ch8SpCwtsLNJ2qmmebnShHPEHI2iOWMvw9M1qKY8qlwbGQbBSHux8FwV3MfTOoLKt8Io6HNQ/iIUu4ML9iW9UDeJgLkf1zQTLSSdV4t2CcHZ4xQzsA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=A/BNi9dr; arc=none smtp.client-ip=209.85.221.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="A/BNi9dr" Received: by mail-wr1-f46.google.com with SMTP id ffacd0b85a97d-39c1efc4577so642624f8f.0 for ; Thu, 24 Apr 2025 07:14:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1745504070; x=1746108870; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3i+gLyWtr6XPiioNUTryX7U5TAHJs0bj5+QGltLz/R0=; b=A/BNi9drAEtZYzDLWDDHNkbHoLQ7G/rSzO6lWtT9wJBxxPLNRLXjqG2CV9Xf5QbQBT Sd9zeNztMZUVOCRA5OflPbZpQ0GAQKMtgYOZVkl6AYSf1iMSjRA5bELWM2/2Ro5G2jLp uJnN4FsqDjPExjGbLQI6o4GhGLVBgLi7/xUjnivg4y9khXjgA3Dmmzsi1UOyqNBUNBT8 MVi1QC4XXmiTDjqog5ZbsG6l9kZaoZlE4hgdKAohqlGXx7MczFhDs/rVxT/XoUHclq03 v+rEjUJ0/gcda3IVfMtpZfSgpUg8mpRx1XLX5Yctye5hH+r06rdmsphQr+bWu8Cqrz5t eCpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745504070; x=1746108870; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3i+gLyWtr6XPiioNUTryX7U5TAHJs0bj5+QGltLz/R0=; b=nFkLWERdMqrbmb6Bn/Xd8w1EPqd7A7E3+FzvEo34J5xKuQsRKWpD+P4UKAWSAS+dEv EHWPuJnJLSfs4SH09EFMY9jkukqkjDpIunIP5Hn+FZ02aELRVcF6S3Fwo8gqX7Jjx3T5 Jl5BypI2SPdDYrbcx7ojO2Mi94CF0o7rWDQ00NW6AvY4vq9usm5qsgHU59knisV5h1Xh F5VTLMBynHEhzKivVScgxYOW5Fy+Hsjstv2FvoRd9Zdct4MMkdhAuQ8STg6U5nTGPRhR kMXk/WRzECFHS166eHPnP9PDU/NlR4mqOj8j8ehva3GQTpsxl40UFzzsJ8LPL+gKu/dX XoYg== X-Forwarded-Encrypted: i=1; AJvYcCWfuHDULefg6NOGzN5qYIuLZRGIG6jqIrbYDp3IWjvQzdNf442yqzk2t40222AAufvO68hC5h568l1FwBbE@vger.kernel.org X-Gm-Message-State: AOJu0YyQO1F35yIDl1leta4NN3h9pZmVLrAlbo9WdJ0dtV0pfV+tP6sS FxCLb7LbIJ1Zako1FsDg0RhvWRidjWzX4BMM6weJI9Z12pz2GhT3fjncu3SOdNs= X-Gm-Gg: ASbGnct30OFupbStl1zr9AObviu95MqVs4WvohlDegOdTw17lBELZVTS+lEXB9iKWMr eddgBpalJPI2yJ3fEad2d9y79w6d9SbHX+hEvxfJD3VjzYiWzg2SUtWzKQ+3J1yazU7wi8dti+c 1oZru95xZCe1r1ENDLbwNKP4TuQmkGvUH/OYvLKFz2FEDiEx0ft3lrZV1Z9UN6Yg09cCI1KBh7a QciRbwWucpuOinKO1HAdy9coWHekUdUcB3r2LEJcFFdrgregdFXRV2oEzpMphKs3SDAujU1mKZK RXWm6lcoqwk2XJmmor8SMysaNzHs0LI7YcJcQCdU3djbyr8h/AxnYcHUIcoIsYhKPVv9y0a+v7F SPuv6AVJfREQtYuAm X-Google-Smtp-Source: AGHT+IFOk65S5hPF/j31g7Of/aiN0lBptA3ImD/zObUzEDJUJE+zLunCG1fROxNheO7aYYEMJ9HW0A== X-Received: by 2002:a5d:5f87:0:b0:38d:b325:471f with SMTP id ffacd0b85a97d-3a06cf5ac50mr2557177f8f.15.1745504070317; Thu, 24 Apr 2025 07:14:30 -0700 (PDT) Received: from seksu.systems-nuts.com (stevens.inf.ed.ac.uk. [129.215.164.122]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a06d4a8150sm2199951f8f.7.2025.04.24.07.14.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Apr 2025 07:14:29 -0700 (PDT) From: Karim Manaouil To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: Karim Manaouil , Alexander Graf , Alex Elder , Catalin Marinas , Fuad Tabba , Joey Gouly , Jonathan Corbet , Marc Zyngier , Mark Brown , Mark Rutland , Oliver Upton , Paolo Bonzini , Prakruthi Deepak Heragu , Quentin Perret , Rob Herring , Srinivas Kandagatla , Srivatsa Vaddagiri , Will Deacon , Haripranesh S , Carl van Schaik , Murali Nalajala , Sreenivasulu Chalamcharla , Trilok Soni , Stefan Schmidt Subject: [RFC PATCH 34/34] KVM: gunyah: enable KVM for Gunyah Date: Thu, 24 Apr 2025 15:13:41 +0100 Message-Id: <20250424141341.841734-35-karim.manaouil@linaro.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250424141341.841734-1-karim.manaouil@linaro.org> References: <20250424141341.841734-1-karim.manaouil@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Enable KVM to act as the interface to create and manage Qualcomm's Gunyah-based virtual machines [1]. [1] https://www.qualcomm.com/developer/blog/2024/08/learn-about-gunyah--qualcomm-s-open-source--lightweight-hypervis Signed-off-by: Karim Manaouil --- arch/arm64/kvm/gunyah.c | 12 ++++++++++++ drivers/virt/gunyah/gunyah.c | 3 +++ include/linux/gunyah.h | 1 + 3 files changed, 16 insertions(+) diff --git a/arch/arm64/kvm/gunyah.c b/arch/arm64/kvm/gunyah.c index 23b9128bf5b1..91f0260ef3f5 100644 --- a/arch/arm64/kvm/gunyah.c +++ b/arch/arm64/kvm/gunyah.c @@ -2071,3 +2071,15 @@ void kvm_arch_free_vm(struct kvm *kvm) kfree(ghvm); } + +int kvm_gunyah_init(void) +{ + int err; + + err = kvm_init(sizeof(struct kvm_vcpu), 0, THIS_MODULE); + if (err) + return err; + + return 0; +} +EXPORT_SYMBOL_GPL(kvm_gunyah_init); diff --git a/drivers/virt/gunyah/gunyah.c b/drivers/virt/gunyah/gunyah.c index 3e795e3ba881..629c812fbac0 100644 --- a/drivers/virt/gunyah/gunyah.c +++ b/drivers/virt/gunyah/gunyah.c @@ -28,6 +28,9 @@ static int gunyah_probe(struct platform_device *pdev) return -ENODEV; } + if (!kvm_gunyah_init()) + pr_info("KVM for Gunyah is available!\n"); + return devm_of_platform_populate(&pdev->dev); } diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h index 26fdfa3174da..7d43449c4547 100644 --- a/include/linux/gunyah.h +++ b/include/linux/gunyah.h @@ -421,4 +421,5 @@ enum gunyah_error gunyah_hypercall_vcpu_run(u64 capid, unsigned long *resume_data, struct gunyah_hypercall_vcpu_run_resp *resp); +int kvm_gunyah_init(void); #endif