diff mbox series

[Part1,v5,17/38] x86/mm: Add support to validate memory when changing C-bit

Message ID 20210820151933.22401-18-brijesh.singh@amd.com
State New
Headers show
Series [Part1,v5,01/38] x86/mm: Add sev_feature_enabled() helper | expand

Commit Message

Brijesh Singh Aug. 20, 2021, 3:19 p.m. UTC 
The set_memory_{encrypt,decrypt}() are used for changing the pages
from decrypted (shared) to encrypted (private) and vice versa.
When SEV-SNP is active, the page state transition needs to go through
additional steps.

If the page is transitioned from shared to private, then perform the
following after the encryption attribute is set in the page table:

1. Issue the page state change VMGEXIT to add the memory region in
   the RMP table.
2. Validate the memory region after the RMP entry is added.

To maintain the security guarantees, if the page is transitioned from
private to shared, then perform the following before encryption attribute
is removed from the page table:

1. Invalidate the page.
2. Issue the page state change VMGEXIT to remove the page from RMP table.

To change the page state in the RMP table, use the Page State Change
VMGEXIT defined in the GHCB specification.

Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 arch/x86/include/asm/sev-common.h |  24 +++++
 arch/x86/include/asm/sev.h        |   4 +
 arch/x86/include/uapi/asm/svm.h   |   2 +
 arch/x86/kernel/sev.c             | 165 ++++++++++++++++++++++++++++++
 arch/x86/mm/pat/set_memory.c      |  15 +++
 5 files changed, 210 insertions(+)

Comments

Borislav Petkov Aug. 25, 2021, 11:06 a.m. UTC | #1 
On Fri, Aug 20, 2021 at 10:19:12AM -0500, Brijesh Singh wrote:
> +	while (hdr->cur_entry <= hdr->end_entry) {

> +		ghcb_set_sw_scratch(ghcb, (u64)__pa(data));

> +

> +		ret = sev_es_ghcb_hv_call(ghcb, NULL, SVM_VMGEXIT_PSC, 0, 0);

> +

> +		/*

> +		 * Page State Change VMGEXIT can pass error code through

> +		 * exit_info_2.

> +		 */

> +		if (WARN(ret || ghcb->save.sw_exit_info_2,

> +			 "SEV-SNP: PSC failed ret=%d exit_info_2=%llx\n",

> +			 ret, ghcb->save.sw_exit_info_2)) {

> +			ret = 1;

> +			goto out;

> +		}

> +

> +		/*

> +		 * Sanity check that entry processing is not going backward.

> +		 * This will happen only if hypervisor is tricking us.

> +		 */

> +		if (WARN(hdr->end_entry > end_entry || cur_entry > hdr->cur_entry,

> +			 "SEV-SNP:  PSC processing going backward, end_entry %d (got %d) cur_entry %d (got %d)\n",


I really meant putting the beginning of that string at the very first
position on the line:

                if (WARN(hdr->end_entry > end_entry || cur_entry > hdr->cur_entry,
"SEV-SNP: PSC processing going backward, end_entry %d (got %d) cur_entry %d (got %d)\n",
                         end_entry, hdr->end_entry, cur_entry, hdr->cur_entry)) {

Exactly like this!

...

> +static void set_page_state(unsigned long vaddr, unsigned int npages, int op)

> +{

> +	unsigned long vaddr_end, next_vaddr;

> +	struct snp_psc_desc *desc;

> +

> +	vaddr = vaddr & PAGE_MASK;

> +	vaddr_end = vaddr + (npages << PAGE_SHIFT);

> +

> +	desc = kmalloc(sizeof(*desc), GFP_KERNEL_ACCOUNT);


And again, from previous review:

kzalloc() so that you don't have to memset() later in
__set_page_state().

> +	if (!desc)

> +		panic("SEV-SNP: failed to alloc memory for PSC descriptor\n");


"allocate" fits just fine too.

> +

> +	while (vaddr < vaddr_end) {

> +		/*

> +		 * Calculate the last vaddr that can be fit in one

> +		 * struct snp_psc_desc.

> +		 */

> +		next_vaddr = min_t(unsigned long, vaddr_end,

> +				   (VMGEXIT_PSC_MAX_ENTRY * PAGE_SIZE) + vaddr);

> +

> +		__set_page_state(desc, vaddr, next_vaddr, op);

> +

> +		vaddr = next_vaddr;

> +	}

> +

> +	kfree(desc);

> +}


-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
Brijesh Singh Aug. 25, 2021, 1:54 p.m. UTC | #2 
On 8/25/21 6:06 AM, Borislav Petkov wrote:
> 

> I really meant putting the beginning of that string at the very first

> position on the line:

> 

>                  if (WARN(hdr->end_entry > end_entry || cur_entry > hdr->cur_entry,

> "SEV-SNP: PSC processing going backward, end_entry %d (got %d) cur_entry %d (got %d)\n",

>                           end_entry, hdr->end_entry, cur_entry, hdr->cur_entry)) {

> 

> Exactly like this!

> 


Noted.

> ...

> 

>> +static void set_page_state(unsigned long vaddr, unsigned int npages, int op)

>> +{

>> +	unsigned long vaddr_end, next_vaddr;

>> +	struct snp_psc_desc *desc;

>> +

>> +	vaddr = vaddr & PAGE_MASK;

>> +	vaddr_end = vaddr + (npages << PAGE_SHIFT);

>> +

>> +	desc = kmalloc(sizeof(*desc), GFP_KERNEL_ACCOUNT);

> 

> And again, from previous review:

> 

> kzalloc() so that you don't have to memset() later in

> __set_page_state().

> 


I replied to your previous comment. Depending on the npages value, the 
__set_page_state() will be called multiple times and on each call it 
needs to clear desc before populate it. So, I do not see strong reason 
to use kzalloc() during the desc allocation. I thought you were okay 
with that explanation.


>> +	if (!desc)

>> +		panic("SEV-SNP: failed to alloc memory for PSC descriptor\n");

> 

> "allocate" fits just fine too.

> 


Noted.

thanks
Borislav Petkov Aug. 25, 2021, 2 p.m. UTC | #3 
On Wed, Aug 25, 2021 at 08:54:31AM -0500, Brijesh Singh wrote:
> I replied to your previous comment. Depending on the npages value, the

> __set_page_state() will be called multiple times and on each call it

> needs to clear desc before populate it.


Ah, now I missed it, sorry.

> So, I do not see strong reason to use kzalloc() during the desc

> allocation.


Yeah, then you don't need it.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
Borislav Petkov Aug. 27, 2021, 5:09 p.m. UTC | #4 
On Fri, Aug 20, 2021 at 10:19:12AM -0500, Brijesh Singh wrote:
> diff --git a/arch/x86/include/asm/sev-common.h b/arch/x86/include/asm/sev-common.h

> index 37aa77565726..3388db814fd0 100644

> --- a/arch/x86/include/asm/sev-common.h

> +++ b/arch/x86/include/asm/sev-common.h

> @@ -74,6 +74,8 @@

>  enum psc_op {

>  	SNP_PAGE_STATE_PRIVATE = 1,

>  	SNP_PAGE_STATE_SHARED,

> +	SNP_PAGE_STATE_PSMASH,

> +	SNP_PAGE_STATE_UNSMASH,


Those two are unused in this set, AFAICT.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
diff mbox series

Patch

diff --git a/arch/x86/include/asm/sev-common.h b/arch/x86/include/asm/sev-common.h
index 37aa77565726..3388db814fd0 100644
--- a/arch/x86/include/asm/sev-common.h
+++ b/arch/x86/include/asm/sev-common.h
@@ -74,6 +74,8 @@ 
 enum psc_op {
 	SNP_PAGE_STATE_PRIVATE = 1,
 	SNP_PAGE_STATE_SHARED,
+	SNP_PAGE_STATE_PSMASH,
+	SNP_PAGE_STATE_UNSMASH,
 };
 
 #define GHCB_MSR_PSC_REQ		0x014
@@ -99,6 +101,28 @@  enum psc_op {
 
 #define GHCB_HV_FT_SNP			BIT_ULL(0)
 
+/* SNP Page State Change NAE event */
+#define VMGEXIT_PSC_MAX_ENTRY		253
+
+struct psc_hdr {
+	u16 cur_entry;
+	u16 end_entry;
+	u32 reserved;
+} __packed;
+
+struct psc_entry {
+	u64	cur_page	: 12,
+		gfn		: 40,
+		operation	: 4,
+		pagesize	: 1,
+		reserved	: 7;
+} __packed;
+
+struct snp_psc_desc {
+	struct psc_hdr hdr;
+	struct psc_entry entries[VMGEXIT_PSC_MAX_ENTRY];
+} __packed;
+
 #define GHCB_MSR_TERM_REQ		0x100
 #define GHCB_MSR_TERM_REASON_SET_POS	12
 #define GHCB_MSR_TERM_REASON_SET_MASK	0xf
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index ecd8cd8c5908..005f230d0406 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -109,6 +109,8 @@  void __init early_snp_set_memory_private(unsigned long vaddr, unsigned long padd
 void __init early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr,
 					unsigned int npages);
 void __init snp_prep_memory(unsigned long paddr, unsigned int sz, enum psc_op op);
+void snp_set_memory_shared(unsigned long vaddr, unsigned int npages);
+void snp_set_memory_private(unsigned long vaddr, unsigned int npages);
 #else
 static inline void sev_es_ist_enter(struct pt_regs *regs) { }
 static inline void sev_es_ist_exit(void) { }
@@ -121,6 +123,8 @@  early_snp_set_memory_private(unsigned long vaddr, unsigned long paddr, unsigned
 static inline void __init
 early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr, unsigned int npages) { }
 static inline void __init snp_prep_memory(unsigned long paddr, unsigned int sz, enum psc_op op) { }
+static inline void snp_set_memory_shared(unsigned long vaddr, unsigned int npages) { }
+static inline void snp_set_memory_private(unsigned long vaddr, unsigned int npages) { }
 #endif
 
 #endif
diff --git a/arch/x86/include/uapi/asm/svm.h b/arch/x86/include/uapi/asm/svm.h
index b0ad00f4c1e1..0dcdb6e0c913 100644
--- a/arch/x86/include/uapi/asm/svm.h
+++ b/arch/x86/include/uapi/asm/svm.h
@@ -108,6 +108,7 @@ 
 #define SVM_VMGEXIT_AP_JUMP_TABLE		0x80000005
 #define SVM_VMGEXIT_SET_AP_JUMP_TABLE		0
 #define SVM_VMGEXIT_GET_AP_JUMP_TABLE		1
+#define SVM_VMGEXIT_PSC				0x80000010
 #define SVM_VMGEXIT_HV_FEATURES			0x8000fffd
 #define SVM_VMGEXIT_UNSUPPORTED_EVENT		0x8000ffff
 
@@ -219,6 +220,7 @@ 
 	{ SVM_VMGEXIT_NMI_COMPLETE,	"vmgexit_nmi_complete" }, \
 	{ SVM_VMGEXIT_AP_HLT_LOOP,	"vmgexit_ap_hlt_loop" }, \
 	{ SVM_VMGEXIT_AP_JUMP_TABLE,	"vmgexit_ap_jump_table" }, \
+	{ SVM_VMGEXIT_PSC,	"vmgexit_page_state_change" }, \
 	{ SVM_VMGEXIT_HV_FEATURES,	"vmgexit_hypervisor_feature" }, \
 	{ SVM_EXIT_ERR,         "invalid_guest_state" }
 
diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c
index 0ddc032fd252..106b4aaddfde 100644
--- a/arch/x86/kernel/sev.c
+++ b/arch/x86/kernel/sev.c
@@ -693,6 +693,171 @@  void __init snp_prep_memory(unsigned long paddr, unsigned int sz, enum psc_op op
 		WARN(1, "invalid memory op %d\n", op);
 }
 
+static int vmgexit_psc(struct snp_psc_desc *desc)
+{
+	int cur_entry, end_entry, ret;
+	struct snp_psc_desc *data;
+	struct ghcb_state state;
+	struct ghcb *ghcb;
+	struct psc_hdr *hdr;
+	unsigned long flags;
+
+	local_irq_save(flags);
+
+	ghcb = __sev_get_ghcb(&state);
+	if (unlikely(!ghcb))
+		panic("SEV-SNP: Failed to get GHCB\n");
+
+	/* Copy the input desc into GHCB shared buffer */
+	data = (struct snp_psc_desc *)ghcb->shared_buffer;
+	memcpy(ghcb->shared_buffer, desc, sizeof(*desc));
+
+	hdr = &data->hdr;
+	cur_entry = hdr->cur_entry;
+	end_entry = hdr->end_entry;
+
+	/*
+	 * As per the GHCB specification, the hypervisor can resume the guest
+	 * before processing all the entries. Checks whether all the entries
+	 * are processed. If not, then keep retrying.
+	 *
+	 * The stragtegy here is to wait for the hypervisor to change the page
+	 * state in the RMP table before guest access the memory pages. If the
+	 * page state was not successful, then later memory access will result
+	 * in the crash.
+	 */
+	while (hdr->cur_entry <= hdr->end_entry) {
+		ghcb_set_sw_scratch(ghcb, (u64)__pa(data));
+
+		ret = sev_es_ghcb_hv_call(ghcb, NULL, SVM_VMGEXIT_PSC, 0, 0);
+
+		/*
+		 * Page State Change VMGEXIT can pass error code through
+		 * exit_info_2.
+		 */
+		if (WARN(ret || ghcb->save.sw_exit_info_2,
+			 "SEV-SNP: PSC failed ret=%d exit_info_2=%llx\n",
+			 ret, ghcb->save.sw_exit_info_2)) {
+			ret = 1;
+			goto out;
+		}
+
+		/*
+		 * Sanity check that entry processing is not going backward.
+		 * This will happen only if hypervisor is tricking us.
+		 */
+		if (WARN(hdr->end_entry > end_entry || cur_entry > hdr->cur_entry,
+			 "SEV-SNP:  PSC processing going backward, end_entry %d (got %d) cur_entry %d (got %d)\n",
+			 end_entry, hdr->end_entry, cur_entry, hdr->cur_entry)) {
+			ret = 1;
+			goto out;
+		}
+
+		/* Verify that reserved bit is not set */
+		if (WARN(hdr->reserved, "Reserved bit is set in the PSC header\n")) {
+			ret = 1;
+			goto out;
+		}
+	}
+
+out:
+	__sev_put_ghcb(&state);
+	local_irq_restore(flags);
+
+	return 0;
+}
+
+static void __set_page_state(struct snp_psc_desc *data, unsigned long vaddr,
+			     unsigned long vaddr_end, int op)
+{
+	struct psc_hdr *hdr;
+	struct psc_entry *e;
+	unsigned long pfn;
+	int i;
+
+	hdr = &data->hdr;
+	e = data->entries;
+
+	memset(data, 0, sizeof(*data));
+	i = 0;
+
+	while (vaddr < vaddr_end) {
+		if (is_vmalloc_addr((void *)vaddr))
+			pfn = vmalloc_to_pfn((void *)vaddr);
+		else
+			pfn = __pa(vaddr) >> PAGE_SHIFT;
+
+		e->gfn = pfn;
+		e->operation = op;
+		hdr->end_entry = i;
+
+		/*
+		 * The GHCB specification provides the flexibility to
+		 * use either 4K or 2MB page size in the RMP table.
+		 * The current SNP support does not keep track of the
+		 * page size used in the RMP table. To avoid the
+		 * overlap request, use the 4K page size in the RMP
+		 * table.
+		 */
+		e->pagesize = RMP_PG_SIZE_4K;
+
+		vaddr = vaddr + PAGE_SIZE;
+		e++;
+		i++;
+	}
+
+	if (vmgexit_psc(data))
+		sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PSC);
+}
+
+static void set_page_state(unsigned long vaddr, unsigned int npages, int op)
+{
+	unsigned long vaddr_end, next_vaddr;
+	struct snp_psc_desc *desc;
+
+	vaddr = vaddr & PAGE_MASK;
+	vaddr_end = vaddr + (npages << PAGE_SHIFT);
+
+	desc = kmalloc(sizeof(*desc), GFP_KERNEL_ACCOUNT);
+	if (!desc)
+		panic("SEV-SNP: failed to alloc memory for PSC descriptor\n");
+
+	while (vaddr < vaddr_end) {
+		/*
+		 * Calculate the last vaddr that can be fit in one
+		 * struct snp_psc_desc.
+		 */
+		next_vaddr = min_t(unsigned long, vaddr_end,
+				   (VMGEXIT_PSC_MAX_ENTRY * PAGE_SIZE) + vaddr);
+
+		__set_page_state(desc, vaddr, next_vaddr, op);
+
+		vaddr = next_vaddr;
+	}
+
+	kfree(desc);
+}
+
+void snp_set_memory_shared(unsigned long vaddr, unsigned int npages)
+{
+	if (!sev_feature_enabled(SEV_SNP))
+		return;
+
+	pvalidate_pages(vaddr, npages, 0);
+
+	set_page_state(vaddr, npages, SNP_PAGE_STATE_SHARED);
+}
+
+void snp_set_memory_private(unsigned long vaddr, unsigned int npages)
+{
+	if (!sev_feature_enabled(SEV_SNP))
+		return;
+
+	set_page_state(vaddr, npages, SNP_PAGE_STATE_PRIVATE);
+
+	pvalidate_pages(vaddr, npages, 1);
+}
+
 int sev_es_setup_ap_jump_table(struct real_mode_header *rmh)
 {
 	u16 startup_cs, startup_ip;
diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index ad8a5c586a35..8e6952d626ec 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -29,6 +29,7 @@ 
 #include <asm/proto.h>
 #include <asm/memtype.h>
 #include <asm/set_memory.h>
+#include <asm/sev.h>
 
 #include "../mm_internal.h"
 
@@ -2009,8 +2010,22 @@  static int __set_memory_enc_dec(unsigned long addr, int numpages, bool enc)
 	 */
 	cpa_flush(&cpa, !this_cpu_has(X86_FEATURE_SME_COHERENT));
 
+	/*
+	 * To maintain the security gurantees of SEV-SNP guest invalidate the memory
+	 * before clearing the encryption attribute.
+	 */
+	if (!enc)
+		snp_set_memory_shared(addr, numpages);
+
 	ret = __change_page_attr_set_clr(&cpa, 1);
 
+	/*
+	 * Now that memory is mapped encrypted in the page table, validate it
+	 * so that is consistent with the above page state.
+	 */
+	if (!ret && enc)
+		snp_set_memory_private(addr, numpages);
+
 	/*
 	 * After changing the encryption attribute, we need to flush TLBs again
 	 * in case any speculative TLB caching occurred (but no need to flush