| Message ID | cover.1695921656.git.lukas@wunner.de |
|---|---|
| Headers | show |
| Series | PCI device authentication | expand |
This looks great Lukas, some forward looking review comments below. Lukas Wunner wrote: > Authenticate PCI devices with CMA-SPDM (PCIe r6.1 sec 6.31) and > expose the result in sysfs. This enables user-defined policies > such as forbidding driver binding to devices which failed > authentication. > > CMA-SPDM forms the basis for PCI encryption (PCIe r6.1 sec 6.33), > which will be submitted later. > > The meat of the series is in patches [07/12] and [08/12], which contain > the SPDM library and the CMA glue code (the PCI-adaption of SPDM). > > The reason why SPDM is done in-kernel is provided in patch [10/12]: > Briefly, when devices are reauthenticated on resume from system sleep, > user space is not yet available. Same when reauthenticating after > recovery from reset. > > One use case for CMA-SPDM and PCI encryption is confidential access > to passed-through devices: Neither the host nor other guests are > able to eavesdrop on device accesses, in particular if guest memory > is encrypted as well. Note, only for traffic over the SPDM session. In order for private MMIO and T=1 traffic to private memory, coordination with the platform TSM is mandated by all the known TSM (CPU/Platform security modules). This has implications for policy decisions later in this series. > Further use cases for the SPDM library are appearing on the horizon: > Alistair Francis and Wilfred Mallawa from WDC are interested in using > it for SCSI/SATA. David Box from Intel has implemented measurement > retrieval over SPDM. > > The root of trust is initially an in-kernel key ring of certificates. > We can discuss linking the system key ring into it, thereby allowing > EFI to pass trusted certificates to the kernel for CMA. Alternatively, > a bundle of trusted certificates could be loaded from the initrd. > I envision that we'll add TPMs or remote attestation services such as > https://keylime.dev/ to create an ecosystem of various trust sources. Linux also has an interest in accommodating opt-in to using platform managed keys, so the design requires that key management and session ownership is a system owner policy choice. > If you wish to play with PCI device authentication but lack capable > hardware, Wilfred has written a guide how to test with qemu: > https://github.com/twilfredo/spdm-emulation-guide-b >
On Fri, Oct 06, 2023 at 09:06:13AM -0700, Dan Williams wrote: > Lukas Wunner wrote: > > The root of trust is initially an in-kernel key ring of certificates. > > We can discuss linking the system key ring into it, thereby allowing > > EFI to pass trusted certificates to the kernel for CMA. Alternatively, > > a bundle of trusted certificates could be loaded from the initrd. > > I envision that we'll add TPMs or remote attestation services such as > > https://keylime.dev/ to create an ecosystem of various trust sources. > > Linux also has an interest in accommodating opt-in to using platform > managed keys, so the design requires that key management and session > ownership is a system owner policy choice. You're pointing out a gap in the specification: There's an existing mechanism to negotiate which PCI features are handled natively by the OS and which by platform firmware and that's the _OSC Control Field (PCI Firmware Spec r3.3 table 4-5 and 4-6). There are currently 10 features whose ownership is negotiated with _OSC, examples are Hotplug control and DPC configuration control. I propose adding an 11th bit to negotiate ownership of the CMA-SPDM session. Once that's added to the PCI Firmware Spec, amending the implementation to honor it is trivial: Just check for platform ownership at the top of pci_cma_init() and return. Thanks, Lukas
On Sat, 7 Oct 2023 12:04:33 +0200 Lukas Wunner <lukas@wunner.de> wrote: > On Fri, Oct 06, 2023 at 09:06:13AM -0700, Dan Williams wrote: > > Lukas Wunner wrote: > > > The root of trust is initially an in-kernel key ring of certificates. > > > We can discuss linking the system key ring into it, thereby allowing > > > EFI to pass trusted certificates to the kernel for CMA. Alternatively, > > > a bundle of trusted certificates could be loaded from the initrd. > > > I envision that we'll add TPMs or remote attestation services such as > > > https://keylime.dev/ to create an ecosystem of various trust sources. > > > > Linux also has an interest in accommodating opt-in to using platform > > managed keys, so the design requires that key management and session > > ownership is a system owner policy choice. > > You're pointing out a gap in the specification: > > There's an existing mechanism to negotiate which PCI features are > handled natively by the OS and which by platform firmware and that's > the _OSC Control Field (PCI Firmware Spec r3.3 table 4-5 and 4-6). > > There are currently 10 features whose ownership is negotiated with _OSC, > examples are Hotplug control and DPC configuration control. > > I propose adding an 11th bit to negotiate ownership of the CMA-SPDM > session. > > Once that's added to the PCI Firmware Spec, amending the implementation > to honor it is trivial: Just check for platform ownership at the top > of pci_cma_init() and return. This might want to be a control over the specific DOE instance instead of a general purpose CMA control (or maybe we want both). There is no safe way to access a DOE to find out if it supports CMA that doesn't potentially break another entity using the mailbox. Given the DOE instances might be for something entirely different we can't just decide not to use them at all based on a global control. Any such control becomes messy when hotplug is taken into account. I suppose we could do a _DSM based on BDF / path to device (to remain stable across reenumeration) and config space offset to allow the OS to say 'Hi other entity / firmware are you using this DOE instance?" Kind of an OSC with parameters. Also includes the other way around that the question tells the firmware that if it says "no you can't" the OS will leave it alone until a reboot or similar - that potentially avoids the problem that we access DOE instances already without taking care about this (I dropped ball on this having raised it way back near start of us adding DOE support.) If we do want to do any of these, which spec is appropriate? Link it to PCI and propose a PCI firmware spec update? (not sure they have a code first process available) or make it somewhat generic and propose an ACPI Code first change? Jonathan > > Thanks, > > Lukas > >
On Mon, Oct 09, 2023 at 12:33:35PM +0100, Jonathan Cameron wrote: > On Sat, 7 Oct 2023 12:04:33 +0200 Lukas Wunner <lukas@wunner.de> wrote: > > On Fri, Oct 06, 2023 at 09:06:13AM -0700, Dan Williams wrote: > > > Linux also has an interest in accommodating opt-in to using platform > > > managed keys, so the design requires that key management and session > > > ownership is a system owner policy choice. > > > > You're pointing out a gap in the specification: > > > > There's an existing mechanism to negotiate which PCI features are > > handled natively by the OS and which by platform firmware and that's > > the _OSC Control Field (PCI Firmware Spec r3.3 table 4-5 and 4-6). > > > > There are currently 10 features whose ownership is negotiated with _OSC, > > examples are Hotplug control and DPC configuration control. > > > > I propose adding an 11th bit to negotiate ownership of the CMA-SPDM > > session. > > > > Once that's added to the PCI Firmware Spec, amending the implementation > > to honor it is trivial: Just check for platform ownership at the top > > of pci_cma_init() and return. > > This might want to be a control over the specific DOE instance instead > of a general purpose CMA control (or maybe we want both). > > There is no safe way to access a DOE to find out if it supports CMA > that doesn't potentially break another entity using the mailbox. > Given the DOE instances might be for something entirely different we > can't just decide not to use them at all based on a global control. Per PCIe r6.1 sec 6.31.3, the DOE instance used for CMA-SPDM must support "no other data object protocol(s)" besides DOE discovery, CMA-SPDM and Secured CMA-SPDM. So if the platform doesn't grant the OS control over that DOE instance, unrelated DOE instances and protocols (such as CDAT retrieval) are not affected. E.g. PCI Firmware Spec r3.3 table 4-5 could be amended with something along the lines of: Control Field Bit Offset: 11 Interpretation: PCI Express Component Measurement and Authentication control The operating system sets this bit to 1 to request control over the DOE instance supporting the CMA-SPDM feature. You're right that to discover the DOE instance for CMA-SPDM in the first place, it needs to be accessed, which might interfere with the firmware using it. Perhaps this can be solved with the DOE Busy bit. > Any such control becomes messy when hotplug is taken into account. > I suppose we could do a _DSM based on BDF / path to device (to remain > stable across reenumeration) and config space offset to allow the OS > to say 'Hi other entity / firmware are you using this DOE instance?" > Kind of an OSC with parameters. Also includes the other way around that > the question tells the firmware that if it says "no you can't" the OS > will leave it alone until a reboot or similar - that potentially avoids > the problem that we access DOE instances already without taking care > about this PCI Firmware Spec r3.3 table 4-7 lists a number of _DSM Definitions for PCI. Indeed that could be another solution. E.g. a newly defined _DSM might return the offset in config space of DOE instance(s) which the OS is not permitted to use. > (I dropped ball on this having raised it way back near start > of us adding DOE support.) Not your fault. I think the industry got a bit ahead of itself in its "confidential computing" frenzy and forgot to specify these very basic things. > If we do want to do any of these, which spec is appropriate? Link it to PCI > and propose a PCI firmware spec update? (not sure they have a code > first process available) or make it somewhat generic and propose an > ACPI Code first change? PCI Firmware Spec would seem to be appropriate. However this can't be solved by the kernel community. We need to talk to our confidential computing architects and our representatives at the PCISIG to get the spec amended. Thanks, Lukas
On 10/10/23 00:49, Lukas Wunner wrote: > On Mon, Oct 09, 2023 at 12:33:35PM +0100, Jonathan Cameron wrote: >> On Sat, 7 Oct 2023 12:04:33 +0200 Lukas Wunner <lukas@wunner.de> wrote: >>> On Fri, Oct 06, 2023 at 09:06:13AM -0700, Dan Williams wrote: >>>> Linux also has an interest in accommodating opt-in to using platform >>>> managed keys, so the design requires that key management and session >>>> ownership is a system owner policy choice. >>> >>> You're pointing out a gap in the specification: >>> >>> There's an existing mechanism to negotiate which PCI features are >>> handled natively by the OS and which by platform firmware and that's >>> the _OSC Control Field (PCI Firmware Spec r3.3 table 4-5 and 4-6). >>> >>> There are currently 10 features whose ownership is negotiated with _OSC, >>> examples are Hotplug control and DPC configuration control. >>> >>> I propose adding an 11th bit to negotiate ownership of the CMA-SPDM >>> session. >>> >>> Once that's added to the PCI Firmware Spec, amending the implementation >>> to honor it is trivial: Just check for platform ownership at the top >>> of pci_cma_init() and return. >> >> This might want to be a control over the specific DOE instance instead >> of a general purpose CMA control (or maybe we want both). >> >> There is no safe way to access a DOE to find out if it supports CMA >> that doesn't potentially break another entity using the mailbox. >> Given the DOE instances might be for something entirely different we >> can't just decide not to use them at all based on a global control. > > Per PCIe r6.1 sec 6.31.3, the DOE instance used for CMA-SPDM must support > "no other data object protocol(s)" besides DOE discovery, CMA-SPDM and > Secured CMA-SPDM. > > So if the platform doesn't grant the OS control over that DOE instance, > unrelated DOE instances and protocols (such as CDAT retrieval) are not > affected. > > E.g. PCI Firmware Spec r3.3 table 4-5 could be amended with something > along the lines of: > > Control Field Bit Offset: 11 > > Interpretation: PCI Express Component Measurement and Authentication control > > The operating system sets this bit to 1 to request control over the > DOE instance supporting the CMA-SPDM feature. > > You're right that to discover the DOE instance for CMA-SPDM in the > first place, it needs to be accessed, which might interfere with the > firmware using it. Perhaps this can be solved with the DOE Busy bit. > > >> Any such control becomes messy when hotplug is taken into account. >> I suppose we could do a _DSM based on BDF / path to device (to remain >> stable across reenumeration) and config space offset to allow the OS >> to say 'Hi other entity / firmware are you using this DOE instance?" >> Kind of an OSC with parameters. Also includes the other way around that >> the question tells the firmware that if it says "no you can't" the OS >> will leave it alone until a reboot or similar - that potentially avoids >> the problem that we access DOE instances already without taking care >> about this > > PCI Firmware Spec r3.3 table 4-7 lists a number of _DSM Definitions for > PCI. Indeed that could be another solution. E.g. a newly defined _DSM > might return the offset in config space of DOE instance(s) which the OS > is not permitted to use. > > >> (I dropped ball on this having raised it way back near start >> of us adding DOE support.) > > Not your fault. I think the industry got a bit ahead of itself in > its "confidential computing" frenzy and forgot to specify these very > basic things. > > >> If we do want to do any of these, which spec is appropriate? Link it to PCI >> and propose a PCI firmware spec update? (not sure they have a code >> first process available) or make it somewhat generic and propose an >> ACPI Code first change? > > PCI Firmware Spec would seem to be appropriate. However this can't > be solved by the kernel community. How so? It is up to the user to decide whether it is SPDM/CMA in the kernel or the firmware + coco, both are quite possible (it is IDE which is not possible without the firmware on AMD but we are not there yet). But the way SPDM is done now is that if the user (as myself) wants to let the firmware run SPDM - the only choice is disabling CONFIG_CMA completely as CMA is not a (un)loadable module or built-in (with some "blacklist" parameters), and does not provide a sysfs knob to control its tentacles. Kinda harsh. Note, this PSP firmware is not BIOS (which runs on the same core and has same access to PCI as the host OS), it is a separate platform processor which only programs IDE keys to the PCI RC (via some some internal bus mechanism) but does not do anything on the bus itself and relies on the host OS proxying DOE, and there is no APCI between the core and the psp. > We need to talk to our confidential > computing architects and our representatives at the PCISIG to get the > spec amended. > > Thanks, > > Lukas
On Tue, Oct 10, 2023 at 03:07:41PM +1100, Alexey Kardashevskiy wrote: > On 10/10/23 00:49, Lukas Wunner wrote: > > PCI Firmware Spec would seem to be appropriate. However this can't > > be solved by the kernel community. > > How so? It is up to the user to decide whether it is SPDM/CMA in the kernel > or the firmware + coco, both are quite possible (it is IDE which is not > possible without the firmware on AMD but we are not there yet). The user can control ownership of CMA-SPDM e.g. through a BIOS knob. And that BIOS knob then influences the outcome of the _OSC negotiation between platform and OS. > But the way SPDM is done now is that if the user (as myself) wants to let > the firmware run SPDM - the only choice is disabling CONFIG_CMA completely > as CMA is not a (un)loadable module or built-in (with some "blacklist" > parameters), and does not provide a sysfs knob to control its tentacles. The problem is every single vendor thinks they can come up with their own idea of who owns the SPDM session: I've looked at the Nvidia driver and they've hacked libspdm into it, so their idea is that the device driver owns the SPDM session. AMD wants the host to proxy DOE but not own the SPDM session. We have *standards* for a reason. So that products are interoperable. If the kernel tries to accommodate to every vendor's idea of SPDM ownership we'll end up with an unmaintainable mess of quirks, plus sysfs knobs which were once intended as a stopgap but can never be removed because they're userspace ABI. This needs to be solved in the *specification*. And the existing solution for who owns a particular PCI feature is _OSC. Hence this needs to be taken up with the Firmware Working Group at the PCISIG. > Note, this PSP firmware is not BIOS (which runs on the same core and has > same access to PCI as the host OS), it is a separate platform processor > which only programs IDE keys to the PCI RC (via some some internal bus > mechanism) but does not do anything on the bus itself and relies on the host > OS proxying DOE, and there is no APCI between the core and the psp. Somewhat tangentially, would it be possible in your architecture that the host or guest asks PSP to program IDE keys into the Root Port? Or alternatively, access the key registers directly without PSP involvement? Thanks, Lukas
On 10/10/23 19:19, Lukas Wunner wrote: > On Tue, Oct 10, 2023 at 03:07:41PM +1100, Alexey Kardashevskiy wrote: >> On 10/10/23 00:49, Lukas Wunner wrote: >>> PCI Firmware Spec would seem to be appropriate. However this can't >>> be solved by the kernel community. >> >> How so? It is up to the user to decide whether it is SPDM/CMA in the kernel >> or the firmware + coco, both are quite possible (it is IDE which is not >> possible without the firmware on AMD but we are not there yet). > > The user can control ownership of CMA-SPDM e.g. through a BIOS knob. > And that BIOS knob then influences the outcome of the _OSC negotiation > between platform and OS. > > >> But the way SPDM is done now is that if the user (as myself) wants to let >> the firmware run SPDM - the only choice is disabling CONFIG_CMA completely >> as CMA is not a (un)loadable module or built-in (with some "blacklist" >> parameters), and does not provide a sysfs knob to control its tentacles. > > The problem is every single vendor thinks they can come up with > their own idea of who owns the SPDM session: > > I've looked at the Nvidia driver and they've hacked libspdm into it, > so their idea is that the device driver owns the SPDM session. > > AMD wants the host to proxy DOE but not own the SPDM session. > > We have *standards* for a reason. So that products are interoperable. There is no "standard PCI ethernet device", somehow we survive ;) > If the kernel tries to accommodate to every vendor's idea of SPDM ownership > we'll end up with an unmaintainable mess of quirks, plus sysfs knobs > which were once intended as a stopgap but can never be removed because > they're userspace ABI. The host kernel needs to accommodate the idea that it is not trusted, and neither is the BIOS. > This needs to be solved in the *specification*. > > And the existing solution for who owns a particular PCI feature is _OSC. > Hence this needs to be taken up with the Firmware Working Group at the > PCISIG. I do like the general idea of specifying things, etc but this place does not sound right. The firmware you are talking about has full access to PCI, the PSP firmware does not have any (besides the IDE keys programming), is there any example of such firmware in the PCI Firmware spec? From the BIOS standpoint, the host OS owns DOE and whatever is sent over it (on AMD SEV TIO). The host OS chooses not to compose these SPDM packets itself (while it could) in order to be able to run guests without having them to trust the host OS. >> Note, this PSP firmware is not BIOS (which runs on the same core and has >> same access to PCI as the host OS), it is a separate platform processor >> which only programs IDE keys to the PCI RC (via some some internal bus >> mechanism) but does not do anything on the bus itself and relies on the host >> OS proxying DOE, and there is no APCI between the core and the psp. > > Somewhat tangentially, would it be possible in your architecture > that the host or guest asks PSP to program IDE keys into the Root Port? Sure it is possible to implement. But this does not help our primary use case which is confidential VMs where the host OS is not trusted with the keys. > Or alternatively, access the key registers directly without PSP involvement? No afaik, for the reason above. > > Thanks, > > Lukas
On Tue, 10 Oct 2023 23:53:16 +1100 Alexey Kardashevskiy <aik@amd.com> wrote: > On 10/10/23 19:19, Lukas Wunner wrote: > > On Tue, Oct 10, 2023 at 03:07:41PM +1100, Alexey Kardashevskiy wrote: > >> On 10/10/23 00:49, Lukas Wunner wrote: > >>> PCI Firmware Spec would seem to be appropriate. However this can't > >>> be solved by the kernel community. > >> > >> How so? It is up to the user to decide whether it is SPDM/CMA in the kernel > >> or the firmware + coco, both are quite possible (it is IDE which is not > >> possible without the firmware on AMD but we are not there yet). > > > > The user can control ownership of CMA-SPDM e.g. through a BIOS knob. > > And that BIOS knob then influences the outcome of the _OSC negotiation > > between platform and OS. > > > > > >> But the way SPDM is done now is that if the user (as myself) wants to let > >> the firmware run SPDM - the only choice is disabling CONFIG_CMA completely > >> as CMA is not a (un)loadable module or built-in (with some "blacklist" > >> parameters), and does not provide a sysfs knob to control its tentacles. > > > > The problem is every single vendor thinks they can come up with > > their own idea of who owns the SPDM session: > > > > I've looked at the Nvidia driver and they've hacked libspdm into it, > > so their idea is that the device driver owns the SPDM session. > > > > AMD wants the host to proxy DOE but not own the SPDM session. > > > > We have *standards* for a reason. So that products are interoperable. > > There is no "standard PCI ethernet device", somehow we survive ;) > > > If the kernel tries to accommodate to every vendor's idea of SPDM ownership > > we'll end up with an unmaintainable mess of quirks, plus sysfs knobs > > which were once intended as a stopgap but can never be removed because > > they're userspace ABI. > > The host kernel needs to accommodate the idea that it is not trusted, > and neither is the BIOS. > > > This needs to be solved in the *specification*. > > > > And the existing solution for who owns a particular PCI feature is _OSC. > > Hence this needs to be taken up with the Firmware Working Group at the > > PCISIG. > > I do like the general idea of specifying things, etc but this place does > not sound right. The firmware you are talking about has full access to > PCI, the PSP firmware does not have any (besides the IDE keys > programming), is there any example of such firmware in the PCI Firmware > spec? From the BIOS standpoint, the host OS owns DOE and whatever is > sent over it (on AMD SEV TIO). The host OS chooses not to compose these > SPDM packets itself (while it could) in order to be able to run guests > without having them to trust the host OS. As a minimum I'd like to see something saying - "keep away from discovery protocol on this DOE instance". An ACPI _OSC or _DSM or similar could do that. It won't be needed for every approach, but it might for some. Then either firmwware knows what to do, or a specific driver does. If your proxy comes up late enough that we've already done (and cached) discovery protocols results then this might not be a problem for this particular approach as we have no reason to rerun discovery (other than hotplug in which case there is lots of other stuff to do anyway). For your case we need some hooks for the PSP to be able to drive the SPDM session but that should be easy to allow. I don't think precludes the hypervisor also verifying the hardware is trusted by it along the way (though not used for IDE). So if you are relying on a host OS proxy I don't thing you need to disable CONFIG_CMA (maybe something around resets?) Potentially the host OS tries first (maybe succeeds - that doesn't matter though nothing wrong with defense in depth) and then the PSP via a proxy does it all over again which is fine. All we need to do is guarantee ordering and I think we are fine for that. Far too many possible models here but such is life I guess. > > >> Note, this PSP firmware is not BIOS (which runs on the same core and has > >> same access to PCI as the host OS), it is a separate platform processor > >> which only programs IDE keys to the PCI RC (via some some internal bus > >> mechanism) but does not do anything on the bus itself and relies on the host > >> OS proxying DOE, and there is no APCI between the core and the psp. > > > > Somewhat tangentially, would it be possible in your architecture > > that the host or guest asks PSP to program IDE keys into the Root Port? > > Sure it is possible to implement. But this does not help our primary use > case which is confidential VMs where the host OS is not trusted with the > keys. > > > Or alternatively, access the key registers directly without PSP involvement? > > No afaik, for the reason above. > > > > > > Thanks, > > > > Lukas >
On 12/10/23 20:15, Lukas Wunner wrote: > On Tue, Oct 10, 2023 at 03:07:41PM +1100, Alexey Kardashevskiy wrote: >> But the way SPDM is done now is that if the user (as myself) wants to let >> the firmware run SPDM - the only choice is disabling CONFIG_CMA completely >> as CMA is not a (un)loadable module or built-in (with some "blacklist" >> parameters), and does not provide a sysfs knob to control its tentacles. >> Kinda harsh. > > On AMD SEV-TIO, does the PSP perform SPDM exchanges with a device > *before* it is passed through to a guest? If so, why does it do that? Yes, to set up IDE. SEV TIO is designed in a way that there is one stream == set of keys per the PF's traffic class. It is like this - imagine a TDISP+SRIOV device with hundreds VFs passed through to hundreds VMs. The host still owns the PF, provides DOE for the PSP, the PSP owns a handful of keys (one will do really, I have not fully grasped the idea when one would want traffic classes but ok, up to 8), and hundreds VFs work using this few (or one) keys, and the PF works as well, just cannot know the IDE key (==cannot spy on VFs via something like PCI bridge/retimer or logic analyzer). It is different than what you are doing, DOE is the only common thing so far (or ever?). btw the PSP is not able to initiate SPDM traffic by itself, when the host decides it wants to setup IDE (via a PSP in SEV TIO), it talks to the PSP which can return "I want to talk to the device, here are req/resp buffers", in a loop, until the PSP returns something else. > Dan and I discussed this off-list and Dan is arguing for lazy attestation, > i.e. the TSM should only have the need to perform SPDM exchanges with > the device when it is passed through. Well, I'd expect that in most cases VF is going to be passed through and IDE setup is done via PF which won't be passed through in such cases as it has to manage VFs. > So the host enumerates the DOE protocols Yes. > and authenticates the device. No objection here. But PSP will need to rerun this, but still via the host's DOE. > When the device is passed through, patch 12/12 ensures that the host > keeps its hands off of the device, thus affording the TSM exclusive > SPDM control. If a PF is passed through - I guess yes we could use that, but how is this going to work for a VF? > I agree that the commit message of 12/12 is utterly misleading in that > it says "the guest" is granted exclusive control. It should say "the TSM" > instead. (There might be implementations where the guest itself has > the role of the TSM and authenticates the device on its own behalf, > but PCIe r6.1 sec 11 uses the term "TSM" so that's what the commit > message needs to use.) This should work as long as DOE is still available (as of today). > However apart from the necessary rewrite of the commit message and > perhaps a rename of the PCI_CMA_OWNED_BY_GUEST flag, I think patch 12/12 > should already be doing exactly what you need -- provided that the > PSP doesn't perform SPDM exchanges before passthrough. If it already > performs them, say, on boot, I'd like to understand the reason. In out design this does not have to happen on the host's boot. But I wonder if some PF host driver authenticated some device and then we create a bunch of VFs and pass the SPDM ownership of the PF to the PSP to reauthentificate it again - the already running PF host driver may become upset, may it? 12/12 assumes the host driver is VFIO-PCI but it won't be, VFs will be bound to VFIO-PCI. Hope this all makes sense. Thanks, > > Thanks, > > Lukas
On Thu, 12 Oct 2023 22:18:27 +1100 Alexey Kardashevskiy <aik@amd.com> wrote: > On 12/10/23 20:15, Lukas Wunner wrote: > > On Tue, Oct 10, 2023 at 03:07:41PM +1100, Alexey Kardashevskiy wrote: > >> But the way SPDM is done now is that if the user (as myself) wants to let > >> the firmware run SPDM - the only choice is disabling CONFIG_CMA completely > >> as CMA is not a (un)loadable module or built-in (with some "blacklist" > >> parameters), and does not provide a sysfs knob to control its tentacles. > >> Kinda harsh. > > > > On AMD SEV-TIO, does the PSP perform SPDM exchanges with a device > > *before* it is passed through to a guest? If so, why does it do that? > > Yes, to set up IDE. SEV TIO is designed in a way that there is one > stream == set of keys per the PF's traffic class. > > It is like this - imagine a TDISP+SRIOV device with hundreds VFs passed > through to hundreds VMs. The host still owns the PF, provides DOE for > the PSP, the PSP owns a handful of keys (one will do really, I have not > fully grasped the idea when one would want traffic classes but ok, up to > 8), and hundreds VFs work using this few (or one) keys, and the PF works > as well, just cannot know the IDE key (==cannot spy on VFs via something > like PCI bridge/retimer or logic analyzer). It is different than what > you are doing, DOE is the only common thing so far (or ever?). > > btw the PSP is not able to initiate SPDM traffic by itself, when the > host decides it wants to setup IDE (via a PSP in SEV TIO), it talks to > the PSP which can return "I want to talk to the device, here are > req/resp buffers", in a loop, until the PSP returns something else. > > > Dan and I discussed this off-list and Dan is arguing for lazy attestation, > > i.e. the TSM should only have the need to perform SPDM exchanges with > > the device when it is passed through. > > Well, I'd expect that in most cases VF is going to be passed through and > IDE setup is done via PF which won't be passed through in such cases as > it has to manage VFs. > > > So the host enumerates the DOE protocols > > Yes. > > > and authenticates the device. > > No objection here. But PSP will need to rerun this, but still via the > host's DOE. > > > When the device is passed through, patch 12/12 ensures that the host > > keeps its hands off of the device, thus affording the TSM exclusive > > SPDM control. > > If a PF is passed through - I guess yes we could use that, but how is > this going to work for a VF? > > > I agree that the commit message of 12/12 is utterly misleading in that > > it says "the guest" is granted exclusive control. It should say "the TSM" > > instead. (There might be implementations where the guest itself has > > the role of the TSM and authenticates the device on its own behalf, > > but PCIe r6.1 sec 11 uses the term "TSM" so that's what the commit > > message needs to use.) > > This should work as long as DOE is still available (as of today). > > > However apart from the necessary rewrite of the commit message and > > perhaps a rename of the PCI_CMA_OWNED_BY_GUEST flag, I think patch 12/12 > > should already be doing exactly what you need -- provided that the > > PSP doesn't perform SPDM exchanges before passthrough. If it already > > performs them, say, on boot, I'd like to understand the reason. > > In out design this does not have to happen on the host's boot. But I > wonder if some PF host driver authenticated some device and then we > create a bunch of VFs and pass the SPDM ownership of the PF to the PSP > to reauthentificate it again - the already running PF host driver may > become upset, may it? 12/12 assumes the host driver is VFIO-PCI but it > won't be, VFs will be bound to VFIO-PCI. Hope this all makes sense. Thanks, Without some experiments with real drivers, will be hard to be sure, but I'd expect it to be fine as the host driver bound after attestation (or what's the point?) In this patch set attestation only happens again on a reset or kicking it because of new certs. For reset, your PSP should be doing it all over again anyway so that can happen after the host driver has dealt with the reset. For the manual poking to retry attestation, if the model is we don't load the driver until the attestation succeeds then that should be fine (as driver not loaded). The lock out needed for PF pass through doesn't apply given we are poking it from the PSP via the host. So I think patch 12 is irrelevant to your usecase rather than a problem. May well be dragons in the corner cases. If we need a lockout for after the PSP gets involved, then fair enough. Jonathan > > > > > > Thanks, > > > > Lukas >
On Thu, 12 Oct 2023 15:13:31 +0200 Samuel Ortiz <sameo@rivosinc.com> wrote: > On Thu, Oct 12, 2023 at 11:15:42AM +0200, Lukas Wunner wrote: > > On Tue, Oct 10, 2023 at 03:07:41PM +1100, Alexey Kardashevskiy wrote: > > > But the way SPDM is done now is that if the user (as myself) wants to let > > > the firmware run SPDM - the only choice is disabling CONFIG_CMA completely > > > as CMA is not a (un)loadable module or built-in (with some "blacklist" > > > parameters), and does not provide a sysfs knob to control its tentacles. > > > Kinda harsh. > > > > On AMD SEV-TIO, does the PSP perform SPDM exchanges with a device > > *before* it is passed through to a guest? If so, why does it do that? > > SPDM exchanges would be done with the DSM, i.e. through the PF, which is > typically *not* passed through to guests. VFs are. > > The RISC-V CoVE-IO [1] spec follows similar flows as SEV-TIO (and to > some extend TDX-Connect) and expects the host to explicitly request the > TSM to establish an SPDM connection with the DSM (PF) before passing one > VF through a TSM managed guest. VFs would be vfio bound, not the PF, so > I think patch #12 does not solve our problem here. > > > Dan and I discussed this off-list and Dan is arguing for lazy attestation, > > i.e. the TSM should only have the need to perform SPDM exchanges with > > the device when it is passed through. > > > > So the host enumerates the DOE protocols and authenticates the device. > > When the device is passed through, patch 12/12 ensures that the host > > keeps its hands off of the device, thus affording the TSM exclusive > > SPDM control. > > Just to re-iterate: The TSM does not talk SPDM with the passed > through device(s), but with the corresponding PF. If the host kernel > owns the SPDM connection when the TSM initiates the SPDM connection with > the DSM (For IDE key setup), the connection establishment will fail. > Both CoVE-IO and SEV-TIO (Alexey, please correct me if I'm wrong) > expect the host to explicitly ask the TSM to establish that SPDM > connection. That request should somehow come from KVM, which then would > have to destroy the existing CMA/SPDM connection in order to give the > TSM a chance to successfully establish the SPDM link. Agreed - I don't see a problem with throwing away the initial connection. In these cases you are passing that role on to another entity - the job of this patch set is done. I'm not clear yet if we need an explicit lock out similar to the VFIO one for PF pass through or if everything will happen in a 'safe' order anyway. I suspect a lockout on the ability to re attest is necessary if the PF driver is loaded. Perhaps just dropping the +#if IS_ENABLED(CONFIG_VFIO_PCI_CORE) and letting other PF drivers or another bit of core kernel code (I'm not sure where the proxy resides for the models being discussed) claim ownership is enough? Jonathan > > Cheers, > Samuel. > > [1] https://github.com/riscv-non-isa/riscv-ap-tee-io/blob/main/specification/07-theory_operations.adoc > > >
On Thu, Oct 12, 2023 at 04:32:21PM +0100, Jonathan Cameron wrote: > On Thu, 12 Oct 2023 15:13:31 +0200 > Samuel Ortiz <sameo@rivosinc.com> wrote: > > > On Thu, Oct 12, 2023 at 11:15:42AM +0200, Lukas Wunner wrote: > > > On Tue, Oct 10, 2023 at 03:07:41PM +1100, Alexey Kardashevskiy wrote: > > > > But the way SPDM is done now is that if the user (as myself) wants to let > > > > the firmware run SPDM - the only choice is disabling CONFIG_CMA completely > > > > as CMA is not a (un)loadable module or built-in (with some "blacklist" > > > > parameters), and does not provide a sysfs knob to control its tentacles. > > > > Kinda harsh. > > > > > > On AMD SEV-TIO, does the PSP perform SPDM exchanges with a device > > > *before* it is passed through to a guest? If so, why does it do that? > > > > SPDM exchanges would be done with the DSM, i.e. through the PF, which is > > typically *not* passed through to guests. VFs are. > > > > The RISC-V CoVE-IO [1] spec follows similar flows as SEV-TIO (and to > > some extend TDX-Connect) and expects the host to explicitly request the > > TSM to establish an SPDM connection with the DSM (PF) before passing one > > VF through a TSM managed guest. VFs would be vfio bound, not the PF, so > > I think patch #12 does not solve our problem here. > > > > > Dan and I discussed this off-list and Dan is arguing for lazy attestation, > > > i.e. the TSM should only have the need to perform SPDM exchanges with > > > the device when it is passed through. > > > > > > So the host enumerates the DOE protocols and authenticates the device. > > > When the device is passed through, patch 12/12 ensures that the host > > > keeps its hands off of the device, thus affording the TSM exclusive > > > SPDM control. > > > > Just to re-iterate: The TSM does not talk SPDM with the passed > > through device(s), but with the corresponding PF. If the host kernel > > owns the SPDM connection when the TSM initiates the SPDM connection with > > the DSM (For IDE key setup), the connection establishment will fail. > > Both CoVE-IO and SEV-TIO (Alexey, please correct me if I'm wrong) > > expect the host to explicitly ask the TSM to establish that SPDM > > connection. That request should somehow come from KVM, which then would > > have to destroy the existing CMA/SPDM connection in order to give the > > TSM a chance to successfully establish the SPDM link. > > Agreed - I don't see a problem with throwing away the initial connection. > In these cases you are passing that role on to another entity - the > job of this patch set is done. Right. As long as there's a way for the kernel to explicitly drop that ownership before calling into the TSM for asking it to create a new SPDM connection, we should be fine. Alexey, would you agree with that statement? > I'm not clear yet if we need an explicit lock out similar to the VFIO > one for PF pass through or if everything will happen in a 'safe' order > anyway. I suspect a lockout on the ability to re attest is necessary > if the PF driver is loaded. > > Perhaps just dropping the > +#if IS_ENABLED(CONFIG_VFIO_PCI_CORE) > and letting other PF drivers or another bit of core kernel code > (I'm not sure where the proxy resides for the models being discussed) > claim ownership is enough? If we agree that other parts of the kernel (I suspect KVM would do the "Connect to device" call to the TSM) should be able to tear the established SPDM connection, then yes, the claim/return_ownership() API should not be only available to VFIO. Cheers, Samuel.
On 13/10/23 16:03, Samuel Ortiz wrote: > On Thu, Oct 12, 2023 at 04:32:21PM +0100, Jonathan Cameron wrote: >> On Thu, 12 Oct 2023 15:13:31 +0200 >> Samuel Ortiz <sameo@rivosinc.com> wrote: >> >>> On Thu, Oct 12, 2023 at 11:15:42AM +0200, Lukas Wunner wrote: >>>> On Tue, Oct 10, 2023 at 03:07:41PM +1100, Alexey Kardashevskiy wrote: >>>>> But the way SPDM is done now is that if the user (as myself) wants to let >>>>> the firmware run SPDM - the only choice is disabling CONFIG_CMA completely >>>>> as CMA is not a (un)loadable module or built-in (with some "blacklist" >>>>> parameters), and does not provide a sysfs knob to control its tentacles. >>>>> Kinda harsh. >>>> >>>> On AMD SEV-TIO, does the PSP perform SPDM exchanges with a device >>>> *before* it is passed through to a guest? If so, why does it do that? >>> >>> SPDM exchanges would be done with the DSM, i.e. through the PF, which is >>> typically *not* passed through to guests. VFs are. >>> >>> The RISC-V CoVE-IO [1] spec follows similar flows as SEV-TIO (and to >>> some extend TDX-Connect) and expects the host to explicitly request the >>> TSM to establish an SPDM connection with the DSM (PF) before passing one >>> VF through a TSM managed guest. VFs would be vfio bound, not the PF, so >>> I think patch #12 does not solve our problem here. >>> >>>> Dan and I discussed this off-list and Dan is arguing for lazy attestation, >>>> i.e. the TSM should only have the need to perform SPDM exchanges with >>>> the device when it is passed through. >>>> >>>> So the host enumerates the DOE protocols and authenticates the device. >>>> When the device is passed through, patch 12/12 ensures that the host >>>> keeps its hands off of the device, thus affording the TSM exclusive >>>> SPDM control. >>> >>> Just to re-iterate: The TSM does not talk SPDM with the passed >>> through device(s), but with the corresponding PF. If the host kernel >>> owns the SPDM connection when the TSM initiates the SPDM connection with >>> the DSM (For IDE key setup), the connection establishment will fail. >>> Both CoVE-IO and SEV-TIO (Alexey, please correct me if I'm wrong) >>> expect the host to explicitly ask the TSM to establish that SPDM >>> connection. That request should somehow come from KVM, which then would >>> have to destroy the existing CMA/SPDM connection in order to give the >>> TSM a chance to successfully establish the SPDM link. >> >> Agreed - I don't see a problem with throwing away the initial connection. >> In these cases you are passing that role on to another entity - the >> job of this patch set is done. > > Right. As long as there's a way for the kernel to explicitly drop that > ownership before calling into the TSM for asking it to create a new SPDM > connection, we should be fine. Alexey, would you agree with that > statement? Yes, sounds right. >> I'm not clear yet if we need an explicit lock out similar to the VFIO >> one for PF pass through or if everything will happen in a 'safe' order >> anyway. I suspect a lockout on the ability to re attest is necessary >> if the PF driver is loaded. >> >> Perhaps just dropping the >> +#if IS_ENABLED(CONFIG_VFIO_PCI_CORE) >> and letting other PF drivers or another bit of core kernel code >> (I'm not sure where the proxy resides for the models being discussed) >> claim ownership is enough? > > If we agree that other parts of the kernel (I suspect KVM would do the > "Connect to device" call to the TSM) should be able to tear the > established SPDM connection, then yes, the claim/return_ownership() API > should not be only available to VFIO. Correct. I just want to make sure that DOE mailboxes stay alive and nothing in the host kernel relies on SPDM being still active after ownership is transferred to the TSM==PSP. > > Cheers, > Samuel.