From patchwork Mon Dec 9 04:55:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 850007 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 00485194C96; Mon, 9 Dec 2024 04:56:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733720220; cv=none; b=QSE+QRK4NenxEOasqnnfrmnyXYjwxr0cA3Ch5HJhuDG+sNzgE0CFQdVC8AatR1uh1sLpCNMbpqmW5nevxl7Xgv58HyqSV48WX/VMBO37Z65EQm5PBER+NiSHhLrlAghgYdw5LSmMN+k8YqXQ3FEOmk7DQQ5dyvGK1WwPNh0nEUs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733720220; c=relaxed/simple; bh=gQxL93tDqeQ81DXvyKU5Qq7RuWyMVm0Ywpf4ghREIMw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lEixG1EQDWNzHENoLWD+AIPmsBCMbN2miLOB6uRjOE0gEncb20E/F6lPEMzr7OP/aDSF9Z5h3BeCRkqRYhRH4dc1CPsVQp3rVbz5hM0eUVcaJSMd5ixb8sEm0S7WSZ4zzaXhO8ytnnLi7bx2Hk8Thkr232RD7CybR6JQ6mKXrPU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qr5kenwq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qr5kenwq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E318DC4CEDF; Mon, 9 Dec 2024 04:56:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1733720219; bh=gQxL93tDqeQ81DXvyKU5Qq7RuWyMVm0Ywpf4ghREIMw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qr5kenwqysiWhiX+f6geZf4nH7oz07Jw+hXafL00bQahJ8Xd2qHe7XGKwX5RYi4Zc zHEtoq29/pQ6s+pWbwbZoSW4Y9J94uRjvGDuW8zXfGi8PJJZERMkvKGGo8THu0izAP IooBmluizx1cRtKVCj5RH9SVe6zQg6PzD9cvvnvotv5/41a5GI9eBhkhJVNPMpMHpQ F4fTt/B3+7AudfMW29w0ymYHoP7HD4RcNDLgY/KiLh7kGTft0JCDOmVrkCsZ61xSAz wDvjShYyKqR6Hsw9KQyk5p/ZMBmZnlAOuYrWOyaMTMSGgdg6MmXwHaeOvmkec/5WQs f+oaAs6omzB3Q== From: Eric Biggers To: linux-block@vger.kernel.org, linux-fscrypt@vger.kernel.org, linux-mmc@vger.kernel.org, linux-scsi@vger.kernel.org, linux-arm-msm@vger.kernel.org, Bartosz Golaszewski , Gaurav Kashyap Cc: Adrian Hunter , Alim Akhtar , Avri Altman , Bart Van Assche , Bjorn Andersson , Dmitry Baryshkov , "James E . J . Bottomley" , Jens Axboe , Konrad Dybcio , Manivannan Sadhasivam , "Martin K . Petersen" , Ulf Hansson Subject: [PATCH v9 01/12] ufs: crypto: add ufs_hba_from_crypto_profile() Date: Sun, 8 Dec 2024 20:55:19 -0800 Message-ID: <20241209045530.507833-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241209045530.507833-1-ebiggers@kernel.org> References: <20241209045530.507833-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Eric Biggers Add a helper function that encapsulates a container_of expression. For now there are two users but soon there will be more. Signed-off-by: Eric Biggers --- drivers/ufs/core/ufshcd-crypto.c | 6 ++---- include/ufs/ufshcd.h | 8 ++++++++ 2 files changed, 10 insertions(+), 4 deletions(-) base-commit: f486c8aa16b8172f63bddc70116a0c897a7f3f02 diff --git a/drivers/ufs/core/ufshcd-crypto.c b/drivers/ufs/core/ufshcd-crypto.c index a714dad82cd1f..0cb425ef618e8 100644 --- a/drivers/ufs/core/ufshcd-crypto.c +++ b/drivers/ufs/core/ufshcd-crypto.c @@ -50,12 +50,11 @@ static int ufshcd_program_key(struct ufs_hba *hba, static int ufshcd_crypto_keyslot_program(struct blk_crypto_profile *profile, const struct blk_crypto_key *key, unsigned int slot) { - struct ufs_hba *hba = - container_of(profile, struct ufs_hba, crypto_profile); + struct ufs_hba *hba = ufs_hba_from_crypto_profile(profile); const union ufs_crypto_cap_entry *ccap_array = hba->crypto_cap_array; const struct ufs_crypto_alg_entry *alg = &ufs_crypto_algs[key->crypto_cfg.crypto_mode]; u8 data_unit_mask = key->crypto_cfg.data_unit_size / 512; int i; @@ -97,12 +96,11 @@ static int ufshcd_crypto_keyslot_program(struct blk_crypto_profile *profile, static int ufshcd_crypto_keyslot_evict(struct blk_crypto_profile *profile, const struct blk_crypto_key *key, unsigned int slot) { - struct ufs_hba *hba = - container_of(profile, struct ufs_hba, crypto_profile); + struct ufs_hba *hba = ufs_hba_from_crypto_profile(profile); /* * Clear the crypto cfg on the device. Clearing CFGE * might not be sufficient, so just clear the entire cfg. */ union ufs_crypto_cfg_entry cfg = {}; diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index d7aca9e61684f..91b4f95d6c8ea 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -1200,10 +1200,18 @@ static inline size_t ufshcd_sg_entry_size(const struct ufs_hba *hba) #define ufshcd_set_sg_entry_size(hba, sg_entry_size) \ ({ (void)(hba); BUILD_BUG_ON(sg_entry_size != sizeof(struct ufshcd_sg_entry)); }) #endif +#ifdef CONFIG_SCSI_UFS_CRYPTO +static inline struct ufs_hba * +ufs_hba_from_crypto_profile(struct blk_crypto_profile *profile) +{ + return container_of(profile, struct ufs_hba, crypto_profile); +} +#endif + static inline size_t ufshcd_get_ucd_size(const struct ufs_hba *hba) { return sizeof(struct utp_transfer_cmd_desc) + SG_ALL * ufshcd_sg_entry_size(hba); } From patchwork Mon Dec 9 04:55:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 850006 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D772198E69; Mon, 9 Dec 2024 04:57:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733720221; cv=none; b=RLkmCdWiroQhUd8y5myL4iUXL4Ih1IyN0pG/C51iCHYIpFCf4pWcKM2aFswkLZC36U4gkhGLnb1U7JvMrB3uYeH9P1mLs4d7vc1atNnoXg77wOPYkYZqJ658WORLs/JPvn+v4/0yvuJjKWdmRGulFTswRnwZGVt13I+DgVjzmhM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733720221; c=relaxed/simple; bh=IIyT5D4ce1fMq74iTKkTD8qR0aHoyIZUofIUVRvWCzM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XS/eJ84oHNzE7BzyCbNRfj1mguD3hCxdonx28Hv7jSA7VSTZ70A1WGlDgqTsekliGneRaQT2n2wjOjBWUDFaRvrpZkbccaahAyHHANonEl0j+M7Zl49RUZDCWt6okkx9SnpWxr/siaQ+RF01eb8qApfwVotZj1h8IZUbtidnNjo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qwSAvq9I; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qwSAvq9I" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 54DEAC4CEE0; Mon, 9 Dec 2024 04:57:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1733720220; bh=IIyT5D4ce1fMq74iTKkTD8qR0aHoyIZUofIUVRvWCzM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qwSAvq9IO/5bBFQi1UZjIl+9AhxK5K2k9pY+DafLiQ2pXws6ShkHYA5Gk+QbsaAaL b3SzOEz4CB+42E96URRk8cg4RucZwNn7o3Yt6YzaJ542f+byD21K6K/Mx26eHTiVUT FABJCyON6/z3B+tYrd8YAv7Ngax2+9iTCGOB8CgIgzUIhCUUkklx7tZg/rHnrLnJx1 hU9w0IOh2u/8WZV2TMKgysZX1pxUPCvtV+Sy/HiPBPNwsZTzTA/5s6Wrla+sc2+BKX GLaXu2d/OiCJfe4crtayGcqkbBNmE7/6GwbV03xxBB3C1cNpeciPZgGe2TEiVVnPPe Wh3qx4S7e3Y1A== From: Eric Biggers To: linux-block@vger.kernel.org, linux-fscrypt@vger.kernel.org, linux-mmc@vger.kernel.org, linux-scsi@vger.kernel.org, linux-arm-msm@vger.kernel.org, Bartosz Golaszewski , Gaurav Kashyap Cc: Adrian Hunter , Alim Akhtar , Avri Altman , Bart Van Assche , Bjorn Andersson , Dmitry Baryshkov , "James E . J . Bottomley" , Jens Axboe , Konrad Dybcio , Manivannan Sadhasivam , "Martin K . Petersen" , Ulf Hansson Subject: [PATCH v9 03/12] mmc: crypto: add mmc_from_crypto_profile() Date: Sun, 8 Dec 2024 20:55:21 -0800 Message-ID: <20241209045530.507833-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241209045530.507833-1-ebiggers@kernel.org> References: <20241209045530.507833-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Eric Biggers Add a helper function that encapsulates a container_of expression. For now there is just one user but soon there will be more. Signed-off-by: Eric Biggers --- drivers/mmc/host/cqhci-crypto.c | 5 +---- include/linux/mmc/host.h | 8 ++++++++ 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/mmc/host/cqhci-crypto.c b/drivers/mmc/host/cqhci-crypto.c index d5f4b6972f63e..2951911d3f780 100644 --- a/drivers/mmc/host/cqhci-crypto.c +++ b/drivers/mmc/host/cqhci-crypto.c @@ -23,14 +23,11 @@ static const struct cqhci_crypto_alg_entry { }; static inline struct cqhci_host * cqhci_host_from_crypto_profile(struct blk_crypto_profile *profile) { - struct mmc_host *mmc = - container_of(profile, struct mmc_host, crypto_profile); - - return mmc->cqe_private; + return mmc_from_crypto_profile(profile)->cqe_private; } static int cqhci_crypto_program_key(struct cqhci_host *cq_host, const union cqhci_crypto_cfg_entry *cfg, int slot) diff --git a/include/linux/mmc/host.h b/include/linux/mmc/host.h index f166d6611ddb9..68f09a955a902 100644 --- a/include/linux/mmc/host.h +++ b/include/linux/mmc/host.h @@ -588,10 +588,18 @@ static inline void *mmc_priv(struct mmc_host *host) static inline struct mmc_host *mmc_from_priv(void *priv) { return container_of(priv, struct mmc_host, private); } +#ifdef CONFIG_MMC_CRYPTO +static inline struct mmc_host * +mmc_from_crypto_profile(struct blk_crypto_profile *profile) +{ + return container_of(profile, struct mmc_host, crypto_profile); +} +#endif + #define mmc_host_is_spi(host) ((host)->caps & MMC_CAP_SPI) #define mmc_dev(x) ((x)->parent) #define mmc_classdev(x) (&(x)->class_dev) #define mmc_hostname(x) (dev_name(&(x)->class_dev)) From patchwork Mon Dec 9 04:55:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 850005 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C43801991C9; Mon, 9 Dec 2024 04:57:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733720222; cv=none; b=VtZQw5d2qMxfqsXHuYBLjBzSfN1BKKYDNYGR+cgC0Qgs0AzIuSygFasBehKrDEq8vPm6uz/CKOvUHVAsIbCn4+x41NhMiou++dQyDTmxyUY0XRI2Ggpvgtzgnx33rZta2sfQReXwFaKQE/nznyW6KoMydujk1tXc/wDMQwiZckw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733720222; c=relaxed/simple; bh=ZweEoiVNeRw4qgQBQf6Ehu9c8g3ZN+76feLZJsrMhd4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EMe93RnTmzvu0+2mKtGtVaiBaQ6Bsk3rqITmZR6/UgtCfVjNCPQuUSbhE5RHc0kuJqoWa2Ed6w03yXVoLaWBKFSj+bh1GndFij95SI/0H44PAane34U8xNeKGfwvCPdj9oAlz9Dze0Ref+4YMNj4LvdaKINaNwZEzlPJNniSh8I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kDItb8Jz; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kDItb8Jz" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BDA9BC4CEE2; Mon, 9 Dec 2024 04:57:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1733720222; bh=ZweEoiVNeRw4qgQBQf6Ehu9c8g3ZN+76feLZJsrMhd4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kDItb8JzAtRXAl5AWdNW1GuU2E2ku4TJhOWIqFS3wRqiWJ9LsP799iRaAELeDFjjK QVJ/Idu5u8kWcptRYRjeGkUO27I0Knhp7d89roHDK76Qd0Od99a7k0u5vF7w5gXCAq RSFmSDG4//kFOlVmqK3kcwXa8Lv7CpKtKigwn11r27kSYtsi9zvSEekAANknOjxAcw khilnGTLrOmUF6yFlG3nrcDFwSyB6yCXDd6jH0Fv9B7tahpIgOxYHr6togg33I+yVO 8A3/LRIkUm+MQv2Mp4jGO1kfGoQaMlkkUNMVTCWJ63mizPatuIKF5UUY5GWgQ6Q/l4 8/gB9QWaHXrNA== From: Eric Biggers To: linux-block@vger.kernel.org, linux-fscrypt@vger.kernel.org, linux-mmc@vger.kernel.org, linux-scsi@vger.kernel.org, linux-arm-msm@vger.kernel.org, Bartosz Golaszewski , Gaurav Kashyap Cc: Adrian Hunter , Alim Akhtar , Avri Altman , Bart Van Assche , Bjorn Andersson , Dmitry Baryshkov , "James E . J . Bottomley" , Jens Axboe , Konrad Dybcio , Manivannan Sadhasivam , "Martin K . Petersen" , Ulf Hansson , Bartosz Golaszewski Subject: [PATCH v9 05/12] firmware: qcom: scm: add calls for wrapped key support Date: Sun, 8 Dec 2024 20:55:23 -0800 Message-ID: <20241209045530.507833-6-ebiggers@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241209045530.507833-1-ebiggers@kernel.org> References: <20241209045530.507833-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Gaurav Kashyap Add helper functions for the SCM calls required to support hardware-wrapped inline storage encryption keys. These SCM calls manage wrapped keys via Qualcomm's Hardware Key Manager (HWKM), which can only be accessed from TrustZone. QCOM_SCM_ES_GENERATE_ICE_KEY and QCOM_SCM_ES_IMPORT_ICE_KEY create a new long-term wrapped key, with the former making the hardware generate the key and the latter importing a raw key. QCOM_SCM_ES_PREPARE_ICE_KEY converts the key to ephemerally-wrapped form so that it can be used for inline storage encryption. These are planned to be wired up to new ioctls via the blk-crypto framework; see the proposed documentation for the hardware-wrapped keys feature for more information. Similarly there's also QCOM_SCM_ES_DERIVE_SW_SECRET which derives a "software secret" from an ephemerally-wrapped key and will be wired up to the corresponding operation in the blk_crypto_profile. These will all be used by the ICE driver in drivers/soc/qcom/ice.c. Signed-off-by: Gaurav Kashyap Signed-off-by: Bartosz Golaszewski [EB: merged related patches, fixed error handling, fixed naming, fixed docs for size parameters, fixed qcom_scm_has_wrapped_key_support(), improved comments, improved commit message.] Signed-off-by: Eric Biggers --- drivers/firmware/qcom/qcom_scm.c | 214 +++++++++++++++++++++++++ drivers/firmware/qcom/qcom_scm.h | 4 + include/linux/firmware/qcom/qcom_scm.h | 8 + 3 files changed, 226 insertions(+) diff --git a/drivers/firmware/qcom/qcom_scm.c b/drivers/firmware/qcom/qcom_scm.c index 72bf87ddcd969..180220d663f8b 100644 --- a/drivers/firmware/qcom/qcom_scm.c +++ b/drivers/firmware/qcom/qcom_scm.c @@ -1277,10 +1277,224 @@ int qcom_scm_ice_set_key(u32 index, const u8 *key, u32 key_size, return ret; } EXPORT_SYMBOL_GPL(qcom_scm_ice_set_key); +bool qcom_scm_has_wrapped_key_support(void) +{ + return __qcom_scm_is_call_available(__scm->dev, QCOM_SCM_SVC_ES, + QCOM_SCM_ES_DERIVE_SW_SECRET) && + __qcom_scm_is_call_available(__scm->dev, QCOM_SCM_SVC_ES, + QCOM_SCM_ES_GENERATE_ICE_KEY) && + __qcom_scm_is_call_available(__scm->dev, QCOM_SCM_SVC_ES, + QCOM_SCM_ES_PREPARE_ICE_KEY) && + __qcom_scm_is_call_available(__scm->dev, QCOM_SCM_SVC_ES, + QCOM_SCM_ES_IMPORT_ICE_KEY); +} +EXPORT_SYMBOL_GPL(qcom_scm_has_wrapped_key_support); + +/** + * qcom_scm_derive_sw_secret() - Derive software secret from wrapped key + * @eph_key: an ephemerally-wrapped key + * @eph_key_size: size of @eph_key in bytes + * @sw_secret: output buffer for the software secret + * @sw_secret_size: size of the software secret to derive in bytes + * + * Derive a software secret from an ephemerally-wrapped key for software crypto + * operations. This is done by calling into the secure execution environment, + * which then calls into the hardware to unwrap and derive the secret. + * + * For more information on sw_secret, see the "Hardware-wrapped keys" section of + * Documentation/block/inline-encryption.rst. + * + * Return: 0 on success; -errno on failure. + */ +int qcom_scm_derive_sw_secret(const u8 *eph_key, size_t eph_key_size, + u8 *sw_secret, size_t sw_secret_size) +{ + struct qcom_scm_desc desc = { + .svc = QCOM_SCM_SVC_ES, + .cmd = QCOM_SCM_ES_DERIVE_SW_SECRET, + .arginfo = QCOM_SCM_ARGS(4, QCOM_SCM_RW, QCOM_SCM_VAL, + QCOM_SCM_RW, QCOM_SCM_VAL), + .owner = ARM_SMCCC_OWNER_SIP, + }; + int ret; + + void *eph_key_buf __free(qcom_tzmem) = qcom_tzmem_alloc(__scm->mempool, + eph_key_size, + GFP_KERNEL); + if (!eph_key_buf) + return -ENOMEM; + + void *sw_secret_buf __free(qcom_tzmem) = qcom_tzmem_alloc(__scm->mempool, + sw_secret_size, + GFP_KERNEL); + if (!sw_secret_buf) + return -ENOMEM; + + memcpy(eph_key_buf, eph_key_buf, eph_key_size); + desc.args[0] = qcom_tzmem_to_phys(eph_key_buf); + desc.args[1] = eph_key_size; + desc.args[2] = qcom_tzmem_to_phys(sw_secret_buf); + desc.args[3] = sw_secret_size; + + ret = qcom_scm_call(__scm->dev, &desc, NULL); + if (!ret) + memcpy(sw_secret, sw_secret_buf, sw_secret_size); + + memzero_explicit(eph_key_buf, eph_key_size); + memzero_explicit(sw_secret_buf, sw_secret_size); + return ret; +} +EXPORT_SYMBOL_GPL(qcom_scm_derive_sw_secret); + +/** + * qcom_scm_generate_ice_key() - Generate a wrapped key for storage encryption + * @lt_key: output buffer for the long-term wrapped key + * @lt_key_size: size of @lt_key in bytes. Must be the exact wrapped key size + * used by the SoC. + * + * Generate a key using the built-in HW module in the SoC. The resulting key is + * returned wrapped with the platform-specific Key Encryption Key. + * + * Return: 0 on success; -errno on failure. + */ +int qcom_scm_generate_ice_key(u8 *lt_key, size_t lt_key_size) +{ + struct qcom_scm_desc desc = { + .svc = QCOM_SCM_SVC_ES, + .cmd = QCOM_SCM_ES_GENERATE_ICE_KEY, + .arginfo = QCOM_SCM_ARGS(2, QCOM_SCM_RW, QCOM_SCM_VAL), + .owner = ARM_SMCCC_OWNER_SIP, + }; + int ret; + + void *lt_key_buf __free(qcom_tzmem) = qcom_tzmem_alloc(__scm->mempool, + lt_key_size, + GFP_KERNEL); + if (!lt_key_buf) + return -ENOMEM; + + desc.args[0] = qcom_tzmem_to_phys(lt_key_buf); + desc.args[1] = lt_key_size; + + ret = qcom_scm_call(__scm->dev, &desc, NULL); + if (!ret) + memcpy(lt_key, lt_key_buf, lt_key_size); + + memzero_explicit(lt_key_buf, lt_key_size); + return ret; +} +EXPORT_SYMBOL_GPL(qcom_scm_generate_ice_key); + +/** + * qcom_scm_prepare_ice_key() - Re-wrap a key with the per-boot ephemeral key + * @lt_key: a long-term wrapped key + * @lt_key_size: size of @lt_key in bytes + * @eph_key: output buffer for the ephemerally-wrapped key + * @eph_key_size: size of @eph_key in bytes. Must be the exact wrapped key size + * used by the SoC. + * + * Given a long-term wrapped key, re-wrap it with the per-boot ephemeral key for + * added protection. The resulting key will only be valid for the current boot. + * + * Return: 0 on success; -errno on failure. + */ +int qcom_scm_prepare_ice_key(const u8 *lt_key, size_t lt_key_size, + u8 *eph_key, size_t eph_key_size) +{ + struct qcom_scm_desc desc = { + .svc = QCOM_SCM_SVC_ES, + .cmd = QCOM_SCM_ES_PREPARE_ICE_KEY, + .arginfo = QCOM_SCM_ARGS(4, QCOM_SCM_RO, QCOM_SCM_VAL, + QCOM_SCM_RW, QCOM_SCM_VAL), + .owner = ARM_SMCCC_OWNER_SIP, + }; + int ret; + + void *lt_key_buf __free(qcom_tzmem) = qcom_tzmem_alloc(__scm->mempool, + lt_key_size, + GFP_KERNEL); + if (!lt_key_buf) + return -ENOMEM; + + void *eph_key_buf __free(qcom_tzmem) = qcom_tzmem_alloc(__scm->mempool, + eph_key_size, + GFP_KERNEL); + if (!eph_key_buf) + return -ENOMEM; + + memcpy(lt_key_buf, lt_key, lt_key_size); + desc.args[0] = qcom_tzmem_to_phys(lt_key_buf); + desc.args[1] = lt_key_size; + desc.args[2] = qcom_tzmem_to_phys(eph_key_buf); + desc.args[3] = eph_key_size; + + ret = qcom_scm_call(__scm->dev, &desc, NULL); + if (!ret) + memcpy(eph_key, eph_key_buf, eph_key_size); + + memzero_explicit(lt_key_buf, lt_key_size); + memzero_explicit(eph_key_buf, eph_key_size); + return ret; +} +EXPORT_SYMBOL_GPL(qcom_scm_prepare_ice_key); + +/** + * qcom_scm_import_ice_key() - Import key for storage encryption + * @raw_key: the raw key to import + * @raw_key_size: size of @raw_key in bytes + * @lt_key: output buffer for the long-term wrapped key + * @lt_key_size: size of @lt_key in bytes. Must be the exact wrapped key size + * used by the SoC. + * + * Import a raw key and return a long-term wrapped key. Uses the SoC's HWKM to + * wrap the raw key using the platform-specific Key Encryption Key. + * + * Return: 0 on success; -errno on failure. + */ +int qcom_scm_import_ice_key(const u8 *raw_key, size_t raw_key_size, + u8 *lt_key, size_t lt_key_size) +{ + struct qcom_scm_desc desc = { + .svc = QCOM_SCM_SVC_ES, + .cmd = QCOM_SCM_ES_IMPORT_ICE_KEY, + .arginfo = QCOM_SCM_ARGS(4, QCOM_SCM_RO, QCOM_SCM_VAL, + QCOM_SCM_RW, QCOM_SCM_VAL), + .owner = ARM_SMCCC_OWNER_SIP, + }; + int ret; + + void *raw_key_buf __free(qcom_tzmem) = qcom_tzmem_alloc(__scm->mempool, + raw_key_size, + GFP_KERNEL); + if (!raw_key_buf) + return -ENOMEM; + + void *lt_key_buf __free(qcom_tzmem) = qcom_tzmem_alloc(__scm->mempool, + lt_key_size, + GFP_KERNEL); + if (!lt_key_buf) + return -ENOMEM; + + memcpy(raw_key_buf, raw_key, raw_key_size); + desc.args[0] = qcom_tzmem_to_phys(raw_key_buf); + desc.args[1] = raw_key_size; + desc.args[2] = qcom_tzmem_to_phys(lt_key_buf); + desc.args[3] = lt_key_size; + + ret = qcom_scm_call(__scm->dev, &desc, NULL); + if (!ret) + memcpy(lt_key, lt_key_buf, lt_key_size); + + memzero_explicit(raw_key_buf, raw_key_size); + memzero_explicit(lt_key_buf, lt_key_size); + return ret; +} +EXPORT_SYMBOL_GPL(qcom_scm_import_ice_key); + /** * qcom_scm_hdcp_available() - Check if secure environment supports HDCP. * * Return true if HDCP is supported, false if not. */ diff --git a/drivers/firmware/qcom/qcom_scm.h b/drivers/firmware/qcom/qcom_scm.h index e36b2f67607fc..097369d38b84e 100644 --- a/drivers/firmware/qcom/qcom_scm.h +++ b/drivers/firmware/qcom/qcom_scm.h @@ -126,10 +126,14 @@ struct qcom_tzmem_pool *qcom_scm_get_tzmem_pool(void); #define QCOM_SCM_OCMEM_UNLOCK_CMD 0x02 #define QCOM_SCM_SVC_ES 0x10 /* Enterprise Security */ #define QCOM_SCM_ES_INVALIDATE_ICE_KEY 0x03 #define QCOM_SCM_ES_CONFIG_SET_ICE_KEY 0x04 +#define QCOM_SCM_ES_DERIVE_SW_SECRET 0x07 +#define QCOM_SCM_ES_GENERATE_ICE_KEY 0x08 +#define QCOM_SCM_ES_PREPARE_ICE_KEY 0x09 +#define QCOM_SCM_ES_IMPORT_ICE_KEY 0x0a #define QCOM_SCM_SVC_HDCP 0x11 #define QCOM_SCM_HDCP_INVOKE 0x01 #define QCOM_SCM_SVC_LMH 0x13 diff --git a/include/linux/firmware/qcom/qcom_scm.h b/include/linux/firmware/qcom/qcom_scm.h index 4621aec0328c2..983e1591bbba7 100644 --- a/include/linux/firmware/qcom/qcom_scm.h +++ b/include/linux/firmware/qcom/qcom_scm.h @@ -103,10 +103,18 @@ int qcom_scm_ocmem_unlock(enum qcom_scm_ocmem_client id, u32 offset, u32 size); bool qcom_scm_ice_available(void); int qcom_scm_ice_invalidate_key(u32 index); int qcom_scm_ice_set_key(u32 index, const u8 *key, u32 key_size, enum qcom_scm_ice_cipher cipher, u32 data_unit_size); +bool qcom_scm_has_wrapped_key_support(void); +int qcom_scm_derive_sw_secret(const u8 *eph_key, size_t eph_key_size, + u8 *sw_secret, size_t sw_secret_size); +int qcom_scm_generate_ice_key(u8 *lt_key, size_t lt_key_size); +int qcom_scm_prepare_ice_key(const u8 *lt_key, size_t lt_key_size, + u8 *eph_key, size_t eph_key_size); +int qcom_scm_import_ice_key(const u8 *raw_key, size_t raw_key_size, + u8 *lt_key, size_t lt_key_size); bool qcom_scm_hdcp_available(void); int qcom_scm_hdcp_req(struct qcom_scm_hdcp_req *req, u32 req_cnt, u32 *resp); int qcom_scm_iommu_set_pt_format(u32 sec_id, u32 ctx_num, u32 pt_fmt); From patchwork Mon Dec 9 04:55:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 850004 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E98A6199938; Mon, 9 Dec 2024 04:57:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733720224; cv=none; b=IHpCu1FqIKXwhyTrnPvp5pNonxbmE5/HODzvpMGZNKM8dorgqjFcCfOWSuJD5kJjO9JH6C7y+K2po4bp8DFeiSdd+l1XNxVkWo12+eqoZashoTICXUCBpJu37kv49YiycDp5h7uBjqnGpXTTZZYoaYgh6Bm49ht9fgaTJaLvuYs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733720224; c=relaxed/simple; bh=P+IYcSv+3ZPjU0tFb0vZaSHTd6TYe+nZefGBRRX9nu4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gJcKrjx1xiWkfe1c9y6VJAgfKrgf1rusBd0eBoZ4tzeeHirE2QvhDU6YfyaZBOqYmaMxDgk+MM9tb2AYnPLfUQJZxRMxKoYohYcVOzImEfQvhmMpVopS6Z8bnpxymAZlQh3+zrrn2P8harFfWx77OONaesgxOZTUluH/aDcILyY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dG62w0hF; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dG62w0hF" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 399DAC4CED1; Mon, 9 Dec 2024 04:57:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1733720223; bh=P+IYcSv+3ZPjU0tFb0vZaSHTd6TYe+nZefGBRRX9nu4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dG62w0hFnPIssARss1U5qKCySvQndVlbrTdDj2Vh5VmvYYOUti5uoldgMekZdGRZc ubRUQhzopTyDdkh0/lKOwFvWSyzuD0N3DeBW5qW4ZHTz81PKwn2EBFFMIdoWRUyDDg PMsP9FYPabryHC5OHoWZ+Oe6Lhu7X4jPCWS1VPLtqJP97buJ+uqZjlip/rzijLUJgj ZZVMGIqpgIiEZOq35VZEgmUYjJBxqfkgnWW8v0YdrluvqZe3PCwvsCSpFMnL9rz3To UtVpaYyjL555GCEKugM12MBADdOVR/6jWqnWjeoYXZ/boTs8lYiViGqjW7UHWaTFTz 9zszgJWtp4K0w== From: Eric Biggers To: linux-block@vger.kernel.org, linux-fscrypt@vger.kernel.org, linux-mmc@vger.kernel.org, linux-scsi@vger.kernel.org, linux-arm-msm@vger.kernel.org, Bartosz Golaszewski , Gaurav Kashyap Cc: Adrian Hunter , Alim Akhtar , Avri Altman , Bart Van Assche , Bjorn Andersson , Dmitry Baryshkov , "James E . J . Bottomley" , Jens Axboe , Konrad Dybcio , Manivannan Sadhasivam , "Martin K . Petersen" , Ulf Hansson Subject: [PATCH v9 07/12] blk-crypto: add basic hardware-wrapped key support Date: Sun, 8 Dec 2024 20:55:25 -0800 Message-ID: <20241209045530.507833-8-ebiggers@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241209045530.507833-1-ebiggers@kernel.org> References: <20241209045530.507833-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Eric Biggers To prevent keys from being compromised if an attacker acquires read access to kernel memory, some inline encryption hardware can accept keys which are wrapped by a per-boot hardware-internal key. This avoids needing to keep the raw keys in kernel memory, without limiting the number of keys that can be used. Such hardware also supports deriving a "software secret" for cryptographic tasks that can't be handled by inline encryption; this is needed for fscrypt to work properly. To support this hardware, allow struct blk_crypto_key to represent a hardware-wrapped key as an alternative to a raw key, and make drivers set flags in struct blk_crypto_profile to indicate which types of keys they support. Also add the ->derive_sw_secret() low-level operation, which drivers supporting wrapped keys must implement. For more information, see the detailed documentation which this patch adds to Documentation/block/inline-encryption.rst. Signed-off-by: Eric Biggers --- Documentation/block/inline-encryption.rst | 219 +++++++++++++++++++++- block/blk-crypto-fallback.c | 7 +- block/blk-crypto-internal.h | 1 + block/blk-crypto-profile.c | 46 +++++ block/blk-crypto.c | 53 ++++-- drivers/md/dm-table.c | 1 + drivers/mmc/host/cqhci-crypto.c | 8 +- drivers/mmc/host/sdhci-msm.c | 1 + drivers/soc/qcom/ice.c | 2 +- drivers/ufs/core/ufshcd-crypto.c | 7 +- drivers/ufs/host/ufs-exynos.c | 3 +- drivers/ufs/host/ufs-qcom.c | 1 + fs/crypto/inline_crypt.c | 4 +- include/linux/blk-crypto-profile.h | 20 ++ include/linux/blk-crypto.h | 72 ++++++- 15 files changed, 411 insertions(+), 34 deletions(-) diff --git a/Documentation/block/inline-encryption.rst b/Documentation/block/inline-encryption.rst index 90b733422ed46..f03bd5b090d89 100644 --- a/Documentation/block/inline-encryption.rst +++ b/Documentation/block/inline-encryption.rst @@ -75,14 +75,14 @@ Constraints and notes Basic design ============ We introduce ``struct blk_crypto_key`` to represent an inline encryption key and -how it will be used. This includes the actual bytes of the key; the size of the -key; the algorithm and data unit size the key will be used with; and the number -of bytes needed to represent the maximum data unit number the key will be used -with. +how it will be used. This includes the type of the key (raw or +hardware-wrapped); the actual bytes of the key; the size of the key; the +algorithm and data unit size the key will be used with; and the number of bytes +needed to represent the maximum data unit number the key will be used with. We introduce ``struct bio_crypt_ctx`` to represent an encryption context. It contains a data unit number and a pointer to a blk_crypto_key. We add pointers to a bio_crypt_ctx to ``struct bio`` and ``struct request``; this allows users of the block layer (e.g. filesystems) to provide an encryption context when @@ -299,5 +299,216 @@ and disallow the combination for now. Whenever a device supports integrity, the kernel will pretend that the device does not support hardware inline encryption (by setting the blk_crypto_profile in the request_queue of the device to NULL). When the crypto API fallback is enabled, this means that all bios with and encryption context will use the fallback, and IO will complete as usual. When the fallback is disabled, a bio with an encryption context will be failed. + +.. _hardware_wrapped_keys: + +Hardware-wrapped keys +===================== + +Motivation and threat model +--------------------------- + +Linux storage encryption (dm-crypt, fscrypt, eCryptfs, etc.) traditionally +relies on the raw encryption key(s) being present in kernel memory so that the +encryption can be performed. This traditionally isn't seen as a problem because +the key(s) won't be present during an offline attack, which is the main type of +attack that storage encryption is intended to protect from. + +However, there is an increasing desire to also protect users' data from other +types of attacks (to the extent possible), including: + +- Cold boot attacks, where an attacker with physical access to a system suddenly + powers it off, then immediately dumps the system memory to extract recently + in-use encryption keys, then uses these keys to decrypt user data on-disk. + +- Online attacks where the attacker is able to read kernel memory without fully + compromising the system, followed by an offline attack where any extracted + keys can be used to decrypt user data on-disk. An example of such an online + attack would be if the attacker is able to run some code on the system that + exploits a Meltdown-like vulnerability but is unable to escalate privileges. + +- Online attacks where the attacker fully compromises the system, but their data + exfiltration is significantly time-limited and/or bandwidth-limited, so in + order to completely exfiltrate the data they need to extract the encryption + keys to use in a later offline attack. + +Hardware-wrapped keys are a feature of inline encryption hardware that is +designed to protect users' data from the above attacks (to the extent possible), +without introducing limitations such as a maximum number of keys. + +Note that it is impossible to **fully** protect users' data from these attacks. +Even in the attacks where the attacker "just" gets read access to kernel memory, +they can still extract any user data that is present in memory, including +plaintext pagecache pages of encrypted files. The focus here is just on +protecting the encryption keys, as those instantly give access to **all** user +data in any following offline attack, rather than just some of it (where which +data is included in that "some" might not be controlled by the attacker). + +Solution overview +----------------- + +Inline encryption hardware typically has "keyslots" into which software can +program keys for the hardware to use; the contents of keyslots typically can't +be read back by software. As such, the above security goals could be achieved +if the kernel simply erased its copy of the key(s) after programming them into +keyslot(s) and thereafter only referred to them via keyslot number. + +However, that naive approach runs into a couple problems: + +- It limits the number of unlocked keys to the number of keyslots, which + typically is a small number. In cases where there is only one encryption key + system-wide (e.g., a full-disk encryption key), that can be tolerable. + However, in general there can be many logged-in users with many different + keys, and/or many running applications with application-specific encrypted + storage areas. This is especially true if file-based encryption (e.g. + fscrypt) is being used. + +- Inline crypto engines typically lose the contents of their keyslots if the + storage controller (usually UFS or eMMC) is reset. Resetting the storage + controller is a standard error recovery procedure that is executed if certain + types of storage errors occur, and such errors can occur at any time. + Therefore, when inline crypto is being used, the operating system must always + be ready to reprogram the keyslots without user intervention. + +Thus, it is important for the kernel to still have a way to "remind" the +hardware about a key, without actually having the raw key itself. + +Somewhat less importantly, it is also desirable that the raw keys are never +visible to software at all, even while being initially unlocked. This would +ensure that a read-only compromise of system memory will never allow a key to be +extracted to be used off-system, even if it occurs when a key is being unlocked. + +To solve all these problems, some vendors of inline encryption hardware have +made their hardware support *hardware-wrapped keys*. Hardware-wrapped keys +are encrypted keys that can only be unwrapped (decrypted) and used by hardware +-- either by the inline encryption hardware itself, or by a dedicated hardware +block that can directly provision keys to the inline encryption hardware. + +(We refer to them as "hardware-wrapped keys" rather than simply "wrapped keys" +to add some clarity in cases where there could be other types of wrapped keys, +such as in file-based encryption. Key wrapping is a commonly used technique.) + +The key which wraps (encrypts) hardware-wrapped keys is a hardware-internal key +that is never exposed to software; it is either a persistent key (a "long-term +wrapping key") or a per-boot key (an "ephemeral wrapping key"). The long-term +wrapped form of the key is what is initially unlocked, but it is erased from +memory as soon as it is converted into an ephemerally-wrapped key. In-use +hardware-wrapped keys are always ephemerally-wrapped, not long-term wrapped. + +As inline encryption hardware can only be used to encrypt/decrypt data on-disk, +the hardware also includes a level of indirection; it doesn't use the unwrapped +key directly for inline encryption, but rather derives both an inline encryption +key and a "software secret" from it. Software can use the "software secret" for +tasks that can't use the inline encryption hardware, such as filenames +encryption. The software secret is not protected from memory compromise. + +Key hierarchy +------------- + +Here is the key hierarchy for a hardware-wrapped key:: + + Hardware-wrapped key + | + | + + | + ----------------------------- + | | + Inline encryption key Software secret + +The components are: + +- *Hardware-wrapped key*: a key for the hardware's KDF (Key Derivation + Function), in ephemerally-wrapped form. The key wrapping algorithm is a + hardware implementation detail that doesn't impact kernel operation, but a + strong authenticated encryption algorithm such as AES-256-GCM is recommended. + +- *Hardware KDF*: a KDF (Key Derivation Function) which the hardware uses to + derive subkeys after unwrapping the wrapped key. The hardware's choice of KDF + doesn't impact kernel operation, but it does need to be known for testing + purposes, and it's also assumed to have at least a 256-bit security strength. + All known hardware uses the SP800-108 KDF in Counter Mode with AES-256-CMAC, + with a particular choice of labels and contexts; new hardware should use this + already-vetted KDF. + +- *Inline encryption key*: a derived key which the hardware directly provisions + to a keyslot of the inline encryption hardware, without exposing it to + software. In all known hardware, this will always be an AES-256-XTS key. + However, in principle other encryption algorithms could be supported too. + Hardware must derive distinct subkeys for each supported encryption algorithm. + +- *Software secret*: a derived key which the hardware returns to software so + that software can use it for cryptographic tasks that can't use inline + encryption. This value is cryptographically isolated from the inline + encryption key, i.e. knowing one doesn't reveal the other. (The KDF ensures + this.) Currently, the software secret is always 32 bytes and thus is suitable + for cryptographic applications that require up to a 256-bit security strength. + Some use cases (e.g. full-disk encryption) won't require the software secret. + +Example: in the case of fscrypt, the fscrypt master key (the key that protects a +particular set of encrypted directories) is made hardware-wrapped. The inline +encryption key is used as the file contents encryption key, while the software +secret (rather than the master key directly) is used to key fscrypt's KDF +(HKDF-SHA512) to derive other subkeys such as filenames encryption keys. + +Note that currently this design assumes a single inline encryption key per +hardware-wrapped key, without any further key derivation. Thus, in the case of +fscrypt, currently hardware-wrapped keys are only compatible with the "inline +encryption optimized" settings, which use one file contents encryption key per +encryption policy rather than one per file. This design could be extended to +make the hardware derive per-file keys using per-file nonces passed down the +storage stack, and in fact some hardware already supports this; future work is +planned to remove this limitation by adding the corresponding kernel support. + +Kernel support +-------------- + +The inline encryption support of the kernel's block layer ("blk-crypto") has +been extended to support hardware-wrapped keys as an alternative to raw keys, +when hardware support is available. This works in the following way: + +- A ``key_types_supported`` field is added to the crypto capabilities in + ``struct blk_crypto_profile``. This allows device drivers to declare that + they support raw keys, hardware-wrapped keys, or both. + +- ``struct blk_crypto_key`` can now contain a hardware-wrapped key as an + alternative to a raw key; a ``key_type`` field is added to + ``struct blk_crypto_config`` to distinguish between the different key types. + This allows users of blk-crypto to en/decrypt data using a hardware-wrapped + key in a way very similar to using a raw key. + +- A new method ``blk_crypto_ll_ops::derive_sw_secret`` is added. Device drivers + that support hardware-wrapped keys must implement this method. Users of + blk-crypto can call ``blk_crypto_derive_sw_secret()`` to access this method. + +- The programming and eviction of hardware-wrapped keys happens via + ``blk_crypto_ll_ops::keyslot_program`` and + ``blk_crypto_ll_ops::keyslot_evict``, just like it does for raw keys. If a + driver supports hardware-wrapped keys, then it must handle hardware-wrapped + keys being passed to these methods. + +blk-crypto-fallback doesn't support hardware-wrapped keys. Therefore, +hardware-wrapped keys can only be used with actual inline encryption hardware. + +Testability +----------- + +Both the hardware KDF and the inline encryption itself are well-defined +algorithms that don't depend on any secrets other than the unwrapped key. +Therefore, if the unwrapped key is known to software, these algorithms can be +reproduced in software in order to verify the ciphertext that is written to disk +by the inline encryption hardware. + +However, the unwrapped key will only be known to software for testing if the +"import" functionality is used. Proper testing is not possible in the +"generate" case where the hardware generates the key itself. The correct +operation of the "generate" mode thus relies on the security and correctness of +the hardware RNG and its use to generate the key, as well as the testing of the +"import" mode as that should cover all parts other than the key generation. + +For an example of a test that verifies the ciphertext written to disk in the +"import" mode, see the fscrypt hardware-wrapped key tests in xfstests, or +`Android's vts_kernel_encryption_test +`_. diff --git a/block/blk-crypto-fallback.c b/block/blk-crypto-fallback.c index 29a205482617c..f154be0b575ad 100644 --- a/block/blk-crypto-fallback.c +++ b/block/blk-crypto-fallback.c @@ -85,11 +85,11 @@ static struct bio_set crypto_bio_split; /* * This is the key we set when evicting a keyslot. This *should* be the all 0's * key, but AES-XTS rejects that key, so we use some random bytes instead. */ -static u8 blank_key[BLK_CRYPTO_MAX_KEY_SIZE]; +static u8 blank_key[BLK_CRYPTO_MAX_RAW_KEY_SIZE]; static void blk_crypto_fallback_evict_keyslot(unsigned int slot) { struct blk_crypto_fallback_keyslot *slotp = &blk_crypto_keyslots[slot]; enum blk_crypto_mode_num crypto_mode = slotp->crypto_mode; @@ -117,11 +117,11 @@ blk_crypto_fallback_keyslot_program(struct blk_crypto_profile *profile, if (crypto_mode != slotp->crypto_mode && slotp->crypto_mode != BLK_ENCRYPTION_MODE_INVALID) blk_crypto_fallback_evict_keyslot(slot); slotp->crypto_mode = crypto_mode; - err = crypto_skcipher_setkey(slotp->tfms[crypto_mode], key->raw, + err = crypto_skcipher_setkey(slotp->tfms[crypto_mode], key->bytes, key->size); if (err) { blk_crypto_fallback_evict_keyslot(slot); return err; } @@ -537,11 +537,11 @@ static int blk_crypto_fallback_init(void) int err; if (blk_crypto_fallback_inited) return 0; - get_random_bytes(blank_key, BLK_CRYPTO_MAX_KEY_SIZE); + get_random_bytes(blank_key, sizeof(blank_key)); err = bioset_init(&crypto_bio_split, 64, 0, 0); if (err) goto out; @@ -559,10 +559,11 @@ static int blk_crypto_fallback_init(void) goto fail_free_profile; err = -ENOMEM; blk_crypto_fallback_profile->ll_ops = blk_crypto_fallback_ll_ops; blk_crypto_fallback_profile->max_dun_bytes_supported = BLK_CRYPTO_MAX_IV_SIZE; + blk_crypto_fallback_profile->key_types_supported = BLK_CRYPTO_KEY_TYPE_RAW; /* All blk-crypto modes have a crypto API fallback. */ for (i = 0; i < BLK_ENCRYPTION_MODE_MAX; i++) blk_crypto_fallback_profile->modes_supported[i] = 0xFFFFFFFF; blk_crypto_fallback_profile->modes_supported[BLK_ENCRYPTION_MODE_INVALID] = 0; diff --git a/block/blk-crypto-internal.h b/block/blk-crypto-internal.h index 93a141979694b..1893df9a8f06c 100644 --- a/block/blk-crypto-internal.h +++ b/block/blk-crypto-internal.h @@ -12,10 +12,11 @@ /* Represents a crypto mode supported by blk-crypto */ struct blk_crypto_mode { const char *name; /* name of this mode, shown in sysfs */ const char *cipher_str; /* crypto API name (for fallback case) */ unsigned int keysize; /* key size in bytes */ + unsigned int security_strength; /* security strength in bytes */ unsigned int ivsize; /* iv size in bytes */ }; extern const struct blk_crypto_mode blk_crypto_modes[]; diff --git a/block/blk-crypto-profile.c b/block/blk-crypto-profile.c index 7fabc883e39f1..1b92276ed2fcc 100644 --- a/block/blk-crypto-profile.c +++ b/block/blk-crypto-profile.c @@ -350,10 +350,12 @@ bool __blk_crypto_cfg_supported(struct blk_crypto_profile *profile, return false; if (!(profile->modes_supported[cfg->crypto_mode] & cfg->data_unit_size)) return false; if (profile->max_dun_bytes_supported < cfg->dun_bytes) return false; + if (!(profile->key_types_supported & cfg->key_type)) + return false; return true; } /* * This is an internal function that evicts a key from an inline encryption @@ -460,10 +462,48 @@ bool blk_crypto_register(struct blk_crypto_profile *profile, q->crypto_profile = profile; return true; } EXPORT_SYMBOL_GPL(blk_crypto_register); +/** + * blk_crypto_derive_sw_secret() - Derive software secret from wrapped key + * @bdev: a block device that supports hardware-wrapped keys + * @eph_key: the hardware-wrapped key in ephemerally-wrapped form + * @eph_key_size: size of @eph_key in bytes + * @sw_secret: (output) the software secret + * + * Given a hardware-wrapped key in ephemerally-wrapped form (the same form that + * it is used for I/O), ask the hardware to derive the secret which software can + * use for cryptographic tasks other than inline encryption. This secret is + * guaranteed to be cryptographically isolated from the inline encryption key, + * i.e. derived with a different KDF context. + * + * Return: 0 on success, -EOPNOTSUPP if the block device doesn't support + * hardware-wrapped keys, -EBADMSG if the key isn't a valid + * hardware-wrapped key, or another -errno code. + */ +int blk_crypto_derive_sw_secret(struct block_device *bdev, + const u8 *eph_key, size_t eph_key_size, + u8 sw_secret[BLK_CRYPTO_SW_SECRET_SIZE]) +{ + struct blk_crypto_profile *profile = + bdev_get_queue(bdev)->crypto_profile; + int err; + + if (!profile) + return -EOPNOTSUPP; + if (!(profile->key_types_supported & BLK_CRYPTO_KEY_TYPE_HW_WRAPPED)) + return -EOPNOTSUPP; + if (!profile->ll_ops.derive_sw_secret) + return -EOPNOTSUPP; + blk_crypto_hw_enter(profile); + err = profile->ll_ops.derive_sw_secret(profile, eph_key, eph_key_size, + sw_secret); + blk_crypto_hw_exit(profile); + return err; +} + /** * blk_crypto_intersect_capabilities() - restrict supported crypto capabilities * by child device * @parent: the crypto profile for the parent device * @child: the crypto profile for the child device, or NULL @@ -483,14 +523,16 @@ void blk_crypto_intersect_capabilities(struct blk_crypto_profile *parent, parent->max_dun_bytes_supported = min(parent->max_dun_bytes_supported, child->max_dun_bytes_supported); for (i = 0; i < ARRAY_SIZE(child->modes_supported); i++) parent->modes_supported[i] &= child->modes_supported[i]; + parent->key_types_supported &= child->key_types_supported; } else { parent->max_dun_bytes_supported = 0; memset(parent->modes_supported, 0, sizeof(parent->modes_supported)); + parent->key_types_supported = 0; } } EXPORT_SYMBOL_GPL(blk_crypto_intersect_capabilities); /** @@ -519,10 +561,13 @@ bool blk_crypto_has_capabilities(const struct blk_crypto_profile *target, if (reference->max_dun_bytes_supported > target->max_dun_bytes_supported) return false; + if (reference->key_types_supported & ~target->key_types_supported) + return false; + return true; } EXPORT_SYMBOL_GPL(blk_crypto_has_capabilities); /** @@ -553,7 +598,8 @@ void blk_crypto_update_capabilities(struct blk_crypto_profile *dst, { memcpy(dst->modes_supported, src->modes_supported, sizeof(dst->modes_supported)); dst->max_dun_bytes_supported = src->max_dun_bytes_supported; + dst->key_types_supported = src->key_types_supported; } EXPORT_SYMBOL_GPL(blk_crypto_update_capabilities); diff --git a/block/blk-crypto.c b/block/blk-crypto.c index 4d760b092deb9..b55b3d8bffa00 100644 --- a/block/blk-crypto.c +++ b/block/blk-crypto.c @@ -21,28 +21,32 @@ const struct blk_crypto_mode blk_crypto_modes[] = { [BLK_ENCRYPTION_MODE_AES_256_XTS] = { .name = "AES-256-XTS", .cipher_str = "xts(aes)", .keysize = 64, + .security_strength = 32, .ivsize = 16, }, [BLK_ENCRYPTION_MODE_AES_128_CBC_ESSIV] = { .name = "AES-128-CBC-ESSIV", .cipher_str = "essiv(cbc(aes),sha256)", .keysize = 16, + .security_strength = 16, .ivsize = 16, }, [BLK_ENCRYPTION_MODE_ADIANTUM] = { .name = "Adiantum", .cipher_str = "adiantum(xchacha12,aes)", .keysize = 32, + .security_strength = 32, .ivsize = 32, }, [BLK_ENCRYPTION_MODE_SM4_XTS] = { .name = "SM4-XTS", .cipher_str = "xts(sm4)", .keysize = 32, + .security_strength = 16, .ivsize = 16, }, }; /* @@ -74,13 +78,19 @@ static int __init bio_crypt_ctx_init(void) goto out_no_mem; /* This is assumed in various places. */ BUILD_BUG_ON(BLK_ENCRYPTION_MODE_INVALID != 0); - /* Sanity check that no algorithm exceeds the defined limits. */ + /* + * Validate the crypto mode properties. This ideally would be done with + * static assertions, but boot-time checks are the next best thing. + */ for (i = 0; i < BLK_ENCRYPTION_MODE_MAX; i++) { - BUG_ON(blk_crypto_modes[i].keysize > BLK_CRYPTO_MAX_KEY_SIZE); + BUG_ON(blk_crypto_modes[i].keysize > + BLK_CRYPTO_MAX_RAW_KEY_SIZE); + BUG_ON(blk_crypto_modes[i].security_strength > + blk_crypto_modes[i].keysize); BUG_ON(blk_crypto_modes[i].ivsize > BLK_CRYPTO_MAX_IV_SIZE); } return 0; out_no_mem: @@ -313,21 +323,24 @@ int __blk_crypto_rq_bio_prep(struct request *rq, struct bio *bio, } /** * blk_crypto_init_key() - Prepare a key for use with blk-crypto * @blk_key: Pointer to the blk_crypto_key to initialize. - * @raw_key: Pointer to the raw key. Must be the correct length for the chosen - * @crypto_mode; see blk_crypto_modes[]. + * @key_bytes: the bytes of the key + * @key_size: size of the key in bytes + * @key_type: type of the key -- either raw or hardware-wrapped * @crypto_mode: identifier for the encryption algorithm to use * @dun_bytes: number of bytes that will be used to specify the DUN when this * key is used * @data_unit_size: the data unit size to use for en/decryption * * Return: 0 on success, -errno on failure. The caller is responsible for - * zeroizing both blk_key and raw_key when done with them. + * zeroizing both blk_key and key_bytes when done with them. */ -int blk_crypto_init_key(struct blk_crypto_key *blk_key, const u8 *raw_key, +int blk_crypto_init_key(struct blk_crypto_key *blk_key, + const u8 *key_bytes, size_t key_size, + enum blk_crypto_key_type key_type, enum blk_crypto_mode_num crypto_mode, unsigned int dun_bytes, unsigned int data_unit_size) { const struct blk_crypto_mode *mode; @@ -336,25 +349,37 @@ int blk_crypto_init_key(struct blk_crypto_key *blk_key, const u8 *raw_key, if (crypto_mode >= ARRAY_SIZE(blk_crypto_modes)) return -EINVAL; mode = &blk_crypto_modes[crypto_mode]; - if (mode->keysize == 0) + switch (key_type) { + case BLK_CRYPTO_KEY_TYPE_RAW: + if (key_size != mode->keysize) + return -EINVAL; + break; + case BLK_CRYPTO_KEY_TYPE_HW_WRAPPED: + if (key_size < mode->security_strength || + key_size > BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE) + return -EINVAL; + break; + default: return -EINVAL; + } if (dun_bytes == 0 || dun_bytes > mode->ivsize) return -EINVAL; if (!is_power_of_2(data_unit_size)) return -EINVAL; blk_key->crypto_cfg.crypto_mode = crypto_mode; blk_key->crypto_cfg.dun_bytes = dun_bytes; blk_key->crypto_cfg.data_unit_size = data_unit_size; + blk_key->crypto_cfg.key_type = key_type; blk_key->data_unit_size_bits = ilog2(data_unit_size); - blk_key->size = mode->keysize; - memcpy(blk_key->raw, raw_key, mode->keysize); + blk_key->size = key_size; + memcpy(blk_key->bytes, key_bytes, key_size); return 0; } bool blk_crypto_config_supported_natively(struct block_device *bdev, @@ -370,12 +395,14 @@ bool blk_crypto_config_supported_natively(struct block_device *bdev, * blk-crypto-fallback is enabled and supports the cfg). */ bool blk_crypto_config_supported(struct block_device *bdev, const struct blk_crypto_config *cfg) { - return IS_ENABLED(CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK) || - blk_crypto_config_supported_natively(bdev, cfg); + if (IS_ENABLED(CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK) && + cfg->key_type == BLK_CRYPTO_KEY_TYPE_RAW) + return true; + return blk_crypto_config_supported_natively(bdev, cfg); } /** * blk_crypto_start_using_key() - Start using a blk_crypto_key on a device * @bdev: block device to operate on @@ -394,10 +421,14 @@ bool blk_crypto_config_supported(struct block_device *bdev, int blk_crypto_start_using_key(struct block_device *bdev, const struct blk_crypto_key *key) { if (blk_crypto_config_supported_natively(bdev, &key->crypto_cfg)) return 0; + if (key->crypto_cfg.key_type != BLK_CRYPTO_KEY_TYPE_RAW) { + pr_warn_once("tried to use wrapped key, but hardware doesn't support it\n"); + return -EOPNOTSUPP; + } return blk_crypto_fallback_start_using_mode(key->crypto_cfg.crypto_mode); } /** * blk_crypto_evict_key() - Evict a blk_crypto_key from a block_device diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c index bd8b796ae683b..3e2ab66a46aef 100644 --- a/drivers/md/dm-table.c +++ b/drivers/md/dm-table.c @@ -1248,10 +1248,11 @@ static int dm_table_construct_crypto_profile(struct dm_table *t) blk_crypto_profile_init(profile, 0); profile->ll_ops.keyslot_evict = dm_keyslot_evict; profile->max_dun_bytes_supported = UINT_MAX; memset(profile->modes_supported, 0xFF, sizeof(profile->modes_supported)); + profile->key_types_supported = ~0; for (i = 0; i < t->num_targets; i++) { struct dm_target *ti = dm_table_get_target(t, i); if (!dm_target_passes_crypto(ti->type)) { diff --git a/drivers/mmc/host/cqhci-crypto.c b/drivers/mmc/host/cqhci-crypto.c index 4736564286859..99456585d63bd 100644 --- a/drivers/mmc/host/cqhci-crypto.c +++ b/drivers/mmc/host/cqhci-crypto.c @@ -82,15 +82,15 @@ static int cqhci_crypto_keyslot_program(struct blk_crypto_profile *profile, cfg.crypto_cap_idx = cap_idx; cfg.config_enable = CQHCI_CRYPTO_CONFIGURATION_ENABLE; if (ccap_array[cap_idx].algorithm_id == CQHCI_CRYPTO_ALG_AES_XTS) { /* In XTS mode, the blk_crypto_key's size is already doubled */ - memcpy(cfg.crypto_key, key->raw, key->size/2); + memcpy(cfg.crypto_key, key->bytes, key->size/2); memcpy(cfg.crypto_key + CQHCI_CRYPTO_KEY_MAX_SIZE/2, - key->raw + key->size/2, key->size/2); + key->bytes + key->size/2, key->size/2); } else { - memcpy(cfg.crypto_key, key->raw, key->size); + memcpy(cfg.crypto_key, key->bytes, key->size); } cqhci_crypto_program_key(cq_host, &cfg, slot); memzero_explicit(&cfg, sizeof(cfg)); @@ -204,10 +204,12 @@ int cqhci_crypto_init(struct cqhci_host *cq_host) profile->dev = dev; /* Unfortunately, CQHCI crypto only supports 32 DUN bits. */ profile->max_dun_bytes_supported = 4; + profile->key_types_supported = BLK_CRYPTO_KEY_TYPE_RAW; + /* * Cache all the crypto capabilities and advertise the supported crypto * modes and data unit sizes to the block layer. */ for (cap_idx = 0; cap_idx < cq_host->crypto_capabilities.num_crypto_cap; diff --git a/drivers/mmc/host/sdhci-msm.c b/drivers/mmc/host/sdhci-msm.c index 8684a5f3357ba..27b24fb8e8809 100644 --- a/drivers/mmc/host/sdhci-msm.c +++ b/drivers/mmc/host/sdhci-msm.c @@ -1845,10 +1845,11 @@ static int sdhci_msm_ice_init(struct sdhci_msm_host *msm_host, if (err) return err; profile->ll_ops = sdhci_msm_crypto_ops; profile->max_dun_bytes_supported = 4; + profile->key_types_supported = BLK_CRYPTO_KEY_TYPE_RAW; profile->dev = dev; /* * Currently this driver only supports AES-256-XTS. All known versions * of ICE support it, but to be safe make sure it is really declared in diff --git a/drivers/soc/qcom/ice.c b/drivers/soc/qcom/ice.c index 04d5884574c54..78780fd508f0b 100644 --- a/drivers/soc/qcom/ice.c +++ b/drivers/soc/qcom/ice.c @@ -182,11 +182,11 @@ int qcom_ice_program_key(struct qcom_ice *ice, unsigned int slot, if (blk_key->size != AES_256_XTS_KEY_SIZE) { dev_err_ratelimited(dev, "Incorrect key size\n"); return -EINVAL; } - memcpy(key.bytes, blk_key->raw, AES_256_XTS_KEY_SIZE); + memcpy(key.bytes, blk_key->bytes, AES_256_XTS_KEY_SIZE); /* The SCM call requires that the key words are encoded in big endian */ for (i = 0; i < ARRAY_SIZE(key.words); i++) __cpu_to_be32s(&key.words[i]); diff --git a/drivers/ufs/core/ufshcd-crypto.c b/drivers/ufs/core/ufshcd-crypto.c index 694ff7578fc19..9e63a9d3cb7e2 100644 --- a/drivers/ufs/core/ufshcd-crypto.c +++ b/drivers/ufs/core/ufshcd-crypto.c @@ -70,15 +70,15 @@ static int ufshcd_crypto_keyslot_program(struct blk_crypto_profile *profile, cfg.crypto_cap_idx = cap_idx; cfg.config_enable = UFS_CRYPTO_CONFIGURATION_ENABLE; if (ccap_array[cap_idx].algorithm_id == UFS_CRYPTO_ALG_AES_XTS) { /* In XTS mode, the blk_crypto_key's size is already doubled */ - memcpy(cfg.crypto_key, key->raw, key->size/2); + memcpy(cfg.crypto_key, key->bytes, key->size/2); memcpy(cfg.crypto_key + UFS_CRYPTO_KEY_MAX_SIZE/2, - key->raw + key->size/2, key->size/2); + key->bytes + key->size/2, key->size/2); } else { - memcpy(cfg.crypto_key, key->raw, key->size); + memcpy(cfg.crypto_key, key->bytes, key->size); } ufshcd_program_key(hba, &cfg, slot); memzero_explicit(&cfg, sizeof(cfg)); @@ -183,10 +183,11 @@ int ufshcd_hba_init_crypto_capabilities(struct ufs_hba *hba) goto out; hba->crypto_profile.ll_ops = ufshcd_crypto_ops; /* UFS only supports 8 bytes for any DUN */ hba->crypto_profile.max_dun_bytes_supported = 8; + hba->crypto_profile.key_types_supported = BLK_CRYPTO_KEY_TYPE_RAW; hba->crypto_profile.dev = hba->dev; /* * Cache all the UFS crypto capabilities and advertise the supported * crypto modes and data unit sizes to the block layer. diff --git a/drivers/ufs/host/ufs-exynos.c b/drivers/ufs/host/ufs-exynos.c index 13dd5dfc03eb3..6a415d9bdc85a 100644 --- a/drivers/ufs/host/ufs-exynos.c +++ b/drivers/ufs/host/ufs-exynos.c @@ -1318,10 +1318,11 @@ static void exynos_ufs_fmp_init(struct ufs_hba *hba, struct exynos_ufs *ufs) dev_err(hba->dev, "Failed to initialize crypto profile: %d\n", err); return; } profile->max_dun_bytes_supported = AES_BLOCK_SIZE; + profile->key_types_supported = BLK_CRYPTO_KEY_TYPE_RAW; profile->dev = hba->dev; profile->modes_supported[BLK_ENCRYPTION_MODE_AES_256_XTS] = DATA_UNIT_SIZE; /* Advertise crypto support to ufshcd-core. */ @@ -1364,11 +1365,11 @@ static inline __be64 fmp_key_word(const u8 *key, int j) static int exynos_ufs_fmp_fill_prdt(struct ufs_hba *hba, const struct bio_crypt_ctx *crypt_ctx, void *prdt, unsigned int num_segments) { struct fmp_sg_entry *fmp_prdt = prdt; - const u8 *enckey = crypt_ctx->bc_key->raw; + const u8 *enckey = crypt_ctx->bc_key->bytes; const u8 *twkey = enckey + AES_KEYSIZE_256; u64 dun_lo = crypt_ctx->bc_dun[0]; u64 dun_hi = crypt_ctx->bc_dun[1]; unsigned int i; diff --git a/drivers/ufs/host/ufs-qcom.c b/drivers/ufs/host/ufs-qcom.c index c0e72d2de378a..865193d9bc474 100644 --- a/drivers/ufs/host/ufs-qcom.c +++ b/drivers/ufs/host/ufs-qcom.c @@ -146,10 +146,11 @@ static int ufs_qcom_ice_init(struct ufs_qcom_host *host) if (err) return err; profile->ll_ops = ufs_qcom_crypto_ops; profile->max_dun_bytes_supported = 8; + profile->key_types_supported = BLK_CRYPTO_KEY_TYPE_RAW; profile->dev = dev; /* * Currently this driver only supports AES-256-XTS. All known versions * of ICE support it, but to be safe make sure it is really declared in diff --git a/fs/crypto/inline_crypt.c b/fs/crypto/inline_crypt.c index 40de69860dcf9..7fa53d30aec30 100644 --- a/fs/crypto/inline_crypt.c +++ b/fs/crypto/inline_crypt.c @@ -128,10 +128,11 @@ int fscrypt_select_encryption_impl(struct fscrypt_inode_info *ci) * crypto configuration that the file would use. */ crypto_cfg.crypto_mode = ci->ci_mode->blk_crypto_mode; crypto_cfg.data_unit_size = 1U << ci->ci_data_unit_bits; crypto_cfg.dun_bytes = fscrypt_get_dun_bytes(ci); + crypto_cfg.key_type = BLK_CRYPTO_KEY_TYPE_RAW; devs = fscrypt_get_devices(sb, &num_devs); if (IS_ERR(devs)) return PTR_ERR(devs); @@ -164,11 +165,12 @@ int fscrypt_prepare_inline_crypt_key(struct fscrypt_prepared_key *prep_key, blk_key = kmalloc(sizeof(*blk_key), GFP_KERNEL); if (!blk_key) return -ENOMEM; - err = blk_crypto_init_key(blk_key, raw_key, crypto_mode, + err = blk_crypto_init_key(blk_key, raw_key, ci->ci_mode->keysize, + BLK_CRYPTO_KEY_TYPE_RAW, crypto_mode, fscrypt_get_dun_bytes(ci), 1U << ci->ci_data_unit_bits); if (err) { fscrypt_err(inode, "error %d initializing blk-crypto key", err); goto fail; diff --git a/include/linux/blk-crypto-profile.h b/include/linux/blk-crypto-profile.h index 90ab33cb5d0ef..7764b4f7b45b9 100644 --- a/include/linux/blk-crypto-profile.h +++ b/include/linux/blk-crypto-profile.h @@ -55,10 +55,24 @@ struct blk_crypto_ll_ops { * Must return 0 on success, or -errno on failure. */ int (*keyslot_evict)(struct blk_crypto_profile *profile, const struct blk_crypto_key *key, unsigned int slot); + + /** + * @derive_sw_secret: Derive the software secret from a hardware-wrapped + * key in ephemerally-wrapped form. + * + * This only needs to be implemented if BLK_CRYPTO_KEY_TYPE_HW_WRAPPED + * is supported. + * + * Must return 0 on success, -EBADMSG if the key is invalid, or another + * -errno code on other errors. + */ + int (*derive_sw_secret)(struct blk_crypto_profile *profile, + const u8 *eph_key, size_t eph_key_size, + u8 sw_secret[BLK_CRYPTO_SW_SECRET_SIZE]); }; /** * struct blk_crypto_profile - inline encryption profile for a device * @@ -82,10 +96,16 @@ struct blk_crypto_profile { * specifying the data unit number (DUN). Specifically, the range of * supported DUNs is 0 through (1 << (8 * max_dun_bytes_supported)) - 1. */ unsigned int max_dun_bytes_supported; + /** + * @key_types_supported: A bitmask of the supported key types: + * BLK_CRYPTO_KEY_TYPE_RAW and/or BLK_CRYPTO_KEY_TYPE_HW_WRAPPED. + */ + unsigned int key_types_supported; + /** * @modes_supported: Array of bitmasks that specifies whether each * combination of crypto mode and data unit size is supported. * Specifically, the i'th bit of modes_supported[crypto_mode] is set if * crypto_mode can be used with a data unit size of (1 << i). Note that diff --git a/include/linux/blk-crypto.h b/include/linux/blk-crypto.h index 5e5822c18ee41..0e63287e2175f 100644 --- a/include/linux/blk-crypto.h +++ b/include/linux/blk-crypto.h @@ -4,10 +4,11 @@ */ #ifndef __LINUX_BLK_CRYPTO_H #define __LINUX_BLK_CRYPTO_H +#include #include enum blk_crypto_mode_num { BLK_ENCRYPTION_MODE_INVALID, BLK_ENCRYPTION_MODE_AES_256_XTS, @@ -15,43 +16,94 @@ enum blk_crypto_mode_num { BLK_ENCRYPTION_MODE_ADIANTUM, BLK_ENCRYPTION_MODE_SM4_XTS, BLK_ENCRYPTION_MODE_MAX, }; -#define BLK_CRYPTO_MAX_KEY_SIZE 64 +/* + * Supported types of keys. Must be bitflags due to their use in + * blk_crypto_profile::key_types_supported. + */ +enum blk_crypto_key_type { + /* + * Raw keys (i.e. "software keys"). These keys are simply kept in raw, + * plaintext form in kernel memory. + */ + BLK_CRYPTO_KEY_TYPE_RAW = 1 << 0, + + /* + * Hardware-wrapped keys. These keys are only present in kernel memory + * in ephemerally-wrapped form, and they can only be unwrapped by + * dedicated hardware. For details, see the "Hardware-wrapped keys" + * section of Documentation/block/inline-encryption.rst. + */ + BLK_CRYPTO_KEY_TYPE_HW_WRAPPED = 1 << 1, +}; + +/* + * Currently the maximum raw key size is 64 bytes, as that is the key size of + * BLK_ENCRYPTION_MODE_AES_256_XTS which takes the longest key. + * + * The maximum hardware-wrapped key size depends on the hardware's key wrapping + * algorithm, which is a hardware implementation detail, so it isn't precisely + * specified. But currently 128 bytes is plenty in practice. Implementations + * are recommended to wrap a 32-byte key for the hardware KDF with AES-256-GCM, + * which should result in a size closer to 64 bytes than 128. + * + * Both of these values can trivially be increased if ever needed. + */ +#define BLK_CRYPTO_MAX_RAW_KEY_SIZE 64 +#define BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE 128 + +#define BLK_CRYPTO_MAX_ANY_KEY_SIZE \ + MAX(BLK_CRYPTO_MAX_RAW_KEY_SIZE, BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE) + +/* + * Size of the "software secret" which can be derived from a hardware-wrapped + * key. This is currently always 32 bytes. Note, the choice of 32 bytes + * assumes that the software secret is only used directly for algorithms that + * don't require more than a 256-bit key to get the desired security strength. + * If it were to be used e.g. directly as an AES-256-XTS key, then this would + * need to be increased (which is possible if hardware supports it, but care + * would need to be taken to avoid breaking users who need exactly 32 bytes). + */ +#define BLK_CRYPTO_SW_SECRET_SIZE 32 + /** * struct blk_crypto_config - an inline encryption key's crypto configuration * @crypto_mode: encryption algorithm this key is for * @data_unit_size: the data unit size for all encryption/decryptions with this * key. This is the size in bytes of each individual plaintext and * ciphertext. This is always a power of 2. It might be e.g. the * filesystem block size or the disk sector size. * @dun_bytes: the maximum number of bytes of DUN used when using this key + * @key_type: the type of this key -- either raw or hardware-wrapped */ struct blk_crypto_config { enum blk_crypto_mode_num crypto_mode; unsigned int data_unit_size; unsigned int dun_bytes; + enum blk_crypto_key_type key_type; }; /** * struct blk_crypto_key - an inline encryption key - * @crypto_cfg: the crypto configuration (like crypto_mode, key size) for this - * key + * @crypto_cfg: the crypto mode, data unit size, key type, and other + * characteristics of this key and how it will be used * @data_unit_size_bits: log2 of data_unit_size - * @size: size of this key in bytes (determined by @crypto_cfg.crypto_mode) - * @raw: the raw bytes of this key. Only the first @size bytes are used. + * @size: size of this key in bytes. The size of a raw key is fixed for a given + * crypto mode, but the size of a hardware-wrapped key can vary. + * @bytes: the bytes of this key. Only the first @size bytes are significant. * * A blk_crypto_key is immutable once created, and many bios can reference it at * the same time. It must not be freed until all bios using it have completed * and it has been evicted from all devices on which it may have been used. */ struct blk_crypto_key { struct blk_crypto_config crypto_cfg; unsigned int data_unit_size_bits; unsigned int size; - u8 raw[BLK_CRYPTO_MAX_KEY_SIZE]; + u8 bytes[BLK_CRYPTO_MAX_ANY_KEY_SIZE]; }; #define BLK_CRYPTO_MAX_IV_SIZE 32 #define BLK_CRYPTO_DUN_ARRAY_SIZE (BLK_CRYPTO_MAX_IV_SIZE / sizeof(u64)) @@ -85,11 +137,13 @@ void bio_crypt_set_ctx(struct bio *bio, const struct blk_crypto_key *key, bool bio_crypt_dun_is_contiguous(const struct bio_crypt_ctx *bc, unsigned int bytes, const u64 next_dun[BLK_CRYPTO_DUN_ARRAY_SIZE]); -int blk_crypto_init_key(struct blk_crypto_key *blk_key, const u8 *raw_key, +int blk_crypto_init_key(struct blk_crypto_key *blk_key, + const u8 *key_bytes, size_t key_size, + enum blk_crypto_key_type key_type, enum blk_crypto_mode_num crypto_mode, unsigned int dun_bytes, unsigned int data_unit_size); int blk_crypto_start_using_key(struct block_device *bdev, @@ -101,10 +155,14 @@ void blk_crypto_evict_key(struct block_device *bdev, bool blk_crypto_config_supported_natively(struct block_device *bdev, const struct blk_crypto_config *cfg); bool blk_crypto_config_supported(struct block_device *bdev, const struct blk_crypto_config *cfg); +int blk_crypto_derive_sw_secret(struct block_device *bdev, + const u8 *eph_key, size_t eph_key_size, + u8 sw_secret[BLK_CRYPTO_SW_SECRET_SIZE]); + #else /* CONFIG_BLK_INLINE_ENCRYPTION */ static inline bool bio_has_crypt_ctx(struct bio *bio) { return false; From patchwork Mon Dec 9 04:55:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 850003 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 63E3019ABC2; Mon, 9 Dec 2024 04:57:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733720225; cv=none; b=W2I5Wm13r+ACrya16Ct5CIvjRZLbP6kjWhCp05MXRw8jMd4P4epmRgStDnPcaFN9kaABdmHHKO2UxSumqZly/sjp+bXLGqiK770b03VAKjZz1fgUtXhJcACVQIkoNbP3Ui6epk25xBAnRIH+bP2CR8W4gCijptSPGZA+8a0Bzbw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733720225; c=relaxed/simple; bh=G8HcVy3gmCxQzbHnywh0N7xLf8OEIhHMqK+oUpShbT0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gG31eB+v+Gg5EjSwPepu9jPbo/fIYLv447G397AJhIEmDKk92o7HMnm41/xwzhzviU/jLDj42kc0HjcYZ77HRNqS7kt6TMxfJQ9FelsWSgDkBzuMRgmyi4gVW7zjki/SxTU20KxXdzIhNcWwlYK79hTGehwhhFasA1wssigo7zU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UfTyJmDO; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UfTyJmDO" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A6C15C4CEE8; Mon, 9 Dec 2024 04:57:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1733720225; bh=G8HcVy3gmCxQzbHnywh0N7xLf8OEIhHMqK+oUpShbT0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UfTyJmDOn+A24o0bKvk1bMD8JDODL/95wm6oOFgouqqCyaxlrlhkI+Gi/xTIWQMhJ 6LxShd4NxLTaJ0sDZvSaYSYXKAXdhf8ueK7iAfX/hMSBSrS2JFHfM2pT/aOeoGfSHy AhDXY7awg7UtpAr8TZmL0/Orn5B7ic/XWyjpt73UhH3n5BOwbtNW9EVGKY71/jCS5N x1rPOEIKciv/XvdWvH3u889XDLWitXvjQBtoJ0wGwMl1bZnjWup5Xc03zVkaSKDE29 s8NBJDMNkDk7FIkUZaOZYFWvUx1ch/V61BgM29I2g126zmTjihx8RiMlJyI4sJWGPu hD/o3WU/MuEtQ== From: Eric Biggers To: linux-block@vger.kernel.org, linux-fscrypt@vger.kernel.org, linux-mmc@vger.kernel.org, linux-scsi@vger.kernel.org, linux-arm-msm@vger.kernel.org, Bartosz Golaszewski , Gaurav Kashyap Cc: Adrian Hunter , Alim Akhtar , Avri Altman , Bart Van Assche , Bjorn Andersson , Dmitry Baryshkov , "James E . J . Bottomley" , Jens Axboe , Konrad Dybcio , Manivannan Sadhasivam , "Martin K . Petersen" , Ulf Hansson Subject: [PATCH v9 09/12] blk-crypto: add ioctls to create and prepare hardware-wrapped keys Date: Sun, 8 Dec 2024 20:55:27 -0800 Message-ID: <20241209045530.507833-10-ebiggers@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241209045530.507833-1-ebiggers@kernel.org> References: <20241209045530.507833-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Eric Biggers Until this point, the kernel can use hardware-wrapped keys to do encryption if userspace provides one -- specifically a key in ephemerally-wrapped form. However, no generic way has been provided for userspace to get such a key in the first place. Getting such a key is a two-step process. First, the key needs to be imported from a raw key or generated by the hardware, producing a key in long-term wrapped form. This happens once in the whole lifetime of the key. Second, the long-term wrapped key needs to be converted into ephemerally-wrapped form. This happens each time the key is "unlocked". In Android, these operations are supported in a generic way through KeyMint, a userspace abstraction layer. However, that method is Android-specific and can't be used on other Linux systems, may rely on proprietary libraries, and also misleads people into supporting KeyMint features like rollback resistance that make sense for other KeyMint keys but don't make sense for hardware-wrapped inline encryption keys. Therefore, this patch provides a generic kernel interface for these operations by introducing new block device ioctls: - BLKCRYPTOIMPORTKEY: convert a raw key to long-term wrapped form. - BLKCRYPTOGENERATEKEY: have the hardware generate a new key, then return it in long-term wrapped form. - BLKCRYPTOPREPAREKEY: convert a key from long-term wrapped form to ephemerally-wrapped form. These ioctls are implemented using new operations in blk_crypto_ll_ops. Signed-off-by: Eric Biggers --- Documentation/block/inline-encryption.rst | 32 ++++ .../userspace-api/ioctl/ioctl-number.rst | 2 + block/blk-crypto-internal.h | 9 ++ block/blk-crypto-profile.c | 57 +++++++ block/blk-crypto.c | 143 ++++++++++++++++++ block/ioctl.c | 5 + include/linux/blk-crypto-profile.h | 53 +++++++ include/linux/blk-crypto.h | 1 + include/uapi/linux/blk-crypto.h | 44 ++++++ include/uapi/linux/fs.h | 6 +- 10 files changed, 348 insertions(+), 4 deletions(-) create mode 100644 include/uapi/linux/blk-crypto.h diff --git a/Documentation/block/inline-encryption.rst b/Documentation/block/inline-encryption.rst index f03bd5b090d89..004b230c80ada 100644 --- a/Documentation/block/inline-encryption.rst +++ b/Documentation/block/inline-encryption.rst @@ -490,10 +490,42 @@ when hardware support is available. This works in the following way: keys being passed to these methods. blk-crypto-fallback doesn't support hardware-wrapped keys. Therefore, hardware-wrapped keys can only be used with actual inline encryption hardware. +All the above deals with hardware-wrapped keys in ephemerally-wrapped form only. +To get such keys in the first place, new block device ioctls have been added to +provide a generic interface to creating and preparing such keys: + +- ``BLKCRYPTOIMPORTKEY`` converts a raw key to long-term wrapped form. It takes + in a pointer to a ``struct blk_crypto_import_key_arg``. The caller must set + ``raw_key_ptr`` and ``raw_key_size`` to the pointer and size (in bytes) of the + raw key to import. On success, ``BLKCRYPTOIMPORTKEY`` returns 0 and writes + the resulting long-term wrapped key blob to the buffer pointed to by + ``lt_key_ptr``, which is of maximum size ``lt_key_size``. It also updates + ``lt_key_size`` to be the actual size of the key. On failure, it returns -1 + and sets errno. + +- ``BLKCRYPTOGENERATEKEY`` is like ``BLKCRYPTOIMPORTKEY``, but it has the + hardware generate the key instead of importing one. It takes in a pointer to + a ``struct blk_crypto_generate_key_arg``. + +- ``BLKCRYPTOPREPAREKEY`` converts a key from long-term wrapped form to + ephemerally-wrapped form. It takes in a pointer to a ``struct + blk_crypto_prepare_key_arg``. The caller must set ``lt_key_ptr`` and + ``lt_key_size`` to the pointer and size (in bytes) of the long-term wrapped + key blob to convert. On success, ``BLKCRYPTOPREPAREKEY`` returns 0 and writes + the resulting ephemerally-wrapped key blob to the buffer pointed to by + ``eph_key_ptr``, which is of maximum size ``eph_key_size``. It also updates + ``eph_key_size`` to be the actual size of the key. On failure, it returns -1 + and sets errno. + +Userspace needs to use either ``BLKCRYPTOIMPORTKEY`` or ``BLKCRYPTOGENERATEKEY`` +once to create a key, and then ``BLKCRYPTOPREPAREKEY`` each time the key is +unlocked and added to the kernel. Note that these ioctls have no relevance for +raw keys; they are only for hardware-wrapped keys. + Testability ----------- Both the hardware KDF and the inline encryption itself are well-defined algorithms that don't depend on any secrets other than the unwrapped key. diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst index 243f1f1b554a2..b9d385e3c7bc5 100644 --- a/Documentation/userspace-api/ioctl/ioctl-number.rst +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst @@ -83,10 +83,12 @@ Code Seq# Include File Comments 0x10 00-0F drivers/char/s390/vmcp.h 0x10 10-1F arch/s390/include/uapi/sclp_ctl.h 0x10 20-2F arch/s390/include/uapi/asm/hypfs.h 0x12 all linux/fs.h BLK* ioctls linux/blkpg.h + linux/blkzoned.h + linux/blk-crypto.h 0x15 all linux/fs.h FS_IOC_* ioctls 0x1b all InfiniBand Subsystem 0x20 all drivers/cdrom/cm206.h 0x22 all scsi/sg.h diff --git a/block/blk-crypto-internal.h b/block/blk-crypto-internal.h index 1893df9a8f06c..ccf6dff6ff6be 100644 --- a/block/blk-crypto-internal.h +++ b/block/blk-crypto-internal.h @@ -81,10 +81,13 @@ int __blk_crypto_evict_key(struct blk_crypto_profile *profile, const struct blk_crypto_key *key); bool __blk_crypto_cfg_supported(struct blk_crypto_profile *profile, const struct blk_crypto_config *cfg); +int blk_crypto_ioctl(struct block_device *bdev, unsigned int cmd, + void __user *argp); + #else /* CONFIG_BLK_INLINE_ENCRYPTION */ static inline int blk_crypto_sysfs_register(struct gendisk *disk) { return 0; @@ -128,10 +131,16 @@ static inline bool blk_crypto_rq_is_encrypted(struct request *rq) static inline bool blk_crypto_rq_has_keyslot(struct request *rq) { return false; } +static inline int blk_crypto_ioctl(struct block_device *bdev, unsigned int cmd, + void __user *argp) +{ + return -ENOTTY; +} + #endif /* CONFIG_BLK_INLINE_ENCRYPTION */ void __bio_crypt_advance(struct bio *bio, unsigned int bytes); static inline void bio_crypt_advance(struct bio *bio, unsigned int bytes) { diff --git a/block/blk-crypto-profile.c b/block/blk-crypto-profile.c index 1b92276ed2fcc..f6419502fcbee 100644 --- a/block/blk-crypto-profile.c +++ b/block/blk-crypto-profile.c @@ -500,10 +500,67 @@ int blk_crypto_derive_sw_secret(struct block_device *bdev, sw_secret); blk_crypto_hw_exit(profile); return err; } +int blk_crypto_import_key(struct blk_crypto_profile *profile, + const u8 *raw_key, size_t raw_key_size, + u8 lt_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]) +{ + int ret; + + if (!profile) + return -EOPNOTSUPP; + if (!(profile->key_types_supported & BLK_CRYPTO_KEY_TYPE_HW_WRAPPED)) + return -EOPNOTSUPP; + if (!profile->ll_ops.import_key) + return -EOPNOTSUPP; + blk_crypto_hw_enter(profile); + ret = profile->ll_ops.import_key(profile, raw_key, raw_key_size, + lt_key); + blk_crypto_hw_exit(profile); + return ret; +} + +int blk_crypto_generate_key(struct blk_crypto_profile *profile, + u8 lt_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]) +{ + int ret; + + if (!profile) + return -EOPNOTSUPP; + if (!(profile->key_types_supported & BLK_CRYPTO_KEY_TYPE_HW_WRAPPED)) + return -EOPNOTSUPP; + if (!profile->ll_ops.generate_key) + return -EOPNOTSUPP; + + blk_crypto_hw_enter(profile); + ret = profile->ll_ops.generate_key(profile, lt_key); + blk_crypto_hw_exit(profile); + return ret; +} + +int blk_crypto_prepare_key(struct blk_crypto_profile *profile, + const u8 *lt_key, size_t lt_key_size, + u8 eph_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]) +{ + int ret; + + if (!profile) + return -EOPNOTSUPP; + if (!(profile->key_types_supported & BLK_CRYPTO_KEY_TYPE_HW_WRAPPED)) + return -EOPNOTSUPP; + if (!profile->ll_ops.prepare_key) + return -EOPNOTSUPP; + + blk_crypto_hw_enter(profile); + ret = profile->ll_ops.prepare_key(profile, lt_key, lt_key_size, + eph_key); + blk_crypto_hw_exit(profile); + return ret; +} + /** * blk_crypto_intersect_capabilities() - restrict supported crypto capabilities * by child device * @parent: the crypto profile for the parent device * @child: the crypto profile for the child device, or NULL diff --git a/block/blk-crypto.c b/block/blk-crypto.c index b55b3d8bffa00..2f6e0294eddc1 100644 --- a/block/blk-crypto.c +++ b/block/blk-crypto.c @@ -465,5 +465,148 @@ void blk_crypto_evict_key(struct block_device *bdev, */ if (err) pr_warn_ratelimited("%pg: error %d evicting key\n", bdev, err); } EXPORT_SYMBOL_GPL(blk_crypto_evict_key); + +static int blk_crypto_ioctl_import_key(struct blk_crypto_profile *profile, + void __user *argp) +{ + struct blk_crypto_import_key_arg arg; + u8 raw_key[BLK_CRYPTO_MAX_RAW_KEY_SIZE]; + u8 lt_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]; + int ret; + + if (copy_from_user(&arg, argp, sizeof(arg))) + return -EFAULT; + + if (memchr_inv(arg.reserved, 0, sizeof(arg.reserved))) + return -EINVAL; + + if (arg.raw_key_size < 16 || arg.raw_key_size > sizeof(raw_key)) + return -EINVAL; + + if (copy_from_user(raw_key, u64_to_user_ptr(arg.raw_key_ptr), + arg.raw_key_size)) { + ret = -EFAULT; + goto out; + } + ret = blk_crypto_import_key(profile, raw_key, arg.raw_key_size, lt_key); + if (ret < 0) + goto out; + if (ret > arg.lt_key_size) { + ret = -EOVERFLOW; + goto out; + } + arg.lt_key_size = ret; + if (copy_to_user(u64_to_user_ptr(arg.lt_key_ptr), lt_key, + arg.lt_key_size) || + copy_to_user(argp, &arg, sizeof(arg))) { + ret = -EFAULT; + goto out; + } + ret = 0; + +out: + memzero_explicit(raw_key, sizeof(raw_key)); + memzero_explicit(lt_key, sizeof(lt_key)); + return ret; +} + +static int blk_crypto_ioctl_generate_key(struct blk_crypto_profile *profile, + void __user *argp) +{ + struct blk_crypto_generate_key_arg arg; + u8 lt_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]; + int ret; + + if (copy_from_user(&arg, argp, sizeof(arg))) + return -EFAULT; + + if (memchr_inv(arg.reserved, 0, sizeof(arg.reserved))) + return -EINVAL; + + ret = blk_crypto_generate_key(profile, lt_key); + if (ret < 0) + goto out; + if (ret > arg.lt_key_size) { + ret = -EOVERFLOW; + goto out; + } + arg.lt_key_size = ret; + if (copy_to_user(u64_to_user_ptr(arg.lt_key_ptr), lt_key, + arg.lt_key_size) || + copy_to_user(argp, &arg, sizeof(arg))) { + ret = -EFAULT; + goto out; + } + ret = 0; + +out: + memzero_explicit(lt_key, sizeof(lt_key)); + return ret; +} + +static int blk_crypto_ioctl_prepare_key(struct blk_crypto_profile *profile, + void __user *argp) +{ + struct blk_crypto_prepare_key_arg arg; + u8 lt_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]; + u8 eph_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]; + int ret; + + if (copy_from_user(&arg, argp, sizeof(arg))) + return -EFAULT; + + if (memchr_inv(arg.reserved, 0, sizeof(arg.reserved))) + return -EINVAL; + + if (arg.lt_key_size > sizeof(lt_key)) + return -EINVAL; + + if (copy_from_user(lt_key, u64_to_user_ptr(arg.lt_key_ptr), + arg.lt_key_size)) { + ret = -EFAULT; + goto out; + } + ret = blk_crypto_prepare_key(profile, lt_key, arg.lt_key_size, eph_key); + if (ret < 0) + goto out; + if (ret > arg.eph_key_size) { + ret = -EOVERFLOW; + goto out; + } + arg.eph_key_size = ret; + if (copy_to_user(u64_to_user_ptr(arg.eph_key_ptr), eph_key, + arg.eph_key_size) || + copy_to_user(argp, &arg, sizeof(arg))) { + ret = -EFAULT; + goto out; + } + ret = 0; + +out: + memzero_explicit(lt_key, sizeof(lt_key)); + memzero_explicit(eph_key, sizeof(eph_key)); + return ret; +} + +int blk_crypto_ioctl(struct block_device *bdev, unsigned int cmd, + void __user *argp) +{ + struct blk_crypto_profile *profile = + bdev_get_queue(bdev)->crypto_profile; + + if (!profile) + return -EOPNOTSUPP; + + switch (cmd) { + case BLKCRYPTOIMPORTKEY: + return blk_crypto_ioctl_import_key(profile, argp); + case BLKCRYPTOGENERATEKEY: + return blk_crypto_ioctl_generate_key(profile, argp); + case BLKCRYPTOPREPAREKEY: + return blk_crypto_ioctl_prepare_key(profile, argp); + default: + return -ENOTTY; + } +} diff --git a/block/ioctl.c b/block/ioctl.c index 6554b728bae6a..faa40f383e273 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -13,10 +13,11 @@ #include #include #include #include #include "blk.h" +#include "blk-crypto-internal.h" static int blkpg_do_ioctl(struct block_device *bdev, struct blkpg_partition __user *upart, int op) { struct gendisk *disk = bdev->bd_disk; @@ -618,10 +619,14 @@ static int blkdev_common_ioctl(struct block_device *bdev, blk_mode_t mode, mode | BLK_OPEN_STRICT_SCAN); case BLKTRACESTART: case BLKTRACESTOP: case BLKTRACETEARDOWN: return blk_trace_ioctl(bdev, cmd, argp); + case BLKCRYPTOIMPORTKEY: + case BLKCRYPTOGENERATEKEY: + case BLKCRYPTOPREPAREKEY: + return blk_crypto_ioctl(bdev, cmd, argp); case IOC_PR_REGISTER: return blkdev_pr_register(bdev, mode, argp); case IOC_PR_RESERVE: return blkdev_pr_reserve(bdev, mode, argp); case IOC_PR_RELEASE: diff --git a/include/linux/blk-crypto-profile.h b/include/linux/blk-crypto-profile.h index 7764b4f7b45b9..a719a0aea122f 100644 --- a/include/linux/blk-crypto-profile.h +++ b/include/linux/blk-crypto-profile.h @@ -69,10 +69,52 @@ struct blk_crypto_ll_ops { * -errno code on other errors. */ int (*derive_sw_secret)(struct blk_crypto_profile *profile, const u8 *eph_key, size_t eph_key_size, u8 sw_secret[BLK_CRYPTO_SW_SECRET_SIZE]); + + /** + * @import_key: Create a hardware-wrapped key by importing a raw key. + * + * This only needs to be implemented if BLK_CRYPTO_KEY_TYPE_HW_WRAPPED + * is supported. + * + * On success, must write the new key in long-term wrapped form to + * @lt_key and return its size in bytes. On failure, must return a + * -errno value. + */ + int (*import_key)(struct blk_crypto_profile *profile, + const u8 *raw_key, size_t raw_key_size, + u8 lt_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]); + + /** + * @generate_key: Generate a hardware-wrapped key. + * + * This only needs to be implemented if BLK_CRYPTO_KEY_TYPE_HW_WRAPPED + * is supported. + * + * On success, must write the new key in long-term wrapped form to + * @lt_key and return its size in bytes. On failure, must return a + * -errno value. + */ + int (*generate_key)(struct blk_crypto_profile *profile, + u8 lt_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]); + + /** + * @prepare_key: Prepare a hardware-wrapped key to be used. + * + * Prepare a hardware-wrapped key to be used by converting it from + * long-term wrapped form to ephemerally-wrapped form. This only needs + * to be implemented if BLK_CRYPTO_KEY_TYPE_HW_WRAPPED is supported. + * + * On success, must write the key in ephemerally-wrapped form to + * @eph_key and return its size in bytes. On failure, must return a + * -errno value. + */ + int (*prepare_key)(struct blk_crypto_profile *profile, + const u8 *lt_key, size_t lt_key_size, + u8 eph_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]); }; /** * struct blk_crypto_profile - inline encryption profile for a device * @@ -161,10 +203,21 @@ unsigned int blk_crypto_keyslot_index(struct blk_crypto_keyslot *slot); void blk_crypto_reprogram_all_keys(struct blk_crypto_profile *profile); void blk_crypto_profile_destroy(struct blk_crypto_profile *profile); +int blk_crypto_import_key(struct blk_crypto_profile *profile, + const u8 *raw_key, size_t raw_key_size, + u8 lt_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]); + +int blk_crypto_generate_key(struct blk_crypto_profile *profile, + u8 lt_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]); + +int blk_crypto_prepare_key(struct blk_crypto_profile *profile, + const u8 *lt_key, size_t lt_key_size, + u8 eph_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]); + void blk_crypto_intersect_capabilities(struct blk_crypto_profile *parent, const struct blk_crypto_profile *child); bool blk_crypto_has_capabilities(const struct blk_crypto_profile *target, const struct blk_crypto_profile *reference); diff --git a/include/linux/blk-crypto.h b/include/linux/blk-crypto.h index 0e63287e2175f..c1ef8c3cea64e 100644 --- a/include/linux/blk-crypto.h +++ b/include/linux/blk-crypto.h @@ -6,10 +6,11 @@ #ifndef __LINUX_BLK_CRYPTO_H #define __LINUX_BLK_CRYPTO_H #include #include +#include enum blk_crypto_mode_num { BLK_ENCRYPTION_MODE_INVALID, BLK_ENCRYPTION_MODE_AES_256_XTS, BLK_ENCRYPTION_MODE_AES_128_CBC_ESSIV, diff --git a/include/uapi/linux/blk-crypto.h b/include/uapi/linux/blk-crypto.h new file mode 100644 index 0000000000000..97302c6eb6afe --- /dev/null +++ b/include/uapi/linux/blk-crypto.h @@ -0,0 +1,44 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +#ifndef _UAPI_LINUX_BLK_CRYPTO_H +#define _UAPI_LINUX_BLK_CRYPTO_H + +#include +#include + +struct blk_crypto_import_key_arg { + /* Raw key (input) */ + __u64 raw_key_ptr; + __u64 raw_key_size; + /* Long-term wrapped key blob (output) */ + __u64 lt_key_ptr; + __u64 lt_key_size; + __u64 reserved[4]; +}; + +struct blk_crypto_generate_key_arg { + /* Long-term wrapped key blob (output) */ + __u64 lt_key_ptr; + __u64 lt_key_size; + __u64 reserved[4]; +}; + +struct blk_crypto_prepare_key_arg { + /* Long-term wrapped key blob (input) */ + __u64 lt_key_ptr; + __u64 lt_key_size; + /* Ephemerally-wrapped key blob (output) */ + __u64 eph_key_ptr; + __u64 eph_key_size; + __u64 reserved[4]; +}; + +/* + * These ioctls share the block device ioctl space; see uapi/linux/fs.h. + * 140-141 are reserved for future blk-crypto ioctls; any more than that would + * require an additional allocation from the block device ioctl space. + */ +#define BLKCRYPTOIMPORTKEY _IOWR(0x12, 137, struct blk_crypto_import_key_arg) +#define BLKCRYPTOGENERATEKEY _IOWR(0x12, 138, struct blk_crypto_generate_key_arg) +#define BLKCRYPTOPREPAREKEY _IOWR(0x12, 139, struct blk_crypto_prepare_key_arg) + +#endif /* _UAPI_LINUX_BLK_CRYPTO_H */ diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 7539717707337..07180da44e13d 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -201,14 +201,12 @@ struct fsxattr { #define BLKDISCARDZEROES _IO(0x12,124) #define BLKSECDISCARD _IO(0x12,125) #define BLKROTATIONAL _IO(0x12,126) #define BLKZEROOUT _IO(0x12,127) #define BLKGETDISKSEQ _IOR(0x12,128,__u64) -/* - * A jump here: 130-136 are reserved for zoned block devices - * (see uapi/linux/blkzoned.h) - */ +/* 130-136 are used by zoned block device ioctls (uapi/linux/blkzoned.h) */ +/* 137-141 are used by blk-crypto ioctls (uapi/linux/blk-crypto.h) */ #define BMAP_IOCTL 1 /* obsolete - kept for compatibility */ #define FIBMAP _IO(0x00,1) /* bmap access */ #define FIGETBSZ _IO(0x00,2) /* get the block size used for bmap */ #define FIFREEZE _IOWR('X', 119, int) /* Freeze */ From patchwork Mon Dec 9 04:55:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 850002 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 29CE419C566; Mon, 9 Dec 2024 04:57:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733720227; cv=none; b=Ou9JykLgt5jCg2xJRMaejswEcFgWbrIsY5gN5Zkf5hXTOENHgTcTZpzK/8ktd55heqfKQkheQcl7KpdtzvTcos8d7Fb+QUbgfkZDmoeWd/6aD8CMRYnOYjI8ZxwN0wtL48v8UPWey9mzwZRFKf8JkUd9RkDJq3HxjUcBeseax7c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733720227; c=relaxed/simple; bh=yUfSugXPgQzw+K+tiazu/rUo5KvmiZ91WFLNZDhbTLw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DHxpJDKjXTQRNa7XrUe7y0vA4aekFHCd5+2MAl4ynJ2hpK4AkoZsUM7B+yAKFyPllSE4NvHogKe0Hgrego15AWb/1FZMFcv2Bfg75gEspw43oAIxSdATdV67e0/OfH6AE3hV+rpopU4LluiC5ORT7fO4t8Lm+WeKPpQWsqK7kuA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=l6piNQnD; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="l6piNQnD" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 18562C4CEE2; Mon, 9 Dec 2024 04:57:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1733720226; bh=yUfSugXPgQzw+K+tiazu/rUo5KvmiZ91WFLNZDhbTLw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=l6piNQnDXfsUCyrOIgqH2GLk6nqxQu2rZt33o7Es8VX023V3BDfndhlgz76PMD2c+ 1JmJZyXdJ2ANz/OAIsK9kw8JM9Hsl5DkGPQmN1ooD9hxaUdjY0MwFAMZiBEuUm2Tv0 Zoq4IGy6o49ySikkZI6NUueJFqCCLbXg262Fkn2liue9rDGVXIywt5xHw9miK7C8H4 r/p9zsg5t/mhoT1IEBTJgh86whSA7zWxh+Lpe8tsdhQmLlQ4TMLcJ6XYr21AIpdFQb 5uBTqTVQ2Yt8hdy7OI1G9DMI72WsSHdNkjygw34k2Iv2mUp/TepING7/6KylnE9mlU 17P3GbrxOxsIg== From: Eric Biggers To: linux-block@vger.kernel.org, linux-fscrypt@vger.kernel.org, linux-mmc@vger.kernel.org, linux-scsi@vger.kernel.org, linux-arm-msm@vger.kernel.org, Bartosz Golaszewski , Gaurav Kashyap Cc: Adrian Hunter , Alim Akhtar , Avri Altman , Bart Van Assche , Bjorn Andersson , Dmitry Baryshkov , "James E . J . Bottomley" , Jens Axboe , Konrad Dybcio , Manivannan Sadhasivam , "Martin K . Petersen" , Ulf Hansson , Bartosz Golaszewski Subject: [PATCH v9 11/12] soc: qcom: ice: add HWKM support to the ICE driver Date: Sun, 8 Dec 2024 20:55:29 -0800 Message-ID: <20241209045530.507833-12-ebiggers@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241209045530.507833-1-ebiggers@kernel.org> References: <20241209045530.507833-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Gaurav Kashyap Qualcomm's ICE (Inline Crypto Engine) contains a proprietary key management hardware called Hardware Key Manager (HWKM). Add HWKM support to the ICE driver if it is available on the platform. HWKM primarily provides hardware wrapped key support where the ICE (storage) keys are not available in software and instead protected in hardware. When HWKM software support is not fully available (from Trustzone), there can be a scenario where the ICE hardware supports HWKM, but it cannot be used for wrapped keys. In this case, raw keys have to be used without using the HWKM. We query the TZ at run-time to find out whether wrapped keys support is available. The selection of HWKM vs non-HWKM mode has to be made at boot time, so add a module parameter qcom_ice.use_wrapped_keys=1 that enables HWKM. Signed-off-by: Gaurav Kashyap Signed-off-by: Bartosz Golaszewski [EB: merged related patches, fixed the module parameter to work correctly, fixed error handling, improved log messages, improved comments, improved commit message, fixed various names.] Signed-off-by: Eric Biggers --- drivers/mmc/host/sdhci-msm.c | 5 + drivers/soc/qcom/ice.c | 360 ++++++++++++++++++++++++++++++++++- drivers/ufs/host/ufs-qcom.c | 5 + include/soc/qcom/ice.h | 12 ++ 4 files changed, 377 insertions(+), 5 deletions(-) diff --git a/drivers/mmc/host/sdhci-msm.c b/drivers/mmc/host/sdhci-msm.c index 27b24fb8e8809..ac79d9e18b3cf 100644 --- a/drivers/mmc/host/sdhci-msm.c +++ b/drivers/mmc/host/sdhci-msm.c @@ -1833,10 +1833,15 @@ static int sdhci_msm_ice_init(struct sdhci_msm_host *msm_host, if (IS_ERR_OR_NULL(ice)) return PTR_ERR_OR_ZERO(ice); msm_host->ice = ice; + if (qcom_ice_using_hwkm(ice)) { + dev_warn(dev, "HWKM mode unsupported; disabling inline encryption support\n"); + return 0; + } + /* Initialize the blk_crypto_profile */ caps.reg_val = cpu_to_le32(cqhci_readl(cq_host, CQHCI_CCAP)); /* The number of keyslots supported is (CFGC+1) */ diff --git a/drivers/soc/qcom/ice.c b/drivers/soc/qcom/ice.c index 78780fd508f0b..bfdda3339dfe9 100644 --- a/drivers/soc/qcom/ice.c +++ b/drivers/soc/qcom/ice.c @@ -20,34 +20,106 @@ #include #define AES_256_XTS_KEY_SIZE 64 +/* + * Wrapped key sizes that HWKM expects and manages is different for different + * versions of the hardware. + */ +#define QCOM_ICE_HWKM_WRAPPED_KEY_SIZE(v) \ + ((v) == 1 ? 68 : 100) + /* QCOM ICE registers */ #define QCOM_ICE_REG_VERSION 0x0008 #define QCOM_ICE_REG_FUSE_SETTING 0x0010 #define QCOM_ICE_REG_BIST_STATUS 0x0070 #define QCOM_ICE_REG_ADVANCED_CONTROL 0x1000 +#define QCOM_ICE_REG_CONTROL 0x0 +#define QCOM_ICE_LUT_KEYS_CRYPTOCFG_R16 0x4040 + +/* QCOM ICE HWKM registers */ +#define QCOM_ICE_REG_HWKM_TZ_KM_CTL 0x1000 +#define QCOM_ICE_REG_HWKM_TZ_KM_STATUS 0x1004 +#define QCOM_ICE_REG_HWKM_BANK0_BANKN_IRQ_STATUS 0x2008 +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_0 0x5000 +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_1 0x5004 +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_2 0x5008 +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_3 0x500C +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_4 0x5010 + +/* QCOM ICE HWKM reg vals */ +#define QCOM_ICE_HWKM_BIST_DONE_V1 BIT(16) +#define QCOM_ICE_HWKM_BIST_DONE_V2 BIT(9) +#define QCOM_ICE_HWKM_BIST_DONE(ver) QCOM_ICE_HWKM_BIST_DONE_V##ver + +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V1 BIT(14) +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V2 BIT(7) +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE(v) QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V##v + +#define QCOM_ICE_HWKM_BOOT_CMD_LIST1_DONE BIT(2) +#define QCOM_ICE_HWKM_BOOT_CMD_LIST0_DONE BIT(1) +#define QCOM_ICE_HWKM_KT_CLEAR_DONE BIT(0) + +#define QCOM_ICE_HWKM_BIST_VAL(v) (QCOM_ICE_HWKM_BIST_DONE(v) | \ + QCOM_ICE_HWKM_CRYPTO_BIST_DONE(v) | \ + QCOM_ICE_HWKM_BOOT_CMD_LIST1_DONE | \ + QCOM_ICE_HWKM_BOOT_CMD_LIST0_DONE | \ + QCOM_ICE_HWKM_KT_CLEAR_DONE) + +#define QCOM_ICE_HWKM_V1_STANDARD_MODE_VAL (BIT(0) | BIT(1) | BIT(2)) +#define QCOM_ICE_HWKM_V2_STANDARD_MODE_MASK GENMASK(31, 1) +#define QCOM_ICE_HWKM_DISABLE_CRC_CHECKS_VAL (BIT(1) | BIT(2)) +#define QCOM_ICE_HWKM_RSP_FIFO_CLEAR_VAL BIT(3) + +#define QCOM_ICE_HWKM_CFG_ENABLE_VAL BIT(7) /* BIST ("built-in self-test") status flags */ #define QCOM_ICE_BIST_STATUS_MASK GENMASK(31, 28) #define QCOM_ICE_FUSE_SETTING_MASK 0x1 #define QCOM_ICE_FORCE_HW_KEY0_SETTING_MASK 0x2 #define QCOM_ICE_FORCE_HW_KEY1_SETTING_MASK 0x4 +#define QCOM_ICE_LUT_KEYS_CRYPTOCFG_OFFSET 0x80 + +#define QCOM_ICE_HWKM_REG_OFFSET 0x8000 +#define HWKM_OFFSET(reg) ((reg) + QCOM_ICE_HWKM_REG_OFFSET) + #define qcom_ice_writel(engine, val, reg) \ writel((val), (engine)->base + (reg)) #define qcom_ice_readl(engine, reg) \ readl((engine)->base + (reg)) +#define QCOM_ICE_LUT_CRYPTOCFG_SLOT_OFFSET(slot) \ + (QCOM_ICE_LUT_KEYS_CRYPTOCFG_R16 + \ + QCOM_ICE_LUT_KEYS_CRYPTOCFG_OFFSET * slot) + +static bool qcom_ice_use_wrapped_keys; +module_param_named(use_wrapped_keys, qcom_ice_use_wrapped_keys, bool, 0660); +MODULE_PARM_DESC(use_wrapped_keys, + "Use HWKM for wrapped keys support if available on the platform"); + struct qcom_ice { struct device *dev; void __iomem *base; struct clk *core_clk; + u8 hwkm_version; + bool use_hwkm; + bool hwkm_init_complete; +}; + +union crypto_cfg { + __le32 regval; + struct { + u8 dusize; + u8 capidx; + u8 reserved; + u8 cfge; + }; }; static bool qcom_ice_check_supported(struct qcom_ice *ice) { u32 regval = qcom_ice_readl(ice, QCOM_ICE_REG_VERSION); @@ -61,12 +133,22 @@ static bool qcom_ice_check_supported(struct qcom_ice *ice) dev_warn(dev, "Unsupported ICE version: v%d.%d.%d\n", major, minor, step); return false; } - dev_info(dev, "Found QC Inline Crypto Engine (ICE) v%d.%d.%d\n", - major, minor, step); + if (major >= 4 || (major == 3 && minor == 2 && step >= 1)) + ice->hwkm_version = 2; + else if (major == 3 && minor == 2) + ice->hwkm_version = 1; + else + ice->hwkm_version = 0; + + if (ice->hwkm_version == 0) + ice->use_hwkm = false; + + dev_info(dev, "Found QC Inline Crypto Engine (ICE) v%d.%d.%d, HWKM v%d\n", + major, minor, step, ice->hwkm_version); /* If fuses are blown, ICE might not work in the standard way. */ regval = qcom_ice_readl(ice, QCOM_ICE_REG_FUSE_SETTING); if (regval & (QCOM_ICE_FUSE_SETTING_MASK | QCOM_ICE_FORCE_HW_KEY0_SETTING_MASK | @@ -111,31 +193,109 @@ static void qcom_ice_optimization_enable(struct qcom_ice *ice) * because (a) the BIST is a FIPS compliance thing that never fails in * practice, (b) ICE is documented to reject crypto requests if the BIST * fails, so we needn't do it in software too, and (c) properly testing * storage encryption requires testing the full storage stack anyway, * and not relying on hardware-level self-tests. + * + * However, we still care about if HWKM BIST failed (when supported) as + * important functionality would fail later, so disable hwkm on failure. */ static int qcom_ice_wait_bist_status(struct qcom_ice *ice) { u32 regval; + u32 bist_done_val; int err; err = readl_poll_timeout(ice->base + QCOM_ICE_REG_BIST_STATUS, regval, !(regval & QCOM_ICE_BIST_STATUS_MASK), 50, 5000); - if (err) + if (err) { dev_err(ice->dev, "Timed out waiting for ICE self-test to complete\n"); + return err; + } + if (ice->use_hwkm) { + bist_done_val = ice->hwkm_version == 1 ? + QCOM_ICE_HWKM_BIST_VAL(1) : + QCOM_ICE_HWKM_BIST_VAL(2); + if (qcom_ice_readl(ice, + HWKM_OFFSET(QCOM_ICE_REG_HWKM_TZ_KM_STATUS)) != + bist_done_val) { + dev_err(ice->dev, "HWKM BIST error\n"); + ice->use_hwkm = false; + err = -ENODEV; + } + } return err; } +static void qcom_ice_enable_hwkm_mode(struct qcom_ice *ice) +{ + u32 val = 0; + + /* + * When ICE is in standard (hwkm) mode, it supports HW wrapped + * keys, and when it is in legacy mode, it only supports raw keys. + * + * Put ICE in standard mode, ICE defaults to legacy mode. + * Legacy mode - ICE HWKM slave not supported. + * Standard mode - ICE HWKM slave supported. + * + * Depending on the version of HWKM, it is controlled by different + * registers in ICE. + */ + if (ice->hwkm_version >= 2) { + val = qcom_ice_readl(ice, QCOM_ICE_REG_CONTROL); + val = val & QCOM_ICE_HWKM_V2_STANDARD_MODE_MASK; + qcom_ice_writel(ice, val, QCOM_ICE_REG_CONTROL); + } else { + qcom_ice_writel(ice, QCOM_ICE_HWKM_V1_STANDARD_MODE_VAL, + HWKM_OFFSET(QCOM_ICE_REG_HWKM_TZ_KM_CTL)); + } +} + +static void qcom_ice_hwkm_init(struct qcom_ice *ice) +{ + /* Disable CRC checks. This HWKM feature is not used. */ + qcom_ice_writel(ice, QCOM_ICE_HWKM_DISABLE_CRC_CHECKS_VAL, + HWKM_OFFSET(QCOM_ICE_REG_HWKM_TZ_KM_CTL)); + + /* + * Give register bank of the HWKM slave access to read and modify + * the keyslots in ICE HWKM slave. Without this, trustzone will not + * be able to program keys into ICE. + */ + qcom_ice_writel(ice, GENMASK(31, 0), HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BBAC_0)); + qcom_ice_writel(ice, GENMASK(31, 0), HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BBAC_1)); + qcom_ice_writel(ice, GENMASK(31, 0), HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BBAC_2)); + qcom_ice_writel(ice, GENMASK(31, 0), HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BBAC_3)); + qcom_ice_writel(ice, GENMASK(31, 0), HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BBAC_4)); + + /* Clear HWKM response FIFO before doing anything */ + qcom_ice_writel(ice, QCOM_ICE_HWKM_RSP_FIFO_CLEAR_VAL, + HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BANKN_IRQ_STATUS)); + ice->hwkm_init_complete = true; +} + int qcom_ice_enable(struct qcom_ice *ice) { + int err; + qcom_ice_low_power_mode_enable(ice); qcom_ice_optimization_enable(ice); - return qcom_ice_wait_bist_status(ice); + if (ice->use_hwkm) + qcom_ice_enable_hwkm_mode(ice); + + err = qcom_ice_wait_bist_status(ice); + if (err) + return err; + + if (ice->use_hwkm) + qcom_ice_hwkm_init(ice); + + return err; } EXPORT_SYMBOL_GPL(qcom_ice_enable); int qcom_ice_resume(struct qcom_ice *ice) { @@ -147,22 +307,71 @@ int qcom_ice_resume(struct qcom_ice *ice) dev_err(dev, "failed to enable core clock (%d)\n", err); return err; } + if (ice->use_hwkm) { + qcom_ice_enable_hwkm_mode(ice); + qcom_ice_hwkm_init(ice); + } return qcom_ice_wait_bist_status(ice); } EXPORT_SYMBOL_GPL(qcom_ice_resume); int qcom_ice_suspend(struct qcom_ice *ice) { clk_disable_unprepare(ice->core_clk); + ice->hwkm_init_complete = false; return 0; } EXPORT_SYMBOL_GPL(qcom_ice_suspend); +/* For v1 the ICE slot is calculated in TrustZone. */ +static int translate_hwkm_slot(struct qcom_ice *ice, int slot) +{ + return (ice->hwkm_version == 1) ? slot : (slot * 2); +} + +static int qcom_ice_program_wrapped_key(struct qcom_ice *ice, unsigned int slot, + const struct blk_crypto_key *bkey) +{ + struct device *dev = ice->dev; + union crypto_cfg cfg = { + .dusize = bkey->crypto_cfg.data_unit_size / 512, + .capidx = QCOM_SCM_ICE_CIPHER_AES_256_XTS, + .cfge = QCOM_ICE_HWKM_CFG_ENABLE_VAL, + }; + int hwkm_slot; + int err; + + /* It is expected that HWKM init has completed before programming wrapped keys */ + if (!ice->use_hwkm || !ice->hwkm_init_complete) { + dev_err_ratelimited(dev, "HWKM not currently used or initialized\n"); + return -EINVAL; + } + + hwkm_slot = translate_hwkm_slot(ice, slot); + + /* Clear CFGE */ + qcom_ice_writel(ice, 0x0, QCOM_ICE_LUT_CRYPTOCFG_SLOT_OFFSET(slot)); + + /* Call trustzone to program the wrapped key using hwkm */ + err = qcom_scm_ice_set_key(hwkm_slot, bkey->bytes, bkey->size, + cfg.capidx, cfg.dusize); + if (err) { + pr_err("%s:SCM call Error: 0x%x slot %d\n", __func__, err, + slot); + return err; + } + + /* Enable CFGE after programming key */ + qcom_ice_writel(ice, cfg.regval, QCOM_ICE_LUT_CRYPTOCFG_SLOT_OFFSET(slot)); + + return err; +} + int qcom_ice_program_key(struct qcom_ice *ice, unsigned int slot, const struct blk_crypto_key *blk_key) { struct device *dev = ice->dev; union { @@ -178,10 +387,18 @@ int qcom_ice_program_key(struct qcom_ice *ice, unsigned int slot, dev_err_ratelimited(dev, "Unsupported crypto mode: %d\n", blk_key->crypto_cfg.crypto_mode); return -EINVAL; } + if (blk_key->crypto_cfg.key_type == BLK_CRYPTO_KEY_TYPE_HW_WRAPPED) + return qcom_ice_program_wrapped_key(ice, slot, blk_key); + + if (ice->use_hwkm) { + dev_err_ratelimited(dev, "Unsupported raw key when in HWKM mode\n"); + return -EINVAL; + } + if (blk_key->size != AES_256_XTS_KEY_SIZE) { dev_err_ratelimited(dev, "Incorrect key size\n"); return -EINVAL; } memcpy(key.bytes, blk_key->bytes, AES_256_XTS_KEY_SIZE); @@ -200,14 +417,137 @@ int qcom_ice_program_key(struct qcom_ice *ice, unsigned int slot, } EXPORT_SYMBOL_GPL(qcom_ice_program_key); int qcom_ice_evict_key(struct qcom_ice *ice, int slot) { - return qcom_scm_ice_invalidate_key(slot); + int hwkm_slot = slot; + + if (ice->use_hwkm) { + hwkm_slot = translate_hwkm_slot(ice, slot); + + /* + * Ignore calls to evict key when HWKM is supported and hwkm + * init is not yet done. This is to avoid the clearing all + * slots call during a storage reset when ICE is still in + * legacy mode. HWKM slave in ICE takes care of zeroing out + * the keytable on reset. + */ + if (!ice->hwkm_init_complete) + return 0; + } + + return qcom_scm_ice_invalidate_key(hwkm_slot); } EXPORT_SYMBOL_GPL(qcom_ice_evict_key); +bool qcom_ice_using_hwkm(struct qcom_ice *ice) +{ + return ice->use_hwkm; +} +EXPORT_SYMBOL_GPL(qcom_ice_using_hwkm); + +/* + * Derive a software secret from a hardware-wrapped key. The key is unwrapped in + * hardware from TrustZone and a software key/secret is then derived from it. + */ +int qcom_ice_derive_sw_secret(struct qcom_ice *ice, + const u8 *eph_key, size_t eph_key_size, + u8 sw_secret[BLK_CRYPTO_SW_SECRET_SIZE]) +{ + return qcom_scm_derive_sw_secret(eph_key, eph_key_size, + sw_secret, BLK_CRYPTO_SW_SECRET_SIZE); +} +EXPORT_SYMBOL_GPL(qcom_ice_derive_sw_secret); + +/** + * qcom_ice_generate_key() - Generate a wrapped key for inline encryption + * @ice: ICE driver data + * @lt_key: buffer for the resulting long-term wrapped key + * + * Make an SCM call into TrustZone to generate a wrapped key for storage + * encryption using HWKM. + * + * Return: the size of the resulting wrapped key on success; -errno on failure. + */ +int qcom_ice_generate_key(struct qcom_ice *ice, + u8 lt_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]) +{ + size_t wk_size = QCOM_ICE_HWKM_WRAPPED_KEY_SIZE(ice->hwkm_version); + int err; + + if (WARN_ON_ONCE(wk_size > BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE)) + return -EINVAL; + + err = qcom_scm_generate_ice_key(lt_key, wk_size); + if (err) + return err; + + return wk_size; +} +EXPORT_SYMBOL_GPL(qcom_ice_generate_key); + +/** + * qcom_ice_prepare_key() - Prepare a wrapped key for inline encryption + * @ice: ICE driver data + * @lt_key: longterm wrapped key that was generated or imported. + * @lt_key_size: size of the longterm wrapped_key + * @eph_key: buffer for the resulting ephemerally-wrapped key + * + * Make an SCM call into TrustZone to prepare a wrapped key for storage + * encryption by rewrapping a long-term wrapped key with a per-boot ephemeral + * key using HWKM. + * + * Return: the size of the resulting wrapped key on success; -errno on failure. + */ +int qcom_ice_prepare_key(struct qcom_ice *ice, + const u8 *lt_key, size_t lt_key_size, + u8 eph_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]) +{ + size_t wk_size = QCOM_ICE_HWKM_WRAPPED_KEY_SIZE(ice->hwkm_version); + int err; + + if (WARN_ON_ONCE(wk_size > BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE)) + return -EINVAL; + + err = qcom_scm_prepare_ice_key(lt_key, lt_key_size, eph_key, wk_size); + if (err) + return err; + + return wk_size; +} +EXPORT_SYMBOL_GPL(qcom_ice_prepare_key); + +/** + * qcom_ice_import_key() - Import a raw key for inline encryption + * @ice: ICE driver data + * @raw_key: raw key that will be imported + * @raw_key_size: size of the raw key + * @lt_key: buffer for the resulting long-term wrapped key + * + * Make an SCM call into TrustZone to import a raw key for storage encryption + * and generate a long-term wrapped key using HWKM. + * + * Return: the size of the resulting wrapped key on success; -errno on failure. + */ +int qcom_ice_import_key(struct qcom_ice *ice, + const u8 *raw_key, size_t raw_key_size, + u8 lt_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]) +{ + size_t wk_size = QCOM_ICE_HWKM_WRAPPED_KEY_SIZE(ice->hwkm_version); + int err; + + if (WARN_ON_ONCE(wk_size > BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE)) + return -EINVAL; + + err = qcom_scm_import_ice_key(raw_key, raw_key_size, lt_key, wk_size); + if (err) + return err; + + return wk_size; +} +EXPORT_SYMBOL_GPL(qcom_ice_import_key); + static struct qcom_ice *qcom_ice_create(struct device *dev, void __iomem *base) { struct qcom_ice *engine; @@ -239,13 +579,23 @@ static struct qcom_ice *qcom_ice_create(struct device *dev, if (!engine->core_clk) engine->core_clk = devm_clk_get_enabled(dev, NULL); if (IS_ERR(engine->core_clk)) return ERR_CAST(engine->core_clk); + engine->use_hwkm = qcom_scm_has_wrapped_key_support() && + qcom_ice_use_wrapped_keys; + if (!qcom_ice_check_supported(engine)) return ERR_PTR(-EOPNOTSUPP); + if (engine->use_hwkm) + dev_info(dev, "QC ICE HWKM (Hardware Key Manager) enabled"); + else if (qcom_ice_use_wrapped_keys) + dev_warn(dev, "HWKM not supported. Not supporting wrapped keys.\n"); + else + dev_info(dev, "HWKM not enabled. Supporting raw keys."); + dev_dbg(dev, "Registered Qualcomm Inline Crypto Engine\n"); return engine; } diff --git a/drivers/ufs/host/ufs-qcom.c b/drivers/ufs/host/ufs-qcom.c index 865193d9bc474..c2fd025d04384 100644 --- a/drivers/ufs/host/ufs-qcom.c +++ b/drivers/ufs/host/ufs-qcom.c @@ -132,10 +132,15 @@ static int ufs_qcom_ice_init(struct ufs_qcom_host *host) } if (IS_ERR_OR_NULL(ice)) return PTR_ERR_OR_ZERO(ice); + if (qcom_ice_using_hwkm(ice)) { + dev_warn(dev, "HWKM mode unsupported; disabling inline encryption support\n"); + return 0; + } + host->ice = ice; /* Initialize the blk_crypto_profile */ caps.reg_val = cpu_to_le32(ufshcd_readl(hba, REG_UFS_CCAP)); diff --git a/include/soc/qcom/ice.h b/include/soc/qcom/ice.h index 4cecc7f088b4b..f352c78d27a15 100644 --- a/include/soc/qcom/ice.h +++ b/include/soc/qcom/ice.h @@ -15,7 +15,19 @@ int qcom_ice_enable(struct qcom_ice *ice); int qcom_ice_resume(struct qcom_ice *ice); int qcom_ice_suspend(struct qcom_ice *ice); int qcom_ice_program_key(struct qcom_ice *ice, unsigned int slot, const struct blk_crypto_key *blk_key); int qcom_ice_evict_key(struct qcom_ice *ice, int slot); +bool qcom_ice_using_hwkm(struct qcom_ice *ice); +int qcom_ice_derive_sw_secret(struct qcom_ice *ice, + const u8 *eph_key, size_t eph_key_size, + u8 sw_secret[BLK_CRYPTO_SW_SECRET_SIZE]); +int qcom_ice_generate_key(struct qcom_ice *ice, + u8 lt_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]); +int qcom_ice_prepare_key(struct qcom_ice *ice, + const u8 *lt_key, size_t lt_key_size, + u8 eph_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]); +int qcom_ice_import_key(struct qcom_ice *ice, + const u8 *raw_key, size_t raw_key_size, + u8 lt_key[BLK_CRYPTO_MAX_HW_WRAPPED_KEY_SIZE]); struct qcom_ice *of_qcom_ice_get(struct device *dev); #endif /* __QCOM_ICE_H__ */