[v3,0/3] crypto: qat - fix dm-crypt related issues

This set fixes the issues related with the dm-crypt + QAT driver
use-case.

The first patch fixes a potential dead-lock that might occur when using
dm-crypt + QAT in out of memory conditions. The datapaths of the aead
and skcipher implementations have been changed to use pre-allocated
buffers that are part of the request contexts.
The also removes the CRYPTO_ALG_ALLOCATES_MEMORY flag from the aead and
skcipher implementations.

The second patch refactors the submission logic in preparation for the
introduction of a backlog queue to handle crypto requests with the
CRYPTO_TFM_REQ_MAY_BACKLOG flag set.

The third patch addresses a stall in the dm-crypt + QAT usecase by
adding support for the CRYPTO_TFM_REQ_MAY_BACKLOG flag. If the HW queue
is full, the driver enqueues the request in a list and resubmit it at
a later time, avoiding losing it.

Changes from v1:
 - Patch #3: removed worqueues. Requests in the backlog queue are now
   resubmitted in the context of the callback of a previously submitted
   request. This reduces the CPU utilization as the resubmit backlog
   function always finds a free slot in the HW queues when resubmits a
   job.

Changes from v2:
  - Patch #3: added initialization of list element before insertion in
    backlog list.
  - Patch #3: changed read order of pointer to backlog structure in the
    callback. The pointer to the backlog structure, which is part of the
    request context, needs to be read before the request is completed.
    This is since the user might free the request structure after the
    request is completed.
  - Removed patch 4 to keep the algorithms disabled as there is now a
    regression on the rsa implementation after commit f5ff79fddf0e
    ("dma-mapping: remove CONFIG_DMA_REMAP") is showing a regression
    unrelated to this set.

Giovanni Cabiddu (3):
  crypto: qat - use pre-allocated buffers in datapath
  crypto: qat - refactor submission logic
  crypto: qat - add backlog mechanism

 drivers/crypto/qat/qat_common/Makefile        |   1 +
 drivers/crypto/qat/qat_common/adf_transport.c |  11 ++
 drivers/crypto/qat/qat_common/adf_transport.h |   1 +
 .../qat/qat_common/adf_transport_internal.h   |   1 +
 drivers/crypto/qat/qat_common/qat_algs.c      | 151 ++++++++++--------
 drivers/crypto/qat/qat_common/qat_algs_send.c |  86 ++++++++++
 drivers/crypto/qat/qat_common/qat_algs_send.h |  11 ++
 drivers/crypto/qat/qat_common/qat_asym_algs.c |  57 ++++---
 drivers/crypto/qat/qat_common/qat_crypto.c    |   3 +
 drivers/crypto/qat/qat_common/qat_crypto.h    |  39 +++++
 10 files changed, 273 insertions(+), 88 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_common/qat_algs_send.c
 create mode 100644 drivers/crypto/qat/qat_common/qat_algs_send.h

On Sun, Apr 10, 2022 at 08:47:05PM +0100, Giovanni Cabiddu wrote:
> If requests exceed 4 entries buffers, memory is allocated dynamically.
> 
> In addition, remove the CRYPTO_ALG_ALLOCATES_MEMORY flag from both aead
> and skcipher alg structures.
> 

There is nothing that says that algorithms can ignore
!CRYPTO_ALG_ALLOCATES_MEMORY if there are too many scatterlist entries.  See the
comment above the definition of CRYPTO_ALG_ALLOCATES_MEMORY.

If you need to introduce this constraint, then you will need to audit the users
of !CRYPTO_ALG_ALLOCATES_MEMORY to verify that none of them are issuing requests
that violate this constraint, then add this to the documentation comment for
CRYPTO_ALG_ALLOCATES_MEMORY.

- Eric