mbox series

[v8,00/22] Add support for the SDM845 Camera Subsystem

Message ID 20210315155942.640889-1-robert.foss@linaro.org
Headers show
Series Add support for the SDM845 Camera Subsystem | expand

Message

Robert Foss March 15, 2021, 3:59 p.m. UTC
This series implements support for the camera subsystem found in
the SDM845 SOCs and the Titan 170 ISP. The support is partial
in that it implements CSIPHY, CSID, and partial VFE support.

The Titan generation of the ISP diverges a fair amount from the
design of the previous architecture generation, CAMSS. As a result
some pretty invasive refactoring is done in this series. It also
means that at this time we're unable to implement support for all
of the IP blocks contained. This is due to a combination of legal
considerations with respect to the IP and its owner Qualcomm and
time & man hour constrains on the Linaro side.

The CSIPHY (CSI Physical Layer) & CSID (CSI Decoder) support is
complete, but the VFE (Video Front End, which is referred to as IFE
(Image Front End) in the Titan generation of ISPs) only has support
for the RDI (Raw Dump Interface) which allows the raw output of
the CSID to be written to memory.

The 2nd interface implemented in the VFE silicon is the PIX
interface, and camss does not support it for this generation of ISPs.
The reason for this is that the PIX interface is used for sending
image data to the BPS (Bayer Processing Section) & IPE (Image
Processing Engine), but both of these units are beyond the scope
of enabling basic ISP functionality for the SDM845.

Since the Titan architecture generation diverges quite a bit from
the CAMSS generation, a lot of pretty major refactoring is carried
out in this series. Both the CSID & VFE core paths are made more
general and hardware version specific parts are broken out.
The CSIPHY didn't require quite as radical changes and therefore
keeps its current form.

Tested on:
 - Qcom RB3 / db845c + camera mezzanine, which is SDM845 based
 - db410c + D3 Camera mezzanine, which is APQ8016 based
 
Branch:
 - https://git.linaro.org/people/robert.foss/linux.git/log/?h=camss_sdm845_v1
 - https://git.linaro.org/people/robert.foss/linux.git/log/?h=camss_sdm845_v2
 - https://git.linaro.org/people/robert.foss/linux.git/log/?h=camss_sdm845_v3
 - https://git.linaro.org/people/robert.foss/linux.git/log/?h=camss_sdm845_v4
 - https://git.linaro.org/people/robert.foss/linux.git/log/?h=camss_sdm845_v5
 - https://git.linaro.org/people/robert.foss/linux.git/log/?h=camss_sdm845_v6
 - https://git.linaro.org/people/robert.foss/linux.git/log/?h=camss_sdm845_v7
 - https://git.linaro.org/people/robert.foss/linux.git/log/?h=camss_sdm845_v8



Robert Foss (22):
  media: camss: Fix vfe_isr_comp_done() documentation
  media: camss: Fix vfe_isr comment typo
  media: camss: Replace trace_printk() with dev_dbg()
  media: camss: Add CAMSS_845 camss version
  media: camss: Make ISPIF subdevice optional
  media: camss: Refactor VFE HW version support
  media: camss: Add support for VFE hardware version Titan 170
  media: camss: Add missing format identifiers
  media: camss: Refactor CSID HW version support
  media: camss: Add support for CSID hardware version Titan 170
  media: camss: Add support for CSIPHY hardware version Titan 170
  media: camss: Refactor VFE power domain toggling
  media: camss: Enable SDM845
  dt-bindings: media: camss: Add qcom,msm8916-camss binding
  dt-bindings: media: camss: Add qcom,msm8996-camss binding
  dt-bindings: media: camss: Add qcom,sdm660-camss binding
  dt-bindings: media: camss: Add qcom,sdm845-camss binding
  MAINTAINERS: Change CAMSS documentation to use dtschema bindings
  media: dt-bindings: media: Remove qcom,camss documentation
  arm64: dts: sdm845: Add CAMSS ISP node
  arm64: dts: sdm845-db845c: Configure regulators for camss node
  arm64: dts: sdm845-db845c: Enable ov8856 sensor and connect to ISP

 .../devicetree/bindings/media/qcom,camss.txt  |  236 ----
 .../bindings/media/qcom,msm8916-camss.yaml    |  256 ++++
 .../bindings/media/qcom,msm8996-camss.yaml    |  387 ++++++
 .../bindings/media/qcom,sdm660-camss.yaml     |  398 ++++++
 .../bindings/media/qcom,sdm845-camss.yaml     |  371 +++++
 MAINTAINERS                                   |    2 +-
 arch/arm64/boot/dts/qcom/sdm845-db845c.dts    |   23 +-
 arch/arm64/boot/dts/qcom/sdm845.dtsi          |  135 ++
 drivers/media/platform/qcom/camss/Makefile    |    6 +
 .../platform/qcom/camss/camss-csid-170.c      |  599 +++++++++
 .../platform/qcom/camss/camss-csid-4-1.c      |  328 +++++
 .../platform/qcom/camss/camss-csid-4-7.c      |  404 ++++++
 .../platform/qcom/camss/camss-csid-gen1.h     |   27 +
 .../platform/qcom/camss/camss-csid-gen2.h     |   39 +
 .../media/platform/qcom/camss/camss-csid.c    |  627 +--------
 .../media/platform/qcom/camss/camss-csid.h    |  161 ++-
 .../qcom/camss/camss-csiphy-3ph-1-0.c         |  179 ++-
 .../media/platform/qcom/camss/camss-csiphy.c  |   66 +-
 .../media/platform/qcom/camss/camss-ispif.c   |  119 +-
 .../media/platform/qcom/camss/camss-ispif.h   |    3 +-
 .../media/platform/qcom/camss/camss-vfe-170.c |  786 +++++++++++
 .../media/platform/qcom/camss/camss-vfe-4-1.c |  144 +-
 .../media/platform/qcom/camss/camss-vfe-4-7.c |  277 ++--
 .../media/platform/qcom/camss/camss-vfe-4-8.c | 1195 +++++++++++++++++
 .../platform/qcom/camss/camss-vfe-gen1.c      |  742 ++++++++++
 .../platform/qcom/camss/camss-vfe-gen1.h      |  117 ++
 drivers/media/platform/qcom/camss/camss-vfe.c |  847 +-----------
 drivers/media/platform/qcom/camss/camss-vfe.h |  128 +-
 .../media/platform/qcom/camss/camss-video.c   |   52 +
 drivers/media/platform/qcom/camss/camss.c     |  410 +++++-
 drivers/media/platform/qcom/camss/camss.h     |   15 +-
 31 files changed, 7027 insertions(+), 2052 deletions(-)
 delete mode 100644 Documentation/devicetree/bindings/media/qcom,camss.txt
 create mode 100644 Documentation/devicetree/bindings/media/qcom,msm8916-camss.yaml
 create mode 100644 Documentation/devicetree/bindings/media/qcom,msm8996-camss.yaml
 create mode 100644 Documentation/devicetree/bindings/media/qcom,sdm660-camss.yaml
 create mode 100644 Documentation/devicetree/bindings/media/qcom,sdm845-camss.yaml
 create mode 100644 drivers/media/platform/qcom/camss/camss-csid-170.c
 create mode 100644 drivers/media/platform/qcom/camss/camss-csid-4-1.c
 create mode 100644 drivers/media/platform/qcom/camss/camss-csid-4-7.c
 create mode 100644 drivers/media/platform/qcom/camss/camss-csid-gen1.h
 create mode 100644 drivers/media/platform/qcom/camss/camss-csid-gen2.h
 create mode 100644 drivers/media/platform/qcom/camss/camss-vfe-170.c
 create mode 100644 drivers/media/platform/qcom/camss/camss-vfe-4-8.c
 create mode 100644 drivers/media/platform/qcom/camss/camss-vfe-gen1.c
 create mode 100644 drivers/media/platform/qcom/camss/camss-vfe-gen1.h

-- 
2.27.0

Comments

Hans Verkuil March 16, 2021, 9:36 a.m. UTC | #1
On 15/03/2021 16:59, Robert Foss wrote:
> In order to support Qualcomm ISP hardware architectures that diverge
> from older architectures, the CSID subdevice drivers needs to be refactored
> to better abstract the different ISP hardware architectures.
> 
> Signed-off-by: Robert Foss <robert.foss@linaro.org>
> Reviewed-by: Andrey Konovalov <andrey.konovalov@linaro.org>
> ---
> 
> 
> Changes since v1:
>  - kernel test robot: Add missing include, interrupt.h
> 
> Changes since v4:
>  - Andrey: Removed whitespace from some includes
>  - Andrey: Removed unused enum
> 
> Changes since v5:
>  - Andrey: Fixed test pattern selection logic
>  - Andrey: Align test mode enum values with v4l mode selection return values
>  - Andrey: r-b
>  - Move Titan 170 test modes to the the Titan 170 commit
>  - Fixed test pattern boundary check
> 
> Changes since v7:
>  - Hans: Fix checkpatch.pl --strict warnings
> 
> 
> 
>  drivers/media/platform/qcom/camss/Makefile    |   2 +
>  .../platform/qcom/camss/camss-csid-4-1.c      | 328 ++++++++++
>  .../platform/qcom/camss/camss-csid-4-7.c      | 404 ++++++++++++
>  .../media/platform/qcom/camss/camss-csid.c    | 608 +-----------------
>  .../media/platform/qcom/camss/camss-csid.h    | 129 +++-
>  5 files changed, 885 insertions(+), 586 deletions(-)
>  create mode 100644 drivers/media/platform/qcom/camss/camss-csid-4-1.c
>  create mode 100644 drivers/media/platform/qcom/camss/camss-csid-4-7.c
> 

<snip>

> diff --git a/drivers/media/platform/qcom/camss/camss-csid.h b/drivers/media/platform/qcom/camss/camss-csid.h
> index 479ac1f83836..613ef377b051 100644
> --- a/drivers/media/platform/qcom/camss/camss-csid.h
> +++ b/drivers/media/platform/qcom/camss/camss-csid.h
> @@ -11,6 +11,7 @@
>  #define QC_MSM_CAMSS_CSID_H
>  
>  #include <linux/clk.h>
> +#include <linux/interrupt.h>
>  #include <media/media-entity.h>
>  #include <media/v4l2-ctrls.h>
>  #include <media/v4l2-device.h>
> @@ -44,18 +45,42 @@
>  #define DATA_TYPE_RAW_16BIT		0x2e
>  #define DATA_TYPE_RAW_20BIT		0x2f
>  
> -enum csid_payload_mode {
> -	CSID_PAYLOAD_MODE_INCREMENTING = 0,
> -	CSID_PAYLOAD_MODE_ALTERNATING_55_AA = 1,
> -	CSID_PAYLOAD_MODE_ALL_ZEROES = 2,
> -	CSID_PAYLOAD_MODE_ALL_ONES = 3,
> -	CSID_PAYLOAD_MODE_RANDOM = 4,
> -	CSID_PAYLOAD_MODE_USER_SPECIFIED = 5,
> +#define CSID_RESET_TIMEOUT_MS 500
> +
> +enum csid_testgen_mode {
> +	CSID_PAYLOAD_MODE_DISABLED = 0,
> +	CSID_PAYLOAD_MODE_INCREMENTING = 1,
> +	CSID_PAYLOAD_MODE_ALTERNATING_55_AA = 2,
> +	CSID_PAYLOAD_MODE_ALL_ZEROES = 3,
> +	CSID_PAYLOAD_MODE_ALL_ONES = 4,
> +	CSID_PAYLOAD_MODE_RANDOM = 5,
> +	CSID_PAYLOAD_MODE_USER_SPECIFIED = 6,
> +	CSID_PAYLOAD_MODE_NUM_SUPPORTED_GEN1 = 6, /* excluding disabled */
> +};
> +
> +static const char * const csid_testgen_modes[] = {
> +	"Disabled",
> +	"Incrementing",
> +	"Alternating 0x55/0xAA",
> +	"All Zeros 0x00",
> +	"All Ones 0xFF",
> +	"Pseudo-random Data",
> +	"User Specified",
> +};

This gives this sparse warning:

'csid_testgen_modes' defined but not used [-Wunused-const-variable=]

This array needs to be moved to camss-csid.c and declared as an extern
here. Also, this menu array needs to be terminated with a NULL, and the
right capitalization needs to be used (first character of each word must
be a capital). This is a suggested patch I made to verify that this solves
this issue, but really both patch 9 and 10 need to be modified.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
---
 drivers/media/platform/qcom/camss/camss-csid.c | 14 ++++++++++++++
 drivers/media/platform/qcom/camss/camss-csid.h | 13 +------------
 2 files changed, 15 insertions(+), 12 deletions(-)

diff --git a/drivers/media/platform/qcom/camss/camss-csid.c b/drivers/media/platform/qcom/camss/camss-csid.c
index fb94dc03ccd4..1513b3d47fc2 100644
--- a/drivers/media/platform/qcom/camss/camss-csid.c
+++ b/drivers/media/platform/qcom/camss/camss-csid.c
@@ -27,6 +27,20 @@

 #define MSM_CSID_NAME "msm_csid"

+const char * const csid_testgen_modes[] = {
+	"Disabled",
+	"Incrementing",
+	"Alternating 0x55/0xAA",
+	"All Zeros 0x00",
+	"All Ones 0xFF",
+	"Pseudo-Random Data",
+	"User Specified",
+	"Complex Pattern",
+	"Color Box",
+	"Color Bars",
+	NULL
+};
+
 u32 csid_find_code(u32 *codes, unsigned int ncodes,
 		   unsigned int match_format_idx, u32 match_code)
 {
diff --git a/drivers/media/platform/qcom/camss/camss-csid.h b/drivers/media/platform/qcom/camss/camss-csid.h
index c2a025f6846b..81a3704ac0e3 100644
--- a/drivers/media/platform/qcom/camss/camss-csid.h
+++ b/drivers/media/platform/qcom/camss/camss-csid.h
@@ -62,18 +62,7 @@ enum csid_testgen_mode {
 	CSID_PAYLOAD_MODE_NUM_SUPPORTED_GEN2 = 9, /* excluding disabled */
 };

-static const char * const csid_testgen_modes[] = {
-	"Disabled",
-	"Incrementing",
-	"Alternating 0x55/0xAA",
-	"All Zeros 0x00",
-	"All Ones 0xFF",
-	"Pseudo-random Data",
-	"User Specified",
-	"Complex pattern",
-	"Color box",
-	"Color bars",
-};
+extern const char * const csid_testgen_modes[];

 struct csid_format {
 	u32 code;
Robert Foss March 16, 2021, 11:12 a.m. UTC | #2
Hey Hans,

Thanks for looking into this.

On Tue, 16 Mar 2021 at 10:36, Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
> On 15/03/2021 16:59, Robert Foss wrote:
> > In order to support Qualcomm ISP hardware architectures that diverge
> > from older architectures, the CSID subdevice drivers needs to be refactored
> > to better abstract the different ISP hardware architectures.
> >
> > Signed-off-by: Robert Foss <robert.foss@linaro.org>
> > Reviewed-by: Andrey Konovalov <andrey.konovalov@linaro.org>
> > ---
> >
> >
> > Changes since v1:
> >  - kernel test robot: Add missing include, interrupt.h
> >
> > Changes since v4:
> >  - Andrey: Removed whitespace from some includes
> >  - Andrey: Removed unused enum
> >
> > Changes since v5:
> >  - Andrey: Fixed test pattern selection logic
> >  - Andrey: Align test mode enum values with v4l mode selection return values
> >  - Andrey: r-b
> >  - Move Titan 170 test modes to the the Titan 170 commit
> >  - Fixed test pattern boundary check
> >
> > Changes since v7:
> >  - Hans: Fix checkpatch.pl --strict warnings
> >
> >
> >
> >  drivers/media/platform/qcom/camss/Makefile    |   2 +
> >  .../platform/qcom/camss/camss-csid-4-1.c      | 328 ++++++++++
> >  .../platform/qcom/camss/camss-csid-4-7.c      | 404 ++++++++++++
> >  .../media/platform/qcom/camss/camss-csid.c    | 608 +-----------------
> >  .../media/platform/qcom/camss/camss-csid.h    | 129 +++-
> >  5 files changed, 885 insertions(+), 586 deletions(-)
> >  create mode 100644 drivers/media/platform/qcom/camss/camss-csid-4-1.c
> >  create mode 100644 drivers/media/platform/qcom/camss/camss-csid-4-7.c
> >
>
> <snip>
>
> > diff --git a/drivers/media/platform/qcom/camss/camss-csid.h b/drivers/media/platform/qcom/camss/camss-csid.h
> > index 479ac1f83836..613ef377b051 100644
> > --- a/drivers/media/platform/qcom/camss/camss-csid.h
> > +++ b/drivers/media/platform/qcom/camss/camss-csid.h
> > @@ -11,6 +11,7 @@
> >  #define QC_MSM_CAMSS_CSID_H
> >
> >  #include <linux/clk.h>
> > +#include <linux/interrupt.h>
> >  #include <media/media-entity.h>
> >  #include <media/v4l2-ctrls.h>
> >  #include <media/v4l2-device.h>
> > @@ -44,18 +45,42 @@
> >  #define DATA_TYPE_RAW_16BIT          0x2e
> >  #define DATA_TYPE_RAW_20BIT          0x2f
> >
> > -enum csid_payload_mode {
> > -     CSID_PAYLOAD_MODE_INCREMENTING = 0,
> > -     CSID_PAYLOAD_MODE_ALTERNATING_55_AA = 1,
> > -     CSID_PAYLOAD_MODE_ALL_ZEROES = 2,
> > -     CSID_PAYLOAD_MODE_ALL_ONES = 3,
> > -     CSID_PAYLOAD_MODE_RANDOM = 4,
> > -     CSID_PAYLOAD_MODE_USER_SPECIFIED = 5,
> > +#define CSID_RESET_TIMEOUT_MS 500
> > +
> > +enum csid_testgen_mode {
> > +     CSID_PAYLOAD_MODE_DISABLED = 0,
> > +     CSID_PAYLOAD_MODE_INCREMENTING = 1,
> > +     CSID_PAYLOAD_MODE_ALTERNATING_55_AA = 2,
> > +     CSID_PAYLOAD_MODE_ALL_ZEROES = 3,
> > +     CSID_PAYLOAD_MODE_ALL_ONES = 4,
> > +     CSID_PAYLOAD_MODE_RANDOM = 5,
> > +     CSID_PAYLOAD_MODE_USER_SPECIFIED = 6,
> > +     CSID_PAYLOAD_MODE_NUM_SUPPORTED_GEN1 = 6, /* excluding disabled */
> > +};
> > +
> > +static const char * const csid_testgen_modes[] = {
> > +     "Disabled",
> > +     "Incrementing",
> > +     "Alternating 0x55/0xAA",
> > +     "All Zeros 0x00",
> > +     "All Ones 0xFF",
> > +     "Pseudo-random Data",
> > +     "User Specified",
> > +};
>
> This gives this sparse warning:
>
> 'csid_testgen_modes' defined but not used [-Wunused-const-variable=]

Thanks for supplying a patch. I'll merge it into patch 9 & 10.

>
> This array needs to be moved to camss-csid.c and declared as an extern
> here. Also, this menu array needs to be terminated with a NULL, and the
> right capitalization needs to be used (first character of each word must
> be a capital). This is a suggested patch I made to verify that this solves
> this issue, but really both patch 9 and 10 need to be modified.
>
> Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
> ---
>  drivers/media/platform/qcom/camss/camss-csid.c | 14 ++++++++++++++
>  drivers/media/platform/qcom/camss/camss-csid.h | 13 +------------
>  2 files changed, 15 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/media/platform/qcom/camss/camss-csid.c b/drivers/media/platform/qcom/camss/camss-csid.c
> index fb94dc03ccd4..1513b3d47fc2 100644
> --- a/drivers/media/platform/qcom/camss/camss-csid.c
> +++ b/drivers/media/platform/qcom/camss/camss-csid.c
> @@ -27,6 +27,20 @@
>
>  #define MSM_CSID_NAME "msm_csid"
>
> +const char * const csid_testgen_modes[] = {
> +       "Disabled",
> +       "Incrementing",
> +       "Alternating 0x55/0xAA",
> +       "All Zeros 0x00",
> +       "All Ones 0xFF",
> +       "Pseudo-Random Data",
> +       "User Specified",
> +       "Complex Pattern",
> +       "Color Box",
> +       "Color Bars",
> +       NULL
> +};
> +
>  u32 csid_find_code(u32 *codes, unsigned int ncodes,
>                    unsigned int match_format_idx, u32 match_code)
>  {
> diff --git a/drivers/media/platform/qcom/camss/camss-csid.h b/drivers/media/platform/qcom/camss/camss-csid.h
> index c2a025f6846b..81a3704ac0e3 100644
> --- a/drivers/media/platform/qcom/camss/camss-csid.h
> +++ b/drivers/media/platform/qcom/camss/camss-csid.h
> @@ -62,18 +62,7 @@ enum csid_testgen_mode {
>         CSID_PAYLOAD_MODE_NUM_SUPPORTED_GEN2 = 9, /* excluding disabled */
>  };
>
> -static const char * const csid_testgen_modes[] = {
> -       "Disabled",
> -       "Incrementing",
> -       "Alternating 0x55/0xAA",
> -       "All Zeros 0x00",
> -       "All Ones 0xFF",
> -       "Pseudo-random Data",
> -       "User Specified",
> -       "Complex pattern",
> -       "Color box",
> -       "Color bars",
> -};
> +extern const char * const csid_testgen_modes[];
>
>  struct csid_format {
>         u32 code;
> --
> 2.30.1
>
> Regards,
>
>         Hans