| Message ID | cover.1585852001.git.alexander.riesen@cetitec.com |
|---|---|
| Headers | show |
| Series | media: adv748x: add support for HDMI audio | expand |
Hi Alex, On 18/06/2020 17:32, Kieran Bingham wrote: > Hi Alex, > > On 02/04/2020 19:35, Alex Riesen wrote: >> As all known variants of the Salvator board have the HDMI decoder >> chip (the ADV7482) connected to the SSI4 on R-Car SoC, the ADV7482 >> endpoint and the connection definitions are placed in the common board >> file. >> >> For the same reason, the CLK_C clock line and I2C configuration (similar >> to the ak4613, on the same interface) are added into the common file. >> >> Signed-off-by: Alexander Riesen <alexander.riesen@cetitec.com> >> >> -- >> >> v5: Add dummy ssi4 node to the rcar sound card in r8a77961, as the >> devices (Salvator-X 2nd version with R-Car M3 W+) also reference >> salvator-common.dtsi. >> Suggested-by: Geert Uytterhoeven <geert@linux-m68k.org> >> >> v2: Also add ssi4_ctrl pin group in the sound pins. The pins are >> responsible for SCK4 (sample clock) WS4 and (word boundary input), >> and are required for SSI audio input over I2S. >> >> The adv748x shall provide its own implementation of the output clock >> (MCLK), connected to the audio_clk_c line of the R-Car SoC. >> >> If the frequency of the ADV748x MCLK were fixed, the clock >> implementation were not necessary, but it does not seem so: the MCLK >> depends on the value in a speed multiplier register and the input sample >> rate (48kHz). >> >> Remove audio clock C from the clocks of adv7482. >> >> The clocks property of the video-receiver node lists the input >> clocks of the device, which is quite the opposite from the >> original intention: the adv7482 on Salvator X boards is a >> provide of the MCLK clock for I2S audio output. >> >> Remove old definition of &sound_card.dais and reduce size of changes >> in the Salvator-X specific device tree source. >> >> Declare video-receiver a clock producer, as the adv748x driver >> implements the master clock used I2S audio output. >> >> Suggested-by: Geert Uytterhoeven <geert@linux-m68k.org> >> >> v2: The driver provides only MCLK clock, not the SCLK and LRCLK, >> which are part of the I2S protocol. >> >> Suggested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> >> --- >> .../boot/dts/renesas/r8a77950-salvator-x.dts | 3 +- >> arch/arm64/boot/dts/renesas/r8a77961.dtsi | 1 + >> .../boot/dts/renesas/salvator-common.dtsi | 47 +++++++++++++++++-- Once again I'm back trying to test this series, and one issue I've had is that the board I have (r8a77951-salvator-xs.dts) isn't included in this DT update. For v6, Should we include the relevant changes to all the following? arch/arm64/boot/dts/renesas/r8a77950-salvator-x.dts arch/arm64/boot/dts/renesas/r8a77951-salvator-x.dts arch/arm64/boot/dts/renesas/r8a77960-salvator-x.dts arch/arm64/boot/dts/renesas/r8a77965-salvator-x.dts arch/arm64/boot/dts/renesas/salvator-x.dtsi And perhaps handle the salvator-xs in a second (yet very similar) patch? arch/arm64/boot/dts/renesas/r8a77951-salvator-xs.dts arch/arm64/boot/dts/renesas/r8a77960-salvator-xs.dts arch/arm64/boot/dts/renesas/r8a77961-salvator-xs.dts arch/arm64/boot/dts/renesas/r8a77965-salvator-xs.dts arch/arm64/boot/dts/renesas/salvator-xs.dtsi I think I've added the relevant entries to my dtb, but I haven't successfully captured audio yet. I can see the device being listed through arecord: kbingham@salvator-xs:~$ arecord -l **** List of CAPTURE Hardware Devices **** card 0: rcarsound [rcar-sound], device 0: rsnd-dai.0-ak4613-hifi ak4613-hifi-0 [] Subdevices: 0/1 Subdevice #0: subdevice #0 card 0: rcarsound [rcar-sound], device 3: rsnd-dai.3-adv748x-i2s adv748x.4-0070-3 [] Subdevices: 1/1 Subdevice #0: subdevice #0 But as yet, everything I try to record fails or is empty silence. Debugging ... -- Regards Kieran >> 3 files changed, 45 insertions(+), 6 deletions(-) >> >> diff --git a/arch/arm64/boot/dts/renesas/r8a77950-salvator-x.dts b/arch/arm64/boot/dts/renesas/r8a77950-salvator-x.dts >> index 2438825c9b22..e16c146808b6 100644 >> --- a/arch/arm64/boot/dts/renesas/r8a77950-salvator-x.dts >> +++ b/arch/arm64/boot/dts/renesas/r8a77950-salvator-x.dts >> @@ -146,7 +146,8 @@ &sata { >> &sound_card { >> dais = <&rsnd_port0 /* ak4613 */ >> &rsnd_port1 /* HDMI0 */ >> - &rsnd_port2>; /* HDMI1 */ >> + &rsnd_port2 /* HDMI1 */ >> + &rsnd_port3>; /* adv7482 hdmi-in */ > > Ah - that was confusing at first... but HDMI0 and HDMI1 are *outputs*, > where of course the adv7482 is an input ;-) > > > Otherwise, I can't spot anything else yet so: > > Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com> > > But I fear there may have been some churn around here, so it would be > good to see a rebase too. > > -- > Kieran > > > >> }; >> >> &usb2_phy2 { >> diff --git a/arch/arm64/boot/dts/renesas/r8a77961.dtsi b/arch/arm64/boot/dts/renesas/r8a77961.dtsi >> index be3824bda632..b79907beaf31 100644 >> --- a/arch/arm64/boot/dts/renesas/r8a77961.dtsi >> +++ b/arch/arm64/boot/dts/renesas/r8a77961.dtsi >> @@ -861,6 +861,7 @@ rcar_sound,src { >> rcar_sound,ssi { >> ssi0: ssi-0 { }; >> ssi1: ssi-1 { }; >> + ssi4: ssi-4 { }; >> }; >> }; >> >> diff --git a/arch/arm64/boot/dts/renesas/salvator-common.dtsi b/arch/arm64/boot/dts/renesas/salvator-common.dtsi >> index 98bbcafc8c0d..ead7f8d7a929 100644 >> --- a/arch/arm64/boot/dts/renesas/salvator-common.dtsi >> +++ b/arch/arm64/boot/dts/renesas/salvator-common.dtsi >> @@ -460,7 +460,7 @@ pca9654: gpio@20 { >> #gpio-cells = <2>; >> }; >> >> - video-receiver@70 { >> + adv7482_hdmi_in: video-receiver@70 { >> compatible = "adi,adv7482"; >> reg = <0x70 0x71 0x72 0x73 0x74 0x75 >> 0x60 0x61 0x62 0x63 0x64 0x65>; >> @@ -469,6 +469,7 @@ video-receiver@70 { >> >> #address-cells = <1>; >> #size-cells = <0>; >> + #clock-cells = <0>; /* the MCLK for I2S output */ >> >> interrupt-parent = <&gpio6>; >> interrupt-names = "intrq1", "intrq2"; >> @@ -510,6 +511,15 @@ adv7482_txb: endpoint { >> remote-endpoint = <&csi20_in>; >> }; >> }; >> + >> + port@c { >> + reg = <12>; >> + >> + adv7482_i2s: endpoint { >> + remote-endpoint = <&rsnd_endpoint3>; >> + system-clock-direction-out; >> + }; >> + }; >> }; >> >> csa_vdd: adc@7c { >> @@ -684,7 +694,8 @@ sdhi3_pins_uhs: sd3_uhs { >> }; >> >> sound_pins: sound { >> - groups = "ssi01239_ctrl", "ssi0_data", "ssi1_data_a"; >> + groups = "ssi01239_ctrl", "ssi0_data", "ssi1_data_a", >> + "ssi4_data", "ssi4_ctrl"; >> function = "ssi"; >> }; >> >> @@ -733,8 +744,8 @@ &rcar_sound { >> pinctrl-0 = <&sound_pins &sound_clk_pins>; >> pinctrl-names = "default"; >> >> - /* Single DAI */ >> - #sound-dai-cells = <0>; >> + /* multi DAI */ >> + #sound-dai-cells = <1>; >> >> /* audio_clkout0/1/2/3 */ >> #clock-cells = <1>; >> @@ -758,8 +769,19 @@ &rcar_sound { >> <&cpg CPG_MOD 1020>, <&cpg CPG_MOD 1021>, >> <&cpg CPG_MOD 1019>, <&cpg CPG_MOD 1018>, >> <&audio_clk_a>, <&cs2000>, >> - <&audio_clk_c>, >> + <&adv7482_hdmi_in>, >> <&cpg CPG_CORE CPG_AUDIO_CLK_I>; >> + clock-names = "ssi-all", >> + "ssi.9", "ssi.8", "ssi.7", "ssi.6", >> + "ssi.5", "ssi.4", "ssi.3", "ssi.2", >> + "ssi.1", "ssi.0", >> + "src.9", "src.8", "src.7", "src.6", >> + "src.5", "src.4", "src.3", "src.2", >> + "src.1", "src.0", >> + "mix.1", "mix.0", >> + "ctu.1", "ctu.0", >> + "dvc.0", "dvc.1", >> + "clk_a", "clk_b", "clk_c", "clk_i"; >> >> ports { >> #address-cells = <1>; >> @@ -777,6 +799,21 @@ rsnd_endpoint0: endpoint { >> capture = <&ssi1 &src1 &dvc1>; >> }; >> }; >> + rsnd_port3: port@3 { >> + reg = <3>; >> + rsnd_endpoint3: endpoint { >> + remote-endpoint = <&adv7482_i2s>; >> + >> + dai-tdm-slot-num = <8>; >> + dai-tdm-slot-width = <32>; >> + dai-format = "left_j"; >> + mclk-fs = <256>; >> + bitclock-master = <&adv7482_i2s>; >> + frame-master = <&adv7482_i2s>; >> + >> + capture = <&ssi4>; >> + }; >> + }; >> }; >> }; >> >> > -- Regards -- Kieran
Hi Kieran, Kieran Bingham, Tue, Aug 25, 2020 16:57:04 +0200: > On 18/06/2020 17:32, Kieran Bingham wrote: > > On 02/04/2020 19:35, Alex Riesen wrote: > >> As all known variants of the Salvator board have the HDMI decoder > >> chip (the ADV7482) connected to the SSI4 on R-Car SoC, the ADV7482 > >> endpoint and the connection definitions are placed in the common board > >> file. > >> > >> For the same reason, the CLK_C clock line and I2C configuration (similar > >> to the ak4613, on the same interface) are added into the common file. > >> ... > >> --- > >> .../boot/dts/renesas/r8a77950-salvator-x.dts | 3 +- > >> arch/arm64/boot/dts/renesas/r8a77961.dtsi | 1 + > >> .../boot/dts/renesas/salvator-common.dtsi | 47 +++++++++++++++++-- > > Once again I'm back trying to test this series, and one issue I've had > is that the board I have (r8a77951-salvator-xs.dts) isn't included in > this DT update. > > For v6, Should we include the relevant changes to all the following? Ok. I shall add them as a separate patch though, as I have no way to verify those boards (and some verification seem to be in order...) > arch/arm64/boot/dts/renesas/r8a77950-salvator-x.dts > arch/arm64/boot/dts/renesas/r8a77951-salvator-x.dts > arch/arm64/boot/dts/renesas/r8a77960-salvator-x.dts > arch/arm64/boot/dts/renesas/r8a77965-salvator-x.dts > arch/arm64/boot/dts/renesas/salvator-x.dtsi > > And perhaps handle the salvator-xs in a second (yet very similar) patch? > > arch/arm64/boot/dts/renesas/r8a77951-salvator-xs.dts > arch/arm64/boot/dts/renesas/r8a77960-salvator-xs.dts > arch/arm64/boot/dts/renesas/r8a77961-salvator-xs.dts > arch/arm64/boot/dts/renesas/r8a77965-salvator-xs.dts > arch/arm64/boot/dts/renesas/salvator-xs.dtsi > > I think I've added the relevant entries to my dtb, but I haven't > successfully captured audio yet. > > I can see the device being listed through arecord: > > kbingham@salvator-xs:~$ arecord -l > **** List of CAPTURE Hardware Devices **** > card 0: rcarsound [rcar-sound], device 0: rsnd-dai.0-ak4613-hifi ak4613-hifi-0 [] > Subdevices: 0/1 > Subdevice #0: subdevice #0 > card 0: rcarsound [rcar-sound], device 3: rsnd-dai.3-adv748x-i2s adv748x.4-0070-3 [] > Subdevices: 1/1 > Subdevice #0: subdevice #0 > > But as yet, everything I try to record fails or is empty silence. > > Debugging ... Does it fail somewhere in the ASoC infrastructure? If so, how'd you find out where exactly and what fails? Asking, because when I was writing this code I ended up adding quite a bit of tracing into the SoC core to figure that out, and I just hope there is a better way to get at the diagnostics. > >> diff --git a/arch/arm64/boot/dts/renesas/r8a77950-salvator-x.dts b/arch/arm64/boot/dts/renesas/r8a77950-salvator-x.dts > >> index 2438825c9b22..e16c146808b6 100644 > >> --- a/arch/arm64/boot/dts/renesas/r8a77950-salvator-x.dts > >> +++ b/arch/arm64/boot/dts/renesas/r8a77950-salvator-x.dts > >> @@ -146,7 +146,8 @@ &sata { > >> &sound_card { > >> dais = <&rsnd_port0 /* ak4613 */ > >> &rsnd_port1 /* HDMI0 */ > >> - &rsnd_port2>; /* HDMI1 */ > >> + &rsnd_port2 /* HDMI1 */ > >> + &rsnd_port3>; /* adv7482 hdmi-in */ > > > > Ah - that was confusing at first... but HDMI0 and HDMI1 are *outputs*, > > where of course the adv7482 is an input ;-) I shall add an "output" to HDMI0 and HDMI1. > > Otherwise, I can't spot anything else yet so: > > > > Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com> Thanks! > > But I fear there may have been some churn around here, so it would be > > good to see a rebase too. Of course, I shall rebase on top of linux-media/master. Should I wait with submission until you get data out of your boards? Regards, Alex
This adds minimal support for accessing the HDMI audio provided through the I2S port available on ADV7481 and ADV7482 decoder devices by ADI. The port carries audio signal from the decoded HDMI stream. Currently, the driver only supports I2S in TDM, 8 channels a 24bit at 48kHz. Furthermore, only left-justified, 8 slots, 32bit/slot TDM, at 256fs has been ever tried. An ADV7482 on the Renesas Salvator-X ES1.1 (R8A77950 SoC) was used during development of this code. Changes since v4: - rebased on v5.6 - Add dummy ssi4 node to the rcar sound card, as the r8a77961 devices also reference salvator-common.dts. Suggested-by: Geert Uytterhoeven <geert@linux-m68k.org> Changes since v3: - use clk_hw instead of clk Suggested-by: Stephen Boyd <sboyd@kernel.org> - formatting improvements and use const where possible - removed implementation of log_status and EDID setting ioctls, those will be submitted as separate patches. Suggested-by: Hans Verkuil <hverkuil-cisco@xs4all.nl> Changes since v2: - prepare/enable the clock when it is used, as it seems nothing else does this otherwise - give the clock a unique name to ensure it can be registered if there are multiple adv748x devices in the system - remove optionality note from clock cell description to ensure the device description matches the real device (the line is always present, even if not used) Changes since v1: - Add ssi4_ctrl pin group to the sound pins. The pins are responsible for SCK4 (sample clock) WS4 and (word boundary input), and are required for SSI audio input over I2S. Reported-by: Geert Uytterhoeven <geert@linux-m68k.org> - Removed the audio clock C from the list of clocks of adv748x, it is exactly the other way around. Reported-by: Geert Uytterhoeven <geert@linux-m68k.org> - Add an instance of (currently) fixed rate I2S master clock (MCLK), connected to the audio_clk_c line of the R-Car SoC. Explicitly declare the device a clock producer and add it to the list of clocks used by the audio system of the Salvator-X board. Suggested-by: Geert Uytterhoeven <geert@linux-m68k.org> - The implementation of DAI driver has been moved in a separate file and modified to activate audio decoding and I2S streaming using snd_soc_dai_... interfaces. This allows the driver to be used with just ALSA interfaces. - The ioctls for selecting audio output and muting have been removed, as not applicable. Suggested-by: Hans Verkuil <hverkuil-cisco@xs4all.nl> I have left implementation of the QUERYCAP in, as it seems to be required by v4l-ctl to support loading of EDID for this node. And setting the EDID is one feature I desperately need: there are devices which plainly refuse to talk to the sink if it does not provide EDID they like. - A device tree configuration without audio port will disable the audio code altogether, supporting integrations where the port is not connected. - The patches have been re-arranged, starting with the generic changes and changes not related to audio directly. Those will be probably sent as a separate series later. - The whole series has been rebased on top of v5.6-rc6 Alex Riesen (9): media: adv748x: fix end-of-line terminators in diagnostic statements media: adv748x: include everything adv748x.h needs into the file media: adv748x: reduce amount of code for bitwise modifications of device registers media: adv748x: add definitions for audio output related registers media: adv748x: add support for HDMI audio media: adv748x: prepare/enable mclk when the audio is used media: adv748x: only activate DAI if it is described in device tree dt-bindings: adv748x: add information about serial audio interface (I2S/TDM) arm64: dts: renesas: salvator: add a connection from adv748x codec (HDMI input) to the R-Car SoC .../devicetree/bindings/media/i2c/adv748x.txt | 16 +- .../boot/dts/renesas/r8a77950-salvator-x.dts | 3 +- arch/arm64/boot/dts/renesas/r8a77961.dtsi | 1 + .../boot/dts/renesas/salvator-common.dtsi | 47 ++- drivers/media/i2c/adv748x/Makefile | 3 +- drivers/media/i2c/adv748x/adv748x-afe.c | 6 +- drivers/media/i2c/adv748x/adv748x-core.c | 45 +-- drivers/media/i2c/adv748x/adv748x-csi2.c | 8 +- drivers/media/i2c/adv748x/adv748x-dai.c | 278 ++++++++++++++++++ drivers/media/i2c/adv748x/adv748x-hdmi.c | 6 +- drivers/media/i2c/adv748x/adv748x.h | 65 +++- 11 files changed, 436 insertions(+), 42 deletions(-) create mode 100644 drivers/media/i2c/adv748x/adv748x-dai.c