| Message ID | 20220920034545.2820888-1-jason.zhu@rock-chips.com |
|---|---|
| Headers | show |
| Series | [1/1] ASoC: soc-dai: export some symbols | expand |
On Wed, Sep 21, 2022 at 10:37:06AM +0800, Jason Zhu wrote: > 在 2022/9/20 20:47, Mark Brown 写道: > > On Tue, Sep 20, 2022 at 11:45:45AM +0800, Jason Zhu wrote: > > > Sometimes we need to make some dais alive when close the card, like > > > VAD, so these functions must be exported so that they can be called. > > I'm not sure I fully understand the use case here - why wouldn't > > the core know about the audio stream being kept active? For > > something like VAD I'd expect this to be just working like a > > normal audio path, if there's a DSP consuming the audio stream > > then it'll keep everything open. If there is a good use case I > > suspect it'll be clearer if you send the users along with this > > patch. > Thanks. For example, we use the VAD(Voice Activity Detect) & PDM( > Pulse Density Modulation) to record sound>. The PDM is used to > record and copy data to DDR memory by DMA when the system is alive. > The VAD is used to detect voice from PDM and copy data to sram > (The sram is small) when the system is sleep. If the VAD detect > specific sound, wake up the system and continue to record sound. > The data can not be lost in this process. So we attach VAD & PDM > in the same card, then close the card and wake up VAD & PDM again > when the system is goto sleep. Like these code: This sounds like a very normal thing with a standard audio stream - other devices have similar VAD stuff without needing to open code access to the PCM operations? > When the system is waked up, open the sound card. The data in sram > is copied firstly and close the vad. Then use the DMA to move data > to DDR memory from PDM. Generally things just continue to stream the voice data through the same VAD stream IIRC - switching just adds complexity here, you don't have to deal with joining the VAD and regular streams up for one thing.
On 9/26/22 17:33, Mark Brown wrote: > On Mon, Sep 26, 2022 at 09:52:34AM +0200, Pierre-Louis Bossart wrote: >> On 9/26/22 03:34, Jason Zhu wrote: >>> 在 2022/9/23 20:55, Mark Brown 写道: > >>>>> The data can not be lost in this process. So we attach VAD & PDM >>>>> in the same card, then close the card and wake up VAD & PDM again >>>>> when the system is goto sleep. Like these code: > >>>> This sounds like a very normal thing with a standard audio stream - >>>> other devices have similar VAD stuff without needing to open code access >>>> to the PCM operations? > >>> At present, only VAD is handled in this way by Rockchip. > > The point here is that other non-Rockchip devices do similar sounding > things? > >>>> Generally things just continue to stream the voice data through the same >>>> VAD stream IIRC - switching just adds complexity here, you don't have to >>>> deal with joining the VAD and regular streams up for one thing. > >>> Yes, this looks complicated. But our chip's sram which is assigned to VAD >>> >>> maybe used by other devices when the system is alive. So we have to copy >>> >>> sound data in sram firstly, then use the DDR(SDRAM) to record sound data. > >> There are other devices that requires a copy of the history buffer from >> one PCM device and a software stitching with the real-time data coming >> from another PCM device. It's not ideal but not uncommon either, even >> for upcoming SDCA devices, combining data from 2 PCM devices will be an >> allowed option (with additional control information to help with the >> stitching). > > If this is something that's not uncommon that sounds like an even > stronger reason for not just randomly exporting the symbols and open > coding things in individual drivers outside of framework control. What > are these other use cases, or is it other instances of the same thing? > > TBH this sounds like at least partly a userspace problem rather than a > kernel one, as with other things that tie multiple audio streams > together. I would tend to agree, the stitching can be either handled in DSP firmware or in user-space. In the first case the kernel would expose a single PCM to userspace, and in the second there would be two separate PCM devices. The kernel drivers would typically do nothing other than deal with moving captured data if/when available. I also don't get the notion of 'keeping some DAIs alive when closing the card', maybe the idea is to redefine what 'D3' means or have an 'active standby' power state that doesn't exist today. That would in contrast be something the frameworks know about.
在 2022/9/26 23:33, Mark Brown 写道: > On Mon, Sep 26, 2022 at 09:52:34AM +0200, Pierre-Louis Bossart wrote: >> On 9/26/22 03:34, Jason Zhu wrote: >>> 在 2022/9/23 20:55, Mark Brown 写道: >>>>> The data can not be lost in this process. So we attach VAD & PDM >>>>> in the same card, then close the card and wake up VAD & PDM again >>>>> when the system is goto sleep. Like these code: >>>> This sounds like a very normal thing with a standard audio stream - >>>> other devices have similar VAD stuff without needing to open code access >>>> to the PCM operations? >>> At present, only VAD is handled in this way by Rockchip. > The point here is that other non-Rockchip devices do similar sounding > things? No. Usually, the vad is integrated in codec, like rt5677, and is linked with DSP to handle its data. If DSP detects useful sound, send an irq to system to wakeup and record sound. Others detect and analysis sound by VAD itself, like K32W041A. >>>> Generally things just continue to stream the voice data through the same >>>> VAD stream IIRC - switching just adds complexity here, you don't have to >>>> deal with joining the VAD and regular streams up for one thing. >>> Yes, this looks complicated. But our chip's sram which is assigned to VAD >>> >>> maybe used by other devices when the system is alive. So we have to copy >>> >>> sound data in sram firstly, then use the DDR(SDRAM) to record sound data. >> There are other devices that requires a copy of the history buffer from >> one PCM device and a software stitching with the real-time data coming >> from another PCM device. It's not ideal but not uncommon either, even >> for upcoming SDCA devices, combining data from 2 PCM devices will be an >> allowed option (with additional control information to help with the >> stitching). > If this is something that's not uncommon that sounds like an even > stronger reason for not just randomly exporting the symbols and open > coding things in individual drivers outside of framework control. What > are these other use cases, or is it other instances of the same thing? Maybe in this case: One PDM is used to record sound, and there is two way to move data. Use the VAD to move data to sram when system is sleep and use DMA to move data when sytem is alive. If we seperate this in two audio streams, we close the "PDM + VAD" audio stream firstly when system is alive and open "PDM + DMA" audio stream. This process maybe take long time that PDM FIFO will be full and lost some data. But we hope that data will not be lost in the whole proces. So these must be done in one audio stream. > TBH this sounds like at least partly a userspace problem rather than a > kernel one, as with other things that tie multiple audio streams > together. Yes, userspace can tie multiple audio stream together to avoid doing complicated things in kernel. This is good method!