| Message ID | 20200821125848.1092958-1-dwlsalmeida@gmail.com |
|---|---|
| Headers | show |
| Series | media: vidtv: Implement a virtual DVB driver | expand |
Hi Daniel, Em Fri, 21 Aug 2020 09:58:44 -0300 "Daniel W. S. Almeida" <dwlsalmeida@gmail.com> escreveu: > From: "Daniel W. S. Almeida" <dwlsalmeida@gmail.com> > > This series is work in progress. It represents the current work done on a > virtual DVB driver for the Linux media subsystem. I am new to the media > subsystem and to kernel development in general. > > This driver aims to: > - Serve as template for new DVB driver writers > - Help userspace application writers in general > - Push fake audio/video to userspace for testing > purposes > - Push debug information to userspace via debugfs > > Current state for this driver: > - Driver generates PSI information (PAT, PMT, SDT) > - Driver generates PCR packets > - Driver generates NULL packets for padding > - PCM audio stream is decoded by ffmpeg, but no audio is heard yet. > > changes in v10: > s302m encoder got reworked Thanks for all the hard work on it. Very much appreciated! I finally found some time to test it. For now, just a quick test from my side, without passing any arguments to the driver. See enclosed. While there are still a few issues there, I'm considering applying this series for Kernel 5.10. My plan is to write some patches on the top of yours, in order to address the problems I'll find on it. If not something more critical won't be solved in time, we may still add it at staging/media. Let's see. The problems I noticed so far are: 1. rmmod doesn't work; 2. dvbv5-stats doesn't seem to be working properly, as it is reporting signal strengh with a percentage (same for Carrier S/N ratio); 3. dvbv5-zap wrote an empty audio file (without -P flag). Probably there are still some issues at the program channel descriptor or service; 4. The provider service field is null. Perhaps we could add some string there, like "linuxtv.org". 5. Maybe we could also add a simple NIT table, just to avoid dvbv5-scan to wait for it until timeout. Also, it probably makes sense to add a debugfs interface in order to allow injecting errors at the stream at runtime. Regards, Mauro --- $ dvbv5-scan -v foobar using demux 'dvb0.demux0' Device Dummy demod for DVB-T/T2/C/S/S2 (/dev/dvb/adapter0/frontend0) capabilities: CAN_FEC_1_2 CAN_FEC_2_3 CAN_FEC_3_4 CAN_FEC_4_5 CAN_FEC_5_6 CAN_FEC_6_7 CAN_FEC_7_8 CAN_FEC_8_9 CAN_FEC_AUTO CAN_GUARD_INTERVAL_AUTO CAN_HIERARCHY_AUTO CAN_INVERSION_AUTO CAN_QAM_16 CAN_QAM_32 CAN_QAM_64 CAN_QAM_128 CAN_QAM_256 CAN_QAM_AUTO CAN_QPSK CAN_TRANSMISSION_MODE_AUTO DVB API Version 5.11, Current v5 delivery system: DVBC/ANNEX_A Supported delivery systems: DVBT DVBT2 [DVBC/ANNEX_A] DVBS DVBS2 Frequency range for the current standard: From: 51.0 MHz To: 2.15 GHz Step: 62.5 kHz Tolerance: 29.5 MHz Symbol rate ranges for the current standard: From: 1.00 MBauds To: 45.0 MBauds Failed to guess country from the current locale setting. ERROR command BANDWIDTH_HZ (5) not found during retrieve Cannot calc frequency shift. Either bandwidth/symbol-rate is unavailable (yet). Scanning frequency #1 330000000 FREQUENCY = 330000000 MODULATION = QAM/AUTO INVERSION = AUTO SYMBOL_RATE = 6940000 INNER_FEC = AUTO DELIVERY_SYSTEM = DVBC/ANNEX_A Got parameters for DVBC/ANNEX_A: FREQUENCY = 330000000 MODULATION = QAM/AUTO INVERSION = AUTO SYMBOL_RATE = 6940000 INNER_FEC = AUTO DELIVERY_SYSTEM = DVBC/ANNEX_A Lock (0x1f) Signal= 0.00% C/N= 0.00% UCB= 0 postBER= 0 dvb_read_sections: waiting for table ID 0x00, program ID 0x00 dvb_parse_section: received table 0x00, extension ID 0x0744, section 0/0 dvb_parse_section: table 0x00, extension ID 0x0744: done PAT | table_id 0x00 | section_length 13 | one 2 | zero 0 | syntax 1 | transport_stream_id 1860 | current_next 1 | version 1 | one2 3 | section_number 0 | last_section_number 0 |\ 1 program pid | pid 0x0101: service 0x0880 Program #0 ID 0x0101, service ID 0x0880 dvb_read_sections: waiting for table ID 0x02, program ID 0x101 dvb_parse_section: received table 0x02, extension ID 0x0880, section 0/0 dvb_parse_section: table 0x02, extension ID 0x0880: done PMT | table_id 0x02 | section_length 24 | one 2 | zero 0 | syntax 1 | transport_stream_id 2176 | current_next 1 | version 1 | one2 3 | section_number 0 | last_section_number 0 |- pcr_pid 0200 | reserved2 7 | descriptor length 0 | zero3 0 | reserved3 15 |\ |- stream 0x0111: ISO/IEC 13818-1 Private Data (6) | descriptor length 6 | 0x05: registration_descriptor | 42 53 53 44 BSSD |_ 1 streams dvb_read_sections: waiting for table ID 0x40, program ID 0x10 ERROR dvb_read_sections: no data read on section filter ERROR error while reading the NIT table dvb_read_sections: waiting for table ID 0x42, program ID 0x11 dvb_parse_section: received table 0x42, extension ID 0x0744, section 0/0 dvb_parse_section: table 0x42, extension ID 0x0744: done SDT | table_id 0x42 | section_length 48 | one 2 | zero 1 | syntax 1 | transport_stream_id 1860 | current_next 1 | version 1 | one2 3 | section_number 0 | last_section_number 0 | network_id 1860 | reserved 255 |\ |- service 0x0880 | EIT schedule 0 | EIT present following 0 | free CA mode 0 | running status 4 | descriptor length 31 | 0x48: service_descriptor | service type 1 | provider '(null)' | name 'S302m: Sine Wave PCM Audio' |_ 1 services Service S302m: Sine Wave PCM Audio, provider (null): digital television Storing as channel S302m: Sine Wave PCM Audio $ dvbv5-zap -c dvb_channel.conf "S302m: Sine Wave PCM Audio" -m ... PID FREQ SPEED TOTAL 0 25.07 p/s 36.826 Kbps 55 KB 17 25.07 p/s 36.826 Kbps 55 KB 257 25.07 p/s 36.826 Kbps 55 KB 273 1335.19 p/s 1.915 Mbps 2943 KB 512 25.07 p/s 36.826 Kbps 55 KB 8191 16775.59 p/s 24.062 Mbps 36974 KB TOT 18211.08 p/s 26.121 Mbps 40138 KB Lock (0x1f) Signal= 0.00% C/N= 0.00% UCB= 0 postBER= 0 $ dvbv5-zap -c dvb_channel.conf "S302m: Sine Wave PCM Audio" -t 30 -o pcm_audio.ts using demux 'dvb0.demux0' reading channels from file 'dvb_channel.conf' service has pid type 06: 273 tuning to 330000000 Hz (0x00) Signal= 0.00% C/N= 0.00% UCB= 0 postBER= 0 Lock (0x1f) Signal= 0.00% C/N= 0.00% UCB= 0 postBER= 0 Lock (0x1f) Signal= 0.00% C/N= 0.00% UCB= 0 postBER= 0 Record to file 'pcm_audio.ts' started Forcing program stop due to timeout or terminate signal $ ls -l pcm_audio.ts -rw-r--r-- 1 mchehab mchehab 0 Sep 11 09:46 pcm_audio.ts $ dvbv5-zap -c dvb_channel.conf "S302m: Sine Wave PCM Audio" -t 30 -o pcm_audio.ts -P using demux 'dvb0.demux0' reading channels from file 'dvb_channel.conf' service has pid type 06: 273 tuning to 330000000 Hz pass all PID's to TS dvb_set_pesfilter 8192 dvb_dev_set_bufsize: buffer set to 6160384 Lock (0x1f) Signal= 0.00% C/N= 0.00% UCB= 0 postBER= 0 Lock (0x1f) Signal= 0.00% C/N= 0.00% UCB= 0 postBER= 0 Record to file 'pcm_audio.ts' started received 103809840 bytes (3379 Kbytes/sec) Lock (0x1f) Signal= 0.00% C/N= 0.00% UCB= 0 postBER= 0 $ ls -lh pcm_audio.ts -rw-r--r-- 1 mchehab mchehab 100M Sep 11 09:54 pcm_audio.ts
Hi Daniel, First of all, thank you for all your work on this. I see that Mauro merged this v10, and after testing I posted a patch for the Kconfig, since that seems to be wrong. I have some more questions below, my apologies if that's been asked before. On 21/08/2020 14:58, Daniel W. S. Almeida wrote: > From: "Daniel W. S. Almeida" <dwlsalmeida@gmail.com> > > This series is work in progress. It represents the current work done on a > virtual DVB driver for the Linux media subsystem. I am new to the media > subsystem and to kernel development in general. > > This driver aims to: > - Serve as template for new DVB driver writers > - Help userspace application writers in general > - Push fake audio/video to userspace for testing > purposes > - Push debug information to userspace via debugfs > > Current state for this driver: > - Driver generates PSI information (PAT, PMT, SDT) > - Driver generates PCR packets > - Driver generates NULL packets for padding > - PCM audio stream is decoded by ffmpeg, but no audio is heard yet. > > > changes in v10: > s302m encoder got reworked > > The Virtual Digital TV Driver (vidtv) > > Background > ---------- > > Vidtv is a virtual DVB driver that aims to serve as a reference for driver > writers by serving as a template. It also validates the existing media DVB > APIs, thus helping userspace application writers. > > Currently, it consists of: > > - A fake tuner driver, which will report a bad signal quality if the chosen > frequency is too far away from a table of valid frequencies for a > particular delivery system. > > - A fake demod driver, which will constantly poll the fake signal quality > returned by the tuner, simulating a device that can lose/reacquire a lock > on the signal depending on the CNR levels. > > - A fake bridge driver, which is the module responsible for modprobing the > fake tuner and demod modules and implementing the demux logic. This module > takes parameters at initialization that will dictate how the simulation > behaves. > > - Code reponsible for encoding a valid MPEG Transport Stream, which is then > passed to the bridge driver. This fake stream contains some hardcoded content. > For now, we have a single, audio-only channel containing a single MPEG > Elementary Stream, which in turn contains a SMPTE 302m encoded sine-wave. > Note that this particular encoder was chosen because it is the easiest > way to encode PCM audio data in a MPEG Transport Stream. > > Building vidtv > -------------- > vidtv is a test driver and thus is **not** enabled by default when > compiling the kernel. > > In order to enable compilation of vidtv: > > - Enable **DVB_TEST_DRIVERS**, then > - Enable **DVB_VIDTV** > > When compiled as a module, expect the following .ko files: > > - dvb_vidtv_tuner.ko > > - dvb_vidtv_demod.ko > > - dvb_vidtv_bridge.ko Why the dvb_ prefix? All virtual drivers just start with 'vi'. And wouldn't it make more sense to call dvb_vidtv_bridge.ko just vidtv.ko? Just like the other virtual media drivers? > > Running vidtv > ------------- > When compiled as a module, run:: > > modprobe dvb_vidtv_bridge > > That's it! The bridge driver will initialize the tuner and demod drivers as > part of its own initialization. > > You can optionally define some command-line arguments to vidtv, see the > documentation for more info. > > Testing vidtv with v4l-utils > ============================ For regression testing of vidtv during the daily build it would be great if the contrib/test/test-media script can be enhanced to include vidtv. This is run during the daily build with a kernel that has lockdep and many other checks enabled, so it is very helpful to verify that no regressions happened. Note that this script relies on the /dev/mediaX devices to run the tests. I noticed that vidtv doesn't appear to create a /dev/mediaX device, even though CONFIG_MEDIA_CONTROLLER_DVB=y. This is definitely something that would be good to support in vidtv. Regards, Hans > > Start by installing v4l-utils and then modprobing vidtv:: > > modprobe dvb_vidtv_bridge > > If the driver is OK, it should load and its probing code will run. This will > pull in the tuner and demod drivers. > > Using dvb-fe-tool > ----------------- > > The first step to check whether the demod loaded successfully is to run:: > > $ dvb-fe-tool > > This should return what is currently set up at the demod struct, i.e.:: > > static const struct dvb_frontend_ops vidtv_demod_ops = { > .delsys = { > SYS_DVBT, > SYS_DVBT2, > SYS_DVBC_ANNEX_A, > SYS_DVBS, > SYS_DVBS2, > }, > > .info = { > .name = "Dummy demod for DVB-T/T2/C/S/S2", > .frequency_min_hz = 51 * MHz, > .frequency_max_hz = 2150 * MHz, > .frequency_stepsize_hz = 62500, > .frequency_tolerance_hz = 29500 * kHz, > .symbol_rate_min = 1000000, > .symbol_rate_max = 45000000, > > .caps = FE_CAN_FEC_1_2 | > FE_CAN_FEC_2_3 | > FE_CAN_FEC_3_4 | > FE_CAN_FEC_4_5 | > FE_CAN_FEC_5_6 | > FE_CAN_FEC_6_7 | > FE_CAN_FEC_7_8 | > FE_CAN_FEC_8_9 | > FE_CAN_QAM_16 | > FE_CAN_QAM_64 | > FE_CAN_QAM_32 | > FE_CAN_QAM_128 | > FE_CAN_QAM_256 | > FE_CAN_QAM_AUTO | > FE_CAN_QPSK | > FE_CAN_FEC_AUTO | > FE_CAN_INVERSION_AUTO | > FE_CAN_TRANSMISSION_MODE_AUTO | > FE_CAN_GUARD_INTERVAL_AUTO | > FE_CAN_HIERARCHY_AUTO, > } > > .... > > Using dvb-scan > -------------- > > In order to tune into a channel and read the PSI tables, we can use dvb-scan. > > For this, one should provide a configuration file known as a 'scan file', > here's an example:: > > [Channel] > FREQUENCY = 330000000 > MODULATION = QAM/AUTO > SYMBOL_RATE = 6940000 > INNER_FEC = AUTO > DELIVERY_SYSTEM = DVBC/ANNEX_A > > NOTE: > The parameters depend on the video standard you're testing. > > NOTE: > Vidtv is a fake driver and does not validate much of the information > in the scan file. Just specifying 'FREQUENCY' and 'DELIVERY_SYSTEM' > should be enough for DVB-T/DVB-T2. For DVB-S/DVB-C however, you > should also provide 'SYMBOL_RATE'. > > Assuming this channel is named 'channel.conf', you can then run:: > > $ dvbv5-scan dresden_dvbc_channel.conf > > Using dvb-zap > ------------- > > dvbv5-zap is a command line tool that can be used to record MPEG-TS to disk. The > typical use is to tune into a channel and put it into record mode. The example > below - which is taken from the documentation - illustrates that:: > > $ dvbv5-zap -c dvb_channel.conf "trilhas sonoras" -r > using demux '/dev/dvb/adapter0/demux0' > reading channels from file 'dvb_channel.conf' > service has pid type 05: 204 > tuning to 573000000 Hz > audio pid 104 > dvb_set_pesfilter 104 > Lock (0x1f) Quality= Good Signal= 100.00% C/N= -13.80dB UCB= 70 postBER= 3.14x10^-3 PER= 0 > DVR interface '/dev/dvb/adapter0/dvr0' can now be opened > > The channel can be watched by playing the contents of the DVR interface, with > some player that recognizes the MPEG-TS format, such as *mplayer* or *vlc*. > > By playing the contents of the stream one can visually inspect the workings of > vidtv, e.g.:: > > $ mplayer /dev/dvb/adapter0/dvr0 > > > Daniel W. S. Almeida (4): > media: vidtv: implement a tuner driver > media: vidtv: implement a demodulator driver > media: vidtv: add a bridge driver > media: Documentation: vidtv: Add ReST documentation for vidtv > > .../driver-api/media/drivers/index.rst | 1 + > .../driver-api/media/drivers/vidtv.rst | 417 +++++ > MAINTAINERS | 8 + > drivers/media/test-drivers/Kconfig | 16 + > drivers/media/test-drivers/Makefile | 1 + > drivers/media/test-drivers/vidtv/Kconfig | 11 + > drivers/media/test-drivers/vidtv/Makefile | 9 + > .../media/test-drivers/vidtv/vidtv_bridge.c | 546 +++++++ > .../media/test-drivers/vidtv/vidtv_bridge.h | 60 + > .../media/test-drivers/vidtv/vidtv_channel.c | 306 ++++ > .../media/test-drivers/vidtv/vidtv_channel.h | 76 + > .../media/test-drivers/vidtv/vidtv_common.c | 89 ++ > .../media/test-drivers/vidtv/vidtv_common.h | 33 + > .../media/test-drivers/vidtv/vidtv_demod.c | 440 ++++++ > .../media/test-drivers/vidtv/vidtv_demod.h | 73 + > .../media/test-drivers/vidtv/vidtv_encoder.h | 96 ++ > drivers/media/test-drivers/vidtv/vidtv_mux.c | 479 ++++++ > drivers/media/test-drivers/vidtv/vidtv_mux.h | 160 ++ > drivers/media/test-drivers/vidtv/vidtv_pes.c | 398 +++++ > drivers/media/test-drivers/vidtv/vidtv_pes.h | 189 +++ > drivers/media/test-drivers/vidtv/vidtv_psi.c | 1352 +++++++++++++++++ > drivers/media/test-drivers/vidtv/vidtv_psi.h | 593 ++++++++ > .../media/test-drivers/vidtv/vidtv_s302m.c | 552 +++++++ > .../media/test-drivers/vidtv/vidtv_s302m.h | 90 ++ > drivers/media/test-drivers/vidtv/vidtv_ts.c | 137 ++ > drivers/media/test-drivers/vidtv/vidtv_ts.h | 130 ++ > .../media/test-drivers/vidtv/vidtv_tuner.c | 427 ++++++ > .../media/test-drivers/vidtv/vidtv_tuner.h | 43 + > 28 files changed, 6732 insertions(+) > create mode 100644 Documentation/driver-api/media/drivers/vidtv.rst > create mode 100644 drivers/media/test-drivers/vidtv/Kconfig > create mode 100644 drivers/media/test-drivers/vidtv/Makefile > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_bridge.c > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_bridge.h > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_channel.c > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_channel.h > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_common.c > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_common.h > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_demod.c > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_demod.h > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_encoder.h > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_mux.c > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_mux.h > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_pes.c > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_pes.h > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_psi.c > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_psi.h > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_s302m.c > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_s302m.h > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_ts.c > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_ts.h > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_tuner.c > create mode 100644 drivers/media/test-drivers/vidtv/vidtv_tuner.h >
Hi Daniel, On Fri, 21 Aug 2020, Daniel W. S. Almeida wrote: > From: "Daniel W. S. Almeida" <dwlsalmeida@gmail.com> > > Digital TV devices consist of several independent hardware components > which are controlled by different drivers. > Each media device is controlled by a group of cooperating drivers with the > bridge driver as the main driver. > > This patch adds a bridge driver for the Virtual Digital TV driver [vidtv]. This is now commit f90cf6079bf67988 ("media: vidtv: add a bridge driver") in the media tree. noreply@ellerman.id.au reported the following error for an m68k allmodconfig build: ERROR: modpost: "__udivdi3" [drivers/media/test-drivers/vidtv/dvb-vidtv-bridge.ko] undefined! Presumably this fails on other 32-bit platforms, too. > --- /dev/null > +++ b/drivers/media/test-drivers/vidtv/vidtv_mux.c > +static u32 vidtv_mux_check_mux_rate(struct vidtv_mux *m) > +{ > + /* > + * attempt to maintain a constant mux rate, padding with null packets > + * if needed > + */ > + > + u32 nbytes = 0; /* the number of bytes written by this function */ > + > + u64 nbytes_expected; /* the number of bytes we should have written */ > + u64 nbytes_streamed; /* the number of bytes we actually wrote */ > + u32 num_null_pkts; /* number of null packets to bridge the gap */ > + > + u64 elapsed_time_msecs = jiffies_to_usecs(m->timing.current_jiffies - > + m->timing.past_jiffies); > + > + elapsed_time_msecs = min(elapsed_time_msecs, (u64)VIDTV_MAX_SLEEP_USECS / 1000); > + nbytes_expected = div64_u64(m->mux_rate_kbytes_sec * 1000, MSEC_PER_SEC); Seriously?!? You multiply by 1000 first, followed by a division by 1000 using an expensive 64-by-64 division? > + nbytes_expected *= elapsed_time_msecs; > + > + nbytes_streamed = m->mux_buf_offset; > + > + if (nbytes_streamed < nbytes_expected) { > + /* can't write half a packet: roundup to a 188 multiple */ > + nbytes_expected = roundup(nbytes_expected - nbytes_streamed, TS_PACKET_LEN); drivers/media/test-drivers/vidtv/vidtv_mux.o: In function `vidtv_mux_tick': vidtv_mux.c:(.text+0x788): undefined reference to `__udivdi3' This is a 64-by-32 division, hence it should use a helper from <linux/math64.h>. However, I'm wondering if "nbytes_expected - nbytes_streamed" is guaranteed to be a "small" number, hence a 32-by-32 division would be sufficient? > + num_null_pkts = nbytes_expected / TS_PACKET_LEN; Likewise. > + nbytes += vidtv_mux_pad_with_nulls(m, num_null_pkts); > + } > + > + return nbytes; > +} > --- /dev/null > +++ b/drivers/media/test-drivers/vidtv/vidtv_s302m.c > +static void vidtv_s302m_compute_pts(struct vidtv_encoder *e) > +{ > + u64 count = e->sample_count; > + struct vidtv_access_unit *au = e->access_units; > + > + while (au) { > + count += au->num_samples; > + > + au->pts = count * > + CLOCK_UNIT_90KHZ / e->sampling_rate_hz; drivers/media/test-drivers/vidtv/vidtv_s302m.o: In function `vidtv_s302m_encode': vidtv_s302m.c:(.text+0x2ac): undefined reference to `__udivdi3' Likewise. > + > + au = au->next; > + } > +} Gr{oetje,eeting}s, Geert -- Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Hi Geert, Thanks for bringing this to my attention. >> + u32 nbytes = 0; /* the number of bytes written by this function */ >> + >> + u64 nbytes_expected; /* the number of bytes we should have written */ >> + u64 nbytes_streamed; /* the number of bytes we actually wrote */ >> + u32 num_null_pkts; /* number of null packets to bridge the gap */ >> + >> + u64 elapsed_time_msecs = jiffies_to_usecs(m->timing.current_jiffies - >> + m->timing.past_jiffies); >> + >> + elapsed_time_msecs = min(elapsed_time_msecs, >> (u64)VIDTV_MAX_SLEEP_USECS / 1000); >> + nbytes_expected = div64_u64(m->mux_rate_kbytes_sec * 1000, MSEC_PER_SEC); > > Seriously?!? > > You multiply by 1000 first, followed by a division by 1000 using an > expensive 64-by-64 division? This entire function is broken and needs a do-over :) > using an expensive 64-by-64 division? I am new to kernel development. I wasn't even aware that this was expensive, to be honest. >> + if (nbytes_streamed < nbytes_expected) { >> + /* can't write half a packet: roundup to a 188 multiple */ >> + nbytes_expected = roundup(nbytes_expected - nbytes_streamed, TS_PACKET_LEN); > > drivers/media/test-drivers/vidtv/vidtv_mux.o: In function `vidtv_mux_tick': > vidtv_mux.c:(.text+0x788): undefined reference to `__udivdi3' > > This is a 64-by-32 division, hence it should use a helper from > <linux/math64.h>. > > However, I'm wondering if "nbytes_expected - nbytes_streamed" is > guaranteed to be a "small" number, hence a 32-by-32 division would be > sufficient? I think so. I will send a patch to address the things you pointed out in this email. -- thanks -- Daniel
Hi Daniel, On Tue, Sep 15, 2020 at 3:26 PM Daniel W. S. Almeida <dwlsalmeida@gmail.com> wrote: > >> + u32 nbytes = 0; /* the number of bytes written by this function */ > >> + > >> + u64 nbytes_expected; /* the number of bytes we should have written */ > >> + u64 nbytes_streamed; /* the number of bytes we actually wrote */ > >> + u32 num_null_pkts; /* number of null packets to bridge the gap */ > >> + > >> + u64 elapsed_time_msecs = jiffies_to_usecs(m->timing.current_jiffies - > >> + m->timing.past_jiffies); > >> + > >> + elapsed_time_msecs = min(elapsed_time_msecs, > >> (u64)VIDTV_MAX_SLEEP_USECS / 1000); > >> + nbytes_expected = div64_u64(m->mux_rate_kbytes_sec * 1000, MSEC_PER_SEC); > > > > Seriously?!? > > > > You multiply by 1000 first, followed by a division by 1000 using an > > expensive 64-by-64 division? > > This entire function is broken and needs a do-over :) > > > using an expensive 64-by-64 division? > > I am new to kernel development. I wasn't even aware that this was > expensive, to be honest. All divisions involving 64-bit data are expensive, especially on 32-bit platforms. That's why we have the helpers in <linux/math.h>. Most of them implement simplified variants, which are less expensive. > >> + if (nbytes_streamed < nbytes_expected) { > >> + /* can't write half a packet: roundup to a 188 multiple */ > >> + nbytes_expected = roundup(nbytes_expected - nbytes_streamed, TS_PACKET_LEN); > > > > drivers/media/test-drivers/vidtv/vidtv_mux.o: In function `vidtv_mux_tick': > > vidtv_mux.c:(.text+0x788): undefined reference to `__udivdi3' > > > > This is a 64-by-32 division, hence it should use a helper from > > <linux/math64.h>. > > > > However, I'm wondering if "nbytes_expected - nbytes_streamed" is > > guaranteed to be a "small" number, hence a 32-by-32 division would be > > sufficient? > > I think so. > > I will send a patch to address the things you pointed out in this email. Thanks, looking forward to it! Gr{oetje,eeting}s, Geert -- Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Hi Mauro, On Wed, Sep 16, 2020 at 9:01 AM Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote: > Em Tue, 15 Sep 2020 15:35:00 +0200 > Geert Uytterhoeven <geert@linux-m68k.org> escreveu: > > On Tue, Sep 15, 2020 at 3:26 PM Daniel W. S. Almeida > > <dwlsalmeida@gmail.com> wrote: > > > >> + u32 nbytes = 0; /* the number of bytes written by this function */ > > > >> + > > > >> + u64 nbytes_expected; /* the number of bytes we should have written */ > > > >> + u64 nbytes_streamed; /* the number of bytes we actually wrote */ > > > >> + u32 num_null_pkts; /* number of null packets to bridge the gap */ > > > >> + > > > >> + u64 elapsed_time_msecs = jiffies_to_usecs(m->timing.current_jiffies - > > > >> + m->timing.past_jiffies); > > > >> + > > > >> + elapsed_time_msecs = min(elapsed_time_msecs, > > > >> (u64)VIDTV_MAX_SLEEP_USECS / 1000); > > > >> + nbytes_expected = div64_u64(m->mux_rate_kbytes_sec * 1000, MSEC_PER_SEC); > > > > > > > > Seriously?!? > > > > > > > > You multiply by 1000 first, followed by a division by 1000 using an > > > > expensive 64-by-64 division? > > > > > > This entire function is broken and needs a do-over :) > > > > > > > using an expensive 64-by-64 division? > > > > > > I am new to kernel development. I wasn't even aware that this was > > > expensive, to be honest. > > > > All divisions involving 64-bit data are expensive, especially on 32-bit > > platforms. That's why we have the helpers in <linux/math.h>. Most > > of them implement simplified variants, which are less expensive. > > I agree that 64-bit math is something that should be used with some > care. However, it is almost unavoidable do to 64-bit divisions for > digital TV. Sure. If 64-bit math is needed, it should be used. The macros (and the link failure on 32-bit) exist to (a) make sure people think twice before using 64-math, and (b) let people pick a less-expensive variant if that is sufficient. Gr{oetje,eeting}s, Geert -- Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
From: "Daniel W. S. Almeida" <dwlsalmeida@gmail.com> This series is work in progress. It represents the current work done on a virtual DVB driver for the Linux media subsystem. I am new to the media subsystem and to kernel development in general. This driver aims to: - Serve as template for new DVB driver writers - Help userspace application writers in general - Push fake audio/video to userspace for testing purposes - Push debug information to userspace via debugfs Current state for this driver: - Driver generates PSI information (PAT, PMT, SDT) - Driver generates PCR packets - Driver generates NULL packets for padding - PCM audio stream is decoded by ffmpeg, but no audio is heard yet. changes in v10: s302m encoder got reworked The Virtual Digital TV Driver (vidtv) Background ---------- Vidtv is a virtual DVB driver that aims to serve as a reference for driver writers by serving as a template. It also validates the existing media DVB APIs, thus helping userspace application writers. Currently, it consists of: - A fake tuner driver, which will report a bad signal quality if the chosen frequency is too far away from a table of valid frequencies for a particular delivery system. - A fake demod driver, which will constantly poll the fake signal quality returned by the tuner, simulating a device that can lose/reacquire a lock on the signal depending on the CNR levels. - A fake bridge driver, which is the module responsible for modprobing the fake tuner and demod modules and implementing the demux logic. This module takes parameters at initialization that will dictate how the simulation behaves. - Code reponsible for encoding a valid MPEG Transport Stream, which is then passed to the bridge driver. This fake stream contains some hardcoded content. For now, we have a single, audio-only channel containing a single MPEG Elementary Stream, which in turn contains a SMPTE 302m encoded sine-wave. Note that this particular encoder was chosen because it is the easiest way to encode PCM audio data in a MPEG Transport Stream. Building vidtv -------------- vidtv is a test driver and thus is **not** enabled by default when compiling the kernel. In order to enable compilation of vidtv: - Enable **DVB_TEST_DRIVERS**, then - Enable **DVB_VIDTV** When compiled as a module, expect the following .ko files: - dvb_vidtv_tuner.ko - dvb_vidtv_demod.ko - dvb_vidtv_bridge.ko Running vidtv ------------- When compiled as a module, run:: modprobe dvb_vidtv_bridge That's it! The bridge driver will initialize the tuner and demod drivers as part of its own initialization. You can optionally define some command-line arguments to vidtv, see the documentation for more info. Testing vidtv with v4l-utils ============================ Start by installing v4l-utils and then modprobing vidtv:: modprobe dvb_vidtv_bridge If the driver is OK, it should load and its probing code will run. This will pull in the tuner and demod drivers. Using dvb-fe-tool ----------------- The first step to check whether the demod loaded successfully is to run:: $ dvb-fe-tool This should return what is currently set up at the demod struct, i.e.:: static const struct dvb_frontend_ops vidtv_demod_ops = { .delsys = { SYS_DVBT, SYS_DVBT2, SYS_DVBC_ANNEX_A, SYS_DVBS, SYS_DVBS2, }, .info = { .name = "Dummy demod for DVB-T/T2/C/S/S2", .frequency_min_hz = 51 * MHz, .frequency_max_hz = 2150 * MHz, .frequency_stepsize_hz = 62500, .frequency_tolerance_hz = 29500 * kHz, .symbol_rate_min = 1000000, .symbol_rate_max = 45000000, .caps = FE_CAN_FEC_1_2 | FE_CAN_FEC_2_3 | FE_CAN_FEC_3_4 | FE_CAN_FEC_4_5 | FE_CAN_FEC_5_6 | FE_CAN_FEC_6_7 | FE_CAN_FEC_7_8 | FE_CAN_FEC_8_9 | FE_CAN_QAM_16 | FE_CAN_QAM_64 | FE_CAN_QAM_32 | FE_CAN_QAM_128 | FE_CAN_QAM_256 | FE_CAN_QAM_AUTO | FE_CAN_QPSK | FE_CAN_FEC_AUTO | FE_CAN_INVERSION_AUTO | FE_CAN_TRANSMISSION_MODE_AUTO | FE_CAN_GUARD_INTERVAL_AUTO | FE_CAN_HIERARCHY_AUTO, } .... Using dvb-scan -------------- In order to tune into a channel and read the PSI tables, we can use dvb-scan. For this, one should provide a configuration file known as a 'scan file', here's an example:: [Channel] FREQUENCY = 330000000 MODULATION = QAM/AUTO SYMBOL_RATE = 6940000 INNER_FEC = AUTO DELIVERY_SYSTEM = DVBC/ANNEX_A NOTE: The parameters depend on the video standard you're testing. NOTE: Vidtv is a fake driver and does not validate much of the information in the scan file. Just specifying 'FREQUENCY' and 'DELIVERY_SYSTEM' should be enough for DVB-T/DVB-T2. For DVB-S/DVB-C however, you should also provide 'SYMBOL_RATE'. Assuming this channel is named 'channel.conf', you can then run:: $ dvbv5-scan dresden_dvbc_channel.conf Using dvb-zap ------------- dvbv5-zap is a command line tool that can be used to record MPEG-TS to disk. The typical use is to tune into a channel and put it into record mode. The example below - which is taken from the documentation - illustrates that:: $ dvbv5-zap -c dvb_channel.conf "trilhas sonoras" -r using demux '/dev/dvb/adapter0/demux0' reading channels from file 'dvb_channel.conf' service has pid type 05: 204 tuning to 573000000 Hz audio pid 104 dvb_set_pesfilter 104 Lock (0x1f) Quality= Good Signal= 100.00% C/N= -13.80dB UCB= 70 postBER= 3.14x10^-3 PER= 0 DVR interface '/dev/dvb/adapter0/dvr0' can now be opened The channel can be watched by playing the contents of the DVR interface, with some player that recognizes the MPEG-TS format, such as *mplayer* or *vlc*. By playing the contents of the stream one can visually inspect the workings of vidtv, e.g.:: $ mplayer /dev/dvb/adapter0/dvr0 Daniel W. S. Almeida (4): media: vidtv: implement a tuner driver media: vidtv: implement a demodulator driver media: vidtv: add a bridge driver media: Documentation: vidtv: Add ReST documentation for vidtv .../driver-api/media/drivers/index.rst | 1 + .../driver-api/media/drivers/vidtv.rst | 417 +++++ MAINTAINERS | 8 + drivers/media/test-drivers/Kconfig | 16 + drivers/media/test-drivers/Makefile | 1 + drivers/media/test-drivers/vidtv/Kconfig | 11 + drivers/media/test-drivers/vidtv/Makefile | 9 + .../media/test-drivers/vidtv/vidtv_bridge.c | 546 +++++++ .../media/test-drivers/vidtv/vidtv_bridge.h | 60 + .../media/test-drivers/vidtv/vidtv_channel.c | 306 ++++ .../media/test-drivers/vidtv/vidtv_channel.h | 76 + .../media/test-drivers/vidtv/vidtv_common.c | 89 ++ .../media/test-drivers/vidtv/vidtv_common.h | 33 + .../media/test-drivers/vidtv/vidtv_demod.c | 440 ++++++ .../media/test-drivers/vidtv/vidtv_demod.h | 73 + .../media/test-drivers/vidtv/vidtv_encoder.h | 96 ++ drivers/media/test-drivers/vidtv/vidtv_mux.c | 479 ++++++ drivers/media/test-drivers/vidtv/vidtv_mux.h | 160 ++ drivers/media/test-drivers/vidtv/vidtv_pes.c | 398 +++++ drivers/media/test-drivers/vidtv/vidtv_pes.h | 189 +++ drivers/media/test-drivers/vidtv/vidtv_psi.c | 1352 +++++++++++++++++ drivers/media/test-drivers/vidtv/vidtv_psi.h | 593 ++++++++ .../media/test-drivers/vidtv/vidtv_s302m.c | 552 +++++++ .../media/test-drivers/vidtv/vidtv_s302m.h | 90 ++ drivers/media/test-drivers/vidtv/vidtv_ts.c | 137 ++ drivers/media/test-drivers/vidtv/vidtv_ts.h | 130 ++ .../media/test-drivers/vidtv/vidtv_tuner.c | 427 ++++++ .../media/test-drivers/vidtv/vidtv_tuner.h | 43 + 28 files changed, 6732 insertions(+) create mode 100644 Documentation/driver-api/media/drivers/vidtv.rst create mode 100644 drivers/media/test-drivers/vidtv/Kconfig create mode 100644 drivers/media/test-drivers/vidtv/Makefile create mode 100644 drivers/media/test-drivers/vidtv/vidtv_bridge.c create mode 100644 drivers/media/test-drivers/vidtv/vidtv_bridge.h create mode 100644 drivers/media/test-drivers/vidtv/vidtv_channel.c create mode 100644 drivers/media/test-drivers/vidtv/vidtv_channel.h create mode 100644 drivers/media/test-drivers/vidtv/vidtv_common.c create mode 100644 drivers/media/test-drivers/vidtv/vidtv_common.h create mode 100644 drivers/media/test-drivers/vidtv/vidtv_demod.c create mode 100644 drivers/media/test-drivers/vidtv/vidtv_demod.h create mode 100644 drivers/media/test-drivers/vidtv/vidtv_encoder.h create mode 100644 drivers/media/test-drivers/vidtv/vidtv_mux.c create mode 100644 drivers/media/test-drivers/vidtv/vidtv_mux.h create mode 100644 drivers/media/test-drivers/vidtv/vidtv_pes.c create mode 100644 drivers/media/test-drivers/vidtv/vidtv_pes.h create mode 100644 drivers/media/test-drivers/vidtv/vidtv_psi.c create mode 100644 drivers/media/test-drivers/vidtv/vidtv_psi.h create mode 100644 drivers/media/test-drivers/vidtv/vidtv_s302m.c create mode 100644 drivers/media/test-drivers/vidtv/vidtv_s302m.h create mode 100644 drivers/media/test-drivers/vidtv/vidtv_ts.c create mode 100644 drivers/media/test-drivers/vidtv/vidtv_ts.h create mode 100644 drivers/media/test-drivers/vidtv/vidtv_tuner.c create mode 100644 drivers/media/test-drivers/vidtv/vidtv_tuner.h