RPi2 LiveTV Problem - VDR Restart itself over watchdog #545
Comments
|
Pretty sure this is not a firmware issue. The firmware won't cause error messages to appear in an arm side log. |
|
It could be a firmware thing if it is decoding that throws an error and then the client doesn't behave nicely. What is the timeline your log for when you interacted with it? Did you do something at 16:43:17, or is 20:40:15 when you "returned home" and changed channel? My thoughts are that a state transition might stall on the OMX decoder, and VDR just restarts as a way to get around it. Without significantly more logs from the OMX interactions it's not possible to say. A full dump of the thread states when the watchdog gives up would be even better if you felt like using GDB or similar. |
|
I interacted with RPi2 on the evening at 20:45:15 and that was time I did returned home and changed channel. The stream from the ServusTV HD should be in the MPEG-4 format. I'll try to create a dump of the thread states when the watchdog gives up, but from VDR itself there is no userdump crated because its killed by watchdog and if I disable VDR's watchdog nothing will happen. And I am not so gut to use VDR through the gdb fully. Last night there was again problem that VDR restarts itself on zap. Though this time LiveTV was ok (video and audio in sync). But switch to other channels was not ok. Not that much to see in the log, as I was using --log=2 (which I did increased now to --log=3). I did also compiled the VDR output plugin rpihddevice with "make DEBUG_BUFFERSTAT=1" to have more stuff logged. Please not that here is the log from two days 16 and 17 Feb. I switched to another channel at Feb 17 20:13:47. Until that Audio/Video was ok on that ServusTV HD channel.
|
|
Today same thing, today (as I installed debug packages) there is an coredump. ilclient is an abstraction layer from OMX IL and was developed from Broadcom ?? - maybe someone have an idea why the VDR is hanging there?
|
|
Yes, ilclient was developed at Broadcom as a wrapper to simplify some of the IL calls. VDR is executing a PortDisable on port 130 (video_decode input port), which also flushes the codec. Completing that command requires all buffers to be returned to the buffer supplier - by the looks of it, your client is likely to be the buffer supplier (it calls UseBuffer instead of AllocateBuffer). You can try enabling a load of GPU logging by running Whilst not bullet-proof, ilclient and the IL services are pretty resilient as long as you communicate with them in the correct way. Running on other than the supported Raspbian distribution also means we're less likely to try to reproduce, but will give some guidance. |
|
New day without interacting with VDR and the then when changing to another channel watchdog kills VDR. Tonight there was no coredump created. I have just VDR log (see it below). Here is the output from vcdbg log msg (is there any parameter like -T for dmesg to show actual time?)
|
|
I've not had the time to analyse the logs fully, but nothing stands out. In all your examples you've left VDR tuned to "ServusTV HD Oesterreich". Does it happen on other channels? Preferably using a different codec, as this may be something in the MPEG4 side that is throwing a wobbly. I just want to reduce the number of variables. Does VDR need a Pi2? If it will run under Raspbian I may try setting up a spare B+ with a DVB-S2 tuner on 28.2degE (as that is where my satellite dish points). It's likely to be slow and painful though if it really takes hours to fail. |
|
I've observed the error as well for a couple of times and it does not relate to the channel nor the video codec. But it always takes a day or more to happen, so it's hard to debug. Personally I use Gentoo for development also on the Raspberry, so it's definitely not about Arch-Linux. VDR runs perfectly fine on a B+, I wrote down the basic steps for the installation: #463 (comment) Thanks a lot for your help! |
|
OK, I'll try to set up a rig. Is there a way to disable the watchdog that is resetting VDR on timeout? I want the system to freeze in the bad state so I can grab logs and examine state. |
|
Sure, just start vdr manually from the console and add Let me know if you have any further question. You'll find my email in rpihddevice's source, don't hesitate to PM me. |
|
Cheers. I'll have a look. Hopefully it's just a buffer gone astray somewhere which will block PortDisable completing. |
|
Today there was no crash, but i was not able to switch from one another FTA mpeg4 to ServusTV HD channel, it was posible after few retrys.. See this gist paste, PS: does reboot of RPi2 change the 'vcgencmd set_logging level=0xc0' to default ones? If not what would be default one 'level=' Thank you all for this time. |
|
Yes, a reboot resets the log level. Default is probably 0x40010000. |
|
Ok, i set it back, there is something weird in 'dmesg -T'
I just stopped VDR, then flushed cache 'sudo /opt/vc/bin/vcgencmd cache_flush' , set the logging as written before 'sudo /opt/vc/bin/vcgencmd set_logging level=0xc0' Let see tomorrow if there will be something. |
|
So I disabled 'wachtdog=0' and tried today to switch channel, firstly I was 'ServusTV HD', then switched 'R9 Oesterreich HD' and this was working, then switching back to 'ServusTV HD' and then it then I lost video/audio. As there is no watchdog any more active even after 5min nothing is happening. Here is VDR and 'vcdbg log msg'. Can you see anything? |
|
That's a much more useful log, but nothing looks out of place. I know I want everything, but the stack traces would have been really useful there to see what VDR was waiting for. I'm just setting up a rig, though having realised I've got no spare satellite feeds in useful places it'll be DVB-T2 rather than DVB-S2. Seeing as this appears to be a decoder issue that shouldn't make any difference. |
|
Can i do the stack traces over gdb if befor switching to another channel I hook gdb to VDR process? If you have an guidelines what to do I can try that. |
|
I'm fighting to get this to do anything useful. (I can't quite believe that rpi-update has been omitted from the Raspbian Jessie Lite image either - that one's just annoying and seems an odd omission. I guess they want to use apt-get dist-upgrade instead). |
|
I'd followed your instructions which started VDR as a service, but seems not to have enabled the rpihddevice plugin. Trying to start it manually with -P rpihddevice I get a UI up, but it won't allow me to specify the telnet remote control. |
|
Some runes deduced thanks to Google. I've got caught out that you're using vcgencmd to check for MPEG2 support - I'd hacked my firmware in the IL component rather than enable a key (whistles innocently!). "video format is not supported". (We'll see if you can convince me that VDR is better than tvheadend. I think not as I mainly want it as a recording box with multiple DVB-T, DVB-T2, and DVB-S2 tuners. It's on an x86 box at the moment as I don't think a Pi would cope). |
Sure, if you have used apt-get to install VDR, there should be init scripts installed already. But since I'm not a friend of aptitude nor systemd, I'm afraid I can't help you here.
great to hear - hope you sleeped well!
I'm not going to try to convince anybody. ;-) I'm running a separate VDR instance on my x86 server in the cellar for recording, my Pi's are solely used as clients - and therefore they're perfect. And VDR is just a great PVR software: Slim, fast, stable, without any glamour but - in my view - the highest usability I've ever seen for this kind of software. |
|
So after 36 hours of no interaction, there was it again. Firstly switch to another channel was ok but LiveTV on that channel was not ok "ERROR: TS packet not accepted in Transfer Mode" then i switch back to ServusTV HD and the watchdog activated after 180 seconds. There was no coredump created, I tried to interact with gdb but I am not sure if there is anything useful. First 'vcdbg log msg' was when I have had "ERROR: TS packet not accepted in Transfer Mode" the second one after watchdog. After VDR restartet the ServusTV HD was working again just fine. |
|
Here so after 24h of no inetraction and trying to switch fort and back there is it again. Please look in gist inline comment: gistpaste Do you need anymore log or is this enough what i posted? Edit: Packages (6) Edit: Here is link to switch from another channel to ServusTV HD after 5 second on above channel, which failed: gistpaste |
|
There's nothing obvious in any of these logs, so little point adding more of the same. |
|
On first switch watchdog was activated: gistpaste |
|
Today after arriving home and Powering TV i found the Live Image was still and the audio croping. Here is gistlog: If you dont see anything, can i get somehow debug firmware to be able to create better log? |
|
While my productive Raspbperry 2 is running for weeks without any troubles, the new Raspberry 3 shows this issue quite frequently. I think it happens more often when stopping a replay, where buffer usage is hitting the maximum, whereas in live TV mode, the amount of used buffers is kept to a low value. Let me know if you need some more debug logs! |
|
I obtain the same problem as reufer. Those crashes happen more often when using Raspberry Pi 3 rather than RPI 2. |
|
With kernel a1a3be1 I was not able to reproduce the issue anymore - I guess the modified locking did the trick! |
|
@vrabac is this still an issue for you with latest (rpi-update) firmware/kernel? |
|
@popcornmix i'll try the new firmware/kernel ASAP and will write here the results, sorry for late replay! |
|
Everything seems to be fine right now with the kernel a1a3be1 Thank you |
Current situation:
VDR 2.3.1 is running for one day without zapping to any channel, at the evening when I come home if I do zap to another channel, VDR will restart itself over watchdog (after 90 seconds). Sometimes when I arrive at home the picture will be in freeze state (but sometimes there is audio going without any glitches), and again after rezap with VDR will restart itself over watchdog.
It happens on FTA channels.
I already contacted the VDR Developer and here is his statement:
If the RasPi is tuned to the same channel for an extended period of time, it shows exactly this behavior.
It can mostly be avoided by switching channels every couple of hours.
I don't think this is a VDR core problem, because it doesn't happen with other output devices (like the TT S2 6400, for instance).
Must be a firmware problem in the RasPi.
I am not including any other log then from VDR, as i am not sure what everything is needed to Debug this issue.
Here some Details:
Hardware:
Raspberry PI 2 Model B 1GB
PSU 5V / 2A - Micro USB Stecker
SkyTV Ultimate V 2016 (DVB-S2 USB Card)
Transcend TS32GUSDHC10E Class 10 Extreme-Speed microSDHC 32GB
SW:
ArchLinuxArm
Kernel: 4.1.17-4-ARCH
linux-raspberrypi-4.1.17-4
raspberrypi-firmware-20160216-1
raspberrypi-firmware-bootloader-20160216-1
raspberrypi-firmware-bootloader-x-20160216-1
raspberrypi-firmware-examples-20160216-1
raspberrypi-firmware-tools-20160216-1
The text was updated successfully, but these errors were encountered: