Raspberry hangs & reboots
Hi,
as already mentioned we're using Emteria in the private space to use a Raspberry as a Dart Scoreboard (using Scorer Apps).
Unfortunately we often have the problem that the Raspberry first hangs and then reboots loosing the current game stats. We have tested various power cables and various Raspberrys to eliminate error sources but now we don#t have any further idea. For sure we could spend some money to buy an official version of Emteria but you may understand that we first would like to get a confirmation that it really runs as expected. So do you have any tips/hints for use how to debug the system to find the error?
Thanks
0
Comments
Is same application running all the time or do you switch them occasionally? In the first case you should really check the power supply. We have recently performed a test, having an application running for about 7-9 days. It did hang once, we have recorded the problem and opened an issue at the official repo (https://github.com/raspberrypi/linux/issues/2392), but this problem has never returned. We have stopped the test after about 5 more days of continuous runtime. How long does it stay alive in your setup?
We're far away from that up-time. As stated we're using the raspberry as a Dart Scoreboard for our steel dart games and therefore a session is from 1 up to max. 6 hours.
We are only using one app here and this app is working absolutely fine on an Android phone and an Android tablet without any issues.
For the power supply .... we're using the official one for the Raspberry 3 with 5.1V and 2.5A ... to eliminate the possibility of a faulty one, we have tried at least three different power supplies with the same specifications.
Do you have any other idea?
It's not yet confirmed, but we got the feeling that it's a bot more stable when we deactivate all lan connections (cable and WIFI).
Official power supply should be fine. Did you try using another USB cable? Is anything else connected (GPIO/USB/touchscreen)?
Trying another USB cable is not an option as the cable is fix to the power supply and cannot be changed. Also when we tried the other power supplies for sure another cable was fixed to them =:-)
Connected is currently only a touchscreen:
DELL S2240T
But this cannot cause the issue as we also faced the problem when a simple TV was connected via HDMI. So independent if the touchscreen or a TV was connected, the Raspberry hangs.
Hmm, I see. We will release a new version tonight/tomorrow.
As soon as it is online, please do
I see you are from Germany. May I ask where exactly you are located?
Thanks a lot for your help so far (I will continue to reply in english for the benefit of internation readers. Will come to your question regarding my location later in this post).
What I now have done:
(only connected Logitect K400r keyborad incl. touchpad and HDMI)
So since now Emteria is running as a licensed version without Evaluation popup and support of updates, I guess =:-)
Questions which I now have:
We need to have the Google Apps (gapps) installed on the Raspberry to get our Dart Scoreboard app installed.For this we download the latest version from opengapps.org. (Platform: ARM / Android: 7.1 / Variant: Pico ... which is absuletly sufficient) and after downloading we restart the device in the recovery mode to get the package installed.
Question: Is that ok to do this now or would you first like to get some logs extracted?
We need to install the Dart Scoreboard app ( My Dart Training ) via Google Playstore.
Question: Is that ok to do this now or would you first like to get some logs extracted?
We also need to install Google drive on the device to import/export our dart statistics and also to get some caller sounds.
Question: Is that ok to do this now or would you first like to get some logs extracted?
Currently I'm running Emteria on a Raspberry which only is connected to HDMI TV and Logitech keyboard, but on Sunday we would like to use our mentioned touchscreen (Dell).
Question: Is that ok to do this now or would you first like to get some logs extracted?
Ok ... so far that's all from our side =:-) Thanks a lot again for the great support here!
(OT) - Off-Topic
Und nun zu der Frage, wo ich "located" bin =:-) Wir sind vier Leute, die sich regelmäßig zum Steel Dart spielen treffen und hier hatten wir nach einer Möglichkeit gesucht einfach Statistiken unserer Spiele zu erstellen. Auf unserer Suche haben wir dann die oben genannte App gefunden und sie erst auf einem Tablet eingesetzt. Nachteil: Man konnte nur schwer seine eigene Score sehen und daher hatten wir nach einem entsprechenden Touchbildschirm gesucht. Unsere Idee war, einen Raspberry an einen Touchscreen zu installieren und diese Vorrichtung dann in der Nähe der Scheibe aufzuhängen. So kann der jeweilige Spieler direkt seine Score sehen und ggf. auch wie er die Double-Outs werfen soll.
Wir leben in der Nähe von Dortmund ... genauer in direkter Nähe des Kamener-Kreuzes.
Der Name "Dieter Dart" ist natürlich nur unser Pseudoname und ich persönlich heiße Andre. Ich habe Informatik studiert und habe schon die eine oder andere Android-Variante auf dem Raspberry ausprobiert, aber Emteria ist definitiv die stabilste Variante.
Thanks for your answer. I am out of office today and will respond in detail as soon as I get back to my PC. Regarding installation of additional apps: please leave the board in this initial state for 10-20 hours. If everything goes well and it doesn't freeze or reboot, you can go ahead and install what you need. If it happens later, it will probably be caused by additional apps.
BTW: you don't need OpenGApps to download apps from Google Play Store. You can use the F-Droid in combination with Yalp Store: https://help.emteria.com/kb/emteria-os-fdroid-and-yalp-store
Hi Igor,
I like to give you a short update here to show what we found out so far.
Last Sunday we used the Pi again to gather our dart scorings and unfortunately the Pi crashed again from time to time. So far as we could see, this was NOT caused by overheating or something like this. There was no regularity which we could identify …. but … my personal feeling is that the issue is NOT caused by Emteria but by the Google Apps which we have to install. Your hint regarding F-Dorid and Yalp was an excellent one but unfortunately we have to install the Gapps on the Pi as otherwise the app will be only available as freeware with a lot of disturbing ads displayed within the app. For sure I also have contacted the app developer regarding this issue but as the app bases on the Google-API from the Google Play-Services to check in-app buys, we don’t have an option here so far. From my feeling I would say that the PI with Emteria was absolutely stable before we installed gapps.
… and again there is a BUT =:-)
It seems that the app is only checking once if the app is the full bought version or the freeware one, means that when the confirmation from Google Play-Services has been done, the service is no longer required as there seems to be set any flag/pointer within the app config which states that it is the premium version. Therefore I first tried to just install the Google-Playservices which seems to be impossible without doing this via Recovery mode as all found versions of the Google Play-Services APKs are marked as incompatible. Therefore I used again the way via Recovery Mode and installed the opengapps package. I started the Dart app and it was marked as premium.
And now there was a new version of Emteria available (vers. 0.5.10)
I installed this OTA update via Recovery Mode which also removes the bloody gapps again, which is excellent. Starting the Dart app again shows that it is still marked as premium without ads and furthermore I can see that the Google Apps have been removed.
So maybe this could be the way for us to get a working version running on the Pi =:-))))
Next Friday we will meet again to play a bit dart and after that I can report if this is the solution or if the Pi still hangs/reboots also when the gapps are not installed any longer.
Cross your fingers!
Hi Igor,
we now have some more new details after our next test and unfortunately I have to say that we still have the same issue. On Friday we played again darts and after some time we had the first reboot. To be honest we don’t have any idea where this could come from, as our system now have the following setup:
We used our Emteria on another Raspberry and therefore we can eliminate that the problem is caused by any hardware issue as we see this behavior on both Raspberry’s. As already stated we are using an official power supply and we tested two different of them. Therefore an issue with the power supply can also be eliminated. Next to the App … we have the app also running on an tablet and here we never have realized a crash; therefore I guess we also can eliminate the app as the problem root.
Last but not least the touch screen. IMHO this seems to be the only item which maybe could cause the problem. Just before the Pi reboots, we only hear some knocking sounds but we also can see that the touch is not recognized (we enabled “fingertouch displayed” via developer menu). Then after some seconds up to a minute the Pi reboots. In the time when the Pi hangs, we also realized that the clock stands still which could be an indicator that really the Pi hangs.
So now up to you again =:-)))) Do you have any ideas for us?
FYI: We now also set max_usb_current=1 in the config.txt as the touchscreen is connected via HDMI to display the screen and to enable the sound but connected via USB to use the touch. This has nit yet tested by us.
Hi Andre,
Thanks for the update. It sounds very weird.
Is it possible for you to setup a single device to stay in idle all the time? If I understood you correctly, you now have a device with the required application and you only use it on weekends. Can you connect it to a power source, start the app and leave it untouched for a couple of days?
I agree, that the problem is probably not the hardware and/or power supply. But I am not yet convinced it is not the sdcard and/or the application itself. What is the name of this application? I'll also setup a test on our side.
Hi Igor,
you're right ... it's a bit weird.
Anyway ... I will continue to test and troubleshot the issue. I now have done some more tests to validate that the SD card does not have a problem. I created an image via Win32 Disk Imager for Windows from our installed Emteria including the app and wrote this to another SD Card. This should give us some information if the issue is caused by a failed SD card.
Regarding the "idle test" ... I now have started one of my Raspberry with this new card and started the App. I will leave this till the weekend but not sure if this will cause the problem .... my feeling is that maybe the problem is caused by the touchscreen. I will make a video next time when the Raspberry hangs while using our app.
Regarding the app ... you can find it here:
My Dart Training
We have bought the premium version of the app without disturbing ads.
I will keep you informed regarding my tests. And thanks again for your support here.
You are very welcome. I hope we can identify the issue... After your message we have prepared a dedicated RPi for a test... and for now its running quite good:
Can you please share a link with more information about the touchscreen you use?
EDIT:
Sorry, I see you have mentioned it previously, it is the DELL S2240T, correct? How is the touch connected? USB? Did you change the video configuration (pixel format / density / resolution)?
Hi Igor,
now I have some more details for your regarding our issue. I followed your idea and started the Raspberry here at my home with only a TV connected and a keyboard.
I have started to my Dart Training app and leave the system as it is to validate if it’s working fine. From time to time I checked if the system is still running and I got the feeling that it’s running fine (still could see the screen from the app without a reboot). Unfortunately I did not realize that the system itself seems to hang again. I started the Pi on Tuesday and on Friday I thought it would be a good idea to make a screenshot from the up-time screen but then I realized that the system was frozen.
This brings me to the conclusion that the issue also is not caused by our usual connected touchscreen as the system hangs at my home with only a TV connected.
Then when we played again yesterday (Sunday), I also had the idea to test another app. I tried to install the app via YALP but now the system also hangs here.
[See video here]
So this will bring me to the conclusion that the freeze is also not caused by the My Dart Training app as the app was not running at that time.
As we thought that the last test confirms that not the app is causing the problems, we used again our standard app and funny enough the system was stabile for the whole evening (we played approx. 5 hours).
As a summary here’s a list what we have tested/validated so far:
1. USB Power Adapter
We have checked three different USB Power Adapters.
2. Raspberry PI
We have tested three different Raspberry PIs (all are model 3B)
3. Google Apps
We have not installed the gapps on the system
4. My Dart Training app
System also crashed in YALP without running My Dart Training app
5. Touchscreen
System also crashed with only a TV connected
6. LAN Connections
Regarding your question: Yes, it’s correct. We are using a DELL S2240T for the display and input. The touch is connected via USB and we did not changed the video settings/config.
Thank you very much for your help and support. I really appreciate that.
We are also trying to identify the problem at our side. After discarding several false positives we are pretty sure it all can be traced back to that one graphical memory issue (it explains your freeze during the installation of new apps with Yalp).
We added a couple of small improvements last week, such that the latest release is running quite stable for a while now:
However, we currently run it with a lower resolution to reduce the memory consumption. We will release a new version this week.
BTW: it seems to be the best way to monitor the system state: Settings -> About -> Status. At the bottom of the list there is a timer showing the uptime. As long as it is changing, the system is fine.
... and once again .. thank you for your excellent support here. I now have started the RPi again connected to a TV and I will check the idle time here leaving the dart app suspended in the background.
When you say that you have reduced the resolution, which we did not tried yet, which resolution did you use? 800x600? We also could give this a try to see if this will bring an improvement. Am I right that we can adjust the resolution by editing the config files like described in another entry here in the forum using a linux distro?
BTW. I now also enabled the button to allow beta updates... so if you need someone to try new betas, feel free to let me know =:-)
Short Intermediate Status
RPi is up and running since 33:30:33 without any hick ups.
https://drive.google.com/open?id=1fd5YSj4nWiYYZLf-9KxIsizprfjQj8ia
and we have some apps in the background:
https://drive.google.com/open?id=1L1iLzwsnagUdPJOnTFd2Mzo_auXZcPSq
Interesting. It could mean the memory is allocated by the application and not by the operating system... We have found a small leak in the driver, but we can't find how it is caused. It is currently our #1 priority
BTW: Our test system is running for over a week now:
our RPi was now stable for more than 86 hours:
https://drive.google.com/open?id=1aPLlKWooEKGH1W0QVTvVzGipqdK8TOWp
as the weekend is not far away and we will use the RPi again as our score board, I now have shutdown the RPi and will change the resolution to a lower one to test it again. Well keep you posted.
@Dieter_Dart, could you do a normal run with the latest version (0.5.12). We have introduced several important fixes, which should improve the behavior.
Hi,
sorry that I did not provide any update earlier but I was extremely busy the last days.
Nevertheless I like to provide some new details.
We had version 1.5.11 installed on our RPi and on Saturday (April 14th) we played dart from approx 2100 till approx 0400 (so the Pi was in use for around 7 hours) and we did not had any crash. The system was more or less stable. The only item which we realized at the beginning was that the used app crashed once or maybe two times .... but after that it was pretty stable.
Then last Sunday (April 22nd) we played from around 1900 till 2400 also without any crash. The only thing which we have seen during the use is that sometimes the app "stuttered". It seems that the system completely hangs again but after some seconds it came back and we could continue to use it.
I will now follow your recommendation and install the new version (1.5.12) to test that as well. Not sure if we could test it directly next weekend .... not sure if we will meet to play dart but I will keep you updated
Thanks a lot again so far for your excellent support here!
Thanks for the feedback. Could you please explain this in more detail:
So the application was running on the side and you didn't actually use it for your game? It would be great to have a bugreport after a crash or stuttering, but that requires a working internet connection. Would that be possible?
I will try
We started the system (Emteria) and started the My Dart Training app. As we had a guest player, we added this player to the list and while doing so the app was automatically closed, but only the app itself. The system continued to work. To avoid that some artifacts are left in the memory from the crashed app, we decided to quickly reboot the system. After reboot we used the same way to add the guest player and it crashed again and again we decided to reboot. Then in the third try, it worked.
IMHO this was not an issue of the system but more an issue of the app as only the app crashed. Unfortunately we have not created a bug report for this behavior but we will do so next time if this happens again. An internet connection should work. Usually we have Wifi available there which we only switch off during we play to avoid that this causes issues but to create a bug report we can enable it again.
Great, thank you. At least we can find out why the app crashes and maybe submit the problem to its developer.
short update:
Last Sunday we used again the RPi to count our games and before I started the App, I realized that a new version of Emteria has been published (1.5.13) so therefore I first updated to the new version. The update went very smooth without any issues. After updating the Emteria system, we have started to play darts and the system was very stable without any hiccups or reboots. Only thing which we observed ... the caller sounds (within the dart app, you can use MP3 sounds for the numbers) are chopped. For sure it's a bit tricky to explain this
As an example .... when the caller has to say "onehundredandfourty" you will hear "onehundredandfo" ... for sure this is not a big issue but would like to let you know.
Thanks for reporting, the sound issue should be fixed now. This change will be included into the next release.
Next update from the dart side
Yesterday we met again to play dart. We started at 2000 and played for around 7 hours. The system was the whole time stable and also the sound issue seems to be fixed. Currently we have v0.5.14 installed and this really seems to provide a fix for the sound issue. So from my point of view it seems that everything is now working as expected.
One additional side note ... at the beginning I tried to show a short video from YouTube by opening the browser and browsing to www.youtube.de. Just when the video tried to start, the RPi rebooted. Similar behavior which we had before. While I tried it again, it worked. I guess that this maybe again is an memory issue, but it is not relevant for our use
Just would like to let you know.