There is no cron in Android. Since have the full control of the system, we can add such tasks on many levels (kernel, android handler notifications, sleep functions, system deamons and more).
Thanks @kalkov but I went with a easier solution. Since I enabled ssh on the first day I just use a cron on my debian desktop and scheduled a reboot every 4 hours...
One of our business clients is running Fully Kiosk 24/7 for several months now without any issues. If it is a problem with our OS, we would be happy to fix it.
If you have SSH access to it, could you please run "getprop"?
You are running the latest version 0.6.0. The only freeze reason I was aware of was the leak of the GPU memory, but that should be fixed since 0.5.X. Just to make sure this problem didn't came back, we could monitor the memory state:
cat /d/dri/0/bo_stats
Run this command shortly after reboot and one more time 3 hours later. Do you know the frequency of freezes, if you don't reboot it? It would be great to see the memory state shortly before it happens.
Well, if I added it all up correctly, there is almost 117 MB allocated (plus 10 MB of cache). Since Raspberry Pi only has 256 MB in total, it should be OK for now.
I am not aware of a ready-to-use tail command for such cases, but you could run something like
while true; do cat /d/dri/0/bo_stats; sleep 10; done
Hi, as I wrote in another thread about GPU issues, I am experiencing the total system freezes in some graphics applications too. I am using display 800x480 so I think, it cannot be caused by insufficient GPU RAM. Mapfactor navigator switched to software rendering works fine but if I turn on hardware rendering then system hangs. Sometimes after one minute, sometimes after one hour.
@Martin said:
I think, it cannot be caused by insufficient GPU RAM. Mapfactor navigator switched to software rendering works fine but if I turn on hardware rendering then system hangs.
This is exactly what GPU memory is used for.
@sgrayban said:
Here is the stats output of the last run.
I use two navigation applications. The first one is Mapfactor Navigator using free open street maps. This application is great for testing because it enables switching between software and hardware rendering (menu->settings->advanced settings) and it contains simulation mode (menu->tools->simulation) - so you can simulate X-hours trip on the table in lab. This application froze together with whole system every time if I used hardware rendering, but time of freezing is very random. Maybe month ago, system froze immediately after start of Mapfactor. Yesterday, I sent my Raspberry to 6-hours virtual trip (in simulation mode) to another country. Device arrived to virtual destination but couple of minutes after arriving system froze again, in time when content of display was static.
Second application is Waze. It would be better for reproducing this issue, because it freezes sooner, mostly in one hour, but it happens always in the car if I am driving. I don't know, if issue is reproducible in the lab, maybe some mock location tool could help. But Waze has second Emteria-specific issue - map flashing. It is reproducible very easily by zooming out in the map or switching to 3D. It's possible that map flashing and freezing has one common source... maybe.
Hi, I found good source of freezing yesterday. If I attach GPS receiver and run Waze on the table (so device is not moving), switch map to 2D (3D is not working) and start navigation to somewhere then whole system froze in 10 minutes. Always. I checked bo_stats and all objects took about 130MB. Logcat showed nothing. @kalkov , please could you verify if it is SD card controller issue or something else?
Comments
Why would you want to use cron? In Android, you can just write an app and have it do something periodically.
There is no cron in Android. Since have the full control of the system, we can add such tasks on many levels (kernel, android handler notifications, sleep functions, system deamons and more).
What you need for your application is JobScheduler: https://developer.android.com/topic/performance/scheduling
Thanks @kalkov but I went with a easier solution. Since I enabled ssh on the first day I just use a cron on my debian desktop and scheduled a reboot every 4 hours...
@sgrayban, may I ask why you need to reboot your Pi every 4 hours?
System hangs if it runs for long time. I haven't been able to figure out why.
That shouldn't happen. Which OS version are you using? Do you still have a SSH access after it freezes?
Not sure of the version but when it freezes it locks everything. No ssh or a ping.
You can check the version in Settings -> Emteria.
Do you have any application running all the time?
I have a few running in the background but Fully Browser Kiosk is running 24/7
I'm not at the location the PI is so I can't see it.
One of our business clients is running Fully Kiosk 24/7 for several months now without any issues. If it is a problem with our OS, we would be happy to fix it.
If you have SSH access to it, could you please run "getprop"?
You are running the latest version 0.6.0. The only freeze reason I was aware of was the leak of the GPU memory, but that should be fixed since 0.5.X. Just to make sure this problem didn't came back, we could monitor the memory state:
cat /d/dri/0/bo_stats
Run this command shortly after reboot and one more time 3 hours later. Do you know the frequency of freezes, if you don't reboot it? It would be great to see the memory state shortly before it happens.
It's random... sometimes just a few minutes, 30 minutes or hours later.
Is there a way to tail a log and keep a running window open so I can capture it at failure time ?
current stats are.........
rpi3:/ # cat /d/dri/0/bo_stats kernel: 5120kb BOs (1) V3D shader: 320kb BOs (80) binner: 16384kb BOs (1) resource: 13864kb BOs (64) resource 65536x1@8/0: 448kb BOs (7) scanout resource 1152x864@32/0: 7776kb BOs (2) mesa cache: 5012kb BOs (18) resource 1048576x1@8/0: 4096kb BOs (4) resource 1152x864@32/0: 34992kb BOs (9) resource 576x576@32/0: 1296kb BOs (1) resource 131072x1@8/0: 512kb BOs (4) resource 128x1@8/7: 32kb BOs (4) resource 1024x512@8/10: 2052kb BOs (3) resource 1x1@32/0: 148kb BOs (12) resource 81x81@32/6: 240kb BOs (4) resource 128x20@8/7: 8kb BOs (1) resource 96x20@8/6: 8kb BOs (1) resource 80x18@8/6: 8kb BOs (1) resource 64x20@8/6: 8kb BOs (1) resource 64x18@8/6: 8kb BOs (1) resource 112x14@8/6: 8kb BOs (1) resource 80x14@8/6: 8kb BOs (1) resource 64x12@8/6: 16kb BOs (2) resource 48x14@8/5: 8kb BOs (1) resource 27x27@32/4: 32kb BOs (4) resource 14x20@8/4: 16kb BOs (2) resource 20x19@32/4: 8kb BOs (1) resource 21x19@32/4: 8kb BOs (1) resource 19x19@32/4: 8kb BOs (1) resource 1280x864@32/0: 4320kb BOs (1) resource 1152x54@32/0: 864kb BOs (3) resource 1152x27@32/0: 432kb BOs (3) resource 1152x810@32/0: 3648kb BOs (1) resource 162x162@32/7: 464kb BOs (2) resource 1152x896@32/10: 13528kb BOs (2) tiling shadow 1152x864: 3888kb BOs (1) userspace BO cache: 5012kb BOs (18)Well, if I added it all up correctly, there is almost 117 MB allocated (plus 10 MB of cache). Since Raspberry Pi only has 256 MB in total, it should be OK for now.
I am not aware of a ready-to-use tail command for such cases, but you could run something like
while true; do cat /d/dri/0/bo_stats; sleep 10; done
Just tested it - works
But a newline would be great:
while true; do cat /d/dri/0/bo_stats; echo ""; sleep 10; doneI'm surprised that android doesn't have a syslog.
I modified it
while true; do cat /d/dri/0/bo_stats;echo "##############NEW STATS###########";echo ""; sleep 10; doneAndroid has "logcat" (or "logcat -d"), which doesn't include kernel information. You could also do "dmesg", but I think there is no continuous output.
@Martin Yup I think that's the issue.
I left the PI running all night with the screen powered off. The minute I turned the screen on the system froze.
Here is the stats output of the last run.
##############NEW STATS########### kernel: 5120kb BOs (1) V3D shader: 272kb BOs (68) binner: 16384kb BOs (1) RCL: 4kb BOs (1) BCL: 4kb BOs (1) kernel BO cache: 32kb BOs (6) resource: 20048kb BOs (76) resource 65536x1@8/0: 384kb BOs (6) scanout resource 1152x864@32/0: 7776kb BOs (2) mesa cache: 15840kb BOs (114) resource 1048576x1@8/0: 3072kb BOs (3) resource 1152x864@32/0: 34992kb BOs (9) resource 576x576@32/0: 1296kb BOs (1) resource 131072x1@8/0: 384kb BOs (3) resource 128x1@8/7: 24kb BOs (3) resource 1024x512@8/10: 1368kb BOs (2) resource 1x1@32/0: 148kb BOs (12) resource 1152x896@32/10: 13528kb BOs (2) resource 128x20@8/7: 8kb BOs (1) resource 96x20@8/6: 8kb BOs (1) resource 80x18@8/6: 8kb BOs (1) resource 64x20@8/6: 8kb BOs (1) resource 64x18@8/6: 8kb BOs (1) resource 112x14@8/6: 8kb BOs (1) resource 80x14@8/6: 8kb BOs (1) resource 64x12@8/6: 16kb BOs (2) resource 48x14@8/5: 8kb BOs (1) resource 1024x704@32/10: 4184kb BOs (1) resource 512x128@32/0: 1280kb BOs (5) resource 176x20@8/7: 12kb BOs (1) resource 162x162@32/7: 464kb BOs (2) tiling shadow 1152x864: 7776kb BOs (2) resource 384x384@32/0: 4032kb BOs (7) userspace BO cache: 15840kb BOs (114) ##############END STATS###########This is exactly what GPU memory is used for.
The allocation size is only around 135 MB, but I see that a lot of objects were created in the meantime. I'm afraid the freeze could be caused by the fragmentation of GPU memory (https://library.softwareverify.com/memory-fragmentation-your-worst-nightmare/).
What is the fastest way of reproducing this issue?
No idea.... The screen is a Planar PT1985P-BK 19" Touchscreen LCD Monitor
I use two navigation applications. The first one is Mapfactor Navigator using free open street maps. This application is great for testing because it enables switching between software and hardware rendering (menu->settings->advanced settings) and it contains simulation mode (menu->tools->simulation) - so you can simulate X-hours trip on the table in lab.
This application froze together with whole system every time if I used hardware rendering, but time of freezing is very random. Maybe month ago, system froze immediately after start of Mapfactor. Yesterday, I sent my Raspberry to 6-hours virtual trip (in simulation mode) to another country. Device arrived to virtual destination but couple of minutes after arriving system froze again, in time when content of display was static.
Second application is Waze. It would be better for reproducing this issue, because it freezes sooner, mostly in one hour, but it happens always in the car if I am driving. I don't know, if issue is reproducible in the lab, maybe some mock location tool could help.
But Waze has second Emteria-specific issue - map flashing. It is reproducible very easily by zooming out in the map or switching to 3D. It's possible that map flashing and freezing has one common source... maybe.
@Martin, thanks for the details. We will start with Mapfactor and perform several tests.
My pleasure
I switched to software rendering in Fully Kiosk and the system is a lot more stable. Might want to add that to the docs @kalkov
@sgrayban, what exactly do you mean by words "lot more stable"? Is it still crashing but after longer time period?
It still hangs but its so random and far apart that I don't get that upset over it. Nothing like it was doing in the past.
It hung only twice last week. Before I switched it would crash every hour.
Still shouldn't happen. We have the Mapfactor app installed on one of our devices for testing. Waiting for the next crash...
We have finally got a freeze, unfortunately, it wasn't related to the GPU:
This rather looks like a old known RPi sdcard controller issue
We will repeat the test again to see the GPU performance.
Thanks for message @kalkov. Please could you send me some reference to description of "old known sdcard controller issue"?
Please see https://help.emteria.com/kb/devices-rpi-3#known-issues
Hi, I found good source of freezing yesterday. If I attach GPS receiver and run Waze on the table (so device is not moving), switch map to 2D (3D is not working) and start navigation to somewhere then whole system froze in 10 minutes. Always. I checked bo_stats and all objects took about 130MB. Logcat showed nothing. @kalkov , please could you verify if it is SD card controller issue or something else?
I'll try to reproduce it. We had multiple MapFactor tests, but non of them ever freezed