SystemDataScope
SystemDataScope is a GUI for visualization of collectd datasets. Together with collectd and RRDtool, it provides a system monitoring solution for Sailfish. Upgrade and this release notes are below. See also below for possible problem with SFOS 2.0.2 upgrade.
SystemDataScope reads the system performance data gathered by collectd. Graphs are generated using RRDtool and shown through provided GUI. If collectd is started by a GUI user through systemd, there is a support for starting/stopping the daemon.
Through process plugin of collectd, each program can be monitored with its RAM, CPU and disk access logged and shown on graphs through longer timespan.
The recorded statistics are kept with different time resolutions allowing to keep short-term (hours) as well as long-term data (year).
Reports
SystemDataScope can be used to generate report images that can be used for giving feedback on performance of your device or monitoring results on an app(s) that you selected in collectd configuration. By selecting "Report" in pulley menu, you would initiate generation of the graphs that will be stored, in Sailfish, under /home/nemo/Documents/SystemDataScope/TIMESTAMP. When the report graphs are all ready, a message will appear in SystemDataScope. With the file manager or Gallery app, select and send graphs that you want to share and delete the folder /home/nemo/Documents/SystemDataScope when not needed. This folder will be created if its missing the next time you would need a report. An example report graph is shown as one of the screenshots below.
Default configuration
The collected statistics are organized in a tree. In the beginning, a list of graphs on a top level are shown. By pressing on a graph, a detailed information regarding that statistics is shown on a list, if it exists. At present, using such scheme, there are three levels used.
The default configuration for collectd provides data on:
- CPU
- CPU usage (system, user, nice, idle, ...)
- Amount of time spent in CPU sleep
- CPU sleep (time; suspend attempts; duration of an average suspend)
- Used CPU frequencies
- Used CPU idle states
- Battery (charge, current, energy, power consumption, temperature, time to full/low, voltage)
- Memory and storage
- RAM usage (free, used, cached, buffered, ...)
- Storage usage
- root partition used and free
- disk stats (I/O time, traffic, operations per second, ...)
- Network
- Overall traffic
- Traffic on cellular and WiFi interfaces
- For each interface: traffic, accumulated traffic over displayed time period, errors, packets per second, packets dropped
- Radio details:
- Cellular and used internet radio (wifi or cellular) signal strength
- Whether WiFi or Bluetooth are switched on
- Bluetooth radio: connected, visible, enabled
- Processes and Load
- Number of processes in different states (running, stopped, zombies, ...)
- Fork rate
- Load
- You could monitor the processes you are interested in. For each of them, you get data on CPU usage, RAM usage (rss, code size, virtual memory used, ...), number of running threads and processes, disk traffic induced by the process
- Uptime
SystemDataScope is build to support multiple different log configurations. The configuration is setup through JSON file that can be autogenerated, specified by user, or downloaded from URL (local or remote). This should facilitate development of the graph generation scripts that are tailored for specific user setups.
Since some statistics are populated as you use the phone, such as used mobile network type, regenerate the configuration after a while again to see the corresponding radio signal strength.
The current implementation is developed on Linux Desktops and Sailfish OS mobile devices.
If something does not work, please send your bug reports via GitHub by opening an Issue (see GitHub page below). It is an open-source project and its an easiest way to contribute to it. If you have improvement suggestions, send them also via GitHub by opening a new issue. Developers are welcome to join.
Upgrade notes:
To get support for new collectd plugins, just regenerate graph configuration. Note that the new stats have to be supported by the generation script.
Upgrade notes for users of collectd versions 2016.07.17-6 and earlier:
For details, see collectd OpenRepos page ( https://openrepos.net/content/rinigus/collectd ), under upgrade notes. In short, for default collectd configuration users, please delete the old datasets and start with the new default configuration (will delete earlier data). For that, stop collectd and remove /home/nemo/.local/share/collectd ( in terminal: rm -rf /home/nemo/.local/share/collectd ) and start collectd again. While the data will be lost from earlier recordings, new databases will be initialized allowing you to record data for 1 year.
SFOS upgrade to 2.0.2 series
This may concern users who have been using collectd/SystemDataScope before upgrading SFOS to 2.0.2. On my device, the upgrade from 2.0.1 to 2.0.2.51 resulted in a change of a hostname from Jolla to Sailfish. As a result, collectd started to record under different folder with the graphs not updated in SystemDataScope. If you have the same problem, stop collectd, move your records to the new folder under .local/shared/collectd, delete old subfolder from where you moved the records, and regenerate the graph definitions in Settings. Start collectd and your old records will be updated as before.
Hopefully, Jolla will not be changing SFOS hostname default frequently. Otherwise, you could also set the hostname or specify folder in /etc/collectd.conf .
Homepage: https://github.com/rinigus/systemdatascope
License: MIT
Official TMO thread: http://talk.maemo.org/showthread.php?t=97756
Donations: https://liberapay.com/rinigus http://rinigus.github.io/donate-bitcoin
Screenshots:
Category:
Keywords:
Application versions:
| Attachment | Size |
|---|---|
 systemdatascope-0.2.5-1.armv7hl.rpm | 108.14 KB |
| systemdatascope-0.2.5-1.i486.rpm | 113.85 KB |
| systemdatascope-0.2.7-1.armv7hl.rpm | 109.42 KB |
| systemdatascope-0.2.7-1.i486.rpm | 115.11 KB |
| systemdatascope-0.3.0-1.armv7hl.rpm | 110.65 KB |
| systemdatascope-0.3.0-1.i486.rpm | 116.52 KB |
| systemdatascope-0.4.0-1.armv7hl.rpm | 116.52 KB |
| systemdatascope-0.4.0-1.i486.rpm | 123.72 KB |
| systemdatascope-0.4.1-10.9.1.jolla_.i486.rpm | 122.77 KB |
| systemdatascope-0.4.1-10.9.1.jolla_.armv7hl.rpm | 116.92 KB |
Changelog:
0.4.1
- Add support for swap statistics
0.4.0
- Add report generation
- Add some support for changes in Sailfish Ambience
- Support for merging statefs_battery into battery plugin
- Redesign of Time dialog
0.3.0
- Support for new variable name in cpusleep, cpuidle, and cpufreq plugins. NB! Regenrate graph definitions after upgrade and allowing collectd some time to make new databases.
- Add progress bar as an indicator for graph generation
- Add subpage title if the click on the graph opens new set of graphs
- Small fixes
0.2.7
- Support for cpufreq plugin
- Support for cpuidle plugin
- Support for suspend plugin
0.2.5
- plugins covering WiFi, Bluetooth and cellular radios
- reorganization of the graphs
For earlier versions, see github page.
Comments
kaulian
Tue, 10/04/2016 - 17:21
Permalink
Hello,
Since the 0.4 battery graph does not work for my oneplusx port.
rinigus
Tue, 10/04/2016 - 18:20
Permalink
Hi! Thanks for reporting. Battery plugin has been changed due to the merge with collectd upstream.
To fix it, let's see what you have in your configuration.
Please let me know if it helped. I am sure we'll get the battery stats working for you. If you still have problems, please pastebin:
Thank you for reporting the problem! There is some flux still expected in few plugins that I developed for collectd, but it will stabilize eventually when they will make it into upstream.
Looking forward to hear back from you,
rinigus
kaulian
Wed, 10/05/2016 - 14:21
Permalink
Hi,
Regenerate SystemDataScope graph done and grah is now here
Thanks
rinigus
Wed, 10/05/2016 - 14:28
Permalink
Excellent! Thank you for reporting back :)
erlen
Sun, 09/25/2016 - 15:32
Permalink
Since the last system update, it not working anymore. no graph printed.
All the systems are launched at the boot.
Do you have an idea how to fixe it ?
rinigus
Thu, 09/29/2016 - 19:48
Permalink
Hi! Have you tried the proposed fix?
rinigus
rinigus
Sun, 09/25/2016 - 15:55
Permalink
For me, the latest upgrade changed default hostname from Jolla to Sailfish. As a result, collectd started recording into /tmp/collectd/Sailfish directory leaving /tmp/collectd/Jolla intact. As a result, SystemDataScope (SDS) was plotting old data from Jolla subdir with the symptoms similar to what you describe. My issue is described in the end of openrepos SDS page, under "SFOS upgrade to 2.0.2 series". So, I suspect that you have been hit with the same issue.
To check if its true, see what directories do you have under /tmp/collectd. If you have 2, that is a problem. To resolve it,
Please let me know if this was your problem and if it fixed it. If it is not, please paste data from Status (there is a pulldown menu in Status to copy it to clipboard). You could either paste it here or as a new issue in github.
erlen
Thu, 09/29/2016 - 22:00
Permalink
Thanks, it works for all excepted as before for the battery, where I have no data, but since I use the program (few months) I have the same problem.
rinigus
Thu, 09/29/2016 - 22:34
Permalink
Good! OK, you are supposed to have battery data as well. So, its either something is wrong with the script that makes figures or maybe collectd is not configured properly.
Maybe we should try to fix battery as well. For that we could either use openrepos or, if you don't mind, maybe you could open an issue on github. The info that I would need:
* please paste data from Status. That's very easy to do on phone: Go to pulley menu, select status, Go to pulley menu again, select "Copy to clipboard". Then you could copy that to github issue, or send it to your pc via email, ...
* please send copy of your collectd conf. That's at /etc/collectd.conf . If you see that you have some other /etc/collectd.conf.rpm{something} you could look on the differences between them. Its possible that you don't have a battery plugin enabled, for example.
In respect to battery plugin, there will be changes in the next version. Namely, statefs extension has been incorporated into upstream and we get full functionality via collectd battery plugin (don't have to use separate statefs_battery). So, you may as well wait a bit and configure it when the next release will be out.
In general, please let me know if something does not work. Otherwise I cannot fix or help you out. On my phone, it all works and I expect that all should work on other devices as well (maybe with exception of suspend attempts due to possible differences in kernels).
cheers,
rinigus
gsalone
Wed, 08/03/2016 - 14:51
Permalink
When trying to install this, I get an error:
cvs-1.11.23-1.1.4.armv7hl.rpm not found on medium 'https://releases.jolla.com/releases/latest/jolla/armv7hl/'
Any hint how to get this?
rinigus
Wed, 08/03/2016 - 15:53
Permalink
No, not really. Sorry, for me it just installed it as a dependency. What's your device? Is it Jolla* or some ported one?
gsalone
Fri, 08/12/2016 - 00:58
Permalink
First Jolla phone. Not ported one.
Having said that, it installed beautifully on Jolla C.
rinigus
Sun, 08/28/2016 - 15:55
Permalink
Hi! I have updated rrdtool package with the reduced number of dependencies. You could try it again and I hope it would fix your problem. Please let me know if it still does not work.
rinigus
Fri, 08/12/2016 - 07:59
Permalink
I am currently testing the reduction of the number of dependencies by the packages. It looks that rrdtool dependency on some standard packages induced rather large chain of additional package dependencies on sailfish. since I am working on getting the collectd plugins merged upstream, it would take some time before I get these changes published as a new release.
PamNor
Tue, 07/05/2016 - 11:26
Permalink
SystemDataScope is now running and displaying data in Overview page.
When collectd is started two folders is created:
/tmp/collectd/jolla
/tmp/collectd/Jolla
To get it Work:
* Collectd folder while running: This must be set to /tmp/collectd/Jolla. Seems that this folder is hardcoded.
Python script has to be run against /tmp/collectd/Jolla
Unable to display data when collectd is stopped
Edit1:
Deleted folders:
/tmp/collect
~/.local/share/collectd
collectd folder wile running: /tmp/collectd/Jolla
collectd folder while stopped: ~/.local/share/collectd/Jolla
Start collectd: OK. Grid in Overview. Only one folder (Jola) in /tmp/collectd
Stop collectd and close SystemDataScope
Start SystemCollectDataScope: No grids in Overview
~/.local/share/collectd/Jolla -> full of folders and data.
/tmp/collectd -> empty.
Edit2
Collectd folder when stopped -> /home/nemo/.local/share/collectd/Jolla
Now working offline.
Thanks for great work and support.
rinigus
Tue, 07/05/2016 - 13:50
Permalink
PamNor, thank you for reporting back and making it work! the feedback is very helpful and I'll try to make the app usage simpler in future.
rinigus
Tue, 07/05/2016 - 07:01
Permalink
Reply to Edit1:
I have uploaded my Settings screeshot as an additional screenshot in app description. The difference that I see is a use of ~ in your settings as opposed to /home/nemo in mine. Please set
collectd folder while stopped: /home/nemo/.local/share/collectd/Jolla
I think this should fix it. Alternatively, you could delete folder name fully (do it only for 'while stopped'), close GUI and start it again. Then it should detect the correct folder. For auto-detection to work, there should be no space in folder name.
From your description, colletd is running and stop/starting as it should.
Please report back and tell if it helped. Thank you for your reports!
rinigus
Mon, 07/04/2016 - 22:59
Permalink
Thank you for the feedback!
This is strange that you get two directories under /tmp/collectd: jolla and Jolla. I have only Jolla and it maybe due to the hardware adaptation on Nexus 4. Which phone/tablet do you have?
Just a bit info for background. When you start collectd, systemd runs a small script that takes folder from /home/nemo/.local/share/collectd and syncs it to /tmp/collectd. As soon as you stop collectd, the same script syncs it back. NB! If your device crashes, you would loose the recorded data since the last stop. I have been trying to get systemd timers to sync data, but they seem to be very unreliable
In your case, for some reason, you have 2 subfolders in /tmp/collectd. One of them has a real data: /tmp/collectd/Jolla. What's in the other one, no clue (maybe you could post ls /tmp/collectd/jolla ?). To see data when you stop, go to Settings and set the corresponding folder to /home/nemo/.local/share/collectd/Jolla (same Jolla as in /tmp).
If you see graphs in Overview then you could navigate to more specific graphs by touching the graph. The graphs that can be navigated 'into' are marked with a small right arrow on the right of the graph.
It seems that configuration part is rather confusing and messy. If you have an idea how to improve it, please go to GitHub and open an issue. There it would be appropriate to discuss how to improve the GUI
PamNor
Mon, 07/04/2016 - 20:05
Permalink
Running python script
Loading file:///home/nemo/myconf.json
No graphs in overview
Only pulley menus.
Status:
collectd: running
RRDtool: running
Running SystemDataScope from terminal give error like:
[W] unknown:189 -file:///usr/lib/qt5/qml/Sailfish/Silica/DialogHeader.qml:189: TypeError: Cannot read property 'BackIndicatorDown' of null
edit1
myconf.json 9.96kb
/tmp/collectd/Jolla
/home/nemo/.local/share/collectd/jolla
Have to start my laptop to link you downloadlink from dropbox. could takr some time.
hmm..
rinigus
Mon, 07/04/2016 - 20:12
Permalink
Regarding the warning - I have seen it and it seems harmless (everything works with it)
rinigus
Mon, 07/04/2016 - 20:11
Permalink
10kb is not much - mine with the default settings is towards 300k. So, something got undetected. I get 10k size, if I use a wrong directory. Did you specified as a root for collectd /tmp/collectd ? You should get a directory below that. On my Nexus 4, its
/tmp/collectd/Jolla
So, the script should be on my phone
systemdatascope-makeconfig /tmp/collectd/Jolla > myconf.json
If your phone sets different hostname, you would get something different under /tmp/collectd
rinigus
Mon, 07/04/2016 - 19:50
Permalink
That's should be easier to fix.
After running a script, you should get myconf.json in your home dir: /home/nemo/myconf.json
That's if you redirected its output to that file. So, when you load, you should use:
file:///home/nemo/myconf.json
Thank you for testing, its of a great help!
EDIT: now I can see that you have put the correct path in URL. Can you send the json config by pastebin or some alternative?
EDIT2: or at least how large is it and whether its similar to https://github.com/rinigus/systemdatascope/blob/master/configs/sfos-n4.json
EDIT3: And what are you directories in Settings?
PamNor
Mon, 07/04/2016 - 16:25
Permalink
Running Sailfishos 2.0.1.11 (arm).
rrdtool: installed
from terminal:
systemdatascope
..
...
[w] unknown:-1 - file:///usr/share/systemdatascope/qml/main.qml: file not found
[edit]
I cannot find qml folder. main.qml i in folder systemdatascope
rinigus
Mon, 07/04/2016 - 16:38
Permalink
PamNor, please try the new version. It should have fixed the wrong location of QML (that's what happens when you develop for desktop & sailfish at the same time and don't want to include desktop QMLs into sailfish package) :)
Please let me know if it helped
PamNor
Mon, 07/04/2016 - 16:46
Permalink
SystemDataScope is starting now.
I will continue testing .
Thanks.
rinigus
Mon, 07/04/2016 - 16:47
Permalink
Thank you very much for reporting it!
rinigus
Mon, 07/04/2016 - 16:26
Permalink
PamNor, excellent! It seems to be a bug due to packaging. Please uninstall SDS GUI and I'll update it to a new version. Just wait a bit...
rinigus
Mon, 07/04/2016 - 16:09
Permalink
very strange - maybe something went wrong during install. QML files should be there. If you do
ls /usr/share/systemdatascope/qml
do you see the files? There should be main.qml, other QMLs from https://github.com/rinigus/systemdatascope/tree/master/qml, directory Platform.silica and a symbolic link Platform->Platform.silica .
If these are absent, maybe something happened during install. Try to reinstall it. Maybe download RPM manually and install with
pkcon install-local ....rpm
PamNor
Mon, 07/04/2016 - 15:26
Permalink
Gui start loading for 30 sec and then quits. SystemDataScope process continue to run. Tried to start collectd from terminal in forehand,but no difference. chmod 666 /var/log/collectd.log, no difference.
rinigus
Mon, 07/04/2016 - 15:46
Permalink
PamNor, sorry to hear about. Let's try to fix it. In general, GUI should run without collectd started and is supposed to display you a help screen. So, let's keep collectd out of the way and fix GUI first.
Just few questions:
As for collectd, it should be started via systemctl --user and, as far as I have seen, all its messages end up in system journal (journalctl as devel-su). But let's fix GUI startup first.