Snipek
Voice assistant for Sailfish OS based on Snips software
Release 2.0 highlights:
- Support for Snips installation on the Sailfish OS phone. It gives truly offline voice assistant.
- Built-in skills: Date & Time, Call History
- Works on Jolla 1
Important Notice. Staring from January 31, 2020 Snips assistant is not longer publicly available. It means that Snipek further development will be very challenging and most likely not possible any more.
Snipek is a Sailfish OS app that provides remote microphone and speaker capability for Snips voice platform.
Snips is unique because it doesn’t require external cloud service. All processing (including voice) is entirely done locally, on a device where Snips is installed.
To make this app works you have to separately install Snips software on your Sailfish OS phone or on any another device in your local network (e.g. Raspberry Pi, Debian computer).
Snips installation
Two installation options are supported:
- Local (Snips is installed on Sailfish OS phone together with Snipek)
- Remote (Snips is installed on a different device in your local network)
When Snipek and Snips are running on the same device (on Sailfish OS phone), no network connectivity is required. Your phone can be in off-line mode and the voice assistant sill works. To install Snips on Sailfish OS follow this guide.
Keep in mind that Snips is not an open source software. The source code is not publicly available and the use of Snips is governed by Snips Terms of Use.
Snips does not provide binaries for x86 (32-bit) CPU architecture, therefore local installation on Jolla Tablet and any other non-ARM Sailfish OS device is not possible right now.
More information about Snips installation options are outlined here.
Snipek skills
Snipek app connects to Snips as an additional audio server. It sends audio stream captured from the microphone and plays audio data received from Snips. It also provides few built-in skills that enable phone control with the voice commands. Skills are the capabilities of voice assistant i.e. the things that assistant can do with a voice command.
Currently implemented skills are:
- Date & Time (Reads current time or date)
- Call History (Reads events from call history e.g. all missed calls)
Languages and translations
Snips software supports following languages and therefore only following languages Snipek can be translated to:
- German
- English
- Spanish
- French
- Italian
- Japanese
- Portuguese (Brazil)
Currently only English language is provided in Snipek.
Bugs reporting
This is "work in progress" project in beta/experimental stage, so likely contains a number of bugs.
Any comments, issue reports are highly appreciated.
Bugs and feature requests can be reported on GitHub or just via email snipek@mkiol.net.
Snipek is a free application.
Screenshots:
Category:
Keywords:
Application versions:
| Attachment | Size | Date |
|---|---|---|
 harbour-snipek-2.0.1-1.armv7hl.rpm | 201.18 KB | 04/06/2020 - 10:10 |
| harbour-snipek-2.0.2-1.armv7hl.rpm | 395.39 KB | 11/06/2020 - 14:32 |
Changelog:
version 2.0.2 (beta)
- Bug fixes and small improvements
version 2.0.1 (beta)
- Fixes for bugs discovered on the recent SFOS release
version 2.0.0 (beta)
- support for Snips installation on the Sailfish OS phone. It gives truly offline voice assistant.
- built-in skills: Date & Time, Call History
- works on Jolla 1
version 1.0.0
- support for multiple Snipek instances connected to single Snips system
Comments
objectifnul
Thu, 2020/06/04 - 12:57
Permalink
Snips downloaded successfully for local install (Xperia X).
snips_start.sh failed though:
Using directory: /home/nemo/.cache/harbour-snipek/harbour-snipek/snips.
Stopping Snips processes...
Check if all needed files are present in /home/nemo/.cache/harbour-snipek/harbour-snipek/snips directory.
Current installation version is 1.0.0 and this script version is 1.0.0.
Starting Snips...
~/.cache/harbour-snipek/harbour-snipek/snips ~
~
Check if all Snips processes are running.
Process mosquitto is running.
Process mosquitto stopped.
----
snipek.log below:
[D] :0 - Logging to file enabled
[D] :0 - MQTT init, state: MqttAgent::MqttState(MqttDisconnected)
[D] :0 - MQTT new URL: "tcp://127.1.0.0:1883"
[D] :0 - MQTT current URL: "tcp://127.1.0.0:1883"
[D] :0 - Staring MQTT thread
[D] :0 - New MQTT action: 1
[D] :0 - Connecting MQTT agent for URL: "tcp://127.1.0.0:1883"
[D] :0 - MQTT ID: "snipek-Cf85"
[D] :0 - MQTT connected changed: false
[D] :0 - Suspend listening requested
[D] :0 - Listening update, current: false desired: false
[W] :0 - Failed to connect MQTT, return code: -1
[D] :0 - Finishing MQTT thread
[D] :0 - MQTT connected changed: false
[D] :0 - Suspend listening requested
[D] :0 - Listening update, current: false desired: false
[D] :0 - Reconnecting attempt: 1
[D] :0 - MQTT init, state: MqttAgent::MqttState(MqttDisconnected)
[D] :0 - MQTT new URL: "tcp://127.1.0.0:1883"
[D] :0 - MQTT current URL: "tcp://127.1.0.0:1883"
[D] :0 - Staring MQTT thread
[D] :0 - New MQTT action: 1
[D] :0 - Connecting MQTT agent for URL: "tcp://127.1.0.0:1883"
[D] :0 - MQTT ID: "snipek-Cf85"
[D] :0 - MQTT connected changed: false
[D] :0 - Suspend listening requested
[D] :0 - Listening update, current: false desired: false
[W] :0 - Failed to connect MQTT, return code: -1
[D] :0 - Finishing MQTT thread
[D] :0 - MQTT connected changed: false
[D] :0 - Suspend listening requested
[D] :0 - Listening update, current: false desired: false
[D] :0 - Reconnecting attempt: 2
[D] :0 - MQTT init, state: MqttAgent::MqttState(MqttDisconnected)
[D] :0 - MQTT new URL: "tcp://127.1.0.0:1883"
[D] :0 - MQTT current URL: "tcp://127.1.0.0:1883"
[D] :0 - Staring MQTT thread
[D] :0 - New MQTT action: 1
[D] :0 - MQTT connected changed: false
[D] :0 - Suspend listening requested
[D] :0 - Listening update, current: false desired: false
[D] :0 - Connecting MQTT agent for URL: "tcp://127.1.0.0:1883"
[D] :0 - MQTT ID: "snipek-Cf85"
[W] :0 - Failed to connect MQTT, return code: -1
[D] :0 - Finishing MQTT thread
[D] :0 - MQTT connected changed: false
[D] :0 - Suspend listening requested
[D] :0 - Listening update, current: false desired: false
[D] :0 - Reconnecting attempt: 3
[D] :0 - MQTT init, state: MqttAgent::MqttState(MqttDisconnected)
[D] :0 - MQTT new URL: "tcp://127.1.0.0:1883"
[D] :0 - MQTT current URL: "tcp://127.1.0.0:1883"
[D] :0 - Staring MQTT thread
[D] :0 - New MQTT action: 1
[D] :0 - MQTT connected changed: false
[D] :0 - Suspend listening requested
[D] :0 - Listening update, current: false desired: false
[D] :0 - Connecting MQTT agent for URL: "tcp://127.1.0.0:1883"
[D] :0 - MQTT ID: "snipek-Cf85"
[W] :0 - Failed to connect MQTT, return code: -1
[D] :0 - MQTT connected changed: false
[D] :0 - Suspend listening requested
[D] :0 - Listening update, current: false desired: false
[D] :0 - Reconnect attempts limit reached
[D] :0 - Finishing MQTT thread
[D] :0 - Checking if Snips is installed...
[D] :0 - proces exit: 0
[D] :0 - Checking if Snips is running...
[D] :0 - proces exit: 1
[D] :0 - Starting Snips...
[D] :0 - proces exit: 1
[D] :0 - Starting Snips...
[D] :0 - proces exit: 1
mkiol
Thu, 2020/06/04 - 14:05
Permalink
Could you please do following tests:
#1
$ /usr/share/harbour-snipek/snips/snips_start.sh
$ ps -A | grep mosquitto
Does `mosquitto` process is running?
#2
$ /usr/share/harbour-snipek/snips/snips_start.sh -k
$ export LD_LIBRARY_PATH=~/.cache/harbour-snipek/harbour-snipek/snips
$ ~/.cache/harbour-snipek/harbour-snipek/snips/mosquitto
Do you see any error from mosquitto?
objectifnul
Thu, 2020/06/04 - 17:13
Permalink
#1 returns no output. ps doesn't list mosquitto.
#2:
[nemo@XperiaCed ~]$ /usr/share/harbour-snipek/snips/snips_start.sh -k
Using directory: /home/nemo/.cache/harbour-snipek/harbour-snipek/snips.
Stopping Snips processes...
snips-asr: no process found
snips-tts: no process found
snips-dialogue: no process found
snips-nlu: no process found
snips-hotword: no process found
snips-asr: no process found
mosquitto: no process found
Done. Snips successfuly stopped.
[nemo@XperiaCed ~]$ export LD_LIBRARY_PATH=~/.cache/harbour-snipek/harbour-snipek/snips
[nemo@XperiaCed ~]$ ~/.cache/harbour-snipek/harbour-snipek/snips/mosquitto
1591278990: mosquitto version 1.4.10 (build date Wed, 13 Feb 2019 00:45:38 +0000) starting
1591278990: Using default config.
1591278990: Opening ipv4 listen socket on port 1883.
1591278990: Opening ipv6 listen socket on port 1883.
objectifnul
Thu, 2020/06/04 - 23:00
Permalink
Something is weird in snipek.log:
[D] :0 - MQTT current URL: "tcp://127.1.0.0:1883"
Why not 127.0.0.1 instead?
mkiol
Fri, 2020/06/05 - 11:02
Permalink
This is a little embarrassing... but for some reason it works fine on all my SFOS devices.
Could please try this test package:
https://github.com/mkiol/Snipek/raw/master/binary/harbour-snipek-2.0.1-2...
objectifnul
Fri, 2020/06/05 - 12:13
Permalink
Tried on both Xperia X and Jolla phone. Same: couldn't get snips running (openrepos package or github package). So, there is something in my both device configurations that prevents snips from working, but I have no precise idea of what to be investigated.
mkiol
Fri, 2020/06/05 - 12:30
Permalink
Hmmm, let's reset the configuration to default.
Could you please:
- uninstall Snipek
- remove both: ~/.config/harbour-snipek and ~/.cache/harbour-snipek
- install Snipek (version from github)
- run script (as nemo): /usr/share/harbour-snipek/snips/snips_download.sh
- check if Snipek works
objectifnul
Fri, 2020/06/05 - 23:10
Permalink
Done. Same result, snips still not running after snipek configured "local" then started snips (within snipek) without success. I suspect my config is missing some hidden dependency (also tried with all patches unapplied).
Additional info: snips could be started (green strip in settings) provided my customized 'hosts' file (about 10,000 entries) is disabled and my wlan connection is turned off.
Even so, snipek still notifies 'connection problem'
mkiol
Sat, 2020/06/06 - 18:05
Permalink
I've prepared new test build with better logging.
Could you please install this package:
https://github.com/mkiol/Snipek/raw/master/binary/harbour-snipek-2.0.1-3...
After installing, please:
1. start Snipek and enable logging to file (propably is already enabled)
2. restart Snipek
3. Go to the settings page and set Local snips
4. close the app
5. send to me (snipek@mkiol.net) 3 log files that are in home dir (snipek.log, snips_start.log, snips_download.log)
objectifnul
Thu, 2020/06/11 - 11:58
Permalink
It's working now (https://github.com/mkiol/Snipek/blob/master/binary/harbour-snipek-2.0.1-...) with minimal built-in skills. How are we supposed to help building and sharing more skills? (despite Sonos' strategy obviously aiming to kill Snips)
mkiol
Thu, 2020/06/11 - 14:37
Permalink
> How are we supposed to help building and sharing more skills? (despite Sonos' strategy obviously aiming to kill Snips)
Acctually I don't know yet. I'm investigating how to migrate Snipek to different non-Snips "back-end" but didn't find any solution so far. With Snips, most likely further development is not possible any more.
objectifnul
Thu, 2020/06/11 - 16:27
Permalink
In other words, this is a dead project?
mkiol
Thu, 2020/06/11 - 20:08
Permalink
Snips is mostly dead. Snipek might be but currently it is in a "paused" state.
toxip
Sun, 2019/12/08 - 10:27
Permalink
Well, it was fun while it lasted but seems like this project will be dead along with snips. Loved this app but with the recent snips news it seems like the days of open snips for makers are over.
https://forum.snips.ai/t/important-message-regarding-the-snips-console/4145
mkiol
Sun, 2019/12/08 - 13:37
Permalink
I'm surprised and sad. I didn't notice it earlier... Indeed, this information makes Snipek continuation very difficult...
Ezreal
Sat, 2019/12/07 - 03:08
Permalink
nice！Sailfish is becoming smart!
jollex
Sat, 2019/11/09 - 13:43
Permalink
Wow that would be amazing!! Everything embeded
lutinotmalin
Sun, 2019/09/22 - 11:46
Permalink
I really don't understand how to use that app that seems wonderful. How do I connect to Snips ? What should I write for the IP, the port, the site ID ? Shall I create an account somewhere ?
mkiol
Mon, 2019/09/23 - 17:51
Permalink
To make this app works you have to have Snips server somewhere in your local network. The easiest way is to use Raspberry Pi because it is officially supported platform. The guide how to do it is here: https://docs.snips.ai/getting-started/quick-start-raspberry-pi (relatively easy but not trivial). Alternatively you can also install Snips on any Debian box.
My installation is based on Raspberry Pi 2 and it works great.
mkiol
Wed, 2019/09/25 - 12:19
Permalink
Just a info. I'm also investigating how to port Snips server to SFOS and provide standalone voice assistant that works without any network connectivity. Maybe it will be included in the next Snipek version... Right now server side (installed on RPi) is needed.
toxip
Mon, 2018/12/17 - 19:57
Permalink
I had trouble setting this up. My raspbi doesn't have any speakers or mic and this doesn't work either. What am I doing wrong?
mkiol
Mon, 2018/12/17 - 21:41
Permalink
Speaker and mic attached to a raspberrypi device are not needed because your phone (powered with Snipek) will provide both. Of course, you can use Snipek and raspberrypi's mic in the one system - they are not mutually exclusive. Snips can handle multiple audio sources.
Maybe your Snips installation is not properly setup? Did you follow Quick Start Raspberry Pi guide? What "sam status" command shows? Here is my output:
toxip
Tue, 2018/12/18 - 06:37
Permalink
So the problem was my custom hotword. I couldn't get it to work at all but by switching to hey snips I could manage to get the thing respond. Also the Xperia X microphone didn't even work for the record (probably too much noise?) so I was using a headset mic for that. Still, even with the headset it didn't recognise the hotword. I would like to use a personal hotword as it'd be much cooler but that's a minor compromise.
Now the app is working great on my Xperia X but unfortunately the mic and speakers didn't work with Jolla 1 at all despite being connected. I would've wanted to repurpose my Jolla 1 as an always on station for the system but unfortunately couldn't do it because of this bug. Perhaps you're using some weird setup to get the mic data that it doesn't work for J1? Anyway, thanks for this app and introducing me to this wonderful platform. Had a great deal of fun playing around with it ^^
mkiol
Tue, 2018/12/18 - 23:53
Permalink
You are 100% right. Snipek doesn't work on Jolla 1. My fault, I've tested it only on Jolla C and XperiaX. The problem with Jolla 1 is a very slow audio processing. Accually, it works but there is huge delay ~20-30s. I'm investigating it and maybe will be able to fix it. Many thanks for detecting this bug.
toxip
Wed, 2019/01/09 - 03:54
Permalink
Need to wait until I get my Jolla C from other side of the world. Meanwhile I would like to ask, is there any chance to build the desktop app on Windows? I'm trying myself but I don't know how to. Some paho-mqtt things seem to give me an error. Any chance you could help me or crossbuild for windows? I'd love to get this running on my Windows PC which has my headset plugged in. Thanks!
mkiol
Sun, 2019/01/13 - 19:57
Permalink
Right, so currently on a desktop only Linux is supported because it was trival to port SFOS code. To be honest I don't have much experience with Windows and is is more important I don't have any Windows-running machine or VM. In theory I could install one but is it would be quite a big effort...
jollex
Mon, 2018/12/17 - 03:12
Permalink
Many thx! I discovered snips with your apps!
Historyscholar
Tue, 2018/11/27 - 11:28
Permalink
Good