Speech Note

Rating: 
5
Your rating: None Average: 5 (13 votes)

App for note taking with speech to text.

Speech Note converts speech to text using Coqui STT engine (a fork of Mozilla's DeepSpeech) and various acoustic and language models.

    All voice processing is entirely done locally on the device. Internet connection is only required for model download during app initial configuration. Speech Note respects your privacy and provides truly offline speech-to-text capability.

    DeepSpeech models for particular language can be downloaded directly from the app. Following models are currently configured for download:

    • Catalan / ca
    • Czech / cs
    • English / en
    • German / de
    • Spanish / es
    • French / fr
    • Italian / it
    • Polish / pl
    • Russian / ru
    • Ukrainian / uk
    • Chinese / zh-CN

    + many experimental models (Dutch, Yoruba, Amharic, Basque, Turkish, Thai, Slovenian, Romanian, Portuguese, Latvian, Indonesian, Greek, Hungarian)

    The exact sources are listed here.

    The quality of speech recognition strongly depends on acoustic model. In general it is not perfect but for some languages is surprisingly fine. I would be grateful for any feedback how good speech transcription is for individual models.

    Known issues:

    • Jolla Tablet: does not work at all because I don't know how to build STT library for i486 architecture.
    • Jolla 1: speech transcription is slow and sometimes app crashes due to low memory error
    • PinePhone: very unstable and sometimes causes crash of PulseAudio server

    Any comments, ideas, translations, issue reports are highly appreciated.

    Tip: To play and test any model which is not yet configured for download, add model description to `$HOME/.local/share/harbour-dsnote/harbour-dsnote/models.json` file on the device.

    Translations (both Speech Note and Speech Keyboard):
    All translations are very welcome. There are three ways to contribute:
    - [preferred] Transifex project
    - Direct github pull request
    - Translation file sent to me via e-mail: dsnote@mkiol.net

    Source code: https://github.com/mkiol/dsnote
    Bugs, Feature requests: https://github.com/mkiol/dsnote/issues or just email: dsnote@mkiol.net

    Application versions: 
    AttachmentSizeDate
    File harbour-dsnote-1.5.1-1.armv7hl.rpm1.27 MB17/11/2021 - 10:00
    File harbour-dsnote-1.5.1-1.aarch64.rpm1.34 MB17/11/2021 - 19:28
    File harbour-dsnote-1.6.0-1.aarch64.rpm1.39 MB09/12/2021 - 21:32
    File harbour-dsnote-1.6.0-1.armv7hl.rpm1.31 MB09/12/2021 - 21:32
    File harbour-dsnote-1.6.1-1.armv7hl.rpm1.31 MB10/12/2021 - 20:52
    File harbour-dsnote-1.6.1-1.aarch64.rpm1.39 MB10/12/2021 - 20:52
    Changelog: 

    1.6.1

    • New German language model "Deutsch (Aashish Agarwal)" (experimental). This model might be even better than the currently configured default. I would be greateful for the feedback.

    1.6.0

    • New and default listening mode: One sentence (Clicking on the bottom panel starts listening, which ends when the first sentence is recognized)
    • Cover action (When 'One sentence' mode is set, cover displays action to enable/cancel listening.)
    • Improved language viewer
    • Coqui STT lib update (v1.1.0)
    • Bug fixes and performance improvements (e.g. App starts much quicker with multiple languages enabled)

    1.5.1

    • Fix: Languages configuration wasn't loaded when app was installed for the first time

    1.5.0

    • Fix for ARM64 - now app should work
    • Model for Catalan language
    • Many "experimental" models for various languages: Dutch, Yoruba, Amharic, Basque, Turkish, Thai, Slovenian, Romanian, Portuguese, Latvian, Indonesian, Greek, Hungarian. Most of these models provide very bad accuracy :(

    1.4.0

    • Russian and Ukrainian models
    • D-Bus API and service for 3rd-party app integration (e.g. Speech Keyboard)

    1.3.0

    • Czech language model and translation (many thanks to Lukáš Karas for the contribution)
    • New additional models: French (Common Voice), Italian (Mozilla Italia)

    1.2.0

    • Option to transcribe audio file
    • Minor UI fixes and improvements

    1.0.1

    • support for Jolla 1, Jolla C and PinePhone (alpha)
    • speech recognition accuracy is much improved thanks to DeepSpeech library update to version '0.10.0-alpha.3'
    • UI minor fixes

    Comments

    JayJay's picture

    Real nice work! The app is really cool. Is there any option to customize the vocabulary (i would need german medical language with drug recognition and medical vocabulary... is there maybe a file i can download or buy? If not... That would be an awesome new feature if i could add new vocabulary myself :-)

    rdomschk's picture

    Perfect Work!  A big Thank You from me...

    inta's picture

    Thanks for the great work, now it runs on arm64 and it works really well. :)

    inta's picture

    Languages still do not load here. Is there anything I have to clean up? I removed the settings folder from .config and the models dir inside Downloads.

    mkiol's picture

    Sorry, silly me. I forgot to upload 1.5.1 package for aarch64. It should be available in a moment.

    inta's picture

    The app does not "hang" anymore on startup and uninstall works, but the language list in the settings is empty (Xperia 10 II), so I can not choose a model to get started with.

    mkiol's picture

    Fixed in 1.5.1. Would be grateful for check if problem is resolved. Thanks.

    mkiol's picture

    Oh dear. I know what is wrong. I will fix it tomorrow.

    inta's picture

    @robthebold 10 II, so @mkiol could be right that this is an arm64 issue. Never mind, force uninstall worked and I'll try it again if you need someone to test it.

    dubliner's picture

    While version 1.3 worked flawlessly under SFOS 3.4, it seems the new version 1.4 runs into a problem. All I get is "Language is not configured". When I open the settings, there are "no languages", nothing is displayed.

    Curiously, the old "Downloads/DeepSpeech models" directory was still there, populated with "de.scorer  de.tflite  en.scorer  en.tflite". Pointing the "Location on language files" to that directory does not make any difference.

    I also tried deleting "Downloads/DeepSpeech models" as well as ".config/harbour-dsnote" to get a fresh start. Unexpectedly, that ".config/harbour-dsnote" is not re-created after starting DeepSpeech Note.

    Starting from the CLI I receive this output:

    $ harbour-dsnote
    [D] unknown:0 - cannot load translation: "C" "/usr/share/harbour-dsnote/translations"
    [D] unknown:0 - cannot load default translation
    [D] unknown:0 - starting configuration
    [D] unknown:0 - Using Wayland-EGL
    [W] unknown:0 - cannot open models file
    [W] unknown:0 - cannot open lang models file
    [D] unknown:0 - [app => dbus] call KeepAliveService
    [W] unknown:247 - file:///usr/lib/qt5/qml/Sailfish/Silica/private/TextBase.qml:247: TypeError: Cannot call method 'createObject' of null
    [W] unknown:0 - cannot reload service because is's not running
    [D] unknown:0 - [dbus => app] signal ModelsPropertyChanged
    [D] unknown:0 - [dbus => app] signal StatusPropertyChanged: 2
    [D] unknown:0 - [dbus => app] signal ModelsPropertyChanged
    [D] unknown:0 - [dbus => app] signal StatusPropertyChanged: 1
    [D] unknown:0 - [app => dbus] get DefaultModel
    [D] unknown:0 - [app => dbus] get CurrentTask
    [W] unknown:0 - ignore update speech
    [D] unknown:0 - [app => dbus] call KeepAliveService

    Any help would be appreciated, especially since I really love this application!

    dubliner's picture

    Update: When I copied ".config/harbour-dsnote" and ".local/share/harbour-dsnote" as well as "Downloads/DeepSpeech models" from another phone running SFOS 4.2 it works!!! Yay!

    Not sure, though, why the ".local/share/harbour-dsnote" directory was not created and populated on the first try?!

    P.S. Now Speech keyboard is not working on the SFOS 3.4 phone. I get the logo (three vertical lines) with strikethrough symbol.

    robthebold's picture

    I installed this on my Xperia 10 II, can't seem to make it work . . . When I start the app, I see an error "Unable to start service" pop up. As I'd expect for this error, speech recognition doesn't work, and when I go to Settings, there are no languages to choose from.

    I was going to uninstall and reinstall the app, but Storeman can't uninstall it and when I try to uninstall from terminal a "scriptlet" fails, saying it can't stop the service because it isn't running and uninstalling fails.

    I've also tried starting the service manually from the terminal but that didn't work. I'm not totally sure I did that right, though: as root I tried "systemctl start harbour-dsnote.service" and "systemctl start --user harbour-dsnote.service" and fails with message "Unit harbour-dsnote.service not found."

    "rpm -rl harbour-dsnote"  led me to check to make sure /usr/lib64/systemd/user/harbour-dsnote/ exists, and it does.

    Any ideas on how I can fix this or debug further? If more details are needed I can find my glasses and copy/paste stuff from terminal

    mkiol's picture

    I'm sorry for this mess. Most likely something is wrong with arm64 package. To be honest, I did not test it because I don't have any arm64 device yet.

    To force uninstall run following in a terminal:

    devel-su rpm --erase --allmatches --noscripts harbour-dsnote harbour-dskeyboard
    

    I will investigate what went wrong tomorrow. Sorrrry.

    inta's picture

    I tried to install this app and the keyboard app, but the list of languages in the settings is empty. I cannot remove this app, it fails with the message that the service is not running. Any idea how to fix that?

    robthebold's picture

    I didn't realize you posted this issue before me -- I'm getting the same problem. What device are you using?

    PamNor's picture

    @mkoli. I'll continue search for Norwegian *.tflite file. Keep up your good work.

    PamNor's picture

    Is there a possibility to get speech model for Norwegian language?
    https://www.google.com/url?sa=t&source=web&cd=&ved=2ahUKEwi53uqAnpj0AhVC...

    mkiol's picture

    I really would like to add such support but unfortunately I wasn't able to find any DeepSpeech model for Norwegian (usually file with *.tflite extension) :(

    lispy's picture

    A big Thankyou for the Transcribe Audio File feature. Made my day!!!

    eson's picture

    How about more language models, Swedish in perticular? ;)

    mkiol's picture

    I've tried but unfortunately I didn't find any available DeepSpeech model for Swedish. If you find one I will be pleased to add it.

    eson's picture

    Well, knowing exactly nothing about the matter, I found these links on the net. Maybe you've already seen them or they are totally useless?

    https://github.com/AlexandrosFerles/Swedish-Language-Automatic-Speech-Re...

    https://github.com/se-asr/model

    https://medium.com/@klintcho/creating-an-open-speech-recognition-dataset...

     

    Thanks anyway for your good work as allways!

    mkiol's picture

    Sorry for the late reply. Indeed this project provides model for Swedish. Unfortunately it was trained for older version of DeepSpeech and therefore it is not compatible. Sadly, there is no simple way to convert it to new one. The only solution is to repeat the training, which is possible but requires access to source material (voice samples) and significant computing power.

    defactofactotum's picture

    Now working on pinephone with sfos4.2. But the microphone disconnects after every input.

    Fuchur's picture

    It really is working very well and a very nice app.
    One thing I really would love to see is to be have a button on the keyboard or an own keyboard layout which would include it to the keyboard input.

    That would just be great :).

    lispy's picture

    Really works. I like it. My wife has to convert a huge audiofile to text but pushing the button for an hour sadly doesn't cut it for her. Can you imagine an audiofile import of sorts? Or maybe make the button sticky?

    mkiol's picture

    There are to modes (Settings->"Speech detection mode"). In "Automatic" mode, you don't have to hold the button. App will (in most cases ;-) automaticaly detect that speaking begins.

    defactofactotum's picture

    Thanks for the keyboard fix! It still doesn't work on my pinephone - it worked briefly in Italian but with very bad recognition, then stopped again. Another suggestion: would it be possible to add words to the database? I imagine this is probably a huge and complicated task....

    defactofactotum's picture

    Also does not work on pinephone. Suggestion for keyboard behaviour: at the moment it's possible to edit text in the middle of a line but after typing one letter the cursor snaps back to end of line. When an entire word is wrong this is very laborious.

    mkiol's picture

    Thank for suggestion.

    In the meantime, I've managed to fix Jolla 1, Jolla C and PinePhone issue. Moreover with alpha version of DeepSpeech accuracy of recognition is much improved. Stay tuned for next release :)

    Pages