To test this feature, visit your live site.

VOSK - Offline Speech To Text

I haven't made a tutorial video on this becasue a lot of the infomation is covered in this video here from a different creator (NOT ME!):

I literally copied the instructions and code from that YouTube video and then modified it slighty to work with my JARVIS program. So I suggest everyone interested in applying VOSK to their program view and give that creator a like for their video.

A very brief recap of the video is:

Pip install vosk

This should be self explanitory if you've used python before.

Download STT model(s)

Go to https://alphacephei.com/vosk/models and find the Speech-To-Text (STT) file you want to use for your language and download it. Depending on your choice it might take a while. These can be large files. The files will be loaded to memory when being used. The larger files are more accurate but take up more RAM. If you are using something like a Raspberry Pi 4, use the smaller model designed for single board computers or Android.

I noticed that if you are using a higher quality mic or a headset mic, the small model works well. I have a cheap conference style mic on my desk specifically for JARVIS so I use the larger model. (But I'm odd, so there's that. ) If you don't have enough ram and accidently load a large file, your pc will freeze for several minutes and then the load will fail and dump the file from memory, unfreezing the computer. So you can't break anything but you may feel like you have. (Yes I tried it, becasue as mentioned before... I'm odd.)

Use the code from the video.

The presenter does a good job explaining the process and I don't know if I would do better. There is 1 part in the video that I found a fix for their error. He has an issue with the mic not being able to be split between other apps and crashing his assistant program. To fix that in his code change

data = stream.read(4096)

data = stream.read(4096, exception_on_overflow = False)

This will fix allow the mic to just pass the exception. If it's being used elsewhere it just won't pick up what you said and you have to repeat. Better than the whole thing crashing. This is what allows me to use JARVIS on the same PC I am recording videos on at the same time.

I hope this makes sense. If enough peolpe like this I'll probably make a full video out of it, but no need to wait for me to spend time editing a video and you can use it now for yourself.

5 answers0 replies

Comments (5)

Mohamed Amir

Aug 27, 2023

where exactly do you put the code in jarvis

James Slagle

Aug 27, 2023

•

Replying to

It’s kind of up to you on how you want it to run.

I load the vosk file into a global variable at the very beginning.

Then I have all the rest of the code in a function TakeCommand().

That works the fastest for responses, but it keeps the file loaded to memory the entire time the assistant is running. Depending on your system that might be okay or not be best for you.

Mohamed Amir

Aug 27, 2023

Replying to

can you copy and paste that part of your code so i can see because that youtuber didnt show how to implement it on jarvis

James Slagle

Aug 28, 2023

•

Replying to

It's probably becasue there are different ways you can do it. What works for me is I have it load the files and recognizer at the beginning of the program into global variables.

MODEL = Model(r"C:\Jarvis\vosk-model-en-us-0.22\vosk-model-en-us-0.22") # Larger Model

RECOGNIZER = KaldiRecognizer(MODEL, 16000)

And then I have the "TakeCommand" function later in the code:

TakeCommand():

    global MODEL

    global RECOGNIZER

    recognizer = RECOGNIZER

    TALK = True

    mic = pyaudio.PyAudio()

    stream = mic.open(format=pyaudio.paInt16, channels=1, rate=16000, input=True, frames_per_buffer=8192)

    stream.start_stream()

    AIon() # Random Hello Function

    print("J.A.R.V.I.S.: Ready to assist ")

    while TALK == True:

        data = stream.read(4096, exception_on_overflow = False)

        if recognizer.AcceptWaveform(data):

            text = recognizer.Result()

            text_to_be_analyzed = text[14:-3] # this is YT code to change dictionary to text maybe(?) can fix this later

            print("User: " + text_to_be_analyzed)

            TALK = False

     return text_to_be_analyzed

Whenever you call this function it says a random "hello" (In the AIon function elsewhere)and then will l listen until it hears something that registers in the model and then it will stop listening and pass that to the next part. Otherwise the model kept listening to JARVIS's own responses back got confused.

Just remember that this is how I did it for my program. You very likely will have to modify yours depending on what you want. (I also removed any reference to my talking GUI in this code becasue that would really depend on what you are using and how.)

Mohamed Amir

Aug 27, 2023

love to see it. I really appreciate it.

VOSK - Offline Speech To Text

My Account

Information

Support S-Tech