is it possible to run openai-whisper ggml model on raspberry pi hardware?

[nyadla-sys]

is it possible to run this gghml model on raspberry pi hardware?

[nyadla-sys]

@ggerganov could you please help on this ?

[ggerganov]

It will probably work - why don't you give it a try?

[ggerganov]

Good news!

I just tried it on a Raspberry Pi 4 Model B from 2018 and it works!

The tiny.en model takes 140 sec to transcribe a 30 sec audio, but I think this can be improved, because I disabled all SIMD instructions to make it compile. I will improve this in the following days.

If you want to try it, use the raspberry branch:

git clone https://github.com/ggerganov/whisper.cpp
cd whisper.cpp
git checkout reaspberry
make tiny.en

[nyadla-sys]

I don't currently have a Raspberry Pi board, but I will run as soon as I get one.

[ggerganov]

You can try running it on whatever Raspberry you have - use the same instructions.

[nyadla-sys]

@ggerganov Thanks and appreciate for your quick response

[ggerganov]

Some more experiments - enabling NEON instructions reduces the time all the way down to just ~15 seconds to process a 30 second audio.

[nyadla-sys]

this is awesome

[nyadla-sys]

@ggerganov is it possible to do the audio streaming on Raspberry Pi and live convert it to captions?

[WilliamTambellini]

@ggerganov
perhaps have a look at
https://github.com/oneapi-src/oneDNN
It would resolve all the compilation issues by letting onednn do the optimizations for the local CPU at runtime whatever the cpu model/brand.

[nyadla-sys]

@ggerganov On a linux computer, I tried the following commands, and streaming performed as expected.
$ git clone https://github.com/ggerganov/whisper.cpp.git
$ bash ./download-ggml-model.sh tiny.en
$ sudo apt-get install libsdl2-dev
$ make
$ make stream -lSDL2
$ ./stream -m models/ggml-tiny.en.bin