Home Automation Services

Rhasspy Voice Assistant with ReSpeaker 4 Mic array on Raspberry 4

Step-by-step guide to setup a local voice assistant using Rhasspy Voice Assistant with ReSpeaker 4 Mic array on Raspberry 4 with Home Assistant

Prerequisites

Step 1

Install Raspberry Pi OS Lite, a “headless” and lightweight version of the Raspberry Pi operating system using Raspberry Pi Imager on a 32GB SD card. Enter WiFI and device name and credentials into Pi Imager config. I named mine rhasspy. This RaspiOS version is the only one I could get working with the ReSpeaker drivers.

Plug the ReSpeaker HAT, Headphones, and SD card into the Raspberry Pi and power on the device. Determine the IP address.

Step 2

Login to the Pi using SSH (I use putty).

  • Confirm your OS version is 5.10.103-v7l+ and update apt. Install git if necessary.
  • Install the ReSpeaker microphone drivers.
  • Confirm driver install with arecord -L
uname -r
sudo apt-get update
sudo reboot
sudo apt-get install --yes git
git clone https://github.com/HinTak/seeed-voicecard
cd seeed-voicecard
sudo ./install.sh
sudo reboot
rhasspy@rhasspysat1:~ $ arecord -L
null
    Discard all samples (playback) or generate zero samples (capture)
jack
    JACK Audio Connection Kit
pulse
    PulseAudio Sound Server
default
playback
ac108
usbstream:CARD=Headphones
    bcm2835 Headphones
    USB Stream Output
sysdefault:CARD=seeed4micvoicec
    seeed-4mic-voicecard, bcm2835-i2s-ac10x-codec0 ac10x-codec0-0
    Default Audio Device
dmix:CARD=seeed4micvoicec,DEV=0
    seeed-4mic-voicecard, bcm2835-i2s-ac10x-codec0 ac10x-codec0-0
    Direct sample mixing device
dsnoop:CARD=seeed4micvoicec,DEV=0
    seeed-4mic-voicecard, bcm2835-i2s-ac10x-codec0 ac10x-codec0-0
    Direct sample snooping device
hw:CARD=seeed4micvoicec,DEV=0
    seeed-4mic-voicecard, bcm2835-i2s-ac10x-codec0 ac10x-codec0-0
    Direct hardware device without any conversions
plughw:CARD=seeed4micvoicec,DEV=0
    seeed-4mic-voicecard, bcm2835-i2s-ac10x-codec0 ac10x-codec0-0
    Hardware device with all software conversions
usbstream:CARD=seeed4micvoicec
    seeed-4mic-voicecard
    USB Stream Output
rhasspy@rhasspysat1:~ $

Step 3

Install Rhasspy

  • Install docker
  • Add rhassy user to the docker group
  • Reboot

Note

Use these instructions for Raspberry Pi Zero.

curl -sSL https://get.docker.com | sh
sudo usermod -aG docker rhasspy
sudo reboot

docker pull rhasspy/rhasspy

docker run -d \
      -p 12101:12101 \
      --name rhasspy \
      --restart unless-stopped \
      -v "$HOME/.config/rhasspy/profiles:/profiles" \
      -v "/etc/localtime:/etc/localtime:ro" \
      --device /dev/snd:/dev/snd \
      rhasspy/rhasspy \
      --user-profiles /profiles \
      --profile en

Step 4

Configure Rhasspy Base

  • For the base station, set MQTT to “External” and configure the details of your broker. Next, set Dialogue Management to “Rhasspy” and select your preferred Speech to Text, Intent Recognition, and Text to Speech services. Make sure to save settings, restart
  • I used the Mosquitto MQTT broker already setup in Homeassistant addon for Rhasspy. I added a new user called rasspy_user as well.
  • Rhasspy, download profile artifacts, update sentences, and train the profile on your base station

Configure Rhasspy Satellite

  • On your satellite, start by changing your “siteId” in the Settings to “satellite”. Afterwards, set MQTT to “External” and configure the details of your broker. Set the speech to text, intent recognition, and (optionally) the text to speech services to “Hermes MQTT”. Make sure to disable “Dialogue Management”, and enable audio recording, wake word, and audio playing.
  • Setting a service to “Hermes MQTT” means that Rhasspy will expect some service connected to your broker to handle the appropriate MQTT messages for speech to text, etc.
  • Under each service (including Dialogue Management), add the site id of each of your satellites to the “Satellite siteIds” text box (separated by commas). This will cause that particular service to response to MQTT messages coming from that satellite.
  • Adding the satellite site id(s) to the dialogue manager is crucial, since it will catch wake word detections, engage the ASR and NLU systems, and dispatch TTS audio playback to the appropriate site.

Step 5

Configure LEDs

  • Activate SPI: sudo raspi-config; Go to “Interfacing Options”; Go to “SPI”; Enable SPI; Exit the tool
  • Get APA102 LEDs Library and examples

Configure Rhasspy Satellite

  • On your satellite, start by changing your “siteId” in the Settings to “satellite”. Afterwards, set MQTT to “External” and configure the details of your broker. Set the speech to text, intent recognition, and (optionally) the text to speech services to “Hermes MQTT”. Make sure to disable “Dialogue Management”, and enable audio recording, wake word, and audio playing.
  • Setting a service to “Hermes MQTT” means that Rhasspy will expect some service connected to your broker to handle the appropriate MQTT messages for speech to text, etc.
  • Under each service (including Dialogue Management), add the site id of each of your satellites to the “Satellite siteIds” text box (separated by commas). This will cause that particular service to response to MQTT messages coming from that satellite.
  • Adding the satellite site id(s) to the dialogue manager is crucial, since it will catch wake word detections, engage the ASR and NLU systems, and dispatch TTS audio playback to the appropriate site.
cd /home/rhasspy
git clone https://github.com/respeaker/4mics_hat.git
cd /home/rhasspy/4mics_hat
sudo apt install python-virtualenv          # install python virtualenv tool
virtualenv --system-site-packages ~/env     # create a virtual python environment
source ~/env/bin/activate                   # activate the virtual environment
pip install spidev gpiozero           # install spidev and gpiozero

Next Steps

  • Setup UDP audio streaming
  • Setup Raven custom wake work
  • Integrate with HA entities

Explore related posts:

, ,

If you found this post useful, please donate to help this site create more useful content.

Useful References