Spracherkennung & Sprachsynthese auf Arduino Due meistern

Komponenten und Materialien

• Arduino Due (×1) – 3,3 V, 2× DAC, DMA
• SparkFun Electret Microphone Breakout (×1)
• SparkFun Mono Audio Amp Breakout (×1)
• Lautsprecher 0,25 W, 8 Ω (×1)
• Steckplatine (×1)
• LED 5 mm rot (×3)
• Widerstand 330 Ω (×3)
• Jumper-Kabel (×1 Set)

Werkzeuge

• Lötkolben

Software & Dienste

• BitVoicer Server 1.0 (kostenpflichtig, ~9,90 $) – Spracherkennung & Synthese Server (Windows/Linux/Mac).

Projektbeschreibung

Erweiterung eines vorherigen Projekts: Nun wird Sprache nicht nur erkannt, sondern auch synthetisiert – dank der zwei DACs und DMA des Arduino Due. BitVoicer Server übernimmt die schwere Arbeit (ASR/TTS), der Due streamt Audio hin und her.

Im Demo-Video: LEDs simulieren Klaviertasten (C, D, E) zu einem Jingle. Synchronisation via Sonic Visualizer (Zeitmarken der Tastenanschläge).

Signalfluss

1. Elektret-Mikrofon + Breakout → Verstärkung.
2. Due ADC digitalisiert Audio, puffert.
3. Audio-Stream → BitVoicer Server (Serial, 115200 Baud).
4. Server erkennt Sprache, mapped auf Befehle.
5. Befehle (Byte-Arrays) → Due zurück.
6. Bei TTS: Server sendet Audio-Stream → Due queued in BVSSpeaker → DAC + DMA → Audio-Amp → Lautsprecher.

Verdrahtung (Wichtig: Due = 3,3 V!)

• Mikrofon-Breakout an 3,3 V (nicht 5 V!) – AREF-Pin nicht bridgen (Due nutzt intern 3,3 V Referenz).
• Audio-Amp an DAC0 (Pin A0) oder DAC1.
• LEDs an D6, D9, D10 (PWM).
• GND gemeinsam.

Arduino-Sketch (BVS_Demo2.ino)

#include <BVSP.h> #include <BVSMic.h> #include <BVSSpeaker.h> #include <DAC.h>

#define BVSM_AUDIO_INPUT 7
#define RED_LED_PIN 6
#define YELLOW_LED_PIN 9
#define GREEN_LED_PIN 10

const unsigned long STATUS_REQUEST_TIMEOUT = 3000;
const unsigned long STATUS_REQUEST_INTERVAL = 4000;
const int MIC_BUFFER_SIZE = 64;
const int SPEAKER_BUFFER_SIZE = 128;
const int RECEIVE_BUFFER_SIZE = 2; // Pin, Value

BVSP bvsp;
BVSMic bvsm;
BVSSpeaker bvss;

byte micBuffer[MIC_BUFFER_SIZE];
byte speakerBuffer[SPEAKER_BUFFER_SIZE];
byte receiveBuffer[RECEIVE_BUFFER_SIZE];

bool playLEDNotes = false;
unsigned int playStartTime = 0;

void setup() {
  pinMode(RED_LED_PIN, OUTPUT); pinMode(YELLOW_LED_PIN, OUTPUT); pinMode(GREEN_LED_PIN, OUTPUT);
  digitalWrite(RED_LED_PIN, LOW); digitalWrite(YELLOW_LED_PIN, LOW); digitalWrite(GREEN_LED_PIN, LOW);
  Serial.begin(115200);
  bvsp.begin(Serial, STATUS_REQUEST_TIMEOUT, STATUS_REQUEST_INTERVAL);
  bvsp.frameReceived = BVSP_frameReceived;
  bvsp.modeChanged = BVSP_modeChanged;
  bvsp.streamReceived = BVSP_streamReceived;
  bvsm.begin();
  bvss.begin(DAC); // Due DAC
}

void loop() {
  bvsp.keepAlive();
  bvsp.receive();

  if (bvsp.isSREAvailable()) {
    if (!bvsm.isRecording) { bvsm.setAudioInput(BVSM_AUDIO_INPUT, DEFAULT); bvsm.startRecording(); }
    if (bvsm.available) {
      if (bvsp.inboundMode == FRAMED_MODE) bvsp.setInboundMode(STREAM_MODE);
      int bytesRead = bvsm.read(micBuffer, MIC_BUFFER_SIZE);
      bvsp.sendStream(micBuffer, bytesRead);
    }
  } else {
    if (bvsm.isRecording) bvsm.stopRecording();
  }
  bvss.play();
  if (playLEDNotes) playNextLEDNote();
}

void BVSP_frameReceived(byte dataType, int payloadSize) {
  if (dataType == DATA_TYPE_BINARY) {
    if (bvsp.getReceivedBytes(receiveBuffer, RECEIVE_BUFFER_SIZE) == RECEIVE_BUFFER_SIZE)
      analogWrite(receiveBuffer[0], receiveBuffer[1]);
  } else if (dataType == DATA_TYPE_BYTE) {
    if (bvsp.getReceivedByte() == 255) { playLEDNotes = true; playStartTime = millis(); }
  }
}

void BVSP_modeChanged() {
  if (bvsp.outboundMode == FRAMED_MODE) bvss.finishPlaying();
}

void BVSP_streamReceived(int size) {
  int bytesRead = bvsp.getReceivedStream(speakerBuffer, SPEAKER_BUFFER_SIZE);
  bvss.enqueue(speakerBuffer, bytesRead);
}

void playNextLEDNote() {
  unsigned long elapsed = millis() - playStartTime;
  allLEDsOff();
  if (elapsed >= 11500) { analogWrite(RED_LED_PIN, 0); playLEDNotes = false; }
  else if (elapsed >= 9900) analogWrite(RED_LED_PIN, 255);      // C
  else if (elapsed >= 9370) analogWrite(RED_LED_PIN, 255);      // C
  else if (elapsed >= 8900) analogWrite(YELLOW_LED_PIN, 255);   // D
  else if (elapsed >= 8610) analogWrite(RED_LED_PIN, 255);      // C
  else if (elapsed >= 8230) analogWrite(YELLOW_LED_PIN, 255);   // D
  else if (elapsed >= 7970) analogWrite(YELLOW_LED_PIN, 255);   // D
  else if (elapsed >= 7470) analogWrite(RED_LED_PIN, 255);      // C
  else if (elapsed >= 6760) analogWrite(GREEN_LED_PIN, 255);    // E
  else if (elapsed >= 6350) analogWrite(RED_LED_PIN, 255);      // C
  else if (elapsed >= 5880) analogWrite(YELLOW_LED_PIN, 255);   // D
  else if (elapsed >= 5560) analogWrite(RED_LED_PIN, 255);      // C
  else if (elapsed >= 5180) analogWrite(YELLOW_LED_PIN, 255);   // D
  else if (elapsed >= 4890) analogWrite(YELLOW_LED_PIN, 255);   // D
  else if (elapsed >= 4420) analogWrite(RED_LED_PIN, 255);      // C
  else if (elapsed >= 3810) analogWrite(GREEN_LED_PIN, 255);    // E
  else if (elapsed >= 3420) analogWrite(RED_LED_PIN, 255);      // C
  else if (elapsed >= 2930) analogWrite(YELLOW_LED_PIN, 255);   // D
  else if (elapsed >= 2560) analogWrite(RED_LED_PIN, 255);      // C
  else if (elapsed >= 2200) analogWrite(YELLOW_LED_PIN, 255);   // D
  else if (elapsed >= 1930) analogWrite(YELLOW_LED_PIN, 255);   // D
  else if (elapsed >= 1470) analogWrite(RED_LED_PIN, 255);      // C
  else if (elapsed >= 1000) analogWrite(GREEN_LED_PIN, 255);    // E
}

void allLEDsOff() {
  analogWrite(RED_LED_PIN, 0); analogWrite(YELLOW_LED_PIN, 0); analogWrite(GREEN_LED_PIN, 0);
}

BitVoicer Server Konfiguration

Lösungsobjekte (Import via .sof Dateien)

1. Locations: z. B. „Home“.
2. Devices: „Mixed“ Device „ArduinoDUE“, Kommunikationseinstellungen: Data Rate 8000 samples/s begrenzen (Pufferüberlauf vermeiden).
3. BinaryData: Je LED-Zustand ein Objekt (z. B. ArduinoDUEGreenLedOn = [9, 255]). Insgesamt 18 Objekte.
4. Voice Schemas: Sätze definieren (z. B. „turn on green light“, „play a little song“). Befehle sequenzieren: erst Byte 255 (Start LED-Noten), dann Audio-Stream (TTS oder aufgenommenes Jingle).

Dateien: Device.sof, VoiceSchema.sof – im Anhang.

Audio-Format für TTS

BitVoicer akzeptiert nur 8-bit mono PCM, 8000 Hz. Konvertierung z. B. via Online-Converter.

Fazit & Ausblick

Projekt demonstriert bidirektionales Audio-Streaming mit Spracherkennung und -synthese auf dem Due. Nächstes Ziel: WiFi-Arduino steuert zwei weitere Arduinos per Sprache – Spielideen willkommen!