2024 Akraino Spring Summit

2 hour on-site discussion
  Agenda(Idea)
    Presentation session
    Akraino 2024 activities
    Collaboration with other communities

Welcome note
Yin Ding TSC Chair
Haruhisa Fukano TSC Co-Chair

Jeff Brower,
CEO, Signalogic

Small Language Model for Device AI Applications

Device AI applications running at the AI Edge on very small form-factor devices (for example pico ITX), and without an online cloud connection, need to perform automatic speech recognition (ASR) under difficult conditions, including background noise, urgent or stressed voice input, and other talkers in the background. For robotics applications, background noise may also include servo motor and other mechanical noise. Under these conditions, efficient open source ASRs such as Kaldi and Whisper tend to produce "sound-alike" errors, for example:

  in the early days a king rolled the stake
  
which contains two (2) sound-alike errors that must be corrected to "in the early days a king ruled the state". Sound-alike errors are particularly problematic for robotics applications in which the robot OS requires precise API commands, for example a robotaxi has stalled and must be instructed to "move forward 20 feet, to the right 10 feet, raise the hood, and turn off the engine". A first responder may use a portable backpack device and give commands "get off the road in that turn-out up ahead and shut it down" or similar. Any sound-alike errors in voice commands make translation to machine-readable APIs problematic.

To address this issue independently of ASR model, Signalogic is developing a Small Language Model (SLM) to correct sound-alike errors, capable of running in a very small form-factor and under 10W, for example using two (2) Atom CPU cores. The SLM must run every 1/2 second and with a backwards/forwards context of 3-4 words. Unlike an LLM, a wide context window, domain knowledge, and extensive web page training are not needed.