2024 Akraino Spring Summit

Abstract - Small Language Model for Device AI Applications

Device AI applications running on very small form-factor edge devices (for example pico ITX), and without a cloud connection, need to perform automatic speech recognition (ASR) under difficult conditions, including background noise, urgent or stressed voice input, and other talkers in the background. For robotics applications, background noise may also include servo motor and other mechanical noise. Under these conditions, ASRs such as Kaldi and Whisper tend to produce "sound-alike" errors, for example:

  in the early days, a king rolled the stake
  
which contains two (2) sound-alike errors that must be corrected to "in the early days a king ruled the state". Sound-alike errors are particularly problematic for robotics applications in which the robot OS requires precise API commands, for example a robotaxi has stalled and must be instructed to "move forward 20 feet, to the right 10 feet, raise the hood, and turn off the engine". A first responder may give these commands as "get off the road in that turn-out up ahead and shut it down" or similar. Any sound-alike errors in voice commands make translation to machine APIs problematic.

To address this issue, a Small Language Model (SLM) is needed to correct sound-alike errors, capable of running in a very small form-factor and under 10W, for example using two (2) Atom CPU cores. The SLM must run every 1/2 second and with a backwards/forwards context of 3-4 words. Unlike an LLM, a wide context window, domain knowledge, and extensive web page training is not needed.