2024 Akraino Spring Summit

Abstract - Small Language Model for Device AI Applications

Device AI applications running on very small form-factor edge devices (for example pico ITX), and without a cloud connection, need to perform automatic speech recognition (ASR) under difficult conditions, including background noise, urgent or stressed voice input, and other talkers in the background. For robotics applications, background noise may also include servo motor and other mechanical noise. Under these conditions, ASRs such as Kaldi and Whisper tend to produce "sound alike" errors, for example:

  in the early days, a king rolled the stake
  
which contains two (2) sound-alike errors, and should be corrected to "in the early days a king ruled the state". Sound-alike errors are particularly problematic for robotics applications in which the robot OS requires precise commands, for example a stalled robotaxi must be instructed "move forward 20 feet, to the right 10 feet, raise the hood, and turn off the engine". A first responder may give these commands as "get off the road in that turn-out up ahead and shut it down" or similar. Any sound-alike errors in the first responder's commands make translation to machine commands problematic.

To address this problem, a Small Language Model (SLM) is needed to correct sound-alike errors, and capable of running in a very small form-factor and under 10W, for example using two (2) Atom CPU cores. The SLM must run every 1/2 second and with a backwards/forwards context of 3-4 words.