Overview

While neural network based Automatic Speech Recognition (ASR) systems have reached impressive milestones, these systems remain notoriously data-hungry, often degrading sharply when faced with languages that lack massive, labelled data. This tutorial investigates the critical challenge of developing efficient speech recognition for the world’s diverse languages by exploring the synergy between speech processing and Large Language Models (LLMs). We move beyond traditional architectures to introduce modern LLM-ASR methods, organized according to whether the speech–language interface is continuous or discrete.
The tutorial first introduces the basics in pursuing data-efficient multilingual and crosslingual speech recognition and the "principled unsupervised learning" philosophy that drives modern LLMs. We then transition into a detailed examination of continuous-interface approaches, where speech encoders are bridged to LLMs via adapters and projection layers. Following this, we explore discrete-interface approaches, highlighting LLM-based generative error correction, LLM-based phoneme-to-grapheme methods, and innovative "lexicon-free" training techniques. At the end of this tutorial, participants will have a clear taxonomy and a practical roadmap for leveraging the reasoning capabilities of LLMs to build inclusive, highperforming speech technology with limited resources.