Presentation Information
[A-8-20]Evaluation of a Transcription Error Correction Method Using a Large Language Model in a Local Environment
◎△Hinata Tsujiura1, Haruka Kataoka2, Akihito Kohiga1, Takahiro Koita2 (1. Doshisha Univ., 2. Graduate School of Doshisha Univ.)
Keywords:
Automatic Speech Recognition,Large Language Model
Methods that apply a large language model (LLM) to correct automatic speech recognition (ASR) transcription errors have been widely studied. While cloud services offer high-performance models, sending audio or transcripts with named entities raises privacy concerns.
Prior work masks audio segments with named entities before sending the masked audio to the cloud. However, detection errors expose sensitive information, and local detection accuracy is often insufficient, limiting privacy protection.
This study performs both ASR and LLM-based correction locally, avoiding data transmission to the cloud. Since applying the LLM to all utterances may cause overcorrection, we apply it only to low-confidence utterances via confidence-based filtering.
Using Whisper tiny and Qwen2.5-7B on SLUE-VoxPopuli, applying the LLM to all utterances worsened the word error rate (WER), but filtering reduced WER below ASR alone at a certain threshold. Confusion matrix analysis showed the filtering suppressed overcorrection but missed many utterances requiring correction. The method shows potential for privacy protection with modest accuracy gains, though more precise filtering remains future work.
Prior work masks audio segments with named entities before sending the masked audio to the cloud. However, detection errors expose sensitive information, and local detection accuracy is often insufficient, limiting privacy protection.
This study performs both ASR and LLM-based correction locally, avoiding data transmission to the cloud. Since applying the LLM to all utterances may cause overcorrection, we apply it only to low-confidence utterances via confidence-based filtering.
Using Whisper tiny and Qwen2.5-7B on SLUE-VoxPopuli, applying the LLM to all utterances worsened the word error rate (WER), but filtering reduced WER below ASR alone at a certain threshold. Confusion matrix analysis showed the filtering suppressed overcorrection but missed many utterances requiring correction. The method shows potential for privacy protection with modest accuracy gains, though more precise filtering remains future work.