Zipf's law suggests a three-pronged approach to inclusive speech recognition
ECE 590SIP, February 1, 2023, Mark Hasegawa-Johnson
Abstract:
Zipf's law is commonly glossed by the aphorism "infrequent words are frequent," but in practice, it has often meant that there are three types of words: frequent, infrequent, and out-of-vocabulary (OOV). Speech recognition solved the problem of frequent words in 1970 (with dynamic time warping). Hidden Markov models worked well for moderately infrequent words, but the problem of OOV words was not solved until sequence-to-sequence neural nets de-reified the concept of a word. Many other social phenomena follow power-law distributions. The number of native speakers of the N'th most spoken language, for example, is 1.44 billion over N to the 1.09. In languages with sufficient data, we have shown that monolingual pre-training outperforms multilingual pre-training. In less-frequent languages, multilingual knowledge transfer can significantly reduce phone error rates. In languages with no training data, unsupervised ASR methods can be proven to converge, as long as the eigenvalues of the language model are sufficiently well separated to be measurable. Other systems of social categorization may follow similar power-law distributions. Disability, for example, can cause speech patterns that were never seen in the training database, but not all disabilities need do so. The inability of speech technology to work for people with even common disabilities is probably caused by a lack of data, and can probably be solved by finding better modes of interaction between technology researchers and the communities served by technology.
Bio:
Mark Hasegawa-Johnson is a William L. Everitt Faculty Fellow of Electrical and Computer Engineering at the University of Illinois in Urbana-Champaign. He has published research in speech production and perception, source separation, voice conversion, and low-resource automatic speech recognition.