Thursday, 5 December 2024, 13:00 – 14:00

Beyond Words: Non-Linguistic Behavior Generation for Human-like Conversational AI

Invited Speaker

Koji Inoue
Kyoto University

Abstract

The development of multimodal large language models (MLLMs), such as ChatGPT-4, has significantly enhanced the capabilities of conversational AI, enabling a wide range of practical applications. Despite these advancements, these models still face challenges in generating non-linguistic behaviors that are essential for the naturalness and fluidity of human conversations. This talk will explore key non-linguistic behaviors including turn-taking, backchanneling, and laughter by tracing the evolution of research from traditional machine-learning approaches to modern Transformer-based models. Additionally, it will address the generalizability of a Transformer-based model in realizing a foundational model for non-linguistic behavior generation as MLLMs continue to advance.

Biography

Koji Inoue received his Ph.D. from the Graduate School of Informatics at Kyoto University, Japan, in 2018, following his graduation from Kurume National College of Technology in 2013. He is currently an assistant professor at Kyoto University. In 2023, he served as a visiting researcher at KTH Royal Institute of Technology in Sweden. His research interests include conversational AI, spoken dialogue systems, and human-robot interaction. His team has developed a spoken dialogue system for android ERICA, which earned the NETEXPLO Innovation 2022 Award.