IX Lab Research - 003
This episode discusses the 2024 paper: Mishra, A., Shukla, S., Torres, J., Gwizdka, J., & Roychowdhury, S. (2024). Thought2Text: Text Generation from EEG Signal using Large Language Models (LLMs) (No. arXiv:2410.07507). arXiv. https://doi.org/10.48550/arXiv.2410.07507
This research is conducted under Dr. Abhijit Mishra as the main Principal Investigator with students Shreya Shukla and Jose Torres and contributions from Dr. Jacek Gwizdka and Dr. Shounak Roychowdhury.
Summary
Researchers at the University of Texas at Austin are developing technology to translate brainwaves into text using electroencephalography (EEG) and large language models (LLMs). The system employs a three-stage process: training an EEG encoder to extract features, fine-tuning LLMs on multimodal data (images and text), and further refining the LLMs with EEG embeddings for text generation. Experiments using a public EEG dataset demonstrate the effectiveness of this approach, surpassing chance performance and showing promise for future applications in assistive technologies and neuroscience. While the technology shows promising results, it's still in its early stages and faces challenges such as noise in EEG data, limited spatial resolution of EEG, and ethical concerns about privacy and bias. Potential applications include assistive technology for communication impairments and advances in healthcare and education.
IX Lab website: https://ixlab.us/
Dr. Jacek Gwizdka website: https://jacekg.ischool.utexas.edu/
The audio for this conversation has been generated by AI using: https://notebooklm.google/
Music intro created by a human (C) Jacek Gwizdka
