(See the preprint on bioRxiv)
In a nutshell
Do the human brain and computers share similar semantic representation?
Can Natural Language Processing (NLP) models predict human neural activition in the language domain?
Do NLP models differ in their predictability?
How do we join forces between neuroscience and computer science to advance our understanding for the computational mechanisms underlying semantic processing?
Semantic representation, a crucial window into human cognition, has been studied independently in neuroscience and computer science. A deep understanding of neural computations in the human brain and the revolution to a strong artificial intelligence appeal for a necessity of joint force in the language domain.
Recently, common representations have been identified in the human visual system and in deep-learning neural network models. However, because of limited understanding of language processing as well as constrained capacity of neuroscientific approaches in language research, bridging between human brain and artificial intelligence in the language domain is difficult and scarce. In this project, we investigated the representational formats of comparable lexical semantic features between these two complex systems with fine temporal resolution neural recordings.
We found semantic representations generated from computational models significantly correlated with EEG responses at an early stage of a typical semantic processing time window in a two-word semantic priming paradigm. Moreover, three selected computational models differentially predicted EEG responses along the dynamics of word processing in the human brain. Our study provided a finer-grained understanding of the neural dynamics underlying semantic processing and developed an objective biomarker for assessing human-like computation in computational models. Our novel framework trailblazed a promising way to bridge across disciplines in the investigation of higher-order cognitive functions in human and artificial intelligence.