PEAKS2IMAGE: RECONSTRUCTING FMRI STATISTI-CAL MAPS FROM PEAKS

Abstract

Neuroscience strives to overcome the lack of power due to the small sample size of imaging-based studies. An important step forward has been the creation of large-scale public image repositories, such as NeuroVault. Such repositories allow images to be compared across studies and automatically associated with cognitive terms. Yet, this type of meta-analysis faces a major roadblock: the scarcity and inconsistency of image annotations and metadata. Another resource containing rich annotations is the neuroscientific literature. However it only yields a handful of brain-space coordinates per publication, those of the main activity peaks reported in each study. This has led the community to mostly perform meta-analysis based on these reported coordinates. In this work, we propose Peaks2Image, a neuralnetwork approach to reconstruct continuous spatial representations of brain activity from peak activation tables. Peaks2Image thus associates rich annotations from the neuroscientific literature with dense brain reconstructions. Using those reconstructions, we train a decoder using tf-idf features as labels, leading to a much broader set of decoded terms than current image-based studies. We validate the decoder on 43,000 NeuroVault images, successfully decoding 65 out of 81 concepts in a zero-shot setting.

1. INTRODUCTION

Cognitive neuroscience aims to map cognitive processes onto brain regions. Functional Magnetic Resonance Imaging (fMRI) is one of the most powerful techniques available to identify such associations. This approach measures brain activity while subjects perform cognitive tasks in an MRI scanner, and then contrasts brain signals associated with different mental conditions. Statistical testing of these contrasts then identifies brain regions where the neural activity elicited by the conditions probed is significantly different. Unfortunately, the high cost of data acquisition limits the number of participants and tasks involved in each study (Poldrack et al., 2017) . A small number of participants causes low statistical power and a high proportion of false discoveries (Ioannidis, 2005; Button et al., 2013) . Analyzing a restricted set of cognitive tasks introduces the risk of over-interpreting statistical effects that are not specific to the mental functions under study (Poldrack, 2011) . Meta-analysis consists in aggregating the results of several studies to find effects that are reported consistently (Wager et al., 2007) . It helps overcome the challenge of small sample sizes and uncovers more reliable associations between brain activity and mental function. When feasible, meta-analysis should use the full statistical brain maps produced by the original studies (Salimi-Khorshidi et al., 2009) . Unfortunately, the vast majority of studies does not share the actual brain images. Instead, only the locations of the peaks of activation are reported. These are communicated in the form of tables, in scientific publications, containing stereotactic coordinates -3D coordinates in a standard spatial referential for the brain. This results in a poor representation of brain activity -most of the information contained in the original statistical maps is lost. Meta-analyses that rely on peak activation coordinates reported in publications are called Coordinate-Based Meta-Analyses (CBMA). Recent efforts to openly share full brain images and statistical maps such as NeuroVault (Gorgolewski et al., 2015) or OpenNeuro (Gorgolewski et al., 2017) , could facilitate Image-Based Meta-Analysis (IBMA) -relying on the brain images rather than coordinates. However, annotations available on those large databases are scarce and inconsistent, leading to difficulties to capture the semantics of the cognitive processes associated with images (Menuet et al., 2022) . There is no standard,

