Auditory EEG Viability for Brain Passage Retrieval

Can listening-based EEG signals enable effective brain-based information retrieval?

cs.IREEG / BCI3 Training Regimes20 Subjects

1.000

Auditory R@1 (Aud-Only)

0.878

Cross-Sensory R@1

1.000

Combined MRR

Retrieval Performance Summary

Training	Evaluation	R@1	R@5	MRR	NDCG@10
Visual-Only	Visual	1.000	1.000	1.000	1.000
Visual-Only	Auditory	0.878	0.958	0.911	0.926
Auditory-Only	Visual	0.995	1.000	0.997	0.998
Auditory-Only	Auditory	1.000	1.000	1.000	1.000
Combined	Visual	1.000	1.000	1.000	1.000
Combined	Auditory	1.000	1.000	1.000	1.000

Auditory EEG is viable for Brain Passage Retrieval. Auditory-only training achieves perfect retrieval, and combined cross-sensory training eliminates the 12.2% gap seen when using visual-only training on auditory stimuli.

R@1 by Training Regime

MRR by Training Regime

Full Recall Comparison

Cross-Sensory Transfer

Visual-only training transfers to auditory evaluation with R@1=0.878 (an 12.2% gap), while combined training eliminates this gap entirely. This demonstrates that visual and auditory EEG share underlying semantic representations suitable for passage retrieval.

Modality Gap Analysis

Experiment Setup

Subjects	20
EEG Channels	64
Trials per condition	600
Visual channels	Occipital (O1, O2, Oz) + Parietal
Auditory channels	Temporal (T7, T8, TP7, TP8) + Frontal

Implications

Accessibility: Auditory BPR enables brain-based retrieval for users with visual impairments
Voice interfaces: Supports thought-driven search during conversational interactions
Data augmentation: Visual EEG data can augment limited auditory datasets through combined training
Channel specificity: Temporal lobe channels are most important for auditory BPR