Self-Calibrating BCIs: Ranking and Recovery of Mental Targets Without Labels
Reference Demo: Basics of Self-Calibrating Interfaces (no BCI)
Abstract
We consider the problem of recovering a mental target (e.g., an image of a face) that a participant has in mind from paired EEG (i.e., brain responses) and image (i.e., perceived faces) data collected during interactive sessions without access to labeled information. The problem has been previously explored with labeled data but not via self-calibration, where labeled data is unavailable. Here, we present the first framework and an algorithm, CURSOR, that learns to recover unknown mental targets without access to labeled data or pre-trained decoders. Our experiments on naturalistic images of faces demonstrate that CURSOR can (1) predict image similarity scores that correlate with human perceptual judgments without any label information, (2) use these scores to rank stimuli against an unknown mental target, and (3) generate new stimuli indistinguishable from the unknown mental target (validated via a user study, N=53). We release the brain response data set (N=29), associated face images used as stimuli data, and a codebase to initiate further research on this novel task.
(Left) CURSOR and human scores correlate strongly with similarity. [Blue] Predicted scores (S) against ground truth similarity (d). [Pink] Human-assigned scores to sets of 10 facial images morphing away from the target [right axis]. (Right) Faces at a distance below 1.6 suggest perceptual indistinguishability to the target. The vertical dotted lines show ranking and optimization performance, both recovering faces indistinguishable from the target.
Rank-related performance for different estimators and data sizes. (Left) Correlation coefficient (R) between scores and ground-truth distances. (Middle) Target rank out of 60. (Right) Euclidean distance to the target for top ranked candidate. The bottom dotted line is theoretical random performance. With LR rankings, human subjects could barely distinguish top-ranked faces from the target face.
(Left) Optimization converges to distance below 1 from the target. Candidates distance (d) to the target against iteration number. We can recover near-target faces even when near-target stimuli are absent in the dataset. Best candidates found after 1000 iterations shown against ablation distances ('d removed') around the target. (Right) Top-ranked (middle) and optimized (right) images per estimators from one randomly selected run and four randomly selected target faces (left). Images from CURSOR using Linear Regression (LR) are indistinguishable from the target image, while controls S-LR and Dummy exhibit visual differences.
Poster
BibTeX
@article{grizou2025self,
title={Self-Calibrating BCIs: Ranking and Recovery of Mental Targets Without Labels},
author={Grizou, Jonathan and de la Torre-Ortiz, Carlos and Ruotsalo, Tuukka},
journal={Advances in Neural Information Processing Systems},
volume={},
pages={},
year={2025}
}