Abstract
Continuous affect tracking is central to closed loop affective brain-computer interfaces and adaptive human computer interaction. Continuous EEG-fNIRS regression is well suited to this setting because EEG captures rapid neural dynamics and fNIRS provides complementary hemodynamic information, but direct fusion often underperforms EEG-only baselines when neurovascular coupling (NVC) introduces subject- and region-dependent delays. PhysioSync addresses this by performing HRF-like temporal alignment and residual-aware reliability control before multimodal fusion. A learnable dual-gamma kernel maps EEG-derived drive to the fNIRS timeline through finite window causal convolution, and the resulting residuals modulate the contribution of hemodynamic evidence during fusion. A linear-complexity Mamba encoder provides long-context EEG representation. On REFED, PhysioSync reduces Avg MAE by 5.8% relative to the best fusion baseline and by 7.2% relative to EEG-only when evaluated across three within-subject folds, while showing lower error across the next three annotation steps. With lightweight task-specific adaptations, ENTER and ZBra-music results indicate conditional applicability to discrete classification rather than uniform transferability, with performance depending on EEG coverage and HbO/HbR chromophore decomposition. Overall, these results support PhysioSync as a subject-calibrated EEG-fNIRS fusion framework for continuous physiological affect tracking in adaptive affective BCI settings, with applicability shaped by calibration, signal quality, and modality coverage.