Handling of missing EOG/ECG channels

[skjerns]

I have a dataset in which for some participants, the EOG or ECG is missing.

In this case, the ICA ECG detection raises a critical error, even when on_error='continue'

│15:38:49│ ⏳️ sub-01 No ECG-related ICs detected, this is highly suspicious. A manual check is suggested. You may wish to lower "ica_ecg_threshold".
│15:38:50│  sub-01 A critical error occurred. The error message was: 

Starting post-mortem debugger.
<traceback object at 0x7f24665ff080>
> /zi/home/simon.kern/anaconda3/lib/python3.11/site-packages/mne_bids_pipeline/steps/preprocessing/_06a2_find_ica_artifacts.py(219)find_ica_artifacts()
-> assert all([ch_name in raw.ch_names for ch_name in ch_names])

Is there any way to include the same participant in the pipeline and simply skip the ICA-ECG removal (without adding a dummy channel to the original data). Or to make on_error work as indented?

[larsoner]

For EOG/ECG missing, I think once you get this message:

│15:38:49│ ⏳️ sub-01 No ECG-related ICs detected, this is highly suspicious. A manual check is suggested. You may wish to lower "ica_ecg_threshold".

it shouldn't actually emit an error anyway. But actually the error isn't during ECG component detection, but during the start of EOG looks like:

mne-bids-pipeline/mne_bids_pipeline/steps/preprocessing/_06a2_find_ica_artifacts.py

Lines 218 to 226 in 9ab8472

	# EOG component detection
	epochs_eog = None
	eog_ics: list[int] = []
	eog_scores = np.zeros(0)
	for ri, raw_fname in enumerate(raw_fnames):
	raw = mne.io.read_raw_fif(raw_fname, preload=True)
	if cfg.eog_channels:
	ch_names = cfg.eog_channels
	assert all([ch_name in raw.ch_names for ch_name in ch_names])

So this does bring up the question of how to handle when the channel name is missing for a given subject.

To me the best behavior here would be:

1. When the EOG channel is missing, raise a proper error rather than assert something
2. Give a way to specify EOG channels on a per-subject basis, and/or set ICA EOG detection on/off on a subject-by-subject basis

(2) could be fairly easily solvable I think if we allowed (and pydantic handles nicely):

eog_channels : Sequence[str] | dict[str, Sequence[str]]

where the keys are subject names. Then your code could be:

eog_channels = defaultdict(lambda: ["EEG001"])
eog_channels["sub-01"] = []

or similar. WDYT?

(For on_error not working, I opened #1022)

[larsoner]

cc @hoechenberger @drammock as this ☝️ brings up a larger issue of how to handle subject-by-subject variability for variables in general. The idea I propose above is to have a dict[str, ...] where you can map subject name keys to values. In the case above it's for eog_channels but you could imagine this generalizing to all sorts of other stuff like number of projectors, etc. so it would be good to make sure this approach makes sense before proceeding.

[skjerns]

Coming back to this issue: I have added the missing channels as flat lines, however then the pipeline fails with

│10:16:35│ ⏳️ sub-01 Creating EOG epochs …
│10:16:35│  sub-01 A critical error occurred. The error message was: can't multiply sequence by non-int of type 'float'

And this prevents the ICA solution from being saved.

│10:16:52│  sub-01 A critical error occurred. The error message was: missing in_files["ica"] = /data/fastreplay/Fast-Replay-MEG-bids/derivatives/sub-01/meg/sub-01_task-main_proc-ica_ica.fif

Any idea how I can still use that subject?

[larsoner]

Coming back to this issue: I have added the missing channels as flat lines, however then the pipeline fails with

This looks like a bug, most likely in MNE-Python (and most likely someplace we do np.zeros(...) instead of np.zeros(..., int) or similar, then an inplace multiply). Can you paste the whole traceback?

[larsoner]

... you might need to run with --debug to get the whole traceback printed

[skjerns]

Ah, yes, it is indeed an error of mne not numpy :) somehow the saved data contains NaNs