CRITICAL: FIX FOR SPINDLES DETECTION! (raphaelvallat#55)

* CRITICAL: FIX FOR SPINDLES DETECTION! * Updated no.convolve in notebook * Removed notebook * Updated changelog
soupeeli · Mar 15, 2022 · d1cb4aa · d1cb4aa
1 parent 2383628
commit d1cb4aa
Show file tree

Hide file tree

Showing 3 changed files with 86 additions and 68 deletions.
diff --git a/docs/changelog.rst b/docs/changelog.rst
@@ -8,6 +8,15 @@ What's new
 
 ----------------------------------------------------------------------------------------
 
+v0.6.1 (March 2022)
+-------------------
+
+This release fixes a CRITICAL BUG with the spindles detection. Specifically, the :py:func:`yasa.spindles_detect` could return different results depending on the sampling rate of the data.
+For example, downsampling the data from 256 Hz to 128 Hz may have significantly reduced the number of detected spindles. As explained in `issue 54 <https://github.com/raphaelvallat/yasa/issues/54>`_, this bug was caused by a floating-point error
+in :py:func:`np.convolve` when calculating the soft spindle threshold. Tests seem to indicate that only certain sampling frequencies were impacted, such as 200 Hz, 256 Hz or 400 Hz. Other sampling frequencies such as 100 Hz and 500 Hz were seemingly not affected by this bug. Please double-check any results obtained with :py:func:`yasa.spindles_detect`!
+
+.. warning:: We recommend all users to upgrade to this new version ASAP and check any results obtained with the :py:func:`yasa.spindles_detect` function!
+
 v0.6.0 (February 2022)
 ----------------------
 

diff --git a/notebooks/01_spindles_detection.ipynb b/notebooks/01_spindles_detection.ipynb
diff --git a/yasa/detection.py b/yasa/detection.py
@@ -581,6 +581,12 @@ def spindles_detect(data, sf=None, ch_names=None, hypno=None,
 
     For better results, apply this detection only on artefact-free NREM sleep.
 
+    .. warning::
+        A critical bug was fixed in YASA 0.6.1, in which the number of detected spindles could
+        vary drastically depending on the sampling frequency of the data. Please make sure to check
+        any results obtained with this function prior to the 0.6.1 release.
+
+
     References
     ----------
     The sleep spindles detection algorithm is based on:
@@ -719,9 +725,12 @@ def spindles_detect(data, sf=None, ch_names=None, hypno=None,
 
         # The detection using the three thresholds tends to underestimate the
         # real duration of the spindle. To overcome this, we compute a soft
-        # threshold by smoothing the idx_sum vector with a 100 ms window.
+        # threshold by smoothing the idx_sum vector with a ~100 ms window.
+        # Sampling frequency = 100 Hz --> w = 10 samples
+        # Sampling frequecy = 256 Hz --> w = 25 samples = 97 ms
         w = int(0.1 * sf)
-        idx_sum = np.convolve(idx_sum, np.ones(w) / w, mode='same')
+        # Critical bugfix March 2022, see https://github.com/raphaelvallat/yasa/pull/55
+        idx_sum = np.convolve(idx_sum, np.ones(w), mode='same') / w
         # And we then find indices that are strictly greater than 2, i.e. we
         # find the 'true' beginning and 'true' end of the events by finding
         # where at least two out of the three treshold were crossed.