Very large difference in sorting using SC2 vs Lupin with dense recordings

[b-grimaud]

I've been comparing SC2 and Lupin on the same 3 minutes-long recording of a cerebellar slice on a 3Brain BioCAM.

Running both sorters with default params and their own preprocessing, apart from removing the calibrating channel and switching from unsigned to signed, the outputs are extremely different.

SC2 results in 4187 units, distributed across the entire chip :

Of those, most seem contaminated with an average isi_violations_ratio of 18.8, and if I apply a threshold of 0.5% of contaminating spikes, only 120 units remain.

Zooming in, it seems that basically every channel has its own unit :

(I labelled each localization directly with the unit ID, in a less-than-elegant way).

Looking at the templates, those units look like this :

Whereas units founds outside of channels and in more believable clusters look like this :

I'm pretty sure those "channel units" are actually signal from the calibrating electrode that gets propagated through the entire chip. Considering how it looks, I'm not surprised they get detected as spikes.

Lupin, on the other hand, returns 191 units.

The distribution across the chip is a lot more believable, but the units are still very contaminated, with an average isi_violations_ratio of 11.6, and only 38 units retained after the threshold described above.

Based on this, I had the following questions :

• Is there something specific to Lupin that allows it to effectively filter out those noisy units ? I've thought that applying a filter based on SNR or some template metric on the output of SC2 could help, but I haven't been successful so far.
• Despite a seemingly more conservative sorting, the units that Lupin returns are still very contaminated. Is there something that can be done to help ? I've tried tuning some of the clustering parameters without success. I've also tried swapping iterative-hdbscan for iterative-isosplit in SC2 but the run then crashes at the clustering step.

Thanks !

[yger]

Thanks a lot for the feedback!

Lupin has indeed some built-in cleaning functions that were introduced, and that might be different in SC2. This takes place at two places:

• at the clustering level, templates that have too few spikes are discarded automatically (min_firing_rate params)
• after the clustering (in internal_sorters/lupin.py L360). here the idea is that any templates that would have a too small snr and/or be not properly centered will be discarded. The rationale is that fitting via template matching will be bad anyway for such very small/noisy templates, so we should not really bother keeping them at this stage.

However, I'm surprised that there is such a discrepancy, because the two sorters should be really similar. Lupin is built on SC2 with some optimizations, but the core should be similar, and the two should have the same cleaning steps, with more or less the same default params (for the cleaning). The only difference is the clustering, and I'm surprised hdbscan would produce so many garbage. Clearly, the one template per channel, with a very small SNR is an obvious artefact that should be either not found, either removed. You are using the master branch?

[b-grimaud]

Yes, I'm running on the main branch. I've tried upping the min_snr threshold, but I'm not exactly sure how much of an impact it has, because this is the distribution of SNRs as calculated by quality_metrics in the analyzer :

Considering the default threshold for min_snr is 5, I assume they are calculated differently.