Add catalogue-based probe construction for OpenEphys#406
Add catalogue-based probe construction for OpenEphys#406alejoe91 merged 32 commits intoSpikeInterface:mainfrom
Conversation
for more information, see https://pre-commit.ci
…' into separate_open_ephys
for more information, see https://pre-commit.ci
read_openephys_binary and catalogue-based probe construction for OpenEphys… adc check in tests, plugin_channe_keys -> settings_channel_keys
for more information, see https://pre-commit.ci
|
@h-mayorquin I reviewed this carefully and made som last changes.
|
|
@jsiegle confirmed that electrode_index is left->right / bottom->top / left shank->right shank |
…erface into separate_open_ephys
Thanks
This LGTM. I added Josh's comment on the code as well. |
for more information, see https://pre-commit.ci
…' into separate_open_ephys
|
@h-mayorquin I further simplified the |
|
@h-mayorquin with the last commit I added a re-ordering of the ADC annotations ( Exploring this a bit more, it seems that the channel order in the oebin file corresponds 1-1 to this Maybe an even simpler solution would be to use that field to slice and re-order contacts? |
|
Yes, we agreed in meeting that #409 makes more sense. A solution that uses less files is better. |
This adds
read_openephys_binary(settings_file, oebin_file, stream_name), a new reader for Open Ephys binary format recordings that uses bothsettings.xml(probe geometry) andstructure.oebin(binary data layout) to build probes. It also refactorsread_openephysto use catalogue-based probe construction, fixing contact_id consistency with SpikeGLX. See #405 for the broader discussion on separating probe construction from wiring.Both readers now follow the same pattern as
read_spikeglx(introduced in PR #383): build the full probe from the catalogue viabuild_neuropixels_probe, then combine it with data from the source files to determine which electrodes are active and how they map to the data. For SpikeGLX, the source file is the.metafile (IMRO table). For Open Ephys, thesettings.xmlprovides electrode positions (viaELECTRODE_XPOS/ELECTRODE_YPOS) or electrode indices (viaSELECTED_ELECTRODES), which are matched against the catalogue probe geometry to select the active contacts.read_openephys_binaryadditionally reads thestructure.oebinto setdevice_channel_indices. As proposed in SpikeInterface/spikeinterface#4435, the oebin contains per-channelelectrode_indexmetadata (added in neuropixels-pxi plugin v0.5.0, January 2023) that maps each binary file column to its electrode index. When available and non-zero,read_openephys_binaryuses this for wiring. For older recordings whereelectrode_indexis missing or all zeros, it falls back to identity wiring, which is correct because the binary.datfile is always written in channel-number order (as confirmed by Josh Siegle). The existingread_openephysis kept for backward compatibility and always sets identity wiring since it has no oebin.Because both readers now derive
contact_idsfrom the catalogue probe, this also fixes #394 (OpenEphyscontact_idsinconsistent with SpikeGLX). The previous code reverse-engineered electrode IDs from positions, which produced different IDs than SpikeGLX for probes where OpenEphys positions don't exactly match the canonical geometry. @chrishalcrow identified the affected datasets in the existing test data (OE_Neuropix-PXI-NP-Ultra,OE_6.7_enabled_disabled_Neuropix-PXI,OE_Neuropix-PXI-QuadBase). A new consistency test verifies that for these and other datasets, thecontact_idsfromread_openephysare a subset of the catalogue probe'scontact_ids.Other changes:
read_openephys_binaryis annotated withplugin_channel_key(the XML CHANNELS key, e.g. "CH0", "CH29"), distinct from neo'schannel_namewhich comes from the oebin and uses different naming conventions.settings.xml+structure.oebinfiles.