Tutorials (Python)

Run the tutorial in Colab:

This tutorial walks you through how to use the pyDANT package to track neurons across sessions. It is designed to help you prepare your data and run the code effectively. Before starting, make sure pyDANT is installed correctly. If you have not installed pyDANT yet, please refer to the Installation section.

Prepare the data

An example dataset is available here. You can download it and follow the steps below to run the full example. You can also use your own dataset by following the same workflow.

To use pyDANT, organize your data in a folder with the following structure:

data_folder
├── channel_locations.npy
├── waveform_all.npy
├── session_index.npy
├── peth.npy (optional)
└── spike_times/
    ├── Unit0.npy
    ├── Unit1.npy
    ├── Unit2.npy
    ├── ...
    └── UnitN.npy

The data files should follow the formats below:

Filename	Shape	Explanation
`session_index.npy`	(n_unit,)	indicates the session. It should start from 1 (for compatibility with MATLAB) and be continuous without any gaps.
`waveform_all.npy`	(n_unit, n_channel, n_sample)	the mean waveform of each unit in μV. All units must share the same set of channels
`channel_locations.npy`	(n_channel, 2)	x and y coordinates of each channel in μm. The y coordinate typically represents depth
`peth.npy`	(n_unit, n_point)	optional, peri-event time histogram for each unit
`spike_times/UnitX.npy`	(n_spike,)	spike times in milliseconds

Crucially, the waveforms used in this analysis must not be whitened, unlike the waveforms processed by Kilosort. Avoid direct use of waveforms from temp_wh.dat and refrain from using whitening_mat_inv.npy or whitening_mat.npy from Kilosort2.5 / Kilosort3 to “unwhiten” data. These matrices do not correspond to Kilosort’s original whitening process (see this issue).

We recommend analyzing data from different brain regions (e.g., cortex and striatum) separately, as they may exhibit distinct drifts and neuronal properties. Please generate a separate data folder for each brain region.

Copy settings.json and mainDANT.py from the pyDANT package into your data folder. A typical layout is shown below:

data_folder
├── settings.json
├── mainDANT.py
├── channel_locations.npy
├── waveform_all.npy
├── session_index.npy
├── peth.npy (optional)
└── spike_times/
    ├── Unit0.npy
    ├── Unit1.npy
    ├── Unit2.npy
    ├── ...
    └── UnitN.npy

Edit the settings

To run pyDANT, edit the settings.json file in your data folder first. At a minimum, you should specify the following fields:

{
    "path_to_data": ".", // path to the input data folder
    "output_folder": ".\\DANT_Output", // output folder
}

If you do not want to use the PETH feature, remove it from both the motionEstimation and clustering sections. After editing, the settings can look like this:

// parameters for motion estimation
"motionEstimation":{
    "features": [
        ["Waveform", "AutoCorr"],
        ["Waveform", "AutoCorr"]
    ] // features used for motion estimation each iteration. Choose from "Waveform", "AutoCorr", "ISI", "PETH"
},

and

// parameters for clustering
"clustering":{
    "max_distance": 100, // um. Unit pairs with distance larger than this value in Y direction will be considered as different clusters
    "features": ["Waveform", "AutoCorr"], // features used for final clustering. Choose from "Waveform", "AutoCorr", "ISI", "PETH"
    "n_iter": 10 // number of iterations for the clustering algorithm
},

Also edit mainDANT.py to specify the path to the settings file:

path_settings = r'./settings.json' # Path to your settings.json file

To learn more about the settings, please refer to the Change default settings section. Careful tuning can help improve tracking results.

Run the code

Run mainDANT.py in your Python environment from the terminal or command prompt:

python mainDANT.py

The tracking results should appear in the output folder specified in settings.json. A typical output layout looks like this:

data_folder
├── settings.json
├── mainDANT.py
├── channel_locations.npy
├── waveform_all.npy
├── session_index.npy
├── peth.npy (optional)
├── spike_times/
└── DANT_Output/
    ├── IdxCluster.npy
    ├── ClusterMatrix.npy
    ├── SimilarityMatrix.npy
    ├── ...
    └── Figures/

Understand the output

Along with several intermediate files, the main output is stored in the DANT_Output folder. The most important files are listed below:

Field name	Shape	Explanation
`IdxCluster.npy`	(n_unit,)	cluster index for each unit.
`ClusterMatrix.npy`	(n_unit x n_unit)	cluster assignment matrix. `ClusterMatrix(i,j) = 1` means unit `i` and `j` are in the same cluster.
`MatchedPairs`	(n_pairs x 2)	unit index for all matched pairs.
`SimilarityMatrix`	(n_unit x n_unit)	weighted sum of the similarity between each pair of units.

The most important file is IdxCluster.npy, which assigns a unique cluster ID to each unit (-1 for non-matched units). You can use it to extract matched units across sessions. To learn more about the output, please refer to the Input and Output section.

Tracking is complete. You can now move on to cross-session analysis with the tracked neurons.