Real-time markerless video tracking of body parts in mice using deep neural networks

Citations Over Time

Abstract

ABSTRACT Markerless and accurate tracking of mouse movement is of interest to many biomedical, pharmaceutical, and behavioral science applications. The additional capability of tracking body parts in real-time with minimal latency opens up the possibility of manipulating motor feedback, allowing detailed explorations of the neural basis for behavioral control. Here we describe a system capable of tracking specific movements in mice at a frame rate of 30.3 Hz. To achieve these results, we adapt DeepLabCut – a robust movement-tracking deep neural network framework – for real-time tracking of body movements in mice. We estimate paw movements of mice in real time and demonstrate the concept of movement-triggered optogenetic stimulation by flashing a USB-CGPIO controlled LED that is triggered when real time analysis of movement exceeds a pre-set threshold. The mean time delay between movement initiation and LED flash was 93.44 ms, a latency sufficient for applying behaviorally-triggered feedback. This manuscript presents the rationale and details of the algorithms employed and shows implementation of the system using behaving mice. This system lays the groundwork for a behavior-triggered ‘closed loop’ brain-machine interface with optogenetic stimulation of specific brain regions for feedback.

Related Papers

• → Functional ultrasound imaging of the spreading activity following optogenetic stimulation of the rat visual cortex(2021)16 cited
• → Application of Optogenetics in Gene Therapy(2018)4 cited
• Optogenetic cardiac pacing and its applications(2021)
• → Optogenetic Cardiac Pacing and Its Applications(2021)
• → The effect of optogenetic activation of astrocytes on the hippocampal neurons activity(2021)