Breakthrough Deep Learning Model Enhances Handheld 3D Medical Imaging

When paired with photoacoustic imaging in photoacoustic and ultrasound imaging (PAUS) systems, its capabilities are further enhanced. However, traditional ultrasound imaging only captures limited two-dimensional areas, restricting understanding of complex three-dimensional anatomical structures. While some transducers provide 3D imaging, they are expensive and have narrow fields of view. An alternative, the 3D freehand technique, stitches together multiple 2D scans but depends on external sensors to track transducer motion. These sensors are often bulky, costly, and inaccurate, limiting the technique’s clinical utility. Now, researchers have developed a new artificial intelligence (AI)-powered approach that enables real-time 3D reconstruction of ultrasound images without relying on external motion sensors.

The deep learning model, named MoGLo-Net, has been developed by researchers from the Pusan National University (Busan, South Korea) and automatically tracks the motion of the ultrasound transducer using only tissue speckle data from B-mode ultrasound images. The system includes two main components: a ResNet-based encoder and an LSTM-based motion estimator. The encoder applies correlation operations to detect motion patterns from consecutive images and incorporates a novel self-attention mechanism to highlight relevant local features informed by global context. These refined features are then passed to the LSTM estimator, which tracks motion over time with long-term memory, ensuring accurate 3D spatial mapping. Customized loss functions were used to improve model precision. This approach eliminates the need for external motion sensors and allows seamless 3D reconstruction from standard 2D ultrasound scans.

The researchers validated MoGLo-Net using both proprietary and publicly available datasets across varied conditions and demonstrated that it outperformed current state-of-the-art models in all tested metrics. In a groundbreaking achievement, the team also integrated ultrasound and photoacoustic data using MoGLo-Net to generate 3D images of blood vessels—an industry first. The results, published in IEEE Transactions on Medical Imaging, indicate that this model significantly improves the quality and accessibility of 3D imaging, potentially transforming diagnostic imaging and interventional procedures by offering accurate, efficient, and cost-effective solutions. The team plans to continue refining the model and exploring broader clinical applications to further democratize access to advanced ultrasound imaging.

“Our model holds immense clinical potential in diagnostic imaging and related interventions,” said Prof. MinWoo Kim, Associate Professor, lead researcher of the study. "By offering clear 3D images of various bodily structures, this technology can help make medical procedures safer and more effective. Importantly, by removing the need for bulky sensors, this technology democratizes the use of ultrasound, making it accessible to clinics where specialists may not be available."

Related Links:
Pusan National University