Google Research Combines AR And A Microscope For Cancer Detection

Google has used its expertise in deep learning and augmented reality to create something that can make a real difference in the medical community. Google has created an augmented reality microscope (ARM) that aids in the detection of cancer. The microscope is a prototype right now and Google thinks that it can help accelerate adoption of deep learning tools for pathologists around the world.

The AR microscope is a modified light microscope that enables real-time image analysis and presentations of results of machine learning algorithms directly in the field of view for the medical professional. One of the most important parts of the platform for the ARM is that the tech needed can be retrofitted into existing light microscopes that are in hospitals and clinics around the world already.

Google also notes that the ARM system uses low-cost and readily available components and doesn't need whole slide digital versions of tissue being analyzed to operate. The computation components and deep learning models like those built on TensorFlow allow a range of pre-trained models to run on the ARM platform.

Google says that the user will view the sample through an eyepiece on the ARM, just as they do on a traditional microscope. The machine learning algorithm projects its output into the optical path of the user in real-time. The projection of the digital information is superimposed on the image of the specimen to help the viewer in localizing or quantifying the features of interest on the slide.

The presentation implementation runs at 10 frames per second allowing the model output to seamlessly update as the user scans the tissue by moving the slide or changing magnification. ARM can give the user a variety of visual feedback including outlining detected tumor regions with a green contour or placing arrows and text on the field of view. Google thinks that ARM has the potential for a significant impact on global health and is working on accelerating the adoption of the tech.

SOURCE: Google