Researchers generate 3D images using just one photon per pixel

Lisa Zynga,

Every time you take a photograph, your camera detects more than a billion photons. For a basic one-megapixel camera, that's more than 1,000 photons per pixel. Now in a new study, researchers have developed an algorithm that is so efficient that it can generate high-quality 3D images using a single-photon camera that detects just one signal photon per pixel.

The researchers, led by Jeffrey Shapiro, a professor of electrical engineering and computer science at the Massachusetts Institute of Technology (MIT), along with coauthors at MIT, Politecnico di Milano, and Boston University, have published a paper on the new photon-efficient approach to imaging with a single-photon camera in a recent issue of Nature Communications.


"Our work shows that we can use these new photon-counting cameras at much lower fluxes and much higher efficiencies than previously thought," Shapiro told

Compared to other camera-based 3D imaging techniques that have recently been developed, the new framework has the highest photon efficiency to date, resulting in a visibly better reconstruction accuracy and an order of magnitude better depth resolution.

RELATED Scared out of its skin, ancient insect escaped sticky doom

To demonstrate how the new single-photon imaging algorithm works in low- environments, the researchers illuminated a scene of interest (such as a mannequin and sunflower) with a pulsed laser that emits low-light pulses every 50 nanoseconds. A light diffuser spatially spreads out the pulses so that they flood the entire scene.


A single-photon camera then captures the light reflected by the illuminated objects, along with some background light that the researchers added to simulate a realistic environment. The key element of the camera is a single-photon avalanche diode (SPAD) array, which detects incoming photons and records their precise times of arrival.

In order to reconstruct a scene from a very small number of detected photons, the new algorithm accounts for the fact that a scene's features are statistically correlated. Whereas most camera systems treat each pixel independently, the new algorithm recognizes that neighboring pixels often have strong spatial correlations, which are punctuated by sharp boundaries that separate two different features (such as the mannequin's shirt and the sunflower in the test scene). By solving a mathematical problem of convex optimization, the new algorithm exploits these correlations to generate high-resolution 3D images using very few photons.

RELATED Massive 3D map of faraway galaxies to aid search for dark energy

The images in this experiment have a final resolution of 384 x 384 pixels, and an average of one signal photon and one background light photon are detected per pixel.

Another advantage of the new single-photon camera is that images can be captured very quickly, making it useful for applications that require fast and accurate imaging using extremely small amounts of light.


By requiring vastly fewer photons than conventional cameras, single-photon cameras have potential applications in low-light conditions, such as biological imaging, astronomy, and providing 3D vision for self-navigating advanced robotic systems, such as unmanned aerial vehicles and exploration rovers.

RELATED Researchers take a closer look at the 'ouzo effect'

In the future, the researchers plan to pursue various applications by collaborating with researchers with different specialties. Working with MIT's chemistry department, for example, they have begun to apply their photon-efficient approach to fluorescence imaging, an imaging modality that has many applications in biology.

They are also pushing the technology to its limits under DARPA sponsorship as part of DARPA's Revolutionary Enhancement of Visibility by Exploiting Active Light-fields (REVEAL) program. REVEAL's goal is to use all the information available in the light field to see into occluded spaces without using mirrors.

"We're going to use laser light and single-photon detection to see around a corner," Shapiro explained. "Our approach will rely on diffuse reflections from a wall. Because the light returned from the hidden objects will be very weak, photon efficiency is essential."

RELATED Bacteria work together to avoid the pitfalls of aging

More information: Dongeek Shin et al. "Photon-efficient imaging with a single-photon camera." Nature Communications. DOI: 10.1038/ncomms12046

See more details at the researchers' project page.

Latest Headlines