Researchers from Penn State and Johns Hopkins, backed by a $100,000 grant from the Bill and Melinda Gates Foundation, are developing technology that could read peoples' vital signs from up to four feet away using only a cellphone camera. 

The team has published a detailed account of the work in the open access journal Biomedical Optics Express. But the funding from the foundation will allow the team to take trips to India and Sierra Leone to test the technology in low-resource areas. The first of these trips is scheduled for this month, with another set for March 2019.

"Our mobile-based application seeks to expand beyond a 'wellness app' classification by the FDA, to an FDA-approved tool that can be used by patients and healthcare officials for measuring vitals at a distance and in varying environments and populations all across the globe," Conrad Tucker, leader of the study and associate professor of engineering design and industrial engineering at Penn State, said in a statement. "This pilot grant is a first step towards this goal, as it will enable the team to evaluate the technology that we've developed in real-world settings involving hardware, environmental and societal constraints."

Specifically, the technology, called VideoVitals, uses a combination of photo-plethysmography and computer vision to detect heart rate based on minute skin color changes. The computer vision helps to account for differences in environmental lighting, skin pigmentation, and camera quality, and to complete a measurement even when the subject is moving.

Why it matters

Being able to measure vital signs reliably and accurately from a distance of four feet — with a low-cost, readily available device — could be helpful in responding to epidemics of highly contagious diseases like Ebola or tuberculosis. 

VideoVitals is being developed specifically with these kinds of use cases in mind. Rather than relying on top of the line smartphone cameras, for instance, it's been tested on the Xiaomi Redmi 6 and the Samsung Galaxy J7 Prime, cellphones that are typically used in countries like India and Sierra Leone, where the team will travel.

"It's one thing to demonstrate the feasibility of an algorithm in a controlled environment in a research lab. It's another thing for technology to function as intended in the field, where researchers have less control of the human and environmental factors," Tucker said in a statement. "This will be an excellent opportunity to integrate user-centered design, concepts that form the foundation for many of the engineering design courses that are taught in SEDTAPP here at Penn State."

What's the trend

The underlying technology, photo-plethysmography, is the same technology the Apple Watch uses to detect heart rate. Using it as a distance isn't completely novel. 

In fact, when Fujitsu announced a technology for reading heart rate at a distance based on facial coloration changing in 2013, MobiHealthNews rounded up six previous attempts, including one from Phillips. MIT Researchers were able to set up the same thing with a Microsoft Kinect.

A few things that are novel here are the fact that it is being designed to work with a variety of low-cost cameras, and the addition of computer vision for facial feature detection could improve the accuracy significantly.