Hemoglobin screening using a smartphone camera.


Ubicomp 2016 (Best Paper Award Top 1% of Submitted Papers): Edward Jay Wang, William Li, Doug Hawkins, Terry Gernsheimer, Colette Norby-Slycord, Shwetak Patel

EMBC 2017: Edward Jay Wang, William Li, Junyi Zhu, Rajneil Rana, Shwetak Patel

We present HemaApp, a smartphone application that noninvasively monitors blood hemoglobin concentration using the smartphone’s camera and various lighting sources. Hemoglobin measurement is a standard clinical tool commonly used for screening anemia and assessing a patient’s response to iron supplement treatments. A blood-screening tool based on unmodified smartphones has the advantage of being easily deployable and enables previously unconsidered treatment management options given the lack of such technology.  HemaApp can help community health workers in developing countries screen for iron-deficient anemia caused by malnutrition.  Beyond improved deployability in remote areas, the reuse of smartphones also aids people being treated for cases of anemia and need to monitor their condition at home. Often, these patients are treated with iron supplements and return to the hospital for a blood test every few weeks to ensure their treatment is effective. A smartphone hemoglobin test is convenient for at-home monitoring and does not require a patient to purchase a specialized blood testing device that costs hundreds to thousands of dollars. This allows both the patient and the doctor to track the effectiveness of these treatments much more easily and frequently.


HemaApp's fundamental operation is a measurement of the absorption property of the blood in the finger of the user. This is somewhat similar to the operations of a pulse oximeter. The major difference is that HemaApp focuses on measuring the concentration of hemoglobin versus plasma, whereas traditional pulse oximeters look at concentration differences between oxygenated and deoxygenated hemoglobin. After a user places their finger over the LED and camera, multiple light sources are cycled through. A video is recorded for each light source. The algorithm then extracts the R, G, and B time series waveform for each video by averaging each RGB channel independently for each frame. The algorithm then extracts machine learning features including peak and trough measurements for each light source, and also interaction terms between light sources. Finally, a SVM based regression is applied to estimate the hemoglobin concentration for the user.


HemaApp Iterations


The first prototype of HemaApp used a combination of built-in and external lighting supplement to enable the optical blood constituent measurement. Our first test revealed that our concept is promising, in a 31 patients ranging from 6 – 77 years of age, yielding a 0.82 rank order correlation with the gold standard blood test. In screening for anemia, HemaApp achieve a sensitivity and precision of 85.7% and 76.5%. Both the regression and classification performance compares favorably with our control, an FDA-approved noninvasive hemoglobin measurement device called the Masimo Pronto. This work was awarded the Best Paper Award at the 2016 Ubicomp. 


However, this first exploration still required an external light, making it less attractive of an option for the ultimate goal of getting hemoglobin monitoring to the masses. To achieve this vision, we further improved our camera spectrometry algorithm and machine learning to remove the dependence on an external light. In our second exploration of HemaApp, we demonstrate that the use of white LED can generate similar performance. As we continue to look towards improving our system, I want to reintroduce the use of infrared based spectral measurement. To do so, we have started exploring custom-kernel based solution to use the infrared proximity sensor as an IR pulse sensor. 

In our continued efforts to better understand hemoglobin measurements, I have expanded the capability of HemaApp through intensive characterizations. Some examples include testing against different populations and under extreme oxygen deprivation in a human-desaturation study.



One of the major challenges of this work is in the data collection. As the system is a data driven approach to the problem, training data is needed. In our first two iterations as described, we performed in-clinic and in-lab studies. These studies were conducted with the help of the doctors at University of Washington Medical Center and Seattle Children's Hospital. HemaApp was used to collect data from 31 participants from the leukemia in-patients of the two clinics and also general population at the university. 

As a continuation to this work, we are now looking at new avenues to significantly increase the sample pool for the system to learn from. This has required significant engineering to encapsulate all the workings of HemaApp into a deployable app that can be used by more than just the researcher. In the summer of 2017, the team began working with several NGOs in Peru to explore the usability of the system in the field. I had the opportunity to engage with multiple clinics in the Lima region through the help of Christopher Westgard (currently with Elementos, Peru). We also performed a three day joint anemia screening campaign and data collection in the Amazonian community near Iquitos, Peru. Through this work, we are now looking for ways to improve the data collection system and algorithm of our system in order to further our data collection efforts with new partners from the Fred Hutchinson Research Center, Bloodworks, and Elementos, Peru .