Food Logging

Trade-off between Automation and Accuracy in Mobile Photo Recognition Food Logging

2017.06 | Chinese CHI 2017


Food logging can help users understand their food choices and encourage healthier eating habits. However, current apps still pose many usability challenges, including tedious manual text entry of food names. Recently, advances in computer vision and deep learning are enabling automatic food recognition for instant and convenient logging. However, as a nascent technology, this suffers from inaccuracy, which may lead to poor adoption or misuse. We investigated the trade-off between accuracy and convenience of automatic photo recognition in comparison to manual search logging. Specifically, we have developed a mobile app prototype that integrates both photo recognition and search logging capabilities, and conducted formative investigations on the usability and usage of automatic photo recognition in food logging in a series of studies: online requirements survey, usability lab study, and 1-week field trial in an Asian country. Participants were interested in convenient, automatic photo logging, but dominantly used manual search logging due to a lack of data coverage and accuracy. We identified reasons for poor accuracy and highlight complications in using inaccurate automatic photo logging. We further discuss opportunities for design and technology to address these challenges


user experience, field study, food logging, food journals, mobile applications, image recognition


Proceedings of the Fifth International Symposium of Chinese CHI

Project Info



Brian Y. Lim, Xinni Chng, Shengdong Zhao