Acceptability, Usability, and Quality of a Personalized Daily Meal Plan Recommender System: The Case of Virtual Dietitian

Introduction

The domain of nutrition science across the entire research spectrum, from the basic science to clinically relevant dietary knowledge, has been immersed to advance our understanding of variability in individual metabolic responses to nutrients, food and diet components that could lead to personalization of nutrition interventions that are translatable in public health policies [1]. From nutrigenomics to deep phenotyping, many experts attempt to discover inter-variability (e.g., phenotypic, medical, nutritional [2]) in order to manufacture a blueprint for tailored dietary recommendations. In addition, knowledge from this inter-variability could be used for creating dynamic nutritional interventions [3] with consideration to fluctuating parameters and uncertain impacts in an individual's internal and external environment, i.e., interaction between nutrients and biological processes [4], and the differences in nutrition response [5] to a specific design of a dietary intervention. For nutrition science, there are still so many things to learn when it comes to the personalization of dietary interventions. Still, this does not stop health experts and nutrition researchers in attempting to measure its impact on an individual level. With this objective, various nutrition systems were proposed.

Despite the large number of proposed food recommender systems (e.g., Plan-Cook-Eat [6] – the initial version of VD) that aims to incorporate the concept of personalized nutrition, existing artifacts remain preliminary in the nutritional health context largely due to lack of integrated nutrition knowledge [7]. Similar findings emerged during the preliminary analysis of VD [8], where nutrition knowledge was regarded as a vital and necessary component to make people a smarter decision-maker when it comes to dietary choices. With considerations to these shortcomings, the first version of VD [9] was created in compliance with these findings. In this version of VD, the Nutrition Care Process and Model (NCPM) (see Figure 1) was used as a grounding framework for the overall systems architecture in conjunction with forward chaining algorithm for filtering of recipes according to users' preferences and restrictions. Nevertheless, the beta evaluation for this version of VD divulged important modifications that are needed to accomplish and/or integrate on the nutrition system as per the feedback from experts (information technology practitioners and registered dietitians). To comply with the requirements, another sprint of development was accomplished. This study reports the systems modification from beta evaluation as well as the assessment of the revised VD by non-expert users for the live version in three factors: acceptability, usability, and quality. Finally, this paper is the last part of a two-year study covering the development of a personalized nutrition system.

Background of the Study

The Emerging Field of Personalized Nutrition

The inter-individual variability, whether on a micro (e.g., microbiota, microbiome, metabolome, phenotype, genotype, etc.) or macro level (e.g., gender, behavior, ethnicity, culture, age, environment etc.), and knowledge furtherance of human genome sequencing, the field of nutrition science steered to a prevalent belief that personalized nutrition was imminent to adopt. This is a mere effort to shadow the footstep of medical science in pioneering its recent precision medicine [10, 11, 12, 13] – a medical model that customizes healthcare instead of a one- medicine-fits-all approach. Meanwhile, nutrition research is presently entering the subfield of personalized nutrition [14, 15, 16], where dietetics professionals are using it as an approach to aid individuals in formulating unique dietary interventions and guidelines. As a result, the nutritional recommendations being proposed are beneficial to everyone regardless of their differential responses of dietary intake variability, as largely echoed in biomarker values on nutrigenomics research [17]. Conversely, the target populations for personalized nutrition range from diseased to healthy individuals with consideration to corresponding groups (children, elderly, pregnant women, athletes, etc.). The promise of personalized nutrition is that it resolutely takes into account the specific characteristics of its recipients, which cannot be accomplished using one-size-fits-all approaches that are applied traditionally.

Challenges in Personalized Nutrition Care

While in the course of advancing personalized nutrition care (see Figure 2), numerous challenges are encountered across the healthcare industry [18]. As a new discipline, there is a reluctance to embrace the concept [19], mainly due to the apprehension of being unable to manage the complexity and overwhelming quantity of biological and related information all throughout the NCPM. In the International Dietetics and Nutrition Terminology (IDNT) alone, there are more than 60 nutrition diagnoses that describe nutrition problems, more than 170 nutrition monitoring and evaluation parameters to measure outcomes, and more than seventy vocabularies just to describe nutrition interventions [20]. The main application of IDNT is to document nutrition care in the medical record as a complementary tool of NCPM. Moreover, there are 301 terms under assessment, 60 terms in diagnosis, 74 terms in intervention, and 277 in monitoring and evaluation. As such, this new discipline of personalized nutrition can benefit from building a n-of-1 computational infrastructure that takes into account the systems-based approach for the recognition of components involved in the optimal health and human well-being. As a result, this would prescribe the collection of data from every person every day or periodically over a duration of time [21]. Defeating these challenges is truly a formidable prospect and winning this battle is burdensome, more so if those in the field act individually with no support. Therefore, there is a great necessity for a technological platform through healthcare information technology that could assists dietetics professionals in an extremely complex and crowded arena of personalized nutrition care process. Such nutrition platform could promote evidence-based research by extending access to the data derived from successful nutrition strategies [22].

Virtual Dietitian: A Personalized Nutrition System

In response to the existing inadequate nutrition problems and challenges of personalized nutrition, VD was proposed [8] and developed [9]. VD is a nutrition system grounded on NCPM that aims to personalize nutrition care through the recommendation of tailored meal plans. As shown on Figure 3, VD has input requirements: knowledge, recommendation, and nutrition care. These requirements serve as the source of core knowledge (e.g., nutrition science research, nutritional guides, and domain experts) and personalization variables as entered by users (e.g., preferences, restrictions, and personal goals). While the knowledge requirements function as VD's core knowledge and reasoning, it is also used to complement other requirements, such as the nutrition care requirements processed on the NCPM and recommendation requirements processed using a recommender technique and a rule-based reasoning. All information, reasoning, and knowledge are stored in a knowledge base, and the rules and logic formed from this complex structured and unstructured intelligence are stored on the inference engine, which serve as the core mechanisms for personalizing meal plan recommendations and dietary interventions under a unified system called VD.

Methodology

The core method of this study depends on the utilization of a mixed-method research design where an exploration of both quantitative and qualitative feedback was achieved to unearth deeper comprehension of a phenomenon. In nutrition and dietetics research, quantitative and qualitative research are collectively used so weaknesses in one approach could be compensated by strengths in the other [23]. Based on the characteristics of data emerged from both designs, facts and truths possess objective elements as far as quantitative part is concerned while the qualitative part borders on the subjective features as narrated by perceptions and meanings of people. Hence, the combination of both approaches leads to a more complete understanding of a phenomenon.

Study Design

With respect to the mixed-method approach, this study followed a convergent parallel research design, where both qualitative and quantitative elements are concurrently done in the same phase of the research process. As described [24], this design treats both methods equally, analyzes the two components independently, and interprets results together. For this study, quantitative statistical results and qualitative findings were corroborated together to support one another as well as compare the outcomes to better understand how it achieve such results. It is important to note that the artifact being evaluated is the revised VD, which is the version after it was modified according to experts' feedback (see Table 1). Lastly, a probability sampling with a 95% confidence level and a margin of error of ± 5% was adopted in this study.

Research Instruments

Quantitative and qualitative data were collected using a survey containing a demographic questionnaire, open-ended questions, Acceptability Questionnaire (AQ) [25], Nutrition App Quality Evaluation (AQEL) [26], and System Usability Scale (SUS) [27]. The open-ended questions include "What is your general impression of the website application?" and "In your opinion, will Virtual Dietitian help you achieve a good nutritional status? How so?" to name a few. For AQ, the content of the instrument was based on the questionnaire used in the evaluation of myfood24 (Measure Your Food on One Day) among British adolescents. AQ was used to assess the acceptability of VD. Meanwhile, AQEL was created for evaluating nutrition apps' qualities, which was the same tool used in the preliminary analysis of VD [8]. It has 25 items divided into five factors such as skill development, behavior change, knowledge acquisition, app function, and purpose. AQEL was used to measure the quality of VD. Finally, SUS was developed in response to the insufficiency of objective usability (effectiveness and efficiency) as primary metrics of evaluating user satisfaction with systems [28]. Originally, it has 10 alternating (positive and negative) items. However, it was converted into all positive SUS for this study to prevent usual mistakes and misinterpretations [29]. Moreover, the adjective rating scale [30] was adopted for ease of score interpretation. SUS was used to assess the usability of VD.

Data Collection and Analysis

Due to restrictions posed by COVID-19 pandemic as of the time the evaluation was conducted, data collection was performed via online channels (e.g., Google Forms, Google Meet). Quantitative data were described and analyzed using the statistical analysis program SPSS. Descriptive analysis techniques were utilized for the summarization of scores and users' profile information, with the calculation of mean and standard deviation in variables with parametric distribution. Using its own scoring system, on the other hand, SUS was calculated by multiplying each score into two [27]. As stated on the guide, scores should not be treated as percentages. To identify if there was a significant difference between scores (usability, quality, and acceptability) and variables related to users' profile (monthly income households, living condition, physical activity, nutritional status, and average meals per day), Kruskal-Wallis H test was applied. Although this was just a supplemental analysis, it was included in the study to show a different perspective on how VD was evaluated by users. Further, qualitative data underwent document analysis to corroborate quantitative results by extracting data based on the constructs given in a specific instrument.

Prior to the start of data collection, provisions to protect the participants' privacy and to maintain data confidentiality were incorporated on all of the instruments. Although there was no health-related personal information being collected (which is typically subjected to legal/ethical regulations like HIPAA), the inclusion of such provisions aims to protect the anonymity of participants. Further, on reporting qualitative feedback, each participant was assigned with a random yet identifiable number (e.g., P2 means participant number 2) as part of the promise of anonymizing the human subjects. A checkbox is also present at the end of online instruments, which signifies the submission of an informed consent. The complete details of this informed consent were linked beside a checkbox. Finally, the ethical principles as outlined by the university was used in conjunction with the Declaration of Helsinki, whenever and whatever is appropriate from these two. This includes promoting the rights of all participants involved in the study, as described above, and submission of a research protocol for review and approval of the research ethics committee of the university, to name a few.

Results and Discussion

The main goal of this study was to assess the revised VD following another sprint of systems development as required by experts [8]. A total of 397 non-expert users participated in the evaluation of VD (see Table 2). Majority of the users were 18-25 years old (n = 261, 65.74%), living with family (n = 278, 70.03%), have a household income of PHP 38,081 – PHP 66,640 per month (n = 200, 50.38%), sedentary (n = 135, 34.01%), overweight (n = 204, 51.39%), and consume an average of four meals a day (n = 168, 42.32%).

Acceptability

Based from the assessment, 87% of the respondents like the design and layout of VD. This is considered a vital result since design and layout is a factor that determines quality of a website [31]. Furthermore, in a health-related website, the quality influences the intention to use, trust, and perceived usefulness by its users [32]. One participant even noted that the "website displays recipes in an enticing way which makes me want to eat the food". On the other hand, 96% of the respondents found the food and recipe databases every useful as it "contains a lot of helpful information such as nutrition data for food and ingredients for recipes". Similar feedback was received from the Phase 1 evaluation [8] by healthcare professionals. Moreover, 69% of the respondents agreed that the meal plans generated by VD were based on their preferences. However, the database must be populated with more recipes to ensure that there are more varieties in the meal plan. As users noted, there are "sometimes same recipes in one meal plan" or the "meal plan seems like same after a few days". Meanwhile, 95% of the respondents strongly agreed that they can easily see and track their Body Mass Index (BMI) and weight history in the app. This is due to the strategic placement of the BMI and weight history widget in the user dashboard. On the other hand, collecting recipes based on users' preferred grouping was found very useful by 75% of the respondents. Instead of a simple "love" button or bookmark, users can create their own collections, add recipes to that collection, and see those recipes in their own dedicated webpage. Just like in Phase 1, grocery list generator was part of the assessment in Phase 2. In this evaluation, 77% of the respondents found it very useful as well. With this feature, they can "easily print it out and go to the supermarket to buy foods then follow the steps on the website". The same number of respondents have strongly agreed in the ease of use of the settings page where they can change their preferences and restrictions. Lastly, 95% of the participants have strongly agreed that the usage of VD to achieve their weight and fitness goals are acceptable.

Quality

In terms of behavior change, VD received a quality score of 4.02 ± 0.21. Some comments from users include:

I could eat healthy meals from now on because of the good recommendations from this app [P22]

I like to eat foods especially snacks with a lot of protein because I need it in sports and this will help me in that scenario [P44]

For the knowledge factor, VD received a quality score of 4.39 ± 0.19. Some comments from users include:

Learning the amount of nutrients in every meal is very helpful for someone who is not expert [P3]

This is something that should be teach in a school so that students will be healthy all the time [P45]

My first impression in the app is about the nutrition data for each meal which I think is a cool feature to add to teach people about what they are eating and if it is healthy or not [P67]

For the app function factor, VD received a quality score of 4.89 ± 0.34. Some comments from users include:

As a student of Information Technology, I like how the website is designed and developed and it gives different system features perfect for people who are concerned with their health. [P32]

The website loads very fast, the layout is clean, the design looks very professional, and so I like Virtual Dietitian. Plus it promotes health [P53]

For skill development, VD received a quality score of 3.96 ± 0.40. Some comments from users include:

My favorite food to cook is sinigang na baboy and because of this app I realize that there are different ways on how to cook it and the nutrition fact every variation of the meal has [P19]

I do not know anything about nutrition or how to select meals with nutritious ingredients and this website is for sure will give me the skills to be health-conscious in terms of foods [P56]

For the app function factor, VD received a quality score of 4.89 ± 0.34. Some comments from users include:

I believe this website will help many people [P3]

My impression about this website is that it is very useful especially for people who need healthy meal plans for their daily living [P66]

Usability

During phase 1 evaluation, VD received a usability score of 83.4 as rated by experts using SUS. For Phase 2, usability score of VD increased to 87.1 after performing all necessary modifications as emerged from the previous evaluation. For non-expert users, they think that they would like to use VD (79, Good) as a personalized nutrition system ([P132]: the app makes nutrition knowledge accessible for everyone so I would like to use it). In addition, they also found its usage to be very simple (81, Excellent) and easy to use (80, Good) as user requirements were carefully considered ([P212]: VD is easy to use because all nutrition-related things you need are already given). Various functions were also well-integrated (86, Excellent), which was noted during the previous phase ([P31]: the system is easy to use because all the modules are easily found and connected to one another). Consequently, they think that they could use VD without the support of a technical person (92, Excellent) and without having to learn something new (89, Excellent). Some participants also noted that "I am not a techy person but I easily learned how to use it [P161]", and that "I feel like I know how to use it the first time I see it [P212]". Further, they would imagine that most people would learn to use VD very quickly (91, Excellent), that it is very intuitive (92, Excellent) as well as consistent (89, Excellent). Lastly, users are very confident in using VD (92, Excellent) ([P22]: I feel like I can easily use VD even if I am not that good in technology like apps).

Supplemental Evaluation

A supplemental evaluation was performed to determine if there are statistically significant differences between the acceptability and app quality scores in Phase 2 and variables linked to participants' profile such as average daily meals, living condition, current physical activity, nutritional status, and monthly household income. The Kruskal-Wallis H Test revealed there were significant differences between all the assessment scores and variables linked to users' profile (p < 0.05). This means that each participant, regardless, whether they are overweight or underweight, or lives with family or alone, has their own exclusive view and assessment of VD. Subsequently, pairwise comparisons were performed as well using Dunn's [33] procedure and Bonferroni correction was made for multiple comparisons (see Figure 5). Although this series of analysis were not part of the objective of the study, it still presents an avenue for future researchers to explore it even more, on how variables linked to evaluators affect their views and opinions about personalized nutrition systems. By doing so, healthcare professionals, website and mobile app developers and other experts can have a strong basis on how to design and develop future nutrition applications. Lastly, it could result to a more personalized nutrition system in terms of design and architecture by considering user profiles.

Conclusion and Recommendations

This study focused on the systems modification and final testing of VD, which is a nutrition intervention tool as a way of responding to inadequate nutrition problems. Final version of VD was subjected to an evaluation by non-expert users in terms of acceptability, usability, and quality. It is essential to note that although this is the last part of a two-year study for the development of a personalized nutrition system, there are still enhancements to be recommended for future works. For instance, one of the most common concerns is the inclusion of diseases as a variable for generating meal plans. With this, users have an assurance that meals being recommended by a nutrition system are not detrimental for them. Although not feasible as of this moment, another future work that may be included is the inclusion of dynamic pricing for each food for the calculation of budget to ensure that a meal recommender system can filter the meals based on the budget constraints of users. This requires a strong coordination between public and private sectors to ensure that the price of each food is always updated in the system in accordance with the market. Future studies may also explore the application of computer vision on how to use to determine the physical condition of patients. As emerged during knowledge acquisition, nutrition-focused physical examination is the traditional method often used by dietitians in the clinical settings to determine the inaccuracies in anthropometric measurements. Lastly, if the core idea is to align the personalized nutrition system with NCP, computer applications dedicated for collecting and storing biochemical assessment via laboratory measurements could be integrated with a meal planner to ensure specific nutritional deficiencies will be considered during nutrition diagnosis. By completing the cycle of NCP, the PES (Problem, Etiology, Symptoms) statement automatic generation could easily be integrated as well. Despite the above recommendations for future works, the final version of VD is already a functional personalized meal plan recommender system, which gathered a positive response from its target users. Everyone deserves access to the nutrition knowledge, either from a professional or virtual dietitian, that may help in making one's health much better.