Mobile Bookkeeper: Personal Financial Management Application with Receipt Scanner Using Optical Character Recognition

Introduction

In a struggling economy, yet with a plethora of spending temptations, personal financial management is considerably a much-needed technique every economic pawn and players should master. Yet, organizing finances can be complex at times given the polymorphism of financial information (bills, statements, invoices, pay stubs, receipts, to name a few). As such, people often have difficulties even finding any of these financial documents when looked-for [1], not to mention the tracking and monitoring of personal cash flow. Movement of money becomes important only when it is going in, but tend to be forgotten when it is going out as people underestimate the amount of cash they spend on purchasing unplanned and off-budget items [2] despite the presence of mental budget. In a series of field experiments administered at convenience stores, the result highlighted the impact of discount stores to consumer behaviour when every merchandise appears to be a good deal that makes consumers purchase more items [3]. Most consumers fall prey to this marketing ploy despite the absence of a long-term benefit of frugality, as the spending temptation and the satisfaction it delivers is hard to resist. A potential solution to overcome this kind of financial dilemma is by providing feedback on spending behaviour with info on cumulative spending [4] like in a budgeting application [1].

As conceived by this potential solution, the purpose of this study is to develop such budgeting application called "Mobile Bookkeeper" with Optical Character Recognition (OCR) for the receipt scanner feature. The implementation of OCR was considered in order to translate printed financial documents like receipts digitally without the hassle of typing the information needed by Mobile Bookkeeper to generate useful financial reports. Such report is dedicated in providing feedback on personal spending behaviour with information on cumulative spending and expenses. After the development of Mobile Bookkeeper, satisfaction and usability tests were performed with the assistance of mobile app experts, finance professors, and other participants using ISO 25062 [5] and ISO 9241 [6] standards, and QUIS 7.0 questionnaire [7], which follows the empirical study in evaluating usability of mobile applications [8]. As little mobile app prototype and research have been published in the development of mobile budget application with OCR technology, and its usability and satisfaction as perceived by its target users, this study means to contribute to literature and provide a baseline for mobile application developers, practitioners, and researchers who will conduct similar project and research in the future. The literature gap this study is trying to solve is the addition of receipt scanner in budget applications by means of OCR.

Related Studies

Drawing from literature, three concepts were analyzed to establish the foundation of this paper. First was the systems review and dissection of existing finance management tools and budget applications to determine software features and characteristics, and what generally makes these tools usable for its target users. The findings served as a template for the system foundation of the mobile application development. Afterwards, the OCR technology was examined with the intention of finding out its implementation procedures in mobile platforms as well as the process of improving text recognition accuracy – a common issue in OCR integration due to conditions of uneven lighting, blurring, tilting, scene complexity, rotation, typography, warping, multilingual environment, and other factors affecting the image quality [9]. Moreover, phases of OCR were also critically reviewed from pre-processing up to post-processing phase. Finally, an inspection on an empirically grounded usability evaluation for mobile applications was performed to regulate possible issues as well as mobile limitations. In addition, this is done to guarantee that a proper evaluation process and instrument was employed for assessing satisfaction of potential users and the general usability of Mobile Bookkeeper.

Finance Tools and Budget Applications

The very existence of budgeting apps is about effortlessly managing finances by means of tracking bills, establishing saving goals, developing budgeting plans, storing financial documents digitally, and growing financial awareness in general. One example of a budgeting app in mobile platform is PocketGuard [10] which objective is to make a hassle-free money management. The strength of this mobile budget app lies on connecting financial accounts in one place to see what is really going on with one’s finances. Mint by Intuit [11] is another example of budget app that aims to achieve the same thing by automatically updating and categorizing information from bank accounts to retirement accounts. In the context of software, BudgetPulse [12] offers comprehensive graphs and advanced money management like easily tracking recurring activities and splitting large transactions into categories. All of these finance tools and budget apps, mobile or software, are comprehensive in terms of its features but users cannot fully maximize the potential when they cannot or simply will not enter transaction details from a receipt. Consequently, the aforesaid software arsenal is missing an OCR technology.

Optical Character Recognition (OCR)

OCR, in a simplest definition, is the conversion of text on paper into electronic form which is performed in numerous fields such as banking for automatic processing of cheques, legal industry for digitizing documents, business for receipt imaging, and healthcare for converting patient information, to name a few [13]. A detailed analysis of OCR technology [9] implied that implementing computerized OCR does not occur without challenges, and it is critical that these issues must be spotted and solved with a head-on approach. Tricky, image-related issues such as skewness, blurring, and tilting are solvable by undergoing pre-processing stage before the actual conversion where pictures are cleaned up using image processing techniques [14] such as skew alteration, lighting correction, binarization, histogram equalization, and noise reduction. Further, to obtain high character recognition rate, the procedure of converting text from paper to digital form should undergo various processes: (1) segmentation to isolate texts from its background, (2) normalization to reject data that is unnecessary without a loss on the important ones [15], (3) feature extraction to mine important features by utilizing neural network [16] or Euler number [17], (4) classification to distribute inputs using Support Vector Machine, statistical techniques, or a combination of classifiers, and (5) a postprocessing phase for a round of manual double checking of data through spell checking or content matching. With the abovementioned phases, the implementation of OCR into an application like Mobile Bookkeeper may result to a more and high accurate character recognition rate [9].

Usability Evaluation of Mobile Applications

At its core and in the context of mobile apps, usability is all about ease-of-use of mobile applications aligned with its target goals. Nielsen’s heuristics [18] has been considered as the most infamous usability evaluation framework, however, the advent of smartphones has introduced several issues such as connectivity, display resolution, limited processing power, data entry method, and screen size [19] that critically renders the traditional way of usability testing insufficient. Needless to say, the usability evaluation of Mobile Bookkeeper calls for a more suitable framework. Fortunately, a framework that was empirically evaluated for mobile applications is existing and ready to be used as a basis [7]. The said framework used software quality standards ISO 9241-11:1998 [6] and ISO 25062:2006 [5] for the foundation of usability evaluation (objective measures) and Questionnaire for User Satisfaction Interaction (QUIS 7.0) [7] for user’s satisfaction (subjective measures). Considering the literature, Mobile Bookkeeper’s usability evaluation was grounded with empirically validated framework that is focused on mobile applications.

Methods

This developmental-usability study focused on designing and developing a budget app with OCR technology. Testing the usability of the mobile app and the satisfaction of users upon using it was also part of the objective. To achieve these objectives, a project methodology following agile approach was needed to ensure that the evaluation of users have a role throughout the development of Mobile Bookkeeper. Hence, Mobile Application Software Agile Methodology (MASAM) [20] was employed as the project methodology as it endorses a simple but effective development life cycle for mobile app development. Moreover, MASAM is a collective strength of Rational Unified Process, Extreme Programming (XP), and Agile Unified Process. Lastly, mobile-related issues such as multi-platform availability, connectivity, display resolution, limited processing power, and screen size [19] were put into consideration when MASAM was developed.

System Design

Mobile Bookkeeper was developed using an open-source framework for building cross-platform mobile applications known as Apache Cordova [21]. As an alternative on relying on platform-specific APIs of mobile operating systems, apps developed using Apache Cordova exploited the potentials of HTML5, CCS3, and JavaScript. This is usually not the main choice for developers since OCR libraries were not freely available on web-based languages. In fact, OCR libraries like Tesseract work best with mobile applications created using native APIs. Fortunately, there is now a browserbased version of this OCR library, Tesseract.js [22], that can arm web-based languages with text recognition. Apart from this, the OCR integration on the application via the receipt scanner feature undergoes several phases as shown on Figure 3. Each phase of receipt scanner feature was based from the related works discussed on the related studies. On the other hand, the feature for expense report generator (see expense report app screen on Figure 2) used visualization techniques to present data into a more coherent and meaningful way.

Study Design

This development and usability study was started on the first month of 2019 and lasted 12 weeks where the first eight weeks was allotted for the app development period while the remaining month was for the usability-testing period. Tasks that are predetermined were given to participants as a to-do list before the usability and satisfaction evaluation instead of just letting them explore Mobile Bookkeeper on their own. This is pivotal since the metrics on the objective measures as derived from the ISO standards deal with time, unlike the subjective measures with usual quantitative measurement.

Participants, Procedures, and Privacy

The participants in this study constituted 12 mobile app experts (instructors and practitioners), 3 finance professors, and 15 regular app users selected using purposive sampling technique. Application package files were stored in a private server and the web address was given to the participants in a private email. To enable the download button in the website, users must answer a basic questionnaire first describing their knowledge in mobile applications and smartphones. Those who evaluated Mobile Bookkeeper at home were instructed to time their tasks concerning the objective measures. After exploration and app usage, participants were given another private link for the satisfaction questionnaire. Data privacy and anonymity were guaranteed to the participating users.

Usability Evaluation

As mentioned from the previous section of this paper, the usability evaluation of Mobile Bookkeeper was based from an empirically tested framework employing software quality standards ISO 9241-11:1998 [6] and ISO 25062:2006 [5] for the objective measures, and QUIS 7.0 [7] for the subjective measures. The objective measures of the usability evaluation are effectiveness and efficiency, which are based on the ISO standards while the subjective measure is the app satisfaction via six scales and six interface factors based on QUIS 7.0 questionnaire. The effectiveness attribute has the following metrics: data entry time, tasks time, response time, time to learn and use, and time to install. The efficiency attribute, on the other hand, has only two metrics: number of errors, and completion rate. Effectiveness and efficiency were measured by recording the time during app usage. The satisfaction attribute, apart from the overall satisfaction score, has the following measures: screen evaluation, learning, terminology and information, technical manual and online help, usability and user interface (UI), and application capabilities.

Result and Discussion

The principal findings of the design and development of Mobile Bookkeeper reported below were founded from the results of mobile application usability (ISO standards) [5, 6] and user satisfaction (QUIS 7.0 questionnaires) [7]. On a side note, the accuracy rate of text recognition was not measured in this study since the objective was to develop a mobile app, and not to propose a new algorithm for OCR. Therefore, the results of users’ assessment to Mobile Bookkeeper were focused on usability and satisfaction as discussed below:

Usability (Objective Measures)

Table 1 summarizes the mean of data collected for each usability metric using predefined tasks as mentioned on the study design section. It took an average of 4 seconds to enter data (scanned document or take pictures) for the execution of text conversion. While taking photos or uploading media files are supposed to be faster than this, it is important to note that participants were cautioned on the low rate of text recognition concerning photos with complex scene or color issues, which may have affected their speed. The predefined tasks that covers receipt data conversion were performed on an average of almost two minutes albeit the data entry time is sufficiently fast, perhaps, because the tasks also involved moving from one screen or feature to another. Hence, layout arrangement and navigation menu must be checked in order to determine what went wrong. Once the data entry, which is part of the tasks, was completed, it took an average of 34 seconds to convert the text on receipts to a digital format. As usual, this result depends on image-related factors such as quality, size, resolution, and text density, which is regularly low for documents like receipt. The time to comprehend the tasks and begin performing it (time to learn and use) took an average of 40 seconds, which can be attributed to the use of "mobile onboarding" – a technique used by User Experience (UX) experts to aid first-time app users. The app installation time is also fast with an average of 32 seconds since the app is lightweight and does not have much media files except for the assets dedicated for branding. The number of errors were recorded as well to determine the challenges faced by users. In total, there were three recorded errors such as: (1) login failure, (2) incorrect total expense amount (due to incorrect text conversion), and (3) a non-functional submit button. All of these were then corrected after the evaluation. Lastly, the completion rate of participants who successfully performed and achieved the goal of each task is 100%, which dictates that Mobile Bookkeeper, a budget tool with receipt scanner through OCR, did pass the usability evaluation.

Satisfaction (Subjective Measures)

Table 2 summarizes the mean of data obtained for each satisfaction measure, which have been classified on a scale of 9 according to the QUIS 7.0 questionnaire [7]: 9 denotes excellent, 6–8 denotes very good, 4–6 denotes good, 2–4 denotes fair and 1–2 denotes poor. The screen evaluation, as rated by the participants, garnered a 7.47 (very good) score, which indicates that the User Interface (UI) element designs and layout are not confusing to users, typography is easy to read, and the organization of information is very clear. For the terminology and information, participants rated it as 7.31 (very good) since the terms on the application are consistent and related to the task they were doing, and error messages were helpful and clear. Moreover, the learning measure got the top score, 8.56 (very good), together with usability and UI, which signifies that Mobile Bookkeeper and its receipt scanner feature using OCR was easy to learn to operate, the output was helpful, and tasks can be performed in a candid manner. In the context of usability and UI, it can only mean that the use of color, system feedback, messages and reports and system clatter were pleasing for app users. Application capabilities, on the other hand, garnered a 6.72 (very good) rating, which indicates that Mobile Bookkeeper is reliable and the features of the application (see Figure 2 for sample app screens) are enough to meet the needs of its users including the integration of OCR technology to a mobile finance app. Lastly, the overall reaction of users to Mobile Bookkeeper is equals to 7.75 (very good) which means that the mobile app was generally acceptable and had successfully met its goals. The result is a proof that users find such mobile application as a helpful tool for personal finance management [1].

Conclusion

In summary, a budgeting application with integration of OCR technology, Mobile Bookkeeper, was developed for personal financial management. The role of the application is to deliver feedback not only on cumulative spending but the cash flow in general that could mediate and solve common financial dilemmas [4]. The application of OCR was placed in order to convert printed financial documents like receipts digitally without the disturbance of typing the information one after another into the app just to generate useful financial reports. The development of Mobile Bookkeeper through MASAM, and its evaluation using ISO standards and QUIS 7.0, drawn a conclusion that the aforesaid mobile application was usable and satisfactory for the participants of the study.