Data Collection in Participants’ Pockets: Research with Apps
By Janet Salmons
Dr. Salmons is the Research Community Manager for Methodspace, and serves as Mentor in Residence for June. Her most recent book from SAGE Publishing is Doing Qualitative Research Online. If ordering from SAGE, use MSPACEQ222 for a 20% discount, valid through the end of June 2022.
Some emerging methods use tools that most of us have close at hand.
This month we are exploring new methods now emerging to study new problems and/or make use of new technologies. In the past, paper was used for diary or other methods that involved tracking thoughts, feelings, or behaviors over time. Paper has advantages in terms of flexibility, with the option to doodle or draw as well as to write. But now that many people have smartphones, applications, commonly known as apps, can be used to collect data for qualitative, quantitative, or mixed methods studies.
Variations of diary methods are being adapted for use with apps. Researchers note the flexibility and advantages for this type of research:
Diary researchers use pre-existing or solicited diaries, documents made up of regular dated entries. Such documents are particularly useful for researchers who would like to observe specific phenomena but are unable to do so because they took place before the research started or are intrinsically difficult to observe. Diaries can also provide ways of accessing voices that are often silenced in public discourses such as those of gay men, members of countercultures, or women. Diaries can be used as a source of data in a variety of research designs, including experimental and survey research that use structured diaries to collect specific data from selected groups as well as interpretive, ethnographic research that examines the ways in which diarists narrate their life and experiences. (Alaszewski, 2019).
While pre-existing diaries might be available through an archive, researchers using diary-style methods with apps are generally soliciting responses. They want to tap into the experience of the participant in the moment, and see how those experiences change over a defined period of time. As Jones (2019) observes:
[A] key strength of using diaries as a research method is the contemporary nature of the data provided, especially its longitudinal component. This data record created at the present or a proximal time to events helps overcome researchers’ concerns with regards to possible retrospective bias.
The diary method of data collection is popular with researchers who are looking for participants to provide a contemporary record of their own actions, thoughts, and behaviour on a regular basis over a period of time. The unstructured diary and more free form can provide rich and meaningful qualitative data, whilst the more structured diary entry is favoured by those researchers looking to discover answers, comments, and reflections on specific areas of interest. One benefit of using a diary is that it is both self-reported and contemporary in nature with a longitudinal time frame. Diary entries are made at either specific intervals such as daily, weekly, or monthly or after specific occurrences such as journeys, visits, or activities.
Some researchers adapt existing smartphone applications, including email or messaging. For example, the researcher can prompt the participants to complete their diary entry on a daily, weekly, or ad hoc basis and link to the app where the response can be entered. Those with larger projects and more time and resources create customized apps.
Two more advantages: the researcher need not be physically present, making it possible to conduct such studies with participants in far-flung locations. Also, in many cases the need for transcription might be eliminated when answers are written, and apps can make analysis simpler, because the data were already entered into a system.
Take a look at these multidisciplinary examples to learn about methods, advantages, and disadvantages researchers are discovering when collecting data using apps. Unless noted, they are open access.
Apte, D., & Upadhyay, A. K. (2022). Cross-Directional Hybrid Space: A Non-Locative Approach to Mobile Social Media Studies. Social Media + Society. https://doi.org/10.1177/20563051221098342
Abstract. The article revisits the theory of hybrid space and looks more closely at the process of integration of physical and virtual spaces. It argues that the formation of hybrid space is not unidirectional but cross-directional. It also aims to address some of the concerns raised by Humphreys about how studies on mobile social networks have mainly been location-based, while increasingly, the use of socializing apps is non-locative. This concern is addressed primarily by offering a non-location-based approach to study mobile spatiality and sociality. This approach involves studying spatial integration afforded by smartphone chat apps with their “ubiquitous connectivity, portability, and fluidity.” While content analysis showed that cross-directional hybrid space exists and that the elements from the physical space migrate to the virtual space, the use of space-event text matrix further helped in unraveling the nature of cross-directionality more closely.
Bokonda, P. L., Ouazzani-Touhami, K., & Souissi, N. (2020). A Practical Analysis of Mobile Data Collection Apps. International Journal of Interactive Mobile Technologies, 14(13).
Abstract. Nowadays, data collection has become an activity inherent in the emergence of any organization. The digital age has enabled the development of mobile data collection apps that are becoming increasingly common around the world. But faced with the growing number of apps offered, Data Managers are often challenged by with the choice of the solution that best suits their case. This study meets this need by providing clear, precise and verified information on each of the selected solutions. The study presents, analyzes and compares four mobile data collection solutions. To achieve an effective comparison, we first chose to collect and select papers on each of the solutions, and then to install and test each of them by executing a data collection process, all the way from the form creation to the visualization of collected data. The comparison presented in this paper was based on technical aspects but also on other important aspects to help users make a good decision.
Chen, J., & Neo, P. (2019). Texting the waters: An assessment of focus groups conducted via the WhatsApp smartphone messaging application. Methodological Innovations, 12(3). https://doi.org/10.1177/2059799119884276
Abstract. Focus groups are a well-used research method in the social sciences. Typically, they are conducted in person to generate research insights through group discussion and interaction. As digital technologies advance, there have been efforts to consider how to conduct focus groups in an online format, often using computer-based tools such as email, chat and videoconferencing. In this article, we test the potential of smartphone-based mobile messaging as a new method to elicit group-level insights. Based on empirical analysis and comparison of in-person and WhatsApp group chat focus groups conducted in Singapore, we find that WhatsApp group chat does have the potential to generate well-elaborated responses and group interaction, particularly among younger, digitally fluent participants. However, the quantity and richness of the conversation still do not match that of the in-person focus groups, and further innovation may be needed to improve mobile messaging as a qualitative research method.
Daum, T., Buchwald, H., Gerlicher, A., & Birner, R. (2019). Times Have Changed: Using a Pictorial Smartphone App to Collect Time–Use Data in Rural Zambia. Field Methods, 31(1), 3–22. https://doi.org/10.1177/1525822X18797303
Abstract. One challenge of collecting socioeconomic data, such as data on time-use, is recall biases. While time-use researchers have continuously developed new methods to make data collection more accurate and easy, these methods are difficult to use in developing countries, where study participants may have low literacy levels and no clock-based concepts of time. To contribute to the closing of this research gap, we developed a picture-based smartphone app called Time-Tracker that allows data recording in real time to avoid recall biases. We pilot tested the app in rural Zambia, collecting 2,790 data days. In this article, we compare the data recorded with the app to data collected with 24-hours recall questions. The results confirm the literature on recall biases, suggesting that using the app leads to valid results. We conclude that smartphone apps using visual tools provide new opportunities for researchers collecting socioeconomic data in developing countries.
Dieter, M., Gerlitz, C., Helmond, A., Tkacz, N., van der Vlist, F. N., & Weltevrede, E. (2019). Multi-Situated App Studies: Methods and Propositions. Social Media + Society. https://doi.org/10.1177/2056305119846486
Abstract. This article discusses methodological approaches to app studies, focusing on their embeddedness and situatedness within multiple infrastructural settings. Our approach involves close attention to the multivalent affordances of apps as software packages, particularly their capacity to enter into diverse groupings and relations depending on different infrastructural situations. The changing situations they evoke and participate in, accordingly, make apps visible and accountable in a variety of unique ways. Therefore, engaging with and even staging these situations allows for political-economic, social, and cultural dynamics associated with apps and their infrastructures to be investigated through a style of research we describe as multi-situated app studies. This article offers an overview of four different entry points of enquiry that are exemplary of this multi-situated approach, focusing on app stores, app interfaces, app packages, and app connections. We conclude with nine propositions that develop out of these studies as prompts for further research. Dieter, M., Gerlitz, C., Helmond, A., Tkacz, N., van der Vlist, F. N., & Weltevrede, E. (2019). Multi-Situated App Studies: Methods and Propositions. Social Media + Society. https://doi.org/10.1177/2056305119846486
Faghih Imani, A., Harding, C., Srikukenthiran, S., Miller, E. J., & Nurul Habib, K. (2020). Lessons from a Large-Scale Experiment on the Use of Smartphone Apps to Collect Travel Diary Data: The “City Logger” for the Greater Golden Horseshoe Area. Transportation Research Record, 2674(7), 299–311. https://doi.org/10.1177/0361198120921860 (Library access only.)
Abstract. Smartphones offer a potential alternative to collect high-quality information on the travel patterns of individuals without burdening the respondents with reporting every detail of their travel. Smartphone apps have recently become a common tool for travel survey data collection around the world, especially for multiday surveys. However, there still exists a lack of systematic assessment of issues related to smartphone app-based surveys, such as the impact of app design or the recruitment method on the collected data. Through a large-scale experiment (named the City Logger), this paper assesses the data produced by the City Logger app, to better understand recruitment avenues (targeted invitation versus crowdsourcing), and examine differences in respondents’ travel behavior recruited through crowdsourcing methods. The paper also examines how app design, and particularly the user input method for trip validation, influences participants’ responses. The results indicate that, while crowdsourcing recruitment is promising, it might not yet be the best way to capture a true representation of the population. For app design, a combination of real-time and travel diary approaches is recommended. An ideal app would prompt users real-time and create a travel diary, so users can validate, edit, or delete the recorded information.
Hoplamazian, G. J., Dimmick, J., Ramirez, A., & Feaster, J. (2018). Capturing mobility: The time–space diary as a method for assessing media use niches. Mobile Media & Communication, 6(1), 127–145. https://doi.org/10.1177/2050157917731484
Abstract. As mobile media have grown more advanced, and mobile Internet access has increased to a near-ubiquitous state, media use is often described as occurring “anytime, anywhere.” Consequently, measuring media use and understanding competition and coexistence within such an environment is a constant challenge for researchers. To help address this issue the present study explicates a method for measuring media use and competition, the time–space diary, and reports a methodological study testing the robustness of this method across 3 diary instruments. Following a summary of concepts central to mobile media use and measurement, this study reports findings from data collected using 3 types of time–space diaries. Results indicate no significant difference between diary methods (paper and pencil, audio recorder, mobile device) on the likelihood of reported media use, and minimal differences in patterns of competitive superiority, supporting the reliability of the method. Paper-and-pencil diaries are argued to offer the greatest use flexibility relative to audio recorders and mobile devices, and received higher ease of use scores relative to mobile devices. The article concludes with a discussion of the utility of the time–space diary method for emerging mobile media research which must account for media use in novel times and places, as well as multichannel media consumption.
Kaufmann, K., & Peil, C. (2020). The mobile instant messaging interview (MIMI): Using WhatsApp to enhance self-reporting and explore media usage in situ. Mobile Media & Communication, 8(2), 229–246. https://doi.org/10.1177/2050157919852392
Abstract. How do people use media technologies in everyday life and how do they make sense of them? This is one of the core questions in media and communication studies, gaining even more in relevance in light of the rapidly changing, convergent media environment. Yet, rich and context-sensitive data about media and technology use are difficult to generate when media consumption is increasingly pervasive, ubiquitous, and often goes on in passing. At the same time, the full potential of smartphones in qualitative research has not yet been realized. The mobile instant messaging interview (MIMI) introduced and assessed in this paper is intended to fill this gap by exploiting some of the unique communication and multimedia features offered by mobile instant messaging apps. Drawing on diary techniques and on the tried and tested mobile experience sampling method (MESM), the MIMI uses WhatsApp for an in situ exploration of distinct settings and situations of social action (e.g., media usage). To substantiate the approach, the results of a pilot study conducted with young smartphone users are presented, discussing the advantages and drawbacks of mobile instant messaging interviews in detail, from the researcher’s as well as the participant’s point of view.
Lev-On, A., & Lowenstein-Barkai, H. (2019). Viewing diaries in an age of new media: An exploratory analysis of mobile phone app diaries versus paper diaries. Methodological Innovations, 12(1). https://doi.org/10.1177/2059799119844442
Abstract. This exploratory study inquires into the validity and reliability of dedicated mobile phone diary applications. We developed Watchy, a dedicated mobile viewing diary application, and compared users’ compliance and usage patterns with those of users of the paper viewing diaries. Participants received paper diaries or installed mobile diary apps, with or without daily reminders, to document their viewings over a 4-day period. Documentation was more extensive in the smartphone app with reminder group compared to the paper diary group. Reminders increased documentation rates. Extent of documentation decreased as the experiment progressed for mobile app users. Findings suggest that mobile viewing diaries are an important tool for viewing studies, yet their use requires careful planning.
Miatello, A., Mulvale, G., Hackett, C., Mulvale, A., Kutty, A., & Alshazly, F. (2018). Data Elicited Through Apps for Health Systems Improvement: Lessons From Using the myEXP Suite of Smartphone and Web Apps. International Journal of Qualitative Methods. https://doi.org/10.1177/1609406918798433
Abstract. A promising approach to meeting the need in many jurisdictions for timely, in-depth qualitative health systems experience data, is to elicit feedback through smartphone and web applications (apps). Apps offer an appealing tool to elicit data from patients and family members who may feel stigma when receiving some services and a power imbalance when providing feedback to health-care providers. In this article, we examine the effectiveness of a suite of smartphone and web apps called myExperience (myEXP) that were created to gather care experiences of youth, family members, and service providers as part of an experience-based co-design (EBCD) study in Ontario involving youth with mental disorders. We analyzed data from 12 triads of youth (aged 16–24), family members, and service providers gathered between August 2015 and December 2016. We used qualitative content analysis to understand participant feedback on the myEXP apps and identify thematic categories that emerged from experience data elicited through the myEXP apps. We found overall that the myEXP apps were more effective at eliciting experience data from youth compared with family members and service providers. Rich experience data were gathered from youth about treatment plans in real time through the apps. The apps also showed important promise as reflective tools for all participants. They may offer advantages in research that seeks to improve responsiveness in service delivery and build mutual understanding. The apps also offer choice in how data are elicited, encourage more candid feedback and help to overcome stigma, which are important considerations for some vulnerable populations. For service redesign research using approaches such as EBCD, apps offer real-time data gathering that can complement and enhance traditional approaches such as retrospective interviews and observation.
Revilla, M., Paura, E., & Ochoa, C. (2021). Use of a research app in an online opt-in panel: The Netquest case. Methodological Innovations, 14(1). https://doi.org/10.1177/2059799120985373
Abstract. The increasing use of mobile devices in the frame of online surveys has been accompanied by the development of research apps. These research apps have the potential to facilitate the process for respondents (e.g. being able to complete surveys when Internet is not available provides more freedom on when and where participants can participate) and fieldwork companies (e.g. the possibility to use push notifications could lead to higher participation rates). However, previous research suggests that panelists may also be reluctant to install an app. In this study, we answer research questions related to the knowledge and use of the Netquest app. We found that a majority of panelist did not know about the app and although sending invitations significantly increased its installation, the overall total of respondents installing the app remained low. Furthermore, the profile of those who installed the app differs from those who did not. The participation of panelists after they installed the app seems stable. The main reason for installing the app is comfort while the main reason for not installing relates to space/battery usage. Most of those who did not install could accept to install the app.
Riggleman, S. (2021). Using Data Collection Applications in Early Childhood Settings to Support Behavior Change. Journal of Special Education Technology, 36(3), 175–182. https://doi.org/10.1177/0162643420942763 (Library access only.)
Abstract. Social–emotional development in early childhood (EC) is an important factor to their later development and adjustment. While all young children display unwanted behaviors at some time during development, challenging behaviors that occur across settings and over a period of time should be identified and intervened; thus, data collection efforts need to be efficient and accurate as time can be a barrier. There are currently applications for children in a K–12 setting that can be used to collect data and share positive behavior support plans; however, they may not necessarily be tailored to the EC population. A variety of data collection apps will be discussed as they relate to identifying a behavior to data collection skills. Suggestions will be given on what apps should contain for data collection and how to make them applicable to EC settings as well as collaborating with multiple professionals.
Roessger, K. M., Greenleaf, A., & Hoggan, C. (2017). Using data collection apps and single-case designs to research transformative learning in adults. Journal of Adult and Continuing Education, 23(2), 206–225. https://doi.org/10.1177/1477971417732070 (Library access only.)
Abstract. To overcome situational hurdles when researching transformative learning in adults, we outline a research approach using single-case research designs and smartphone data collection apps. This approach allows researchers to better understand learners’ current lived experiences and determine the effects of transformative learning interventions on demonstrable outcomes. We first discuss data collection apps and their features. We then describe how they can be integrated into single-case research designs to make causal inferences about a learning intervention’s effects when limited by researcher access and learner retrospective reporting. Design controls for internal validity threats and visual and statistical data analysis are then discussed. Throughout, we highlight applications to transformative learning and conclude by discussing the approach’s potential limitations.
References
Alaszewski, A. (2019). Diaries. In P. Atkinson, S. Delamont, A. Cernat, J.W. Sakshaug, & R.A. Williams (Eds.), SAGE Research Methods Foundations. https://dx.doi.org/10.4135/9781526421036760284
Jones, A. (2019). Email Diaries. In P. Atkinson, S. Delamont, A. Cernat, J.W. Sakshaug, & R.A. Williams (Eds.), SAGE Research Methods Foundations. https://dx.doi.org/10.4135/9781526421036823709
