Do as I tweet, not as I do: comparing physical activity data between fitness tweets and Healthy People 2020

Introduction

The promotion and monitoring of physical activity have been a focus of public health efforts in recent years. However, objectively measuring population-level physical activity is challenging because it requires tracking a large number of people using expensive devices and imposing strict data-collection protocols (1). That said, emerging technology can provide reliable and valid alternative surveillance tools for self-reported measures of physical activity (1). Although there is an increase in the number of studies using integrated sensor technology to collect physical-activity data on a population level, there is little technical guidance for researchers who want to use this technology within their research (2).

According to Graham and Hipp, “Physical activity measurement research is achieving greater ease of use, precision and scope by incorporating emerging technologies. These emerging technologies are noteworthy because they can: greatly increase external validity of measures and findings through ease of use and transferability; significantly increase the ability to analyze patterns; improve the ongoing, systematic collection and analysis of public health surveillance due to real-time capabilities; and address the need for research about the cyber infrastructure required to cope with big data.” (3).

In 2010, the U.S. Department of Health and Human Services published the fifth instalment of the national report on health and wellness, reflecting the strong state of the science supporting the health benefits of regular physical activity based on the accomplishments of previous Healthy People initiatives (4). The report, entitled Healthy People 2020, introduced new 10-year objectives for health promotion and disease prevention. New to the objectives is “myHealthyPeople,” a challenge for technology application developers. The research discussed here reflects an attempt to meet that challenge. The use of the fitness tweet classification model which was developed for this study enables researchers to collect ongoing data in real time, which is a sharp contrast to phone interviews that rely on participant recall. The biggest challenge in using technology to track physical activity lies in accounting for the fact that many users are inconsistent in their use of the tracking devices.

One component of Healthy People 2020 involves physical activity, suggesting that Americans should engage in at least 150 minutes per week of moderate-intensity physical activity to obtain substantial health benefits and more than 300 minutes per week to obtain more extensive health benefits.

Current baseline and targets are presented in Table 1.

Table 1 Healthy people 2020 baseline and targets
Full table

In 2008, when the goals and objectives for Healthy People 2020 were first developed, 43.5% of American adults met the goal of 150 min per week of moderate-intensity physical activity, with only 28.4% reaching 300 min per week (5).

Methods

For this research project, a comparison between the collected physical-activity data provided in the Healthy People 2020 report and physical-activity data collected from five mobile fitness apps (Nike+, DailyMile, MyFitnessPal, Endomondo and RunKeeper) as publicly shared over Twitter was conducted.

Each mobile fitness app used in this research had a standard word phrase for the automatic sharing of physical activity using fitness tweets that include time and/or distance of the physical activity. In addition, some mobile fitness apps included in the standard word phrase a shortened URL that directed back to the mobile fitness app’s user page.

On that page, additional information not included in the fitness tweet could be collected (Figure 1). A data-scraping script was written to collect this information. Once the data were collected, a data cleaning removed low totals (less than 15 min of reported physical activity over 28 weeks) and high totals (more than 30,000 min of reported physical activity over 28 weeks) in order to account for one-time users or user error and invalid results stemming from technology-related issues (e.g., a fitness app being left open after a workout is completed, which would inflate the numbers and skew the data).

Figure 1 User page from shortened URL link in fitness tweet.

Data

Data for this research was from two data sets:

• Healthy People 2020;
• Fitness Tweet Classification Data Set.

Results from the Healthy People survey were compiled using telephone interviews of self-reported physical activity from the previous week. There are considerable concerns about this methodology, as physical-activity questionnaires show limited reliability and validity (6). Even so, they have long been considered the only feasible means of collecting data in large populations, despite the fact that researchers know that responses can be influenced by cultural factors, language barriers and recall accuracy, particularly in older populations (6). One aim of this research study is to explore the use of Twitter as a more reliable and valid alternative.

The fitness tweet classification data set was collected from mobile fitness app users who shared their physical activity over Twitter. Over 184 days, 2,856,534 tweets were collected in 23 different languages. However, for the purposes of this study, only the English-language tweets were analysed, resulting in a total of 1,982,653 tweets by 165,768 unique users.

The fitness tweet classification model (7) was used to classify each tweet into main categories of activity, blarney and conversation and then into subcategories as shown in Figure 2.

Figure 2 Fitness tweet classification model.

Results

In total, 102,544 users mentioned workout duration in their tweets, accounting for 2.4 million min of physical activity. The addition of workout type, duration and distance allowed additional analysis to be conducted. Physical activity is a sporadic and complex behaviour to measure, but previous research suggests that three days of accelerometer data, four days of pedometer data or 4 days of physical-activity logs are needed to reliably measure physical-activity levels in older adults (8).

As demonstrated in Figure 3, the data collected from fitness tweets using the five mobile fitness apps suggest lower percentages of people achieving both the 150 to 300 and 300+ min levels. The lower percentage for the 150 to 300 min range was expected, as it is difficult to know whether a person used their mobile fitness app during every workout session. What was a bit more surprising was the lower percentages for the 300+ min levels, as the more active population might be expected to be more dedicated users of their wearable devices and mobile fitness apps.

Figure 3 Average weekly minutes of physical activity by app.

Consider, for example, users of Nike+, which could be considered the most physically active overall group, as Nike targets athletic shoe buyers through social media channels. Analysis of the fitness tweet conversations also indicated a higher number of mobile fitness app users using Nike+ to train for 5K, 10K, half-marathon and full-marathon events. That being the case, the hypothesis was that of the five mobile fitness apps, Nike+ would be one of the more used mobile fitness apps within the 300+ min category due to the daily training regimens of the participants. Figure 4 highlights the data analysis suggesting that weekly Nike+ users fitness tweet an average of 81 min per week of physical activity. In fact, RunKeeper, which is also geared toward runners, reported the second lowest average weekly minutes of physical activity, with just over 104 min per week.

Figure 4 Mobile fitness app vs. Healthy People 2020.

The overall variance in the data derived from those who completed the Healthy People 2020 survey and mobile fitness app fitness tweeters could be due to users not sharing all of their physical activity via Twitter and/or an overestimation of weekly minutes of exercise collected during the phone surveys for Healthy People 2020. Table 2 shows how one aspect of physical-activity data collected from Twitter can be presented. To maintain confidentiality of the users, Twitter user names were replaced with generic ‘User xxx’ labels. It is important to determine the user’s first user date of the mobile fitness app within the data-collection period, as the data-collection timeframe is just a snapshot over time. A user could have already been using the mobile fitness app and sharing the data before the start of the data-collection period. Cells that contain the label “X” indicate that the first use date of the mobile fitness app by the user occurred after the week header. For example, the first use date for User 13 occurred sometime in week 4. It is also important to be able to determine gaps of weekly usage over time, showing that a user is not consistent in the sharing of physical-activity data from mobile fitness apps using Twitter, or that the user simply did not exercise for a time due to injury, illness, vacation, etc.

Table 2 User tables with first use date, total minutes and weekly status
Full table

Discussion

This case study presents a comparison between weekly minutes of physical activity derived from Healthy People 2020 survey results and fitness activity tweets of mobile fitness app users, and provides physical-activity researchers an alternative method of data collection that could be more reliable than self-reported physical-activity survey data. The issue of why this research yielded lower percentages of physical activity than the Healthy People phone survey remains unaddressed. Is it possible that the Twitter data is more accurate and that people are over-reporting their activity levels over the phone? Can further research derive a means of accounting for any under-reporting that is taking place via Twitter? Recall bias is a considerable issue in phone surveys, as people tend to overestimate their physical activity and underestimate their sedentary time; thus, researchers have developed ways to account for this bias when analysing the resulting data (9). This needs to be done for Twitter-based data as well, but ongoing, real-time data analysis is an invaluable resource for researchers that should eventually prove to be more reliable then recall-based phone surveys.

This active data collection could provide numerous benefits when compared to passive data collection. For example, some evidence already suggests that the knowledge that their activities are being monitored could impact participants’ weekly minutes of physical activity (8). While this may be problematic in a research setting, as described above, it can lead to true lifestyle change in individuals who use social media to motivate themselves to stay on track.

Obtaining information from social media allows for crowdsourced participation, which can provide much more data diversity in terms of greater range of age, geography and ethnicity of users. Moylan, Derr and Lindhorst found that mobile technology was especially useful in reaching out to participants who were previously inaccessible due to geography or physical disability (10). Employing Twitter and other social media as data-collection tools could help researchers obtain information that users might not remember or be willing to disclose face-to-face or over the telephone. Ahlwardt found that patients are often willing to reveal information about their personal healthcare experiences on Twitter, allowing healthcare providers to glean insight on how to improve communication with patients and treat them more effectively (11). Because users are often relaying information in real time, some researchers posit that the personal details users share may be more accurate than data collected by traditional methods.

Social media data collection also provides the added benefit of allowing researchers to access more people in their target population over a shorter amount of time. Furthermore, Casler and College [2013] discovered that participants who signed up for studies online performed behavioural tasks just as well as people who participated face-to-face or over the phone (12). Information collected over social media has also provided additional useful healthcare data, from the presentation of menopause symptoms in women to the prevalence of children with ulcerative colitis. Healthcare practitioners can also access additional information through these methods, including demographics, current medication lists and potential diagnoses.

Limitations

A common limitation in this type of research lies in the fact that adolescents and adults do not always accurately report physical-activity levels (13), with many underreporting sedentary behaviours and over-reporting exercise (9).

As this research used all Twitter data (i.e., users were not assigned a specific mobile fitness app to share physical-activity data), one cannot assume that when a user reported zero minutes of weekly activity, this means that the user actually performed no physical activity during that period. There could have been any number of user, device, data collection or Twitter errors. Other than sending a tweet to each individual user, it would be difficult to determine the reason for the lack of data.

An additional limitation is the actual definition of physical activity. During the original data collection for the Healthy People project, depending on how the question was phrased, the respondent may have answered in one of two ways. First, he or she could have provided the number of minutes he or she performed traditional physical activity by going to the gym or going outside for a run, for example. Second, the respondent could have included all physical activity, including non-structured activity such as walking in a mall. The data set of the Fitness Tweets would suggest that this data is a collection of exercise-type activity rather than ongoing measurement, as such measurement would be difficult due to battery issues throughout the day.

While we are confident that during the data-collection process we had access to the Twitter firehose allowing for the collection of all publicly available tweets, there is no way to verify this without actually purchasing all of the tweets. There remains a challenge in the extraction of useful data within these repositories through data mining and knowledge discovery (14). Researchers could enhance our model by purchasing commercially available data sets for analysis in future studies.

While we created a very potential tool for large-scale research by collecting physical-activity data from Twitter, the demographics used in this research could suggest a bias in terms of the users of the mobile fitness apps and thus under-represent certain groups. If researchers wish to use Twitter and mobile fitness apps for physical-activity research, additional steps would need to be taken to ensure that all groups are represented in the data samples collected from Twitter.

These findings and interpretations should be regarded as exploratory and speculative, as they represent what can be potentially done in a short development time and with ease of use for non-computer programming health-promotion researchers.

Future work

Advancements in technology design for both smartphones and wearables allow for continuous monitoring of physical activity without a drain in battery life. Depending on the sharing ability, physical activity could be measured by hour or even by minute, thus providing an even greater detail of recorded physical activity.

One benefit of using the fitness tweet classification model was that the database included 184 days of continuous data collection, which stands in stark contrast to the one-week recall used in the Healthy People project. While not every subject had daily physical-activity measures, the same is true with the survey respondents in the Healthy People project. One future area of work could be the determination of how many days’ worth of fitness tweets would be needed to reliably measure physical activity.

Future research could also involve a study that uses fitness tweeting as a more effective data-collection tool, with participants understanding what is being measured and the need to share all physical-activity sessions—as opposed to passive data collection. Knowing they are being monitored could impact participants’ weekly minutes of physical activity, but perhaps not in longer-term studies. Because this type of research can be conducted on an ongoing basis, the phenomenon of study participants outperforming their usual activity levels should dissipate over time as they return to their usual behavioural patterns.

Conclusions

Technologies currently used in other fields could be adopted for physical-activity measurement. This research used one such technology—Twitter—and created a method to collect physical-activity data from publicly available tweets. The precise measurement of physical activity, including type, amount, context and place is essential for increasing physical activity (2). While this approach shows promise in data collection, future research on how to account for user inconsistency in terms of reporting physical activity is needed before Twitter-based data can be considered truly reliable, but it is clear that Twitter, other forms of social media and smartphone apps are here to stay. Health and fitness professionals and researchers in this area would benefit from leveraging the ever-growing population of users in their work.

Acknowledgements

This research was conducted under grant number SFI/08/CEI1380 from the Science Foundation Ireland and IRCSET.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Informed Consent: Informed Consent was not required because data collected and provided could not be tracked back to individuals.

References

1. Lim S, Wyker B, Bartley K, et al. Measurement error of self-reported physical activity levels in New York City: assessment and correction. Am J Epidemiol 2015;181:648-55. [PubMed]
2. Hurvitz PM, Moudon AV, Kang B, et al. Emerging technologies for assessing physical activity behaviors in space and time. Front Public Health 2014;2:2. [PubMed]
3. Graham DJ, Hipp JA. Emerging technologies to promote and evaluate physical activity: cutting-edge research and future directions. Front Public Health 2014;2:66. [PubMed]
4. Healthy People. Healthy People 2020 Objectives. Available online: http://www.healthypeople.gov/2020/topics-objectives/topic/physical-activity/objectives
5. Carlson SA, Fulton JE, Schoenborn CA, et al. Trend and prevalence estimates based on the 2008 Physical Activity Guidelines for Americans. Am J Prev Med 2010;39:305-13. [PubMed]
6. Shephard RJ. Limits to the measurement of habitual physical activity by questionnaires. Br J Sports Med 2003;37:197-206; discussion 206. [PubMed]
7. Vickey TA, Ginis KM, Dabrowski M. Twitter classification model: the ABC of two million fitness tweets. Transl Behav Med 2013;3:304-11. [PubMed]
8. Hart TL, Swartz AM, Cashin SE, et al. How many days of monitoring predict physical activity and sedentary behaviour in older adults? Int J Behav Nutr Phys Act 2011;8:62. [PubMed]
9. Celis-Morales CA, Perez-Bravo F, Ibañez L, et al. Objective vs. self-reported physical activity and sedentary time: effects of measurement method on relationships with risk biomarkers. PLoS One 2012;7:e36345. [PubMed]
10. Moylan CA, Derr AS, Lindhorst T. Increasingly mobile: How new technologies can enhance qualitative research. Qual Soc Work 2015;14:36-47. [PubMed]
11. Ahlwardt K, Heaivilin N, Gibbs J, et al. Tweeting about pain: comparing self-reported toothache experiences with those of backaches, earaches and headaches. J Am Dent Assoc 2014;145:737-43. [PubMed]
12. Casler K, College M. Separate but equal ? A comparison of participants and data gathered via Amazon’ s MTurk, social media, and face-to-face behavioral testing. Comput Human Behav 2013;29:2156-60.
13. Chinapaw MJ, Slootmaker SM, Schuit AJ, et al. Reliability and validity of the Activity Questionnaire for Adults and Adolescents (AQuAA). BMC Med Res Methodol 2009;9:58. [PubMed]
14. Arias M, Arratia A, Xuriguera R. Forecasting with Twitter Data. ACM Trans Intell Syst Technol 2015;5:8.