Hot, horny and healthy—online intervention to incentivize HIV and sexually transmitted infections (STI) testing among young Mexican MSM: a feasibility study

Introduction

Almost 180,000 people were living with human immunodeficiency virus (HIV) or AIDS in Mexico in 2019 (1), and men who have sex with men (MSM) accounted for approximately 54% of new HIV infections in 2017 (2). Nevertheless, it is estimated that between 60% and 68% of all MSM living with HIV in Mexico do not know their HIV status (3,4). Encouraging Mexican MSM to learn about, and get tested for HIV is essential not only to initiate early treatment and reduce complications related to acquired immune deficiency syndrome (AIDS) but also to avoid new infections (5).

Traditional HIV-related behavior change programs include risk reduction, information, education, and communication, and appeal to individuals’ innate desire to remain healthy and avoid infections (6). Such programs are responsible for many of the global successes in HIV prevention. However, the benefits achieved by these traditional approaches to behavior change may have plateaued, as evident in the worldwide resurgence of HIV among MSM (7,8). New strategies are needed to motivate behavior change.

HIV testing for MSM in Mexico remains a challenge, in part because of the stigma and discrimination they face in their daily lives and the perceived discrimination in health care services (9). Thus, innovative approaches are needed to increase the uptake of health prevention services in this population.

Gamification, which attempts to harness the motivational forces of games to achieve non-game objectives, is a creative and unconventional approach to behavior change. Game elements including badges, points or leaderboards, and social components such as competition or connection can be used to engage the participants into non-game activities that they might be otherwise reluctant to perform (10,11). Further, unlike other approaches to behavior change, a gamification intervention offers a unique context in which tools and mechanisms from behavioral economics and psychology such as incentives, priming, nudging, and social networks can be integrated (10).

Gamified interventions have been used in various health domains (12-14), including HIV (15). For instance, in “EpicAllies”, young black MSM living with HIV and transgendered women who have sex with men and living with HIV, create an online persona or “avatar” that gains strength and earns rewards (badges, points) when they report better antiretroviral therapy (ART) adherence (16,17). Another intervention combined financial incentives with gamification (i.e., offering points for participation redeemable for weekly raffles or showing performance graphs); which had significant effects on self-reported adherence among drug-using participants (18). A group of researchers in the United States used gamification to design an intervention to incentivize HIV and sexually transmitted infections (STI) screening among young MSM (19) and found acceptance but moderate level of engagement from the participants (20).

The aim of this study was to develop a phone-based game and linked online platform with gamification elements to incentivize HIV and STI testing and increase HIV and STI knowledge among young Mexican MSM. This paper describes the intervention development, its implementation process, and feasibility assessment.

Methods

The study consisted of three phases. The first phase was a formative research, in which we conducted focus groups and rapid prototyping to determine the most effective and appropriate design for the intervention. The second phase consisted of the piloting and implementation of the intervention, and in the third phase we conducted a feasibility evaluation of the intervention.

In this section we will first present the methodology for the first two phases. Then, we will describe the methodology of the feasibility evaluation through three dimensions: acceptability, potential demand and, the challenges of the implementation.

Formative research phase

We used a qualitative approach to design each component of the intervention. From November 2014 to February 2015 a moderator and an assistant moderator conducted six focus group discussions (FGDs) with 39 MSM aged 18–40 in Mexico City. Participants were recruited by our partner Inspira Cambio A.C., a non-governmental organization (NGO) specialized in HIV and STI prevention. The following criteria were used to select the participants: Mexican men between 18 and 35 years old, self-identifying as MSM or gay and currently living in Mexico City. After analyzing the first two FGDs, we decided to include men between 36 and 40 years old, since data suggested they also used digital platforms for dating and looking for information on sexual health. All participants provided written informed consent before starting the FGDs and all six FGDs were audio-recorded.

We conducted the process of data collection, analysis, and prototyping in three steps. For the first step, we conducted two FGDs every 2 weeks, after which we abridged transcribed the audios and conducted a discourse analysis. The second step included sending the U.S.-based design team a detailed report with the main findings. Then, within 1 week, we had a virtual team meeting in which we assessed the FGD’s findings and the potential changes according to their technical feasibility, budget, and time constraints. The third step included modifying the mock-ups and getting the prototypes ready to be tested in the following FGDs.

Because of the expertise of two members of the team regarding the use of gamification, some of the elements of the intervention were decided a priori. These elements included the use of a point system, a leaderboard, and for the game, the use of game characters or “partner profiles”.

For the first two FGDs, we used the findings of a literature review as a guide to explore subjects’ experiences towards the hookup process and the specific characteristics of their ideal sexual partners for casual encounters. For these focus groups, we used magazine clippings with more than 100 men pictures. We asked participants to select their ideal casual sexual partners, and then elicited the specific sexual partners’ characteristics (e.g., personality, body, hobbies, sexual preferences, etc.) that most appealed to participants and would lead to a successful hookup. After the first two FGDs, we identified four main profiles of ideal sexual partners, and the design team developed the first draft of mock-ups for the game characters and the platform visuals.

For the third and fourth FGDs, we validated the identified profiles, further interrogated the hookup process, and received feedback on the first version of the platform visuals. As a result, we added two more profiles and developed a more comprehensive platform, game visuals, and dynamics. For the last two FGDs, participants assessed several elements of the platform. They provided detailed feedback on key features of the intervention (e.g., points, badges, platform mechanics, etc.). After the FGDs, further decisions on the intervention’s aesthetics, mechanics, and implementation were made based on team consensus, findings from the FGDs, and consultation with our partner NGO.

During the six FGDs we also explored the mechanics of the three factors that according to Social Cognitive Theory would influence behavior: cognitive influence, socioenvironmental influence, and supporting factors (21). For the first factor, we explored what men knew about HIV, STI, and sexual practices and what they would like to learn about. For the second factor, we explored the barriers experienced when getting tested for HIV or STI, how appealing they found an intervention in which they could learn more about those topics, the common misconceptions and beliefs that surrounded the HIV test, and safe sex practices in the context of casual sexual encounters. Finally, for the third component, we tested the gamification elements and incorporated the participants’ feedback into the final version of the intervention.

For the analysis, a team of two researchers conducted discourse analysis (22). We used a discourse analysis approach to interpret the language captured in the FGDs in light of the social and cultural context (22) that surrounds MSM sexuality in Mexico. This approach allowed the intervention’s components to be relatable and genuine to our participants’ reality. For example, we identified that the word “partner” had different meanings when speaking about sexual encounters. While it could be related to a stable and romantic relationship, it was also associated with a casual sexual encounter in which the emotional connection was not important. This insight was key for the intervention because what men looked in a casual partner and how they proceeded in a casual encounter was different from a scenario where they looked for a stable partner.

We developed seven a priori codes based on several elements of the intervention: partners’ profiles, game mechanics and visuals, HIV and STI knowledge, the experience of accessing an HIV or STI test, platform mechanics, points, and leaderboard. We also used in-vivo coding since some of the phenomena analyzed were particular to the context of Mexican MSM; for example, the in-vivo code “partner”. Through the process of constant comparison across codes, we came up with four main categories, which led to the final version of the intervention: hookup process, HIV and STI knowledge, barriers around HIV and STI testing, and gamification elements.

Implementation: intervention components

The intervention consisted of four main conceptual components: gamification, information, linkage to prevention and testing services, and an online sex game.

Figure 1 outlines how the various components interacted to create one unified approach to behavior change, which can be explained using some of the major constructs for Social Cognitive Theory. According to this theory, health behavior is influenced by personal cognitive, socioenvironmental and behavior supporting factors (21). Constructs such as self-efficacy, outcome expectations, and knowledge are part of personal cognitive factors. Socioenvironmental influences include observational learning, social support, normative beliefs, physical and social barriers, and opportunities. Supporting factors can include reinforcement and punishment.

Figure 1 Intervention components and rationale behind HIV and STI testing and safer sexual behavior change.

Each of the intervention components combined several elements to influence health behavior in ways that would lead to safer sexual practices and increased HIV and STI testing.

• Gamification. Consisted of using game elements to motivate and reinforce behavioral changes in real-life situations. This was accomplished through the use of an online interactive platform that engaged participants through the use of points, badges, a leaderboard, and competition to motivate participants to pick up a sex-kit at an NGO partner and get tested for HIV, syphilis, or both.
• Information. Media and strategies used to transmit knowledge to participants. Information was provided to participants through many channels: quizzes, game feedback, WhatsApp messages, and videos.
• Linkage to prevention and testing services. The process of linking participants to services offered by an NGO partner that specializes in HIV and STI prevention, detection, and linkage to treatment services. This component also included personalized assistance by a host and the delivery of a prevention kit (sex-kit).
• Game. Component of the intervention that simulates how Mexican MSM hookup through a game downloaded to an iPhone or Android phone. The objective of the game was to negotiate a virtual sexual encounter. Players succeed by figuring out what their partner wanted, asking about serostatus, and selecting safer sex practices such as condom use.

We implemented the conceptual components through different intervention elements, which we divided into web-based activities and in-person activities. The in-person activities included participants interacting with other people in real life (inviting friends, receiving phone text messages, going to the clinic to get tested, or picking up a sex-kit). The web-based activities took place through an online platform (Figure S1) and a smartphone application which could only be accessed by the participants after signing an electronic informed consent form.

Figure S1 Online platform.

Web based intervention elements

• Leaderboard. This element ranked participants based on the total points they earned across the platform and real-life activities. It also allowed them to compare their statistics to those of other participants (Figure S2).
• Points. Participants were rewarded for online and real-world activities such as answering quizzes, playing games, inviting friends, and getting tested for HIV or Syphilis. Some examples of the point system include: 5 points for doing a quiz even with wrong answers, 150 points for inviting a friend that enrolled in the study, 500 points for going to the clinic for the sex-kit, 750 points for getting tested for syphilis and 1,250 points for getting tested for HIV.
• Badges. These were displayed on the web platform when earned for participation in activities such as registering in the platform, answering a quiz, playing the game several days in a row or a certain number of times, inviting friends and getting them to register in the platform, picking up their sex-kit, or reaching a certain amount of points. The design of the badges was based on a collection of MSM underwear (Figure S3).
• Quizzes. These were based on a set of scenarios in a chat-format with a virtual friend who had a virtual talk with the participants regarding HIV, STI, sexuality, and gay culture (Figure S4). At the end of the talk, the virtual friend asked participants a question related to the topic. Quizzes were updated every 2 days. Participants were able to choose from different predetermined answers. Regardless of the answer, the chat provided short and friendly feedback on the selected answer. After the feedback message, participants were given the opportunity to indicate whether they liked or disliked the question.
• Game. The game sought to reproduce a hookup experience while educating players and normalizing asking about serostatus and use of condom. Participants chose an avatar and a virtual character as a potential sexual partner. These characters had different profiles based on their physical appearance, personalities, hobbies and sexual practices.
• Conversation room. There was a “conversation” screen in which the participant could flirt with the virtual character. The participant could choose from a set of possible dialogs to capture the interest of the virtual character, and if he succeeded then he would go to a sex-room.
• Sex-room and information bar at the sex-room. In order to win the game, the participant had to find the combination of sexual acts sought by his virtual partner while using safer sex practices. At the top of each AVATAR there was a “health bar” and at the bottom a “pleasure bar” that went up or down depending on the sex-acts chosen by the player. There was a bar at the bottom of the sex room that showed information facts regarding HIV, STI, and sexual health. Although the game had a time limit, if the participant put the mouse cursor on the information bar, the countdown paused. There was also an “HIV button” that simulated asking for the HIV status of the participant’s virtual sexual partner, and a “condom button” that gave participants the option to virtually use a condom. When clicked, these two buttons added points to the participant score (Figure S5).
• Results screen and final feedback. After the participant reached certain score or the time ended, he would either lose or win, and a results screen would appear. This screen included feedback on the real-life risk of the sex acts chosen in the sex-room. The participant would get a score depending on the selected acts and if he elected to use or not use a condom. This information was portrayed into two faces with traffic light colors showing the participant’s and his partner’s risk (Figure S6).

Figure S2 Intervention element: Leaderboard.

Figure S3 Intervention element: Badges.

Figure S4 Intervention element: Quizzes.

Figure S5 Game: sex-room.

Figure S6 Results screen and final feedback.

In-person intervention elements

• Invite friends. Participants could invite friends by adding their e-mails in a “invite your friends” section. Their friends received the player’s code and used it when signing-up, so the person who invited them would earn points and a badge (Figures S2,S3).
• VIP-pass. By using a “VIP-pass” provided through the platform, participants could visit the clinic; be given priority for HIV and syphilis testing, and pick-up a safe-sex kit (Figure S7A).
• Host. Participants were greeted at the clinic by an attractive MSM trained in topics related to HIV, STI, and the elements of the platform. While the participants were waiting to get tested they could talk about sexual health, HIV, STI, questions regarding the platform, or any other topic they wanted to address.
• WhatsApp videos and messages. The host at the clinic sent videos and messages through WhatsApp certain days a week to all the participants. The content included explanations on how to use elements of the intervention and the same type of information contained in the quizzes regarding sexual health, HIV, STI, or getting tested.
• Sex-kit. Participants had the option to pick up a “sex-kit” at the clinic which included prevention products, such as lubricant and condoms, and a small erotic toy (Figure S7B).
• HIV and syphilis rapid tests. The participants could get tested for HIV and syphilis in the clinic that was operated by Inspira Cambio A.C. A trained volunteer provided counseling and applied the rapid test. In less than 30 minutes the participant received his results. If the participant was positive for HIV, the volunteer navigated him to a clinic to start free ART. If the participant was positive for syphilis, he was referred to a clinic to receive medical care. If the participant was negative, he would still get post-testing counseling and HIV prevention supplies such as lubricant and condoms or other information regarding HIV, STI, and sexual health.

Figure S7 VIP-pass and Sex-kit.

Point system and leaderboard design

• Point system. The point system considered the riskiness for HIV and STI implied on each of the participants’ choices and rewarded the preventive or healthy choices. The point system assumed that the average participant would play the game 20 times, answer 96 quizzes (16 per week for 6 weeks), invite a maximum of 10 friends, and would get tested only once for HIV and Syphilis. According to those assumptions, we designed the point system so that the relative contribution of the scores would be game 30%, quizzes 30%, invite friends 14%, HIV and Syphilis testing 21%, and picking up the prevention kit 5%.
• Leaderboard. The main objective of the leaderboard was that participants could compare their performance with their peers and that friendly competition would motivate increased participation.

Implementation: pilot

From October to November 2015, we piloted the intervention for five weeks among 61 MSM living in Mexico City, aged between 18 and 35 years old. Eligible men were recruited through the following approaches: peer recruitment, inviting participants from a previous study, participants from phase one of this study, engaging MSM opinion leaders to recruit among their social networks, and recruiting among existing clients of the NGO partner.

Participants were given a link to the platform, and after signing an online informed consent form, were allocated to one of two groups, with or without non-cash incentives. We did not randomize the intervention to the participants due to technical restrictions of the platform. The first participant was assigned by the platform to group one (no incentives) and the next participant to group two (non-cash incentives). This 1:1 allocation sequence was followed for the rest of the participants. In group one the participants experienced the standard intervention that included the previously described conceptual components, while in group two in addition to the standard intervention the top five winners won prizes, based on total points earned: first place an iPad, and the remaining four places gift cards ranging from 300–1,000 Mexican Pesos (17 to 56 US dollars approximately).

Feasibility evaluation

Once the pilot intervention was completed, we assessed the feasibility of the intervention over three dimensions: acceptability, demand and implementation (Figure 2), in accordance with the taxonomy proposed by Bowen et al. (23). We used a mixed-methods approach as described below.

• Qualitative approach. The week after the pilot ended we conducted semi-structured interviews with ten participants. Interviews were audio-recorded and transcribed. We developed a codebook with four codes to assess the feasibility dimensions. After reaching an inter-coder agreement of 92%, the two researchers who conducted the interviews coded the interviews using the software MAXQDA11 Plus. We then described the main topics underlying each coded segment and categorized them to further identify the main themes that were associated with each conceptual component of the intervention.
• Quantitative approach. To describe the demand dimension, we gathered data from the platform regarding the patterns of use for each conceptual component. We conducted a descriptive analysis differentiating by study group. We estimated parametric and non-parametric tests of differences in proportions, means, and medians. In order to summarize and visualize the results, we classified the feasibility of each conceptual component into one of three categories (Table 1).

Figure 2 Feasibility dimensions analysis.

Table 1 Summary of feasibility dimensions and conceptual components
Full table

We defined each feasibility dimension as follows: acceptability considers the appropriateness of the intervention, which is the perception of how appealing, relatable, and suitable each component was for the participant as an MSM or for other MSM in Mexico City. This dimension also evaluates the participants’ satisfaction, the degree to which participant’s expectations or needs were fulfilled and the enjoyment that was derived from using that component of the intervention. Acceptability also accounts for the intention to continued use, e.g., the willingness of the participant to keep using each component. Finally, improvements refer to any adjustments that participants identified as relevant for a future version of the intervention.

Demand quantitatively describes the actual patterns of use of each component, and qualitatively the determinants that led participants to use them or not. Demand also includes the initial expectations that led participants to use each conceptual component.

Implementation explores the unplanned obstacles faced by the participants in each component and the extent to which the resources used to implement them were found sufficient and pertinent. These resources were (I) “human”, people involved in participants’ real-life activities, (II) “material”, physical objects or products of the intervention components, (III) “aesthetic”, design, graphic features, and art, (IV) “mechanic”, logistics and functional mechanisms, (V) “virtual”, programming and platform administration, and (VI) “theoretical”, the main purposes or the ideas behind each component.

Results

Overall, participants in both groups had similar baseline characteristics. Men were highly educated in both groups; the majority had education level of college or higher. Also, most men had been tested for HIV during the last 12 months (80.3%). The only significant difference was age; participants that received non-cash incentives were younger than the reference group (24.1 vs. 25.7 years) (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx).

During the intervention, both groups had similar patterns of use of the platform. In both groups, participants had high acceptance of the quizzes (83.4%). About half of them played at least one game or answered at least one quiz. Men of both groups earned a similar number of badges and points. However, there was a notable difference between the maximum number of points earned by the participants from the incentives group; it was 7.5 times higher than the maximum amount of points earned by the group without incentives (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx).

Regarding the outcomes, although 12 men went to the clinic to pick up the prevention kit, only five got tested for HIV and Syphilis, and there was no significant difference between groups.

Gamification

Acceptability

Gamification elements—points, badges, prizes, competition—were seen in general as a fun, exciting and motivating by the participants (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx). Nevertheless, each element had different levels of acceptability.

The most appropriate element was the badges, whose design was considered witty and appealing. Although participants recalled the points system as a feedback mechanism of their performance (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx), they felt the points became repetitive, predictable, unchallenging, and were not considered enough incentive for them to stay engaged.

Although the element of competition was appealing for some players, in general, it had low acceptability because it did not fulfill players’ expectations regarding interaction with other players. Finally, when participants of the incentives group found themselves playing only to win the prize, they felt that the main intention behind the intervention had been lost—learning new information about HIV, STI and sexual health in a fun and appealing environment.

The main improvements suggested by the participants were to incorporate more competition, more interaction among players, more updates and new elements, increasing the level of difficulty, and reorganizing the points system to reward more exciting and fun features.

Demand

Overall, 61 participants enrolled in the platform. The median number of points earned per player was 450 (range: 0–204,000), and the mean number of badges was 2 (range: 0–9) (Table 2). When players noticed through the leaderboard that several participants stopped gaining points, they felt unmotivated to keep using the game because of lack of competition (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx).

Table 2 Socioeconomic and behavioral characteristics and patterns of use: points, badges, quizzes, game, and testing
Full table

Implementation

Unforeseen technical failures allowed us to identify the extent to which some gamification elements were important for participants. Those failures revealed that there was a negative reaction in participants’ motivation when a gamification element was defective. For example, some participants referred having faced technical problems with the point system updates and as a result, expressed demotivation and confusion (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx). In some instances, they even reached out to the support team, which was not able to solve their issues as quickly as expected. Some participants were not able to win points for inviting their friends due to technical problems, which lead participants to stop using this element (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx). Regarding badges, even though there were instructions in the platform on how to obtain them (mechanic resources), some participants were not able to find them, which lead to uncertainty and confusion.

Information

Acceptability

In general, participants perceived the platform as a good-quality information source and were satisfied to have participated in what they described as a project that aimed to create a positive change in MSM health by distributing useful information about sexual health.

The quizzes were the element of the platform that participants liked most (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx). They found the format attractive and easy to read, unlike other conventional forms like a questionnaire or plain text on a web page. Men found dialogs fun, simple, straightforward, and realistic (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx). They perceived quizzes as a reliable, anonymous, and non-judgmental way to learn more about HIV, sex, sexuality, and MSM culture. They were delighted to receive immediate feedback on their responses (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx). Despite the technical problems experienced by the players when updating the quizzes, most of them expressed their intention to keep using them if these issues were solved in the future.

Regarding other elements, participants approved receiving WhatsApp messages that reinforced specific health messages. Some participants appreciated the feedback and information about sexual practices and risks obtained through the game during the hook-up process and the virtual sexual encounter. Improvements suggested by the participants included: lower the frequency of WhatsApp messages; include more topics in the quizzes, such as sexual and human rights; and provide information for HIV positive participants.

Demand

Participants were curious about the way the platform would address topics such as sexuality, sexual health, casual sex, and HIV prevention. They expected an educational, innovative and fun discussion. Overall, 50% of participants answered at least one quiz, and the median number of quizzes per participant was four (Table 2). Technical problems had a negative effect for this element of the platform; some even caused participants to stop using a preferred element and try another one instead.

Implementation

In general, participants found the theoretical, mechanical and aesthetic resources used to develop the quizzes relevant and sufficient (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx). In contrast, virtual resources were largely insufficient due to technical challenges (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx).

Linkage to prevention and testing services

Acceptability

Most participants felt comfortable and satisfied with the HIV testing process and the service quality received at the NGO. They described the service as warm, professional, and supportive (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx). Some participants found the host fun and kind, and felt he created a comfortable environment to discuss topics related with the intervention and HIV/AIDS—while waiting for the tests results. For others, the host did not add any value to the process.

In general, participants were satisfied with the components of the sex-kit and found them useful and fun. The main suggestions were focused on adding instructions on the items contained in the kit and improving the quality of the lubricant bottles.

Demand

The VIP pass was attractive and useful for participants (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx). For several men, the sex-kit was a motivation to attend the clinic. The clinic’s schedule was as a barrier for some participants because it interfered with school or work (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx). Although 19% of the participants went to get their sex-kit, only 8% got tested for HIV and Syphilis (Table 2), this was due to not wanting to get tested or because they got tested a few days ago.

Implementation

One challenge we identified during implementation was that some participants had been recently tested for HIV or Syphilis and others had been previously diagnosed as HIV positive, which made them unable to win points for HIV testing (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx). This limitation reflected deficiencies in the mechanic resources.

Game

Acceptability

Participants were pleased by the idea behind the game and how it conveyed sexual risk. However, they found the game to be to be lacking in challenge and complexity (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx). Many players complained that the game offered few options for them to win, which limited the game’s re-playability (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx). They also indicated that the game mechanics were not intuitive.

The dialogues between the AVATARS were seen as appropriate and relatable by some participants, while others found them annoying or unrealistic. Suggestions included improving the dialogues, making the artwork more appealing and consistent, and increasing the game’s difficulty.

Demand

Almost 50% of participants played the game more than once, and the median number of games played was five (Table 2). Technical issues were such that in some cases they prevented users from being able to play the game (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx).

Implementation

The implementation of this component was hindered by technical problems. The downloading process was complicated, confusing, and lengthy (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx). In addition, participants assessed the aesthetic, mechanic, and virtual resources as insufficient and inappropriate (http://fp.amegroups.cn/cms/a61752208ea034c0523de7db113fc73c/mhealth.2020.03.01-1.docx).

Feasibility dimensions and conceptual components summary

In order to summarize and visualize the results section, we classified the feasibility of each conceptual component into one of three categories. Each category was represented by one, two, or three symbols (Table 1).

Discussion

Main results

In general, the conceptual components of the intervention were perceived as acceptable, which suggests that the formative phase captured our participants’ needs and perceptions. However, we experienced technical challenges during the execution of the intervention, which along with our participants’ high standards and expectations impacted their patterns of use.

All the conceptual components had one or several elements that entertained, motivated, or provided knowledge to the participants. The elements of the gamification component—particularly the badges, were described as entertaining and motivating. Participants highly accepted the game’s objective and rationale. The most successful element of the information component was the quizzes, because of their simplicity, realistic scenarios, topics addressed, anonymity, non-judgmental format, and immediate feedback. Participants were satisfied with the high quality of the testing services; they perceived them as fast, warm, and professional. Men were also satisfied with the sex-kit and felt motivated by the VIP pass.

On the other hand, there were aspects of each component and their implementation that demotivated or annoyed the participants. Across all the conceptual components, we experienced several technical issues related to the programming of the platform, the game, and the mobile application, which demotivated participants and made them stop using the elements. Concerning the gamification component, the point system choices did not work as expected because we assumed a higher engagement than what we achieved. Poor engagement was likely a result of the technical issues of the platform and the gamification elements lacking complexity, dynamism, and sustained challenge. Nevertheless, the leaderboard did achieve its objective because, for instance, some participants deduced through the leaderboard that other men had stopped playing because they were no longer accumulating points, and for some others, the leaderboard ignited their sense of competition to defeat their peers. However, the greater amount of points earned by the group that received non-cash incentives might reflect their higher motivation to win the price.

Finally, the main feedback on the quizzes from the information component was to improve the uploading process and to have a greater variety of topics. With respect to the linkage to prevention and testing services component, the two main observations were the opening hours of the clinic that offered the testing services, which were not always convenient for participants, and improving the quality of some elements of the kit.

Limitations

Because of the study design, we were not able to quantitatively measure the acceptability dimensions that we assessed. However, we were able to describe the quantitative patterns of use, as well as the qualitative perceptions towards each conceptual component of the intervention.

Eighty percent of the participants referred having been tested for HIV within the last 12 months prior to enrollment, which is not representative of the Mexican MSM population, where only 32–40% of those living with HIV are aware of their status (3,4). Thus, the intervention likely tapped into a population that is more informed and aware about the importance of testing for HIV. Nevertheless, we were able to test the gamification components, and capture the reactions and feedback from the participants.

Given the time and resources constraints, the testing phase of the intervention by the research team lasted less than a week. Therefore, we were not able to identify or predict several critical technical issues that emerged during the implementation phase.

The platform server collapsed one week before the planned time of implementation (six-weeks) due to a high demand from two players. In consequence, we were unable to conduct the planned follow-up. Despite the above, we were able to gather information to assess the acceptability dimensions addressed.

Comparison with prior work

Our qualitative findings suggest that gamification elements motivated participants to use the intervention components, as posited by the theory of gamification (10). Also, when the sense of competition was lost, it made some participants stop using the intervention. This matches the literature of social motivation (10) which states that a key piece of motivation is the peer comparison of performance.

While the most satisfying factors of the quizzes were the information that the participants learned and the format and aesthetic in which it was transmitted, the least favorite was the lack of sustained challenge. The approach of Ang and Rao [2004] (24) regarding the “hierarchy of the players’ needs” can explain this outcome. The approach mentions that when players’ lower levels of needs are fulfilled (e.g., understanding the rules needed to win and keep playing), they escalate to higher levels of the pyramid such as the need of esteem—feeling satisfaction while playing the game. After users fulfill those needs, they start expecting more complex challenges. The previous framework also allows explaining why the users might have stopped using the game given the lack of fulfillment of the challenge need.

Finally, other authors (18) have also reported technical issues when implementing interventions with gamification components, as well as the dissatisfaction and frustration that this lead among their participants. As Horvath et al. [2013] mention, we agree that when possible, studies developing this kind of interventions should allocate enough time and resources to pilot-test the components several weeks before the intervention is launched and address any technical issues that might hinder users’ engagement.

Conclusions

By analyzing the feasibility of the intervention with the dimensions proposed by Bowen et al. [2009], we were able to understand that an intervention with gamification elements to motivate HIV and STI testing in young Mexican MSM is feasible with certain modifications in the conceptual components.

Several factors could have improved the process of designing and implementing the intervention, starting with reducing the complexity of the intervention. Second, because of different time zones, working with a Mexico-based technical partner to develop and implement the platform would have been useful to improve communication and solve technical failures more rapidly. Likewise, since the platform was developed in Spanish and the teams outside Mexico spoke only English, the time spent in translating findings, reports, and contents could have been invested in other activities if working with Mexico-based partners. Finally, designating a testing period of at least four weeks would have been helpful to identify critical technical issues.

In order to successfully implement and scale an intervention of this kind, more time and attention must be devoted to ensuring that the technical aspects of the project are working smoothly and easy for participants to download and play. Also, a re-design of the point system would be needed to increase difficulty as the players obtain more experience. This could be calculated either by considering the total amount of points gained so far by the participant or the number of times that a participant used an element of the platform.

Additionally, the sophistication and expectations of our target audience were such that the level of challenge of the information and game components needed to be modified. They also wanted the graphic design of the whole project to be comparable to that of commercially available products. Although the latter might be difficult to achieve in public health, it is what our sample of young, urban, and highly educated MSM from Mexico City were expecting. Having learned from this experience, we implemented a simplified version of a gamification platform for MSM living with HIV in two other cities in Mexico (25) from 2017–2019, which garnered fewer complaints about the quality of the graphics.

Acknowledgments

The authors would like to thank the participants of the different phases and to Inspira Cambio A.C. for their help recruiting participants during the formative research phase and the implementation implementation phase. To Quinnipiac University Game Design Collaborative for their contribution to the game design and development. To Conducttr for their support with the development of the platform. To Reva Grimball, who participated in the formative research phase of the study, to María Alejandra Cortés Ortiz who participated in the implementation phase and to Tania Aramburo Muro for the administrative support of the project.

Funding: This study was funded by the Bill and Melinda Gates Foundation OPP1107942. The funder did not have any role in the writing of the protocol, the design of the study, data collection or any other part of the study.

Footnote

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/mhealth.2020.03.01). ZAR reports grants from Bill and Melinda Gates Foundation, during the conduct of the study. LCR reports grants from Bill & Melinda Gates Foundation, during the conduct of the study. RB reports grants from Bill and Mellinda Gates Foundation, during the conduct of the study. SBA reports grants from Bill and Melinda Gates Foundation, during the conduct of the study. The other author has no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Secretaría de Salud, Centro Nacional para la Prevención y el Control del VIH y el Sida (Censida), Dirección General de Epidemiología. Vigilancia Epidemiológica de casos de VIH/SIDA en México Registro Nacional de Casos de SIDA Actualización al Cierre de 2019 In: Epidemiología DGd, editor. 2019:23.
2. López-Castañeda M. Los derechos humanos de los hombres homosexuales, ante el VIH. In: Humanos CNdlD, editor. Ciudad de México, México: Comisión Nacional de los Derechos Humanos; 2018:28.
3. Bautista-Arredondo S, Colchero MA, Romero M, et al. Is the HIV Epidemic Stable among MSM in Mexico? HIV Prevalence and Risk Behavior Results from a Nationally Representative Survey among Men Who Have Sex with Men. PLoS One 2013;8:e72616. [Crossref] [PubMed]
4. Secretaría de Salud, Centro Nacional para la Prevención y el Control del VIH y el Sida (Censida). Informe nacional del monitoreo de compromisos y objetivos ampliados para poner fin al sida (Informe GAM) 2018.
5. Rodger AJ, Cambiano V, Bruun T, et al. Sexual activity without condoms and risk of hiv transmission in serodifferent couples when the hiv-positive partner is using suppressive antiretroviral therapy. JAMA 2016;316:171-81. [Crossref] [PubMed]
6. Munro S, Lewin S, Swart T, et al. A review of health behaviour theories: how useful are these for developing interventions to promote long-term medication adherence for TB and HIV/AIDS? BMC Public Health 2007;7:104. [Crossref] [PubMed]
7. van Griensven F, de Lind van Wijngaarden JW, Baral S, et al. The global epidemic of HIV infection among men who have sex with men. Curr Opin HIV AIDS 2009;4:300-7. [Crossref] [PubMed]
8. Beyrer C, Sullivan P, Sanchez J, et al. The increase in global HIV epidemics in MSM. AIDS 2013;27:2665-78. [Crossref] [PubMed]
9. Consejo Nacional para Prevenir la Discriminación. Encuesta Nacional sobre Discriminación en México Enadis 2010 Resultados generales. México, D.F.: Consejo Nacional para Prevenir la Discriminación 2011 June 2011.
10. Richter G, Raban DR, Rafaeli S. Studying Gamification: The Effect of Rewards and Incentives on Motivation. In: Reiners T, Wood LC, editors. Gamification in Education and Business. Cham: Springer International Publishing; 2015:21-46.
11. Deterding S, Sicart M, Nacke L, et al. Gamification. using game-design elements in non-gaming contexts. CHI '11 Extended Abstracts on Human Factors in Computing Systems; Vancouver, BC, Canada. 1979575: ACM; 2011:2425-8.
12. Miller AS, Cafazzo JA, Seto E. A game plan: Gamification design principles in mHealth applications for chronic disease management. Health Informatics J 2016;22:184-93. [Crossref] [PubMed]
13. Kato PM, Cole SW, Bradlyn AS, et al. A video game improves behavioral outcomes in adolescents and young adults with cancer: a randomized trial. Pediatrics 2008;122:e305-17. [Crossref] [PubMed]
14. Cafazzo JA, Casselman M, Hamming N, et al. Design of an mHealth app for the self-management of adolescent type 1 diabetes: a pilot study. J Med Internet Res 2012;14:e70. [Crossref] [PubMed]
15. Hightow-Weidman LB, Muessig KE, Bauermeister JA, et al. The future of digital games for HIV prevention and care. Curr Opin HIV AIDS 2017;12:501-7. [Crossref] [PubMed]
16. LeGrand S, Muessig KE, McNulty T, et al. Epic Allies: Development of a Gaming App to Improve Antiretroviral Therapy Adherence Among Young HIV-Positive Men Who Have Sex With Men. JMIR Serious Games 2016;4:e6. [Crossref] [PubMed]
17. LeGrand S, Muessig KE, Platt A, et al. Epic Allies, a Gamified Mobile Phone App to Improve Engagement in Care, Antiretroviral Uptake, and Adherence Among Young Men Who Have Sex With Men and Young Transgender Women Who Have Sex With Men: Protocol for a Randomized Controlled Trial. JMIR Res Protoc 2018;7:e94. [Crossref] [PubMed]
18. Horvath KJ, Oakes JM, Rosser BR, et al. Feasibility, acceptability and preliminary efficacy of an online peer-to-peer social support ART adherence intervention. AIDS Behav 2013;17:2031-44. [Crossref] [PubMed]
19. Mejia CM, Acland D, Buzdugan R, et al. An Intervention Using Gamification to Increase Human Immunodeficiency Virus and Sexually Transmitted Infection Screening Among Young Men Who Have Sex With Men in California: Rationale and Design of Stick To It. JMIR Res Protoc 2017;6:e140. [Crossref] [PubMed]
20. McCoy SI, Buzdugan R, Grimball R, et al. Stick To It: pilot study results of an intervention using gamification to increase HIV screening among young men who have sex with men in California. mHealth 2018;4:40. [Crossref] [PubMed]
21. Kelder SH, Hoelscher D, Perry CL. How individuals, environments, and health behaviors interact Social Cognitive Theory. In: Glanz K, Rimer BK, Viswanath K. editors. Health Behavior: Theory, Research, and Practice. 5th ed. San Francisco, California: Jossey-Bass; 2015:159-81.
22. Paltridge B. What is discourse analysis? Discourse Analysis: An Introduction. 2nd ed. Bloomsbury discourse. London UK and NY United States: Bloomsbury Academic; 2012:1-14.
23. Bowen DJ, Kreuter M, Spring B, et al. How We Design Feasibility Studies. Am J Prev Med 2009;36:452-7. [Crossref] [PubMed]
24. Ang CS, Krishna Rao R. Theories of learning: A computer game perspective. 2004.
25. Bautista S. A Habit-formation and Gamification Intervention to Improve ART Adherence Among MSM HIV Patients in Mexico (FUERTES). ClinicalTrials.gov. 2018. Accessed 01/01 2018.https://clinicaltrials.gov/ct2/show/NCT03410680?term=Sergio+bautista&cond=Hiv&rank=1