1 San Jose State University, USA
Citation:
Dao, M. (2020). Conceptualizing participatory evaluation in sport for development: A researcher’s perspective on processes and tensions from Vietnam. Journal of Sport for Development. Retrieved from https://jsfd.org/
ABSTRACT
Participatory Evaluation (PE) has been adopted as a methodology in Sport for Development (SFD); however, there is a wide scope of conceptualizing how and what a PE research process may entail. Specifically, more nuance and insight are needed regarding how PE is a formidable research process between SFD researcher and SFD organizational staff. The purpose of this paper is to illustrate the organic and planned methodological processes of conducting a PE. Hence, this study provides empirical insight into conducting a PE with an SFD project located in Vietnam. Drawing from my fieldwork, I detail the initiation of the PE, the process of establishing methods, data collection, data analysis, and results, and then comment on the researcher-to-practitioner tensions that arose. The data highlights that while participatory research (PE in this case) is increasing, there are processual considerations and limitations that need to be accounted for in the field. Inasmuch, this paper adds to pertinent methodological discussions by providing an in-depth account of PE research in SFD practice.
INTRODUCTION
Methodological negotiations and deliberations continue to be present and central to considerations for ways forward within Sport for Development (SFD) contexts (Darnell et al., 2016). SFD scholars acknowledge both the need for more descriptive and empirical based stories of SFD research (Welty Peachey et al., 2019) and the nuances of participatory based research (Hayhurst et al., 2015; Spaaij et al., 2018; Reis et al., 2015). Yet, not many have provided insights into their experiences about the complexities of attempting participatory research in SFD (see Spaaij et al., 2018). The purpose of this paper is to add to this ongoing methodological discussion in SFD.
Building on the work of Halsall and Forneris (2016) who conducted a participatory evaluation (PE) with a Right to Play program and Oatley and Harris’s (2020) depiction of stakeholder experiences of PE, this paper details what a PE research project may entail and how it may be a formidable collaborative research approach in SFD. Traditionally, participatory evaluation, like other participatory research, is usually inclusive of primary beneficiaries of a particular research setting (Cousins, 1996). However, to unpack collaborative research efforts between SFD researchers and SFD practitioners, this project centered PE from an organizational perspective. Empirical insight into the methodological process of conducting a PE with SFD project staff members in Vietnam highlights the triumphs, difficulties, and tensions of such an endeavor. Notably, the nuances of breaking down the researcher-researched power dynamics, where SFD organizational stakeholders are integrally involved in the research process from beginning to end are detailed.
Herein, writing this paper follows a moral ethics founded in participatory research broadly whereby researchers ought to share stories of the research to stimulate the senses and depict realities that occurred (Green et al., 1995; Frisby et al.,1997; Simpson, 2007). Importantly, to conform to a moral ethics of writing about participatory research (Frisby et al., 1997; Greenwood et al., 1993), I write an earnest description of what it looks like to do participatory research with an SFD project.
The remainder of this paper proceeds in four parts. First, I outline PE as an approach to research, summarizing the definition, processes, and benefits of this form of participatory research. Second, I contextualize the methodological background by focusing on discussions and studies that have implemented PE in previous SFD literature. Third, PE is conceptualized through my own experiences in Vietnam highlighting how and why PE is a useful, potentially impactful, and insightful approach to SFD research. Finally, I conclude with the implications of this methodology on SFD research and suggesting ways forward for this methodology.
PARTICIPATORY EVALUATION
Here, I discuss PE as a methodological approach that has been used to examine SFD practices. PE is a research approach within applied social science research in which various organizational stakeholders and outside researchers act in a concerted effort to share evaluation responsibilities in pursuit of both understanding and improving what is being studied (Cousins, 1996; Cousins & Earl, 1992; Fawcett et al., 2003). As a methodology, PE transforms power relations by paying attention to researcher dynamics in which ownership, agency, and ability are afforded to organizational stakeholders or local beneficiaries involved in the research (Cousins & Whitmore, 1996; King et al., 2007). Cousins and Whitmore (1996) emphasize that from an epistemological nature that the research knowledge and evaluation data are valid only when informed by practitioner perspectives. Similarly, King (2005) has stated that
participatory evaluation is an overarching term for any evaluation approach that involves program staff or participants actively in decision making and other activities related to the planning and implementation of evaluation activities. (p. 291)
Thus, PE can be suitable for research with SFD projects. Suarez-Balcazar and Harper (2003) state that PE is rooted in participatory action research (PAR), in which there is the active participation of stakeholders in the evaluation process and the shared power of critical decisions. To distinguish the different research approaches between PE and PAR that have been conducted previously in SFD, Cousins and Earl (1996) emphasize that PAR is normative and ideological in form and function, hence PAR can be abstract and loose in developing the research project (Minkler, 2000). PE, on the other hand, is formative to the development of the research project. There is a special focus on the development of participants in the research process and improvement of the organization. Thus, PAR as an umbrella concept to participatory research aims to create inclusive spaces where participants draw from their experiences to guide and inform research and present questions that they want to address in order to enact positive change (Kidd & Kral, 2005; McTaggart, 1994; Tandon, 1981). PE, however, assesses an organization’s effectiveness and discovers ways in which the organization can be improved (Cousins & Whitmore, 1996). For PE to take shape in SFD, processes should be inclusive, ranging from stakeholders delivering programs to stakeholders on the receiving end.
Process of Participatory Evaluation
Fawcett et al. (2003) describe the PE process as one in which those doing the work are aiming to understand and, most importantly, improve, a certain area of concern or challenge. The involvement of stakeholders such as staff or participants of a respected organization requires their participation in creating evaluation questions, designing the methods, participating in data collection, and reporting the findings (Daigeneault & Jacob, 2009). Since the organization (i.e., SFD project) itself has a legitimate stake in the research, to discover organizational improvements its members are deeply involved. There are five requirements that researchers and partner organizations should be aware of when conducting PE (Cousins & Earl, 1992):
- An evaluation must be recognized as important by the organization.
- The organization must have the time and resources (financial and personnel to carry out an evaluation).
- The organization ought to commit to learning about itself in the research process to engender its improvement.
- PE activities must be motivated by program beneficiaries, local partners, or an organization’s staff members.
- Appropriate training must be given to people who may not have the knowledge to carry out research.
Researchers and organizations that use PE intentionally involve different groups of people as participants in order to capture their experiences and knowledge. As a result, participants in PE differ in terms of demographics, positions, and responsibilities. Of course, levels of involvement and how people are involved in PE varies across populations (Cousins & Earl, 1992; Cousins & Whitmore, 1996). Particularly, it needs to be noted that PE between organizations differs from a PE conducted with program beneficiaries and this may depend on the problem being addressed (USAID, 2011). For example, if the aim is to uncover what hinders program implementation, then organizational staff may be involved. On the other hand, if the objective is to determine a program’s impact on local communities, then beneficiaries may be the preferred participants. Nonetheless, roles may vary, and thus participation fluxes through phases of a PE.
In the context of SFD, determining who gets to participate in a PE and why may conjure up a conundrum laden with complexities of power, privilege, and voice (Coalter, 2007; Levermore, 2011 Kay, 2012; Schulenkorf et al., 2013). Stakeholders’ interests vary (Hartman & Kwauk, 2011; Lindsey & Jeanes, 2014, and the level of involvement of stakeholders may be divergent. It is important, then, to recognize that PE goals and outcomes are not synonymous among all actors, since participants may want to improve programming while organizational staff may be motivated to obtain more funding (Cousins & Whitmore, 1996; Fawcett et al., 2003; King, 2005). This may result in conflicting and inequitable moments of participation. In this paper, the PE was between myself and the organizational stakeholders of an SFD project in Vietnam, particularly staff and government authorities.
REVIEW OF LITERATURE
Several scholars have acknowledged and applied a PE methodology to SFD research (see Halsall & Forneris, 2016; Harris, 2018; Oatley and Harris, 2020; Welty Peachey and Burton, 2017). Welty Peachey and Burton (2017) acknowledge the potential application of PE to SFD stating that PE may offer critical stakeholders an opportunity to be involved in the process and thus improve the quality of any evaluation. For example, Halsall and Forneris (2016) provide a compelling depiction of PE in a paper detailing their work with indigenous youth leaders from a Right to Play program. Utilizing photovoice methods, their PE provided results that highlight youth participants’ thoughts on experiences, successes, challenges, and opportunities for improvement regarding a youth-led community event. Of particular importance was that Halsall and Forneris (2016) facilitated a space for youth leaders to engage in leadership roles and develop research skills, such as being involved in determining the focus on their PE.
Similarly, Harris (2018) showcases a PE in which training was provided to SFD program staff in the United Kingdom to conduct their evaluations. In doing so, Harris’s participants developed knowledge and experience to engage in specific research projects. Oatley and Harris (2020) shared that due to the divergent experiences and knowledge of the stakeholders (e.g., founders, coaches, and young people), a PE design needed to be flexible where levels of involvement differed. The researchers needed to create bonds with people willing to be involved in the research and, importantly, to be more sympathetic to participants’ disagreements and knowledge. PE as shown by Hallsall and Forneris (2016) and Harris (2018), can be a useful methodology to explore impacts of SFD projects while also providing participants opportunities to develop and showcase skills. However, although Oatley and Harris (2020) provide a vivid depiction of PE experiences, they state “there is a lack of research on, and application of, PE in practice within SFD programmes” (p. 20). That said, further exploration and critique of PE across varying SFD programs is needed to capture insights and understandings to actualize PE within SFD.
There are recommendations to improve and conduct effective evaluations in SFD. For example, Welty Peachey and Burton (2017 advocate for engaging in long-term assessments. Complementing this notion, Halsall and Forneris (2016) state that their PE would have benefitted from a longitudinal time frame because of their use of photovoice methodology. Moreover, considering that Halsall and Forneris (2016) conducted a PE with the specific program Right to Play, it is unsure if these findings informed Right to Play as a major international SFD organization broadly. To extend PE discussion in SFD, it will be informative to see how PE results may inform SFD project practice. Given the nature of analysis, whether it is at the organizational level or local beneficiaries’ level, different methods are utilized in PE (Cousins & Earl, 1992; USAID, 2011); therefore, there is no one way to conduct a PE. PE research in SFD has provided rich descriptions focused on local beneficiaries (Halsall & Forneris, 2016) and organizational stakeholders’ experiences (Harris, 2018; Oatley & Harris, 2020).
Further attention to various aspects of PE that provide empirical guidance on how PE may enhance SFD practices, programs, or activities is needed. A concerted conversation about the PE process regarding how PE materializes and the intricacies of deciding the research purpose, methods, and overall experience is needed. In particular, Halsall and Forneris (2016) describe that they reached out to youth program leaders and community leaders of a specific Right to Play program to conduct a PE. Comparably, it would be interesting to explore how PE derives when an SFD organization proposes a project to an academic about conducting a PE and in doing so holds substantial expertise or authority in the process. It is thus important to continue discussions of PE, as well as other forms of participatory research, to advance and learn best practices of this methodology for SFD practice (Spaaij et al., 2018; Welty Peachey & Cohen, 2016).
Accompanying this notion, rarely has SFD research written in depth about the actual processes of researching with people from the research setting (Darnell, 2014; Spaaij et al., 2018; Reis et al., 2015; Mwaanga & Prince, 2016). Welty Peachey and Cohen (2016) call for more first-hand accounts of barriers, challenges, and successes of research partnerships between SFD scholars and organizations. In doing so, Spaaij et al. (2018) state that experiences detailing collaboration between researchers and SFD organizations can help make participatory research more common and effective in SFD research. What I offer here is an account of participatory research inclusive of the successes and challenges so that others can learn from my experience and proceed with a more inclusive and positive participatory research agenda in SFD. By doing so I add to the recommendation of Nicholls, Giles, and Sethnna (2010) about participatory research in SFD: “we would contend that what is needed is not more research, but better research” (p. 260).
PARTICIPATORY EVALUATION WITHIN SFD: A CASE STUDY OF FOOTBALL FOR ALL IN VIETNAM
Much like Puma et al. (2009) and Sanchez et al. (2010) who depict the promising aspects and shortcomings of PE community-based projects more broadly, attention is needed to the diverse application of PE. As the paper unfolds, focus is centered on the initiation of the PE, the process of establishing methods, data collection, data analysis, and results, and then finishing with a comment on tensions. Spaaij et al. (2018) note that participatory research may be challenging for junior scholars (i.e., Ph.D. students) due to time, in addition to financial and logistical constraints. With that in mind, this paper utilizes experiences in a PE research project undertaken for my Ph.D. in 2016-2017 to understand better how to implement this methodology with an SFD organization. Ultimately, future researchers can learn from my inexperience, mistakes, and insights to conduct better PE with SFD stakeholders.
Background of Football for All in Vietnam
The PE centered on Football for All in Vietnam (FFAV), a football (soccer)-based SFD project based in Huế City, Thừa Thiên Huế province, Vietnam (Thừa Thiên Huế Province has nine districts). FFAV was originally established through an agreement that was initiated by Norwegian authorities between the Football Association of Norway (NFF) and the Vietnam Football Federation (VFF) (Football for All in Vietnam, 2018). The vision of FFAV is to enable children to play football and experience life-skills education through the development of grassroots football programs across Vietnam (Football for All in Vietnam, 2018). To achieve this vision, FFAV seeks to educate parents, volunteers, and teachers and to empower local communities to create sustainable grassroots football structures (fieldnotes, December 2016).
In May 2016, a decision by NFF changed the entire landscape of FFAV. NFF, along with other Norwegian funders, implemented an exit process that would eventually cease all funding and logistical support to FFAV after December 31, 2018 (fieldnotes, August 2016). By way of this decision, leadership and decision-making processes veered from Norway to Vietnamese employees of FFAV. Ultimately, the result of the handover process was for FFAV as an SFD project to relinquish responsibility of all programs and activities to Vietnamese partners and communities by the end of 2018. Thus, in May 2016, FFAV shifted the focus of creating and implementing activities to how they can support and strengthen local capacities to continue activities. This plan provided that, from 2016 to 2018, FFAV would do everything in the project’s capacity to equip government partners and local communities in continuing activities. Hence, the PE’s primary purpose was to determine the appropriate steps in the remaining two years (2016-2018) of funding support to provide the tools and resources for sustainability in the local communities across the Thừa Thiên Huế province. Under the impression that activities in the communities would cease without funding from the local communities, FFAV decided an evaluation was necessary to gather the opinions of local communities on how best to continue activities without FFAV support. The hope was that the PE would result in actionable results/next steps.
This specific PE was about the handing off of activities to local partners, how that transition would go, and the new stakeholders’ thoughts on how this would continue. The provincial Department of Education and Training (DoET) and the district-level Sub-Department of Education and Training (Sub-DoET) were identified as authorities that would assist local communities continue activities. The goal was that, once FFAV dissolved as an SFD project and all staff went their separate ways, local communities across Thừa Thiên Huế could continue activities with their own resources and possible support from the DoET and the respected Sub-DoET. Notably, the purpose of the PE was not merely to gather information that would lead to funding but rather what skills and resources did the local communities have and would need to develop to continue FFAV activities.
Ethical Approval and Consent
Three layers of consent were needed to conduct academic research with FFAV and in Vietnam. First, with a letter of support from NFF and FFAV, ethics approval was granted by the Ethics Review Board (ERB) of University of Toronto. Support from FFAV was provided beginning in summer 2015 when I first visited the project in Vietnam to discuss the possibility of moving to Vietnam to conduct dissertation research. Second, formal consent was given by FFAV after confirmation that my dissertation proposal was approved allowing me to travel to Vietnam during my Ph.D. The vice-director notified each staff member of FFAV of my arrival and that I would be researching as a volunteer. In turn, FFAV staff consent was enlisted by the vice-director, as we would be working together in various capacities. Consequently, FFAV staff also acted as gatekeepers to local communities (see Collison et al., 2016) due to their longstanding relationships with people from the province. Third, because permission to undertake social science fieldwork in Vietnam needs governmental authorization (Bonnin 2010; Turner, 2010), formal documentation was submitted to the Vietnamese Consulate resulting in a work visa being granted. Formally, I registered with the local Thừa Thiên Huế Department of Foreign Affairs and police force in order to visit local communities.
Questions regarding my research position and power certainly warrant discussion. I must disclose that my own historical and cultural background is founded in the land of my ancestors, Vietnam. I am a Vietnamese person born in America to parents who immigrated after America’s war in Vietnam. Plummer (2001) states that conducting ethical research is based on making decisions in different situations and drawing from culture and history, and it is not a pattern of just following the rules. As someone with historical, cultural, and social connections to Vietnam, I was attentive to the hierarchical structure embedded in society. Thus, I navigated my position based on age, status, and experience. For example, while I was an academic researcher from North America, I was conscious that I was not an expert of Vietnamese life nor how research is conducted in Vietnam (see Dao, 2020 for more insight). In recognizing this, I constantly conversed with the power and privileges that I had as a researcher and outsider in the research setting (Frisby et al.,2005). Donnelly (2007), Frisby et al. (2005), and Plummer (2001) all state that researchers who contemplate their power do so by having a greater social and self-awareness of the research process, especially in communities and in relationships with participants.
The ability to engage and be open with my position in Vietnam as a person and researcher possibly created an inclusive research agenda in SFD research (Schulenkorf et al., 2020). Being aware of my position guided me in a participatory model toward ethics. I felt a social responsibility to acknowledge that my experiences differed from FFAV and to create an atmosphere for their knowledge to be represented honestly (see Manzo & Brightbill, 2007). Hence, throughout the research process, I made sure to be open, honest, and straightforward about my intentions with FFAV to establish research openly and in line with the cultural values.
FFAV Staff
All but one of the Vietnamese staff members were người Huế (a person from Huế) and most came from each of the nine districts that make up the province. Many have faced or experience the same social conditions that local FFAV program beneficiaries encounter everyday (e.g., living in rural districts or encountering gendered customs). Additionally, in Vietnamese culture, connection to one’s ancestral land is important (Lam, 2005), and there is a collective identity that binds people together and communal values that inspire people to be loyal to the place of their birth (King, 2005; Nguyen, 2016; Raffin, 2005). That said, one FFAV staff member expressed that working for FFAV was an avenue to give back to the province in which he was raised. Another staff member raised in A Lưới District (rural district bordering Laos) applied to work for FFAV because she saw how the project was creating spaces for young girls to play football. FFAV staff members’ commitment and devotion to their province reinforced their determination to assist local communities in continuing activities.
To depict the PE in this paper, all FFAV staff have been provided a pseudonym in alignment with Vietnamese respect regarding seniority and order. See table 1 for a complete list of pseudonyms and demographics of FFAV staff.
Research Process: Initiating the PE
In my preparation to conduct research with FFAV, there was no intention to assess the project. However, after my first visit to FFAV in July 2015, the landscape of FFAV dramatically shifted. Hence, in planning to move to Thừa Thiên Huế and conduct research with FFAV, senior leadership asked me to assist with their evaluation. By not formulating a concrete research plan in advance and acknowledging FFAV’s input on the research, my actions aligned with participatory research (Kidd & Kral, 2005; Mellor, 2007) and the PE was the foundation of my doctoral research. Thus, the decision to do a PE was organic and grew from the idea that FFAV was going to conduct an evaluation regardless of my involvement. But, due to my position as an outside academic, it was decided I could offer a different perspective and assist with the research. Along these lines, as an academic researcher with my research expertise, FFAV staff and I provided our distinct knowledge and experience to the research process.
Following Cousins and Earl’s (1992) five requirements of a PE, FFAV and I found that an evaluation was important due to the current handover climate. The importance of a PE resulted in coordinating and planning with FFAV staff. Meetings and discussions helped unite FFAV staff and me as a research team, allowing us to collaborate on why this PE was important and necessary and set a clear definition and plan as to how we were going to implement the PE. Fieldnotes from one of the first meetings about the PE in September 2016 described,
Today we had a meeting about the evaluation. All of the management team and myself were sat in the meeting room to discuss the evaluation of FFAV. Anh Hai facilitated the meeting and spoke for most of the time. . . . The discussion then opened to the group where everyone provided input on what they believed were relevant information to address sustainability. With Anh Hai leading the discussion, he honed in on needing to get evidence of FFAV success to show partners such as DoET and to get information on the difficulties that may prohibit local communities from continuing activities. In the end, it was determined that three areas we wanted data to inform are to: (1) assess FFAV’s impact on children and the education sector; (2) understand difficulties and challenges for FFAV from 2016 to 2018; (3) obtain recommendations for sustainability from local communities. We believed focusing on obtaining data to inform these three areas would help meet the objectives of presenting data to provincial and national partners. Another important resolution was that this evaluation is something I will take lead on and coordinate, but I am not entirely responsible, and it is not entirely about my Ph.D. With that said, FFAV is immensely invested in the research. Lastly, it was officially made known that Chi Ba and I will be working on this whole research project.
During this meeting, we coalesced as a group to create a vacillating dialogue to conclude the most appropriate areas of concern for the PE. Gibbon (2002) and Minkler (2000) state that in forms of participatory research participants must become coresearchers so a mutual gain is shared and processes of learning can occur. The more we engaged in meetings with each other and talked about sustainability and our research, the more we became connected as coresearchers. Also, in being transparent, the meeting confirmed FFAV initiated the PE; however, I would take the lead due to my research background. In the end, FFAV wanted to learn what the experiences of the communities of Thừa Thiên Huế were with FFAV programs and activities, drawing on areas in which there were positives, negatives, and room for improvement.
However, the fieldnotes above highlight a point of contention of the PE. That is, “needing to get evidence of FFAV success to show partners such as DoET” strayed from focusing on the needs and ways to assist local communities in taking ownership of activities. By doing so, the focus of the PE shifted from the local communities’ concerns to FFAV wanting to keep FFAV activities going. As a novice in participatory research, I missed this evaluation tension by focusing too much on aiding FFAV with its research endeavors.
Research Process: Methods, Data Collection, Data Analysis, and Actionable Results
The PE here was a research partnership between FFAV and me. This section explores the processes of how this relationship went about selecting appropriate methods, analyzing data, and disseminating results of a PE.
Creation of Methods. Humble and Smith (2007) advise that methods should be beneficial for enacting change and should be influenced by stakeholders. Then through discussions, and within the research parameters, the chosen methods must fit the questions and interests of the local communities (Fawcett et al., 2003). Identifying appropriate methods and frameworks ought to be done in conjunction with academic researchers and their partners (Fawcett et al., 2003; Frisby et al., 2005). To capture a wide range of data, FFAV leadership decided on province-level data collection with various stakeholder groups via semistructured interviews and focus groups.
USAID (2011) instructs that those key informants who make decisions should be interviewed in a semistructured format, while participants who have knowledge and experience of what is being evaluated are better suited for focus groups. In this PE, stakeholders included: government officials from the provincial DoET, headmasters, physical education teachers, parents, and youth participants of FFAV programs. In discussions with FFAV, we decided that semistructured interviews with government officials and community leaders (e.g., headmasters) would provide insight into what local and key Vietnamese informants perceived as successes of FFAV and barriers to maintaining activities. Interview questions varied by group (e.g., government authorities, headmasters, and physical education teachers), but they primarily centered on the perceptions of local beneficiaries regarding FFAVs influence in the province and the reasons communities could be able to continue activities without the project. For example, questions for headmasters focused on what FFAV brought to their school that may not have been present before. In contrast, focus groups with parents sought to draw out what would deter them from allowing their children to participate in activities.
Focus groups were determined to be the best method for physical education teachers, parents and young people because it would allow these groups to speak among themselves about their experiences with FFAV. Of particular importance were that these groups were those implementing activities (e.g., physical education teachers) and those identified as direct beneficiaries (e.g., young people). Thus, it was pivotal to garner the sentiments of their experiences with FFAV and specifically asked them what actions they could take to help continue activities.
Last, regarding methods, as an outsider of FFAV, my expertise was utilized through direct observations of FFAV programs and activities. Direct observations are recorded details of what is seen and heard of a program including but limited to activities, discussions, and physical surroundings (USAID, 2011). Similar to Atkinson’s (2012) conception of participant observation, through active participation as a volunteer with FFAV and assisting with the PE, direction observations occurred as I paid attention to interactions, comments, and expressions during FFAV SFD activities. This process allowed for documentation of criticism as well as supportive moments of FFAV.
Admittedly, while I was able to assist with creating interview guides and write reports or fieldnotes, FFAV staff demonstrated their knowledge by providing input and educating me about how to conduct research in Vietnam. Chi Ba was one person whose advice and experience researching Thừa Thiên Huế was worth much more than, I, the “trained researcher” could provide. Reverting to preparing the PE, I certainly did not have the experience and knowledge of the everyday lived reality in Thừa Thiên Huế that FFAV staff held. Many times, I witlessly went about the research in an academic mindset. For example, when drafting interview guides, I often used academic jargon challenging to translate to Vietnamese. In correcting me, Chi Ba advised keeping the questions straightforward so participants could understand the questions and answer coherently. Here are fieldnotes from a December 2016 meeting between Chi Ba and me where we discussed the interview guides:
Chi Ba is so good at this stuff. She is definitely taking part in the participatory nature of this research. Her comments on the interview guides are great. She has knowledge and experience that is making this research process and evaluation that much better. It’s insightful when she is able to comment. Being able to work with her through these things is something I believe participatory research needs. Her knowledge and experience in FFAV, while also being a participant in my study is something greatly needed in SFD research. Just got some focus group guides back from Chi Ba and she is way better at facilitating these things compared to me. She’s got so much experience in all of this it’s crazy.
As depicted in this excerpt, this specific PE could not have been carried out without the knowledge and experience of FFAV staff. Chi Ba’s experience was undeniably crucial to how the PE was conducted. This example reflects how FFAV staff and I learned from each other throughout the project (Greenwood et al., 1993; Ponic et al., 2010; Scott et al., 2005). Mostly as a less knowledgeable outsider to FFAV, it was essential that FFAV staff led the PE and that I provided input when necessary. This dynamic helped us be open about appreciating each other’s knowledge and experience, leading to mutual learning moments. Indeed, the PE and participatory research developed due to Chi Ba’s willingness to help and provide her knowledge.
Data Collection. Spaaij et al. (2018) note that no two participatory projects are alike, meaning every instance and experience of data collection within a PE will occur differently, based on disparate needs, resources, and settings. PE as a methodology is constantly adapting to be inclusive of the construction of research questions and methods, as well as to create spaces to talk about the research issues and process. The PE here had us embark on trips to each district of Thừa Thiên Huế where each trip presented different encounters; thus, it was important to have interchangeable data collection processes and flexible schedules.. My fieldnotes from our November 2016 trip to Nam Đông district describe the situation:
We get to Nam Đông around 8:30 am and the leader of the SuB-DoET in Nam Đông, at a coffee shop where we meet. . . . At 9:15 we go to the first school and FFAV Club of the day, Huong Huu primary school. We get to the school and are greeted by the headmaster and a FFAV coach. We walk to the headmaster’s office where there is a large rectangular table with water and tangerines for us to enjoy. Introductions are made by physical expert introducing the school’s staff, FFAV staff, and the local parents. Anh Ba from FFAV formally introduces why we are doing the evaluation to the school and thanking everyone for their time.
Anh Ba makes his remarks about why we are doing this evaluation to the school and thanking them for their time. . . . It’s decided that I along with the volunteer who is helping me translate will interview the headmaster, Chi Ba and Em Hai will do the focus group with the children, Anh Ba will interview the coaches, and Anh Tam will interview the parents. . . . The interviews start as people go with their groups to different locations.
Illustrated above was a routine data collection trip consisting of introductions and a schedule of events where local beneficiaries were advised of the purpose and how the PE would occur. Data collection trips consisted of a multitude of moving parts and potential chaos. Plans were disrupted when someone from the local community was late or did not have the adequate time devoted to the PE. In learning from these experiences, as a research team, we learned to adapt the plans at the moment, conduct interviews in whatever spaces were open, and ensure everyone felt comfortable and accommodated. Car rides after data collection trips were where we often decompressed and conversed about the day’s research. Fieldnotes from after the November 2016 PE trip to Nam Đông district describe these conversations:
We finished data collection in Nam Đông around 4 PM and were back on our way to Hue City. 5 of us total in a little sedan driving through the central highlands. Anh Ba jokes that he wants to go to market. He is known as “Mr. Market” because everywhere he goes, he likes to visit the local market of a district or community to see if they sell anything special (e.g., fruits, produce, or clothes). I asked how today went for everyone. There’re some murmurs of it was fine and ok. Chi Ba was more vocal and said that we have a schedule but sometimes the local community members cannot abide by these times. So future research trips need to be flexible. “Em sua” said her focus group with the children from the secondary school was fun because they were very vocal and nonchalantly were not worried about activities continuing. This was interesting because they seemed to acknowledge that they could continue activities on their own. Right here, Chi Ba said we should start asking older youth how they would sustain activities in future trips. We all agreed that would be smart.
As illustrated in the fieldnotes, there were natural components of debriefing and talking about the research process. By eliciting the epistemology on researching with FFAV, the fieldnotes illustrate how this research project was able to collaboratively and effectively set up data collection processes. Insight into organization and car rides home may allow other SFD researchers embarking on PE with program staff to envision better how a PE can occur.
While there were many moving parts to data collection trips, special attention is focused on conducting semistructured interviews with an FFAV intern. Morrell (2008) and Romero et al. (2008) describe that education and learning processes increase when participants are active and contribute to the research. In the field, everyone involved was capable of conducting this PE because it allowed spaces for creating new knowledge and gaining valuable experiences (Veugelers, 2017). An example of this was when one of the FFAV interns who helped me with an interpreter started to ask probing questions during a November 2016 interview with a headmaster of a primary school where FFAV activities are housed:
Having Em Ba as a translator has been an amazing experience because I can see her growing and learning methods by conducting the interviews with me. Throughout the day she also probed and asked other questions that she saw as relevant during the interviews. As I said before eventually PhDs from abroad should probably not doing this kind of research and it should be done by locals. Maybe this is a step to doing something that will create that local research agenda.
Em Ba being able to participate in an inclusive research space possibly allowed her to feel comfortable conducting the interview on her own. I was both surprised and overjoyed once she took over and probed the interviewee because she was usually quiet. So, while my position was the academic researcher, I was certainly not the only person capable to conduct an interview. Em Ba’s ability to exhibit her research abilities at the moment emphasizes how this PE data collection allowed for FFAV staff to provide their expertise and knowledge.
Data Analysis. Being the “trained researcher” of the PE, I was primarily responsible for data analysis, in which a thematic analysis was undertaken to draw out common perceptions of participants (Smith & Bryan, 2004). Data analysis in participatory research may span the spectrum from more traditional models where only one person (usually the academic researcher) conducts the analysis alone or where some, but not all coresearchers are able or willing to help with the analysis (Cahill, 2007). Thus, similar to Reid et al. (2006) who utilized member-checking in their PAR project, FFAV staff members were regularly checked to clarify what was emerging from the data.
Through a thematic analysis, themes emerged inductively through constant coding of data (Gratton & Jones, 2004; Smith & Bryan, 2004). Braun et al. (2016) outline six phases of thematic analysis ranging from familiarization and coding of data; theme development, refinement and naming; and writing up and results. The familiarization and coding of data, as well as theme development and refinement were two specific phases in which this PE consisted of moments of participatory data analysis. First, I immersed myself in the data to familiarize myself with the most common codes and themes. As I read through the data, I noted common themes that appeared among the different participant groups. In addition to indicating common themes in the data, I highlighted different specific quotes, moments, and notes that exemplified these themes. Following Torre et al. (2007) who said everyone should be encouraged to participate in the data analysis was co-constructed with FFAV. Hence during this first step, FFAV staff members were often asked what they thought about themes emerging from the data analysis, and they would inform me of their perspective or clarify perceptions. For example, clarification was needed from a staff member about their notes on if physical education teachers wanted less training from FFAV or whether physical education teachers were concerned about how money from FFAV was being distributed among the province’s clubs. He clarified that some physical teachers wanted less training from FFAV on grassroots football and would rather have the funding for these training be distributed to support their FFAV football clubs.
In turn and aligned with Braun et al.’s (2016) phase of theme development and refinement, I coded and created labels for data that evoked connections to the purpose of the PE. As themes developed and were constantly being refined, I made sure to review the emerging themes with FFAV staff. Chi Ba was one specific person whom I turned to for peer debriefing (Lincoln & Guba, 1985). While Chi Ba was not necessarily a disinterested peer, as should be in peer debriefing, her removal from the data analysis process allowed her to guide me and inquire about the emerging themes. The constant discussion of themes with FFAV staff culminated once it was perceived that data saturation was complete.
Unfortunately, data analysis yielded tensions around how data and notes were collected. Since we did not audio-record interviews with local participants, as FFAV staff advised that recording participants may cause them to be wary, data were collected via handwritten notes. Reflecting on the process as a collective group, we did not have clarity on how to write down data or take fieldnotes. The lack of clarity on how to collect data would prompt me to ask FFAV staff for more clarity on their notes and, at times, become frustrated when writing up the analysis. This is depicted in my fieldnotes from December 2016:
I hate to say it but some of the interview data that I’m looking at are just bullet points. There’s no substance to the data where I can sift through and try and find some themes. But I got to go with it. It’s part of the research process that we are doing. This was done in a participatory manner and so it goes that data is the way it is.
Reflecting on the process of the PE, I realize that although FFAV staff and I had discussed what methods to use and how to conduct interviews, we had not discussed how the data would be collected. In turn, the PE process could have used more discussion on the best way to capture data. Specifically, since I was the one writing the report, I could have been more direct on what approach to data collection would allow me best to balance authority and expertise in PE research (Daigneault & Jacob, 2009). Further conversations about best practices for data collection were needed to ensure clarity and consistency.
Actionable Results. As noted by Cousins and Earl (1992), a main outcome of PE is to produce results that would lead to practical actions for an organization to implement. Thus, the data collected for this PE revealed broadly that government officials were reluctant about assuming the responsibility of FFAV programs and activities and that overall local communities believed they could sustain activities without FFAV but had qualms over needing the training to be self-sufficient. For example, participants from local communities generally specified they did not have the resources, nor did they feel comfortable about fundraising for their FFAV football clubs. Headmasters, teachers, and parents, who understood that they would shoulder the responsibilities, requested that FFAV provide training to develop skills in fundraising and marking. Thus, FFAV visited each district to deliver training to local stakeholders on how to fundraise. Fieldnotes from a May 2017 fundraising workshop describe this situation:
In order to build the capacity and inspire local communities to take on the challenge of self-financing their activities, the fundraising team has worked hard at bringing new methods to engage participants. There is a session that asks the participants to break into groups and discuss the challenges to fundraising in their communities, as well as ways to overcome these fundraising challenges. . . . Some challenges brought forth in the workshops include areas of districts are not financially stable, very little corporations to interact with, and people do not have experience in fundraising methods; thus, to remedy the lack of experience the fundraising team created role-playing scenarios. Role-playing helps position the participants into scenarios where they may be seeking money. For example, participants are asked to read letters of financial support to potential donors then after the whole group discusses any mistakes, weaknesses or improvements that they saw. . . . As many participants have no experience in fundraising it is important for them to have exposure in writing letters, approaching potential donors, and to be familiar on how to handle challenges.
While the specific purpose of this paper was on the process of conducting and implementing a PE, it is important to bring attention to what came about in this PE. FFAV realized that they were asking local communities to continue activities; however, adults from local communities indicated they would need proper guidance on topics such as fundraising, as the different districts face different challenges. Moreover, to present this data, in staying with participatory modes of disseminating results in a public sphere (Fine, 2016; Frisby et al., 1997), working with Chi Ba I helped write a report of the PE to be presented to provincial authorities. With the occasional peek over to speak with Chi Ba, who sat two desks away from me, this report was the culmination of the PE. This project started with me not knowing a PE was going to be part of my Ph.D. research, to going through the process of conducting research with FFAV, to finishing with a report.
Research Tensions
As described in the processes of data analysis, there were often times of discomfort and tension, and there were meetings where authority was positioned. It would be disingenuous to write and not acknowledge the uncomfortable moments. The most tense-filled research experience occurred during the second meeting I had with the vice-director of FFAV. I was talking about ethnography and participant observations and how doing these methods would benefit “my own data.” She stopped me mid-sentence and said, “Michael, this is too much about your research than it is about the FFAV evaluation.” In unthinkable fashion during my first week, I had put forward the attitude of an academic know-it-all. My fieldnotes from that September 2016 meeting reflect my feelings:
I messed up today. I came in big with my academic background. The biggest event that happened today was my conversations with (FFAV vice-director) about the areas of focus and research for the evaluation. She had asked me to talk about methods and I mentioned ethnography and how on my own I was collecting ethnographic data for my own data. I explained that I was going to use this as my own data in writing about the research process. I think we got into a clear disagreement that got into a rather heated discussion about what needs to be done in regard to the evaluation. I definitely came across more concerned about my dissertation and she even said that I was speaking too much about my own research than the FFAV evaluation. I really don’t know what I’m doing or if I belong here.
Reflecting on this moment, in the beginning stages of this PE I needed a moral and ethical engagement to FFAV. Regrettably, the focus was on research that would produce a dissertation neglecting the needs of FFAV and local stakeholders. Overall, in participatory research, let alone PE, there needs to be a commitment to the collaborators in research for actionable change (Barab et al., 2004; Mellor, 2007). At this moment, my use of academic language distanced me from the people with whom I was conducting research, and instead I needed to centralize their words and reality to demonstrate the current situation (Torre et al., 2007). Gratefully, this meeting occurred during the beginning of my research with FFAV. Throughout the next 10 months, I became more aware of my positionality by reflecting on my fieldnotes and by having honest conversations with FFAV staff. I caution others that before they enter a research space to be aware of their position and power, as well as to begin thinking of the research “space” as a place that many people occupy (Atkinson, 2012; Barab et al., 2004; Mellor, 2007). Once a researcher enters a space where people are living, the space changes and becomes a place where they work with others to recreate the world that is studied (Fine et al., 2003). When doing any kind of participatory work, there will always be tension. Accordingly, aligned with Warhman and Zach (2016), SFD researchers need to take risks and become more engaged within the research setting to have an emic approach informed by those whose questions need answers.
DISCUSSION
PE has become an increasingly useful methodology in SFD. In this paper, I highlight a different perspective of PE by focusing on a research relationship between myself and FFAV. The insights provided detail the first-hand account of participatory research with an SFD project (Welty Peachey & Cohen, 2016) by specifically focusing on the PE methodological process. Acknowledging the rich depictions of PE provided by Halsall and Forneris (2016) and Oatley and Harris (2020), consideration must be paid to how PE is conceptualized in different ways with different people. In turn, the experience serves as a reminder that individuals, especially novices, doing participatory research may encounter obstacles and uncertainty (Spaaij et al., 2018). More stories that exemplify the complex nature of PE are needed to improve the facilitation of this methodological process in SFD research.
Illustrated here are a researcher’s perspectives on initiating a PE, the creation of methods and data collection, and tensions that arose during research; participatory research can be a difficult but worthwhile endeavor. Similarities are drawn to both Halsall and Forneris (2016) and Oatley and Harris (2020), who highlight the ability and knowledge of stakeholders in the research process. In contrast, the brief results indicate that the PE motivated FFAV to provide varied training that may assist local communities in continuing activities. However, as exemplified, in PE, tensions stem from what groups are involved and the constant shifts in power and agency by those involved in the research. For example, the PE at hand was a research relationship between myself and FFAV; thus, the research process was primarily centered on the SFD organizational level.
Two important caveats must be made. One is that even though this PE was intended to gather data that FFAV would use to help local communities sustain FFAV activities once the handover was complete, local community members (e.g., headmasters, teachers, parents, and youth participants) were excluded from the PE process. This potentially leads to a discrepancy between the actions of FFAV and the needs of local communities. This also leads to the critique that FFAV staff (myself included) were more concerned with the legacy of the project instead of the lived reality that may deter local communities from continuing activities on their own. Second, this paper was written from the perspective of the researcher. Future papers of PE should be written with organizational staff to depict clearly the research relationship.
Broadly, reverting to implications in SFD research, this paper is intended to highlight how PE and participatory research can be done. That is to say, while participatory research has been suggested and carried out (see Burnett, 2008; Hayhurst et al., 2015; Reis et al., 2015; Spaaij, 2009 Whitley et al., 2014) there is still room to improve and conduct better research (Nicholls at al., 2010). First-hand commentary into the research partnership between FFAV and myself may allow for future researchers to acknowledge the challenges and successes they may face with an SFD organization (Welty Peachey & Cohen, 2016). One specific research suggestion is to embark on an all-inclusive PE similar to Hall and Oatley (2020) that specifically examines the relationship between SFD project and SFD beneficiaries. Indeed, to conceptualize and unpack this direct relationship in PE may yield enlightening insight to how PE can be conducted between SFD organizational staff and the intended beneficiaries. As well, an SFD PE project that directly examines sustainability and ownership of activities within local communities is greatly needed. A research project of such importance will provide insight to how and why certain locales aim to undertake SFD activities from their partners.
CONCLUSION
The purpose of this article was to illustrate how one may conduct participatory research, and by extension evaluation, with an SFD project. For Tandon (1981) and Mellor (2007), the main purpose of participatory research is to increase knowledge about a localized social setting with the hope and goal for positive change to occur. In utilizing my experiences with PE as a place of departure and argument, this paper outlines PE and the potential and difficulties of this method as one particular option that can be utilized in SFD research, both as an answer for the many calls for increased participatory research and as an example of the power and potential of collaboration and inclusion of often silenced voices. In practice, the application of PE can be utilized by individual SFD projects and organizations, as outlined by their challenges and needs. The potential for academic research and SFD practice to converge in PE can undoubtedly lead to benefits that will speak to both sides, presenting data that can answer a diverse array of questions and concerns.
ACKNOWLEDGEMENTS
Deepest gratitude to the two anonymous reviewers for their supportive comments in getting this article in publication form. Much love to Adam Ali, Adaurennaya Onyewuenyi, Katie Houston, and Patricia Posey for their insightful comments. Thank you to FFAV for allowing me into their project and allowing me to conduct research with them. It is because of the knowledge and determination of FFAV that this participatory evaluation was allowed to be carried out.
REFERENCES
Atkinson, M. (2012). The empirical strikes back: Doing realist ethnography. In K. Young & M. Atkinson (Eds.), Qualitative Research on Sport and Physical Culture (pp. 23–49). Emerald Group.
Barab, S. A., Thomas, M.K., Dodge, T., Square, K., & Newell, M. (2004). Critical design ethnography: Designing for change. Anthropology & Education Quarterly, 35(2), 254–268. https://doi.org/10.1525/aeq.2004.35.2.254
Bonnin, C. (2010). Navigating fieldwork, practicalities and ethics in the upland borderlands of Northern Vietnam. Asia Pacific Viewpoint, 51(2), 179–192. https://doi.org/10.1111/j.1467-8373.2010.01423.x
Braun, V., Clarke, V. & Weate, P. (2016). Using thematic analysis in sport and exercise research. In B. Smith & A. C. Sparkes (Eds.), Routledge handbook of qualitative research in sport and exercise (pp. 191–205). Routledge.
Burnett, C. (2008). Participatory action research (PAR) in monitoring and evaluation of sport for-development programmes: Sport and physical activity. African Journal for Physical Health Education, Recreation and Dance, 14(3), 225–239.
Cahill, C. (2007). Participatory data analysis. In S. Kindon, R. Pain, & M. Kesby (Eds.), Participatory action research approaches and methods: Connecting people, participation and place (pp. 181–187). Routledge.
Cameron, J. & Gibson, K. (2005). Participatory action research in a poststructuralist vein. Geoforum, 36(1), 315–331. https://doi.org/10.1016/j.geoforum.2004.06.006
Coalter, F. (2007). A wider social role for sport: who’s keeping the score? Routledge.
Collison, H., Giulianotti, R., Howe, D. P, Darnell, S. (2016). The methodological dance: Critical reflections on conducting a cross-cultural research project on “Sport for Development and Peace.” Qualitative Research in Sport, Exercise and Health, 8(5), 413–423. https://dx.doi.org /10.1080/2159676X.2016.1206610
Cousins, J. B. (1996). Consequences of researcher involvement in participatory evaluation. Studies in Educational Evaluation, 22(1), 3–27.
Cousins, J. B. & Earl, L. M. (1992). The case for participatory evaluation. Educational Evaluation and Policy Analysis, 14(4), 397–418. https://doi.org/10.3102/01623 737014004397
Cousins, J. B. & Whitmore, E. (1996). Framing participatory evaluation. New Directions for Evaluation, 80(1), 5–24. https://doi.org/10.1002/ev.1114
Daigneault, P. M. & Jacob, S. (2009). Toward accurate measurement of participation: Rethinking the Conceptualization and Operationalization of Participatory Evaluation. American Journal of Evaluation, 30(3), 330–348. https://doi.org/10.1177/1098214009340580
Dao, M. (2020). “Nước Việtnam là Quê Hương Của Tôi”: historical, political and sociocultural implications of the Vietnamese-American researcher in sport for development. Qualitative Research in Sport, Exercise and Health. https://doi.org/10.1080/2159676X.2020.1753804
Darnell, S. C. (2014). Ethical challenges and principle hurdles in sport-for-development work. In K. Young & C. Okada (Eds.), Sport, social development and peace: Vol. 8. Research in the sociology of sport (pp. 1–18). Emerald Group.
Darnell, S. C., Whitley, A. M., & Massey, V. W. (2016). Changing methods and methods for change: Reflections on qualitative research in sport for development and peace. Qualitative Research in Sport, Exercise and Health, 8(5), 571–577. http://dx.doi.org/10.1080/2159676X.2016.1214 618
Donnelly, J. (2007). Development ethics and moral engagement: Some political and ethical dilemmas in development research. In M. Smith (Ed.), Negotiating boundaries and borders: Vol. 8. Studies in qualitative methodology (pp. 107–129). Emerald Group.
Fawcett, S. B., Boothroyd, R., & Schultz, J. (2003). Building capacity for participatory evaluation for community initiatives. Journal of Prevention & Intervention in the Community, 26(2), 21–36. https://doi.org/10.1300/J005v26n02_03
Fine, M. (2016). Just methods in revolting times. Qualitative Research in Psychology, 13(4), 347–365. https://doi.org/10.1080/14780887.2016.1219800
Fine, M., Torre, M. E., Boudin, K., Bowen, I., Clark, J., Hylton, D., Martinez, M., Missy, Roberts, R. A., Smart, P., & Upegui, D. (2003). Participatory action research: From within and beyond prison bars. In P. M. Camic, J. E. Rhodes, & L. Yardley (Eds.), Qualitative research in psychology: Expanding perspectives in methodology and design (p. 173–198). American Psychological Association. https://doi.org/10.1037/10595-010
Football for All in Vietnam. (2018, November 6). http://www.ffav.com.vn/wpcontent/uploads/2016/07/2016/application/pdf/FFAV-Project-description.pdf
Frisby, W., Crawford, S., & Dorer, T. (1997). Reflections on participatory action research: The case of low-income women accessing local physical activity services. Journal of Sport Management, 11(1), 8–28. https://doi.org/10. 1123/jsm.11.1.8
Frisby, W., Reid, C. J., Millar, S. & Hoeber, L. (2005). Putting “Participatory ” into participatory forms of action research, Journal of Sport Management, 19(1), 367–386. https://doi.org/10.1123/jsm.19.4.367
Gibbon, M. (2002). Doing a doctorate using a participatory action research framework in the context of community health. Qualitative Health Research, 12(4), 546–558. https://doi.org/10.1177/104973202129120061
Gratton, C. & Jones, I. (2004). Research methods in sport studies. Routledge.
Green, L. W., George, M. A., Daniel, M., Frankish, C. J., Herbert, C. P., Bowie, W. R., & O’Neill, M. (1995). Study of participatory research in health promotion: Review and recommendations for the development of participatory research in health promotion in Canada. Royal Society of Canada.
Greenwood, D. J., Whyte, W.F., Harkavy, I. (1993). Participatory action research as a process and as a goal. Human Relations, 46(2), 175–192. https://doi.org/10.1177/ 001872679304600203
Halsall, T. G. & Forneris, T. (2016). Behind the scenes of youth-led community events: A participatory evaluation approach using photovoice in a Canadian First Nation community. Youth Engagement in Health Promotion, 1(2), 1–40. https://jps.library.utoronto.ca/index.php/youthengage/ article/view/0203
Harris, K. (2018). Building sport for development practitioners’ capacity for undertaking monitoring and evaluation: Reflections on a training programme building capacity in realist evaluation. International Journal of Sport Policy and Politics, 10(4), 795–814. https://doi.org/ 10.1080/19406940.2018.1442870
Hartmann, D. & Kwauk, C. (2011). Sport and development: An overview, critique, and reconstruction. Journal of Sport and Social Issues 35(3), 284–305. https://doi.org/10.1177 /0193723511416986
Hayhurst, L. M. C., Giles, A. R., Radford, W. M., & The Vancouver Aboriginal Friendship Centre Society. (2015). I want to come here to prove them wrong: Using a post-colonial feminist participatory action (FPAR) approach to studying sport, gender, and development programmes for urban Indigenous young women. Sport in Society, 18(8), 1–16. https://doi.org/10.1080/17430437.2014.997585
Humble, D. & Smith, M. (2007). What counts as development research? In M. Smith (Ed.) Negotiating boundaries and borders: Vol. 8. Studies in qualitative methodology (pp. 13–34). Emerald Group.
Jeanes, R. & Lindsey, I. (2014). Where’s the “evidence?” Reflecting on monitoring and evaluation within sport-for-development. In K. Young & C. Okada (Eds.), Sport, social development and peace, (pp. 197–217). Emerald Group.
Kay, T. (2012) Accounting for legacy: Monitoring and evaluation in sport in development relationships. Sport in society: Cultures, commerce, media, politics, 15 (6), 888–904. https://doi.org/10.1080/17430437.2012.708289
Kidd, S. & Kral, M. (2005). Practicing participatory action research. Journal of Counseling Psychology, 52(2), 187–195. http://dx.doi.org/10.1037/0022-0167.52.2.187
King, J. A. (2005). Participatory evaluation. In S. Mathison (Ed.), Encyclopedia of evaluation (pp. 291–294). Sage.
King, J. A., Cousins, J. B., & Whitmore, E. (2007). Making sense of participatory evaluation. New Directions for Evaluation, 114(1), 83–115. https://doi.org/10.1002/ev.226
Lam, A. 2005. Perfume dreams: Reflections of the Vietnamese diaspora. Heyday Books.
Levermore, R. (2011). Evaluating sport-for-development: Approaches and critical issues. Progress in Development Studies, 11(4), 339–53. https://doi.org/10.1177/14649934 1001100405
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Sage.
Manzo, L. C. & Brightbill, N. (2007). Toward a participatory ethics. In S. Kindon, R. Pain, & M. Kesby (Eds.), Participatory action research approaches and methods: Connecting people, participation and place (pp. 33–40). Routledge.
McTaggart, R. (1994). Participatory action research: Issues in theory and practice. Educational Action Research, 2(3), 205–222. https://doi.org/10.1080/0965079940020302
Mellor, M. (2007), Researching for change. In M. Smith (Ed.) Negotiating boundaries and borders: Vol. 8. Studies in qualitative methodology (pp. 177–195). Emerald Group.
Minkler, N. (2000). Using participatory action research to build health communities. Public Health Report, 115(2–3), 191–197. https://doi.org/10.1093/phr/115.2.191
Morrell, E. (2008). Six summers of YPAR: Learning, action, and change in urban education. In J. Cammarota & M. Fine (Eds.). Revolutionizing education: Youth participatory action research in motion (pp. 155–184). Routledge.
Mwaanga, O. & Prince, S. (2016). Negotiating a liberative pedagogy in sport development and peace: Understanding consciousness raising through the Go Sisters programme in Zambia. Sport, Education and Society, 21(4), 588–604. https://doi.org/10.1080/13573322.2015.1101374
Nicholls, S., Giles, A., Sethna, C. (2010). Perpetuating the “lack of evidence” discourse in sport for development: Privileged voices, unheard stories and subjugated knowledge. International Review for the Sociology of Sport, 46(3), 249–264. https://doi.org/10.1177/1012690210378273
Nguyen, D. Q. (1994). Where the ashes are: The odyssey of a Vietnamese family. University of Nebraska Press.
Nguyen, V. T. (2016). Nothing ever dies: Vietnam and the memory of war. Harvard University Press.
Oatley, C. & Harris, K. (2020). How, and why, does participatory evaluation work for actors involved in the deliver of a sport-for-development initiative? Managing Sport and Leisure. Advance online publication. https://doi.org/10.1080/23750472.2020.1735942
Plummer, K. (2001). Documents of life 2. SAGE Publications.
Ponic, P., Reid, C., & Frisby, W. (2010). Cultivating the power of partnerships in feminist participatory action research in women’s health. Nursing Inquiry, 17(4), 324–335. https://doi.org/10.1111/j.1440-1800.2010.00506.x
Puma, J., Bennett, L., Cutforth, N., Tombari, C., & Stein, P. (2009). A case study of a community based participatory evaluation research (CBPER) project: Reflections on promising practices and shortcomings. Michigan Journal of Community Service Learning, 15(2), 34–47. http://hdl.handle.net/2027/spo.3239521.0015.203
Raffin, A. (2005). Youth mobilization in Vichy Indochina and its legacies, 1940-1970. Lexington Books.
Reid, C., Tom, A., & Frisby, W. (2006). Finding the “action” in feminist participatory action research. Action Research, 4(3), 315–332. https://doi.org/10.1177/14767503 06066804
Reis, C. A., Viera, C. M., de Sousa-Mast, F. R. (2015). “Sport for Development” in developing countries: The case of the Vilas Olimpicas do Rio de Janeiro. Sport Management Review, 3(17), 1–13. https://doi.org/10.1016/j. smr.2015.01.005
Romero, A., Cammarota, J., Dominguez, K., Valdez, L., Ramirez, G., & Hernandez, L. (2008). “The opportunity if not the right to see”: The social justice education project. In J. Cammarota & M. Fine (Eds.). Revolutionizing education: Youth participatory action research in motion (pp. 131–151). Routledge.
Sanchez, V., Carrilo, C., & Wallerstein, N. (2011). From the ground up: Building a participatory evaluation model. Progress in Community Health Partnerships: Research, Education, and Action, 5(1), 45–52. https://doi.org/ 10.1353/cpr.2011.0007
Schulenkorf, N., Sugden, J., & Burdsey, D. (2013). Sport for development and peace as contested terrain: Place, community, ownership. International Journal of Sport Policy and Politics, 6(3), 1–18. https://doi.org/10. 1080/19406940.2013.825875
Schulenkorf, N., Edwards, D., & Hergesell, A. (2020). Guiding qualitative inquiry in sport-for-development: The sport in development settings (SPIDS) research framework. Journal of Sport for Development, 8(14), 53–69. https://jsfd.org/2020/05/01/
Simpson, K. (2007). Hearing voices? Negotiating multiple ethical commitments in development research. In M. Smith (Ed.), Negotiating boundaries and borders: Vol. 8. Studies in qualitative methodology (pp. 155–173). Emerald Group.
Scott, S., Miller, F., & Lloyd, K. (2005). Doing fieldwork in Development Geography: Research culture and research spaces in Vietnam. Geographical Research, 44(1), 28–40. https://doi.org/10.1111/j.1745-5871.2006.00358.x
Suarez-Balcazar, Y. & Harper, G.W. (2003). Community-based approaches to empowerment and participatory evaluation. Journal of Prevention & Intervention in the Community, 26(2), 1–4. https://doi.org/10.1300/J005v 26n02_01
Smith, P. & Bryan, K. (2004). Participatory evaluation: Navigating the emotions of partnerships. Journal of Social Work Practice, 19(2), 195–209. https://doi.org/10.1080/0 2650530500144881
Spaaij R. (2009). The social impact of sport: Diversities, complexities and contexts. Sport in Society, 12(9), 1109-–1117. https://doi.org/10.1080/17430430903137746
Spaaij, R., Schulenkorf, N., Jeanes, R., & Oxford, S. (2018). Participatory research in sport-for-development: Complexities, experiences and (missed) opportunities. Sport Management Review, 21(1), 25–37. https://doi.org/10.1016/ j.smr.2017.05.003
Tandon, R. (1981). Participatory evaluation and research: Main concepts and issues. In F. Walter & R. Tandon (Eds.), Participatory research and evaluation (pp. 15–34). Indian Social Institute.
Torre, M. E., Fine, M., Alexander, N., Billups, A., B., Blanding, Y., Genao, R., Marboe, E., Salah, T., & Urdang, K. (2007). Participatory action research in the contact zone. In J. Cammarota & M. Fine (Eds.), Revolutionizing education: Youth participatory action in motion (pp. 23–44). Routledge.
Turner, S. (2010). Challenges and dilemmas: Fieldwork with upland minorities in socialist Vietnam, Laos and southwest China. Asia Pacific Viewpoint, 51(2), 121–134. https://doi.org/10.1111/j.1467-8373.2010.01419.x
USAID (2011). Performance monitoring and evaluation–Tips: Conducting participatory evaluation. https://pdf.usaid .gov/pdf_docs/pnadw101.pdf
Veugelers, W. (2017). The moral in Paulo Freire’s educational work: What moral education can learn from Paulo Freire. Journal of Moral Education, 46(4), 412–421. https://doi.org/10.1080/03057240.2017.1363599
Warhrman, H. & Zach, S. (2016). The value of emic research in sport for development and peace programs. Sport, Education and Society, 23(4), 1–14. https://doi.org/10.1080/13573322.2016.1183192
Welty Peachey, J. & Burton, L. (2017). Servant leadership in Sport for Development and Peace: A way forward. Quest, 69(1), 125–139. https://doi.org/10.1080/0033629 7.2016.1165123
Welty Peachey, J. & Cohen, A. (2016). Research partnerships in sport for development and peace: Challenges, barriers and strategies. Journal of Sport Management, 30(3), 282–297. https://doi.org/10.1123/jsm. 2014–0288
Welty Peachey, J., Schulenkorf, N., and Spaaij, R. (2019). Sport for social change: Bridging the theory-practice divide. Journal of Sport Management, 33(5), 361–365. https://doi.org/10.1123/jsm.2019-0291
Whitley, M. A., Forneris, T., & Barker, B. (2014). The reality of evaluating community-based sport and physical activity programs to enhance the development of underserved youth: Challenges and potential strategies. Quest, 66(2), 218–232. https://doi.org/10.1080/00336297.2013.872043
Journal of Sport for Development 