Yarning with the Stars Project: An Indigenous evaluation protocol for a sport for development and peace program

Authors

Rose Whitau1, Helen Ockerby1

1 Shooting Stars: Education Through Netball, Glass Jar Australia, Australia

Citation:

Whitau, R. & Ockerby, H. (2019). “Yarning with the Stars Project: An Indigenous evaluation protocol for a sport for development and peace program.” Journal of Sport for Development, 7(13), 46–54. Retrieved from https://jsfd.org/

Download article as PDF

ABSTRACT

In Australia, the gap between Aboriginal and Torres Strait Islander students and their non-Indigenous peers is significant in terms of attendance, retention to Year 12, and literacy and numeracy skills, with the gap widening in regional and remote contexts. School-based, “academy-style” engagement programs work to close this gap by providing holistic support services to Aboriginal and Torres Strait Islander students while requiring a certain level of school attendance by program participants. Shooting Stars is an engagement program based in seven remote and regional schools in Western Australia, where it uses netball and other incentives to engage Aboriginal and Torres Strait Islander girls in their education, while promoting their health and wellbeing. Shooting Stars evaluates the efficacy of its services through collation of attendance data, participant case studies, and yarning circles. The methods used in the yarning circles research were developed over 18 months in collaboration with Shooting Stars participants, localized Shooting Stars steering committees, and Shooting Stars staff. This paper presents the evaluation protocols for the Shooting Stars program, focusing on the yarning circles’ methods in order to provide a framework or model of Indigenous evaluation methods for others working within this space.

INTRODUCTION

First, it is right that we introduce ourselves. We, the authors, are both Indigenous women: RW is of New Zealand Māori (Kāi Tahu, Kāti Mamoe, Waitaha) and Pākehā (New Zealand European) descent and HO is of Aboriginal Australian (Bardi-Jawi and Nyoongar) descent. We both work for the Shooting Stars program from which we receive financial remuneration. RW was the regional manager for the Shooting Stars program in the Mid West, Gascoyne, and Goldfields regions from January 2016 to January 2018, was a contract researcher for the Shooting Stars program from January 2018 to August 2019, and is now the Shooting Stars research manager. HO has been the regional manager of the Kimberley region for the Shooting Stars program since January 2016. We have given much of our blood, sweat, and tears to the establishment of the Shooting Stars program, and we are heavily invested in the program and the participants’ success.

Background

In Australia, the gap between Aboriginal and Torres Strait Islander1 students and their non-Indigenous peers is significant in terms of attendance, retention to Year 12, and literacy and numeracy skills, with the gap widening in regional and remote contexts (Australian Bureau of Statistics, 2017; Commonwealth of Australia, 2017; Commonwealth of Australia, 2018). School-based, “academy-style” engagement programs, such as the Clontarf Foundation and Role Models Australia, work to close this gap by providing holistic support services to Aboriginal students while requiring a certain level of school attendance by program participants. Funding is currently biased toward boys’ programs, with $40 million granted to the mentoring and support of young men in June 2017, compared to $9 million for young women (Commonwealth of Australia, 2017). In 2017, the Standing Committee on Indigenous Affairs found that the common elements among successful engagement programs were “flexibility, cultural safety, buy-in from the family and connection with community”; yet how the success of these programs was measured was not defined, with the committee also stating that they were consistently surprised and concerned about the lack of data available regarding attendance and education outcomes for Indigenous students” (Commonwealth of Australia, 2017, p. 47).

Engagement programs that use sport as a “hook” are part of the Sport for Development and Peace (SDP) movement. Distinct from sports development, which aims to develop sport per se, SDP aims to use sport to tackle development issues—in this case, engagement with education (Coalter, 2009; Darnell, 2012). There are currently two main critiques of the SDP movement. The first is that there are few theoretical frameworks for evaluation across both top-down broad-scale and bottom-up local-scale initiatives (Black, 2010; Kay, 2009; Kidd, 2008). The benefits of sport are often listed, usually within Western, “white,” and colonial modes of understanding, without evaluation. The second is that development initiatives, and SDP programs by proxy, tend to propagate Western, colonial paradigms, with white developers developing the black underdeveloped world. This is particularly true when working in Indigenous contexts, where Indigenous populations are targeted, yet Indigenous voices are rarely heard within SDP theorisation, policy, and evaluation processes (Black, 2010; Kay, 2009; Kidd, 2008; Rossi & Rynne, 2014). It is beyond the scope of this paper to discuss the history of social/economic development and its idiomatic pitfalls; however, what can readily be drawn from the literature is that development programs need to be both embedded within and driven by the communities within which they serve, and these programs need to undergo regular self-reflection (where that reflection is undertaken collaboratively by staff, participants, and their communities).

The Shooting Stars Program and the Yarning with the Stars Project

Shooting Stars, an initiative of Glass Jar Australia and Netball Western Australia, uses netball and other incentives as rewards to encourage young Aboriginal girls to improve their school attendance, while promoting their health and wellbeing. First established as a pilot project in Halls Creek in July 2014, Shooting Stars now reaches over 350 girls and is embedded in seven remote/regional schools across Western Australia: Carnarvon Community College, Derby District High School, Halls Creek District High School, Leonora District High School, Meekatharra District High School, Mullewa District High School, and Narrogin Senior High School (see Figure 1). At all of the schools except for Narrogin (16% Aboriginal), students are predominantly Aboriginal (75 to 100%).

Figure 1 – Map of Western Australia with Shooting Stars sites and approximate participant numbers at time of writing, with inset of Australia

Figure 1 - Map of Western Australia with Shooting Stars sites and approximate participant numbers at time of writing, with inset of Australia

The Shooting Stars program is site specific, with staff adapting a key set of deliverables to the requirements and interests of the respective host school, community, and students. Figure 2 depicts the program logic model, including program activities, outcomes, and evaluation tools. The program comprises two netball and two health and wellbeing sessions per week per age group. Netball is used as a hook to engage potential participants. The health and wellbeing aspect of the Shooting Stars program is grounded within the Shooting Stars curriculum framework, underscored by the three values of the Shooting Stars program: pride, respect, and success. Once registered with the program, participants are encouraged to improve or maintain their school attendance through a system of structured rewards, such as interschool sports carnivals, bush trips, and movie nights. Rewards are also used to reinforce positive classroom behavior, and teachers are actively encouraged to provide feedback to Shooting Stars staff on participant behavior in class. In conjunction with the proactive program based on these deliverables, staff work responsively within the school context, providing support for participants. For example, Shooting Stars staff will advocate for participants during behavior management meetings and Shooting Stars staff are regularly engaged by their host school to mitigate behavior issues throughout the school day. The Shooting Stars room at each site is a safe space for all Shooting Stars participants and is often the place nominated by the school for participants on behavior management plans to de-escalate.

Figure 2 – The Shooting Stars program logic model: Program activities, outcomes, and evaluation tools

Figure 2 - The Shooting Stars program logic model: Program activities, outcomes, and evaluation tools

The four key outcomes of the Shooting Stars program are:

  1. Participants maintain/improve their school attendance (target minimum 80%);
  2. Participants maintain/improve a positive attitude toward their education, health and wellbeing, and future;
  3. Empower Aboriginal Women: The number of Aboriginal women gainfully employed or undertaking higher education is increased (e.g., grow the number of young women completing Year 12 at each site; increase Glass Jar Australia’s Aboriginal staff rate to 100%);
  4. Embed the Shooting Stars Program in Communities: Family, communities, local service providers, and schools are engaged in Shooting Stars program delivery of events and reward trips.

Shooting Stars evaluates the success of the program through collation of attendance data, participant case studies, and yarning circles. Attendance data collection is straightforward: the data are taken directly from the Department of Education’s Integris system. Participant case studies are shared only with the main funding body—the Department of Prime Minister and Cabinet (PMC)—under strict confidentiality agreements. This paper presents the protocols for the yarning circles research—the Yarning with the Stars project, which draws on Western and Indigenous research methods to provide qualitative evaluation for this program. The three key aims of the Yarning with the Stars project, are:

  1. To evaluate the progress and efficacy of the program in order to complete funding requirements;
  2. To enable communities and participants to drive the direction and content of their local program by creating activities that directly influence outcomes;
  3. To disseminate the results of the yarning circles method externally in order to:
    1. Provide other organizations working in the Indigenous education/youth/sport space with a framework or model for applying Indigenous evaluation methods;
    2. To grow the capacity of Shooting Stars staff and other Aboriginal and Torres Strait Islander women to conduct and drive local research initiatives; and,
    3. To validate Shooting Stars yarning methods in order to build the prestige of the program and enhance future funding opportunities

METHODS

Essential aspects of Indigenous research include: respect; inclusive decision making; equality of input and control (including the objectives, processes, and data interpretation); privileging Indigenous voices, perspectives, and knowledge systems; and benefits for all who participate (Kovach, 2009; Smith, 1999). We selected yarning circles as the primary mode for data collection because they provide a culturally responsive research space in which both information can be collected and relationships can be built (Bessarab & Ng’andu, 2010). To yarn is to converse—in a mutual, reciprocal dialogue, where interjections, interpretations, and additions are welcome (Geia, Hayes, & Usher, 2013; Rogers, 2017). Yarning is being used more and more frequently as a qualitative research tool (e.g., Bessarab and Ng’andu, 2010; Carlson & Frazer, 2018; Rogers, 2017). The yarning circle, a “focussed, directed discussion based on principles of respect, inclusion and democratic participation” (Carlson & Frazer, 2018, p. 44) has an ancient history in Aboriginal Australia. Similar practices, such as hui in Aotearoa/New Zealand, occur in other Indigenous nations throughout the world (Aseron, Greymorning, Miller, & Wilde, 2013).

In the Shooting Stars Yarning with the Stars project, there are three groups of people involved in the yarning process: Shooting Stars staff, Shooting Stars steering committees, and Shooting Stars program participants. Shooting Stars staff are predominantly Aboriginal (85% at time of writing), all are women, and all come from diverse backgrounds in terms of age, cultural heritage, work background, education, and life experience. At each site, Shooting Stars steering committees are comprised of Shooting Stars staff, host school representatives, Aboriginal community representatives, and relevant local stakeholders. Steering committee representatives are also diverse in terms of age, cultural heritage, work background, education, and life experience. Shooting Stars program participants are all girls and young women, predominantly Aboriginal (75 to 100% depending on site demographics), vary in age and education from Year 3 to Year 12, and currently live in a remote/regional area.

In considering this research and its diverse participants, we identified that a culturally responsive, decolonized methodology, which gives epistemological equality to western, Aboriginal, and other ways of knowing, being, and doing, was paramount. We selected relatedness as a theoretical framework because it is situated within Aboriginal ways of knowing, being, and doing, and is both inclusive and subjective, requiring “identities, interests, and connections to determine our relatedness, drawing on what we know through elders and community, as proper ways of being” (Martin, 2003, p. 210). Relatedness is a suitable framework since the central tenet of the Shooting Stars program is the relationships connecting individuals and communities, with staff embedded within the communities where this research is conducted. Comprising both insiders and outsiders in these communities, we, as Shooting Stars staff, are able to use relatedness to acknowledge our position and privilege others’ knowledge alongside our own. Similarly, the program participants are from diverse Indigenous and non-Indigenous communities, which are often quite transient. Relatedness provides an inclusive, subjective space for the participants to relate to each other and staff and a method by which to privilege their ways of knowing, being, and doing. Finally, relatedness theory helps us to understand that the implications of this research will continue past its undertaking (Martin, 2003), which belies the sustainability that is built into the program so that it will have a positive legacy beyond the discontinuation of funding.

There are two types of yarning circles conducted in this research: “yarning about yarning” yarns, in which the yarning circle methods are discussed, and yarning circles, in which participants’ attitudes toward their education, health and wellbeing, and future are gauged. Yarning circles are audio recorded (no video or photos are taken for safety and confidentiality). The audio files are transcribed by Digital Transcripts. We, the authors, cross-code the transcripts to establish a collaborative coding tree, using NVivo for Mac qualitative software, and prepare biannual reports on the yarning circles, in which data are presented thematically, with representative quotes, word clouds, tables, and figures.

Yarning in Ten Steps

We developed a simple ten-step process to aid staff when conducting yarning circles (see Figure 3). This ten-step process divides yarning into three stages in relation to netball session planning, which staff are all familiar with. The first phase, the “warm up,” comprises planning, informing parents/caregivers and participants in order to get their consent, and preparing the participants for yarning. The preparation phase could occur in the moments prior to the yarning circle, for example, an icebreaker activity, or could involve several health and wellbeing sessions over several weeks. These preparation methods have varied and evolved across time and between sites from simple questions and brainstorming on butcher’s paper to photovoice (e.g., Wang, 2006), where participants take photos that represent specific elements and bring these to the yarning circle for discussion. Staff have used both Indigenous tools (e.g., message stick) and non-Indigenous tools (e.g., scaffolding and differentiation techniques) to make sure that the yarning space is always safe, inclusive, and accessible to participants.

Figure 3 – Yarning with the Stars project: Yarning in ten steps

Figure 3 - Yarning with the Stars project: Yarning in ten steps

The second phase, “the game,” is the yarning circle itself, where staff turn the audio on and reintroduce the yarning circle topic. During this phase staff act as a yarning circle “umpire,” creating and monitoring boundaries with the participants to keep them safe and facilitating the conversation to help it flow. While staff have done all that they can in the first phase to ensure that the circle will be safe, it is after the topic has been reintroduced that staff will set yarning circle boundaries with the girls. Staff start each yarn by discussing not just the purpose of the yarn and how it will work, but also by identifying, and helping the participants to identify their relatedness. Together, the facilitator and the participants establish what they know to be appropriate and proper ways of acting in the circle—the “rules” that the umpire upholds for the duration of the game. The umpire as facilitator has been a very useful analogy for our staff, who were initially feeding a lot of answers into the yarning circles but who now understand that like the umpire, the facilitator’s role is not to play the game, but to enable the game/yarn to proceed in fairness and safety.

Facilitation does not stop when the audio turns off—consensus must be facilitated in the third and final phase or “cool down,” which also comprises uploading the yarning circle audio to a secure folder and reflecting on the yarning circle process. The session plan template, which staff use for netball training, health and wellbeing classes, and yarning circles, includes a small section for staff to write two to three sentences about what worked and did not work.

Participant and Steering Committee Yarning Circles

From December 2016 to December 2018, the main topic across participant and steering committee yarning circles was school— what participants’ attitudes were toward school and what barriers they faced in attending school and/or remaining in class. The results have been used to improve program delivery. Reports generated during this time period were discussed during steering committee yarns, where they were used as context for the yarning circle about participants’ attitudes toward school and the barriers that the participants had discussed. From these yarns, staff and steering committees selected two of the top five barriers identified across the Shooting Stars program sites for Shooting Stars to focus on in 2019: bullying and alcohol/drugs. Shooting Stars has now employed a full-time Aboriginal curriculum developer both to write health and wellbeing sessions specifically for this purpose and to train staff to deliver these sessions.

Shooting Stars has also used yarning circles to make program changes and additions based on direct feedback from steering committees and participants. Changes to program delivery vary from the adjustment of netball training times and the inclusion of others (such as boys or girls from other schools) in netball games to the development of the Shooting Stars Leadership Project. The development of the Shooting Stars Leadership Project by the Shooting Stars leaders themselves has further embedded yarning within the program’s logic model, since yarning now directly contributes to empowering Aboriginal women to lead projects within their communities and to make the changes that they have identified. For example, in Carnarvon, one student noticed a lack of education around sexually transmitted diseases among her peers, so for her Leadership Project in 2018, she organized for a nurse to run a health and wellbeing session on sexual health. The yarning circles enable participants to drive the direction and content of their local program in three ways: through analysis of summary reports of participant attitudes; through direct feedback from steering committees and participants; and via the Leadership Project, in which the participants both identify the need and drive the change. The yarning circles are themselves a tool for empowering the participants, since they provide a platform for student voice and agency, where student feedback is honored and acted upon (Bahou, 2011; Bamblett, Harrison, & Lewis , 2010; Fielding, 2004).

Yarning circles have been a central part of the Shooting Stars evaluation protocol since December 2016. The appropriate approvals had been sought from host schools and institutions, and the results of the yarning circles were shared internally or with key stakeholders (schools, funding bodies, steering committees) under strict confidentiality agreements. In August 2018, a formal ethics approval was sought from, and granted by, the Australian Institute of Aboriginal and Torres Strait Islander Studies, so that the results of the research could be publicly disseminated moving forward.

Challenges and Limitations

The reality of staff-driven research has proven to be both the greatest limiting factor for this project and its greatest asset. Our relationships with our participants and communities are not simply an advantage over an external researcher, who must spend a large quantity of time building rapport with their participants (see, e.g., Sharif, 2001), it is through these relationships, and our relatedness, that we are truly able to decolonize our program’s theorization, policy, practice, and evaluation. However, facilitating participant and steering committee yarns provides an additional administrative pressure on staff, who are already time poor. Operating within the school context, staff must respond to behavioral events ad hoc, which can throw a day’s planning out the window. Therefore, while staff have planned and discussed the tools they will use for their yarning circles and how they will make them contextual, staff time and access to those participants might become restricted and improbable during the week that those activities were planned. While running yarns on reward camps does alleviate some of this pressure, only participants who are readily engaged in the program and have earned the camp (participants with school attendance 80% and above or an improvement of 20% or more from the previous term and those who have made positive behavior decisions in class) will be present in the yarn. Additionally, staff capacity, turnover, and time between opportunities for training mean that staff are at different levels of understanding and engagement with the research, its methods, and its possibilities.

The biggest learning for us has been a reflection on how the professional development around yarning circles is delivered to and with Shooting Stars staff. In the beginning, we facilitated the first staff yarn without ever having conducted a yarning circle ourselves. We assumed that because social and therapeutic yarning (Bessarab & Ng’andu, 2010) is the Shooting Stars Program Coordinator’s bread and butter that the yarning circle would be a straightforward activity. We discussed theory, the reasons behind the yarning circles—with little reference or instruction to staff in how they would be delivered on the ground, besides ethical necessities such as consent and participant safety. Over time, and with staff feedback, we have learned what practical information is useful for staff, who have the skills and relationships to facilitate yarning circles successfully but are often daunted by the prospect of “doing research” and are self-conscious in front of the audio recorder.

CONCLUSIONS

The methods described here are predominantly based on Indigenous principles and practices, while combining Indigenous (yarning circles, storytelling, knowledge-sharing through art and objects) and non-Indigenous (scaffolding learning, differentiation, audio-recording and transcription, thematic analysis) data collection and processing methods. The perspectives, protocols, and cultural values of Aboriginal people, particularly women, have been central to the development of this learning and research process. The research aims and methods, while restricted by funding reporting requirements, are flexible and open to negotiation from all three groups (Shooting Stars staff, steering committees, and participants) in various stages throughout the research process.

Yarning circles are not just a culturally responsive way to evaluate program success and to tailor engagement and/or Sport for Development and Peace programs to local contexts, they provide a platform for student voice and self-determination. The ten steps presented here could be related to any sport and delivered by any coach familiar with session planning, warming up/cooling down, and the role of an umpire; however, cultural competency training would be required for non-Indigenous staff members, and all staff would need support in applying the theoretical framework of relatedness. Yarning itself can be as simple as sitting down with a cup of tea and a question or more complex, with participants bringing photos that they have taken of a specific theme. Other organizations can also adapt the methods to their specific circumstances. For Shooting Stars, the essential element is the relationship that is built between facilitator and participant before, during, and after the yarning has taken place.

NOTES

1Henceforth “Aboriginal and Torres Strait Islander” will be abbreviated to “Aboriginal.”

CONFLICTS OF INTEREST

Both of the authors work for the Shooting Stars program, from which we receive financial remuneration. We are heavily invested in the program and the participants’ success.

FUNDING

The Shooting Stars program is funded by the Department of Prime Minister and Cabinet under the Indigenous Advancement Strategy. The Yarning with the Stars Project receives additional funding from Horizon Power and Lotterywest. None of these funding bodies contributed to the research process.

ACKNOWLEDGEMENTS

First, we would like to thank our participants, staff, steering committees, schools, and communities for their participation in this research project. Figure 1 was produced by Ferdinando Handojo. Figures 2 and 3 were produced by Ashton Murphy. We would like to thank Professor Colleen Hayward AM and two anonymous reviewers for their invaluable comments on earlier versions of this manuscript.

REFERENCES

Aseron, J., Greymorning, S. N., Miller, A., & Wilde, S. (2013). Cultural safety circles and Indigenous Peoples’ perspectives: Inclusive practices for participation in higher education. Contemporary Issues in Education Research6(4), 409-416.

Australian Bureau of Statistics. (2017). Schools, Australia, 2017, Key Findings. Available from http://www.abs.gov.au/AUSSTATS/abs@.nsf/Latestproducts/4221.0Main%20Features22017?opendocument&tabname=Summary&prodno=4221.0&issue=2017&num=&view.

Bahou, L (2011). Rethinking the challenges and possibilities of student voice and agency. Educate (January), 2-14.

Bamblett, M., Harrison, J. & Lewis, P. (2010). Proving culture and voice works: Towards creating the evidence base for resilient Aboriginal and Torres Strait Islander children in Australia. International Journal of Child and Family Welfare, 13(1-2), 98-113.

Bessarab, D., & Ng’andu, B. (2010). Yarning about yarning as a legitimate method in Indigenous research. International Journal of Critical Indigenous Studies, 3(1), 37-50.

Black, D. R. (2010). The ambiguities of development: Implications for “development through sport.” Sport in Society 13(1), 121-129. doi: 10.1080/17430430903377938

Carlson, B., & Frazer, R. (2018). Yarning circles and social media activism. Media International Australia169(1), 43-53.

Coalter, F. (2009). Sport-in-Development: Accountability or development? In R. Levermore and A. Beacon (Eds.), Sport and International Development (pp. 55-75). London: Palgrave MacMillan.

Commonwealth of Australia. (2017). House of Representatives, Standing Committee on Indigenous Affairs. The power of education: From surviving to thriving. Educational opportunities for Aboriginal and Torres Strait Islander students.

Commonwealth of Australia. (2018). Department of the Prime Minister and Cabinet. Closing the Gap Prime Minister’s Report 2018.

Darnell, S. (2012). Sport for development and peace: A critical sociology. London & New York: Bloomsbury.

Fielding, M. (2004). Transformative approaches to student voice: Theoretical underpinnings, recalcitrant realities. British Educational Research Journal, 30(2), 295-311.

Geia, L., Hayes, B. & Usher, K. (2013). Yarning/Aboriginal storytelling: Towards an understanding of an Indigenous perspective and its implications for research practice. Contemporary Nurse, 46(1):13-17.

Kay, T. (2009). Developing through sport: Evidencing sport impacts on young people. Sport in society, 12(9), 1177-1191.

Kidd, B. (2008). A new social movement: Sport for development and peace. Sport in Society, 11(4), 370-380.

Kovach, M. (2009). Indigenous methodologies. Toronto: University of Toronto Press.

Martin, K (2003). Ways of knowing, ways of being and ways of doing: A theoretical framework and methods for indigenous research and indigenist research. Journal of Australian Studies, 76, 203-214.

Rogers, J. (2017). Photoyarn: Aboriginal and Maori girls’ researching contemporary boarding school experiences. Australian Aboriginal Studies, (1), 3-13.

Rossi, T., & Rynne, S. (2014). Sport development programmes for Indigenous Australians: Innovation, inclusion and development, or a product of “white guilt”? Sport in Society, 17(8), 1030-1045. doi: 10.1080/17430437.2013.838355

Sharif, B. (2001). The ambiguity of boundaries in the fieldwork experience: Establishing rapport and negotiating insider/outsider status. Qualitative inquiry7(4), 436-447.

Smith, L. T. (1999). Decolonizing methodologies: Research and Indigenous peoples. London: Zed Books.

Wang, C. C. (2006). Youth participation in photovoice as a strategy for community change. Journal of community practice14(1-2), 147-161.