METHODS

Intervention

The overall objectives of the netball programs in Samoa and Tonga are to build local governance capacity, provide leadership opportunities for women and sustainably increase recreational physical activity to improve health across all age-groups. We propose to focus on the health-related objectives and evaluate two components of the programs being delivered for adults:

a) Social Mixed Netball

The Social Mixed Netball League is known as “Business House” in Samoa and the “Corporate Cup” or “Fiefia” in Tonga. Despite minor differences between countries, it typically comprises two or three playing “seasons” each year that go for six to ten weeks and are conducted in the major towns of Samoa (Apia) and Tonga (Nuku’alofa). Groups of individuals representing local organisations self-select 12 team members and then voluntarily register to participate in a netball league that welcomes all genders. Previous iterations have often been over-subscribed with approximately twice as many teams registering as places available in the competition. Selection of the participating teams in each “season” is decided on a “first-come, first-served” basis and the number of teams included is determined by locally available infrastructure and resources. Participation in the Social Mixed Netball League typically involves:

• Separating the selected teams into two equal-sized pools;
• One or two games of organised netball per week for each team at a central location (30minutes/game);
• Additional voluntary training sessions that are self-organised.

(NOTE: Social Mixed teams typically comprise people with mixed abilities representing local workplaces).

b) Grassroots Netball Development

The local netball federations have adopted a multi-faceted program approach for promoting netball participation by adults throughout Samoa and Tonga. Grassroots Netball Development typically includes:

• Providing equipment that enables communities to participate in recreational netball;
• Delivering coach, player and administrator development workshops designed to build capacity and encourage local autonomy for promoting sustained netball participation;
• Organising mass participation short tournaments/ carnivals (e.g. one-day competition) that attract participants from multiple villages to a central location for playing and promoting netball;
• Facilitating a home & away netball series where teams from several neighbouring villages play one game against each other at home and one game away;
• Supporting club netball series where registered clubs are graded and compete against each other.

(NOTE: The tournaments / carnivals, home & away series and club netball series typically cater for all genders and include both open and senior age categories. In Tonga, the senior category is known as “Over 35” and only includes participants whose age or body mass index are higher than 35).

The proposed actions, outputs, impacts and outcomes of the Social Mixed Netball League and the Grassroots Netball Development program are consistent with a published theoretical framework for sport-for-development programs.¹³ Although there is some overlap with the stated objectives of governance and empowerment, we have developed a logic model that focuses on the health-related outcomes (Figure 1).¹⁴

Evaluation design
a) Social Mixed Netball League (Figure 2)

The oversubscription of registered teams provides an opportunity to conduct a natural experiment. The first teams to register will be allocated to the intervention group until local infrastructure and resource capacity is reached. All subsequent teams that register will be wait-listed and given priority entry to the subsequent “season”. All adults in the registered teams will have their netball access, netball participation, recreational physical activity, body composition and mental well-being assessed before and after the program. Further follow-up assessment will occur after a six-week “wash-out” period. We will also conduct a parallel process evaluation for intervention delivery and reach in the participating teams.

b) Grassroots Netball Development (Figure 3)

We will conduct periodic monitoring in an evaluation of multiple intervention communities. Six communities targeted by the Grassroots Netball Development program in each of Samoa and Tonga will be identified. People from each of these communities will have their netball access, netball participation, recreational physical activity, body composition and mental well-being assessed at 12-month intervals. We will also conduct a parallel process evaluation of intervention delivery and reach in each of the communities. Small focus group discussions with community representatives and semi-structured interviews with program personnel will be conducted in parallel at 12 monthly intervals to further substantiate these process indicators and triangulate the quantitative data.

Target Population
Despite different intervention recruitment procedures the target population is consistent for both the Social Mixed Netball League and the Grassroots Netball Development program.

Inclusion criteria:

• Adults (aged older than 18 years);
• Eligible to participate in the netball programs;
• (NOTE: Participation in the “Over 35” programs requires an age or body mass index higher than 35).

Exclusion criteria:

• Professional advice against vigorous-intensity physical activity due to health contraindications;
• Diagnosed severe mental illness.

Sample Size and Selection
All sample size and selection procedures will be determined by pragmatic considerations of program structure and capacity. The following calculations provide guidance and a recruitment target.

a) Social Mixed Netball League

The required sample size (n=392 per country) was calculated using relevant physical activity data from a previous validation study of the selected metric and by applying a standard formula based on: a normal distribution (z); probability of type I error (α=0.05); probability of type II error (β=0.20); estimated standard deviation (σ=52.96 mins/day)15; expected effect size (δ=15.00 mins/day). To allow for approximately 15% attrition, all members of first 20 teams in the intervention group (n=240 per country) and all members of the first 20 wait-listed teams (n=240 per country) will be included in the evaluation.

b) Grassroots Netball Development

The required sample size (n=146 per country) was calculated using relevant physical activity data from a previous validation study of the selected metric and applying a standard formula based on the following factors: a normal distribution (z); probability of type I error (α=0.05); probability of type II error (β=0.20); estimated standard deviation of within-group change (σ=64.60 mins/day)¹⁶ ; expected effect size (δ=15.00 mins/day). To allow for approximately 15% attrition, a convenience sample of 30 people will be selected from each of the six participating communities (n=180 per country) for quantitative data collection. Study participants will be the same people at each assessment period. The participating communities will be selected based on previous engagement with the national netball federations and agreement from village elders and leaders to participate in the evaluation. Additionally, qualitative assessment of a purposeful sample of program participants, coaches, umpires, staff and other stakeholders from the national netball federations will be conducted until data saturation is reached.

Outcomes
The primary and secondary outcomes are consistent for both the Social Mixed Netball League and the Grassroots Netball Development program. The primary outcomes are netball access and netball participation. The secondary outcomes are recreational physical activity, body composition and mental well-being.

Data Collection Procedures (Quantitative)
Researchers from the University of Sydney will collaborate with health workers and train a local evaluation team in both Samoa and Tonga to collect the necessary quantitative data. These results will be recorded using iPads in the order outlined below and automatically uploaded to a central database. The survey items will be translated into the local language to facilitate communication and delivered in the form of an interview. Prior to commencing data collection, the interviewer will explain the purpose of the evaluation to the participant and then answer any questions before obtaining their informed consent to proceed.

The local evaluation team will use the same measurement tools to collect demographic data and results for the primary and secondary outcomes at all time points for both the Social Mixed Netball League and the Grassroots Netball Development program.

Data will be collected individually and commence with the following identification and demographic information:

• Full name (anonymised to program implementers and data analysts);
• Gender (male / female / other Samoa: fa’afafine, Tonga: leities);
• Age (years);
• Village of residence (including postcode if available);
• Highest completed education level (none / primary school / secondary school / university degree);
• Main work status in past 12 months (government / private sector / non-government organisation / self-employed / non-paid volunteer / student / homemaker / retired / unemployed).

The survey items used to collect data on each of the outcome measures have been derived from various sources (Table 1).

Netball access will be assessed using a series of novel survey items about local capacity to engage in ongoing netball activities. All questions have been developed and piloted in the local communities and are required to be answered (Table 1: survey items 1.1-1.6).

Netball participation will be assessed using a series of novel survey items about playing frequency and comparisons to pre-intervention. The questions have been developed and piloted in the local communities and will vary according to the assessment timing and program component (Table 1: survey items 2.1-2.4).

Mental well-being will be evaluated using the WHO-5 Well-Being Index, which is a globally validated self-report measure.¹⁷ All questions are required to be answered (Table 1: survey items 3.1-3.5).

We will also apply a globally validated one-item happiness measure, which has been modified to include a visual analogue scale to improve discriminating power and reduce gender distortion (Table 1: survey item 4.1).¹⁸ This will subsequently be converted to a score on a linear 100 point scale for analysis.

Recreational physical activity will be assessed using the relevant section of the Global Physical Activity Questionnaire (GPAQ), which is a globally validated self-report measure that has been adapted to the local context in Samoa and Tonga.^15,16 The questions asked will vary according to the responses provided and minutes of moderate and vigorous intensity recreational physical activity will be calculated for analysis (Table 1: survey items 5.1-5.2).

Body height and weight will be assessed objectively and used to calculate Body Mass Index (BMI) by applying standardised methods previously used in low- and middle-income settings.¹⁹ All measures will be performed by locally trained assessors using consistent apparatus for all participants (i.e electronic scales, telescopic ruler).

The timing / location of data collection and process indicators will vary according to program component:

a) Social Mixed Netball League

All data will be collected face-to-face by a member of the evaluation team at a location that is convenient for the registered participants (e.g. workplace, training venue, village hall). Arrangements to complete baseline measurements will be made immediately after a team registers for the program. Follow-up measurements can commence immediately after the “season” is complete and will take place at the same location as the baseline assessments.

Process indicators for program delivery and reach will include a weekly record of player participation in games and training. Each team captain will be provided with a weekly template to complete and submit to a member of the local evaluation team who will collate these in a previously established excel spreadsheet.

b) Grassroots Netball Development

All data will be collected face-to-face by a member of the evaluation team at a convenient time and location for the participating members from each community. Arrangements to complete baseline measurements will be made as soon as the evaluation team has been trained. Follow-up measurements will be scheduled at a time that is convenient for the participants and will take place at the same location as the baseline assessments.

Process indicators for program delivery will be recorded throughout the year using previously established excel spreadsheets and collated annually by a member of the local evaluation team. It will include the collection of the following data for each participating community across all age groups:

• Equipment provision (number of netballs and other equipment provided);
• Training workshops (number of participants and types of netball workshops);
• Mass participation tournaments / carnivals (number of participants and tournaments /carnivals);
• Home and away series (number of participants, games and training sessions);
• Club Netball Series (number of registered clubs / players and series / games played).