Consultant - Solomon Islands Swipe Safe Evaluation

Consultant’s Terms of Reference for Swipe Safe Solomons Project Evaluation.

1. Organisational context

ChildFund is an independent and non‐religious international development organisation that works to reduce poverty for children in developing communities. We work in partnership with children and their communities to create lasting change by supporting long‐term community development, responding to humanitarian emergencies and promoting children’s rights. Our vision is a world without poverty where all children and young people can say ‘I’m safe. I am educated. I have a future’.

ChildFund Australia implements programs with a range of local partners in Cambodia, Laos, Myanmar, Vietnam, Papua New Guinea, Timor‐Leste, and other Pacific Island countries. Our work is funded through community sponsorship, government grants as well as donations from individuals, trusts and foundations, and corporate organisations.

ChildFund Australia is a member of the ChildFund Alliance – a global network of 11 child-focused development and humanitarianorganisations which assists more than 23 million children and their families in over 70 countries. ChildFund Australia is a registered charity, an accredited member of the Australian Council for International Development and fully accredited by the Department of Foreign Affairs and Trade (DFAT), which manages the Australian Government’s overseas aid program.

ChildFund in the Pacific

ChildFund Australia works with governments, civil society partners and communities in PNG, Timor Leste and the Solomon Islands to deliver child protection, education and healthcare programs. We respond to emergencies in Pacific Island nations and are committed to building strengthening capacity so that locals can respond first.

2. Background

The project “Strengthening Online Safety for Young People in the Pacific 2020-2022: Phase 2” seeks to increase protective environments for children online and offline in the Pacific region with focus on Solomon Islands, by addressing limited professional capacity in child protection and by building holistic online safety projects with young people across the Pacific. This project is funded by the DFAT Cyber and Critical Technology Cooperation Program. It is a joint project between ChildFund Australia and PLAN International Australia.

Some major outcomes of this project are:

• Young women and men with and without disabilities participate, establish and work together to implement stronger approaches to online safety
• Male and female parents and caregivers are equipped with knowledge and skills to enable them to be online safety partners for their young people
• Community entities where girls and boys with and without disabilities can access and interact with the online world such as schools, churches and businesses are empowered to create safe environments that promote online safety
• The Ministry of Communication and Aviation is supported to coordinate cross-sector collaborations to create safer online environments for young women and young men.
• Pacific police forces will engage with and be informed by youth participatory approaches for online safety.
• Improved evidence-base of Pacific young women and young men’s experiences of online risk and protective mechanisms to inform local, national, and regional online safety approaches.

In this project, ChildFund's flagship Swipe Safe training package is rolled out to equip children, young people, parents/caregivers and community entities with knowledge and skills to increase protective environments for children online.

The project has adopted a phased implementation approach which incorporates ongoing research, consultation with local stakeholders, the small-scale piloting of activities and then building in the overall learning to future phases and iterations.

3. Purpose

The purpose of the next phase of Swipe Safe is to evaluate the scalability of the program and training to introduce to other areas of the Solomon Islands or other Pacific country contexts.

• Why the evaluation is being undertaken at this point in time?

The evaluation is planned at the end of phase 2 of the project, in order to collect and consolidate information on outcomes and learnings, which will then inform the further development and implementation of the program in additional contexts.

• The objectives of the evaluation (what you and other key stakeholders need to know/investigate)

• Testing the effectiveness of current project interventions.

• Investigate the extent to which the project’s interventions are integrated into systems.

• Investigate the level of political support/buy in for the project and activities including improved awareness and support of government resources and the impact of advocacy efforts.

• Viability of the current project model for scale up and replicability in other Pacific locations, and/or in new regions. This should include assessing stand-alone interventions as well as the full training package.

• What the evaluation will be used for

  • The evaluation will be used to support the design and development of the next phase of Swipe Safe in the Solomon Islands, through its assessment of project activities, and the viability of replicating these activities in other locations, as well as key challenges, issues, and lessons learnt.
  • Key findings and learnings will also be used to inform and support the delivery of online safety trainings and advocacy work under the “Swipe Safe South-East Asia" regional project, which is currently in its first year of implementation.
  • More broadly, the evaluation will inform the possibility to expand project activities to new locations by providing insight into the replicability of project activities and the impact they have had.

4. Scope of Evaluation

This end-of-project evaluation will cover the whole of the project and its entire implementation period from July 2019 to June 2023. The evaluation must be conducted in accordance with ChildFund Australia’s guidelines and policies of child protection and ethical standards.

The consultant is required to undertake the following:

• Design an evaluation plan and prepare a schedule for the evaluation by consulting with ChildFund Australia and PLAN International Australia’s Child Protection and MEL Advisers, as well as with partner staff in the Solomon Islands;
• Ensure that the conduct of the evaluation provides opportunities for participation of stakeholders and capture the views and opinions of key stakeholders (children and young people (especially those with a disability), youth facilitators, parents and caregivers, community entities, government representatives, and other target groups);
• Review relevant primary and secondary sources of information associated with project proposal documents, project variations, periodic reports, Living Lab reports, activity result monitoring forms, and other relevant project-related materials;
• Design, lead, and document focus group discussions (FGDs), key informant interviews (KIIs), and change stories with the different stakeholder groups as stated in the project proposal;
• Ensure that all quantitative and qualitative data collected must be classified by location, age, sex, and whether they do or don’t have a disability. Care must be taken to ensure that the analysis of findings consider and reflect the views of marginalised groups;
• Analyse available information and document findings and recommendations;
• Present findings to ChildFund Australia and PLAN International for discussion and feedback. The evaluation should provide evidence including case stories, direct quotes, and high-quality photos in support of the findings arising; and
• A final report will be produced in English, along with a summary outlining key findings and recommendations.

The relevant staff of ChildFund Australia and PLAN International Australia will:

• Provide relevant documents to the consultant;
• Organise logistics;
• Organise field/site visits; and
• Participate in technical review (feedback will be consolidated).

The consultancy will be carried out in Honiara, Solomon Islands. In-depth interviews and focus group discussions with youth facilitators, parents and caregivers and community entities who directly participated in the project will be conducted. Interviews with officials and representatives from the Solomon Islands Police Force and the Ministry of Communication and Aviation will also be required.

The evaluation is taking place at the end of a 3-year project (+1 year extension).

Please see Annex 1 for the Key Questions and OECD Evaluation Criteria.

1. Methodology

This evaluation will employ a mixed-methods approach and will mostly derive both quantitative and qualitative data from the project’s in-built monitoring and evaluation system. These data sources will be reviewed as part of the desk assessment and will comprise of:

1. Testing results of facilitated SwipeSafe trainings for children and young people, parents and caregivers, and community entities (YoY comparative data by cohort);
2. SwipeSafe Living Lab data collected by Western Sydney University;
3. Activity reports for key activities and/or deliverables;
4. Monitoring reports;
5. Periodic project reports (quarterly progress, 6-monthly, and annual narrative reports; as well as output and activity tracking);
6. Change stories or case studies;
7. Various agenda and meeting minutes;
8. Periodic fiscal reports and results of internal and/or CFA/PLAN-led audits.

Discrete sets of qualitative data will however be required for the sole purpose of the evaluation and will comprise of the below methods:

1. Focus Group Discussions (across children and young people, youth facilitators, parents and caregivers, and representatives of community entity stakeholder groups)
2. In-Depth Interviews (Solomon Islands Police, Solomon Islands Ministry of Aviation and Communications officials, and select representatives from the above stakeholder groups; as well as interviews with relevant ChildFund Australia and PLAN International staff).

6. Deliverables and Indicative Timetable

Note that this is subject to negotiation with the Consultant

**Indicative dates/****Outputs and Activities/**Number of Days

10 February 2023

4 to 5 days

·       Discussion of work plan with the Consultant

·       Gathering of relevant ChildFund Australia and PLAN International documents

·       Document review

·       Consultant to develop proposal for data collection and analysis. The proposal includes methodology for data collection, timing, and key people to be interviewed)

·       Consultant to design and set interviews/workshops with relevant staff

Date —

6 to 7 days

·       Conduct focus group discussions and interviews

·       Analysis, synthesis and triangulation of data

Date —

8 days

·       Produce draft evaluation report

Date —

2 days

·       Present and Review evaluation report with relevant staff

End date negotiable

2 days

·       Produce final report

·       Produce external facing summary of findings from evaluation

Total number of days

23 - 25 days

7. Management and Reporting Arrangement

The Consultant will report to Lauren Miles, ChildFund Australia Child Protection Advisor.

All reports must be written in English and provided in an electronic format (Microsoft Word).

8. Confidentiality

All discussions and documents relating to this ToR will be treated as confidential by the selected parties.

9. Child Safeguarding

The Consultant will undertake the Services to a high standard; use its best endeavors to promote the best interests of ChildFund; protect the reputation of ChildFund and work in a manner consistent with the mission, vision and policies of ChildFund (see Child Safeguarding Policy/Child Safeguarding Code of Conduct PSEAH policy and Employee Code of Conduct). ChildFund Australia has a zero-tolerance policy to abuse, exploitation and harassment in all its forms.

10. Counter-Terrorism and Anti-Money Laundering

ChildFund Australia acknowledges its obligation under the Australian laws relating to counter-terrorism and anti-money laundering.  In order to meet its obligation, the consultant is obligated to provide information required for ChildFund Australia to undertake counter terrorism screening before engagement.  The consultant’s name, date & place of birth and ID number will be checked against Department of Foreign Affairs and Trade (DFAT) consolidated list, National Security Australia list, World Banks listing and the Asian Development bank listing to ensure not engage with entities or individuals appearing on the lists.

11. Conflict of Interest

The Consultant must declare any financial, personal, family (or close intimate relationship) interest in matters of official business which may impact on the work of ChildFund

12. Fraud and Corruption prevention and awareness

ChildFund Australia has a zero approach to fraud and corruption act. The successful consultant will be required to comply with ChildFund Australia’s fraud and corruption prevention and awareness Policy and act against any form of fraud or corruption and not offer, promise, give or accept any bribes.

13. Insurance

The successful applicant will be required to have in place insurance arrangements appropriate to provision of the requirement in this TOR including (without limitation) travel insurance.

14. Acknowledgment and Disclaimer

ChildFund, its Board and staff make no express or implied representation or warranty as to the currency, reliability or completeness of the information contained in this ToR.  Nothing in this ToR should be construed to give rise to any contractual obligations or rights, expressed or implied, by the issue of this ToR or the submission of Expression of Interest in response to it.  No contract would be created until a formal written contract is executed between ChildFund and a selected consultant.

--------------------------------------------

Selection Criteria for Consultant

ChildFund Australia is seeking a consultant with knowledge and experience in project evaluations  (preferably in Solomon Islands and /or Pacific Island Countries). The Consultant must be available for distance and face-to-face meetings, as necessary.

The Expression of Interest should include resume, referees, proposal containing competence for the required work, professional fee, approaches/methodologies and timeline based on the Terms of Reference.

Submit CV and proposal by 6 February 2023 to Lauren Miles at Lmiles@childfund.org.au

Annex 1

Proposed Evaluation Approach

1. The EOP will focus on scalability and replicability; and will test the viability of the current model for scale-up.
2. In order to get a sense of to what extent the current set of strategies can be scaled and/or replicated, testing the effectiveness of the intervention will be required as a prerequisite; followed by an analysis of the various strategies employed through the scale-up domains.

Key Evaluation Domains

• Evidence of impact

This domain will investigate the project’s effectiveness in achieving project outcomes, the evidence of changes to knowledge attitudes, and practices, as well as systems change. It will respond to the below key evaluation questions:

1. What knowledge, practice, systems, and policy change/s have the project contributed to? What strategies contributed to these? (Relevant OECD Evaluation Criteria: Effectiveness)
2. How effective were the interventions and/or strategies employed? Which of these worked well, and those that didn’t? Why?
3. Were these changes experienced by women and girls and children with a disability? What strategies and / or interventions contributed to these? (Relevant OECD Evaluation Criteria: Effectiveness, Efficiency)
4. Did a specific part of the intervention achieve a greater impact than another?
5. To what extent does the intervention respond to the Child Centred Indicators?

(Relevant OECD Evaluation Criteria: Impact, Effectiveness)

• System-level (policy) embeddedness

This domain will investigate the extent to which the project’s interventions are integrated into systems, as well as providing evidence for contextual relevance, alignment with national priorities, and seek examples of effective local resource mobilisation. It will respond to the below key evaluation questions:

1. How effective was the intervention in linking key online safety actors? What strategies were used by the project to enable this?

(Relevant OECD Evaluation Criteria: Relevance )

1. Is the intervention design and objectives aligned with the needs of the government? What opportunities exist for continuation/sustainability of the activities?

(Relevant OECD Evaluation Criteria: Coherence, Sustainability)

• Political support/buy-in

This domain will investigate the extent to which the project’s strategies and interventions improved awareness and support from government resources, and investigate the ways in which advocacy efforts have resulted to government-allocated resources to potentially initiate scaling. It will respond to the below key evaluation questions:

1. How aware are relevant government officials of the intervention? Has there been any indications of public support from government officials to the intervention or the theme/issue that the intervention directly addresses?

(Relevant OECD Evaluation Criteria: Impact, Relevance)

1. How effective are the intervention’s advocacy strategies, or in raising the awareness and/or securing the support of government? What worked and what didn’t?

(Relevant OECD Evaluation Criteria: Effectiveness)

• Community sustainability/embeddedness

This domain will investigate how the project contributed to or enabled integration of interventions across local and national systems, applicability of introduced elements across a range of settings, and understand compatibility by understanding the extent it has leveraged or improved existing structures and/or practices. This domain will also investigate the level of stakeholder buy-in and the intervention’s perceived value; and will thus respond to the below key evaluation questions:

1. What changes have the target participants seen in the community? How did these changes come about? To what extent are these changes attributed to the intervention? How are they planning to sustain these changes?

(Relevant OECD Evaluation Criteria: Impact, Sustainability, Relevance)

1. KEQ: To what extent did the intervention address the needs of target stakeholder groups?

(Relevant OECD Evaluation Criteria: Relevance)

1. KEQ: How well did the intervention consider the context with which it was being implemented?

(Relevant OECD Evaluation Criteria: Effectiveness)

1. KEQ: Did the intervention target the appropriate stakeholder group? Were there other groups that could have been included? To what extent does the intervention address a key need/problem/issue?

(Relevant OECD Evaluation Criteria: Relevance)

• Replication in other contexts and Institutional implementation capacity

This domain will investigate promising indications of continuing/ongoing activities existing outside of the project’s targeted scope, the fidelity of its various implementation components, and level of transferability, or ability, of key institutional partners and actors to sustain the outcomes of the intervention. This domain will respond to the below key evaluation questions:

KEQ: Were there any changes/variations to project components in the course of project implementation?

(Relevant OECD Evaluation Criteria: Efficiency)

1. KEQ: Did the design and implementation of the intervention consider the available capacities?
2. KEQ: Which strategies or elements of the intervention present promising indications of target stakeholder ability to ensure that resulting outcomes are sustained?

(Relevant OECD Evaluation Criteria: Sustainability)

As above, clarity whether we mean political I.e ministerial or departmental/I.e line ministry