Competition versus relationship: Youth services seen from New Public Management or ecological perspectives
In the youth services field, moves to the market-oriented, competition-based policies of New Public Management (NPM) and New Public Governance (NPG) have impacted on service funding, service delivery, working with other services and accounting for value. Under these models, services are purchased, organisations are producers of social outcomes and service users are rational and self-interested consumers, and evaluations focus on quantifiable effects, often measured in cost/benefit terms. While recognised as being central to the youth services field, relationships between services are expected, but undermined, through the processes of NPM and NPG. An alternative to these quantification and finance-oriented rationales is to take an ecological approach that sees services as inextricable from the relationships in which they are enmeshed. These relationships are integral to service achievements and service identities. Evaluation strategies that support story-telling are more effective in allowing the significance of these relationships to be heard. This paper aims to demonstrate that ecological perspectives can help to re-view the service landscape in valuable ways and help to make visible important elements that go unnoticed when competitive and market-based approaches are used. Two key findings from a recent evaluation process presented here illustrate how the ecological experience of services might be recognised, and implications of this for service development and evaluation are briefly discussed.
Key words: youth services, ecological perspective, New Public Governance, New Public Management, evaluation, Indigenous youth
The contemporary policy context of youth services is fraught with contradictions leading to a schizophrenic existence: they are expected to collaborate, while at the same time are forced to compete for their existence; expected to be innovative, while promising specific program outputs and outcomes; and encouraged to be local, while economies of scale privilege state-wide or national, corporatised agencies. This conflict appears to be predicated upon a competitive and market approach to service development and delivery, which tends towards service monopolies, while recognising that meeting complex human needs is more than one agency alone can achieve. A small number of services survive even though many and different services are required. Some of these tensions are identified by the Australian Productivity Commission’s (2017) recommendations for reforms of the human services sector. The issues are more strongly stated by Smith and Phillips (2016).
This paper argues that funded youth services (like all other funded human services) are largely thought about, funded and evaluated atomistically, as though they are individual and independent units that are expected to connect and work together, but whose achievements (or failures) are the result of their individual efforts and competencies, or lack there-of. An alternative perspective might be that services are one of many overlapping and mutually influential elements of an ecology, where each element is interdependent and intersubjective; each service shapes, and is shaped by, others with which it comes into contact. Further, they each rely upon others for their outcomes and each is what it is because of the existence of other elements in this ecology. This has implications for just and effective service funding as well as for program evaluation. This paper aims to demonstrate that an ecological perspective can help to re-view the service landscape in valuable ways. Ecological perspectives can help to make visible important elements that go unnoticed when competitive and market-based approaches are used.
To begin this discussion, this paper uses some contemporary literature about the influence of New Public Management (NPM) and New Public Governance (NPG) on the service landscape. It then describes different ecological perspectives, arguing for those that promote intersubjectivity and interdependence. The paper next illustrates how this was reflected in recent evaluative work for an Indigenous Justice Program on the Central Coast of NSW and considers some implications for ways that we see, fund and evaluate programs.
In Australia, the UK, North America and New Zealand, human services policy and provision have been substantially reformed by NPM and NPG (Carson & Kerr 2014). Beginning in the 1980s, NPM increased the degrees of external management of funded services through increasingly regulated and competitive forms of service purchasing and contracting (Barraket, Keast & Furneaux 2016; see also Considine, Nguyen & O’Sullivan 2018). Following this, since the late 1990s, NPG has been developing as a process by which governments are more consciously integrating human services into strategies of governance. Such services are more deliberately utilised by governments to achieve their vision of public value and the social good (Barraket, Keast & Furneaux 2016).
NPM has focused on the marketisation and commodification of human services delivery (Healy 2009; Sercombe 2015; Smith 2018), while service users are reconceptualised as rational and self-interested consumers participating in transactional activities to meet their requirements (Healy 2009; Sercombe 2015; Smith 2018). Service providers are expected to be enterprising, entrepreneurial and competitive (Smith & Phillips 2016). At the same time, funding bodies expect them to collaborate and form strategic partnerships, although the extent to which this is happening in a committed way is not clear (Carson & Kerr 2010). Within a competition and market model, smaller and community-based organisations are at a competitive disadvantage compared to “big charity” (Onyx, Cham & Dalton 2016, p.175), where economies of scale support operational efficiencies and a perception of more reliable risk management infrastructure (Onyx, Cham & Dalton 2016). In NPM, reporting requirements are substantially increased and accountability is oriented towards quantification of activities and impacts in order to meet the dictates of performance-based contracts (Onyx, Cham & Dalton 2016; Sercombe 2015; Smith 2018). Mechanisms such as unit costs, which provide a quantification of the amount the purchaser (the “funding” body) is willing to pay for service provision to each individual client, may be used to set service delivery levels. Additionally, rather than telling the stories of practice that could reflect the part a human service plays in “the art of living well” (Sercombe 2015, p.104), cost/benefit analysis and social return on investment are used to articulate a service’s value (Onyx, Cham & Dalton 2016).
Sercombe (2015) argues that all knowledge-making involves a process of selecting what to pay attention to and what to ignore; what to admit and what to omit. Under NPM, human services work is framed by risk management rather than relationship-building (Healy 2009) and care work is devalued (Carson & Kerr 2010; Healy 2009). In this context, non-government organisations were found to have difficulties attracting suitable staff and were confronted by high levels of staff turnover due to a range of factors, including uncertain program funding and lower pay levels (Carson & Kerr 2010). NPM seeks a knowledge base about repeatable, transferable and efficient practices against which services can be compared, and to eliminate the messy, less controlled and idiosyncratic dimensions of professional judgement and decision-making. Rather than being technicians in a precision factory, though, Sercombe (2015, p.111) argues that youth workers, and by consequence the programs in which they operate, “work in the infinitely variable flux of real relationships in an ever-changing social matrix, where those variations are precisely the things that are important”. NPM and NPG’s reductionism and commercialism invite the analogy of organisations as factories manufacturing units of “human service”, which are supplied into a consumer market. They are compelled to account for their efficiency as well as quality, compete for market share through strategies that include brand recognition and loyalty, and (ac)count their value in terms of outcome measures, weighed up in terms of costs versus benefits.
An alternative to this commercialisation might be to see organisations, services or programs through a metaphor of ecology. Ecological perspectives see, at a minimum, individuals (whether a person or an organisation) not in isolation and in competition with each other, but situated in contexts and formed in relationships. Bronfenbrenner’s “bioecological model of human development”, for example, sees human development occurring through dynamic relationships (process), with the individual and context having influence each upon the other (Bronfenbrenner & Morris 2007). The model includes different levels of influence, conceived as layers, hierarchically organised by proximity and strength of influence, each larger one containing the previous one(s). The important role of time as a feature of the bioecological framework and the (generally neglected) recognition that “objects and symbols” (Bronfenbrenner & Morris 2007, p.797) are part of the two-way relational developmental process are other important elements of the Bronfenbrenner approach. Moving away from individual human development to more collective issues, human/social approaches to ecology recognise interconnectedness of all entities and the potentially wide-ranging and unpredictable consequences that can occur across the system from change in any one area (Rotello 1997). Another dimension in a socio-ecological frame is geography – all forms of “environment”, including physical location. This is an ecological element specifically named by Levins and Lopez (1999) in their consideration of ecological perspectives on health in the US and the need for a non-reductionist and holistic approach. Finally, ecological approaches emphasise dynamism (White, Rudy & Gareau 2015) – that is, things are never entirely stable or still, but are continually changing (slowly or rapidly), adjusting or radically reforming.
Bronfenbrenner (1979, 2007) and others who use systems approaches to ecology suggest multiple independent units interacting in a particular context of time and place, thus retaining a degree of reductionism, even within a perspective of holism. Disturbing this approach are those that either soften or completely erase the boundaries between ecological entities and remove the backdrop of context (see, for example, Morton 2007). In these approaches, entities may be seen as having a general degree of coherence but also uncertain boundaries or borders, having edges that are, at best, porous (Latour 2017). Following from this understanding, there is a recognition of continual interpenetration between elements, such that “the internal becomes part of the external” (Levins & Lopez 1999, p.288) and the external part of the internal. For Latour (2017), writing about Gaia, ideas of inside/outside are erased, as are the ideas of levels of ecology containing other levels, as per Bronfenbrenner’s (1979) “nesting”. Subsequently, there is no place for an observer to stand outside of the ecology. Further, extrapolating from Latour’s (2017, p.63) statement that “you can no longer distinguish between organisms and their environment”, ultimately that which belongs, or is attributable, to any particular entity is scarcely discernible. Within this ecological view, agency also becomes much more distributed, multi-directional, and the origin of action and/or change harder to identify. These ecological perspectives invite us to see entities as being entirely reliant upon relationships for their ability to be what they are and their ability to do what they do. These relationships include interactions with other entities, dependence upon non-human elements (Barnett 2017), while geography/place and time/history (Levins & Lopez 1999) are also significantly influential. Finally, ecologies do not pre-exist their being named. That is, an ecology is formed as a consequence of its being named/described by a participant/observer and, like all other entities in this ecological view, any ecology of interest has particularly porous boundaries.
While competitive, commercial and quantitative approaches to human service delivery could utilise systems’ ecological perspectives on the basis that they generally retain high degrees of individualism, with inherent reductionism and attribution of agency, “distributed” ecological perspectives do not lend themselves to such uses. When thinking about how to account for services and the differences they make for people or places, providing people with opportunities to give a narrative description and tell stories is likely to provide a better fit.
The importance of an ecological perspective was highlighted in a recent piece of evaluation work the author undertook for an Indigenous Justice Program on the Central Coast of NSW. In the rest of this paper, the author presents a description of that program, some indications of general findings from the evaluation, a discussion of two key findings, and closes with a brief section on implications of this argument.
Background to the program
Stopping the Revolving Door is a program provided on the Central Coast of NSW by Regional Youth Support Services Inc. (RYSS). In the local area it is more commonly known as the Indigenous Justice Program (IJP). The program was originally funded in 2013 by the Australian Government’s Attorney General’s Department, as part of the broader Indigenous Justice Program, which funds “projects that seek to reduce contact of Indigenous Australians with the criminal justice system and improve community safety … by reducing offending, victimisation and incarceration of Indigenous people” (Attorney-General’s Department 2013, p.57). It started operation in July 2013.
RYSS IJP provides intensive case management services, including the option of independent housing, for young Indigenous people who have been in the juvenile justice system or are at risk of entering it. It was modelled on a non-Indigenous program already delivered by the service. The IJP grant application followed consultation with, and support from, key local Aboriginal groups. Some of those initial relationships have remained, such that individuals remained connected with the program and served as important consultants at key points in the development and delivery of the program.
The program design process, in which the author was involved, targeted “criminogenic needs” (Spiranovic et al. 2015). It sought to integrate a knowledge base of strategies for addressing the various factors empirically associated with offending that are amenable to change (Stewart et al. 2014), such as drug and alcohol issues, homelessness and housing instability, lack of education and training, disconnection from culture and family, and mental and physical health issues. Specific cultural dimensions were integrated as explicit elements of the model, as well as in all aspects of delivery.
Evaluation of the IJP
The initial development of the IJP included a plan for evaluation, which, in turn, included a mid-point process evaluation (see below) and an outcomes evaluation set for the end of the original three-year contract. To support the evaluation, a logic model was developed that depicted the links between needs, actions and short- and longer-term outcomes for the young client group. The actions were focused on making a difference to a range of issues identified as connected with offending for young people (Indigenous young people specifically, where this information was available), with the intended outcomes being directly aligned with the actions. The logic model acknowledged the intrinsically relational nature of the young clients’ lives, with connection to culture, community and family all included as important actions and results for the program. The logic model provides a reference point for evaluation by clearly identifying what the program is aiming to do and what it is aiming to achieve. Multiple methods are needed to identify the degree of success in these areas, as well as how these methods were achieved, for whom they were better and less well achieved, as well as the experiences of the program.
In 2014/15, half way through the program’s three-year timeline, a process evaluation looking at the implementation of the IJP was undertaken. This assessment found the program was being delivered as intended, with positive levels of client engagement and promising short-term outcomes. It found that the program had a clear theory of change – as well as being focused on individual needs and providing individualised case planning. The service partners who participated in the evaluation reported being pleased with the program, experienced its collaborations as effective, and saw it meeting the needs of the young people it was serving. Cultural engagement was part of the program but did not take the place of meeting other needs such as accommodation and living skills and addressing health and wellbeing needs.
In mid-2016, at the end of the three years of operation, an evaluation of the outcomes of the IJP commenced. This ongoing work is a multi-stage evaluation, with a series of different steps being undertaken over a period of two years. The first phase of the evaluation, described and reported briefly here, took a “story-telling” approach. It focused on hearing about service providers’ relationship with the RYSS IJP over time and their views about which young people, generally, the program did and did not work for and why. These focusing questions are consistent with the realist evaluation approach pioneered by Pawson and Tilley in 1997 and developed by others since (see, for example, Westhorp 2014; Westhorp, Stevens & Rogers 2016). This story-telling method, in addition to being culturally meaningful, sought to make space for specific details over generalisations and interpersonal elements over data/technical information, which Sercombe (2015) identifies as reflecting an epistemological approach consistent with a youth work/professional perspective.
Method and results
For this evaluation project, information was gathered during a series of focus groups and interviews. Three small focus group events were held, with four people participating at each. The participants for these groups were drawn first from the host organisation (four workers from RYSS who were directly involved in the delivery or running of the program) and then from staff of other agencies who worked with IJP as service delivery partners. Following these meetings, service delivery partners who could not be at the focus groups were contacted and interviews were arranged; three of these latter interviews were held. In total, 16 workers participated in the focus groups and interviews. The services they represented included a community housing provider, an Aboriginal health service, a family program, a mentoring program, a program supporting young people in out-of-home care, and government agencies providing child protection and juvenile justice services.
Following data collection and initial analysis, a meeting was held to present the initial findings to, and to get comment from, the evaluation participants. All participants in the focus groups and interviews were invited to this meeting, as well as two additional people who had been consulted at different points in the evaluation process. Twelve people attended this session. During this meeting, an overview of the first analysis of information was presented. Some of the important points from this presentation included:
- Routine reporting data showing that the IJP’s clients had consistently needed support for issues including alcohol and other drugs, mental and physical health, and anger management. They had also accessed support to increase their independent living skills, education and training, and ability to maintain accommodation. In addition, culture-specific supports had been required, including “cultural identity support” and connections to community and family. While a degree of this assistance was provided within IJP, external service referrals were common and helped to meet the needs of the young people in the program. There were few needs for which there was no service available for referral.
- Young people and service partners saw IJP as a valuable program that made a difference for young people. Illustrative statements included:
- Young people who haven’t taken up IJP have ended up in the adult (corrections) system. (Participant 8, partner agency)
- If he hadn’t got stable accommodation, with support, he would have been dead. (Participant 6, partner agency)
- Participants were able to identify young people the program worked for and young people for whom it did not work as well. In many cases, success was dependent upon a good alignment between the high service intensity and the young person’s level of need.
- Staff recruitment and retention had been a significant issue for this program. This impacted on service stability. Despite staff changes, the broader stability and intimacy of the organisation meant that young people were not without a reliable point of contact.
The discussion that followed this presentation covered many points including the evaluation process (see Limitations below) and reflection on the evaluation findings. From this latter content, two syntheses of the findings were articulated. First, the RYSS IJP not only made a difference for young people in the program, it had also strengthened a substantially stretched local network of services providing support for Indigenous young people with complex needs and offending histories. In this way, IJP made a positive difference for the service sector, not just for young service users. Second, the evaluation found that the successes of the IJP were born of a fragile harmony, reliant upon a number of interdependent and precarious factors, some of which were not under the control of RYSS; that is, the program was most likely to be successful when the following elements were present and enduring:
- The young person entered the program with a sufficient degree of readiness for change.
- The young person remained engaged with the program.
- There was a good match between the intensive support offered and the level of support the young person needed.
- There was enough stability of IJP staff to remain engaged with the young person and maintain consistency of worker/relationship.
- There were enough suitable local support services to meet the young person’s needs which could not be met within RYSS.
- Collaborative service relationships were maintained, whether through formal agreements (such as memoranda of understanding, (MOUs)) or informal processes.
- The balance between practical and cultural supports was informed by the young person’s needs and readiness, not predetermined.
Discussion – evaluations and ecologies
The evaluative activity reported here is one part of a larger process discerning the effects of the RYSS IJP over time. Future evaluation activities will include review of (re)offending and case management data, as well as qualitative measures such as hearing from young people in the program through methods such as interviews or focus groups. Although these evaluation activities and the findings are not yet comprehensive, they provide an opportunity to make two comments about the program and relationships. The first is about how programs are conceptualised, the second about how programs are evaluated.
NPM creates an operational context of marketisation, competition and efficiency-based contracting (see Smith & Phillips 2016 for a thorough description of this situation) where relationships are functional/utilitarian. In contrast, the two key findings described above suggest that the RYSS IJP not only affects the lives of the young people who come into contact with it, but also affects, and is affected by, the services and communities within which it is enmeshed. The findings indicate that the RYSS IJP is one element in the local service “ecology”, where each program shapes and changes the others around it (they are inter-subjective) as well as being reliant each upon the other (they are inter-dependent). Westhorp, Stevens and Rogers (2016) describe this as programs being complex systems that are embedded in complex systems, with each affecting the other and impacting on what they can achieve. Applying these insights to service development and service “contracting” would invite processes that are fundamentally relational in nature, that recognise interdependencies and appreciate that outcomes arise within a field, not out of any one service.
Without an evaluation approach that allows these “relational” features to be seen, vital factors shaping program delivery and outcomes might be missed. Evaluations that measure reoffending (see, for example, Poynton & Menendez 2015; Ringland 2016) or even multiple aspects of a program but not its external relationships (such as, Cunneen & Luke 2007; Spiranovic et al. 2015) miss an aspect of vital importance to these programs. Realist evaluation, on the other hand, is a method that attends to a service’s setting – its ecology – by asking what about a service’s context allowed or inhibited its mechanisms to fire in order to achieve the outcomes it did or did not achieve (Westhorp 2014; Westhorp, Stevens & Rogers 2016) and can address some of this limitation. Accepting Sercombe’s (2015, p.107) view that how we look changes what we see in the “fog of limitless connection” of any situation, this article encourages evaluation methods that incorporate attention to relationships, understood in ecological terms of inter-subjectivity and interdependence, as a constitutive dimension of program identity and efficacy.
The evaluative activities described here were not without limitations and flaws. During the participants’ meeting, where initial findings were shared, and in conversations following it, Indigenous participants identified that the evaluation had treated the young people as isolated units within their lives and not considered the connecting people, places and cultural elements that make them who they are. Despite best intentions, the evaluation had not paid sufficient attention to the relationships that were important to the community. In addition, it was pointed out that the evaluation’s focus was on funding body interests and not on community priorities. These are important limitations of this evaluation and aspects to be addressed in future work. The importance of learning what the community values, and ensuring this is addressed in evaluation, is stressed in writing about evaluation of Indigenous programs (see, for example, Hudson 2017; Rossingh & Yunupingu 2016).
Carson and Kerr (2014) describe a complex and variable landscape of policy-making and funding approaches for Australian human services over the past 30 or so years. During this time, state interactions with service providers have fluctuated between dictating terms and cooperation. More recently, agencies have been increasingly expected to operate within business models, with conceptualisations of services now based on market forces. Carson and Kerr (2014) conclude that the service delivery context has become competition-oriented, with a local devolution of responsibility but a general centralisation of decision-making power. Ecological approaches to the human services field (including youth services) start from a non-reductionist premise that foregrounds relationships with other services, with place, with time, with workers and service clients and with non-human elements. Recognising these entanglements and enmeshments in the process of funding/contracting, as well as in evaluation is a complicated but more just and more accurate way to approach youth/human services.
The author acknowledges the numerous local Indigenous people who contributed to the evaluation described here, as well as recognising that this evaluation was undertaken and the article written on the traditional lands of the Darkinjung people. The individual people and services who participated in the evaluation activities are also acknowledged – the understandings presented here arose from their contributions. Finally, to the many people, including the reviewers, who have commented upon, provided advice and made suggestions for this article, thank you.
Attorney-General’s Department 2013, Annual report 2012–13, Commonwealth of Australia, Canberra, Australia, retrieved from, https://www.ag.gov.au/Publications/AnnualReports/Annualreport2012-13/Documents/AGD_AR2012-13Full.pdf
Barnett, J.T. 2017, ‘Impurities: Thinking ecologically with safe’, Communication, Culture & Critique, v.10, n.2, pp.203-20.
Barraket, J., Keast, R. & Furneaux, C. 2016, Social procurement and new public governance, Routledge, New York.
Bronfenbrenner, U. 1979, The ecology of human development: Experiments by nature and design, Harvard University Press, Cambridge, MA.
Bronfenbrenner, U. & Morris, P.A. 2007, The bioecological model of human development, John Wiley & Sons, Hoboken, NJ.
Carson, E. & Kerr, L. 2010, ‘Contractualism, workforce-development and sustainability in the community-services sector in Australia’, Third Sector Review, v.16, n.1, pp.69-86.
—— 2014, Australian social policy and the human services, Cambridge University Press, Melbourne.
Considine, M., Nguyen, P. & O’Sullivan, S. 2018, ‘New public management and the rule of economic incentives: Australian welfare-to-work from job market signalling perspective’, Public Management Review, v.20, n.8, pp.1186-204.
Cunneen, C. & Luke, G. 2007, ‘Recidivism and the effectiveness of criminal justice interventions: Juvenile offenders and post release support’, Current Issues in Criminal Justice, v.19, n.2, pp.197-210.
Healy, K. 2009, ‘A case of mistaken identity: The social welfare professions and New Public Management’, Journal of Sociology, v.45, n.4, pp.401-18.
Hudson, S. 2017, ‘Evaluating Indigenous programs: A toolkit for change’, Research Report 28, retrieved from, https://www.cis.org.au/app/uploads/2017/06/rr28.pdf
Latour, B. 2017, ‘Why Gaia is not a God of totality’, Theory, Culture & Society, v.34, n.2-3, pp.61-81.
Levins, R. & Lopez, C. 1999, ‘Toward an ecosocial view of health’, International Journal of Health Services, v.29, n.2, pp.261-93.
Morton, T. 2007, Ecology without nature: Rethinking environmental aesthetics, Harvard University Pres, Massachusetts.
Nutbeam, D., Harris, E. & Wise, M. 2010, Theory in a nutshell: A practical guide to health promotion theories, 3rd edn, McGraw-Hill Australia, Sydney, Australia.
Onyx, J., Cham, L. & Dalton, B. 2016, ‘Current trends in Australian nonprofit policy’, Nonprofit Policy Forum, v.7, n.2, pp.171-88.
Poynton, S. & Menendez, P. 2015, ‘The impact of the NSW Intensive Supervision Program on recidivism’, Crime and Justice Bulletin, n.186, pp.1-16, retrieved from, http://www.bocsar.nsw.gov.au/Documents/CJB/Report-2015-The-impact-of-the-NSW-Intensive-Supervision-Program-on-recidivism-cjb186.pdf
Productivity Commission 2017, ‘Introducing competition and informed user choice into human services: Reforms to human services’, Productivity Commission Inquiry Report, n.85, retrieved from, https://www.pc.gov.au/inquiries/completed/human-services/reforms/report/human-services-reforms.pdf
Ringland, C. 2016, ‘Participation in PCYC Young Offender programs and re-offending’, Crime and Justice Bulletin, n.191, retrieved from, http://www.bocsar.nsw.gov.au/Documents/CJB/Report-2016-Participation-in-PCYC-Young-Offender-programs-and-re-offending-cjb191.pdf
Rossingh, B. & Yunupingu, Y. 2016, ‘Evaluating as an outsider or an insider: A two-way approach guided by the knowers of culture’, Evaluation Journal of Australasia, v.16, n.3, pp.5-14.
Rotello, G. 1997, Sexual ecology: AIDS and the destiny of gay men, Dutton, New York, USA.
Sercombe, H. 2015, ‘The watchmaker’s chainsaw: Why New Public Management is the wrong tool for youth work (and most of the professions)’, Journal of Applied Youth Studies, v.1, n.1, pp.97-121.
Smith, S.R. 2018, ‘The future of nonprofit human services’, Nonprofit Policy Forum, v.8, n.4, pp.369-89.
Smith, S.R. & Phillips, S.D. 2016, ‘The changing and challenging environment of nonprofit human services: Implications for governance and program implementation’, Nonprofit Policy Forum, v.7, n.1, pp.63-76.
Spiranovic, C., Cockburn, H., Bartels, L. & Julian, R. 2015, ‘Outcome measures for evaluating the effectiveness of juvenile justice programs’, Victoria University Law and Justice Journal, v.5, n.1, pp.23-33.
Stewart, J., Hedwards, B., Richards, K., Willis, M. & Higgins, D. 2014, Indigenous Youth Justice Programs Evaluation, Australian Institute of Criminology, retrieved from, https://aic.gov.au/publications/special/005
Westhorp, G. 2014, Realist impact evaluation: An introduction, Methods Lab, retrieved from, https://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/9138.pdf
Westhorp, G., Stevens, K. & Rogers, P.J. 2016, ‘Using realist action research for service redesign’, Evaluation, v.22, n.3, pp.361-79.
White, D.F., Rudy, A.P. & Gareau, B.J. 2015, Environments, natures and social theory: Towards a critical hybridity, Palgrave, London/New York.
Chris Krogh is a lecturer in human services at the University of Newcastle, Australia. Informed by his earlier work as a practitioner, Chris’ ecologically-informed research interests include the power of professional writing and the human impacts of changes to physical places. email@example.com
 This is not to suggest that there is no place for numerical data and statistics. These are likely to be valuable for measuring change with a person, place, group or thing. These data are less useful for accounting for the difference any one service, program or activity has made.
 A logic model is a diagrammatic way of depicting the different elements of a program, what it is aiming to do and what it is aiming to achieve. These are often presented in terms of existing situation, resources, activities, outputs and outcomes (shorter, mid-range, and longer-term).
 A theory of change is a logical way of working to convert the activities undertaken to results for the program participants (Nutbeam, Harris & Wise 2010).