1. Introduction
Emerging global consensus around the risks posed by climate change has led many countries, including the United Kingdom (UK), to implement carbon reduction commitments. A key strut of such plans is the increasing penetration of renewable electricity generation, and the associated storage required to help manage variability. While renewable generation need not necessarily be highly decentralised, there are a variety of reasons why this situation may arise and may (or may not) be desirable. These include individuals and organisations deciding to invest in onsite distributed energy resources, and the need to meet substantially increased electricity demands associated with heating and transport while minimising costly infrastructure upgrades (HM Government, 2017).
Countries around the world are therefore exploring how distributed energy resources (DER) can be optimally combined and coordinated to support affordable and secure decarbonisation (Burger et al., 2020). In the UK, the concept of “smart local energy systems” has gained prominence as a concept around which to arrange this idea of coordinated decentralisation. Their development is being explored and promoted through the £100m Government-funded Prospering from the Energy Revolution programme (PFER) (Innovate UK, 2018). Important to note is that while effective delivery of energy services is a key objective of SLES, they are expected to go well beyond this in the benefits they can deliver. For example, they are seen as a way of creating high value employment and reducing the cost of energy (UK Research and Innovation, 2022).
The emergence of SLES at meaningful scale would entail a transformation of the way most countries’ energy systems are structured, planned, and operated. This is not a change that can be delivered solely by existing energy industry incumbent actors. Action and engagement is required for a wide variety of stakeholders, including system users, technology installers, and local and national policymakers. There are big implications for the nature and scale of technical infrastructure. And the investment required for such a transformation will be substantial in scale, and likely diverse and novel in source. We therefore view the transformation of energy systems in the UK towards SLES, if it happens, as a “societal project” on a very large scale.
In this paper we describe the development of a “Theory of Change” (ToC) for the societal project of the emergence of SLES, and consider the insights and challenges provided by such an approach. As discussed further in the next section, ToC is a well-established way of describing in detail how changes are expected to come about that are needed to deliver intended outcomes (Center for Theory of Change, n.d.). We have two main aims:
To reflect on the process, benefits, and challenges of using ToC to structure knowledge emerging from a large interdisciplinary research consortium, EnergyREV.
To explain how the ToC can be applied by stakeholders such as policymakers, civil society organisations, and in industry to inform and evaluate progress towards delivery of successful SLES.
Following a short section on the background and principles of ToC, we then set out the approach we followed to construct the EnergyREV ToC. The ToC is then introduced, and we consider what insight it brings to the potential for emergence of SLES. We also discuss opportunities the approach brings for coordinating and navigating large, multidisciplinary research projects as well as the challenges we encountered in its development and communication. We conclude with a discussion of the policy implications the ToC raises for SLES.
2. Theory of Change
Theory of Change is now a widely used approach to support the development and evaluation of projects and organisations aiming to create change in the world. While definitions vary, it encompasses a process of (usually) collaborative working to identify key intended longer term outcomes, and pathways by which planned activities are expected to deliver these outcomes (Stein and Valters, 2012). Originally used most in the community initiative and international development sectors, it is now widely used across a range of topics as a tool for project/programme planning and evaluation. It is increasingly seeing use in topics related to energy and low carbon transitions (Dreyer et al., 2021; World Benchmarking Alliance, 2022), and is commonly used as a planning and evaluation tool for energy and climate policymaking (Department of Business, Energy and Industrial Strategy, 2020; Scottish Government, 2021).
The key feature of ToC approaches is to make the logic of interventions explicit – that is, the intermediate steps by which activities lead to outcomes, the assumptions underlying these, important aspects of the context, and associated risks and unintended consequences (Stein and Valters, 2012). Through doing this it can help collaborators identify and agree on key project outcomes, and the activities which might have the greatest chance delivering them. It can also highlight ways that activities might need to be done differently, or in combination with other supporting activities, to maximise the chances of success (depending on contextual factors) and minimise risk (Allen et al., 2017).
Explicit description of the logic for change also makes ToC approaches particularly amenable to process evaluation (Rogers, 2014). Process evaluation involves investigating to what extent the activities and intermediate steps of an intervention are happening, before real impacts would be observable (or where they are hard to observe) (Moore et al., 2015). This allows early recognition of whether a programme appears to be working, or whether a change of tack may be needed, thus reducing the chances of failure.
As part of the PFER government funding programme introduced in the previous section, funding was made available for a large, interdisciplinary research consortium known as EnergyREV. Involving researchers from 18 academic institutions across the UK, it would conduct research related SLES across technical, social, and economic domains. The scale and breadth of the consortium presented a challenge for how best to synthesise the knowledge generated through the work. It was decided to employ a ToC approach to provide structure to this process of knowledge synthesis, and to help provide a logically organised “way in” to the findings and ideas for external stakeholders. ToC approaches have previously been used to inform the design and conduct of research projects (Vogel, 2012), but the driver for its application within EnergyREV was different. Here it is not intended to reflect the research activities that are planned, but rather the “societal project” of the emergence of SLES with intended outcomes – the focus of study, rather than the study itself. In this way, as described above, the aim is to provide a common object which different disciplinary and institutional teams across a large consortium can interact with and update together to reflect something of a consensus view.
3. Method
This section describes the process by which the EnergyREV Theory of Change was constructed. As outlined above, the value of ToC arises not just from the final product or diagram, but through the process of collaborative development. We were therefore keen to build it with plenty of opportunity for discussion between members across the consortium. However, the consortium was highly geographically dispersed and multidisciplinary. Whole consortium meetings were only feasible on a infrequent basis, and time available for discussion of any single part of the project was highly constrained. This limited the opportunity to hold traditional ToC workshops (Breuer et al., 2014). Instead, we followed a staged process. Initial meetings were held with consortium work packages individually. Work package members were introduced to the concept and purpose of the ToC, and a facilitated discussion held with them on:
from the perspective of their work package, the key long-, medium-, and short-term outcomes that SLES should deliver, and the preceding conditions needed for these to come about
the most important stages within their work package area required for SLES to emerge that can deliver these outcomes
what negative or unintended outcomes they were concerned about
the exogenous assumptions underpinning their expectations
any key contextual factors they expected to affect outcomes
We used notes of these meetings to construct sub-ToCs showing the conditions needed for SLES to come about and deliver good outcomes from the perspective of each work package. An example provisional sub-ToC for the business and financial practices work package is shown in
Figure 1. These mini-ToCs were shared with the relevant work package and feedback and edits requested.
The next stage of ToC development was conducted as part of a whole consortium meeting. We prepared materials in advance including printouts of the individual work package sub-ToCs, and large posters showing all the sub-ToCs collected on the same page. In the first half of the workshop work packages had a further opportunity to reflect on and refine their own sub-ToCs, for example by adding conditions and connections. In particular, they were asked to try ensure that they only included conditions which they believed would be
necessary to the delivery of SLES and subsequent good outcomes in the vast majority of SLES. Or framed another way, that the absence of such conditions would seriously inhibit the likelihood of successful delivery. In the second half, participants were divided into new (cross-work-package) groups and asked to indicate what they saw as the key connections between sub-ToCs. For example, this might be where they viewed a condition in one sub-ToC as an essential pre-condition to a condition in another work package area (
Figure 2). Through the course of this process, consortium members had the opportunity to consider and express the key pathways and outcomes in their own area of work, think about co-dependencies with other work packages, and see where other consortium members saw co-dependencies with their work package.
Informed by the edits and connections suggested by consortium members in the workshop, we developed a provisional EnergyREV ToC (see (Fell et al., 2020). Moving from right to left, it shows ultimate, high level intended outcomes of the emergence of SLES across the UK; the necessary preconditions of these that SLES must provide; and the necessary preconditions for SLES to emerge. It is organised in a number of sectoral layers referred to as “challenge areas” with interconnections between them. A number of important contextual factors and general assumptions are also listed. The provisional ToC, and subsequent developments, were created and maintained in the online diagramming service Lucidchart. The content of the ToC, and how it can be used, are discussed further in the next section.
The provisional ToC was based on the expert expectations of the consortium members. The next stage of ToC development was to populate it with evidence emerging from the work of the consortium, amending its structure to reflect the evidence where appropriate. We did this on a semiregular basis throughout the life of the project. Evidence could be added in two ways: in support of a causal link between two conditions, or simply relevant to a condition. We extracted key findings from EnergyREV outputs and summarised them in text boxes which were added to the ToC. Specific findings were then associated with specific conditions or links by means of clickable features on the ToC, toggling the evidence into and out of view (
Figure 3). In this way it was possible to include a large quantity of extracted evidence on the ToC diagram without obscuring it.
The ToC also incorporates other layers which can be toggled on and off. One shows the policy or regulator precursors which EnergyREV evidence suggests are needed to underpin certain necessary conditions. The other shows metrics that can be used to assess progress against delivery of certain conditions, as compiled by the EnergyREV work package task with developing evaluation approaches for SLES. A summary ToC is also provided as a layer, which presents the main conditions in each challenge area, summarised into single boxes.
The ToC was further reviewed by the whole consortium at a subsequent meeting, this time conducted online due to the Covid-19 pandemic. This resulted in some changes to the layer structure, served to check changes that had been made to the structure following addition of evidence, and offered an opportunity to add previously unrecognised links between EnergyREV evidence and the ToC. The ToC continues to be maintained on an ad hoc basis as new EnergyREV evidence becomes available.
The next section summarises the key functionality of the interactive ToC. We then highlight some key use cases we envisage and reflect on challenges we have observed in employing the ToC approach in this context.
4. Results and Discussion
4.1. Functionality and Overview
The most recent version of the interactive ToC is available online at
https://tinyurl.com/energyrev-toc. A full version of its state at the point of submission is provided in the supplementary material.
Figure 4 gives an impression of the overall ToC (the online or supplementary versions should be used to view detail).
When users navigate to the ToC they are greeted by a welcome screen with basic instructions on how to navigate the LucidChart interface. LucidChart allows for basic interactivity through the toggling on and off of layers of the chart when users click on specified hotspots. Users are invited to either hide or reveal the high level summary ToC, and then close the navigation instructions. The summary can then by hidden or revealed using the appropriate control on the chart. Further more detailed instructions appear to the left of the ToC and are permanently available.
As described in the methods section, the ToC is broken into eight challenge areas collected into three broader groups, as follows:
- ◦
Technology/system interactions
- ◦
Data and digital
- ◦
Users
- ◦
Skills
- ◦
Business, finance, and organisations
- ◦
Heating and cooling
- ◦
Mobility
- ◦
Natural ecosystems
A central vertical line crossing through all the challenge areas represents the point at which a SLES is delivered. Everything to the left of the line represents the preconditions seen as necessary to achieve delivery of SLES, which are preceded by their own necessary preconditions, and so on. To the right of the central line are conditions required for SLES to deliver intended beneficial outcomes, including enabling prosperous communities across the country, meeting carbon reduction targets, and growth in productive exports. These appear in green boxes at the far right hand of the diagram (and are repeated for each challenge area as appropriate to avoid cluttering with lines). Also depicted are negative and unintended consequences associated with the delivery of (or failure to deliver) specific conditions.
Table 1 shows a high level summary of the these conditions.
Users are able to show or hide all or a selection of the underlying EnergyREV evidence. This is achieved by clicking a show/hide all evidence button, or clicking hotspots associated with specific links or boxes on the detailed ToC. There are two separate evidence hotspot icons:
A green arrow indicates the existence of EnergyREV evidence specifically supporting the existence of a link between two conditions in the ToC.
A blue circle indicates EnergyREV evidence or recommendations relevant to a condition on the ToC, but not directly supporting it.
When either of these icons are clicked, a text box appears connected to the icon with an arrow. Each text box represents a single EnergyREV output (unless otherwise indicated), and it may connect to more than one icon. That is, it may be relevant to more than one link or condition on the ToC. The text box contains the title of the output (with hyperlink to the original document), lead author name, a line summarising the approach or method, and a numbered list of key findings. Where a specific finding relates to the link or condition, the number of the finding in the list is included in the arrow or circle icon to allow ease of tracing. At the time of writing over 20 EnergyREV outputs have been added to the ToC.
As described in the methods section, users are also able to show or hide layers showing key policy and governance preconditions, and possible evaluation criteria that could be used to measure progress against the conditions or links. The policy/governance boxes are also linked to EnergyREV evidence where possible. Evaluation metrics are based on the work of an EnergyREV work package with this topic as its focus (Francis et al., 2020).
4.2. Applications
The ToC is designed to have applications both within and beyond the EnergyREV project. A key external use is to inform strategy and planning in relation to SLES. The starting proposition of the EnergyREV ToC is that, in order to successfully deliver SLES with the intended benefits, it is necessary to deliver all of the conditions identified. Organisations involved in SLES planning, delivery, and policy can therefore work through the ToC and consider:
Whether there are special contextual reasons which mean that a particular condition is not necessary for delivery of their SLES, and;
Where they agree with a condition:
whether the condition is already met in the locality
if not, whether there are plans to deliver it, involving which actions, by whom
The rationale for this approach is that it is better to consider and dismiss a condition as irrelevant, than to neglect to consider a condition which may prove to have been necessary to achieve intended outcomes. We have previously developed materials which organisations can use to structure workshops on this topic (Fell et al., 2020).
The ToC also provides a novel way for users to access research and guidance in the form of project outputs. Conventionally these are stored on websites and are browsable by topic/type or can be returned by searches. The ToC provides an alternative to this in that it provides access to summaries and links to outputs in context and at a glance. This is intended to make it easier both to identify literature relevant to the users’ interests and stage of SLES progress.
As discussed in the section on background to ToC, the approach can also usefully inform process evaluation. Because of the scope and ambition of truly systemic SLES projects, ultimate outcomes might only be expected to be delivered years into the future. Informed by the ToC, users can identify intermediate indicators and metrics that allow monitoring and evaluation of the extent to which planned activities are being carried out and leading to short- or medium-term outcomes. This can provide early insight into whether planned approaches are working, or need to be amended.
The interactive ToC has performed, and has potential to perform, a number of roles within and beyond the EnergyREV project. During its development, the ToC was able to act as a “boundary object” (REF) between the various work packages of the consortium. EnergyREV is similar to other large research projects in that it involves multiple teams of researchers at different institutions and across a range of disciplines. Unless ways can be found to meaningfully connect and relate the different workstreams together, there is a risk that teams lose sight of what others are working on, and for interdisciplinary insights to be missed. This can be partly addressed through conventional meetings with update presentations, but these present limited opportunity for consideration of the mutual impacts of work, findings, and recommendations across teams. The ToC was used several times as a focal object during consortium meetings as the basis for collaborative activities (beyond those designed to develop the ToC as described in the Methods section). For example, one exercise involved identifying the key actor(s) associated with each condition in the ToC, and prioritising the conditions based on importance and urgency.
4.3. Challenges
Our work in developing and communicating the ToC has highlighted several challenges. A key difficulty has been in effectively communicating about the potential value of the EnergyREV ToC to private sector stakeholders. As described above, the approach originated in development studies as a way of helping to plan and evaluate projects. It has subsequently found quite wide use in the spheres of policy evaluation and academic research. However, it does not appear to be a widely used approach in businesses. On describing the approach we encountered critique around how it could bring value to a business looking to profit from delivering products and services in a SLES. In part, this is probably due to the aims of this ToC being different – that is, to provide a system-level view of the change needed to deliver successful SLES. Beyond scoping out the potential role organisations could play in this change, it does little to provide information on the specific actions that they should take to achieve this. This is because these are likely to be highly context specific and also varied. However, this is not to say that the ToC could not be developed to provide more obvious value to stakeholders such as businesses. Because of the ability to add links and layers, it would be possible add business case studies. Alternatively, it could be used as a resource to inform use of more familiar business tools such as the Business Model Canvas (REF).
While it may appear trivial, the name “Theory of Change” may also present a barrier. It gives the appearance of a tool that may be of theoretical (but not practical) use. Consideration could be given to alternative framings for certain audiences.
A related challenge has been balancing complexity and interpretability. As a ToC for the “societal project” of the emergence of smart local energy systems, there is a great range of activities and factors that could potentially be included. They are also linked to each other in complex ways, which often involve feedbacks. In order to create a legible and usable final product, it has been necessary to omit a lot of nuance and detail. Again, this is fine where the usefulness is around gaining a whole system overview, but is less useful in more applied cases. For example, there are likely to be many necessary conditions on route to getting financing for aspects of a SLES project that are simplified into a single box. This makes the ToC useful for gaining a high-level, whole system view of things, and subsequently drilling down into detail where necessary – but not suitable for conveying more detailed guidance in its own right.
It is important to be transparent about challenges to the validity of the EnergyREV ToC. It was originally constructed based on the expectations of consortium members. Subsequently evidence emerging from their research was brought to bear on it. However, so far, much of the evidence is not of a nature that is able to fundamentally test the existence of causal links; rather it adds useful detail. In the case of findings that specifically support links in the ToC, in some cases this emerges from modelling work that may, in part, have been informed by similar expectations to those that informed the ToC originally. It will only be genuinely possible to assess the overall validity of the expectations mapped out in the ToC as more examples of SLES arise and are made subject to study, such that the existence (or not) of our expected necessary conditions can be established. In its current form, the ToC should be viewed as a structured representation of evidence-informed expert views, rather than a mapping of empirically demonstrated causal links.
The EnergyREV Theory of Change is designed to be a living document, and – funding permitting – has potential to be developed in a number of ways. Beyond operational updating to include more recent outputs, the most important of these concern the ways in which users can interact with it. The complexity of the current display (as described above) may present a barrier to some applications. But the ToC can be used as a starting point to develop a range of tools to address more specific uses and audiences, and such work is underway.
5. Conclusions and Policy Implications
There is significant policy momentum behind the development of smart local energy systems (SLES), whereby more energy services are provided by energy generated and managed locally. As well as supporting decarbonisation, such SLES are often envisaged as providing wider benefits for communities, such as high quality employment. However, neither the successful delivery of SLES, nor their ensuing benefits, is assured. This paper has described the rationale for, construction process, and delivery of a Theory of Change for the “societal project” of how SLES are expected to deliver desired outcomes. Broken down into a range of challenge areas (grouped under: technology and system coordination; people and organisations; and key services) it presents short-, medium-, and longer-term necessary conditions for delivery of successful SLES based on the expectations of expert participants in the Energy REV research consortium. The structure and content has been updated over the life of the project to include new research evidence from the consortium. In addition to the pathways and evidence summaries, the ToC also includes detail on key policy and governance conditions, and suggested metrics for evaluation.
The ToC raises a range of specific policy implications. Embedded within it are a range of policy and governance measures which may help foster necessary conditions in the development of successful SLES. For example, the condition of having incentives and market structures that allow coordination of distributed energy resources (DER) may be enabled by measures such as: changing regulation so that customers are no longer limited to a single electricity supplier; allowing assets to participate in multiple markets; creating a level playing field for DER (e.g. ability to realise value of supporting black starts); and a network charging regime which supports effective DER coordination. These points are linked back to the underlying evidence, where this has emerged from the work of the EnergyREV consortium. The ToC also highlights more thoroughgoing policy reform that may be necessary to underpin successful SLES. For example, this involves the devolution of more responsibility and resource to local authorities to help support SLES delivery, as highlighted in the work of (Tingey and Webb, 2020).
The ToC also clearly highlights is the necessity of creating the right conditions across multiple domains to maximise the likelihood of successful SLES delivery. It makes salient the point that policy measures in individual areas – or even in energy policy more generally – are extremely unlikely to be enough to yield meaningful progress on SLES unless they are made in the context of complementary measures elsewhere. This calls for joined up policymaking that brings together different departments of government, and national and local policymakers. While this is notoriously challenging, the ToC also has the potential to act as a “boundary object” where each actor can see their own remits represented in the context of the whole.
Finally, the ToC has potential to inform process evaluation, albeit at a high level. SLES will usually be quite large, long term projects consisting of multiple initiatives and interventions. It may not be possible to evaluate whether they have truly delivered their intended outcomes for many years. Process evaluation provides the opportunity to check whether the conditions are being observed that we would expect to see on the path to delivery of the intended outcomes. Policymakers can draw on the ToC to inform and assess plans for process evaluation – again, with attention being drawn to the multisectoral nature of the challenge.
Acknowledgements
We gratefully acknowledge the input of all members of the EnergyREV consortium in helping to construct the Theory of Change. This work was funded by UK Research and Innovation Grant No EP/S031863/1 “Energy Revolution Research Consortium—Core—EnergyREV”, administered by the Engineering and Physical Sciences Research Council (EPSRC).
Abbreviations
PFER |
Prospering from the Energy Revolution (programme) |
SLES |
Smart local energy systems |
ToC |
Theory of Change |
References
- Allen, W.; Cruz, J.; Warburton, B. How Decision Support Systems Can Benefit from a Theory of Change Approach. Environ. Manag. 2017, 59, 956–965. [Google Scholar] [CrossRef]
- Breuer, E.; De Silva, M.J.; Fekadu, A.; Luitel, N.P.; Murhar, V.; Nakku, J.; Petersen, I.; Lund, C. Using workshops to develop theories of change in five low and middle income countries: Lessons from the programme for improving mental health care (PRIME). Int. J. Ment. Health Syst. 2014, 8, 15. [Google Scholar] [CrossRef]
- Burger, C.; Froggatt, A.; Mitchell, C.; Weinmann, J. Decentralised Energy, Ubiquity Press. Ubiquity Press: 2020. [CrossRef]
- Center for Theory of Change, n.d. What is Theory of Change? What is Theory of Change? Available online:. Available online: https://www.theoryofchange.org/what-is-theory-of-change/ (accessed on 2 March 2022).
- Department of Business, Energy and Industrial Strategy. Contracts for Difference evaluation phase 2 annexes evaluation of allocation round 3. Department of Business, Energy and Industrial Strategy. 2020.
- Dreyer, B.C.; Riemer, M.; Spadafore, B.; Marcus, J.; Fernandes, D.; Taylor, A.; Whitney, S.; Geobey, S.; Dennett, A. Fostering Cultures of Sustainability in a Multi-Unit Office Building: A Theory of Change. Front. Psychol. 2021, 12. [Google Scholar] [CrossRef]
- Fell, M.; Maidment, C.; Vigurs, C.; Shipworth, D. Developing an Organising Framework: How do we create successful smart local energy systems? EnergyREV report). 2020.
- Francis, C.; Costa, A.S.; Thomson, R.C.; Ingram, D.M. Developing a multi-criteria assessment framework for smart local energy systems. EnergyREV. 2020.
- HM Government. Upgrading Our Eergy System: Smart systems and flexibility plan. London, UK. HM Government. Upgrading Our Eergy System: Smart systems and flexibility plan. London, UK. 2017.
- Innovate, UK. Prospering from the energy revolution: Full programme details [WWW Document]. 2018. Available online: https://www.gov.uk/government/news/prospering-from-the-energy-revolution-full-programme-details (accessed on 18 May 2022).
- Moore, G.F.; Audrey, S.; Barker, M.; Bond, L.; Bonell, C.; Hardeman, W.; Moore, L.; O’Cathain, A.; Tinati, T.; Wight, D.; Baird, J. Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015, 350, 1258. [Google Scholar] [CrossRef]
- Rogers, P. Theory of Change (Methodological Briefs Impact Evaluation No. 2). Unicef. 2014.
- Scottish Government. Net Zero Nation: Public engagement strategy from climate change. Scottish Government. 2021.
- Stein, D.; Valters, C. Understanding Theory of Change in International Development (No. JSRP Paper 1). The Justice and Security Work Programme. 2012.
- Tingey, M.; Webb, J. Net zero localities: Ambition & value in UK local authority investment. EnergyREV. 2020.
- UK Research and Innovation. Smart local energy systems: The energy revolution takes shape. UK Research and Innovation. 2022.
- Vogel, I. ESPA guide to working with Theory of Change for research projects. Ecosystem Services for Poverty Alleviation. 2012.
- World Benchmarking Alliance. Theory of change [WWW Document]. World Benchmarking Alliance. 2022. Available online: https://www.worldbenchmarkingalliance.org/theory-of-change/ (accessed on 27 October 2022).
|
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).