1 Introduction

Urban living labs (ULLs) have arisen in cities as a response to a pressing challenge (Marvin et al., 2018): How can cities provide economic prosperity and social cohesion while achieving environmental sustainability? In this perspective, the core idea of ULLs is that urban sites can provide a learning arena within which the co-creation of innovation can be pursued between research organisations, public institutions, private sectors, and community actors (Liedtke et al., 2012). Not only in practise, but also in academic spheres, the concept of ULLs has increasingly gained interest in recent years (Schuurman, 2015; Hossain et al., 2019).

Yet, despite the growth of ULLs and their experimentation, their nature and purpose as an empirical phenomenon is still not fully understood (Bulkeley et al., 2016). This is partly because the acceleration and normalisation of ULLs in practise has proceeded much more rapidly than the development of evidence and theoretical understanding about them (Bulkeley et al., 2016; Marvin et al., 2018). As such, international comparison and systematic learning is lacking on how ULL impacts can be scaled up to achieve transformative changes (Marvin et al., 2018), and how they can effectively facilitate urban sustainability transitions (Evans & Karvonen, 2013; Nevens et al., 2013). Such transitions are about changes in markets, policy, culture, technologies, and infrastructure, as well as in human behaviours and practises (Bulkeley et al., 2010; Frantzeskaki & Loorbach, 2010; Schaffers & Turkama, 2012; Voytenko et al., 2016).

A key point therein is to examine the role of (urban) experiments to govern these transitions, and in doing urban innovation and governance (Marvin et al., 2018) to gradually transform stable regimes (Kemp et al, 1998; Schot & Geels, 2008). Existing regimes or systems seem to be difficult to pry off because they are stabilised by processes that create path dependencies (Grin et al., 2010; Loorbach & Rotmans, 2010; Neef et al., 2017). ULLs are one way to effect change (Schaffers & Turkama, 2012; Marvin et al., 2018), because they are similar in approach to “transition management” (Loorbach & Rotmans, 2010), and centre on the use of experiments, including less directed processes in which innovation and ideas are demonstrated, tested, and experienced for gain (Kemp et al., 1998; Bulkeley & Castán Broto, 2012). The degree to which these experiments lead to regime transitions seem to depend on growing social networks, innovations, and learnings that they establish (Brown & Vergragt, 2008). Existing research, however, mainly focusses on the aims and workings of ULLs instead of critically reviewing their implications (Bulkeley et al., 2016), their essence (Hossain et al., 2019), or to what extent they shape new governance modes (Marvin et al., 2018). Some challenges in ULLs, therefore, link with temporality and unpredictable outcomes (Hossain et al., 2019), such as financial sustainability (Gualandi & Romme, 2019), scalability, diffusion, and impact (Puerari et al., 2018; von Wirth et al., 2018), and the redistribution of agency and risks (Loorbach & Rotmans, 2010; Smith & Raven, 2012; Burch et al., 2018).
This study addresses this research gap by focussing on how the relationship between funding, stakeholder roles, and process outcomes in ULLs can contribute to transformative changes. The main research question is: How does the trinity of funding options, stakeholder roles, and outcomes in ULLs influence their impact creation for transformative changes in cities? Tensions between these aspects were observed by Hodson and colleagues (2018) in the UK, which are still present in today’s ULL practises (Scholl & de Kraker, 2021).

The paper is structured as follows. First, it elaborates on current literature about ULLs and the trinity under study to explore and identify current approaches and theories. Second, it explains and justifies the methodology chosen in the literature review and comparative case study in the context of the city of Groningen. Then, it provides the results of the empirical study focussed on funding options, stakeholder roles, outcomes created, and impact. Lastly, the paper presents the importance of trust building in ULLs to overcome the particular challenge under study, highlighting its theoretical and practical implications, as well as limitations and recommendations for further research.

2.0 Literature Review

2.1 Origin and positioning of urban living labs

Although the origin of the living lab movement can be traced back to the 1960s, and later, the founding of the European Network of Living Labs in 2006 (Hossain et al., 2019), the emergence of ULLs more generally started following the 2008 Global Economic Crisis. Since then, cities have struggled to find solutions to challenges faced via three sets of issues: 1) there is no singular pathway towards urban sustainability (De Jong et al., 2015), 2) interest increased in the potential of experimentation in place-based contexts to overcome rigidity in existing socio-technical systems based on private contexts (Chesbrough, 2006; Almirall & Wareham, 2011), and 3) various stakeholders, like research and technology institutions, started to see urban environments as places to support local communities, as well as grassroots initiatives that align with national innovation (Paroutis et al., 2014; Luque-Ayala & Marvin, 2015; Marvin et al., 2018). In fact, ULLs and various other parts of cities can be positioned as a form of experimentation towards a broader shift in the character of urban governance (Bulkeley et al., 2016; Evans et al., 2017; Steen & van Bueren, 2017), and as such seems to be able to enhance learning about placed-based contexts to achieve changes in socio-technical and socio-ecological systems by continuously enrolling new sites and actors (Liedtke et al., 2012; Baccarne et al., 2014; Bulkeley et al., 2016; Voytenko et al., 2016; Scholl & Kemp, 2016; Astbury & Bulkeley, 2018; Marvin et al., 2018; Steenbergen & Frantzeskaki, 2018).

2.2 Defining urban living labs

Defining ULLs has been an ongoing challenge, both in scientific studies and in practise. While Voytenko and co-authors (2016) argued that there is no universal ULL definition, Chronéer and colleagues (2019) in contrast developed a “unified” definition of living labs (LLs). In general, however, these two streams are identifiable by their opposition. On the one hand, LL definitions stem from open innovation theory and co-creation (see for example, Westerlund & Leminen, 2011; Leminen, 2013). On the other hand, ULL definitions stem from management transition and urban governance, thereby viewing the concept as a governance model in which experimentation and learning can be centred (See for example, Bulkeley & Castán Broto, 2012).
In this study, we use the following definition: “Urban living labs constitute a form of experimental governance whereby urban stakeholders develop and test new technologies, products, services and ways of living to produce innovative solutions to the challenges of climate change, resilience and urban sustainability” (Bulkeley & Castán Broto, 2012; interpreted by Voytenko et al., 2016), because it shows two aspects. First, ULLs constitute a form of experimental governance with and among urban stakeholders. Second, it underpins the shared focus on finding solutions to today’s urban challenges and reaching urban sustainability. Both are relevant, because urban and societal challenges nowadays need collaborative efforts across sectors as well as between disciplines (Evans et al., 2015; Bulkeley et al., 2016; Voytenko et al., 2016; Marvin et al., 2018; Menny et al., 2018; Hossain et al., 2019).

2.3 Urban living lab typologies

Discussion remains open regarding ULL typologies, since the cases investigated so far have differed, and the criteria that form the basis for the typologies are still not widely agreed upon. Neef and colleagues (2017), for example, differ between “Product Oriented Labs” and “Urban Transition Labs” based on the theoretical foundations of open and user innovation. Leminen and colleagues (2012), as well as Bondarenko and co-authors (2019) proposed five types of ULLs based on the stakeholder who drives the activities: Enabler, Provider, Utilizer, User, and Researcher-driven. Additionally, and as used in this current study, Marvin and colleagues (2018) empirically compared 50 ULLs across Europe, differentiating between three types of ULLs based on their geographical scale and urban dimension, see Table 1.

Table 1. Strategic, Civic and Organic ULL characteristics (Marvin et al., 2018)

Strategic ULLs contain larger scale technological development programmes procured by state intermediaries and involving private partners. They are often state sponsored, including private investments to build local capacity and enhance international competitive position. Civic ULLs involve municipal governments and local stakeholders. Their goals tend to have a strong local character and focus on urban priorities. Hence, co-funding in these LLs is widely used, in combination with private investments and national or European subsidies. Organic ULLs focus on specific local and contextual issues, like social needs or urban poverty, on the community- and neighbourhood level, and link these with grassroots activities literature in socio-technical innovation (Seyfang & Smith, 2007). The key actors are civil society and non-profit organisations that try to mobilise residents around various projects.

2.4 Stakeholder roles

ULLs are, indeed, associated with open innovation and user innovation (Hossain et al., 2019), which are extremes of the user involvement spectrum (Leminen, 2013). Open innovation functions on the idea that businesses cannot operate on their own, and instead look for external resources to improve their developments (Chesbrough, 2006). User innovation highlights the necessity of both passive and active roles by citizens in innovation processes (see for example, Bergvall-Kåreborn & Ståhlbröst, 2009). Both roles are needed to identify needs and ideas, as well as to validate and formalise learning outcomes (Menny et al., 2018).
As such, scientists have tried to come up with stakeholder roles. Often referred to and used for typologies are the Enabler, Provider, Utilizer, User and Researcher roles (Westerlund & Leminen, 2011; Leminen et al., 2012; Schuurman et al., 2016; Bondarenko et al 2019). Enablers are organisations that make things happen and that support ULL activities in resource terms. Providers are development organisations that provide something to ULLs like knowledge or expertise. Public or private organisations that use ULLs as a strategic business development tool are Utilizers. Users reflect the end-users of products or citizens involved in an urban context. Researchers are both providers of knowledge, as well as generators of new scientific knowledge in diverse fields, like urban policy.

2.5 Funding options and outcomes created

Recently, The Funding Mix Framework (FMF, Figure 1) is set up by Gualandi and Romme (2019), who provide a first holistic view of the relationship between stakeholders, value creation, and funding options. It consists of four funding methods: Pay per service (PPS), Subsidies (SUB), Out of Network Funds (ONF), and Cross Financing (CRF). PPS revenue arises from services in ULLs, mostly paid by private partners that seek economic value. SUB are often given by public partners to serve the strategic level of ULLs. ONF are equal to SUB, however, provided by partners not involved in the ULL constellation, like EU funding. CRF involves new ways of funding, such as renting out the physical space of ULLs.
Additionally, the authors argue that value created can be economic, business, and public. The first is about tangible and measurable outcomes, like new start-ups generated (Baccarne et al., 2014). The second is an extension of economic value, such as training provided. The third is about non-financial impacts of ULLs that, following Baccarne and co-authors (2014), relate to realizing policy goals. In these terms, “public value” is considered the most important in ULLs (Guzman et al., 2013), as they have a strong focus on social value creation and civic engagement (Baccarne et al., 2014). The social acceptance of innovation and consumer practises therefore seems to be a crucial accelerator of sustainability transitions (Schaffer & Turkama, 2012; Markard et al., 2020; Stoeglehner, 2020), while it is as difficult to measure as urban safety, environmental awareness (Ståhlbröst, 2012), or the early adoption of new practises (von Wirth et al., 2018).
Moreover, long-term funding is needed for long-term operation, knowledge accumulation, scalability, and impact creation (Guzman et al., 2013; Veeckman et al., 2013; Evans et al., 2015). Indeed, ULL outcomes and studies have focussed on incremental rather than radical outcomes (Hossain et al., 2019), since successful ULLs are inherently local, (Burch et al., 2018), and from there viewed as the starting point for scalability and transformation at different scales (Astbury & Bulkeley, 2018). Despite this, Mai (2018) showed that it is small scale ULLs that struggle hardest to achieve appropriate funding. Thus, business models have remained underdeveloped and unsustainable because they depend on public funding that requires strict justification, via project-based injections, or funding from universities and regional development agencies (Schaffer & Turkama, 2012).

Figure 1. The Funding Mix Framework. Source: Gualandi & Romme (2019).

2.6 Impact creation and transformative changes

Von Wirth and colleagues (2018) showed three ideal-typical ways of creating transformative changes in ULLs: Embedding, Translating and Scaling. The first is about the adoption and integration of an approach or outcome in existing local structures. The second is about elements of experiments or lessons learned being replicated and reproduced elsewhere. The third is about experiments becoming “bigger in terms of content and remit” (Ibid). Herein, transformative changes are viewed as the de-institutionalisation of existing socio-technical structures, along with new more sustainable ones being created, diffused, mainstreamed, and institutionalised again (von Wirth et al., 2018).

2.7 Current debate on solving the particular challenge

The current debate on solving the issue between funding, stakeholder roles, and outcomes, provides two positions. First, researchers are calling for a shared ideology within ULLs to help operations in such a way that complementarity stands above competition between stakeholders (see for example, Mangan & colleagues, 2009; or Gualandi & Romme, 2019). Second, debate continues about the various types of agency and power of stakeholders involved. For example, Burch and co-authors (2018) explained that ULLs redistribute agency and power to non-traditional urban stakeholders, while Menny and colleagues (2018) introduced a cyclical process of redistributed power throughout different ULL phases. Also, Savini and Bertolini (2019) demonstrated that ULLs relate to the political dynamics of institutional stability and change. Thus, it seems that opportunity lies in a change of mindset towards shared ideologies, which requires reviewing the concepts and meaning of agency and power.

3.0 Methodological Notes

3.1 Research design

This study uses a qualitative research design and combines a retrospective literature review with a comparative case study analysis. Indeed, case studies are arguably the most used methodology in ULL research (Greve et al., 2020). Yet, the method of case study here seems useful to gain a holistic overview of the context under investigation (Punch, 2013), and is particularly appropriate for collecting data to study a novel phenomenon in an explorative manner (Yin, 2002). This reflects the situation in ULL literature, since it is recent, quickly expanding, and contains competing definitions (Greve et al., 2020).
3.2 Research approach: literature review and comparative case study

First, a retrospective literature review was made to understand the historical development of ULLs, starting with two key documents. First, the Urban Living Labs: Experimenting with city futures scientific reader from Marvin and colleagues (2018), and second, an academic paper “A Systematic Review of Living Lab Literature” by Hossain and co-authors (2019). Both documents pay particular attention to LLs in urban contexts and as means for sustainable development. Subsequently, the “snowball” method and “pearl-growing” method were both applied to find additional relevant literature regarding the topic under study, by focussing on recognized authors and often mentioned key terms. Herein, no resource type scope was applied, such as restricting to academic articles only. As such, the authors ended up with a wide range of resource types. In this way, the snowball method allowed for a relatively fast and holistic exploration of the recent history of ULL literature, including the identification of often mentioned terms. The key terms (Urban Living Labs, Impact, Diffusion, Outcomes, Value, Stakeholder Roles, Funding Model, Financing) were then put into Google Scholar in the pearl-growing method for additional literature.

Second, a comparative case study (Punch, 2013) was conducted using semi-structured interviews to collect detailed data of cases by understanding and accessing stakeholder perspectives of the situation, and also to mutually explore the research question. Three cases were selected first that met four specific ULL criteria, and second which were labelled as an “organic”, “civic”, or “strategic” ULL. The criteria were established for assessment while reading in online policy documents, on their website, and about the mission, vision, and goals of the ULL. As far as general ULL criteria were concerned, the cases: 1) were geographically embedded in a particular location, 2) had to consist of urban stakeholders in the co-creation triple- or quadruple helix model, 3) focused on urban sustainability, and 4) used experiments or test moments to generate knowledge or learn about urban sustainability.

Additionally, the specific criteria to label the cases as an organic, civic, or strategic ULL are as follows. The strategic ULL, 1) operates on a city, regional, or national scale, 2) has involvement or a link to innovation agencies or agreements, 3) is financed by a lump sum in the relatively short-term (although not necessarily), 4) falls into a wider sustainability strategy, and 5) is competitive in nature. The civic ULL, 1) operates within a city scale, 2) is focussed on local urban priorities, and 3) consists of a clear partnership between urban stakeholders that initiated the ULL. The organic ULL, 1) is active on a community- or neighbourhood level, 2) is not initiated by governmental parties, 3) focusses on specific local contextual issues that link with social needs or ideological values of the initiators, and 4) is a strategic niche (Seyfang & Smith, 2007).

3.3 City context and case study descriptions

The case studies for our research were located in the city of Groningen. It is the largest and youngest city in the Netherlands’ north. The surrounding rural areas are entitled krimpregio’s by the National government, meaning they face a declining population and related urban challenges. Moreover, Groningen is a typical student city with one in four being a student. The economy of the city has mainly been focussed in recent years on services and energy, such as the natural gas company GasUnie. Focus is currently shifting to tourism, ICT, energy, and the environment, like the Hydrogen Valley HEAVENN EU-project. The cases are described below.

Organic ULL: Stichting Paddepoel Energiek (SPE)

SPE is a citizen initiative to improve energy neutrality in the neighbourhood Paddepoel, originally built in the 1960s to show opportunities in typical old Dutch neighbourhoods and buildings. It achieved some success, since Paddepoel became a partner in the EU-project “Making City”, although it is often associated as a not-to-be-in neighbourhood. Their main aim was to turn Paddepoel into an energy neutral neighbourhood in 2035, both technically and socially. Also, everyone living in Paddepoel could join the initiative and learn from their energy coaches about energy production and consumption. Additionally, the neighbourhood collaborated with the municipality of Groningen, educational institutions, and local businesses that develop, for example, solar panels. The period we studied was roughly during its “stichting” [1] period, from February 2016 until October 2019.

Civic ULL: Urban Gro Lab (UGL)

The UGL is a partnership of the municipal department of urban development and the Faculty of Spatial Sciences at the University of Groningen. Together they envisioned the city as itself a ULL in which research could be conducted that focuses on local socio-spatial challenges by bridging science and practise. Thus, it tried to function as a source of knowledge and inspiration by collaborating with urban stakeholders and involving citizens. As such, the UGL was run by a yearly changing lab coordinator. The UGL existed for almost five years and was purposefully intended to be and function as an ULL for spatial research and innovation. The period under study was from November 2015 until it stopped in November 2018.
Strategic ULL: Welcoming International Talent (WIT)

A “Gentlemen’s Agreement” called “Het Akkoord van Groningen” between the province of Groningen, the municipality of Groningen, the knowledge institutes in the city, and the University Medical Centre Groningen has existed since 2005. It is a cross-party collaborative platform for joint coordination and decision-making that agreed upon envisioning a sustainable future for the city of Groningen as a knowledge city by focussing on various themes. The focus in this study was laid on “internationalization”. The ULL aimed to make Groningen “stickier” by attracting, retaining, and integrating international residents and students better in the city, while maintaining a high level of social cohesion and liveability to enhance the innovate capacity. The WIT finds it basis within wider sustainability strategies at the EU, regional, and city levels. For example, the EU Cohesion Policy and the Next City Policy document. Also, Groningen is a “European Good Practise” city in the URBACT WIT Transfer Network. The period under study was from November 2016 (the review moment “Gentlemen’s Agreement” together with local policies) until November 2019. The ULL is still active and functioning.

3.4 Data collection and analysis

In total, six interviews and one mail questionnaire were conducted with eight interviewees in October and November 2019. The interviews lasted between 38 and 90 minutes and were taken at the work location of the interviewees, except for one in Groningen city centre. The mail questionnaire contained the same questions as the interviews and was applied based on the preference of respondents. In this study, the questionnaire is therefore viewed as a kind of “interview held by mail”, and as such included in the data analysis. The interviewees were governmental employees (2), governmental trainees who coordinated activities (2), civil initiators (2), a university employee (1), and a private sector person (1). The interviewees were chosen as they had leading positions in the Groningen ULL activities and projects. The semi-structured interviews were transcribed manually and analysed using the coding and memoing methodology (Punch, 2013). Analysis began by scoring out irrelevant information. Then, codes were attached to specific pieces of texts, resulting in 42 to 82 codes per interview. Next, all codes were clustered to find cross-connections between codes, clusters, and interviews. Alongside of this, the memoing technique (Punch, 2013) was used to put memos on different spots in the transcripts to move from the empirical to the conceptual level while analysing the data collected.

3.5 Ethical considerations

Prior to the interviews, interviewees received an interview guide and interview permission statement. The interview guide concerns an introduction to the research, its objectives, and questions asked. With the interview permission statement, respondents were asked to agree to recording the interview, and to the use of information and data collected. Transcriptions were provided to the respondents for approval, or any changes needed regarding their anonymity or answers given.

4.0 Comparative Case Study Results

4.1 Stakeholder roles in urban living labs

In all cases a core group was identified that was responsible for the funding and operation of the ULL (see blue contours in Figure 2). In the SPE and WIT, the core group existed out of the Enabler and Utilizer roles, whereas in the UGL the core group encompassed the Enabler, Provider and Researcher roles because of the partnership between the municipality and university. Municipalities were in all cases involved in the core group as Enablers, which was in line with observations from Scholl and Kemp (2016). In SPE and WIT, the Utilizers were involved because of the funding possibilities and expertise provision, which was already inherent in the UGL partnership. The Researcher role in SPE was absent since it did not intend to generate new knowledge per se, while the UGL and WIT did.

Figure 2. Stakeholder roles in the organic, civic, and strategic ULLs. The blue contour implies the core group of stakeholders most responsible for funding and operation activities. The positioning of each stakeholder is based on comparing the theoretical description with practical operation. The figures highlight the importance of the Enabler role in ULLs, as well as the potentially unconditional Researcher role in organic ULLs. Source: Authors.

Figure 3. Theoretical funding model in the organic, civic, and strategic ULLs. Most important observations are, 1) the eligibility of ONF in the strategic ULL, 2) the general focus on public outcomes in civic and strategic ULLs compared to organic ULLs, and 3) the sporadic and ambiguous use of CRF. Source: Authors based on Gualandi and Romme (2019).

4.2 Funding in urban living labs

The SPE is mostly financed by PPS and SUB methods via private businesses and the municipality. The UGL is also financed by PPS and SUB methods. The WIT is financed by all methods available and focuses on various outcomes. While it is acknowledged that the FMF is not a static model, notably PPS in the UGL, and PPS and SUB in WIT contrast to the FMF, since the PPS streams focussed on public outcomes, and the SUB obligated private outcomes. CRF was found to be complementary to other main funding streams aligning with the FMF. For example, payments by the housing association in SPE that informed and advised tenants as side-activity. Moreover, WIT seems most eligible to receive ONF due to legitimation reasons as the geographical location they serve is bigger compared to organic or civic ULLs.
Additionally, the interviewees mentioned university funding, political will, and the role of civil servants as important in funding provision. The Dutch educational system of universities has limited funding for “experiments” given that their core task is doing scientific research. Thus, political will is important for budgeting ULL projects in the coming years to ensure continuation. Meanwhile, the role of civil servants was criticized by the interviewees. In general, it was thought that they handle too strict justification criteria for subsidies provided, thereby limiting the freedom of the ULL to “experiment”, especially in the cases of SUB and ONF. These observations are in line with the SWOT analysis of LLs made by Guzman and colleagues (2013), and the accountability discussion raised by Astbury and Bulkeley (2018).

4.3 Outcomes in urban living labs

Increased social networks and mutual learning were found in all cases and indicated as important by interviewees, whether or not focussed on in advance. These are clearly felt outcomes, though not directly measurable. In fact, Brown and Vergragt (2008) argued that both are of immense importance if ULLs want to contribute to transformative changes. Indeed, most value strived for in ULLs is of public value (see black circles in Figure 3) to aim for societal and urban improvements. In addition, from our research we found that when economic value was pursued it was done by private parties or for specific short-term services. The interviewees argued that while some ULL experiments were seen as failures, they still brought outcomes that one might benefit from in the future, or that potentially could initiate wider transitions in provision systems, regardless of the type of ULL involved.

4.4 Impact creation in urban living labs

Impact creation seems to depend on the interests of outsiders (at translating) and the geographical scale the ULL is already active at (scaling), together with the available possibilities to embed lessons in local structures or organisations (at embedding). The SPE scaled and translated outcomes, without initiating them. Rather, interest came in from outsiders, and the further obligation to become a stichting made them do it. The UGL embedded outcomes in local structures within its geographical focus. However, their impact remained sporadic due to a lack of long-term vision, which ranged from products created, experiences gained, networks built, or education improved. The WIT embedded and translated lessons learned via either the integration of outcomes in local structures, like policy plans, or via the URBACT Network to other city contexts. Scaling was not observed as the WIT already focused on (inter-)regional scale and “everyone” in the Akkoord van Groningen.

5.0 Conclusion

5.1 Theoretical implications: urban living labs and transformative changes

In this study, the research question was presented: How does the trinity of funding options, stakeholder roles, and outcomes in ULLs influence their impact creation for transformative changes in cities? Since the 2008 Global Economic crisis, ULLs have emerged in the urban context to learn collectively about urban development as a long-term process. While ULL practitioners and scientists have been aware of the potentials inherent in ULLs for several years (Bergvall-Kåreborn & Ståhlbröst, 2009; Almirall & Wareham, 2011; Westerlund & Leminen, 2011), the continuous searches to limit individual political and financial risks among urban stakeholders have hindered the potential of ULLs. This study therefore emphasized that it is not individual aspects in the trinity highlighted that improve impact creation in ULLs, rather trust building among stakeholders in ULLs and their place-based contexts seems necessary to contribute to transformative changes in the long-run. That way, ULLs can strengthen their foreseen role as a form of experimentation in a broader shift involving urban governance, and as such can achieve gradual changes in socio-technical and socio-ecological systems based on a continuous learning process among actors and urban sites.

5.2 Theoretical implications: concepts and methods

The theoretical contribution of this study is mainly twofold. First, it follows up on Greve and colleagues (2020) who pointed to the opportunity for scholars to apply unused theoretical approaches in ULL literature. Hence, this study has shown the importance of, for example, the exploration in practise of the FMF and ULL typology of Marvin and colleagues (2020) to enhance the theoretical understanding of ULLs in certain domains. We thus call for researchers to explore existing theoretical approaches more extensively, instead of continually seeking to provide new approaches, categories, or models. A good example of this is the recent study by Kalinauskaite and co-authors (2021), who further developed Schuurman’s three-layer model (2015), which is still underutilized as a conceptual model for organising (U)LLs. Second, this study confirms and adds to the current theoretical debate about how to overcome the issue under study: a change in mindset towards shared ideologies, which requires reviewing the meanings of agency and power. In this effort, we recommend complementing shared ideologies with individual wishes and needs. Likewise, pairing the concepts of “power” and “justification”, as we observed that constant pressure to safeguard and legitimise expenditures creates power dynamics in ULLs between stakeholders involved.

5.3 Practical implications

The main practical contribution of this study to ULLs is the emphasis on trust building among stakeholders toward overcoming the issue under study. More specifically, five points emerge: First, public grant providers on various political levels are challenged to rethink their selection criteria for subsidy approvals to guarantee strategic long-term funding in ULLs that can be complemented with project-based private investments. Second and consequently, new ways to measure effectiveness of ULL activities are needed to indicate successes and failures, both quantitative and qualitative, and that allow for deviation on the individual and collective level. In fact, this study shows that increased (social) networks and accumulated learning must be integrated in evaluation criteria. Third, municipalities are challenged to empower ULL initiators politically by reviewing the concepts of “agency” and “power”, especially in organic and civic ULLs, as those are active topics that municipalities already treat with concern. Fourth, ULL stakeholders should view the level of abstractness in objectives as facilitators in collaboration by envisioning shared strategic goals, while providing room for individual outcomes to ensure continuous momentum for all stakeholders. While the importance of these anchor points (Leminen et al., 2017) is acknowledged and recognized on the operational level, they have yet to be incorporated on a strategic level. Fifth, the ULL community must become aware of the fragmentation of views about what ULL’s are and aren’t. Thus, thinking critically about when ULLs are needed will help improve their applicability in practise. Recently, Greve and colleagues (2020) emphasised this as well by exploring the overall landscape of LL research and its potential areas of fragmentation and isolation.

5.4 Limitations and recommendations for further research

We also recognize important limitations to the study. First, this study was limited to the city of Groningen, and the inclusion of three ULLs indicated as organic, civic, and strategic ULLs. Further research should also focus on comparison between different local contexts, as well as between similar types of ULLs. Second, we held only a limited number of interviews. More experiences from a multitude of people and disciplines should be collected to enrich our understanding of the topic under study. Third, we support with the call of Greve and colleagues (2020) to start using more diverse, as well as quantitative research methods in ULL literature, alongside the often used and in this study applied case study methodology. For example, discourse analysis or ethnographic research designs may help researchers to analyse and experience how (spoken) agency and power is orchestrated in ULL daily practises. Additionally, we call for more research into the power dynamics in ULLs, especially in terms of political power, such as found in studies like Savini and Bertolini (2019). Lastly, we call for more research into the self-sustaining character of ULLs. More specifically, 1) to investigate appropriate ways to embed ULLs in local context needs, by identifying problems and potentials to improve social adoption and ownership of ULLs that help guarantee continuity over time, especially regarding organic and civic ULLs, and, 2) to investigate how ULLs can be self-sustained financially in order to make them less dependent on subsidies, which have led to certain consequences observed and investigated in this study.

Notes:

[1] A stichting is a Dutch legal type of organisation that focusses primarily on societal or social goals, rather than monetary profit. A board must be formed in a stichting, while a supervisory board is not needed per se. A stichting is often funded by donations, loans, or subsidies and does not have members or shareholders.

Previously published

A previous version of this paper was published in the Conference Proceedings of the Digital Living Lab Days 2021 Conference of the European Network of Living Labs. It is available online here: https://issuu.com/enoll/docs/dlld_2021_-_proceedings.

---

References

Almirall, E., & Wareham, J. 2011. Living Labs: arbiters of mid- and ground-level innovation. Technology Analysis & Strategic Management, 23(1): 87–102. https://doi.org/10.1080/09537325.2011.537110

Astbury, J. and Bulkeley, H. 2018. 7. Bringing Urban Living Labs to Communities; Enabling processes of transformation. In Marvin, S., Bulkeley, H., Lindsay, M., McCormick, K. and Voytenko Palgan, Y. (Eds.), Urban Living Labs Experimenting With City Futures: 106-125. Routledge.

Baccarne, B., Schuurman, D., Merchant, P. and De Marez, L. 2014 The Role of Urban Living Labs in a Smart City. The XXV ISPIM Conference – Innovation for Sustainable Economy and Society, Dublin, Ireland, 8-11 June. http://hdl.handle.net/1854/LU-5646684

Bergvall-Kåreborn, B., & Ståhlbröst, A. 2009. Living Lab: an open and citizen-centric approach for innovation. International Journal of Innovation and Regional Development, 1(4): 356. https://doi.org/10.1504/ijird.2009.022727

Bondarenko, O., Schuurman, D., & de Kinderen, W. 2019. From Local to Global – Scaling up your Living Lab Activities to a Sustainable Living Lab Organization: Experiences from City Pulse. Case presentation at the annual Open Living Lab Days Conference 2019, Thessaloniki, Greece.

Brown, H. S., & Vergragt, P. J. 2008. Bounded socio-technical experiments as agents of systemic change: The case of a zero-energy residential building. Technological Forecasting and Social Change, 75(1): 107–130. https://doi.org/10.1016/j.techfore.2006.05.014

Bulkeley, H., Broto, C. V., Hodson, M., & Marvin, S. 2010. Cities and Low Carbon Transitions (Routledge Studies in Human Geography Book 35). Routledge.

Bulkeley, H., & Castán Broto, V. 2012. Government by experiment? Global cities and the governing of climate change. Transactions of the Institute of British Geographers, 38(3): 361–375. https://doi.org/10.1111/j.1475-5661.2012.00535.x

Bulkeley, H., Coenen, L., Frantzeskaki, N., Hartmann, C., Kronsell, A., Mai, L., Marvin, S., McCormick, K., van Steenbergen, F., & Voytenko Palgan, Y. 2016. Urban living labs: governing urban sustainability transitions. Current Opinion in Environmental Sustainability, 22: 13–17. https://doi.org/10.1016/j.cosust.2017.02.003

Burch, S., Graham, A. and Mitchell, C. 2018. 11. Agency, Space and Partnerships: Exploring key dimensions of urban living labs in Vancouver, Canada. In Marvin, S., Bulkeley, H., Lindsay, M., McCormick, K. and Voytenko Palgan, Y. (Eds.), Urban Living Labs Experimenting With City Futures: 189-209. Routledge.

Chesbrough, H. W. 2006. Open Innovation: The New Imperative for Creating and Profiting from Technology. Harvard Business Review Press.

Chronéer, D., Ståhlbröst, A., & Habibipour, A. 2019. Urban Living Labs: Towards an Integrated Understanding of their Key Components. Technology Innovation Management Review, 9(3): 50–62. https://doi.org/10.22215/timreview/1224

De Jong, M., Joss, S., Schraven, D., Zhan, C., & Weijnen, M. 2015. Sustainable–smart–resilient–low carbon–eco–knowledge cities; making sense of a multitude of concepts promoting sustainable urbanization. Journal of Cleaner Production, 109: 25–38.

https://doi.org/10.1016/j.jclepro.2015.02.004

Evans, J., Jones, R., Karvonen, A., Millard, L., & Wendler, J. 2015. Living labs and co-production: university campuses as platforms for sustainability science. Current Opinion in Environmental Sustainability, 16: 1–6. https://doi.org/10.1016/j.cosust.2015.06.005

Evans, J., & Karvonen, A. 2013. ‘Give Me a Laboratory and I Will Lower Your Carbon Footprint!’ - Urban Laboratories and the Governance of Low-Carbon Futures. International Journal of Urban and Regional Research, 38(2): 413–430. https://doi.org/10.1111/1468-2427.12077

Evans, J., Karvonen, A., & Raven, R. (2017). The Experimental City (Routledge Research in Sustainable Urbanism). Routledge.

Frantzeskaki, N., & Loorbach, D. 2010. Towards governing infrasystem transitions. Technological Forecasting and Social Change, 77(8): 1292–1301. https://doi.org/10.1016/j.techfore.2010.05.004

Greve, K., Leminen, S., de Vita, R., & Westerlund, M. 2020. Unveiling the diversity of scholarly debate on living labs: A bibliometric approach. International Journal of Innovation Management, 24(08): 2040003. https://doi.org/10.1142/s1363919620400034

Grin, J., Rotmans, J., & Schot, J. 2011. Transitions to Sustainable Development: New Directions in the Study of Long Term Transformative Change (Routledge Studies in Sustainability Transitions). Routledge.

Gualandi, E., & L. Romme, A. G. 2019. How to Make Living Labs More Financially Sustainable? Case Studies in Italy and the Netherlands. Engineering Management Research, 8(1): 11. https://doi.org/10.5539/emr.v8n1p11

Guzmán, J. G., del Carpio, A. F., Colomo-Palacios, R., & de Diego, M. V. 2013. Living Labs for User-Driven Innovation: A Process Reference Model. Research-Technology Management, 56(3): 29–39. https://doi.org/10.5437/08956308x5603087

Hodson, M., Evans, J. and Schliwa, G. 2018. 3. Putting Urban Experiments into Context; Integrating urban living labs and city-regional priorities. In Marvin, S., Bulkeley, H., Lindsay, M., McCormick, K. and Voytenko Palgan, Y. (Eds.), Urban Living Labs Experimenting With City Futures: 37-51. Routledge.

Hossain, M., Leminen, S., & Westerlund, M. 2019. A systematic review of living lab literature. Journal of Cleaner Production, 213: 976–988. https://doi.org/10.1016/j.jclepro.2018.12.257

Kalinauskaite, I., Brankaert, R., Lu, Y., Bekker, T., Brombacher, A., & Vos, S. 2021. Facing Societal Challenges in Living Labs: Towards a Conceptual Framework to Facilitate Transdisciplinary Collaborations. Sustainability, 13(2): 614. https://doi.org/10.3390/su13020614

Kemp, R., Schot, J., & Hoogma, R. 1998. Regime shifts to sustainability through processes of niche formation: The approach of strategic niche management. Technology Analysis & Strategic Management, 10(2): 175–198. https://doi.org/10.1080/09537329808524310

Leminen, S. 2013. Coordination and Participation in Living Lab Networks. Technology Innovation Management Review, 3(11): 5–14. https://doi.org/10.22215/timreview740

Leminen, S., Rajahonka, M., & Westerlund, M. 2017. Towards Third-Generation Living Lab Networks in Cities. Technology Innovation Management Review, 7(11): 21–35. https://doi.org/10.22215/timreview/1118

Leminen, S., Westerlund, M., & Nyström, A. G. 2012. Living Labs as Open-Innovation Networks. Technology Innovation Management Review, 2(9): 6–11. https://doi.org/10.22215/timreview/602

Liedtke, C., Jolanta Welfens, M., Rohn, H., & Nordmann, J. 2012. LIVING LAB: user‐driven innovation for sustainability. International Journal of Sustainability in Higher Education, 13(2): 106–118. https://doi.org/10.1108/14676371211211809

Loorbach, D., & Rotmans, J. 2010. The practice of transition management: Examples and lessons from four distinct cases. Futures, 42(3): 237–246. https://doi.org/10.1016/j.futures.2009.11.009

Luque-Ayala, A., & Marvin, S. 2015. Developing a critical understanding of smart urbanism? Urban Studies, 52(12): 2105–2116. https://doi.org/10.1177/0042098015577319

Mai, L. 2018. 12. Placing Sustainability in Communities; Emerging urban living labs in China. In Marvin, S., Bulkeley, H., Lindsay, M., McCormick, K. and Voytenko Palgan, Y. (Eds.), Urban Living Labs Experimenting With City Futures: 210-230. Routledge.

Mangan, A., Rukanova, B., Henningson, S., Kipp, A., Klein, S. and Tan, Y.-H. 2009. Technological Innovation in Learning Settings: Power and Politics in Living Labs. Paper presented at the International Conference on Organizational Learning, Knowledge and Capabilities (OLKC), Amsterdam, Netherlands, 26-28 April. http://www.feweb.vu.nl/olkc2009/Papers/1CAMangan.pdf

Markard, J., Geels, F. W., & Raven, R. 2020. Challenges in the acceleration of sustainability transitions. Environmental Research Letters, 15(8): 081001.

https://doi.org/10.1088/1748-9326/ab9468

Marvin, S., Bulkeley, H., Mai, L., McCormick, K., & Voytenko, Y. (Ed.). 2018. Urban Living Labs: Experimenting with City Futures. Routledge.

Menny, M., Palgan, Y. V., & McCormick, K. 2018. Urban Living Labs and the Role of Users in Co-Creation. GAIA - Ecological Perspectives for Science and Society, 27(1): 68–77. https://doi.org/10.14512/gaia.27.s1.14

Neef, M. R., Verweij, S., Gugerell, K. and Moen, P.L.R. 2017. ‘Wegwijs in Living Labs in Infrastructuur en Ruimtelijke Planning: Een Theoretische en Empirische Verkenning’, Groningen: Rijksuniversiteit Groningen.

Nevens, F., Frantzeskaki, N., Gorissen, L., & Loorbach, D. 2013. Urban Transition Labs: co-creating transformative action for sustainable cities. Journal of Cleaner Production, 50: 111–122. https://doi.org/10.1016/j.jclepro.2012.12.001

Paroutis, S., Bennett, M., & Heracleous, L. 2014. A strategic view on smart city technology: The case of IBM Smarter Cities during a recession. Technological Forecasting and Social Change, 89: 262–272. https://doi.org/10.1016/j.techfore.2013.08.041

Puerari, E., de Koning, J., von Wirth, T., Karré, P., Mulder, I., & Loorbach, D. 2018. Co-Creation Dynamics in Urban Living Labs. Sustainability, 10(6): 1893.

https://doi.org/10.3390/su10061893

Punch, K. F. 2013. Introduction to Social Research: Quantitative and Qualitative Approaches, 3rd Edition. SAGE Publications Ltd.

Savini, F., & Bertolini, L. 2019. Urban experimentation as a politics of niches. Environment and Planning A: Economy and Space, 51(4): 831–848.

https://doi.org/10.1177/0308518x19826085

Schaffers, H., & Turkama, P. 2012. Living Labs for Cross-Border Systemic Innovation. Technology Innovation Management Review, 2(9): 25–30.

https://doi.org/10.22215/timreview/605

Scholl, C., & De Kraker, J. 2021. Urban Planning by Experiment: Practices, Outcomes, and Impacts. Urban Planning, 6(1): 156–160.

https://doi.org/10.17645/up.v6i1.4248

Scholl, C., & Kemp, R. 2016. City Labs as Vehicles for Innovation in Urban Planning Processes. Urban Planning, 1(4): 89–102.

https://doi.org/10.17645/up.v1i4.749

Schot, J., & Geels, F. W. 2008. Strategic niche management and sustainable innovation journeys: theory, findings, research agenda, and policy. Technology Analysis & Strategic Management, 20(5): 537–554. https://doi.org/10.1080/09537320802292651

Schuurman, D. 2015. Bridging the gap between Open and User Innovation?: exploring the value of Living Labs as a means to structure user contribution and manage distributed innovation. Ghent University. Faculty of Political and Social Sciences; Vrije Universiteit Brussel. Faculty of Economic and Social Sciences, Ghent; Brussels, Belgium.

Schuurman, D., Baccarne, B., Marez, L. D., Veeckman, C., & Ballon, P. 2016. Living Labs as open innovation systems for knowledge exchange: solutions for sustainable innovation development. International Journal of Business Innovation and Research, 10(2/3): 322.

https://doi.org/10.1504/ijbir.2016.074832

Seyfang, G., & Smith, A. 2007. Grassroots innovations for sustainable development: Towards a new research and policy agenda. Environmental Politics, 16(4): 584–603. https://doi.org/10.1080/09644010701419121

Sharp, D., & Raven, R. 2021. Urban Planning by Experiment at Precinct Scale: Embracing Complexity, Ambiguity, and Multiplicity. Urban Planning, 6(1): 195–207.

https://doi.org/10.17645/up.v6i1.3525

Smith, A., & Raven, R. 2012. What is protective space? Reconsidering niches in transitions to sustainability. Research Policy, 41(6): 1025–1036.  https://doi.org/10.1016/j.respol.2011.12.012

Ståhlbröst, A. 2012. A set of key principles to assess the impact of Living Labs. International Journal of Product Development, 17(1/2): 60. https://doi.org/10.1504/ijpd.2012.051154

Steen, K., & van Bueren, E. 2017. The Defining Characteristics of Urban Living Labs. Technology Innovation Management Review, 7(7): 21–33. https://doi.org/10.22215/timreview/1088

Stoeglehner, G. 2020. Integrated spatial and energy planning: a means to reach sustainable development goals. Evolutionary and Institutional Economics Review, 17(2): 473–486.

https://doi.org/10.1007/s40844-020-00160-7

van Steenbergen, F. and Frantzeskaki, N. 2018. 13. The Importance of Place for Urban Transition Experiments; Understanding the embeddedness of urban living labs. In Marvin, S., Bulkeley, H., Lindsay, M., McCormick, K. and Voytenko Palgan, Y. (Eds.), Urban Living Labs Experimenting With City Futures: 231-247. Routledge.

Veeckman, C., Schuurman, D., Leminen, S., & Westerlund, M. 2013. Linking Living Lab Characteristics and Their Outcomes: Towards a Conceptual Framework. Technology Innovation Management Review, 3(12): 6–15. https://doi.org/10.22215/timreview748

von Wirth, T., Fuenfschilling, L., Frantzeskaki, N., & Coenen, L. 2018. Impacts of urban living labs on sustainability transitions: mechanisms and strategies for systemic change through experimentation. European Planning Studies, 27(2): 229–257. https://doi.org/10.1080/09654313.2018.1504895

Voytenko, Y., McCormick, K., Evans, J., & Schliwa, G. 2016. Urban living labs for sustainability and low carbon cities in Europe: towards a research agenda. Journal of Cleaner Production, 123: 45–54. https://doi.org/10.1016/j.jclepro.2015.08.053

Westerlund, M., & Leminen, S. 2011. Managing the Challenges of Becoming an Open Innovation Company: Experiences from Living Labs. Technology Innovation Management Review, 1(1): 19–25. https://doi.org/10.22215/timreview489

Yin, R. K. 2002. Case Study Research: Design and Methods, 3rd Edition (Applied Social Research Methods, Vol. 5). SAGE Publications, Inc.