Introduction

An emerging and innovative way to organize projects in health technology and innovation has arisen, called a “living lab”. Living labs are small public-private partnerships where multiple stakeholders, including end-users, collaborate around shared challenges in a real-life setting (Geenhuizen, 2014; Westerlund et al., 2018a; Westerlund et al., 2018b; Hossain et al., 2019). Some authors conceptualize and define living labs according to their methods, processes, business model, or outcomes (Veeckman et al., 2013; Hossain et al., 2019). Even though no consensus on a single definition of “living labs” has yet been reached, this useful definition is provided by Westerlund and Leminen (2011):

“They [living labs] are physical regions or virtual realities where stakeholders form public-private-people partnerships (4Ps) of firms, public agencies, universities, institutes, and users all collaborating for creation, prototyping, validating, and testing of new technologies, services, products and systems in real-life contexts.”

In other words, the formerly prominent “university laboratory” has been moved, or is in the process of being moved, either physically or virtually. Two major benefits of this have been described in the literature that apply to the use case at stake in this research, that is, the wards of a nursing home. Firstly, contextual and situational factors both come to be understood as part of the innovation process, making it more responsive to the changing demands of stakeholders in their context. Secondly, because of the real-life setting, close collaboration with, and meaningful participation¹ by relevant stakeholders may be facilitated.

As such, living labs may become a solution to a very old problem: how to facilitate the meaningful participation of stakeholders in science and technology? Dell’Era and Landoni (2014) suggest that living labs operate between the methodologies of “user-centred design” and “participatory design”. Other authors write about “co-design” and “co-creation” (Almirall et al., 2012; Brankaert & den Ouden, 2017; Schuurman & Protic, 2018). All these methods share as a common feature the active participation of end-users in the innovation process. This meaningful participation of stakeholders holds a claim as one of the pillars of successful living labs (Hyysalo & Hakkarainen, 2014).

The motivations for stakeholders to participate in living labs are diverse and depend on the type of stakeholder. Companies and organizations usually participate to design, test, refine, or implement their products and services (Leminen & Westerlund, 2012; Schuurman et al., 2016). End users, according to Leminen et al., (2014), may assume various roles in the process; for example, a ‘tester’ contributes to new technologies by testing prototypes, whereas a co-creator is an active participant in the design or implementation of a technology. However, the motivations of end-users to participate, and the way in which they were actually involved, often remain implicit in literature (Leminen et al., 2012; Leminen, 2013; Schuurman et al., 2015).

Nevertheless, some authors provide more detailed descriptions of end-user or stakeholder participation in living labs. Hyysalo and Hakkarainen (2014) stress that user participation within living labs does not happen effortlessly or automatically. Swinkels et al. (2018) discuss how, over time, patient perspectives became of secondary importance, eventually favoring the perspectives of professionals. In such cases, the ‘participation potential’ of living labs is accepted as a given, intrinsic characteristic of the methodology. The process of stakeholder participation, as a guide or inspiration for living labs, nevertheless remains sparsely published in the living lab literature.

In this article, I aim to contribute to the living lab literature by providing an account of my experiences as a participation researcher and stakeholder in a living lab in the Netherlands. The living lab Care Robotics (LLCR) falls under the umbrella of the Medical Delta: a consortium of public and private organizations and knowledge institutes in the Delta region of the Netherlands (Medical Delta, 2019). The LLCR aims to improve the wellbeing of patients and quality of care through meaningful technological innovation mainly within elderly care. It works to achieve this through close collaboration between universities, care organizations, SME’s, and the end-users in projects where technologies are designed, tested, and implemented in a real-life setting. The strong focus on meaningful knowledge circulation necessarily involves researchers, lecturers, and students from associated knowledge institutes. In this sense, the active participation of all stakeholders, including lay persons, is itself central to the goals and methodology of the LLCR.

My Role as a Researcher

I² am a postdoctoral researcher at a university of applied sciences. I have a background in participatory research and patient participation. With that expertise I am involved with the living lab Care Robotics to facilitate the participation of ‘end-users’ in a number of projects within the living lab. As a participation researcher, the promise of living labs appeals to me as well. For the first time, the context of health care delivery has been acknowledged as essential for meaningful innovation and technology. Within the patient participation tradition, it is commonly accepted that the context wherein people live, get sick, receive treatment and care is essential for understanding their perspectives. This promise of literally moving innovation towards the context where it will be applied, sparked my interest. Are living labs finally the “magic bullet” that will facilitate meaningful and significant participation of patients in healthcare innovation?

I chose to participate in this project specifically in order to facilitate participatory processes in the living lab. During the year, I attended all of the project’s meetings, advised on participatory methods, conducted interviews and observations, and presented intermediate findings to the project team. Additionally, because of the LLCR desire for strong knowledge circulation, students from several programs were involved at all stages. For most of these student projects, I was the main supervisor.

My observations presented in this article are the result of ongoing qualitative analysis of all documents created during this year, including meeting transcripts, observation notes, and student reports. I wrote this article as a case study where the LLCR and the project were the objects of my qualitative exploration. This way of working bears close resemblance to the basic tenets of Participatory Action Research.

In this article, however, I would like to share my perspectives and reflections on the project and extrapolate them into lessons learned for the future. I will first provide a summary of the chronological course of events after which I will reflect on three key moments that illustrate why participation may be the intention, but is more complicated to achieve in the process of a project.

Project Background

Before the project began, the board of the care organization established four pillars on which future innovation projects should rest. One of those pillars was “care for freedom”.

It is commonly acknowledged that the experience of freedom increases the wellbeing of people who receive home care or live in nursing homes (Scherder et al., 2010). Specifically, in psychogeriatric care, restrictions in freedom for those who develop the urge to wander may be detrimental. These residents may experience lower quality of life, become agitated or even aggressive, and negatively influence the happiness of those living and working in the ward. This problem is often described as “unrest”, the state where multiple residents become agitated.

Ensuring freedom in a closed psychogeriatric ward is complex. Without proper supervision, residents may wander off, get lost, or fall and injure themselves. This project aimed to increase the freedom for residents given the current characteristics of the facility. These specific problems of this facility were known, and experienced, by all layers in the organization. However, no actual research was done involving the experiences and perspectives of all relevant stakeholders. To further explore this problem, collaboration with the LLCR was initiated. The project was kickstarted by a funding voucher from the Medical Delta. For this voucher, a project proposal was written where the core problem was identified as how to balance the tension between ensuring freedom for residents whilst safeguarding their safety.

Three guiding principles were defined for the course of the project:

• All relevant stakeholders should participate in the project in a meaningful way
• The focus should be on exploring the problem and its context
• The focus should not be on possible technological solutions.

Chronological Course of Events

A summary of the chronological course of events can be found in Table 1. Since this project aimed for meaningful participation of all relevant stakeholders, a project team with a broad diversity of stakeholders was established. Included as part of the team were two ward managers, the location manager, the innovation manager, the geriatric psychologist, and three researchers (of whom I was one) from the university of applied sciences. The project team met regularly, on average every other week, to maintain close supervision on the projects’ progress.

The first step by the project team was to create a student project for ten weeks (group 1). The students explored the perspectives of all relevant stakeholders on the issue of freedom given the facility’s characteristics. A group of students started with a ten-week project for this step. Under the supervision of researchers (SS and RdeV), the students conducted semi-structured interviews with various stakeholders, and did participatory observations of both day and night shifts at the closed psychogeriatric ward. Their results were eventually presented to the project team, and the project team used them to move on to the next stage.

Since no new key insights surfaced from this stage of exploration, the project group brainstormed the best way forward. The brainstorm sessions resulted in two shared insights. First, only a few residents develop the urge to wander. These residents can then get agitated and cause unrest among others in the wards. Thus, if these residents can be moved to a more attractive environment, both their wellbeing and the wellbeing of the residents and employees on the wards can potentially be restored. Following this idea, part of the main building of the facility was selected to be redesigned to be attractive for residents with dementia who develop the urge to wander.

Second, we developed a shared narrative about the “courage to experiment” without being paralyzed by the fear of risk and failure. This narrative became a new guiding principle for the project’s future. These two ideas were discussed in eight semi-structured interviews with the project team, as well as with employees from both the psychogeriatric wards and the facility’s central building.

The project for student group 2 consisted of two steps. First, the students observed residents and talked with them to explore what they like doing or liked doing when they were younger. Second, using this information, they then redesigned part of the central building, and participated by wandering around with residents with dementia to see whether they liked the new situation. Drawing from my background in participatory research, I coached the students on working together with employees and residents and advised them on how to share decision making power. Eventually, with the help of the student findings, we were then able to do “small experiments” at the facility to see how our ideas worked out in practice.

End Result: A television screen

One of the end results of the project can be found in Figure 1. A television screen was bought for the facility to experiment with interactive technology in an attempt to make the new ward more attractive to residents with dementia. Using imagery from their past, the idea was that this screen could serve as a “pleasant distraction” for residents to decrease their urge to wander and lower their levels of agitation. As can be judged from the image, however, a sharp contrast exists between the assumed possibilities of the screen, and the crude reality of day-to-day health care delivery. While a good idea from the project perhaps in theory, the television screen did not turn out to be useful in practice.

Using three key moments below, I aim to provide an explanation for this. The first part of the answer is that the idea of interactive technology as a possibility was explicitly discussed at the start of the project. It was just decided that it should not be the starting point. In other words, this idea was surreptitiously present during the entire project, it just resurfaced later in the project. A second important observation is that the project focus shifted unnoticedly from “freedom for people with dementia” to the “residents with an urge to wander”. This was a key shift in thinking that impacted the project’s outcomes.

Figure 1. A “interactive” television screen, as end result of the project. The Dutch notice states: “No channels installed. Select ‘install’ and press ‘OK’ to install TV-channels.”

Table 1. The project’s chronological course of events and their main outcomes

The first student project assignment was written in tandem with the innovation manager and two ward managers from the facility. This assignment concealed a pre-existing perspective on the issue: the building itself prevents employees from ensuring freedom for their residents. For example, the assignment for the first student project mentioned:Setting the Stage

A mismatch emerged between the ideas of the developers of the facility, and the possibilities of employees to deliver care. Residents live in small, closed wards and cannot move freely. Employees often work in isolation because one ward will usually have one health worker. Therefore, it is more difficult for employees to ensure the freedom and safety of their residents.

This excerpt from the project’s texts illustrates that a shared perception was held regarding the problem at that moment. As the students started their interviews with stakeholders, the project group expected to get a better sense of the problems the work floor experienced. However, no new insights emerged immediately, and instead all of our preconceptions were reproduced by our participants. In hindsight, this can be easily explained. Our stakeholders were consulted on subjects that we predefined in the project assignment: the contrast between “freedom” and the building’s “restrictions” .

However, careful analysis of the interviews with health professionals showed a different perspective on the situation. They felt restricted in multiple aspects of delivering care. Most of these exchanges revolved around issues that were either desirable or undesirable for their residents. They talked about continuity of care, visiting a local restaurant, taking a walk outside, and the importance of seeing familiar faces. This showed a different dichotomy: restrictions caused by the building, rules and regulations versus the delivery of ‘good care’.

For example, consider the following quotation:

“[W]ell, our staff has a lot of tasks at the moment, administration is increasing and things are getting more and more busy, but that takes time away from the residents, you know, attention for the residents. I would like to have more attention for my residents and spend less time on administration (practical nurse).”

The actual sentiments of employees on the work floor clearly did not revolve around the single issue of ensuring freedom alone. They instead explained that they were happy with their work when their residents are happy. They reflected on how the residents could achieve this wellbeing, and how oftentimes this was hampered. In other words, their struggle revolved around their perspective on “good care” and the extent to which they felt they were able to deliver it. In sum, the project group never diverted from their initial perspective on the issue. As a result, my view is, to put it bluntly: residents and healthcare staff were only consulted³, but not actually heard. To be clear, their perspectives were indeed expressed and made explicit, but interpreted through our framework, and therefore had no influence on the direction of the project.

Removing Key Players

Before student group 2 started, the ward managers served as an important part of the project group. They functioned as a linchpin with healthcare staff on the wards. Their role was twofold. First, they set up meetings for interviews with staff members and planned observations at the wards. They also informed employees about the project’s process. Second, they were able to bring to the table perspectives, ideas, and reservations from the employees. Moreover, being experienced health professionals, they were able to steer the project group towards ideas that would likely have higher chances of success in a nursing home setting. In practice, they turned out to represent the healthcare staff in the project.

Up to then, the project had resulted in new insights and the construction of a narrative shared by all members of the project group. However, as the project’s end-date drew closer, the pressure to deliver concrete results increased. The group reached consensus that these results could be achieved by experimenting with our ideas in the real-life setting of day-to-day practice. In other words, the courage to experiment changed due to pressure to experiment.

To facilitate experimentation, the project needed closer supervision, which demanded greater flexibility from the project group. Together with the ward managers, the project group decided to move on with a smaller project group, in which the ward managers would no longer be present. A pragmatic argument was laid out: it proved difficult to schedule meetings with all project group members due to different schedules in the nursing home and at the university. Moreover, this decision made sense since close collaboration with healthcare staff would already be at the center of the activities. The ‘experiments’ would thus be carried out by the students from group 2, making them the new linchpin with the work floor.

During an evaluation meeting with the group 2 students, they discussed how the experiments were more difficult than expected for them to perform in reality:

“I remember stepping into the ward and told them that I was here to bring the residents to the other location for our observations. And she was like: “okay, you can take these residents outside”; and she pointed at people in wheelchairs. But that was not the point ... I don’t know whether they expected us to go outside for a walk with the residents for fun. Then you have to set your boundaries, but maybe they weren’t up to date with our project (bachelor of nursing student).”

Multiple issues soon became apparent. To the students it was unclear with whom to make appointments or to plan observations and interviews. Moreover, people on the work floor were not aware of their presence, nor the reason for them being there. Looking back, the ward managers were the linchpin with the work floor, who should not have been cut out of the process. In previous projects they both communicated with the employees and were also their voice during the project’s group meetings. Removing them from the project group distanced the project group from the work floor.

Reclaiming the Stage

I should return here to remind that the idea of adding an interactive screen to the ward was discussed from the start of the project. During initial conversations, potential technological solutions to the problem of freedom in a restricted place were discussed abundantly. We decided to abandon the focus on technology to instead facilitate an open exploration of the problem, and also give space to our stakeholders to come up with possible solutions. However, as the pressure to deliver results increased, the idea of interactive technology as a potential solution resurfaced.

As described above, the focus of the project moved towards making the target location more attractive to residents with the urge to wander. Combined with pressure to produce results, the need to make rapid decisions increased as the schedule progressed as well. This resulted in a top-down management process where the project group made all the decisions.

From the interview excerpt shown below, we can distill that the students would have preferred a different course of action than what was eventually taken:

“Student 1: I have another bottleneck … [The organization] had a steering role in the project, how shall I put it?

Student 2: well, that we weren’t completely free, that we were controlled a little.

Student 1: we had the executive role, and they had the decision-making role. Looking back, we should maybe have voiced our opinion more strongly.

Interviewer: can you give an example?

Student 1: well, probably about the television. A lot of ideas were discussed [with residents and employees], also for the patio. We should have said that we would have liked to focus on something else, but it was already decided that the TV would come first.

Student 2: We saw more potential in the patio. But it was also quite unclear what we were going to do for [the organization] and what they expected from us. Eventually we had the meeting where they said: “ok, go and observe some people, four days a week, and carry out this many interviews”. We did not have freedom in that.

Interviewer: looking back, would you have done things differently?

Student 1: we talked about that in class. As a group of students, we found it very hard to say what we wanted, because you don’t want to be put at a disadvantage you know. The same that an intern would not say to his employer that he is not doing things the right way.”

These reflections by the students of group 2 provide an important insight into the dynamics of the project group. The first thing that should be addressed is the power imbalance that the students described. They felt ‘out of place’ to go against the ideas of the project group. Thus, they consented to the television screen. In this case, the power imbalance was probably reinforced by the pressure to deliver results pushing the students into more of an operational role. They came to feel that they just had to carry out tasks that were set out for them. In the end, the students did not feel there was enough space to express their ideas to the project group, or to help steer the project towards another, perhaps more successful course of action.

This was a crucial moment during the trajectory of the project. The students were supposed to be the spokespersons for the employees and residents. Moreover, they were expecting to be involved in the decisions that were being made during the project. However, the decision making solely happened by the project group members, and focused on pre-existing ideas. In retrospect, this was the moment when the project took a turn away from health staff and residents, and instead started going its own way.

The enthusiasm to accept technology as a real solution provides an excellent example for these situations. The television screen seemed to wait for the perfect moment when the project group would adopt it. After being put up, the screen played a central role in the immediate projects to follow. It continued to spark hope in various people around the organization, as can be judged from the following quotation:

“We collectively confirmed that the first floor of the facility would become the center of activities. We have to create a challenging environment. ... We already purchased a television ... and the students would explore how we can make the environment attractive for residents, so that they are challenged to be more active”

The question I keep asking myself with the benefit of hindsight is: what would have happened if we would have started with the screen getting installed in the first place? Would the situation not have been better off that way? Maybe, then the screen would have been properly integrated into the ideas of the organization. The problem with those questions, and those hopes, is that apparently even I fell prey to the appeal of technology. Even I, along the way, lost sight of the core issue of the project: ensuring freedom for elderly with dementia living at a closed psychogeriatric ward. This insight of what was lost gave me a feeling that the project had failed, as if, for a year, I had achieved nothing. At key moments, when important decisions were made during the project, I had simply gone with the flow of the group, bowing to the promise of technology. In other words, I stood by and watched.

Discussion and Conclusions

In this autoethnography, I described my experiences with stakeholder participation in a living lab to contribute to the literature on living labs. I used three key moments to show that the project’s progress combined with the promise of technology were together the driving force behind crucial project decisions. Thus, even for me as a participant researcher, it proved difficult to facilitate active and meaningful participation of stakeholders in the project. I will elaborate on this issue first and conclude the discussion with a suggestion to conceptualize living labs as situated participatory practices. 

Stakeholder participation, especially focused on end-users, is central to the living lab literature. However, as Schuurman et al. (2015) argue, this participation is only implicitly present in the literature, yet is rarely explicitly elaborated on. Even then, users oftentimes are more like passive participants in testing or validating prototypes (Leminen et al., 2015). One notable exception is the study by Logghe and Schuurman (2017), where action research was used to facilitate participation. Other examples highlight issues regarding end-user participation. In a qualitative case study comparison, Franz et al. (2015) demonstrated a deficit in the traditional co-creation methodology, as the project team was responsible for most decisions about defining the problem, as well as selecting stakeholders and methods. Hyysalo and Hakkarainen (2014) show that collaboration between designers and users is often a hard and frustrating process. They conclude that user involvement should have started at the outset of the project. Both of these examples resonate with my experiences in the project above. The initial framing of the problem happened with the project team only and no other stakeholders involved, which influenced the process later on. In conclusion, stakeholder participation seems to be at the core of the living lab methodology. Yet through studies by those who address it, signals show that it may not happen as naturally as it seems.

A possible explanation may be the focus of living labs on end-results. Many publications take an instrumental perspective on living labs as a means to create impact or meaningful and successful innovations (van Geenhuizen, 2013; Schuurman et al., 2016; Keyson et al., 2017). Hossain et al. (2019) categorize living lab outcomes mainly as making contributions to innovations. In this sense, the success of a living lab depends on its ability to boost innovations. In our project, I observed that as time went on the pressure to deliver tangible results increased. The decision to remove certain stakeholders from the project group was made to speed up the process and deliver results in time. The time pressure thus created a dynamic that favoured a push for tangible results at the expense of active stakeholder participation. Swinkels et al., (2018) showed a similar dynamic where the focus gradually shifted towards improving eHealth technology and away from active patient involvement, mainly because patient involvement was more time consuming than initially expected. Stakeholder participation processes can often be complex and time consuming and require hard work (Hyssalo & Hakkarainen, 2014). It may therefore be tempting when pressured to shift focus to the end results, even though participation is at the heart of the living lab methodology.

In sum, I elaborated on two barriers to meaningful stakeholder participation in this project. First, not all stakeholders participated from the outset of the project, and therefore the motivations and needs of staff members and residents were excluded from the project’s design. This resulted in dissonance between the perspectives of employees and the project group regarding the problem definition. Second, under time pressure combined with a desire for tangible results, more stakeholders were excluded as time passed from actively participating in the project. This all set the stage for an end-result that, in hindsight, seems ridiculous: a television screen.

In this sense, from the perspective of living labs seen as being facilitators for meaningful innovations, one could conclude that the project was a failure. However, the project’s value still lies in the hard work of the individuals who participated, rather than in the technological artifact itself. The meaningful end result can thus be seen as a shared understanding of the process that lead to the television screen. The shared understanding, however, was is not having found a suitable solution for the problem, and thus gaining a shared new perspective that could help on future projects. In the end then, it can be argued that in fact I did not (just) stand by and watch; instead, I was part of an exploratory process that inspired real change in organizational culture. We created a culture where stakeholder participation became a topic of interest and importance.

In the previous paragraph I suggested that value can also be derived from the living lab process itself. For example, Janssen et al. (2015) claim that the value of new innovations is due to “continuous work on developing, adapting, implementing and translating innovations”. We believe this shows it is important to realize that stakeholder participation processes depend on their specific context and the individuals who participate. Living labs, where people work on innovations, can thus be conceptualized as situated practices. The notion of ‘situated practice’ was originally defined as “the part of pedagogy that is constituted by immersion in meaningful practices within a community of learners who are capable of playing multiple and different roles based on their backgrounds and experiences” (The New London Group, 1996). Nowadays the term is applied to other fields than pedagogy, relevant examples being technology studies (Orlikowski, 2000), and design (Paton & Dorst, 2011; Crompton, 2019). In these contexts, situatedness describes how meaning or value is dependent on the social, historical, and physical context. Such an approach is central to living labs that aim to provide a ‘real-life setting’ that includes contextual factors relevant both to the innovation itself and to the stakeholders (Coorevits & Jacobs, 2017; Hossain et al., 2019). In situated practices, the innovation process itself is a source of meaning and value. As noted earlier, the value of the chosen television screen itself was understood as being limited; however, through our work it became a vehicle for mutual learning and meaningful changes in the organizational culture of a closed psychogeriatric ward.

Along the same line, we can more clearly view stakeholder participation as a situated process. For example, the students’ struggles to get in touch with employees, and their reservations to suggest alternative ideas, resulted from the dynamics of that specific time and context. Their role as spokespersons for the work floor was shaped by these dynamics. Similarly, Hyysalo and Hakkarainen (2014) state, “the question is whether the parties engaged in living lab collaboration are willing to go through all the work needed to create the specific and particular relationships by which the relevant information can be made visible and transferred to the other party”. Dedding and Slager (2013) draw focus to this aspect by defining participation as a “situational and interactive process”, wherein ‘participation’ is revealed as a situated practice, an act of doing.

To conclude, we believe that the focus on living labs as situated participatory practices has analytical benefits in that it draws focus to the process of the work around participation and on the context in which it happens. These benefits may help researchers, and other stakeholders, to better understand how value and meaning is, or is not, produced in living labs. Furthermore, we believe it helps provide insight into stakeholder motivations and needs to participate in living lab projects, as well as ways in which meaningful participation is facilitated or hindered. Additionally, meaning and value can be broadened into topics like stakeholder learning and changes in organizational culture. In line with Hakkarainen and Hyssalo (2013), our hope is that in the near future other scholars will publish open and honest, even sometimes difficult accounts of their work on stakeholder participation within living labs. Providing such narrative descriptions of living lab projects, as we have tried to do in this paper by analyzing them as situated participatory practices, may provide invaluable examples, insights, and inspiration for other researchers in the field.

Notes

1. In this article we define “participation” as “a situational and interactive process in which all stakeholders in research and/or policy are in dialogue, doing justice to the lived experiences, knowledge and competences of all actors, especially individuals whose daily life and body are at stake, in all phases of the process, aiming for improvements in quality of care and a more inclusive society.” This is an official translation from the definition in Dedding and Slager (2013).
2. In this article I use the active forms of “I” and “we”. In the text, “I” refers to my (main author) own personal observations or decisions, while “we” refers to decisions made by the project team, as well as our collective interpretations and conclusions from this project.
3. The term “consultation” for a level of participation in this context was coined by Arnstein (1969), which she incorporated into a ladder for citizen participation in governmental projects. The higher on the “rungs” of the ladder, the more power is transferred to citizens. According to Arnstein (1969), consultation that asks citizens for their opinions and perspectives can be considered as a form of “tokenism”, where no “real” power is transferred to those who are consulted, and decision-makers can still ignore the opinions and perspectives of citizens.

Acknowledgements

We would like to thank all of the employees from Pieter van Foreest, at the location Wijdevogelhof, for their time and efforts to facilitate this project. Special thanks go to Ad Blom, Fons van Hoek, Shewa Sidiq, José Bezemer, Nancy Hengelveld-Klomp and Dick Rijken. We are grateful for the reviewers’ thorough and constructive feedback.

Funding

A small research voucher was granted by the Medical Delta to kickstart the project.

---

References

Almirall, E., Lee, M., & Wareham, J. 2012. Mapping Living Labs in the Landscape of Innovation Methodologies. Technology Innovation Management Review, 2(9): 12-18. http://doi.org/10.22215/timreview/603

Arnstein, S.R. 1969. A Ladder Of Citizen Participation. Journal of the American Institute of Planners. Journal Of The American Institute Of Planners, 35(November 2012): 37-41. https://doi.org/10.1080/01944366908977225

Brankaert, R., & den Ouden, E. 2017. The Design-Driven Living Lab: A New Approach to Exploring Solutions to Complex Societal Challenges. Technology Innovation Management Review, 7(1): 44-51. http://doi.org/10.22215/timreview/1049

Coorevits, L., & Jacobs, A. 2017. Taking Real-Life Seriously: An Approach to Decomposing Context Beyond “Environment” in Living Labs. Technology Innovation Management Review, 7(1): 26-36. http://doi.org/10.22215/timreview/1047

Crompton, A. 2019. Inside Co-production: Stakeholder meaning and situated practice. Social Policy and Administration, 53(2): 219-232. https://doi.org/10.1111/spol.12466

Dedding, C.W.M., & Slager, M T. 2013. De rafels van participatie in de gezondheidszorg: van participerende patiënt naar participerende omgeving. Amsterdam: Boom Lemma uitgevers.

Dell’Era, C., & Landoni, P. 2014. Living lab: A methodology between user-centred design and participatory design. Creativity and Innovation Management, 23(2): 137-154.

Franz, Y., Tausz, K., Thiel, S., & Yunus, M. 2015. Contextuality and Co-Creation Matter : A Qualitative Case Study Comparison of Living Lab Concepts in Urban Research. Technology Innovation Management Review, 5(12): 48-55. http://doi.org/10.22215/timreview/952

Geenhuizen, M. van. 2014. Critical factors in health innovation in cities : from ivory tower to living lab. Int. J. Global Environmental Issues, 13(June 2013): 258-280. https://doi.org/10.1504/IJGENVI.2014.064508

Hakkarainen, L., & Hyysalo, S. 2013. How Do We Keep the Living Laboratory Alive? Learning and Conflicts in Living Lab Collaboration. Technology Innovation Management Review, 3(12): 16-22. http://doi.org/10.22215/timreview/749

Hossain, M., Leminen, S., & Westerlund, M. 2019. A systematic review of living lab literature. Journal of Cleaner Production, 213: 976-988. https://doi.org/10.1016/j.jclepro.2018.12.257

Hyysalo, S., & Hakkarainen, L. 2014. What Difference Does a Living Lab Make? Comparing Two Health Technology Innovation Projects. CoDesign, 10(3-4): 191-208. https://doi.org/10.1080/15710882.2014.983936

Janssen, M., Stoopendaal, A.M.V., & Putters, K. 2015. Situated novelty: Introducing a process perspective on the study of innovation. Research Policy, 44(10): 1974-1984. https://doi.org/10.1016/j.respol.2015.06.008

Keyson, D.V, Morrison, G.M., Baedeker, C., & Liedtke, C. 2017. Living Labs to Accelerate Innovation. In D. V Keyson, O. Guerra-Santin, & D. Lockton (Eds.), Living Labs: Design and Assessment of Sustainable Living: 55-61. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-33527-8_5

Leminen, S. 2013. Coordination and Participation in Living Lab Networks. Technology Innovation Management Review, 3(11): 5-14. http://doi.org/10.22215/timreview/740

Leminen, S., Nyström, A.-G., & Westerlund, M. 2015. A typology of creative consumers in living labs. Journal of Engeneering and Technology Management, 37: 6-20.

https://doi.org/10.1016/j.jengtecman.2015.08.008

Leminen, S., & Westerlund, M. 2012. Towards innovation in Living Labs networks. International Journal of Product Development, 17(1-2): 43-49. https://doi.org/10.1504/IJPD.2012.051161

Leminen, S., Westerlund, M., & Nyström, A.-G. 2012. Living Labs as Open-Innovation Networks. Technology Innovation Management Review, 2(9): 6-11. http://doi.org/10.22215/timreview/602

Leminen, S., Westerlund, M., & Nyström, A. G. 2014. On becoming creative consumers - user roles in living labs networks. International Journal of Technology Marketing, 9(1): 33. https://doi.org/10.1504/IJTMKT.2014.058082

Logghe, S., & Schuurman, D. 2017. Action Research as a Framework to Evaluate the Operations of a Living Lab. Technology Innovation Management Review, 7(2): 35-41. http://doi.org/10.22215/timreview/1056

Medical Delta. 2019. Creating real-life experimental environments. https://www.medicaldelta.nl/en/living-labs.

Orlikowski, W.J. 2000. Using Technology and Constituting Structures: A Practice Lens for Studying Technology in Organizations. Organization Science, 11(4): 367-472. https://doi.org/10.1287/orsc.11.4.404.14600

Paton, B., & Dorst, K. 2011. Briefing and reframing: A situated practice. Design Studies, 32(6): 573-587. https://doi.org/10.1016/j.destud.2011.07.002

Scherder, E.J.A., Bogen, T., Eggermont, L.H.P., Hamers, J.P.H., & Swaab, D.F. 2010. The more physical inactivity, the more agitation in dementia. International Psychogeriatrics, 22(8): 1203-1208. https://doi.org/10.1017/S1041610210001493

Schuurman, D., De Marez, L., & Ballon, P. 2016. The Impact of Living Lab Methodology on Open Innovation Contributions and Outcomes. Technology Innovation Management Review, 1(6): 7-16. http://doi.org/10.22215/timreview/956

Schuurman, D., Marez, L. De, & Ballon, P. 2015. Living Labs : a systematic literature review. Open Living Lab Days 2015, proceedings. URL: http://hdl.handle.net/1854/LU-7026155

Schuurman, D., & Protic, S. 2018. Living Labs versus Lean Startups: An Empirical Investigation. Technology Innovation Management Review, 8(12): 7–16. http://doi.org/10.22215/timreview/1201

Swinkels, I.C.S., Huygens, M.W.J., Schoenmakers, T.M., Nijeweme-D’Hollosy, W.O., Velsen, L. van, et al. 2018. Lessons learned from a living lab on the broad adoption of eHealth in primary health care. Journal of Medical Internet Research, 20(3): 1-10. http://doi.org/10.2196/jmir.9110

The New London Group. 1996. A pedagogy of mutliliteracies: designing social futures. Harvard Educational Review, 66(1). https://doi.org/10.17763/haer.66.1.17370n67v22j160u

van Geenhuizen, M. 2018. A framework for the evaluation of living labs as boundary spanners in innovation. Environment and Planning C: Politics and Space, 36(7): 1280-1298. https://doi.org/10.1177/2399654417753623

Veeckman, C., Schuurman, D., Leminen, S., & Westerlund, M. 2013. Linking Living Lab Characteristics and Their Outcomes: Towards a Conceptual Framework. Technology Innovation Management Review, 3(12): 6-15. http://doi.org/10.22215/timreview/748

Westerlund, M., & Leminen, S. 2011. Managing the Challenges of Becoming an Open Innovation Company: Experiences from Living Labs. Technology Innovation Management Review, 1(1): 19-25. http://doi.org/10.22215/timreview/489

Westerlund, M., Leminen, S., & Habib, C. 2018. Key Constructs and a Definition of Living Labs as Innovation Platforms. Technology Innovation Management Review, 8(12): 51-62. http://doi.org/10.22215/timreview/1205

Westerlund, M., Leminen, S., & Rajahonka, M. 2018. A Topic Modelling Analysis of Living Labs Research. Technology Innovation Management Review, 8(7): 40-51. http://doi.org/10.22215/timreview/1170