Factors influencing the implementation of mental health recovery into services: a systematic mixed studies review

Abstract

Background

Countries around the world have committed in policy to transforming their mental health services towards a recovery orientation. How has mental health recovery been implemented into services for adults, and what factors influence the implementation of recovery-oriented services?

Methods

This systematic mixed studies review followed a convergent qualitative synthesis design and used the best-fit framework synthesis method. Librarians ran searches in Ovid- MEDLINE, Ovid-EMBASE, Ovid-PsycInfo, EBSCO-CINAHL Plus with Full Text, ProQuest Dissertations and Theses, Cochrane Library, and Scopus. Two reviewers independently screened studies for inclusion or exclusion using DistillerSR. Qualitative, quantitative, and mixed methods peer-reviewed studies published since 1998 were included if they reported a new effort to transform adult mental health services towards a recovery orientation, and reported findings related to implementation experience, process, or factors. Data was extracted in NVivo12 to the 38 constructs of the Consolidated Framework for Implementation Research (CFIR). The synthesis included a within-case and a cross-case thematic analysis of data coded to each CFIR construct. Cases were types of recovery-oriented innovations.

Results

Seventy studies met our inclusion criteria. These were grouped into seven types of recovery-oriented innovations (cases) for within-case and cross-case synthesis. Themes illustrating common implementation factors across innovations are presented by CFIR domain: Intervention Characteristics (flexibility, relationship building, lived experience); Inner Setting (traditional biomedical vs. recovery-oriented approach, the importance of organizational and policy commitment to recovery-transformation, staff turnover, lack of resources to support personal recovery goals, information gaps about new roles and procedures, interpersonal relationships), Characteristics of Individuals (variability in knowledge about recovery, characteristics of recovery-oriented service providers); Process (the importance of planning, early and continuous engagement with stakeholders). Very little data from included studies was extracted to the outer setting domain, and therefore, we present only some initial observations and note that further research on outer setting implementation factors is needed.

Conclusion

The CFIR required some adaptation for use as an implementation framework in this review. The common implementation factors presented are an important starting point for stakeholders to consider when implementing recovery-oriented services.

Background

Mental health recovery is increasingly the focus of mental health policy, guidelines, and action plans worldwide. Recovery arose from the consumer survivor movement in the late 1980s as mental health service users began publishing on their own recovery experiences [1]. Personal recovery is not to be confounded with clinical recovery, the latter concept referring to measurable disease-focused outcomes such as a reduction in symptoms or hospital days. Personal recovery in contrast is defined as “a way of living a satisfying, hopeful, and contributing life even with limitations caused by illness” (p.527) [2]. Despite widespread adoption of the personal recovery concept, the challenge has been to operationalize the principles of personal recovery into services [3, 4], so that responsibility for recovery becomes a shared responsibility. While traditional mental health services focus on professional control, patient dependency, self-stigma, and hopelessness, the focus of recovery-oriented services is on client empowerment, choice, collaborative professional/client relationships, and community integration. In promoting a life beyond services, recovery also meets a key ethical obligation to honour the personhood and citizenship of people with mental illness.

Research on recovery has proliferated over the past two decades with studies on personal recovery [5,6,7,8,9,10,11], recovery-oriented services [12,13,14,15,16,17,18,19], and provider competencies [20,21,22,23,24]. Conceptual frameworks have been produced [25,26,27,28,29,30] and standardized measures have been developed [31,32,33]. Research has linked recovery to existing theories, e.g. empowerment theory [34, 35], the strengths model [36], capabilities theory [37,38,39], positive psychology [40,41,42], person-centered practice [43, 44], and co-production [45, 46]. Guidelines for recovery-oriented service provision are available [47,48,49,50,51,52,53].

To date, systematic reviews in mental health recovery have focused on conceptualizing personal recovery [25, 29, 54,55,56,57,58,59,60], measurement instruments [28, 61, 62], conceptualizing recovery-oriented practice [63], and intervention effectiveness [64,65,66]. One review described what influences the implementation of peer support work specifically [67]. However, no known systematic review, to date, has been published on how recovery has been implemented into services from an implementation science perspective. To address this knowledge gap, it was deemed appropriate to employ a systematic mixed studies review to ensure that we captured the breadth of evidence across research designs. This review seeks to address the question: How has mental health recovery been implemented into services for adults, and what factors influence the implementation of recovery-oriented services?

Methods

Synthesis design

This systematic mixed studies review follows a convergent qualitative synthesis design [68]. Based on Hong et al.’s classification of convergent synthesis design sub-types, ours can be described as “data-based”, meaning that findings from qualitative, quantitative, and mixed methods studies were extracted concurrently, analysed using the same method, and the review findings are presented together (p.7) [69]. We applied the best-fit framework synthesis method [70]. We chose the Consolidated Framework for Implementation Research (CFIR) [71] as the best-fit framework for this synthesis based on it being a germinal compilation of factors known to influence implementation and our aim being to systematically synthesize the factors known to influence the implementation of recovery-oriented services. The CFIR framework includes 38 constructs grouped into five domains: intervention characteristics, outer setting (outside or beyond the organization), inner setting (within the organization), characteristics of individuals, and process [71]. We used a hybrid deductive-inductive approach [68] that is consistent with the best-fit framework synthesis method [72]. The CFIR framework was used for data extraction by deductively coding findings from primary studies to the 38 constructs. Data within each CFIR construct was inductively coded thematically. Thematic synthesis methods are a common approach to mixed studies convergent qualitative synthesis design [68]. Currently, no published reporting guideline exists for systematic mixed studies reviews. We were guided by relevant elements of both the PRISMA [73] and ENTREQ [74] reporting guidelines (see Additional file 1). A comparison of the published protocol [75] and this review can be found in Additional file 2. A core team of three reviewers (MP, MW, ES) worked closely together on the review and kept a process log to document over 90 review meetings between December 2017 and the date of submission.

Searches

A health sciences librarian conducted comprehensive searches in the following databases: Ovid- MEDLINE, Ovid-EMBASE, Ovid-PsycInfo, EBSCO-CINAHL Plus with Full Text, ProQuest Dissertations and Theses, Cochrane Library, and Scopus from January 1, 1998, to December 20, 2016, using a combination of keywords and MeSH terms. 1998 was chosen as the start date because it was in 1998 that recovery was first defined in an international policy document [76]. The search strategy was peer reviewed by another health sciences librarian using the Peer Review of Electronic Search Strategies checklist [77]. A third librarian (FF) updated the searches on July 25, 2018. No functional limits other than the start date were applied.

The search strategy was developed for MEDLINE (see Additional file 3), and a modification of this strategy was used to search the other databases. Two librarians executed all final searches (initial and updated (FF)), exported the results into EndNote and removed duplicates from the search results. A PRISMA flow chart was used to track the number of studies at each stage of the review. The table of contents of Implementation Science, Psychiatric Services, Psychiatric Rehabilitation, Journal of Mental Health, Administration and Policy in Mental Health and Mental Health Services were searched from January 2012 to March 2017. These journals were selected for having published several studies of interest to our review question. Eight researchers/experts in recovery and system transformation, from different countries, were contacted in July 2017 and asked to share any known empirical studies on the implementation of recovery into services published in the past 2 years or in press. No additional studies were identified through these means.

Study inclusion and exclusion criteria

We included peer-reviewed studies that reported on the implementation process, factors, and experience when implementing new efforts to transform services for adults with serious mental illness towards a recovery-orientation. All study inclusion and exclusion criteria can be found in Table 1. All studies were independently screened over two stages for inclusion by two of three reviewers (MP, MW, ES) using DistillerSR software [78] (stage one: title and abstract. stage 2: full-text screening). Disagreements were resolved in meetings including a third reviewer.

Table 1 Inclusion and exclusion criteria

Study quality assessment

The Mixed Methods Appraisal Tool (MMAT) [79] was used to critically appraise all included studies. MMAT is a validated tool for appraisal of all study designs including mixed methods studies [80]. Studies were not excluded based on critical appraisal. Two reviewers independently appraised each study, using the MMAT template [79] and compared appraisals to arrive at a consensus (BV, TD, CP).¹ Sensitivity analyses answer the question “are the findings robust to the decisions made in the process of obtaining them?” [81]. In this case, we conducted a sensitivity analysis to determine whether our decision not to exclude studies based on quality shaped the findings (e.g. are some findings based solely on lower quality evidence?). One reviewer (MW) applied Houghton et al.’s approach [82] to sensitivity analysis using matrix coding queries in NVivo 12 to visualize the number of studies by MMAT score categories supporting each theme presented in the findings section. The data coded to each theme were plotted against the MMAT score categories in the query. We scored the MMAT by counting “yes” responses and using five as the common denominator since both qualitative and quantitative studies are appraised based on five questions. Mixed methods studies are appraised on 15 questions so scores out of 15 were converted to scores out of five. This led to some scores with decimal points (e.g. 3.33). To simplify we assigned each study to one of four score categories: 0, 1–2, 3–4, and 5. Dividing studies into categories based on the number of critical appraisal criteria met is consistent with other studies that have applied sensitivity analysis to qualitative synthesis findings [70, 82, 83].

Data extraction strategy

Descriptive data such as country, setting, objective, recruitment, data collection methods, theoretical framework, analysis, sample, and characteristics of participants were extracted to a form created in DistillerSR. Given that we used a qualitative approach to convergent synthesis [68] we approached the extraction of study findings in the following way: in qualitative studies, or the qualitative branch of mixed-methods studies, we extracted findings (quotes and authors’ descriptions) from results and discussion sections. In quantitative studies, or the quantitative branch of a mixed-methods study, we extracted the study authors’ own narrative descriptions and summaries of their quantitative results. Extraction of study findings to the CFIR was done in NVivo12 [84]. The data extraction template consisting of the five domains and 38 constructs of the CFIR [71], and their definitions, was pilot tested on five articles. Modifications were made including adding “authors and/or research participants descriptions of…” before each definition, adding an additional construct under a process called engaging with stakeholders, and adding an “additional information” code to each domain so that data that did not fit any of the constructs could be coded there and included in the thematic analysis (see Additional file 4 for data extraction framework used, including definitions).

In NVivo12, we created codes and sub-codes for each domain and construct and sub-construct and included the definitions in the “description” field of each code for easy access during extraction. PDFs of included articles were imported, a case node was created for each article, and all were coded to the case node “included studies”. A case classification sheet was created with descriptive information about the studies (country, perspectives). This process enabled the use of data exploration features in NVivo12, such as framework matrices and matrix coding queries. Data was extracted by coding sections (e.g. a sentence, quote or paragraph) of the PDF to corresponding CFIR constructs. If a section of data illustrated more than one construct, it was coded to each. Factual information provided by the authors that related to CFIR constructs (this usually appeared in background and methods sections) were extracted to a second coding tree but not used in this review.

One reviewer (MW) carried out data extraction. Five studies were co-extracted by a second review (MP, ES) for quality control. Three reviewers (MP, MW, ES) met weekly over this stage to discuss extraction and interpretations of CFIR constructs. Questions about interpreting distinctions between domains and constructs were clarified in a meeting with authors of the CFIR.

Data synthesis and presentation

Due to the wide variety of innovations implemented with the aim of transforming services, three reviewers (MP, MW, ES) worked to conceptually group the 70 included studies into similar types of innovations, as a precursor to analysis. Each innovation group became a “case” and we created case nodes for each innovation group in NVivo12. This facilitated analysis and synthesis conceptually by enabling the reviewers to embed themselves in the extracted data case by case. Table 2 shows each innovation group (case) and the number of studies in each (it also highlights studies that were coded to more than one innovation group (case)—e.g. a study of peer workers doing personal recovery planning. An additional case node was created called “perspectives on implementation of recovery-oriented services in general” for those studies with a broader focus and without enough description of the intervention to enable grouping. These, alongside innovation categories with only one supporting study, were not included in the within-case and cross-case synthesis. In total, 55 studies representing seven innovation groups (cases) were included in a within-case and cross-case analysis (thematic coding within CFIR constructs) and synthesis (writing summarized review findings). Analysis and synthesis in NVivo12 was undertaken by one reviewer (MW) who met weekly with two reviewers to discuss emergent findings and co-interpret data (MP, ES). The following details the steps and procedures of the within-case and cross-case analysis and synthesis.

Table 2 Innovation groups, definitions, and corresponding studies

Within-case analysis

The data extracted to each CFIR construct for each separate innovation group (case) were thematically coded. In Nvivo12, we generated a framework matrix table for each innovation group into which we wrote-up summarized review findings under each theme while easily viewing the extracted data coded to the theme. Summarized review findings were saved to seven documents, one per innovation group (case). The principal investigator (MP) read and commented on all within-case review findings and read all the data underlying each finding to verify that the findings were adequate representations of the extracted data.

Cross-case analysis

The seven documents containing the within-case review findings for each innovation group were imported to NVivo12 as the data set for the cross-case analysis and synthesis. Each document was coded to its corresponding innovation case node and each finding coded to its corresponding CFIR domain and supporting studies. This ensured traceability within NVivo12 between the summarized findings and the underlining data extracted. For cross-case analysis within-case findings from all innovation groups were thematically coded by CFIR domain, starting with intervention characteristics and ending with process. First, summarized findings were coded to the categories “common” (findings that emerged across innovations) and “specific” (findings that related specifically to one innovation). This review focuses on factors common across innovations (cases) that influenced implementation.² The within-case findings categorized as “common” were grouped into cross-case themes. The list of emergent themes was divided into primary and secondary themes. Primary themes were those that occurred across the most innovations (cases) and were best supported by the data. An NVivo12 framework matrix was created for each domain to assist with merging innovation-specific findings into a single narrative illustrating a cross-case theme. Theme names were refined in the process. These represent the final results of this synthesis (Table 3). To reduce the length of the manuscript, we do not report on two primary themes within inner setting (financial issues and staff time) since we believe these factors are widely known and reported on in the implementation science literature. The sustainability of funding and staff perceptions of the time they have for implementation are also fundamental factors to consider when implementing new innovations. We briefly summarize outer setting rather than present findings because this was the least well-supported domain and requires further research.

Table 3 CFIR domains, synthesis themes, and corresponding CFIR constructs data were extracted to

Results

Included studies

Figure 1 is the PRISMA flow chart representing the stages of study selection for this review. In total 70 studies (publications) met our inclusion criteria and were included in this review (see reference list in Additional file 5). Eleven studies originated from four research projects [18, 85, 88, 97, 103, 124, 145,146,147,148, 151] but each publication was treated as an individual study. We reflected on whether this decision impacted our findings by using the Query function in NVivo12 to see how many publications contributing to a theme were from the same study. Only four of the 13 themes presented in the synthesis section below were contributed to by two publications from the same study. We therefore conclude that none of the themes are overrepresented by data emanating from a single project.

Fig. 1

PRISMA flow chart. V&E, vocation and employment; IMR, illness management and recovery; ACT, assertive community treatment

Results of study quality assessment and sensitivity analysis

Of the 70 included studies, 55 were categorized as qualitative, six mixed methods, six quantitative descriptive, one quantitative non-RCT, one quantitative RCT, and one as both quantitative descriptive and qualitative for the purposes of the MMAT assessment. When only part of the findings were relevant to this review, the study was categorized according to the methods used to produce these findings only (hence the categorizations may not match the design of the whole study). For example, if a study collected qualitative and quantitative data but only the qualitative component related to our review question, we categorized the study as qualitative for MMAT. We categorized as mixed methods studies that self-described as such or that collected, analysed, and integrated both quantitative and qualitative data. Studies that did not demonstrate any integration and did not self-label as mixed methods were categorized as “quantitative and qualitative” and both sets of questions in MMAT were used. Overall, the majority of included studies were good quality studies, with 35 studies scoring 5, and 27 scoring as 3–4. There were just as many or more “Can’t Tell” responses compared to “No” suggesting that in some studies lower appraisals may reflect issues in reporting rather than actual quality. Resources were not available to contact authors to clarify “Can’t Tell” appraisals. Appraisals can be found in Additional file 6.

Figure 2 shows the results of a sensitivity analysis of the themes presented under each CFIR domain. We conclude that none of the themes are based solely on lower quality studies and that all themes are well supported by higher quality studies.

Fig. 2

Results of the sensitivity analysis showing how many studies of each appraisal category contribute to each theme. MMAT, Mixed Methods Appraisal Tool

Synthesis

Although we extracted data from and appraised all 70 studies, we chose not to include 15 studies in the within and cross-case synthesis either because they were in innovation groups containing only one study, or they were studies exploring perspective on implementing recovery-oriented services in general (see Table 2). For the 55 studies categorized into one of the seven innovation groups, Table 3 lists by CFIR domain the themes representing common implementation factors synthesized from across these studies. The table also shows the corresponding CFIR constructs to which the data underlying the theme were coded to at the data extraction phase. Illustrative quotes from contributing studies for each theme can be found in Additional file 8. In the following, we describe the general pattern observed across multiple innovations and provide innovation-specific examples in tables (one per CFIR domain).

Intervention characteristics

Flexibility

Across innovations, flexibility was highlighted as an important intervention characteristic that enhanced adaptability and was sometimes seen to provide a relative advantage over traditional services [85, 88, 89, 93, 101, 104, 109, 110, 114, 115, 117, 125,126,127, 129,130,131, 133, 134]. For example, having flexible program content that service providers and service users could pick and choose from and tailor to their own, and their clients’ needs were valued [85, 88, 89, 93, 115, 117, 125, 127, 129]. Flexible role definition for service providers (including peer workers) delivering recovery-oriented services was also valued as it enabled tailoring services to service user and community needs [101, 104, 109, 110, 131]. Service providers could more easily adapt to the needs of service users if the elements surrounding the innovation’s delivery, such as location, mode, timing, frequency, structure, or length were flexible [93, 101, 109, 110, 125, 126, 134]. Flexible funding was another characteristic of some interventions that service providers found helpful in adapting their support to service users’ needs [114, 130, 133]. Table 4 provides examples of flexibility for each innovation group.

Table 4 Intervention characteristics: themes and examples from each innovation group

Relationship building

Relationship building is a key element of innovations aimed at transforming services towards a recovery orientation. A common characteristic of recovery-oriented innovations is the appointment of a service provider who takes on a role predicated upon building trusting relationships with service users (e.g. facilitator) [85, 87, 89, 90, 93, 109, 116, 118, 120, 122]. However, relationship building is also an element of these innovations’ complexity from the point of view of service providers, both in terms of managing relationships with co-workers and service users, and ending relationships with service users [99, 102, 104, 105, 107, 114, 118, 126, 133]. Table 4 provides examples for each innovation group.

Lived experience

Across innovations, the inclusion of people with lived experience of mental health challenges was a valued aspect of designing and packaging recovery-oriented innovations [89, 90, 98, 103, 104, 106, 109, 114,115,116, 125, 128, 129, 135, 137]. When the source of the intervention was a person or group of people with lived experience, this was viewed positively [109, 115, 116]. Those with lived experience were perceived to have a relative advantage over other staff when it came to working in a recovery-oriented way [98, 103, 104, 106, 109, 114]. Table 4 provides examples for each innovation group.

Outer setting

Very little data was extracted to outer setting. The question of how well organizations knew the needs and resources of their clients (as it is framed in CFIR) was rarely directly studied or reported on in the findings. We extracted data to patient needs and resources that spoke generally about service users’ needs but because we judged that this data did not capture the meaning implied by the CFIR construct we do not present it here. External policies were a topic covered descriptively in the background sections of articles rather than directly studied and reported on in findings, and so relatively little data on the topic contributed to this review. One finding worth mention is that external policies of funders and governments regarding billing for services (in the USA especially) caused difficulties for peer worker and service navigation and coordination innovations when these new roles and services did not easily fit existing funding structures [96, 134].

Inner setting

Traditional biomedical vs. recovery-oriented approach

Data extracted and synthesized to three CFIR constructs (compatibility, culture, learning climate), spoke to the overall theme of the challenge with implementing recovery into services that espouse the medical model. Across innovations stakeholders perceived there to be compatibility issues between traditional organizational culture (described in terms of hierarchies, unequal power relations, paternalism, and punishment) and recovery-oriented innovations [94, 98, 99, 103, 104, 108, 111, 115, 119, 122, 124, 131, 134,135,136,137]. An important aspect of traditional organizational or service culture is the way in which staff understand their roles and the priorities of their job. Traditional roles and priorities, such as dealing with acute episodes of mental illness, a focus on medication prescription, and managing risk, were not easily compatible with implementing recovery into services [18, 86, 92, 100, 108, 135]. Many peer workers described a culture of stigma towards mental illness in the organizations in which they worked (and the field of mental health in general) that affected both them and the clients they served [96, 99, 102, 108]. The learning climate for service users—that is, how safe services users feel to try new recovery-oriented services—was sometimes poor because traditional organizational culture from the point of view of some service users is one of distrust and fear due to past negative experiences (e.g. unpleasant or forced treatments) [119, 124, 132]. Specific examples by innovation group are in Table 5.

Table 5 Inner setting: themes and examples from each innovation group

The importance of organizational and policy commitment to recovery-transformation

The compatibility between the wider organizational commitment to recovery-oriented transformation and the recovery innovation being implemented was important for staff [85, 86, 115, 119, 124, 135], especially peer workers [94, 98, 99, 102, 107, 115]. Staff, including peer workers, expressed concern that if recovery-oriented innovations are implemented into a wider organizational setting that does not espouse the same recovery values, then the success of the innovation will be hampered [99, 115, 119, 124, 135]. Leadership and staff buy-in for an innovation is enhanced by a perception that the innovation is compatible with existing organizational and service goals [85, 86]. Specific examples by innovation group are in Table 5.

Staff turnover

Structural characteristics of organizations such as staff turnover and difficulty recruiting and retaining staff were mentioned as implementation challenges across innovation groups [18, 119, 130, 132, 135, 139]. If staff or managers change frequently, the climate for implementation may be compromised by increased workloads, staff stress, and changes to teams’ skill mix [18], and can lead to inefficiencies in building capacity and a continued vision for recovery in the organization [18, 119, 130, 135, 139]. Specific examples by innovation group are in Table 5.

Lack of resources to support personal recovery goals

An aspect of many recovery-oriented innovations is to support service users in their own personal recovery journeys by facilitating access to the resources and services they wish to make use of. A challenge which sometimes arose across different innovation groups was a lack of available resources beyond the innovation both within the organization and in the community for supporting personal recovery goals, such as relevant programming, services, placements, and accommodation [18, 119, 130, 132,133,134,135]. Specific examples by innovation group are in Table 5.

Information gaps about new roles and procedures

The need for additional guidance and training to help clarify roles and specific procedures within innovations was mentioned across studies. This need was mentioned not only for service providers delivering an innovation but also for those who supervised them, other service providers working alongside them, and the service users with whom they worked [91, 92, 95, 96, 98, 100, 104, 108, 110, 119, 122, 128, 130, 131, 133, 134]. When recovery innovations involve the implementation of new roles, lack of access to information about the new role can cause a number of difficulties in the workplace including added stress, confusion, difficult relationships, and work [104, 108, 110, 130, 131, 133, 134]. Service users, new staff, and existing staff also noted needs for greater access to information about new procedures, roles, or services on offer [91, 92, 94, 97, 100, 102, 108, 119, 122, 128]. Specific examples by innovation group are in Table 5.

Interpersonal relationships

As reported under intervention characteristics, relationship building is both a key design feature and a source of complexity for recovery-oriented innovations. Interpersonal relationships play out in the inner setting and can be helped or hindered by existing relationships. Since the CFIR framework does not have a construct related to relationships in the inner setting we developed an additional construct called “interpersonal relationships” from the data extracted to additional information. Building good interpersonal relationships between existing staff and newly hired service providers taking-up new roles is an important factor for innovation success [98, 105, 106, 108, 114, 131]. So are positive interpersonal relationships (described as trusting, respectful, mutual esteem, supportive, and caring) between staff providing recovery-oriented services and service users [109, 117, 122, 130, 133]. Managing expectations within the service provider-service user relationship was also an important element of positive interpersonal relationships and fostering a positive implementation climate [85, 88, 119, 121, 122]. Pre-existing interpersonal relationships between staff or staff and service users can at times facilitate [85, 88, 93] or pose certain challenges [99, 102, 126] to implementation. Specific examples by innovation group are in Table 5.

Characteristics of Individuals

Variability in knowledge about recovery

The issue of variability in understandings of the concept of recovery was mentioned across studies [18, 86, 92, 94, 99, 102, 109, 113, 117, 119, 121, 122, 129, 135,136,137, 139]. While a good understanding of recovery principles was specifically noted in some studies [109, 117, 122, 135], in others, some non-peer service providers still confounded personal and clinical recovery [92, 94, 99, 102], and expressed a belief that not all service users could participate in recovery-oriented services because they lacked some necessary quality or level of wellness [86, 129, 136, 137, 139], thus demonstrating a lack of familiarity with the facts, truths, and principles of a recovery-orientation. Examples by innovation group are in Table 6.

Table 6 Characteristics of Individuals: themes and examples from each innovation group

Characteristics of recovery-oriented service providers

The experience of the innovations and their relative success were considered to be closely tied to the specific attributes of the service provider delivering the recovery-oriented service [99, 104, 109, 111, 117, 121, 122, 124, 130, 131, 133]. Positive experiences and implementation success were related to positive personal attributes including being respectful, encouraging, helpful, collaborative, warm, patient, understanding, dependable, trustworthy, professional, good at communicating and listening, hardworking, able to build rapport, empathetic, positive, and optimistic [99, 106, 109, 111, 117, 124, 130, 131, 133]. Negative attributes associated with negative experiences were service providers being patronizing in their approach, not genuine in their compassion or formulaic and generic in their approach, or having done an inadequate job discussing recovery [109, 121, 122]. Examples by innovation group are in Table 6.

Process

The importance of planning

The importance of planning was exemplified in examples of good and poor planning of the implementation process. Good planning was about anticipating known or expected challenges and building in processes for mitigating them [91, 95, 102, 103, 108, 109] Where implementation challenges were encountered these were associated with inadequate planning, particularly around the availability of protocols, guidelines, and/or clear information on roles, information management, and training [95, 112, 130, 134]. Examples by innovation group are in Table 7.

Table 7 Process: themes and examples from each innovation group

Early and continuous engagement with stakeholders

The importance of engaging with a variety of stakeholders early and continuously in the implementation process was mentioned in a number of studies across innovations [91, 109, 117, 119, 121, 124, 127, 130, 134]. Some studies highlighted successful engagement [91, 109, 127] whereas others noted that early and continuous engagement with stakeholders was lacking and needed [94, 109, 117, 119, 121, 124, 130, 134]. Successful engagement was around involving key stakeholders in selecting the innovation [91], refining associated materials [109], and leveraging resources [127]. Examples by innovation group are in Table 7.

Discussion

Our review is, to the best of our knowledge, the first systematic review on the implementation of recovery-oriented services. Our synthesis has demonstrated how recovery has been operationalized into different innovations, and the common factors that influence its implementation. In terms of the characteristics of the recovery-oriented innovations, flexibility, relationship building, and lived experience are important factors to consider when designing innovations. At the level of organizations, traditional biomedical culture, staff turnover, available resources to support personal recovery goals, gaps in access to knowledge and information about new roles and procedures, and interpersonal relationships are essential factors to anticipate and plan for. The evidence to date also notes the issue of individual variation in recovery knowledge and the characteristics that make up recovery-oriented service providers. Finally, planning is key, as is engaging early with stakeholders and continuing to do so over the course of implementation. In this section. we will discuss some of these, and the CFIR constructs they relate to, in more detail, including how these findings compare to reviews of other interventions that used CFIR.

It is important to note that this was not a review of all the programs and services that exist in recovery, or the effectiveness of innovations, but rather was a review of research that has studied the process, experience, or factors that shape implementation. Some of the innovations identified are well established in some countries (peer workers) and some are new (e-innovations, recovery colleges). All help to operationalize recovery guidelines. Unlike innovations such as new treatments, the aim of recovery innovations is to transform mental health services towards a recovery orientation. By virtue of this, they are complex innovations primarily targeting deep culture change, not simply at the individual behavioural level but at the organization and system level. Many countries have committed in policy to implementing a recovery-orientation into their services [154] but knowing how to do this is the more challenging question. The literature synthesized here demonstrates how recovery as a policy, strategy, or approach has been operationalized into new recovery-specific innovations such as new training programs, new services, and new roles in the service system. Most aim to change wider organizational culture through these specific actions and many studies identified the importance of embedding implementation of these types of innovations within a wider commitment to recovery transformation [18, 94, 98, 99, 102, 107, 115, 119, 124, 135]. For transformation to happen, widespread change across organizations, from paperwork, to language, to hiring structures, need to change [155]. However, taking this on all at once is a daunting task, and decision-makers may prefer to make a start by introducing more tangible innovations like those included in this synthesis.

Like other systematic reviews, on different topics however, we used the CFIR as both a framework for extraction and synthesis [156,157,158,159,160,161,162,163]. Comparing our results to those of these reviews highlights important differences in what some constructs mean in the context of implementing recovery-oriented services compared to other interventions. For example, complexity in other reviews related to things like the length of consultations [156], difficult changes to workload, routines, and priorities [159], technical aspects like screening, resources, and number of professionals involved [157], and challenges with software and hardware [160]. In the case of recovery, complexity also related to managing relationships since a core characteristic of the innovations was making a change to the way service providers and service users interact. Similarly, culture was not reported on in depth in other reviews, whereas in our review it is highly significant across studies. This is likely because recovery is about system transformation and organizational culture change, and is quite a radical departure from traditional mental health services [48, 155]. Similar to other reviews using CFIR, included studies contributed the least data to outer setting [159, 161], and process [162].

Overall the CFIR, as a compilation of factors known to influence implementation, worked well as a data-extraction and synthesis framework, suggesting that implementation factors in the context of recovery-oriented services are similar to innovations in health and social care. However, in order to synthesize the evidence on implementing recovery into services we did have to adapt the CFIR framework- which itself is a contribution consistent with the best-fit framework synthesis method [72]. We replaced more medicalized terminology in the CFIR like “patients” and “intervention” with “service user” and “innovation”. Services should not intervene on someone’s recovery; they should support it through providing recovery-oriented services [49]. We also observed that service users (in CFIR terms “patients”) are inherently framed as outside the inner setting in CFIR—located explicitly in the outer setting in the 2009 version of CFIR we used [71]. The language of the CFIR definitions for the inner setting imply that organizational staff are the focus of the inner setting. However, from a recovery perspective, service users are actors within the inner setting and we took this view when extracting data. The result was that issues like service users perspectives’ on the learning climate were extracted to the inner setting and not to the outer setting or domain of individuals.

In terms of specific constructs, we added two. One was the sub-construct “engaging stakeholders” within the engaging construct in the process domain. While the “engaging” construct focuses on how stakeholders are attracted to participate in the innovation, the idea of engagement in the studies in this review related more closely to the recovery-oriented principle of co-production, that is engaging with stakeholders such as staff and service users to design and develop innovations [164, 165]. It is important to note that we used the 2009 version of CFIR [71], whereas more recent iterations have included the constructs “key stakeholders” and “patients and consumers” [166]. However, from a recovery perspective, we see no reason why consumers/service users should not inherently be considered key stakeholders. We also developed an additional construct within inner setting called “interpersonal relationships” out of data that did not fit elsewhere. Interpersonal relationships can refer to staff relationships (including with managers) or staff and service user relationships, goes beyond issues of communication and networks, and is an important part of the implementation climate. In the case of recovery, which aims to transform the way service users and service providers relate to one another [167, 168], the state of these relationships before and during implementation is an important implementation factor to consider.

Review limitations

Because of the conceptual ambiguity surrounding recovery, and frequent misuse of the term [169], we had to keep our search criteria broad and found it conceptually challenging to determine when described programs were truly recovery-oriented and new, and when the word recovery was tokenistic or in fact referring to clinical recovery. Primary articles were inconsistent in providing a referenced definition of recovery-oriented services. We may have excluded studies that other reviewers would have included. Since we chose not to include systematic reviews or dissertations in this review, in retrospect we should not have searched the Cochrane Library or the ProQuest Dissertations and Theses databases.

Another conceptual challenge in this review was deciding on what constituted an implementation study in a field (mental health recovery) that has only recently begun intersecting with implementation science. Only 16 of the included studies used the word implementation in their title or keywords. This justifies why we did not rely on this label in our search to locate studies, but also signifies we are in an early stage of implementation research in this area. Another indication of this is the fact that we could only identify six studies that used an implementation-related framework in their research [103, 109, 112, 119, 134, 139]. By extension, the use of standard implementation terminology like that provided in the CFIR was infrequent. It was up to the reviewer extracting data to become intimately familiar with the CFIR construct definitions and see reference to them in the primary studies when the primary studies themselves were, for the most part, not using this terminology. If resources had been available, ideally two reviewers could have independently extracted data.

Another limitation may have been our decision to exclude pre-implementation studies, program descriptions, and grey literature. These may have included additional information on implementation factors, described other types of innovations, and have widened the geographical spread of studies. Finally, it is important to acknowledge that critical appraisal is a contested topic [170] but also a fundamental step in a systematic review [171]. Since critical appraisal is the result of two individuals’ judgements, our sensitivity analyses should be interpreted with due acknowledgement that scores could have been different had two other reviewers applied the MMAT.

Recommendations for future research

The evidence base on the implementation of recovery into services to date has allowed us to identify important factors but not to study their exact mechanisms or effects, for example how exactly poor flexibility might lead to poor outcomes, or how greater flexibility may lead to better implementation outcomes. Research relating implementation factors to implementation outcomes is needed, as is research relating implementation barriers to implementation strategies. Stakeholders involved in implementation efforts can use tools like the CFIR-ERIC Matching Tool v.1 [166] to help prioritize strategies to consider including in their implementation plans. In this review, we screened program descriptions from regions like South America, and predict that within the coming years we will see additional research publications from non-English language countries evaluating the implementation of recovery. Future reviews and updates should pay particular attention to this emerging literature. Future research should also empirically study research participants’ perspectives on outer setting and process CFIR constructs. This synthesis found that we have the least evidence on these two domains. So far, primary study authors have tended to report outer setting and process issues factually as part of background or program description rather than explicitly targeting them in data collection. Lastly, mental health recovery researchers considering using the CFIR in their research may want to adopt some of the adaptations we describe in the discussion. These adaptations made the CFIR more compatible with mental health recovery in this review and may prove useful for future primary implementation research on recovery.

Conclusions

This systematic mixed studies review has highlighted the factors known to influence the implementation of recovery-oriented services based on the evidence available to date. There are many types of innovations that operationalize recovery-transformation of services. This review identifies the factors that decision makers should consider in the domains of intervention characteristics, outer setting, inner setting, individuals, and process, regardless of the specific recovery-oriented innovation selected for implementation.