Effectiveness of Public-Private Partnerships: A Systematic Literature Review

Questions about the ‘effectiveness’ of Public-Private Partnerships (PPPs) have been raised for quite some time now. Although the implementation and performance problems are given some attention while carrying out systematic literature reviews on PPPs, there is no systematic literature review to the best of our knowledge that analyses the ways in which the effectiveness of PPPs is measured in the existing literature. This review of 107 articles intends to contribute to this gap by reviewing the peer-reviewed articles/papers from the year 2000 to 2016. The review particularly investigates three areas (a) the ways in which the effectiveness of PPP is defined, (b) methods used by the researchers to measure the effectiveness of PPPs, and (c) theories/perspectives explaining PPP effectiveness. The results indicate that the effectiveness of PPPs is not clearly determined in the existing literature. Moreover, there is no consensus on what counts as an effective PPP. Regarding measurement methods, the overwhelming majority of researchers have focused only on the identification of factors as are opposed to the actual measurement of effectiveness. With respect to theoretical perspectives, governance theories are the ones that are used in most cases. Based on the findings of this systematic literature review, it is recommended to use six broad 1 University of the Punjab, Pakistan; 2 University of the Punjab, Pakistan; 3 University of the Punjab, Pakistan. Correspondence concerning this article should be addressed to Zulfiqar Ali, University of the Punjab, Pakistan. E-mail: zulfiqarkharalias@gmail.com Effectiveness of PPPs: Systematic Literature Review | 105 Journal of Management and Research (JMR) Volume 7(2): 2020 categories of methods/factors identified as a starting point to measure PPP effectiveness.

on PPP performance, the effectiveness of PPPs has still been a relatively less focused area of research. Researchers have mixed opinions about the conceptualization of the effectiveness of PPPs. For instance, (Widdus, 2001) displayed serious reservations regarding the success of the health of PPPs in all circumstances. A larger governmental role has been indicated as a key plank for securing public interest through PPPs (Jamali, 2004). Likewise, accountability, transparency, management of contracts, and safeguarding the interest of all stakeholders have been declared as of significant importance for the effectiveness of PPPs Singh & Prakash, 2010). While there is some evidence of improved efficiency through PPPs, quality research regarding their effectiveness and implications for the public is still negligible (Hodge & Greve, 2009;Teicher, et al., 2006). Moreover, governance aspects of PPPs are still under-researched as compared to their commercial significance (Hodge & Greve, 2010;Hodge, 2004). There is a need to redress this imbalance. This issue can be better understood through conducting a systematic literature review (hereafter review) of the effectiveness of PPPs covering both commercial and governance aspects.
The performance of PPPs has been highlighted as a key area of future research (Ismail, 2011). Further, researchers have found effectiveness as one of the five most important lines of inquiries in the PPP literature (Torchia, et al., 2015). To the best of our knowledge, there is hardly any review specifically dealing with the effectiveness of PPPs. The effectiveness of PPPs, similar to other public policy arenas, is a contested area (Higgins & Huque, 2015). Despite the rhetoric about the effectiveness of PPPs, there is a mounting criticism regarding the outcomes of such arrangements (Hodge & Greve, 2010). This is indicative of the fact that despite PPP being a buzz word, there is a lot more required to conceptually clarify the effectiveness of PPP. This review seeks to bridge this critical gap and provides insight for informed practice and policy decision making in the PPP sector. Objectives guiding this review are as under:  To explore the theoretical underpinnings of the effectiveness of PPP.
 To define effectiveness and identify methods used by researchers to measure the effectiveness of Public-Private Partnership. The above-stated objectives lead to the following three research questions: 1. What theory/perspective explains the effectiveness of PPP? 2. How effectiveness of PPP has been defined in the literature? 3. What are the methods used by the researchers to measure the effectiveness of a Public Private Partnership (PPP)? To answer the above-mentioned questions, a review of papers/articles from the period 2000 to 2016 was conducted. The review provides a wealth of writings on the effectiveness of PPPs during the period under review. Given the diversity of sectors and geographic spread of PPPs, this review acts as a mosaic presenting a rich description of various perspectives, aspects, models of PPP effectiveness. An effort has been made to follow the "Preferred Reporting Items for Systematic Reviews and Meta-Analysis" (hereon referred to as PRISMA) (Voorberg, et al., 2015).
The results are presented, and recommendations given taking into account various perspectives e.g., public, private, and other stakeholders. This provides critical insight for future PPP projects. As far as the PPP effectiveness is concerned, it has not been clearly and precisely defined in the literature under review. Effectiveness is a broad term that covers the whole spectrum of PPP setting, implementation, performance, and outcomes. Based on the papers included in the review, six broad categories of measurement methods/factors including, the institutional, economic, financial and technical, social and relational, legal and regulatory, region-specific, and leadership factors have been identified. Coming together, the presence or absence of these methods/models will determine the success or otherwise of PPP. In the end, based on the results of the review some recommendations have been offered for effective PPP implementation.
The structure of the review follows a four-part schema. The introduction is followed by the research strategy. The results of the review are presented in the third section. Lastly, a conclusion and recommendations are given.

Research Strategy 2.1. Literature Search Strategy
Before the initiation of the review process, broad themes in PPP, and a specific period for the review were identified through a scoping study. Quite a few seminal researchers have indicated a need to inquire into the claim of the effectiveness of PPP (Hodge, 2009). To make the review authentic, a two-fold review search strategy on the analogy of PRISMA, as far as possible, consisting of study eligibility and report eligibility criteria were adopted.
The search period for articles concerning the effectiveness of PPP is 2000 to 2016. The selection of the year 2000 as a starting point for the instant review was due to the 'Millennium Development Declaration of the United Nations General Assembly 2000, wherein all the member countries were encouraged to ensure partnership and involvement of the private sector for sustainable social development.
Initially, when the researchers searched for "effectiveness of public-private partnership in ISI Web of Science, only three results emerged and that too reduced to a mere 1 (one) when Social science and English language filters were applied. To further explore the area, ISI Web of Science was used. Public-Private Partnership was written in Boolean in the search area. This was followed by one of the nine other keywords in the subfield. To ensure a complete and holistic picture of effectiveness in PPPs, the above-mentioned search words were searched in the topic as opposed to the title or abstract. The last search was made on 21-05-2016. The report eligibility criteria are as under:

Record Selection
The search resulted in 379 articles in total with Effectiveness (29), Evaluation (28), Outcomes (31) Performance (78), Implementation (51), Assessment (13), Networks (40), Collaboration (29) Results (80). Out of 379 articles, 247 were ignored as they were being repeated. Abstracts of all 132 articles were read. Having read the abstracts of all 132 articles, based on relevance, a total of 107 articles were read, examined, and analyzed in detail.
Data extraction of the selected record constitutes a critical aspect of a systematic literature review. While explaining data extraction, (Rutter, et al., 2010) observe that it is essential "To extract the findings from each study in a consistent manner to enable later synthesis, and to extract information to enable quality appraisal so that the findings can be interpreted" (p. 47). To this end, data were extracted on excel spreadsheets making it possible to retrieve, compare, contrast, analyze, and appraise the information at the later stages of the review. In addition to the information gathering on three critical aspects essentially important for this review i.e., theories and perspectives used by researchers to explain effectiveness, the definition of PPP effectiveness, methods/models used by researchers to measure effectiveness, and data were gathered on some other important aspects as well. These other important aspects include information related to the year of publication, journal, country of publication, sector, type of paper, approach/strategy, and research design. International peer-reviewed journal articles on the effectiveness of PPP were included. All books were excluded.

Results of the Systematic Review
In the first place, salient features of records of the review are given as under: The review includes papers from a wide array of journals forming a cross-section of disciplines. The top 5 journals based on the highest number of publications are given below.

Top 5 Countries
In addition to the variety in research journals, the papers represent writings from 36 different countries across the globe. USA (16), Australia (11), China, and the UK (8 each) are the countries with the most papers in the review record. The top 5 countries having the highest number of publications are mentioned below.

Range of PPP Sectors
With regard to the sectoral division of PPPs, papers were divided into three parts. First, there are papers which are concerned with infrastructure PPPs (29.91%). Secondly, there is a small number of papers (5.61%) which cater for both infrastructure and Service Delivery PPPs. Major Chunk of papers (64.49%), however, is concerned with Service Delivery PPPs from a broad assortment of sectors/areas of public policy.

Year Wise Publications Trend
Analysis of data reveals that out of 107 entirely read articles, the maximum number (20) were written in the year 2012 followed by 19 in the year 2015. Conceptual 17%

Research Designs
Broadly speaking, there are 13 different categories and subcategories of designs used by the researchers in the articles analyzed in the review. Forty-seven (47) articles' design was case study and its different versions (38 articles 35.51%) as case study, Comparative case study (5 articles 4.67%), mixed-method case study-triangulation (3 articles 2.80%) and Case study-ethnographic approach (1 article 0.93%) is at the top of the preferred research designs adopted by the researchers. These are followed by 18 conceptual articles/papers (16.82%). A preponderant number of empirical articles are reflective of an increasing trend in PPP-related literature to seek empirical evidence to measure the performance and implementation of PPPs. Mixed Methods design used in 5 articles (4.67%) is also reflective of a new trend. Again, there is evidence of using the Delphi Survey in empirical articles. There are 4 articles (3.74%) in this review study where the Delphi survey has been used. This is yet again evidence of the increasingly scientific nature of inquiry in PPP literature. Another very important strand is a non-participant observation design which is evident in one (1) article. In the proceeding lines of this part, the researchers seek to answer three questions framed to undertake the systematic literature review namely definition of effectiveness in PPP, methods/models used to measure it, and the theories/perspectives of the effectiveness of PPP.

Theories/Perspectives of PPP Effectiveness
There are a whole plethora of theoretical perspectives that researchers have analyzed PPP through. These theories/perspectives have been broadly classified into the following fourteen (14) categories:  (2016) One of the most important and widely used theoretical perspectives is the conglomeration of economic and management theories under the rubric of New Public Management (NPM) such as Transaction Cost Economics (TCE) and Public Choice Theories. This is primarily due to these theories' efficiency and effectiveness claim (De Schepper, Haezendonck, & Dooms, 2015b;Dowling & Kent, 2015a;Nisar, 2013).
Recently, the NPM inspired diffusion of PPPs has been overshadowed by the myriad of Governance theories/perspectives. From amongst the selected papers, about 39 (36.5%) are related to governance aspects of PPP evaluation. Bhuiyan (2010) has observed that failure to observe good governance and politico-administrative culture of a country may cause alternative forms of governance like PPPs to be established in service delivery sectors. Researchers tend to agree that governance and network theories and their various dimensions are helpful in understanding PPP setting, implementation and outcomes (Aars & Fimreite, 2005;Andon, 2012;da Cruz & Marques, 2012;Lippi et al., 2008;Mauldin, 2012;Parola et al., 2013;van den Hurk & Verhoest, 2015;van der Heijden, 2013;Velotti et al., 2012;Wetterberg, 2011).
Two perspectives are of import in response to the debate about theoretical underpinnings of PPP effectiveness. These are the critical and constructivist perspectives. As we have seen that many of the papers (56) are Qualitative while seven (7) are Mixed Method studies. This is reflective of an increasing attempt by the researchers to interpret the phenomenon of PPP through variety of actors/stakeholders. A case in point is various studies on PPPs in Kazakhstan with the stakeholders' perspective and (Mouraviev & Kakabadse, 2014. This emphasis on interpretation has also led to Constructivist Perspective. For instance, (Menashy, 2016) has suggested that shared normative beliefs and perspectives about world in the Civil Society Organizations may be a key element of effective functioning of PPPs in education sector. Again, there is a growing skepticism regarding the rhetoric about PPP effectiveness leading to adoption of Critical perspective (Hodge & Coghill, 2007;Kumari, 2016;Wong et al., 2015).

Definition of PPP Effectiveness
Having read the entire 107 articles it appears that not a single paper has clearly defined the 'effectiveness' of PPP. Similar to the other areas of management and social sciences, effectiveness in PPP also appears to be a vague, highly contested, and undefined term. There is a need for further conceptual and empirical research in order to define the 'effectiveness' of PPP.

Methods used to Measure the Effectiveness
With regard to the methods used by the researchers to measure the effectiveness of a PPP, the review reveals that there are very few studies that have based their findings on some rigorous and comprehensive empirical work. Researchers for the most part have focused on the identification of factors encouraging PPP or suppressing it. Andrews and Entwistle (2010) have found a partnership with private sector organizations to be negatively associated with effectiveness and equity. Availability and access to data is a major problem when it comes to the measurement of effectiveness in PPP projects (Chen, Daito, & Gifford, 2016). Different methods used to measure effectiveness are tabulated as under: Based on the variety of factors/ measurement methods as in Table: 9 these diverse measures/factors are categorized in the following 6 different categories:  (Hayllar, 2010;Ofek, 2015;Higgins & Huque, 2015;Buse & Harmer, 2007;Hodge & Coghill, 2007;Panayides et al., 2015;Becker, 2012;Willem & Lucidarme, 2014;MacDonald, 2012;Wong et al., 2015;Buse & Harmer, 2007;Alasad & Motawa, 2016;Abramov, 2009a;Aaronson, 2011;Loosemore & Cheung, 2015) (Nisar, 2013;Mishra et al., 2013;Andon, 2012;Hadzic et al., 2015;Zhang, 2014;Xu et al., 2012;Samii, Van Wassenhove, & Bhattacharya, 2002

Institutional Factors
This group of methods/factors provides the foundation for a sound PPP. To begin with, accountability figures at the core of PPP effectiveness. Along with effective community participation, accountability is one of the two core components of Good Governance Theory (Hayllar, 2010).
Accountability in PPP has been analyzed by the researchers in two broad dimensions. Firstly, it is about the larger debate where alternative governance mechanisms like PPP are seen as an attempt to avoid accountability from the parliament and other democratic institutions (Hodge & Coghill, 2007). It has been suggested to devise a fairly complex accountability model geared at enhanced transparency may further increase the potential benefits of Privatization and PPP. Hodge and Coghill (2007) have come up with an accountability pyramid indicating various tiers of accountability required for complex arrangements like PPP. Secondly, concerns about accountability in PPPs stem from the realization that a variety of actors pursue different goals and perspectives (Ruuska & Teigland, 2009) which make the PPP arrangements a very complex and tricky one requiring innovative accountability regimes (Becker, 2012). This increased focus on accountability is in line with the concern shared by many researchers to make governments accountable to the representative institutions of the people like the parliaments (Hayllar, 2010). Besides, trust (MacDonald, 2012;Wong et al., 2015), broad government effectiveness, regulatory effectiveness (Panayides et al., 2015), balanced representation of people, transparency, awareness of citizens, capacity building (Aaronson, 2011;Abramov, 2009b) and holistic system thinking (Loosemore & Cheung, 2015) are some other key institutional methods/factors researchers have used to analyze the effectiveness of PPP.

Economic, Financial, and Technical methods/factors
There are researchers who in line with the traditions of New Public Management driven Transaction Cost Economics (TCE) have sought to measure effectiveness strictly in economic terms (Chow, 2014;Nisar, 2013). These are the factors which provided the impetus to the PPPs as it rests on the claim of less transaction cost as compared to the traditional modes of public management. The studies included in the review provide a good view of a variety of economic, financial, and technical methods that are critical to understanding PPP effectiveness. Value for Money (Siemiatycki, 2010) and Risk management (Hwang, Zhao, & Gay, 2013;Ke, Wang, Chan, & Lam, 2010;Xu et al., 2012) are two of the pivotal concepts in the economic and financial analysis of PPP. Researchers, however, have recommended Value at risk (VaR) (Mishra et al., 2013) and Whole of life (Nisar, 2013) approaches to risk for better outcomes of PPPs. Likewise, accounting related problems, affordability for the citizens, and end-users and specific technical criteria constitute other important factors for a viable PPP. Recently, accounting-related issues of PPP have been a special focus of research. Andon (2012) has recommended five steps to ensure effective accounting of PPPs. These steps are (a) underlying nature and rationale for PPPs; (b) processes and procedures aiding decisions to undertake PPPs; (c) processes and procedures for expost evaluations of PPPs; (d) the merit and worth of PPPs; and (e) PPP regulation and guidance. Similarly, the technical capacity of the regulator has been proved as critical for PPP success (Zhang, 2014;Zhang et al., 2009).

Social and Relational Factors
PPPs are embedded in a particular context. Researchers have identified a variety of methods that relate to the social and relational aspects of PPP and are of immense importance for effective PPP implementation. In this regard, literature from the stream of networks and alliances included in the records was of great help. Chowdhury et al. (2011) have highlighted the importance of 'Power' in networks like PPP in the energy sector. As to the contract between all parties of PPP arrangement its flexibility has been proved as a contributing factor for the effectiveness of PPP in the health sector . Researchers have also indicated methods like Innovation & Sustainability ), multi-actor complexity (van den Hurk & Verhoest, 2015, managing partnership and performance measurement (Nisar, 2013), public satisfaction (Alasad & Motawa, 2016;Lieberherr et al., 2012;Thuen & Laerum, 2005), selection of suitable partners (Kumari, 2016) effective communication , alignment of stakeholders' values, the achievement of common goals , conflict management in partnership (Siemiatycki, 2010), cooperation and competition and role of nonstate actors .
Stakeholders of PPP have a critical role in the effectiveness or otherwise of PPPs. Much of the government's success in PPP implementation and performance depends on the effective management of stakeholders' value. If the government's role is overbearing, it will reduce the role of the private party leading to serious implications for the collaborative and co-productive character of PPP (Mouraviev & Kakabadse, 2015a). On the other hand, neglect on the part of the government may lead to lots of missed opportunities for the stakeholders (Mouraviev & Kakabadse, 2014).
The political dimension of PPP effectiveness i.e., Political complexity and political violence have been equally addressed by the researchers. For instance, (Sachs et al., 2007;& van den Hurk & Verhoest, 2015) have accounted for the political dimension of PPP for their effective implementation. The effectiveness of PPP has also been analyzed through the prism of social capital .

Legal and Regulatory Factors
Researchers have also highlighted a variety of legal and regulatory impediments in the way of effective PPP management such as the excessive regulatory role of the government, irregularities in the PPP legal framework which are critical from the private sector perspective (Mouraviev & Kakabadse, 2015a).

Region-Specific Factors
Geographic and regional specificities have a role to play in the effective implementation of any reforms. In his research on PPPs in the European context, (Hadzic et al., 2015) has suggested that PPPs in Europe have to be in line with the European Union's specific requirements for being effctive and result oriented.

Leadership Factors
Of no less importance are the leadership factors i.e., strong leaders (MacDonald, 2012), the role of the manager in renegotiation (Leviakangas et al., 2015), involving the private sector as network owner (Ruuska & Teigland, 2009) and employee engagement measures (Reissner & Pagan, 2013).
The records reveal that 47.92% (47) of the articles have sought to measure PPP effectiveness in one way or the other. While most of these studies were never aimed at exclusive measurement of PPP effectiveness or evaluation, very few (Andrews & Entwistle, 2010;Barr, 2007; sought to exclusively measure and evaluate PPP effectiveness. The studies, for the most part, are focused on the identification and discussion of factors responsible for the success or failure of PPP. Further, the focus appears more on process of PPP as opposed to its outcomes (Andon, 2012;Mouraviev & Kakabadse, 2015b;van den Hurk & Verhoest, 2015;Velotti et al., 2012).
Given the predominant nature of studies focusing on the identification and elaboration of factors of PPP as against their outcome, it is extremely hard to come up with conclusive evidence regarding measurement methods of PPP effectiveness. Having examined the records, it appears that the researchers have taken the PPP effectiveness for granted. A lesser number of Quantitative papers (24%) also lend credence to the finding that measurement of PPP effectiveness is a controversial and tricky issue.

Conclusion and Future Research Directions
Wide spread appeal of PPPs notwithstanding, skepticism abounds when it comes to effectiveness of PPPs. Critics have raised serious questions on the capacity and implementation acumen of the public sector regarding PPPs. With this backdrop, this review was undertaken to examine the latest research on the subject and come up with the answers to three questions dealing with definition of PPP effectiveness, methods to measure effectiveness of PPP and theories and perspectives explaining PPP effectiveness.
Since PPP effectiveness was not easily defined, the search strategy included 9 key words alongside 'Public Private Partnership'. This opened up access to the relevant literature on 'networks', 'collaborations' streams making access to wide range of ideas and concepts possible in addition to PPP and its relevant search words. As far as the PPP effectiveness is concerned, it has not been clearly and precisely defined in the literature under review. Effectiveness is a broad term which covers the whole spectrum of PPP setting, implementation, performance and outcomes. Regarding measurement methods/models of PPP, the records reveal that 47.92% (47) of the articles have sought to measure PPP effectiveness in one way or the other. However, only a few were based on exclusive measurement and evaluation of PPP effectiveness. Various stages of PPP process, however, point at factors strengthening or weakening PPPs. The literature review suggests no clear protocol to evaluate effectiveness of PPP save (Barr, 2007). He has devised an eight principles of the health of PPP evaluation which covers the entire gamut of PPP implementation. Based on the papers included in the review, six broad categories of factors including, the institutional, economic, financial and technical, social and relational, legal and regulatory, region-specific and leadership factors have been identified. Coming together, the presence or absence of these methods/models will determine the success or otherwise of PPP.
The NPM inspired diffusion of PPPs has been overshadowed by the myriad of governance theories/perspectives. From amongst the selected papers, about 39 (36.5%) are related to governance aspects of PPP evaluation. Bhuiyan (2010) has observed that failure to observe good governance and politico-administrative culture of a country may cause alternative forms of governance like PPPs to be established in service delivery sectors.
In order to further enhance our understanding of PPP effectiveness, following recommendations are presented. In the first place, given the paucity of research on the actual outcomes to measure effectiveness of PPPs, there has to be a concerted effort to define and measure PPP effectiveness. Mere rhetoric as to their effectiveness is not sufficient. Secondly, a renewed focus on studies measuring effectiveness quantitatively or in measurable terms is a must. There is, however, a caveat to it. Mathematical computing models or advance economic techniques may be good to understand commercial claims of PPP, but they are no substitute to the credible research on their governance outcomes. Future research to measure effectiveness, therefore, need center on measuring effects of a series of methods/factors. In this regard, six broad categories of methods/factors of effective PPP identified in this review may constitute a good beginning. Nonetheless, as more research on the topic is carried out, some of the factors may be merged or deleted altogether.
With the ever increasing attraction of PPP across the globe, there is an urgent need to buttress the PPP appeal with concrete evidence of their success from the field. Democratic accountability and accounting practices in PPPs need be beefed up as perils for governments to fast encumbering the coming generations with major economic and financial liabilities are higher (Higgins & Huque, 2015). To this end, questions regarding the rationale of effectiveness (why?) and its audience (for whom?) becomes critical. End users especially citizen's perspective in PPP measurement is getting increasing attention satisfaction (Aaronson, 2011;Alasad & Motawa, 2016;Lieberherr et al., 2012;Thuen & Laerum, 2005). Again, as is evident from 40% of papers from Asia and Africa, PPP provides the governments with a huge opportunity in developing and transitional countries to raise the living standard of their population. This, however, is a double-edged sword. Records in the review suggest that if PPPs are not dexterously conceived, designed, and implemented, their effectiveness would be seriously compromised. PPP governance, therefore, should cover the whole implementation process, the performance (both process and product performance) and must be mindful of the political complexity involved in such project (van den Hurk & Verhoest, 2015, Mouraviev & Kakabadse, 2015a).
Despite the utmost care, certain important studies might have escaped attention of the author. This may be partly due to complicated nature of the 'effectiveness' of PPP. As mentioned in the outset of the methodology/search strategy section, initial search of PPP effectiveness on ISI web of Science resulted in just three (3) results. Further, since all books and gray literature was excluded, generalizability of the findings have somewhat compromised. The inclusion of record like books and gray literature would have made this review unmanageable. Time happens to be another delimiting factor. As review was to be submitted within a very short span of time, there may be certain areas were left unattended. Nevertheless, as far as possible, attempt was made to objectively search all such words that would yield results relevant to the review questions.
Be that as it may, to the best of author's knowledge, this is by far the first Systematic Literature review on the effectiveness of PPP. There appears not a single review exclusively treating PPP effectiveness. Secondly, this review has sought to integrate both economic and governance aspects of PPP. Hence, the findings/results are more credible. Thirdly, contrary to the previous reviews focusing on a single sector such as health, transport or infrastructure, this review has cut across the sectoral divide and presented results from all sectors across the infrastructure and service delivery divide. This review, therefore, will be of significance for both academicians and practitioners of PPP.