Social Science Research and Reforms of International Institutions

I. Introduction

After a period of robust growth with the creation of a significant number of new international courts and tribunals, international law has now come to a phase of backlashes and recalibration. Several African countries have withdrawn from the International Criminal Court (ICC) amid criticism of the court’s bias against African countries.1 One of the rallying points during the Brexit campaign was the Court of Justice of the European Union’s (CJEU’s) jurisdiction over the United Kingdom, jurisdiction which has come to an end following the finalization of Brexit.2 Latin American countries such as Bolivia, Ecuador, and Venezuela have withdrawn from the International Centre for Settlement of Investment Disputes (ICSID) Convention after a series of claims being filed by foreign investors against these countries at ICSID.3 Other countries have, en masse, terminated the bilateral investment treaties into which they entered.4 Even the World Trade Organization’s (WTO) dispute settlement system, which was once viewed as the “crown jewel” of the WTO, has now become partially paralyzed because of the United States’ continuous objections to the reappointment of Appellate Body Members.5

In the meantime, countries are actively engaging in discussions of reforming international courts and tribunals. More than forty-five countries are participating in discussions of possible reforms of investor-state dispute settlement (ISDS) under the auspices of Working Group III of the United Nations Commission on International Trade Law (UNCITRAL).6 Major WTO member countries also agree that the system needs reform and have started dialogues on this topic.7 Other international courts such as the ICC and CJEU have also been the subject of reform proposals propelled by criticism against these institutions.8

As Abebe, Chilton, and Ginsburg note in their article, the evolution of scholarship on international law has always been influenced by real-world problems.9 While earlier literature takes a largely theoretical approach to studying problems arising from international law practice, in the past two decades, there has been an empirical turn in international law scholarship.10 Nevertheless, until recently, this line of empirical research has been largely motivated by high-level questions from prior theoretical debates, such as how international law is produced and whether international law matters.11 While empirically assessing these issues can have implications for larger normative questions, it often does not speak directly to which normative prescriptions should be adopted to address real-world problems in international law.

This Essay argues that the backlash international law is currently facing, and the ongoing reforms of international institutions underscore the need for more social science research that is geared toward examining the causes of existing institutional problems. Abebe, Chilton, and Ginsburg describe social science research as identifying a research question, developing a specific hypothesis that can be empirically assessed, identifying a research design and data to assess the validity of the hypothesis, and presenting results while acknowledging the assumptions upon which they are based and the level of uncertainty associated with those results.12 Building on their framework, this Essay proposes that to help inform the reform of various international institutions, more social science research should approach an existing institutional problem by asking which factors may have caused the problem, generating testable hypotheses based on a potential cause, and developing a research design that allows one to draw causal inferences about the effect of this cause.

Understanding the causes of a problem helps inform more tailored institutional reforms that specifically address these underlying causes. This is particularly important in the context of reforms of international legal institutions because of the wide variety of stakeholders. For example, during the ISDS reform discussions at UNCITRAL Working Group III, while countries generally agreed that the existing ISDS system needs reform, their positions diverged considerably as to which reforms to pursue.13 After multiple rounds of discussions over the past three years, countries have identified six major areas of concerns over ISDS: “excessive costs, excessive duration of proceedings, lack of consistency in legal interpretation, incorrectness of decisions, lack of arbitral diversity, and lack of independence, impartiality, and neutrality of ISDS adjudicators.”14 With respect to each of these concerns, the Working Group has put forward various reform options.15 Countries have expressed divergent views on these options and have yet to reach consensus on the adoption of any reforms.16 Social science research that clearly identifies the causes of existing problems will help facilitate consensus building amongst countries and provide guidance in terms of which reform option may be best suited to address a particular problem. Indeed, each reform option comes with its own trade-offs.17 This makes it even more important to obtain a comprehensive understanding of the existing problem first, instead of rushing to implement reforms that may not get at the real causes of the problem.

As Abebe, Chilton, and Ginsburg note in their article, social science research is pluralistic in methods.18 Both qualitative and quantitative methods can be instrumental in enhancing our understanding about the causes of a particular problem in the study of international law. However, the emphasis of these two types of methods is slightly different. Qualitative methods are more often applied to locate the potential causes of a problem, whereas quantitative methods are more often applied to assess the effect of a particular causal factor.19 For example, one may conduct interviews to understand what has caused a particular problem and, on the other hand, apply a reduced-form analysis using observational data to identify the effect of a potential cause. Social scientists should embrace the use of both methods and, as Abebe, Chilton, and Ginsburg suggest in their article, pick the method most appropriate to the problem at hand.20

In applying quantitative methods to the study of international law, one major challenge is causal identification. International law, by its nature, is influenced by various forces working together. It can be difficult to tease out the effect of a particular factor in a clean manner. In addition, unlike studies evaluating policy merits or effectiveness in the domestic setting that can leverage variations at the jurisdictional level (for example, a difference in differences study examining the employment effect of minimum wage increases in New Jersey versus Pennsylvania), it is difficult for international law scholars to find such variations to figure out whether a policy or institutional option is desirable or not, as international law is meant to be international and universal.

That said, this does not mean causal identification is impossible in the study of international law. As Shaffer and Ginsburg pointed out almost a decade ago, empirical work in international law should be guided by conditional international law theory, which focuses on the conditions under which international law is produced and has effects.21 One empirical strategy for conditional international law theory is to leverage variations in the contexts in which international law operates, such as variations in the underlying treaty provisions, in the legal claims advanced during dispute settlement, and in the international law participants themselves. With the increased availability of fine-grained international law data and the development in identification strategies, one can employ a research design that exploits variations in the aforementioned dimensions to draw causal inferences in studying questions related to existing problems, such as why certain countries chose to terminate the international investment treaties they signed or what has led to prolonged proceedings in ISDS and WTO dispute settlement.

Importantly, one should always be clear about the assumptions being made and any limitations associated with the methodology or results. Because international law actors may not be familiar with sophisticated statistical methods, transparency on methodology and caution against overclaiming can help alleviate potential concerns about the credibility of the results.

The challenges of causal inferences in the international law context make it important to combine quantitative analysis with qualitative methods which contribute to the development of theories guiding quantitative research and provide valuable insights where quantitative methods have limitations.  Another area that awaits more future work is the replication of prior research findings, which will generate more confidence that reform proposals made on the basis of existing social science research are, in fact, supported by robust empirical evidence.

In the remainder of this Essay, I use the excessive duration and costs problem of ISDS proceedings as an example to discuss how social science research may be applied to explore causes of a problem and shed light on potential solutions.

II. Social Science Research and the Excessive Duration and Costs Problem of ISDS

ISDS is a dispute settlement mechanism that allows foreign investors to bring claims against sovereign states before arbitral tribunals for alleged violations of the latter’s investment protection obligations. ISDS has long been criticized for its lengthy and costly proceedings.22 According to UNCITRAL, an average ISDS case lasts for approximately 3.75 years, which translates into average litigation and arbitration costs of millions of dollars for each side of the dispute.23 The excessive length (and relatedly, excessive costs) of investor-state arbitration, which was designed to be a cost- and time-effective dispute settlement system, has given rise to wide criticism from countries participating in the process.24 However, while there has been a recent increase in empirical research documenting this problem,25 few studies have empirically examined the causes of the excessive duration and costs problem.26 Different diagnoses of the causes may point to different prescriptions to address the problem. In particular, if one cause for prolonged proceedings is that countries are unwilling to settle cases because of domestic political pressure, then perhaps more institutional reform efforts should be focused on dispute prevention and mitigation at the domestic level rather than case management reforms or other procedural changes at the international level.

In a new article, I examine this potential cause by exploring the influence of domestic political pressure on state settlement behavior in ISDS cases.27 The overall settlement rate in ISDS is much lower than what is typical in other litigation settings—only around twenty percent of all concluded ISDS cases were settled.28 News reports and surveys of ISDS practitioners show that states are averse to settlement, which tends to generate public criticism for capitulating to the demands of foreign investors and “selling out” using public money.29 In a 2018 survey of ninety-seven experienced practitioners and government officials who have participated in ISDS proceedings, the most frequently mentioned obstacle to settlement in investor-state disputes was the desire to shift the blame to a third-party adjudicator, so that the government would not have to take responsibility for compensating foreign investors with public money.30 These qualitative findings suggest that domestic political pressure may have caused respondent states to delay settlement or forego settlement opportunities altogether, which has the effect of substantially extending the length of arbitration proceedings.

To identify the effect of anticipated domestic public pressure on case settlement, I exploit variation in election timing in the respondent states and use it as a proxy for the government’s sensitivity to domestic public pressure. Electoral disapproval is more likely to translate into loss of political power as elections approach. Hence, elected officials likely become increasingly cautious with settling with foreign investors in the run-up to elections. On the other hand, the time left until the next election should be exogenous to case quality. Thus, the research design allows one to draw causal inferences about the effects of domestic political pressure on state settlement decisions.

In the article, I find that a state becomes less likely to settle an ISDS case as it gets closer to the next election of the state leader. This finding suggests that state settlement decisions in ISDS are not made solely based on case merits.  Instead, case settlement also appears to be affected by a political calculus which fluctuates based on election timing. Such political influence leads to delay, and in cases where domestic political pressure is high enough, a failure to settle. This corroborates prior qualitative findings and points to domestic political influence on state settlement decisions as a cause of lengthy and costly ISDS proceedings.

This finding has direct normative implications regarding pursuing reforms to address the problem of excessive duration (and relatedly, excessive costs) associated with ISDS proceedings. UNCITRAL has identified several possible measures to address concerns about excessive duration and costs, including promotion of dispute prevention and mitigation policies, implementing stricter timelines, adding new procedural rules to prevent disputing parties from delaying the process, establishing advisory centers to provide legal advice to countries, and providing arbitrators with case management training.31 A majority of these measures aim to shorten case duration and reduce associated costs through reforming various procedural aspects of ISDS.32   While such procedural reform efforts can be valuable, the finding that countries’ settlement decisions are affected by domestic political pressure highlights the importance of mechanisms that insulate respondent state decision makers in ISDS proceedings from such domestic political pressure. Without such mechanisms in place, even if reforms are made to improve case management and streamline the proceedings, case duration and costs may not be substantially reduced when domestic political pressure forces respondent states to continue litigation without settlement.

One way to insulate settlement decision makers from domestic political pressure is to delegate the decision-making power to a specialized agency or commission and have legal experts there issue detailed reports to explain the reasoning behind the settlement decision, illustrating why settlement is the desirable strategy in a particular case. These legal experts should be less susceptible to domestic political pressure as compared to politicians, and collective decision may further relieve them of concerns about potential repercussions from the domestic public. An expert report with detailed reasoning also helps elected officials justify the settlement decision to domestic audiences, which can mitigate potential domestic backlash against settlement with foreign investors.

A few Latin American countries have already established mechanisms serving similar purposes.33 Peru, for example, established an inter-agency commission, Coordination and Response System for International Investment Disputes (SICRECI), which specializes in the prevention and handling of investor-state disputes.34 Among other things, this commission is responsible for “assessing the possibility of reaching a settlement in the direct negotiation stage and participating in these negotiations.”35 The work of the commission is supported by a Technical Secretariat, whose core functions include “conducting an initial assessment of the dispute and preparing a preliminary report that is submitted to the other members; preparing reports on courses of action and strategies and any other information necessary for the Commission to perform its duties.”36 In addition, the commission also works closely with the Peruvian entities responsible for concluding investment agreements to impose new requirements on foreign investors. Under these new requirements, investors have to present detailed information about the dispute at the time of dispute notification, so as to “facilitate the complete and full understanding of the dispute by the Special Commission . . . [to] increase the possibility of achieving a satisfactory outcome in the negotiation stage.”37

By requiring legal experts in the Special Commission to issue reports detailing the reasoning for the litigation (or settlement) strategies, based on all of the available information about the dispute, these institutional arrangements have the potential of insulating the decision makers from domestic political pressure so that they can make settlement decisions based on case merits rather than political whims. In this way, more cases, which otherwise would not be settled due to intense domestic political pressure, will be settled, likely at earlier stages of the dispute.  Indeed, Peru credits its system with averting around 300 potential arbitration proceedings.38  The facilitation of early settlement (when it is demonstrated to be more desirable than litigation) relieves respondent states of the burden of spending considerable public funds and resources defending investor claims, therefore contributing to the reduction of duration and costs of ISDS proceedings.

Another mechanism that may serve similar purposes is to have a third party independently assess the facts of a dispute and issue a report of its fact findings, upon which respondent state governments can rely to make settlement decisions. In this regard, ICSID’s most recent rules amendment proposals contain a stand-alone set of rules for fact-finding, which offer states and investors the opportunity to constitute a committee to make objective findings of fact that could resolve their dispute.39 Reports resulting from these fact-finding proceedings can provide basis for respondent state governments’ settlement decisions and therefore alleviate their concerns about domestic backlash (to the extent that the report implies settlement is likely more desirable than litigation). Of course, strict time limits need to be imposed so that the fact-finding will not become another costly and drawn-out process itself.

III. Social Science Research and Other Aspects of ISDS Reform

The social science approach can be extended to study a broader range of problems in the international investment law sphere that are pertinent to ongoing ISDS reform discussions. For example, many critics consider the recent explosion of ISDS cases to be a major factor in contributing to a number of countries’ terminations of bilateral investment treaties.40 Countries learn about the consequences of their treaty commitments through their experiences in ISDS disputes, which propels them to terminate these treaties.41 This has brought a lot of blame and controversy to ISDS, which is now at the center of a legitimacy crisis. However, an alternative or additional cause of the problem may be that these countries did not carefully negotiate the terms of these treaties and, as a result, are more likely to renege on their prior commitments. These two causes implicate different reform options to address the problem. While one suggests that more should be done to improve the fairness and quality of ISDS decisions, which is what most reform proposals in the area have centered on so far, the other points to the importance of having countries spend more effort and use more expertise in negotiating treaties. More social science research on this issue will help redirect reform efforts to needed places.

As another example, also in the context of ISDS reform, while countries share concerns over lack of impartiality and independence among arbitrators, their views diverge when it comes to deciding which reforms to pursue to address this problem. Some countries advocate for an overhaul of the ISDS system by replacing investor-state arbitration with a multilateral investment court, whereas other countries favor retaining the existing system but instituting more incremental reforms that redress these specific concerns.42 More social science research on what causes biased decisions can help provide states with a clearer picture regarding which path to pursue. For instance, if there is no evidence that reappointment incentives per se, which are inherent to ad hoc appointment—the defining feature of arbitration—lead to biased decisions, then perhaps more incremental reforms, such as refining the arbitrator’s code of conduct, address the problem better than a systematic overhaul does.43

IV. Conclusion

In a time when international law is facing significant challenges and undergoing rapid changes, Abebe, Chilton, and Ginsburg’s call for more social science research in the field is timely and important. The social science research on international law has so far largely focused on documenting existing problems and examining high-level theoretical questions. This Essay posits that to inform reforms of international institutions, more social science research should be focused on identifying the causes of these problems and proposing policies or reforms that specifically address those underlying causes. This requires us to explore new frontiers of the conditional international law theory and exploit variations in both the substance of international law and international law participants. In an international setting where decision-making requires the consensus of a wide range of stakeholders, obtaining a comprehensive understanding of what caused existing problems is both important and necessary for implementing tailored reforms that help the field of international law overcome challenges at a pivotal time.

• 1South Africa: Withdrawal from the Rome Statute of the International Criminal Court, U.N. Doc. CN.786.2016.TREATIES-XVIII.10 (Oct. 25, 2016); Burundi: Withdrawal from Rome Statute of the International Criminal Court, U.N. Doc. C.N.805.2016.TREATIES-XVIII.10 (Oct. 28, 2016); Gambia: Withdrawal from the Rome Statute of the International Criminal Court, U.N. Doc. C.N.862.2016.TREATIES-XVIII.10 (Nov. 11, 2016).
• 2Eleanor Bennett, UK Government Trumpets Renewed ‘Control of Our Laws’ and End of ECJ Jurisdiction After Making EU Brexit Deal, Jurist (Dec. 24, 2020), https://perma.cc/ZDG4-L363.
• 3Matthew Weiniger, Christian Leathley & Joanne Greenaway, Venezuela Follows Bolivia and Ecuador with Plans to Denounce ICSID Convention, Lexology (Jan. 19, 2012), https://perma.cc/RSE4-7CU4.
• 4Kavaljit Singh & Burghard Ilge, India Overhauls Its Investment Treaty Regime, Fin. Times (Jul. 15, 2016), https://perma.cc/GPR7-BNHU; see also Ben Bland & Shawn Donnan, Indonesia to Terminate More than 60 Bilateral Investment Treaties, Fin. Times (Mar. 26, 2014), https://perma.cc/BVQ4-PUKR.
• 5Cosette Creamer, From the WTO’s Crown Jewel to Its Crown of Thorns, 113 AJIL Unbound 51, 51 (2019).
• 6Julian Arato, ISDS Reform: Working Group III Gets Down to Brass Tacks, Int’l Econ. L. & Pol’y Blog (Oct. 21, 2019), https://perma.cc/7KEZ-H29J.
• 7Isabelle Icso, USTR Nominee on the WTO: U.S. ‘Can’t Afford’ Continued Disengagement, Inside U.S. Trade (Feb. 26, 2021), https://perma.cc/3QLV-P2HD; Hannah Monicken, EU Moves Closer to U.S. on Appellate Body, Other WTO Reforms In New Strategy, Inside U.S. Trade (Feb. 18, 2021), https://perma.cc/U4L7-B7UB.
• 8Douglas Guilfoyle, Reforming the International Criminal Court: Is It Time for the Assembly of State Parties to Be the Adults in the Room?, EJIL: Talk! (May 8, 2019), https://perma.cc/CYS3-4RGQ.
• 9Daniel Abebe, Adam Chilton & Tom Ginsburg, The Social Science Approach to International Law, 22 Chi. J. Int’l L. 1, 5 (2021).
• 10Gregory Shaffer & Tom Ginsburg, The Empirical Turn in International Legal Scholarship, 106 Am. J. Int’l L. 1, 1 (2012).
• 11See id. at 2–3, 12.
• 12Abebe et al., supra note 9, at 12–13.
• 13Anthea Roberts, Incremental, Systematic, and Paradigmatic Reform of Investor-State Arbitration, 112 Am. J. Int’l L. 410, 414–15 (2018).
• 14Daniel Behn, Malcolm Langford & Laura Letourneau-Tremblay, Empirical Perspectives on Investment Arbitration: What Do We Know? Does It Matter?, 21 J. World Inv. & Trade 188, 190 (2020).
• 15Working Group III: Investor-State Dispute Settlement Reform, U.N. Comm’n on Int’l Trade L., https://perma.cc/DKT7-A97T.
• 16See id.
• 17Sergio Puig & Gregory Shaffer, Imperfect Alternatives: Institutional Choice and the Reform of Investment Law, 112 Am. J. Int’l L. 361, 361 (2018).
• 18Abebe et al., supra note 9, at 4.
• 19See Hebert Smith, Effects of Causes and Causes of Effects, Some Remarks from the Sociological Side, 43 Socio. Methods & Rsch. 406, 409 (2014).
• 20See Abebe et al., supra note 9, at 15–17.
• 21Shaffer & Ginsburg, supra note 10, at 1.
• 22See generally Susan D. Franck, Arbitration Costs: Myths and Realities in Investment Treaty Arbitration (2019).
• 23Secretariat, U.N. Comm’n on Int’l Trade L., Possible Reform of Investor-State Dispute Settlement (ISDS) — Cost and Duration: Note by the Secretariat, ¶¶ 48, 56, U.N. Doc. A/CN.9/WG.III/WP.153 (Aug. 31, 2018).
• 24Franck, supra note 22, at 181–84.
• 25See, e.g., José Manuel Álvarez Zárate et al., Duration of Investor-State Dispute Settlement Proceedings, 21 J. World Inv. & Trade 300, 303–04 (2020).
• 26For an exception, see Behn et al., supra note 14, at 18–21.
• 27Weijia Rao, Domestic Politics and Settlement in Investor-State Arbitration, J. Legal Studies (forthcoming).
• 28Investment Dispute Settlement Navigator, Inv. Pol’y Hub, https://perma.cc/L989-QYAN (last updated Jul. 31, 2020).
• 29See, e.g., Bette Hileman, Canada Capitulates on MMT, Settles with Ethyl, Chem. & Eng’g News (Jul. 27, 1998), https://pubs.acs.org/doi/pdf/10.1021/cen-v076n030.p013a; Uchenna Awom & Patience Akpuru, Nigeria: Malabu Deal Latest—Shell’s Dirty Lies, AllAfrica (May 24, 2012), https://perma.cc/D7DH-HN4U; Seraphina Chew, Lucy Reed & J Christopher Thomas, Report: Survey on Obstacles to Settlement of Investor-State Disputes 1 (NUS Ctr. for Int’l L. Working Paper, Paper No. 2018/022, 2018), https://perma.cc/8V28-SEUS.
• 30Chew et al., supra note 29, at 12.
• 31Secretariat, U.N. Comm’n on Int’l Trade L., supra note 23, ¶ 101.
• 32Addressing concerns about time and costs through procedural reforms has also been the focus of ICSID’s ongoing rules amendment project. See Meg Kinnear, Continuity and Change in the ICSID System: Challenges and Opportunities in the Search for Consensus, https://perma.cc/98ZQ-3CPA.
• 33See U.N. Conf. on Trade & Dev., Investor-State Disputes: Prevention and Alternatives to Arbitration, at 86, U.N. Doc. UNCTAD/DIAE/IA/2009/11, U.N. Sales No. E.10.II.D.11 (2010) https://perma.cc/66C3-NCES.
• 34Ricardo Ampuero Llerena, Peru’s State Coordination and Response System for International Investment Disputes, Inv. Treaty News (Jan. 14, 2013), https://perma.cc/WK9T-56WR.
• 35See id.
• 36Id.; see also U.N. Conf. on Trade & Dev., Best Practices in Investment for Development: How to Prevent and Manage Investor-State Disputes: Lessons from Peru, at 35 (2011) https://perma.cc/5H2H-7JGG.
• 37See Llerena, supra note 34. 
• 38See Nancy A. Welsh & Andrea Kupfer Schneider, The Thoughtful Integration of Mediation into Bilateral Investment Treaty Arbitration, 18 Harv. Negot. L. Rev. 71, 118 (2013).
• 39ICSID, Proposals for Amendment of the ICSID Rules Working Paper 4 (2020), 193–204, https://perma.cc/9NXJ-DLNS.   
• 40See Yoram Z. Haftel & Alexander Thompson, When Do States Renegotiate Investment Agreements? The Impact of Arbitration, 13 Rev. Int’l Org. 25, 33 (2018).
• 41See id.
• 42See Roberts, supra note 13, at 411.
• 43See Weijia Rao, Are Arbitrators Biased in ICSID Arbitration? A Dynamic Perspective, 66 Int’l Rev. L. & Econ. (forthcoming 2021).