Experimentally Testing the Effectiveness of Human Rights Treaties

International human rights law is a field concerned with causality. While scholars in other fields argue about how laws can be changed to maximize their effectiveness, scholars of international human rights law still regularly debate whether major international agreements have had any effect on state behavior. Some scholars have examined empirical records and concluded that under the right conditions, the ratification of human rights treaties is associated with improved human rights practices,11

but others have found that the ratification of human rights treaties does not have any influence on the likelihood that states will violate human rights.22

One thing that scholars agree on, however, is that if international human rights treaties were to change state behavior, it would not be because of the usual mechanism that drives compliance with international law: reciprocity.33

This is because one state’s failure to respect the rights of its citizens cannot be corrected by another state reciprocally violating the rights of its own citizens. As a result, scholars have examined other mechanisms—including domestic politics, empathetic enforcement by powerful states, pressure from international treaty bodies, and the influence of non-governmental organizations—that may lead ratification of human rights agreements to result in changes to state behavior. After over a decade of empirical research on the effectiveness of human rights treaties, however, there is disagreement on whether each of these alternative mechanisms leads to improved rights practices.

This lack of consensus is in part due to the substantial obstacles that stand in the way of using observational data to study the causal effect of human rights treaties on state practices.44

Perhaps the most difficult of these obstacles to overcome is the fact that states are not randomly assigned commitments to human rights treaties, but instead select agreements based on their expected behavior. There is near universal ratification of the major human rights agreements, and most states have also subjected themselves to a series of overlapping international and domestic legal obligations that all seek to protect human rights. Furthermore, convincing evidence has recently emerged showing that the data used by researchers to measure human rights practices is systematically biased because of changes in standards used for reporting human rights abuses over time.55 It has proven difficult enough to design an observational study that accounts for one of these problems, but designing a study that accounts for all of them simultaneously has so far proven to be close to impossible.

Given these obstacles to using observational data, and the importance of the topic, scholars have begun to use experimental methods to study the effects of commitments to human rights agreements. This is because experimental studies can be designed to test the plausibility of the mechanisms theorized as ways that the ratification of human rights treaties may change state behavior. Although experimental methods are unlikely to definitively establish whether international treaties improve human rights outcomes, the handful of experimental studies conducted to date have found that information on commitments to international agreements does have a modest impact on public opinion.66

By doing so, these studies have provided some qualified support for the theorized mechanisms through which treaties may improve human rights.

There are, however, several reasons why these experiments have extremely limited generalizability. This is not only because of the general problem that subjects often behave differently in experimental settings than they do in the real world, but also because the human rights experiments that have been conducted to date have largely relied on surveys administered to samples of Americans. As a result, although the evidence produced by these studies is informative, it does not offer any conclusive answers on whether human rights treaties change state behavior.

That said, these early studies have demonstrated that using experimental methods to study human rights may help bring new evidence to this important debate. Future work should build upon this foundation by expanding: the countries in which experiments are conducted; the topics that are studied; and the type of experiments being conducted. By building on existing studies in this way, experimental research on human rights may be able to provide crucial evidence that supplements what has been produced using observational research methods.

This Essay proceeds as follows. Section II discusses mechanisms that may lead states to improve their human rights practices as a result of signing human rights agreements. Section III summarizes the evidence from observational studies on whether those mechanisms have led states to improve their human rights practices. Section IV outlines why observational studies face difficulties trying to empirically test the effects of human rights treaties. Section V presents experimental studies that have tried to examine the effect of human rights agreements. Section VI discusses future directions that experimental research on human rights should take.

A central question in the study of international law is whether states change their behavior because of commitments to international agreements.77

This question has taken on unique importance in the human rights context because the mechanism that drives compliance in many areas of international law—reciprocity—is not present.88 In most areas of international law, the logical response to State A breaking a commitment is for State B to apply pressure on State A to change its behavior by reciprocating.99 For example, if State A were to violate an investment agreement by blocking firms from State B from acquiring their domestic companies, State B can respond by doing the same thing to firms from State A.1010 In the human rights context, however, the same logic does not hold.1111 Unlike investment agreements, if State A were to torture its citizens in violation of an international human rights treaty, State B will not try to convince State A to change course by torturing its own citizens. The implication is that although it is possible that international human rights treaties have helped to improve human rights practices, it is not because of the possibility of reciprocity. Instead, for commitments to human rights treaties to produce changes in state behavior, it would have to be through mechanisms other than reciprocity.

Figure 1 illustrates some of the mechanisms through which commitments to international human rights treaties could result in changes to state behavior. The solid lines represent the commitments that states make to human rights treaties.1212

The dashed lines represent four mechanisms that might create pressure for states to change their human rights practices after having signed a human rights treaty: (1) ratification changes the domestic politics of countries that are at least partially democratic; (2) other countries coercing states to comply with their commitments; (3) treaty bodies pressuring the state to improve its human rights practices; and (4) non-governmental organizations (NGOs) advocating for compliance.

Figure 1: Mechanisms Through Which Human Rights Treaties May Change State Behavior

[[{"fid":"511","view_mode":"full","fields":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":false,"field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""},"type":"media","field_deltas":{"1":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":false,"field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""}},"attributes":{"class":"media-element file-full","data-delta":"1"}}]]

The primary mechanism that has been theorized as driving compliance with human rights treaties—illustrated by the dashed loop below State A—is that ratification of human rights treaties may change the domestic political landscape of partially democratic countries in ways that make it easier for activists and opposition political parties to lobby for improved rights practices.1313

For all but the most powerful states, new human rights treaties create exogenous shocks to countries’ domestic agendas that force governments to decide whether to sign the treaty and make a public commitment to human rights. This shock may create an opening for domestic groups to lobby for improved rights practices, and if the agreement is signed, it may improve the ability of domestic groups to pressure the government to live up to the commitments it made. This pressure may come in many forms. For example, it may be human rights activists bringing litigation to try and have the courts pressure the government to bring human rights practices into compliance with the treaty, or it may be opposition political parties attacking the government for failure to live up to prior international commitments.

A second mechanism that may drive compliance with human rights treaties—illustrated by the dashed lines between countries—is “empathetic enforcement” by powerful states.1414

Although states are not obligated to apply strong pressure on repressive states to comply with human rights agreements, they may choose to do so. This pressure could come in many forms. For example, developed countries might condition foreign aid upon recipient states improving human rights practices, or be more likely to militarily intervene in countries that violate human rights commitments they have made. If powerful states were more likely to apply these forms of pressure against repressive states that had previously committed to human rights treaties, this would be one way that treaty commitment could lead to improved rights practices.

A third mechanism that may drive compliance with human rights agreements—illustrated by the dashed line from the treaty to the states—is pressure applied by the international treaty bodies themselves.1515

By voluntarily signing onto international human rights agreements, states subject themselves to monitoring by treaty bodies. The treaty bodies then have the power to produce reports that recommend ways for the states to improve human rights, or to issue statements chastising states for failure to live up to their commitments. Although the international treaty bodies cannot back up these statements with explicit sanctions, the reports may indirectly influence human rights practices.

A fourth mechanism that may lead states to improve human rights practices after ratifying international agreements—illustrated by the dashed line from the NGOs to State A—is the influence of non-governmental organizations.1616

There are a large number of NGOs dedicated to trying to improve human rights practices around the world. This includes organizations that cover a broad range of issues like Amnesty International and Human Rights Watch, as well as organizations focused on specific issue areas. These organizations use a range of tactics to pressure governments to improve their rights practices. These tactics include filing reports with treaty bodies, lobbying powerful governments to pressure repressive states, and naming and shaming states that violate human rights.

Despite years of scholarship researching the four mechanisms discussed in the previous Section, it is still disputed whether any of the four are viable channels through which treaty commitments may result in improved human rights practices. In this Section, I will briefly discuss some of the empirical research that has used observational data to test whether these four mechanisms have caused states to change their behavior.1717

First, empirical research has examined whether domestic political mechanisms may lead states to improve their human rights practices after ratifying international treaties. In the most prominent research testing this theory, Beth Simmons tested the effectiveness of thirteen human rights treaties.1818

Simmons found evidence that ratification was associated with improved human rights practices for seven of the thirteen agreements. Simmons’ research did not, however, suggest that the effectiveness of ratification was consistent for all countries. Instead, Simmons found that ratification of human rights treaties is not associated with improved human rights in stable democracies or stable autocracies, but is associated with better rights practices in transitioning democracies. According to Simmons, the reason that transitioning democracies change their behavior after ratifying human rights treaties is that the existence of the agreement helps domestic groups pressure their governments to be more respectful of rights. In addition to Simmons’ research, two recent empirical studies by political scientist Yonathan Lupu have used sophisticated empirical techniques to test whether domestic mechanisms increase the effectiveness of human rights treaties. Lupu’s studies have found that compliance with the International Covenant on Civil and Political Rights (ICCPR) has increased when there are more veto points in the state’s legislature,1919 or when there is an independent judiciary that can evaluate claims that certain rights are being violated.2020 Although Simmons’ and Lupu’s research has produced perhaps the most promising evidence that committing to human rights treaties may drive states to change their behavior, it is worth noting that it has been suggested that these studies at best show that ratification has made a marginal improvement in rights practices for a small number of countries on a handful of issues.2121

Second, empirical research has tested whether empathetic enforcement might cause states to improve their human rights behavior. One body of research testing this theory has examined whether aid donors have punished states that violate human rights by withholding aid.2222

Although that line of scholarship has produced mixed results, in the most recent research on the topic, political scientist Richard Nielsen found that states only selectively sanction human rights violators. That is, although aid is sometimes withheld from states that violate human rights, “[d]onors seem willing to abandon human rights norms when their friends and allies violate human rights.”2323 Based on this finding and other similar research, scholars have largely been skeptical that states would actually improve their human rights practices because of rewards or punishments from more powerful countries.2424

Third, there has been limited research trying to test whether pressure from human rights treaty bodies leads states to improve their human rights practices. Since human rights bodies do not have any strong enforcement powers and countries do not uniformly take their obligations to report to human rights bodies seriously,2525

it is perhaps unsurprising that few scholars have tried to link activities of human rights bodies to improved rights practices. In one study on the topic, James Lebovic and Erik Voeten tested whether resolutions condemning a state’s human rights practices passed by the U.N. Commissions on Human Rights (UNCHR) resulted in that state receiving lower levels of foreign aid.2626 Their study found that resolutions condemning human rights practices are associated with states receiving lower amounts of aid from multilateral organizations like the World Bank, but that resolutions condemning human rights practices are not associated with aggregate bilateral aid. Even if UNCHR resolutions result in lower foreign aid, however, it does not necessarily follow that the resolutions cause improvements in human rights. As a result, despite this finding, Beth Simmons has argued that “[n]o one has shown any significant external enforcement behind the provisions of international human rights treaties of a kind that might plausibly account for the patterns of compliance observed across a number of rights areas.”2727

Fourth, a handful of empirical studies have tried to test whether pressure from NGOs has led to improved human rights practices. This research has focused on whether efforts to “name and shame” countries for violation of human rights has led to better practices. In what is perhaps the most widely cited study on the subject, Emilie Hafner-Burton found that efforts to spotlight human rights violations by NGOs are associated with increased repression.2828

Hafner-Burton explains this finding by arguing that oppressive governments have incentive to crack down on opposition groups after they are empowered by the attention created by international organizations. In contrast to Hafner-Burton’s finding, Anna Schrimpf found that there is likely no relationship (but perhaps a positive relationship) between NGO attention and human rights practices.2929 Finally, in related research, Judith Kelley and Beth Simmons found that including states on “watch lists” is associated with those states being likely to then criminalize human trafficking.3030

As the preceding discussion hopefully made clear, a consensus among empirical researchers has not emerged on the question of whether ratification of international human rights agreements improves state behavior. Although this is likely in part because empirical scholarship on international law is relatively new, it is also because there are a number of factors that make it incredibly difficult to study the causal effects of human rights treaties using observational data.3131

First, observational research on human rights treaties has difficulty accounting for selection bias. Selection bias exists when the assignment of a treatment variable is systematically related to the dependent variable of interest. For example, if the subjects given Drug X in a clinical trial were picked to receive the drug because they were the healthiest subjects in the group, any positive effect of the drug may be attributable to the fact that the receipt of the Drug X (the treatment) was related to likely health outcomes (the dependent variable). The same problem exists in studying human rights treaties. Countries do not enter into treaties randomly. Instead, countries voluntarily choose to enter into human rights agreements based on their preferences and planned policies. To be clear, it might be the case that countries decide to enter into agreements when they do not plan on complying, or that they enter into the agreements when they do. Either is problematic for empirical research. Although empirical researchers have developed strategies to try and account for this selection bias,3232

ultimately there is only so much that can be done given the fact that countries voluntarily opt into human rights regimes and thus any later human rights practices are endogenous to the decision to ratify the treaties.

Figure 2: Number of Countries Party to Major Human Rights Treaties

[[{"fid":"516","view_mode":"full","fields":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":false,"field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""},"type":"media","field_deltas":{"2":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":false,"field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""}},"attributes":{"class":"media-element file-full","data-delta":"2"}}]]

Second, even if it were possible to account for selection bias, a basic requirement of causal inference is variance. Simply put, there must be variance in the explanatory variable of interest to be able to assess the impact that it has on a given dependent variable. For example, if everyone that contracted a new disease was given Drug X, it would be impossible to know the effectiveness of Drug X (which is exactly why clinical trials randomly give some subjects placebos). This basic requirement of causal inference is a major hurdle for studying human rights treaties because there is very little variance in applicability of the major international human rights agreements (and no variance in the applicability of the parts of human rights law that are considered part of customary international law). As Figure 2 shows, nearly every country has signed onto the major human rights treaties. To further illustrate this point, the red line is the number of countries that are currently members of the U.N. Since there are very few countries that are not party to the major human rights agreements, and countries that are not party to the agreement are likely not representative of the countries where human rights agreements are likely to have an effect, it is thus very difficult to understand the effectiveness of these agreements on human rights practices.

Figure 3: Ratification History of Major Human Rights Agreements

[[{"fid":"521","view_mode":"full","fields":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":false,"field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""},"type":"media","field_deltas":{"3":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":false,"field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""}},"attributes":{"class":"media-element file-full","data-delta":"3"}}]]

Third, the current lack of variance in ratification of major human rights treaties would not be as much of a problem if historically there had been variance in which countries were party to the treaties. If that were the case, it would be possible to study the period between when the treaty had opened for ratification and when nearly all countries joined the agreements. For the first major human rights treaties, this was the case. As Figure 3 shows, countries continued to ratify the Convention on Elimination of Racial Discrimination (CERD), the International Covenant on Civil and Political Rights (ICCPR), and the International Covenant on Economic, Social, and Cultural Rights (ICESCR) for the entire latter half of the twentieth century.3333

As Figure 3 also shows, however, more recent human rights treaties have been ratified incredibly quickly. For example, the most recent major human rights treaty—the Convention on the Rights of the Child (CRC)—became widely adopted in about five years.3434 Although these increasingly short ratification windows may be a positive development for the human rights movement, they are problematic for human rights researchers who need longer windows before widespread ratification to study.

Figure 4: Countries that Have Made Commitments to Eliminate Torture

[[{"fid":"526","view_mode":"full","fields":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":false,"field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""},"type":"media","field_deltas":{"4":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":false,"field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""}},"attributes":{"class":"media-element file-full","data-delta":"4"}}]]

Fourth, another difficulty with conducting empirical research on the effectiveness of human rights treaties is that countries frequently have overlapping legal obligations. The problem is that it is only possible to understand the effect of a given variable if it is possible to isolate it from other variables that may be driving the outcome of interest. For example, if everyone that contracted a new disease were given both Drug X and Drug Y, it would be impossible to isolate the effects of Drug X and the effects of Drug Y (not to mention the effect of their interaction). The same problem often exists with international legal commitments. Countries often have signed multiple agreements that regulate human rights practices, have joined multiple organizations that monitor human rights compliance, and also have domestic laws that prohibit the same conduct as the international agreement. To illustrate this point, Figure 4 shows the number of countries that have signed the Convention Against Torture (CAT) and the number of countries that have provisions in their constitutions that prohibit torture.3535

As the figure shows, there is nearly the same number of countries with each restriction as of 2015. These kinds of overlaps can make it difficult to tell whether changes (for better or worse) in human rights practices are attributable to international treaties or other sources of law.

Fifth, beyond the previous four concerns related to the ratification of human rights treaties, it is also the case that there are frequently only flawed measures of human rights practices available to use as dependent variables. Human rights treaties seek to protect a large number of rights. In fact, as Figure 5 shows, the number of rights protected by individual treaties ranges from eleven to fifty-eight. Currently, however, there is only data available on a handful of rights for a large number of countries. The result is that it is difficult to know whether a given treaty has improved rights because there are not good measures for all of the rights that any given treaty tries to protect.

                         Figure 5: Number of Rights Protected in Major Human Rights Agreements

[[{"fid":"531","view_mode":"full","fields":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":false,"field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""},"type":"media","field_deltas":{"5":{"format":"full","field_file_image_alt_text[und][0][value]":false,"field_file_image_title_text[und][0][value]":false,"field_caption[und][0][value]":"","field_photo_credit[und][0][value]":""}},"attributes":{"class":"media-element file-full","data-delta":"5"}}]]

Moreover, even if there was unbiased data for one of these rights—say the right of freedom of speech—it might be the case that the treaty improves protection for that right while making things worse for another dozen rights. That said, it is likely the case that all of the current data sources are biased. Recent research has shown that the most widely used dataset for human rights—the CIRI dataset—is systematically biased because reporting standards have changed over time.3636

This not only calls into question the results of studies that have relied on the CIRI data,3737 but also suggests that any other research that uses data on rights practices based on reports may be biased as well. Since the data on human rights practices has been shown to be biased and only covers a few rights, taken together these facts should make it clear that it is difficult to use conventional methods of empirical research to study the effectiveness of human rights agreements.

One advantage of experimental research is that experiments can be designed to help test the mechanisms discussed in Section II that have been theorized as driving compliance with human rights agreements while accounting for the barriers to inference discussed in Section IV. Of course, my claim is not that experiments can address all of these problems at once.3838

Instead, they can bring new evidence to evaluate the effectiveness of human rights agreements. To date, experimental studies have been designed to test the plausibility of all of the four mechanisms that were previously outlined: (1) domestic politics; (2) empathetic enforcement; (3) treaty body pressure; and (4) NGO advocacy.

There have been at least two experiments conducted that tested whether prior commitments to international agreements increased support for policy reforms that would make a state comply with its human rights obligations. In the first experiment designed to test this mechanism, Wallace conducted a survey on American adults to determine whether information on prior commitments to international law decreased support for torture.3939

The survey told respondents that the U.S. often captures combatants from the opposing side during conflicts, and that those combatants may have information on future attacks. The survey further told respondents that some officials believe that using interrogation methods that include torture on the combatants may be a useful way to obtain information on future attacks. The survey then randomly told half of the sample that those interrogation methods would violate international law and treaties that the U.S. had previously committed to.4040 After reading this information, respondents were asked whether they supported the use of interrogation techniques that include torture. Wallace found that respondents given information on international law treatment were six percent less supportive of using torture than the respondents who did not receive the information. This survey suggests that even on an issue where people have strong views, information on international law can have a statistically significant effect on the views of Americans.

The second study designed to test the Domestic Politics mechanism is Chilton.4141

Chilton also conducted a survey on a sample of American adults to test the effect of information about international law, but made two changes to the design used by Wallace. First, the survey focused on a purely domestic topic to reduce the possibility that respondents changed their view in light of international law because they thought that retaliations may result from non-compliance. Second, the survey included a placebo treatment to see whether the legal commitments, instead of the general idea of human rights, is what lowered support for violations of human rights treaties. The survey specifically focused on the use of solitary confinement in American prisons. Respondents were told that solitary confinement is often used in American prisons, and that supporters of the practice argue that it is necessary to maintain prison discipline. There were then three randomly assigned treatments: the control group was not given any information on human rights; the placebo group was told that critics argue that the use of solitary confinement “violates the human rights of the prisoners held in solitary confinement;” and the international law group was told that critics argue that the use of solitary confinement “violates international human rights treaties that the United States has signed.” The idea is that the placebo group invoked the idea of human rights generally without relying on a specific legal commitment, thus making it possible to determine whether the existence of specific treaties—as opposed to general norms—is what influences opinion. The results of the survey suggested that there was no difference between the control and placebo groups in their level of support for using solitary confinement, but that there was a small but statistically significant difference with the international law treatment. Respondents that received international law treatment were roughly four percent more likely to support reforming the use of solitary confinement.

Several experiments have tested whether information on international law makes respondents more likely to support efforts to change the policies of states that violate human rights. For example, Putnam and Shapiro (2013) tested respondents’ willingness to support economic sanctions against a foreign country because of its human rights violations.4242

In a survey of American adults, Putnam and Shapiro told respondents about the use of forced labor in Myanmar. After being given varying amounts of information on the severity of the human rights violations in Myanmar, respondents were also given different information about the status of international law. The experiment suggested that information on international law increased support for government sanctions, but did not increase respondents’ willingness to say that they would personally boycott goods from Myanmar. Although the complete findings of the experiment were mixed, the authors did find qualified support for the idea that respondents learned that human rights violations were against international law.4343

In another experiment that tests support for empathetic enforcement, Tomz and Weeks examined factors that influence support for military intervention.4444

In the survey, respondents were told about a hypothetical country developing nuclear weapons. Respondents were randomly told about the country’s regime type—that is, whether or not it was democratic—as well as about the country’s human rights record. Half of the respondents were told that “[t]he country does not violate human rights; it does not imprison or torture its citizens because of their beliefs,” while the other half were told that “[t]he country violates human rights; it imprisons or tortures some of its citizens because of their beliefs.” The respondents were then asked about their willingness to support military strikes against the country. The results suggest that respondents were dramatically more likely to support military strikes when the country violated human rights. Specifically, the experiment revealed a twenty-three percent increase in support for military strikes against democracies that violate human rights, and a sixteen percent increase in support military strikes against autocracies that violate human rights.

At least one experiment has tested whether treaty body pressure increases support for reforms aimed at improving human rights. Anjum, Chilton, and Usman conducted an experiment in Pakistan that tested whether learning that policy reforms aimed at improving women’s rights were proposed by the U.N. increases support for the reforms.4545

In the experiment, roughly six hundred subjects were recruited in Pakistan to take an in-person survey on women’s rights. The respondents were asked whether they supported four reforms that the U.N. Committee on the Elimination of Discrimination Against Women recommended that Pakistan should implement. Half of the subjects were told that the reforms were proposed by the U.N., while half of the subjects were not told who proposed the reforms. The results of the experiment revealed that learning that the U.N. supported the reforms increased support for them. The results of the experiment thus provide some evidence that when the U.N. supports a policy reform, it might increase its domestic support.

A recent experiment has also tested how international human rights law may help with NGO advocacy. Hafner-Burton, LeVeck, and Victor conducted an experiment with experienced NGO professionals that had experience working on environmental issues and human rights.4646

For the human rights treatments, the experiment asked the sample of NGO professions how much more difficult they believed it would be to accomplish their advocacy goals if either the ICCPR (which is a binding agreement) or the Universal Deceleration of Human Rights (which is a non-binding agreement) did not exist. The results suggested that they believed that their job would be more difficult if the laws did not exist, and that it would be even more difficult if the binding ICCPR did not exist. Moreover, the respondents indicated that the reason that their jobs would be particularly difficult is that removal of the law would make litigation and accountability efforts more difficult. These results thus provide evidence that NGO advocates at least believe that the presence of international human rights law helps them to accomplish their goals.4747

Although the experimental research on human rights has suggested that information on human rights agreements may be able to change public opinion, there are serious limitations to the studies that have been conducted to date. Most notably, the fact that nearly all of the experiments on human rights have been conducted using vignette studies with respondents in the U.S. seriously limits the generalizability of the findings.4848

This method of research is still in its infancy, however, and could learn a great deal from the more developed experimental literatures in development economics, American politics, and international relations. I will briefly outline three ways that future experimental studies on human rights could improve on prior research.

First, experiments should be conducted in transitioning democracies. There is little evidence to suggest that human rights treaties have improved the rights practices of stable democracies or stable autocracies. Instead, the best available evidence suggests that human rights agreements are most likely to change the behavior of developing democracies.4949

That is, countries like those in Eastern Europe or Central America that are not well-established, stable democracies, but that have moved towards democracy in the last few decades. To date, however, nearly all of the experiments on the effectiveness of human rights have been conducted in the U.S.5050 This is a major shortcoming of the small experimental literature on human rights because the U.S. is not only a stable democracy, but also because Americans may have idiosyncratic views towards international human rights agreements. There have been a large number of experiments conducted by political scientists in developing countries in the last several years,5151 and there is a great deal that scholars interested in international human rights law could learn from these efforts.

Second, experimental research should explore a wider range of topics. For better or worse,5252

there are dozens of rights that are protected by international human rights agreements.5353 One of the advantages of experimental research is that a study can be designed to test whether international commitments can increase support for providing any right that is included within a treaty. Since observational studies can only test the effectiveness of rights where cross-country data is available, this is one comparative advantage of experimental research. This advantage should be leveraged, and studies should be conducted to test whether the impact that information has on international law is conditional on the nature of the right at issue.

Third, scholars conducting experiments on human rights should start to utilize field experiments. Although the survey experiments that have been conducted provide new data on the influence of human rights agreements on public opinion, it is not clear whether those differences in opinion translate into meaningful changes in the respondents’ behavior, and in turn, into changes in public policies. Researchers in a number of fields related to international law—including international economic regulation, international relations, and development economics—have begun to conduct field experiments to determine whether experimental interventions result in changes to real world behavior.5454

These experiments are thus able to reduce the concern that using experimental methods will produce internally valid results, but that they will not have any real-world applicability.

Of course, even if future experimental research on the effectiveness of human rights law moves in the direction that I have outlined, it does not mean that any one study will be able to definitely demonstrate whether human rights treaties have improved human rights around the world. Instead, experimental methods can produce evidence that complements empirical research using observational data. This would make it possible to triangulate on the answer to whether human rights treaties improve respect for rights.