Who said or what said? Estimating ideological bias in views among economists

Abstract

There exists a long-standing debate about the influence of ideology in economics. Surprisingly, however, there are very few studies that provide systematic empirical evidence on this critical issue. Using an online randomised controlled experiment involving 2,425 economists in 19 countries, we examine the effect of ideological bias among economists. Participants were asked to evaluate statements from prominent economists on different topics, while source attribution for each statement was randomised without participants’ knowledge. For each statement, participants either received a mainstream source, an ideologically different less-/non-mainstream source, or no source. We find that changing source attributions from mainstream to less-/non-mainstream, or removing them, significantly reduces economists’ reported agreement with statements. This contradicts the image economists have/report of themselves, with 82% of participants reporting that in evaluating a statement one should only pay attention to its content. Our analysis provides clear evidence for the existence of ideological bias as well as of authority bias among economists. We also find significant heterogeneity in our results by gender, country, PhD completion country, research area and undergraduate major, with patterns consistent with the existence of ideological bias.

1. Introduction

In her 1987 American Economic Association presidential address, the influential American economist, Alice Rivlin, suggested to her fellow economists that one of the critical steps towards improving the interaction between what economists do and political and policy processes is for economists ‘to be more careful to sort out, for ourselves and others, what we really know from our ideological biases’ (Rivlin, 1987).

Rivlin’s plea to discipline is perhaps more important today than at any other time in our recent past. Today’s polarised political and social spheres are increasingly shaped and dominated by ideological forces that often paint starkly different realities about critical issues we face. The future of human societies, human autonomy and our modern democracies will be determined by the ideological battles on critical issues such as climate change, rampant inequality, corporate oligarchy, the future of work, the rise of surveillance capitalism, and the loss of human privacy and autonomy.

At the same time, economists and their views, beliefs and ideologies play a major role in how we characterise and understand these challenges, and how we formulate solutions to them. It is well documented that economists occupy a unique position of power among social scientists, and have widespread and significant influences on public discourses (e.g. Appelbaum, 2019; Fourcade et al., 2015; Hirschman and Berman, 2014; Wright, 2019). This occurs through various channels, including their broad professional authority, their unique institutional position and presence in policy-making organisations and elite networks, and their role in shaping the cognitive infrastructure of policy-making, which involves using the dominant mainstream discourse and its language and styles of reasoning to establish economic policy devices that produce knowledge and help make decisions (Hirschman and Berman, 2014).

There seems to exist a perception among economists, however, that this exclusive position and widespread influence is driven by the ‘uniquely scientific’ and ‘superior’ nature of the discipline (Fine and Milonakis, 2009; Fourcade et al., 2015; Wright, 2019). For example, Edward Lazear, a prominent American economist who served as Chairman of the Council of Economic Advisors from 2006 to 2009, celebrates ‘Economic Imperialism’ by boasting that ‘[b]y almost any market test, economics is the premier social science’ and that ‘[e]conomics is not only a social science, it is a genuine science. Like the physical sciences …’ (Lazear, 2000, p. 99). Similarly, Richard Freeman, another prominent economist, argues that ‘sociologists and political scientists have less powerful analytical tools and know less than we do’ (Freeman, 1999, p. 141). Not surprisingly, in a survey of economics graduate students in elite programmes in the USA, Colander (2005) finds that 77% agree with the statement that ‘economics is the most scientific of the social sciences’.

This perception regarding the ‘superiority’ of the discipline is deeply rooted in a dominant view in the mainstream (Neoclassical) economics that emphasises the positivist view of science and characterises economists as dispassionate, objective, unbiased and non-ideological. Milton Friedman describes in his influential 1953 essay that ‘positive economics is, or can be, an “objective” science, in precisely the same sense as any of the physical sciences.’ (Friedman, 1953, p. 4). Similarly, Armen Alchian asserts that ‘[i]n economics, we have a positive science, one completely devoid of ethics or normative propositions or implications. It is as amoral and non-ethical as mathematics, chemistry, or physics’ (see Freedman, 2016, p. 39). While fewer economists are seen today waving the flag of positive economics, as Boland (1991, p. 88) suggests, this is mainly because ‘there is no [more] territory to dispute and thus no need to wave one’s flag’.

There exists, however, a long-standing debate about the influence of ideology in economics, which some argue has resulted in rigidity in the discipline, rejection and isolation of alternative views, and narrow pedagogy in economic training (e.g. Backhouse, 2010; Chang, 2014; Colander, 2005; Dobb, 1973; Fine and Milonakis, 2009; Fullbrook, 2008; Galbraith, 1989; Krugman, 2009; Morgan, 2015; Romer, 2015; Rubinstein, 2006; Stiglitz, 2002; Thompson, 1997). Unfortunately, despite its critical implications, this issue has been largely ignored/suppressed within mainstream economics for the last few decades. Moreover, surprisingly for a discipline that emphasises the importance of hard evidence, there is no empirical evidence for the claim of objectivity and ideological neutrality.

We believe that having a better understanding of the role of ideology in economics has several important implications. First, it will help us investigate the extent to which the theoretical arguments behind the positivist methodology of neoclassical economics are consistent with empirical evidence. Second, our study will provide a general context for the growing evidence that suggests value judgments and political orientation of economists affect various aspects of their academic life, including research (Jelveh et al., 2018; Saint-Paul, 2018), citation network (Önder and Terviö, 2015), faculty hiring (Terviö, 2011), and their views on both public policy and economic methodology (e.g. Beyer and Pühringer, 2019; Fuchs et al., 1998; Mayer, 2001; van Dalen, 2019). Third, we believe that our analysis can contribute to the advancement of critical knowledge by inviting a debate about the central views, biases, structures and practices in the profession.

In order to examine ideological biases among economists, we use an online randomised controlled experiment involving 2,425 economists in 19 countries.¹ More specifically, we ask participants in our online survey to evaluate statements from prominent (mainly mainstream) economists on a wide range of topics (e.g. fairness, intellectual property, globalisation, economic methodology, women in economics). While all participants receive identical statements, source attribution for each statement is randomised without the participants’ knowledge. For each statement, participants randomly receive either a mainstream source (Control Group), a relatively less-/non-mainstream source (Treatment 1), or no source attribution at all (Treatment 2).

We then measure whether economists agree/disagree with identical statements to different degrees when statements are attributed to authors who are widely viewed to adhere to different views/ideologies or when no source attributions are provided. Implementing two different treatments allows us to distinguish between the influences of ideological bias and authority bias.

We find clear evidence that changing or removing source attributions significantly affects economists’ level of agreement with statements. More specifically, we find that changing source attributions from mainstream to less-/non-mainstream on average reduces the agreement level by 7.3% (or 22% of a standard deviation). These results hold for 12 out of 15 statements evaluated by participants, across a wide range of topics and ideological distances between (the genuine and the false) sources. Similarly, we find that removing mainstream source attributions on average reduces the agreement level by 11.3% (or 35% of a standard deviation).² These result holds for all 15 statements evaluated by participants.

We also find that these results stand in sharp contrast with the image economists have or project of themselves. More specifically, in an accompanying questionnaire at the end of the survey, a majority of participants (82%) report that a statement should be evaluated based on its content only, as opposed to its author (0.5%), or a combination of both (18%), in sharp contrast with how they actually evaluate statements. These findings, along with other evidence we provide in Section 5.2 are all consistent with the existence of strong ideological/authority bias among economists.

We also examine whether our results vary systematically by characteristics such as gender, country, area of research, country where PhD was completed and undergraduate major. We find that the estimated ideological bias among female economists is around 40% smaller than their male counterparts. We also find systematic and significant heterogeneity across other groups, with some economists exhibiting no ideological bias and some others showing very strong bias. In addition, the heterogeneity patterns found in our results remain consistent with the existence of ideological bias.

The remainder of the paper proceeds as follows. Section 2 provides a brief overview of the discussion about economics and ideology. Section 3 describes our experimental design. Section 4 discusses our data and empirical methodology. Section 5 presents and discusses our results. Section 6 concludes.

2. Economics and ideology: a brief overview

Our study is rooted in and motivated by a long-standing debate about the influence of ideology in economics. Milberg (1998, p. 243) elegantly summarises this debate by stating that ‘the history of economic thought can in fact be read as a series of efforts to distance knowledge claims from the taint of ideology, a continuing struggle to establish the field’s scientific merit’.

About a century ago, Irving Fisher, in his presidential address to the American Economic Association, raised his concern about ideological bias in economics by stating that, ‘academic economists, from their very open-mindedness, are apt to be carried off, unawares, by the bias of the community in which they live’ (Fisher, 1919, p. 10). Other prominent economists such as Joseph Schumpeter and George Stigler also made substantial contributions to this discussion over the next few decades (see Schumpeter, 1949; Stigler, 1959, 1960, 1965, for examples). However, the change in the nature of economic discourse, the increasing use of mathematics and statistics and the increasing dominance of the positivist methodology, represented by Friedman’s ‘Methodology of Positive Economics’, have gradually reduced—almost to nothing—the concern with ideological bias in economics, which has gradually given way to a consensus that ‘economics is, or can be, an objective science’.³

Critics, however, argue that the increasing reliance of economics on mathematics and statistics has not freed the discipline from ideological bias; it has simply made it easier to disguise it (e.g. Lawson, 2012; Myrdal, 1954). There also exists evidence that economics has not successfully rid itself of ideological bias. For example, Hodgson and Jiang (2007) argue that due to ideological bias in economics, the study of corruption has been mainly limited to the public sector, when there is abundant evidence of corruption in the private sector. Another example is in a 2006 interview with David Card by the Minneapolis Fed.⁴ Talking about his decision to stay away from the minimum wage literature after his earlier work on the topic, which according to the article ‘generated considerable controversy for its conclusion that raising the minimum wage would have a minor impact on employment’, he laments that one of the reasons was that ‘it cost me a lot of friends. People that I had known for many years, for instance, some of the ones I met at my first job at the University of Chicago, became very angry or disappointed. They thought that in publishing our work we were being traitors to the cause of economics as a whole’.

Other prominent manifestations of ideological bias in economics include: the so-called fresh-water/salt-water divide in macroeconomics (Gordon and Dahl, 2013) and its impact on citation networks (Önder and Terviö, 2015) as well as faculty hiring (Terviö, 2011); the conflicts between liberal/conservative camps on distribution-efficiency trade-off; the Borjas versus Card debate on immigration, and the ideologically charged debates over the controversial book by Thomas Piketty (2014) or over the article by Paul Romer (2015, p. 89) in which he argued that ‘mathiness lets academic politics masquerade as science’. Finally, recent results from the Professional Climate Survey conducted by the American Economic Association also highlight some of the challenges in the profession that could be driven by ideological bias. For example, 58% of economists feel that they are not included intellectually within the field of economics.⁵ In addition, 25% of economists report that they have been discriminated against or treated unfairly due to their research topics or political views.

There also exists a long-standing charge laid mainly by non-neoclassical economists regarding the prevalence of ideological bias among neoclassical economists (e.g. Backhouse, 2010; Fine and Milonakis, 2009; Fullbrook, 2008; Morgan, 2015; Thompson, 1997). For example, McCloskey (2017) argues that economics has ‘deliberately clad itself in a garb of positivism, even when scholars knew the critical importance of the historical, social, and political embeddedness of their interventions’.

There are also studies that point to the ideological biases in economics training. Based on a survey of graduate students in economics, Colander (2005) argues that graduate training in economics induces conservative political beliefs in students. Allgood et al. (2012, p. 248) also find evidence that ‘undergraduate coursework in economics is strongly associated with political party affiliation and with donations to candidates or parties’. Using laboratory experiments, other studies find that compared to various other disciplines, economics students are more likely to be selfish (Frank et al., 1993, 1996; Frey et al., 1993; Rubinstein, 2006), free-riding (Marwell and Ames, 1981), greedy (Wang et al., 2011) and corrupt (Frank and Schulze, 2000).

Frey et al. (1993, p. 271) attribute these patterns to the economic training which ‘neglects topics beyond Pareto efficiency […] even when trade-offs between efficiency and ethical values are obvious’. Frank et al. (1993, p. 159) highlight the exposure of students to the self-interest model in economics where ‘motives other than self-interest are peripheral to the main thrust of human endeavor, and we indulge them at our peril’. Rubinstein (2006, p. C1) argues that ‘students who come to us to “study economics” instead become experts in mathematical manipulation’ and that ‘their views on economic issues are influenced by the way we teach, perhaps without them even realising’. Stiglitz (2002) also argues that ‘[economics as taught] in America’s graduate schools … bears testimony to a triumph of ideology over science’.

There exists, however, little systematic empirical evidence for (or against) ideological biases among economists. One well-known study by Gordon and Dahl (2013) uses a series of questions from the IGM Economic Expert Panel to examine consensus among prominent economists. Their study, however, relies on an extremely small and non-representative sample which includes 51 economists from the top seven economics departments. They find strong evidence of professional consensus and ‘no evidence to support a conservative versus liberal divide’ (p. 635). However, van Gunten et al. (2016) re-analyse the same data using an alternative model of ideological alignment and find that despite consensus among economists on many issues, there exists a ‘consistent ideological dimension in economists’ beliefs’ (Van Gunten et al., 2016., p. 1046) and show that the ‘ideological distance between economists is related to partisan and departmental affiliations—as well as to the similarity of respondents’ informal social networks’ (van Gunten et al., 2016, p. 1029). They suggest that ‘one implication of our findings is that consumers of economic expertise must exercise healthy skepticism faced with the claim that professional opinion is free of political ideology’ (van Gunten et al., 2016, p. 1046).

Jelveh et al. (2018) use observed political behaviour (i.e. campaign contributions and petitions signings) of politically active US economists and the phrases from their academic articles to construct and validate a political ideology index for economists. They find that predicted ideology is ‘robustly correlated with field of specialization as well as various department characteristics’ (Jelveh et al., 2018, p. 3). They also document a ‘robust correlation between author ideology and magnitudes of reported policy relevant elasticities’ (Jelveh et al., 2018, p. 1). Re-analysing the data used by Gordon and Dahl (2013), they also find a significant correlation between the responses of the economists and their predicted political ideologies.

van Dalen (2019) uses an online survey of Dutch economists to examine the effect of the personal values of economists on their positive or normative economic views. He finds evidence that highlights a clear lack of consensus among economists as well as a significant influence of personal values on their economic views and judgments. Finally, Beyer and Pühringer (2019) use petitions signed by economists as an indicator for ideological preferences to analyse the social structure of the population of politically engaged economists. They find ‘a very strong ideological division among politically engaged economists in the USA, which mirrors the cleavage within the US political system’ (Beyer and Pühringer, 2019, p. 30).

In our study, we rely on a much larger and significantly more diverse sample of economists from all over the world. In addition, our use of a survey allows us to collect a rich set of background characteristics from our participants. As a result, we can conduct a much more systematic and detailed analysis compared to previous studies. Moreover, while most previous studies focus their analysis on a narrow definition of ideology (i.e. American-centric Democrat vs. Republican political orientation) we treat ideology as a more complex and multi-dimensional concept and examine a clearly defined manifestation of this complex notion. Finally, our use of a Randomised Controlled Trial allows us to go beyond correlations and examine the casual effect of ideological bias among economists.

3. Experimental design

It is well understood that examining issues such as the impact of bias, prejudice or discrimination on individual attitudes and decisions is very challenging. This is due to the complex and often unconscious nature of these types of attitudes and behaviour, and the fact that some socially relevant patterns of human behaviour can only be studied if individuals are caught off guard (Cooper, 1976; Hertwig and Ortmann, 2008; Weber and Cook, 1972; Weiss, 2001). For example, reliable empirical evidence of discrimination in the labour market has only been made available, thanks to a field experimentation literature that has relied on the use of deception using correspondence or audit studies, for example, through sending out fictitious resumes and applications (see Bertrand and Duflo, 2017; Riach and Rich, 2002, for a review; also see Currie et al., 2014, for another example of experimental audit studies with deception).

We take a similar approach, namely using a deception mechanism in our survey. Before proceeding, however, we should note that the use of deception both in Western scientific systems as well as Indigenous systems of knowledge has a long history as a method to unveil the inner tensions of human perspective, behaviour and relations, as well as to unveil hidden knowledge (Orr and Orr, 2022). The use of deception in experiments is common in many disciplines such as marketing, psychology, sociology and political science to name a few, and is permitted in virtually all research ethics applications if certain conditions are met by researchers.⁶ The situation, however, is different in economics, especially among experimental economists, with strong views expressed on whether/when deception can be used. As Cooper (2014, p. 113) suggests, however, ‘only an extremist would claim that experimenters (or economists in general) should never use deception’.

In essence, our study is not any different from the field experimentation studies with deception that seem to be well accepted in economics, evidenced by their publication in top economic journals as well as their citation counts.⁷ The main difference between our study and the correspondence studies is that our target population are economists as opposed to potential employers, or doctors in the case of Currie et al. (2014). One could even argue that our study lacks some of the potential disadvantages that could be associated with correspondence or audit studies. For example, flooding the market with fictitious resumes could negatively affect the chances of some real candidates for being reasonably evaluated by employers. Similarly, sending fake patients to hospitals (see Currie et al., 2014) clearly consumes public resources and affects real patients.

Nevertheless, there exist some concerns regarding the use of deception in experiments which could be categorised into ethical and methodological. Getting into discussions around the ethical permissibility of deception in experiments is beyond the scope of this paper since it would inevitably require us to get into discussions around ethics and morality. However, suffice it to say that the diversity of views and practices on this issue clearly suggests that there exists no clear consensus on this matter. Moreover, as Wilson (2014) points out, economists’ reluctance, or opposition, to the use of deception does not seem to be based on any deontic aversion to dishonesty, but rather due to their methodological concern regarding the potential effect deception might have on participants.

The main methodological concern with deception in experiments is that experience with deception, either firsthand or secondhand, might create suspicion or resentment among participants. It is hypothesised that this could produce two potentially negative consequences. First, it could affect participants’ future willingness to join other experiments or surveys. Second, for those who participate, their behaviour could be distorted by suspicion and second guessing, which would impair experimental control. It is also argued that these two mechanisms could impose negative externalities on experimenters, in general, over time by destroying the credibility of their work.

The empirical evidence regarding the potential methodological costs associated with the use of deception, however, remains inconclusive.⁸ For example, on the question of whether deception breeds resentment, there are studies such as Christensen (1988) or Kimmel (1998) which conclude that having been exposed to deception, participants do not perceive being harmed and do not become resentful. There is even evidence indicating that ‘participants in deception experiments report having enjoyed the experience more, having felt less bored, and having perceived more educational benefit from their participation than participants in nondeception experiments’ (Hertwig and Ortmann, 2008, p. 68). On the other hand, there also exist studies that provide less optimistic findings. Conducting a review of the literature, Hertwig and Ortmann (2008) conclude that ‘undoubtedly, the available empirical evidence does not allow us to finally settle the methodological debate on deception, and there is room for honest differences in evaluating the ultimate impact of deception’.

We would also like to suggest that considering the context of our experiment and the type of deception used, our experiment is likely to affect participants in positive ways. More specifically, if they learn that using altered sources with different views will induce ideological bias in responses, this could encourage economists to avoid judgment based on sources and pay more attention to the content of an argument or idea. This is especially important given the fact that in many cases exhibited individual bias could be unconscious or unintentional, and gaining knowledge about such attitudes and behaviour is potentially extremely valuable. We believe finding out about their own biases is certainly a positive outcome for economists who (at least claim to) strive to be objective and ideology free. This also seems to be consistent with the standard most participants in our study hold themselves to.⁹

Moving on to our experimental design, we employ a randomised controlled experiment embedded in an online survey. Participants are asked to evaluate 15 statements presented to them by choosing one of the following options: strongly agree, agree, neutral, disagree and strongly disagree. They are also asked to choose a confidence level on a scale from 1 to 5 for their selected answer. These statements are on a wide range of topics in economics and are mainly from prominent mainstream economists.

It is important to note that most of our statements are not clear-cut, one-dimensional, neutral statements such as those in the IGM Expert Panel analysed by Gordon and Dahl (2013), which according to van Gunten et al. (2016) includes ‘softball’ items that are not designed to elicit ideology. We believe that, given the complex nature of ideological bias, it is more likely to be revealed by individuals in situations where the issues discussed challenge one’s prior (ideological) views, or when discussions are around dense, complex and multi-dimensional issues, which make it easier for individuals to present their views while concealing their ideological underpinnings.

All participants in our survey receive identical statements in the same order. However, source attribution for each statement is randomised without participants’ knowledge. For each statement, participants randomly receive either a mainstream source (Control Group), or a relatively less-/non-mainstream source (Treatment 1), or no source attribution (Treatment 2). Section 1 in our Online Appendix provides a complete list of statements and sources.

Participants who are randomised into Treatment 2 for the first statement remain there for the entire survey. However, those who are randomised into control group or Treatment 1 are subsequently re-randomised into one of these two groups for each following statement. Moreover, those randomised into Treatment 2 were informed, before starting to evaluate the statements, that ‘All the statements that you are going to evaluate are made by scholars in economics, and do not necessarily reflect the views of the researchers. We have not provided the actual sources of these statements to make sure they are evaluated based on their content only’.

Three points are important to highlight here. First, our dichotomisation of the sources into ‘mainstream’ and ‘less-/non-mainstream’ is meant to simplify and summarise the relative ideological differences between sources, even though we believe these differences are more appropriately understood as a continuum rather than a dichotomy. Of course, it is well understood that this classification does not readily apply to some sources, such as older ones (e.g. Marx or Engels) or sources from other disciplines (e.g. Sandel or Freud) in the same way it applies to others. However, to remain consistent and avoid confusion for the reader, we stick to the same naming convention for all sources.

Second, statements were carefully selected so that their attribution to fictitious sources is believable by participants. All selected statements were also relatively obscure so the misattribution would not be easily noticed by participants. While we cannot rule out the possibility that some participants might have identified some of the misattributions, this seems to be a very rare incidence, at least based on the emails we received.¹⁰ Nevertheless, participants’ identification of misattributions would either make them stop the survey or continue to its completion. As we explain in more detail in our data section, the first group is not part of our analysis since we restrict our sample to only those who completed the entire survey. As for the second group, they will either consider the misattribution a mistake and will therefore evaluate the statement based on the real source, or will become suspicious that this is part of our survey design to examine bias. Both of these scenarios will lead to an underestimation of the true bias. In the first case, any potential bias the treatment could have induced is eliminated by the discovery of the true source, while in the second case the self-awareness about the purpose of the study will induce people to reveal less bias.

Third, the two sources for each statement were carefully paired such that they can be easily associated with commonly known but different views (such as different schools of thoughts, political leanings, disciplines, attitudes towards mainstream economics, etc.). In addition, for each source, we also provide information on their discipline, their affiliation and the title of one of their publications. This information is carefully provided to accentuate the ideological differences between each pair, especially in cases where sources might be less known. For example, while some economists might not know Richard Wolff or Anwar Shaikh, knowing that they are affiliated with the University of Massachusetts Amherst or the New School for Social Research, two famous heterodox schools in economics, makes it more likely to induce an ideological reaction. Similarly, titles of selected publication for each source, such as Rethinking Marxism, The Crisis of Vision in Modern Economic Thought or What Money Can’t Buy: The Moral Limits of Markets, serve the same purpose. We, therefore, assume that most of our participants, who are mainly academics with a PhD in economics, possess the minimal amount of knowledge required to identify the ideological differences between each pair of sources.

We also included a few pairs where these ideological differences are arguably less distinct, such as Rodrick versus Krugman or Deaton versus Piketty. This is to examine whether smaller ideological differences, especially between two economists who are both mainstream, can also induce ideological reactions. Finally, three of the less/non-mainstream sources selected are not economists (i.e. Freud, Sandel and Gigerenzer), although they are prominent scholars in their fields. These sources are carefully paired with equally prominent economists (Hayek vs. Freud, Sen vs. Sandel, Thaler vs. Gigerenzer) to empirically test the common view that ‘economists tend to look down on other social scientists, as those distant, less competent cousins’.¹¹ The statements used in these three cases were also carefully selected so that they do not favour expertise in economics and scholars from these other disciplines can equally weigh in on the discussion.

Although we believe our assumption regarding the ability of participants to identify the ideological differences between sources is quite reasonable, we nevertheless implement several empirical tests to provide further support for this assumption. Findings from these tests are discussed in detail in Section 2 in our Online Appendix and clearly suggest that participants in our sample are able to locate the ideological coordinates of the ‘mainstream’ versus ‘less-/non-mainstream’ sources, and react to them according to their own personal views and ideologies.

4. Data

The target population for this study were economists from 19 different countries.¹² We used Economics Departments, Institutes and Research Centres in the World (EDIRC) website, which is provided by the Research Division of the Federal Reserve Bank of St. Louis, to identify economic institutions (economics departments, government agencies, independent research institutions and think tanks) in each target country. We then used the website of each institution to manually extract the email addresses of economists to invite them to participate in our survey. The survey was conducted between October 2017 and April 2018.

In many cases during email extraction, especially in the case of multidisciplinary departments, research institutions and government agencies, we could not distinguish between economists and non-economists from the information provided on the website. In these cases, we asked our team of research assistants to extract all listed email addresses. Our rationale was that sending email invitations to some non-economists was clearly better than risking excluding some economists, especially since this exclusion could be systemically related to the type of institution and lead to sample selection. We made sure however that non-economists who received the survey invitation were self-filtered out by making it clear in our email invitation, as well as on the first page of the survey, that economists are the survey target population.

As a result, we are not able to provide a reliable estimate of the participation rate in our survey since that would require information on the total number of economists in the target population, which is considerably smaller than the total number of email addresses we extracted online. This calculation is further complicated by the fact that we received a considerable number of auto-replies from people who had left their institution, were on sabbatical, parental or sick leave, or temporarily had no access to their email. With these in mind, a very rough (under)estimate of the participation rate in our survey is around 15%. Although we cannot measure a reliable participation rate for our survey, our summary statistics (Table A3 in our Online Appendix) suggest that we have a very diverse group of economists in our final sample. We have also reported the distribution of responses by institution of affiliation in the USA, Canada and the UK in Figures A4–A6 in our Online Appendix as examples to show that participants in our survey come from a very diverse group of institutions in each country and are not limited to certain types of institutions.

Participants in our survey were required to complete each page in order to proceed to the next page. However, participation in the survey was entirely voluntary and participants were assured that, if they decided to withdraw at any point during the survey, their responses will not be used. For this reason, we are not allowed, by the terms of our ethics approval, to use data collected from people who did not complete the entire survey. As a result, we have restricted our sample to participants who completed the entire survey.¹³ Our final sample includes 2,425 economists from 19 different countries. We run several tests to ensure that our focus on participants who completed the entire survey does not introduce sample selection bias in our results and we find no evidence of such a bias. See Section 3 in our Online Appendix for a thorough discussion.

We are interested in examining whether the average agreement level with a given statement is systematically different between those who randomly received a less-/non-mainstream source or no source, relative to those who received a mainstream source. To do this, we estimate a linear regression model which is essentially equivalent to comparing average agreement levels between our three different groups. The additional benefits of using a linear regression model are that it makes it easy to test whether any differences that might emerge between groups are statistically significant. Moreover, it allows us to account for any differences in agreement levels between groups which could be systematically related to potential differences in observed characteristics among participants in different groups.

In our baseline analysis, we estimate by OLS the following model:

where $yij$ is the reported agreement level of participant $i$ with statement $j$ and is coded as 1 for ‘strongly disagree’, 2 for ‘disagree’, 3 for ‘neutral’, 4 for ‘agree’ and 5 for ‘strongly agree’; $S1ij$ and $S2ij$ are indicators that are equal to one if for statement $j$ participant $i$ randomly received a less-/non-mainstream source, or no source, respectively. Therefore, the omitted category which serves as the comparison group is those who received a mainstream source. We also include several individual-level control variables (⁠$Xi$⁠) in some of our specifications.¹⁴ If our randomisation is carried out properly, however, including these control variables should not affect our results (and, as reported, later on, we find that they don’t).

Our two parameters of interest to be estimated are $γ1$ and $γ2$ which simply measure the average difference in agreement level between those who receive a less-/non-mainstream source and no source, relative to those who receive a mainstream source, respectively. We should note that, while this regression model allows us to test whether any estimated differences in average agreement levels are statistically significant, it does not tell us much about the magnitude or the substantive size of these differences. For example, imagine that our estimated Treatment 1 effect is equal to −0.20 points (measured in units of agreement level) and is statistically significant. To get an idea about the magnitude of the effect, however, we need to have another measure to compare this effect to.

One suggested option is to use the standard deviation of agreement level in our sample. Therefore, a relatively small standard deviation of 0.5 implies relatively small variability in reported agreement levels across participants which in comparison makes the estimated effect of −0.20 large (40% of a standard deviation). In contrast, a larger standard deviation of 1.5 implies larger variability in agreement level and therefore a much smaller estimated effect (13% of a standard deviation). We, therefore, measure and report our estimated effects in units of standard deviation in agreement level, rather than in simple units of agreement level. Another advantage of such standardisation is that it facilitates the comparison of estimated effects across different subsamples (see Section 5.3), as well as across different studies—for example, for the purpose of a meta-analysis.

5. Results

5.1 Main findings

Figure 1 displays the probability of different agreement levels for each statement among all participants regardless of their group assignment. Results suggest that there is a significant dissensus on the wide variety of issues evaluated by economists. We find similar patterns if we restrict the sample to economists who only received mainstream sources or no sources. Table 1 displays the results from linear models that estimate how these agreement levels are influenced by our two treatments. Column (1) uses a simplified model with no additional control variables, while Columns (2)–(4) add observed personal and job characteristics as well as individual fixed effects.¹⁵

We find clear evidence that changing source attributions from mainstream to less-/non-mainstream significantly reduces the agreement level by 0.26 points. This is around one-fourth of a standard deviation or a 7.3% reduction in an average agreement level of 3.6 in our control group. Our results also suggest that removing mainstream sources (i.e. providing no source) also significantly reduces the agreement level by 0.41 points (a 1l.3% reduction which is equal to 35% of a standard deviation).

Estimates reported in other columns suggest that controlling for different characteristics and individual fixed effects does not change our results, which provides further support that our randomisation protocol was implemented properly. In addition, results from our specification with individual fixed effects suggest that our estimate of Treatment 1 effect is unlikely to suffer from sample selection bias due to non-random attrition across treatment groups. Moreover, results reported in Table A11 in our Online Appendix suggest that clustering the standard errors at the individual level does not have any appreciable impact on our results.

It is worth noting that our choice of OLS models was mainly driven by the fact that their results are easier to summarise and communicate. However, we also estimate our treatment effects using ordered logit models which are perhaps more appropriate given the ordered nature of our dependent variable. Results from these models are discussed in Section 5.1 in our Online Appendix and are similar to those from OLS models. Finally, we also conduct some analyses regarding the statistical power and the reproducibility of our findings which we report in Section 6 in our Online Appendix.

5.2 Ideological/authority bias or unbiased Bayesian updating?

It could be argued that our results presented and discussed above might not be necessarily driven by ideological bias induced by attributed sources. In order to organise and examine different potential explanations for our results, we use Bayesian updating as a guiding framework that fits reasonably well with different elements of our experiment.

More specifically, our experiment involves evaluating statements in an environment with imperfect information about the validity of the statements. This imperfect information could be due to not having enough knowledge about the subject, lack of conclusive empirical evidence, the statements being open to interpretation, etc. Bayesian updating models suggest that in such an environment, individuals make judgment using a set of prior beliefs that are updated using Bayes’ rule as new information arrives. In the context of our study, this translates into prior beliefs held by economists on each statement’s validity being updated according to a signal they receive regarding the validity of the statement, in form of an attributed source.

It is important to note, however, that the process of updating the priors could be both biased or unbiased. Bayes’ Theorem does not say anything about how one should interpret the signals received in the process of updating priors and therefore does not preclude the influence of cognitive or ideological biases in interpreting signals and updating priors (Bartels, 2002; Bullock, 2009; Fryer et al., 2019; MacCoun and Paletz, 2009). Unbiased Bayesian updating requires the processing of information to be independent from one’s priors (Bullock, 2009; Fischle, 2000; Taber and Lodge, 2006). In contrast, under ideologically biased Bayesian updating, one would selectively assign more weight to information that is more likely to confirm one’s ideological views (Bartels, 2002; Gentzkow and Shapiro, 2006; Taber and Lodge, 2006).

In the context of our study, lower agreement level associated with less-/non-mainstream sources could be attributed to unbiased Bayesian updating under the assumption that mainstream sources systematically provide objectively more credible signals regarding the validity of the statements. Alternatively, mainstream sources could be perceived as more credible not based on objective evaluations unrelated to priors, but rather based on the fact that mainstream sources are more likely to confirm the survey respondent’s mainstream views.

There exists extensive evidence that suggests that individuals tend to agree more with findings or views that are more (less) likely to confirm (disconfirm) their beliefs (e.g. Bartels, 2002; Bullock, 2009; Fryer et al., 2019; MacCoun, 1998, MacCoun and Paletz, 2009). This is broadly referred to as the confirmation bias. Beliefs that one seeks to confirm have different natures and could be formed by ingrained, ideological or emotionally charged views. If beliefs that an individual is trying to confirm or validate are shaped by their ideological views, we are dealing with what is often referred to as ideological bias.

For example, MacCoun and Paletz (2009) conduct an experiment to examine how ordinary citizens evaluate hypothetical research findings on controversial topics. They find that, when findings challenge their prior beliefs, people are more sceptical of the findings. Their results also suggest that ‘citizens, especially those holding conservative beliefs, tended to attribute studies with liberal findings to the liberalism of the researcher, but citizens were less likely to attribute conservative findings to the conservatism of the researcher’ (MacCoun and Paletz, 2009, p. 43). They interpret this as effects of ‘partisanship and ideology’.

Determining whether less/non-mainstream sources are more or less objectively credible than mainstream sources is of course extremely difficult, if not impossible, since both groups include individuals who are prominent scholars in their fields. Therefore, one major problem with unbiased Bayesian updating as a potential explanation is that there are no objective measures that could be used to assess the credibility of our sources. Any claims of systematic differences between these sources in terms of credibility are inevitably based on subjective metrics that correlate with where one stands relative to mainstream views and its academic norms. It is exactly for this reason that the norms of modern science suggest that any serious evaluation of an argument should be based on the content of the argument as opposed to the source attributed to it (MacCoun, 1998; Merton, 1973).

In fact, economists in our sample strongly agree with this view. More specifically, as part of a questionnaire that appears at the end of the survey, a strong majority of participants (around 82%) report that ‘a claim or argument should be rejected only on the basis of the substance of the argument itself’. Around 18% report that both the views of the author as well as the substance of the argument matter, with only a tiny minority (around 0.5%) reporting that the views of the author should be the only basis to reject an argument.

Despite the serious limitations of unbiased Bayesian updating as a potential explanation, we nevertheless propose three empirical tests that would allow us to further examine the validity of biased versus unbiased updating. We develop a Bayesian updating model in Section 4 in our Online Appendix and, due to space constraint, discuss here the results from the test whose conclusion is the clearest. Discussion of the other two tests can be found in Section 5.2 in our Online Appendix.

5.2.1 Biased or unbiased updating? Differences by political orientation

In this section, we examine whether our estimated treatment effects vary across different groups with different political orientations/ideologies. If the reduction in agreement level associated with changes in sources is based on objective differences in the credibility of the sources, this objective difference should not depend on one’s political views. As a result, our estimates should not vary systematically by political orientation under unbiased updating. In fact, if anything, those on the right should be less affected by changing the source attributions since they are significantly more likely to report that a statement should be evaluated based on its content only (86.7%, compared to 73.8% at the far left).

Our less-/non-mainstream sources often represent views or ideologies that are (politically) to the left of mainstream sources.¹⁶ Therefore, if our results are driven by ideological bias, reduction in agreement level should be larger among those more to the right of the political spectrum since altering the sources creates a larger contrast with their prior beliefs which will in turn induce a larger ideological reaction among this group.¹⁷

We estimate linear models where we allow the effect of each treatment to vary by political orientation. Political orientation is reported by participants on a scale from −10 (far left) to 10 (far right). We use the reported values to group people into five categories.¹⁸ Results from this model are reported in Table 2. Estimates reported in Column (1) suggest that there exists a very significant difference in the average agreement level among economists with different political orientations, even when the sources are all mainstream. For example, the average agreement level among economists categorised as left is one-fourth of a standard deviation lower than those categorised as far left. This already large difference increases consistently as we move to the far right, reaching a difference of 60% of a standard deviation, which is an increase of 150%. This strong effect of political orientation, which does not change after controlling for observed characteristics, seems to be a clear manifestation of ideological bias.

Estimates reported in Column (2) suggest an even more drastic effect by political orientation, and provide further evidence that our estimated treatment effects are driven by ideological bias. More specifically, for those on the far left, Treatment 1 only reduces the average agreement level by 4.6% of a standard deviation, which is less than one-fourth of the overall effect we reported in Table 1. However, moving from the far left to the far right consistently and significantly increases this effect, with the effect being almost seven times (678 %) larger at the far right compared to the far left (−0.36 vs. −0.046, respectively). We reject the null hypothesis that the effect at the far left (left) is equal to the effect at the far right at 0.1% (5%) confidence level. We also reject the null that the effects are equal across all five groups (F-statistic = 17.27) or across all four groups excluding the far left (F-statistic = 3.12).

Our estimates reported in Column (3) suggest that for every given category of political orientation, removing the sources has a larger effect on reducing the agreement level compared to altering the sources. However, this difference becomes smaller as we move towards right. This suggests that for those more oriented towards right receiving a less-/non-mainstream source or no source at all makes little difference.

Another important pattern to highlight is that, while the estimated effect of Treatment 1 consistently and very significantly increases as we move to the far right, we fail to reject the null that the estimated effect of Treatment 2 is similar across all five groups. This lack of systematic difference in the estimated effect of Treatment 2 by political orientation could be due to the fact that removing sources induce what is known as authority bias. Authority bias is the tendency to assign more credibility to views that are attributed to an authority figure. This implies that, under authority bias, political orientation does not necessarily influence the agreement level, which is consistent with our estimated effect of Treatment 2.

Given that we found a robust and significant estimated effect for both Treatments 1 and 2, up until this point, we could not rule out that both ideological bias and authority bias contribute to each treatment effect. However, our finding that the estimated effect of Treatment 2 does not follow the same meaningful pattern by political orientation as Treatment 1 is consistent with the distinction between ideological bias and authority bias. In other words, it suggests that there are important differences in underlying forces driving our estimated effects of Treatment 1 and Treatment 2, with the former (latter) more likely to be driven by ideological (authority) bias.

It is reasonable to argue, however, that the self-reported measure of political orientation used to categorise people depends on political environments and contexts that could vary significantly from one country to another. For example, someone who is considered a centrist or centre-right in the UK could perhaps be categorised as left in the USA. This could complicate the interpretation of our results. To address this issue, we use participants’ answers to a series of questions at the end of our survey that are designed to identify their political and economic typology.¹⁹ More specifically, we regress our self-reported political orientation measure on a series of indicators created based on answers to these questions. We then use predicted values from this regression to categorise people into five groups based on their distribution quintiles. These results are reported in Table A10 in our Online Appendix and remain similar to those discussed above.

Before we conclude this section, we would like to address another potential explanation for our results. As an alternative explanation for our overall findings, it could be argued that, given the low-stake nature of our survey, economists did not have the incentive to exert much effort and read each statement carefully. Therefore, when the attributed source for a statement was a prominent mainstream economist who they recognised and trusted as a scholar, they glossed over the statement and relied on the source for their evaluation. As a result, statements attributed to mainstream sources received a higher level of agreement. However, the results discussed in this section stand in contrast to this hypothesis. These results suggest that even conditional on sources being mainstream, there still exists a significant difference in agreement level by political orientation. Furthermore, Treatment 1 has systematically and significantly different impacts on people with different political ideologies. In other words, in evaluating the statements, the identified ideological contours of different sources clearly interact with participants’ own political ideologies.

In addition, if this alternative explanation is valid, then one of its implications is that participants in our control group should have spent less time completing the survey compared to those in the two treatment groups. However, our estimates reported in Table A7 in our Online Appendix suggest that there are no differences in average survey completion time between control group and Treatment 1. Consistent with these results, our estimates in Table A9 in our Online Appendix also suggest that restricting our sample to individuals with different survey completion times (a potential proxy for different levels of effort exerted to read the statements) also has no impact on our results.

5.3 Heterogeneity analysis

In this section, we examine how our estimated treatments vary by statement as well as by different characteristics including gender, country of residence, country where PhD was completed, undergraduate major and main research area. It is interesting and important to understand how the biases we have found in our analysis vary across different groups. This could help to shed more light on some of the factors underlying ideological/authority bias. As is discussed in more detail in the next sections, we find evidence of significant and systematic heterogeneity in our estimated treatment effects. Consistent with our previous results, these differences remain consistent with the existence of ideological/authority bias among economists.

5.3.1 Heterogeneity by statement

First, we examine the effect of our two treatments on agreement level separately for each statement. These results are summarised in Figure 2. Consistent with our overall results, we find that for all but three statements, Treatment 1 significantly reduces the agreement level. The estimated effects range from around one-tenth of a standard deviation to around half of a standard deviation. Interestingly, we find that the largest reduction in agreement level for Treatment 1 occurs for Statement 6, which is arguably the statement that is most critical of mainstream economics and its methods, and also brings up the issue of ideological bias in mainstream economics. This again is consistent with ideological bias where views that are more likely to disconfirm previously held beliefs are more strongly discounted when the source is less-/non-mainstream.

Regarding the three statements with no reduction in agreement level (i.e. Statements 1, 3 and 7), one potential explanation is that the ideological distance between the (real and fake) sources is not large enough to induce ideological bias. Taking a closer look at the sources for these statements seems to suggest that this is indeed a plausible explanation. The sources for these statements are Dani Rodrick versus Paul Krugman, Hayek versus Freud and Irving Fisher versus Kenneth Galbraith. Interestingly and consistent with authority bias, we find that, for the same three statements, removing the mainstream source significantly reduces the agreement level, highlighting again the difference in driving forces behind the estimated effects of Treatment 1 and Treatment 2.

Results displayed in the right panel of Figure 2 suggest that removing the source attributions significantly reduces the agreement level for all 15 statements. Similar to our results reported in Table 1, the estimated effects of Treatment 2 are larger than those of Treatment 1 in almost all of the statements.

5.3.2 Heterogeneity by gender

Next, we examine gender differences in our estimated treatment effects. These results are reported in Table 3. Column (1) suggests that the average agreement level among female economists in the control group is 6% of a standard deviation higher than their male counterparts. In addition, we find that the estimated ideological bias is 44% larger among male economists as compared to their female counterparts (24% of a standard deviation vs. 14%, respectively), a difference that is statistically significant at 0.1%. This difference holds even after including our extensive set of indicators for political orientation and political/economic typology. We also find similar results when we estimate gender differences in treatment effects separately for each statement. In 9 out of 15 statements, the estimated ideological bias is larger for men than for women (see Figure A3 in our Online Appendix).

These results seem to be consistent with evidence from psychology which suggests women exhibit less confirmation bias (Meyers-Levy, 1986; Bar-Tal and Jarymowicz, 2010). Gordon and Dahl (2013) also find evidence that male economists are less cautious in expressing an opinion. We find, however, that the gender difference in authority bias is much smaller (34% of a standard deviation for men vs. 35% for women) and statistically insignificant. In other words, removing mainstream sources altogether seems to affect men and women in similar ways.

We would also like to highlight the estimated gender differences for Statement 5, which discusses the issue of gender gap in economics. The statement reads: ‘Unlike most other science and social science disciplines, economics has made little progress in closing its gender gap over the last several decades. Given the field’s prominence in determining public policy, this is a serious issue. Whether explicit or more subtle, intentional or not, the hurdles that women face in economics are very real’. Overall, and without considering group assignment, there exists a very large gender difference in agreement with this statement. More specifically, conditional on observed characteristics, the average agreement level among male economists is 0.78 points lower than female economists, which is equivalent to 64% of a standard deviation in agreement level.

Taking group assignment into account, female economists who receive Carmen Reinhart as the source (i.e. control group) report an agreement level that is on average 0.73 points (60% of a standard deviation) higher compared to their male counterparts. Moreover, while switching the source to the left-leaning economist/sociologist Diane Elson does not affect the agreement level among female economists (estimated effect is 0.006 points), it significantly decreases the agreement level among male economists by 0.175 points (around 15% of a standard deviation). One potential explanation is that when it comes to the important issue of the gender gap in economics, which involves female economists at the personal level, women put aside ideology and focus on the content of the statement as opposed to its source.

These results also highlight a large divide between male and female economists in their perceptions and concerns regarding the gender gap in economics. This is of critical importance since the discussion around the gender problem in economics, although long overdue, has finally received some attention in the discipline. For example, during the 2019 American Economic Association (AEA) meeting, in one of the main panel discussions titled ‘How can economics solve its gender problem?’, several top female economists talked about their own struggles with the gender problem in economics. This is following the appointment of an Ad Hoc Committee by the Executive Committee of the AEA in April 2018 to explore ‘issues faced by women […] to improve the professional climate for women and members of underrepresented groups’.²⁰ It is well understood that approaching and solving the gender problem in economics first requires a similar understanding of the problem by both men and women. However, our results suggest that there exists a very significant divide between male and female economists in their recognition of the problem. In a discipline dominated by men, this makes it challenging to meaningfully address the gender problem.

5.3.3 Heterogeneity by country of residence/PhD completion

Next, we examine how our estimated effects vary by country of residence. These results are reported in Table 4. Estimates reported in Column (1) suggest that even when sources are mainstream, and conditional on observed characteristics, there are significant differences in average agreement level by country (we reject the null of equality at 0.1%). On one side, we have economists in South Africa, France and Italy who hold the highest level of agreement with the statements, while on the other side we have economists in Austria and the USA who hold the highest level of disagreement. These results are consistent with Frey et al. (1984) who also find significant differences across five countries in views among economists.

Similarly, and for both treatments, we find that the estimated treatment effects vary significantly across countries, ranging from around half of a standard deviation to zero. We also reject the null that the estimated effects of Treatment 1/Treatment 2 are the same across countries at the 0.1% confidence level. More specifically, we find that economists in Austria, Brazil, and Italy exhibit the smallest ideological bias (for Brazil and Austria, the estimated effects are also statistically insignificant). On the other side of the spectrum, we find that economists in Ireland, Japan, Australia and Scandinavia exhibit the largest ideological bias. Economists in countries such as Canada, the UK, France and the USA stand in the middle in terms of the magnitude of the estimated ideological bias. In addition, when we examine the effect of authority bias, these countries maintain their positions in the distribution, although the estimated effects of authority bias remain larger than the estimated effects of ideological bias for most of the countries.

Table 5 reports results that examine heterogeneity by country/region where a PhD was completed. We find that economists who completed their PhD in Asia, Canada, Scandinavia and the USA exhibit the strongest ideological bias, ranging from 39% to 25% of a standard deviation. On the opposite end, we find that economists with PhD degrees from South America, Africa, Italy, Spain and Portugal exhibit the smallest ideological bias (statistically insignificant for South America and Africa). These results are somewhat consistent with those reported in Table 4 and suggest that some of the countries where economists exhibit the largest/smallest ideological bias are also those that induce the strongest/weakest ideological bias in their PhD students (e.g. Brazil, Italy and Scandinavia). In addition, we find that our estimated effects of authority bias, while larger in size, largely follow the same patterns as our estimates of ideological bias.

5.3.4 Heterogeneity by area of research

In Table 6, we take up the issue of heterogeneity by the main area of research. Results reported in Column (1) suggest that, similarly to previous heterogeneity patterns and conditional on observed characteristics, there are significant differences in agreement level among economists from different research areas, even when attributed sources are all mainstream. Estimates reported in Columns (2) and (3) suggest that economists whose main area of research is history of thought, methodology, heterodox approaches; cultural economics, economic sociology, economic anthropology; or business administration, marketing, accounting exhibit the smallest ideological and authority bias.²¹ We find, however, that economists whose main area of research is macroeconomics, public economics, international economics and financial economics are among those with the largest ideological bias, ranging from 33% to 26% of a standard deviation.

Another interesting point to highlight is that, while for economists in all research areas the estimated effect of ideological bias is smaller or similar to the estimated effect of authority bias, macroeconomists are the only group for whom the estimated ideological bias is significantly larger than the estimated authority bias (one-third vs. one-fifth of a standard deviation). This is potentially driven by the fact that our less-/non-mainstream sources induce a stronger reaction in macroeconomists than when we remove the sources altogether.

5.3.5 Heterogeneity by undergraduate major

Lastly, we examine heterogeneity by undergraduate major. As we discussed before, there exists growing evidence that suggests economic training, either directly or indirectly, could induce ideological views in students (e.g. Allgood et al., 2012; Colander, 2005; Colander and Klamer, 1987; Rubinstein, 2006). Consistent with these studies, results reported in Table 7 suggest that economists whose undergraduate major was economics or business/management exhibit the strongest ideological bias (one-fourth of a standard deviation). However, we find that economists with an undergraduate major in law; history, language and literature; or anthropology, sociology, psychology, exhibit the smallest ideological bias (statistically insignificant in all three cases).²²

6. Conclusion

In this study, we use an online randomised controlled experiment involving 2,425 economists in 19 countries to examine the influence of ideological and authority bias on views among economists. Economists who participated in our survey were asked to evaluate statements from prominent economists on different topics. However, source attribution for each statement was randomised without participants’ knowledge. For each statement, participants either received a mainstream source, a less-/non-mainstream source, or no source. We find clear evidence that changing source attributions from mainstream to less-/non-mainstream, or removing source attributions, significantly lowers economists’ level of agreement with statements. This contradicts the image economists have/report of themselves, with 82% of participants reporting that in evaluating a statement one should only pay attention to its content. These findings along with other evidence we provide in Section 5.2 point to the existence of strong ideological and authority bias among economists. We also find significant heterogeneity in our results by gender, country, PhD completion country, research area and undergraduate major, with patterns consistent with the existence of ideological bias.

Scholars hold different views on whether economics can be a ‘science’ in the strict sense and free from ideological biases. However, ideological bias that could result in endorsing or denouncing an argument on the basis of its author’s views rather than its substance, is unhealthy and in conflict with scientific tenor and the subject’s scientific aspiration, especially when the knowledge regarding rejected views is limited.

Furthermore, it is hard to imagine that the biases that our results uncover will only manifest themselves in a low-stakes environment, such as our experiment, without spilling over to other areas of academic life.²³ After all, political scientists, sociologists and psychologists have long established the widespread influence of such biases on various important domains of our lives. In addition, there already exists growing evidence that value judgements and political ideology of economists affect not just research (Jelveh et al., 2018; Saint-Paul, 2018) but also citation networks (Önder and Terviö, 2015), faculty hiring (Terviö, 2011), and economists’ positions on positive and normative issues related to public policy (e.g. Beyer and Pühringer, 2019; Fuchs et al., 1998; Mayer, 2001; van Dalen, 2019; Van Gunten, 2015). Given this, the biases revealed by our results could play an important role in suppressing plurality, narrowing pedagogy and delineating biased research parameters in economics. We believe that recognising their own biases, especially when there exists evidence that biases could operate through implicit or unconscious modes, is the first step for economists in their attempt to be objective and ideology free. This is also consistent with the standard to which most economists in our study hold themselves.

Another important step to minimise the influence of ideological biases is to understand their roots. It is well understood that ideological bias is fundamentally knowledge based. Mainstream economics, as the dominant and most influential strand in economics, plays a major role in producing discourses, which constitutes knowledge among economists. It can, therefore, shape ideological views among economists via channels through which it produces and disseminates economic discourses.

One of these important channels is economics education, through which economic discourses are disseminated to students and future economists. It affects the way students process information, identify problems and approach these problems in their research. Not surprisingly, this training may also affect the policies they favour and the ideologies they adhere to. For example, Colander and Klamer (1987) and Colander (2005) surveyed graduate students at top-ranking graduate economic programs in the USA and find that, according to these students, techniques are the key to success in graduate school, while understanding the economy and knowledge about economic literature only help a little. This lack of depth in the knowledge acquired, not only in economics but in any discipline or among any group of people, makes individuals lean more easily and blindly along ideological lines.

Another important channel is the social structures and norms in the profession. While social structures and norms exist in all academic disciplines, economics seems to stand out in at least several respects, characterised by the centralisation of power and the creation of incentive mechanisms for research in a way that hinders plurality, encourages conformity and promotes adherence to the dominant (ideological) views. Wright (2019) highlights several features of the discipline that make the internal hierarchical system in economics ‘steeper and more consequential’ compared to that in most other academic disciplines. These features include (i) particular significance of journal ranking, especially the ‘Top Five’, in various key aspects of academic life including receiving tenure (Heckman and Moktan, 2020), securing research grants, receiving invitation to seminars and conferences, and getting request for professional advice; (ii) dominant role of ‘stars’ in the discipline (Goyal et al., 2006; Offer and Söderberg, 2016); (iii) governance of the discipline by a narrow group of economists (Fourcade et al., 2015); (iv) strong dominance of both editorial positions and publications in high-prestige journals by economists at highly ranked institutions (Colussi, 2018; Fourcade et al., 2015; Heckman and Moktan, 2020; Wu, 2007); and (v) the strong effect of the ranking of one’s institution, as a student or as an academic, in career success (Han, 2003; Oyer, 2006).

Some might object that economists are human beings and therefore these biases are inevitable. However, the differences we find in the estimated effects across personal characteristics such as gender, political orientation, country and undergraduate major clearly suggest that there are ways to limit those ideological effects and ways to reinforce them. Furthermore, admitting that as economists we are also susceptible to different biases in our economic views and analysis should at least cast some doubts on the strong emphasis often put on positivism and objectivity by mainstream economics. We believe, as Mckenzie (1981, p. 715) suggests, that ‘[t]he social importance of economic analysis is enhanced when the necessary normative context of economic analysis is openly admitted. The social scientists, as distinguished from the research technician, not only must test the relevance of his science, but also must be prepared to test in open forum the normative context within which his science is conducted’.

We thank John List, Syngjoo Choi, Carey Doberstein, Julien Picault, Michele Battisti, Erik Kimborough, Ali Rostamian, Leonora Risse, Sam van Noort and anonymous reviewers for helpful comments and discussions. We would also like to thank Michele Battisti, Julien Picault, Taro Yokokawa and Allan Vidigal for their help in translating the survey into Italian, French, Japanese and Brazilian Portuguese, respectively. We are also grateful to our research assistant, Jay Bell, for his assistance. We would also like to thank all the people who took the time to participate in this survey. Funding for this project was provided by Social Sciences and Humanities Research Council of Canada (SSHRC) FAS# F16-00472. This study obtained ethics approval from the Behavioral Research Ethics Board at the University of British Columbia (#H16-01480). This study is registered in the AEA RCT Registry and the unique identifying number is AEARCTR-0003246.

Bibliography

Footnotes