3 The Comparative Agendas Project: The Evolving Research Interests and Designs of the CAP Scholarly Community

This book describes and presents the evidence gathered in the framework of the Comparative Agendas Project (CAP). In this chapter, we adopt a slightly different approach. Instead of presenting the countries and the evidence they’ve gathered, and instead of showing what can be done with the agenda data, we shift our attention to what has been done with the agenda data during the last dozen years or so. We examine the evolving research interests of the scholars participating in the CAP community and the designs they used for their work. We then discuss what these past trends may suggest for the future of CAP research—a future that is very bright, indeed.

At the heart of the CAP community are conferences, organized since 2006, that occur each year in changing venues. The first conference was organized in Aarhus in 2006, the last so far in Edinburgh in 2017. In the years between, the CAP community met in Paris (2007), Barcelona (2008), Den Hague (2009), Seattle (2010), Catania (2011), Reims (2012), Antwerp (2013), Konstanz (2014), Lisbon (2015), and Geneva (2016). The work presented at these yearly events no doubt grasps the gist of the work undertaken with CAP data. All country teams have been represented at nearly every CAP conference, and as soon as a new CAP project starts in a new country, the team is invited to attend the next year’s conference in order to formally present the new country’s project, its aims and data. As a consequence, the number of participants in the CAP conference has increased considerably, from only thirteen in 2006 to seventy-nine in 2017. The greatest increase in participants and papers took place in 2009, in Den Hague, when the number of participants nearly doubled from the previous year (up from 32 in Barcelona in 2008 to 60). The Netherlands are, after all, lovely in June. Since Reims (2012), the number of participants has more or less stayed the same and there is no more growth to be noted. Looking at the work that has been presented at these yearly conferences gives us a good overview of what CAP scholars have been up to with the agenda data they’ve collected.

This chapter presents simple and descriptive analyses of the abstracts of papers that were presented at one of eleven CAP conferences—there have been twelve CAP conferences but the data from Paris in 2007 are missing (we blame the excellent French wine). In total, our analysis draws upon exactly 398 papers presented/co-authored by more than two hundred different individuals. We, the authors of this chapter, in line with the CAP content analysis approach, coded the abstracts on a number of variables. We did not implement any inter-coder reliability procedures but agreed on the variables and how to code them before we started our work. Some of the paper abstracts were unclear, vague or sometimes even entirely missing.¹ As a consequence, the total number of conference papers in the tables and figures below varies, as missing data varies across variables. Note that in most cases we content-analyzed the abstracts that were submitted before the conference in order to be accepted to present a paper. The actual paper that was eventually presented later at the conference (and that was published even much later in a journal or book, if at all) could have been different; our experience is that even among the excellent CAP community, there can often be some slippage between promises and delivery.

The essence of the CAP is attention to issues. Political attention is a scarce resource as politicians have limited time, energy, money, staff, etc. to deal with all the problems in a society. At the same time, and maybe exactly because it is scarce, political attention is consequential; it is an absolute precondition for policy change. So, CAP scholars investigate the causes and consequences and dynamics of political attention to issues. Political agendas are manifold, though. All political actors and institutions more or less have their own agenda, their own prioritized list of issues they devote more or less attention to. The political agenda does not exist. Over the years, CAP scholars have addressed issue attention with regards to a wide variety of political agendas, broadly defined. Figure 3.1 lists the number of CAP conference papers examining each major agenda, noting that many papers examine more than one agenda and thus might be included multiple times.

The figure shows that the policy agendas approach has been applied to no less than thirteen different agendas, ranging from congressional hearings to street protests, from parliamentary questions to law making and budgeting. CAP scholars used all these agendas to take quantitative measure of the amount of attention paid to a given issue or issues. This was mostly done by counting the number of individual items (e.g., laws, street demonstrations) per time unit devoted to one of the CAP codebook issues, but sometimes also by counting money (e.g., budgets) or sentences or quasi-sentences (e.g., manifestos, State of the Union addresses) spent on issues. The figure underpins the versatility of the agenda approach that has been applied to a very wide variety of political attention venues.

The graph indicates that the media agenda is by far the most examined agenda in the CAP network. Of all 398 papers presented at CAP conferences, eighty-seven contained media data (22 percent). This is a remarkable result, since the media cannot be considered as a pure sang policy agenda. Rather, the media are often used as an indicator of societal demands and of incoming signals. In a way, the media are supposed to capture the external information that is injected into the political system. As a consequence, the mass media are mostly an independent variable in agenda research; only in rare cases are they the dependent variable.

Further, the graph shows that the media are definitely not the only ‘societal’ agenda in the body of work that CAP has produced over the years. The darker bars in the figure refer to societal agendas—these are not policy agendas in the strict sense—that are mostly used as alternative independent variables in some studies. The large interest in societal agendas suggests that a good deal of CAP studies are concerned with the responsiveness of the core policy agendas to external information. Party manifestos (45), parties forming the prime link between societal demands and the institutional political system, score high also. The same applies to public opinion (57) that was most often operationalized through the classic most important problem measure.

The grey bars in the graph refer to, what we could call, ‘hard core’ policy agendas; they mostly form the dependent variable of the studies. The agenda most commonly used within this category is the agenda of laws (53). Agenda scholars have also shown a large interest in bills (39), parliamentary questions (37), executive orders (36), and executive speeches (35). What we could call the most viscose and most consequential agenda of them all, the yearly budget (Baumgartner et al., 2009), appears in twenty-six studies.

Thus, our analysis of what CAP scholars are interested in shows a notable interest in agendas that go beyond what one would spontaneously mention when thinking about policy agendas. Or, in other words, the CAP community’s take on politics and policymaking is broad and does not remain confined to the classic, institutional policy agendas situated at the end of the policy cycle. Figure 3.1 is thus a first indicator of the fact that the CAP network produces mainstream political science research examining the political process and is not just interested in policy output.

This observation can be put in a longitudinal perspective, as well. Figure 3.2 shows—as an example of the attention to societal signals—the percentage and number of papers in which the media agenda was coded. The bars are labeled with the absolute number while the size of the bar is proportional to the number of papers presented that year. The graph shows a general rise in work including the media from 2012 onwards in absolute numbers; in percentages, the media agenda seems to have received a little less attention during the most recent years. In 2017 in Edinburgh, our last observation, seven of the in total forty-eight papers deal, among others, with the mass media (16 percent).

Apart from the question of which agendas CAP scholars are interested in, a second important distinction between studies is whether scholars try to explain the attention for issues on a given agenda. In fact, a large share of the initial agenda work as inspired by Frank Baumgartner and Bryan Jones (1993; 2005) was, at least empirically, descriptive in nature. Its aim was to show that political attention is stochastic and spiked with long periods of stasis alternating with short bursts of attention. It is difficult to conclude based on short abstracts whether the (to be written) paper is descriptive or rather explanatory. As a simple solution, we coded the abstracts for whether they mention the fact that the paper will assess the impact from one agenda on another. We can be reasonably confident that these inter-agenda papers are explanatory in nature. Figure 3.3 contains the evidence.

A large group of papers (198) assesses the impact of one agenda on another. Incorporating at least two agendas, these papers are basically interested in how attention for an issue jumps from one agenda to another. Prime examples here are studies dealing with how media attention leads to parliamentary questions or how executive speeches foreshadow legislation. Other CAP papers (152) do not tackle inter-agenda effects. These papers study one agenda only, sometimes seeking to explain variance in that one agenda as a function of other variables in a given country (e.g., economic conditions), and other times in a comparative perspective. Examples are studies dealing with legislation in two countries finding that the same issues become the object of legislation in the two countries at about the same time.

Yet again, we see quite a dramatic change over time. Figure 3.4 shows the number of inter-agenda papers over time and displays a secular increase. More and more, CAP scholars have become interested in the dynamics of agenda-setting by examining how agendas influence each other. At the last conference in 2017 (Edinburgh), twenty-six of the forty-eight papers were somehow looking into inter-agenda effects (54 percent) while the average for the entire period is 36 percent. This clear trend reflects the increasing value CAP scholars have put on explaining why issue attention goes up or down over time. We’ve known for a while that attention for issues is irregular and punctuated, and increasingly the CAP community has turned to explaining why this could be the case. Inter-agenda influence is one of the prime candidates.

The previous section explored the evolving interest of the CAP community over time: the agendas under study and the increasingly explanatory take on the agenda process. But what designs have students of agenda-setting been using?

To start with, to what extent has CAP work been comparative? The network of CAP scholars has been growing organically, with country teams gradually joining the common endeavor but (intentionally) without strong centralized efforts and (unintentionally) without consistent funding to integrate the country data in a common comparative dataset. Prior to the coordinated effort among all participating projects to edit their data in accordance with the common CAP codebook (Bevan, 2017) and then the launch in 2016 of the www.comparativeagendas.net website, comparative work depended on the willingness of country teams to share data and on their ability to put together integrated databases. This challenge has seriously hampered the pace of development of comparative work in the CAP community. Indeed, most studies deal with evidence from one country only. We found only eighty comparative studies; that is, studies containing evidence of more than one country (20 percent). A majority of work in the CAP community is not comparative in nature (58 percent). Figure 3.5 has the evidence.

Figure 3.6 shows that the structural shift towards comparative work has not occurred yet. Despite the concerted efforts of the CAP community to unify a central coding scheme and the strong professional relationships between researchers of different countries, there is simply no increase in comparative work over time. On the contrary, the percentage of CAP conference papers examining comparative evidence has gone down over time. That said, as we will elaborate in the conclusion, we expect a major shift towards comparative agenda work in the years to come.

Although a good deal of the CAP work is not comparative, the community is highly international, with scholars from many different countries gathering agenda data according to CAP standards, attending CAP conferences, and publishing about their own or other countries. Still, the country project triggering the whole CAP undertaking has been the US Policy Agendas Project, which has been ongoing from the 1990s onwards. CAP has been spreading to other countries, but exactly how international is the work produced by CAP? Instead of coding the nationality of CAP attendees, we simply coded the abstracts for which country (or countries) the paper examined, including two US states: Florida and Pennsylvania. Note that comparative papers are double (or triple, quadruple, etc.) counted. Figure 3.7 summarizes the evidence.

The data show that US evidence dominates the CAP community. Over the years, ninety-one out of 398 papers have used (at least) US evidence (23 percent); nearly one-quarter of all papers deal with US data. All in all, this is not an extremely large share, knowing the head start of the US country team with its project dating back at least ten years compared to the second country’s project. A number of European countries with early-starting and long-lasting agenda projects are also well represented in the country list: United Kingdom (41), Belgium (38), Denmark (32), Spain (32), the Netherlands (28), Switzerland (26), and France (24). Together with the Americans, these (Western) European countries have formed the core of the CAP community for many years. These results strongly underpin the US-European bias in agenda work. Still, testifying to the international character of the CAP community, papers presented at CAP conferences dealt with no less than twenty-nine distinct countries, with thirteen of these countries outside of the United States and Western Europe.

Figure 3.8 focuses on the United States only, sketching the number of CAP papers with US evidence over time. The raw number goes up with the years, but in fact as a percentage of all papers the US focus has not noticeably increased. But neither has it noticeably decreased. Thus, it is not the case that CAP scholars are slowly turning away from US evidence. On the contrary, while the number of countries with CAP evidence is slowly expanding, US data remain the most used sources of evidence among CAP scholars.

Finally, there is a clear distinction in CAP work between studies dealing with a select number of issues (or just one issue) and other work dealing with all issues that have been CAP-coded. In fact, the coding of political material based on the CAP codebook typically leads to a dataset that spans the entire universe of all possible policy issues. In a sense, all that happens in politics is turned into one of 233 issue codes (or one of the 21 topic codes). Some studies have taken this entire dataset and have looked at all issues. These typically are studies looking for an overall pattern of agenda-setting across issues. Other studies have, in contrast, looked at a single issue or a small number of issues. These studies were more interested in a specific policy domain because of its substantial interest or they selected a smaller number of issues because of theoretical reasons. The former strategy of examining all issues can be considered to be typical for political scientists more generally; they are less interested in specific policies but more in the general political process. The latter strategy of focusing on only one or a few issues is typical for policy scholars who are mostly interested in the output of the political system and who care about some specific issue(s) for substantial reasons. So, is CAP more of a general political science or rather more of a specific policy science community? Figure 3.9 has the evidence.

The studies using the full dataset including all issues are clearly prevailing. No less than 60 percent of all papers presented at CAP conferences encompass all issues. This pattern suggests that the CAP community may be more of a political science than of a policy science group. There is less interest in substantive issues but more in how issues in general are processed in the political system. Issues are mostly just cases that are used to track the political process. Still, a sizable minority of studies (30 percent) focus on one or only a few issues.

Again, we looked over time at the difference between papers that did and did not encompass all issues to see whether there is an evolution through the years. There are some indications of a trend, but it is definitely not secular, as Figure 3.10 shows. The last Edinburgh conference in 2017 seems to be an outlier, with exceptionally few all-issue papers (21 percent). Looking at the whole series suggests that the trend has been generally upward, though, especially compared to the early years. This finding suggests that, over time, the CAP community may have evolved even more into a mainstream political science group. The interest in specific issues was never very high but it appears even to have decreased over the years.

Our data suggest that the scholars of the Comparative Agendas Project are mainly interested in the political process and less in public policy. The interest in the political process has been present from the very beginning of the project but seems to have become even stronger over the years. Indeed, CAP has become mainstream political science. Moreover, the agenda data are increasingly used to analyze how agendas influence each other and, thus, to track political power. Arguably the most central research agenda within the CAP community is dealing with the responsiveness of political institutions (and their agendas) to societal signals coming from outside the political system, sensu stricto. So, ultimately, what drives much of the agenda work seems to be a concern about democratic representation. The main weakness of the network is its relatively limited attention for comparison. By far most of what has been done with CAP data is single-country only and does not compare across countries; there is no trend towards more comparative work. Further, while the number of countries with agenda data is definitely growing, the original American and Western European countries are still dominating the CAP conferences.

This chapter looked back and sketched the (short) history of CAP by examining the papers presented at the yearly conferences. Can we draw conclusions about the future development of CAP from it? It is of course most simple to predict that the next years will bring more of the same and that the tendencies witnessed in previous years will continue to manifest themselves in the years to come. Yet, there are some signs that the next few years will also bring structural changes to CAP—we could call them ‘punctuations’ (although CAP scholars have never been very good at predicting when exactly the spike of change is going to occur, not even afterwards).

The most important crucial new thing is the data website launched in the summer of 2016. While before the CAP data were not centralized and could only be collected by asking each country responsible for its data and then by following by a painstaking exercise to construct a comparative and comparable dataset, the free availability of all CAP data in a standardized format is expected to strongly boost the comparative use of the CAP data in the years to come. Indeed, the collected dataset is so vast that it likely is the biggest dataset available in the whole of political science (with millions of observations). It is not hard to predict that these data will be used for non-strictly agenda-setting purposes as well. So, the variety of research questions, designs, and purposes for which CAP data will be used is surely about to increase further.

The increased centralization and standardization of the dataset could have an impact on the functioning of the CAP network as well. The network grew organically, as new country teams popped up and sought connection with people already doing agendas work. A hallmark of the CAP conferences has been an openness to newcomers. Also, despite some key senior researchers responsible for the development of the CAP community, these researchers were careful not to take dictatorial charge, fostering instead a rotating set of temporary leaderships coupled with a tradition of the conferences moving from one venue to another. It remains to be seen whether such an informal structure without formal leadership or central funding can survive in the long term, especially when a common dataset has to be maintained. The maintenance and updating of the common dataset is the most obvious main challenge for the years to come. At present, there is a core of comparable data covering many years, but this evidence will soon be outdated. Shall country teams find the energy and the funding to update their datasets? This challenge might prove daunting. At the same time, as more and more scholars use the now easy-to-access CAP data, the need and will to regularly update these datasets will be felt even more acutely. And by any measure, the quality and commitment (and likeability) of the CAP community is superb. We are optimistic about its future.