Market failure in the diffusion of clinician-developed innovations: The case of off-label drug discoveries

Abstract

1. Introduction

Research on product development by individual consumers in the household sector have shown that, in over 90% of cases, these innovators do not protect their innovation designs from free adoption by others. (This research involved nationally representative surveys carried out in the UK by von Hippel et al. (2012), in the USA and Japan by Ogawa and Pongtanalert (2011), in Finland by de Jong et al. (2015), in Canada by de Jong (2013), and in South Korea by Kim (2015).) The availability of innovation designs and innovations ‘for free’ can, of course, increase social welfare. However, there is also a downside to this pattern. An individual innovator who does not protect his or her innovation from free-riding adopters may at the same time have no or little incentive to invest in actively diffusing the innovation-related design to free riders. After all, by definition there is no market connection between the innovator and the free adopter under the conditions just described. As a consequence, benefits that the free-riding adopter may gain from the innovation are not shared with the developer: they are an externality from that individual’s point of view. And, as a further consequence, the amount that the free-revealing innovator invests in diffusion may be too low from the perspective of social welfare.

To understand the ‘right’ level of spending on diffusion in this case, we can consider the innovation developer together with the pool of potential free-riding adopters as a combined ‘system’ for which we are seeking to maximize benefits. We assume that investments in diffusion of freely revealed innovations will lower adoption costs for free riders. We also assume that additional investments will lower adopters’ costs at a declining rate. (For example, the first hour I spend improving the documentation for my freely revealed software-coded innovation might help clarify things a lot for free adopters, the second hour would contribute somewhat less additional clarity etc.) System benefit is then maximized at the point where an additional dollar of investment in diffusion by innovators or adopters reduces adoption costs by a dollar across all adopters (von Hippel 2016).

The situation described above is more generally described in economics under the heading of ‘market failure’. A market failure exists when another possible outcome can make a market participant better off without making someone else worse off (Krugman and Wells 2006). In turn, market failures are regarded are a form of inefficiency, especially of information and resources, that call for government intervention and remedy (Bator 1958; Cowen 1988).

In previous work, we have shown evidence for this form of market failure in the case of household sector innovators in Finland (de Jong et al 2015). In this paper, for the first time we explore the phenomenon beyond household sector innovators. We will consider whether medical clinicians who make important discoveries regarding new uses for drugs also underinvest in diffusing their findings ‘for free’ to other clinicians and their patients.

The remainder of this paper is organized as follows. We describe the context of our research study in Section 2. We then describe the pathways by which freely revealed innovations diffuse in general, and also specifically in our study context (Section 3). Next, we discuss the likelihood of market failure with respect to diffusion in the specific context of free innovation by clinicians (Section 4). We then review the literature on user innovation and diffusion (Section 5). In Section 6 we explain our research methods, and in Section 7 we present our analyses and findings. Finally, in Section 8 we discuss general and specific implications of the market failure found with respect to the diffusion of freely revealed innovations.

2. Research context

Prior empirical research has shown that, in the USA, a large fraction (about 60%) of valuable new ‘off-label’ applications for Food and Drug Administration (FDA)-approved pharmaceuticals are discovered by practicing clinicians via direct observation of clinical outcomes, or by reports to them from their patients (Demonaco et al. 2006).

The distinction between on and off-label usage of FDA-approved drugs is frequently mentioned in medical practice. To understand what is meant, it is important to know that, in the USA, developers of novel drugs wishing to sell them on the market must first apply to the FDA for approval to do so. In their application, they must explain what the drug or device is to be used for, and demonstrate to the satisfaction of the FDA that it is both safe and efficacious for that use. Successful completion of the FDA application and approval process is usually very expensive and time consuming. In the case of new drugs, the average total development costs from bench to market exceed US$800 million and can take 10–15 years to accomplish (Drews 2000).

When the approval process has been successfully completed, the drug is only approved for advertisement and sale by the producer for ‘labeled’ uses—those whose efficacy and safety have been successfully demonstrated to the FDA. However, once a drug or device is on the market for any FDA-approved use, clinicians will often discover additional medically valuable uses for that item. These are termed ‘off-label’ uses. Consider for example, the discovery of the value of the drug Propofol as an antipruritic (anti-itch) drug related to surgical anesthesia.

Anesthesiologists often control patient pain by using an epidural (injection or infusion near the spinal cord) drug administration technique. Many patients initially suffer from intense itching when morphine or another narcotic is administered in this way. This intense itching can be very troublesome and is difficult to treat with the usual drugs.

Propofol is a sedative–hypnotic drug with the approved, on-label use of ‘inducing’ anesthesia. In 1992, anesthesiologists published the results of the first use of Propofol for the relief of itching due to epidurally administered narcotics (Borgeat et al. 1992). The first author was contacted to determine the events leading up to discovery of this novel off-label use. He explained that the initial clinical observation was serendipitous. A patient who complained of intense itching, nausea and vomiting related to epidural morphine was treated with the coincident administration of subtherapeutic doses of Propofol, and reported immediate cessation of itching. This serendipitous experiment was then intentionally repeated on other patients with the same clinically beneficial results. This valuable off-label use of Propofol is common in hospitals across the USA today.

3. Innovation diffusion pathways

Clinicians’ use of FDA-approved drugs in off-label applications is a very important part of medical practice. Data suggests that in fields such as chemotherapy and prescriptions for children, off-label use of drugs accounts for as much as 85% of total prescriptions (Jong et al. 2000). Indeed, for some diseases such as non-small cell lung cancer and cystic fibrosis, off-label uses of existing drugs are either the only drug therapies available or are therapies of choice (Poole and Dooley 2004). Of course, only some off-label discoveries prove to have clinical value with the passage of time and the accumulation of clinical experience (Abbot and Ayres 2014).

The development and diffusion paths followed by innovations developed by those who use them have been explored, and are shown in Fig. 1 (Baldwin et al. 2006). At the top of Fig. 1, we see that these individuals may choose to diffuse information regarding their developments to their peers for free (Benkler 2002; 2006). We define peer-to-peer diffusion as occurring via information that is freely revealed independent of markets. Free revealing is defined to exist when the information is provided without any restriction upon, or charge for, the information itself. (The reasons why innovators might act in this manner are: if it takes effort to restrict access, and if adoption by others does not damage the innovator’s interests, it is reasonable to not invest in restricting access (Baldwin and von Hippel 2011; Harhoff et al. 2003; Allen 1983).) The second pathway is diffusion of the innovation by the innovator to a commercializing firm which, in turn, diffuses innovations to adopters via the marketplace.

Figure 1.

Pathways via which clinician innovations diffuse

Open in new tabDownload slide

In terms of the present study, if a clinician discovers a novel off-label use for an approved drug then, as can be seen in Fig. 1, the first pathway for diffusing that information is to directly inform peer clinicians about the new use for free. The second pathway involves informing a pharmaceutical firm of the new off-label use with a view towards commercial diffusion. In practice today, diffusion is usually entirely up to the innovating and adopting clinicians via information freely distributed on the peer-to-peer channel shown in Fig. 1. Although the drug producer would benefit from advertising the new application—doing so would probably increase sales—the producer is prohibited from doing this by FDA regulations. The FDA requires new clinical trials for any new use before it will allow a producer to advertise that use. As a practical matter, producers seldom find additional clinical trials economically justifiable, and instead rely on clinicians to diffuse information regarding new applications to peers off-label.

However, the commercialization pathway shown in Fig. 1 is not entirely inactive in the case of discoveries of novel off-label uses for FDA-approved drugs. In rare cases, the newly discovered off-label application can be so potentially commercially valuable that clinicians making the discovery will choose to patent the use and work with a pharmaceutical firm to conduct formal clinical trials followed by sales to adopting clinicians. In other words, these individuals follow the second diffusion pathway shown in Fig. 1. For example, the cosmetic effects of Botox were serendipitously noticed by a patient who was being treated with that drug for an entirely different medical problem. The general value of this serendipitous discovery was then recognized by and acted upon by the patient’s doctors. They studied, understood and patented the novel, off-label application, and then worked with a pharmaceutical firm to conduct clinical trials and commercialize it as an FDA-approved, ‘on-label’ application (Carruthers and Carruthers 1992, Coondoo and Sengupta 2015).

4. The potential for market failure in peer-to-peer diffusion

Active efforts to diffuse information about a newly discovered off-label application of a drug via the peer-to-peer pathway can require the investment of significant amounts of time by the innovating clinicians. First, the evidence of efficacy that they present to colleagues will often need to be richer and more robust than the private evidence they may use to guide their own office practice. (Within their own office, they may simply say, ‘It is known to be a safe drug—I observed that it worked very well in the case of Patient X for the novel off-label application A—so trying it on Patient Y to treat a similar condition seems both safe and reasonable’.) Second, the clinician must diffuse this evidence for his innovation and its efficacy to peer clinicians via informal meetings, and/or via presentations at conferences and/or via publication in medical journals. Since, as was discussed in Section 1, innovation-related information is provided for free by innovating clinicians in the case of peer-to-peer diffusion, there is potential for a market failure with respect to diffusion of the innovation. (A market failure exists when another possible outcome can make a market participant better off without making someone else worse off (Krugman and Wells 2006)).

It should be noted that clinicians are motivated to develop an off-label innovation primarily by a wish to use it to help their own patients in their own medical practices. This benefit can be obtained without any investment in diffusing the innovation. They may also have other motivations, such as altruism or reputational benefits, that can reward diffusion. For example, reputational benefits are of interest to some, and obtaining these requires some degree of information diffusion (Lakhani and Wolf 2005, von Hippel 2016). But even in such a case, it is not clear that the type of diffusion incentivized is that of maximal clinical value to patients. For example, the enhancement of the medical reputation of the clinician—and thus his or her incentive to diffuse—may depend upon how technically interesting the new off-label application is, rather than on how clinically useful it is. Further, only some clinician innovators, perhaps those with academic appointments at teaching hospitals in addition to their clinical appointments, may be motivated to invest in diffusion to gain reputational rewards.

In other words, under the conditions just described, the benefits that practicing clinicians and their patients might obtain from adopting new off-label drug innovations may be mostly or entirely an externality for the clinicians who discovered the new applications. In turn, these circumstances create the possibility of a market failure with respect to the diffusion of user-developed innovations. That is, the innovators’ active diffusion efforts, incurred at some incremental costs beyond what is required to develop the innovation for themselves, can lower the combined costs of the innovator and follow-on adopters. However, due to the absence of a market or other mechanism to give clinician innovators a share of adopter benefits, their incentives to invest in diffusion may be lower than is socially desirable.

5. Previous studies of user innovation and diffusion

Clinician innovators who freely diffuse their innovations peer-to-peer are in a situation analogous to that facing individuals in the household sector who develop innovations for personal use (user innovators) and reveal them for free to potential adopters. User innovators are defined as those who develop an innovation in order to benefit from in-house use. Producer innovators are defined as those who develop an innovation in order to benefit from sale to others (von Hippel 2005).

National representative surveys of citizens above age 18 show that millions of individuals develop or modify consumer products to better serve their personal needs (see Table 1). These same studies report upon the fraction of user innovations that diffuse. In these surveys, diffused innovations may have only been transferred to one or a few adopters, or been diffused quite widely. Even given this encompassing definition of diffusion, one can see that the rate of diffusion of user-developed innovations does not exceed one out of four. In professional work contexts, empirical findings have not been dissimilar. Reported user innovation frequencies range from 15.3% in a sample of small and medium-sized enterprises (SMEs) in the UK (Flowers et al. 2010) to 54.0% in a sample of Dutch high-tech SMEs (de Jong and von Hippel 2009). Reported diffusion rates, however, did not exceed 25% in any of these studies.

It should be noted that, on their own, these figures for diffusion are not evidence for under-diffusion. Many of the reported user innovations are probably only of interest to the innovator. In such cases, non-diffusion may simply reflect a lack of adopter interest.

In a recent study of 176 individual end consumers in Finland, findings in line with a market failure in peer-to-peer diffusion were found in the case of user innovations that consumers develop to satisfy needs they face in everyday life (de Jong et al. 2015). In the case of peer-to-peer diffusion, by definition there was no market connection between innovators and free adopters. To assess investments made to support peer-to-peer diffusion among these innovators, the authors of the study asked whether the innovator had made any effort to diffuse his/her innovation, listing several different types of effort that might qualify, such as posting on the internet. Very little effort of any kind was mentioned by respondents. Accordingly, for analysis the definition of the presence of diffusion effort was set as a dichotomous variable with a very low threshold. Diffusion effort to peers was indicated if he/she simply had revealed the innovation to any other individuals.

The authors found a pattern of diffusion and diffusion effort that was in line with the presence of a market failure (see Table 2). First, there was no significant relationship between the likelihood of peer-to-peer diffusion and the general value of the innovation. Second, there was no significant relationship between the likelihood that innovators were making an effort to diffuse to peers (by revealing their innovation to other potential users) and the general value of the innovation. Both findings are in line with there being a market failure in the case of peer-to-peer diffusion. If the value of an innovation to others is entirely an externality for individual innovators, there is no reason why the diffusion effort should be correlated with the general value of the innovation (de Jong et al. 2015).

6. Research methods

To explore the extent and effectiveness of diffusion efforts made by clinicians who discovered a novel, off-label clinical use for an FDA-approved drug, we collaborated with Sermo, a commercial online community platform exclusively for physicians. At the time of our research (May 2014) Sermo had an enrolled membership of 270,000 US physicians, more than one-third of all physicians practicing in the USA. Sermo offers a range of services to its members, such as providing a channel to seek advice about difficult medical cases from peers nationwide. In addition, the site posts and records responses to surveys. Some polls are paid for by a commercial sponsor, such as a pharmaceutical firm, to address a question of interest to that firm. Other short surveys (which must contain 10 questions or less) are supported by Sermo at no charge when site managers think survey findings would be of interest to their clinician members. With this criterion in mind, Sermo allowed us to post, without charge, a short survey on the subject of possible off-label drug application discoveries by Sermo members. We agreed with Sermo that the survey would remain active until about 1,000 clinicians had responded.

An invitation to take our survey and a link to it was posted on the front page of the Sermo site from 5 May 2014 to 2 June 2014. During that period, 1,008 physicians at least began the survey. The key screener question at the start of our survey was: ‘Have you ever discovered one or more valuable off-label uses for FDA-approved drugs that, to your knowledge, was novel at the time you discovered it?’. If a respondent’s answer to this question was no, he or she was directed to the end of the survey and thanked. If their answer was yes, the remainder of the survey questions was presented to them. If respondents reported multiple off-label discoveries, we asked them to focus on their most recent case, as we assumed that this would produce the most accurate recollection of events related to their discoveries. Of the 1,008 physicians who began the survey, 382 claimed at least one off-label discovery during their career. Of these, 309 went on to complete the survey. These 309 individuals comprised our study sample.

Most of the innovating respondents (55%) specialized in internal medicine, with the remainder being distributed across a range of medical specialties. We found that 81% worked in a community-based practice, 14% in an academic hospital, and 5% worked in other clinical settings. The respondents who did (n = 309) and did not (n = 699) report an off-label discovery were not significantly different (assessed with χ²-tests) with respect to their medical specialty and the clinical setting in which they practiced. The questions we asked, the variables derived from them, and response frequencies are shown in Table 3.

Key variables in the analysis reported hereafter are the perceived general value of off-label discoveries, effort made to diffuse these discoveries, and observed diffusion. Perceived general value and diffusion observed were perception-based measures. As for diffusion effort, we recorded the four most common practices that American clinicians can engage in to spread their discoveries: presentations at local grand rounds or morbidity/mortality conferences, reports at regional professional meetings, reports at national professional meetings, and publication in medical journals.

We included a control variable if the respondent had communicated his/her discovery to local colleagues who were mostly in the same hospital. In contrast to the four diffusion activities described above, informing local colleagues is nearly costless, but is not generally perceived to be instrumental in accomplishing widespread diffusion of medical innovations. Other control variables were the context in which the clinician worked (assuming academic clinicians are more likely to diffuse) and the time passed since the discovery was made (assuming that the older the discovery, the more likely it will have spread). Finally, we collected data on a range of diffusion motives to more deeply analyze why clinicians would refrain from diffusing their off-label discoveries.

7. Analysis and findings

In order to investigate the potential market failure in a professional work context, we analyzed the determinants of the degree of diffusion effort invested by clinician-discovers, and the extent of diffusion observed for their freely revealed off-label discoveries. The key variable in our analyses is the perceived general value of the off-label discoveries claimed. From a social welfare perspective, diffusion effort and diffusion are only economically desirable when innovations have value to adopters beyond the initial developer. To empirically explore if market failure with respect to the diffusion of freely revealed innovations occurs, three conditions should be met:

• First, in line with our argument that diffusion is mostly an externality to freely revealing innovators, we anticipate that these individuals will invest little effort to diffuse their innovations.

• Second, and for the same reason, we expect to find that the level of diffusion effort made by clinicians discovering new uses for FDA-approved drugs will not increase along with the perceived general value of their off-label discoveries.

• Third, to document that a lack of diffusion effort is of consequence with respect to social welfare, we expect to find that diffusion effort is positively related to diffusion levels attained, especially in the case of generally valuable innovations.

7.1 Descriptive analysis

Table 4 offers key descriptive statistics for the variables employed in our analyses. It can be seen that the average discoverer invested little effort to inform others. The average number of diffusion activities was 0.27 out of four possible activities asked about: participation in local grand rounds or morbidity/mortality conferences, presenting at regional professional meetings, presenting at national professional meetings, and publication in medical journals. This finding is in line with the first condition stated above. Only the nearly costless effort of informing local colleagues was practiced more often (by 66%), but as our subsequent analysis will show, informing local medical colleagues did not significantly affect diffusion.

Table 4 also shows that diffusion effort did not increase along with the level of clinical value clinicians saw in their discoveries (r = 0.032, p >.10). More specifically, for discoveries deemed to be of minor, moderate and major importance, the average number of diffusion activities was 0.27, 0.24 and 0.33, respectively. This is exactly what we would expect to see if innovators view the potential benefits accruing to adopters as an externality.

Table 4 also shows the significant positive relationship between perceived general value of an off-label discovery and diffusion attained (r = 0.305, p <.01). It should be noted that this significant relationship does not imply that market failure is absent. First, some off-label discoveries may well be made simultaneously by multiple clinicians, while treating patients with similar diseases. Proactive diffusion effort would imply that their discoveries would have spread more quickly. Moreover, the significant relationship does not imply that diffusion reaches its full potential. Indeed, a follow-up analysis (available on request) showed that the extent of diffusion to ‘many’ or ‘nearly all’ clinicians was 48% for discoveries of major importance—so about half of these discoveries had not yet diffused. To more thoroughly analyze the second and third conditions of market failure outlined above, we next estimate a range of regression models.

7.2 Regression models

Regression models allow us develop and explore our data in detail (see Table 5). All models included as control variables whether the respondent worked in an academic setting, the date of the claimed discovery, and whether the respondent engaged in the costless diffusion activity of informing his/her local colleagues. In addition the four models also include several specific motives (stated in the negative as possible inhibitors of diffusion effort) that might reduce innovating clinicians’ diffusion effort. All models had a significant fit at the 0.1% level. As the (multiplicative) parameters of ordinal logit regressions (models I and II) and negative binomial regressions (III and IV) are difficult to interpret, we also provide marginal effect parameters. These can be found in the Appendix (Tables A.1 and A.2).

Model I shows that diffusion effort is only weakly related to diffusion observed for all innovations taken together (b = 0.355, p <.10). However, as we see in model II, there is a significant positive interaction between perceived general value and diffusion effort (b = 0.505, p <.05). After interpreting the marginal effect parameters (bottom rows of Table A.1), we learned that diffusion effort is significantly associated with diffusion only for innovations of moderate and (especially) major general value. Effort exerted to diffuse innovations of minor importance had no significant effect on diffusion attained. This makes sense as diffusion can only be expected if there is some value to others. The finding in model II is in line with the third condition indicating market failure that we outlined earlier.

Model III confirms our initial finding that diffusion effort is unrelated to the perceived general value of off-label discoveries (b = 0.14, p >.10). Respondents were as likely to put effort into informing others about minor discoveries as they were about major ones. This confirms the second condition that we formulated above.

Model III showed that academic clinicians were more likely to put effort into diffusion, engaging in 0.457 more diffusion activities than non-academics on average (see Table A.2). We therefore followed up with an exploratory analysis to investigate any differences between academic and non-academic clinicians. In model IV, we found that the extra effort by academics was directed to discoveries of minor and moderate general clinical value. (Negative interaction coefficient between perceived general value and the dummy for academic respondents of b = −0.941, p <.05). The bottom rows of Table A.2 show that higher diffusion effort by academics relative to other physicians is focused on efforts to diffuse information regarding minor (dy/dx = 0.838, p <.05) and moderate discoveries (dy/dx = 0.328, p <.05) while for major discoveries the difference with non-academics was negligible.

As we noted earlier, effort devoted to publishing in medical journals or presenting innovations at medical conferences has no impact on diffusion in the case of innovations of minor clinical importance: beyond awareness, potential adopters need to see potential clinical value before they will adopt. Academic clinicians may know this. However, it is likely that clinicians working in academic settings, to a greater degree than community-based clinicians, have reputation-building reasons to invest in diffusing information in professional fora. Off-label discoveries can also be of interest to academics for their scientific interest as well as their clinical value.

Finally, we note our findings regarding reasons given for not exerting diffusion effort within currently available diffusion channels, and discoverers’ stated likelihood of participating in alternate diffusion channels which we proposed (see Table 4). Reasons significantly associated with low diffusion (see model I) were inaccessible channels (‘I am not likely to get access to medical journals/conferences to report my findings’) and unwillingness to invest personal time. Both of these reasons are, in effect, statements that the effort required to diffuse a discovery is not worth the personal cost. Again, this is what we would expect if there is a market failure with respect to diffusion. In line with this finding, discoverers mainly said (see Table 4) that they were significantly more likely to use diffusion channels that demanded less effort from them, with 71% saying that they were likely to use a process of reporting to an online physician-only community at a cost of 30 minutes. However, academic clinicians differed in this regard. In a follow-up descriptive analysis we found that academics expressed a slightly higher preference than community-based clinicians for diffusion via medical conferences and publications, even though these involve much higher time investments (output available on request). As mentioned earlier, it seems likely that conference presentations and professional publications offer reputational gains to academic clinicians that offset their higher cost.

8. Discussion

Market failure exists when individuals’ or organizations’ pursuit of pure self-interest leads to results that are not efficient—that can be improved upon from the societal point of view. To the extent that freely revealed innovations are generally useful, their diffusion enhances social welfare. However, due to lack of incentives, diffusion is an externality from the free innovator’s point of view. Market failure with regard to lack of diffusion of freely revealed innovations has been demonstrated empirically in a sample of individual end consumers (de Jong et al. 2015).

In this paper we analyzed if this market failure hypothesis could also be encountered in professional work contexts where some types of innovations also are made available ‘for free,’ and where professionals are not directly compensated for devoting effort to diffusion of those innovations. Drawing on survey data of 309 clinicians in the USA, we documented three conditions indicative of market failure. First, in line with our argument that diffusion is mostly an externality to individual innovators who freely reveal, we found that clinicians invested little in efforts to diffuse their off-label discoveries. Second, we found that their level of diffusion effort was not related to the perceived general value of their innovations: those were independent of the perceived level of benefit for potential adopters. Finally, to document that a lack of diffusion effort due to market failure is of consequence with respect to social welfare, we found that diffusion effort by innovators increased diffusion when their off-label discoveries were generally valuable.

The investment of personal uncompensated time required of clinician innovators to carry out any of the four types of innovation diffusion modes we considered, were only in the range of tens to a few hundred hours. Drawing on responses to our ‘uncompensated time’ question, we find that publication is the most costly diffusion channel for clinicians from an uncompensated time perspective. Respondents who reported publishing their off-label discovery had on average spent 183 hours to accomplish that.

We find that a relatively minor increase in investment in diffusion effort will produce a major increase in diffusion accomplished. Thus, in the case of off-label discoveries of major clinical importance, our marginal effect estimates indicate that an increase in engaging in only one additional diffusion task (out of the four we measured), is associated with an increase in diffusion accomplished to ‘many’ or ‘nearly all’ other clinicians of 20.3% (see Appendix, Table A.1). It seems clear that the associated increase in social benefit would outweigh this relatively minor amount of additional personal investments by innovators. But, as we have argued, because benefit to adopters is an externality to innovators, these additional welfare-increasing private investments are not made today. Thus, we conclude that the market failure with regard to the peer-to-peer diffusion of user innovations exists in a professional work context of off-label drugs discoveries by clinicians, and has a negative impact on net social welfare.

In the remainder of this discussion, we first consider how general this market failure is likely to be with respect to the diffusion of freely revealed innovations. Second, we consider policy implications, and offer suggestions for future research.

8.1 Factors ameliorating market failure

In our study we explored a professional work context where user innovations could only be diffused peer-to-peer. In the USA, government regulations prohibit pharmaceutical firms from promoting off-label uses of drugs they sell that have not been subjected to clinical trials and approved by the FDA. In this and similar cases where innovators do not seek commercial returns, whether from the innovation itself or from commercially sold complements, market failure will exist by construction: there is no market involved that could incentivize freely revealing innovators to invest in diffusion so that a portion of the adopters’ benefits would transfer to them. However, there are factors present in real-world conditions that can partially or entirely offset this market failure. These include costless and low-cost diffusion, and offsetting investments by adopters.

With respect to costless and low-cost diffusion, there is usually an amount of diffusion of innovation-related information that will occur without cost to a freely revealing innovator. Market failure applies to investments to diffuse information beyond what can be diffused without cost. For example, a biker may at no cost diffuse some information about a mountain biking innovation simply by riding it openly in a race where some information will be observable to other participating bikers—the innovation is partially ‘self-revealing’ (Strandburg 2004, 2008). As a second example, an innovator may at trivial cost freely reveal the software she developed for herself by posting her code on a website ‘as is’. Clearly, however, additional information would be useful to potential adopters in both these cases. That is, an adopters’ costs to replicate a briefly glimpsed biking innovation would be greatly reduced if the innovator put effort into creating and diffusing a detailed set of engineering drawings—drawings that the innovator might not need for his own in-house development activities. Similarly, adopters of software code might benefit from additional written documentation that the user, having created the code and intimately knowing what is in it, would not need for herself.

With respect to motives for, and sources of, investments that can offset innovators’ investment shortfalls with respect to their freely revealed innovations, recall the diffusion channels accessible to user innovators shown in Fig. 1. The market failure we are discussing occurs when individuals diffuse their information via free revealing, with potential adopters being peers and/or producers. As was shown in Fig. 1, there are other options for diffusion and also investment possibilities by adopters that can offset this market failure. First, individual innovators can choose to sell their information rather than freely revealing it, thus gaining access to a share of adopter benefits via a market transaction. This can happen when they sell their innovations to existing firms or start their own firms to commercialize their innovations.

It is also possible for adopters to invest in obtaining and enhancing freely revealed innovation information in addition to, or instead of, diffusion investments made by the innovator him or herself. In the case of multiple adopters each making similar adoption investments and keeping what they have learned private, this pattern is less socially efficient than the freely revealing innovator making such an investment usable by all. However, this inefficiency can be avoided, and often is in open source communities, when a non-innovating individual invests in improving documentation as a contribution to the community, and reveals that contribution to all. Inefficiency is also avoided when a producer makes this investment, spreading the costs of obtaining information from a individual innovator over all product purchasers.

8.2 Implications and suggestions

Despite the ameliorating factors mentioned above, there remain situations where the type of market failure we explore in this paper will result in underinvestment by freely revealing innovators in diffusion of their innovations from the perspective of social welfare. Major cases of this type are innovations that, while generally valuable, only diffuse peer-to-peer (de Jong et al. 2015). Thus, due to regulatory cost considerations, the diffusion of off-label applications of FDA-approved drugs that was the subject of study in this paper are generally not diffused commercially. Thus, peer-to-peer diffusion depends on the efforts of clinician-developers. Many techniques essential to practice in fields from surgery to mountain biking are also not diffused commercially, in this case because ‘competing with free’ peer-to-peer diffusion is unprofitable for commercializers (von Hippel 2016). Again, investment in peer-to-peer diffusion (likely to be underinvestment) is left to the innovators themselves and/or community-minded peers and adopters.

In such cases, we argue, it is appropriate to address the market failure via public policy, in order to increase social welfare. Consider the following analogy. Nations around the world go to major expense to provide a combination of R&D subsidies and intellectual property rights (IPR) protection to innovation developers. They do this to offset a market failure: information developed at private cost may spill over from innovators to imitators. Without subsidy and/or monopoly protection from imitators, it is reasoned that innovators’ returns from their innovation investments will be negatively affected, and the amount of innovation will accordingly drop to levels below a social optimum. Indeed, even today there is an impetus in public policy in many countries to strengthen IPR in order to foster innovation based upon this reasoning (Blaxill and Eckardt 2009).

Traditionally, and in contrast, policy measures are seldom applied to support innovation diffusion after development. This is reasonable for producer innovators who seek to profit from selling their innovations—there is no market failure with respect to diffusion. However, as was discussed in our literature review, it is now understood that individuals, in addition to producers, are a major source of innovations that are of great value to the economy. Modeling has also shown that an economy in which user-developed innovations diffused peer-to-peer both compete with and complement producer offerings with resultant increases in social welfare (Gambardella et al. 2016). Given that user innovation enhances social welfare, an important question for policy-makers immediately emerges: Are government policies currently at least even-handed with respect to freely revealed innovations and producer innovation and diffusion? Or do they on balance tilt towards supporting producer innovation? We suspect the latter is the case. In this light, it becomes important as a matter of equity as well as social welfare improvement to explore for market failures with respect to this newly assessed source of innovation, and to respond to those discovered.

How can this be done? Potential solutions are often easy to identify, because existing diffusion mechanisms were not designed with the diffusion of free innovations in mind, and so are visibly and grossly defective for that purpose. Both general and situation specific remedies are possible. Generic remedies could usefully be linked to the magnitude of benefit potential adopters of freely revealed innovations would experience. An example would be a subsidy based upon demonstration of diffusion expenditure, or a prize that is awarded after a period of time based upon diffusion benefits realized by adopters. Such a prize could be general, or could be restricted to specific fields of high social value and high levels of freely revealed innovation like medicine. For this approach to be effective in selectively motivating diffusion efforts in the case of innovations that are in fact of general value, the innovator should have access to knowledge regarding the likely extent of demand for his or her innovation. This is inherently likely to be the case. For example, individual user innovators have better insights into the potential demand in their own fields from similar users than do many others. However, it also could be enhanced by even better information. For example, part of the subsidy or prize infrastructure could be services to assist freely revealing innovators to estimate likely demand based on the field of application of their innovation and other data.

Situation-specific remedies could be addressed to specific fields where social welfare gains are projected to be especially important—as in the case of diffusion of off-label drug discoveries. Clinician users are a logical source for valuable discoveries in that field. As Brodell (2000: 1192) notes:

In this era of outcome studies and evidence-based medicine, the value of case reports, physician intuition, and serendipity is often overlooked. All science is rooted in observations, and full-time clinicians are in an ideal position to observe unusual cases, develop rational explanations for the findings, and follow progress to determine if their hypotheses appear to be valid.

In addition, diffusion of discoveries at no cost to peers is in line with norms in the medical field. Indeed, historical roots going back to the Hippocratic tradition encourage physicians to pass on what they know to other physicians (Huth 1985). In our questionnaire, we offered four diffusion channel options to clinician discoverers, and found that 71% of them would be willing to contribute to an online reporting site at a cost to themselves of 30 minutes. This is more than twice as many as would be willing to contribute via other diffusion channels (see Table 2). As is the case with professional refereeing applied to articles submitted to medical journals, such a site could provide tools for peer-to-peer evaluation of the clinical and scientific value of off-label discoveries along with information diffusion. Public support for the development of such a site would be an example of a specific remedy to diffusion incentive shortfalls in a specific, socially important field.

Apart from policy, there are some new research opportunities emerging from this study and other studies of market failure in the diffusion of freely revealed innovations. First, we would encourage investigations of the circumstances in which freely revealing individual innovators do put effort into diffusion, that is, empirical analyses of the factors that alleviate the market failure. Second, we suggest that empirical contributions should include other types of professional work contexts in which commercial diffusion is not a viable option, as in the case of technique innovations developed by users of medical devices (Hinsch et al. 2014). More generally, the problem of market failure in relation to incentives for freely revealing innovators to diffuse is an interesting topic with respect to studies of general diffusion mechanisms, and to studies of mechanisms for and barriers to diffusion in specific areas. We look forward to further explorations.

References

Appendix