The Ways We’ve been Measuring Patent Scope are Wrong

The Ways We’ve been Measuring Patent Scope are Wrong: How to Measure and Draw Causal Inferences with Patent Scope Abstract According to surveys, the top 10% of patents are worth more than a thousand times as much as the bottom 10%. This isn’t surprising since a patent’s value derives from its ability to exclude rival products from the market, and patents vary widely in their ability to do this. At the same time as there is great variation in the value of patents that do issue, there is almost no variation between patents that are ‘just’ worth filing and those that aren’t – both are nearly worthless. This presents a challenge for innovation scholars, because we have much better tools for measuring whether or not something is patented than we do for how much scope a patent protects. Thus our empirical tools are weakest precisely where they are most important. This paper presents an easy-to-use measure of patent scope that is grounded both in patent law and in the practices of patent attorneys. Our measure counts the number of words in the patents’ first claim. The longer the first claim, the less scope a patent has. This is because a longer claim has more details – and all those details must be met for another invention to be infringing. Hence, the more details there are in the patent, the greater are the opportunities for others to invent around it. We validate our measure by showing both that patent attorneys’ subjective assessments of scope agree with our estimates, and that the behaviour of patenters is consistent with it. Our validation exercise also allows us to examine the performance of previous measures of patent scope: the number of patent classes, the number of citations made by future patents, and the number of claims in a patent. We find them all to be uninformative (no useful correlation with scope) or misleading (negative correlation with scope). To facilitate drawing causal inferences with our measure, we show how it can be used to create an instrumental variable, patent examiner Scope Toughness, which we also validate. We then demonstrate the power of this instrument by examining standardessential patents. We show that an (exogenous) diminishment of patent scope leads to patents being much less likely to be declared standard-essential. I. Introduction "Broad patent claims are similar to a property fence; the bigger the scope of the claims, the more intellectual property is covered by the claims” - ARC Intellectual Property Law blog This paper provides two tools for patent scholars to help them study patents and their value. The first is a measure of patent scope, the extent of the legal coverage that a patent provides. The second is an econometric instrument, a measure of a patent examiner’s scope toughness on the extent of scope given to a patent, which allows for causal inference on the effect of getting more or less patent scope. To ensure that our measures are capturing the phenomena that we intend, we ground them in (i) the details of patent law, (ii) the rules of the U.S. Patent and Trademark Office (USPTO), and (iii) the practice of attorneys prosecuting patents. We also do extensive validation work to show that our measures accord with the views of practitioner experts and with the observed behavior of patent filers. Next, we compare our measure with other approaches for measuring patent scope suggested in the literature. Finally, we provide an example of how our instrument can be used, showing that patents with (exogenously) narrower scope are less likely to be declared standards-essential. Patents are valuable because of what they protect. If they can exclude competitors from a valuable market or from important technologies, they can generate greater profit margins or faster growth for the patent holder, becoming extremely valuable. In contrast, if they cannot restrict competitors, their value can be negligible. The enormous range of patent values can be seen in Figure 1, based on a survey of patent holders by Gambardella, Harhoff, and Verspagen (2008). --------Insert Figure 1 about here-------The ability of a patent to exclude others arises from a particular part of a patent: the claims. “Patent claims define the scope of patent protection. Whether a claim covers a product accused of infringement frequently determines whether a patent holder will be rewarded or left with nothing." (DiPietrantonio, 2011). If those claims are to be valuable then they must be sufficiently broad – that is, they must not only stop a competitor from directly copying the invention but also prevent that competitor from producing a readily substitutable variation of the invention. For example, a patent on an engine would be worth little if a competitor could escape that protection by making the same engine in a different color. Patent attorney Thomas Kulaga explains: “Broad patents typically offer more protection against infringers than a narrow patent” because they are “oftentimes very hard to design around”. (Kulaga). We show that the breadth of a patent’s scope can be measured by counting the number of words in its first claim2, with more words corresponding to less scope.3 This relationship holds because a competitor’s offering must meet every condition of a claim in order to infringe it, so a longer claim implies more conditions that must be met for a patent to be violated. For example, a patent on an “engine” would be broader than a patent on a “car engine” and both would be broader than a patent on a “200 horsepower car engine.” This correspondence between claim length and patent scope is widely acknowledged by practitioners, who report that “longer claims are necessarily more narrow.” (Quinn, 2015) Ours is not the first attempt to measure patent scope. Common alternatives have included (1) the number of patent classes to which a patent is assigned, (2) the number of citations from future patents to that one, and (3) the number of claims listed on the patent. We test our measure, and these alternatives, against the judgment of patent attorneys as to the scope of patent coverage. We find that our measure is highly predictive (p < 0.001) and explains nearly half of all the variation in patent scope, as judged by experts. In contrast, we find that the number of patent classes assigned to a patent is negatively correlated with patent scope, the reverse of what is typically assumed. We find that the number of citations to a patent is only incredibly weakly (and statistically insignificantly) related to patent scope, and that in practice it only explains 0.04% of the variance in patent scope, as judged by experts. We do find that the number of claims in a patent is predictive of patent scope, but that this relationship is much weaker than that of our measure and only explains 7% of the variation in patent scope. From these findings we conclude that our measure is highly predictive of patent scope, and that traditional measures are either misleading or lack explanatory power. Inspired by the work of Sampat and Williams (2015) we also use our measure to construct an econometric instrument, scope toughness. This reflects how tough a patent examiner at the USPTO is on giving scope when evaluating their patents. We construct our instrument in such a way that it is exogenous to the patent being evaluated, and therefore can be used for causal inference insomuch as patent examiner assignment is random (see Sampat and Williams (2015) and Righi and Simcoe (2017) for discussions of this). Importantly, our measure of scope toughness is notably different from the measure introduced by Sampat and Williams (2015). Their measure is based on the likelihood of patents being granted at all. In cases where technology is very discrete, such as gene patenting where they apply it, we believe 2 3 Technically we are interested in the first independent claim, although by law the first claim in a patent must be independent. Concurrent work done independently by Marco, Sarnoff and deGrazia (Working Paper) has also proposed a related measure. that theirs is a good measure. However, we do not find it compelling for more general application because, as Figure 1 shows, a large number of patents have very little value. If one examiner is tougher than another, it is presumably because they are disallowing more of these low-value patents. This may cause large differences in the number of patents that are allowed, but it will reflect only a very small share of the value being protected by the patent system, certainly less than 5% and perhaps more like 1-2%.4,5 More important for the operation of the patent system is how patent examiners treat valuable patents. These are patents with little to no danger of not being awarded a patent of some sort, but where differences in scope can vary their value enormously. Our measure captures this margin – and we show that, as might be expected, patent examiner behavior on patents of little value is notably different than their behavior on valuable ones. Finally, we provide an example of the usage of our scope toughness instrument. We examine how exogenous changes in patent scope that arise from random assignment of patent examiners affect the likelihood of patents being declared standards essential. We find that a one standard deviation increase in examiner toughness is associated with a 7.1%*** lower probability of a patent being declared standard-essential. The purpose of our paper is provide useful tools for patent scholars that are grounded in the details of the legality and practice of patent prosecution and that are properly validated with experts. The following sections lay out these arguments and provide the empirical support for them. II. Measuring patent scope A. Previous work Scholars have long recognized the importance of patent scope. (Chang, 1995; Cohen & Lemley, 2001; Gilbert & Shapiro, 1990; Green & Scotchmer, 1995; Klemperer, 1990; Lerner, 1994; Merges & Nelson, 1990; Scotchmer, 1996). As noted by Merges and Nelson (1990), “The economic significance of a patent depends on its scope: the broader the scope, the larger the number of competing products and processes that will infringe the patent.” However, empirical analysis of the effects of patent scope presents significant measurement challenges due to the difficulties involved in accurately quantifying such an amorphous and idiosyncratic construct. 4 5 Based on calculations by the authors on the Gambardella et al data. Sampat and Williams (2015)’s finding of very little effect on downstream outcomes is consistent with this. An ideal measure of patent scope would accurately measure the extent of the legal right to exclude afforded by a patent. It would also be both validated and easy to use without being a subject matter expert. Further, it would be computable ex ante at the time a patent is issued. Finally, such a measure would facilitate both performing cross-patent comparisons and drawing causal inferences about the effects of patent scope. Scholars have proposed various measures of patent scope, including counting the number of patent citations received by a patent, the number of claims in a patent, or the number of distinct International Patenting Classification (IPC) subclasses to which a patent is assigned. (Lerner, 1994). Our analysis shows that all of these measures are problematic. The legal basis of patent citations does not support its use as a direct measure patent scope. Patent citations are selected by patent applicants and examiners ostensibly based on the content of the cited patent’s description of the invention (the specification) rather than the cited patent’s claims (the protection provided by the patent). Therefore, a well-described patent may garner more citations, irrespective of patent scope. Moreover, patent citations present numerous empirical challenges as a measure of claim scope. First, citations are accrued over time and are thus unavailable for computing a measure of patent scope immediately after a patent issues. Second, citations are subject to numerous selection effects specific to firms, technologies, patent families, and time. (Kuhn, Younge, & Marco, 2016; Marco, 2007). Third, patent citations have also been used to measure constructs as diverse as private patent value, knowledge flows, and technological impact, and using them to measure patent scope risks conflating scope with these other constructs. This conflation is particularly problematic for causal inference because it means that any regression of patent citations on commercial outcomes would need to control for all these other effects to get an unbiased estimate. Since many of these factors are codetermined (e.g. market growth leading to more company investment and more competitors citing the patent), such controls will almost always be insufficient. The number of claims fares no better as a measure of patent scope. A patent’s legal right to exclude is defined by its broadest claims, which are referred to as “independent” claims. If a patent includes more than a handful of independent claims, or if the independent claims differ substantially in their scope, the patent examiner will typically force the patent applicant to split the patent application by filing a “divisional application” on the theory that substantially different claims should be subject to distinct examination processes. Most claims in a typical patent are “dependent” claims. These serve as fallback positions in the event that an independent claim is ruled invalid, but by law each dependent claim must be narrower than the independent claim on which it depends. Because applicants pay extra for each claim beyond 20, patents with many claims may cover more valuable technology but are not necessarily any broader than patents with fewer claims. A more nuanced measure of patent scope proposed in the literature is the number of IPC classes to which a patent is assigned. (Lerner, 1994). The logic behind this measure is that patents that relate to many different technologies are broader than those that are specific to a more limited range of technology. However, a patent may be assigned to many IPC classes if it lies at a technological border, even if it represents only an incremental improvement on prior approaches and thereby affords only a limited legal right to exclude. Further, patent classes are complex, discrete, and vary in granularity across technology. For these and other reasons, the literature includes substantial debate about their use in empirical applications. (Younge & Kuhn, 2015). In sum, none of these measures accurately captures the scope of the legal right to exclude afforded by a patent’s claims. Each of the proposed measures is also subject to selection effects that render the measures unsuitable to drawing causal inferences. Finally, to the knowledge of the authors, none of the proposed measures have been validated against expert evaluations. Later in the paper, we demonstrate that each of these measures also correlates poorly with our validated measure of patent scope. B. Perspective of a filer The legal right to exclude afforded by a patent is defined by its claims. “Each claim is in a very concise single-sentence format.” (Yan, 2013). Under the “every element” test, a competing product or process must exhibit every element included in a patent claim in order to infringe that claim. For this reason, “[v]irtually every word in a claim is important.” (Yan, 2013). Patents directed to similar technology can vary considerably in claim scope. According to one practitioner, “Patent claims can be ‘broad’—protecting a whole concept that would be difficult for someone else to duplicate without infringing your patent, or ‘narrow’—protecting only a very specific configuration of your invention.” (Goldstein). Another practitioner notes that "[a] broader claim is usually more valuable than a narrower claim. In the context of patent claims, ‘broader’ means ‘reciting fewer features’.” (Funnell, 2014). This is because infringers must meet every condition of the patent to be infringing—the more conditions, the more room there is to invent around. “Every patent practitioner knows that shorter claims are usually broader than long claims. This can be stated as a ‘rule of thumb:’ If your claim is longer than your thumb, it is too easy for an infringer to get around it.” (Beem, 2015). Under U.S. patent law, the broadest claim, which is necessarily an independent claim, should be presented first.6 Subsequent independent claims are thus typically either more narrow or identical in scope. For instance, a first independent claim may cover a method, while a later independent claim may encompass a computer that performs the method. If a subsequent independent claim differs significantly in scope from the first independent claim, the patent examiner typically requires the applicant to pursue those different claims in different patent applications.7 In addition to these independent claims, a patent may include some number of dependent claims that serve as fallback positions and which, by law, must be narrower than their corresponding independent claims.8 For these reasons, in this article we focus on the first independent claim as providing the best indication of a patent’s scope.9 A patent’s scope is defined by a back-and-forth process known as patent examination. An applicant files a patent with a full technical description and an initial set of claims. The patent is then sorted into an appropriate examining division at the USPTO based on its technology and assigned to an examiner. Exactly which examiner is assigned to which patent and how this happens is an important question, which we revisit in Section V.C. But once an examiner is assigned, they search the prior art to identify reasons to reject the patent, such as the claims reciting a concept that is either not new or is obvious in view of what is already known. A patent applicant whose claims are rejected cannot add any new matter to the technical description of the patent application but can amend the claims (typically by narrowing them) to overcome the rejection. A patent examiner can never finally reject an application—only the applicant can decide to stop pursing a patent application. Most patent applications (about 80%) are initially rejected, but most patent applications (again about 80%) ultimately issue as patents. As we will show, the patent examination process causes the claims of most patent applications to be narrowed considerably between filing and issuance. Further, as noted by Cockburn, Kortum, and Stern (2002), Sampat and Williams (2015), and others, examiners vary considerably in their characteristics, which provides variation useful for constructing a causal instrument. 37 C.F.R. 1.75(g). 35 U.S.C. § 121. 8 35 U.S.C. § 112(d). 9 Alan Marco, Joshua Sarnoff and Charlie deGrazia have pursued an alternate calculation that focuses on the shortest independent claim. We do not adopt this definition in part because we judge that in most situations it will be equivalent but also because it adds considerable complexity to empirical analysis. For example, in this article we compare the scope before and after patent examination. However, because claims can be deleted or re-ordered during examination, the, the shortest claim at filing may be completely different from the shortest claim at issuance, whereas the first claim at issuance is likely to be an amended version of the first claim at filing. Accordingly, focusing on the first claim allows for more straightforward analysis with more interpretable results. 6 7 C. Our measure We measure patent scope using the number of words in the first independent claim of a patent. This measure provides several advantages. From an institutional standpoint, it is grounded in both the legal impact of patent claims and anecdotal evidence from practitioners about how patent claims are drafted. From an empirical standpoint, it is clearly observable both at the time a patent application is filed and when the resulting patent is issued. Although concurrent, independent work has proposed claim length as a measure of patent scope (see, e.g., (Harhoff, 2016); Marco, Sarnoff, and deGrazia (2016)10), to our knowledge it has neither been validated against expert judgments nor employed for causal inference. We will present evidence in Section IV that claim length accurately reflects the opinion of expert reviewers regarding claim scope. Further, as will be discussed in Section V, variation in patent examiner characteristics can be used to support causal inferences regarding the effects of patent scope. Another advantage of claim length is that it is a characteristic shared by all patents. However, different technologies are described differently. Further, institutional details arising from administrative organizational structure may give rise to further between-technology differences in claim structure. To parcel out these differences, we normalize claim length at the level of the USPTO examining division, known as an “art unit”. The resulting measure is standardized in the sense that a patent having a claim scope value of 1 is one standard deviation narrower than other patents in the same area of technology. III. Data A. Data Sources The data for this project comes from four principal sources. First, to observe the changes made to patent claims during their prosecution, we collect and parse the text of both the published U.S. patent application and the issued U.S. patent for each of 947,191 patents. Second, to observe various other characteristics of these patents relevant to our analysis, we link each patent with bibliographic data extracted from metadata XML files published by the USPTO. Third, for our validation exercise, we find seven patent attorneys who generously agreed to volunteer their time, give them 140 randomly selected U.S. patents (in total) to review, and have them complete a questionnaire for each patent they review 10 The measure noted by Harhoff (2016) provides very similar results as ours, but with a slightly different specification. reporting on the patent’s scope relative to other patents in the same area of technology. Fourth, we rely on firm disambiguation data provided by Hanley (2015) to link each patent with a firm identifier. We link each patent with claims data from the USPTO patent claims dataset and bibliographic data extracted from bulk data documents published by the USPTO. The bulk data files contain a wealth of information about each patent such as the name of the patent examiner who examined the patent and the dates on which a patent was filed, published, and issued. We partition these data based on the patent examining division of the USPTO. Patents are divided into eight administrative units, known as Patent Technology Centers, responsible for examining utility patent applications. The technology centers correspond to broad areas of technology, such as “Communications” and “Semiconductors” and help to organize patent examiners into logical units. The technology centers include: 1. Biotechnology and Organic Chemistry 2. Chemical and Materials Engineering 3. Computer Architecture, Software, and Information Security 4. Computer Networks, Multiplex Communication, Video Distribution, and Security 5. Communications 6. Semiconductors, Electrical and Optical Systems and Components 7. Transportation, Construction, Electronic Commerce, Agriculture, National Security and License & Review 8. Mechanical Engineering, Manufacturing, Products Each technology center includes subdivisions, known as “art units”, which focus on a more-granular technology, such as “Internal Combustion Engines” and “Electrical Heating.” Together, the eight technology centers include ~450 art units. Each art unit includes between 1 and 83 patent examiners. Although U.S. law mandates that patent applications are initially owned by their inventors, most patent applications are assigned by the inventors to a firm prior to the issuance of a patent. For each patent that is so assigned prior to issuance we link each patent to a firm identifier via firm disambiguation data provided by Hanley (2015). Firms are commonly listed in assignment documents as a company name in text, but the same company may appear in different ways across different patent documents due to typographical errors and other inconsistencies. The firm disambiguation data resolves these inconsistencies and assigns each firm a distinct identifier. Finally, in early 2017 we collected data from legal experts by conducting a survey of seven patent attorneys within a law firm specializing in the drafting and examination of patent applications. We selected a random sample of patents stratified on changes to patent scope according to our measure. We then asked each attorney to rate using a Likert scale the scope of each patent as a whole at the time of filing, at the time of issuance, and as a change between filing and issuance. B. Data Coverage We begin by selecting all patents issued between January 1, 2005 and December 31, 2012, inclusive, that were filed on or after January 1, 2001. We restrict the sample to patents issued in 2005 or later to take advantage of the improved bibliographic data available from the USPTO for patents issued on or after that year. We also limit the data to patents issued by 2012 in order to reduce rightward truncation bias in certain outcome measures. Virtually every patent issued in this range is included in the data set, with three exceptions. First, we do not observe claim scope data for patent applications that were not published prior to issuance. Patents whose applications were unpublished include those filed before January 1, 2001. Patent applications filed before that date were not published prior to issuance, and observing the claims of these patents at the time of filing is not possible using our current methodology. However, a change to U.S. patent law caused patents filed in 2001 or later to be published by default 18 months after filing. The claims in each published patent match those submitted with the application prior to beginning patent examination. Patents whose applications were unpublished include those where the patent applicant opted out of pre-grant publication. U.S. law allows patent applicants to make this choice provided that the applicant pays a fee and agrees not to file the patent application in foreign jurisdictions. In practice, about 10% of issued patents are not published prior to issuance; we have nothing to say about these patents because we do not observe their claims at filing. Second, we exclude from our sample each patent that claims priority to an earlier-filed patent application. Each of these “continuing” patent applications is part of an ongoing interaction between the patent applicant and the patent office. A continuing patent application is normally assigned to the same patent examiner who examined its parent. At the same time, the patent applicant can use information gleaned from the examination of the parent application to formulate the claims of the continuing application. Thus, our assumption that patent applicants are quasi-randomly assigned to patent examiners would not hold for this set of patent applications. Third, we exclude from our main sample patents examined by the biotechnology technology center because of the way language is used in their claims. Patents examined by art units within the USPTO’s biotechnology center are the most likely to use Markush language, where lists are used to make a patent broader, e.g. “a compound consisting of drug A and a drug selected from the group consisting of: drug B, drug C, and drug D”.11 Amending a Markush claim to eliminate one of these alternatives will narrow and shorten the claim, reversing the normal relationship between claim length and scope, as discussed in Section 2.1 above. Moreover, biotechnology patent claims use sequence numbers to incorporate by reference genetic sequences listed in the technical description of the application and other idiosyncratic claim drafting phenomena that can confound measurements of claim length. The original data set includes all patents for which we observe claim scope data, which virtually all patents issued from 2005 to 2012 that are published prior to issuance. After imposing the restrictions described above, we are left with 1,061,863 patents remaining in the final data set. C. Summary Statistics Table 1 provides summary statistics for the words present in the first independent claim. The data shows that patent claims tend to lengthen considerably between filing and issuance. Figure 2 plots the distribution of the number of words in the first claim of each issued patent in the data, both at the time of filing and at the time of issuance. The average patent has 130 words in the first claim at the time of filing and 181 words in the first claim at the time of issuance. --------Insert Figure 2 and Table 1 about here-------As discussed in Section III.A, each patent is assigned to one of about 450 art units, which each includes a number of patent examiners who specialize in the area of technology associated with the art unit. For technological and administrative reasons, there is considerable variation across different art units and within an art unit between individual examiners. For example, Figure 3 plots the number of patent examiners within art units in each USPTO Technology Center in our sample. Although most art units include fewer than 20 patent examiners, some art units include more than 60. Figure 4 plots the examiner-level average number of words added to the first independent claim for the 10 largest art unit in each Technology Center. Both the mean and the variance of the examiner-level average differ considerably, even within a Technology Center. --------Insert Figure 3 and Figure 4 about here-------- Preliminary testing in our data suggests that 27% of biotechnology patents employ Markush claim language, while 12% of chemical patents do so. In other technology centers, fewer than 2% of patents include such language in the claims. 11 IV. Validation A. Our measure We validate our measure of patent scope in two ways. First, we directly compare our measure of patent scope with patent attorney evaluations for a randomly selected sample of patents. Second, we evaluate how well our measure of scope predicts private patent value as evidenced by patent renewal rates. We conducted a survey of seven patent attorneys within a law firm specializing in the drafting and examination of patent applications. Each patent attorney was asked three questions for each of twenty randomly selected patent applications: 1) If this patent had been issued with the claims as written in the patent publication, what is your best estimate of this patent’s scope as compared to other issued patents in similar technologies? 2) Based on the claims in the patent that was issued, what is your best estimate of this patent’s scope as compared to other issued patents in similar technologies? 3) Compare the first independent claim in the issued patent to the first independent claim in the patent publication. How much narrower / broader is the scope of the first independent claim in the issued patent? The answer to each question was provided on a 0-10 Likert scale, with 0 (10) labeled as “Very Narrow” (“Very Broad”) for the first two questions and “Much Narrower” (“Much Broader”) for the third question. The survey instrument is shown in the Appendix, Figure A1. Figure 5 and Figure 6 present the survey responses graphically. Figure 5 plots for each patent the change in patent scope as reported by the practitioner against our measure of the change in patent scope (normalized number of words added), while Figure 6 plots for each patent the final patent scope as reported by the practitioner (i.e. survey question 2) against our measure of the change in patent scope (i.e. survey question 3). In both plots, the survey responses suggest that our measure of patent scope is consistent with patent attorney evaluations.12 12 Recall that, for the reasons discussed in the Data section, we exclude the Biotechnology Technology Center from this analysis. --------Insert Figure 5 and Figure 6 about here-------Table 2 presents the survey results, including six regression specifications correlating our measures with survey responses. Specifications 1, 2 and 3 correlate the number of words in the first claim with rater evaluations of the scope of each issued patent. Each additional 100 words in a patent claim correlates with a 1.3*** point reduction in patent scope according to survey respondents’ 0-10 scale (Specification 1). Specification 2 shows that this number is virtually unchanged if we add in fixed effects to control for differences in means between art units. In specification 3 we further control of artunit differences by controlling for both differences in the mean and variance between art units, which we do by creating an art-unit normalized measure of words added (a Z-score). When normalized this way, a 1 standard deviation increase in the number of words added correlates with a 1.3*** reduction in patent scope, again on the 0-10 scale, according to survey respondents. Specifications 4, 5, and 6 show that similar effects are observed for changes to the number of words between patent filing and issuance, with a 100-word increase in claim length corresponding to a patent 1.5*** points narrower, and a 1 standard deviation increase in words added corresponding to a patent 1.2*** points narrower according to survey respondents. In addition to all of these effects being highly significant, they also have considerable explanatory power, explaining between 30 and 50% of the variance in patent scope (as judged by our expert evaluators). The survey results thus support the claim that our measures accurately measure both absolute and relative claim scope as perceived by patent attorneys. --------Insert Table 2 about here-------Next, we compare our measure of patent scope to private patent value as evidenced by patent holder behavior. A patent holder is more likely to pay to keep in force a patent that is more valuable, and practitioners report that broader patents are more valuable. Thus, if our measure is accurate, then patents having broader patent scope under our measure should be more likely to be renewed. Figure 7 shows that this is indeed the case. Most patents are renewed regardless of scope, but renewal rates are higher for broader patents, particularly for the more expensive second renewal fee. --------Insert Figure 7 about here------We normalize our measure at the level of the art unit to account for any systematic technological and institutional differences in how inventions are claimed. While such a fine-grained approach is the most-compelling in theory, it is nevertheless also more challenging for practitioners to work with. As such, we also explore the extent to which claim length can proxy for patent scope even without such granular normalization. Table 3 uses different measures of final patent scope and changes to patent scope to predict the likelihood that the second renewal fee is paid. As the results show, the length of the first claim is a highly significant predictor of renewal fee payments regardless of whether claim length is normalized across all patents (Specifications 1 and 5), within USPTO technology center (Specifications 2 and 6), within USPTO art unit (Specifications 3 and 7), or within the intersection of USPTO art unit and main class (Specifications 4 and 8). --------Insert Table 3 about here-------This suggests the remarkable conclusion that scholars needn’t limit their analyses to within art unit or within main class effects, but can instead compare patents across the whole patent office by simply counting the words in the first independent claim. It is not at all obvious that this pattern should be true; the words to describe the scope of a car and a solar panel needn’t be at all comparable. Nevertheless, the empirical results indicate that it does seem to be broadly true. The effect strengthens at coarser levels of analysis. This suggest that the “between” variation for technologies is even more correlated than “within” variation. Put another way, technologies that have more words in the first independent claim are also more likely to have higher abandonment rates at renewal and this effect is stronger than the relationship between words and renewal within an art unit. Our results suggest that scholars should be able to predict renewal rates (and perhaps other commercial outcomes) by just counting the words in the first claim.13 To facilitate others using this measure effectively, we include Table A2 in the Appendix, which can be used by empiricists to map their patent data to scope percentiles in the overall patent distribution. B. Other measures In Section II.B, we discussed reasons why other measures of patent scope previously proposed in the literature present significant problems. Our validation exercise provides the opportunity to test these arguments. Table 4 presents the results of ordinary least squares regression models that use the number of IPC subclasses14 to which a patent is assigned, the number of claims in a patent, and the number of forward citations received by a patent to predict the patent’s scope as reported in the survey. --------Insert Table 4 about here-------As shown in Table 4, the number of IPC subclasses to which a patent is assigned is negatively correlated with its overall claim scope, although the relationship is not significant. This is the reverse of the relationship typically assumed in the literature (see, e.g., Lerner (1994)), suggesting that IPC is a misleading measure of scope. The number of claims in a patent is significantly and positively correlated with claim scope, but the magnitude of the coefficient, 0.065, is quite small. This implies that an increase of 15.4 claims is associated with an increase in patent scope of 1 survey point in a 0-10 Likert scale. But the average patent includes only 16.9 claims, and thus very large changes in the numbers of claims would be needed to produce any significant variation in predicted patent scope. Thus, not surprisingly, the number of claims can only explain 7% of the variance in patent scope as judged by our survey of experts. Accordingly, while the number of claims is at least correlated with patent scope, we find it to be uninformative for practical use. We similarly find that the number of citations to a patent is not a useful measure of that patent’s scope. Our survey validation exercise suggests that there is only a very slight (0.004) correlation between the number of citations and the scope of a patent and that this relationship is not statistically significant. To put that coefficient in perspective, ~250 citations would be needed to increase the 13 We do, however, provide the important caveat that while this measure of patent scope is useful for cross-technology comparisons, one cannot build a comparable exogenous examiner-based equivalent (discussed later) for causal inference due to non-randomness in examiner assignment. 14 The International Patent Classification System has five levels: section, class, subclass, main group, and subgroup. We count the number of unique classifications at the subclass level. predicted scope by 1 point in the 0-10 Likert scale on the survey. In practice, the modal and median patent have no citations within 4 years of issuance, and even the average patent only has 1.4. So variation in citations in not a good measure of patent scope. These findings, plus the theoretical issues discussed in Section II.B, suggest that none of the traditional measures of patent scope are informative for empirical studies of the effect of receiving a broader patent. V. Causal inference A. Context Having introduced a measure of patent scope, there may be a natural tendency to want to run regressions of the form shown in equation (1). Unfortunately, such a regression would not produce unbiased estimates because the quality of the underlying invention is likely to induce commercial outcome changes through multiple channels, not just patent scope. For example, a good invention might lead a company to invest more in the development of the product, which would lead to better commercial outcomes, irrespective of the scope of the patent. (1) 𝐶𝑜𝑚𝑚𝑒𝑟𝑐𝑖𝑎𝑙𝑂𝑢𝑡𝑐𝑜𝑚𝑒 = 𝛼 + 𝛽 ∗ 𝑃𝑎𝑡𝑒𝑛𝑡𝑆𝑐𝑜𝑝𝑒 + 𝜖 A causal graph (Figure 8), such as those advocated by Jordan (2003) and Pearl (2009), illustrates why this is likely to be true. As Figure 8 suggests, invention quality is likely to influence commercial outcomes both via patent scope and other factors. These alternate channels create an omitted variable problem that is very hard to control for, and which will lead to biased effect estimates if one were to try to directly analyze the effect of patent scope on commercial outcomes. --------Insert Figure 8 about here------To resolve this endogeneity problem, we introduce an instrument: patent examiner scope toughness. In this we apply the approach suggested by Cockburn et al. (2002) and taken by Sampat and Williams (2015) in that we will focus on the role of patent examiners in the determination of a patent’s scope. Recall that “often claims are lengthened through the addition of more specific language during prosecution at the request of the patent examiner.” (Quinn, 2015). Such lengthened claims will, on average, cause those patents to have less scope. As we will show, some examiners are systematically tougher across all the patents that they face, while others are systematically more lenient. This variation implies that a patent evaluated by such a more stringent examiner is treated differently, not because of the characteristics of the patent or the invention it describes, but because of characteristics of the examiner. Insomuch as examiners are randomly assigned (which we will address below), this variation provides the exogenous shock to patent scope that is needed for a causal analysis. B. Our measure: Patent Examiner Scope Toughness We define our measure of patent examiner Scope Toughness as the average number of words added to the first claim in an examiner’s patents (excluding the patent being evaluated), normalized by the art unit that the examiner is in. This formula is shown mathematically in equation (2), where 𝐸𝑥𝑎𝑚𝑖𝑛𝑒𝑟𝑆𝑐𝑜𝑝𝑒𝑇𝑜𝑢𝑔ℎ𝑛𝑒𝑠𝑠𝑝 is the examiner’s toughness on patent p being evaluated by examiner e, in art unit a. where 𝑚𝑒𝑎𝑛(𝑊𝑜𝑟𝑑𝑠𝐴𝑑𝑑𝑒𝑑)𝑖 represents the mean of words added by that examiner (e) or by all examiners in that art unit (a), and where the subscript (≠ 𝑝) indicates that the focal patent is excluded from all calculations. This exclusion is important because without it the focal patent would impact the result mechanically, inducing bias. In sum, our measure is a leave-one-out average of the words added by a particular examiner, which is then converted into a Z-score. (2) 𝑚𝑒𝑎𝑛(𝑊𝑜𝑟𝑑𝑠𝐴𝑑𝑑𝑒𝑑)𝑒 −𝑚𝑒𝑎𝑛(𝑊𝑜𝑟𝑑𝑠𝐴𝑑𝑑𝑒𝑑)𝑎 ) 𝑠𝑡𝑑𝑒𝑣(𝑊𝑜𝑟𝑑𝑠𝐴𝑑𝑑𝑒𝑑)𝑎 ≠𝑝 𝐸𝑥𝑎𝑚𝑖𝑛𝑒𝑟𝑆𝑐𝑜𝑝𝑒𝑇𝑜𝑢𝑔ℎ𝑛𝑒𝑠𝑠𝑝 = ( The motivation for normalizing within an art unit is that, as shown previously in Figure 4, art units exhibit considerable differences in both the mean and variance in words added. This variation probably reflects both substantive reasons (different treatment of the subject matter) and administrative differences (either derived from subject-area differences, or idiosyncratic to the examiners working there). We find abundant evidence that patent examiner Scope Toughness is a strong instrument. Figure 9 and Figure 10 illustrates these in summary form, showing the number of words in the patents, at filing and issuance respectively, by the top 25% toughest patent examiners in an art unit as compared to the bottom (most lenient) 25%. While the number of words in the claims at filings for the two are almost identical, the patents assigned to the tougher examiners have their scope narrowed much more, yielding much longer claims at issuance. This can also be seen in more continuous form in Figure 11. Table 5 shows that examiner Scope Toughness (i.e. word added in an examiner’s other patents) is highly predictive of the words added to the focal patent during examination (with F-scores greater than 1,900). These results conclusively demonstrate that patent examiner Scope Toughness has a strong “first stage” in predicting changes in the words added to patent claims, and thus to the scope of those patents. --------Insert Figure 9, Figure 11, Figure 11, and Table 5, about here------- C. Random Assignment and Balance There is substantial debate about whether the assignment of examiners to patents within the USPTO is random, or as good as random. Ultimately this is a claim about the internal operations of the patent office, which we cannot observe. Nevertheless, those who have studied it in detail, including those with visiting positions there, have concluded that the assignment process is as good as random. (Farre-Mensa, Hegde, & Ljungqvist, 2016; Lemley & Sampat, 2012; Sampat & Williams, 2015). Patent examiners themselves lend support to the view that there isn’t any selection on the applicant side, reporting that it is also nearly impossible for applicants to pick their examiners: (Key, 2015) Question: “Is there any way for an inventor to influence who examines his or her patent?” Answer: "No, you have absolutely no influence over who reviews an application. I don't know anyone who has been successful at getting a new examiner assigned to a case." Nevertheless, recent work by Righi and Simcoe (2017) that evaluates the types of patents assigned to different examiners, find that “examiners specialize in particular technologies, even within relatively homogenous art units,” suggesting that assignment isn’t always random, but rather that particular types of inventions are assigned to specialists at higher-than-random rates. At the same time, they “find no evidence that certain examiners specialize in applications that have greater importance or broader claims.” We do not take a strong point of view on this issue in general, but rather consider it in our context. If examiners are randomly assigned, then we should find that the patents assigned to tough and lenient examiners are different by no more than chance. If, however, we find that they are different, we need to evaluate by how much – since that will affect the extent of bias that could be introduced. We test for balance between the type of patents examined by tough and lenient examiners both in means and in distributions. Table 6 shows the mean differences in application between the top 25% toughest and the bottom 25% (most lenient) examiners.15 We find that, of average, the toughest examiners receive applications with 4 additional words in the claims. Because of our enormous sample Ideally this comparison would examine all applications by these examiners, but data availability concerns restrict the sample to issued patents. 15 size, this difference is highly significant in a test of difference of means, but it is also quite small, representing less than a 3% normalized difference between them.16 --------Insert Table 6Error! Reference source not found. about here-------The extreme closeness of these distributions at filing is supportive of the general contention of quasi-random assignment of patents to examiners. But the strong statistical significance of differences in means suggests there may be either rare instances of significant non-randomness or very subtle nonrandomness more broadly, in line with the findings of Righi and Simcoe (2017). That said, in practice we would suggest that these balances are close enough to limit the potential bias introduced by the instrument across large groups of patents,17 but that future work that uses them should be cognizant of, and test, what effect these small imbalances might have on their estimated coefficients, particularly when dealing with small samples. D. Impact of getting a tougher examiner Getting one’s patent assigned to a tough examiner should have a number of effects. First, that difference should manifest as a more difficult negotiation process between the examiner and the applicant. Table 7 examines this, showing that for each standard deviation in examiner scope toughness negotiations take longer (a.k.a. pendency – 0.29 years***18), involve more back and forth communications (0.37 more office actions***), and involve the examiner being more likely to find reasons to reject the patent application because it is not novel (1.5% greater chance of a 102 rejection***) or is just an obvious extension of other work (7.6% greater chance of a 103 rejection***). --------Insert Table 7 about here-------More importantly, getting a tough examiner should affect the value of a patent, and thus decrease the applicant’s willingness to pay to issue or maintain the protection that it provides. Table 8 shows that indeed this is the case. Patents evaluated by a one standard deviation tougher examiner are abandoned before issuance 5.4%*** more often, after which their owners are 0.3%*** less likely to pay $1,600 to Intuitively, the normalized difference corresponds to a standard deviation of the pooled data set. Importantly, this claim is one about covariate balance. If random assignment holds, then the observed covariate balance is likely to also hold for unobservables. If, however, assignment is largely non-random then such assumptions about unobservables cannot be made and our claim must be limited to the bias introduced by the observable factors. In cases where there is some randomness, but it is not complete (as seems likely given the literature), the potential bias will be between these extremes. 18 This correlation (also shown in Figure 14) also suggests that looking at the pendency of an examiner might be another way of constructing an examiner toughness metric. We find that it is an acceptable metric, but has both more theoretical issues and less empirical predictiveness, and thus we focus on words added. 16 17 maintain the patent after 3.5 years, and 3.4%*** less likely to pay $3,600 to maintain it after 7.5 years.19 The net effects on maintenance fee payments are different across different technology classes, as Figure 12 shows. Patents in Computers, Communications and Semiconductors are most likely to be abandoned if they’ve had a tough examiner. --------Insert Table 8 and Figure 12 about here-------It is perhaps more revealing to examine this by the identity of the patentee. Table 9 and Table 10 show this for the first and second renewal fees respectively, with public firms broken into the top and bottom half of assets at patent filing and private firms broken up into whether they have more or less than 50 previous patent applications (since asset data isn’t available). These results show that even at the first renewal fee, both large public and large non-public firms are renewing patents less if they have less scope. This is consistent with them having more sophisticated patenting operations, and thus being able to identify a fraction of their patents that they wish to let lapse. Smaller firms and universities show no such abandonment pattern, perhaps reflecting that they either cannot or don’t wish to do similarly. In contrast, by the second, more expensive renewal fee, all applicants except universities are disproportionately abandoning patents with less scope, with large firms continuing to do so more aggressively. --------Insert Table 9 and Table 10 about here-------One might also expect that broader patents are more likely to be cited by future patents. We find mixed results for this, with even small changes in the specification able to produce large changes in the magnitude or sign of the resulting estimates. A more detailed look at the data clarifies why this should be, since examination by a tough examiner also leads to other conflating effects: decreased likelihood of issuance, increased pendency, and decreased scope, all of which can feed back into changes in citations.20 E. Comparison with Sampat and Williams (2015) measure While our inspiration for this measure came from Sampat and Williams (2015), our measure is different from theirs both mathematically and substantively. Mathematically the most import differences are our usage of words added (instead of abandonment rates) and our normalization at the Source: USPTO. USPTO Fee schedule. (http://www.uspto.gov/learning-and-resources/fees-and-payment/uspto-fee-schedule). Our panel is too short to accurately measure effects at the third maintenance fee payment. 20 Discussed in further detail in the Appendix. 19 art unit level. Substantively, and more importantly, the two measures characterize patent examiners based on different sets of patents that they are evaluating. Our measure characterizes patent examiners by how they treat the scope of valuable patents, where the question is how broad a patent will be granted, not whether one will be. Sampat and Williams’ measure characterizes patent examiners by how they treat patents that might be allowed or might not be. Since ultimately only an applicant can abandon an application, by definition such patents must have valuations close to zero. Because these two measures consider such different types of patents, one might imagine that examiners treat them differently. Figure 13 shows that this is indeed the case, with examiners that are tough on valuable patents having notably different toughness on lowvalue ones. As Figure 13 shows, the two measures have some correlation, but there is also lots of variation around them. An area of particular difference is the examiners that are only moderately tough on scope (0-1 standard deviations), but have enormous differences in the share of abandonments that they cause. --------Insert Figure 13 about here-------In addition to comparing these measures directly, we can also ask which does better at predicting the scope changes that an examiner will impose on patents they evaluate. Table 11 shows that both measures are highly predictive (<0.1%), but that Scope Toughness predicts ~4x as much of the variance (R2 of 3.1% vs 0.8%). --------Insert Table 11 about here-------It is clear that both our measure and that of Sampat and Williams (2015) have value. Theirs is likely to be most useful where the issue of patent issuance (or not) is the chief contention, for example if the existence of patents is used as a signal by a small firm trying to raise venture capital. In contrast, we contend that our measure is better suited for evaluating effects of the protection offered by patents – in particular the ability to exclude competitors. Most of the value from the patent system comes from a set of highly valuable patents. For these patents, the question isn’t typically will they get a patent, but how much coverage will it provide and therefore how effective will it be in generating additional profits for the applicant. In these contexts, our measure’s focus on the scope of coverage and on evaluating patent examiners by how they treat valuable patents is particularly useful. The next section presents an example of how our instrument can be applied.21 VI. Application: Standards Essential Patents We demonstrate the power of examiner toughness instrument based on patent scope by looking at standards-essential patents. Technical standards can be essential for facilitating the adoption of new technology and for coordinating to produce subsequent innovations that are compatible. (Merges & Kuhn, 2009; Simcoe, 2016). They can also have substantial effects on competition and follow-on innovation. (Lampe & Moser, 2016). Thus, it is of substantial interest to understand what drives the inclusion of patents into a standard. Here we test whether a patent’s scope affects whether it is declared “essential” to a technical standard. Crucially, by using our instrument this test is exogenously varying the scope of protection, but not the innovativeness or contribution from the technology underlying the patent. To analyze this question we start with a data set, developed by Bekkers, Catalini, Martinelli, and Simcoe (2012), of 14,057 patents that have been declared standards essential by their owners. This data “provides a full overview of all disclosed IPR at standard setting organizations world-wide” and is “[b]ased on the archives of thirteen major SSOs”. Most art units at the USPTO lack of patented standards in this data, so we limit our analyses to the subsample of in art units where at least one patent has been declared standards essential. Using a direct OLS regression of the examiner toughness instrument on inclusion of patents in a standard22 we find that for every additional standard deviation of toughness of the patent examiner, a patent is 7.1% less likely to be incorporated into a standard. Because so few patents (<.01%) are declared standards essential, we estimate the null distribution with a Fisher’s Exact Test (i.e. a permutation test) to build up a distribution of outcomes under the hypothesis that patent scope has no effect on the likelihood of being declared standards essential.23 Figure 15 plots the results of this 21 It would be appealing to be able to apply our metric to Sampat and Williams’ dataset. They found precise nulls in their estimation of the effect of patent coverage on downstream innovation. It would be interesting to ask if we considered patent examiners based on their treatment of more valuable patents (which many pharmaceutical patents surely are), whether we would get a different result. Unfortunately, the presence of Markush language and less randomness in the assignment of patent examiners (Righi and Simcoe) make this much more difficult. However, an enterprising researcher might find a way to identify these systematically and then validate our measure in the rest of biotechnology, in which case they would be able to check this. 22 Notice, this is the intent-to-treat estimator, rather than the treatment-on-the-treated estimator. We report this one because we suspect that it will be the most useful to practitioners. 23 We randomly permute examiner toughness across all patents while holding constant whether a patent is declared standards essential. Then, we regress the outcome (i.e. whether the patent is declared standards essential) on examiner toughness. We repeat this process 10,000 times to construct the distribution of outcomes. analysis. We find that only 0.52% of the permutations of examiner toughness produce as large an estimated coefficient as the real one, and thus our estimate has a 0.52% significance level and conclude that patent scope plays an important role in whether a patent is essential to the use of a technological standard, even apart from the technology underlying the patent. --------Insert Figure 15 about here----- VII. Conclusion This paper addresses a fundamental challenge for patent scholars: evaluating the effects of greater patent coverage. Traditionally such evaluations have been difficult because the scope of protection has been difficult to measure and because, even if scope were measured, omitted variable biases would preclude drawing causal inferences about the effects of patent scope. This paper introduces two new tools for patent scholars: an easy-to use way of measuring patent scope and a way to use this as an econometric instrument to produce unbiased casual estimates. Our measure of patent scope is the number of words in a patent’s first claim, with longer claims producing narrower scope. This approach is grounded in the practice and law of patenting: a product, composition, or process must exhibit all the elements in a claim to infringe it, and therefore adding more elements (and hence words) almost always makes it narrower. We extensively validate this measure, showing that it agrees with (i) qualitative pronouncements by practitioners, (ii) patent evaluations by patent attorneys, and (iii) the behavior of applicants. Our validation exercise also allows us to evaluate previously proposed measures of patent scope, including: counting the number of patent classes, counting the patent’s citations, and counting the number of claims in a patent. We find that they are all either misleading or uninformative. In particular the number of patent classes is negatively correlated with scope, exactly contrary to traditional assumptions. The citations to a patent are so weakly (and statistically insignificantly) related scope as to be of no use to patent scholars. Finally, we find that the number of claims in a patent is correlated with patent scope, but that this relationship is quite weak and explains little variation, meaning that it provides virtually no information across the patents that most scholars work with. We also introduce a new instrument: patent examiner scope toughness, which provides an exogenous source of variation in patent scope that can be used for causal inference. We validate this instrument, showing that it is predictive of the behavior of patent applicants, and that it is a strong first stage when used as an instrument. In contrast to previous instruments which focus on abandonment rates on marginal patents, our instrument reflects the behavior of patent examiners on all patents, including the important patents that comprise the vast majority of the value in the patent system. Further, we show that scope toughness explains about 4 times more of the variation in changes in patent scope than measures based on abandonment. Finally, we provide an example of the usage of our instrument, showing that a patent that provides exogenously less scope (by being assigned to an examiner of one standard deviation more scope toughness ) is 7.1% less likely to be declared standards essential.