Attributes of a Good Theory

IR Theory & Research Notes Attributes of a Good Theory I. Basic Elements of a Theory A. Normative Foundations 1. Why do you (others) in the field care about this issue? 2. What values/who’s interests are at stake? 3. What outcomes/goals might be affected by addressing the question? B. Positive Elements: From Problem Definition to Execution “A problem is not a research question. It is easy to assume that stating a topic or a theme or an issue is equivalent to asking a research question. It is not. A research question is literally a sentence that ends in a question mark and in which every word counts, one that points in two directions—toward the theoretical framework that justified the question and toward the empirical evidence that will answer it. A research question is a commitment to a way of framing the problem for the research process to begin or continue, since research questions usually change in the course of the project.” (Robert R. Alfred, The Craft of Inquiry, Oxford U. Press, 1998, p. 25) Example: A problem. Since 1980 average workers’ wages have not kept up with productivity growth. A research question: What explains why there has been a consistent lag in wages relative to productivity growth since 1980? 1. Defining the Question(s) a. Does it pose an interesting intellectual puzzle? b. Does resolving it have important consequences? c. Do you find it of deep intellectual interest—Can you see it through the dark nights of doubt and criticism that you will encounter? d. What broader debate is it embedded in? e. Does it have actionable implications for actors of interest to you? f. Can it be focused and reduced to a question that is testable and refutable? g. What literatures and intellectual traditions is it positioned in or can be used to frame the question and examine past research relevant to the question? 2. Conceptual Definitions: Can you define the key constructs: a. Dependent variable: (Y) Question of interest you are seeking to explain. b. Independent variable(s): (X) Causal variable(s) that cause variation in the dependent variable. c. Intervening Variable(s): (I)Variable that are positioned intermediate in a “nomonological network” between independent & dependent variables. The correlation between X and Y goes toward zero when I is controlled, but for theoretical purposes X remains important to your theory. Intervening variables should help to explain the process by which variations in X cause variations in Y. Otherwise a problem of spurious (uninteresting but real) correlation exists and the theory becomes less powerful or interesting. d. Moderator Variable(s): (M) Variable(s) that influence when, or under what conditions X causes variations in Y. Slopes of regression coefficient of y on x are significantly different across different levels or values of M. 3. Can you identify operational measures of the constructs for which data can be collected? Key attributes of these measures include: a. Construct validity: Does the measure capture the meaning of the construct? 1. Does it meet tests of reliability? 2. Does it meet the face and content validity judgements (i.e., is it intuitively sensible to other experts in the field?) 3. Does it meet the convergent and discriminant validity tests? 4. Can you offer a clear proposition(s) for why and how variations in X causes variations in Y. Propositions relate two or more constructs. a. Here one wants to go beyond prediction to propose causal processes relating the variables (not just the sun will come up soon after the birds start singing—a reasonable prediction but not an explanation). 5. Can you generate testable hypotheses to relate the operational measures that are consistent with the propositions? II. Is it Testable? A. Research Design(s) 1. Theorizing as an Iterative Process a. Case studies, enthnographies, interviews, observations, and other qualitative, historical, and institutional research are invaluable for generating and deepening hypotheses and placing a theory in a broader contexts—historical, political and institutional, etc.—in which it is situated. b. Theories are never complete explanations. (Unless they are some deductive law that when X always Y). We have few if any of these in the social sciences or particularly in our field. 2. Choosing a Research Design and Method Appropriate to the Problem a. No one best design, method. Qualitative and quantitative methods; historical, laboratory, ethnography, case studies, longitudinal and cross sectional quantitative analyses are all tools to fit to the problem. Multiple methods are strongly encouraged. b. Key features are to rule out alternative explanations by attending to threats to internal validity and being clear about the domain of the theory—the external validity or range of observations or conditions over which the theory is generalizeable. 3. Testing: Reducing the Theory to a Model fitted to a particular empirical setting. a. No theory can be tested in one study. Identifying empirical opportunities to test key parts of the theory that build incrementally on what’s been done before is critical to moving from broad question to a doable study. b. Often the process works in reverse. Knowing the theory of interest, data possibilities that turn up suggest theoretical questions that can be tested. One can work up the ladder of abstraction as well as down. Insights in field work often happen that suggest new theoretical questions or redefine prior ones. 4. Publishing the Theory and Tests. a. A good published paper will focus on one aspect or question. It might have several sub-questions, but unless you can define the question of interest—what’s this paper about—in one sentence, its probably not focused precisely enough to pass muster. (Same goes for a thesis). b. A single paper as described above may be part of a larger series of related papers, drafts, etc. that address different subsets of the theory, use different methods, different data, different measures, rule out different alternative explanations, or explore different consequences of a phenomenon. Each of these deepen and broaden the theory. Trying to put too many of these in one paper loses the power of the idea. c. Your immediate colleagues should be your most vigorous and helpful critics. Give it to them to read and expect criticism. Never submit a paper for peer reviewed publication until it has been read and criticized by valued colleagues. But, don’t feel all comments have to be addressed. Remember what is of central interest to you. Stay focused but listen to alternative points of view and address those you can or, after more thought, those you agree with or recognize you have to contend with. d. It’s not a theory until its published! No paper is perfect. Don’t submit something until you are comfortable with it. Don’t expect the reviewers to do your work in editing it down or finding the flaws that can be corrected in a revised submission. But, don’t sit on it forever worrying if its perfect.