Information Credibility, Political Sophistication, and the Punishment of Corruption in Brazil

“Can Citizens Discern? Information Credibility, Political Sophistication, and the Punishment of Corruption in Brazil” Online Appendix 1   Vignettes Pure Control Imagine que você vive num bairro como o seu, mas numa cidade diferente do Brasil. Vamos chamar o Prefeito dessa cidade em que você mora de Carlos. Agora imagine que o Prefeito Carlos está concorrendo à reeleição. Durante os quatro anos em que foi Prefeito a cidade teve várias melhorias, com crescimento econômico e melhores serviços públicos de saúde e transporte. No Corruption [Pure control plus] Também nessa cidade, todo mundo diz que o Prefeito Carlos não aceitou suborno para fechar contratos com fornecedores da Prefeitura. Corruption without Source [Pure control plus] Também nessa cidade, todo mundo diz que o Prefeito Carlos aceitou suborno para fechar contratos com fornecedores da Prefeitura. Credible Source / Specific Accusations [Pure control plus] Também nessa cidade, uma auditoria do governo federal diz que o Prefeito Carlos aceitou suborno para fechar contratos com fornecedores da Prefeitura. Less Credible Source / Specific Accusations [Pure control plus] Também nessa cidade, o partido de oposição diz que o Prefeito Carlos aceitou suborno para fechar contratos com fornecedores da Prefeitura. Credible Source / Less Specific Accusations [Pure control plus] Também nessa cidade, uma auditoria do governo federal diz que ocupantes de cargos na Prefeitura aceitaram suborno para fechar contratos com fornecedores da Prefeitura. Less Credible Source / Less Specific Accusations [Pure control plus] Também nessa cidade, o partido da oposição diz que ocupantes de cargos na Prefeitura aceitaram suborno para fechar contratos com fornecedores da Prefeitura. 2   Main Follow Up Questions Na sua opinião, qual a chance de você votar para o Prefeito Carlos? Grande chance; alguma chance; pouca chance; ou nenhuma chance? Agora que nota de 1 a 7 você daria para o Prefeito Carlos, sendo que 1 significa que você acha ele “um Prefeito muito ruim” e 7 significa que você acha ele “um Prefeito excelente”? 3     Sampling Procedure Our survey was included as the second module of the May 2013 IBOPEBus survey. The IBOPEBus is a monthly omnibus survey that uses a probabilistic sample of geographic areas to obtain a representative sample of the over-16-years-old Brazilian population. The sampling frame is based on the 2010 census, the 2011 Pesquisa Nacional por Amostra de Domicilios (National Household Survey), and 2012 data from the Tribunal Superior Eleitoral (National Electoral Tribunal). 140 cities were sampled using a probability-proportional-to-size (PPS) method within 25 strata that are defined by 25 of Brazil‘s 27 states. (The survey rotates on a monthly basis among three small states in the northern region of the country.) Census tracts were selected using PPS with stratification across zones of major metropolitan areas. Enumerators recruited individual respondents in public or semi-public places according to a quota scheme designed to produce a representative sample of the national population in terms of age, gender, and employment characteristics (sector of the economy and employment status). Interviews are conducted face-to-face during working days, evenings, and weekends. 4     Randomization Procedure and Balance Checks The seven vignettes were to be randomly assigned to survey respondents within each sampling strata. Since seven respondents were sampled from each census tract, each vignette was to be assigned once per census tract. Unfortunately, rather than assigning the vignettes in random order, they were assigned in the same order – from the first through the seventh – within each of the sampled census tracts. If different types of respondents were recruited earlier in the day as compared to later in the day, this failed randomization could imply a correlation between observable or unobservable characteristics of respondents and their treatment status. While we cannot comment on correlations between treatment status and unobservable characteristics, we examine here whether or not any observable pre-treatment characteristics predict selection into the treatment categories. To do this, we use two methods. Both indicate a degree of variation in observed characteristics across treatment groups that is consistent with what could be generated by chance. We nonetheless replicate results reported in Tables 3 and 4 in the paper using regression analyses controlling for possible confounding covariates and report those below. In both cases, the results are substantively unchanged from the simpler differencein-means tests reported in the text. Here, we explain our two methods for checking balance. First, we run two multinomial logit models where the seven categories of treatment assignment defined the outcome variable. 1 We compare a null model with no predictors to a model with predictors for gender, age, education, social class, income, an indicator for whether or not the respondent is catholic, a variable measuring how often the respondent talks about politics, a variable measuring how often the respondent reads the news, a variable representing the respondent’s score on a two-question measure of political knowledge, and a set of indicators for whether or not the respondent identifies with one of the three major political parties in Brazil, the PT, the PSDB, or the PMDB. The table below presents the results of the multinomial logit model. The chi-squared statistic indicates that the model with the set of predictor variables is not statistically distinguishable from a null model without any predictor variables at all (p < 0.85). Outcome Male (0/1) Vignette 1 0.07 (0.18) Vignette 3 -0.11 (0.18) Vignette 4 0.13 (0.18) Vignette 5 0.11 (0.18) Vignette 6 0.15 (0.18) Vignette 7 0.15 (0.18)                                                                                                                         1  We  do  not  use  the  test  proposed  by  Hansen  and  Bowers  (2008)  because  that  is  appropriate  only  for  dichotomous   treatments.  For  experiments  with  multiple  treatments,  using  multinomial  logit  as  described  here  is  the  preferred   method  (Bowers,  personal  communication,  2014).  For  consistency  with  balance  tests  reported  elsewhere  in  the   literature,  we  also  report  the  results  of  difference  of  means  tests  below.   5     Age Category 0.01 (0.06) -0.06 (0.06) 0.00 (0.06) -0.09 (0.06) -0.04 (0.06) -0.04 (0.06) Education -0.05 (0.10) -0.05 (0.10) 0.01 (0.10) -0.08 (0.10) 0.09 (0.10) 0.00 (0.10) Social Class 0.44** (0.17) 0.09 (0.17) 0.05 (0.17) 0.19 (0.17) 0.10 (0.17) 0.10 (0.17) Income -0.13 (0.12) 0.03 (0.13) -0.03 (0.12) -0.16 (0.13) -0.16 (0.13) -0.13 (0.13) Catholic (0/1) -0.29 (0.19) -0.27 (0.19) -0.33* (0.19) -0.18 (0.19) -0.14 (0.19) -0.12 (0.19) Talk about Politics 0.18 (0.13) 0.19 (0.13) -0.04 (0.13) 0.12 (0.13) -0.12 (0.13) -0.03 (0.13) Read the News -0.00 (0.11) -0.02 (0.11) -0.01 (0.11) -0.05 (0.11) -0.01 (0.11) 0.02 (0.11) Political Knowledge Index -0.04 (0.14) 0.09 (0.14) 0.02 (0.14) 0.21 (0.14) 0.03 (0.14) 0.14 (0.14) PT Identifier (0/1) 0.10 (0.21) 0.02 (0.21) 0.31 (0.20) -0.05 (0.21) -0.16 (0.21) -0.02 (0.21) PSDB Identifier (0/1) -0.35 (0.42) -0.27 (0.41) -0.02 (0.40) -0.66 (0.46) -0.28 (0.42) -0.26 (0.42) PMDB Identifier (0/1) 0.46 (0.34) 0.16 (0.37) -0.15 (0.40) 0.17 (0.36) 0.37 (0.35) 0.47 (0.35) Constant -0.77 (0.48) -0.14 (0.48) 0.06 (0.48) 0.29 (0.48) 0.37 (0.48) 0.13 (0.48) N Pseudo R2 Chi-Squared p-value for H0: No difference between the models 1,795 0.01 59.65 0.85 Second, we examine balance on these same observable covariates by calculating the mean value of each covariate in the overall data and the mean value and 95 percent confidence interval of each covariate for each of the seven treatment categories. These results are plotted below. 6     Looking at the extent to which the value of each by-treatment mean is different from the overall mean in the data, we find that three out of the 84 tests indicate differences significant at the 0.05 level or better; this is no more than what we would expect to see by random chance. 7     Replication of Results using Regression Analysis Our examination of variation on observable covariates detailed above suggests that, in spite of the failure of our randomization procedure, differences on observable characteristics across treatment groups are no greater than we would expect by random chance. Nonetheless, we replicate the analysis in Tables 2, 3, and 4 from the paper using regression analyses that control for covariates. As can be seen below, the results are extremely similar and substantively equivalent in all three cases. How likely are you to vote for the mayor? (N) Average Response (Standard Error) Estimated Difference from Control Conditions Estimated Difference from Unsourced Accusations Estimated Difference from Less Credible Accusations Credible 2.07 -1.28 -0.05 -0.24 Accusations (0.05) (0.07) (0.09) (0.07) (N=553) [p < 0.01] [p < 0.57] [p < 0.01] Less Credible 2.37 -1.02 0.21 Accusations (0.05) (0.07) (0.09) -(N=547) [p < 0.01] [p < 0.02] No Source of 2.18 -1.22 Corruption (0.07) (0.08) --Accusations [p < 0.01] (N=278) Pure 3.38 Control/Control (0.04) ---with Clean Mayor (N=560) Replication of Table 2 with Controls Note: Cells in columns 2-4 present point estimates on treatment indicators defined by the row and column titles that are included in linear regressions also controlling for male, age, education, social class, income, catholic, talk about politics, attention to the news, political knowledge, PT identity, PSDB identity, and PMDB identity. Robust standard errors in parentheses. p-values in brackets. 8     How likely are you to vote for the mayor? Illiterate / less than primary Coefficient on Treatment Indicator for Credible versus Less Credible Accusations -0.08 (0.25) [p < 0.75] Complete primary; incomplete middle -0.08 (0.14) [p < 0.55] Complete middle; incomplete secondary -0.33 (0.14) [p < 0.03] Complete secondary At least some tertiary -0.25 (0.14) [p < 0.09] -0.53 (0.17) [p < 0.01] Replication of Table 3 with Controls Note: Cells present point estimates on a treatment indicator from linear regression models for each group controlling for male, age, social class, income, catholic, talk about politics, attention to the news, political knowledge, PT identity, PSDB identity, and PMDB identity. Robust standard errors in parentheses. p-values in brackets. How likely are you to vote for the mayor? Coefficient on Treatment Indicator for Credible versus Less Credible Accusations No Questions Right -0.24 (0.09) [p < 0.01] One Question Right -0.10 (0.16) [p < 0.54] Two Questions Right -0.53 (0.17) [p < 0.01] Replication of Table 4 with Controls Note: Cells present point estimates on a treatment indicator from linear regression models for each group controlling for male, age, social class, income, catholic, talk about politics, attention to the news, political knowledge, PT identity, PSDB identity, and PMDB identity. Robust standard errors in parentheses. p-values in brackets. 9   Replication of Main Results using Only Vignettes that Specifically Accuse the Mayor As reported in the text, for the analysis presented there, the credible and less credible conditions include vignettes that explicitly mention the mayor (specific vignettes) as well as those that mention municipal officials (less specific vignettes). Here, we replicate the results from Tables 2, 3, and 4 using only those vignettes that mention the mayor. The results from the replicated Table 2 are almost identical, and in fact somewhat stronger, than those presented in the text. In the replicated Table 3, we once again see no significant difference in mayoral support among respondents with the lowest level of education and significant differences in support among the rest of respondents. As compared to the table presented in the main text, however, there is not a clear monotonic increase in differentiation across education categories. The conditional average treatment effect for three of the education groups is marginally significantly different from the conditional average treatment effect for the lowest education group. In the replicated Table 4, we see the same pattern as in the main text, where the group that answered only one political knowledge question correctly has the smallest difference between how they react to the more and less credible accusations, while the group that answered no questions right is in the middle, and the group that answered both questions right has the largest difference (as we would expect). As in the main text, the differences between the CATEs are not statistically significant. 10   How likely are you to vote for the mayor? (N) Average Response (Standard Error) Estimated Difference from Control Conditions Estimated Difference from Unsourced Accusations Estimated Difference from Less Credible Accusations Credible 1.97 -1.41 -0.21 -0.38 Accusations (0.06) (0.07) (0.09) (0.09) (N=279) [p < 0.01] [p < 0.03] [p < 0.01] Less Credible 2.36 -1.02 0.18 Accusations (0.07) (0.07) (0.10) -(N=278) [p < 0.01] [p < 0.07] No Source of 2.18 -1.20 Corruption (0.07) (0.07) --Accusations [p < 0.01] (N=278) Pure 3.38 Control/Control (0.04) ---with Clean Mayor (N=560) Replication of Table 2 with Specific Vignettes Only Note: Cells in columns 2-4 present difference-in-means tests among the means reported in column 1. Standard errors in parentheses. p-values from a t-test of the null hypothesis of no difference in means between the two groups in brackets. How likely are you to vote for the mayor? Illiterate / less than primary Complete middle; incomplete secondary 2.41 (0.14) N=68 Complete secondary At least some tertiary 2.08 (0.21) N=35 Complete primary; incomplete middle 2.34 (0.15) N=59 Less Credible Accusations 2.44 (0.12) N=73 2.40 (0.18) N=43 Credible Accusations 2.25 (0.22) N=28 1.85 (0.13) N=69 1.97 (0.12) N=72 2.16 (0.11) N=77 1.58 (0.15) N=33 Difference -0.16 (0.30) 0.59 0.48 (0.20) 0.02 0.44 (0.18) 0.02 0.28 (0.17) 0.10 0.47 (0.24) 0.01 0.10 0.21 0.02 p-value on H0: No Difference p-value on H0: No -0.08 Difference between CATE and CATE for Lowest Education Group Replication of Table 3 with Specific Vignettes Only 11   Note: p-values for the null hypothesis on the conditional average treatment effect (CATE) for each group are based on difference-in-means t-tests. p-values for differences across the CATEs are based on the randomization inference tests described in Gerber and Green (2012). How likely are you to vote for the mayor? Less Credible Accusations Credible Accusations Difference No Political Knowledge Questions Right 2.31 (0.09) N=159 1.92 (0.08) N=174 0.39 (0.12) 0.01 -- One Political Knowledge Question Right 2.25 (0.15) N=56 2.31 (0.15) N=65 0.06 (0.21) 0.79 0.17 Both Political Knowledge Questions Right 2.56 (0.14) N=54 1.86 (0.14) N=49 0.70 (0.20) 0.01 0.20 p-value on H0: No Difference p-value on H0: No Difference between CATE and CATE for No Questions Right Group Replication of Table 4 with Specific Vignettes Only Note: p-values for the null hypothesis on the conditional average treatment effect (CATE) for each group are based on difference-in-means t-tests. p-values for differences across the CATEs are based on the randomization inference tests described in Gerber and Green (2012). 12   Party Identification and Responses to Credibility PT Sympathizers and Responses to Credibility As discussed in the text, this table shows that PT partisans are not especially skeptical of information attributed to the opposition (less credible accusations). How likely are you to vote for the mayor? Not a PT sympathizer PT sympathizer Less Credible Accusations 2.36 (0.06) N=404 2.12 (0.04) N=405 2.38 (0.09) N=143 1.97 (0.08) N=148 Difference .24 0.42 p-value on H0: No Difference 0.00 0.00 Credible Accusations 13     Alternative Measure of Sophistication: Political Interest In the text, we present results for more versus less sophisticated voters using education and political knowledge as measures of sophistication. Here we present results for an additional measure of sophistication, political interest. Political interest is a measure of how frequently a respondent reports discussing politics with her family and friends. Respondents had the option of answering “very frequently,” “frequently,” “rarely,” or “never.” For the purpose of analysis below, we group together “very frequently” and “frequently” responses into a single “high interest” category. For this measure, we see the same overall pattern as presented in the main text: more sophisticated respondents differentiate more between the credible and less-credible treatments. Also as in the text, the effect seems to be driven primarily by more interested respondents being more forgiving of less credible accusations (H2b), rather than being more punitive of more credible accusations (H2a). 14     How likely are you to vote for the mayor? Less Credible Accusations Credible Accusations Difference Never Discuss Politics 2.18 (0.09) N=187 1.98 (0.07) N=205 0.20 Rarely Discuss Politics 2.40 (0.07) N=232 2.13 (0.07) N=235 0.27 Frequently Discuss Politics 2.60 (0.10) N=124 2.16 (0.11) N=104 0.43 p-value on H0: No Difference 0.07 0.00 0.00 p-value on H0: No Difference -0.64 0.20 between CATE and CATE for Lowest Interest Group Replication of Table 3/4 using Political Interest as a Proxy for Sophistication. Note: p-values for the null hypothesis on the conditional average treatment effect (CATE) for each group are based on difference-in-means t-tests. p-values for differences across the CATEs are based on the randomization inference tests described in Gerber and Green (2012). How likely are you to vote for the mayor? Less credible accusations Lowest Interest Highest Interest Difference p-value on H0: No difference between lowest and highest education 0.01 2.18 2.60 0.41 (0.08) (0.10) N=187 N=124 More credible 1.99 2.16 0.18 accusations (0.07) (0.11) N=205 N=104 Replication of Table 3a/4a using Political Interest as a Proxy for Sophistication. 0.17 15   Replication of Results Using Feeling Thermometer Outcome As an alternative outcome variable, we measured respondent’s attitudes toward the mayor using a feeling thermometer. In English, the question asked, “What grade, on a scale from 1 to 7, would you give to Mayor Carlos, where 1 means that you think he is a terrible mayor, and 7 means that you think he is an excellent mayor?” The original Portuguese question is reported in section 2 above. The patterns that we observe are the same as with the vote intention variable. Across all of the treatment conditions, respondents express less enthusiasm for the mayor. All of these differences are statistically significant. The responses on the feeling thermometer between the more and less credible accusations and between the less credible accusations and the unsourced accusations are statistically distinguishable from each other, whereas the responses between the unsourced and the credible treatments are not statistically distinguishable. Using education as the proxy for political sophistication, we again find that discernment between sources is increasing in education with statistically significant differences between the more and less educated groups observed for all groups below the least educated and with the difference being the largest in magnitude for the most educated group. Using political knowledge as the proxy for political sophistication, we again find that the most and least knowledgeable do separate but that the respondents who answered a single political knowledge question right rated the mayors in the credible condition more highly in a way that makes their level of discernment appear lower. 16   Average Estimated Estimated Feeling Response Difference from Difference from thermometer Control Unsourced (1=terrible mayor; (Standard Error) [95% CI] Conditions Accusations 7=great mayor) (N) Credible 3.56 -1.93 -0.03 Accusations (0.08) (0.10) (0.13) (N=572) [3.41, 3.71] [p < 0.01] [p < 0.82] Less Credible 4.01 -1.48 0.42 Accusations (0.08) (0.10) (0.13) (N=572) [3.85, 4.16] [p < 0.01] [p < 0.01] No Source of 3.59 -1.90 Corruption (0.11) (0.12) -Accusations [3.38, 3.80] [p < 0.01] (N=286) Pure 5.49 Control/Control (0.07) --with Clean Mayor [5.36, 5.62] (N=572) Replication of Table 2 for Feeling Thermometer Outcome. Estimated Difference from Less Credible Accusations -0.45 (0.11) [p < 0.01] --- -- Illiterate/Less than primary Complete primary; incomplete middle Complete middle; incomplete secondary Complete secondary At least some tertiary 3.81 (0.25) N=64 3.98 (0.15) N=148 3.95 (0.16) N=130 4.18 (0.16) N=137 4.01 (0.18) N=93 Credible 3.90 (0.25) N=68 3.49 (0.15) N=132 3.48 (0.16) N=147 3.61 (0.15) N=148 3.42 (0.19) N=77 Difference -0.08 0.48 0.47 0.56 0.59 0.01 0.03 Feeling thermometer (1=terrible mayor; 7=great mayor) Not credible p-value on 0.82 0.03 0.04 H0: No Difference Replication of Table 3 for Feeling Thermometer Outcome 17   No Political One Political Knowledge Knowledge Questions Right Question Right 3.94 4.03 (0.10) (0.17) N=340 N=125 Credible Accusations 3.52 3.80 (0.09) (0.19) N=368 N=110 Difference 0.41 0.23 (0.14) (0.25) 0.01 0.36 p-value on H0: No Difference Replication of Table 4 for Feeling Thermometer Outcome Feeling thermometer (1=terrible mayor; 7=great mayor) Less Credible Accusations Both Political Knowledge Questions Right 4.21 (0.17) N=107 3.43 (0.19) N=94 0.78 (0.25) 0.01 18