add NYPD data to the paper

ms/face_of_crime.tex

@@ -63,16 +63,16 @@
 
 
 \begin{abstract}
-\noindent Race, gender, and crime are inextricably linked in people's minds. And television programming is thought to strongly influence how they are linked. We investigate the extent to which popular television programming perpetuates stereotypical linkages by tallying the race and gender of criminals and victims in three popular series of the most successful criminal procedural franchise on television---Law \& Order. Using data from a census of the shows from aired seasons of \textit{Special Victims Unit} and \textit{Criminal Intent} series, and data from seven seasons of the \textit{Original} series, we find that whites and women are overrepresented (and blacks and men underrepresented), both as victims and as criminals. In particular, blacks are dramatically underrepresented both as criminals and as victims, with actual arrest rate and violent victimization rate of blacks nearly 300\% and 200\% respectively of the commensurate numbers for the show. 
+\noindent Race, gender, and crime are inextricably linked in people's minds. These linkages exist in part because of what is shown of television. We investigate the extent to which popular television programming perpetuates stereotypical linkages between race, gender, and crime. We tally the race and gender of criminals and victims in three popular series of the most successful criminal procedural franchise on television---Law \& Order. Using data from a census of the shows from aired seasons of \textit{Special Victims Unit} and \textit{Criminal Intent} series, and data from seven seasons of the \textit{Original} series, we find that whites and women are overrepresented (and blacks and men underrepresented), both as victims and as criminals. In particular, blacks are dramatically underrepresented both as criminals and as victims, with actual arrest rate and violent victimization rate of blacks nearly 300\% and 200\% respectively of the commensurate numbers for the show. 
 \end{abstract}
 \clearpage
 \doublespace
 
-Among the most foundational of the cognitive biases is the tendency to overweight accessible information \citep{tversky1973availability, iyengar2010news, iyengar1990accessibility}. For instance, one of the reasons comparisons with peers weigh heavily in people's assessments of their own lives is because information about peers is much more readily available. Given the importance of accessible information in attitude formation and decision making, what is covered on television has been a natural focus of inquiry. 
+Among the most foundational of the cognitive biases is the tendency to overweight accessible information \citep{tversky1973availability, iyengar2010news, iyengar1990accessibility}. Given the importance of accessible information in attitude formation and decision making, what is covered on television has been a natural focus of inquiry. 
 
-One particularly important topic in that inquiry has been whether television programming propagates aversive racial stereotypes. Common longstanding aversive stereotypes about blacks are that they are lazy, poor, unintelligent and violent \citep{katz1933racial}. And much research shows that such stereotypes shape political attitudes \citep{sniderman1995scar, hurwitz1997public, peffley1997racial, dixon2006psychological} and affect economic behavior \citep{bertrand2004emily}. 
+One particularly important topic in that inquiry has been whether television programming perpetuates aversive racial stereotypes. Common longstanding aversive stereotypes about blacks are that they are lazy, poor, unintelligent and violent \citep{katz1933racial}. And much research shows that such stereotypes shape political attitudes \citep[see, for e.g.,][]{sniderman1995scar, hurwitz1997public, peffley1997racial, dixon2006psychological} and affect economic behavior \citep[see, for e.g.,][]{bertrand2004emily}. 
 
-Vitally, research also suggests that these stereotypes are correlated with exposure to television programming \citep{busselle2002television, entman2001black, armstrong1992tv}. For instance, exposure to television news is correlated with the extent to which college students endorse that blacks are lazy \citep{busselle2002television}. Similarly, \citet{armstrong1992tv} find that college students who consume more television news were more likely to believe that blacks had lower socio-economic outcomes. (Albeit, ``TV drama exposure was associated with beliefs that Black Americans had a relatively higher socio-economic standing.'') A comprehensive study of how blacks are portrayed in the media by \citet{entman2001black} reached similar conclusions---it found that nonfictional portrayals of African Americans were associated with negative stereotypes of blacks, including, for instance, the perception that they are hostile.
+Vitally, research also suggests that these stereotypes are correlated with exposure to television programming \citep{busselle2002television, entman2001black, armstrong1992tv}. For instance, exposure to television news is correlated with the extent to which college students endorse that blacks are lazy \citep{busselle2002television}. Similarly, \citet{armstrong1992tv} find that college students who consume more television news are more likely to believe that blacks had lower socio-economic outcomes. (Albeit, ``TV drama exposure was associated with beliefs that Black Americans had a relatively higher socio-economic standing.'') A comprehensive study of how blacks are portrayed in the media by \citet{entman2001black} reached similar conclusions---exposure to nonfictional portrayals of African Americans was associated with negative stereotypes of blacks.
 
 Given the longstanding concern, numerous researchers have studied how blacks are covered on various television programs \citep[for e.g.,][]{entman2001black, eschholz2004images}. One particular focus of the effort---given the stereotype of blacks as violent and criminal, and given the impact such stereotypes may have on policy consequential attitudes related to punitiveness---has been on describing the extent to which blacks are overrepresented as criminals, especially as violent criminals. 
 
@@ -84,7 +84,7 @@
 
 \section*{Learning from Television}
 
-We are constantly learning from our environment, though not always effortfully. One salient place we learn from is the mass-media. For instance, local news, the most common kind of news programming that people tune into \citep{pew2004}, carries lots of episodic coverage of violent crime \citep[see, for instance,][]{gross2006covering, klite1997local}. And watching multiple daily reports of violent crime may cause them to think that violent crime is `common'---a phrase that the person internally maps to a number much larger than actual incidence of violent crime.
+We constantly learn from our environment, though not always effortfully. One salient place we learn from is the mass-media. For instance, local news, the most common kind of news programming that people tune into \citep{pew2004}, carries lots of episodic coverage of violent crime \citep[see, for instance,][]{gross2006covering, klite1997local}. And watching multiple daily reports of violent crime may cause them to think that violent crime is `common'---a phrase that the person internally maps to a number much larger than actual incidence of violent crime.
 
 Similar, though subtler, mechanism likely holds when someone watches one of the tens of hundreds of crime dramas on television. A person watching a crime drama, aware of its fictional nature, may still inadvertently think that the drama accurately `mirrors' society. This inference may be encouraged by prior interpretation and acceptance of imprecise flawed statements such as `art reflects life' as true. Increasingly common mass-media tropes such as realism in depiction and explicit statements designed to cue realism may also increase the chances that people take fictional portrayals as reasonably accurate depictions of reality. As a case in point, all Law \& Order \textit{Special Victims Unit} series episodes start with the statement: ``...In New York City, the dedicated detectives who investigate these vicious felonies are members of an elite squad known as the Special Victims Unit. These are their stories.'' It is not unreasonable to assume that someone listening to the statement may come to think that the crimes covered in the drama represent a broad, perhaps representative, set of all ``sexually based crimes.''
 
@@ -101,7 +101,7 @@ \section*{Data and Measurement}
 
 We coded victims' and criminals' race as white, black, Asian, or Hispanic, following the general census guidelines.\footnote{\url{http://www.census.gov/topics/population/race/about.html}} Characters of Middle-Eastern descent were coded as white, but a note marking their specific background is included in the data. And under `Hispanics,' we included non-Hispanic Latinos such as Brazilians.
 
-In addition to the data on the race and gender of the victims and the criminals from the shows, we collected data on a few baselines. In particular, from the 1990, 2000, and 2010 census, we collected data on the proportion of whites, blacks and men in both the US and New York City. Like \citet{dixon2000overrepresentation}, from the Federal Bureau of Investigation's Uniform Crime Report (UCR), we gathered data on the percentage of blacks, whites and men arrested for \textit{All Crimes}, \textit{Violent Crimes}, and \textit{Sexually Based Crimes}. Lastly, from the Bureau of Justice Statistics' National Crime Victimization Survey (NCVS) \citep{powers2016national, victimization1998national}, we gathered data on the race and gender of victims of crime.  
+In addition to the data on the race and gender of the victims and the criminals from the shows, we collected data on a few baselines. From the 1990, 2000, and 2010 census, we collected data on the proportion of whites, blacks and men in both the US and New York City. Like \citet{dixon2000overrepresentation}, from the Federal Bureau of Investigation's Uniform Crime Report (UCR), we gathered data on the percentage of blacks, whites and men arrested for \textit{All Crimes}, \textit{Violent Crimes}, and \textit{Sexually Based Crimes}. From NYPD's Crime and Enforcement Activity in New York City Report (NYPD),\footnote{\url{http://www.nyc.gov/html/nypd/html/analysis_and_planning/crime_and_enforcement_activity.shtml}} we collected data on percentage of blacks, whites and Hispanics who were suspected of (and were victims of) murder, rape, assault and robbery each year between 2008 and 2015. Lastly, from the Bureau of Justice Statistics' National Crime Victimization Survey (NCVS) \citep{powers2016national, victimization1998national}, we gathered data on the race and gender of victims of crime.  
 
 The UCR data on race of alleged criminals is available for 1995--2014. (For 2015, only preliminary data is currently available. And the preliminary release does not include information on race.) In UCR, race is split into white, black, American Indian or Alaskan Native, Asian, and Native Hawaiian or Other Pacific Islander. The totals in these categories include Hispanics; data on Hispanic background only began to be collected in 2013. 
 

scripts/02_agg_analysis.R

@@ -72,6 +72,12 @@ murder_v_means        <- colMeans(murder_victims)
 ucr        <- read.csv("data/ucr/race_gender_criminals.csv")
 ucr_means  <- colMeans(ucr)
 
+# NY Data  
+ny        <- read.csv("data/ny_enforcement/nyc_enforcement.csv")
+names(ny) <- tolower(names(ny))
+ny        <- as.data.frame(lapply(ny, function(x) as.numeric(gsub("%", "", x))))
+ny_means  <- colMeans(ny, na.rm=T)
+
 # -----------
 # Custom theme
 
@@ -154,7 +160,7 @@ out_murder <-
 lo_murder %>%
 group_by(series) %>%
 summarise(male=sum(n_c_male, na.rm=T), female=sum(n_c_female, na.rm=T), pfemale = female*100/sum(male, female))
-rout_murder <- out_mrdraslt[,c("series", "pfemale")]
+rout_murder <- out_murder[,c("series", "pfemale")]
 
 # Criminals by Gender
 out <- 
@@ -206,13 +212,14 @@ group_by(series) %>%
 summarise(white=sum(n_v_white, na.rm=T), black=sum(n_v_black, na.rm=T), pblack = black*100/sum(white, black))
 rout <-  out[,c("series", "pblack")]
 
-# UCR and NCVS
+# UCR and NCVS and NY 
 ucr_v_race  <- data.frame(series="UCR",  pblack = murder_v_means["murder_victim_black"])
 ncvs_v_race <- data.frame(series="NCVS", pblack = ncvs_means[c("rape_sexual_assault_black", "serious_violent_victimization_black")])
+ny_v_race   <- data.frame(series="NYPD", pblack=ny_means[c("homicides_black_victim", "rapes_black_victim")])
 
 # All/Rape/Murder 
-rout2 <- cbind(rbind(rout, rout_murder, rout_rape, ucr_v_race, ncvs_v_race), crime = c(rep("All",3), rep("Murder", 3), "Rape", "Murder", "Rape", "Serious Violent Victimization"))
-rout2$series <- factor(rout2$series, levels=c("UCR", "NCVS", "SVU", "Criminal Intent", "Original"))
+rout2 <- cbind(rbind(rout, rout_murder, rout_rape, ucr_v_race, ncvs_v_race, ny_v_race), crime = c(rep("All",3), rep("Murder", 3), "Rape", "Murder", "Rape", "Serious Violent Victimization", "Murder", "Rape"))
+rout2$series <- factor(rout2$series, levels=c("NYPD", "UCR", "NCVS", "SVU", "Criminal Intent", "Original"))
 
 ggplot(rout2, aes(series, pblack, color=crime)) +
 geom_point(pch=16, size=3, alpha=.55) + 
@@ -251,10 +258,11 @@ rout <- out[,c("series", "pblack")]
 
 # UCR 
 ucr_c_race <- data.frame(series="UCR",  pblack = ucr_means[c("all_crime_perc_black", "homicides_perc_black", "rape_perc_black")])
+ny_c_race   <- data.frame(series="NYPD", pblack=ny_means[c("homicides_black_suspect", "rapes_black_suspect")])
 
 # All/Rape/Murder 
-rout2 <- cbind(rbind(rout, rout_murder, rout_rape, ucr_c_race), crime = c(rep("All",3), rep("Murder", 3), "Rape", c("All", "Murder","Rape")))
-rout2$series <- factor(rout2$series, levels=c("UCR", "NCVS", "SVU", "Criminal Intent", "Original"))
+rout2 <- cbind(rbind(rout, rout_murder, rout_rape, ucr_c_race, ny_c_race), crime = c(rep("All",3), rep("Murder", 3), "Rape", c("All", "Murder","Rape"), "Murder", "Rape"))
+rout2$series <- factor(rout2$series, levels=c("NYPD", "UCR", "NCVS", "SVU", "Criminal Intent", "Original"))
 
 ggplot(rout2, aes(series, pblack, color=crime)) +
 geom_point(pch=16, size=3, alpha=.55) + 

scripts/03_ts_analysis.R

@@ -50,6 +50,11 @@ names(murder_victims) <- tolower(names(murder_victims))
 # UCR
 ucr        <- read.csv("data/ucr/race_gender_criminals.csv")
 
+# NY Data  
+ny        <- read.csv("data/ny_enforcement/nyc_enforcement.csv")
+names(ny) <- tolower(names(ny))
+ny        <- as.data.frame(lapply(ny, function(x) as.numeric(gsub("%", "", x))))
+
 # Victims by Gender
 # ----------------------------
 
@@ -222,6 +227,7 @@ w_out <- out[,c("pblack", "series", "year")] %>% spread(series, pblack)
 all_out_v_race <- 
 w_out %>% 
 left_join(ncvs[,c("year", "serious_violent_victimization_black", "rape_sexual_assault_black", "aggravated_assault_black")], by="year") %>%
+left_join(ny[,  c("year", "homicides_black_victim", "rapes_black_victim", "assaults_black_victim", "robberies_black_victim")], by="year") %>%
 left_join(murder_victims[,c("year", "murder_victim_black")], by="year") %>%
 left_join(census[,c("year", "black_us", "black_ny")], by="year")
 
@@ -233,7 +239,11 @@ c(year = "Year",
   serious_violent_victimization_black = "NCVS__Serious Violent Victimization",  
   rape_sexual_assault_black           = "NCVS__Rape or Sexual Assault",        
   aggravated_assault_black            = "NCVS__Aggravated Violent Victimization",            
-  murder_victim_black                 = "UCR__Murder Victims",               
+  murder_victim_black                 = "UCR__Murder Victims", 
+  homicides_black_victim              = "NYPD__Homicides", 
+  rapes_black_victim                  = "NYPD__Rape", 
+  assaults_black_victim               = "NYPD__Assault",
+  robberies_black_victim              = "NYPD__Robbery",               
   black_us                            = "Census__US",  
   black_ny                            = "Census__NY"
 )
@@ -245,13 +255,14 @@ colMeans(all_out_v_race[,5:8], na.rm=T)
 
 print_2heading_xtable(all_out_v_race, 
 	separator = "__",
-	digits=c(0,0,rep(1,9)),
+	digits=c(0,0,rep(1,13)),
 	caption="Share of Black Victims in Law \\& Order, and the Real World, and Share of Blacks in the Population", 
 	label="tab:v_race",
 	caption.placement="top",
 	size="\\tiny", 
+	floating.environment = "sidewaystable",
 	heading_command = NULL,
-	xtable.align = c("l", rep("c", 10)),
+	xtable.align = c("l", rep("c", 14)),
 	sanitize.text.function = function(x){x},
 	table.placement="!htb",
 	file="tabs/v_race.tex")
@@ -286,19 +297,24 @@ w_out <- out[,c("pblack", "series", "year")] %>% spread(series, pblack)
 # Append data
 all_out_c_race <- 
 w_out %>% 
-left_join(ucr[,c("year", "all_crime_perc_black", "violent_crime_perc_black", "homicides_perc_black", "rape_perc_black", "assault_perc_black")], by="year") %>%
-left_join(census[,c("year", "black_us", "black_ny")], by="year")
+left_join(ucr[, c("year", "all_crime_perc_black", "violent_crime_perc_black", "homicides_perc_black", "rape_perc_black", "assault_perc_black")], by="year") %>%
+left_join(ny[,  c("year", "homicides_black_suspect", "rapes_black_suspect", "assaults_black_suspect", "robberies_black_suspect")], by="year") %>%
+left_join(census[, c("year", "black_us", "black_ny")], by="year") 
 
 renames <- 
 c(year = "Year", 
   "Criminal Intent" = "Law and Order__Criminal Intent",
   Original = "Law and Order__Original",                           
   SVU = "Law and Order__SVU",
   all_crime_perc_black      = "UCR__All Crime",
-  violent_crime_perc_black = "UCR__Violent Crime",
+  violent_crime_perc_black  = "UCR__Violent Crime",
   homicides_perc_black      = "UCR__Homicides",
   rape_perc_black           = "UCR__Rape",
   assault_perc_black        = "UCR__Assault",
+  homicides_black_suspect   = "NYPD__Homicides", 
+  rapes_black_suspect       = "NYPD__Rape", 
+  assaults_black_suspect    = "NYPD__Assault",
+  robberies_black_suspect   = "NYPD__Robbery", 
   black_us                  = "Census__US",  
   black_ny                  = "Census__NY"
 )
@@ -307,13 +323,13 @@ names(all_out_c_race) <- renames[match(names(all_out_c_race), names(renames))]
 
 print_2heading_xtable(all_out_c_race, 
 	separator = "__",
-    digits=c(0, 0,rep(1,10)),
+    digits=c(0, 0, rep(1,14)),
 	caption="Share of Black Criminals in Law \\& Order, and the Real World, and Share of Blacks in the Population", 
 	label="tab:c_race",
 	caption.placement="top",
 	size="\\tiny", 
 	heading_command = NULL,
-	xtable.align = c("l", rep("c", 11)),
+	xtable.align = c("l", rep("c", 15)),
 	sanitize.text.function = function(x){x},
 	table.placement="!htb",
 	file="tabs/c_race.tex")
@@ -333,3 +349,4 @@ cust_theme
 direct.label(p, list(last.points, cex=.8, alpha=1, hjust = 0, vjust = -.75))
 
 ggsave("figs/all_criminals_by_race_ts.pdf", dpi=450, width=7.5)
+