Some Economics of Stop and Frisk

If almost every time the police stopped and frisked someone, they found an illegal weapon or drugs, or identified a wanted criminal, then complaints about the practice would ring hollow. On the other hand, if the police almost never found evidence of a crime during a stop and frisk, then complaints about the practice would take on a sharpened urgency.

Other evidence would be nice, too. It would be nice to know on what grounds the police are making decisions to stop and frisk, and whether some reasons for stop and frisk are more likely or less likely to lead to evidence of a crime. It would be nice to know the extent to which the well-known racial differences in stop-and-frisk are related to the practice occurring more in higher poverty, higher crime areas, which also have a racial imbalance. It would be nice to have some evidence on whether stop-and-frisks are more likely to lead to evidence of a crime for whites or blacks.

Sharad Goel, Justin M. Rao And Ravi Shroff offer some evidence and analysis on these kinds of questions in their research paper "Precinct Or Prejudice? Understanding Racial Disparities In New York City’s Stop-And-Frisk Policy," which appeared earlier this year in  the Annals of Applied Statistics (2016, 10:1, 365–394).

The authors have data on 2.9 million stops conducted by New York City police officers between 2008 and 2012. They write:

"Following a stop, officers complete a UF-250 stop-and-frisk form, recording various aspects of the stop, including demographic characteristics of the suspect, the time and location of the stop, the suspected crime and the rationale for the stop (e.g., whether the suspect was wearing clothing common in the commission of a crime). ... After an individual is stopped, officers may conduct a frisk (i.e., a quick patdown of the person’s outer clothing) if they reasonably suspect the individual is armed and dangerous; officers may additionally conduct a search if they have probable cause of criminal activity. Frisks and searches occur in 56% and 9% of cases, respectively. An officer may decide to make an arrest (6% of instances) or issue a summons (6% of instances), all of which is recorded on the UF-250 form."

Of course, it's sensible to be skeptical about the quality of this evidence. For example, one might raise questions about how frequently or accurately these UF-250 forms are filled out. One answer to this concern is that because of past court cases, the NYPD has some explicit emphasis on filling out the forms, and filling them out accurately. Also, with data on several million forms, one should be able to learn something, even if lessons should be drawn with appropriate caution.

The researchers focus most of their discussion in this study on the 760,000 cases where the reason for the stop was suspicion of criminal possession of a weapon. This group of stops is useful to study because it's the largest single reason for such stops, and because the data shows whether a weapon was actually found, or not, which focuses on a specific crime, rather than jumbling all crimes together. The authors look at data mostly from 2009-2010, and calculate what factors listed on the UF-250 form--including personal characteristics of the suspect, the specific factors that the officer observed that led to the stop, and the location as determined by the police precinct-- make it more or less likely that a weapon was actually found.  The result is a big messy statistical calculation, which for 2009-2010 includes 301,000 stops and 7,705 different variables (the large number of variables is because they look at a bunch of potential variables both individually and in how the variables might interact with each other).

Here's the payoff:  The authors can use the answers from their calculations on the first few years of the data to predict how likely any given stop-and-frisk looking for criminal possession of a weapon was to find such a weapon in the 2011-2012 data. For example, if an officer in a certain precinct stopped someone for criminal possession of a weapon because the person was acting furtively, and wearing suspicious clothing at a certain time of day in a certain  precinct, what was the chance (based on data from the earlier time period) that the person actually had a weapon? If an officer in another precinct stopped someone for criminal possession of a weapon who was near a crime scene, fitted a witness report, and changed direction when the officer came into view, what is the chance that that person actually had a weapon?  The researchers write (citations, footnotes, and references to figures omitted for readability):

We find that in 43% of the approximately 300,000 CPW [criminal possession of a weapon] stops between 2011 and 2012, there was at most a 1% chance of finding a weapon on the suspect. We note that the recovered weapons are typically knives, with guns constituting approximately 10% of found weapons. ...

In particular, consistent with past results, the overall hit rates for blacks and Hispanics (2.5% and 3.6%, resp.) are considerably lower than for whites (11%). In other words, these results indicate that when blacks and Hispanics are stopped, it is typically on the basis of less evidence than when white suspects are stopped. Moreover, while 49% of blacks stopped under suspicion of CPW [criminal  possession of a weapon] have less than a 1% chance of in fact possessing a weapon, the corresponding fraction for Hispanics is 34%, and is just 19% for stopped whites. Thus, if we equate reasonable suspicion with a particular probability threshold (say 1%), a far greater fraction of stops of blacks and Hispanics are unwarranted than are stops of whites. ... 

However, ... whites and minorities are typically stopped in different contexts, and so differing hit rates may not be the result of racial bias. Indeed, as we discuss below, stop-and-frisk is an extremely localized tactic, heavily concentrated in high-crime, predominantly black and Hispanic areas, and so lower tolerance for suspicious activity (and hence lower hit rates) in these areas could account for the racial disparity. ...
[T]here is an almost one-to-one correspondence between areas with heavy use of stop-and-frisk. While this is a natural and possibly effective policing strategy, a consequence of the tactic is that individuals who live in high-crime areas, but who are not themselves engaged in criminal activity, bear the costs associated with being stopped. ... [T]hese high-crime areas are overwhelmingly black and Hispanic. Accordingly, the cost of stop-and-frisk is largely shouldered by minorities. ... [W]e see that the racial composition of stopped individuals is similar to the racial composition of the neighborhoods in which stop-and-frisk is heavily employed. Thus, the striking racial composition of stopped CPW [criminal possession of a weapon] suspects (61% are black, 30% are Hispanic and 4% are white) appears at least qualitatively attributable to selective use of stop-and-frisk in minority-heavy areas ...

In a more detailed analysis of the data, they find that the differing use of stop-and-frisk across neighborhoods accounts for part of gap by which blacks and Hispanics are stopped and frisked more than whites, but not for all of it. 

Another intriguing aspect of the study is that it can answer the question of what reasons--and remember, these are the reasons given by the police themselves--are more likely to uncover a concealed weapon. The UF-250 report lists 18 specific "stop circumstances" (there's also a category for "other," which they ignore). The 18 circumstances are: suspicious object, fits description, casing, acting as lookout, suspicious clothing, drug transaction, furtive movements, actions of violent crime, suspicious bulge, witness report, ongoing investigation, proximity to crime scene, evasive response, associating with criminals, changed direction, high crime area, time of day, sights and sounds of criminal activity.

The question is whether some of these are more likely, as revealed by the actual evidence, to lead to actual discovery of a criminal concealed weapon than others. The authors look at these 18 factors together with each of the 77 police precincts and also whether the stop-and-frisk happened at a public housing location, a transit stop, or elsewhere. Notice that this is not an exercise in 20:20 hindsight: instead, it's looking at the circumstances that police actually reported seeing at the time, and then seeing what worked. Basically, they find that three  of the circumstances were good predictors of a criminal concealed weapon: suspicious object, sights and sounds of criminal activity, suspicious bulge. The other 15 circumstances were either barely connected to finding a concealed weapon, or not connected at all.

An obvious policy choice suggests itself here. The NYPD are often using stop-and-frisk to look for weapons based on the policy observing certain circumstances that are quite unlikely to be associated with a concealed weapon. If the NYPD no longer stopped people on suspicion of  suspicion of criminal possession of a weapon based on furtive movements, acting as a lookout, changing direction, and many of the other reasons given, it could focus on the circumstances that are more likely to actually end up finding a concealed weapon. The authors write:

In particular, we show that one can recover 50% of weapons by conducting only the 6% of CPW [criminal possession of a weapon] stops with the highest ex ante hit rate, and 90% of weapons by conducting 58% of CPW stops. These ex ante hit rates are based only on information observable to officers prior to the stop decision, and so it is at least in theory possible to implement such a strategy. Further, since low hit rate stops disproportionately involve blacks and Hispanics, optimizing for weapons recovery would simultaneously bring more racial balance to stop-and-risk.To facilitate adoption of such strategies by police departments, we develop stop heuristics that approximate our full statistical model via a simple scoring rule. Specifically, we show that with a rule consisting of only three weighted stop criteria, one can recover the majority of weapons by conducting 8% of stops.

An obvious concern about these results is that perhaps stopping and frisking people for criminal possession of a weapon is just an excuse, but it's nonetheless a useful excuse for reducing the rate of crime. The authors discuss the point this way:

A possible objection to our approach is that even for CPW [criminal possession of a weapon] stops, recovering weapons is not the only—or perhaps not even the primary—goal of the police. Officers, for example, may simply consider stops a way to advertise their presence in the neighborhood or a means to collect intelligence on criminal activity in the area, regardless of how many weapons are directly recovered. Stops conducted for these alternative motives could quite plausibly deter individuals from carrying weapons and might lead to information helpful in solving cases, both of which presumably would lower the incidence of violent crime over time. In the instances we consider, however, the explicitly stated reason for a stop is suspicion of criminal possession of a weapon, not one of the various other reasons that may or may not withstand legal or public scrutiny, and so it seems most natural to consider whether individuals were in fact likely to be carrying weapons. Moreover, as we have previously noted, simply because a strategy may be effective does not make it legal. ... A related worry is that “criminal possession of a weapon” is a catchall category for a variety of criminal offenses, and so by focusing on whether a weapon was found, we underestimate the value of a stop. Addressing this issue, we observe that our results are qualitatively similar if we instead use arrests [for any reason] as the outcome variable, mitigating cause for concern.

The authors instead draw the conclusion that stop-and-frisk can be a useful tool for police work, but when it comes to criminal possession a weapon, it's a tool that's being overused. They conclude:

By focusing on the relatively small number of high hit rate situations—situations that can be reliably identified via statistical analysis—one may be able to retain many of the benefits of stop-and-frisk for crime prevention while mitigating constitutional violations. This observation has the potential to not only improve New York City’s stop-and-frisk program, but could also aid similar policies throughout the country.