What Do They Know and When Do They Know It?
Using Public Opinion to Evaluate Institutional Performance

1. In assessing President Nixon's culpability in Watergate, then senator Howard Baker, noted that the Watergate Committee needed to know "what the President knew, and when did he know it?"
1. Same question must be asked about the public if we are to rely upon them to evaluate public institution's OPERATION.
2. Distinguish between IMAGE and OPERATION.
1. If reform/change is based on IMAGE as opposed ACTUAL operation, changes are unlikely to achieve their goals
2. IMAGE is important, but if the IMAGE is not based in OPERATION changing OPERATION will be unlikely to change IMAGE.
3. Ohio Rules of Superintendence example



2. Public often does not have a good base of information to evaluate institutions.
1. Own area is courts, and so my examples will focus on that setting
2. Courts are typically seen as slow, expensive, inefficient, inattentive, etc.



3. Wisconsin study of "consumer satisfaction"
1. Three surveys
2. Not expecting the pattern we found, so not designed specifically to pick it up
3. "Conventional Wisdom" from 1978 Yankelovich survey





4. Closer one is to direct experience with courts, the more positive courts are evaluated
1. A study in Virginia several years ago found a similar pattern



4. Crime, and the treatment of criminals by the courts, has been a major subject of concern in the U.S. since at least the 1960s.





1. Often appears among the "most important problems" facing the country as measured by public opinion polls, particularly in recent years [BLUE IN GRAPH]
2. Public has unwavered in its view that too little is spent to fight crime [RED ON GRAPH]





3. Public has for some time felt that courts have been too lenient in dealing with criminals [BLUE ON GRAPH]
4. Yet the public seems unaware that the courts have been sending increasing numbers of persons to prison





1. Same graph as previous but adding Inmates/UCI Arrests (PURPLE)
2. NOTE: Typical sentences have not increased substantially over the last 20 years, except for drug offenses (particularly at the federal level), so in that sense the public may be right.



5. General patterns of what political scientists call "diffuse" support of the courts has remained high vis-a-vis the U.S. Supreme Court





1. BLUE line shows relatively high percentage of population reporting that they have a very favorable or mostly favorable view of the SC
2. For the court system as a whole, the picture is more mixed, with favorable views (RED) only recently surpassing unfavorable (GREEN)



6. It would seem safe to say that the public has more day to day contact with lower courts than with the Supreme Court. In fact, the public has little knowledge of what the Supreme Court actually does in terms of the decisions it makes.





1. Explain the survey (1992)
1. GREEN for Casey v. Planned Parenthood
2. BLUE for St. Paul cross burning case
3. RED for the Cippolino tobacco case





2. Compare to presidential events
1. At first glance looks similar, but this is because the scaling of the vertical axis differs





3. Here is the court graph scaled to the same as the presidential events graph
1. EMPHASIZE that Casey was one of the most anticipated decisions, and talked about decisions, in years.
4. The absence of knowledge about Supreme Court decisions is not surprising given that the court (1) does not permit television coverage, and (2) the national television media devotes little time to the Court

TELEVISION NEWS COVERAGE OF
THE SUPREME COURT AND OTHER NATIONAL INSTITUTIONS






1. This graph shows the coverage of the Court, the President, and Congress
1. Top shows the number of stories (LEFT) or amount of time (RIGHT) per month for three time periods
2. Bottom shows the distribution
2. The share for the Court is shown in GREEN-its to see!
5. This illustrates one of the key problem in using public opinion surveys to evaluate institutions: The public lacks information.



7. The other key problem is that the information provided by the media is often flawed, either inherently in the definition of what constitutes news or because of flawed reporting
1. A recent study of reporting of Supreme Court decisions by the major Television Networks found that news reports frequently misreported that the Supreme Court had decided a case, when the Court had in fact simply declined to decide the case.
2. However, what is more important, is that the routine is not news, and hence the public has little knowledge of the routine within the Courts. The public hears about the Courts only when something big or unusual happens.
3. This is clearly illustrated in the reporting of verdicts of civil juries.
1. I found five surveys over the last dozen years or so that asked respondents whether or not they believed that jury awards in personal injury cases were "too large" or "excessive". Depending on how the question was worded, between 45% and 58% of the respondents saw awards as too large.
2. This is not surprising given what people hear about awards from the media.
3. Several studies have compared cases reported in the press to populations of cases









4. The BLUE in the top graph show the Median jury awards reported in national magazines, New York Times, and New York Newsday. The comparisons to populations of awards are shown in RED (hard to see because they are so much lower).
1. The smallest median in reported cases was about $1.8 million; the largest population median was about $200,000, and the most representative samples found medians of $50,000 to $60,000
2. Note that the differences would be even starker if I used means rather than medians
5. The bottom graph shows the probability of plaintiffs winning. Note that in the cases reported in the news media, the plaintiff win rate is 80% or more (BLUE) while the overall win rate is around 50%, and under 30% in some areas such as auto product liability (shown in the graph) and medical malpractice (not shown in the graph).



8. My conclusion is very straight-forward: before relying on public opinion-based evaluations of institutions, we need to ask what is reasonable to expect the public to know about the institution given the information sources they have available?
1. It may be important to know whether or not the public views institutions positively or negatively, even if we do not rely upon that information to structure reform or change efforts.
2. However, the implications of uninformed evaluations may have more to do with institutional legitimacy, and the ability of institutions to move in controversial directions, than it has to do with whether or not the institutions are doing a good or bad job of what we want them to do.