"Number Jumble Clouds Judgment of
Drug War" - The Washington Post Gets Critical?
Differing Surveys, Analyses Yield Unreliable Data
http://search.washingtonpost.com/wp-srv/WPlate/1998-01/02/111l-010298-idx.htmlEd. Note: While this article is important for what it says, it is even
more important for having originated in the Washington Post, which is slavish in
following the prohibitionist party line. This is the second "deviationist" major
article to appear in the Post recently.(See: "Hooked on Dogma" Amazingly, Washington Post Deviates
from the Prohibitionist Party Line ) Retired
Krimlinologists who used to make a career reading between the lines of Pravda
could possibly put their skills to work here. Could there be glasnost at the
Post? Of course, the original purpose of glasnost was to make communism work, so
the Post quotes only one anti-prohibitionist. All the rest are academics determined to
make it work. They want prohibition "with a human face." The truth is very
dangerous.
By Jeff Leen
Washington Post Staff Writer
January 2, 1998; Page A01 (Note: page 1!)
As the election season began gearing up in late 1991, President George
Bush got an unsettling bit of front-page news:
The number of habitual cocaine users in the United States had jumped an
astounding 29 percent in a single year, from 662,000 to 855,000, according to the National
Institute on Drug Abuse (NIDA). Bush had aggressively pushed his administration's
anti-drug effort. Now, he had little to show for it.
But the bad news, widely reported by newspapers across the country, was wrong.
NIDA had miscounted in its annual National Household Survey on Drug Abuse, one of the
nation's "leading drug indicators." A year later, without fanfare, the number of
habitual users was revised back down to 625,000.
"Problems with statistical imputation," the General Accounting Office
concluded in a 1993 report on the miscalculation that received
little public attention. "We certainly think that more adequate quality
control procedures could have caught findings of such significant policy relevance."
The 1991 cocaine mistake stands out as just one example of the tenuous grasp
scientists, politicians, the media and the public have in evaluating America's 25-year
crusade against drugs. Different methods of calculating the number of drug users continue
to produce widely gyrating estimates, including those contained in the 1997 White House
drug strategy report that variously gives the number of habitual cocaine users as 582,000
and 2.2 million.
In spending a proposed $16 billion on the federal drug war in 1998 -- a 400
percent increase since 1986 -- lawmakers will rely on reams of data that often attempt to
impose statistical order on a chaotic social problem that defies easy analysis. Extensive
federally funded efforts to accurately assess the subterranean drug world have led to
contradictory findings and occasional statistical curiosities, such
as a 79-year-old female respondent whose avowed heroin usage in one survey resulted in a
projection of 142,000 heroin users, 20 percent of the national total.
"It's clear that these things are badly mismeasured
and nobody cares about it," said Peter Reuter, the former co-director of drug
research for the non-profit RAND think tank and now a University of Maryland professor. "That's because drug policy isn't a very analytically serious
business."
Measuring the drug war with any precision is a daunting task. Hard-core drug
users are hard to find, much less question, and people frequently lie on drug-use surveys
-- one study shows two-thirds of teenagers giving deceptive answers. Since surveys
typically receive only a small number of positive responses, analysts risk making
substantial errors in creating projections for the entire nation. Survey results sometimes
include warnings acknowledging these obstacles, such as "subject to large sampling
error" or "great caution should be taken."
But the caveats often are downplayed or ignored, either
by those issuing the data or by journalists and others promulgating the information.
In reporting the apparent 1991 jump in habitual cocaine use, for example, the White
House's Office of Drug Control Policy noted that the statistics were both "cause for
concern" and "highly unreliable."
The difficulty in measuring and evaluating the nation's illegal drug problem
made it harder to set policy, stoked partisan rhetoric and confused the public, drug
analysts say. Many experts, for example, believe cocaine and crack use are in decline, and
the federal household survey indicates that overall drug use is down 49 percent from its
peak of 25 million monthly users in 1979; yet many Americans still perceive the drug war
as perennially lost.
"You really can't tell from the big debate that goes on in public what the
big picture is," said David Musto, a Yale University medical historian who has
studied drug trends for three decades. "When I tell people about it, they're
completely surprised by the fact there has been a decline since 1980."
That big picture can be obscured by drug statistics that are "often
incomplete, erratic and contradictory," in the words of two RAND researchers funded
by the government to measure cocaine consumption. The first problem of drug war analysis
is the sheer number of measurements -- there are more than 50 federal drug-related
"data systems" with hundreds of "drug variables" produced by an array
of federal agencies.
For cocaine alone there are national statistics on casual use (at least once a
year), current use (at least once a month), frequent or habitual use (at least once a
week), crack use and use broken down by age, race and sex. There are stats on tonnage
consumed, purity, price per gram, price per kilo, patients reporting cocaine problems in
emergency rooms, patients seeking treatment and so forth.
"It's not that one thing is better than the other," said Eric Wish,
director of the Center for Substance Abuse Research at the University of Maryland.
"They all give a different piece of the puzzle, and they need to be put together. But
because of federal turf issues, it's more of an adversarial process than a collaborative
relationship."
Reuter said he has pointed out discrepancies in the habitual cocaine-use
figures in the national strategy report in the past, but the discordant numbers keep
appearing. On page 11 of the 1997 strategy, the count of habitual cocaine users is given as 582,000, a number that "has not changed markedly
since 1985." But in a chart on page 227 of the strategy's budget summary, the number
of such users is given as 2,238,000.
"I can't seem to get the machinery that cranks out these reports to pay
attention to these inconsistencies," Reuter said.
An official with the Office of National Drug Control Policy blamed the 1997
inconsistency on "sloppy writing." But the precise reasoning behind it gives a
glimpse into the problem of gauging the drug war. The warring numbers in this case come
out of different measuring methodologies -- one based on the household survey, the other
on urine tests of jail inmates -- that give radically different results.
"The truth is probably somewhere in the middle," said Joe Gfroerer,
who manages the household survey for the federal Substance Abuse and Mental Health
Services Administration (SAMHSA). "It's just a difficult thing to estimate."
Jared Hermalin, the GAO project manager who uncovered the 1991 cocaine mistake,
said: "There's every reason to believe that maybe the numbers are not absolutely
correct but the trends are correct. That's the main thing we need to know."
In recognition of the need for better analysis, the office of national drug
policy director Barry R. McCaffrey has proposed a comprehensive Performance Measurement
System intended, for the first time, to standardize measurement of the drug war.
"Facts should drive policy, but they haven't until very, very lately, with
McCaffrey," Sen. Joseph R. Biden Jr. (D-Del.), a longtime critic of the household
survey's measurement of hard-core cocaine use, said in an interview.
The proposed system shows just how complex measuring the drug war is. It
contains one mission statement with five goals, 32 objectives and 99 "targets"
that will be tracked by more than 111 "measures."
Even when the data is not marred by obvious statistical flaws, the sheer
profusion of it can baffle those looking for simple answers on whether the drug war has
been a success or failure. There is consensus that overall drug use,
as well as marijuana and cocaine use specifically, have declined dramatically since the
1970s. But that clarity soon clouds when researchers delve deeper.
For example, according to the household survey, current (monthly) cocaine use
decreased in the 1980s -- and was often cited as a sign of success; but, also according to
the household survey, hard-core (weekly) use did not drop, and that was cited as a sign of
failure. More recently, even as the household survey shows that the overall number of
cocaine users has declined (success), emergency room data shows that the number of people
seeking medical treatment for cocaine problems is rising (failure) as chronic addicts age
and their health deteriorates. And the household survey may show that overall drug use is
down (success), but a high school survey shows that teenage
marijuana use is up (failure).
For the past 25 years, the nation's most prominent gauge of illegal drug use
has been the national household survey, begun by NIDA in 1972 and taken over by SAMHSA in
1992. Government workers annually conduct one-hour, in-person interviews with a randomly
selected sample of 18,000 people, age 12 and up. From the answers, statisticians
extrapolate the size of the nation's drug-taking population.
The second most-publicized measurement is the NIDA-sponsored, 22-year-old
"Monitoring the Future" survey. Each year, more than 51,000 high school students
at more than 400 public and private schools are polled about their drug use.
In the 1970s and 1980s, the household and high school surveys were treated as
national news on the state of the drug war, particularly in tracking the rise of marijuana
and cocaine.
"I've been looking at the household survey and the high school survey for
years and years," said Eric Sterling, a former House Judiciary Committee staff
counsel now with the Criminal Justice Policy Foundation. "They have an effect like
electric shock on a dead frog's leg. There's a spasm people have when they get this data.
People, certainly on Capitol Hill, look to respond."
In the mid-1980s, the advent of crack played havoc with the existing
measurement system. Simply put, there was no measurement in place for crack use -- crack
was so new that the household survey did not start asking about it until 1987.
Faced with an unprecedented national outcry after the overdose death of
University of Maryland basketball star Len Bias on June 19, 1986, Congress rushed through
a law punishing crack cocaine possession at a rate 100 times that of powder cocaine.
Without hard data, lawmakers relied heavily on high-pitched media accounts, some of which
"were not supported by data at the time and in retrospect were simply
incorrect," the U.S. Sentencing Commission later concluded in a comprehensive study
on "Cocaine and Federal Sentencing Policy."
"It was really the opposite of science," said Sterling, who wrote the
draft version of the crack law when he served with the Judiciary Committee. "It was
mythology-driven. It was said repeatedly that there were 3,000 new crack addicts every
day. These kinds of numbers would get thrown out and repeated without anybody doing the
arithmetic or asking: `How does this number relate to anything we know about the usage?'
"
The lawmakers believed -- erroneously, it would later turn out -- that crack
had killed Bias. (Testimony from someone who was with Bias when he died pointed to powder
cocaine.) Congress reacted so strongly to crack in part because it believed it was dealing
with a rapidly spreading "crack epidemic."
Yet the household survey eventually estimated that crack use stabilized almost
immediately and never approached the levels that powder cocaine had -- crack stood at
668,000 monthly users in 1996 compared with more than 5 million for powder cocaine in
1985, according to survey figures.
But the statistical data eventually provoked just as much criticism as the
absence of data did. Crack use turned out to be harder to measure than powder cocaine use.
Like heroin, crack quickly concentrated among poor urban addicts. Many of them lived on
the streets, where they would not be counted by the household survey.
"The household survey and the school survey are pretty useless for
measuring hard drug use in the population," said Wish, the University of Maryland
research center director.
By the late 1980s, drug researchers like Wish thought that the nation's cocaine
problem was breaking into two distinct groups: mainly white suburbanites who used cocaine
casually on weekends and mainly black urban addicts who used crack or cocaine daily. For
casual users, Bias's death seemed to have the effect of scaring millions off cocaine; the
household survey indicated that after 1985 the number of monthly cocaine users plummeted
70 percent.
Yet the trend in hard-core usage is still being sorted out.
In 1990, just as the Bush administration had begun touting the decline in
casual use, then-Senate Judiciary Committee Chairman Biden produced a report counting
habitual cocaine users at 2.2 million. That was nearly triple the household survey's
estimate.
Biden's numbers had come from what would eventually emerge as a third leading
indicator of the nation's drug use -- the Justice Department's Drug Use Forecasting (DUF)
program, started in 1987. The DUF program collects voluntary urine samples from 30,000
jail inmates in 23 cities across the country each year to test for cocaine and other
drugs. Biden's figures were extrapolations from these urine tests.
Mark Kleiman, a Harvard researcher who supervised the Biden committee's work,
subsequently acknowledged that the methodology was "not precise." But he said
conservative assumptions were used to come up with numbers that gave a clearer picture of
the nation's cocaine use.
But the GAO and household study researchers like Gfroerer say that the DUF
urine tests cannot be used to extrapolate larger numbers because they are not part of a
randomly selected scientific sample.
"DUF really isn't representative of anything," Gfroerer said.
"The way it's collected, you can't project it out to any population."
Although the household survey is based on a randomly selected sample, it also
has limitations, according to some researchers. Only a tiny percentage of people admit to
heroin and cocaine use, and they must then become the basis for projections into the
millions of users. For example, of 32,594 people surveyed in 1991,
only 127 admitted to using heroin in the past year, according to the GAO. From this number
the survey projected 701,000 heroin users nationwide.
Thus, small errors in the way the survey is carried out can be magnified. That
means yearly shifts of a few hundred thousand in a projected user population of a million
are statistically insignificant because they could be explained by possible errors in
sampling, reporting or extrapolation, Gfroerer said.
The GAO found such problems in the 1991 cocaine and heroin figures. For heroin,
further investigation revealed that 53 of the 127 users counted in the survey were
inappropriately "imputed" -- researchers made a subjective decision to count
them even though they gave contradictory answers. When the error was later corrected, the
number of heroin users dropped 46 percent to 381,000.
Moreover, of the 701,000 annual heroin users originally estimated in 1991,
142,000 were derived from the survey response of a lone 79-year-old white woman. Her
answer was weighted in an effort to make the survey result more representative of the
nation's population; but the resulting statistical projection accounted for one-fifth of
all the estimated heroin users in the United States that year, according to the GAO.
"The bottom line is [that] to make projections from the household survey
to the number of heroin users in the country is probably not a good idea," said
Hermalin, the GAO project manager. "Cocaine [estimation] is dangerous, too."
In 1994, the household survey was revamped to make it more accurate at counting
hard-core drug use, but Gfroerer said the difficulty was "only partially"
corrected.
"The basic issue of understating of hard-core drug use, those problems are
exactly as they have been," Gfroerer said. "We still feel it's important to
collect these data as part of the survey. The real issue is how you report them."
MEASURING THE DRUG WAR: For the last 25 years, progress in America's battle
against illegal drugs has been measured primarily by four "leading drug
indicators."
SURVEY: National Household Survey on Drug Abuse
BEGUN: 1972
SPONSOR: Substance Abuse and Mental Health Services Administration (SAMHSA)
FREQUENCY: Every three years since 1972, yearly since 1990
YEARLY FUNDING: $5.7 million
METHODOLOGY: Questionnaires given to a randomly selected sample of 18,000
households; answers are reported anonymously
STRENGTHS: Best measure of overall national drug trends, especially marijuana
and first-time users
REPORTED LIMITATIONS: Undercounts hard-core heroin and cocaine users who don't
live in households; some people lie on surveys; sampling errors distort results
OVERALL TRENDS REPORTED: Overall drug use down 49 percent since 1979 peak;
cocaine down 70 percent since 1985 peak; marijuana down 58 percent since 1979 peak; crack
stable since 1988
SURVEY Monitoring the Future Study
BEGUN 1975
SPONSOR National Institute on Drug Abuse
FREQUENCY Yearly
YEARLY FUNDING $4 million
METHODOLOGY University of Michigan researchers poll more than 51,000 eighth-,
10th- and 12th-grade students at more than 429 schools
STRENGTHS Best early warning system for drug use among youth
REPORTED LIMITATIONS Doesn't count dropouts; undercounts
non-white students; some students lie on surveys
OVERALL TRENDS REPORTED For high school seniors, overall use down 37 percent
since 1979 peak; cocaine use down 70 percent since 1985 peak; marijuana
use up 84 percent since 1992 but still down 41 percent since 1978 peak; heroin use
up sharply since 1985
SURVEY Drug Abuse Early Warning Network
BEGUN 1978
SPONSOR SAMHSA
FREQUENCY Yearly
YEARLY FUNDING $2.5 million
METHODOLOGY Data collected from patients in more than 500 emergency rooms in 21
cities
STRENGTHS Best measure of people with chronic or acute drug problems
REPORTED LIMITATIONS Not a representative sample; counts people seeking
treatment along with people who overdose; counts suicide attempts with legal drugs. Ed. Note: There is another serious problem not reported here, especially
for marijuana. These numbers are about "mentions" of a drug, not necessarily the
cause of the emergency.
See: Shalala Uses ER Stats to Lie About
Marijuana --Again -- as "Drug-related" ER Visits Decline )
OVERALL TRENDS REPORTED More and more drug users are ending up in emergency
rooms -- a record 531,827 in 1995, up 43 percent since 1990
SURVEY Drug Use Forecasting program
BEGUN 1987
SPONSOR Department of Justice
FREQUENCY Quarterly
YEARLY FUNDING $2.4 million
METHODOLOGY More than 30,000 jail inmates at 23 U.S. cities submit to voluntary
urine tests
STRENGTHS Best measure of drug use among the criminal population
REPORTED LIMITATIONS Isn't a scientific sample; can't be used to extrapolate
national figures
OVERALL TRENDS REPORTED Crack and cocaine use declining among U S. arrestees;
17 cities reported declines in percentage of positive cocaine urine tests in 1996
SIX WAYS OF LOOKING AT COCAINE USE
Over the past 10 years, different efforts funded by the federal government have
produced wildly different estimates of the number of hard-core (weekly) cocaine users in
the United States.
The most conservative estimates come from the National Household Survey on Drug
Abuse, which was revised in 1994 to better measure hard-core use. (A mistake in 1991 led
to the report of an erroneous single-year jump of 200,000 hard-core users.)
Other studies combining the household survey findings with urine-test data from
jail inmates have produced larger figures but differing trendlines.
A RAND study released in 1994 showed a slow upward trend.
A 1995 study by Abt Associates Inc. showed falling and rising trends. When Abt
improved and revised its methodology in a 1997 study, it counted more hard-core users than
ever.
SOURCES: National Household Survey on Drug Abuse, Office of National Drug
Control Policy, General Accounting Office, RAND, Abt Associates Inc.
THE BIG PICTURE (Extrapolated from highly flawed
numbers.)
The number of illicit drug users has declined sharply since 1985.
IN MILLIONS
1996: 13 million
The amount of federal money spent on drug control efforts continues to increase
rapidly.
IN BILLIONS
1998: $16 billion requested
SOURCE: Office of National Drug Control Policy, National
Household Survey on Drug Abuse
© Copyright 1998 The Washington Post Company
|