Digest Topics

Authoring & editing
Budgeting, reporting, & metrics
Case studies
Content management
Enterprise knowledge management
Government applications
Graphics & posters
Learning & Education
Legal applications
Ontologies & topic maps
Personal knowledge management
Point of view articles
Publishing
Reviews
Studies & surveys
Research & searching
Slides & presentations
Taxonomies & metadata
Technologies & tools
Trends & strategies
Usability & user behavior
see also A - Z index

In this occasional column, Montague Institute Founder Jean Graef comments on one or more of the Digest articles. See also other POV articles.

Less search, more "brave thinkers"

After spending a year compiling my SharePoint notes and slides into a book, I'm dismayed to learn that my efforts may be irrelevant. That's the implication of a special issue of the Atlantic magazine devoted to "brave thinkers" — people who think out of the box and have the courage to follow through on their ideas. One of them, a Greek doctor who reports on the credibility of medical research, charges that as much as 90 per cent of the published information that doctors rely on is flawed. What's the use of searching if the stuff you find is wrong or misleading? What's the good of finding "experts" if they work in a flawed ecosystem?

The lead Atlantic article, "Lies, Damned Lies, and Medical Science," illustrates the effect of information on systemic risk in the health care field. The subject of the article is John Ioannidis, whose 2005 paper “Why Most Published Research Findings Are False” turned the medical establishment on its head. Some of the eye-opening findings include the following:

• Patients with Albanian names were three times more likely to have healthy appendices removed in a Greek teaching hospital than patients with Greek names.

• 80% of non-randomized medical research studies (the most common type) turn out to be wrong, as do 25% of randomized trials and 10% of large randomized trials.

• Of 45 of the most highly regarded research findings over the last 13 years, 41% had been convincingly shown to be wrong or significantly exaggerated.

Ioannidis says that the reason these flawed studies do make more of us seriously ill is due to the fact that most medical interventions and advice don't address life-and-death issues but rather aim to make us marginally healthier.

But why do they happen in the first place? The article lists the usual suspects:

• drug companies and medical manufacturers choosing topics that compare their products to those already known to be inferior;
• surgical residents eager to get more "scalpel time;"
• researchers who want to get their grants approved;
• conflicts of interests (i.e. researchers receiving funds from businesses with a vested interest in the outcome);
• poorly designed research studies;
• misunderstanding and misinterpretation due to the fact that researchers and doctors speak different languages.

Shocking as the findings are, they are not the most interesting part of the article — at least to an information junkie like me. For example, why didn't the medical establishment embrace Ioannidis' work instead of running him out of town on a rail? According to one prominent medical blogger, "Not a single one of my surgical colleagues was the least bit surprised or disturbed by [Ioannidis'] findings." Ioannidis thinks this is because he didn't mention specific researchers, thus giving scientists and physicians the opportunity to commiserate about the problem without having to acknowledge personal responsibility. He goes on to say, "There may not be fierce objections to what I'm saying, but it's difficult to change the way that everyday doctors, patients, and healthy people think and behave." We could solve much of the problem, he thinks, if the world simply stopped expecting scientists (and other "experts" for that matter) to be right.

Interesting insights from other articles in the same issue include the following:

• Military intelligence journalists. Michael T. Flynn, the top military intelligence officer in Afghanistan, reported publicly that too many intelligence analysts are "ignorant, incurious, and disengaged." He demanded that 2,000 additional intelligence analysts be brought into Afghanistan. Their objective: to gather and report field intelligence as journalists would.

• Choosing the right publication. Flynn took some flack from Defense Secretary Gates for publishing through a Washington think tank instead of military channels. Flynn's response: military channels are too cumbersome. The report required a wider readership and a faster publication schedule.

• Choosing the right time. An awarding-winning Ghanaian journalist, who uses his penchant for disguises to get stories (and protect himself), has four completed stories on hold waiting the right time to publish them. "When the country is dull and feeling that everything is all right, I send [a story to get them] thinking that nothing is all right, that we are all jokers." See Smuggler, Forger, Writer, Spy.

A key implication of the Atlantic issue is the need to pay more attention to the strategies and techniques of journalism, not just in public discourse but also in business. It's not that organizations haven't recognized the need for better business intelligence or the value of using journalistic techniques to build morale and influence public perception. That's why there are corporate libraries, competitive intelligence (CI) units, and corporate communications departments, but each has its own job to do and, for the most part, they are responders rather than initiators. CI staff that do uncover counter-intuitive trends and uncomfortable truths are often not heeded by senior managers.

Knowledge management as practiced in most organizations is not the answer either. There's a limit to the value of information that is hoovered up through search technology, surfaced with knowledge sharing incentives, or gathered with "crowd sourcing" techniques. What's needed is more creativity, integrity, and effort on the front end (creating and verifying information) as opposed to the back end (capturing, sifting, organizing, and sharing information). We need to pay less attention to "experts" anointed by the status quo and more attention to people thinking outside of the box, experimenting with new ideas in the field, and publishing reports referencing data that can be validated by an independent third party.

I think it will be difficult to get an investigative journalism function funded in most public or private bureaucracies, but it may be possible for the IT function to introduce information quality initiatives through the back door. That's what happened with triple bottom line reporting, data quality management, content "curation," taxonomies and metadata management. Funding is forthcoming when user and stakeholder complaints are loud enough to reach the ears of senior managers.

But what should information quality programs look like? At the very least we should be educating managers about the relationship between misinformation and risk, where to look for ideas that challenge the status quo, how to conduct basic information research, how to select and use publishing tools, and how to use thesauri to bridge the language gaps between disciplines. Ultimately, it might also be necessary to fund external agencies that exist for that purpose (e.g. FactCheck.org). The trick is to make them truly independent, willing to challenge the status quo, and challenge vested interests — almost a Consumer Reports for information. Failure to do so perpetuates both corporate and systemic risk at unacceptable levels.

Related articles

Content curation in context (2010)

The truth about expertise databases (2007)

Managing the triple bottom line (2003)

Corporate scholarship: an oxymoron? (1999)

For original articles by Jean Graef, see the Montague Institute Review.