Pattern Recognition

Images often provide compelling evidence in R&D, and in intelligence gathering in general (e.g., the CIA). A picture is worth a thousand words: ultrasound, X rays, microarrays, micro imaging, tomography, MRI, PET scans. These are becoming household names. In the hands of experienced technicians who understand the strengths and limitations of the technologies, they can be a very effective means of gathering evidence, short-cutting the often convoluted arguments for causality we make in their absence.

But images can be too compelling: researchers relying on images too easily jump to hasty, ill-formed decisions to the exclusion of other, more subtle evidence that can be contradictory. Satellite images showing Sadaam Hussain was building a chemical weapons capability in Iraq were compelling, and swamped more subtle evidence to the contrary.

Images have a key shortcoming when used by highly educated scientists and clinicians: these individuals have a super developed network of cubbyholes in their brains just waiting to file that next piece of data. Visual data is often under analyzed before being automatically filed away and absorbed into the overall body of evidence+.

Situs inversus, the left-to-right transposition of all internal organs, has been known since 1788. Dextrocardia – a variation in which only the heart is on the wrong side of the body – was described even earlier, in 1643. Dr. Maldjian, [M.D., Associate Professor of Radiologist] ... sees several cases a year at the University of Medicine and Dentistry [of New Jersey]. It's a shock every time you see one of those images,’ he said ... The patient had been in for three successive chest X rays over three years .. each time, the radiologist reading the X ray assumed that the film [had been reversed], so the case was read as normal. Maldjian, Pierre (2007). cited in... Mirror-Image Humans Walk Among Us With No Warning

Physicians aren't used to seeing the heart on the wrong side and will blind themselves to even the left-right markings on the image to conform what they’re seeing to their beliefs.

If we misinterpret evidence from compelling images, we risk falling into a cycle of Confirmation error+ and Narrative fallacy+ for all other evidence. Images and graphics pose the additional risk that they quickly establish themes and principles that help us organize all the evidence in our minds: we remember the image and use it as a catalog for non image evidence. The nature of images makes it too easy to ignore contradictory, non image evidence.

Experts in image interpretation can often get it wrong. A study of fingerprint matches by examiners show instead they were merely opinions – fingerprint matches did not exist outside the subjective perceptions of the examiners.

In 1995 a comprehensive review of [fingerprint] examiner proficiency was performed. The results were astonishing. Only 44 percent ... scored perfectly. Six of the 156 examiners reported false negatives ... thirty-four examiners, 22 percent of the total, reported false positives. Cole, Simon A. (2002). Suspect Identities, p. 281

This and other studies point to the need for independent interpretation of any evidence based on images that is deemed critical to an overall body of evidence. To their credit, fingerprint experts moved to rely on a less-precise point system. It was easier to agree on points of agreement in a fingerprint, rather than on an exact match, but this reduced the weight of the fingerprint evidence in a court of law.

People look day after day into microscopes or at the image outputs of a machine reader. It numbs the mind. If they stare at the images repeatedly and long enough, patterns and matches begin to emerge where none exist. For this reason pilots are trained to first look at the terrain, and only then to look for a match in the aeronautical maps (personal communication from flight instructor). Mechanical errors and tolerance for discrepancies are more frequent the longer you stare at the images. Even the best, most mature, imaging tool will not be allowed to replace personal responsibility. ‘The system said so’ is not an excuse for mistakes.

Scientific images are often snapshots – points in time – that require further expert interpretation to make them come alive. Automation can help, but it cannot replace human interpretation. Training and testing of users of imaging technologies are key to maintaining the integrity of image-based evidence. You cannot assume individuals remain proficient in interpreting the images from these technologies, no matter how well-established the technology. Participation in teaching societies or accreditation with continuing education is mandatory for users of imaging technologies.

Automation brings its own special issues.

Today’s crop of senior latent fingerprint examiners rose through the ranks as novice examiners who spent several years classifying, filing, and retrieving [manual fingerprint cards]. Latent fingerprint examiners universally attest that this long period of [manual card] work allowed them to develop the visual skills necessary for more advanced fingerprint work: how to orient themselves among the whorls of fingerprinting patterns and how to visually analyze fingerprint patterns ... crucial when examiners try to read a poor-quality latent print. Essentially, this period of [manual card work] taught examiners to see fingerprint patterns in a way quite different from the way the rest of us see them...[Computer automation] brought about a profound change in the training of latent fingerprint examiners. [Manual cards] no longer serves as a training ground for examiners. It is not clear how this aspect of training is going to be replaced ... [Computer automation] threatens to erode the skill level of latent fingerprint examiners. Cole, Simon A. (2002). Suspect Identities, p. 256

As late as 1997 it was noted that latent fingerprint evidence might not be able to pass the Daubert criteria: it could be declared inadmissible evidence+ in a court of law.

As illustrated above, technologies need to be designed that much better encourage a learning experience for novices. Today many of technologies are essentially unapproachable – they are too complicated. The must be learned by rote. Technology that includes a ‘learning experience’ could include, for example: hotlinks to relevant scientific literature that uses the technology at its foundation; smart questioning of the user; tracking user skill levels; etc. One client recommended leveraging tricks of the video gamer industry: you gain access to higher levels of automated support only once you demonstrate competency at lower levels.

Images are very compelling, and as such required even greater care in their use. No single piece of evidence should be allowed to become too important in our overall body of evidence. The more compelling the evidence, the greater should be our caution. Observational data and theory are inextricably linked. Get the theory wrong and the observational data will be misinterpreted, no matter how compelling it may appear on the surface.