 |
 |
 |
 |
| |
|||||
|
|
|
Home > News & Events > MITRE Publications > The MITRE Digest > | |||||||||||||||||||
| Storytelling The Oldest Art Helps New Science May 2002
Insights on Information Visualization On the surface, the word storytelling seems oddly out of place in the high-tech swirl of MITRE research and development. However, just beneath the surface, nestled right in there with the likes of information technology and systems engineering, storytelling, an integral aspect of the relatively new field of information visualization, is on a quick and dramatic rise into prominence. The brain's prodigious capacity for multisensory information fusion—when data is presented to it in story form—is under intense scrutiny by both industry and government. With over 4 million years of development in nature's R&D lab of evolution, the inner workings of the brain's storytelling capabilities are only now slowly revealing themselves to scientific inquiry. The findings are as important as they are startling—so much so they may profoundly affect the entirety of information technology. The brain, it seems, heartily concurs with the boast from that most keen observer of information, Sherlock Holmes: "My mind rebels at stagnation. Give me problems, give me work, give me the most abstruse cryptogram or the most intricate analysis, and I am in my own proper atmosphere."—The Sign of the Four Like the brainy sleuth before him, MITRE's Nahum Gershon, a Senior Principal Scientist, is also in his own proper atmosphere, integrating the elements of storytelling with information visualization, and then utilizing both to provide real-world solutions where other technology has come up short. And the challenges for this nascent field are plentiful. Over the past decade, a commonly held and growing concern has arisen over what is loosely called information overload. Simply put, we're being overwhelmed by the data flow hurtling at us from our own computers. Countering that viewpoint are many who believe that the brain is far from being overwhelmed, that it is how we convey information to the brain rather than the quantity of information that confounds comprehension. In response, George Robertson in 1993, then of Xerox PARC, coined the term "information visualization" to define his interactive techniques of using aspects of imaging and graphics to transform data, information, and knowledge into a form that relies on the human visual system to perceive its meaning. A picture is worth a thousand words; moving pictures, maybe a few billion! Information visualization converts the avalanche of raw computer data to enable faster and easier comprehension. Take, for instance, the use of information visualization – in real time – on a stream of raw data listing the sale prices and addresses of residential properties. The process dynamically transforms the residences, as each comes onto the market for sale, into icons that represent home types (e.g., single-family, multi-dwelling), colors the icons to represent price ranges, and then places individual icons at street addresses on a map. In a single glance, the eye instantly absorbs the embedded information, say, where are the locations of the most expensive single-family homes or are the sales orderly over time or is there a stampede to sell? Information visualization converts the avalanche of raw computer data – the overload of text, numbers, and symbols – to enable faster and easier comprehension. The salient realization here is that the brain's ability to absorb information is strikingly prodigious if information is properly tuned to maximize the sense of vision. Ongoing research in information visualization, known now as "infoviz," continues to spawn ever-newer interactive techniques in its quest to perfect spatial metaphors to represent these streams of abstract information. Prominent among them is storytelling. If Nahum Gershon's view of information visualization holds true, the millennia-old art of storytelling could well be the linchpin of this new field. Instead of utilizing a single sense, such as vision, Gershon enlists all of the brain's multisensory talents to the cause. And to that view, agreement is quickly mounting from current brain research, emerging cognitive studies, and recent advances in technology. Gershon, a leading exponent of storytelling's unparalleled ability to convey information, said "A well-told story conveys great quantities of information in relatively few words in a format that is easily assimilated by the listener or viewer." Anyone even vaguely familiar with Shakespeare, a Richard Feynman physics lecture, or Aesop's fables, has experienced the enduring magic of storytelling to convey information. Genius aside, this is the brain's own proper atmosphere. This is not a world of bits and bytes, but rather an object-oriented world in which the brain contextualizes information by common sense understanding and then pays attention to the deltas between assumption, past experience, and current observation. It is a powerful world where everything happens in an instant, and Gershon seeks to graft it to infoviz. If Gershon's research succeeds, computer-based storytelling could well revolutionize the way people interact with computers. To underscore the importance of infoviz on the one million terabytes of data generated globally each year, Communications of the ACM devoted its entire August 2001 issue to "Visualizing Everything." In it Gershon's article, "What Storytelling Can Do for Information Visualization," detailed the impact of storytelling. Into the infoviz interactive mix of graphical metaphors to represent information, Gershon introduces sound, both voice-over narration and sound effects, to craft the information flow into a story. After all, he reasons, the ancient art of storytelling is the brain's own natural way of quickly conveying information; why not use it? A story also allows the brain to infer information not stated explicitly and to use continuity to predict future actions. Secondly, he adds visual techniques common to Hollywood movies: aerials to establish an overall scene, zooms to focus attention, dissolves to show the passage of time, and intercutting between images to compare and contrast. Then comes animation, either supporting live-action footage or as standalone motion graphics. Such techniques help the brain to order and compress information quickly, which, in turn, hastens its transformation into knowledge, which then, with the aid of common sense, creates understanding and the ability to make decisions. Bran Ferren, co-chairman and chief creative officer of Applied Minds, Inc., is explicit about the impact of storytelling: "It's interesting to note," he said during his presentation for the MITRE Technology Speaker Series, "that every time an effective new storytelling technology has been introduced, it has changed our world. Examples include language, writing, the telegraph and telephone, newspapers, radio, television, and most recently, the Internet." The implications of Ferren's pronouncement on the potential of Gershon's storytelling research are astounding. In terms of applications to the real world, consider the massive databases and fast decision-making critical to industries like transportation, telecommunications, or energy transmission. If infoviz and storytelling can substantially enhance reaction to and restoration of, say, electricity during a blackout then its value is incalculable. In addition, the power of storytelling is not lost on the U.S. Army, who, in conjunction with the University of Southern California, created the on-campus Institute for Creative Technologies, which utilizes animation and storytelling to help train military officers. And more recently, following the September 11th attacks on the Pentagon and World Trade Center, the Los Angeles Times reported that federal government enlisted the talents of Hollywood writers to craft stories of terrorist scenarios and story-based solutions to thwart those schemes. That the ear helps the eye to read and that the eye assists hearing – which is at the core of storytelling's multisensory approach to conveying information – have been confirmed by recent scientific research. Harvard University's Alvaro Pascual-Leone, in working with people blind from birth, found that they used their visual cortex to read Braille, and with that came the discovery of multisensory neurons. These are brain cells that react to many senses all at once instead of just to one. In a New Scientist article, "Infinite Sensation," he further claims that routinely "the brain can mix up the senses to solve particular problems": sensory integration for better clarity and detection and for making sense out of ambiguous information. These are the sensory qualities that Gershon claims are necessary to properly comprehend high-speed streams of information and data, and exactly the qualities storytelling brings to infoviz. Now the stream of raw data listing the sale prices and addresses of residential properties is enhanced by sound, snippets of video, a variety of camera angles, and animation techniques. And with the changes, the information stream enters into the brain's own proper atmosphere. Comprehension is thereby enhanced and with it comes increased control by the human observer. All of which just may turn out to be the answer to information overload. Let the computers do what they do best, and then serve up to the brain – in storytelling form – information that allows the brain do what it does best: synthesize, analyze, and make decisions. By grafting storytelling to computer information visualization and adding a human language interface (see, " Chasing HAL"), critical advantages could be brought into play for the military, as well as crisis management in homeland defense. Imagine a field commander readying troops for battle. A small viewing screen snaps down from the visor on his or her helmet. A field map slides into view, and a voice asks what sectors of the map are needed, and then also inquires about the intended action and the time it will commence. The commander relates the necessary map coordinates for a pre-dawn assault on enemy positions. Suddenly a raw data flow begins to emanate simultaneously from orbiting satellites, drone aircraft, and from manned aircraft, mixing with ground observations from robotic behind-the-lines devices and forward human observers, and further mixed with intelligence reports. The raw data is converted to infoviz via storytelling and made available to the commander. In addition to infoviz on the adversary, the commander can also order infoviz on his or her own resources: artillery and smart munitions, air cover, reinforcements, and, if necessary, evacuation alternatives. The commander can also role-play attack plans and enhance those plans with a database of historical infoviz as to how past commanders, in similar situations, achieved success. Ferren considers this type of integrated, multisensory, interactive display environment an absolute necessity for battlefield visualization. He contends that present-day visual display systems are overly cluttered with icons and poor graphics and, as such, lead unnecessarily to information overload. "We should be trying to communicate information," he says, "not giving the commander an eye test." The Advanced Technology Office of the U.S. Defense Advanced Research Projects Agency (DARPA) has a mission afoot called "Command Post of the Future," which is in wholehearted agreement with multisensory interactivity. As DARPA succinctly puts it: "Dynamically tailored visualizations will improve decision speed and quality" and achieve the program's goal to "shorten the commander's decision cycle to stay ahead of the adversary's ability to react." As Ferren reasons, "Silent movies died because talkies did a better job." For Gershon and his research team, storytelling may well cause much the same result in visualizing information.
Page last updated: May 21, 2002 | Top of page |
Solutions That Make a Difference.® |
|
|
