Mario Valle Web

Visualization and art

The artists see the unseen and communicate it. That is exactly the same goals of scientific visualization.

There are obvious relations between science and art and between scientific visualization and art. Here I collected material, thinklets and references to explore this issue and to decide if this area is worth pursuing.

This research can be trivial or interesting depending on how I would consider the relationship between science and art: "…we serve any enquiry into art and science badly if our criterion is superficially the influence of science on art, or the influence of art on science", reminds us the paper From science in art to the art of science by Martin Kemp (Nature 434, pag. 308-309).


So which directions should I investigate?

Possible questions

After collecting references and ideas, what goal I want to reach? What questions should I pose? For example:

Scientists and artists

[Leonardo da Vinci]

Leonardo the scientist and Leonardo the artist were one man, not two. Many of the qualities of a successful scientist and a talented artist are the same:

[Artists common themes]

We can add also those common interests and themes:


What specific characteristics artists have?

  1. An eye trained very well in pattern recognition.
  2. An unbiased eye that sees what usually scientists do not see because they are blocked by been able to see only what they are expected to find.
  3. An artist also knows what (details) are not important and should be ignored.

Visual literacy: skill developed in interpreting, judging, responding to and using visual representations of reality. If you want to appreciate an artistic artifact, you should learn to see.

Researchers in many fields are becoming aware that in order to do really creative work, they may need to go back to visual approaches once again.

Very high level and creative achievement in the sciences has often come from the neurological resources linked to success in the arts.

Which is the process of visual understanding?

Artists work to achieve variations in visual expression, and may at times strive for the viewer to experience the emotional turmoil embedded in the art form. The artists’ tools of line, shape, form, contrast, color, scale, composition, and movement are manipulated in order to affect the emotions and, at its zenith, the passions in others. But like the scientist, the artists’ creations, to be successful, must maintain the intellectual components which are based upon the building blocks of structure, and the elements and principles of design.

This “visual communication” is computer graphic’s link between scientists and artists, often those with differing approaches to the same information.

Steve Jobs: Real Artists Ship Products.

Art often convey a meaning to which we can relate.

Creative insight does not happen in a vacuum. Artists need models, examples, sketches. Artists benefit from rapid test of new ideas and from multiple approaches to the same problem (See Picasso’s Guernica preparatory sketches).

"Can the Arts offer alternatives in setting research agendas, interpreting results and communicating findings?" from Stephen Wilson. Surely it can! An artist is an originator of ideas.

"What I knew for myself in my music system, and Peter so delicately reminded me in the domain of paint programs, was that the ten years that this has taken are nothing special - at least when contrasted to the years that the artists themselves have invested in developing their unique skills. While the essence of the artist is reflected in their work, it is rooted in skill – skill which is hard earned, and therefore worthy of respect by the instrument builder, or “luthier.” But it is precisely these same skills which are so poorly captured by most computer-based tools. (From: Buxton, W. (1997). Artists and the Art of the Luthier. Computer Graphics: The SIGGRAPH Quarterly, 31(1), 10-11.)

“There is no science without fancy, nor art without facts”— Vladimir Nabokov

Science and art core differences

Prof. Kemp warns us of one of the fundamental differences between art and science. And again this warning helps us to go beyond trivial comparisons between these two fields.

“Many authors, particularly those aiming to communicate to an audience outside their immediate professional orbit, use artistry to stimulate engagement, impact and excitement. For many scientists, these dimensions are barely implicit, but for others the aesthetic motive is consciously present throughout.”

“A work of art always remains open for interpretation, drawing the spectator into the shape of the artist’s visualization, but without being able to exert fixed control over the feelings it induces. There is always room for the beholder’s share.”

“Scientists may wish to engage the reader or spectator in a wonderful journey of imaginative visualization, but in the final analysis they wish to communicate an interpretation that embodies testable content in an unambiguous way.”

There is a continuum between an artist and a scientist. But there are also notable differences between those two kinds of person:

To establish a dialog a common frame of reference should be set.

(From: Colwell, B.: Frames of Reference, Computer – Volume 38, Issue 6, May 2005, Pages: 9-11)

An artist has his personal agenda and a very particular point of view: communicate part of himself he wants the world to perceive.

Specialist often has a myopic perspective that precludes the comprehension of larger important patterns. Also unconnected knowledge quickly slips away. Unfortunately the environment for research in most universities is one which rewards increased specialization rather than encourages interdisciplinary cooperation.

Arts integrate, specialized science disintegrate knowledge into a myriad of different fields that lost communication between them.

The role of mental images in science

Many of our scientific models originated as psychical images whose forms could not be communicated to others except perhaps notionally. Because objectivity and reproducibility are the ultimate goals of scientific models, scientists and engineers who create and use these models rarely allude to this psychical process and hence effectively deny the motivation and inspiration behind this creative visual experience. J. W. Gibbs, James Clerk Maxwell, Albert Einstein, and Richard Feynman were exceptions in that they would openly talk about their “visual mental models.”

The discovery process reported by scientists and engineers using visual tools closely parallels comments made by researchers who used a visual cognitive process to create scientific models. In both cases the “minds eye” is used to gain insight into complex abstract processes.

Visualization is science and art

[Left vs. Right Brain]

This old IBM advertising expresses really well the role of visualization and computing…

…using an image normally related to artistic orientation.


Visualization facts

Visualization is that initial step taken to establish a clearer pictorial representation of the problem and continue the development of the picture before the formal mathematics is done.

“Computing, and in particular supercomputing, without visualization, is like assembling a jigsaw puzzle in the dark”. Richard Weinberg

Visualization and art

Which are common themes between an artist and who does scientific visualization? The same commonalities found between an artist and a scientist.

Images can resemble art, but the primary purpose is to communicate information.

The challenges facing information visualization researchers often involve finding innovative graphic and interactive techniques to represent the complexity of information structures.

Then from Einstein: “If we trace out what we behold and experience through the language of logic we are doing science; if we show it in forms whose interrelationships are not accessible to our conscious thought but are intuitively recognized as meaningful, we are doing art.”

An important risk is about fidelity: visualization is about seen the unseen, or about seen the inexistent? Is what I see contained in the data or is it added by the visualization algorithms?

There is also the reverse problem: there are people that are 3D-blind. So why try to get an accurate picture if they don’t see it right? The consensus is that to “teach people to see” we must learn from those in the fine arts. If we listen to the artists then we can go into a new phase with our pictures.

Artists can tell us what works and what does not works in a visual representation.

As for art, the interpretation and evaluation of patterns is entirely subjective.

Is visualization science?

Here are some ideas from the panel: “Is Visualization Struggling under the Myth of Objectivity?” from IEEE Visualization '95.

The problems that visualization has to fit in the mold of science derive between other things from: the divorce of human error and perception from the process of discovery, denial of emotional interpretation, poor aesthetic judgment and other subtle misdirections. Of particular interest will be those choices and presumptions born out of an automatic observance of the commandment of scientific objectivity.

Visualization works out of the grey area of human perception and cognition. So far it has not been found to be self-consistent; there are no rules which precisely govern what approach will work under a given set of conditions. Visualization is not strictly repeatable; one user may perceive a relationship (which may not be verifiable using visualization techniques) while another sees nothing at all or something entirely different. Visualization relies on subjective interpretation; contextual cues are usually derived from "common" experience which is neither guaranteed to be common or commonly integrated into personal experience. Visualization is resistant to the systematic evaluation and assessment procedures common to science; it still remains difficult to ascertain if a particular instantiation has been "successful". And there are persons who see the compelling nature of the imagery as an open temptation for abuse and trivial indulgence.

So how does it contribute to science at all? Visualization is part of a process of discovery. It appears to aid the intuition in identifying relationships, some of which can be later described by formal analysis. In cases where there is no such analysis, the results of visualization cannot be properly integrated back into the scientific domain.

What art teaches to visualization

Metaphors and mappings

The concept of mapping is closely related to visualization but it makes sense to keep it separate. By representing all data using the same numerical code, computers make it easy to map one representation into another: grayscale image into 3D surface, a sound wave into an image (think of visualizers in music players such as iTunes), and so on. Visualization then can be thought of as a particular subset of mapping in which a data set is mapped into an image.

In such situations designers and their clients have to choose which dimensions to use and which to omit, and how to map the selected dimensions. This apparently rational decision process could be made explicit, instead of presenting only the end result to the user. Art can instead make the method out of irrationality and data visualization can try this also.

Most mappings in both science and art go from non-visual media to visual media. Is it possible to create mappings that will go into the opposite direction?

And if modernism reduced the particular to its Platonic schemas (think of Mondrian, for instance, systematically abstracting the image of a tree in a series of paintings), data visualization is engaged in a similar reduction as it allows us to see patterns and structures behind the vast and seemingly random data sets.

Thus it is possible to think of data visualization as a new abstraction. But if modernist abstraction was in some sense anti-visual – reducing the diversity of familiar everyday visual experience to highly minimal and repetitive structures (again, Mondrian’s art provides a good example) – data visualization often employs the opposite strategy: the same data set drives endless variations of images (think of various visualization plug-ins available for music players such as iTunes.) Thus, data visualization moves from the concrete to the abstract, and then again to the concrete. The quantitative data is reduced to its patterns and structures that are then exploded into many rich and concrete visual images.

I often find myself moved emotionally by visualization projects outcomes. Why? Is it because they carry the promise of rendering the phenomena that are beyond the scale of human senses into something that is within our reach, something visible and tangible?

This promise makes data mapping into the exact opposite of the Romantic art concerned with the sublime. In contrast, data visualization art is concerned with the anti-sublime. If Romantic artists thought of certain phenomena and effects as un-representable, as something which goes beyond the limits of human senses and reason, data visualization artists aim at precisely the opposite: to map such phenomena into a representation whose scale is comparable to the scales of human perception and cognition.

(From: Lev Manovich: The Anti-Sublime Ideal in Data Art, Berlin August 2002)

An already explored strategy has been to use metaphors from other areas such cartography or the natural environment.

The natural environment we inhabit has always been a source of inspiration to artists. For example the organisms in a mature ecosystem live following some rules. And those rules can be mapped to visualization design in the context of the optimization of the cognitive potential of visualization users.

Do not forget that the users live in the real world, not in a metaphorical one. Some unneeded metaphors are nice at first, but are immediately tiring.

Analogy and metaphor are important contributors to creative thinking. An analogy is more creative the smaller the distance within local subspaces and the larger the distance between comparative subspaces.

The concept of metaphor derives straight-forwardly from analogy, in that they both force a different perspective or interpretation of a situation or construct.

Artistic forms comparison

Visual arts
Obvious comparison.
Show what the photographers want to show.
Tell a story, engage audience, create visual writing, ignite imagination.
Structures, symmetries. Communicate information in the absence of visual contact.
Plastic arts
Direct manipulation, 3D, touching and force feedback. Beauty as balance: any addition is superfluous, every subtraction leaves a hole.
Tell a story, focus attention.
Graphic design
Composition, balance, usability. They developed guidelines (related to modes of perception) for the use of type, form and color to optimize visual communication.
Highlights interesting parts, omits unimportant details. Specific features enhancement and highlight.
Optical illusions
Pre-wired interpretations and assumptions.


[Sensemaking and Creativity process comparison]

Do not forget creativity. There are interesting parallels with visualization (and obviously arts).

Graphical methods support
Mindmaps, Concept Maps
Sensemaking cycle and creativity process. Knowledge as a net of relationships
Go out of the box
See the data in unusual ways, force unexpected relationships
Ambiguity to facilitate flexible thinking

A key activity in the process is the exploration of ideas, knowledge, and options. Some examples of aspects of exploration that were identified from empirical studies are summarized here:

From: E. Edmonds, L. Candy: Creativity, art practice, and knowledge, Communications of the ACM, Volume 45, Number 10 (October 2002), Pages 91-95

Ben Shneiderman for his work on human computer interaction and in the area of computer support to creativity.

Creativity and the exploratory visualization process: Which tools are needed? Which visualization tools do I use that help my exploratory visualization tasks?

Margareth Boden makes the point that changing a constraint might be at the core of creative thinking.

Other important characteristics of creative designers:

A key activity in the process is the exploration of ideas, knowledge and options.

Modeling the creative process support systems shows that knowledge and visualization are essential ingredients of creativity work.

Parts of the creativity process:

"The disappointing fact is that the majority of design is not in groundbreaking innovation, but in variations on something successful."

Analogy and metaphor are important contributors to creative thinking. This is accomplished by generating pre-inventive forms or structures and then tweaking (that’s jargon for twisting, forcing, inquiring, etc.) the form to relate it to a novel avenue of thought or procedure. Normally it doesn’t appear to fit immediately. That’s precisely the point; one has to generate novel relationships and perspectives to be creative, and this mechanism forces one to do just that. An analogy is more creative the smaller the distance within local subspaces and the larger the distance between comparative subspaces.

The concept of metaphor derives straight-forwardly from analogy, in that they both force a different perspective or interpretation of a situation or construct.

Mixing disciplines

There is a potential of misinterpretation due to the artistic inherent subjectivity of the process.

Art: pleases the eye. Science: communicates.

Generate new way of thinking. Not only provides new tools for the artist. This is not the point. But a computer could add new artistic dimensions (time) or break established conventions.

An interesting aspect of the collaboration between artists and scientists is the way in which it provides participant with more than one viewpoint about the nature of the creativity process.

The artist (or the visualizer) needs the correct visual vocabulary to make realistic proposals to the scientist.

Techniques from the arts

On the minus side:

General resources

Interesting pages

Image galleries