I. Focus:

In 1995, Joshua Quittner, who has become aware of the narrative possibilities of the Internet, called for a brand new journalism in the online magazine Hotwired. Since then, his call has been heard and further propagated by more and more journalists as well as scientists (Meier 1999: 129; Neuberger 2000: 310; Wagner 1998: 210).

Following Quittners lead, my research deals with the narrative possibilities of web-composition (= ”Web-Inszenierung”), focusing on the phenomenon of the multimodal and multi-media based (hyper-) text and the question how the semiotic and technical possibilities of the Internet can be used sensibly and effectively for online-journalistic article design so that information transferring and opinion building abilities of online-journalistic texts are optimized.

To answer this question, it is unavoidable to also take the user’s side into account, which I did by following Wolfgang Schweiger’s example and including the “uses-and-gratifications-approach” into my research perspective (Schweiger 2003: 294). The user-needs considered here correspond to a general understanding of the functions of journalistic media to inform, comment and entertain.(1)

So, the main question of this article is: How can online-journalism use the possibilities of web-composition in such a way that the resulting articles are able to inform effectively, stimulate opinion building and are interesting to and enjoyed by their recipients? To make suggestions about how online-journalistic article design could be improved to better fulfill the afore mentioned user needs, the following questions have to be answered:

- What are the new requirements on journalistic storytelling that arise through the technical characteristics of the Internet?
- How can the possibilities of web-composition be used in such a way that they ensure maximum readability of texts and highest emotional satisfaction of the users?
- What steps have already been taken in this direction? (What kind of action has still to be taken?)

II. Web-composition and the concept of the "extensive audio-visual event"

For the development of new storytelling forms on the Internet, the following two concepts are central: web-composition and the extensive audio-visual event (= audio-visuelles Gesamtereignis).

Considerations about web-composition have to start with the technical characteristics of the Internet that can influence storytelling in a way that was / is unthinkable in traditional media such as TV, radio or newspaper.

These technical characteristics are hypertext and hypermodality. Hypertext and hypermodality contain a huge design and meaning-making potential that can prove as a big advantage for effective information transfer compared to traditional media.

But to realize this potential, the technical possibilities have to be used sensibly. It is not at all trivial, how the various media that are used to produce the information are applied and in which way the different information parts are combined with each other.

This is where the extensive audio-visual event comes into play. This concept focuses - analogous to its definition by Ursula Hentschläger and Zelko Wiener (2) - on the “inner structure” resp. “coherence” of an online-journalistic product.

A coherent text must be the aim of any media adequate article design, because otherwise the informational surplus value is lost. While Hentschläger and Wiener focus on the creation of suspense – that means a dramaturgically sound structure – I favor the more encompassing concept of web-composition.

Web-composition contains two central aspects, namely the translation of a design into a concrete product, but also the design process itself. The reason why the latter is so very important is that the possibilities for multiplying meaning have already to be considered during the outlining of an online-journalistic article.

Afterwards it’s always difficult – if not impossible – to form a coherent text from a lot of lose and uncoordinated pieces. That also means that it does not make sense to use multi-media for it’s own sake or to take already existing material, and transfer it unchanged onto the Web, like it happens often with TV or radio pieces but also with print articles, which is less terrible but also very problematic (Schröter 2001: 76).

My approach to the question how the technical possibilities of the Internet can be applied in a sensible way is interdisciplinary and unprecedented in its form and systematic outline. The theoretical considerations about effective web-composition are rooted in narrative theory as well as textlinguistics, take research results from media- and perception-psychology into account and refer to elements from hypertext-, multimodality-, usability- and readability-research.

Only through incorporating all these different disciplines into the principle of web-composition, it is possible to develop a concept of a multi-layered, coherent and semantically complex hypertext, which bears higher emotional satisfaction, better readability and an informational surplus value for the recipient compared to traditional article-design.

II.I. Technical building stones of the extensive audio-visual
event and their sensible composition

II.I.I. The hypertext principle

The term was coined by Ted Nelson and contains two central characteristics: non-linear textual organization and electronic publishing form. Non-linearity of textual information in itself is not new.

It has been used for a long time by newspapers and magazines where it is common practice to start an article on the front page and continue it later on (Bucher 1996: 31-61 and Bucher 1999: 9-32).

What is new is the radicalization of this organizational principle through the jumping between information pieces via hyperlinks, as well as the multi-channel information transfer in the sense of hypermedia (Storrer 1999: 35).

The hypertext principle contains the following technical characteristics that can - if applied sensibly - contribute to effective (online-journalistic) information transfer:

    1)   De-linearization
    2)   Interactivity
    3)   Multimodality (3)

Ad 1) De-linearization: This means that information is broken down into smaller chunks, which are then presented in separate modules. De-linearization solves the dilemma that comes with the fact that on the one hand people don’t like to scroll and that therefore information on a page must be kept short(4), but on the other hand, also online-journalism should serve the function to thoroughly inform its users. Furthermore, modularization offers the recipients various entry points into a story and allows several strategies of browsing the information.

While a long text presents the user with the choice for all-or-nothing – reading the whole thing or skipping it – the cluster presentation gives her the freedom to select. (Bucher 1996: 44)(5).

The user can decide upon how deep to dig into the information depending on former knowledge and personal interest. Modularizing the different aspects of a story also makes it much easier to present circumstances and events from different perspectives. Again, the user can either choose from the different perspectives or try to get a complete understanding of things by examining all available points of view. It goes without saying that this individualization of online-journalistic articles is only possible if de-linearization really happens, and the hypertext principle is not misused for cutting a long piece of text into smaller chunks that, although presented on different pages, again follow a linear sequence.

The reception of hypertexts is a very active process, because the user constantly has to make choices in the course of finding her way through the information offer (Bucher 1996: 31-61).

Consequently, interactivity, which is strongly related to the de-linearization and individualization of hypertexts, can enhance comprehension and recollection of texts (Pfammatter 1998b: 59).

The reason for this is that the cognitive stimulation arising while reading a non-linear text is generally bigger because of the constant choice- and sense-making process then in a linear texts, where the user “only” has to reconstruct the path the author has chosen for her (Storrer 1999: 45).

Ad3) Multimodality:

This means that different modes of perception are combined and presented in one medium – the computer (Schmitt 1998: 295).

The advantages resulting from multimodality for effective information transfer will be discussed later, in relation to the second theoretical building stone of web-composition – the hypermodality principle.

II.I.I.I Sensible application of the hypertext-principle

The hypertext-specific gratifications of de-linearization and interactivity for the user can only be realized, when some thought has been invested concerning the sensible hypertextual presentation of a topic (6).

Losing orientation is not restricted to long scroll-texts but it can also happen if a text has been badly modularized.

So, to help the user to integrate the various information pieces, it is recommended to give some guidelines – so called “coherence aids” (see: Bucher 1996:47; Pfammatter 1998b: 64-67; Meier 1999: 45; Storrer 1999: 33-65).

Furthermore, the various information units should be self-contained, which means that textual relations that are constructed through the formal means of grammar should not go beyond the boundary of an information unit. (Pfammatter 1998b: 54f.).

To enhance orientation, it is also important that the granularity of the hypertext is right, meaning the information units are neither too small nor too big (for a rule of thumb see: Pfammatter 1998b: 54f.).

That hypertexts can improve comprehension and recollection of texts through cognitive stimulation only stays true as long as the hypertextual structure does not get too complex. When stimulation turns into stress, the positive effect is certainly lost.

II.I.II The hypermodality principle

The new thing about the Internet compared to traditional media is that it presents a digital platform on which several different media can be combined. Of course the concept of multimodality in itself is not new (Doelker 1998: 37), but only through digitalization, that enables the transformation of all kinds of information into binary code, it has become possible to bring those modes together on one level of presentation.

The conceptual basis for multimodality is the hypertext principle (Pfammatter 1998a: 12), which is the reason that the more suitable term “hypermedia” is often used instead of “multimedia”.

The available media that can now be combined are spoken language, sounds, music, animation, video, text, images, and computer graphics (Pfammatter 1998a: 10).

The call for new storytelling forms on the web includes the use of these elements, meaning that on the producer’s side, journalistic, technical and compositional tasks grow more and more inseparable (Schmitt 1998:297).

Concerning the meaning-making potential of online-texts and the question how it can be most effectively realized, it is now necessary to have a closer look at the hypermodality principle.

Jay Lemke coined the term "hypermodality" by conflating "multimodality" and "hypertext" (Lemke 2002: 301).

For the concept of the extensive audio-visual event, the hypermodality principle is of central importance, because it goes far beyond the technical possibilities of multi-channel information transfer and focuses on the multiple semiotic relations that can arise from the combination of different modes within a singular webpage, but also – in the sense of Lemke’s “traversals” (Lemke 2002: 301) – across pages.

For the extensive audio-visual event, the difference between “mode” and “medium” is essential. Kress and van Leeuwen define “mode” as semiotic resource in a design process.

During the production of a concrete text, this semiotic resource is materialized and takes the form of a medium.

“But design is still separate from the actual material production of the semiotic product or the actual material articulation of the semiotic event. The resources on which design draws, the semiotic modes, are still abstract, capable of being realized in different materialities. Language, for instance, is a semiotic mode because it can be realized either as speech or as writing, and writing is a semiotic mode too, because it can be realized as engraving in stone, as calligraphy on certificates, as print on glossy paper, and all these media add a further layer of signification. The writer of the ‚ethnic conflict’ thriller, apart from using language, also uses the resources of the mode of narrative in designing the thriller. And this mode is separate from the medium of the printed book in which it will be produced.”

I hope this distinction makes it clear, why hypermodality is so much more important to the extensive audio-visual event then hypermediality. It is because hypermodality takes place in the beginning of a design process, not at its end.

The choices you make about the modes you would like to use to transfer meaning should go hand in hand with considerations about how those modes – in their medial form, of course – are going to interact with each other.

The questions which channel should be used for what kind of information and how the different modes can ideally support each other to multiply their meaning should be answered during the outlining of an article.

If that does not happen and thoughts about which medium to choose occur only during production or even worse, when all the elements are already finished, it will be hard if not impossible to make a coherent text from incoherent parts.

It’s like trying to make a puzzle out of a lot of puzzle stones that simply do not match. If the inner structure is lacking, effective information transfer will suffer.

Now it is time to take a closer look at the potential for multiplying meaning that comes with hypermodal article design.

According to Lemke, every semiotic mode draws upon three resources: the presentational, the orientational and the organizational (Lemke 2002: 300).

Presentational meanings are those which present some state of affairs. We construe a state of affairs principally from the ideational content of texts, what they say about processes, events, participants, and circumstances.

For images, one could apply the same terms, recognizing what is shown or portrayed, whether figural or abstract (Kress and van Leeuwen 1996).

It is this aspect of meaning which allows us to interpret the child’s unfamiliar scrawl on paper through his use of the word ‚cat’, or his indecipherable speech through his pantomime of eating.

Orientational meanings are more deeply presupposed; they are those which indicate to us what is happening in the communicative relationship and what stance its participants may have to each other and to the presentational content.

These are the meanings by which we orient to each other in action and feeling, and to our community in terms of point of view, attitudes, and values. In text we orient to the communication situation primarily in terms of speech acts and exchanges: are we being offered something, or is something being demanded of us? Are we being treated intimately or distantly, respectfully or disdainfully? We assess point of view in terms of how states of affairs are evaluated and which rhetorics and discourses are being deployed.

The actual signs range from the mood of a clause (interrogative, imperative) to its modality (uncertainty, insistence), from markers of formality to the lexis of peer-status, from sentence adverbials (unfortunately, surprisingly) to explicit evaluations (it’s terrible that…).

Visually, there is also a presumptive communicative or rhetorical relationship in which the image mediates between creators and viewers and projects a stance or point of view both toward the viewer and toward the content presented in the image.

Organizational meanings are largely instrumental and backgrounded; they enable the other two kinds of meaning to achieve greater degrees of complexity and precision. Most fundamentally, organizational resources for meaning enable us to make and tell which other signs go together in larger units.

These may be structural units, which are contiguous in text or image-space, and usually contain elements which are differentiated in function (subject / predicate in the clause; foreground / background in image composition).

Or they may be cohesive or catenative chains which may be distributed rather than contiguous, and in which similarity and contrast-within-similarity of features tie together longer stretches of text or greater extent of image as a unity or whole (repetition of words and synonyms; unity of palette). (Lemke 2002: 304f.)

The various modes of a text can interact with each other on all three levels of meaning-making, thus not increasing meaning in an additive but a multiplicative sense. This is bound to lead to an informational surplus value for the user that cannot be found in traditional forms of journalistic article design – but again, only if hypermodality is sensibly applied.

Multimodality research, media pedagogics and experiences from the field of e-learning showed how hypermodal texts can enhance effective information transfer for the recipient. The advantages such texts have compared to current forms can be summed up as follows:

- If a lot of complex information is mediated via only one channel, the recipient might well have troubles absorbing it, is getting stressed, bored or both and will look for more agreeable information offers. So, using multiple channels to transfer information is a possibility to reduce the information overload that might accompany monomodality (Weidenmann 2002: 53f.).

- Hypermodal texts make it easier for the recipient to build mental models of circumstances, events and correlations, thus improving text comprehension (Weidenmann 2002: 53-55).

- If the hypertext-principle is enriched by the use of multi-media, information design that presents a range of possible perspectives becomes more authentic, comprehensible and vivid which can improve the user’s ability to form an opinion (Wiedenmann 2002: 61).

- As I have already mentioned earlier, the interactivity that goes along with the reception of hypermodal texts enhances user activity, thus promoting text comprehension and recollection.

II.I.II.I. Sensible application of the hypermodality-principle

Hypermodal text design bears a huge potential for effective information transfer. But again, to realize this potential, an article must have an inner structure that relates the individual elements to each other.

That means that using images complementary to written language improves text recollection only if there is a temporal contiguity between visual and textual representation. If this contiguity exists, the simultaneously applied elements are “mapped” by the user: “The verbal and visual representations are brought in relation to each other and integrated.(7)”.

This results in better text recollection especially after a longer period (Weidenmann 2002: 52).

But multi-media based representations can also confuse and over-stimulate the user, thus having clear disadvantages compared to traditional information design. This happens when the elements of an information offer are badly coordinated and insufficiently synchronized.

To avoid this, the individual elements should not only be related to each other structurally (= level of “organization”), but also content-wise (= level of “presentation”), so that the informational building stones can be mentally integrated by the user and turned into one consistent whole (Weidenmann 2002: 54).

Again, how the various elements should interrelate, ought to be planned before the article is being produced. The application principles that have to be considered when designing a hypermodal text refer to textlinguistics on the one hand and to the psychology of perception on the other hand, because visual design plays an important part in web-composition.

For the framework of the extensive audio-visual event and the following analysis, Kress’ and van Leeuwen’s work about the “Grammar of Visual Design”, mainly their articulation of a so called “integration code” for spatial composition (Kress / van Leeuwen 1996: 183 - 230), proved most useful.

Now, how can an author arrange the individual information building stones of a hypermodal text in such a way that they most effectively support each other (Bucher 1996: 36)?

The basis for this mutual support is the article’s inner structure, which promotes textual cohesion and coherence without limiting the user’s freedom to choose her own path through an information offer, but enabling her to grasp the multiplicative meaning resulting from the interrelations of the individual elements.

The visual design of an article plays a major role in producing textual cohesion and therefore has to be carefully considered when building extensive audio-visual events. If the elements are not adequately arranged on the screen, it becomes difficult to tell if they belong together or not.

The so called “content orbiting”(8) stands for a layout that features the written text of an article in the middle of the page and it’s multimedia elements in a separate column to it’s left or right, which makes it difficult to tell on first sight what still belongs to the article and what is part of the navigation bar or else.

Through this design, the multimedia elements are degraded to mere accessories whose information value appears subordinate to the written text. Their role in the meaning-making process is minimal – or at least difficult to perceive – and it is questionable if the user will notice them at all.

At the heart of the extensive audio-visual event lies the semiotic integration of all individual parts of an article into a meaningful whole. An essential integration code of multimodal texts is the spatial composition (Kress / van Leeuwen 1996: 212).

The following design principles are central to the production of cohesion on the structural level. These design principles also served as variables for the qualitative content analysis.

“Center and Margin”: The information value of an element depends strongly on its position in a layout. The highest information value is reserved for the Center of a composition. The elements that are placed around the Center are called Margins. Their information value is subservient to the information given in the Center. Theses differences in information value depending on position are the reason that some elements are more relevant for the meaning-production of the whole text than others (Kress / van Leeuwen 1996: 206). This distinction in central (i.e. important information) and marginal (i.e. subordinate information) becomes even more drastic when “Framing” is involved.

“Framing”: Framing stands for the separation of distinct information elements from each other. “Framing can […] be realized by by frame lines (the thickness of which then indicates the strength of the framing), by discontinuities of color or shape […] or simply by empty space between the elements” (Kress / van Leeuwen 1996: 216). The recipient perceives the framed elements as separate entities, which do not relate to each other.

“Salience”: Salience indicates the “weight” or importance of an element of a composition. “[…] the greater the weight of an element, the greater its salience”. (Kress / van Leeuwen 1996: 212). Salience can give an element weight, no matter where it is placed in a layout. Salience is produced by a complex interaction of a number of factors like tonal contrast, color contrast, sharpness of focus, size, perspective as well as cultural factors.
(Kress / van Leeuwen 1996: 212)

”Reading Path”: The afore mentioned design principles form the hypothetical reading path. This reading path is planned by the author to lead a reader through the text. It does this by building hierarchies between the individual elements, thus determining the sequence of their reception. “Such reading paths begin with the most salient element, from there move to the next most salient element, and so on.”
(Kress / van Leeuwen 1996: 218).

It certainly cannot be the goal of a hypertext to dictate a very strict reading path, but it still is a good idea to offer suggestions on what to read and where to go next.

II.II The extensive audio-visual event – summary

An extensive audio-visual event is a text in which, through cohesion and coherence, the multiple semiotic relations between its modi - within the three levels of meaning production as well as across them – are interpretable for the recipient as a meaningful whole.

A qualitative analysis of a hypermodal text will show in some detail what kinds of cohesive and coherent ties can exist between the individual elements. Only through this inner structure, that is a result of coherence and cohesion, multimedia allows for more text comprehension compared to traditional forms, contributes to a more effective information transfer and promotes opinion building.

III. Status Quo of online-journalistic article design

III.I Under which circumstances are online-journalistic articles produced?

To whish for the extensive audio-visual event as an alternative to current online-journalistic article design is one thing, but to integrate it into the daily routine of online-journalism is a completely different story.

In the following, I’d like to give an account of my investigations in the online-editorial offices of various well-known Austrian and American mass media.

In autumn 2001 and summer 2002, I visited various editorial offices for several days and interviewed a large number of the editorial staff as well as chiefs of company, web-designers and technicians.(9)

The quintessential thing, I discovered during my field research, was that the circumstances under which online-journalistic articles are produced today in Austrian and American editorial offices, are by and large not at all favorable for the production of extensive audio-visual events, even make them impossible sometimes.

This starts with the fact that all traditional media try to use their websites as a platform to doubly sell their print-, radio-, TV- content,(10) as well as additional content that is produced originally for the web, but they are still lacking a clear idea, what the users want from their online-versions.

This insecurity results from the fact that the Internet still is a very young medium and has not yet developed individual journalistic forms. Traditional media eye it with suspicion, because they think it quick and global but don’t see what it has to offer concerning appealing information design that can match the quality of the more elaborate forms employed by established media like newspaper, radio and TV.

This is a vicious circle, because if nobody ever begins to experiment with the narrative possibilities of the Internet, its information design will stay behind forever and it will never be regarded as an autonomous journalistic medium.

Because the chiefs of companies and chief editors – who know very well that at least their younger audience expects them to have a website – are insecure as to what should be offered online, they are not over-eager to invest much money into the web-platforms.

Another reason for their reluctance to spend money on an online-presence is the fact that there is no significant money to be made with it. Most online-editorial offices must be glad to at least break even.

Every traditional medium I visited, except for the Chicago Tribune, tries to meet the expectations of its audience by investing an absolute minimum of financial and personnel resources.

Where an online-editorial office exists at all, the staff is much smaller than that of the traditional medium. Furthermore, online-journalists generally are much younger than their colleagues working for the traditional medium, have therefore less journalistic experience and are less qualified for their job.

Concerning the production of extensive audio-visual events it is particularly problematic that online-journalists do not show a pronounced understanding or awareness for the design affordances of their medium.

This indicates that the Internet is not taken seriously as an autonomous journalistic medium. It would be a rare exception if an editor in chief would allow an untrained student to produce something for TV- or radio- broadcasting without further instruction or would even let her write an article for a newspaper.

Producing for traditional media is considered high art by traditional journalists, because the affordances of the medium concerning information design have to be met. But for the web, well everybody can produce for the web as long as she is able to handle the editorial system and to keep her sentences short, so the unanimous opinion of traditional journalists.

And for now, this really might be enough because hypermodal text design rarely ever happens. For the creation of extensive audio-visual events, it is especially disquieting that the journalistic core tasks like investigating events and circumstances and producing the actual information pieces, are taken away from online-journalists.

The content for the website is to a large extent, sometimes exclusively, taken from the traditional medium or from external sources like press agencies. It certainly makes sense that the available information is also presented online, but how this happens in practice is very problematic for the creation of extensive audio-visual events.

Except for the Chicago Tribune, traditional- and online-editorial offices do not cooperate, thus prohibiting the composition of extensive-audio-visual events. It works like this: either there is no cooperation at all between core- and online- medium or the finished pieces are taken from the traditional medium and pasted without further modification on the website.(11)

As for radio- and TV-producers, they have to do double work if there are no online-journalists (or if they are only supervising and coordinating the content-transfer but not producing content themselves, as it is often the case), because additionally to their work for the traditional medium, they also have to produce manuscripts of their pieces for web publishing. In every case, the result will not be an extensive audio-visual event.

In the first case the online-journalists are lacking multi-media based material, i.e. the possibilities of web-composition stay unused. In the second case, where the traditional journalists have to serve two masters, producing for the Internet always comes second to producing for the core medium, leaving not enough time for careful web-composition, not to mention that they are not trained to do it and have neither understanding of nor interest in the affordances of the medium.

Therefore TV and radio-websites are dominated by written text containing not much more than short program information and are thus not further analyzed.

III.II What consequences have the production circumstances for online-journalistic article design?

III.II.I Results of the qualitative content analysis

The qualitative content analysis of derStandard.at and chicagortibune.com, which I conducted in spring 2003, showed that at least in the arts and entertainment channels, the possibilities of web-composition are hardly ever used and if they are, they do not correspond to the design criteria of extensive audio-visual events.

The arts and entertainment channel of derStandard.at contained 206 articles, the vast majority of which was conventionally designed, meaning that written text presented the basis for the articles. Only 18% took the form of moderated photo-galleries.(12) 37% of the conventionally designed articles were written text only, without any multimedia element (not even pictures).

A little more over 50% of the articles had at least one picture. Other elements were hardly used.(13) Longer texts were never modularized, but always presented as scroll texts.

Concerning the concept of the extensive audio-visual event, it is important to know, what part the multimedia elements of an article play for the production of a meaningful whole.

Are pictures, videos, animations and written text contributing equally important to the meaning production, or are the multimedia elements semantically subservient to the written text? The content analysis of derStandard.at showed that pictures mostly play the role of accessories.

They are marginally positioned and seldom accompanied by explaining text. Videos are however quite often (60%) integrated in the text, thus seeming to be of equal value to the meaning making than the written text.

It is further interesting that only 16% of the analyzed articles were originally produced by online-journalists. The rest was either taken from the newspaper or external sources like the Austrian Press Agency.

A very similar picture presents the Online-Version of the Chicago Tribune. From the 481 analyzed articles, 97% were conventionally designed. Only 3% took the form of moderated photo-galleries.

Even less articles than on derStandard.at were enriched with pictures. 85% were text-only articles without a single multimedia element. Very sparsely graphics (2%), audios (1%) and videos (2,7%) could be found.

All the multimedia elements were used as mere accessories, being generally positioned in a separate column to the right side of the written text.

Like derStandard.at the chicagotribune.com also did not make use of hypertextual information structuring, although it contained scroll-texts that printed out were over 16 pages long and to which modularization would have been an enormous benefit.

Only 1,45% of all articles, were produced at the online-editorial office.(14)

III.II.II Qualitative text analysis

The following exemplary qualitative text analysis of a chicagotribune.com article throws light upon the question, in how far hypermodal articles fulfill the criteria of extensive audio-visual events for formal and semantic coherence / cohesion.

The article “Rediscovered score pianist’s last legacy” by Howard Reich, was published on www.chicagotribune.com on the 11th of August 2002.

III.II.II.I Organisation

The chicagotribune.com employs a horizontal-vertical grid layout. The page begins with three columns. Because the central column is not only more expansive than the sidebars but also much longer, the left and right columns disappear further down the text. (Printed out the article is six A4 pages long. The three columns only last for one and a half pages. The rest of the article has only one column). As soon as the right sidebar is not needed any more, the written text fills out the now empty space by expanding to this side.

The general background of the page is white, but the left and right columns have a grey background, which distinguishes them quite clearly from the rest of the page. They do not seem to be part of the article because they are placed in different frames. Because of the different background color, the framing is very prominent. At the same time, the similarity between the sidebars relates them to each other. At first glance, they seem to belong together. Both contain – contrary to the central column – blue colored hyperlinks. Only the color of the category headings, which is grey in the left and red in the right column, distinguishes them from each other. Because red has a greater salience than grey, the eye is automatically drawn to the right column first, interpreting the left column as less interesting. This makes sense, because the left sidebar contains the unchanging navigation bar, whereas the information offered in the right sidebar is related to the article. This also corresponds to the observations of Kress and van Leeuwen, namely that the semantic implications of left and right correspond to the information values “Given” and “New”, the left standing for the “Given”, the right representing the “New” (Kress / van Leeuwen 1996: 187). Because the focus of this analysis lies on the article, the navigation bar will not be discussed further.

What is interesting is how the individual elements in the right sidebar are related to, or distinguished from each other. Partly they belong to the particular article, but the first three hyperlinks of the right sidebar can be found in every article of the chicagotribune.com(15). They do not possess any cohesive relation to the article. Seven red headings subdivide the right column into seven distinct sections. The first heading reads “photo gallery” and is placed above the picture of a pianist. The caption that also functions as a hyperlink says “Paul Wittgenstein”. Picture and caption possess a cross-modal cohesive connection through reference. The lexical cohesion between the name “Paul Wittgenstein” in the subheading of the article and the caption connects the written text with the photo gallery. This connection becomes more obvious, when you click on the caption to open the “photo gallery” about Paul Wittgenstein. It contains seven hypertextually connected and linearly sequenced pictures. Each picture is accompanied by a comment that helps to connect it to the corresponding part in the written text of the article. The cohesive relationship between the pictures and the written text is that of cross-modal reference. The pictures refer visually to the written text, thus illustrating it. For example, there is a picture of a painting that shows the Wittgenstein’s drawing room which is described in the text, a picture of Paul Hindemith who is mentioned several times, and a picture of Paul Wittgenstein in front of his house in New York, etc.

The following sections of the right column (“Your Turn”, Graphic” and “Quote”) are lexically cohesive with each other as well as with the written text through repeating the name “Paul Wittgenstein”. All of them contain either the full name of the pianist or at least his last name. The graphic is like the photo-gallery connected to the written text through cross-modal reference. It bears the heading “Wittgenstein’s dealings with the Nazis” and shows (illegible) parts of Wittgenstein’s correspondences with the Nazis, which are mentioned several times throughout the text.

The “Quote” in the right sidebar also appears in full length in the written text, therefore establishing a particularly strong connection between the two columns. The section that follows the “video” heading contains six links to video clips showing the author, Howard Reich, talking about the theme of the article. The links are lexically cohesive through the repetition of the name “Reich”. Because it is obvious that Howard Reich is the author of the article, his name connects the right with the middle column.

The links to the video-clips are further related to each other by using various synonyms of “telling about”. Four links contain the phrase “Reich on” and the variations “Reich comments on”, “Reich talks about”. The things he talks about are also central elements of the written text (“the concerto for the left hand”), thus again relating the videos to the middle column.

The two hyperlinks in the section “On the Web” are also connected to the article, because they lead to external websites containing further information about “left hand piano music” and “Paul Hindemith”. The last section features one link to the collected articles by Howard Reich. The name again serves as lexical cohesion to the written text. Except for the first three hyperlinks, all the elements of the right sidebar are somehow structurally and with regard to their contents related to written text. But only the elements subsumed under the headings “Photo Gallery”, “Graphic”, “Quote” and “Video” are really modules of the article. The section that bears the title “Your Turn” is distinguished from the other article modules, whose text color is black, by its red text color. If we take at this point the presentational meaning into account, it becomes clear that “Your Turn” is not part of the article at all, but contains user comments. A further link leads to more comments and to a field where you can enter your own commentary to be sent to chicagotribune.com.

I think I have now exhaustively explained the cohesive interrelations within the right column as well as between right and middle column and it is time, to have a closer look at the middle column:

By its prominent position on the page, the white background color and its bigger size, the middle column that contains the written text is signified as the core of the article, whereas the sidebars appear subservient to it (Kress / van Leeuwen 1996: 206). The article is announced by a heading in bold face, which is followed by a quite long subheading that also serves as teaser. Six more bold-faced subheadings further divide the text into six distinct paragraphs. The biggest salience on the whole page however, has the advertisement in the middle column. It is placed relatively high on the page and can therefore be seen also on small screens instantly without scrolling. It horizontally covers more than half the middle column, it is bigger than every other picture on the page and it furthermore attracts attention by regularly changing its content. The article almost seems to exist only as a frame for the advertisement. Apart from the visual integration, there is no cohesion between the advertisement and the written text, showing unmistakably that it is not part of the article.

What has been shown above is that the article consists of ten individual modules. The core module is the written text in the middle column, which is accompanied by a graphic, a quote and six video-clips in the right sidebar. Through putting the multimedia elements in a separate column, they are signified very clearly as mere accessories to the core module – i.e. the written text. The visual composition does not promote a complementary reception of the various elements. It is very hard to structurally or substantially relate the pictures in the photo-gallery to the corresponding text passages. Because the text is so long, you need a great memory to recognize the picture that goes with a certain passage. The same problem presents itself when a parallel reception of graphic and text is attempted. Between the video-clips and the adjoining text is also neither structural nor substantial connection. But not only is there no structural relation between the elements in the right sidebar and the neighboring text passages, also within the right column, the individual sections seem to follow an arbitrary sequence.

The hypertextual structure of the article is hierarchical. The hypermedia elements (photo-gallery, graphic, videos) can only be reached from the initial article page (see screenshots). Clicking on a link to an element opens it in a new window. The scrollbar on the right side of the screen functions as vertical vector. It indicates that only a small part of the article is visible on the page. Considering how long a user has to scroll downwards to reach the end of the text, it might have been a good idea to put the information into smaller chunks and to link them hypertextually, thus using modularization not only for the multimedia elements but for the whole article.

III.II.II.II Orientation

The article “Rediscovered score pianist’s last legacy” is a report. The characteristics of this journalistic form are well known and imply that the user can expect objective, unbiased and value-free information. In the case of the article at issue, this is true for all its elements. Only the readers express their opinion in the section “Your Turn”. With the words “What do you think of Wittgenstein’s story”, they are encouraged to present their views. The link with the title “Read more comments or post your own” is a further call for user action. The other links can be interpreted as implicate offers to the recipients. Their titles indicate where they are leading, but express no request to follow them.

III.II.II.III Presentation:

In my analysis of the article’s organization I have already demonstrated that the essential thing of “Rediscovered score pianist’s last legacy” is the middle column. The same holds true on the level of presentation. The written part of the article, which was produced at the print editorial-office of the Chicago Tribune, is completely autonomous. The multimedia elements, which were added at the editorial office, are substantially superfluous. The story is told by the written text only. Die photo-gallery and the graphic only serve as illustrations for the written text. The video-clips, which I will discuss later in more detail, also bear no additional information.

The peg on which the story hangs is the discovery of Paul Hindemith’s “Concerto for the Left Hand” that was believed lost for a long time. The one-armed pianist Paul Wittgenstein once ordered it, but then never performed. It (probably) was too modern for him. Around the discovery of this long lost piece of music, the fascinating, tragic and touching story of the strong-willed pianist Paul Wittgenstein and his family is told.

The three main subjects of the article are:
  1. Paul Wittgenstein’s family: the cream of contemporary artists and intellectuals (Brahms, Casals, Freud) regularly met at the house of the wealthy industrialist Kurt Wittgenstein (Paul’s father). Kurt was determined to make his sons into industrialists and drove three of eight into suicide. He finally gave freer rein to Paul and Ludwig. Ludwig became a famous philosopher, Paul a remarkable pianist.

  2. Paul Wittgenstein’s fate and the “Concerto for the Left Hand”: During the First World War, Paul, who had already started a promising career as a pianist, lost his right arm. With huge will power, he finally managed to create the illusion of playing the piano with two hands with his left hand alone, and made a successful return to international concert stages. He spent a large part of his fortune on hiring the greats of his time to write left-hand piano music for him. Among them was Paul Hindemith, whose “Concerto for the Left Hand” is of special importance to musical history. Therefore, the rediscovery of this piece is sensational.

  3. Paul Wittgenstein’s dealings with the Nazis: To stabilize the German mark, all Austrians were required to exchange their money into German marks deposited in the Reichsbank. The Nazis were of course especially interested in the enormous wealth of the Wittgenstein family, but they couldn’t touch it, because it was in a Swiss trust. So, they used the Wittgenstein’s Jewish origins to exert pressure on the family and to make them deliver their money. Knowing that Paul’s sisters yearned to stay in Vienna, the Nazis offered a deal: If they turned the Swiss francs into German marks at the Reichsbank, they would receive special mischling (half-breed) status, which would allow them to remain in Vienna exempt from the Nuremberg Laws. To get to the money, all siblings had to unanimously agree on how it was to be disposed. Paul, who distrusted the Nazis and had meanwhile emigrated to America, was against this deal, but finally agreed to do everything so that his sister’s could stay in Vienna and survive the Nazi regime unharmed. After the deal was signed, Paul’s sisters reviled him for having been uncooperative with the Nazis. The family was torn apart.

The multimedia elements further elaborate upon these thematical building stones of the article. The photo-gallery is the most general of them, because its pictures illustrate several passages throughout the text. The text that accompanies the pictures corresponds – sometimes even word-for-word – to the passages in the article that are illustrated by the pictures. The first picture is an exception from the chronological order because it has introductory function. It shows Paul Wittgenstein at the piano, thus visually introducing the man whose story is told in the article. The next picture shows Paul and Ludwig Wittgenstein as kids. The one after that is a painting of the Wittgenstein’s drawing room, where famous artists and intellectuals used to meet. The next picture shows Paul Hindemith, the composer of the “Concerto for the Left Hand”. The following picture shows Paul Wittgenstein in front of his house in Long Island, where he lived after having fled from the Nazi-Germany. The last picture shows the pianist with an unknown conductor in Tel Aviv in the late 50is or early 60ies. By illustrating the corresponding passages in the middle column, these pictures specify the described places and persons, allowing the user to get a clearer picture of Paul’s upbringing and living circumstances as well as of the great composer who wrote the “Concerto for the Left Hand” etc. Therein lies their informational surplus value.

That the pictures are accompanied by written comments not only facilitates finding the text passages to which they relate but also allows their autonomous reception. The photo-gallery tells Paul Wittgenstein’s story less fully, but it is per se comprehensive and conclusive. It therefore (and because there are so many pictures that would have to be downloaded were they integrated into the text) seems justified to place the photo-gallery in a separate column. It is only a valuable accessory to the article with limited meaning-making function. But it would have made sense to at least integrate links or thumbnails into the text at the right places that could be clicked on by the interested user to open or enlarge the picture. This would not be interruptive nor would it negatively influence download time, but it would enable the user to anticipate the content of the pictures (namely the illustration of a given text passage) and present her with the freedom to decide if she wanted to see the picture or not. The problem with a photo-gallery is, that you either have to click through all the pictures, without knowing what is coming next, or you risk to miss something that might have interested you, if you leave prematurely – and of course there is the difficulty of relating text and pictures when you first read the article and then click through the photo-gallery or the other way round.

The title of the graphic tells us that it relates to the third theme of the article: “Wittgenstein’s dealings with the Nazis”. It shows excerpts of chronologically ordered papers that document Wittgenstein’s “dealings” with the Nazis. The excerpts are accompanied with comments that enable the user to connect them to the corresponding passages in the middle column. This is necessary, because the excerpts are short and very hard to decipher, which also reduces their explanatory power. Only two of four excerpts fulfill their illustrative function, the other two can hardly be deciphered and a long text is needed to explain them. The information these explanations contain is not new and can also be found in the middle column. This is quite absurd if you think about it: the illustration of the written text needs a written commentary to explain what it tries to illustrate. The gratification for the user, who took the effort to follow the link to the graphic, is absolutely minimal.

The six video-clips deal mainly with Hindemith’s “Concerto for the Left hand”. From link-titles like “Tribune arts critic Howard Reich on how Paul Wittgenstein lost his right arm” it can already suspected that the video shows Howard Reich sitting in a TV studio, talking into the camera. This actually is the case for all six video-clips. In clip one, Reich talks about how Wittgenstein lost his right arm, how he nevertheless was determined to proceed with his career and that this was the reason for him to order the “Concerto”. In clip two Reich reports how Wittgenstein, thanks to his wealthy origin, was able to hire the greats of his time to write piano pieces for him, one of them being Paul Hindemith. His “Concerto” had been lost for the last 70 years. In clip three, Reich is speculating on the reason why Wittgenstein never performed the Hindemith piece: He might not have liked it. In clip four, the author is talking about how the piece was rediscovered in a farmhouse in Pennsylvania where Wittgenstein’s wife had lived until her death. She had enclosed all her husband’s documents in a room, which she had never allowed anybody to enter. Clip five is an exception because it does not deal with the “Concerto”. Reich is talking about Wittgenstein’s dealings with the Nazis and how this tore the family apart, because Paul’s sisters reviled him for having been uncooperative with the Nazis. The “Concerto” is again the subject of the sixth clip and deals with the question about its future. Reich suspects that pianists will compete about the chance to perform and record it first.

The six video clips are extracts from a TV interview with Howard Reich and build – except for the fifth clip – a coherent, conclusive report about the production, disappearance and sensational rediscovery of Paul Hindemith’s “Concerto for the Left Hand”. Because the clips only contain information that is also given in the written text of the article, their additional value for the user is questionable. It seems as if they were added to the article just because they already existed, not because they should help to achieve a certain communicational goal. It is a typical case of “showing videos for the video’s sake”. Apart from the fact that you see the author of the article (a picture would have been sufficient) and that you hear him talk (which is - with all due respect - not tremendously exciting either), their informational surplus value equals zero. For the article, the video-clips are quite superfluous. They make it neither more exciting, nor interesting or amusing. They do not offer new perspectives or elaborate upon old ones. Interviews with Wittgenstein’s heirs, his piano pupils or colleagues would have been much more interesting. Also statements from music experts about the value of Hindemith’s “Concerto” for musical history might have presented an informational surplus value for the users.

III.II.II.IV Conclusion

The length of the article suggests modularizing the information. The fact that the content can be divided into three distinct topics would be very accommodating for that. Cutting the information into smaller chunks would also facilitate a parallel reception of text and multimedia elements. Because the text passages would be shorter, the pictures, graphic, videos etc. could be directly integrated into the text at right place. The user would not have to remember the text passage that relates to a certain picture, which information in the written text is connected to a particular document clipping on the graphic, and what part of the text could have been substituted by a video. It would allow the recipient to decide on the spot if she would like to follow a certain link or not. If the written text is so prominently presented in the middle column and if all the other elements are marginalized, they hardly have a chance to be noticed at all and their role in the meaning making process will be very limited. At best, they serve as illustrations for the written text (like the photo-gallery and at least partly the graphic) at the worst they are superfluous like the six video-clips.

To produce a meaningful whole, in which every element is sensibly applied and positioned, it would be necessary to think of new narrative concepts that differ from traditional article design. It is quite obvious that the article at issue here was conceived as a print article and that the multimedia, which was added for the web-presentation is more or less “found footage”.

IV. The user’s side

To find out, what consequences current inadequate article design can have for text-comprehension and the emotional satisfaction of the users, I conducted a user study with 39 test persons. To test the effects certain design measures can have on emotional satisfaction and text-comprehension, I made five versions of an online-journalistic article about a film-project, each version differing from the other in one formal aspect only. Starting with version 1, that mirrors current article design (= a long scroll-text in the middle and pictures on the sides), I made the following modifications:

- Version 2: multimedia elements were added

- Version 3: the multimedia elements were integrated into the text at the corresponding places

- Version 4: modularization of the long scroll-text (no multimedia in this version)

- Version 5: final version which combines all formal measures taken in versions 2 to 4

Each version was given to eight (in one case seven) test persons.

A complex set of methods was used in the study. The socio-psychological predispositions of the users were especially taken into consideration, because emotional satisfaction and text-comprehension depend not exclusively on medial factors but also to a large extent on interest, motivation, education, social class, gender, foreknowledge etc.. Therefore an “integrative model of comprehension” (=”integratives Verstehensmodell”) (Lutz / Wodak 1987 : 76-85) “which modells text-comprehension as interactive process between text presentation and active user participation (16).” (Lutz / Wodak 1987: 207), built the theoretical basis for my study. To cover the quantitative as well as the qualitative aspects of text-comprehension, I applied psycho-linguistic methods for my empirical investigation that go beyond a mere content analysis. The test began with a questionnaire, which collected personal data of the test persons and asked them about their habits of media reception and their interest in the subject of the article. After a test person had “read” the article, he / she was asked to write down a re-narration of the article. A final questionnaire collected more data about text-comprehension and emotional satisfaction. The test-persons were asked to give reasons for their answers in a short comment. This helped me to differentiate the results.

The following results show how formal measures in online-journalistic article design can influence text-comprehension and emotional satisfaction of the users:

Formal measure: “adding multimedia”
Adding multimedia to the article not only enhanced text-reproduction, but also the emotional satisfaction of the test persons. 82,6 % of the test persons who had article versions with multimedia elements were very satisfied or satisfied, whereas 68,75% of the test persons who had versions without multimedia (only text and pictures) were rather not or not satisfied with the application of multimedia. Most test-persons gave the following reason:

Maybe I’ve overlooked them, but if not, video or audio-clips could have been added, for example sounds to accompany the pictures, or video-clips showing the director at work or in an interview.(17)

This confirms the hypothesis that the users appreciate the sensible application of multimedia and miss is if it there isn’t any.

Formal measure: “integrating the multimedia elements into the article”
When the multimedia elements are integrated into the article at the according place, their positive influence is dramatically enhanced. The test persons clicked on more video-clips if their relation to the text was clear. The increased text-reproduction also suggests a direct connection between text-comprehension and the number of the received videos. The qualitative analysis of these reproductions further showed that the multimedia elements were only then recognized by the test persons as meaningful parts of the article and not just as unnecessary accessories, when they were spatially integrated into the text. If that was not the case, it really didn’t matter that their contents perfectly complemented the written text. Their informational surplus value was lost nonetheless. This is substantiated by the fact that the test users explicitly referred to the videos as illustrations to the written text only in the reproductions of versions 3 and 5 but not concerning version 2. Some test users only mentioned in passing the video’s illustrative function, for example: “cinematic procedures like camera work etc. are described and illustrated by examples”(18) , but some referred to the relationships between text and videos in a detailed and enthusiastic fashion:

[The director] attached great importance to the production of an exceptional trilogy. This is demonstrated by the scene clippings, which are very appealing and fascinating, almost a little disturbing. He [the director] says himself, that a special camera work, which seemingly is not used often resp. was “invented” by him, was very important to him.(19)

It can therefore be claimed that opinion building and interpreting information is only then fully possible when the recipients are enabled to substantially relate the various channels of information transfer to each other, like this test person managed to do: “ “The Kiss“ [=the film’s title] is fascinating due to its emotional story and good camera work. Irfan Rehman knows how to match camera work and music.”(21) This example is in so far interesting as it seems at first as if the test person only repeated information and evaluations from the article (“emotional story”, good camera work”). But the article says nothing at all about camera work that matches the music. The test person came to this opinion solely by looking at the video clips in combination with the preceding text, which probably led his attention to the compositional aspects of the film demonstrated in the video-clips.

The test persons found the integration of the multimedia elements into the text very positive and said several times, whereas they explicitly criticized the positioning of the elements in the sidebar. So, if one undertakes the effort to add multimedia to an online-journalistic article, it should happen in a way the users notice those elements and can relate them without difficulty to the written text.

Formal measure: “modularizing the information”

The modularization of the article did not enhance text-reproduction. The re-narrations of the scroll text were on an average even a little longer than those of the hypertext-versions. The reason for this could not be found in this investigation. But the user-survey showed that the users do not at all reject this form of information presentation. To the contrary, they liked the hypertext-versions much better than the scroll-texts, (21) harshly criticized the scroll text, and even proposed themselves to modularize the text. In comparison, the modularization of information was often explicitly praised.

V. Suggestions for a positive development of online-journalistic article design

The preceding investigations lead to the following conclusions for a positive development of online-journalistic article design:

It is necessary, that online-journalism not only focuses on the immediate transfer of short and short-lived information, but that it establishes a less ephemeral standing leg i.e. hypermodal articles. Especially in the arts and entertainment sector there are many topics that do not lose their interest just because they are no longer up-to-date. They stay fascinating because they are multi-faceted. With such articles it would pay off to invest a little more preparation time to produce extensive audio-visual events. I’d like to emphasize it once again that the effort of adding multimedia is in vain if the multimedia elements are not properly coordinated and cannot be related to each other visually. Videos, audios, animations, pictures and graphics – they all have to be emancipated from their role as mere accessories so that they gain more weight in the production of a meaningful whole. To enhance text-comprehension and emotional satisfaction of the users, the possibilities of the Internet for multi-channel information transfer should be used. This is the only way to keep online-journalistic information offers competitive in the long run.

Of course it is extra work to produce the various elements and to make a coherent article out of them, but my user study showed that it is worth it and that the recipients appreciate it enormously. Therefore, it would make sense to use online-journalists not solely for shoveling content from traditional media onto the Internet, but to let them produce some high quality articles once in a while. This would also strengthen the character of an online-medium. The number of hypermodal articles could be successively increased. From my own experience I know that one has not to be a trained camera (wo)man, cutter, light- or audio-engineer to produce audios or videos suitable for the web. Concerning the technical conditions on the user’s side can be said that firstly, they are constantly improving, and second, the application of multimedia might encourage those, who still log on to the Internet via a 56k modem to get a faster connection, thus producing the infrastructure which is claimed lacking by online-journalists today. Besides, it is possible to design an article in such a way that multimedia elements will present an informational surplus value, but that the article can still be understood without them.

For a while, it might still be possible (or even necessary) to ignore the hypermodal possibilities of the Internet. But sooner or later online-journalism has to account for the technical possibilities of the Internet, in order to further fulfill its function to effectively inform, comment and entertain. When that day comes, creativity is called for as well as scientific reflection, and the courage to step by step turn one’s back on the old and familiar and to think of new, innovative forms of journalistic article-design.