Learning From Multimodal Texts
Willy A Renandya, 14 August 2022
Texts that we produce and consume these days are increasingly multimodal in nature. It is thus only natural that multimodal texts find their way in the language classroom as well as other non-language classrooms (e.g., science and social studies classrooms).
Although traditional texts are still being used in some countries, it is just a matter of time that they will give way to the digitally enhanced multimodal texts.
Multimodal texts are considered more complex and richer than traditional texts, as the former carry textual as well as non-textual information (e.g., still and moving images, audio, different font types and sizes etc.).
With a traditional text, readers only need to focus on the verbal information by skimming and scanning the words in the text.
With a multimodal text, readers need to distribute their attention to the various elements present in the text, i.e., the words, the accompanying images, the colour, font type and size and other visual cues, which the writer used to communicate their message.
It is important to note however while multimodal texts are more complex and require a different kind of processing, it does not necessarily mean that they are more difficult to comprehend than traditional texts.
Indeed, my own perspective is that multimodal texts should be more appealing and at the same time easier to comprehend.
There are of course multimodal texts that are poorly designed, i.e., the different visual elements in the text do not seem to serve any obvious purposes. The writers simply use images and other multimodal elements for decorative purposes but without much reader awareness.
Thus, instead of playing a facilitative role that support a greater reader experience with the text, the multimodal elements distract readers’ attention from the main message that the writer tried to communicate to their readers. When this happens, readers may find it frustratingly difficult to navigate their way into the multimodal text.
So just like any other things in life, there are multimodal texts that are well-designed and those that are poorly designed. Examples are easy to find.
If you have recently attended a workshop or conference presentations, you may have noticed that the the powerpoint slides used by the speakers vary a great deal.
Some may be appealing visually and easy to follow and comprehend. Others however may be too wordy and cluttered with poorly chosen images that take your attention away from the key message of the presentations.
From a language acquisition perspective, a pedagogical text (multimodal or not) should be considerate (i.e., linguistically and visually accessible) and compatible (i.e., relatable).
If the text contains lots of new vocabulary, complex language features and unfamiliar images, for instance, students won’t be able to learn much from the text. In fact, they may stop reading or viewing the text due to an information overload.
Similarly, if the text contains too much new and unfamiliar verbal as well as non-verbal information, students won’t be able to make relevant and meaningful connections to the contents. In other words, their comprehension may be minimal and/or superficial.
The big question for us then is this: how do we select multimodal texts for language teaching/learning purposes? Here are some guiding questions.
Does the text contain relevant, useful and interesting information?
This is a critically important question because students tend to be more engaged when they find the text content appealing. When the content is neither interesting nor relevant, students are less likely to engage in a deeper level of processing of the text. Williams (1986) reminded us that when the text is not interesting, nothing much is possible in the language classroom.
Is the language rich and comprehensible?
We know that the language element of the text is a great source of language learning. When the language is easy to understand and contains lots of instances of authentic lexical and grammatical structures, students can pick them up quite naturally. Exposure to rich and comprehensible language enables students to build implicit knowledge of the language.
Are the visual and other non-verbal elements distracting?
There has to be a balance between textual and non-textual elements. Too many of the latter will only disrupt students’ comprehension process as they have to split their attention to the non-essential elements of the text. Comprehension breakdowns are uncommon when the verbal and non-verbal elements are competing for the readers’ limited attention resources.
Do the visual and other non-verbal elements increase students’ language learning experience?
When visual and other non-verbal elements are nicely designed, students can get a lot of pleasure viewing the multimodal text. This positive feeling can often lead to students wanting to do more reading and viewing both in and out of the classroom. In other words, positive feelings can boost students’ motivation.
Does the multimodal text contain comprehension-enhancing non-verbal elements?
Instead of just serving as decorative purposes, visual elements can be used to support students’ comprehension. This is particularly true when the multimodal text contains new and abstract ideas. Visuals and other non-verbal elements can be added to help represent unfamiliar ideas more efficiently. Like they say, a picture is worth a thousand words. So when properly used, visual elements can greatly support students’ comprehension.
Does the multimodal text contain memory-enhancing non-verbal elements?
Comprehension is important but remembering the text contents is equally important. This is especially so when the text contains important concepts that students need to remember and make use of in their subsequent lessons. Thus, multimodal texts than contain memory-enhancing visuals can boost students’ memory of the taught contents.
Does the text contain verbal and non-verbal elements that help students notice important language features (e.g., spelling, grammatical structure)?
Comprehension is a necessary condition for language acquisition. Students need to immerse themselves in rich and comprehensible language to build a strong linguistic foundation. But comprehension alone is not enough. Students should also notice important language elements, especially those difficult-to-notice language features (e.g., non-count nouns and tense markers). The noticing of these features is one of the key processes that can help students become a more competent users of the language.
Does the text content spark students’ interest to find related information from the web and other learning channels (Youtube videos, Podcast etc)?
As mentioned earlier, students need to experience lots of language in their language acquisition process. While the texts we use in the classroom play an important role, the quantity is too little to trigger the acquisition process. So texts that we use in the classroom should ideally ignite students’ intrinsic desire to do self-selected, independent reading and viewing beyond the classroom. Repeated observations show that students who engage in extramural learning (i.e., independent reading and viewing) become more fluent and confident users of the language.
Conclusion
The questions above serve as general guideline for choosing multimodal texts. Admittedly, it is probably difficult to find texts that meet all of the criteria above. But if we want our students to get the most of a multimodal text, we need to ensure that the text satisfies some of the most important requirements above. The first three (i.e., the contents are appealing, the language is comprehensible and the visual elements are not disorganized) are the most fundamental criteria for selecting multimodal texts. The rest of the criteria are good to have as they can further increase the quality and quantity of learning from multimodal texts.
More reading
One Reply to “Learning from Multimodal Texts”