Resources

Unit 2: Why engage with multimodality?

Chapter overview

Learning objectives

Study unit 2 will help you to:

  • Engage critically with the notion of multimodality
  • Make a case for engaging with multimodality

Overview of Chapter 2

Topics

  • The rationale for multimodality
  • The claim that language is the most important of all modes
  • The claim that language can be studied in isolation

Summary

In Chapter 2 we engaged with, and challenged assumptions about language widely held among those studying language and the general public. We countered the notion that language is the most resourceful, important and widely used of all modes using three arguments:

  • The status of language varies across communities and contexts of use. It is not possible to make general claims about what ‘people’ do with language. Rather, we suggested the need to ask, for whom is language the most resourceful, widely used and important mode of all, and in what contexts of use? We demonstrated how a multimodal frame puts the significance of individual modes, including language, in perspective.
  • Many ‘linguistic’ principles are actually general semiotic principles. We presented examples showing that many of the principles and social structures described by those studying language can be realized in a variety of different ways. These principles and structures are common semiotic principles.
  • Each mode offers distinct possibilities and limitations. We pointed out some of the limitations of language, and possibilities of other modes not shared with language, and proposed that a mode always offers distinct yet always limited potentialities.

We concluded that a multimodal perspective enables you to differentiate between the general and particular in language, and to recognize both what language has in common with other modes and how it is distinctly different.

We went on to problematize the notion that language can be studied in isolation by showing that the meaningful wholes that people produce are almost always multimodal. The elements in a multimodal whole are mutually modifying, making it problematic to attend to language alone.

Study questions

Read Chapter 2 and think about how you would answer the questions below. Make notes of your responses and review them when you have completed the study guide.

  1. What does it mean to consider general semiotic principles? Or: what is the difference between a ‘linguistic’ and a ‘semiotic’ principle/feature?
  2. Watch a ‘tutorial’ on YouTube – what can you say about the status of language in these videos relative to other modes?
  3. What would attention to language alone fail to capture when analysing an interview?
  4. Why is it important to take account of the distinct possibilities and limitations of each mode in a multimodal account?

Exercises

Exercise 2.1: Why look beyond natural language?

  1. Read the below statement by American linguist Jerrold Katz:

    ’I take it as some empirical evidence for the claim that natural languages are effable that speakers almost always find appropriate sentences to express their thoughts, that difficulties in thinking of a sentence are invariably regarded as a failing on the part of the speaker rather than the language, and that there is nothing to indicate that there is any type of information that cannot be communicated by the sentences of a natural language.’ (Katz 1972: 19).

  2. The Oxford Dictionary defines ‘effable’, from Latin for ‘utter’, as ‘able to be described in words’. How would you respond to the principle he called ‘effability’? Can you think of anything that is not ‘effable’?

Tip: Being clear as to what is not ‘effable’ will help to strengthen your rationale for doing a multimodal study.

Exercise 2.2: Evaluating the place of language

  1. Select a multimodal artefact or an everyday context where you can observe people interacting (e.g. a café or a rail station ticket office).
  2. Use the questions below to reflect on the status of language in that particular context of use:
    • For whom is language the most resourceful?
    • In what context is it used?
    • What other modes feature?
    • What is the status of language relative to the other modes in use?
    • What does understanding language as only one part of an ‘ensemble’ reveal?

Tip: Being able to state clearly what a multimodal approach offers in comparison with a solely linguistic one will help to strengthen the rationale for doing a multimodal study.

Exercise 2.3 Critique the rationale for multimodality in a published paper

  1. Search online for some recent publications that have ‘multimodal’ or ‘multimodality’ in the title that relate to your area of study.
  2. Identify a research paper that represents one of the multimodal approaches discussed in this book.
  3. Read the paper and review the case that is made for the use of a multimodal approach. Highlight all elements of the paper that contribute to the rationale.
  4. Address the following questions:
    • What is the main argument made in the paper to support the use of a multimodal approach?
    • What criticisms would you make of the arguments presented?
    • How could the paper’s rationale for a multimodal approach be strengthened?

Exercise 2.4: Make a case for your multimodal study

  1. Imagine that you need to convince someone, maybe a sceptical supervisor, colleague or journal editor of the need to engage with multimodality in your study. Reflect on the following statements:
    • The status of language varies across communities and contexts of use.
    • Many ‘linguistic’ principles are actually general semiotic principles.
    • Each mode offers distinct possibilities and limitations
    • Language cannot be studied in isolation from other modes
  2. Use your responses to these statements to build your case.
  3. Try your argument out on a few colleagues. Listen to their responses, and use their questions, criticisms, and counter-arguments to refine and strengthen your case.

Suggested resources

Further reading

Kress, G. & van Leeuwen, T. (2001). Multimodal Discourse. The Modes and Media of Communication. London; New York: Arnold; Oxford University Press (especially Chapter 1).

Online resources

Interview with Gunther Kress: Why adopt a multimodal approach?

Reference

Katz, J. J. (1972). Semantic Theory. New York: Harper & Row.