Something about multimodality. (quoted from wikipedia)

(1) An introduction to M.A.K Halliday---
Michael Alexander Kirkwood Halliday (often M.A.K. Halliday) (born 1925) is an Australian linguist who developed an internationally influential grammar model, the systemic functional grammar (which also goes by the name of systemic functional linguistics [SFL]).
Halliday was born and raised in England. He took a BA Honours degree in Modern Chinese Language and Literature (Mandarin) at the University of London. He then lived for three years in China, where he studied under Luo Changpei(羅常培) at Peking University and under Wang Li(王力) at Lingnan University, before returning to take a PhD in Chinese Linguistics at Cambridge. Having taught Chinese for a number of years, he changed his field of specialisation to linguistics, and developed systemic functional grammar, elaborating on the foundations laid by his British teacher J. R. Firth and a group of European linguists of the early 20th century, the Prague School. His seminal paper on this model was published in 1961. He became the Professor of Linguistics at the University of London in 1965. In 1976 he moved to Australia as Foundation Professor of Linguistics at the University of Sydney, where he remained until he retired. The impact of his work extends beyond linguistics into the study of visual and multimodal communication, and he is considered to have founded the field of social semiotics. He has worked in various regions of language study, both theoretical and applied, and has been especially concerned with applying the understanding of the basic principles of language to the theory and practices of education. He received the status of emeritus professor(榮譽教授) of the University of Sydney and Macquarie University, Sydney, in 1987, and is currently Distinguished Visiting Professor in the Faculty of Education, University of Hong Kong. With his seminal lecture "New Ways of Meaning: the Challenge to Applied Linguistics" held at the AILA conference in Saloniki (1990), he became one of the pioneers of eco-critical discourse analysis (a discipline of ecolinguistics).
(2) Social semiotics and multimodality---
Social semiotics is currently extending this general framework beyond its linguistic origins to account for the growing importance of sound and visual images, and how modes of communication are combined in both traditional and digital media (see, for example, Kress and van Leeuwen, 1996), thus approaching semiotics of culture (Randviir 2004). Theorists such as Gunther Kress and Theo van Leeuwen have built on Halliday's framework by providing new "grammars" for other semiotic modes. Like language, these grammars are seen as socially formed and changeable sets of available "resources" for making meaning, which are also shaped by the semiotic metafunctions originally identified by Halliday. The visual and aural modes have received particular attention. Accounting for multimodality (communication in and across a range of semiotic modes - verbal, visual, and aural) is considered a particularly important ongoing project, given the importance of the visual mode in contemporary communication.