多模態，隱喻，與詮釋的交會：以政治漫畫與藝術歌曲為佐證

林盈妤

doi:10.6342/NTU.2015.01717

3.238.70.175，Hello！

Close
Search Quick Search Advanced Search Publication Search	Personal Service Member Login Searching History Use Bonus Card SYMSKAN
Browse Browse by University of Graduate Thesis Browse by Pinyin of Graduate Thesis Browse by Conference Proceedings Topic Browse by Conference Proceedings Publisher Browse by Pinyin of Conference Proceedings Browse by eBook Category Browse by eBook Topic Browse by eBook Publisher
Customer Services Authorize Airiti Recommend a Journal

Search Symbol (Half-width)	Description of Search Symbols
Space	"AND" indicates the intertwining of key terms used in a search
Double Quotation Marks ("") ( " " )	Double quotation marks indicate the beginning and end of a phrase, and the search will only include terms that appear in the same order of those within the quotations. Example: "image process" ： " image process "
?	Indicates a variable letter. Entering two ? will indicate two variable letters, and so on. Example: "Appl?", search results will yield apple, apply… e , appl y … ( (often used to English word searches) )
*	Indicates an unlimited number of variable letters to follow, from 1~n. Example: Enter "appl*", search results will yield apple, apples, apply, applied, application…(often used in English word searches) e , appl es , appl y , appl ied , appl ication … ( (often used to English word searches) )
AND、OR、NOT	Boolean logic combinations of key words is a skill used to expand or refine search parameters. (1) AND (1) AND: Refines search parameters (2) OR (2) OR: Expands search parameters (3) NOT: Excludes irrelevant parameters

DOI stands for Digital Object Identifier （ D igital O bject I dentifier ） ,
and is the unique identifier for objects on the internet. It can be used to create persistent link and to cite articles.

Using DOI as a persistent link

To create a persistent link, add「http://dx.doi.org/」「 http://dx.doi.org/ 」 before a DOI.
For instance, if the DOI of an article is 10.5297/ser.1201.002 , you can link persistently to the article by entering the following link in your browser: http://dx.doi.org/ 10.5297/ser.1201.002 。
The DOI link will always direct you to the most updated article page no matter how the publisher changes the document's position, avoiding errors when engaging in important research.

Cite a document with DOI

When citing references, you should also cite the DOI if the article has one. If your citation guideline does not include DOIs, you may cite the DOI link.

DOIs allow accurate citations, improve academic contents connections, and allow users to gain better experience across different platforms. Currently, there are more than 70 million DOIs registered for academic contents. If you want to understand more about DOI, please visit airiti DOI Registration （ doi.airiti.com ）。

Data Source

Humanities > Linguistics
College of Liberal Arts > Graduate Institute of Linguistics

Bibliography Management Tool

Related Links

Report an Issue

Purchase Single Articles

Download PDF

多模態，隱喻，與詮釋的交會：以政治漫畫與藝術歌曲為佐證

The Encounter among Multimodality, Metaphor, and Interpretation: Evidence from Political Cartoons and Poetry Music

林盈妤 , Ph.D　　Advisor：江文瑜　　

英文 DOI： 10.6342/NTU.2015.01717

多模態融合聚變理論；多模態隱喻/轉喻；閱聽者反應；多模態提示；多模態體裁； Multimodal Fusion Model ； multimodal metaphor/metonymy ； audience response ； multimodal cues ； multimodal genre

Abstract │ Reference (130) │ Altmetrics

Abstract 〈TOP〉

近年來認知語言學領域中的隱喻相關研究受到高度重視，語言中的隱喻如何型塑並影響我們的思維與行動，在認知科學領域，心理學，數學，藝術，哲學等領域均獲得廣泛的討論與研究，非語言隱喻也隨之逐漸在圖像領域及多模態/多媒體的研究中受到注意。然而，過去的研究大多著重於語言與其他媒介之間一對一的對應關係以及認知，然而本研究認為多模態與語言之間的關係不只有一對一的關係，應可視為多重組合關係的連續體(continuum)，豐富且複雜，因此本研究提出創新的互動式詮釋理論：多模態融合聚變理論(multimodal fusion model)，提供多模態分析以解釋語言與其他媒介之間一對多的對應關係，探討特定主題及抽象概念如何再現於多模態體裁(multimodal genre)中，以政治漫畫與藝術歌曲為例證，深入了解 (1)『美國牛肉進口』這個熱門的政治議題，如何透過眾多的政治漫畫呈現於不同立場的報紙中？ (2) 經典詩作『偶然』又如何透過樂曲中的聽覺與視覺，再現於四個版本的藝術歌曲中？ 本論文進一步結合閱聽者訪談結果，審視閱聽者如何詮釋並理解多模態隱喻於特殊體裁。本文進一步提出顯著的多模態特性與多模態視覺/聽覺/語言提示(visual/aural/verbal cues) 均可協助促進多模態隱喻的運作與詮釋。
    本研究以多模態融合聚變理論為基本架構，建構了美牛進口相關政治漫畫的多模態語料庫，歷時收集涵蓋台灣兩大報『自由時報』與『聯合報』共56篇政治漫畫。本文主張多模態融合聚變結合視覺，語言，與概念三大模式，衍生自轉喻與隱喻(metonymic-metaphoric)共同組成的兩大網絡：相關化轉喻網路(related metonymic network) 和多元化隱喻網路 (diversified metaphoric network)。本研究證明多模態融合聚變是政治漫畫中重要且重複運用的表現技巧，具有將複雜抽象的政治概念有效率地濃縮封裝(encapsulating)的認知功能，並產生諷刺與幽默的效果。本分析進一步強調轉喻的重要性，證明轉喻與隱喻彼此密切交織(interwoven)於多模態融合聚變中，蘊藏於概念情境(conceptual scenarios) 如政治是遊戲(POLITICS IS WAR)與政治是戰爭(POLITICS IS WAR)的隱喻對應範疇內。本研究發現雖然兩報的批判性訊息與相異的政治立場均浮現自多模態融合聚變理論，卻分別透過顯著的視覺特色與語言情境，更加強化與對比。
    本研究以四首為徐志摩詩作『偶然』譜寫的藝術歌曲為例證，提出以隱喻為基礎的譜面分析，進而發現此體裁中特殊的現象：『隱喻之鏈』(chain of metaphor) ，即歌曲中的多模態隱喻彼此可前後呼應，互相串聯於整首歌曲中，以維持整體的連貫性。另外，本研究發現藝術歌曲中的多模態隱喻可透過轉喻的方式再現(metonymic representation) 於歌曲中（如：模擬雲的視覺形狀於譜面或模擬海浪的節奏於伴奏中）。作者更進一步提出藝術歌曲中的多模態隱喻兩大面向：具體形象(concrete image)與抽象隱喻延伸(abstract metaphorical extension)，透過分析提出相對應的音樂技巧以及具體抽象光譜(concrete-abstract continuum)為四首歌曲分類。本文並從身體經驗(embodied)為基礎的概念隱喻出發，發掘抽象概念如空間關係和情緒均可對應於音樂中，如透過音量/音域高低，音色明暗等音樂特性呈現。
    本研究進一步整合閱聽者反應的量化與質化分析，從閱聽者訪談的研究結果中發現，閱聽者對於政治漫畫的高度理解，來自有效率的人物辨識，背景理解與多模態之間的有效且互動式的結合，而閱聽者對於藝術歌曲的高度理解則來自能分辨藝術歌曲中的具體形象與抽象意涵的詮釋，詮釋過程中所建立的對應關係與閱聽者對於藝術歌曲的喜好度具有正面相關，不同版本的喜好度也間接反應了閱聽者如何感知及詮釋藝術歌曲中語言與聽覺的融合下所產生的音樂效應。
     本論文從分析者與閱聽者的視角，探究視覺，聽覺，與概念如何互動,整合,並透過多模態隱喻詮釋，本研究希望拓展以多模態隱喻為主題的跨領域研究，針對特定的多模態體裁深入分析，並結合認知與語用學的觀點，進一步理解多模態體裁中的詮釋與審美經驗歷程。

Parallel Abstract 〈TOP〉

In the past few decades, studies on how verbal metaphors shape our thoughts have been widely discussed in the filed of cognitive science, cognitive linguistics, psychology, mathematics, art, and philosophy, etc. Under this view, “non-verbal” metaphors have also received considerable attention, while previous studies mainly focused on the mental processing and one-on-one correspondence between language and other mode, especially the pictorial mode. However, this study believes the relations between multimodalities can be viewed as a complex continuum of various kinds of combinations. Therefore, this study proposes an alternative interactive interpretation model, the Multimodal Fusion Model, to account for the one-on-many relationship between language and multimodality (visual and aural modes) and to explore how certain topic/issue has been represented in various forms in distinct genre. Based on the evidence from the multimodal genre of political cartoons and poetry music, we examined (1) how the hotly debated topic, the U.S. beef import issue, has been represented in various political cartoons in different newspapers and (2) how a classic poem (i.e. Serendipity) has been conceptualized in four versions of songs. We further incorporate the audience response analysis from the interview surveys to examine how the audience actually interprets the multimodal genre. Our findings also show how prominent multimodal features and multimodal cues (visual/aural/verbal cues) help facilitate the interpretation process of multimodal metaphor in specific genre.
    This study constructs a multimodal corpus involving 56 political cartoons with regard to U.S. beef import issues as reported in two dominant Taiwanese newspapers, the Liberty Times and United Daily News. Our corpus shows that multimodal fusion combining the visual mode, verbal mode, and the conceptual level evolves from two metonymic-metaphoric networks, i.e., related metonymic network and diversified metaphoric network. The result demonstrates that multimodal fusion is a significant and recurrent representation technique in the genre of political cartoon and has the cognitive function of encapsulating the abstract complex political debates efficiently with irony and humorous effect. In addition, this study emphasizes the important role of metonymy and demonstrates how metonymies and metaphors are interwoven in the process of multimodal fusion, which underlies the metaphorical mappings of conceptual scenarios related to “POLITICS IS GAME” and “POLITICS IS WAR”. Although the critical messages and distinct political stances in two newspapers both emerged through multimodal fusion, they are highlighted and contrasted through prominent visual features and verbal context in political cartoons. 
    Based on the evidence from four versions of poetry music composed for the poem “Serendipity” by Hsu Chih-Mo, this study provides a metaphor-based musical analysis under the scope of multimodal fusion model. We discovered the “chain of metaphors” as an interactive and echoing relationship between metaphors that creates a holistic representation of multimodal metaphor combining the verbal and aural mode in the genre of poetry music. Furthermore, the results show the metonymic representations of metaphor in poetry music have been realized through the “pictorial effect” of partial visualization of cloud image on the score representing CLOUD metaphor and aural simulation of wave rhythm representing SEA metaphor, etc. Through cross-comparison analysis, we propose two major aspects of metaphor, the “concrete image” and “abstract metaphorical extension”, have been highlighted differently through musical techniques and features such as dynamics, pitch, and timbre, etc. in four versions of songs. Moreover, this study provides a concrete-abstract continuum as a categorization of composing patterns and musical effects. Viewing from the embodied perspective, our findings also show that the concrete relationship of space and abstract attributes of emotion can be mapped onto the representations of prominent musical features in poetry music.
   This study further incorporates the quantitative and qualitative analysis from audience response survey. The results show that higher level of audience interpretation in political cartoons relies on the effective visual recognition of caricatures, comprehension of verbal and contextual information, and interactive connection of the verbal mode, visual mode, and conceptual level. On the other hand, the higher level of audience interpretation in poetry music is related to the distinction between the concrete image and abstract metaphorical extension. The results also suggest that the audience’s preference is relevant to the correspondence between the verbal and aural mode, which implicitly reflects how the audience “feels” about the musical effect and aesthetic functions created from multimodal fusion in different versions of songs. 
    From the perspectives of researcher and audience, this study investigates how verbal mode, visual mode, aural mode, and conceptual level are interacted, integrated, and interpreted through multimodal fusion. We hope to shed light on the interdisciplinary studies on multimodality and the interface between the cognitive mechanism and pragmatic multimodal use of language, picture, music, and the aesthetic experience of the multimodality appreciation process.

Reference ( 130 ) 〈TOP〉

Agawu, K. (1992). Theory and practice in the analysis of the nineteenth-century 'Lied'. Music Analysis, 11 (1), 3-36.
連結：
Atrey, P., Hossain, M., El Saddik, A., & Kankanhalli, M. (2010). Multimodal fusion for multimedia analysis: a survey. Multimedia Systems, 16, 345-379.
連結：
Baldry, A. and Thibault, P.J. (2006). Multimodal transcription and text analysis: A multimedia toolkit and coursebook. London: Equinox.
連結：
Barcelona, A. (2002). Clarifying and applying the notions of metaphor and metonymy within cognitive linguistics: an update. In R. Dirven & R. Pörings (Eds.), Metaphor and metonymy in comparison and contrast (pp. 207-278). Berlin: Mouton de Gruyter.
連結：
Benoit, W., & Klyukovski, A., McHale, J., & Airne, D. (2001). A fantasy theme analysis of political cartoons on the Clinton-Lewinsky-Starr affair. Critical Studies in Media Communication, 18, 377–394.
連結：

Altmetrics

〈TOP〉

E-mail ：

When an article is available to download, a notice will be sent to your mailbox address.

E-mail ：