華語文寫作測驗信度與評分規準適切性研究－以流利精通級摘要寫作題型為例

The Reliability and Rubric Relevance of the TOCFL Writing Test - Analyzing Band C Summary Writing as an Example

陳柏熹(Po-Hsi CHEN) ； 彭淑惠(Shu-Hui PENG) ； 藍珮君(Pei-Jiun LAN)

《華語文教學研究》 16卷2期 (2019/06) Pp. 75-102

摘要

寫作測驗因題數少且須仰賴人工評分，使其信度研究更形重要，然而過往華語寫作測驗信度相關文獻甚為少見。緣此，本研究針對流利精通級華語文寫作測驗摘要寫作題型的信度與評分規準的適切性進行探討，運用的分析模式包括斯皮爾曼等級相關分析、類推性理論與多元迴歸分析。研究結果顯示：（1）在採取分析式評分方式的情況下，多數評分者所評定出的整體級分與最後成績呈中度或高度正相關，評分者間信度大致良好。（2）受試者的變異成分最高，可佐證其得分能反映寫作能力。（3）文本由3人評閱，有利兼顧評分品質與經濟效益。（4）在11個變項中，複雜句型、組織、詞彙語法、詞語運用、規範性為預測力較為顯著的變項。

關鍵字

信度 ； 流利精通級 ； 華語文寫作測驗 ； 評分規準 ； 摘要寫作

並列摘要

A writing test is a form of language test that has fewer items in each test and relies heavily on subjective rating. These features of the writing test make the issue of reliability rather significant. This research focuses on the task type of Summary Writing used in the TOCFL Writing Test, Band C. By utilizing the analysis methods associated with Spearman's rank correlation coefficient, multiple regression analysis and generalizability theory, this study aims to discuss the reliability of, and the rubrics associated with, this specific task type. The results of this study can be summarized as follows: (1) By adopting an analytic rating method, the individual scores given by the majority of the raters have high-positive or medium-positive correlations with the final score. Inter-rater reliability is moderately high. (2) The variance components of the test-takers are the highest, which indicates that the scores for this task type can clearly reflect individual test-taker's writing ability. (3) Assigning three raters to rate each test essay can assure the quality and efficiency of the rating process. (4) Among the 11 variables which we have analyzed, 5 of them - complex sentence patterns, organization, vocabulary and syntax, vocabulary switching, and response requirements - can predict test-takers' writing ability more significantly.

並列關鍵字

Band C Test ； reliability ； rubrics ； Summary Writing ； TOCFL Writing Test

參考文獻

Alderson, Charles J., Caroline Clapham, and Dianne Wall. 1995. Language Test Construction and Evaluation. Cambridge: Cambridge University Press.

Alkharusi, Hussain. 2012. Generalizability theory: An analysis of variance approach to measurement problems in educational assessment. Journal of Studies in Education 2.1: 184-196.

Bachman, Lyle F. 1990. Fundamental Considerations in Language Testing. Oxford: Oxford University Press.

Brennan, Robert L. 2000. Performance assessments from the perspective of generalizability theory. Applied Psychological Measurement 24.2: 339-353.

Brookes, Arthur, and Peter Grundy. 1998. Beginning to Write. Cambridge: Cambridge University Press.