創意產品共識評量之信、效度析論

Consensual Assessment for Creative Products: A Review of Reliability and Validity

劉昆夏(Kun-Shia Liu) ； 鄭英耀(Ying-Yao Cheng) ； 王文中(Wen-Chung Wang)

《測驗學刊》 57卷1期 (2010/03) Pp. 59-84

https://doi.org/10.7108/PT.201003.0059

摘要

本研究經由創意產品共識評量之相關文獻的回顧與評析，歸納出當前創意產品共識評量所存在的信、效度問題，包括：(1)使用Cronbach's係數進行評分者一致性分析，導致評分者信度會因創作者能力分布以及評分者人數多寡，而呈現不穩定的現象；(2)僅以領域專家在「整體創造力」評量指標上的共識程度，做為效度之依據，缺乏創造力概念的聚斂效度與區辨效度之檢核；(3)忽略主觀評量過程中評分者效應所造成的評量誤差，導致相同專業背景的評分者，在不同的研究中所呈現的評分者一致性存有相當的差異。最後，本研究以試題反應理論的Rasch多相模式觀點，針對上述的問題，提出未來研究的可行方向，期能改善創意產品共識評量技術與提升評量的品質。

關鍵字

Rasch多相模式 ； 共識評量技術 ； 創造力 ； 評分者效果

並列摘要

The study reviews the literature on consensual assessment of creative products and generalized problems of reliability and validity in current consensual assessment of creative products. Major issues are (a) problems of unstable rater reliability caused by creators' ability distribution and rater's number when Cronbach's was used to analyze rater reliability, (b) problems of lacking examination of convergent validity and discriminant validity in creativity concepts when most researches only used domain experts' consensual level in ”overall creativity” as the basis of validity, and (c) problems of ignoring measurement errors caused by raters' effects during subjective assessing process, resulting in significant differences of rater consistency for raters of the same background in different research findings. Finally, to resolve these problems, the study proposes a feasible direction of applying the many-facet Rasch model for future studies in order to improve consensual assessment technique of creative products and elevating the quality of assessment.

並列關鍵字

consensual assessment technique ； creativity ； many-facet Rasch model ； rater effect

參考文獻

Amabile, T. M.(1979).Effects of external evaluation on artistic creativity.Journal of Personality and Social Psychology.37,221-233.

Amabile, T. M.(1982).Social psychology of creativity: A consensual assessment technique.Journal of Personality and Social Psychology.43,997-1013.

Amabile, T. M.(1983).Social psychology of creativity.New York:Spring-Verlag.

Amabile, T. M.(1996).Creativity in context.Boulder, CC:Westview Press.

Amabile, T. M.,Goldfarb, P.,Brackfield, S. C.(1990).Social influences on creativity: Evaluation, coaction, and surveillance.Creativity Research Journal.3,6-21.