The Validity and Reliability of Student Ratings: Comparison Between Paper-pencil and Online Survey

“學生評鑑教師教學”信效度之探討：紙筆和網路調查之比較

張德勝(Te-Sheng Chang)

《中華心理學刊》 47卷2期 (2005/06) Pp. 113-125

https://doi.org/10.6129/CJP.2005.4702.02

摘要

本研究主要的目的在於比較紙筆調查和網路調查方式對於學生評鑑教師教學信、效度的影響。研究對象為九十學年度第一學期國立花蓮師範學院大學部所開設的624門課，包含一年級198(31.73%)門、二年級161(25.80%)、三年級146(23.40%)以及四年級119(19.07%)門。研究工具為「國立花蓮師範學院教學意見反映調查表」(student Ratings of Instruction form)，共包含四個層面：「準備與計畫」、「教材與內容」、「教法與技巧」、「作業與評鑑」。研究結果顯示，紙筆調查而言，學生評鑑教師教學每一個層面的的內部一致性alpha係數，分別介於.904和.965之間；就網路調查而言，則介於.919和.960之間。就構念效度而言，在紙筆方面，每一個向度之因素解釋量介於13.4%和32.8%之間，每一向度中小題之因素負荷量介於.332和.826之間。而網路方面，則每一個向度的因素解釋量介於16.6%和27.5%之間，每一層面的題目因素負荷量介於.422和.848之間。另外從多項特質多重方法來看，則信度係數高於軸合效度係數，而軸合效度係數也高於區別效度係數。由研究結果得知，無論紙筆調查和網路調查的方式，其所產生的學生評鑑教師教學的信、效度都頗高，而且不會有影響。

關鍵字

學生評鑑教師教學 ； 大學教師評鑑 ； 紙筆調查 ； 網路調查 ； 信度 ； 效度

並列摘要

The purpose of this study was to compare the difference between the paper administration procedure and the online administration procedure regarding the reliability and validity of student ratings of instruction. The instrument was a Student Ratings of Instruction (SRI) form developed by the National Hualien Teachers College, 1995. Using a repeated measure design, the SRI was administrated to 624 undergraduate courses, 198 (31.73%) freshman, 161 (25.80%) sophomore, 146 (23.40%) junior, and 119 (19.07%) senior courses, fall 2001. Results indicate that internal consistency reliabilities and factor loadings for the online method are similar to those obtained for the paper method. Additionally, high between-method correlations (convergent validity coefficients) are obtained for subjects who completed both administration formats, indicating the rank order of courses is basically unchanged across the methods. The paper scores are significantly higher than the online scores on all of the evaluation items, factors, and the total scores. This finding supports the position that the ratings scores are influenced by survey method but the validities and reliabilities are not.

並列關鍵字

Student Ratings of Instruction ； Faculty Evaluation ； Paper Survey ； Online Survey ； Internal Consistency Reliability ； Construct Validity

參考文獻

Sproull, L.,Kiesler, S.(1991).Connections: New ways of working in the networked organization.

Chang, T.(2001).The comparison between web-based and paper-and-pencil surveys of student ratings of instruction.

Cashin, W. E.(1988).Student rating of teaching: A summary of the research.

Ghiselli, E. E.(1964).Theory of psychological measurement.

Marsh, H. W.,Dunkin, M. J.(1992).Students` evaluations of university teaching: A multidimensional perspective.Higher education: Handbook of theory and research.143,233.