設計並建置一個用於高等教育環境中的學習成效評量導引系統

Designing and Developing an Approach for Improving the Consistency of Learning Assurance Evaluation in Higher Education

張惠茹(Hui-ju Chang)

指導教授 ： 陳仲儼

國立中央大學/管理學院/資訊管理學系/碩士(2012年)

摘要

對於AACSB認證中的學習目標而言，商管高教組織通常會使用學生的成績來判斷課程來確保學生的學習成效(Assurance of Learning)是否達成，而學生的成績則仰賴評量人員對於學生的表現進行評量才能得到，也就是說學生的分數品質對於學習成效的判定具有重大且關鍵的影響。但是，當課程的測驗題型採用建構反應試題(Constructed-response item)這種題型時，評量人員須依靠本身經驗及專業能力進行評量，由於評量者效應的存在，特別是共構課程有不同科系教師執教的情況下，評量的一致性便成為一個重要的課題。因此，本研究旨在透過教育訓練的角度，提出一個評量導引架構：LBR評量訓練循環模式(Learning , Behavior and Reaction Circular Training and Assessment Model)。同時透過此架構的三個分數品質指標(嚴格度、集中度、準確度)，協助組織能從不同角度掌握評量分數的品質。在實務應用上，本研究依據所提出的架構，實作出一個應用系統以幫助組織訓練並導引評量人員，使其了解組織所制定之評量準則後在進行評量時能符合組織的標準。本研究亦將LBR評量訓練循環模式應用於中央大學100學年度企業概論期中考的評量中，來展示系統功能與使用情形。最後，針對現有的運用與研究限制進行討論，並提出未來研究的可能方向。

關鍵字

高等教育 ； 商管教育品質認證 ； 評量者一致性 ； 評量者訓練 ； LBR評量訓練循環模式

並列摘要

As for the assurance of learning of AACSB accreditation, the higher-educational organizations of business and management usually use the students’ scores of classes to determine whether they achieve the standards or not. Organizations can get the results after raters evaluate the performances of students. That is, the quality of evaluation of students plays an important role when judging the assurance of learning. However, raters have to score the constructed-response-item tests by their own experiences and professional skills. Owing to the existence of rater effect, the evaluation consistency becomes an important issue when the shared courses taught by different professors in different departments. Therefore, this research aims to propose a rating guidance framework, named LBR (Learning, Behavior and Reaction) circular training and assessment model. Through the three quality indicators (Harshness, Accuracy, Centrality), LBR can assist organizations to handle the quality of scores from different dimensions. This research also implements a system as a real use of LBR to train and guild the raters. After training by LBR, the result of raters’ rating will much more meet the goals of the organization. Finally, discussion and suggestions are presented for the future research.

並列關鍵字

Quality accreditation of Business education ； LBR circular training and assessment model ； Raters training ； Evaluation consistency ； Higher education

參考文獻

[3] 周行一、張逸民、畢文玲（2008）,國際商管教育認證對臺灣商管教育的影響─從政大商管學院申請AACSB認證經驗談起,評鑑雙月刊，11，33-37頁。

[1] 杜佳靜(2009),高等商管教育的新刺激－AACSB International,品質月刊,45卷12期,28-31頁。

[6] Bernardin, H. J. and Pence, E. C.(1980), Effects of rater training. New response sets and decreasing accuracy, Journal of Applied Psychology,65,pp.60-66

[8] Bonk, W. J. and Ockey, G. J. (2003), A many-facet Rasch analysis of the second language group oral discussion. ,Language Testing, 20(1), pp.89-110.

[10] Chen, C.Y. and Chen, P.C.(2009), A holistic approach to managing software change impact, Journal of Systems and Software, 82(12),pp.2051~2067