Title

網路化檔案評量環境下教學者評、學生自評與同儕互評之信效度比較

Translated Titles

Comparisons of Reliability and Validity among Instructor Assessment, Student Self-Assessment and Peer-Assessment under Web-Based Portfolio Assessment Environment

Authors

張基成(Chi-Cheng Chang);吳明芳(Ming-Fun Wu)

Key Words

網路化學習歷程檔案 ; 網路化檔案評量 ; 自評 ; 同儕互評 ; Web-based portfolio ; Web-based portfolio assessment ; Self-assessment ; Peer-assessment

PublicationName

教育資料與圖書館學

Volume or Term/Year and Month of Publication

49卷1期(2011 / 09 / 01)

Page #

135 - 170

Content Language

繁體中文

Chinese Abstract

本研究比較網路化檔案評量環境下教學者評、學生自評與同儕互評的信效度。研究樣本為某高職二年級修習「計算機應用」課程的72位學生。學生使用網路化檔案評量系統進行檔案的製作、觀摩、自評與同儕互評,教師透過系統檢視學生的檔案並評量其學習表現。在三種評量結果之間差異上,教學者評、學生自評、同儕互評三種評量結果有顯著差異,其中教學者評量最為嚴格、其次為學生自評,同儕互評最為寬鬆。在三種評量結果之間一致性上,僅自評與教師評之間具一致性。自評與同儕互評之間、同儕互評與教學者評之間,都不具一致性。在三種評量結果與測驗成績一致性上,教學者評分結果與測驗成績具高度一致性,學生自評與測驗成績亦具高度一致性;而同儕互評結果與測驗成績一致性低。

English Abstract

This study compares the reliable and validity among instructor assessment, student self-assessment and peer-assessment under the Web-based portfolio assessment environment. The research samples consist of 72 students who take ”Computer Application” course of the second grade in some vocational high school. The students use the Web-based portfolio assessment system for creating their portfolio, emulating peers' portfolio, and performing self-assessment and peer-assessment. The teacher uses the system for reviewing and assessing students' learning performances. In regard to the difference among the three assessment approaches, there exists a significant difference among the three approaches. Instructor assessment is most rigorous, while peer-assessment is most slack. In regard to the correlation among the three assessment approaches, there exists a significant consistency between self-assessment and instructor assessment. There exist inconsistencies between self-assessment and peer-assessment as well as peer-assessment and instructor assessment. In regard to the correlation among the three assessment results and test score, there exist a high consistency between instructor assessment and test score. As well, there exist a high consistency between self-assessment and test score. However, the consistency between peer-assessment and test score is inadequate.

Topic Category 人文學 > 圖書資訊學
Reference
  1. High Plains Regional Technology in Education Consortium (2005). Educational technology MA online program rubric: EPortfolio. Retrieved February 22, 2010, from http://www.msarreola.com/Master%20Canvas/Assessment%20&%20Evaluation_files/eport_rubric.pdf
  2. Milford School District, (2009). Information, communication, technology (ICT) portfolio assessment. Retrieved February 22, 2010, from http://www.milfordschools.net/mms/documents/ePortfolio%20Rubric%202009.pdf
  3. 鄒慧英[Tsou, Hui-Ying](2000)。檔案於國語科教學評量之應用[Dangan yu guoyuke jiaoxue pingliang zhi yingyong]。上網日期:2009年8月7日[Retrieved August 7, 2009],檢自[from]:http://www.nknu.edu.tw/~edu/item/item4-article.file/item4-article19.htm
  4. Anderson, L.,Krathwohl, D.(2001).A taxonomy for learning teaching and assessing: A revision of Bloom's taxonomy of educational objectives.New York:Wesley Longman.
  5. Ballantyne, R.,Hughes, K.,Mylonas, A.(2002).Developing procedures for implementing peer assessment in large classes using an action research process.Assessment & Evaluation in Higher Education,27(5),427-441.
  6. Barrett, H.(2006).Using electronic portfolios for classroom assessment.Connected Newsletter,13(2),4-6.
  7. Bloxham, S.,West, A.(2004).Understanding the rules of the game: Marking peer assessment as a medium for developing students' conceptions of assessment.Assessment & Evaluation in Higher Education,29(6),721-733.
  8. Burch, C. B.(1999).Inside the portfolio experience: The student's perspective.English Education,32(1),34-49.
  9. Chang, C. C.(2008).Enhancing self-perceived effects using web-based portfolio assessment.Computers in Human Behavior,24(3),1753-1771.
  10. Chang, C. C.,Tseng, K. H.(2009).Use and performances of web-based portfolio assessment.British Journal of Educational Technology,40(2),358-370.
  11. Chang, C. C.,Tseng, K. H.(2009).Using a web-based portfolio assessment system to elevate project-based learning performances.Interactive Learning Environments,16(2),25-37.
  12. Chang, C. C.,Tseng, K. H.,Yueh, H. P.,Lin, W. C.(2011).Consideration factors and adoption of type, tabulation and framework for creating e-portfolios.Computers & Education,56(2),452-465.
  13. Chen, C. H.(2010).The implementation and evaluation of a mobile self- and peer-assessment system.Computers & Education,55(1),229-236.
  14. Cho, K.,Schunn, C.,Wilson, R.(2006).Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives.Journal of Educational Psychology,98(4),891-901.
  15. Davies, P.(2000).Computerized peer assessment.Innovations in Education and Training International,37(4),346-355.
  16. Derham, C.,Diperna, J.(2007).Digital professional portfolios of preservice teaching: An initial study of score reliability and validity.International Journal of Technology and Teacher Education,15(3),363-381.
  17. Falchikov, N.,Goldfinch, J.(2000).Student peer assessment in higher education: A metaanalysis comparing peer and teacher marks.Review of Educational Research,70(3),287-322.
  18. Gadbury-Amyot, C.(2003).Kansas City,University of Missouri-Kansas City.
  19. Gadbury-Amyot, C.,Kim, J.,Palm R. L.,Mills, E.,Noble, E.,Overman, P.(2003).Validity and reliability of portfolio assessment of competency in a baccalaureate dental hygiene program.Journal of Dental Education,67(9),991-1002.
  20. Gelinas, A. M.(1998).Columbus, Ohio,Ohio State University.
  21. Jenkins, M.(2004).Unfulfilled promise: Formative assessment using computer-aided assessment.Learning and Teaching in Higher Education,1,67-80.
  22. Johnson, R. L.,McDaniel, F.,Willeke, M. J.(2000).Using portfolios in program evaluation:An investigation of interrater reliability.American Journal of Evaluation,21(1),65-81.
  23. Knowles, M. S.,Holton III, E. F.,Swanson, R. A.(2005).The adult learner.Burlington, MA:Elsevier.
  24. Lin, S. J.,Liu, Z. F.,Yuan, S. M.(2001).Web-based peer assessment: Attitude and achievement.IEEE Transactions on Education,44(2),211-223.
  25. Lin, S. J.,Liu, Z. F.,Yuan, S. M.(2001).Web-based peer assessment: Feedback for students with various thinking-styles.Journal of Computer Assisted Learning,17(4),420-432.
  26. Liu, Z. F.(2002).Hsinchu,National Chiao Tung University.
  27. Liu, Z. F.,Lin, S. J.,Chiu, C. H.,Yuan, S. M.(2001).Web-based peer review: The learner as both adapter and reviewer.IEEE Transactions on Education,44(3),246-251.
  28. Miller, P. J.(2003).The effect of scoring criteria specificity on peer and self-assessment.Assessment & Evaluation in Higher Education,28(4),383-394.
  29. Oskay, O.,Schallies, M.,Morgil, I.(2008).A closer look at findings from recent publication.H. U. Journal of Education,35,263-272.
  30. Price, M.,O''Donovan, B.(2006).Improving performance through enhancing student understanding of criteria and feedback.Innovative assessment in higher education,London, UK:
  31. Prins, F. J.,Sluijsmans, D. M. A.,Kirschner, P. A.,Strijbos, J. W.(2005).Formative peer assessment in a CSCL environment: A case study.Assessment & Evaluation in Higher Education,30(4),417-444.
  32. Reckase, M. A.(2002).Portfolio define.Workshop of Portfolio Assessment,Taipei, Taiwan.:
  33. Russell, J. D.,Butcher, C.(1999).Using portfolios in educational technology courses.Journal of Technology and Teacher Education,7(4),279-289.
  34. Sadler, P.,Good, E.(2006).The impact of self- and peer-grading on student learning.Educational Assessment,11(1),1-31.
  35. Sulzen, J.,Young, M.,Hannifin, R.(2008).Reliability and validity of an ecologicallygrounded student teacher electronic portfolio rubric.Proceedings of Society for Information Technology & Teacher Education International Conference,Chesapeake, VA:
  36. Sung, Y. T.,Chang, K. E.,Chiou, S. K.,Hou, H. T.(2005).The design and application of a web-based self and peer-assessment system.Computers & Education,45,187-202.
  37. Sung, Y. T.,Lin, C. S.,Lee, C. L.,Chang, K. E.(2003).Evaluating proposals for experiments: An application of web-based self-assessment and peer-assessment.Teaching of Psychology,30(4),331-334.
  38. Sweat-Guy, R.,Buzzetto-More, N.(2007).A comparative analysis of common e-portfolio platforms and available features.Issues in Informing Science and Information Technology Education,5(1),327-342.
  39. Tabachnick, B. G.,Fidell, L. S.(2007).Using multivariate statistics.Boston, MA:Allyn & Bacon.
  40. Topping, K. J.,Smith, E. F.,Swanson, I.,Elliot, A.(2000).Formative peer assessment of academic writing between postgraduate students.Assessment & Evaluation in Higher Education,25(2),149-169.
  41. Tsai, C. C.,Liang, J. C.(2009).The development of science activities via on-line peer assessment: The role of scientific epistemological views.Instructional Science,37,293-310.
  42. Tsai, C. C.,Lin, S. J.,Yuan, S. M.(2002).Developing science activities through a networked peer assessment system.Computers & Education,38,241-252.
  43. Tseng, S. C,Tsai, C. C.(2007).On-line peer assessment and the role of the peer feedback: A study of high school computer course.Computers & Education,49(4),1161-1174.
  44. Wen, L. M. C.,Tsai, C. C.(2006).University students' perceptions of and attitudes toward (Online) peer assessment.Higher Education,51,27-44.
  45. Yang, Y. F.,Tsai, C. C.(2010).Conceptions of and approaches to learning through online peer assessment.Learning and Instruction,20,72-83.
  46. 王保進(2006)。中文視窗版SPSS 與行為科學研究。台北市=Taipei:心理出版社=Psychological Publishing Co., Ltd.。
  47. 余民寧(2003)。教育測驗與評量:成就測驗與教學評量。台北市=Taipei:心理出版社=Psychological Publishing Co., Ltd.。
  48. 吳明隆、涂金堂(2008)。SPSS與統計應用分析。台北市=Taipei:五南=Wu-Nan。
  49. 卓宜青(2001)。新竹市=Hsinchu,國立交通大學資訊科學系=Department of Computer and Information Science, National Chiao Tung University。
  50. 邱皓政(2008)。量化研究與統計分析:SPSS中文視窗版資料分析範例解說。台北市=Taipei:五南=Wu-Nan。
  51. 張振成(1997)。教學評量的新趨勢:實作評量與檔案評量。中等教育,48(6),90-94。
  52. 曾吉雄(1998)。新式評量:檔案評量。教育資料文摘,48(1),133-137。
  53. 楊國鑫(2002)。新竹市=Hsinchu,國立交通大學資訊科學系=Department of Computer and Information Science, National Chiao Tung University。
  54. 劉旨峰、林珊如、袁賢銘(2002)。網路化學習歷程檔案之學習成效分析及未來展望。第三屆電子化企業經營管理理論暨實務研討會,彰化縣=Changhua:
  55. 劉勝鈺(2003)。新竹市=Hsinchu,國立交通大學網路學習學程碩士班=Degree Program of CS College, National Chiao Tung University。
Times Cited
  1. 黃乙珊(2015)。網路同儕互評對大學生學習動機、自我效能與學習成效之探究 - 以「影像繪圖設計」課程為例。屏東科技大學幼兒保育系所學位論文。2015。1-100。