簡易檢索 / 詳目顯示

研究生: 范瀚之
Fan, Han-Chih
論文名稱: 人工智慧面試官與人類專業面試官對於面試者在錄影面試中進行印象管理辨別程度之比較性研究
The Comparative Study on Accuracy of Identifying Interviewees' Impression Management Used in Recorded Interview by Embedded Artificial Intelligence and Human Interviewers
指導教授: 孫弘岳
Suen, Hung-Yue
口試委員: 陳建丞
Chen, Chien-Cheng
林弘昌
Lin, Hung-Chang
孫弘岳
Suen, Hung-Yue
口試日期: 2021/08/10
學位類別: 碩士
Master
系所名稱: 科技應用與人力資源發展學系人力資源發展碩士在職專班
Department of Technology Application and Human Resource Development_Continuing Education Master's Program of Human Resource Development
論文出版年: 2021
畢業學年度: 109
語文別: 中文
論文頁數: 72
中文關鍵詞: 印象管理臉部表情卷積神經網路人工智慧面試官
英文關鍵詞: Impression management, Facial expression, Embedded artificial intelligence, Convolutional neural networks
研究方法: 實驗設計法
DOI URL: http://doi.org/10.6345/NTNU202101482
論文種類: 學術論文
相關次數: 點閱:88下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 面試是最常見的招募甄選工具,幾乎每個組織都會採用面試。在面試過程當中,面試者會努力運用誠實性或是欺騙性的印象管理(Impression management, IM)技巧來影響面試官的評分,以期提高錄取的機會。隨著科技的發展,不同型態的面試方式如視訊會議或錄影面試,以及面試決策輔助工具像是人工智慧面試官(Embedded artificial intelligence, embedded AI)等也越來越常見。與人工智慧面試官相比,人類專業面試官透過口語與非口語線索如臉部表情(Facial expressions),細微臉部表情(Subtle facial expressions)或微表情 (Micro expressions)來判斷面試者表現的敏感度較低,且無法準確判斷面試者是否有使用印象管理技巧。
    本研究邀請30位曾任或現職為管理職或人力資源之從業者,作為人類專業面試官。再邀請32位有工作經驗之社會人士或即將畢業的學生作為面試者進行錄影面試(又稱非同步視訊面試)並填寫自我印象管理問卷。每位面試者由3位面試官看完錄影面試後進行印象管理評分;再將錄影面試資料由先前研究所開發出能夠自動辨認印象管理技巧之人工智慧面試官來產出印象管理評分。資料收集完全後,將面試者印象管理自評之分數與人工智慧面試官評分與人類專業面試官評分與進行相關分析。
    結論顯示人工智慧面試官評分與面試者自評在自我推銷、自我辯護、誇大不實與避重就輕等四種印象管理構面上都有顯著正相關,而人類專業面試官評分與面試自評在此四個構面上皆無顯著相關。故可得知人工智慧面試官較人類專業面試官更能辨別面試者在印象管理技巧之使用。

    Interview is the most common tool for recruitment and selection which adopted by almost every organization. Interviewees will try their best to use either honest or deceptive impression management (IM) to increase the possibility of getting job offer by influencing interview score of interviewers. With the development of technology there’re more and more methods for conducting interview (e.g. teleconference or asynchronous video interview, AVI) and for interview decision assisting tool (e.g. embedded artificial intelligence, embedded AI). Human expert interviewers are lesser sensitive to identify performance of interviewees according to their verbal and non-verbal clues (e.g. facial expressions, subtle facial expressions or micro-expressions) as well as weather applying impression management or not compared to embedded AI.
    This study invited 30 former or incumbent managers or human resource practitioners as human expert interviewers to evaluate IM score of interviewees together with embedded AI. And invited 32 experienced office workers or fresh graduate students as interviewees to attend AVI and complete self-report IM. Each interviewee was evaluated by 3 human expert interviewers after reviewing his/her recorded video interview and average their scores of IM as final score. On the other hand, each recorded video interview was evaluated by embedded AI developed to automatically identify and recognize IM in previous study as well. Correlation analysis was adopted for understanding the correlation between self-report IM score and IM score evaluated by embedded AI and between self-report IM score and IM score evaluated by human expert interviewer respectively.
    The result shows that IM scores evaluated by embedded AI was highly correlated to self-report IM score but there’s no significant correlation between IM scores evaluated by human expert interviewers and self-report IM score. In conclusion embedded AI can accurately identify IM applied by interviewees compared to human expert interviewers.

    中文摘要 i 目 錄 v 表 次 vii 圖 次 viii 第一章 緒 論 1 第一節 研究背景與動機 1 第二節 研究目的與待答問題 4 第三節 名詞解釋 5 第二章 文獻探討 7 第一節 印象管理 7 第二節 印象管理與臉部表情 9 第三節 人工智慧面試官運用臉部表情判斷印象管理 11 第三章 研究設計與實施 15 第一節 研究架構 15 第二節 研究假設 16 第三節 研究設計與步驟 17 第四節 研究對象 19 第五節 研究工具 20 第六節 資料處理與分析 30 第七節 研究流程 33 第四章 結果與討論 35 第ㄧ節 信效度分析 35 第二節 描述性統計分析 40 第三節 變異數分析 43 第四節 相關分析 46 第五節 研究結果 52 第五章 結論與建議 53 第一節 研究討論 53 第二節 研究貢獻與建議 56 第三節 研究限制與未來研究建議 59 參考文獻 63 一、外文部份 63

    一、外文部份
    Amaral, A. A., Powell, D. M., & Ho, J. L. (2019). Why does impression management positively influence interview ratings? The mediating role of competence and warmth. International Journal of Selection and Assessment, 27(4), 315-327. https://doi.org/10.1111/ijsa.12260
    Arthur Jr, W., Glaze, R. M., Villado, A. J., & Taylor, J. E. (2010). The magnitude and extent of cheating and response distortion effects on unproctored internet‐based tests of cognitive ability and personality. International Journal of Selection and Assessment, 18(1), 1-16.
    Barrick, M. R., Shaffer, J. A., & DeGrassi, S. W. (2009). What you see may not be what you get: relationships among self-presentation tactics and ratings of interview and job performance. Journal of Applied Psychology, 94(6), 1394. https://doi.org/10.1037/a0016532
    Basch, J. M., Melchers, K. G., Kegelmann, J., & Lieb, L. (2020). Smile for the camera! The role of social presence and impression management in perceptions of technology-mediated interviews. Journal of Managerial Psychology. https://www.emerald.com/insight/content/doi/10.1108/JMP-09-2018-0398/full/html
    Blacksmith, N., Willford, J., & Behrend, T. (2016). Technology in the Employment Interview: A Meta-Analysis and Future Research Agenda. Personnel Assessment and Decisions, 2(1). https://doi.org/10.25035/pad.2016.002
    Bolino, M. C., Kacmar, K. M., Turnley, W. H., & Gilstrap, J. B. (2008). A multi-level review of impression management motives and behaviors. Journal of Management, 34(6), 1080-1109. https://doi.org/10.1177/0149206308324325
    Bond Jr, C. F., & DePaulo, B. M. (2006). Accuracy of deception judgments. Personality and Social Psychology Review, 10(3), 214-234. https://doi.org/10.1207/s15327957pspr1003_2
    Bourdage, J. S., Roulin, N., & Tarraf, R. (2018). “I (might be) just that good”: Honest and deceptive impression management in employment interviews. Personnel Psychology, 71(4), 597-632. https://doi.org/10.1111/peps.12285
    Brenner, F. S., Ortner, T. M., & Fay, D. (2016). Asynchronous Video Interviewing as a New Technology in Personnel Selection: The Applicant’s Point of View [Original Research]. Frontiers in Psychology, 7(863). https://doi.org/10.3389/fpsyg.2016.00863
    Buller, D. B., & Burgoon, J. K. (1996). Interpersonal deception theory. Communication Theory, 6(3), 203-242. https://doi.org/10.1111/j.1468-2885.1996.tb00127.x
    Burgoon, J. K. (2018). Microexpressions Are Not the Best Way to Catch a Liar [Opinion]. Frontiers in Psychology, 9(1672). https://doi.org/10.3389/fpsyg.2018.01672
    DeGroot, T., & Gooty, J. (2009). Can nonverbal cues be used to make meaningful personality attributions in employment interviews? Journal of Business and Psychology, 24(2), 179-192. https://doi.org/10.1007/s10869-009-9098-0
    Donnellan, M. B., Oswald, F. L., Baird, B. M., & Lucas, R. E. (2006). The mini-IPIP scales: tiny-yet-effective measures of the Big Five factors of personality. Psychological Assessment, 18(2), 192. https://doi.org/10.1037/1040-3590.18.2.192
    Du, S., Tao, Y., & Martinez, A. M. (2014). Compound facial expressions of emotion. Proceedings of the National Academy of Sciences, 111(15), E1454-E1462. https://doi.org/10.1073/pnas.1322355111
    Ekman, P., & Friesen, W. V. (1969a). Nonverbal leakage and clues to deception. Psychiatry, 32(1), 88-106. https://doi.org/10.1080/00332747.1969.11023575
    Ekman, P., & Friesen, W. V. (1969b). The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Nonverbal communication, interaction, and gesture, 57-106. https://doi.org/10.1515/9783110880021.57
    Ekman, P., O'Sullivan, M., Friesen, W. V., & Scherer, K. R. (1991). Invited article: Face, voice, and body in detecting deceit. Journal of Nonverbal Behavior, 15(2), 125-135. https://doi.org/10.1007/BF00998267
    Elaad, E., & Reizer, A. (2015). Personality correlates of the self-assessed abilities to tell and detect lies, tell truths, and believe others. Journal of Individual Differences. https://doi.org/10.1027/1614-0001/a000168
    Ellis, A. P., West, B. J., Ryan, A. M., & DeShon, R. P. (2002). The use of impression management tactics in structured interviews: A function of question type? Journal of Applied Psychology, 87(6), 1200. https://doi.org/10.1037/0021-9010.87.6.1200
    Escalante, H. J., Kaya, H., Salah, A. A., Escalera, S., Gucluturk, Y., Guclu, U., Madadi, M. (2018). Explaining first impressions: modeling, recognizing, and explaining apparent personality from videos. arXiv preprint arXiv:1802.00745. https://arxiv.org/abs/1802.00745
    Frank, M. G., & Ekman, P. (1997). The ability to detect deceit generalizes across different types of high-stake lies. Journal of Personality and Social Psychology, 72(6), 1429. https://doi.org/10.1037/0022-3514.72.6.1429
    Friesen, E., & Ekman, P. (1978). Facial action coding system: a technique for the measurement of facial movement. Palo Alto, 3(2), 5. https://www.semanticscholar.org/paper/Facial-action-coding-system%3A-a-technique-for-the-of-Ekman-Friesen/1566cf20e2ba91ca8857c30083419bf7c127094b
    Gu, S., Wang, F., Patel, N. P., Bourgeois, J. A., & Huang, J. H. (2019). A model for basic emotions using observations of behavior in Drosophila. Frontiers in Psychology, 10, 781. https://doi.org/10.3389/fpsyg.2019.00781
    Higgins, C. A., Judge, T. A., & Ferris, G. R. (2003). Influence tactics and work outcomes: A meta‐analysis. Journal of Organizational Behavior: The International Journal of Industrial, Occupational and Organizational Psychology and Behavior, 24(1), 89-106. https://doi.org/10.1002/job.181
    Hsia, C.-H. (2018). Improved finger-vein pattern method using wavelet-based for real-time personal identification system. Journal of Imaging Science and Technology, 62(3), 30402-30401-30402-30408. https://doi.org/10.2352/J.ImagingSci.Technol.2018.62.3.030402
    Huffcutt, A., & Culbertson, S. (2011). Interviews APA Handbook of Industrial and Organizational Psychology (Vol. 2: Selecting and Developing Members for the Organization). Washington, DC: American Psychological Association, 185-203. https://doi.org/10.1037/12170-006
    Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255-260. https://science.sciencemag.org/content/349/6245/255.abstract
    Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20(1), 141-151. https://doi.org/10.1177/001316446002000116
    Koo, K.-M., & Cha, E.-Y. (2017). Image recognition performance enhancements using image normalization. Human-centric Computing and Information Sciences, 7(1), 1-11. https://doi.org/10.1186/s13673-017-0114-5
    Langer, M., König, C. J., & Krause, K. (2017). Examining digital interviews for personnel selection: Applicant reactions and interviewer ratings. International Journal of Selection and Assessment, 25(4), 371-382. https://doi.org/10.1111/ijsa.12191
    Langer, M., König, C. J., & Papathanasiou, M. (2019). Highly automated job interviews: Acceptance under the influence of stakes. International Journal of Selection and Assessment, 27(3), 217-234. https://doi.org/10.1111/ijsa.12246
    Leary, M. R., & Kowalski, R. M. (1990). Impression management: A literature review and two-component model. Psychological Bulletin, 107(1), 34. https://doi.org/10.1037/0033-2909.107.1.34
    LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444. https://www.nature.com/articles/nature14539
    Levashina, J., & Campion, M. A. (2007). Measuring faking in the employment interview: development and validation of an interview faking behavior scale. Journal of Applied Psychology, 92(6), 1638. https://doi.org/10.1037/0021-9010.92.6.1638
    Levashina, J., Hartwell, C. J., Morgeson, F. P., & Campion, M. A. (2014). The structured employment interview: Narrative and quantitative review of the research literature. Personnel Psychology, 67(1), 241-293. https://doi.org/10.1111/peps.12052
    Liem, C. C., Langer, M., Demetriou, A., Hiemstra, A. M., Wicaksana, A. S., Born, M. P., & König, C. J. (2018). Psychology meets machine learning: Interdisciplinary perspectives on algorithmic job candidate screening. In Explainable and interpretable models in computer vision and machine learning (pp. 197-253). Springer. https://doi.org/10.1007/978-3-319-98131-4_9
    Lorenzo, G. L., Biesanz, J. C., & Human, L. J. (2010). What is beautiful is good and more accurately understood: Physical attractiveness and accuracy in first impressions of personality. Psychological Science, 21(12), 1777-1782. https://doi.org/10.1177/0956797610388048
    Marcus, B. (2009). ‘Faking’from the applicant's perspective: A theory of self‐presentation in personnel selection settings. International Journal of Selection and Assessment, 17(4), 417-430. https://doi.org/10.1111/j.1468-2389.2009.00483.x
    Mejia, C., & Torres, E. N. (2018). Implementation and normalization process of asynchronous video interviewing practices in the hospitality industry. International Journal of Contemporary Hospitality Management, 30(2), 685-701. https://doi.org/10.1108/ijchm-07-2016-0402
    Melchers, K. G., Roulin, N., & Buehl, A. K. (2020). A review of applicant faking in selection interviews. International Journal of Selection and Assessment, 28(2), 123-142. https://doi.org/10.1111/ijsa.12280
    Nestler, S., & Back, M. D. (2013). Applications and extensions of the lens model to understand interpersonal judgments at zero acquaintance. Current Directions in Psychological Science, 22(5), 374-379. https://doi.org/10.1177/0963721413486148
    Nestler, S., Egloff, B., Küfner, A. C., & Back, M. D. (2012). An integrative lens model approach to bias and accuracy in human inferences: Hindsight effects and knowledge updating in personality judgments. Journal of Personality and Social Psychology, 103(4), 689. https://doi.org/10.1037/a0029461
    Nunnally, J. C. (1978). An overview of psychological measurement. Clinical diagnosis of mental disorders, 97-146. https://doi.org/10.1007/978-1-4684-2490-4_4
    Ötting, S. K., & Maier, G. W. (2018). The importance of procedural justice in human–machine interactions: Intelligent systems as new decision agents in organizations. Computers in Human Behavior, 89, 27-39. https://doi.org/10.1016/j.chb.2018.07.022
    Porter, S., & Ten Brinke, L. (2008). Reading between the lies: Identifying concealed and falsified emotions in universal facial expressions. Psychological Science, 19(5), 508-514. https://doi.org/10.1111/j.1467-9280.2008.02116.x
    Porter, S., Ten Brinke, L., & Wallace, B. (2012). Secrets and lies: Involuntary leakage in deceptive facial expressions as a function of emotional intensity. Journal of Nonverbal Behavior, 36(1), 23-37. https://doi.org/10.1007/s10919-011-0120-7
    Roulin, N., Bangerter, A., & Levashina, J. (2015). Honest and deceptive impression management in the employment interview: Can it be detected and how does it impact evaluations? Personnel Psychology, 68(2), 395-444. https://doi.org/10.1111/peps.12079
    Shen, X., Fan, G., Niu, C., & Chen, Z. (2021). Catching a liar through facial expression of fear. Frontiers in Psychology, 12, 2211. https://doi.org/ 10.3389/fpsyg.2021.675097
    Su, L., & Levine, M. (2016). Does “lie to me” lie to you? An evaluation of facial clues to high-stakes deception. Computer Vision and Image Understanding, 147, 52-68. https://doi.org/10.1016/j.cviu.2016.01.009
    Suen, H.-Y., Chen, M. Y.-C., & Lu, S.-H. (2019). Does the use of synchrony and artificial intelligence in video interviews affect interview ratings and applicant attitudes? Computers in Human Behavior, 98, 93-101. https://doi.org/10.1016/j.cviu.2016.01.009
    Suen, H.-Y., Hung, K.-E., & Lin, C.-L. (2019). TensorFlow-based automatic personality recognition used in asynchronous video interviews. IEEE Access, 7, 61018-61023. https://doi.org/10.1109/ACCESS.2019.2902863
    Suen, H.-Y., Hung, K.-E., & Lin, C.-L. (2020). Intelligent video interview agent used to predict communication skill and perceived personality traits. Human-centric Computing and Information Sciences, 10(1), 1-12. https://doi.org/10.1186/s13673-020-0208-3
    Sun, A., Li, Y., Huang, Y.-M., Li, Q., & Lu, G. (2018). Facial expression recognition using optimized active regions. Human-centric Computing and Information Sciences, 8(1), 1-24. https://doi.org/10.1186/s13673-018-0156-3
    Swider, B. W., Barrick, M. R., & Harris, T. B. (2016). Initial impressions: What they are, what they are not, and how they influence structured interview outcomes. Journal of Applied Psychology, 101(5), 625. https://doi.org/10.1037/apl0000077
    Takalkar, M., Xu, M., Wu, Q., & Chaczko, Z. (2018). A survey: facial micro-expression recognition. Multimedia Tools and Applications, 77(15), 19301-19325. https://doi.org/10.1007/s11042-017-5317-2
    Vrij, A., Granhag, P. A., & Porter, S. (2010). Pitfalls and opportunities in nonverbal and verbal lie detection. Psychological Science in the Public Interest, 11(3), 89-121.
    Wu, Z., Singh, B., Davis, L., & Subrahmanian, V. (2018). Deception detection in videos. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). https://ojs.aaai.org/index.php/AAAI/article/view/11502
    Xin, Y., Kong, L., Liu, Z., Chen, Y., Li, Y., Zhu, H., . . . Wang, C. (2018). Machine Learning and Deep Learning Methods for Cybersecurity. IEEE Access, 6, 35365-35381. https://doi.org/10.1109/access.2018.2836950
    Zeng, Z., Gong, Q., & Zhang, J. (2019). CNN model design of gesture recognition based on tensorflow framework. 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC),
    Zuckerman, M., DePaulo, B. M., & Rosenthal, R. (1981). Verbal and nonverbal communication of deception. In Advances in Experimental Social Psychology (Vol. 14, pp. 1-59). Elsevier. https://doi.org/10.1016/S0065-2601(08)60369-X

    無法下載圖示 電子全文延後公開
    2025/01/01
    QR CODE