Epistemic Stance and Contextualization on MLM and NSP
How Japanese Chatbots Recognize the Long-Distance Cohesion between Utterances
DOI:
https://doi.org/10.52731/liir.v005.221Abstract
Error analysis in non-task-oriented dialogue systems has been discussed from many perspectives, mainly in the fields of Artificial Intelligence and Informatics. However, the current trend of error analysis focuses only on local elements and fails to incorporate the whole discourse, as shown in the masked language model and next language prediction. From a linguistic perspective, there are several reasons for errors and unnatural flow in conversations with a Chatbot. These can be stated as: 1) narrowly defined fragments of discourse and the concept of cohesion, 2) a lack of social intelligence in Chatbots due to the limited variety of corpora, and 3) the algorism uncertainty based on the limited variety of data.
This paper analyses the range of references seen in a Chatbot conversation, via qualitative and quantitative methods, and observes how errors are related to the coverage of references, why and how it occurs, and how this kind of error is related to current architectures.
The hypotheses are examined using two processes: 1) comparing the length of references between Chatbot and human interactions, 2) the frequency of errors in Chatbot conversations based on recognition of turn issues (such as insufficient recognition of references, recognition limited to two turns, and the fixed feedback move in a three-turn exchange structure) based on data from “Airfriend” and real, human-produced conversational data.
References
K. Amino, J. Dong, Z. Zhou, Z., “The Analysis of Conversational Error on the Unsuper vised Chatbot: Applying Linguistic Frameworks of IRF and its 6 Acts,” Bulletin of the Society of East Asian Language and Culture Studies, vol. 2, Academic Bulletin, Kyusyu University, 2023.
W. Chafe and J. Nichols, “Evidentiality: The Linguistic Coding of Epistemology.” Norwood, NJ Ablex, 1986.
M. Coulthard and D. Brazil, “Exchange Structure.” In Coulthard, M. (eds.), Advances in Spoken Discourse Analysis. Routledge, 1992, pp. 50-78.
A. Duranti and C. Goodwin, “Rethinking Context: Language as an Interactive Phenomenon.” In E. Ochs (Ed.), Culture and language development: Language acquisition and language socialization in a Samoan village. Cambridge: Cambridge University Press, 1992.
J. Gumperz, J. Discourse Strategies. Cambridge University Press, 1982.
R. Higashinaka, K. Funakoshi, Y. Kobayashi, M. Inaba, “The Dialogue Breakdown Detection Challenge: Task Description, Datasets, and Evaluation Metrics.” In The Tenth International Conference on Language Resources and Evaluation (LREC' 16), 2016, pp. 3146-3150.
Kobelco Systems Corporation. 2021. “Breakthrough of Natural Language Processing,” 15 Aug. 2022; https://www.kobelcosys.co.jp/column/itwords/20211101/
Ledge.ai. 2020. “What is BERT? Explaining the Mechanism of Natural Language Pro-cessing, which Google proud of,” 12 Aug. 2022; https://ledge.ai/bert/.
Matshakayile-Ndlovu, “The flash-back and the flash-forward, Zambie 29.2, De-pt. Languages Literature, Univ. Zimbabwe, 2001, pp. 189-198.
E. Ochs, E, “Becoming a Speaker of Culture.” in Language Acquisition and Language Socialization: Ecological Perspectives, ed C. Kramsch (New York, NY: Continuum), 2002, pp. 99-120.
H. Sacks, E. Schegloff and G. Jefferson, “A Simplest Systematics for the Organization of Turn-Taking for Conversation. Language” vol. 50, no. 4, 1974, pp. 696-735.
E. Takamori, E., “The comparative study of Japanese and Spanish -The feature of feedback towards opinion,” Tokyo University of Foreign Studies, 2004. “unpublished master’s thesis.”
