IIAI Letters on Institutional Research

A Survey on Self-Perception of Institutional Research Skills and Knowledge by Focusing on the Gap with the Participants Needs of Training Courses

Tetsuya Oishi — 2024-09-14

We conducted a web questionnaire survey to determine the need for training courses for institutional research in FY2021 and conducted individual surveys in FY2022. It was found that there is a need for step-by-step training sessions based on the abilities of the participants of these training courses. Moreover, we conducted an additional training course to determine the essential training for institutional research and the self-perceptions of the participants in FY2023. This paper reports on the background to the survey conducted at this additional training course and the results of the questionnaire survey conducted at this training course.

Integrating Tableau into a First-Year Information Literacy Course: A Practical Approach to Enhancing Data Science Education

Shintaro Tajiri — 2024-09-15

In the face of rapid technological innovation and the increasing importance of data science skills across all fields, higher education institutions must adapt their curricula to equip students with the tools and knowledge required to thrive in a data-driven world. This paper discusses the inte-gration of Tableau, a business intelligence tool for data visualization, into a first-year information literacy course at Hokuriku University as a practical approach to enhancing data science educa-tion. The university launched the “Data Science and AI Education Program” in 2022, which combines traditional information literacy topics with hands-on learning using Tableau, engaging students with real-world datasets from campus stores and cafeterias. The implementation and practice of the program are discussed, highlighting the use of Tableau as a gateway to introduce fundamental data science concepts and problem-solving skills. Although the integration of Tab-leau in university courses is still relatively rare, this paper contributes to the growing evidence of its educational value. Future studies will delve into the program’s effectiveness through student surveys and other data, aiming to contribute to the ongoing discourse on best practices in data science education in the era of accelerating technological innovation.

Enhancing Moodle Insights

Yuji Kobayashi — 2024-09-15

Moodle access logs have been widely used in learning analytics and institutional research to analyze educational activities. However, previous research primarily focused on accessing count data, overlooking the potential of leveraging the time-tracking data made possible by the addi-tion of IntelliBoard functionality to Moodle. This study explores novel applications of Moodle's time-tracking data. Specifically, we examined the relationship between access count and learn-ing time data, as well as the analytical possibilities of these datasets, in the context of a faculty development (FD) training course for newly hired instructors. The results suggest that incorpo-rating time-tracking data enables a more detailed analysis of learner behavior, presenting op-portunities to inform effective instructional design. This study proposes new methods of utilizing Moodle’s time-tracking data to complement the established use of access-count data in learning analytics and institutional research.

How to Enroll Industrial Collaborators in the Nascent Ac-ademic Entrepreneurship Under Uncertainty: A Concept Model

WEI LIU — 2024-09-14

Enrolling industrial collaborators is an ideal method for academia seeking commercialization of their research findings. We define and delineate novelty types of uncertain for academic entre-preneurs in the cooperative entrepreneurship contexts: the struggles to establish social ties with potential collaborators and hardness to strengthen the cooperative relationship. In developing our theory, we employ a two-stage model to uncover how academic entrepreneurs intentionally target desired pre-commitment towards industrial collaborators and then how they achieve sustained commitment to ultimately set up a company. Herein, we add important insights to complete the model. On one hand, individual proximity provides an important path for network construction and is regulated by reputation in the former stage. On the other hand, sufficient information feed-back and knowledge sharing are required to engage both of them in the entrepreneurial journey in the latter one. Overall, we put forward the conceptual the model to illustrate this process.

Assessment of Study Abroad Programs as Co-Curricular Program

Soichiro Aihara — 2024-09-15

Global competency was measured for a university-wide study abroad program using the MGUDS-S Japanese version questionnaire. The survey was administered to 756 participants in the summer study abroad program in 2023. The survey was conducted both pre-test and post-test. First, the validity and reliability of the questionnaire were confirmed with factor analysis and reliability coefficients. Next, a paired t-test and effect sizes were measured. Although the growth in scores was not large, statistically significant differences were found. the MGUDS-S Japanese version questionnaire can be used to measure global competency in study abroad programs.

A Study on the Influence of Community-Based Education on Post-University Workers and Its Time Dependence

Tatsuya Tsumagari — 2024-09-15

This study identified the long-term effects of community-based education on a university la-boratory that conducts "community-based education," a form of university education in collab-oration with the local community. Ninety-three graduates from their first to 16th year in the workforce were asked in a descriptive format whether their learning through activities in the community had been useful to them as members of society. A dataset was created from the free-writing responses of graduates and analyzed using the Structural Topic Model (STM). Five topics were extracted. For the graduates of the laboratories surveyed, the usefulness of learning through activities in the region during their university years can be summarized by these five topics. Multiple regression analysis was conducted on these topics, with the proportion of topics as the dependent variable and the number of years since graduation and the degree of involve-ment with the regional community as independent variables. The results showed that Topic 4 was significantly dependent on the number of years since graduation, and Topic 3 also showed a significant trend. In addition, Topics 1 and 3 were significantly dependent on the degree of in-volvement in the community during the respondent’s university years, and Topic 2 showed a significant trend.

Gender Equality and Digital Education as Catalysts for Economic Growth

Noriko Ito — 2024-02-01

This study conducts a comparative analysis of the economic growth models of Japan and the Nordic five countries, Denmark, Finland, Iceland, Norway, and Sweden, from 1990 to 2022, with a particular emphasis on three aspects: education systems, political participation, and digitalization. Progress and challenges in achieving gender equality, promoting digital education, encouraging entrepreneurship, and fostering innovation are examined to discern how these aspects have affected economic growth in both regions. Through the analysis, the differences in economic growth models of Japan and the Nordic countries, especially in the realms of educational reforms brought about by gender equality and economic growth, are discussed.

Programming Education Methods for Elementary School Students and Their Relation with Personal Preferences

Emi Ichimura — 2024-09-15

This study investigated the differential impact of cooperative and competitive instructional strategies in the programming education for elementary students using the visual program-ming language, Scratch. The methodology involved conducting 40-minute sessions within two distinct educational settings to explore how students' preferences for specific tastes, colors, and school subjects influenced their learning outcomes. These preferences were se-lected from readily accessible elements that could be acquired rapidly, thereby serving as indicators to facilitate a simplified assessment of students' personality traits. The efficacy of the instructional sessions was gauged by evaluating task achievement and ingenuity, which were further linked to personality traits extrapolated from student preferences. The results demonstrated that a competitive setting notably enhanced both achievement and ingenuity. Remarkably, students who preferred competitive environments exhibited higher levels of achievement and ingenuity, whereas most participants predominantly perceived cooperative environments as more enjoyable. No significant relationships emerged between learning outcomes and other preferences, such as gender, favorite color, or chosen academic subjects. This study highlights the critical importance of customizing programming instruction to align it with the individual characteristics and preferences of students to optimize educa-tional effectiveness.

Current Status and Issues Concerning the Formulation and Operation of Mid-Term Plans for Japanese National Universities

Tetsuya Oishi — 2024-09-14

Today, many universities are faced with diverse and rapid changes in the business environment and need to improve their management. Since the formulation and implementation of management plans is effective as a method of management improvement, all Japanese universities are required to formulate mid-term plans. However, the formulation and operation of university management plans has not been established, and it is assumed that mid-term plans are not sufficiently conducive to improving management planning. In light of this situation, we conducted a survey of all universities in Japan to understand the current status of and issues related to mid-term planning. Based on the survey results, the presentation will report on the current roles and issues of mid-term plans of national universities, the focus and issues of the formulation process, and the current status and issues related to their operation after formulation.

Beyond Silos: Eduinformatics as a Catalyst for Dissolving Faculty and Staff Boundaries in Higher Education

Kunihiko Takamatsu — 2024-09-15

This paper introduces eduinformatics as an innovative approach to bridge the longstanding di- vide between faculty and staff within institutions of higher education. By integrating education and information technology, eduinformatics fosters a collaborative environment that transcends traditional siloed structures, strengthening communication and cooperation. We explore the historical context of disciplinary boundaries in Japanese higher education, highlighting the shift towards interdisciplinary studies. Through a conceptual framework and practical applications of eduinformatics, we examine its potential to transform educational practices by facilitating da- ta-driven decision-making and improving student learning outcomes. The paper discusses the significance of eduinformatics in promoting a more holistic and interdisciplinary approach to learning and teaching in higher education.

Proposing a New Field: Institutional Research (IR) Philosophy based on Eduinformatics

Kunihiko Takamatsu — 2024-02-01

Amid the transition from Society 4.0 to Society 5.0, the role of Institutional Research (IR) in higher education is evolving. We have proposed an interdisciplinary field named “Eduinformatics,” which integrates education and informatics, offering fresh insights into data-driven educational strategies. In this study, we further introduce “IR Philosophy” as a novel approach to bridge the gap between the theoretical and practical aspects of IR. By examining the current state of IR in Japanese universities, we emphasize the significance of technical skills and the importance of understanding the broader educational context, termed “contextual knowledge.” Our findings suggest that while technical proficiency is crucial, a profound understanding of the broader educational context, referred to as “issue knowledge,” is equally vital. Furthermore, as we move into the era of Society 5.0, our research underscores the need for a more integrated approach to IR, emphasizing its pivotal role in shaping the future of education.

Experiments of Automatic Scoring Using Generative AI in a Summary Essay Learning System

Takahiro Yamasaki — 2024-09-15

We are developing an e-learning system aimed at improving Japanese language proficiency. This system focuses on the task of summarizing texts. An e-learning system designed for on-demand learning requires an automatic scoring function to provide feedback to users. This paper focuses on the scoring function using generative AI. Generative AI can be utilized in two ways: generating various patterns of correct answers and using it in the scoring process itself. When generating model answers, the user’s responses are compared to these model answers for scoring, so the method of comparison is also considered. This paper reports experimental results on the differences between these applications.

Effects and Limitations of University Information Disclosure: A Study on the Impact on University Choice

Nozomi Yoshida — 2024-09-14

This study aims to find an effective way of career-path selection for high school students by clarifying the impact of such educational information on the university choices of prospective students. In this study, we conducted a questionnaire survey targeting university students to ex-amine whether the information required to be disclosed by universities was perceived at the time of university selection and to what extent this information was helpful in making their university choices. Results showed that the perception of university educational information was overall low, but for the information that was perceived, it was generally shown to help make career choices. Besides, differences in respondents' attributes such as post-graduation career aspira-tions, academic fields at the university, and entrance exam formats also showed variations in perception and usefulness. These results imply the value of the information provided for choosing the college and the major academic field in relation to the future career path.

Analysis of Discrepancies in Learning Awareness of Data Science Across Disciplines

Eriko TANAKA — 2024-09-15

In recent years, there has been a growing demand for data science education alongside rapid advancements in technology, particularly in AI. However, despite various attempts to innovate curricula and teaching methods at universities, there is limited research on students' awareness of their learning needs in this field. Hence, this study investigated the recognition of the importance of data science and willingness to take related courses among a diverse range of students, includ-ing those from both scientific and non-scientific disciplines. The results revealed that although 90% of the students recognized the necessity of data science in the future, less than half were considering taking related courses. In our faculty, we are considering measures, such as compul-sory courses and certification systems, to enhance students' knowledge and skills in data science and make them more accessible to a wider range of students. This paper summarizes the back-ground and findings of the study.

Diploma Supplements in Japanese Higher Education

Satoshi Ozeki — 2024-02-01

Over the last decade, the diploma supplement (DS), a document providing detailed information on degrees’ qualifications, has been introduced into Japanese higher education (HE) as part of a broader reform of its quality assurance system. Scholars argue that the Japanese DS focuses on individual student learning but not on articulation and student mobility, as observed in the European Higher Education Area. However, little is known about DS use in Japan. As such, this study aims to investigate DS implementation in Japanese HE. An online questionnaire was developed to examine the implementation rate and information type included in the DS. The survey targeted all 787 national, public, and private universities offering bachelor’s degrees in Japan. The study obtained a total of 240 responses, resulting in a response rate of 30.5%. Subsequent analysis revealed that 29.6% of the universities had implemented the DS, with higher rates in national and private universities than prefectural and municipal universities. The main reason for DS implementation was to “visualize student learning outcomes” (93.0%), and the most popular information type included in the DS was “indicators of attainment based on diploma policy learning outcomes” (73.2%). This study supports the argument that DS use in Japanese HE is related to student learning outcomes.

Preliminary Study of Teachers’ Opinions on the Effective Use of Digital Textbooks

Fumiko Yagisawa — 2024-09-15

In this study, we conducted preliminary interviews with a teacher with experience in the use of digital textbooks to examine the perspectives necessary for their effective use in the classroom based on the actual use and characteristics of digital textbooks observed by the teacher. We qual-itatively analyzed the responses and organized the results as a case study. The results suggest the following four influencing factors in the effective use of digital textbooks: content and duration of initial instructions to familiarize students with digital textbooks, amount of experience with learning activities using digital textbooks, student attitudes toward learning, and characteristics of the subject matter.

Comparative Analysis of High and Low Performers' Behavior from Research Topic Exploration to Research Outreach

Kaoru Matsumoto — 2024-09-14

This study presents findings from an exploration into the practices of active researchers, en-compassing the entire process from collecting research information to presenting their findings. Participants were researchers from universities, national research and development corporations, and independent administrative agency research institutes in Japan.We asked 486 organizations and received 1442 responses. The findings indicate that high-performing researchers prioritize sharing their work widely. Additionally, the way they gathered information differed highly among different fields of study.

Context-sensitive Classification for Scientific Keywords in Grant Reports

Michiko Yasukawa — 2024-09-15

In the task of institutional research (IR), it is important for each university to identify the latest trends in cutting-edge scientific research and to understand its own strengths. The Grantin-Aid for Scientific Research (KAKENHI), the largest research grant in Japan, makes publicly available the research outline, progress, and keywords of adopted research projects. These open data can be used to analyze research information in IR tasks. Our study in this paper focuses specifically on keyword analysis in research grant reports. Technical terms that describe scientific projects are important clues in analyzing research information. However, state-of-the-art terminology is not easy to process on computers because word occurrences and usages are often polysemous and unpredictable. To deal with this issue, we propose a method for disambiguating keywords by attaching a prefix to each keyword that takes into account the context in which the keyword appears. Such contextual prefixes are expected to enable useful searches for relevant keywords and automatic classification of keywords. Evaluation experiments on real data confirmed the effectiveness of our proposed method.

Identifying the High Risk Duration to the Semester to Drop Out of College Using Dropout Probability

Naruhiko Shiratori — 2024-09-15

This study investigates dropout probability, which quantifies the risk of a student discontinuing their studies, to determine the length of the high-risk phase prior to dropout. The departure of a student from college poses significant adverse impacts on both the individual and the institution. Universities have implemented various dropout prevention measures; however, their effective-ness hinges on timely execution. Through an analysis of dropout probabilities by semester for 173 students who eventually dropped out, it was found that these students were at an increased risk of dropping out an average of 2.97 semesters, or approximately 18 months, before actually leaving the university.

Predicting Student Dropout Risk Using LMS Logs

Takaaki Ohkawauchi — 2024-03-18

Traditionally, the prediction of student dropout in university classes has often been based on stu-dents’ pre-enrollment information or confirmed grade data for each semester after enrollment. However, effective support requires early intervention when signs of dropping out appear. In this study, we propose a model to continuously measure dropout signs using log data accumulated in a learning management system during classes. By applying machine learning to the log data in the learning management system, we could continuously update information on at-risk students with high accuracy from the beginning to the end of the class.

Designing Data Science Courses to Support Non-STEM Undergraduate Students

Mio Tsubakimoto — 2024-09-15

This study employed the Expectancy-Value Theory framework to organize the educational prac-tices of a data science minor course attended by a mixed cohort of humanities and science stu-dents. This approach aims to identify the elements that support humanities students in learning data science. The findings indicate that integrating course content with the students' major fields of study, simplifying the process of setting up technical environments, and offering prompt feed-back through information and communication technology are essential.

Proposed Analytical Process for More Convenient Utilization of Open Data

Soh SAKURAI — 2024-09-15

This study proposes a statistical analysis process for the effective and convenient utilization of open data. Considering the current state of open data use in Japan and other countries, it focuses on the challenge where maximizing data utility is heavily dependent on user skills. The process is validated using easily accessible prefectural tourist data, which is rich in academic research. By applying principal component analysis and regression analysis, the study defines a specific model and proposes a process aimed at enabling more practical and straightforward applications of open data.

Is Dual Enrollment a Predictor of Academic Success?

Jennifer Lude — 2024-02-01

As almost all institutions of higher education continue to face enrollment issues, they must consider evidence-based strategies to sustain and increase recruitment, matriculation, retention, and graduation. Policy makers and educational administrators expect that dual enrollment provides institutions of higher education an opportunity to invest early in student success by encouraging students to pursue postsecondary education while decreasing cost and time to graduation. However, findings across multiple studies suggest that dual enrollment does not necessarily always increase a student’s chance for positive post-secondary school outcomes. This study analyzed data from a R1 southern flagship institution to determine if dual enrollment was a predictor for college graduation. Use of descriptive statistics, ANOVA, and stepwise logistic regression determined that dual enrollment alone did not increase the probability of graduation. Variables impacting graduation and dual enrollment characteristics are discussed and recommendations provided for institutions.

Learning Objectives from the Syllabus and Model Core Curriculum for Medical Education in Japan

Yoshikazu Asada — 2024-09-15

The Model Core Curriculum for Medical Education (MCC) demonstrates the “core” parts of the curriculum for Japanese medical education. Given that MCC is defined as the model curriculum and medical education should be based on the MCC, the current status of the connection between MCC and the university curricula is surveyed. The research consisted of two brief studies: (1) calculating the total number of medical universities that show a connection between their curric-ulum and MCC, and (2) mapping the results of the curriculum at the author’s university and MCC. About 40% of universities show connections between their courses and MCC in syllabi. Although all of the MCC learning objectives in the third tier are covered in the curriculum of the author’s university, some of the learning objectives have few related courses.

A Trial to Convert Graduation Competency Question-naire Data into Key Performance Indicators in Japanese Medical Schools

Koji Tsunekawa — 2024-09-14

Introduction: The monitoring of graduation competencies is a crucial issue for institutional research (IR) for medical schools in terms of accreditation. To enable the comparison of the acquisition of competencies across universities, the current study aimed to achieve better quality assurance by converting data disclosed in graduation competency questionnaires into key performance indicators (KPIs). Methods: First, we conducted a Google search of 82 Japanese universities regarding the disclosure of information on graduation competencies and examined the characteristics of universities that fit the criteria. The graduation competencies of the targeted universities were then examined and categorized into newly revised common national competencies followed by the ranking of data and the calculation of averages. Finally, comparisons of the competency data of the universities were conducted. Results: Twelve universities met the criteria and exhibited significant differences in characteristics such as foundation and name. One of the newly revised common national competencies was missing from the existing graduation competencies at most universities. The study observed significant differences in the level of mastery among the competencies, and variations in ranking between the university of the authors and the national average. Discussion and Conclusion: By examining data disclosed in graduation competency questionnaires for universities, converting them into KPIs became possible. This method may also be used to convert other monitoring data into KPIs.

Consistency of the I-E-O-L Model and the Guidelines for Academic Management

Sayaka Matsumoto — 2024-09-15

The Guidelines for Academic Management (henceforth referred to as the Guidelines) issued by the Ministry of Education, Culture, Sports, Science and Technology (MEXT) require Japanese universities to foster autonomy in learners and university management. As such, one of the ex-pected management processes is to establish institutional research (IR) on teaching and learning to grasp and visualize academic and educational outcomes. However, in many cases, the student surveys used to collect information for this purpose are conducted in a disjointed manner based on the business needs of each administrative department of the university, unrelated to IR. In those cases, data tabulation and analysis are completed within each survey, and it is, therefore, expected that the data is not fully utilized as panel data. In our IR practice, we utilize the I-E-O-L model, an extended version of the I-E-O model, to clarify issues and enhance the effi-ciency and sophistication of such student surveys. However, in this process, there have been instances where the cooperation of the various administrative departments was not forthcom-ing. Consequently, it is necessary to ascertain the consistencies between the I-E-O-L model and the Guidelines and to evaluate the efficacy of utilizing them as a foundation for IR staff to be engaged in student surveys sponsored by each administrative department and to seek collaboration.

Launch Out on a Practical Platform for Institutional Research Toward Sharing of Its Technology and Knowledge

Shotaro Imai — 2024-02-01

Terenzini classified the i ntelligence required f or an Institutional Research (IR) conductor into three tiers. “Technical and analytical intelligence” is a general technique of statistical analysis but the other two are difficult to learn s ince they r equire experience in IR and a deep understanding of the institute to which each belongs. In addition, there is no definitive definition of IR in J apan. These facts confuse IR b eginners. To overcome the problem, we launched a platform for sharing technology and knowledge of IR. An IR beginner can access to learn the methodology of IR and an IR expert can share ingenious ideas and techniques through the platform. We named the platform PAIR (Platform of the Art of Institutional Research) and provided it by GitHub. In this paper, we describe the background, the concept, and the future visions of the PAIR.

Quantitative Quality Evaluation of the Impact of Indentation in Source Code Using Eye-Tracking

Kou Yorimoto — 2024-09-15

This study focuses on the setting of indentation and aims to elucidate its impact on reada-bility through the analysis of program comprehension processes using eye tracking. Within the workload of software lifecycle activities, maintenance tasks are known to occupy a sig-nificant proportion. Among the various stages of maintenance, understanding the content of source code, namely comprehension, is considered the most time-consuming task. Against this backdrop, the ability to comprehend source code is recognized as an important pro-gramming skill. Alongside comprehension, awareness of source code readability is also considered a vital aspect of comprehension learning. Factors influencing the readability of source code include code structure, naming conventions, presence, and quality of com-ments, as well as indentation and placement of parentheses. However, insufficient quantita-tive research has been conducted to demonstrate the impact of these factors on readability. Therefore, this study focuses on the influence of indentation on readability and analyzes the program comprehension process using eye tracking. The results suggest that in the case of small-scale source code, the absence of indentation may not adversely affect slicing.