Back to all keywords
Large language models
4 items tagged with “Large language models”
Publications
- NivTA: Towards a Naturally Interactable Edu-Metaverse Teaching Assistant for CAVEEdu-metaverse is a specialized metaverse dedicated for interactive education in an immersive environment. Its main purpose is to immerse the learners in a digital environment and conduct learning activities that could mirror reality. Not only does it enable activities that may be difficult to perform in the real world, but it also extends the interaction to personalized and CL. This is a more effective pedagogical approach as it tends to enhance the motivation and engagement of students and it increases their active participation in lessons delivered. To this extend, we propose to realize an interactive virtual teaching assistant called NivTA. To make NivTA easily accessible and engaging by multiple users simultaneously, we also propose to use a CAVE virtual environment (CAVE-VR) as a "metaverse window" into concepts, ideas, topics, and learning activities. The students simply need to step into the CAVE-VR and interact with a life-size teaching assistant that they can engage with naturally, as if they are approaching a real person. Instead of text-based interaction currently developed for large language models (LLM), NivTA is given additional cues regarding the users so it can react more naturally via a specific prompt design. For example, the user can simply point to an educational concept and ask NivTA to explain what it is. To guide NivTA onto the educational concept, the prompt is also designed to feed in an educational KG to provide NivTA with the context of the student’s question. The NivTA system is an integration of several components that are discussed in this paper. We further describe how the system is designed and implemented, along with potential applications and future work on interactive collaborative edu-metaverse environments dedicated for teaching and learning.
- HiBench: Benchmarking LLMs Capability on Hierarchical Structure ReasoningStructure reasoning is a fundamental capability of large language models (LLMs), enabling them to reason about structured commonsense and answer multi-hop questions. However, existing benchmarks for structure reasoning mainly focus on horizontal and coordinate structures (\emph{e.g.} graphs), overlooking the hierarchical relationships within them. Hierarchical structure reasoning is crucial for human cognition, particularly in memory organization and problem-solving. It also plays a key role in various real-world tasks, such as information extraction and decision-making. To address this gap, we propose HiBench, the first framework spanning from initial structure generation to final proficiency assessment, designed to benchmark the hierarchical reasoning capabilities of LLMs systematically.HiBench encompasses six representative scenarios, covering both fundamental and practical aspects, and consists of 30 tasks with varying hierarchical complexity, totaling 39,519 queries.To evaluate LLMs comprehensively, we develop five capability dimensions that depict different facets of hierarchical structure understanding. Through extensive evaluation of 20 LLMs from 10 model families, we reveal key insights into their capabilities and limitations: 1) existing LLMs show proficiency in basic hierarchical reasoning tasks; 2) they still struggle with more complex structures and implicit hierarchical representations, especially in structural modification and textual reasoning. Based on these findings, we create a small yet well-designed instruction dataset, which enhances LLMs' performance on HiBench by an average of 88.84\% (Llama-3.1-8B) and 31.38\% (Qwen2.5-7B) across all tasks.The HiBench dataset and toolkit are available
- PockeTA: Virtual Teaching Assistant Platform with Course ContextTeaching assistants (TAs) are essential for scalable, high‑quality university education, yet most institutions face persistent constraints in TA-to-student ratios, limiting timely feedback and personalized support. At the same time, large language models (LLMs) are rapidly being adopted in education, but typical deployments treat them as generic chatbots, lacking course awareness, pedagogical grounding, and meaningful integration with instructors’ workflows. In this work, we present PockeTA, an ubiquitously accessible virtual teaching assistant platform that tightly couples LLM-based multi-agent intelligence with a teacher-controlled, graph-based knowledge base (K-Cube). PockeTA provides a personified avatar interface on both desktop and mobile, supports multimodal interaction (text, document sharing, interactive drawing), and is deeply aware of course structure and materials through K-Cube integration. Student interactions are continuously collected, organized as learning analytics, and visualized to help instructors monitor learning progress, identify misconceptions, and refine teaching. We describe the overall system design, including its multi-agent architecture and knowledge-graph integration, and report initial findings from a beta deployment with university students. Early user feedback indicates that PockeTA complements human TAs by providing always-available, context-aware support, while giving teachers unprecedented visibility and control over the virtual assistant’s behavior and course context. Our results suggest that PockeTA offers a promising path toward scalable, course-grounded virtual TAs that go beyond generic LLM interfaces to support both learners and educators.
- Towards AI-Assisted Immersive Learning:Factor Analysis of Learning Effect in K-CubeEdu-MetaverseThis study examines the impact of an AI-powered Virtual Teaching Assistant (NivTA) within a VR-based Edu-Metaverse (K-Cube), highlighting the roles of social presence, trust, and engagement in shaping learning outcomes. Grounded in Social Presence Theory, the Uses and Gratifications framework, and the Cognitive-Affective Theory of Learning with Media (CASTLE), our AI-Assisted Immersive Learning Framework emphasizes both cognitive and affective dimensions. A user study with 21 participants in a Cave Automatic Virtual Environment (CAVE) setting collected quantitative and qualitative data on trust, social presence, engagement, workload, and learning performance. Partial Least Squares Structural Equation Modeling revealed that heightened social presence fosters trust, which in turn drives behavioral, cognitive, and affective engagement. Notably, cognitive social presence was directly linked to better knowledge test scores, while confidence in test responses stemmed primarily from all forms of engagement. Overall, these findings underscore the significance of nurturing trust and social presence to enhance learner engagement and outcomes in AI-driven immersive educational environments.