For Instructors: Google Colab now with auto-grading, powered by Vocareum.

Create a Free Course
Logo-wrap
Logo-wrap

Log in

Request a Demo

 GPS UC San Diego

Uncategorized
News
Case Study

New AI Support Tool for Math Prep Instruction at GPS

  • The challenge: Many incoming Global Policy and Strategy (GPS) students require a refresher on math concepts they may have not seen in years, and others must learn differential calculus for the first time before the start of the GPS curriculum.
  • A new tool: Developed as a collaboration between GPS, Halıcıoğlu Data Science Institute (HDSI) and Vocareum, Vocareum AI Compass is a personalized learning platform designed to help students learn more effectively through real-time AI support.
  • Results: AI Compass enabled 110 students in two weeks to (a) complete 18,000+ graded multiple choice and free response questions with personalized feedback given, (b) see 846 unique AI-generated questions, and (c) ask the AI Tutor 4175 questions. Engagement with the platform (including interactions with the tutor) is statistically associated with test score improvement between a pre-test and the final exam.
message-square-01 (1)

Background

In the spring of 2024, UC San Diego School of Global Policy and Strategy (GPS) Associate Teaching Professor Gordon McCord along with members of UC San Diego’s HDSI Strategic Research Initiatives Office and the Teaching and Learning Commons met with Vocareum to discuss whether new tools could help address a core pedagogical challenge at GPS. While the GPS curriculum offers a mathematically rigorous graduate-level preparation in international affairs and public policy, students come from a wide variety of backgrounds regarding their previous math training. Many incoming students require a refresher on math concepts they may have not seen in years, and others must learn differential calculus for the first time before the start of the GPS curriculum. For this purpose, GPS created a set of “Prep” courses that a subset of the incoming class is required to take over summer. However, the wide and varied knowledge gaps, as well as the short time available for Prep, creates a challenge in adequately preparing these students for GPS’s curriculum. Our proposition was to leverage generative AI to create a personalized experience to address each learners’ needs. 

To develop a solution, UCSD and Vocareum combined their expertise with faculty insight on current challenges and the best teaching and learning practices to address them, student feedback and experience, and Vocareum’s 10+ years of experience in rapidly developing, iterating, and scaling commercial-grade solutions. This partnership was invaluable in creating the Vocareum AI Compass platform within the year to leverage AI in a controlled and curated manner to meet the needs of students, staff, and faculty for teaching and learning. The first UCSD deployments occurred in summer of 2024 with the first GPS deployment being the Summer 2025 GPS Math Camp Prep course.

Vocareum AI Compass is a personalized learning platform designed to help students learn more effectively through real-time AI support. Through personalized, instructor-controlled AI tutoring experiences, students receive custom feedback and support to master core concepts, overcome knowledge gaps, and build confidence. Faculty can rapidly create content, access real time analytics and AI summaries, and configure AI behavior to best fit their students’ needs. The result: lighter instructor workload, faster student progress, and consistent, equitable support for students.

Vocareum AI Compass student view of an assignment and tutor conversation.
message-square-01 (1)

Development

When conversations first began between GPS and Vocareum, the initial version of the AI Compass platform was being created as mainly a Virtual Tutor. Thanks to our partnership, which included regular meetings with Prof. McCord and the hosting of a focus group with current GPS students, the scope and abilities of the platform grew exponentially. Development included improvement of the virtual tutor to support all languages, context awareness to allow for students to request additional questions and examples with minimal fear of hallucinations, integration of content pages with curated AI tutors for asynchronous preview of content and for students taking courses remotely, and the creation of an instructor digest to provide AI-powered summaries for instructors and Teaching Assistants (TAs) to be able focus their time on what students are struggling with the most. 

The other major area of development was content. Utilizing the AI Compass generation flow, broader mathematical concepts could be refined to match the GPS curriculum and create additional concepts from scratch. This led to the creation of over 800 vetted questions and answers that spanned from introduction to arithmetic to multivariable calculus applications. This was anchored in a visual concept map–tailored to the GPS curriculum and relevant applications–so this large volume of content was navigable for students, allowing them to take advantage based on their needs. 

message-square-01 (1)

Implementation

AI Compass assignments on prerequisite concepts and early Math Camp content were made available early for students to review ahead of time, with the integration of the Virtual Tutor being able to provide guidance and answer questions. Homework and optional review was available during Math Camp, allowing students to seek help with course content at any time. Access for students will continue so that students are able to refer back to math concepts as they complete Econ Prep and other courses in the upcoming academic year.

Concept map for GPS Math Camp
message-square-01 (1)

Capabilities and Strengths

  1. Refined Concept Maps – content is tailored to the GPS curriculum, ensuring content is relevant. Concept relationships allows for the scaffolding of content and provides direction to those with gaps.
  2. Volume of Content – additional questions and practice are available on demand from the tutor, allowing students more support without increases to instructor or teaching assistant workload.
  3. Preview Deployment – early access to embedded instruction and practice problems with integrated AI Tutor support allows for relevant active learning before the course even begins, especially relevant for reentry students and those with gaps in knowledge.
  4. Curated AI Tutor – instructor controlled tutor build to support learning, 24/7 support, personalized feedback on every student response, multilingual support, available alternative to in-person support for those remote or uncomfortable asking questions (math anxiety, sense of belonging, fear over knowledge gaps/level of understanding, etc.)
  5. Instructor support – AI provided instructors with daily summaries of student activity including common questions posed to the tutor, difficult concepts and questions, activity and usage patterns, etc. These summaries serve to help the instructor and teaching assistant tailor in-class content to address concepts with which students are struggling most.
message-square-01 (1)

Results

AI Compass enabled 110 students in two weeks to (a) complete 18,000+ graded multiple choice and free response questions with personalized feedback given, (b) see 846 unique AI-generated questions, and (c) ask the AI Tutor 4175 questions.

Student feedback was used to further improve upon the user experience on the platform, and instructor feedback was used to refine analytics for teaching staff. Feedback included the following comments, highlighting some of the values of AI Compass:

“When it came to homework, the AI Tutor was able to point out the specific step I missed and explain why. Whether it was the patience of the tutor or the privacy of working in my own room, I found it very helpful to ask guiding questions without feeling silly or worried that I was taking too long to reach a conclusion.”

“The AI Tutor gave me a level of confidence I haven’t experienced with in-person tutoring before… It showed me exactly where I was making small mistakes and explained why. The AI Tutor was an invaluable resource that made the learning process less intimidating and more accessible for me.”

“The ability to interact with the AI tutor, ask if I’m in the right direction and if I can’t understand something asking why certain things work the way it does etc. was really helpful in me grasping the newer subjects I wasn’t previously familiar with”

“I would like to see this platform available as an alternative to ChatGPT as we move through our classes.”

Teaching Assistants noted that the daily digest enabled them to focus discussion time on the hardest problems, making pre-planning much easier and the help sessions relevant to more students.

Ideally, we would be able to experimentally test whether AI Compass leads to improved learning outcomes. In the absence of a proper research design, however, we were able to compare student grades in a pre-test (taken on the first day of Math Camp) and the final exam with AI Compass engagement. The engagement with the AI Compass platform (including interactions with the tutor), was statistically associated with test score improvement among students starting in the bottom 75% of the class according to the pre-test. While not proof of the learning impact of the tool, this statistical association between student use of the tool and test score improvement helps motivate future more rigorous estimations of AI Compass’ effects on student learning.

message-square-01 (1)

Next Steps

Given that the GPS Map Prep content has already been developed, this can be used for future cohorts with minimal additional work needed. A potential expansion could allow for more content to be released early and encourage students to complete more guided preparation before they arrive at GPS for Math Camp.

With a strong foundation of relevant math and economics applications done, further development of GPS content, especially the Econ Prep course, is a natural expansion. If funding opportunities for research are available, an encouragement design could be implemented to better measure outcomes tied to the use of the AI Compass tool. Econ Prep and the Fall Microeconomics courses are ideal environments for such research due to their longer duration compared to Math Camp.

message-square-01 (1)

Summary

Thanks to the partnership between UCSD and Vocareum, AI Compass is a personalized learning assistant platform that leverages cutting-edge language models and customizable content to create experiences that align with course objectives and adapt to individual learner needs. The platform’s strength lies in its integrated approach, where faculty or course designers can author or upload content, design assessments based on real time analytics, consult instructor assistants, and configure AI behavior to best fit their students’ needs. Students receive personalized tutoring and assessment experiences. Administrators gain deep insights into learning patterns and outcomes. This comprehensive ecosystem ensures that AI-powered education maintains pedagogical rigor while scaling to meet institutional demands. 


Learn more about AI Compass.

Similar Case Studies

 GPS UC San Diego

 Iowa State University

Miami Dade College