Case Study: Singapore University of Technology and Design (SUTD)
How SUTD is Using Vocareum to Teach Computer Programming

Successfully conducted high-stakes performance-based exams for 400 students concurrently


As computer programming continues to rise in popularity, innovative universities around the world are looking for creative ways to teach large numbers of students at one time. At the Singapore University of Technology and Design (SUTD), for example, computer programming is now one of the most popular courses on campus, attracting hundreds of new students each year. That spike in demand has inspired the SUTD to look for new ways to handle every aspect of this course, especially the administration and grading of programming quizzes and exams.


Dr. Oka Kurniawan, Senior Lecturer at SUTD

Vocareum recently sat down with Dr. Oka Kurniawan, the lead instructor for the university’s compulsory “Digital World” course which uses Python for teaching computer programming, to discuss how the SUTD uses Vocareum to streamline the teaching of so many students at one time and improve the overall classroom experience.

Can you tell us a little about the “Digital World” course?

At SUTD, all students – including those from fields such as architecture – have to learn computer programming, which is the first introduction to computing for all the students. It’s a compulsory course here, and the current batch is around 400 to 500 students, making it the largest computer course in our university.

We don’t have big lectures, but we do have a large amount of common classroom teaching. For example, in one class, it’s about 40 students, and there are usually two instructors in a class, plus one undergraduate teaching assistant. So, normally we have three instructors in a class, and we try to apply some flipped classroom learning where we ask the student to look into the materials before they come to the class.

How does Vocareum fit into that classroom structure?

Vocareum is part of the flipped classroom model at SUTD. For example, we have some online videos and online textbooks. And then when students come to the class, they work on the problems. That’s where we give them exercises, and we discuss how to solve the problems. So, in this way, they write their code in the class where they can consult with the instructors. This is also where they use Vocareum. They use it in the classroom. They also use Vocareum to submit their homework, programming quizzes, and exams.

How do you use Vocareum for performance-based tests and exams?

With every exam, one part is typically written questions and conceptual problems, and the other part generally is programming. That’s where we use the Vocareum system for students to submit their answers as well as to check them using the automated system. That’s why it’s essential for us to have the auto-grading features in the Vocareum system. If we had to grade everything manually, that would be very overwhelming.

We also use the platform for programming quizzes, which happen almost every week. Different class sections have different timings and, fortunately, the system can handle this easily. The tricky time is during the midterm and final exam because all 400 students are having exams at the same time. I can see how it is very challenging to scale up the system so all students can access and work on the exam at the same time.

In previous years, we used our own online platform to handle students’ submissions. But, we found Vocareum to be much more maintainable and has better features. Using Vocareum really helps us: we know who submitted, at what time they submitted, and then we can run an auto-grading job immediately or later on.

We can adjust the settings, and I think it is advantageous also for students. Students can even get feedback from the system. We can comment on the students’ code, and the students can see the comments also from the system. We like the system because it has specialized features that are more suitable for programming assignments. In short, Vocareum really suits the programming and assignments that we would like to offer.

What is the impact of Vocareum on your course??

We are looking into whether we can help students earlier in their course because one of our main challenges is that the background of each student varies significantly from those who have never programmed before to those who have written hundreds to thousands of lines of code. Some of them already learned to program in their polytechnic or on their own. But, others have zero background. So, this is a challenging course to teach because of the wide spectrum of students that we have.

One of the things that we would like to do is to have a monitoring system where we can do early intervention for students who need our help in the earlier weeks. Previously we could only get the performance indicators after the midterm, but we like to do it much earlier. If we can do it in the middle of the first half of the semester, we can do some analytics to the point where we can see if the students need our help. If they do, we can direct them to something like consultation hours that we already have.

What kind of data do you collect through Vocareum?

We are trying to look into two things. One is student activity. We would like to see through the student activity if we can make some correlation as to their performance. So, for example, maybe the better students submit earlier in the week, whereas the weaker students submit later and perhaps they click the run button more often than better students. The other is looking at code quality and similarity among student submissions.  With all this data, hopefully, we can classify students earlier in the semester and provide assistance for those who need it.

As Dr. Kurniawan points out, cloud computing solutions such as Vocareum are revolutionizing the way universities offer programming courses to young undergraduates. For one, it’s making it easier for students, irrespective of which devices they use, to connect on one uniform platform. And, secondly, it’s ensuring that universities, regardless of size or resource level, can meet the growing demand for their courses. Going forward, it’s easy to see that cloud computing solutions will have a major impact on course design and the way students learn.


Screen Shot 2022-10-11 at 4.06.51 PM