Melissa Pappas
Usually taken by first-year graduate students in robotics, “Intro to Robotics” is a course on modeling and planning for how a robot will turn digital code into physical action. Traditionally, the course also provides students with what is often their first hands-on experience in the field through programming a physical robotic arm over the course of multiple exploratory labs. However, as with all courses in the fall 2020 semester, this experience had to be redesigned to work entirely online.
This meant that Cynthia Sung, Gabel Family Term Assistant Professor in the Department of Mechanical Engineering and Applied Mechanics (MEAM), and her head teaching assistants (TAs) had a major task on their hands in August: preparing a virtual simulator that could replicate the hands-on elements of their robotics laboratory.
“The goal was to have this simulator working bug-free no less than two weeks before the start of the course, a small window for error for the functioning of a tool which was to be the pillar of the class assignments and lessons,” says Sung. “We weren’t starting entirely from scratch, but to meet the purposes of our class, we had to do a lot of work.”
The simulator for this course had been designed for a summer course that provided a basic simulation experience for 15 students. To get the simulator up to speed for a class of 80 who needed to use it as a complete replacement for an in-person robotic experience, the TAs had to improve the physical model, visualization, and aspects that would allow the robots to respond to their simulated environment. They took the basic bones of the first simulator and advanced it on multiple levels to fit the needs of their course.
However, a simulator is an inherently challenging environment to learn about real robotic functions. A real robot’s motion is controlled by the code programmed into its central processor, easy enough to replicate in a digital simulation, but the execution of that code is constrained by its physical environment. The class’s simulator was designed to mimic these limitations, but due to its virtual nature, it was hard for students to discern the extent to which it represented the real world.
“It was difficult to show this real-world perspective using an online simulator,” says Jake Welde, a TA for the course and doctoral candidate in MEAM. “With a physical robot, students already understand that there’s a gap between theory and practice. Over the semester, students came to understand that even a state-of-the-art simulator sits somewhere between simple, idealized models and the real world. Understanding which physical phenomena are being modelled or ignored is key to predicting whether your code will work as expected.”
“Normally, the course would introduce students to manipulator arms moving in 3D space, where they could see how robots respond to their environment and physical constraints,” adds Sung. “With the simulator as a new tool for our class, we were able to turn on and off real-world aspects that could allow for a limitless environment. For example, some of our first labs included models with no gravity or self-collision, allowing students to explore the robot better and control the complexity of the simulation. This flexibility of the simulator allowed for students to take risks and be creative in ways they might not have tried in real life.”
As students got used to this exploratory environment, they encountered another layer of complexity. Like any piece of software, the simulator had its own set of unintentional bugs, which were another potential source of confusion for students writing their code.
In time, however, students learned to recognize the various reasons why a piece of code failed: either there was an error in their code, a bug in the simulator stopped correct code from functioning, or a real-world limitation aspect of the simulator was turned on and prevented something physically impossible from happening.
Although challenging and time consuming, students ended up working with the TAs to find and fix the bugs in the simulator and to better understand the ways its unrealistic aspects could be taken advantage of.
“Many students even found that the use of the simulator was good practice for virtual problem solving in future jobs in academia and industry, which was one of the benefits of being forced to use it,” says Sung.
Another challenge of the online version of the course was encouraging student collaboration, which this course usually thrives from in person.
“We had hoped that students would learn from one another, as that has been a key piece in this course. For students that had never been to Penn before, it must have been challenging to have their first semester be virtual,” says Sung. “We did our best as instructors to help students foster collaborations, making teams so that they could work together on assignments and get to know one another.”
The online format did provide forms of student engagement that an in-person course could not.
“Surprisingly, I typically had ten or more students come to my office hours to work on homework or to test code. I imagine that because office hours were virtual, in some ways they were more accessible to students, allowing them to just open their laptops to get help even without planning ahead to show up in person,” says Welde.
As students became more confident with the online simulator, and the instructors became more comfortable with teaching through Zoom, Sung’s TAs, Welde and fellow TA Shane Rozen-Levy, a doctoral candidate in MEAM, spearheaded the design of the final project. They devised a block-stacking competition using the simulator’s virtual robotic arm, in which students were placed in teams and were left to collaborate with their own creativity to develop winning strategies.
“We quickly learned that the online simulator could be manipulated to turn off some of the real-world limitations and we could create code that would make the robot move very fast without worrying about the damage from collisions,” says Zador Pataki, a master’s student on the winning team. “We took advantage of both the system and the theories we learned in class. That was the kind of mentality you needed to win.”
Unexpected strategies highlighted the benefits of the simulated environment that proved to be fairly effective.
“One team’s strategy was to throw blocks at the opposing team’s stack. We didn’t expect students to actually pursue that avenue, but it was exciting to see the diversity and effectiveness of the different strategies,” says Welde. “Even though it was not in person, student engagement was very high during the final competition. Many students were conversing about the strategies they used and how they implemented them. It was refreshing to see them work together and really get the hang of working with robots.”
Although the robots in the competition were not real, the time invested, the teamwork, and the strategies were. Even with these challenges, students felt accomplished and satisfied with the outcome.
“Having to work on a team project completely online was quite challenging. Debugging and code integration are simpler in person, and working online meant many hours spent on Zoom calls trying to explain the function of each chunk of code while the simulation crashed in the background,” says Vera Lee, a master’s student on the winning team. “It was still an incredibly fun experience, though. I didn’t know any of my teammates before taking the course, but the many hours spent working on the project made us fast friends.”
The course proved to be successful in an online format and even gave Sung a new perspective on how to teach the basics of robotics.
“We decided that the simulator, although challenging and sometimes frustrating for students, is actually a great learning and practice tool for working with robotics in the physical realm. We would plan to work with the simulator even in an in-person version of this course,” says Sung.
Despite the challenges of going remote, the course did achieve its learning outcomes.
“The fact that this course was online did not mean that we weren’t able to learn how to program real robots. It actually taught us that there are diverse ways to approach a problem,” says Ayaaz Versi, a senior on the winning team. “Once you know how to problem solve, you can adapt that practice to whatever situation is at hand, whether it be a physical robot or an online simulator, and this course really strengthened our problem-solving skills in many ways.