HP Labs Blog

HP Labs

Summer 2019 Interns at HP Labs – Anika Cheerla

By Simon Firth, HP Labs Correspondent

August 15, 2019

Anika Cheerla joins MIT as an undergraduate this fall where she plans to build on her love of math and computer science. A graduate of Monte Vista High School in Cupertino, California, Cheela is interning this summer in HP’s Artificial Intelligence and Emerging Compute Laboratory where she is working on new approaches to teaching robots how to learn. When she’s not studying or working, Cheerla is an accomplished water polo player. She picked MIT despite the fact that they don’t (yet) have a women’s water polo team.


HP: Tell us what you are investigating this summer

I’m working on robust reinforcement learning. Traditional reinforcement learning is a method via which we get robots to learn on their own. But we have to do most of that training in simulations which can result in what we call “overfitting,” where the robot can’t apply what it learned in the simulation to real life because it's trained only on the simulation. I'm working on finding a way to bridge the gap between simulation and real life by making a more general training algorithm that is less tied to any one simulation.

HP:  What does that require you to do?

Essentially, I’m varying the simulation through something called curriculum learning, where you slowly change parameters like the amount of friction in your simulation to account for the fact that friction can vary in real life situations. I worked on implementing that on several different robot simulations for the first couple of weeks. Then more recently I’ve been working on my own idea, which I call adversarial environment generation.

“The main robot that I've been training … can walk easily on a flat surface, but I’ve been trying to get it to learn how to walk on more difficult terrain like an area with stumps or stairs.”

Anika Cheerla, HP Labs Intern

HP: Tell us about that

This is something that you see in computer vision research, but I haven’t seen applied to reinforcement learning for robotics. The idea is that instead of randomly selecting the environments that a robot has to deal with, you do that intelligently. You deploy a superior learning agent that’s able to determine what kind of environments the robot is currently having difficulty with and then feed those environments to it. That means your robot trains only on the harder environments, so it learns faster.

HP: What sort of jobs are these robots training for?

It really varies. The main robot that I've been training is a bipedal walking robot. It can walk easily on a flat surface but I’ve been trying to get it to learn how to walk on more difficult terrain like an area with stumps or stairs.

HP: How’s the project been going?

Pretty well. I still need to fine tune a lot of my hyper-parameters like how many times we ask the robot to re-started the learning process, but it looks like this method works at least as well as regular curriculum learning on the bipedal walker – and I’m hoping that with more fine tuning, I can show that it’s even better than that.

HP: What's been the biggest challenge for you in doing this work?

I didn't know anything about reinforcement learning coming into this – I’d just done some competitive programming along with a couple of projects in standard machine learning and computer vision, which weren’t as challenging as this work. So there was a very steep learning curve, and I still feel like I have a ton of gaps in my knowledge. But I also feel like I’m starting to think the way an actual reinforcement learning researcher would think about these problems.

HP: Has your experience at HP Labs changed how you are thinking about what you might study in college?

Definitely. The researchers I’m working with in the lab have very diverse engineering backgrounds. That’s made me aware that robotics is a very multidisciplinary field and that it’s not a bad thing at all to have a wide variety of experiences to draw on. So while I came in thinking that I was just interested in computer science and math, I’m now I’m super interested in taking a lot of other classes too.