"In a seminar during my first year of grad school, I heard Anshumali Shrivastava talk about Locality Sensitive Hashing (LSH), a classical randomized algorithm from the ‘90s derived by theoreticians. After that semester, I joined Anshu's lab and started my Ph.D. journey — randomized algorithms for efficient deep learning," said Beidi Chen, a Rice University Computer Science alumna (Ph.D.' 20) who will begin her faculty role at Carnegie Mellon University (CMU) after her postdoctoral research with Chris Ré at Stanford University.
"The most important thing I learned from Anshu and Chris is that many of today's technical advancements are rediscoveries of existing techniques from the literature but on new exciting settings and applications," she said.
"For example, LSH has been out there for 25 years, and I've been working on it for 3-4 years. Also, 2018-2019 was the ‘right’ time to use it for speeding up neural network training because the deep learning model size has been exponentially growing. It was a very timely and demanding publication, and it was one of my first experiences in redesigning the existing technology on CPU to approach emerging problems.
"More recently, I explored Butterfly Matrices, which have existed for more than 50 years, and our group from Stanford has worked on it since 2017. We can speed up neural network training by redesigning these matrices and taking advantage of the power of graphical processing units (GPUs). My experiences with LSH and Butterfly Matrices show me how all research areas are connected; being able to collaborate with researchers in many fields is beneficial to all computer scientists."
Two key reasons Chen decided to stay in academia were the freedom to pursue different or even risky research directions that interest her, and the opportunity to collaborate with various industrial partners. Chen said her background lends itself to different research directions in algorithms, theory, modeling, systems, and hardware. In addition, time spent mentoring students is precious to Chen and another driving factor behind her pursuit of a faculty career. "Becoming a faculty member is similar to being a CEO because you are responsible for your entire research group," she said.
"I can branch out in so many different directions. Research is a lot like launching startup after startup because you go through a full cycle of ideation, exploration, testing, and publishing in one to two years, and then you begin the cycle again.”
That sense of freedom continues to fuel Chen's enthusiasm for research and innovation. She is currently tackling an optimal transport problem and considers herself to be in the pre-theoretical stage of the research, learning the theory and how implementation usually occurs. Then she plans to determine how it can be used in computer vision, biology, and other areas.
Chen explained why she likes her current research community, saying, "Deep learning has been dominating the machine learning (ML) community, and now there are opportunities for applying it to compilers, networks, and even other sciences.
"The ML community is very welcoming and inclusive. As long as you are doing something interesting, everyone is happy to hear about your technique and consider how they might apply it in their area. I just joined Meta as a visiting researcher and met a physicist in our group. It was exciting to hear how he explains work that is similar to mine and see how it impacts other fields."
Chen said she benefited from a series of talks sponsored by Rice’s computer science department. Each Monday in the spring and fall, CS Ph.D. students present short talks about their research to their peers. The seminar series is part of the student communication training as well as a way to showcase the breadth of work in the department. In the department's colloquium series, external researchers give longer and more formal talks, exposing the graduate students to additional topics that might impact their existing work or provide inspiration for new work.
"The opportunity to interact with researchers outside of Rice —true experts in their fields— helped me identify the many different areas where our algorithm work could be applied," said Chen. "And the weekly seminars showcasing the work of our peers were eye-opening for all of us. We learned how to communicate our ideas with others, but we were also learning about cutting-edge research from people in our same building!"
Chen's career has been strongly influenced by Shrivastava and Ré. In 2018, Shrivastava took a sabbatical to work on his research in conjunction with Amazon Search (formerly known as A9) in Palo Alto, California. His research team went with him. "When I talked with graduate students at other universities, they could not believe we were researching with our advisor at Amazon," said Chen. "This was two years before the pandemic; working on your Ph.D. two thousand miles away from your university was rare.
"Beyond our required degree work, being with Anshu in Palo Alto provided the opportunity to communicate with our peers in the industry and find out what they really need, what they are working on or trying to accomplish, and then reshape our research to fit those goals. It was an amazing experience for our whole team."
That Amazon experience was just one of many examples. She said her advisor seemed to enjoy really getting to know each of the graduate students he advised and was careful to listen to them, ask questions beyond their research, and find the best ways of supporting their Ph.D. journey. "Anshu was able to support each of us in the way that we needed. One year, I identified three or four conferences where I wanted to present a paper, and Anshu was fully on board with that." Chen said.
"As a postdoc at Stanford, the expectation is to help Chris manage the group and mentor the junior Ph.D. students. While further improving my technical skills and working on my research, the postdoc experience is a chance to expand my vision and acquire more of the team management skills necessary for academia."
She said, "I got to see the full cycle — from entering graduate student to senior graduate student and mentor. I still keep in touch with some of the students I mentored, and the experience helped prepare me for the faculty job at CMU.”