Four Computer Science Ph.D. students who worked in Rice’s Optimization Lab (OptimaLab), launched and directed by Anastasios “Tasos” Kyrillidis, a Noah Harding Associate Professor in the Computer Science department, have graduated and accepted roles at prestigious companies over the past year.
These alumni, “exemplify the successful fusion of rigorous academic training and impactful industrial application,” according to Kyrillidis.
OptimaLab specializes in increasing the efficiency of very complicated calculations. Kyrillidis said optimization is at “the heart of recent computational advances in machine learning (ML) and artificial intelligence (AI).”
The research conducted by Ph.D. alumni John Chen, Lyle Kim, Cameron Wolfe, and Chen Dun while at Rice included “high-impact papers and critical advancements in both theory and application,” said Kyrillidis. “And their diverse yet complementary expertise—from enhancing machine learning algorithms and exploring quantum computing to pioneering new methods in federated learning—demonstrates the lab's commitment to addressing a broad spectrum of technological challenges.”
John Chen
Chen turned down offers to complete his Ph.D. at other prestigious universities so he could continue his research at Rice under Kyrillidis’ mentorship. “Tasos was very encouraging and supportive for me to pursue my own line of research,” said Chen. “He emphasized the importance of reminding ourselves to work on things that we are interested in, which has become a consideration in my current job.”
Chen has worked at Meta for the past year, specifically focused on AI models for Facebook Marketplace. “I work on a product team,” Chen said, “which means we touch a lot of different ML models. Fortunately, at OptimaLab, I learned how all these different models work and how to train them.”
He is fascinated by all the subtle, under-the-radar implementations that “make our everyday lives better,” and he “admires the technical challenges of increasingly complex forms of AI, such as ML, deep neural networks (DNNs), and generative adversarial networks (GANs).”
Kyrillidis believes that Chen’s involvement in OptimaLab projects such as "Demon: Improved Neural Network Training with Momentum Decay" and "REX: Revisiting Budgeted Training with an Improved Schedule" has “pushed the boundaries of what is achievable in neural network optimization and training efficiency.”
At Meta, Chen’s work includes using “advanced computer vision models to help evaluate and improve images, and using large language models to parse and understand marketplace listings to extract relevant information that can help improve relevance on Marketplace. We also use a variety of machine learning models, which help us understand a user's preferences and listing information to better match users with listings.”
According to Kyrillidis, “John's work highlights a career that bridges theoretical research and practical application seamlessly. His transition from academic settings, where he explored semi-supervised learning and GANs, to industry roles where he applies these concepts to real-world problems, underscores the practical relevance of his research.”
Junhyung Lyle Kim
Kim recently joined Global Technology Applied Research at JPMorgan Chase as a quantum computing research scientist. Its areas of focus include the fields of Quantum Technology, Augmented and Virtual Reality, Cloud Networking, Internet of Things, Blockchain and Cryptography.
Kim’s research interests lie in the intersection of optimization, quantum computing, and AI. For instance, he said, there is speculation that “quantum computing might help with non-convex optimization, which is extremely challenging in classical computing; decoding this speculation would broaden our understanding of both optimization and quantum computing.”
Kim has said that studying optimization often feels like he is “learning a secret language.” Optimization, in a broad sense, is the subject of studying how to minimize, or maximize, a function. An example Kim uses is when we travel: “we try to minimize the traveling time or distance. Similarly, in AI and ML, we try to minimize the loss function – the fit between the observed data and the model. Studying rigorously when and how to optimize a function is tremendously empowering knowledge to have, as I can apply it to virtually any setting.”
Kim arrived at Rice from the University of Chicago with a mathematics and statistics undergraduate background; he never imagined he’d be conducting research about quantum computing. Kyrillidis was working on an optimization problem, called quantum state tomography, related to quantum computing when Kim joined the OptimaLab. “One of my first projects was to accelerate this protocol using a more sophisticated optimization technique,” Kim said. “This was my introduction to the field of quantum computing. I was immediately hooked, and here I am now working as a quantum computing research scientist!”
Kyrillidis believes Kim’s research is set to “influence next-generation AI technologies in profound ways. His work, such as "Fast Quantum State Reconstruction via Accelerated Non-Convex Programming," has been pivotal in advancing the understanding of quantum state tomography.” Kyrillidis noted that Kim’s papers “reflect a rigorous approach to tackling some of the most challenging problems in AI and quantum computing. The breadth of his research, from stochastic algorithms to federated learning, highlights his versatility and commitment to pushing the frontiers of knowledge.”
Kim said that “Tasos taught me the power of perseverance, positivity, and being open-minded. He showed by example — if the manuscript is rejected, it is an opportunity to improve it. With such positivity one can persevere, and eventually the improved work ought to get published. I still operate in this mindset, and instilling it was one of the biggest assets I learned from Tasos.”
Cameron Wolfe
Like John Chen, Wolfe chose Rice specifically to work with Kyrillidis. He had conducted some optimization research as an undergraduate at The University of Texas at Austin and wanted to continue that work under Kyrillidis’ tutelage.
“My research with Tasos had a massive impact on where I am now,” said Wolfe. “Most of my work now [at Netflix] focuses on training and hosting large neural networks. My knowledge of this area began with my work on independent subnetwork training (IST) with Tasos, then continued with my research on neural network pruning, quantization, and other areas.”
According to Kyrillidis, Wolfe's ascent to a Senior Machine Learning (ML) Scientist at Netflix, following his tenure as Director of AI at Rebuy Engine, is a “testament to his ability to translate complex theoretical concepts into actionable, scalable AI solutions. His research during his Ph.D. at Rice, particularly in non-convex optimization, efficient neural network training and online learning, laid a solid foundation for his professional pursuits.”
Wolfe is part of Netflix’s Promotional Writing Data Science and Engineering team, where he works “on tools for creating promotional copy, including summaries and descriptions of shows.” He must be careful what he shares about his work externally, but Wolfe explained that as a senior ML scientist, he helps “define the direction for algorithmic strategies and spearheads the execution of ML solutions, ultimately shaping how titles are promoted on Netflix and discovered by its members.”
Wolfe believes his two biggest academic highlights were earning an oral presentation about his work on neural network pruning at the Learning for Dynamics & Control Conference (L4DC 2022), where less than 5% of papers were accepted, and traveling to Singapore to present his work on hyperparameter tuning for neural networks at the International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2022), hosted by the Institute of Electrical and Electronics Engineers (IEEE).
Kyrillidis noted that Wolfe’s significant contributions to the field while he was at Rice include “influential papers such as ‘GIST: Distributed training for large-scale graph convolutional networks’ and ‘Cold start streaming learning for deep networks’ which have not only advanced the academic community's understanding but also had direct implications for industrial applications in AI-driven content promotion at Netflix.”
“His quick rise through industry ranks and his impactful publications, achieved in a remarkably short span, showcase his ability to handle fast-paced, high-stakes projects in machine learning,” said Kyrillidis.
“Above everything,” said Wolfe, “Tasos taught me to be resilient. Any time we faced adversity, Tasos would tell us to ‘be like a tree.’ We had to grow thick skin and be hard to damage.” Early in his academic career, Wolfe struggled with how competitive the field is and how difficult it is to get papers accepted. “Over time, thanks to Tasos, I learned to deal with it and move forward.”
Chen Dun
Dun currently works at ByteDance, which has a suite of more than a dozen products and services, including TikTok, CapCut, TikTok Shop, Lark, Pico, and Mobile Legends: Bang Bang. ByteDance’s mission is to inspire creativity and enrich life.
According to Kyrillidis, Dun has “carved a niche for himself with his focus on optimizing ML processes for large-scale applications. His work, particularly in federated learning and distributed training systems, addresses some of the most pressing challenges in AI today.”
Collaborating with Microsoft Research as a member of the OptimaLab, Dun’s publications, such as "FedJETs: Efficient Just-In-Time Personalization with Federated Mixture of Experts" and "Sweeping heterogeneity with smart mops: Mixture of prompts for LLM task adaptation,” demonstrate “his innovative approach to making AI more efficient and personalized,” said Kyrillidis.
Dun’s “leadership in developing new methods for distributed learning,” Kyrillidis pointed out, “showcases a commitment to advancing AI while considering privacy and efficiency, aligning with modern technological needs and ethical standards.
Looking Forward
As a group, Chen, Dun, Kim, and Wolfe illustrate the power of a collaborative and innovative research environment, where each member has been able to explore complex ideas and translate these into significant technological advancements.
“The success of these graduates not only enhances the reputation of Rice’s OptimaLab,” Kyrillidis pointed out, “but also sets a benchmark for future research and development in the tech industry, ensuring that the lab's influence will be felt for years to come.”