Two Purdue CS Professors Awarded Facebook Probability and Programming Research Awards

07-01-2019

Computer Science Associate Professor Lin Tan and Assistant Professor Tiark Rompf were each awarded a Facebook Probability and Programming Research Award for their individual projects.

Facebook rewards forward-looking research with $50,000 grants for projects exploring the intersection of programming languages (PL) and machine learning (ML). Out of 66 submissions, 10 awards were chosen, with Purdue Computer Science being one of just of two institutions with multiple awards. 

Tan’s project titled, “Contextual Ensemble Learning for Software Productivity and Reliability,” is a continuationof her long-term research program on the synergy between software dependability and ML, which consists of two directions.

The first direction uses ML approaches to improve software dependability. The second direction leverages software approaches, such as testing and verification, to improve the dependability and quality of ML systems.  

Tan describes her research, “There is a high demand for novel and effective techniques to detect and fix bugs and synthesize source code to improve software dependability and engineering productivity.” For such techniques, the context of the problem is crucial. As Tan explains, “The same source code statement can be correct in many locations and projects, while it is buggy or incorrect in many other locations. Whether it is buggy depends on the context, (e.g., surrounding code and the code that has data and control-dependency relationships with it.)” 

Tan’s research proposes designing new algorithms to represent context to enhance traditional encoder-decoder neural machine translation architectures. The context and the target code each have their own encoder for different representations. Additionally, her research proposes to use ensemble learning to build multiple models and combine them to learn different types of bugs, fixes, and programs to improve effectiveness. “We are excited to contribute to approaches and algorithms that bridge software engineering and machine learning,” said Tan. “We anticipate that the approaches we propose will also be beneficial for application domains other than software dependability and software engineering.”

Rompf’s project titled, “Higher-Order Differentiable Programming with Delimited Continuations” addresses the increasing demand for efficient automatic gradient computation for emerging deep neural network architectures which incorporate dynamic control flow, especially in Natural Language Processing.

Training of deep learning models depends on gradient descent and end-to-end differentiation, typically achieved in a process called backpropagation or reverse-mode automatic differentiation. In Rompf’s research he proposes an implementation of backpropagation using functions with callbacks, where the forward pass is executed as a sequence of function calls and the backward pass as a corresponding sequence of function returns. A key realization is that this technique of chaining callbacks is well known in the PL community as continuation-passing style. Any program can be converted to this form using standard techniques. Therefore, any program can be mechanically converted to compute gradients.

Rompf’s approach achieves the same flexibility as other reverse-mode automatic differentiation techniques, but it can be implemented without any auxiliary data structures besides the function call stack. Also, it can easily be combined with graph construction and native code generation techniques through forms of multi-stage programming, leading to a highly efficient implementation that combines the performance benefits of define-then-run software frameworks, such as TensorFlow, with the expressiveness of define-by-run frameworks like PyTorch.

“We’re at an exciting point in time where foundational ideas from PL research find their way into cutting-edge machine learning and artificial intelligence systems,” said Rompf. “This new project with Facebook will allow us to push the boundary in this space, together with one of the industry leaders.”

Rompf is a co-director and Tan is a member of PurPL - The Purdue University Center for Programming Principles and Software Systems. The research center aims to connect fundamental research in PL and software engineering with domains such as artificial intelligence (AI) and cybersecurity. The founding faculty includes 10 experts in PL, covering all areas from theory to systems, as well as 9 experts in domains spanning AI, ML, security, cryptography, as well as computational science and engineering, with a proven track record of successful collaboration and tech transfer. The center is supported by a consortium of industry partners including Facebook.

 

Writer: Emily Kinsell, 765-494-0669, emily@purdue.edu, @emilykinsell

Sources: Lin Tan, lintan@purdue.edu

Tiark Rompf, tiark@purdue.edu