NSF grant supports ballot integrity research

Computer scientists will use machine learning to read hand-marked ballots

Dan Wallach standing outside by trees on Rice University campus

Computer scientists from Rice University, Texas A&M University and the University of Alabama at Birmingham are preparing to use state-of-the-art artificial intelligence to study the robustness and security of election systems that read hand-marked paper ballots.

Rice’s Dan Wallach, working in conjunction with Texas A&M’s Nitesh Saxena and UAB’s Chengcui Zhang, has been awarded a $400K grant from the National Science Foundation to build an artificial intelligence system called Bubble Aid that can read hand-marked ballots. By training on data from millions of actual ballots, the system will use modern computer vision and machine-learning techniques to recognize hand-marked bubble targets more effectively than existing systems.

"I'm very excited to collaborate on a project that brings together researchers from very different areas to look at the seemingly simple problem of scanning hand-marked paper ballots,” said Wallach, a professor in Rice's Department of Computer Science. “When the goal is to get exceptionally high accuracy, and perhaps even detect fraudulent ballots, our research will hopefully devise ways that we can do this better than ever before."

Even the most sophisticated systems used to scan and process hand-marked ballots primarily look at the average darkness across bubble targets on a sheet. As a result, traditional paper ballot scanners can miss partially filled-in marks or misclassify stray marks and scanner noise as filled-in bubbles. Existing systems are also not inherently designed to identify potentially fraudulent voting cases where a single voter has filled out multiple ballots.

Bubble Aid could improve vote tabulation efficiency and security by helping election officials focus their attention on ambiguous ballots that require the most manual attention. The research could also benefit people who use hand-marked forms in other applications, like standardized testing.

This project was part of an overall $1.2M grant. Read more:

“Collaborative Research: SaTC: CORE: Medium: Bubble Aid: Assistive AI to Improve the Robustness and Security of Reading Hand-Marked Ballots”


Photo by Jeff Fitlow/Rice University

Jade Boyd, Associate Director for Rice University's Office of Public Affairs