In response to the COVID-19 outbreak, the University of Illinois Urbana-Champaign leveraged cloud technology experts at Technology Services to develop a low-cost, automated and fully secure data pipeline to process and maintain COVID-19 testing data.
Garnering much press and attention, the innovative rapid-saliva test for COVID-19 was developed by scientists at the U of I during the early months of the pandemic. By the end of July 2020, the campus testing infrastructure was set up to allow for thousands of tests a day with results available within 24 hours. This was all in preparation for the start of a new academic year unlike any other the university community had ever endured.
At the onset of the Fall 2020 semester, however, positive cases rose dramatically due to the increase of people on campus. The initial systems in place to track and share that data were not scaling quickly enough to meet increased testing demands.
Here is where the Technology Services Data Analytics and Cloud Architecture teams come in. Staff from these groups built a low-cost, automated, efficient, and fully secure testing data pipeline using Amazon Web Services.
This new cloud-based system allows the university to identify and contain outbreak clusters on campus with increased speed and efficiency. It provides a flexible and reliable process that can scale easily, and nearly eliminates the risk of any breakdown in the process.
“In the Fall 2020 and Spring 2021 semesters,we were processing between 10 and 17 thousand tests a day and doing it in the cloud where this is a serverless, distributed environment,” explained Ken Taylor, cloud architect at Technology Services. “We aren’t on a machine that might break or need upgrading or patching, so we get to focus on what’s important: processing and storing the data.”
The SHIELD: Target, Test, Tell program was developed to mitigate outbreaks on campus through frequent testing and communication, rigorous quarantine protocols, and contact tracing. Due to the sensitive nature of protected student and health information, the university has a legal obligation to keep data private, sharing only the information needed to maintain a healthy campus and community by avoiding further spread of COVID-19.
In supporting SHIELD, the new “All COVID Tests” data pipeline shifted a manual process to a fully automated system that sends the appropriate information to the partners who need it.
The complex pipeline begins at the lab with test results. Each sample has a unique order number that maintains privacy for the test taker as results aggregate into positive, negative, and inconclusive categories. The negative results are sent to Mckinley Health Center (McKinley), where they are able to re-connect the appropriate personal health data, alert test takers of their results, and notify the Safer Illinois and Illinois Apps whether these testers have current building access.
Meanwhile, the lab team retests all positive and inconclusive results in case of false-positives. Confirmed COVID-19 cases are placed on a different database that alerts both McKinley and the Champaign Urbana Public Health District (CUPHD). CUPHD handles contact tracing and works with University Housing and others to coordinate proper quarantine protocols.
Perhaps the most surprising part of transitioning to a cloud-based system for this intricate process is the low cost.
“We only use a computer when we need it. Each step takes only a few seconds and we are just paying for those seconds of compute time that we use,” said Taylor.
“It only costs one-quarter of a cent to process a single test which includes storage, computer and database expenses, totaling about $1.20 to $1.50 a day. Adding monitoring, networking, and data lake services, it costs under $20 a day for the entire pipeline to process all of this data – that’s about $550/month or less than $7000/year – to build and maintain this environment.”
One of the key reasons for maintaining all of the COVID-19 testing data in a cloud environment is the research and learning potential. With a centralized, flexible, and reliable place for data, researchers can investigate new and pressing questions about the effects of the pandemic at our university and find new ways to mitigate health risks in the future.
The COVID-19 testing data pipeline was just one of many essential efforts from Technology Services that supported the university’s initiative for a safe return to campus for the Fall of 2020.
The U of I makes COVID-19 testing data available to the public. View the live COVID-19 dashboard.
Interested in using Amazon Web Services for your research? Our cloud support team hosts AWS office hours every Wednesday from 3-5pm, and Friday from 1-3pm. For more information, visit the AWS Office Hours page and sign up for a time (requires NetID): https://go.illinois.edu/awsofficehours