Design Strategy

The Georgia Institute of Technology, North Carolina State University, and DPH collaborated to create a simulation model to simulate the effects of surveillance testing on the spread of Covid-19 on college campuses (Yu, Fujimoto, Keskinocak, & Swann, 2021). This foundational Covid-19 testing model is based on a Susceptible-Infected-Recovered (SIR) epidemiological model, which is extensively used in research for epidemic spread. The foundational model solely considered the effects of surveillance testing as the intervention strategy to reduce the spread of Covid-19 in schools. The simulation model we developed expands on the foundational model, with additional parameters for the effects of masking and vaccination to account for a wider range of intervention strategies that exist in schools. To analyze different combinations of intervention strategies, the model takes in two types of parameters: built-in data and parameters as well as user inputs. The following researched built-in data and parameters are fixed and predetermined, and they are integrated into the back end of the simulation model: transmission rates due to contact with (a)symptomatic, (un)vaccinated individual, rate of recovery for an infected individual, proportion of infections that are (a)symptomatic, isolation compliance (a)symptomatic infected and tested patients, and false negative rates of a rapid test for (a)symptomatic patients. User inputs are parameters that school decision-makers will input into the decision-support tool interface so that the simulation model can project cases using school information: county name, school type, student population size, faculty & staff population size, type of surveillance testing (PCR, Rapid, None), simulation duration (in days), percentage of population wearing masks, percentage of population participating in weekly surveillance testing, percentage of students vaccinated, and percentage of faculty & staff vaccinated. We validated our developed simulation model by conducting sensitivity analysis, county-level validation, and school-level validation. 

We performed two types of sensitivity analysis to assess the robustness of the model: sensitivity analysis with respect to the transmission rate (beta; Bn) and sensitivity analysis with respect to case projections. To assess the sensitivity with respect to transmission rate, each rate was uniformly distributed from the range [0.75Bn, Bn]. This is because, as our transmission rates account for the presence of the highly-infective Delta variant, we wanted to ensure the transmission rates were at most that of Delta. Each transmission rate was sampled 40 times uniformly (greater number of sampling could lead to significant runtime), and the case projections were recorded with all other transmission rates constant. This process was repeated for each transmission rate. After 40 case projections were produced, the mean and standard deviation of these values were taken. The coefficient of variation (CV) was then calculated by taking the ratio of the standard deviation to the mean. The sensitivity with respect to case projections was also assessed to see the variation in how the model predicts cases per day per iteration. In this case, all transmission rates were sampled from the uniformly distributed [0.75Bn, Bn] range, and the model was run 40 times to provide 40 case projections over 14 days. The mean and standard deviation of case projections were calculated per day, i.e. the mean and standard deviation for iterations 1 through 40 on day one, and so on, were respectively recorded. The CV was then calculated by taking the ratio of the standard deviation to the mean.

To validate the model at the county level, we compared the number of observed cases obtained from DPH’s website to the model’s average predicted number of cases for each county. We scaled the observed number of cases at the county level by a uniformly and randomly selected underreporting multiplier between 3.4 and 4.7 (Estimated Covid-19 Burden). We set the masking compliance to 50%, the percentage of the population getting PCR tested to 50%, and used the vaccination percentage by county data posted on DPH’s website prior to running the simulation model, as directed by DPH, to simulate the average policies and practices being seen across all 159 counties. Once these values were set, we ran the simulation model 10 times (as opposed to 40 due to increased run-time errors) for each county for a two-week period and calculated the average number of predicted cases per county. Next, we compared the average number of predicted cases to the scaled number of observed cases for each county, calculating the overall difference between the two values, the percent error of those infected, and the percent error of those not infected. We executed this process for validation five times in order to assess the model’s performance with new data sets of observed cases being posted by DPH.

To validate the model on a school level, we ran the simulation model for 50 schools in Georgia. Each of the schools we tested had varied intervention strategies, community and school types, and population sizes to ensure that the simulation model was representative of the diverse range of Georgia schools. We compared the actual number of Covid-19 cases for each of these schools to the simulation model case projections to assess accuracy.