About the 2019 Democracy Fund / EVIC At Reed College LEO Survey

The 2019 Democracy Fund/EVIC Reed College Survey of Local Election Officials is a follow-up to our 2018 LEO Survey, out of which we produced the Stewards of Democracy report with Democracy Fund. From our previous report:

The 2019 Survey of Local Election Officials is part of our effort to create space for LEOs to be heard. Designed to capture the collective experience of LEOs across the country, the 2019 LEO survey solicited opinions about election administration, access, integrity, and reform.

In this second survey, taking advantage of a less stressful off-year in the political and elections cycle, and in collaboration with the Democracy Fund Elections Team, we provided a deeper dive into what has been called the “origination stories” of American local election officials. Included in the survey is the first-ever set of “job satisfaction” items that were adapted from public administration research on the career experience of local administrators. We want to understand who it is that becomes a steward of democracy, and what this may tell us about access to this important role in US democracy.

The 2019 survey included questions that were asked of the mass public in the 2018 Democracy Fund/Reed College module for the Cooperative Congressional Election Survey (CCES). These questions provided a unique opportunity to compare the attitudes of the mass public and local election officials on election reform and on partisan vs. non-partisan elections for a variety of state and local offices. Future white papers and reports will address these comparisons of LEOs to the mass public.

The Elections & Voting Information Center’s project team is proud to contribute to Democracy Fund’s goal of an accessible, efficient, and secure elections system. We hope the results of this project help identify the key ways that local election administration and the people that serve as administrators support a voter-centric elections system.

The EVIC LEO Survey Project is led by Paul Gronke, EVIC Director and Professor of Political Science at Reed College; Paul Manson (Reed ’01), EVIC Research Director and ABD at the Mark O. Hatfield School of Government at Portland State University; and Jay Lee (Reed ’19), EVIC Postbaccalaureate Research Fellow. EVIC thanks Reed College for its ongoing support of our work.

Methodology

Survey Design and Administration

The 2019 LEO Survey was designed in the Spring of 2019. The survey was mailed late in August 2019, with survey returns occurring over three months, mostly concentrated in the first 6 weeks (n = 876). Reminder emails were sent twice in September and October for jurisdictions if we had an email address for the local election official. The 2019 survey focused on the LEOs themselves with items on career history, job satisfaction, attitudes on various election reform initiatives, and finally personal demographics. The response rate for 2018 was 36% and for 2019 was 29%. The primary source of the difference was the mode of survey. The 2019 survey began with mailed surveys to the entire sample and email follow-up to complete a PDF version of the survey, while our 2018 survey started with an online Qualtrics instrument and followed up with mailed surveys.

Sampling Methodology

Our research builds on two consecutive years of surveys sent to LEOs in 2018 and 2019. We used slightly different sampling procedures between the two years to try and control for the impact of jurisdiction size as measured by total number of registered voters.

The 2018 and 2019 LEO surveys were significant learning opportunities to develop best practices for sampling, surveying, and producing accurate statistical estimates across the “complex quilt” (Brown, Hale, and King 2020) of American elections and election administration. This page and other working papers (listed below) describe these best practices in more detail.

The 2018 Local Election Official Survey utilized a sampling frame built off of a comprehensive list of all local election officials in the country obtained from the US Vote Foundation. This was matched with registered voter totals from 2018 Election Administration and Voting Survey (EAVS) administered by the US Election Assistance Commission. We also used local contacts and websites when necessary for sub-county data. We drew a sample of 3,000 jurisdictions, using sampling proportional to the registered voter population – that is, a given small jurisdiction is less likely to be sampled than a given large jurisdiction.

For the 2019 Survey of Local Election Officials we developed a new sampling frame by building off of the 2018 EAVS and scraping data from Secretary of State websites or similar sites, state-by-state. We made a series of edits to this list of jurisdictions to create proper entries for each local jurisdiction that included a local administrator responsible for election administration. This resulted in a sampling frame with 7,834 local jurisdictions. We drew a sample of n = 3,000 from this list using the random systematic sampling method, with inclusion probabilities proportional to number of registered voters in each jurisdiction. This ensured that all of the largest jurisdictions (> 15,000 registered voters) were included in the sample, and we collected a representative sample of jurisdictions of smaller sizes.

Our sampling method was determined with two goals in mind. First, we wanted our sample to be representative of local election officials. Second, we wanted our sample to be nationally representative of service provision to voters, or put another way, we wanted to assure that we have sufficient coverage of local election officials serving a large and diverse American electorate. As pointed out by previous researchers, “[l]ess than 6% of the local election officials in the United States serve more than two-thirds of the voters in a national election” (Kimball and Baybeck 2013). Therefore, following past practice, we have sampled jurisdictions proportional to the number of registered voters they serve. In practice, what this means is that 100% of the larger jurisdictions (> 15,000 registered voters) fell into our sample.

A key challenge in sampling LEOs is the diversity of jurisdictions, and that most of the LEOs in the United States are concentrated in smaller jurisdictions when measured by number of registered voters. This is connected to state-specific election administration systems; several states in the Midwest and New England administer elections at the municipality or township level, as opposed to the county-level administration elsewhere. For example, the top five states in terms of number of LEOs represent 57% of the sampling frame. However, these five states combined only represent 8% of registered voters in our sampling frame, and for all but one of these states, the median number of registered voters in the jurisdiction is less than 1,800. A graphical representation of what we call the “75:8 problem” is shown below.

Our sample resulted in fewer of the very small jurisdictions than we would have in a purely random sample. This was done purposefully, but using “unequal probability sampling” means that we have to generate survey weights and weight our statistical estimates if we want to make inferences back to the overall LEO population.

Response Rates

The top five states in terms of number of LEOs represent 34% of our sample and 17% of registered voters. The figures below demonstrate this dynamic across the US. This is an important point as we report results later in the paper. We intentionally divide jurisdictions into various size classes in an effort to control for the impact of the size, distribution, and representation of jurisdictions. Alaska and Hawaii are unique spatial cases for election administration: Alaska is divided into four administrative regions for election administration, and Hawaii is organized administratively into four island based counties that we sampled.

Sampling Weights

Sampling weights were used to produce all estimates in this report. The impact of the weights is illustrated in the table below, which presents some demographic information collected in our 2019 survey. The “Unweighted” column presents unweighted averages, the “Weighted” column presents estimates weighted according to the unbiased Horvitz-Thompson estimator (referencing the entire population of LEOs), and the final three columns are unweighted averages within three size categories. As we can see, the weights have only minimal effects on some quantities, but have a substantial impact on our estimated proportion of LEOs receiving salaries less than $50,000.

Other Resources and Working Papers