Evaluating a New Process for Assigning Geographic Residence Codes and Identifying Demographic Information for Workers in a Given Tax Year

Introduction

In 2021, the Office of Research, Evaluation and Statistics (ORES) in the Social Security Administration (SSA) completed development of a new methodology for assigning geographic residence codes and identifying demographic information for nearly all workers with earnings in a given year. Compson (2022) describes the new methodology in detail, and this article evaluates it by comparing it to the methodology SSA currently uses to generate the comprehensive earnings and employment estimates it publishes in its annual statistical publications. ORES has applied the new methodology to tax information for tax years 2014 through 2020,¹ producing a standalone Master Geographic and Demographic (MGD) data file for each year. For that reason, the terms “new methodology” and “MGD process” are used interchangeably throughout this article. ORES considers the development and evaluation of the new methodology to be the first and second steps, respectively, in a multistep process that will result in a dramatic expansion of the sample size used to generate the estimates for its statistical publications.

The evaluation of the MGD methodology consists of two distinct assessments. The first, a procedural assessment, uses internal audit reports to assess the completeness and accuracy of the new methodology in processing tax records for a 7-year span. It involves looking at the number of records processed, the number of unique Social Security numbers (SSNs) represented, the sources of the tax information, the methodology used to assign state and county codes (SCCs), and the results of various imputations used in populating missing data fields. The second assessment compares the MGD-assigned SCCs and demographic identifiers with those assigned under the current methodology. Specifically, this assessment involves comparing the estimated numbers of workers with Social Security taxable earnings and the amounts of those earnings by state, sex, age, and type of earnings (wage and salary, self-employment). This assessment also compares the two methodologies' estimated numbers of workers by county.

In general, the procedural assessment finds that the MGD process is consistent and thorough across the 7 years of tax data analyzed, although it raises minor questions, noted later, that ORES is currently investigating. The comparative assessment finds match rates of nearly 99 percent for workers' state code assignments and sex and age identifications. For county code assignments, the match rate is lower, at 94.5 percent. This result was expected because the MGD process uses more detailed geographic identifiers to assign county codes than were available when the current process was developed.² Thus, in this circumstance, the lower match rate might reflect greater accuracy in the new methodology.

This introduction is followed by five sections. The first section highlights the key points of the MGD process for identifying demographic information and assigning geographic codes for the population of workers in a given tax year. The second section details the procedural assessment of the MGD process for the 7 years of tax data currently available. The third section discusses the methodology for conducting the microdata comparison, presents the preevaluation expectations of the comparisons, and assesses the results for worker counts by type of earnings, sex, age, and state. The fourth section discusses the comparison of the county-level estimates, and the fifth section concludes.

The MGD Process

The MGD process was developed to address the current methodology's limitations in assigning accurate geographic codes to workers' tax records and its reliance on sample data too small in size for the required scope of the work. The current methodology uses administrative microdata (that is, person-level information) from SSA's 1-percent Continuous Work History Sample (CWHS). For tax year 2017, the sample consists of fewer than 1.7 million workers. By contrast, the new methodology culminates in the creation of a standalone MGD data file containing geographic and demographic information for nearly all workers in a given tax year (179 million in 2017). The process generates 32 audit reports for each year. The reports allow ORES to track and evaluate the results of each step in the process of assigning a single SCC to each worker in a given tax year.

Each year, SSA and the Internal Revenue Service (IRS) share information from IRS tax forms for those agencies' respective programmatic needs. To that end, SSA's Office of Enterprise Information Systems (OEIS) receives hundreds of millions of IRS Forms W-2 and W-2c (filed by employers) and millions of Form 1040 Schedule SE (filed by the self-employed) from the IRS.³ As part of its elaborate annual wage reporting process, OEIS extracts all the information SSA needs to administer its programs. In a separate and distinct process undertaken for ORES and SSA's Office of the Chief Actuary, OEIS extracts the full address information reported on these forms and uses Pitney-Bowes' Finalist software to assign an SCC for each record.⁴ The resulting data files are the basis of the MGD process for assigning geographic residence codes for nearly all workers in a tax year.

The MGD process begins with job-level data—that is, records that contain both the worker's SSN and the employer identification number—and converts them to worker-level data, assigning a single SCC to each SSN as it does so. For this article, “number of records processed in a given tax year” is synonymous with the number of jobs in a given tax year, and “number of SSNs” refers to the number of workers in a given tax year. The number of records for each of the data sources (that is, the tax forms) is always greater than the number of SSNs because many individuals hold multiple jobs during the year.

Audits track the number of records or SSNs throughout each step of the MGD process. For example, the audit reports provide the number of records processed for each type of data source (Forms W-2, W-2c, and 1040 Schedule SE) and the number of unique SSNs associated with each data source, on an annual basis and over time.

In general, the audit reports contain the following information:

• The number of records processed and the corresponding number of unique SSNs.
• The number of records and SSNs associated with each type of tax form.
• The number of SCCs assigned to each worker via the OEIS/Finalist process (one, multiple, none).
• The numbers of workers with valid and invalid SSNs.
• The number of individuals with sex and date of birth information assigned.
• The number of unique SSNs affected by each of the MGD methodology's various imputation processes.
• The data source for the SCC assignment for each worker.
• The number of records for each tax year that were processed in a given calendar year.

Once the underlying OEIS/Finalist data are extracted, the MGD process sorts workers into one of the following mutually exclusive data-source categories:

• W-2 only.
• W-2c only.
• Schedule SE only.
• W-2 and W-2c.
• W-2 and Schedule SE.
• Schedule SE and W-2c.
• W-2, W-2c, and Schedule SE.

The audit reports detail the number of records and the number of unique SSNs for each of these data-source categories and compute the total number of unique SSNs in a given tax year.

The next step in the MGD process uses an administrative data master file called the Numerical Identification System (Numident) to identify valid and invalid SSNs and to supply information on each worker's sex and date of birth.⁵ Any SSN in both the MGD file and the Numident file is deemed to be valid. SSNs in the MGD file but not in the Numident file are deemed to be invalid. ORES can record sex and date of birth only for workers whose SSNs appear in the Numident file. For records with invalid SSNs, ORES enters “Missing” in the sex and date of birth data fields. The MGD process creates a data file containing demographic information that is set aside while the process of assigning a single SCC to each worker in a given tax year, described below, proceeds.

First, the MGD process groups workers by the number of SCCs (one, multiple, none) that the OEIS/Finalist process assigned to them. The records for workers with a single SCC assigned by the OEIS/Finalist step are referred to as the “gold-standard file” and for them, the process of assigning an SCC is complete. For workers who were not assigned an SCC, ORES uses the frequency distribution of SCCs in gold-standard file records that share the worker's ZIP Code to try to impute an SCC.⁶ ORES employs a multistep process (briefly summarized later and detailed in Compson 2022) to assign the “best” SCC to the records of workers that have multiple SCCs after the OEIS/Finalist step. The audit reports show which method ORES used to assign a single SCC for each worker.

Once a single SCC has been assigned to each worker, the resulting file is rejoined with the file containing demographic information to create the standalone MGD file for that tax year. The merged file contains the following data fields for each worker:

• SSN.
• SCC.
• Date of Birth.
• Sex.
• Date of Death.
• Date on which Date of Death was posted on the Numident.
• Method of SCC Assignment.

Researchers and policy analysts using the MGD file must consider several important points. First, as noted earlier, the process that OEIS uses to assign SCCs for each job is based on the full address reported on the tax forms and is separate and distinct from the annual wage reporting process undertaken as part of program operations. As a result, the data used to create the MGD file have not been subjected to the cleaning and evaluation techniques that the tax data must undergo before they can be posted to SSA's Master Earnings File (MEF) for programmatic purposes. One result of using the raw tax data is that the MGD file contains invalid or improperly assigned SSNs. The latter may occur if the employer incorrectly enters an SSN when filling out the worker's Form W-2 or W-2c, or a self-employed individual enters the wrong SSN when filing Form 1040 Schedule SE. There is currently no way for ORES to correct such errors in its files.

Second, because the MGD file does not contain any information on the type or amount of earnings reported on the tax forms,⁷ it cannot, by itself, be used to estimate earnings covered or taxable under Social Security and Medicare. ORES is currently developing a new process to generate estimates using a much a larger sample of workers extracted from the MEF or even, possibly, the entire population of workers in a tax year. The MEF and the MGD files together would contain the data necessary to generate the annual earnings estimates.

Third, the MGD file for a given tax year y contains data only for tax records that were processed in calendar year y + 1. For example, the first MGD file contains data for tax year 2017, but only for forms that were processed in 2018. In turn, for processing year 2018, 2017 is the primary tax year. The 2017 MGD file excludes any data for tax year 2017 that were processed in a calendar year other than 2018, and it excludes data for tax years other than 2017 that were processed in 2018. In developing the MGD methodology, ORES decided to focus on the data for a single tax year that were processed in the single calendar year that followed. ORES chose this method despite knowing that some tax year 2017 earnings were processed in 2017 or would not be processed until after 2018. Whether it was possible to include these data in the 2017 MGD file, and if so, how, was yet to be determined.⁸

To illustrate, the MGD file for 2017 contains records for 178,863,694 workers whose tax forms were processed in 2018. However, an additional 2,618,600 workers had earnings in tax year 2017, but their forms were processed in other years, as follows: 233,222 in 2017; 1,737,114 in 2019; 404,899 in 2020; and 243,365 in 2021. Thus, the 2017 MGD file omits up to 1.44 percent of the population of workers with reported earnings in 2017.

This circumstance raises several critical questions. First, are any of these individuals already in the 2017 MGD file? (This can occur for multiple job holders or those with earnings reported on both a Form W-2 and an amended Form W-2c, or because of filer error in entering the tax year.) Identifying these instances can reduce the number of individuals whose records need to be incorporated into the MGD file. Second, to add the records for workers whose tax forms were not processed in 2018 into the 2017 MGD file, how many processing years should be included, and how reliable will those data be? Experience shows that some data for a given tax year may not be reported for several years. However, over time, the number of workers being added trends to zero so the potential effect on the MGD file becomes inconsequential.

Another concern is the reliability of the address information reported on the tax forms processed in later years. For example, if a Form W-2 or W-2c for tax year 2017 is not processed until 2021, the individual may no longer reside in the same location. ORES is evaluating the possibility of incorporating the additional tax information reported in subsequent years to the MGD files. This issue is especially pertinent given that the COVID-19 pandemic led to substantial delays in IRS processing of tax returns from 2020 to April 2023.

The Procedural Evaluation

Table 1 presents the number of records extracted in each processing year from 2015 to 2021 and the number of unique SSNs associated with those records, by type of data source (W-2, W-2c, and 1040 Schedule SE). The number of records is analogous to the number of jobs, and the number of SSNs reflects the number of workers. The number of records far exceeds the number of unique SSNs each year because workers may have multiple jobs, each requiring its own tax form. A worker may have tax forms of more than one type for a given year, or multiple forms of the same type in a year, or both. The total number of unique SSNs for each year overstates the actual number of workers because it includes duplicates (that is, the SSNs of workers with more than one type of tax form). Note the relatively large volume of W-2c records processed in 2017, the decrease in the number of Schedule SE records processed in 2020, and the drop in the numbers of W-2s and associated SSNs in 2021. It is not clear if the lower numbers of Schedule SE records processed in 2020 and W-2s processed in 2021 reflect fewer jobs in the economy or the effect of COVID-19 on employers' ability to timely file W-2s or W-2cs for their employees and the IRS' ability to process Schedule SEs.⁹

As mentioned earlier, data for nonprimary tax years are included in a calendar year's processing workload. Table 2 shows the prevalence of primary-year and nonprimary-year data for each type of tax form in 2015–2021. The number of W-2s processed dropped by nearly 12 million from 2020 to 2021, which is likely due to COVID-19's effect on the labor market and employers' W-2 filings.

The number of W-2cs processed nearly doubled in 2017 and increased sharply in 2021. Part of the increase in W-2c processing in 2017 reflects a large payroll service provider's issuance of corrections to approximately 500,000 records (SSA 2017). The increase in 2021 is most likely a rebound after COVID-19 limited W-2c processing in 2020. The steep increase in 2021 processing of 1040 Schedule SEs for nonprimary tax years most likely reflects IRS efforts to reduce the backlog caused by the pandemic.

As noted earlier, the MGD process uses data from the Numident master file to identify valid and invalid SSNs and to provide information on each worker's sex and date of birth, yet the tax data used in the OEIS process are not subject to the cleaning and verification associated with the programmatic annual wage reporting process. As a result, some of the SSNs in the data extracted for the new process are not in the Numident file and are deemed to be invalid.¹⁰ Table 3 shows the number of valid and invalid SSNs and expresses both numbers as a percentage of the unique SSNs contained in the records processed each year. The percentage of SSNs that are valid is stable over the years.

In the next step of the MGD process, ORES identifies the demographic information for each worker using data from the Numident file. Table 4 shows the volume of records processed for this step and the breadth of the demographic information the records contained, which enabled ORES to identify, in each tax year, the sex and date of birth of nearly 99 percent of workers whose records include a valid SSN.

Table 5 shows, for unique SSNs associated with worker records processed, the number to which the OEIS/Finalist process assigned either one SCC, multiple SCCs, or no SCCs. The number of workers for whom the OEIS/Finalist process assigned a single SCC is dramatically lower for 2015 than all other years and is reflected in the aberrantly high number of workers with no SCC assigned in that year. In addition, the number of workers with multiple assigned SCCs is much lower for 2015 than all other years. These results raise concerns about the quality of the processing-year 2015 MGD data and ORES will carefully evaluate the distribution of the state and county assignments for that year. The record-processing results for the other years are consistent over time.

ZIP Code imputation is the first of several steps ORES takes to assign a single SCC for records that were not assigned an SCC in the OEIS/Finalist process. Table 6 shows that ZIP Code imputation dramatically affects the distribution for 2015, converting many worker records from zero to one assigned SCC. Yet for 2015, the number of workers with multiple SCCs is still much lower than in subsequent years and the number of workers with no SCC is much higher than in later years. The high volume of records that were subject to imputation because they were not assigned an SCC in the OEIS/Finalist process probably accounts for the anomalous 2015 figures. The results for the other years are consistent.

The next step in the MGD process determines, for workers with multiple SCCs, which one is the best to assign. For this, ORES first generates a file containing all the SSNs that have multiple SCCs and extracts the earnings data for each worker from the MEF. The SCC for the location of the worker's highest-paying job is assigned, when that information is available. For the remaining workers, ORES applies one of several additional imputation techniques (detailed in Compson 2022) that involve matching the frequency distribution of employer location and worker SCCs in the gold-standard file to select the best SCC.

Table 7 quantifies the methods by which records received a single SCC assignment. Excluding processing year 2015, the volume of records having a single SCC assigned via each method is consistent over time. The OEIS/Finalist process produces most of the single-SCC assignments, with the resulting gold-standard file constituting at least 94 percent of workers each year. Using the highest-paying job to assign a single SCC for workers with multiple SCCs accounts for at least 4.7 percent and as much as 5.1 percent of workers in a given year. Combined, these techniques enabled ORES to assign a single SCC to at least 99 percent of workers with a valid SSN in 2016–2021. The frequencies of the other imputation techniques are also consistent over time, as is the percentage of SSNs for which ORES could not assign an SCC.

The procedural evaluation of the MGD process shows consistency over time and provides evidence that the process is stable and robust. However, some observations warrant further investigation. Why did so many records have no SCC assigned in 2015, and did that affect the assumed geographic distribution of workers for that year? Why did the number of W-2c records processed increase sharply in 2017? What accounts for the drop, shown in Table 4, in the number of records with a date of death from 5.18 million (3.0 percent of records processed) in 2015 to 1.57 million (0.9 percent) in 2021? Comparing the following tabulation, which shows all U.S. deaths for 2014–2022, with the number of worker records containing a value in the date of death field shown in Table 4 suggests that many deaths from earlier years were not posted until 2015, 2016, and 2017, and that many deaths occurring in 2018 or later have not been posted yet.

The Comparative Evaluation

This section describes the steps ORES took in preparing to compare the current-methodology and new-process estimates, summarizes the results that ORES staff expected the evaluation would produce, and describes the construction and characteristics of the data files used in the evaluation. Then, it discusses the differences between the two methodologies in the estimated number of workers with covered earnings and the amounts of those earnings by state, sex, and age.

Comparison of Current-Methodology and MGD-Process Geographic Estimates

The current methodology provides the estimates that ORES publishes in annual statistical publications. ORES publishes covered employment and earnings estimates by state in the Annual Statistical Supplement to the Social Security Bulletin (hereafter, the Annual Statistical Supplement; see https://www.ssa.gov/policy/docs/statcomps/supplement/index.html) and by state and county in Earnings and Employment Data for Workers Covered Under Social Security and Medicare, by State and County (hereafter, Earnings and Employment; see https://www.ssa.gov/policy/docs/statcomps/eedata_sc/index.html). This evaluation uses the microdata and the estimation methods currently used for those publications (with slight modifications, described later) to generate a data file that allows comparison with the MGD process used for assigning SCCs and identifying demographic information. The evaluation comprises two distinct comparisons. The first comparison focuses solely on estimates of the number of workers and their taxable earnings amounts by state, sex, and age. The second comparison focuses on county-level estimates. It is addressed in a separate section because it is significantly more complex than the state-level comparison.

Chart 1 diagrams the steps ORES currently takes to generate the state- and county-level earnings estimates in its statistical publications.¹¹ The process begins by merging the contents of three distinct component files in the 1-percent CWHS file system: the Assigned State file, the Active file, and the SE file. The Assigned State file contains annual earnings and geographic data at the job level, with one record for each SSN/employer identification number combination. The Active file contains time-series earnings and demographic data at the SSN level for each worker with reported earnings over time. The SE file contains job-level earnings and geographic data for self-employed individuals for a given year. The resulting merged microdata file is called the Assigned State_SE_Active (ASA) file. After various manipulations, this merged file contains worker-level data and includes the following data fields:

• SSN;
• Social Security taxable earnings;
• Medicare taxable earnings;
• Year of birth;
• Sex;
• Employment type (wage and salary, self-employed); and
• SCC (in ASA, the SCC is a strictly numeric code; that is, it does not contain state or county names).

ORES currently uses the merged ASA file to create several summarized data files from which it generates state- and county-level employment and earnings estimates. Generating the county-level estimates requires an extra step because the microdata do not contain the county names associated with the SCCs. Specifically, the summarized county-level data must be joined with a separate data file called the LABELS file that contains both the numeric SCCs and the corresponding county names. Further details are provided below in the section on county-level estimates.

Comparing the current and MGD methodologies involves joining the ASA and MGD files, linking the two files' records by SSN. The resulting joined ASA-MGD file—the evaluation file—contains all the information needed to generate two versions of the earnings tables with state-level estimates. This allows a direct comparison between the current and MGD processes of the estimated number of workers and total earnings amounts by sex, age, and type of earnings.

The ASA microdata file that is used to generate the tax year 2017 earnings tables by state and county contains 1,758,471 SSNs (Table 8). Of those, 1,751,807 SSNs are found in both files and 6,664 are in the ASA but not in the MGD file. Given that the MGD file represents the entire population of workers in a tax year, what explains the 6,664 workers represented in the ASA but not in the MGD file? There are two possible answers.

Recall that the MGD file for a given tax year excludes records for earnings that were not processed in the calendar year following that tax year. For example, in 2018, the MGD process excluded all records containing information for tax years other than 2017. Therefore, some of the MGD file's “missing” SSNs for tax year 2017 were processed in a year other than 2018.¹² As previously noted, tax year 2017 data for 2,618,600 individuals were processed in 2017, 2019, 2020, and 2021, and were therefore omitted from the tax year 2017 MGD file.

Of the 6,664 individuals with a 2017 ASA record but no MGD record, ORES identified 1,001 whose records were processed in 2017, 50 whose records were processed in 2019, and 8 whose records were processed in 2020 or 2021. ORES did not attempt to assign a geographic code or identify the sex and date of birth for the 1,059 individuals whose tax year 2017 information was not processed in 2018.

A second explanation for the “missing” individuals is the possibility that incorrect SSNs were entered on the tax forms. Recall that the MGD process for assigning location codes and identifying sex and date of birth is separate and distinct from the OEIS process that cleans and verifies the information before posting the data to the MEF. For example: In compiling its annual wage reports, OEIS matches the name and SSN shown on Form W-2 to that worker's administrative records. If one of the digits in the SSN was entered incorrectly, OEIS undertakes one or more procedures to assign the W-2 information to the correct worker. However, the MGD process does not have this capability. Instead, ORES simply takes the SSN as given and uses it to assign a geographic code and identify the worker's sex and date of birth using the Numident file. As a result, the record for a worker whose information was incorrectly reported on Form W-2 could be retained in the ASA file but would not be included in the MGD file.

Whatever the cause of the discrepancy, the 6,664 individuals with records missing from the MGD file represent less than 0.4 percent of the 1,758,471 workers in the 2017 ASA file. Therefore, ORES removed them from the merged ASA-MGD file that was used in evaluating the MGD process results.

Table 8 shows the number of workers represented in the ASA microdata file and in the large subgroup who comprise the MGD evaluation file, with detail by earnings type (wage and salary, self-employment). It also distinguishes between workers whose earnings are taxable and are not taxable for Social Security and Medicare.

The evaluation begins by comparing MGD-process estimates with slightly modified versions of those published in Annual Statistical Supplement Tables 4.B10 and 4.B12, which respectively show Social Security– and Medicare-covered workers and taxable earnings, by state.¹³ In the next step, MGD-process estimates are compared with the worker counts and earnings amounts by sex and state found in the modified versions of Earnings and Employment Tables 1 and 4. The third step involves comparing the MGD-process estimates with worker counts and earnings amounts by sex, age, and state, as published in Earnings and Employment Tables 2 and 5. After comparing the state-level estimates, the MGD file's county-level estimates of workers by sex are compared with those in Earnings and Employment Tables 3 and 6.

Preevaluation Expectations

Prior to comparing the estimates produced by the current and MGD processes, ORES expected the outcomes to include larger percentage differences between the methodologies' worker counts and earning amounts for less populous states than for larger ones. To provide a deliberately exaggerated example, consider two hypothetical states: SSA statistical publications estimate that state A has 10,000 workers and state B has 50,000 workers. If the MGD process assigns 2,000 more workers to each state, the estimated number of workers differs by 20 percent in state A but only 4 percent in state B. In such a scenario, the estimated amounts of taxable earnings reported in the states would be similarly affected. In addition, because there are far fewer self-employed individuals (186,697) than wage and salary workers (1,580,879) in the CWHS microdata that underlie the current methodology, smaller absolute changes will likewise generate larger percentage differences for the self-employed than for other workers. ORES expected a similar effect in the estimates by age for the age groups that include comparatively few workers in the CWHS.

ORES also expected match rates between the current methodology and the MGD process to be higher for state assignments than for county assignments. Any worker whose state code does not match in the two files will also have a nonmatching county code, even before considering the several reasons why county assignments within a state may differ between the files. The current methodology assigns state and county codes based on abbreviated geographic identifiers (the first five letters of the city name and the five-digit ZIP Codes reported on tax forms). Although the same abbreviated city name can appear in multiple states, the fact that few (if any) ZIP Codes cross state lines indicates that the current methodology generates reasonably accurate state code assignments.

For county code assignments, however, abbreviated geographic information can be problematic. ZIP Codes speed the flow of mail by designating efficient postal delivery zones which, at the five-digit level, may cross county boundaries. Thus, using only the first five letters of a city name and the five-digit ZIP Code can lead to occasional county code inaccuracies.

Furthermore, under the current methodology, an SCC assigned for a worker with both wage and salary and self-employment earnings might be based on data reported on Form 1040 Schedule SE and on either or both of Forms W-2 or W-2c. When the current methodology was developed more than 30 years ago, the SCC corresponding with the self-employment income was typically assigned because the address reported on Schedule SE was viewed as more reliable than a conflicting address reported on another form. However, the MGD process has revealed that millions of individuals are assigned multiple SCCs in a given tax year and there is no reason to believe that the address reported on Schedule SE is more reliable than the address on the W-2 or W-2c. The MGD process provides several options for assigning an SCC and ORES has determined that the best option is to use the SCC corresponding with the highest-paying job regardless of the type of earnings. For these reasons, differences between the current methodology and the MGD process are more likely in county assignments than in state assignments.

Third, ORES expected very high match rates between the current methodology and the MGD process for worker sex and age. Where discrepancies emerged, ORES expected that the MGD process would be more accurate than the current process. This is because the Numident master file is the sole source of the sex and age information used in the MGD process, while in the current methodology, that information may be drawn from either of two files that are derived from the Numident, rather than from the source file itself.

Evaluating Worker Counts

Of the 1,751,807 individuals represented in the full MGD evaluation file, which includes those with noncovered earnings as well as those with earnings covered by Social Security or Medicare, 98.87 percent have the same state code assigned by the current and MGD processes (not shown). Thus, only 19,875 workers (1.13 percent) have nonmatching state codes. However, among those workers with nonmatching state codes are 2,511 to whom the current methodology assigns one of the following location categories: Armed Forces, International Operations, Other, and Reserves, categories that are not included in the MGD process.¹⁴ Because those categories do not represent a state or U.S. territory, calculating a “true” match rate—one that accounts only for cases in which it is possible for the two state codes to match—requires removing those 2,511 individuals from the total of 1,751,807 workers. The resulting “true” match rate is 99.26 percent, which leaves only 17,364 of 1,749,296 workers whose MGD-process and current-methodology state codes do not match. This result aligns with ORES expectations of high state code match rates given that few ZIP Codes, if any, cross state lines.

The tables that follow compare the numbers of workers and the taxable earnings amounts estimated using the current and the MGD processes for assigning geographic and demographic information. Recall that the current-methodology estimates are slightly modified so that the estimates for both processes are based on the same unadjusted and unweighted raw data from the microdata file derived from the 1-percent CWHS.

Table 9 shows the estimated number of workers with earnings taxable for Social Security—that is, Old-Age, Survivors, and Disability Insurance (OASDI)—by state or other area (as assigned using the current methodology) and type of earnings. It also shows the number and percentage of workers who are assigned the same state codes using the MGD process. Note that these estimates include the workers for whom the current methodology assigned the codes Armed Forces, International Operations, Other, and Reserves. As a result, the state-code match rates are slightly understated.

The match rate for all workers with OASDI taxable earnings is 98.9 percent. It is at least 99 percent in 34 states, at least 98 percent in 46 states, and at least 97 percent in 48 states.¹⁵ The lowest match rates are those for the District of Columbia (95.9 percent), South Dakota (93.8 percent), and Montana (93.3 percent)—states with relatively few workers.

The match rate for all wage and salary workers with OASDI taxable earnings is also 98.9 percent.¹⁶ The match rate is at least 99 percent in 34 states and at least 98 percent in 47 states. The lowest match rates are 96.9 percent for North Dakota, 95.8 percent for the District of Columbia, and around 93 percent for Montana and South Dakota.

The match rate for all self-employed individuals with OASDI taxable earnings is 98.1 percent. In most states, the match rate for self-employed individuals tends to be lower than that for wage and salary workers, likely because self-employed individuals are far less numerous than wage and salary workers in the CWHS. Only three states have a match rate of at least 99 percent, although 39 states have a match rate of at least 98 percent and 49 have a match rate of at least 97 percent. The match rate for Wyoming is 96.1 percent and for the District of Columbia it is 95.0 percent.

Results of the same analysis for workers with earnings covered under the Medicare programs were similar to those for workers covered under OASDI, and this pattern recurred for all subsequent comparisons between the two methodologies. Therefore, the results for Medicare-covered workers are not shown in separate tables and are not discussed hereafter unless they diverge from those for OASDI-covered workers.

Differences in Estimated Worker Counts

Table 10 shows the number of workers for whom the current methodology and the MGD process assigned a state code, by the assigned state or area. For all workers with OASDI taxable earnings, the difference in the number of state assignments ranges from 420 fewer workers estimated in the MGD file for Illinois to 1,331 additional workers estimated in the MGD file for California. For only seven states does the percentage differ by more than 1 percent, with the MGD file assigning fewer workers for six of them: Montana (−6.1 percent), South Dakota (−5.1 percent), the District of Columbia (−2.7 percent), Nebraska (−2.5 percent), North Dakota (−1.2 percent), Delaware (−1.1 percent), and Missouri (1.1 percent). These states all have relatively few workers.

For wage and salary workers with OASDI taxable earnings, the difference in the number of state assignments ranges from 431 fewer workers in the MGD file for Illinois to 1,291 additional workers in the MGD file for California. Because wage and salary workers far outnumber self-employed individuals, their results in Table 10 are similar to those for all workers: MGD assignments for the same seven states differ by more than 1 percent from those of the current methodology (Montana, −6.4 percent; South Dakota, −5.4 percent; the District of Columbia, −2.8 percent; Nebraska, −2.6 percent; North Dakota, −1.3 percent; Delaware, −1.2 percent; and Missouri, 1.2 percent).

For self-employed individuals, the differences in the numbers of state assignments range from 23 fewer workers in the MGD file for Tennessee to 36 additional workers in the MGD file for California. The MGD state code assignments differ by more than 1 percent from the current-methodology assignments in seven states: the District of Columbia (2.3 percent), New Hampshire (1.9 percent), West Virginia (1.3 percent), North Dakota (1.2 percent), Utah (1.2 percent), Wyoming (−1.4 percent), and Montana (−1.8 percent). In a notable departure from the pattern for wage and salary workers, MGD code assignments for the self-employed are more than 1 percent higher than those in the current methodology for five states.

A parallel analysis for workers with Medicare-taxable earnings produced very similar results, with one difference worth noting. The MGD process assigned the District of Columbia code to 1.3 percent more individuals with Medicare-covered self-employment income than the current methodology did (not shown), compared with 2.3 percent more for self-employed individuals with OASDI taxable earnings.

Differences in Estimated Taxable Earnings Amounts

Given the high match rates in the estimated numbers of workers with OASDI taxable earnings for both earnings types, one might expect the estimated taxable earnings amounts by state to be similar under the two methodologies as well. However, some of the workers with different state codes assigned by the MGD process could have earnings that are high enough to alter some of the estimated state-level earnings. Potential state-level shifts in estimated Medicare-covered earnings amounts could be even greater because unlike OASDI-covered earnings, there is no cap on the amount of Medicare earnings subject to the payroll tax.

Table 11 compares the estimated amounts of Social Security taxable earnings for workers whose state code was assigned under the current methodology with those whose state code was assigned under the MGD process.

For all workers, the MGD earnings estimate differs by at least 1 percent from that of the current methodology in 10 states. The MGD estimate is lower in seven of those states: Montana (−5.0 percent), South Dakota (−4.2 percent), the District of Columbia (−3.8 percent), Nebraska (−1.9 percent), Delaware (−1.5 percent), Louisiana (−1.2 percent), and North Dakota (−1.0 percent). The MGD estimate is higher in three states: Missouri (1.3 percent) and New Mexico and Maine (1.1 percent).

For wage and salary workers, the MGD estimate differs by at least 1 percent from that of the current methodology in 11 states. The MGD estimate is lower in eight of those states: Montana (−5.2 percent), South Dakota (−4.3 percent), the District of Columbia (−3.9 percent), Nebraska (−2.0 percent), Delaware (−1.6 percent), Louisiana (−1.2 percent), North Dakota (−1.1 percent), and Ohio (−1.0 percent). In three states, the MGD estimate is at least 1 percent higher than the current-methodology estimate: Missouri (1.4 percent), New Mexico (1.1 percent), and Maine (1.0 percent).

For self-employed individuals, the MGD estimate differs by at least 1 percent from that of the current methodology in 12 states. The MGD estimate is higher for eight of them: the District of Columbia (2.7 percent); Nevada (1.5 percent); Alaska, Indiana, and North Dakota (1.4 percent); Washington and West Virginia (1.1 percent); and Colorado (1.0 percent). The MGD estimate is lower for four states: Montana (−2.3 percent), Wyoming (−1.7 percent), Arkansas (−1.1 percent), and Louisiana (−1.0 percent).

The percentage changes in the estimated amounts of OASDI taxable earnings between the two methodologies are generally small, as was expected; but are the percentage changes in estimated earnings proportional with the percentage changes in the estimated numbers of workers? That is, if the MGD estimate of workers in a given state is 1.5 percent lower than the current-methodology estimate, is there a corresponding decrease in the estimated amount of taxable OASDI earnings?

Table 12 shows the percentage differences between the current-methodology estimates and the MGD-process estimates of both the number of workers with OASDI taxable earnings (from Table 10) and the amounts of those earnings (from Table 11), and presents the percentage-point differences between those two measures. For all workers, the percentage-point difference between the two measures exceeds 0.5 in only four states: Montana and the District of Columbia (1.1 percentage points), South Dakota (0.9 percentage point), and Nebraska (0.6 percentage point). The results for wage and salary workers are similar.

For self-employed individuals, the two measures differ by 0.5 percentage point or more in nine states: Arkansas (1.2 percentage points); New Hampshire (1.1 percentage points); Indiana (0.9 percentage point); Connecticut, Florida, and South Carolina (0.8 percentage point); Nevada and Alaska (0.7 percentage point); and Washington (0.6 percentage point).

Estimates by State and Sex

The evaluation continues by comparing the results of the current methodology and the MGD process for identifying the sex of workers. Table 13 shows that the match rate of the reported sex for all workers is 99.3 percent. However, the MGD file includes two categories of incomplete data, Missing and Unknown, that are not duplicated in the CWHS microdata file. If the records for the 5,237 workers with Missing values and the 648 workers with Unknown values for sex are removed from the MGD file, the match rate is 99.6 percent (not shown).

Table 13 shows that the sex identified in the current methodology and the MGD process matches for at least 99 percent of all workers and for wage and salary workers in all states except Iowa, which has a 98.9 percent match rate. For self-employed individuals, the match rate by sex is lower than 99 percent in seven states. However, the match rate for all seven of those states exceeds 98.4 percent.

Table 14 repeats Table 12 with detail by sex; that is, it shows the percentage differences between the current-methodology estimates and the MGD-process estimates of both the number of workers with OASDI taxable earnings and the amounts of those earnings, and presents the percentage-point differences between those two measures. For all workers, the percentage-point difference between the number of workers and the amount of taxable OASDI earnings exceeds 0.5 in only nine states (Louisiana, Ohio, South Dakota, and Wyoming, for women; the District of Columbia, Maine, Nebraska, and Oklahoma, for men; and Montana, for both).

Estimates by State and Age

Earnings and Employment includes tables showing the numbers of workers with taxable Social Security and Medicare earnings by state or other area, sex, and age. Table 15 compares the worker ages identified using the current methodology and the MGD process and shows that the ages assigned by the MGD process match those identified under the current methodology for 98.9 percent of workers overall. However, the records for 5,734 workers in the MGD file are missing an age value and therefore cannot match the current-methodology age value. Removing these records from consideration would produce a “true” match rate of 99.2 percent. Further, for an additional 0.6 percent of all workers, the age assigned in the MGD file is within 2 years (plus or minus) of the age assigned by the current methodology. Combining the true match rate and the share of workers whose ages are within 2 years of the current-methodology assigned age would result in a 99.8 percent match rate for all workers.

Estimates by age in Earnings and Employment are shown for each of nine age groups. In many states, some of those categories contain relatively few workers. Specifically, five of the age groups (under 20, 60–61, 62–64, 65–69, and 70 or older) contain far fewer workers than the other four. As previously noted, lower numbers of workers in these categories are likely to result in larger percentage differences in the estimates by age between the current methodology and the MGD process. Comparing the differences between the two processes is therefore problematic because many of the larger percentage changes may reflect relatively small changes in the number of workers.

Table 16 shows how the MGD-estimated counts of workers with OASDI taxable earnings by sex and age differ from the current-methodology estimates (after removing the MGD records for 5,734 workers with a missing value for age). Because the differences are slight, the MGD assignment of age requires no further evaluation.

Estimates by County

Earnings and Employment includes 102 tables showing county-level statistics: 51 (one for each state plus one for Puerto Rico) for workers covered under Social Security and 51 for those covered under Medicare. Each table presents worker counts, taxable earnings, and trust fund contributions, by sex, for all workers, wage and salary workers, and self-employed individuals.

Evaluating the results of the MGD process at the county level is much more complex than assessing the estimates shown by state, sex, and age for three primary reasons. First, the current methodology and the MGD process use distinct sets of county codes and names. The current methodology uses SSA-designated SCCs while the MGD process uses Federal Information Processing Standards SCCs. As a result, ORES must confirm the consistency of the county names used in the two methodologies and determine if any counties are identified in one process and not the other. For example, some states recognize independent cities as well as counties.¹⁷ Earnings and Employment includes estimates for those independent cities. Are each of those independent cities also identified in the MGD file?

Second, data nondisclosure requirements significantly affect the quantity of county-level estimates that SSA may publish. More than one-half of the cells showing county-level data in the Earnings and Employment tables are suppressed to comply with disclosure restrictions. Primary cell suppression rules require any unweighted estimate of fewer than 10 workers to be suppressed. For tables that include sex, age, or type-of-earnings breakdowns, SSA must also apply secondary cell suppression. Consider a small county with an unweighted count of 25 workers. If 13 are men and 12 are women, SSA can publish estimates for the total number of workers and workers by sex for this county. However, if 16 of the workers are women and only nine are men, secondary data disclosure rules require SSA to suppress the estimates by sex and publish only the total number of workers for the county (because suppressing only the number of men would leave that value open to computation). Estimates with breakdowns by age and type of earnings only increase the instances that require cell suppression. More than one-half of the estimates of self-employed individuals are subject to primary cell suppression, which requires SSA to apply secondary cell suppression to the corresponding estimates for wage and salary workers.

Third, evaluating county-level estimates is complicated by their sheer volume. In the 2017 edition of Earnings and Employment, the tables showing county-level data for Social Security–covered workers contain 88,182 discreet estimates, as do the tables for Medicare-covered workers.

The comparison of the SCCs assigned via the current methodology and the MGD process takes place in two steps. The first step involves aligning the universe of geographic identifiers: comparing all possible state and county combinations in the two methodologies irrespective of the actual distribution of workers. This step ensures that the SCCs include all possible state and county combinations in both methodologies and not just the combinations found in the CWHS microdata file. This first step allows a direct comparison between the resulting distribution of workers under both methodologies. The second step simply extends the first step by directly comparing the numbers of workers estimated under each methodology.

Identifying All Possible State and County Combinations and Removing Incomplete or Incompatible Records

The universe of state and county combinations is drawn from the current methodology's LABELS file (Chart 1) and the MGD file. Box 1 shows an excerpt from the LABELS file and provides examples of the geographic coding it contains. For example, row 1 shows the codes that designate workers with a missing value for both the state and county, row 2 shows the codes for workers with an “unknown” state code and a missing county value, and row 3 contains the codes for workers with the Alabama state code and a missing county value. Rows 4 through 10 and 69–70 show the data fields that apply when state and county codes are assigned. Row 71 applies the “Statewide” identifier in the county name field and indicates data for all workers in Alabama.

To focus the evaluation on counties, ORES removed LABELS file records with the values American Samoa, Armed Forces, District of Columbia, Guam, International Operations, Northern Mariana Islands, Other, Reserves, UNKNOWN, or Virgin Islands in the STATE_NAME field; and with Statewide or no value in the COUNTY_NAME field. ORES used the resulting adjusted LABELS file in comparing the current methodology with the MGD process.

The 2017 MGD file contains records for 178,863,694 workers. To limit the file to records that are relevant for comparison, ORES removed the records of workers with the values American Samoa, District of Columbia, Federated State of Micronesia, Guam, Marshall Islands, Northern Mariana Islands, Palau, UNKNOWN, or Virgin Islands in the STATE_NAME field; and UNKNOWN in the COUNTY_NAME field.

This step removed records for 1,031,176 workers from the file, leaving 177,832,518 workers represented in the modified MGD file. Those records were then exported to a separate data file that sorts the workers across the U.S. counties, which can be compared with the data from the current methodology's modified LABELS file. In both files, the county-level records are arranged by state.

The comparison begins by ensuring that the entries in the state name data fields are consistent in both files and confirming that the number of observations (that is, counties) in the state tables match. For the tax year 2017 data, this process revealed duplicate entries for Waukesha County in Wisconsin (with the same SCC) and two different SCCs associated with Teton County in Wyoming, enabling ORES to remove the duplicate records from the LABELS file.

Next, ORES compared the county names in the two files and identified nonmatching names. This review revealed mismatches caused by variant spellings of the county names, such as the following:

After standardizing the spelling of county names, ORES identified the following counties (or county equivalents) in the LABELS file but not the MGD file:

Kusilvak Census Area in Alaska and Oglala Lakota County in South Dakota were, until 2015, named Wade Hampton Census Area and Shannon County, respectively. The part of Yellowstone National Park located in Montana was a county equivalent until 1978, when the area was absorbed by two adjacent counties.¹⁸ Administrative districts called municipalities are the Puerto Rican equivalent of counties, but because no municipality is named “Puerto Rico,” that term's appearance in the county-name data field seems to be similar to “Statewide,” or a proxy for the entire territory. Of the four independent cities in Virginia named in LABELS but not in the MGD file, Clifton Forge and South Boston voluntarily dissolved their charters as independent cities (in 2001 and 1994, respectively), and became part of their surrounding counties; Nansemond merged with Suffolk Independent City in 1974; and Emporia remains an independent city. ORES is in the process of standardizing the county names in the two files.

Comparing County Assignments

The final step in compiling the data that allows a comparison of the two methodologies' county assignments is to compare the number of counties allocated to each state via the two processes. The number of allocated counties differed in six states: The current methodology allocated one more county to Alaska, South Dakota, and Virginia than the MGD process did, and the MGD process allocated one more county to Montana, Puerto Rico, and Texas, and two more counties to Virginia, than the current methodology did.

Regarding the counties that are identified in the current methodology but not the MGD process, 35 workers were assigned by the current process to Kusilvak Census Area in Alaska, 54 were assigned to Oglala Lakota County in South Dakota, and 66 were assigned to the independent city of Emporia in Virginia. Conversely, the MGD process assigned 1 worker to Wibaux County in Montana, 2 workers to Aibonito Municipality in Puerto Rico, 4 workers to Borden County in Texas, and 91 workers to Manassas Park Independent City and 79 workers to Poquoson Independent City in Virginia; the current methodology assigned no workers to those areas. The records for these 332 workers in 8 areas were removed from the merged county-comparison file because the evaluation requires the state and county names to align across the two methodologies. The resulting file contains records for 1,731,546 workers and 3,202 counties.

Evaluating the County-Level Estimates

With the preliminary processes complete, the resulting merged file allows a comparison of current-methodology and MGD-process county-level estimates of worker counts by type of earnings. Note that the MGD process, unlike the current methodology, does not generate any county-level estimates if the microdata file has no workers with a given type of earnings in that county. This has a pronounced effect on the number of counties to which self-employed individuals are assigned.

Table 17 shows the numbers and percentages of workers whose records have matching and nonmatching county assignments by type of earnings. More than 97 percent of the individuals represented in the county-comparison file have earnings that are taxable under Social Security. Among all workers with taxable earnings, the county-assignment match rate is 94.5 percent. Workers with OASDI taxable wage and salary earnings account for 90.3 percent of the workers in the county-comparison file. For them, the match rate for county assignments is also 94.5 percent.

Nearly 11 percent of workers represented in the county-comparison file have OASDI taxable self-employment income. Among them, the match rate for county assignments is 92.3 percent. Note that because the number of self-employed individuals is far less than that of wage and salary workers, 62 counties have at least one of the latter but none of the former, resulting in fewer counties assigned for self-employed individuals (3,140) than for wage and salary workers (3,202). Table 18 shows the county-assignment match rates by state.¹⁹ The match rates for all workers range from a high of 99.3 percent for Hawaii to a low of 80.3 percent for Virginia.

As noted earlier, the critical limitation of the current methodology is that data disclosure restrictions require some estimates to be suppressed, and estimates based on a 1-percent sample of self-employed individuals fall under that rule in many counties. Table 19 shows, for each state, the percentage distribution of counties by the number of self-employed workers with Social Security taxable earnings who have records assigned by the current methodology to that county. Nearly 30 percent of Alabama's 67 counties, for example, have fewer than 10 self-employed individuals assigned to them in the current methodology, and primary cell suppression rules require SSA to suppress the estimates for those counties in Earnings and Employment. Although the estimated number of wage and salary workers exceeds 10 in most if not all of those counties, secondary cell suppression rules require SSA to suppress those estimates as well. In total, more than 37 percent of the county estimates for self-employed individuals (and, therefore, also for wage and salary workers) must be suppressed.

Further, as noted earlier, publishing county-level estimates by worker sex requires that a county contain a minimum of 20 self-employed individuals to meet the data disclosure threshold. Adding this restriction requires SSA to suppress more than 60 percent of the county-level estimates for self-employed individuals, and secondary cell suppression applies to the corresponding county estimates for wage and salary workers. Given the complexity of incorporating data disclosure procedures into the large number of county-level estimates that would have to be generated, ORES decided to forgo any attempt to compare the estimates of the amount of taxable OASDI earnings for the two methodologies. These circumstances highlight the importance of using a much larger sample of workers to generate the annual employment and earnings estimates.

Conclusion

This article presents two distinct assessments of the MGD process: a procedural evaluation of the completeness and consistency of the MGD data produced over time and a comparison of current-methodology and MGD-process assignment of residential location and demographic data for earners in tax year 2017. The procedural evaluation shows very consistent outcomes for the MGD process across tax years 2015–2020. Although the procedural evaluation identified some minor issues that ORES is investigating, it found that the MGD process is robust and working as expected. In comparing the estimated number of all workers with taxable earnings, the state code assigned in the MGD process matched that of the current methodology for 98.9 percent of the records (Table 9). As was expected prior to the evaluation, the match rate for county assignments was lower, at 94.5 percent (Table 17). The primary reason for occasional disagreement between the two methodologies is a difference in the level of detail with which geographic information is recorded. The current methodology assigns county codes using only the first five letters of the city name and the five-digit ZIP Codes reported on the workers' tax forms. Additionally, the current process uses the SCCs generated for two different data files within the CWHS system and does not consistently select the code from only one of those files. ORES believes that the MGD process is more accurate because it relies on more recently developed software that uses the full address information reported on workers' tax forms to assign SCCs.

Worker sex and age identified in the MGD process match those identified in the current methodology at very high rates (99.3 percent and 98.9 percent, respectively; Tables 13 and 15). The current methodology extracts age and sex information from either of two different CWHS files. In theory, the values in these files should match. Although the nonmatch rates for sex and age are low, ORES believes that the MGD process is the more accurate of the two methodologies because it assigns sex and age identifiers based on a single authoritative source.

The evaluation's results are encouraging. ORES will continue developing the MGD process to provide a streamlined, modern method of generating its annual earnings estimates using a much larger sample of earners. Using a larger sample will eliminate the need for cell suppression in many instances and enable ORES statistical publications to report county-level estimates with much greater depth and accuracy.