How We Analyzed the Gender Pay Gap at Glassdoor

At Glassdoor, pay transparency is something we believe helps both job seekers and employers alike. In an attempt to put that belief into practice, today we released an analysis of the “gender pay gap” in Glassdoor’s own payroll data. In this post, I’ll explain what we found when we looked for gender bias in Glassdoor’s compensation program. In doing so, I’ll also outline the simple statistical method we used, in the hope that it will help other companies who are committed to studying their compensation data and addressing gender pay equity. The Data As a first step, we gathered basic payroll information from Glassdoor’s HR department. Here is the information we gathered for of our full-time U.S. employees:

•      Gender
•      Birth Year
•      Job Title
•      Job Seniority Level
•      Company Department
•      Score on Two Most Recent Performance Evaluations
•      Current Base Salary
•      Current Bonuses and Incentive Pay

To safeguard privacy, we used anonymized data only, with no names or other personally identifying information. In our analysis, we examined pay for 563 employees as of March 2016.  It’s important to note that like most technology companies, Glassdoor’s employee population is made up of more men (55 percent) than women (45 percent), and women comprise 42 percent of senior leadership, defined as director and above. The Method To examine whether there is evidence of gender bias in Glassdoor’s payroll, we applied the same method outlined in our March 2016 study of the gender pay gap around the world, Demystifying the Gender Pay Gap. Here is the basic idea. We write down a simple model of which factors predict the salary of workers—age, job title, seniority, and so on. Gender is also one of those predictors. In this model, we assigned the gender factor as equal to one if an employee is male and zero if female. If there really is no gender bias in pay, this gender variable shouldn’t add any predictive power to the model. Instead, it should be a statistical zero in the model.[1] To be more specific for the data experts, let Salary_i be the earnings for Glassdoor employee i. We estimate the following regression model by ordinary least squares (OLS):

Salary_i = Controls_iβ₁ + Gender_iβ₂ + ε_i

where Controls_i is a set of controls for worker and job characteristics including age, job title, department, job level, and performance scores on the last two employee reviews. Gender_i is a dummy variable equal to zero for women and one for men. The estimated coefficient for β₂ tells us the male pay advantage (if there is one) after all other job and worker characteristics in Controls_i are accounted for. What We Found First, we looked at the raw or “unadjusted” pay gap. That’s the simple average of men’s pay compared to women’s pay, without accounting for any differences between job titles, seniority, experience, or performance reviews. On average, men earned $107,371 in base pay in 2015, while women earned $85,962 per year. That’s a pay gap of $21,410 or about 19.9 percent. The gap was slightly larger for total compensation: $29,566 or 20.7 percent. This is shown in the table below.

Base Pay	Average Base Pay	Average Total Compensation	Observations
Female	$85,962	$113,107	253
Male	$107,371	$142,673	310
"Unadjusted" Gap	$21,410	$29,566
Percent of Male Pay	19.9%	20.7%

As we explain in our full study, one drawback of this simple view is that it doesn’t really make an apples-to-apples comparison between men and women. Male and female workers might work in systematically different roles in the company—with different salary schedules. So the average pay difference may be due to working in different roles, not gender. To identify whether there is a gender pay gap among similar workers, we need to filter out pay differences due to job title and seniority. That’s what we do in the tables below. In the first table, we look for gender bias in base pay at Glassdoor. Starting from the left column, we show the simple difference in average male and female pay before any statistical controls (what we call the “unadjusted” pay gap). In the second column, we add controls for age (a good proxy for years of experience) and ratings on our last two employee performance reviews. Finally, in the third column we add all controls for job title, department, and job seniority level (what we call the “adjusted” pay gap). As above, with no controls men earn $21,410 more than women on average. However, adding controls for age and performance reviews that gap shrinks to $15,776. Finally, adding all controls the “adjusted” pay gap shrinks to -$425, which is a slight female pay advantage (but not statistically significant). Thus, once we make an apples-to-apples comparison of workers, there’s no material difference in pay by gender at Glassdoor.

Base Pay	No Controls ("Unadjusted")	Adding Controls for Age and Performance	Adding Controls for Department, Seniority Level, and Job Title ("Adjusted")
Male Pay Advantage	$21,410	$15,776	-$425
Statistically Significant?	Yes	Yes	No
P-value	0.00	0.00	0.54

The table below repeats this exercise for total compensation, which includes employee bonuses. As above, with no statistical controls men earn $29,566 more than women on average. But after controlling for differences between jobs and workers the “adjusted” pay gap shrinks to -$375, which is again not a statistically significant difference.

Total Compensation	No Controls ("Unadjusted")	Adding Controls for Age and Performance	Adding Controls for Department, Seniority Level, and Job Title ("Adjusted")
Male Pay Advantage	$29,566	$23,243	-$375
Statistically Significant?	Yes	Yes	No
P-value	0.00	0.00	0.69

What About Bonuses? Another question we asked is whether our system of employee performance reviews is gender biased. Twice per year Glassdoor managers assign performance ratings to employees, and those ratings directly influence bonuses. Are men and women treated fairly by this system? First we looked at whether Glassdoor performance ratings favor men or women. The table below shows the average performance rating for men vs. women from our most recent round of bonuses in May 2016. As is clear from the table, there is no statistically significant difference between performance reviews for men vs. women. On average, women scored 3.11 (on a scale of 1 to 5), while men scored 3.03. So there’s no evidence of bias in our performance rating system.

	Average May 2016 Performance Review Score
Male	3.03
Female	3.11

What about the actual bonuses that Glassdoor employees were paid out? In the table below, we show our statistical analysis of gender bias in bonuses. Before any controls, average male bonuses were about $1,231 more than average female bonuses. But almost all of this difference is due to differences in job seniority—more senior roles have higher bonus percentage targets. Once we control for that fact, the gap reduces to a -$299 female pay advantage, which is not statistically significant.

	No Controls ("Unadjusted")	Adding Controls for Age and Performance	Adding Controls for Department, Seniority Level, and Job Title ("Adjusted")
Male Bonus Advantage	$1,231	$1,073	-$299
Statistically Significant?	Yes	Yes	No
P-value	0.02	0.02	0.27

A Closer Look at Engineers Given the disproportionate number of men versus women in STEM fields, one area we wanted to examine was whether there is a gender pay gap within Glassdoor’s engineering department. Although there’s no evidence of a company wide pay gap, is there a gap among our most technical software developers, database architects and other engineers? To answer that question, we make a slight adjustment to our statistical work above. To test for differences by company department, we include an “interaction term” or gender x department variable in our regression model. That lets us identify whether gender differences in pay are bigger (or smaller) in some departments versus others.[2] When we do so, we find no significant gender pay gap within our engineering department. Men on average earn $2,900 per year more than women in terms of base pay. But after adding statistical controls the pay gap drops to -$3,132—a slight pay advantage for female engineers, but one that isn’t statistically significant. The results for total compensation within our engineering department are nearly identical. Conclusion In recent years, Glassdoor has been a leader in raising awareness of the gender pay gap and the need for greater salary transparency. With this effort, our aim is to move beyond raising awareness toward taking action to address gender equity in the workplace. As a first step, it is important for employers to make the effort to systematically examine their payroll data, using some version of the above method to test for systematic gender bias in compensation. Looking at simple averages alone will not paint a complete picture of gender balance in pay. Doing so can not only help avoid serious pay equity problems down the line, but can also help improve employer brand—building good will with employees through greater transparency about who is earning what, and why. To read more, check out the Glassdoor Blog.  Footnotes [1] We also performed a standard Oaxaca-Blinder decomposition on our sample of Glassdoor payroll data, which is available upon request. For additional background on our methodology, see our complete study, Demystifying the Gender Pay Gap. [2] To estimate the “adjusted” pay gap in department i, we sum together the estimated coefficients from the “male” indicator variable and the “male x department i” variable, a standard way to test for differences among groups. This shows whether any male pay advantage is magnified by being both male and working in department i.