The State of Diversity and Inclusion in the ASF Community: A Pulse Check

Posted by The ASF Community, Foundation

By: Zixuan Feng, Mariam Guizani, Anita Sarma (Oregon State University)

“Our mission is to build equity in our community by developing tools and frameworks to foster inclusion and increase diversity in all phases of Apache projects.

We are guided by the values of respect, empathy, openness, patience, candor & dependability.” 

— [Apache Diversity and Inclusion]

In recent years, there has been a growing recognition of the significance of diversity, equity, and inclusion (DEI) in open-source software (OSS) communities. Diverse OSS communities benefit from a range of backgrounds, qualifications, and perspectives, resulting in more innovative software products and greater social equity. Many OSS communities, such as the Apache Software Foundation (ASF), have launched DEI initiatives. However,  to the best of our knowledge, no empirical research has investigated the evolution of the state of DEI in OSS. Without a pulse check on the state of DEI, the ASF OSS community lacks an understanding of what has improved and what requires more attention. It is crucial to examine the evolution of DEI. 

To answer these questions, ASF hired a team from Bitergia and the Oregon State University to conduct two cross-sectional surveys, the first in 2020 and the second in 2022, to assess the state of DEI in ASF projects, including the difference in contributors’ perspectives and engagement over time. The team adopted a multidimensional approach, examining four key diversity perspectives: gender, English proficiency, seniority, and country of residence. 

We asked:

How much do contributors from traditionally underrepresented groups engage in ASF project activities; face challenges when participating in the ASF (e.g., process challenges, social challenges, project-related challenges); and perceive the state of DEI?

Through sharing the results of this study, we hope to:

  • Contribute to research on DEI by offering insights on how to increase participation and remove obstacles for contributors to the ASF OSS community who identify as coming from underrepresented backgrounds. 
  • Help identify areas of improvement for OSS communities and shed light on how diversity impacts software development. 
  • Provide guidance on designing inclusion initiatives for all contributors.

Survey Approach and Demographics

Our survey included multiple-choice and Likert scale questions. The survey was sent to the ASF Contributor List (7K email addresses). In 2020, we received 624 responses (response rate: 8.32%), and in 2022 it received 432 responses (response rate: 5.76%). The survey had similar demographic profiles across the two years, with around 5% differences among all demographic attributes.  We investigated engagement in ASF activities,  perceptions of DEI, ease of locating policy information, and frequency of locating policy, particularly for underrepresented contributors based on diversity dimensions of gender, English proficiency, seniority, country of residence, compensation, and education background.  We defined “newcomers” as those with less than a year’s experience. “English proficiency” was analyzed based on confidence levels in communication, technical/non-technical discussions, and code reviews. Respondents with “average” or above scores in all categories were categorized as “Comfortable using English.” The current Country location was also used as a proxy for cultural and communication differences and categorized as “Western” or “non-western.” 

Survey Findings: Challenges & Engagement in ASF Activities

Overall, fewer respondents in 2022 frequently encounter challenges in ASF projects, especially those from underrepresented groups (Figure 2). This shows progress has been made in reducing the frequency of challenges faced by underrepresented groups in the ASF community. However, continued efforts are needed to ensure a more inclusive and supportive environment for all contributors.

Figure 2. Do you face challenges when participating in the ASF  community (e.g., language differences, technical expertise, cultural differences, etc)? (sample size: 432).

We analyzed response proportions based on demographics. For example, for questions regarding engaging in OSS activities, we grouped responses into “often” and “rarely to never,” depending on how frequently participants engaged in the listed activities. We then compared the proportion of each demographic group that responded “often” in 2020 to the proportion that responded “often” in 2022. We show the differences between the proportions of responses in the two surveys (2020 vs. 2022), broken down by demographic attributes. The icons   indicate increases or decreases per category.

We found positive trends with increased engagement across most activities for almost all demographic groups (Figure 3). Overall, we find that more contributors who are traditionally underrepresented are increasingly participating in ASF projects, including decision-making. Some of our key findings include:

  • We saw a 30% increase in women respondents in contributing/reviewing code.
  • Respondents who did not feel comfortable using English were more involved in creating/maintaining documentation and decision-making, implying that the contributors who are not comfortable using English are participating in and have a voice in ASF projects.
  • There was an increase in the proportion of non-western respondents engaged in ASF project activities, including being community organizers, and mentors, participating in decision-making, and creating/maintaining documents, which shows that the ASF community is becoming more inclusive of contributors from non-western countries.  

Figure 3. Proportional activity engagement in 2020 vs. 2022.

Survey Findings: Perceptions about the state of DEI

Figure 4. Comparison of respondents’ agreement on the perceptions of DEI in 2020 vs. 2022.

We similarly grouped responses to the questions regarding perceptions of DEI and compared the proportion of each demographic group that responded with “agree,” respectively.

Breaking Stereotypes and Embracing Inclusivity

Both majority and underrepresented groups reported an increasing trend of mentoring others,  as shown in Figure 4. The proportion of respondents who felt members were “patronizing” them reduced across all demographics, with a reduction in double digits for contributors who are from the gender minority group (24%) or who are not comfortable using English (14%). This positive trend reflects that the ASF community is becoming more inclusive and respectful of its members. Respondents across all demographics feel they are sought out as parental figures, while fewer feel patronized.

Overcoming Barriers and Nurturing Mentorship 

We found that contribution barriers have been reduced for all demographics, with a higher proportion of contributors feeling they have an “equal chance” of getting contributions accepted, especially respondents not confident in English and Newcomers. We observed positive changes in responses to the question, “Nothing keeps me from contributing.” Specifically, 29% of women contributors and 14% of those uncomfortable using English reported that nothing stops them from contributing.  These positive trends can help ASF projects attract and retain more contributors. However, when considering finding a mentor they are comfortable with, all underrepresented demographics reported a reduction, despite a higher proportion of contributors from the women, non-binary, and transgender feeling they have a good peer network and mentors with whom they are comfortable.

Enhancing Representation and Adherence to the Code of Conduct 

A higher proportion of gender minority and newcomer respondents felt less represented in the PMC, or decision-making group. However, respondents from non-Western countries or who did not feel comfortable using English reported feeling more represented in the PMC and decision-making groups. We found that awareness of the code of conduct increased for respondents who were uncomfortable using English or were not from Western countries. Still, the proportion of newcomers or those from the gender minority group who were aware of the code of conduct decreased. We recommend listing the code of conduct in projects and emphasizing it in onboarding and training. 

Conclusion: We investigated the evolution of DEI within the ASF community through two cross-sectional surveys. The findings suggest that obstacles and challenges related to diverse backgrounds and experiences are improving for contributors from traditionally underrepresented groups. To continue promoting a diverse and inclusive community, it is essential to continue conducting pulse check-style research to update contributors on the current state of DEI. We plan to continue longitudinal studies to monitor and investigate the state of DEI, attracting more contributors’ attention and eliminating bias.

To view the full report, visit ASF Reports and Statements. Special thanks to Google for sponsoring this research work.

Further reading: The academic research paper, The State of Diversity and Inclusion in Apache: A Pulse Check by Zixuan Feng,Mariam Guizani, Marco A. Gerosa, Anita Sarma