The Unique Roles and Mission of an Open Source Engineering Team in OSS-backed Company

The engineering team who works on the core open source projects at a OSS-backed company is unique. With OSS commercialization getting popular, such engineering team becomes common, e.g. in companies like MongoDB, Elastic, Databricks, Confluent, etc.

Such engineering team is unique in the way that, despite all the normal, common engineering culture and practices, they have special requirements in roles and missions at the OSS companies. Here are the 4 unique areas are critical to the success of the team and the company:

1. Set the Foundation for Core Open Source Software and Project

The OSS team needs to set the foundations for both technical and non-technical areas:

For technical areas:

• design and software quality (reliability, HA, performance, scalability, forward-looking extensibility, etc)
• code quality (code readability, comment, etc)
• test coverage and quality for core software (unit, integration, E2E, etc)
• CI of core software. Usually no CD since this team doesn’t run the service themselves
• software releases (quality, cadence/frequency, maintenance, backward compatibility, etc)

For non-technical areas:

• Software development process and best practices
• PR review process and standard in open source
• Completeness of documentation

Overall, the OSS team in a OSS company is at center of the stage to drive these areas.

2. Foster and Cultivate Open Source Engineering Culture and Bridge OSS and non-OSS Culture within Company

Only people who have deep experience involved in open source understand that there are differences working in OSS v.s. within a wall-guarded company internal codebase.

“Why can’t we make a big decision and pivot in a single day in open source project?! That’s inefficiency!” “Why does someone outside our company have a saying in our project?!” “Why can’t we just change the public API and force users to migrate?” Such questions are good indicator that the asker haven’t had hands-on experience in a large OSS project before, and you need to be cautious of having them run an OSS engineering team.

OSS engineering best practices have very different perspective than those of working in a private codebase, as your audience is critically different - OSS engineers work with a community that do not belong to your company. The community members may have different priorities, use cases, concerns, and timeline than yours. E.g. :

• top-down decisions do not work well in OSS as there’s no management chain or leveling authority; credibility is earned in OSS community thru individual’s past involvement
• decisions needs consensus and reaching consensus can take days, if not weeks, as external stakeholders have different time schedules and concerns you have no control of; participants have to be patient, plan ahead for the timeline, and follow the community process; community does not accept arguments like “this is urgent and we have to decide today” except rare cases like security breach (e.g. recent log4j issue)
• standards and bars are held high across the board, in every details, e.g. design and code review, or as small as coding styles. Things that can sneak thru in company’s private codebase can no longer make it. Newbies in OSS may feel it’s overdoing and get frustrated easily, believing they are being specifically targeted by someone which is not the case at all
• project stability comes first. Minima breaking API changes, any major changes requires careful planning, proactive communication and warning, releases and upgrade are taken seriously and in right cadence (majority people won’t be able to upgrade more than a couple times a year), back compatibility is always desired, define well short term maintenance and long term maintenance support scope, etc
• community health and success is the top consideration, rather than individual companies
• etc

Depending on project, foundations, and licenses, the specific details can vary. E.g. one of the most strict and counter-intuitive rules is the famous “The Apache Way” , which is the standards and best practices to manage Apache Foundation projects. Apache projects have boards and OSS hierarchy to monitor the daily operations and ensure their success. MongoDB and Elastic on the other hand are relatively loose in rules as they are controlled by one company.

The key here is bridging the OSS engineering culture with a company’s general engineering culture, e.g. how to balance the decision making in OSS which can take days/weeks v.s. that within company which needs to be quick? how to communicate and collaborate with engineers working on private code with different expectation of “responsiveness”? how to blend the OSS standards with company internal ones? Not be able to address this will create conflicts and frictions across the board.

3. Own The Advocacy and Channel of User Acquisition, Marketing, and Branding

If your company is built around an OSS project, that project is “the” single most important channel for customer acquisition/engagement, product marketing and branding. Being the engineering face of the company and product to external audience, OSS engineering team must own the OSS project, the advocacy and the channel.

This include, e.g.:

Customer Acquisition & User Engagement:

• listen to and collect customer requirements/requests
• be responsive in answering questions and helping users debug issues
• be respectful and welcoming in communication
• be transparent in defining processes and making decisions, enforce same process and standards across the board
• grow OSS project adoptions
• encourage contribution and grow OSS contributors
• define and drive OSS governance, e.g. define titles and responsibilities (e.g. “committer”/“PMC” in Apache projects, “maintainer” in some projects)

Product Marketing & Branding:

• lead OSS roadmap and priority with community
• organize and speak at meetups, conferences, etc

4. Drive Open Source Business Model and Monetization by Collaborating Closely within Internal Teams

The last but not the least point is the open source engineering team *must* collaborate closely with internals teams. Being named as “open source engineering” can create the wrong illusion that you only need to focus on open source, which is not true. Growing business (revenue, profit, etc) is the ultimate goal of a company, and having a OSS project alone doesn’t make you there. To achieve the goals, the OSS team has to partner with comprehensive, cross-functional internal teams, like service engineering, product, marketing, sales, legal, support, on strategy and execution.

Potential Collaborations:

• with service engineering team on paid products (e.g. SaaS)and areas like reliability/HA, multi-tenancy, deployment, operation easiness, monitoring/alerting, scalability, etc
• with product team on product roadmap and prioritization
• with product and design team on UI and UX
• with product, marketing, sales team to decide which feature should be in OSS version v.s. enterprise/SaaS version. There are usually 3 areas of such features, 1) security, 2) collaboration, 3) DevOps (tools of observability, monitoring/alerting, CI/CD, operations, etc)
• with pre-sales or sales team on lead generation thru OSS channel
• with marketing team on new feature/product announcement
• with support and service engineering teams on customer support, SLA, oncall, etc

The last thing I want to highlight on is that, being unique doesn’t mean such a team should adopt a different engineering culture that is completely different from rest of the engineering team. On contrary, they should share, on higher level, the same values and same mission/visions, and on lower level details, the same performance review criteria, etc. Taking it as a divergence is a total misunderstanding of this blog and the intentions here.