There’s a lot going on behind the scenes when a security patch is released by Umbraco HQ.
As the leading open-source .Net CMS, with millions of live websites, Umbraco HQ needs to balance a mix of stakeholder incentives while maintaining the codebase’s security. This is a challenging feat from a technical, systems, and human point of view, as some of these incentives are in contention with each other.
When a new security issue is raised with Umbraco through its security disclosure process, HQ takes the following steps:
- Review the submission with the disclosing analyst to ensure it’s got all the required details, and to ensure that it’s a legitimate vulnerability
- Investigate the issue and develop a patch for it
- Create a Severity Rating for the issue (none, low, medium, high, critical)
- If the patch is rated high or critical severity, release a heads-up notification to the community that a patch is incoming
- Publish the patch, publish a security advisory blog post, and roll out to Umbraco Cloud
When reviewing an issue and creating a CVE severity rating, there are three main stakeholders to keep in mind, each with discrete incentives: Umbraco HQ; development agencies; and the disclosing security analyst. For Umbraco, the main incentive is to have the disclosure rating be as accurate as possible, as there’s reputational risk to frequently overstating or understating severities. For the security analyst, the main incentive is to achieve a high severity rating, because kudos & career opportunities are improved for analysts with more severe disclosures. For the application maintainer & the development agency, the incentive is to have the severity be as low as possible, so that they don’t need to spend time & effort deploying the fix.
To ensure the voices of each stakeholder is heard, and disclosures are consistently reliable, HQ setup a team this year within the Community Security & Privacy Team, to help HQ identify the correct CVE rating for new security advisories.
When timing a patch deployment, Umbraco HQ face a challenging situation (similar to most open-source projects), where there’s a gap between a patch’s source code being publicly available for developers to see, and the patch binaries being publicly available for maintainers to deploy.
For most security issues, HQ will release a heads-up notification that a patch is coming in a few days (see this example from Oct 2022). This is so that site maintainers can be on-hand to deploy the fix as soon as it becomes available. Security patches are usually published at a predictable time on a Tuesday morning. This schedule is important for Umbraco, as it’s a globally adopted CMS underpinning websites for multinational enterprises, governments, and regulated organisations - prime targets for nefarious activities.
When creating a fix for a security issue, HQ works in private forks of the source-code, hidden from public view. Once the patch is created, tested, and ready for production, HQ needs to merge the private PR into the public repository on GitHub, so that the application’s build system can create a new release version’s binaries, and publish them to Nuget for devs to deploy to production sites. The build system takes a couple of hours. This means that there’s a gap where the fundamental details of the vulnerability are public, and all Umbraco sites are vulnerable.
