The struggle to reduce bug-fixing time is real
There are many reasons why we want a bug fixed as soon as we can, but there are also plenty of reasons why doing it âright nowâ is not an option. This phenomenon starts at the side of the developers. The average time to fix a bug seems to vary depending on the platform the bug was found in. What is one group doing better and can the others take lessons from that? Or is it something we have to take as it comes?
“Bug-fixing time” refers to the time required to fix known bugs. So, on a per bug basis it is the time between being made aware of an existing bug and issuing a fix for the bug. The ability to better understand and predict bug-fixing time can help a project team better estimate software maintenance efforts and better manage software projects.
Reasons to fix ASAP
There are some very obvious reasons why we want to push and install bug fixes as soon as possible.
- Improved security by fixing the vulnerability.
- Even if a vulnerability is found by a researcher taking the high road of responsible disclosure, once the cat is out of the bag, there is a good chance others will be able to duplicate the researcher’s effort. This could result in a zero-day vulnerability.
- When you are working on a new version, a critical bug in the old version is holding you back as long as you donât know how to fix it.
- If the published timeline shows it has taken months to fix a bug it reflects badly on your company, and could lead customers to question whether you care about security.
In general, you can say that the bug-fixing time is an important factor for bug related analysis, such as measuring software quality. Having your software considered to be âbuggyâ does not helps sales in any way. But situations may arise when you need to prioritize what needs to be fixed first.
Differences in platform
Last month, the Project Zero team at Google looked at fixed bugs that were reported between January 2019 and December 2021. During this period, Project Zero reported 376 issues to vendors under their standard 90-day deadline.
When reading the data, it is important to note that the number of issues is too small and not chosen randomly enough to give a full picture, but it gives you an idea at least.
| Vendor | Total bugs | Fixed by day 90 | Fixed during grace period | Exceeded deadline and grace period | Avg days to fix |
|---|---|---|---|---|---|
| Apple | 84 | 73 (87%) | 7 (8%) | 4 (5%) | 69 |
| Microsoft | 80 | 61 (76%) | 15 (19%) | 4 (5%) | 83 |
| 56 | 53 (95%) | 2 (4%) | 1 (2%) | 44 | |
| Linux | 25 | 24 (96%) | 0 (0%) | 1 (4%) | 25 |
| Adobe | 19 | 15 (79%) | 4 (21%) | 0 (0%) | 65 |
| Mozilla | 10 | 9 (90%) | 1 (10%) | 0 (0%) | 46 |
| Samsung | 10 | 8 (80%) | 2 (20%) | 0 (0%) | 72 |
Overall, the data show that almost all of the big vendors here are coming in under 90 days, on average.
Complaints from bug bounty hunters
At this point it should be pointed out that bugs reported by the Project Zero team are reported to vendors directly and will be taken very seriously by the vendors.
Individual bounty hunters, however, have been complaining about getting their bugs accepted. For example, in January we saw CVE-2022-22587, a vulnerability in Appleâs IOMobileFrameBuffer, where a malicious app could execute random code with kernel privileges. This vulnerability ended up being a zero-day vulnerability that was exploited in the wild after one of them posted a Proof-of-Concept (PoC).
Many researchers that donât want to report to vendors directly make use of the Zero-Day-Initiative (ZDI). The ZDI was created to encourage the reporting of zero-day vulnerabilities privately to the affected vendors by financially rewarding researchers, although there have been complaints from researchers that they didnât feel they were taken seriously by the ZDI.
The next step
So, yes, it’s important to fix vulnerabilities ASAP, but why does it take so long sometimes before these fixes and patches get installed?
According to recent podcast guest Jess Dodson, the problem of patching isnât just a problem of resourcesâtime, staffing, fundingâbut also of mindset. For some organizations, refusing to patch almost brings with it a bizarre sense of pride, Dodson said.
Finally, even if you are not a Federal Civilian Executive Branch (FCEB) agency that needs to follow the Binding Operation Directive 22-01, the CISA list known as the Known Exploited Vulnerabilities Catalogcan act as a good guideline for your patch management strategy. This catalog provides FCEB agencies with a list of vulnerabilities that are known to be exploited in the wild and gives the agencies a due date by when the vulnerability needs to be patched in their organization.
