The High Cost of a Tech Defect
Finding a software defect during development can be a nuisance for a technology team. The challenges inherent in discovering a defect late in the development cycle, during QA, or God forbid after being moved to production, can be crippling.
To make matters even more dicey, in multi-tenant environments the implications on the system and reputation with end clients can lead to serious repercussions and inefficiencies.
Test Effectivness Using Defect Containment Efficiency
A good testing environment ensures that automated testing is put in place at the beginning of the project and that user stories are governed by positive and negative test cases. Finding a problem in a story is much more quickly resolved, and the cost of the defect remains low when acceptance criteria is clear and well understood by everyone in the development team.
Having the developer working on the user story discover and resolve a defect is the next best thing. This means that that developer was able to identify a problem in their logic or the logic of the user story and could get that defect resolved early before involving the larger team. Early diagnosis and resolution means knowing what the defect is, who owns the resolution, and how to fix it. It is also critical to create a better test case so that other developers do not encounter the same challenge when they encounter the same problem.
Once a task is handed off to quality assurance (QA), the developer often believes that he or she is done and shifts to the next task. QA reads, interprets, and writes its own test cases. If a defect is found at that point, the QA resource needs to verify the issue with the developer. The process to talk to the developer has a high switching cost as the developer has to stop, context shift back to that issue, determine whether it is legitimate (sometimes requiring the involvement of the product owner), and then fix it. The code is fixed, and again tested, built, deployed, run through the regression suite a second time; all while QA resource is sitting around latent and waiting, or perhaps shifting focus and energy to another piece of code to be tested.
If the code makes it past QA and onto user acceptance testing (UAT) with the business or others, the process becomes even more burdensome. The bug has to be reported to QA who verifies that the defect is real, and the above process from the previous paragraph has to be repeated (developer fixes, tests, sends to QA, and back to UAT). The result is a loss of even more time, resource cost, and efficiency and velocity of delivery.
The worst case scenario is that the problem is identified by the customer once the software solution goes live. The pathway to resolution is riddled with people, communication, process and information friction. The customer reports the issue to a front line business interface such as a sales representative or support center, and the entire UAT to QA to developer process has to be rewound and re-initiated to resolve the defect. At this point, issues around downtime are trivial compared to the reputation damage to the brand as its promise is unfulfilled and customer satisfaction suffers. The hard cost of time, resources, and money inherent in the reconciliation of the defect are secondary.
Key questions to think about when assessing the cost of a defect include:
Is this an emergency fix?
What kind of problem is it?
How do you deploy to that customer (SaaS, etc.)?
Given the timing of a release cycle, how many people encounter the bug in the interim?
What types of testing are being employed? If a developer finds it. If QA finds it. If UAT/internal testing/business. If the customer finds it.
Software development can be a sticky wicket. Ensuring that you have sound test methodologies that include functional and non-functional testing to validate the application under test (AUT) is paramount, and each testing methodology should have a defined test objective, test strategy, and deliverables. Bottom line, the adage “an ounce of prevention is worth a pound of cure” is just as true for modern software development, and can save you, your team, and your company a lot of aggravation.
by Russ Harding, Principal Technical Consultant/Architect