今日推荐英文原文：《What if the Customer Can’t Reproduce Your Bug?》
今日推荐英文原文：《What if the Customer Can’t Reproduce Your Bug?》作者：Sandra Parker
推荐理由：在其他人无法重现可能存在的 bug 时寻找原因的方法
What if the Customer Can’t Reproduce Your Bug?
“What are we doing wrong?”Software quality assurance and testing are complicated procedures that require dedication, technical expertise, an eye for detail, and hours of work. Testers are people who dive deep into your product and have a major impact on the final project’s outcomes. That’s why it is so important that the product owners understand well not only the importance of software testing but its pain points from within for more effective communication.
Today’s article focuses on bug reproduction issues that usually remain behind the scenes for everyone but testers, which isn’t always good for the whole software project’s quality.
“Can’t Reproduce” — What Are We Doing Wrong?Many software testers sooner or later face a problem where a customer/their team can’t reproduce a reported bug. A bug that you spent time and effort investigating. You tried so hard to report it back in a clear and concise manner, and yet they throw the “can’t reproduce” verdict in your face.
Trouble communicating is the top reason why anyone on the team fails to reproduce a bug. Either a vague bug summary was provided or it was a great one that no one wants to read.
Someone might experience troubles reproducing a bug because of a tester’s actions as well. For example, they might have provided a vague bug summary, listed uncertain steps to reproduce it, or enclosed a useless attachment that either doesn’t demonstrate the bug correctly or doesn’t want to open at all. It’s even worse if you didn’t check the bug reproduction before getting to the report and it stopped reproducing after the cache was cleared.
In case you are sure that the bug was reported correctly and can be found by the application end users, you have no other choice but to defend it. This is achieved through communication with all stakeholders who can investigate and fix that bug in the future.
Questions You Should Ask Your QA Team to Find Out Why They Can’t Reproduce Your Bug
- Do you use the same testing environments (platform, browser, browser version, device, OS, screen resolution)?
- What type of internet connection did you and the testing team use?
- Has the team accounted for all the preconditions and steps for bug reproduction that you specified?
- Does your expected result of software behavior match those of the team as well as the expected testing outcomes stated in the project requirements?
- Do you or your team use any third-party apps or browser extensions that could affect the software’s performance?
Let’s Look at All These Questions in More DetailFirst of all, in most cases, the team doesn’t finish reading your bug. In the best-case scenario, a person trying to reproduce your bug has read more than the bug summary and its attachment. That’s why it is no wonder some of the bug report attributes often get missed. Yes, it is as unfortunate as it sounds.
Do your testing environments match?You have to specify which exact environment the team checked your bug in.
It is assumed that the environment of a reproducing bug was stated in the description of it. Therefore, you have to double-check whether your colleague who’s trying to reproduce the bug is using the same platform to do it (browser, device, operating system, screen resolution) as you did.
As an example, we once had a case where our tester reported a bug that could only be reproduced on iPhone 6 but no other versions. That’s why they specified the particular environment in the bug report description. A developer who had to process that bug report didn’t pay attention to the fact that the bug can only be seen on a specified device. Since they only had an iPhone 7 on hand, they couldn’t reproduce the reported issue. Only after we double-checked everything was the misunderstanding finally solved. The bug that used to occur on iPhone 6 exclusively was found and fixed.
What type of internet connection did you and the testing team use?For the most part, such a question should be asked about device-associated bugs because the type of internet connection (wifi, 3G, 4G, etc.) can affect the performance of an application or site, so this information is really important and has to be clarified.
Although, it turned out this issue is not only relevant for mobile. On one of our projects, we found an interesting bug: The online store did not display a map of post offices available for certain delivery methods. We discovered it when everyone was working from home due to the quarantine. The customer team could not reproduce the bug. For them, it seemed as if everything was perfectly displayed. After a few discussions, we decided to put the bug under investigation. After a part of our team got back to the office, we found that the bug could no longer be reproduced.
Since some of the employees continued working remotely, we managed to discover a fascinating fact. With a home internet connection, the bug kept reproducing. Meanwhile, in our office, where the connection was more stable, the post office map was displayed successfully — just like on the customer’s side. That’s when we decided to include the information on the internet connection type and its speed in the bug report.
As you can see, the internet connection itself can be a reason for a bug. In our case, it led to certain customers being unable to place their orders in that online store.
Has the team accounted for all the preconditions and steps for bug reproduction that you specified?Getting back to the issue of the team not paying enough attention to the bug summary and its attachment, sometimes the team misses some important information — namely how to reproduce a particular bug.
Frequently, qualified employees tend to think they have mastered a project’s functionality to the point where they don’t need to examine the steps to reproduce the bug and its preconditions thoroughly. That’s why it’s worth double-checking how exactly developers tried to reproduce your bug and whether they missed a step or didn’t take into account any preconditions.
We had a case where a bug was reproducing only after we marked one out of four checkboxes for one country in the settings of software under testing. We added all this information to the bug description, which our colleague didn’t finish reading, and as a result, they could not reproduce the bug since they were following the usual steps to reproduce one.
Does your expected result of software behavior match those of the team as well as the expected testing outcomes stated in the project requirements?It happens. You reported a bug because it didn’t match the expected result specified in the requirements or even ones that weren’t specified, but you know exactly how the functionality should perform based on your experience, general standards, or just common sense. You should discuss this issue with your colleagues. Maybe someone on the team is relying on inaccurate information about the expected result of the software behavior. Do you or your team use any third-party apps or browser extensions that could affect the software’s performance?
And the last question to discuss is the one that could be asked by the bug report author. Sometimes, certain apps can conflict with each other. Naturally, this provokes software errors that QA analysts classify as bugs. Even VPNs can affect the work. Therefore, you have to make sure during the bug reproduction that there are no third-party applications installed that are able to affect the performance of the tested product and mislead the entire team. For example, you’re using a virtual machine for testing that can be less stable than the original platform or a browser itself. This also has to be clarified.
Our colleague once reported a bug displaying unnecessary web elements on a page. No one else on the team could reproduce that bug. As it was later revealed, the bug author used a spelling assistant application that was reacting to certain web elements.
Wrapping UpThe aim of a software tester is to assure stable and error-free software functionality and prevent users from spotting bugs. That’s why we examine and classify all the bugs you may ever encounter. We take full responsibility for the quality of a tested product, a high-quality bug report if necessary, and the high-quality processing of it.
If you find a bug that seems like a serious reason to cancel the product release and the team cannot reproduce it, you should definitely find out the reasons why by asking the questions listed above. This will not only shape your professional reputation but also affect the image of the entire QA department in the eyes of the development team. A company’s position in front of its end users is a fragile thing that is so hard to get yet so easy to lose through a single serious mistake.