HW9: Chapter 8 and reflections on testing

8.7: Write a scenario that could be used to help design tests for the wilderness weather station system.

Context: According to Chapter 7, Design and Implementation, the wilderness weather station system is a station deployed in a remote area. It records local weather information and periodically transfers this to a weather information system using a satellite link.

Scenario: Alex is a local weatherman in small-town North Dakota. Although his hometown is small, it’s surrounded by even more remote communities that use the wilderness weather system to transmit weather data to aggregators and reporters. Alex is putting together a weekly report in which he includes the forecast for not only his town but also the surrounding areas. His hometown’s data automatically syncs to a weather information system. However, in order to manage all of the up-to-date weather data he needs this report in one place, so he puts in a request to the satellite (SatComms) of the wilderness weather system for data from Area1. The satellite then reports the weather to the weather station and once a summary is retrieved, it’s aggregated into the requested report and returned to the weather information system/database. With the data from the remote areas now synced to the general weather information system, Alex can effectively plan for his presentation by creating graphics and charts and analyzing the data. He repeats this process for Area2 and Area3 through separate requests to the satellite. After each request has been completed, the system records that the data has been sent so that it doesn’t not resend data unncessarily.

8.10: A common approach to system testing is to test the system until the testing budget is exhausted and then deliver the system to the customers. Discuss the ethics of this approach for systems that are delivered to external customers.

I do not think that this is an ethical system. The scenario implies a lack of planning by the testing team to create a comprehensive test-suite within the budget. Particularly because this is a system that will be delivered to an external customer, it’s crucial that testing within the budget is maximized. I think it’s important here to distinguish between the case that the system was tested until the budget ran out according to plan and the case that the system was tested until the budget ran out without covering all of the planned (comprehensive) test cases. If this testing strategy resulted in major holes in testing, it would certainly not be an ethical use of the client’s money. For example, say features were tested separately and sequentially so that testing the last few features was cut out with the budget loss. In such a case, it would be more ethical to test each feature to a lesser degree. If it isn’t possible to test every feature sufficiently within the budget, this should be communicated to the customer as early as possible to make a plan and prioritize which potential issues to investigate.

Reflections on testing:

“Testing and debugging” - this phrase is often thrown around within the computer science/software engineering world. Although testing and debugging are bucketed together, in reality, they are two distinct processes. This week’s readings, Chapter 8 (Software Testing) and a separate chapter centered around testing cleared up a lot of the confusion I had about this distinction by examining it in detail. The chapters focus on several topics related to the nature of testing and debugging.

To summarize the difference: Testing is the process of surfacing bugs. Debugging depends on testing and encompasses the more laborious process of correcting bugs found through it. These two processes taken together consume “at least half of the labor expended to produce a working program” (outside reading pg. 1). The goal of testing is not to show that there are no errors - this is nearly impossible to do, particularly in a large program. Rather it’s to surface as many errors as possible so that they can be rectified and the final product is as error-free as possible. The outside reading explains that this very fact leads many software engineers to dread the testing process; it feels like an admission that they’ve created an imperfect program. It feels defeating that more than half the time spent on the product is spent testing it. After all, if they’d done everything right, they could be assured there would not be any bugs, right? Although it feels this way, it simply is not practical to create a bug-free program, nor is it safe to assume a program is bug-free because all of the right precautions have been taken. As a programmer, it’s your responsibility to assume the worst of, not give the benefit of the doubt to, your work.

The importance of taking on a different role as the programmer and the tester really struck me throughout these readings. It makes so much sense - it’s really difficult to construct tests to find holes in your work when you’ve proudly constructed that work with the goal of preventing holes. I love the way the author of the outside reading puts this: “you must be a constructive schizophrenic.” This is all about being two different people when you program versus when you test. The tester in you needs to be “suspicious, uncompromising, hostile, and compulsively obsessed with destroying, utterly destroying, the programmer’s software.” I can imagine that’s not an easy suit to put on, but I suppose you’d have to do so with the goal of your final project being as sounds as possible.

That brings me to my final point to reflect on which ties to testing with such rage - “complete” testing. Both readings touch on unit testing versus component testing versus system testing. While unit testing should always be 100% complete, larger systems may be approved with as little as 50% or so completeness in the other testing types. The higher up the ladder you go, the more difficult - and eventually, impossible - it becomes to cover all of the bases and prove that no bugs exist. The objective is instead to create a “suitably convincing demonstration” of correctness. This takes a bit of the pressure off and allows companies to stay within their budget. Particularly in a non-safety-critical program, I would imagine that any existing bugs that aren’t detrimental could be discovered and mended throughout the life of the program.

• 2021 11
• 2020 22