15 – The need for early testing in software delivery

Date: 1/2/2020Guests: None
Welcome and greetings
Recap of last episode

  • In the last episode, I discussed the Wyze suite of home automation products. I spoke about my experiences with each item and any issues or gotchas that I encountered while setting them up and using them.

Summary of this episode

  • In this episode, I am going to talk about why you need to start your testing while you are writing the code for your applications. I’ll give you example of how, what the benefits are and what costs are associated with doing it right and doing it…….not so well.

What’s in it for you?

  • After listening to this episode, I hope you understand why it is important to start testing early. Go back and discuss it with your team and start today if you can. Start small and build on it.

Episode Content

  • Why test early?
    • Testing early is one of the simplest way that you can cut some of the expense out of software development.
  • There have been numerous studies that show it is much cheaper to fix a bug or defect before it is released to a testing environment.
  • At the start of a project there are many unknowns and if a team writes all the code and no testing is done on it the first quality profile is found when it is first deployed to any environment.
  • This uncertainty is referred to as the “cone of uncertainty” and is shown in a diagram that looks like a funnel with the large end to the left where the project starts and the smaller end on the right as requirements are redefined more accurately and software defects are fixed and released. The premise is that you don’t know a lot a the start of a project but should know everything about the software and how it meets requirements as you get closer to the production release.
  • The best way to fix defects in the large end of the cone is to perform a super-duper simple set of tests…..are the requirements testable and do they make sense. Requirements reviews are a great way to cut expense at the beginning of the project since no code has been written yet.
  • Once the code is deployed, it is then up to testers to run through test suites (manual or automated) to find what issues exist with the code.
  • At this point it is more expensive in money and effort to fix the defects that are found.
    • This is true because a developer, who may not have looked at the code in weeks, will have to review the code and possibly relearn what it was supposed to do before they can make the changes to fix the defect.
    • The code will then need to be re-compiled and packaged before it can be re-scheduled for release back into the testing environment.
    • The testers will then re-test the code to verify whether the defect was fixed.
    • If not, the defect is once again re-assigned to the development team to fix.
    • You see how this cyclical code, test, fix can be time consuming and expensive.
    • The more optimal path would be to write the code and write a corresponding test to verify its functionality.
  • Types of early testing
    • Unit testing – this is probably the one that most people are familiar with.
      • It is used to test small pieces of code as they are written or modified.
      • They are usually written using a framework such as Junit or on of the xunit variants since they have wide usage and adoption in the industry.
      • They can be very language specific, such as python where you are required to use custom libraries to implement the test framework. The first time that I had to do this for Python, I was pretty lost as my background is in Java.
    • Test Driven Development
      • This type of testing is based on the premise that no code is written before a test case for that code is created. Then just enough code is written to make the test pass.
        • For example: a program needs to determine if a file exists.
          • Test case: does file x exist at location y
          • Code is written to determine if there is a file system at the specified path x and if the file denoted by y exists at that location.
          • Then another test case is written for the next piece of functionality.
          • And the cycle continues.
  • Costs associated with testing
    • When you wait until development is finished and first deployment is completed, there is a cost associated with the number of testers needed and how the tests are run (manual vs. automated).
    • If you are a manual testing shop and have a fairly complex system, the cost to develop the needed test cases is fairly high since the testers must wait until the code is completed and deployed to write their functional test cases (a test case that tests a specific piece of functionality).
    • If you are an automated testing shop and wait until the software is completely deployed, the cost can be just as high as automated test cases, even though they can be re-used once complete, can be very difficult and time consuming to develop the first time.
    • If you write your test cases with your code as it is developed, then you can have a higher degree of confidence that it will meet the needs of the requirements once it is deployed to a test environment and the system integration and functional tests are run against it.
    • Non-functional requirements (system response time, time to failure, http request response time, database response time, etc.) require a different type of testing and is usually very specialized (framework, toolkit, and resources) and complicated to create and run. The earlier that these testers can get the code and start developing their test cases, the cheaper it will be in the long run as this is also a re-usable set of test cases.

So in my opinion, the earlier that you can start writing test cases, the more expense that you can remove from your software development efforts. Don’t wait until your code is complete to let your test partners start creating tests based on what you have. Even if they have to re-write some of the test cases, it is better than starting from the beginning.

Recap of this episode

  • In this episode, I discussed why you should start testing as early as you can in your development cycle. I also discussed a couple of methods of early testing that can help reduce the expense of software development. I then discussed the costs associated with software testing and how “shifting left” can help your organization reduce costs related to development and testing.

Like me Like my podcast – share, like, thumbs-up, review, subscribe
Next Episode: Types of testing