What is Unit Testing?
We write unit tests during the development phase of the code. It is a classic example of white-box testing. This is because to carry out any white-box testing, we need to know the internal mechanism of the code. However, the scope of a unit test case is limited. It can only test one code unit. It cannot test the interactions between different units of code.
Benefits of Unit Testing
The main idea behind writing unit test cases is to segregate different logical parts of the program and test their individual behavior correctly. Unit tests can easily detect a large portion of the bugs when we create them. Moreover, it allows us to automate the testing process and reduce the effort required to detect errors in complex applications. It helps maintain code coverage and detect bugs if existing code is changed.
So what are the benefits?
- It makes the entire process agile. Refactoring an existing code is risky and can introduce bugs in the system. If you have unit tests in place, then refactoring becomes easier.
- It improves the overall quality of the code. It will allow you to identify all possible edge cases and detect bugs before they are sent for integration testing.
- You can find bugs at a very early stage of development. This is so because unit tests are usually written during the development phase and can be resolved without impacting other pieces of code.
- Unit testing simplifies integration and facilitates changes. It allows you to refactor or update code in the future and ensure that the module works correctly.
- They act as documentation of the code units. New team members looking to learn the code’s functionality can easily refer to these test cases to gain a basic understanding of the API.
- They help make debugging very easy. In case a test fails, you know which piece of code to look for and only consider the latest changes in that piece of code.
- Unit tests help in improving the design of the code. If you write the tests first, it allows you to think through the code’s design and helps you find any inconsistencies.
- They help you reduce the cost of manual labor and the time required. If we discover a bug later, like during system or acceptance testing, it would incur a huge cost.
Unit Testing Example
Let’s suppose you have a web application to book airline tickets. Then there will be several modules inside this application, each with several classes and functions. Each of these functions will have its own usage.
Suppose you have the functionality to find out the details of all the flights in a given price range. In this case, based on the input price range, the method should return a list of all the flights whose price falls in that range on a particular day and route. We can test this function as a single unit of code using unit test cases.
So whenever the CI/CD pipeline builds the code, these unit tests must be run to validate the underlying code. Moreover, if we make any changes to this function and it does not follow the expected logic, the test cases will fail to indicate a bug in the new change.
Types of Unit Testing
Predominantly, there are two types of unit testing. It means that there are two ways to create and execute unit test cases.
- Manual Testing
- Automated Testing
In the case of manual testing, there may be an instructional document with a step-by-step approach to testing each feature. However, manual testing is almost obsolete in complex applications.
Most applications now rely on automated testing to test each code unit. There are several test frameworks that allow you to create test cases, such as JUnit, PHPUnit, etc. Once you have created unit tests with these frameworks, you can mock external dependencies and execute these cases to determine whether your logic works as expected.
These frameworks help you to log the runs and create detailed stack traces and failure/success reports. In a CI/CD pipeline, every time you create a new pull request, the code is built, and along with the code, the existing unit test cases also run. Hence, you can determine if there is a bug with the new change.
Tools for Unit Testing
For each programming language, there are several test frameworks and libraries that you can use to create and execute unit tests. Some of the most popular ones are:
- JUnit – It is one of the most popular testing libraries for the Java programming language. It allows you to create assertions to test actual vs. expected outputs. You can mock dependencies and use mocked data.
- NUnit – It is a unit testing library for the .NET language. It is open-source and supports parallel data-driven tests.
- JMockit – This is yet another popular open-source unit testing framework. It has code coverage functionality with path and line metrics. It has features such as API mocking, verification and recording syntax, line coverage, and data coverage.
- EMMA – It allows you to analyze and report Java code. It also supports line and block-code coverage features.
- PHPUnit – We can use it to create unit tests for the PHP language and define assertions.
What is End-to-End Testing?
End-to-end testing is the process of testing the functionality of the entire application from start to finish. It involves emulating the most commonly performed user actions and tests whether the output is desired. We perform end-to-end (E2E) testing after the unit and integration tests are over.
An E2E test involves testing all aspects of the application ranging from technical functionalities to the user interface. We can do automated as well as manual E2E tests. It involves finding out bugs and errors in the applications against the intended requirements.
E2E tests are black box tests, meaning that the tester does not know how the functionalities are programmed. If an end-to-end test is successful, it means that the user has validated the business requirements.
What are the main benefits of end-to-end testing?
Modern applications are complicated and built on internal and external subsystems. We need to validate these products as a whole and not just as individual units. Although each subsystem might pass the tests, they may fail when they interact with the whole system. Hence, conducting an end to end testing will help you ensure that your application is production ready.
The main benefits of end-to-end testing are:
- It confirms the overall health of your application. It validates your application as a whole and provides perspectives on the performance of the application in different user environments.
- E2E testing expands the test coverage by accounting for the interaction between subsystems and services.
- If we do end-to-end testing in an agile way, they are executed in every iteration. This will help to catch more bugs more often. This reduces the possibility of getting bugs buried deep in the production version of the software, where it becomes hard to fix.
- These are tests from a user’s perspective. They uncover bugs and irregularities that are not apparent in unit testing. It verifies the business logic of the application.
- Since bugs are now identified at an earlier stage, it reduces the cost of fixing them in terms of time taken and manual labor. Hence, we can reduce the time to market.
- It avoids risks associated with the underlying subsystems by verifying the system flow.
Example of End-to-End Testing
Let’s consider the previous example. Earlier, we tested individual logical units of the application code. In the case of end-to-end testing, we will test the application as a whole from a user’s point of view. We can do this after the features are deployed in a testing environment.
The testers will simulate the user actions on the application and monitor the output. If we get the expected output, the new version is validated from a user’s perspective. Some examples of user actions include clicking a button, filling up a form, filtering or sorting information, etc. There are several test frameworks that allow you to create automated test cases to simulate such user actions.
Types of End-to-End Testing
Based on the applicability of the E2E testing process, there are two types:
Horizontal E2E Testing
Horizontal E2E testing methodology is quite common, and it spreads horizontally across the context of the applications. We mostly use this in single enterprise resource planning applications.
For example, consider a web-based e-commerce shopping application. It has components such as inventory status, shipping details, accounts, product details, etc. Testing all these components simultaneously falls under horizontal testing.
In our example, our application needs not only to work from a UI perspective but also to be integrated properly with the email, payment systems, and network infrastructure.
That’s why we usually do horizontal E2E testing at the end of the release cycle when the changes related to all the subsystems are done. Hence, we need to do an environment setup before advancing to the testing part.
Vertical E2E Testing
Here, testing happens hierarchically in layers. To achieve quality, we test each of the component layers from start to finish. We use vertical E2E testing to test the complicated and critical components of an application. Components like expensive computing systems where user interfaces are not involved, are targeted explicitly by E2E testing strategies.
In the example above, we can test the core system with the help of unit tests. Then we will test the payment processing and email subsystem before testing the network and database infrastructure, API, and UI. Since vertical E2E testing is granular, it usually follows behavior-driven, test-driven, or continuous testing.
Tools for End-to-End Testing
Some of the most popular test frameworks for end-to-end testing are:
- Selenium – It supports all the major programming languages, including Java, JavaScript, C#, Ruby, Python, etc. You can easily simulate almost all user actions using Selenium. It is an open-source tool, and the Selenium IDE also has a record and playback feature. It consumes significantly less resources, has the option to allow parallel execution, and we can integrate it with other frameworks as well.
- Cypress – It is a popular and versatile testing framework that works effectively with software built using frameworks like React, Angular, etc. Unlike Selenium, Cypress does not use any driver or remote code. Cypress tests run in JavaScript in a web browser. It supports test-driven development (TDD). Besides, it supports dynamic or hot-swaps changes, thus allowing you to create or execute test cases faster.
- Cucumber – It uses Gherkin, which allows you to write test cases using simple natural language. It enables you to avoid writing complex codes for simple test cases and thus makes lives easier for non-technical people. Unlike Cypress, Cucumber mostly supports behavior-driven development (BDD). This allows you to consider the perspective of stakeholders while writing test cases.
Differences between Unit Testing and End-to-End Testing
The Difference between Unit Testing and End-to-End Testing is that Unit testing ensures that the functions or calculations that generate data, a numerical value, a text string, are working as required. End-to-end testing ensures that buttons, forms, updates, links, and the complete workflows work properly.
Both unit tests and end-to-end tests serve their own purposes and have their own usage. However, let’s look at the key differences between them.
Scope of test
As discussed, unit tests focus mainly on individual code units, whereas end-to-end testing tests the application as a whole from a user’s perspective. It’s easy to run unit tests, and we can execute them in each build. Whereas, running a complete end-to-end test on an application takes time.
Type of testing
Unit tests are white-box tests. This is so because if you want to write a unit test to cover a specific function, you need to know the internal implementation of that function. On the other hand, end-to-end tests are black-box tests. These tests intend to simulate user actions on the application and test the functionality, which does not require any knowledge of the implementation.
Testing methodology
Unit tests are mostly automated tests that we can integrate with the CI/CD pipeline. As soon as we push a new code commit, the entire code is built, and the test cases are run to check for any bug because of the new change. Whereas end-to-end tests can be manual as well as automated. We can use several frameworks to make such tests automated or test them manually by simulating user actions.
Testing environment
The developers mostly create unit tests; hence, they run them on the developer machines. On the other hand, we do end-to-end tests in a dedicated testing environment after the subsystems are built.
Performers
Unit tests are performed by the developers who write them. Whereas, a team of testers or QAs execute end-to-end tests in a dedicated environment.
Parallel Run
We can run unit test cases in parallel because they focus on individual code units and are not dependent on each other. In comparison, end-to-end tests have to be performed sequentially.
Access to database and other resources
We usually mock the dependencies in unit tests; hence, they do not require access to any resources. End-to-end tests typically require more resources because the whole purpose of this test is to check whether the entire system works as intended.
Effort
We need less time and cost to create unit test cases as compared to end-to-end tests. Performance tests are complicated, and a lot of subsystems are involved.
Which is better, unit testing or end-to-end testing?
Unit tests and end-to-end tests complement each other. They are different testing strategies with different use cases. Based on the requirement, the correct testing strategy for your application can be determined. But it’s always better to keep both strategies in your bag.
We use unit test cases to find bugs during the early development phase and check whether the individual code units work as expected. We run these test suites every time a build is triggered in the CI pipeline. However, it does not mean you have to write unit tests for all your subsystems. For example, if your API lacks logic, it’s better to go with integration tests rather than writing unnecessary unit tests.
On the other hand, we use end-to-end tests to verify the business requirement of the application. We check whether the functionalities of the application are as expected before going into production. We use E2E testing for large and complicated applications which has multiple subsystems.
Therefore, to decide the best suitable approach for your project, you need to analyze your requirements, compare these strategies, and choose the best one.
Summary
To summarize, both unit and end-to-end tests serve their unique purposes. It’s always better to have them both in your testing strategies.
Writing, maintaining, and understanding unit test cases are very easy. But since it is a white-box testing technique, you must understand the code to properly create unit test cases. Also, we cannot cover all the functionalities of the software by unit test cases.
Hence, to verify whether each and every part of the application works as intended and well together, an end-to-end test just before the release is a must. Although they are very time-consuming to perform, they can help you find issues and performance impacts before a production release.
https://testsigma.com/blog/unit-test-vs-e2e-test/
