An URL will always have a status with 2xx which is valid. There are different HTTP status codes which are having different purposes. For an invalid request, HTTP status is 4xx and 5xx. 4xx class of status code is mainly for client side error, and 5xx class of status codes is mainly for the server response error. We will most likely be unable to confirm if that link is working or not until we click and confirm it.

You should always make sure that there are no broken links on the site because the user should not land into an error page. The error happens if the rules are not updated correctly, or the requested resources are not existing at the server. Manual checking of links is a tedious task, because each webpage may have a large number of links & manual process has to be repeated for all pages. An Automation script using Selenium that will automate the process is a more apt solution.

For checking the broken links, you will need to do the following steps.

Collect all the links in the web page based on tag. Send HTTP request for the link and read HTTP response code. Find out whether the link is valid or broken based on HTTP response code. Repeat this for all the links captured.

Below is the web driver code which tests our use case:

Explaining the code Example

Step 1: Import Packages

Import below package in addition to default packages: Using the methods in this package, we can send HTTP requests and capture HTTP response codes from the response.

Step 2: Collect all links in web page

Identify all links in a webpage and store them in List. Obtain Iterator to traverse through the List.

Step 3: Identifying and Validating URL

In this part, we will check if URL belongs to Third party domain or whether URL is empty/null. Get href of anchor tag and store it in url variable. Check if URL is null or Empty and skip the remaining steps if the condition is satisfied. Check whether URL belongs to a main domain or third party. Skip the remaining steps if it belongs to third party domain.

Step 4: Send http request

HttpURLConnection class has methods to send HTTP request and capture HTTP response code. So, output of openConnection() method (URLConnection) is type casted to HttpURLConnection. We can set Request type as “HEAD” instead of “GET”. So that only headers are returned and not document body. On invoking connect() method, actual connection to url is established and the request is sent.

Using getResponseCode() method we can get response code for the request Based on response code we will try to check link status. Thus, we can obtain all links from web page and print whether links are valid or broken. Hope this tutorial helps you in checking Broken links using selenium.

How to get ALL Links of a Web Page

One of the common procedures in web Testing is to test if all the links present within the page are working. This can be conveniently done using a combination of the Java for-each loop, findElements() & By.tagName(“a”) method. The findElements() method, returns a list of Web Elements with tag a. Using a for-each loop, each element is accessed.

The WebDriver code below checks each link from the Mercury Tours homepage to determine those that are working and those that are still under construction. The output should be similar to the one indicated below.

Accessing image links are done using By.cssSelector() and By.xpath() methods.

TroubleShooting

In an isolated case, the first link accessed by the code could be the “Home” Link. In such case, driver.navigate.back() action will show a blank page as the 1st action is opening a browser. The driver will not be able to find all other links in a blank browser. So IDE will throw an exception and rest of the code will not execute. This can be easily handled using an If loop.