Find and fix broken links using DubBot

DubBot automatically checks links on the webpages it crawls and reports links that return an error or otherwise cannot be successfully reached. The Broken Links section of a Site or Page Set dashboard can help you identify both the link that is failing and the webpage where that link needs to be corrected.

Understanding broken links in DubBot

When working with broken links, it is important to distinguish between two things:

• Broken link: The URL that DubBot attempted to visit and identified as having a problem.

• Affected page: A webpage on your site that contains the broken link.

For example, DubBot might report this as a broken link:

https://example.wisc.edu/old-resource/

That URL may appear on five different pages of your website. To fully remediate the issue, you would need to find and update the link on each of those five affected pages.

DubBot provides two useful views for approaching this work: Issues found and Pages. From the Broken Links panel, the issues found link groups results by the broken link that was discovered, while the pages link displays pages containing broken links. 

Option 1: Start with the broken links

This approach is useful when you want to identify individual broken URLs and determine everywhere they are being used.

1. Open the appropriate Site or Page Set dashboard in DubBot.

2. Locate the Broken Links panel.

3. Select the issues found link.

4. DubBot will display results grouped by the broken link that was identified.

5. If needed, filter the results by Broken Link Reason, such as 404.

6. Select or review a broken link to determine which pages are affected.

7. Open the affected pages and update or remove the link in your website's content management system.

This view is particularly helpful when the same broken URL appears on many pages. Rather than discovering the same problem repeatedly as you move page by page, you can identify the broken destination first and then address each place where it is used.

Example

Suppose DubBot finds:

https://example.wisc.edu/resources/accessibility.pdf

and reports that the link occurs on 12 pages.

If the document has moved to a new URL, you can use the affected-page information to identify all 12 pages that may need to be updated.

Suggested screenshot: Broken Links "Issues found" view with a broken URL and affected-page count highlighted.

Option 2: Start with pages that have broken links

This approach can be more useful when you are responsible for a particular webpage or section of a website and want to work through your content one page at a time.

1. Open the appropriate Site or Page Set dashboard.

2. Locate the Broken Links panel.

3. Select the pages link.

DubBot will open the page inventory already filtered to pages containing broken links. The page list includes the page title, URL and issue counts. You can also filter a page inventory by selecting Pages with Broken Links from the page filter.

4. Select the title or URL of a page you want to review.

5. In the Detail Page View, select Broken Links from the issue categories.

6. Review the broken links DubBot identified on that page. Each reported link includes the status or error DubBot received when attempting to access it. 

7. Determine where that link appears within your webpage and update or remove it in your content management system.

Suggested screenshot: Page inventory filtered to "Pages with Broken Links."

Suggested screenshot: Detail Page View with the Broken Links panel selected.

Which view should I use?

A simple way to choose is:

Use "Issues found" when:
You want to find a broken URL and see everywhere it is being used. This is often the fastest method for cleaning up the same outdated link across multiple pages.

Use "Pages" when:
You are responsible for particular webpages or a section of a site and want to resolve all of the broken links on those pages.

You can move between these approaches as needed.

Prioritize 404 errors

A 404 Not Found response usually means that the requested page or resource no longer exists at that URL. DubBot recommends prioritizing links returning a 404 when reviewing broken-link results. 

For a 404, determine what the link was intended to provide and then:

• Update the link if the content has moved to a new location.

• Replace the link if another resource now provides the same information.

• Remove the link if the resource is no longer available or relevant.

• If the broken URL is on a website you manage, consider whether the old URL should redirect to its new location.

Not every reported link is necessarily broken

Some websites intentionally prevent automated tools such as DubBot from accessing their content. As a result, responses such as 401 Unauthorized or 403 Forbidden may indicate that authentication or permissions are required rather than that the page is actually unavailable. Other server responses may similarly require additional investigation.

When you encounter one of these results:

1. Open the reported URL in your own browser.

2. Verify whether the link works for a typical user.

3. If the URL works correctly, it may be a false positive rather than something that needs to be changed on your webpage.

DubBot provides options for ignoring broken-link results depending on your assigned permissions. However, avoid ignoring a result simply to remove it from the dashboard. Confirm that the link works as expected before treating it as a false positive. Managers and administrators have additional abilities to ignore broken-link results at broader scopes. 

Finding a link that is difficult to locate

Sometimes the destination URL reported by DubBot is not immediately obvious when looking at the webpage. The visible link text may be something like Learn more, Download the guide, or the name of a resource rather than the URL itself.

If you cannot immediately locate the link:

1. Open the affected webpage from DubBot.

2. Note the broken destination URL.

3. Review the page for links that appear likely to point to that resource.

4. Open the page in your content management system and inspect the links within the page content.

5. If necessary, search the page's HTML for part of the broken URL.

Remember that the affected page URL tells you which page you need to edit, while the broken link URL tells you which link within that page needs attention.

Downloading broken-link results

If you have a large number of issues to work through, DubBot also allows results to be downloaded as CSV files. Filtered page lists can be downloaded, and DubBot provides both summary and full-list reports for its primary issue categories. 

A downloaded report can be useful when:

• dividing remediation work among several people;

• tracking a large cleanup project;

• sorting issues by page or URL;

• documenting remediation progress outside of DubBot.

Keep in mind that DubBot's dashboard information is based on its crawl and analysis of your site. After making corrections on your website, those changes will be reflected in DubBot after the content is crawled and analyzed again.

A recommended workflow

For most broken-link cleanup projects:

1. Open the Broken Links panel for your Site or Page Set.

2. Select issues found.

3. Start by filtering for and reviewing 404 errors.

4. Identify the affected pages for each broken URL.

5. Open those pages in your content management system.

6. Update, replace or remove the broken link.

7. Verify that the replacement link works.

8. Continue through the remaining broken URLs.

9. Review other response types to determine whether they represent genuine problems or false positives.

10. Allow DubBot to crawl the site again to verify your corrections.

Using the broken-link view this way allows you to address the source of the problem while also making sure you find every webpage where that problem occurs.