WiscWeb - Preventing a Page / Post from Being Indexed by Search Engines

• Important considerations (it is highly recommended that you review these before getting started)
  • Previously published pages
  • Sitemaps
  • 'noindex' vs 'nofollow'
  • Temporary page blocks for Google
• Steps

Important considerations

For previously published pages where you've recently added the 'noindex' meta tag:

• If a page was created and already indexed by Google, it won’t become un-indexed until Google re-crawls and finds the ‘noindex’ meta tag for that page
  
  
  • For these instances, consider running the URL Inspection Tool to let Google know to recrawl your content. To complete this step, you’ll need to have a Google Search Console account setup and linked to your site: WiscWeb - Adding Google Search Console to Your Site
    
    
  • Tip: If you know you have a page that should not be indexed, consider adding the meta tag upon first page publish.
    
    
• It seems that Google may crawl some of a site but not all so there’s a chance it will update some content in the search results and not catch your ‘noindex’ right away. It won't update content in search results until it crawls that page specifically and finds the 'noindex' tag.

'noindex' in regards to sitemaps:

• All WordPress sites have an automatically generated sitemap that publishes to sitename.wisc.edu/wp-sitemap.xml. It appears that pages designated as 'noindex' will still appear in the sitemap. 
  
  
• Google may not honor pages that are designated as 'noindex' because they appear in the sitemap. 
  
  
• Having “noindex” pages in the sitemap can confuse search engines, resulting in them completely ignoring your sitemap. This would mean relevant content may not show up in Google search results.
  
  
  • Tip: If you have a site where you want most of the content to be in Google search results and only 1 or 2 pages that are designated as 'noindex', it might be best to explore alternatives so you don’t risk your entire site from being ignored by Google.

'noindex' versus 'nofollow':

• Where 'noindex' is telling Google not to index the page, 'nofollow' is asking Google to ignore hyperlinks. This meta tag can be added to the page in the same way as 'noindex'.
  
  
• ‘nofollow’ may be treated differently by Google. Google will only see this as a strong recommendation. In other words, Google may ignore 'nofollow' tags for hyperlinks. 
  
  
• If you want to dive super deep, here's an article that breaks down all of the differences.

Temporarily blocking a page from appearing in Google search results:

• If you are looking to only temporarily block Google from displaying a page in Google search results (for a period of up to 6 months), there is a process you can follow. This involves setting up a Google Search Console account: WiscWeb - Adding Google Search Console to Your Site

Steps

The process of preventing a page from being indexed in the UW Theme involves creating a custom meta tag property. This option will prevent most search engine web crawlers from indexing the page.

1. Scroll down to the bottom of the page and locate the Meta Tags Settings box
   
   
2. Locate the option for creating Custom Meta Tags
   
   
3. Select Add Meta Tag
   
   
4. In the Property field, type “robots”
   
   
5. In the Content field, type “noindex”
   
   
   
   
6. Publish/update the page or post

See Also:

• WiscWeb - Adding Google Search Console to Your Site