Ok, for those who aren’t big SEO techies – there is a difference between crawling and indexing. Its fairly straight forward in that one is the fetch (crawl) and the other is the understanding (index) of the data found. Many people confuse and conflate the too, the web has therefore come to understand them as one – so twitter/reddit etc etc you may see them as one.
This year, Google has had some crawling and indexing issues – this first was really noticed as a wide spread issue in April but seems to happen every few weeks. Though Google seem to think its now fixed (WaHaaay!) – lets see!
This issue is now resolved. Thanks for all your feedback along the way. Wishing you all a calm weekend!
— Google Webmasters (@googlewmc) August 9, 2019
Ok, so GWC say a “large part” is resolved… so not all of it – what does this mean? who knows but we do know things are working again somewhat and that means we can all move on a little… at least for now right?
Hold your horses!
Google Search Guru John Mueller has been doing a lot of webmaster videos recently and meetings! (the new matt cutts – lol)
Some really interesting stuff has come out of it and these two are really interesting …
Firstly….
Mobile Indexing ….
You can’t opt-in or opt-out, Google will decide if you should be mobile indexed or not – and WHEN!
So, ok here is what my experience is of this just as a little guide.
You will get mobile-indexing (generally) if:
- You build a new website
(any new website will be mobile-first indexing) - You redesign your current website
(basically force a huge reindex) - You change your domain, or a significant number of urls
(basically force a huge reindex) - Generally if you wait a while eventually you will just become mobile-first
The second video…
Two Wave Indexing? Gone?
Don’t have a clue what this is… i’ve summaried it below the video 
What is 2-wave indexing?
Basically, after Google has crawled your site currently it will index your site as flat HTML (the first wave) … if your site looks like there is alot of javascript it will then opt to render the website (the second wave) to check nothing appears to have changed in your content – ensuring you deliever to your users what your are to search engines in terms of content. Afterall this is about user experience.
In this video John Mueller and Marting Spitz talk with Bartosz about whether 2-wave is going to disappear, and there is a clear indication that overtime things will return to a single-wave index.
This is to say that whilst there is no timeline for this, all website will be indexed in a single-wave and this will be a rendered index. The reasoning Martin gives is simply that there is no reason not too, the power usage is the same and there is actually a time and power to do so – so it makes sense to do this. – oh and ultimately many sites are already single-wave, as Google will put you in a pattern for a long-term single wave after it trust your site to need rendering as HTML or Javascript.
Does all this sound a little confusing?
Not sure why you need to know about indexing on your website?
Indexing and Crawling are key to any website, if your website can’t be crawled and the information collected by search engines then it can’t be indexed. Learning about Indexing and Crawling can help improve your SEO through a deeper understanding of Technical SEO and your own websites capabilities.
Image – Twins about to crawl by Zaid Abu Taha from Pexels
