Getting indexed by Google and other search engines is of utmost importance for any website. However, it is also worth pointing out that crawl priorities are constantly being changed by search engines. If you crawl too deep you are going to end up getting lots of low quality pages, and if you crawl too shallow you will not get down to the really relevant pages. Since crawl priorities change all the time, there is really no way to make sure that pages get indexed and stay like that.
Here are however, some things you could see to:
- Content duplication - it could be that your meta description tags, page titles or even the content is similar to that on other sites.
- Link authority - you need to see if your site has real high quality links so that your link profile can compare well to that of other sites.
- Site growth rate - is your site's growth rate improving and doing as well as that of competing sites?
- Internal link structure - if search engines have to go through never-ending loops, or if you are linking to low value pages, then you need to make sure that this is avoided.
- Technical issues - make sure you are not feeding search engines with cookies. Try using clean and descriptive URLs.
- Do not waste link equity on low quality pages. The pages that you want to be indexed should be the ones that are being indexed and actively crawled.
- If your site is not building natural link equity over time you need to do something about it.
- Try to focus on internal link equity on your site by linking to the most important categories on it.
- Trimming the site depth could help. You could place multiple options on the same page instead of having several nearly duplicate pages.
- If you are noticing that there are several low value pages getting indexed, try blocking their indexing. You can do this by not linking them or simply integrating the information found on them onto other pages of higher value.
- Restructuring your site is often required so as to change the internal link structure and place more weight on certain sections.
- The use of sitemaps is helpful. Link all quality pages from your site so as to enable search engines to understand the page relationships with an open sitemap.
By trying out these recommendations you should be able to see some improvements in your site's indexing. Eventually you will start seeing a growing traffic trend from search engines when relevant search queries are made. Your aim is to get more high quality traffic, and to have searchers landing on the right page according to the query they made. Applying the tips we discussed above should enable you to see progress. However, do try to put them into practice from time to time so as to keep your site in good shape.