.Google's John Mueller responded to a concern about why Google.com indexes web pages that are prohibited coming from creeping by robots.txt and why the it's risk-free to dismiss the relevant Browse Console records about those creeps.Robot Visitor Traffic To Query Parameter URLs.The individual talking to the concern recorded that crawlers were actually creating web links to non-existent concern parameter URLs (? q= xyz) to web pages with noindex meta tags that are actually likewise obstructed in robots.txt. What triggered the question is actually that Google.com is crawling the hyperlinks to those pages, receiving blocked out by robots.txt (without noticing a noindex robotics meta tag) after that obtaining turned up in Google Look Console as "Indexed, though shut out through robots.txt.".The individual inquired the following concern:." However below is actually the major inquiry: why will Google index web pages when they can't even see the web content? What's the benefit in that?".Google.com's John Mueller confirmed that if they can't creep the page they can not view the noindex meta tag. He also creates an appealing mention of the website: search driver, recommending to neglect the end results because the "common" consumers won't view those results.He created:." Yes, you're right: if we can't creep the webpage, our team can not find the noindex. That stated, if our experts can't creep the pages, at that point there's certainly not a great deal for us to mark. Therefore while you may find several of those webpages along with a targeted site:- query, the average individual will not see all of them, so I would not fuss over it. Noindex is likewise alright (without robots.txt disallow), it only implies the Links will certainly end up being actually crept (as well as end up in the Browse Console document for crawled/not recorded-- neither of these conditions create concerns to the remainder of the site). The vital part is that you don't make them crawlable + indexable.".Takeaways:.1. Mueller's answer confirms the limitations in using the Website: search accelerated search operator for diagnostic explanations. Some of those factors is actually considering that it is actually not connected to the normal hunt index, it is actually a distinct point entirely.Google.com's John Mueller talked about the website search driver in 2021:." The brief answer is that an internet site: concern is actually not suggested to become total, neither utilized for diagnostics objectives.A website question is actually a details kind of search that limits the results to a certain website. It is actually primarily simply words website, a colon, and then the internet site's domain name.This concern restricts the outcomes to a specific website. It's not suggested to be a complete selection of all the webpages coming from that website.".2. Noindex tag without utilizing a robots.txt is great for these sort of conditions where a robot is connecting to non-existent pages that are acquiring found through Googlebot.3. URLs with the noindex tag will certainly produce a "crawled/not catalogued" entry in Look Console which those will not possess a negative effect on the rest of the web site.Read the concern and also address on LinkedIn:.Why will Google.com index web pages when they can't also see the information?Featured Picture through Shutterstock/Krakenimages. com.