My site have that robots.txt:
User-agent: *
Disallow: /account/
Disallow: /actions/
Disallow: /admin/
Disallow: /dashboard/
Disallow: /engine/
Disallow: /entities/
Disallow: /errors/
Disallow: /friends/
Disallow: /languages/
Disallow: /search/
Disallow: /services/
Disallow: /settings/
Disallow: /simplecache/
Disallow: /views/
Disallow: /_css
Disallow: /_graphics
Disallow: /account
Disallow: /actions
Disallow: /admin
Disallow: /engine
Disallow: /entities
Disallow: /errors
Disallow: /languages
Disallow: /mod
Disallow: /search
Disallow: /services
Disallow: /settings
Disallow: /simplecache
Disallow: /vendors
Disallow: /?views
Disallow: /?q
Disallow: /*?*
Disallow: /*?
Disallow: /likes
What you think about it?
Errors? Your suggestions?
info@elgg.org
Security issues should be reported to security@elgg.org!
©2014 the Elgg Foundation
Elgg is a registered trademark of Thematic Networks.
Cover image by RaĆ¼l Utrera is used under Creative Commons license.
Icons by Flaticon and FontAwesome.
- ghumanz@harinderghumanz
ghumanz - 0 likes
- Nikolai Shcherbin@rivervanrain
Nikolai Shcherbin - 0 likes
- ghumanz@harinderghumanz
ghumanz - 0 likes
- ghumanz@harinderghumanz
ghumanz - 0 likes
- Nikolai Shcherbin@rivervanrain
Nikolai Shcherbin - 0 likes
You must log in to post replies.I was using the same but got lots of crawl erros in google webmasters , it shows like 3000 errors like blocked by robots.txt Did you checked there after 2-3 days . please post anything you found there , I am also interested in it.
@ghumanz Errors are possible. This occurs because the blocked search results.
This is done for the best SEO.
Example, check your site in Google as site:yoursite.com and you can to see: yoursite.com/pg/search/?q=%some-keyword%
If you do not want to block search results, remove
Disallow: /search
Disallow: /?q
Disallow: /*?*
Disallow: /*?
Also check that your links do not contain mod. It is used in some plugins (ex., Market)
Otherwise, remove
I deleted from robots.txt
Disallow: /errors/
Disallow: /errors
because it was for me personally.
I thing we need to remove account it will block the link to login.php which is must.
what you think about it??? Is google will provide a like to login even if we block account?
one more thing add /?offset
I think that Google is not necessarily aware of such links:
If for you it is of great importance then need to delete:
I think it does not affect access to the site.
Disallow: /?offset is a great idea!
Need to add it!
Thanks for your advice