User-agent: * #Disallow: / ## but allow only important bots #this for www.xaluan.com robots_xl.txt #User-agent: Googlebot #User-agent: Googlebot-Image #User-agent: Mediapartners-Google ##User-agent: msnbot-media #User-agent: Slurp #User-agent: Yahoo-Blogs #User-agent: Yahoo-MMCrawler #User-agent: facebookexternalhit #User-agent: Facebot # Directories Disallow: *xl_* Disallow: *XL_* Disallow: *iarss* Disallow: *xaluan.net* Disallow: *xaluan.info* Disallow: *xaluan.org* Disallow: *xaluan.mobi* Disallow: *us.xaluan.com* Disallow: /admin.php Disallow: /admin/ Disallow: *xaluano* Disallow: *includes* Disallow: */modules/* Disallow: */opx/* Disallow: /blocks/ Disallow: /Tag/ Disallow: /lucida grande/ Disallow: /tv/ Disallow: /vnhits/ Disallow: /downloads/ Disallow: /seodigger/ Disallow: /language/ Disallow: /index.php/* Disallow: /modules.php/* Disallow: /*cname* #Disallow: /*.rss/* Disallow: /*.php/* Disallow: /*.html/* Disallow: *ad_click* Disallow: *=http:* Disallow: *%3D* Disallow: *php*fb* Disallow: *usg=* Disallow: *url=* Disallow: *t=* Disallow: *qc=* Disallow: *start=* Disallow: *postorder=* Disallow: *postdays=* Disallow: *topicdays=* Disallow: */Hau-truong/* Disallow: */timnhanh/* Disallow: *detail*modules* Disallow: *forums* Disallow: *profile* Disallow: *posting* Disallow: *mode* Disallow: *qsearch* Disallow: *Your_Account* Disallow: *Private_Messages* Disallow: *showad* Disallow: *activate* Disallow: *edit* Disallow: *raovat/-2* Disallow: *raovat/-3* Disallow: *raovat/-4* Disallow: *raovat/-5* Disallow: *xzcal* Disallow: *reply* Disallow: *comment* Disallow: *reportabuse* Disallow: *cityid* Disallow: *adid* Disallow: *ads*.php* Disallow: *pay* Disallow: *mark* Disallow: *Members* Disallow: *groupcp* Disallow: *coppermine* Disallow: *Horoscopes* Disallow: *url* Disallow: *thold* Disallow: *page* Disallow: *print* #Disallow: *font* Disallow: *search* Disallow: *previous* Disallow: *?utm* Disallow: *next* Disallow: *highlight* Disallow: *contact* Disallow: *trangvang* Disallow: *opx* Disallow: *rmcache* Disallow: *events* Disallow: *page* Disallow: *psid* Disallow: *mail* Disallow: *lang* Disallow: *playflash* Disallow: *mobitopic* Disallow: *mcategory* Disallow: *mtopic* Disallow: *new_topic* Disallow: *article_mobi* Disallow: *mobi.php* Disallow: *mobisetpref.php* Disallow: *hot* Disallow: /themes/ Disallow: /*thumbs* Disallow: /themes/Zeonix/ Disallow: */modules/News/comnews.php* Allow: /ads*.txt #Sitemap: /home.rss #Sitemap: http://www.xaluan.com/mobile.rss #User-Agent: Googlebot-Image Disallow: /*thumbs* Disallow: /*templates/* Disallow: /*themes/* Disallow: /videos/files/* Disallow: /*.mp4$ Disallow: /images/news/Image/*xsmall_*img.* Disallow: /images/news/Image/*small_*img.* Disallow: /images/news/Image/*med_*img.* Disallow: /*?url=* #Disallow: /images/news/Image/* Allow: /images/news/Image/*.img.* User-agent: Googlebot-Mobile Disallow: *mobitopic* Disallow: *article_mobi* Disallow: *mobi.php* Disallow: *hot*