分享一个比较全面的robots.txt写法

Posted 2021-03-16 fymuban

Sitemap: http://www.sdfangyuan.cn/sitemap.xml

User-agent: *
Disallow: /*?*
Disallow: /*#*
Disallow: /*%*
Disallow: /*^*
Disallow: /caches/
Disallow: /phpcms/
Disallow: /install/
Disallow: /phpsso_server/
Disallow: /statics/
Disallow: /api/
Disallow: /2018/
Disallow: /*.php
Disallow: /*.asp
Disallow: /m/
Disallow: /404.html
Disallow: /portal.php
Disallow: /index.php
Disallow: /*-typeid-*
Disallow: /*-areaid-0*
Disallow: /*-list-*
Disallow: /*-maxprice-*
Disallow: /*-minprice-*
Disallow: /*-order-*
Disallow: /*-day-*
Disallow: /*-kw--*

User-agent: Baiduspider-image
Allow: /uploads