# robots.txt file from http://www.cnsd.cn/seller/offerlist/news/ # all robots will spider the domain user-agent:* disallow: