...
Code Block |
---|
User-agent: *
Disallow: /browse-subject
Disallow: /browse-author
Disallow: /*/browse-subject
Disallow: /*/browse-author
Sitemap: http://my.dspace.repo.ac.uk/htmlmap
|
Other content to consider blocking
...
...
Code Block |
---|
User-agent: *
Disallow: /browse-subject
Disallow: /browse-author
Disallow: /*/browse-subject
Disallow: /*/browse-author
Sitemap: http://my.dspace.repo.ac.uk/htmlmap
|
...