robots exclusion standard

Related Overviews


'robots exclusion standard' can also refer to...


More Like This

Show all results sharing this subject:

  • Computing


Quick Reference

A standard which is used to define data stored in a text file associated with a Web site. The text in the file describes the access privileges of robots visiting the page: for example, the text could forbid all access to the Web site by robots or restrict it to a small number of pages.

Subjects: Computing.

Reference entries