Robot specification

• robots.txt - Capek obeys the definitions listed in robots.txt files. It revisits daily the robots.txt file of each hostname looking for changed definitions. You can find out more about robots.txt files at:
  • egothor
  • Search Engine World
  • Search Tools.
• speed - It can fetch files at a selectable rate (set by the administrator). In the instance of www.egothor.org the rate is set at a maximum of 1 request per 10 seconds for each hostname.
• frequency - The robot amongst other things calculates the frequency with which the content of pages changes. Pages and web-servers which are deemed to change often are visited more often. The frequency of the visits is selectable by the administrator, and in the instance of www.egothor.org it is set at no more than one visit per 30 minutes.
• user-agent - capek
• complaints - If the robot evidently contravenes the robots.txt specification or there are any other issues please inform us.
• note - Although Capek is prepared for distributed crawling, it runs in a single box mode now, right on this server. We cannot be responsible for instances which do not run on our hardware. We appreciate your patience, and apologise for any inconvenience or frustrations you (may) have experienced.

Contacts

There is one public mailing list where only list members can post:

• egothor-tech@egothor.org - technical topics for Egothor developers and advanced users. Subscribe to egothor-tech