[Egothor-tech] Duplicate default pages

Stuart David Lewis [sdl] sdl at aber.ac.uk
Wed Mar 23 10:27:53 GMT 2005


>>http://www.aber.ac.uk/en/student/travel/
>>http://www.aber.ac.uk/en/student/travel/index.php
>>
>>Could Egothor be improved to know that these are the same?

>Unfortunately, you could only exclude the identical pages manually (in
robot's rules file).

How would I go about doing this? I wouldn't want to exclude pages ending
in '/' or one of the many default pages (default.asp, default.shtml,
default.shtml, index.php, index.html, index.asp, index.aspx, index.wml
to name but a few) as some pages may only be referred to by one of
these. 

Could I use a rewrite rule at all, or would this be tricky because of
the number of possible index pages?

Thanks,


Stuart


More information about the Egothor-tech mailing list