[Egothor-tech] Duplicate default pages
Stuart David Lewis [sdl]
sdl at aber.ac.uk
Wed Mar 23 10:27:53 GMT 2005
>>http://www.aber.ac.uk/en/student/travel/
>>http://www.aber.ac.uk/en/student/travel/index.php
>>
>>Could Egothor be improved to know that these are the same?
>Unfortunately, you could only exclude the identical pages manually (in
robot's rules file).
How would I go about doing this? I wouldn't want to exclude pages ending
in '/' or one of the many default pages (default.asp, default.shtml,
default.shtml, index.php, index.html, index.asp, index.aspx, index.wml
to name but a few) as some pages may only be referred to by one of
these.
Could I use a rewrite rule at all, or would this be tricky because of
the number of possible index pages?
Thanks,
Stuart
More information about the Egothor-tech
mailing list