[Egothor-tech] mg4j / future of egothor

[Leo Galambos]

hp at mokk.bme.hu wrote:

>You know kanaging gigabytes for java (mg4j), don't you. It was a liblarary of
>imlemented of algorithm to build inverted indexes but it emergeces to be a
>search library.
>
>Don't you think that the two projects should join?
>
>  
>

Hi!

Yes, I know that project and it was told me two year ago (by its 
author), that he wouldn't write mg4j if he knew about egothor sooner ;). 
I was talking with him about golomb/gamma coding, I guess. As far as I 
read mg4j API now, he removed the compression module and he's developing 
Java search engine with a nice architecture!

Anyway, I have not any personal problems and I am willing to contribute 
to anything in an academic sphere and join with anyone :). On the other 
hand, mg4j follows the same way as lucene/nutch/etc - some parts of code 
assume that you have a homogeneous environment/format of indices. I 
would rather stay with the current egothor style, where such an 
assumption is not required... It will allow me to construct an engine of 
new parameters, not "OSS Google". We are in an era of 64b computers, 
many users with a fast internet connection and large disks, and software 
platform that can run on any CPU. Therefore, I do think that a _new_ 
engine should be developed.

BTW, do you know how fast is their robot? Can it consume more than 
6-7Mbps with 30 threads? I know, it is not part of mg4j, but they have 
one. IMHO the robot could be the first touching point.

>What is the future of egothor? A good search engine lib for applications or a
>customable search engine app?
>
>  
>

Well, it would be used as:
1) testbed for IR - it implies some library for single/multi-node systems
2) search engine with user's profiles and personal search engine - it 
implies some application
3) classic search engine a la google - it implies some application and 
configuration tool for masses

BTW: Obviously, Point 3) is a subcase of point 2)

Cheers,
Leo

-- 
::egothor
http://www.egothor.org/Main/LeoGalambos