Grub "LookSmart is confident that the number of Grub volunteers will continue to grow, and is hopeful that in time -- perhaps several months -- the system's "distributed crawl" will be capable of indexing all of the Web's estimated 10 billion pages -- every day.
"It will be the first comprehensive index (of the Net)," said Kord Campbell, the programmer behind the Grub software. "We can conceivably crawl every Web page, every day."
By contrast, today's fastest search engines, such as Google or Inktomi, crawl about 150 million pages a day. Google indexes about a third of the Web, and refreshes its index every 30 days, according to LookSmart.
Campbell quietly worked on the software for more than three years in Oklahoma before LookSmart acquired his company and three-person staff in January. Grub was -- and continues to be -- largely an open-source project. "
"If the Grub project attracts enough volunteers, it may be capable of performing the holy grail of Web searches: a real-time "semantic parse" of the Web. Stechert said."
Will Looksmart beat everyone in implementing and deploying the Semantic Web?
No comments:
Post a Comment