... there is always Deep Web Technologies, a software developer spun out of Los Alamos National Laboratory, which claims that it searches the other 98% of Internet content that Google and Yahoo "ignore".
DWT announced that it has been awarded a Phase 1 Small Business Innovation Research grant by the Department of Energy to investigate ways to scale so-called Internet Federated search. DWT describes this as "search to accomodate thousands of simultaneous users seeking information from millions of documents and Web pages from thousands of sources of information in real time."
The DOE says DWT's work could help to develop a search appliance that could search 1,000 distributed database of knowledge in the physical sciences, all in parallel.
DWT was founded in 2002 and powers sites including Science.gov and the E-Print Network.
Linda Leung, lleung@nww.com