Some good info on Wolfram Alpha: Impressive: The Wolfram Alpha “Fact Engine” and See Wolfram Alpha in Action: Our Screenshots. The screenshots suggest that Wolfram Alpha has more in common with Business Intelligence tooling than with search engines. And the database they’re building to support this is a big datawarehouse containing data from selected sources:
Wolfram Alpha isn’t crawling the web and “scraping” information, a process where you try to extract data from a web page. Instead, it’s working with a variety of providers to gather public and private information.
It seems that organizing all the data currently is a manual process, but I would expect that Wolfram Alpha might be able to benefit from the increasing availability of LinkedData. Wolfram Alpha might be an example of what search engines can become when more semantically tagged LinkedData becomes available: BI dashboards for all the data on the internet. One challenge will be to make it flexible enough that everybody can find the information they’re looking for. For example, the weather information presented is nice, but i would want to be able to drill into the windstatistics, and wind-forecast. It’s about time that the NOAA made their weather data available as LinkedData.