The Evolution Of Search

Home
>
Web Development
>
The Evolution Of Search

The Evolution Of Search

There have been many significant changes to the face of search over the last several years with engines becoming more intelligent than ever before. Today's users expect mainly fast, easy, relevant and satisfactory search results. In response to this search engines have responded by giving users more control over search results than ever through the emergence of alternative search engines.

One instance of these so-called alternative search engines goes by the name of Nutch (http://www.nutch.org/). Nutch is a two-year-old open source project, which has been hosted previously at Soundforge and backed by a non-profit organization. Since then it has been determined that the Apache license is the most appropriate, with Nutch no longer requiring the overhead of an independent non-profit organization. The board of directors and the developers both were in favor of the move to the Apache Foundation.

Nutch builds on Lucene technology, which was developed under the watchful eye of Doug Cutting, the primary developer for both of these open source projects. Doug has been working in the field for almost two decades and has spent three years at Apple, four years at Excite as well as 5 years at Xerox PARC, so it is safe to say that Doug definitely knows his stuff. Lucene is suitable for nearly any application that requires full-text search, especially cross-platform. It is a full-featured, high-performance, text search engine library, coded entirely in Java to implement web search. Nutch is an application; you can download it and run it. It adds a crawler and other web-specific stuff to Lucene as well as it's very own search algorithm and a link analysis module. Nutch aims to search the entire web like Google or Yahoo! but has a few tricks up its sleeve thanks to the beauty of open source licensing.

I recently had the privilege to interview Mel Strocen, the CEO of Jayde Online, Inc. (http://www.exactseek.com/), one of the Web's major online publication and search companies. Mel had some very exciting news to report on how Jayde is planning to utilize the Nutch application.

Jayde has been developing a customized version of Nutch for the last eight months and is planning to launch a search engine based on the Nutch technology within the next few weeks. The initial beta version will consist of a network of dedicated servers with an index of between 20 and 30 million website listings.

The real potential of this new search engine, and others using the Nutch technology, lies in the fact that it is open source and uses a "Plug-In Architecture". What this means is that the engine will be perpetually evolving and constantly improving to better facilitate the needs of searchers. One terrific example that shows us just how beneficial this type of open source plug-in technology can be is the FireFox web browser (http://www.firefox.com/).

FireFox, in its short existence has eaten up a significant portion of the once all mighty Internet Explorer's market share. The popularity of this browser is due to the fact that it is constantly making itself smarter. You can now find a plug-in for virtually anything that you require , ranging from web developer, downloading, and search tools to privacy, security, website integration and humorous plug-ins. You name it, there is an extension for it. The extension library consists of nearly six hundred different plug-ins and is growing daily thanks to the help of contributors everywhere.

Now just imagine implementing this type of plug-in technology to a search engine, with one type of plug-in for say searching MP3s and another plug-in for downloading PDFs. The possibilities of this new open source search technology are infinite. Now the term "open-source search engine" may make a lot of people's minds wander towards the idea of Black Hat search engine optimization. The primary developer of Nutch, Doug Cutting, feels that the closed-source advantage is not nearly as much of a factor as one might imagine it to be. The fact that the search engine is open-source allows sp@mmers to be detected far faster than that of closed-source search engines latest sp@m detecting algorithms. Either way, you know that the sp@mmers will eventually figure out how it works, the only difference is how quickly. So the top anti-sp@m techniques, closed or open source, are those that continue to function even when their mechanism is known.

Another type of alternative search engine technology has just recently been released to beta version is "Relevancy Rank" from the Claria Corporation (http://www.claria.com/relevancyrank/about/), the minds behind Gator. I had the pleasure to conduct an interview with the Vice President and Executive Chief of Marketing, Scott Eagle. He had some very interesting things to say about the launch of this new product and what exactly the benefit of Relevancy Rank has to the user. This unique search technology takes the results from the top search engines and applies its very own algorithms to output to the user the most relevant results.

Relevancy Rank is a combination of personalization, localization, time spent at any one site, click through rates as well as conversions. These are all taken into account to provide the most relevant results. "For an example, if you happened to be a zoologist who loved to search for different animals and information relating to animals and you entered the word "Jaguar" you would be returned far different results from say a car enthusiast who searched frequently for different types of vehicles and also typed in the word "Jaguar"", noted Scott. Relevancy Rank helps to provide you with the most relevant results based on your previous search behavior.

With the end users expectations continuing to grow, these twists on the way that results are gathered and displayed are an enormous help in satisfying the user's hunger to get to the results that they are looking for. I am quite anxious to see how these new forms of search technology fair out over the next several months. One thing is for sure, these new technologies are sure to revolutionize the way that web search is conducted and pave a new path for the evolution of search.

Tyler Huston is the SEO Manager for Beanstalk Search Engine Positioning Inc. Beanstalk is proud to offer their guaranteed SEO services to clients from around the world. To keep updated on the latest going's on in the search engine world watch for Tyler's posts on the Beanstalk SEO blog.

In The News:

Top Stories - Google News
Updated : Fri, 10 Sep 2010 12:12:04 GMT+00:00

Top Adviser to Lead Panel on Economy - New York Times

Washington Post

Top Adviser to Lead Panel on Economy
New York Times
WASHINGTON — President Obama on Friday will promote a longtime economic adviser, Austan D. Goolsbee, to chairman of his Council of Economic Advisers, signaling continuity even as a high unemployment rate has left much of the public ...
Obama taps Goolsbee for key economic postUSA Today
Goolsbee Said to Be Picked to Head White House Economic PanelBloomberg
Obama to tap Goolsbee to head Council of Economic AdvisersChicago Sun-Times
AFP -New York Daily News -Reuters
all 579 news articles »

Publ.Date : Fri, 10 Sep 2010 11:01:14 GMT+00:00

Florida pastor tells TODAY undecided on whether to burn Qurans - msnbc.com

Telegraph.co.uk (blog)

Florida pastor tells TODAY undecided on whether to burn Qurans
msnbc.com
GAINESVILLE, Fla. — The pastor behind plans to burn Qurans on the anniversary of Sept. 11 failed to clarify the confusion during an interview with Meredith Viera on NBC's TODAY Show on Friday. According to Jones, he had agreed to call off the event ...
Reaction to Quran burning pours in from around the worldCNN
Pastor Dave Burke: Terry Jones - Saints and MonstersBBC News
Pastor rules out Koran burning 'for now'Telegraph.co.uk
Reuters -USA Today -The Guardian
all 12,214 news articles »

Publ.Date : Fri, 10 Sep 2010 11:54:53 GMT+00:00

Parties and prayers celebrate Eid - BBC News

The Hindu

Parties and prayers celebrate Eid
BBC News
The Muslim community in Wales is celebrating the end of Ramadan, the Islamic holy month of fasting. In Cardiff the Eid el-Fitr holiday was marked with morning prayers around the city's 11 mosques, before families returned home for a day of feasting. ...
Quran controversy casts pall over Eid celebrationsCNN International
Nigeria: Muslims Celebrate Eid-El-Fitr TodayAllAfrica.com
Students as teachers: UH Muslim students talk about faithHouston Chronicle
Xinhua -ABC Online -Denver Post
all 844 news articles »

Publ.Date : Fri, 10 Sep 2010 11:54:36 GMT+00:00

Judge: 'Don't Ask, Don't Tell' Is Unconstitutional - ABC News

Daily Mail

Judge: 'Don't Ask, Don't Tell' Is Unconstitutional
ABC News
A federal judge in California ruled tonight that the “Don't Ask, Don't Tell” policy that bars openly gay service members from serving in the military is unconstitutional. "Defendants faced the burden at trial of showing the “Don't Ask, Don't tell” Act ...
Cal. Judge: DADT UnconstitutionalTIME
Federal judge rules 'don't ask, don't tell' policy is unconstitutionalLos Angeles Times
Calif judge to stop 'don't ask, don't tell' policyThe Associated Press
Washington Post -Reuters -The Star-Ledger - NJ.com
all 1,133 news articles »

Publ.Date : Fri, 10 Sep 2010 11:28:26 GMT+00:00

Combat Game Goes Too Far for Military - New York Times

Telegraph.co.uk

Combat Game Goes Too Far for Military
New York Times
By AG SULZBERGER LEAVENWORTH, Kan. — Sgt. First Class Brian Hampton knows it is just a video game. But the details are unnervingly familiar: the uniforms, the weapons, even the military bases and desert towns where the action is set. ...
Fallen Special Forces soldier to be awarded Medal of HonorLos Angeles Times
Medal of Honor "needs 3 million sales for sequel" - will it get there?The Guardian (blog)
Montgomerian to attend Medal of Honor ceremony honoring late airmanMontgomery Advertiser
USA Today -The Associated Press -PC Magazine
all 396 news articles »

Publ.Date : Fri, 10 Sep 2010 00:53:44 GMT+00:00

1 Dead, 53 Homes Destroyed After Gas Explosion in Calif. - ABC News

Globe and Mail

1 Dead, 53 Homes Destroyed After Gas Explosion in Calif.
ABC News
At least one person has been killed and 53 homes have been destroyed after an apparent gas line explosion in a residential neighborhood south of San Francisco, according to officials. The explosion occurred just after 6:15 pm Thursday ...
Huge blast destroys dozens of homes near San Franciscomsnbc.com
Explosion Destroys Homes south of San FranTIME
Gas line explosion burns San Bruno neighborhoodSan Francisco Chronicle
New York Times -Los Angeles Times (blog) -The Associated Press
all 1,718 news articles »

Publ.Date : Fri, 10 Sep 2010 11:19:02 GMT+00:00

After Apple about-face, Adobe resurrects Flash tool - Computerworld

Reuters

After Apple about-face, Adobe resurrects Flash tool
Computerworld
Computerworld - Adobe on Thursday said it would resurrect a tool that lets developers port Flash applications to the iPhone after Apple did an about-face earlier in the day. The move came just hours after Apple reversed an April ...
Adobe on Apple: Our glass is half fullFortune
Adobe reignites Flash on iOSRegister
Adobe resurrects Flash-to-iPhone app toolCNET
Inquirer -PC World -New York Times
all 1,386 news articles »

Publ.Date : Fri, 10 Sep 2010 10:55:08 GMT+00:00

Add RSS Feeds to My Website
Turnkey Websites

The Evolution Of Search

Web Development