Search Engines and Directories

By Jim Calloway and Courtney Kennaday

Let us be blunt: Once upon a time you didn’t know jack about the Internet. Don’t remember? Hop into Mr. Peabody’s Wayback machine and visit early 1996. You’re at a CLE. The lights are dim and Robert Ambrogi (lawyer and Internet pioneer) is demonstrating the Internet to lawyers in the hinterland. As he speaks, a hand is raised in the audience. It’s you. (Thought we didn’t see you there, didn’t you?) Bewildered, you ask, “Can you start over with how you got there…how you got on the Internet?”

Back in the present, you know all about how to get on the Internet. You pay a phone or cable bill every month that proves it, right? But sometimes, when deadlines loom and time is running out, you wonder how much you really know. It can be hard to see the forest for the trees. You wonder if you are missing something. Or worse: You are finding too much.

For most of us, search engines are where we start our Internet search. The five most popular search engines (according to are Google, Yahoo!, Microsoft, Time Warner/AOL, and Ask. Search engines retrieve information from across the Internet and present the results back in a format we can quickly peruse.

As you can guess, all search engines are not alike. They try to distinguish themselves in functionality by using different complicated formulas or algorithms (a math-sounding word). As lawyer and ABA author Sharon Nelson says, the algorithm used by a search engine is a secret more closely guarded than the recipe for Coca-Cola. categorizes search engines as either crawler-based search engines or human-powered directories (or combinations of both). Crawlers (also called “spiders” or “robots”) “crawl” the web following links and then index what is found. The search engine then sifts through the index in response to a query by you, which it presents to you in a ranked result list. (If you would like to know more about how search engines work, visit

It would be nice if all you had to do to succeed in research was use one good crawler search engine, such as Google. But it’s not enough. This is mainly because most crawlers are unable to find everything on the web. Typically, they miss the contents of thousands of specialized searchable databases. Also, some pages are excluded by search engines for arcane reasons known to few mortals. Websites missed by search engines are known to nerdy-types as the invisible web or the deep web.

Directories and portals pick up the slack left by search engines. They are maintained by humans and are helpful in finding so-called invisible web pages. The definition of a directory is pretty much the same as for a portal: a website that functions as a point of access to other websites. Some examples are Yahoo! Directory (, Best of the Web (, Google Directory (, Open Directory Project (, and FindLaw (

Start Broad, Then Narrow

Basic searching can be as simple as typing a few well-chosen terms into a search engine or directory. The results will tell you if you are on the right track. To choose your search terms well, try to imagine how they would appear in your search results. For most sites, the order in which the terms are typed will affect the search results. If they are likely to appear together, try putting the terms in quotations. “American Bar Association” will limit results to those words together, instead of appearing separately many times. Most major search engines allow the use of quotation marks to search phrases. Quotes are also a way to force a search engine to search for common words that are typically excluded— and, the, of, etc. (Google calls these “stop words.”)

We suggest starting with broad queries and then narrowing the search based on what you find. Some lawyers try to get too specific with their terms in the beginning. Unfortunately, this limits the possible results—not necessarily a bad thing in its own right, but something that can be dangerous. Trying to guess the exact words someone else may have used is a frustrating exercise. No one can imagine all the possible ways the information could appear or how other authors may have phrased what you’re looking for. For many searches, a simple query will yield all that you need. At a minimum, it will point you in the right direction for further searches.

Digging Deeper

If leaving no stone unturned is your objective, then it’s time to roll up your sleeves. You will want to search a number of search engines and directories. To save time, you might be tempted to try a meta-search engine. A meta-search engine is a website that submits your search to multiple search engines, theoretically saving you loads of time and giving you results in one place. Examples of meta-search websites are Dogpile (, Clusty (, (, KartOO (, and Copernic Agent Basic (

In our opinion, meta-search engines are one of those ideas that are good in theory but leave something to be desired in practice. The problem is, where search engines are concerned, one size does not fit all. Using the same query for several different search engines doesn’t work well because not all search engines are alike. Quite often, you get different results than you would if you’d gone directly to the search engine’s home page to do your search. You might want to use a meta-search engine if you’re researching something for personal use, but not if you’re staking your professional reputation on it. Leave meta-search engines out of your search repertoire.

We suggest making a list (on paper) of sites you intend to search. It’s easy to get sidetracked while surfing the web. Start with the major search engines and several directories. You may find that a general directory, like Yahoo! Directory, is needed to find a specific topic area directory missed by search engines.

Advanced Searches

We tend to forget that the major search engines have some advanced search functionality, including features familiar to users of Westlaw or LexisNexis. This can be helpful when you need to narrow your results. Just don’t expect to narrow your results from 14 million to a few hundred.

To locate the advanced search function in popular search engines, take the following steps:

  • Google ( From the Google home page, click on the “Advanced Search” link.
  • Yahoo! Search ( Click on “Options,” then “Advanced Search.”
  • ( Click on “Advanced” below the search field.
  • Live Search from Microsoft ( Click on “Options,” then “Advanced Search.”

You can filter results by excluding terms, adding an “or” component, specifying a date range, searching English sites only, or looking for a specific file format, among other techniques.

A trend among major search engines is to suggest words and phrases as you type. It feels a little like charades. You type a few letters and suggestions start popping up. If you see one you like, you can click on it—saves time and manicures. Yahoo! calls this “Search Assist.” Google Toolbar does it as well.

Don’t be intimidated by the label “advanced.” You may think you are not really an advanced Internet searcher, but remember, it’s the search that’s advanced, not the searcher. Think of “advanced” as meaning “don’t have time to mess around.” Besides, at your billing rate, you are advanced.

Directories are like card catalogs in the library—well, they are if you remember what a card catalog was. Because directories are organized already by topic, you can narrow your search by drilling down in the levels first and then typing your search terms. For an example of this, see the FindLaw legal directory. If you are looking for a U.S. Code section, you can click on “Cases and Codes,” then “U.S. Code,” then type in your keyword to search within the U.S. Code. Another nice thing about directories is they have been around on the Internet for years. No matter what your level of experience on the Internet, you have probably seen or used a directory.

We hope you have enough information now to blaze your own trail as an Internet researcher. We don’t like to get too preachy about how you should search on the Internet—the next thing you know, we’ll be trying to invent some new Dewey Decimal system . . . and who wants that? 

Further Resources

Jim Calloway is director of the Oklahoma Bar Association’s Management Assistance Program; he may be reached at . Courtney Kennaday is a practice management advisor for the South Carolina Bar; she may be reached at . An electronic version of SitesForSoreEyes also appears in the Technology eReport; you can find past columns at

Copyright 2008

Back to Top