I have submitted the pages on my site to the popular search engines, AlthVista, HotBot, and Lycos. These are supposed to examine all my pages, and index them. I have been quite disappointed by the results. In most cases, they have never visited most of my site. Even when I feed them each page one at a time, they fail to incorporate the contents into their index.
The failure of these seach engines has two undesirable effects:
I am currently experimenting with PicoSearch and Atomz. This technology is designed to index individual sites, and provides a search engine for that site. Both search engines seem to work as advertised, and they are both free.
Still, this does not correct the basic problem: people looking for information on topics at my site will not find them with AltaVista or HotBot. These organizatons should be ashamed, AltaVista has never worked properly: it visited and re-indexed my sites much less frequently than claimed, but at least it used to do that. Some time in the Fall of 1999, AltaVista dropped all information on my site from its index. My attempts to get back in so far have been fruitless.
You can check the current status for yourself at AltaVista and at HotBot. At the moment (December 1999) neither one has more than a small fraction of what is on my web, and a lot of what HotBot has is junk that was placed there by FrontPage.