Package: htdig (1:3.2.0b6-3.1)

WWW search system for an intranet or small internet

The ht://Dig system is a complete World Wide Web indexing and searching system for a small domain or intranet. This system is not meant to replace the need for powerful internet-wide search systems like Lycos, Google, or Yahoo!. Instead it is meant to cover the search needs of a single company, campus, or even a particular subsection of a website.

As opposed to some WAIS-based or web-server based search engines, ht://Dig can span several web servers at a site. The type of these different web servers doesn't matter as long as they understand the HTTP 1.0 protocol.

Features:

   * Intranet searching
   * It is free
   * Robot exclusion is supported
   * Boolean expression searching
   * Configurable search results
   * Fuzzy searching (different algorithms supported)
   * Searching of HTML and text files
   * Keywords can be added to HTML documents
   * Email notification of expired documents
   * A Protected server can be indexed
   * Searches on subsections of the database
   * Full source code included
   * The depth of the search can be limited
   * Full support for the ISO-Latin-1 character set

Please note that ht://Dig is a resource-hog, with respect to processor usage, when indexing.

Disk space requirements:

13.000 documents indexed: 150MB disk space with a 'wordlist database'

                               93MB disk space without a 'wordlist'

Multiplying the number of documents to index by 12.000 comes pretty close to the real disk space used.

Tags: Implemented in: C++, User Interface: Command Line, World Wide Web, Networking: Server, Network Protocol: HTTP, Role: Program, Purpose: Searching, World Wide Web: CGI, Works with: Text, Supports Format: HTML, Hypertext Markup Language