For our purpose, however, we need to explore the sites more thoroughly using a host crawler.
we continue our search for files with a host crawler which uses the documents found by the search engines as the starting point.