Can wget estimate a website’s size before mirroring it?

A crawl-based estimate can be very misleading. Dynamically generated URLs may produce far more content than expected, or an effectively endless set of pages, while content hidden behind forms, scripts, authentication, or unlisted links may be missed entirely. The only dependable size is available on the server itself—for example, if all of the site’s files are under `/var/www`, the administrator could use `du -sh /var/www`. From outside the server, treat any result as a rough estimate rather than a guarantee.

There isn’t a reliably accurate way to know the final size without transferring the files. You can perform a spider or header-only pass first, but that still takes time and may require contacting every resource. A second pass to download everything will therefore involve much of the work twice.

The deletion warning is probably about temporary files that wget creates while spidering, or about running it in a directory containing files that conflict with its output. It’s safest to run both the spider check and the actual mirror in a new, empty temporary directory. Spider mode is intended to inspect links, not erase arbitrary files, but avoid testing it in a directory containing anything important. Options such as `--no-clobber` may help prevent overwriting, although they don’t make a size estimate accurate.