How to exclude certain directories while using wget

excludewget

I'd like to download a directory from a FTP, which contains some source codes.
Initially, I did this:

wget -r ftp://path/to/src

Unfortunately, the directory itself is a result of a SVN checkout, so there are lots of .svn directories, and crawling over them would take longer time. Is it possible to exclude those .svn directories?

Best Answer

wget -X directory_to_exclude[,other_directory_to_exclude] -r ftp://URL_ftp_server

SERVER
    |-logs
    |-etc
    |-cache
    |-public_html
      |-images
      |-videos ( want to exclude )
      |-files
      |-audio  (want to exclude)

wget -X /public_html/videos,/public_html/audio ftp:SERVER/public_html/*

Related Solutions

Using client certificates with wget

I studied a week on this, finaly with the help I got from this page.

The command I used to connect is:

wget --ca-cert=/etc/ssl/certs/winhostname.pem --certificate=/etc/ssl/private/linuxhost.pem \
     --private-key=/etc/ssl/private/linuxhost.key https://winhostname.home.net:8443/winhosturl.asmx

Linux – How to symlink folders and exclude certain files

I assume you really mean "symlink everything inside the folder"...

Conceptually, you want something like:

for each file in base folder not in exclude list
    create symlink in target folder pointing back to base folder
done

bash logic to achieve this goes something like

#! /bin/bash

exclude=( "foo.conf" "bar.conf" )

for file in *; do
    for (( index = 0; index < ${#exclude[@]}; index++ )); do
        if [[ ${file} != ${exclude[${index}]} ]]; then
            ln -s ${file} ${target}/${file}
        fi  
    done
done

Assuming you don't know already, the Advanced Bash-Scripting Guide should give you enough to work out what globbing you want (perhaps find) and how to convert that to a function (perhaps a recursive one).

Best Answer

Related Solutions

Using client certificates with wget

Linux – How to symlink folders and exclude certain files

Related Topic