Re: RUNNING FTP IN BATCH MODE


Sung Kwon Chung (sung@june.cs.washington.edu)
28 Aug 88 06:48:11 GMT


Here are two quick solutions to do batch-mode ftp. They are written in
shell script, that means they are of no use for some of you who don't have
the shell, sorry.

The first one (ftp1.sh) is for simple ftp'ing. It works fine for small number
of files. The limit depends on the command buffer size of `mget' and some
other factors unknow to me :-) Is is used like,
        % ftp1.sh host pub a b c d e f g
to get file a, b, ..., g on host:pub.

The second one (doftp) is a bit more complicated, but versatile (and
extensible if you want). doftp uses other two shell scripts (ftp.sh and
waitfor). It requires a file which contains the names of the files to be
ftp'ed. Each line in the file contains one file name. If `files.list'
contains wanted file names on host:pub,
        % doftp host pub files.list
will transfer all of them (of course, if everything goes right).
I used it to bring over 70 odd (irregularly selected) files from
an archive site.

To use them, extract the scripts to their names and do "chmod +x" them.
Please see the notes at the end of this message before trying them.
========================= ftp1.sh =========================
#!/bin/sh
#
# simple ftp
# Usage: ftp1.sh host direcrory file1 file2 file3 .....
#
host=$1; directory=$2; shift; shift;
ftp -n -i $host << ___EOF___
user anonymous guest
cd $directory
mget $*
bye
___EOF___
exit $?
------------------------------------------------------------

The following three files are for the `doftp'.
========================= doftp =========================
#!/bin/sh
# Usage: doftp host directory namelistfile
# needs ftp.sh and waitfor
# Set up the pipes and temp files, and invoke ftp and its command feeder
# ftp-out$$ : conversation during the job, feed-back to ftp.sh
# ftp-log$$ : just for logging the commands
# sung@cs.washinton.edu, 8/22/88
if test "$#" != "3"
then
    echo "doftp host directory namelistfile" 1>&2
    exit 1
fi

oldmask=`umask` #
umask 077 # if we need to make the conversation unreadable
cat >ftp-out$$ </dev/null
cat >ftp-log$$ </dev/null
umask $oldmask

( ftp.sh $2 $3 <ftp-out$$) |
                     tee ftp-log$$ |
                                    ( ftp -v -n $1 >ftp-out$$ 2>&1 )
------------------------------------------------------------
========================= ftp.sh =========================
#!/bin/sh
# ftp.sh: ftp command feeder
#
# Uage: (sleep 1; ftp.sh dir namelistfile <ftp-out) |
# (ftp -v -n host-name >ftp-out)
# See `doftp' and `waitfor'
# sung@cs.washington.edu, 8/22/88

if test "$#" != "2"
then
        echo " Uage: (sleep 1; ftp.sh dir namelistfile <ftp-out) | (ftp -v -n host-name >ftp-out)" 1>&2
        exit 2
fi

echo "user anonymous guest" # login
result=`waitfor "230"` # okay?
if test "$result" = "ERROR"
 then exit 1
fi

echo "cd $1"
result=`waitfor "250"` # 250 CWD successful
if test "$result" = "ERROR"
 then exit 1
fi

# other FTP commands here, if needed

for f in `cat $2` # main loop
do
   destName=$f # file name change may be used here. a:b.c --> a-b.c
   if test ! -f $destName # get it, only when it is not there
   then
      echo "get $f $destName"
      result=`waitfor "226"` # 226 Transfer complete
      case "$result" in
      "OK") echo "got $f" 1>&2 ;;
      "ERROR") echo "failed getting $f, abort the job" 1>&2;
               exit 1;;
      # other cases may need to be considered here
      esac
   fi
done
exit 0
------------------------------------------------------------
========================= waitfor ==========================
#!/bin/sh
# Usage: result=`waitfor string`
# wait for given string from std input while monitoring error messages
# Usually, `string' is an expected message number (e.g., 250). It
# detects 5xx error messages.
# sung@cs.washington.edu, 8/22/88
while true
do
  read aLine
  if test "$aLine" != ""
  then
    case "$aLine" in
    "$1"*) echo "OK"; exit 0 ;;
    *'timed out'*) echo "ERROR"; exit 1 ;; # timeed out message
    5*) echo "ERROR"; exit 1 ;; # 5XX error msg
    *) ;
    esac
  else
   sleep 3
  fi
done
------------------------------------------------------------
The scripts are for anonymous login. For non-anonymous login, you can
change the `user' command line. (Be sure to set proper file protection
in this case.) Or you can use .netrc file and auto-login feature. In this
case, '-n' option is in `doftp' should be removed.

There is much room for improvement especially for error message handling,
and retrial in the case of partial fairlure. So please feel free to
make further extensions or modifications as needed.

Hope this helps some of you.

Sung K. Chung | sung@cs.washington.edu
Dept. of Computer Science, FR-35 | {decvax,ucbvax}!uw-beaver!uw-june!sung
University of Washington |
Seattle, WA 98195 |



This archive was generated by hypermail 2.0b3 on Thu Mar 09 2000 - 14:43:14 GMT