Give up on POP3 UIDs, they're a dead loss.

[~andy/fetchmail] / fetchmail.man

.\" Copyright 1993-95 by Carl Harris, Jr. Copyright 1996 by Eric S. Raymond
.\" All rights reserved.
.\" For license terms, see the file COPYING in this directory.
.TH fetchmail LOCAL
.SH NAME
fetchmail \- retrieve mail from a mailserver using POP2, POP3, APOP, or IMAP
.SH SYNOPSIS
.B fetchmail
[\fI options \fR] \fI [server-host...]\fR
.SH DESCRIPTION
.I fetchmail
is a batch mail retrieval utility intended to be used over on-demand
TCP/IP links (such as SLIP or PPP connections).  It fetches mail from
remote mail servers and forwards it to your local (client) machine's
delivery system, where it can then be be read by normal mail user
agents such as \fIelm\fR(1) or \fIMail\fR(1).
.PP
The
.I fetchmail
program can gather mail from servers supporting POP2 (as specified in RFC
937), POP3 (RFC 1725), IMAP2bis (as implemented by the 4.4BSD imapd
program), and IMAP4 (RFC1730).  It can use (but does not require) the
RPOP and LAST facilities removed from later POP3 versions.
.PP
The behavior of
.I fetchmail
is controlled by comand-line options and a run control file,
.I ~/.fetchmailrc
the syntax of which we describe below.  Command-line options override
.I ~/.fetchmailrc
declarations.
.PP
To facilitate the use of
.I fetchmail
in scripts, pipelines, etc, it returns an appropriate exit code upon 
termination -- see EXIT CODES below.
.SH OPTIONS
Each server name that you specify (following the options on the
command line) will be queried.  If you don't specify any servers
on the command line, each server in your 
.I ~/.fetchmailrc
file will be queried.
.TP
.B \-2
Use Post Office Protocol version 2 (POP2).  See also the 
.B \--protocol
option, below.
.TP
.B \-3
Use Post Office Protocol version 3 (POP3).  See also the
.B \--protocol
option, below.
.TP
.B \-a, --all
POP3 only.  Retrieve both old (previously retrieved) and new messages from 
the mailserver.
.TP
.B \-S host, --smtphost host
Specify an SMTP forwarding host (other than localhost).  Normally
fetched mail is delivered by SMTP over a socket to the client machine
.I fetchmail
is running on (this simulates the way mail would
be delivered to the client by a normal Internet TCP/IP connection).
With this option you can specify another host to deliver to.
.TP
.B \-m mda, --mda mda
Specify a mail delivery agent to use.  See OUTPUT OPTIONS below for a
complete description.
.TP
.B \-o folder, --local folder
Causes retrieved messages to be appended to file named by the folder 
argument. See OUTPUT OPTIONS below for a complete description.
.TP
.B \-c, --stdout
Causes retrieved messages to be written to stdout instead of a mail folder.
See OUTPUT OPTIONS below for a complete description.  You may not specify
both the
.B \-c
and 
.B \-o
options on the same command line.
.TP
.B \-F, --flush
POP3/IMAP only.  Delete old (previously retrieved) messages from the mailserver
before retrieving new messages.
.TP
.B \-f pathname, --fetchmailrc pathname
Specify an alternate name for the .fetchmailrc run control file.
.TP
.B \-k, --keep
Keep retrieved messages in folder on remote mailserver.  Normally, messages 
are deleted from the folder on the mailserver after they have been retrieved
(unless 
.I fetchmail
was compiled with the KEEP_IS_DEFAULT option).  Specifying the 
.B keep 
option causes retrieved messages to remain in your folder on the mailserver.
.TP
.B \-K, --kill
Delete retrieved messages from the remote mailserver.  If 
.I fetchmail
is compiled with the KEEP_IS_DEFAULT option, the
.B kill
option forces retrieved mail to be deleted.
.TP
.B \-l lines, --limit lines
POP3 and IMAP only.  Retrieve no more than the specified number of
lines (POP3) or characters (IMAP) of each message body (plus message
headers). The
.B keep
option is implied by the
.B limit
option -- i.e. messages downloaded with the 
.B limit
option remain on the remote mailserver.
.TP
.B \-p, \--protocol proto
Specify the protocol to used when communicating with the remote 
mailserver.  If no protocol is specified,
.I fetchmail
will try each of the supported protocols in turn, terminating after
any successful attempt.
.I proto 
may be one of the following:
.RS
.IP IMAP
IMAP2bis, a compatible subset of IMAP4.
.IP POP2
Post Office Protocol 2
.IP POP3
Post Office Protocol 3
.IP APOP
Use POP3 with MD5 authentication.
.RE
.TP
.B \-P, --port
The  option permits you to specify a TCP/IP port to connect on.  You
will need to specify this in order to use RPOP authentication.  Otherwise
this option will seldom be necessary as all the supported protocols have
well-established default port numbers.
.TP
.B \-r folder, --remote folder
Causes an alternate mail folder on the mailserver to be retrieved.
The syntax of the folder name is server dependent, as is the default
behavior when no folder is specified.  Fortunately, most POP2 and IMAP
servers have a reasonable default behavior, so use of this option
should be limited to fairly specialized applications.  POP3 does not
support a folder specification in the protocol.
If the
.B remote
option is used in conjunction with the POP3 protocol, the remote folder 
specification is ignored.
.TP
.B \-s, --silent
Silent mode.  Suppresses all progress/status messages that are normally
echoed to stderr during a POP connection.  If both the 
.B silent
and
.B verbose
options are specified, the 
.B verbose
option takes precedence.
.TP
.B \-u name, --username name
Specifies the user idenfication to be used when logging-in to the mailserver.
The appropriate user identification is both server and user-dependent.  
The default is your login name on the machine that is running 
.I fetchmail.
See USER AUTHENTICATION below for a complete description.
.TP
.B \-v, --verbose
Verbose mode.  All control messages passed between 
.I fetchmail
and the mailserver are echoed to stderr.  Specifying
.B verbose
causes normal progress/status messages which would be redundant or meaningless
to be modified or omitted.
.TP
.B \-N, --norewrite
Normally,
.I fetchmail
edits RFC-822 address headers (To, From, Cc, Bcc, and Reply-To) in
fetched mail so that any mail IDs local to the server are expanded to
full addresses (@ and the POP host name are appended).  This enables 
replies on the client to get addressed correctly (otherwise your
mailer might think they should be addressed to local users on the
client machine).  This option disables the rewrite.
.TP
.B \-V, --version
Displays the version information for your copy of 
.I fetchmail.
No POP connection is made.
Instead, for each server specified, all option information
that would be computed if
.I fetchmail.
were connecting to that server is displayed.
.TP
.SH USER AUTHENTICATION
User authentication in 
.I fetchmail
is very much like the authentication mechanism of 
.I ftp(1).
The correct user-id and password depend upon the underlying security
system at the mailserver.  
.PP
If the mailserver is a Unix machine on which you have an ordinary user 
account, your regular login name and password are used with 
.I fetchmail.
If you use the same login name on both the server and the client machines,
you needn't worry about specifying a user-id with the 
.B \-u
option \-\- 
the default behavior will use your login name on the client machine as the 
user-id on the server machine.  If you use a different login name
on the server machine, specify that login name with the
.B \-u
option.  e.g. if your login name is 'jsmith' on a machine named 'mailgrunt',
you would start 
.I fetchmail 
as follows:
.IP
fetchmail -u jsmith mailgrunt
.PP
The default behavior of 
.I fetchmail
is to prompt you for your mailserver password before the POP connection is
established.  This is the safest way to use 
.I fetchmail
and ensures that your password will not be compromised.  You may also specify
your password in your
.I ~/.fetchmailrc
file.  This is convenient when using 
.I fetchmail
in daemon mode or with scripts.
.PP
On mailservers that do not provide ordinary user accounts, your user-id and 
password are usually assigned by the server administrator when you apply for 
a mailbox on the server.  Contact your server administrator if you don't know 
the correct user-id and password for your mailbox account.
.PP
POP3 versions up to the RFC1225 version supported an alternate
authentication mechanism called RPOP intended to remove the security
risk inherent in sending unencrypted account passwords across the net
(in RFC1460 this facility was replaced with APOP).  If you specify the
RPOP protocol and a connection port in the privileged range (1..1024),
.I fetchmail will
ship your password entry to the mail server as an RPOP id.
(Note: you'll need to be running fetchmail setuid root for RPOP to
work --
.I fetchmail
has to bind to a privileged port locally in order for the mail 
server to believe it's allowed to bind to a privileged remote port.)
.PP
RFC1460 introduced APOP authentication.  In this variant of POP3,
you register an APOP password on your server host (the program
to do this with on the server is probably called \fIpopauth\fR(8)).  You
put the same password in your 
.I .fetchmailrc
file.  Each time 
.I fetchmail
logs in, it sends a cryptographically secure hash of your password and
the server greeting time to the server, which can verify it by
checking its authorization database. 
.PP
.SH OUTPUT OPTIONS
The default behavior of 
.I fetchmail
is to ship mail via SMTP to port 25 on the machine it is running on
(localhost), just as though it were being passed in over a normal TCP/IP link.
This normally results in the mail being delivered locally via your
system's default MDA (Mail Delivery Agent, usually
.I /usr/lib/sendmail
but your system may use a different MDA).
.PP
You can force mail to be passed to an MDA directly with the -mda or -m
option.  Some possible MDAs are "/usr/lib/sendmail -oem %s",
"/usr/formail", and "/usr/bin/deliver %s" (if the MDA string contains
%s, that escape will be expanded into your username on the client
machine).  This shouldn't be necessary unless for some reason you
want to bypass your system's default MDA.
.PP
Using the 
.B \-o
option, you can specify a mail folder to which retrieved
messages will be appended;
.I fetchmail
always writes the retrieved messages using Unix mail folder format so
the folder will be parsed correctly by Unix mail programs such as
.I elm
or
.I pine. 
.PP
If you prefer, for example, to have your POP
mail from a machine called 'mailgrunt' stored in the 
.I mbox
file in your home directory, you would start 
.I fetchmail
as follows:
.IP 
fetchmail \-o $HOME/mbox mailgrunt
.PP
Note that the folder specified with
.B \-o
is write-locked while fetchmail is writing to it,   
.PP
.I fetchmail
can be used in a shell pipeline by using the 
.B \-c
option.  In this mode, 
.I fetchmail
writes the retrieved messages to stdout, instead of a mail folder.  This would
allow you, for instance, to pass the incoming mail through a filter that
discards mail marked as 'Precedence: junk'.  Suppose you've written an AWK
script called 'dumpjunk.awk' to implement a junk mail filter.  The appropriate
syntax to retrieve your mail from 'mailgrunt', pass it through the filter,
and write it to a folder called 'realmail' in your home directory would be:

.nf
  fetchmail -c mailgrunt | awk -f dumpjunk.awk >$HOME/realmail
.fi
.PP
The progress/status messages written to stderr when the 
.B \-s
option has not been specified, do not interfere with the message stream, which 
is written to stdout.  You may even use 
.B \-v
and 
.B \-c
together without corrupting the message stream.  It is a good idea to use the
.B \-k
option when using 
.B \-c
to insure that your messages will not be lost if part of the shell pipeline 
does not function incorrectly.  The safest bet would be something like:

.nf
  fetchmail -k -c mailgrunt | myfilter >$HOME/filtered.mail
.fi
.PP
followed by

.nf
  fetchmail -c mailgrunt > /dev/null
.fi
.PP
when you're sure the messages were correctly processed by 'myfilter'.
.PP
.SH DAEMON MODE
The 
.B --daemon
or
.B -d 
option runs 
.I fetchmail
in daemon mode.  You must specify a numeric argument which is a
polling interval in seconds.
.PP
In daemon mode, 
.I fetchmail
puts itself in background and runs forever, querying each specified
host and then sleeping for the given polling interval.
.PP
Simply invoking
.IP
fetchmail -d 900
.PP
will, therefore, poll the hosts described in your 
.I ~/.fetchmailrc
file once every fifteen minutes.
.PP
Only one daemon process is permitted per user; in daemon mode,
.I fetchmail
makes a per-user lockfile to guarantee this.  The option
.B --quit
will kill a running daemon process.
.PP
The
.B -L
or
.B --logfile
option allows you to redirect status messages emitted while in daemon
mode into a specified logfile (follow the option with the logfile name).
This is primarily useful for debugging configurations.
.SH THE RUN CONTROL FILE
The preferred way to set up fetchmail (and the only way if you want to
specify a password) is to write a .fetchmailrc file in your home directory.
To protect the security of your passwords, your ~/.fetchmailrc may not have
more than u+r,u+w permissions; 
.I fetchmail
will complain and exit otherwise.
.PP
Comments begin with a '#' and extend through the end of the line.
Otherwise the file consists of a series of server entries.
Blank lines between server entries are ignored.
Keywords and identifiers are case sensitive.
When there is a conflict between the command-line arguments and the
arguments in this file, the command-line arguments take precedence.
.PP
Legal keywords are:

    server
    protocol (or proto)
    username (or user)
    password (or pass)
    remotefolder (or remote)
    localfolder (or local)
    mda
    smtphost (or smtp)
    keep
    flush
    fetchall
    rewrite
    skip
    nokeep
    noflush
    nofetchall
    norewrite
    noskip
    port
.PP
All these correspond to the obvuious command-line arguments except
two: \fBpassword\fR and \fBskip\fR.
.PP
The \fBpassword\fR option requires a string argument, which is the password
to be used with the entry's server.
.PP
The \fBskip\fR option tells
.I fetchmail 
not to query this host unless it is explicitly named on the command
line.  A host entry with this flag will be skipped when
.I fetchmail
called with no arguments steps through all hosts in the run control file.
(This option allows you to experiment with test entries safely.)
.PP
Legal protocol identifiers are

    auto (or AUTO)
    pop2 (or POP2)
    pop3 (or POP3)
    imap (or IMAP)
    apop (or APOP)
    rpop (or RPOP)
.PP
Basic format is:

.nf
  server SERVERNAME protocol PROTOCOL username NAME password PASSWORD 
.fi
.PP
Example:

.nf
  server pop.provider.net protocol pop3 username jsmith password secret1
.fi
.PP
Or, using some abbreviations:

.nf
  server pop.provider.net proto pop3 user jsmith password secret1
.fi
.PP
Multiple servers may be listed:

.nf
  server pop.provider.net proto pop3 user jsmith pass secret1
  server other.provider.net proto pop2 user John.Smith pass My^Hat
.fi
.PP
Other possibilities (note use of \ to escape newline -- this is all 
one server definition.

.nf
  server pop.provider.net       \e
        proto pop3              \e
        port 3111               \e
        user jsmith             \e
        pass secret1            \e
        localfolder ~/mbox
.fi
If you need to include whitespace in a parameter string, enclose the
string in double quotes.  Thus:

.nf
  server mail.provider.net      \e
        proto pop3              \e
        user jsmith             \e
        pass secret1            \e
        mda "/bin/mail %s"
.fi
Finally, you may have an initial server description headed by the keyword
`defaults' instead of `server' followed by a name.  Such a record
is interpreted as defaults for all quries to use. It may be overwritten
by individual server descriptions.  So, you could write:

.nf
  defaults                      \e
        proto pop3              \e
        user jsmith             \e
        mda "/bin/mail %s"
  server pop.provider.net       \e
        pass secret1            \e
  server mail.provider.net      \e
        pass secret2            \e
.fi
.SH EXIT CODES
To facilitate the use of 
.I fetchmail
in shell scripts and the like, an exit code is returned to give an indication
of what occured during a given POP connection.  The exit code can be tested
by the script and appropriate action taken.
.PP
A simple example follows.  This Bourne shell script executes 
.I fetchmail
and, if some messages were successfully retrieved from a mailserver retrieved
from the command line, it starts the 
.I mail
utility to read those messages.  Otherwise, it prints a brief message, and
exits.
.EX 0
#!/bin/sh

if fetchmail $1
then
  mail
else
  echo "No mail to read."
fi
.EE
.PP
The exit codes returned by 
.I fetchmail
are as follows:
.IP 0
One or more messages were successfully retrieved.
.IP 1
There was no mail awaiting retrieval.
.IP 2
An error was encountered when attempting to open a socket for the POP 
connection.  If you don't know what a socket is, don't worry about it --
just treat this as an 'unrecoverable error'.
.IP 3
The user authentication step failed.  This usually means that a bad 
user-id, password, or RPOP id was specified.
.IP 4
Some sort of fatal protocol error was detected.
.IP 5
There was a syntax error in the arguments to 
.I fetchmail.
.IP 6
Some kind of I/O woes occurred when writing to the local folder.
.IP 7
There was an error condition reported by the server (POP3 only).
.IP 8
Exclusion error.  This means 
.I fetchmail
either found another copy of itself already running, or failed in such
a way that it isn't sure whether another copy is running.
.IP 9
The 
.I fetchmail.
run failed while trying to do an SMTP port open or transaction.
.IP 10
Something totally undefined occured.  This is usually caused by a bug within
.I fetchmail.
Do let me know if this happens.
.PP
When
.I fetchmail
queries more than one host, the returned status is that of the last
host queried.
.SH AUTHOR
.I fetchmail
was originated (under the name `popclient') by Carl Harris at Virginia
Polytechnic Institute and State University (a.k.a. Virginia Tech).
Version 3.0 of popclient was extensively rewritten and improved by
Eric S. Raymond <esr@snark.thyrsus.com>. The program's name was
then changed to
.I fetchmail
to reflect both the presence of IMAP support and the symmetry with sendmail
created by the new SMTP forwarding default.
.PP
.SH FILES
.TP 5
~/.fetchmailrc
default run control file
.TP 5
${TMPDIR}/fetchmail-${HOST}-${USER}
lock file to help prevent concurrent runs.
.SH ENVIRONMENT
For correct initialization, 
.I fetchmail
requires either that both the USER and HOME environment variables are
correctly set, or that \fBgetpwuid\fR(3) be able to retrieve a password
entry from your user ID.
.SH KNOWN PROBLEMS
Use of any of the supported protocols other than APOP requires that
the program send unencrypted passwords over the TCP/IP connection to
the mail server.  This creates a risk that name/password pairs might
be snaffled with a packet sniffer or more sophisticated monitoring
software.
.pp
Running more than one concurrent instance of
.I fetchmail
on the same mailbox may cause messages to be lost or remain unfetched.
(This is a design problem of the POP2, POP3 and IMAP2bis protocols.)
.PP
If, using POP3, you find that messages you've already read on the
server are being fetched, blame RFC1725.  That late version pf the
POP3 protocol specification ill-advisedly removed the LAST command, and
some servers (including the one distributed with at least some
versions of SunOS) follow it (you can verify this by invoking
.I fetchmail -v
and watching the response to LAST early in the query).  The fix is to
install an older POP3 server with LAST or switch to an IMAP server.
.PP
The RPOP support is not yet well tested.
.PP
Send comments, bug reports, gripes, and the like to Eric S. Raymond
<esr@thyrsus.com>.
.SH NOTE
This program used to be called `popclient' (the name was changed
because it supports IMAP now and may well support more remote-fetch
protocols such as DMSP in the future).
.PP
The --password option of previous (popclient) versions has been removed -- it
encouraged people to expose passwords in scripts.  Passwords
must now be specified either interactively or in your
.I ~/.fetchmailrc
file.  The short-form -p option now specifies the protocol to use.
.PP
The reason the password isn't stored encrypted is because this doesn't
actually add protection.  Anyone who's acquired permissions to read your
fetchmailrc file will be able to run 
.I fetchmail
as you anyway -- and if it's
your password they're after, they'd be able to use the necessary decoder from
.I fetchmail
itself to get it.  All encryption would do in this context is give a
false sense of security to people who don't think very hard.
.SH SEE ALSO
mail(1), binmail(1), sendmail(8), popd(8), imapd(8)
RFC 937, RFC 1081, RFC 1082, RFC 1225, RFC 1460, RFC 1725.