From: Matthias Andree Date: Wed, 2 Jan 2013 23:11:10 +0000 (+0100) Subject: Attempt merging from 6.3.24. X-Git-Url: http://pileus.org/git/?p=~andy%2Ffetchmail;a=commitdiff_plain;h=98cfcef26048bba06975e68a1aad05a8bac0d65d Attempt merging from 6.3.24. --- 98cfcef26048bba06975e68a1aad05a8bac0d65d diff --cc NEWS index 6c05420c,d78cdcfd..0d801d15 --- a/NEWS +++ b/NEWS @@@ -37,110 -50,9 +37,109 @@@ removed from a 7.0.0 or newer release. * The --bsmtp - mode of operation may be removed in a future release. * Given that OpenSSL is severely underdocumented, and needs license exceptions, fetchmail may switch to a different SSL library. - -* SSLv2 support will be removed from a future fetchmail release. It has been - obsolete for more than a decade. +-------------------------------------------------------------------------------- + +fetchmail-7.0.0 (not yet released): + +NOTE THIS IS AN ALPHA RELEASE THAT HAS NOT BEEN THOROUGHLY TESTED! + +# MAJOR CHANGES +* The UIDL handler code is now much faster, especially noticable with lots of + mail kept on a POP3 server. Where the 6.3.X code was of O(n^2) complexity, + we're down to O(n log n). + Contributed by Rainer Weikusat, MAD Partners Ltd./MSS GmbH. +* The POP3 code now always uses UIDL, except if "fetchall" is in effect. + Fixes BerliOS Bug #16172. Fixes Debian Bug#345788. +* Fetchmail now enables SSL support by default. If this is undesired, + ./configure --without-ssl should help. +* The OpenSSL code now excludes the SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS option. + This can cause interoperability problems with certain buggy servers, but is + required to defang chosen-plaintext attacks against AES. While probably hard + to mount against fetchmail, let's play it safe rather than be sorry later. + +# FEATURES ADDED +* Fetchmail can now retrieve credentials from PWMD. This needs to be enabled at + compile-time and requires run-time configuration. See README.PWMD for details. + Contributed by Ben Kibbey, author of libpwmd and pwmd. +* Fetchmail now supports a retrieve-error command line or rcfile option that + takes exactly one argument, abort (default), continue or markseen. This + specifies the policy used by fetchmail to handle messages whose bodies + fail to be retrieved due to server errors. Both the continue and markseen + options will skip the message with errors and allow the session to + continue so that subsequent messages can be retrieved. The markseen + option will also mark the message with errors as seen. + The default policy is to abort the session whenever a server error occurs. + Contributed by Craig Brown. +* Fetchmailconf offers cram-md5 and apop authentication. + +# REMOVED FEATURES +* IMAP2 protocol support was removed. +* POP2 protocol support was removed. +* RPOP (not actually a protocol, but a variant of POP3) was removed +* POP3: the uidl option has been removed. It is always on. +* POP3: LAST is no longer used. It was removed from POP3 in 1994, and it could + cause mail loss when the connection was interrupted or if clients besides + fetchmail polled the mailbox. +* Trio was removed, fetchmail expects reasonable stdio.h quality levels. +* Support for systems that do not conform to C89 and POSIX 2001 was removed, + this means that BeOS, EMX, NeXTSTEP quirks are no longer worked around. +* The MX and host alias DNS lookups that fetchmail performs in multidrop mode + have been removed. They were based on the mistaken assumption that the + IMAP/POP3 server was also the MX server, which is rarely the case. They have + never supported IPv6 (including IPv6-mapped IPv4) either. + Non-DNS based alias keywords such as "aka" remain. +* Kerberos IV support was removed. +* fetchmail no longer supports SSL v2, nor the corresponding SSL2 option to + --sslproto. SSLv2 is insecure and had been deprecated 15 years ago. fetchmail + will actively forbid SSLv2 negotiation by means of SSL_OP_NO_SSLv2. + To fix Debian Bug#622054. +* A lot of outdated and/or unsafe-to-use material got dropped from contrib/. + +# REGRESSION FIXES +* The mimedecode feature now properly detects multipart/mixed-type matches, so + that quoted-printable-encoded multipart messages can get decoded. + (Regression in 5.0.0 on 1999-03-27, as a side effect of a PGP-mimedecode fix + attributed to Henrik Storner.) + +# BUG FIXES +* The mimedecode feature failed to ship the last line of the body if it was + encoded as quoted-printable and had a MIME soft line break in the very last + line. Reported by Lars Hecking in June 2011. + Bug introduced on 1998-03-20 when the mimedecode support was added by ESR + before release 4.4.1 through code contributed by Henrik Storner. + Workaround for older releases: do not use mimedecode feature. +* Fetchmail now detects singly-quoted % expansions in the mda option and refuses + to deliver for safety reasons. Fixes Debian Bug#347909. +* The Server certificate: message in verbose mode now appears on stdout like the + remainder of the output. Reported by Henry Jensen, to fix Debian Bug #639807. + +# CHANGES +* A foreground fetchmail can now accept a few more options while another copy is + running in the background. +* APOP is no longer a protocol, but an authentication method. In order to use + it, use protocol POP3 auth APOP, or on the commandline, -p pop3 --auth apop. + If no authentication method is specified, APOP is automatically tried if + offered by the server before we resort to sending the password as clear text. + +# KNOWN BUGS AND WORKAROUNDS + (This section floats upwards through the NEWS file so it stays with the + current release information) +* Fetchmail does not handle messages without Message-ID header well + (See sourceforge.net bug #780933) +* BSMTP is mostly untested and errors can cause corrupt output. +* Sun Workshop 6 (SPARC) is known to miscompile the configuration file lexer in + 64-bit mode. Either compile 32-bit code or use GCC to compile 64-bit + fetchmail. Note that fetchmail doesn't take advantage of 64-bit code, + so compiling 32-bit SPARC code should not cause any difficulties. +* Fetchmail does not track pending deletes across crashes. +* The command line interface is sometimes a bit stubborn, for instance, + fetchmail -s doesn't work with a daemon running. +* Linux systems may return duplicates of an IP address in some circumstances if + no or no global IPv6 addresses are configured. + (No workaround. Ubuntu Bug#582585, Novell Bug#606980.) +* Kerberos 5 may be broken, particularly on Heimdal, and provide bogus error + messages. This will not be fixed, because the maintainer has no Kerberos 5 + server to test against. Use GSSAPI. -------------------------------------------------------------------------------- @@@ -244,6 -216,6 +303,7 @@@ fetchmail-6.3.22 (released 2012-08-29, * [vi] Vietnamese, by Trần Ngọc Quân ++ fetchmail-6.3.21 (released 2011-08-21, 26011 LoC): # CRITICAL BUG FIX diff --cc configure.ac index 6780831d,c379ccb7..89396c31 --- a/configure.ac +++ b/configure.ac @@@ -16,13 -16,11 +16,14 @@@ AC_CONFIG_LIBOBJ_DIR([.] AC_CANONICAL_HOST +dnl keep this before stuff that runs the compiler! +AC_USE_SYSTEM_EXTENSIONS + dnl automake options are in Makefile.am -AC_PREREQ(2.60) +AC_PREREQ(2.64) dnl 2.60 required for AC_USE_SYSTEM_EXTENSIONS - AM_INIT_AUTOMAKE(silent-rules) + AM_INIT_AUTOMAKE([silent-rules -Wall]) + AM_SILENT_RULES dnl python is optional # @@@ -694,7 -916,59 +695,7 @@@ the ]) fi]) - AC_CONFIG_FILES([Makefile po/Makefile.in genlsm.sh]) -dnl ,------------------------------------------------------------------ -dnl Check if we need TRIO -needtrio=0 -if test "$FORCE_TRIO" = "yes" ; then - needtrio=1 - ac_cv_func_vsnprintf=no - ac_cv_func_snprintf=no -fi -if test "x$ac_cv_func_snprintf" != "xyes" ; then - AC_DEFINE(snprintf, trio_snprintf, - [Define to trio_snprintf if your system lacks snprintf]) - needtrio=1 -fi -if test "x$ac_cv_func_vsnprintf" != "xyes" ; then - AC_DEFINE(vsnprintf, trio_vsnprintf, - [Define to trio_vsnprintf if your system lacks vsnprintf]) - needtrio=1 -fi -AM_CONDITIONAL(NEED_TRIO, test "$needtrio" = 1) - -dnl TRIO IEEE compiler option for Alpha -dnl -if test "$needtrio" = 1 ; then - AC_MSG_CHECKING(for IEEE compilation options) - AC_CACHE_VAL(ac_cv_ieee_option, [ - AC_TRY_COMPILE(,[ - #if !(defined(__alpha) && (defined(__DECC) || defined(__DECCXX) || (defined(__osf__) && defined(__LANGUAGE_C__))) && (defined(VMS) || defined(__VMS))) - # error "Option needed" - #endif - ],ac_cv_ieee_option="/IEEE_MODE=UNDERFLOW_TO_ZERO/FLOAT=IEEE", - AC_TRY_COMPILE(,[ - #if !(defined(__alpha) && (defined(__DECC) || defined(__DECCXX) || (defined(__osf__) && defined(__LANGUAGE_C__))) && !(defined(VMS) || defined(__VMS)) && !defined(_CFE)) - # error "Option needed" - #endif - ],ac_cv_ieee_option="-ieee", - AC_TRY_COMPILE(,[ - #if !(defined(__alpha) && (defined(__GNUC__) && (defined(__osf__) || defined(__linux__)))) - # error "Option needed" - #endif - ],ac_cv_ieee_option="-mieee", - ac_cv_ieee_option="none" - ) - ) - ) - ]) - AC_MSG_RESULT($ac_cv_ieee_option) - if test $ac_cv_ieee_option != none; then - CFLAGS="${CFLAGS} ${ac_cv_ieee_option}" - fi -fi -dnl ----------------------------------------------------------------' - + AC_CONFIG_FILES([Makefile po/Makefile.in]) AC_OUTPUT dnl Local Variables: diff --cc fetchmail.c index 2498e471,f7bcfd94..b33cf585 --- a/fetchmail.c +++ b/fetchmail.c @@@ -146,282 -151,6 +146,281 @@@ static void printcopyright(FILE *fp) const char *iana_charset; +#ifdef HAVE_LIBPWMD +static void exit_with_pwmd_error(gpg_error_t error) +{ + gpg_err_code_t code = gpg_err_code(error); + + report(stderr, GT_("pwmd: error %i: %s\n"), code, pwmd_strerror(error)); + + if (pwm) { + pwmd_close(pwm); + pwm = NULL; + } + + /* Don't exit if daemonized. There may be other active accounts. */ + if (isatty(1)) + exit(PS_UNDEFINED); +} + +static int do_pwmd_connect(const char *socketname, const char *filename) +{ + static int init; + gpg_error_t rc; + pwmd_socket_t s; + + if (!init) { + pwmd_init(); + init = 1; + } + + if (!pwm || (pwm && socketname && !pwmd_socket) || + (pwm && !socketname && pwmd_socket) || + (pwm && socketname && pwmd_socket && strcmp(socketname, pwmd_socket))) { + if (pwm) + pwmd_close(pwm); + + pwm = pwmd_new("Fetchmail"); + rc = pwmd_connect_url(pwm, socketname); + + if (rc) { + exit_with_pwmd_error(rc); + return 1; + } + } + + if (run.pinentry_timeout > 0) { + rc = pwmd_setopt(pwm, PWMD_OPTION_PINENTRY_TIMEOUT, + run.pinentry_timeout); + + if (rc) { + exit_with_pwmd_error(rc); + return 1; + } + } + + rc = pwmd_socket_type(pwm, &s); + + if (rc) { + exit_with_pwmd_error(rc); + return 1; + } + + if (!pwmd_file || strcmp(filename, pwmd_file)) { + if (s == PWMD_SOCKET_SSH) + /* use a local pinentry since X11 forwarding is broken. */ + rc = pwmd_open2(pwm, filename); + else + rc = pwmd_open(pwm, filename); + + if (rc) { + exit_with_pwmd_error(rc); + return 1; + } + } + + /* May be null to use the default of ~/.pwmd/socket. */ + pwmd_socket = socketname; + pwmd_file = filename; + return 0; +} + +static int get_pwmd_details(const char *pwmd_account, int protocol, + struct query *ctl) +{ + const char *prot = showproto(protocol); + gpg_error_t error; + char *result; + char *tmp = xstrdup(pwmd_account); + int i; + + for (i = 0; tmp[i]; i++) { + if (i && tmp[i] == '^') + tmp[i] = '\t'; + } + + /* + * Get the hostname for this protocol. Element path must be + * account->[protocol]->hostname. + */ + error = pwmd_command(pwm, &result, "GET %s\t%s\thostname", tmp, prot); + + if (error) { + if (gpg_err_code(error) == GPG_ERR_NOT_FOUND) { + report(stderr, GT_("pwmd: %s->%s->hostname: %s\n"), pwmd_account, prot, pwmd_strerror(error)); + pwmd_close(pwm); + pwm = NULL; + + if (isatty(1)) + exit(PS_SYNTAX); + + return 1; + } + else { + exit_with_pwmd_error(error); + return 1; + } + } + + if (ctl->server.pollname != ctl->server.via) + xfree(ctl->server.via); + + ctl->server.via = xstrdup(result); + + if (ctl->server.queryname) + xfree(ctl->server.queryname); + + ctl->server.queryname = xstrdup(ctl->server.via); + + if (ctl->server.truename) + xfree(ctl->server.truename); + + ctl->server.truename = xstrdup(ctl->server.queryname); + pwmd_free(result); + + /* + * Server port. Fetchmail tries standard ports for known services so it + * should be alright if this element isn't found. ctl->server.protocol is + * already set. This sets ctl->server.service. + */ + error = pwmd_command(pwm, &result, "GET %s\t%s\tport", tmp, prot); + + if (error) { + if (gpg_err_code(error) == GPG_ERR_NOT_FOUND) + report(stderr, GT_("pwmd: %s->%s->port: %s\n"), pwmd_account, prot, pwmd_strerror(error)); + else { + exit_with_pwmd_error(error); + return 1; + } + } + else { + if (ctl->server.service) + xfree(ctl->server.service); + + ctl->server.service = xstrdup(result); + pwmd_free(result); + } + + /* + * Get the remote username. Element must be account->username. + */ + error = pwmd_command(pwm, &result, "GET %s\tusername", tmp); + + if (error) { + if (gpg_err_code(error) == GPG_ERR_NOT_FOUND) { + report(stderr, GT_("pwmd: %s->username: %s\n"), pwmd_account, pwmd_strerror(error)); + + if (!isatty(1)) { + pwmd_close(pwm); + pwm = NULL; + return 1; + } + } + else { + exit_with_pwmd_error(error); + return 1; + } + } + else { + if (ctl->remotename) + xfree(ctl->remotename); + + if (ctl->server.esmtp_name) + xfree(ctl->server.esmtp_name); + + ctl->remotename = xstrdup(result); + ctl->server.esmtp_name = xstrdup(result); + pwmd_free(result); + } + + /* + * Get the remote password. Element must be account->password. + */ + error = pwmd_command(pwm, &result, "GET %s\tpassword", tmp); + + if (error) { + if (gpg_err_code(error) == GPG_ERR_NOT_FOUND) { + report(stderr, GT_("pwmd: %s->password: %s\n"), pwmd_account, pwmd_strerror(error)); + + if (!isatty(1)) { + pwmd_close(pwm); + pwm = NULL; + return 1; + } + } + else { + exit_with_pwmd_error(error); + return 1; + } + } + else { + if (ctl->password) + xfree(ctl->password); + + ctl->password= xstrdup(result); + pwmd_free(result); + } + +#ifdef SSL_ENABLE + /* + * If there is a ssl element and set to 1, enable ssl for this account. + * Element path must be account->[protocol]->ssl. + */ + error = pwmd_command(pwm, &result, "GET %s\t%s\tssl", tmp, prot); + + if (error) { + if (gpg_err_code(error) == GPG_ERR_NOT_FOUND) { + report(stderr, GT_("pwmd: %s->%s->ssl: %s\n"), pwmd_account, prot, pwmd_strerror(error)); + + if (!isatty(1)) { + pwmd_close(pwm); + pwm = NULL; + return 1; + } + } + else { + exit_with_pwmd_error(error); + return 1; + } + } + else { + ctl->use_ssl = atoi(result) >= 1 ? FLAG_TRUE : FLAG_FALSE; + pwmd_free(result); + } + + /* + * account->[protocol]->sslfingerprint. + */ + error = pwmd_command(pwm, &result, "GET %s\t%s\tsslfingerprint", tmp, prot); + + if (error) { + if (gpg_err_code(error) == GPG_ERR_NOT_FOUND) { + report(stderr, GT_("pwmd: %s->%s->sslfingerprint: %s\n"), pwmd_account, prot, pwmd_strerror(error)); + + if (!isatty(1)) { + pwmd_close(pwm); + pwm = NULL; + return 1; + } + } + else { + exit_with_pwmd_error(error); + return 1; + } + } + else { + if (ctl->sslfingerprint) + xfree(ctl->sslfingerprint); + + ctl->sslfingerprint = xstrdup(result); + pwmd_free(result); + } +#endif + + xfree(tmp); + return 0; +} +#endif - int main(int argc, char **argv) { int bkgd = FALSE; @@@ -471,7 -198,7 +470,6 @@@ #define IDFILE_NAME ".fetchids" run.idfile = prependdir (IDFILE_NAME, fmhome); - - outlevel = O_NORMAL; /* @@@ -1130,13 -842,6 +1130,12 @@@ } } +#ifdef HAVE_LIBPWMD + if (pwm) { + pwmd_close(pwm); + pwm = NULL; + } +#endif - /* close connections cleanly */ terminate_poll(0); @@@ -1432,27 -1109,6 +1431,26 @@@ static int load_params(int argc, char * fprintf(stderr,GT_("Warning: multiple mentions of host %s in config file\n"),argv[optind]); ctl->active = TRUE; predeclared = TRUE; - +#ifdef HAVE_LIBPWMD + if (ctl->pwmd_file) { + /* + * Cannot get an element path without a service. + */ + if (ctl->server.protocol <= 1) { + report(stderr, GT_("%s configuration invalid, pwmd_file requires a protocol specification\n"), + ctl->server.pollname); + exit(PS_SYNTAX); + } + + fprintf(stderr, "%s(%i): %s\n", __FILE__, __LINE__, __FUNCTION__); + if (do_pwmd_connect(ctl->pwmd_socket, ctl->pwmd_file)) + continue; + + if (get_pwmd_details(ctl->server.pollname, + ctl->server.protocol, ctl)) + continue; + } +#endif } if (!predeclared) @@@ -1463,32 -1119,8 +1461,31 @@@ * call later on. */ ctl = hostalloc((struct query *)NULL); - +#ifdef HAVE_LIBPWMD + if (cmd_opts.pwmd_file) { + /* + * Cannot get an element path without a service. + */ + if (cmd_opts.server.protocol == 0 || cmd_opts.server.protocol == 1) { + report(stderr, GT_("Option --pwmd-file needs a service (-p) parameter.\n")); + exit(PS_SYNTAX); + } + + fprintf(stderr, "%s(%i): %s\n", __FILE__, __LINE__, __FUNCTION__); + if (do_pwmd_connect(cmd_opts.pwmd_socket, cmd_opts.pwmd_file)) + continue; + + if (get_pwmd_details(argv[optind], cmd_opts.server.protocol, + ctl)) + continue; + } + else + ctl->server.via = + ctl->server.pollname = xstrdup(argv[optind]); +#else ctl->server.via = ctl->server.pollname = xstrdup(argv[optind]); +#endif ctl->active = TRUE; ctl->server.lead_server = (struct hostdata *)NULL; } @@@ -1671,6 -1311,15 +1668,7 @@@ if (!ctl->localnames) /* for local delivery via SMTP */ save_str_pair(&ctl->localnames, user, NULL); -#ifndef HAVE_RES_SEARCH - /* can't handle multidrop mailboxes unless we can do DNS lookups */ - if (MULTIDROP(ctl) && ctl->server.dns) - { - ctl->server.dns = FALSE; - report(stderr, GT_("fetchmail: warning: no DNS available to check multidrop fetches from %s\n"), ctl->server.pollname); - } -#endif /* !HAVE_RES_SEARCH */ + /* * can't handle multidrop mailboxes without "envelope" * option, this causes truckloads full of support complaints @@@ -1788,6 -1444,10 +1786,7 @@@ static void terminate_run(int sig if (ctl->password) memset(ctl->password, '\0', strlen(ctl->password)); -#if !defined(HAVE_ATEXIT) - fm_lock_release(); -#endif + if (activecount == 0) exit(PS_NOMAIL); else @@@ -1897,14 -1570,6 +1896,13 @@@ static int query_host(struct query *ctl return(st); } +static int print_id_of(struct uid_db_record *rec, void *unused) +{ + (void)unused; + + printf("\t%s\n", rec->id); + return 0; +} - static void dump_params (struct runctl *runp, struct query *querylist, flag implicit) /* display query parameters in English */ @@@ -2302,8 -1975,16 +2300,9 @@@ printf(GT_(" No UIDs saved from this host.\n")); else { - struct idlist *idp; - int count = 0; - - for (idp = ctl->oldsaved; idp; idp = idp->next) - ++count; + printf(GT_(" %d UIDs saved.\n"), count); - if (outlevel >= O_VERBOSE) - for (idp = ctl->oldsaved; idp; idp = idp->next) - printf("\t%s\n", idp->id); + traverse_uid_db(&ctl->oldsaved, print_id_of, NULL); } } diff --cc fetchmail.man index 8f80db38,a618fffc..aa8836f5 --- a/fetchmail.man +++ b/fetchmail.man @@@ -2167,12 -2204,12 +2167,10 @@@ Legal protocol identifiers for use wit .sp .nf auto (or AUTO) (legacy, to be removed from future release) - - pop2 (or POP2) (legacy, to be removed from future release) pop3 (or POP3) - sdps (or SDPS) + sdps (or SDPS) (a POP3 variant specific to Demon) + kpop (or KPOP) (a Kerberos-based variant) - imap (or IMAP) - apop (or APOP) - kpop (or KPOP) .fi .sp .PP