What’s New for LISTSERV

LISTSERV Classic and Classic HPO, version 15.5, is able to interface to LDAP servers to authenticate user logins and to insert LDAP attributes in mail-merge distributions. For information on LISTSERV/LDAP integration, please see the LISTSERV-LDAP Documentation, which is available separately.

LISTSERV Classic and Classic HPO version 15.5 is able to execute on-demand Dynamic Queries against either LDAP or traditional DBMS servers, and use the results for access control or mail delivery. For instance, an LDAP query against an Active Directory server could be used to grant list owner privileges to all members of a particular Windows Security Group. A DBMS query could be used to combine employee rosters for two departments and send a single copy of an announcement to each unique employee e-mail address. For more information on DQL, please see the PDF Dynamic Query Documentation, which is available separately.

LISTSERV 15.5 now supports (and has enabled by default) password encryption for its SIGNUP files. This means that installing LISTSERV 15.5 will encrypt the passwords in your SIGNUP files, which is not backwardly-compatible with earlier versions of LISTSERV. As noted above, if you are upgrading to 15.5 from an earlier version, it is STRONGLY RECOMMENDED that you make a full backup of your LISTSERV installation before upgrading it.

Password encryption can be disabled by setting the new site configuration variable SIGNUP_ENCRYPT_PASSWORDS (Boolean) to 0. This can be done prior to upgrading if desired.

When SIGNUP_ENCRYPT_PASSWORDS is set to 1, all SIGNUP entries with a password will be irreversibly encrypted. Newly created passwords will be encrypted, and PWC QUERY will still work, but it will not tell you what the password is. The initial encryption process happens as part of the regular SIGNUP file compression. If you have a lot of passwords, the initial compression will take a bit longer to complete.

If you then return to SIGNUP_ENCRYPT_PASSWORDS=0, encrypted passwords will continue to work and PWC QUERY will continue not to show them, while passwords created after disabling encryption will be in plain text, and PWC QUERY will show them to you. There is no way to recover the one-way encrypted passwords. Either the user must use PW REP or the LISTSERV administrator must use PWC REP and change the password if it is lost.

LISTSERV version 15.5 is able to automatically process AOL Feedback Loop reports (these special reports are sent automatically by AOL to organizations that are on AOL’s whitelist). Once configured, LISTSERV automatically parses the reports and implements the actions required by AOL’s whitelist agreement. This helps preserve whitelist status and reduce the number of spam complaints from AOL users.

Use of this feature assumes you already have a feedback loop in place as part of your AOL whitelist agreement and are able to redirect the feedback loop reports to a new address.

These two "Misc-Options=" values have been introduced to replace "Language= EXCHANGE" and "Language= NOHTML", respectively. This makes it possible to set site defaults (in the site configuration variable DEFAULT_MISC_OPTIONS) that can be overridden by the list owner.

* Language= ...,NOHTML,...
Warning: The NOHTML option is deprecated and may be removed in a future version. Please use "Misc-Options= DISCARD_HTML" instead.

* Language= BAD,...
Warning: There is no message template file by that name. This option will have no effect until a message template file is installed by the LISTSERV administrator.

Because LISTSERV Lite does not support the "Misc-Options=" keyword, for the time being LISTSERV Lite sites will have to continue using the deprecated Language= syntax for these two functions.

Note: "Language= HTML" was not changed, as it controls whether or not administrative messages are generated in HTML format, as opposed to determining what content is acceptable for messages sent to the list.

Documented Restriction: Indexing blocks all other LISTSERV commands/functions until the current list is done. Because it is resource-intensive, the REINDEX command is considered a "maintenance" command as opposed to an "everyday" command, and should not be issued in production environments outside of regularly-scheduled maintenance windows.

The main reason to use REINDEX is to retroactively add RSS abstracts (see below) to an existing list. It may also be useful if an archive file is edited by accident or if there is a corrupt file or the like. LISTSERV Classic non-HPO sites with one or more very large and active list(s) may also wish to run this command when upgrading to HPO.

On non-VM systems where there are no web indexes, normally there is no reason at all to rebuild your database indexes, other than in cases of disk corruption or in an upgrade to HPO coupled with the presence of a list with very large archives.

When rebuilding web indexes, BOTH the HTML files AND the INDxxxx files are reindexed. While there may be legitimate reasons for wanting to rebuild your HTML files (template changes, etc), it is NOT recommended to use the REINDEX command to accomplish that, as the overhead involved in rebuilding the INDxxxx files is prohibitive if they do not actually need rebuilding.

With LISTSERV Classic there is no background support, so REINDEX * should be scheduled for "quiet" periods if you have many lists and/or large archives.

While a particular list has its indexes rebuilt, WA browsing accesses will generally fail and/or you will get "listname.html" not found errors. There is no good way to avoid that since the goal is to recreate the entire index structure from scratch, and during that process files will be deleted and not be available until they are rebuilt.

By default, database indexes are reset and will be rebuilt the next time they are needed; this is a fast operation. Under HPO, you can use the IMMEDIATE option to cause an immediate rebuild of all database indexes if absolutely necessary, but remember that the immediate rebuild will block the server from doing anything else until it completes. However, this can be a useful command if you migrate an entire server or perhaps just a single list during off hours and want to pre-index everything so that the server is snappy when traffic increases in the morning.

• SINCE(date): show only entries from and including a specific date. Use parentheses if you want to supply the date in a multi-word format like SINCE(3 Jun 2006), otherwise something like SINCE 2006-01-01 (without the parentheses) is acceptable.

• UNTIL(date): same as above for the other end of the range.

• AFTER/BEFORE: silent aliases for the above.

• RESET: resets all the matching entries. Use with care. QUIET is implied (no notifications are sent).

The old syntax ('SERVE LIST [options]') is still accepted for everything but the RESET option, which requires you to provide a wildcard specification and therefore requires the new syntax.

Most larger LISTSERV installations probably have thousands of SERVE entries that have accumulated over the years as administrators have battled spam. Only a fraction of the addresses so served off are likely to be in use today, but they continue to make SERVE LIST unmanageable, and continuing to maintain them comes at a resource cost. In addition, many spammer addresses were likely served out prior to the availability of the DROP option, which just causes more spam for the LISTSERV administrator to deal with.

While there is no absolute necessity to purge your list of served-off addresses, the new functionality at least provides you with the ability to "SERVE LIST SINCE" recent history, and avoid having thousands of addresses thrown at you when all you really wanted to know was who was served out in the last week.

The existing RSS support in LISTSERV's web archive interface has been improved by the addition of RSS abstracting. Abstracts are available in your RSS feed by default starting with LISTSERV 15.5. However, it should be noted that the web indexes must be recreated to add the abstracts. This is a one-time operation that could take a while on a large site and is better left to be scheduled by the administrator. (See Section 1.2.3 New REINDEX Command for Archives.) An abstract is either generated implicitly from the existing text of the message, or may be specified explicitly.

In plain text messages, an explicit abstract is specified by using <abstract> and </abstract> tags in the body of the message - typically at the very end, but the explicit abstract may in theory appear anywhere in the message. The closing tag is optional but recommended.

For those using HTML-capable mail clients, if the mail client is unable to provide a user-specified plain-text alternative and instead sends a "canned" message to the effect that mail clients that do not support HTML will not be able to read the message, the "canned" message will be the abstract. If the mail client is not capable of providing a plain-text alternative message part at all, and provides HTML only, no abstract will be available.

For those posting messages using the web interface, a dedicated text box for entering explicit abstracts can also be enabled in the message posting interface. List owners can set the RSSINPUT variable to 1 in the list-specific SKIN template. To enable the dedicated text box for all lists, the server administrator should set the RSSINPUT variable to 1 in the site-wide SKIN template.

There are two parameters: a maximum abstract size, and a minimum abstract size. The second parameter is optional, and if left unset, defaults to 50% of the maximum size. The basic syntax is:

LISTSERV stops at the first paragraph boundary after which it has collected at least 'min' words, adding an ellipsis if there is more text (compliant signatures are ignored). If there is an explicit abstract, the min and max parameters are ignored and the abstract is whatever the user entered. If the stop-on-paragraph-end feature is not desired, simply set "min" to the same value as "max".

In all cases, RSS_ABSTRACT_WORDS and/or RSS-Abstract-Words may be changed at will, with the change taking effect "from now on." In order to make the new value take effect retroactively, the indexes must be rebuilt manually. Because of the resource-intensive nature of the REINDEX command, this will NOT happen automatically.

• LISTSERV 15.5 now intercepts Microsoft Exchange "recall" messages and discards them silently (although there is a notification in the log for debugging purposes).

• For sites that continue to use the obsolete static web archive index page (index.html), starting with LISTSERV 15.5, the HTML description of the list as configured in the list header (if present) is saved to index.html immediately following the number of subscribers (on the same line). There is a '<br>' preceding the text. HTML formatting MUST follow the rules as documented for the list's HTML description, which is normally used only for CataList purposes. There is a single exception which applies only to this function (not to CataList): The HTML description has a hard-coded limit of 4K in order to help prevent DoS attacks. If the HTML description of the list is longer than 4K, the HTML description for the list will not appear in index.html.

For sites that use the dynamic archive index page, the HTML descriptions of all lists (if present) will be shown in javascript-powered floating boxes when you move the mouse over the list name. This provides list owners an opportunity to display more information about their lists on the main archive index page than merely the list title. Site administrators who want to disable this functionality can set the SHOWHTMLDESC variable to 0 in the site-wide SKIN template.

• A new site configuration variable, WWW_SHOW_SUBSCRIBER_COUNT (Boolean) has been added to allow site maintainers to suppress the subscriber count shown per list on the LISTSERV Archives page in the web interface. The default is 1, or enabled (show the counts) as before. To disable showing the subscriber counts, set the variable to 0.

• When an address has been administratively (manually) SERVEed OFF, LISTSERV 15.5 will prevent that address from being ADDed to a list. This applies only to non-bulk ADDs; it remains incumbent on the list owner or site administrator to filter such addresses out of the address lists they use for bulk ADD. (As a reminder, under LISTSERV 14.3 and later it is possible to list all addresses that are currently served off by issuing a SERVE LIST command.)

• A Topics pull-down menu is now available when posting messages using the web interface to lists with Topics defined in the list configuration.

• LISTSERV now writes a null return-path (MAIL FROM:<>) in the RFC821 headers of mail forwarded to list owners via the LISTSERV-request and listname-request pseudo-mailboxes. Previously the original return path was unaltered, which could cause SPF-enabled MTAs to reject -request mail on its way out to the postmaster(s) or list owner(s).