Description of the changes for release 1.8e of LISTSERV®

1.8d introduces "periodic" change-logs, with the ability to automatically rotate them in a manner similar to that used for list archive notebooks. There are several new keyword settings and configuration variables.

· The existing Change-Log= list header keyword syntax is changed to

* Change-Log= Yes[,Yearly|Monthly|Weekly|Daily|Single]

The change to the keyword parser for this new feature necessitated a change to LISTKWD FILE, which means that unix sites upgrading to LISTSERV Classic 1.8e must be particularly careful to download BOTH the `uname`.tar.Z for their unix platform AND common.tar.Z when obtaining the installation kit. (LISTSERV Lite sites upgrading to LISTSERV Lite 1.8e do not have this problem as everything is shipped in one archive.)

· The new site configuration file variable DEFAULT_CHANGELOG_PERIOD defines the default for above. This in turn will default to SINGLE if unspecified, for compatibility.

· (Classic and Classic HPO only) The existing site configuration file variable SYSTEM_CHANGELOG definition was expanded from Boolean (0 or 1) to allow a period to be specified, e.g. SYSTEM_CHANGELOG=MONTHLY. 0 turns it off as before and 1 maps to SINGLE for compatibility.

· NOLIST-xxx change logs are always SINGLE.

· Rotated logs are renamed to CHANGELOG-yyyy[mm[dd|w]]. GET was changed to recognize these as change logs.

· When the feature is introduced (i.e. when upgrading to 1.8e), old logs will be renamed based on the date in the last entry. If the last entry is compatible with the newly introduced period (e.g. logs are YEARLY and the last entry is 2001), the log will continue, even if there were also entries for 2000. The definition of the 2001 log is that it contains all the entries for 2001, not that it is the first log started in 2001. Logs are not split as the feature is introduced.

· If the rename fails (e.g. because you keep changing the period back and forth), the current log continues.

· While LISTSERV on VM will accept all of the above settings without complaint, everything will behave as if you had specified SINGLE. This is because filetypes are limited to 8 characters on VM and it would be difficult to support a different naming convention just for VM.

Usability: New change-logs start with a SUBCOUNT entry

Please note that this feature is not available under LISTSERV Lite.

List-level change-logs on all platforms now start with a SUBCOUNT entry showing the subscriber count at the time the change-log was started. Note carefully that this change does not affect NOLIST or SYSTEM change-logs, or change-logs already started before 1.8e was applied.

An example to show the format is

20011201111349 SUBCOUNT 115

Usability: Moderation OK requests and MIME message display

In previous versions, an OK confirmation request for a message coming to a moderated list displayed the message to be approved in its "raw" format, i.e., there was no attempt made to display/decode MIME attachments that might be present in the message to be approved. LISTSERV 1.8e addresses the problem by including a copy of the first text/plain part (if one exists in the message) for the purpose of quick screening. The following restrictions apply:

1. This is only done for MIME messages (even simple single-part ones, but they must have MIME headers).

2. The text part in question is sent pretty much 'as is', that is, as an extra text/plain part in the message, with all the options and encoding and what not supplied in the original message. The reason is quite simply that it would be a lot of work and, in some extreme cases (incompatible code page, etc), completely impossible, to embed it into the first text/plain part with the LISTSERV message. The drawback is that some mail agents might conceivably only show the first part until you take some kind of clicking action.

It is important to understand that only the first text/plain part is extracted in this fashion. The goal was to make it easier to approve or reject simple text messages, not to build a factory around a simple problem. The ENTIRE message is available at an extra click.

Where security is a concern, it is important to review the ENTIRE original message and not just the plain text part. There could be an obscene GIF or another text part or a text/html part not matching the contents of the text/plain part or whatever. This is why, again, you are given the ENTIRE original message.

List owners using certain email clients (specifically Pine, which handles attachments in a secondary viewing area) may find the new format difficult to use. If preferred, the pre-1.8e behavior may be reverted to by specifying "NOMIME" in the Send= list header keyword; for instance,

* Send= Editor,Hold,NoMIME

Usability: GETPOST and MIME message display

In previous versions, the GETPOST command returned messages that contained MIME attachments in their "raw" form, which could not be extracted automatically by MIME-aware mail clients. Customers who wished to use list notebooks to archive word-processing documents (for instance) found this to be a problem. In LISTSERV 1.8e, attachments returned in messages by way of the GETPOST command will now display as inline clickable links in the individual messages.

Users of certain email clients (specifically Pine, which handles attachments in a secondary viewing area) may find the new format difficult to use. If preferred, the pre-1.8e behavior may be reverted to by specifying "NOMIME" as the last parameter of the GETPOST command, for instance,

GETPOST MYLIST-L 2893-2910 2912 NOMIME

Usability: New JOB card keyword AFTER=

LISTSERV 1.8e introduces the ability to schedule a JOB to run after a specified time, implemented through a new JOB card keyword: AFTER=. There are three valid formats:

1. AFTER=date

2. AFTER=time

3. AFTER="date time"

The first two formats may also be quoted if desired. The third must be quoted since it contains a space. In case 1, the time is 00:00:00; in case 2, the date is today. The date should be specified as yyyy-mm-dd, but other formats may also work, without guarantees. The time is hh:mm[:ss].

If the requested date is in the past, the job is executed immediately. Otherwise, it is placed on hold for execution at the exact specified time. If several jobs are submitted for the same execution time, the order in which they are executed is unpredictable. Generally, this would be a bad practice anyway since there is no guarantee that jobs arrive in the same order as you sent them. If on the other hand you schedule your jobs for one second after the previous one in the desired sequence, execution order will be respected, even if the execution of the first job causes the others to be simultaneously eligible for execution.

Usability: OK confirmation requests now sent with non-null RFC821 MAIL FROM:

In order to work around breakage caused by non-compliant mail servers that block or discard incoming mail with RFC821 "MAIL FROM:<>", OK confirmation request mails now contain "MAIL FROM:<OWNER-LISTSERV@node>".

Usability: New &*URLENCODE(); mail-merge function

Please note that this feature is not available under LISTSERV Lite.

This new LISTSERV 1.8e feature encodes the passed value so that it is valid in a URL (replacing special characters with "%" followed by two hexadecimal digits forming the hexadecimal value of the character). For instance, this works for substitutions such as &*URLENCODE(&ID;); where &ID; represents a field extracted from a database.

Usability: DISTRIBUTE MAIL-MERGE can perform quoted-printable encoding of substitutions on the fly

Please note that this feature is not available under LISTSERV Lite.

DISTRIBUTE MAIL-MERGE has been enhanced to perform QP encoding of substitutions on the fly, as described below. This behavior can be disabled with the new QPAUTO=NO option to the DISTRIBUTE command. You can also specify QPAUTO=YES, but this is the default. The main reason you would want to use QPAUTO=NO is if you had previously kludged QP encoding into your substitution data in order to arrive as the desired result, and you do not want to change your data or scripts. This kind of kludge also tends to have better performance as it makes assumptions that LISTSERV cannot make in the general case. There is nothing wrong with such kludges as long as you know what you are doing. You cannot enable QPAUTO=NO for a list, so kludged bottom banners will need to be dekludged manually.

On the fly encoding is performed by first analyzing the MIME structure of the message, and then replacing &XYZ substitutions with &LSVQP_XYZ if the substitution occurs in a QP attachment. At the same time, a flag is set to generate the LSVQP_XYZ variable with the contents of XYZ, QP encoded. If the MIME structure is in error, you will get QPAUTO=NO behavior but the message will still be distributed (unless some other option blocks it).

Usability: Commands: LISTS OWNED BY, LISTS MODERATED BY

LISTSERV 1.8d was released with an undocumented feature for the LISTS command: LISTS OWNED, which returned a list of local lists owned by the invoker (the feature was originally added as part of the web administration interface and was not intended to be used otherwise). LISTSERV 1.8e adds functionality to this now-documented feature for site maintainers, who can now use it to determine which lists are owned by a given e-mail address. The syntax is

LISTs OWNed [[BY] [internet-address]]

A similar command structure to return a list of local lists that are moderated has also been added in 1.8e:

LISTs MODerated [[BY] [internet-address]]

In both commands:

· The BY token is optional.

· Wildcards are accepted.

· If the address is different from your own, you must be postmaster.

· Postmaster privileges of 'internet-address' are ignored. Thus, LISTS OWNED BY self from a postmaster only shows the lists actually owned by that postmaster, as opposed to showing all the lists on the server (which was the behavior under 1.8d, since postmasters are considered list owners of all lists on the server, even if they do not appear explicitly in an Owner= keyword setting). There is little point in doing LISTS OWNED BY another_postmaster if postmaster privileges are honored. To issue this command you need to be postmaster yourself, and you would then get the same results as with the old LISTS OWNED.

· Execution time will be long if you have lists with "Owner= (biglist)". The only way to avoid this would be to disallow wildcards. HPO could have a dual-path implementation in a future version if this turns out to be a problem.

Usability: Revamped web administration interface

At release time LISTSERV 1.8e will have a completely rewritten and revamped web administration interface. Two significant additions are a new web-based moderation interface and new extensible LISTSERV "wizards".

Documentation: A supplemental guide to the new web interface will be produced after product launch, with a target date of 3Q2002.

Usability: (unix) DB2 now supported as a DBMS datastore

Please note that this feature is not available under LISTSERV Lite.

LISTSERV 1.8e introduces CLI support for DB2 on AIX and other unix platforms on which DB2 is supported.

In order to enable this support, a library flag must be specified at compile/install time. The correct library flag on AIX is -ldb2 , and this is specified in the LISTSERV Makefile in the FLAGS_AIX= macro that resides below the end of the normal "user customization area". By default the macro is blank, so when set for DB2, it will look like this:

CFLAGS_AIX=-ldb2

Other unixes are configured similarly.

It is important to note that LISTSERV's DB2 support will work only if you have carefully followed the DB2 installation guide and performed the post-configuration tasks, in particular the task that creates links to /usr/lib for the various libraries. If this is not done, DB2 will use default directories with version-specific names to find the library, which L-Soft does not and will not support. Should this problem arise you must simply create the links as stated in the DB2 manual; if further information is necessary you must contact IBM for DB2 support.

LISTSERV configuration for the use of the CLI driver is very similar to configuration for use of the ODBC driver. There are three site configuration variables:

CLI_DSN=

CLI_UID=

CLI_AUTH=

which are configured in a similar manner to their ODBC counterparts.

Usability: Multiple simultaneous database connections now supported

Please note that this feature is not available under LISTSERV Lite.

The release version of LISTSERV 1.8d allowed only one database connection to be defined at a time (although that one connection could serve multiple tables, as long as they were contained in the same database). This was inconvenient for some sites which wished to integrate user data kept in multiple databases. Under Windows NT/2000, it was possible under 1.8d to install a supplemental kit that had OCI support and sites could then define both an ODBC and an OCI connection at the same time. This did not prove to be efficient, and during 1.8e development functionality was added to allow multiple simultaneous database connections (and native OCI support for Windows servers was dropped as of 1.8e as noted farther down in this document).

Alternate data sources are specified as follows:

In a List Header:

* DBMS= Yes,...,SERVER(server_alias)

In an ad-hoc DISTRIBUTE job:

DISTRIBUTE MAIL-MERGE DBMS=YES(EMAIL=EMAILADDR,...,SERVER=server_alias)

The SERVER specification is optional; it defaults to SERVER(DEFAULT), which for backward compatibility is identical with the server defined in ODBC_DSN= in the site configuration file.

When defining an alternate server, it is recommended that server_alias be defined as an alpha string as opposed to a numeric string, in order to avoid certain problems in some ODBC drivers. There is no character limit for the server_alias string, but for simplicity's sake it should not be overly long.

If you plan to define multiple database sources, it is strongly recommended that the default values ODBC_DSN=, ODBC_UID=, and ODBC_AUTH= be set deliberately to a known non-existing DSN, regardless of whether or not you are using ODBC. This forces ALL jobs to explicitly declare which server_alias they will use. Jobs which leave this out will thus always fail, but there is no risk of them producing incorrect, potentially embarrassing results by using the wrong DBMS in any real production job.

Alternate ODBC data sources are then defined with additional parameters of the form

ODBC_DSN_server_alias=

ODBC_UID_server_alias=

ODBC_AUTH_server_alias=

For OCI, again you first define your default OCI connection -- and again, it is strongly recommended to set the default OCI_CONNECT=, OCI_UID=, and OCI_PWD= variables to point a known non-existing database -- then alternate OCI data sources are defined with additional parameters of the form

OCI_CONNECT_server_alias=

OCI_UID_server_alias=

OCI_PWD_server_alias=

Similar to ODBC, the default data source for OCI is OCI_CONNECT=, not OCI_CONNECT_DEFAULT=, in order to be backwards compatible with 1.8d syntax.

And finally, for CLI, you first define your default CLI connection -- and again, it is strongly recommended to set the default CLI_DSN=, CLI_UID=, and CLI_AUTH= variables to point a known non-existing DSN -- then alternate CLI data sources are defined with additional parameters of the form

CLI_DSN_server_alias=

CLI_UID_server_alias=

CLI_AUTH_server_alias=

A default data source MUST always be defined; otherwise the driver will disable itself. As indicated above, you do not have to use the default data source for anything and it does not even need to be a valid login, but it needs to be there as an indicator that you have configured ODBC/OCI/CLI and want it to be activated. The driver itself does not know which data source(s) will be accessed until it is called for the first DISTRIBUTE job or list posting after startup.

Please note carefully that not all database types are supported across all platforms. For instance, CLI is not currently supported natively under the Windows NT/2000 version of LISTSERV 1.8e, but note that CLI databases can be accessed via ODBC if need arises.

Usability: TCPGUI access to change-logs

Please note that this feature is not available under LISTSERV Lite.

It is now possible to access change-logs via the TCPGUI through the addition of two options to the non-VM GET command, MSG and COLumns(...). Both are designed primarily for GUI use, and although they will be documented, there was no attempt at making the syntax intuitive to end users, as they are not expected to use this feature. The design goal was a syntax that is easy to generate and parse.

The MSG option (named for compatibility with the REVIEW option by the same name) simply tells LISTSERV to return the contents of the file as "interactive messages," which for TCPGUI will mean that the data will be returned as command output. Thus the option is effective only with text files, as this is not a binary channel.

The COLumns(...) option tells LISTSERV to filter records from the file before sending it. The syntax is the following:

COLumns(colspec1 colfilter1 [colspec2 colfilter2[...]])

'colspec' defines the column to which the filter is to be applied. It can be either a-b (positions a to b, inclusive), a- (a through end of line), a.b (positions a to a+b-1) or Wa (blank-delimited word number a).

'colfilter' is either a-b, a-, -b or a. The latter is equivalent to a-a, whereas the two incomplete forms only check one of the bounds. There must not be any spaces surrounding the hyphen in a-b, and neither a nor b may contain spaces or parentheses.

It is possible to specify as many different columns as desired. Each column must match its respective filter in order for the record to be selected.

It is also possible to specify the same column multiple times (for this purpose, equivalent a-b and a.b specifications are considered the same column). This provides multiple acceptable ranges for the column in

question.

It is important to understand how exactly comparisons take place:

All operands are treated as case-insensitive strings, even if they happen to be valid numbers. Thus, 9.00 is greater than 200.

Comparisons proceed from left to right and stop at the first differing character. There is no concept of column alignment; in particular, leading blanks are significant (this is not applicable if you defined the column as Wa).

If the reference is shorter than the column, the comparison ends successfully when the last character in the reference has been compared. For instance, if you search a changelog for W1 200101-200102, only the first 6 characters will effectively be searched.

If the column is shorter than the reference, it is considered to be padded with blanks. This usually fails the record.

Another way to describe it would be to say that the reference, which is well-defined, takes precedence over the record column, which could vary widely. If the reference is too short, the remainder of the column is ignored. If the column is too short, it will usually not match.

Comparisons succeed for their bounds, i.e. 2000-3000 will select both 2000 and 3000.

Comparisons that are bound to fail will fail as expected. If you look for a string of 4 characters in a column that is defined to be only 3 wide, you will find no match and there will be no error.

If there are no matches, the file is not sent and the message "No matching records" is sent instead as command response. This message is suppressed when the MSG option is used; you simply get an empty response.

To select ADD or DELETE records in 2001 in a changelog, you could use COL(1-4 2001 W2 ADD W2 DELETE).

Note that this feature works with any file, not just change logs. In particular, it works with archived change logs.

At the time this was written (2002/01/04, prior to the public beta), there was a 64KB limitation on the amount of data returned via the web interface, which can be circumvented by sending the command via e-mail or by limiting your search period, and which may be corrected in the final release of the 'wa' CGI stage for LISTSERV 1.8e.

Example: To show all POST operations for MYLIST-L since 1 June 2001,

GET MYLIST-L CHANGELOG (MSG COLUMNS(W1 20010601- W2 POST)

Usability: Change-Log information can be mirrored to DBMS table

Please note that this feature is not available under LISTSERV Lite.

LISTSERV 1.8e allows a copy of change logs to be stored in a DBMS. This requires a pre-existing DBMS connection configured in the usual way (see the Developer's Guide for LISTSERV, chapter 4, if you need guidance), and is controlled by three new site configuration parameters, and a table that must be created manually. The parameters, which are defined in LISTSERV's site configuration file, are CHANGELOG_DBMS, CHANGELOG_DBMS_TABLE, and CHANGELOG_DBMS_CONNECTION . The first two are mandatory while the third is optional.

Note: It is not possible for LISTSERV to write the changelog in multiple different tables based on various combinations of parameters. This can be accomplished on the DBMS side with a stored procedure if required.

Important: The DBMS copy is just that, a copy. The disk files are still generated. Naturally if they are not needed they may periodically be erased with a script.

CHANGELOG_DBMS (mandatory)

This controls which entries are to be copied to the DBMS. Note that only entries that are actually generated can be copied. If a given change-log is disabled, it will not go to the DBMS even if you request it in this variable.

The value can be ALL or a space-separated combination of SYSTEM, NOLIST (matches any NOLIST-xxx), LISTS (matches any list) or the names of individual lists.

CHANGELOG_DBMS_TABLE (mandatory)

This contains five space-separated names:

1. The name of the table in which to store changelog entries.

2. The name of a time-stamp column in which to write the current date and time. This must be a DATE (Oracle), TIMESTAMP (DB2), DATETIME (SQL Server), or whatever else can store both date and time. This cannot be a character string.

3. The name of a VARCHAR or equivalent column storing the name of the list. The maximum size depends on the list names you choose, but it should be at least 40.

4. The name of a VARCHAR column storing the record type (BOUNCE, etc). This should probably be around 40.

5. The name of a VARCHAR column storing the parameters. This ought to be 256 or so.

If any of these parameters is missing, the setting is ignored.

CHANGELOG_DBMS_CONNECTION (optional)

This contains two optional space separated words:

1. The type of driver to be used (CLI, OCI, ODBC). This defaults to your system default driver type.

2. The server to connect to (similar to SERVER=). This defaults to the empty string ie the default server.

Usability: Content filtering

Please note that this feature is not available under LISTSERV Lite.

This feature is primarily intended to filter out-of-office messages and the like. It is not intended as a profanity filter. Attempts to configure it to filter profanity will most likely prove to be futile in the long run and are not recommended by L-Soft.

The new CONTENT_FILTER mail template form, if present, contains filtering rules, one rule per line, empty lines ignored. Each rule has the following format:

[prefix:] pattern

The prefix, if present, can be a mail header tag (e.g. "Subject:"); "Header:" to check the whole header; or "Text:" to search the message text. The latter is the default if no prefix is supplied, it is provided in case the pattern contains a colon in the first word. If there are multiple mail header tags with the specified name (e.g. "Received:"), each such tag is searched and it is enough for one of them to match the pattern. If the requested tag is not present in the header, there is (surprise!) no match. A text search will search every line of the first text/plain part in the message. If there is no text/plain part, there is no match. Again, this is designed to filter read receipts, loops, chain letters, spam, you name it. This is NOT a profanity filter and future versions will not be "enhanced" to make futile attempts at decoding Word documents to look for obscene words.

Regular comparisons such as those described above are not case sensitive. Patterns are standard LISTSERV patterns, ie the asterisk is the wildcard character. If there is no asterisk in the pattern, it is replaced with "*pattern*" much like the SCAN command.

The content filter also supports "exact match" comparisons, which are triggered by a double colon. For instance:

Subject::

There are two significant differences between exact and regular match:

a. You must supply your own wildcard characters in an exact match (if you want to use wildcards, that is). A regular match will insert leading and trailing wildcards if none are found. Thus, an exact match is the only way to make a comparison without wildcards.

b. You can make an exact match for the empty string. Empty regular matches are ignored since they map to a wildcard comparison for **, which would be always true. This also makes it possible to apply an exact match to a message that does not contain a specified header. For instance, if you want all messages to contain a (mythical) KABOOM: RFC822 header, with an exact match you can tell LISTSERV to perform one of the content-filtering actions if the the header is not present. This is not possible with a regular match.

Note however that you cannot differentiate a header with an empty KABOOM field from a header with no KABOOM field.

One of the most handy uses for the exact match syntax is to be able to write a rule to reject messages with blank subject lines. For instance:

Subject::

Action: REJECT Please resubmit your message with a non-blank subject.

Every rule can, optionally, be followed by an action rule. This has the following format:

Action: ALLOW

Action: REJECT reason

Action: DISCARD comment

Action: MODERATE

(The available actions are the same for both regular and exact comparisons.) For instance,

>>> CONTENT_FILTER

Subject: Out of office

Action: REJECT OOO messages are not allowed on this list.

Subject: Auto-Generated:

Action: REJECT

Text: Click here to be removed

Action: REJECT Buzz off, spammer.

Subject::

Action: REJECT Please resubmit with a non-blank subject.

Subject: copyright

Action: MODERATE

To: friend@public.com

Action: DISCARD This guy is a spammer

The default is "Action: REJECT" with no specified reason. REJECT means that the message is rejected. MODERATE means that the message is to be forwarded to the list editor to be manually approved or rejected. DISCARD means that the message is to be dropped on the floor without further processing; any text following DISCARD is echoed to the LISTSERV console (and is thus logged).

ALLOW means that the message is allowed and all remaining rules are ignored. This could be used in moderated lists to allow the list moderator to bypass certain filters, for instance:

>>> CONTENT_FILTER

Subject: Out of office

Action: REJECT OOO messages are not allowed on this list.

Approved-By: JOE@EXAMPLE.COM

Action: ALLOW

Text: Click here to be removed

Action: REJECT Buzz off, spammer.

In the example above, messages with Subject: lines containing "Out of office" are rejected. Messages containing the text "Click here to be removed" are also rejected UNLESS they come from joe@example.com .

The text of the rejection is fetched from the BAD_CONTENT mail template form, with the reason supplied as a variable called &COMMENT.

Usability: Sizelim= may be specified in kilobytes or megabytes

The Sizelim= list header keyword has been enhanced in 1.8e to allow list owners to specify a maximum message size in either kilobytes or megabytes, rather than in lines, which was the only way to do this in earlier versions. For instance:

Sizelim= 100K Reject messages over 100Kbytes

Sizelim= 1M Reject messages over 1Mbyte

As before, the limit operates against the entire message file, including all Internet header lines.

Usability: Top and bottom banners now visible in MIME postings

In LISTSERV 1.8d and earlier, the automatic banners placed on messages via the TOP_BANNER and BOTTOM_BANNER mail template forms fell outside of the MIME wrapper on MIME messages, and thus were not displayed in MIME-aware clients (the banner was physically present in the message but could not be seen unless the "raw" message was viewed). When the banner template forms were introduced in version 1.8b, most people were still using text-based mail clients and the banners displayed correctly for the majority of users. As time has gone by, more and more people have started using MIME-aware mail clients and the old behavior is no longer acceptable.

In LISTSERV 1.8e, banners are now displayed correctly in MIME messages.

Usability, Anti-Spoof: "Subscription= By_Owner,Confirm" available

In LISTSERV 1.8e the ",Confirm" option may now be specified with "Subscription= By_Owner" to force a confirmation from the would-be subscriber before the subscription request is forwarded on to the list owner, i.e.,

* Subscription= By_Owner,Confirm

Be careful to note that the underscore in "By_Owner" is REQUIRED in order for LISTSERV to see and parse the ",Confirm" parameter.

While the existing "Subscription= By Owner" syntax continues to work and be supported, it should be noted that this syntax is deprecated, and that attempts to code "Subscription= By Owner,Confirm" in a list header categorically will be evaluated as "Subscription= By Owner". This is because the LISTSERV parser understands a space not followed by a comma to be the end of the keyword setting, and LISTSERV actually parses "Subscription= By Owner" as "Subscription= By".

Security: (Windows NT/2000/XP, Linux; Classic, Classic HPO) Real-Time Anti-Virus Scanning of Messages and Attachments

Please note that this feature is not available under LISTSERV Lite.

Please note that the anti-virus scanning software is available only for Windows NT/2000/XP and Linux, and that this feature is only available for LISTSERV Classic or LISTSERV Classic HPO sites running those operating systems. An L-Soft maintenance contract is also required.

LISTSERV Version 1.8e supports real-time anti-virus scanning of all messages sent to mailing lists that run under LISTSERV Classic or LISTSERV Classic HPO on Windows NT/2000/XP and Linux servers, including inline uuencoded binaries and MIME attachments in those messages. This is a value-added feature which, in addition to a regularly-licensed LISTSERV Classic or LISTSERV Classic-HPO installation, requires the following:

For sites with perpetual ("EXP=NEVER") LAKs: An additional "maintenance" LAK, meaning that you must purchase maintenance (which includes automatic anti-virus signature updates for the term of the LAK) for LISTSERV in order to use the feature. This LAK will come from your sales representative automatically when a perpetual LISTSERV LAK is purchased with maintenance and must be renewed yearly.
For all sites: A separate F-Secure Anti-Virus key that should be sent to you by your sales representative along with your LISTSERV LAK.

F-Secure Anti-Virus must be installed on your LISTSERV machine to enable anti-virus scanning.

There is a new site configuration variable, ANTI_VIRUS=0|1 , which defaults to 1 (enabled) if the supported AV system is detected.

The anti-virus scanning feature includes:

A 45 day warning when maintenance/AV support is about to expire.

A 5 day warning when virus databases have not been updated.

The above warnings are controlled by LICENSE_WARNING=0|1 as usual. And, as usual, it is not recommended to disable the license warnings as you may miss an expiration date without any warning and/or your anti-virus signature databases may not be kept up to date without your knowledge.

The virus checking capability is enabled if (1) the supported AV system is present, (2) a maintenance LAK is loaded and not expired, and (3) ANTI_VIRUS=0 is not specified in the site configuration file. List owners may not turn off AV checking (design decision - security). List postings containing viruses are always returned to the sender (design decision - the sender ought to be warned) even if filtering is otherwise enabled. However, attachments which have been filtered are not scanned for viruses.

When a virus is detected and the message returned to the sender, LISTSERV uses the BAD_ATTACHMENT mail template form to inform the sender. Note that this mail template form has been modified from the original which shipped with late versions of LISTSERV 1.8d. If list owners have modified BAD_ATTACHMENT, they should be notified to update the copy in their list-level template files.

LISTSERV also scans messages sent to the *-request and owner-* pseudo-mailbox addresses (including listserv-request and owner-listserv) and simply discards such messages if they contain viruses.

Additionally, for sites using LISTSERV's system.changelog, a VIRUS job type was introduced to track intercepted viruses. Assuming that (1) the system.changelog is enabled and (2) LISTSERV's anti-virus scanning facility is enabled, LISTSERV will write records like the following to the system.changelog file for each virus encountered:

20020110152955 VIRUS TEST 6 EICAR-Test-File

20020128144949 VIRUS *LSWAVDD* 1 EICAR-Test-File

The record consists of

Date and time the job was processed
The VIRUS job type
The list to which the message containing the virus was posted (if sent to a -request or -owner mailbox, this part of the record contains *LSWAVDD*, as in the second example above)
LISTSERV's best guess of the number of outbound copies that have been suppressed. This is not always 100% correct, but very close. When LISTSERV does not know, the value 1 is assumed. A value of 0 is possible, for instance if the list had no recipients.
The name or description of the virus as provided by F-Secure

Security: (Windows NT/2000/XP, Linux; Classic, Classic HPO) Real-Time Anti-Virus Scanning for DISTRIBUTE jobs

Please note that this feature is not available under LISTSERV Lite.

Anti-virus checking for DISTRIBUTE jobs (including mail-merge jobs) also has been implemented in LISTSERV 1.8e. This feature is available only if the anti-virus software and maintenance LAK have been installed as described in the preceding section, assuming that the anti-virus feature has not been explicitly disabled.

The AV checking for DISTRIBUTE jobs is implemented by appending a new AV=YES parameter to the DISTRIBUTE command, e.g.,

DISTRIBUTE MAIL[-MERGE] ... AV=YES ...

By specifying AV=YES, you direct LISTSERV to check the message for viruses and terminate distribution if ANY errors occur. This includes internal virus scanner errors that do not necessarily indicate the presence of a virus, out of memory errors, invalid base64 data errors, you name it.

While very safe, there is obviously the drawback that critical messages may be suppressed due to problems within the virus scanner. It should be up to the customer to decide what to do in such cases. AV=YES,FORCE tells LISTSERV to stop ONLY if a virus has been found. The drawback is that viruses may be let through if there was a problem with the virus checker (this includes expired maintenance), insufficient memory, or any other problem that would otherwise have caused LISTSERV to terminate the distribution. For obvious reasons, therefore, L-Soft does not recommend the indiscriminate use of the FORCE option.

AV=NO is also available, but does not need to be explicitly coded. This is the default and the only valid option for DISTRIBUTE of an NJE FILE (VM/VMS only) and DISTRIBUTE RFC822 (although it is doubtful that this DISTRIBUTE type is used by anyone anymore).

This option is not forwarded. It is compatible with the backbone, but checking only occurs at the originating site.

IMPORTANT caveat for mail-merge: for performance reasons, virus checking is done only once, not for each recipient after substitution. It is theoretically possible for a virus to escape undetected if it is constructed from a combination of message text and substitution data, especially if the substitution comes from a database.

Another caveat is that certain types of sophisticated mail-merge jobs may incorrectly raise the invalid MIME data error. This could happen if the base64 data were fetched from a database, or if it were included in a .BB/.EB block. Again, all you need to remember is that LISTSERV scans the message template rather than the X million individual substituted messages, so viruses that do not exist in the template will not be detected. The expected usage for correct scanning is something like this:

.BB whatever

--blah boundary

content-type: application/msword; blah blah

content-transfer-encoding: base64

WDVPIVAlQEFQWzRcUFpYNTQoUF4pN0NDKTd9JEVJQ0FSLVNUQU5EQVJELUFO

...

.EB

Security: ALL-REQUEST restrictions

The ability to post to the ALL-REQUEST alias, which expands to all non-quiet list owners, has been restricted in version 1.8e as follows:

1. A new site configuration variable, ALL_REQUEST_ALLOWED_USERS=, can be used to specify who can mail to ALL-REQUEST. This variable uses the same syntax as POSTMASTER=, i.e., a space-separated list of userid@host specifications.

2. Postmasters are always allowed to mail to ALL-REQUEST, even when not listed in ALL_REQUEST_ALLOWED_USERS.

3. The determination is made conclusively on the MAIL FROM: because:

a. This avoids dealing with header errors. Header error = don't know who sent this = discard silently = unhappy admin who had to send an urgent notice to all his owners.

b. The main point of this change is to block spammers, and spammers typically have non-working or null MAIL FROM: addresses. Needless to say, null doesn't pass the test.

4. When the MAIL FROM: is not null, a new template (REQNAK1) is sent when the message is rejected.

Although a further request was considered for a way to disable ALL-REQUEST entirely, it was decided not to implement it, as the main concern was to block spammers and general users from posting to the alias at random.

Security, Anti-Spam: Information added to "Received:" header for incoming -REQUEST mail

The Received: header generated by LISTSERV for mail it receives on listname-REQUEST pseudo-mailboxes now includes the address of the listname-REQUEST mailbox to which it was delivered. This was done to avoid the impression that LISTSERV allows 3rd party relaying when the listname-REQUEST address is BCC’ed.

Security, Anti-Spoof: Original mail headers or originating IP shown on ADDREQ1, CONFIRM1, ADD2, SIGNOFF1

The Subscription=By_Owner message to the list owner (ADDREQ1) and Subscription=Open,Confirm confirmation message to the subscriber (CONFIRM1) now include the full headers or the original request or the IP address of web-originated requests. This change extends also to the ADD2 template form sent to the list owner when someone subscribes to the list and Notify= Yes, and to the SIGNOFF1 template form sent to the list owner when someone unsubscribes and Notify= Yes.

Security: Attachments= changes to block inline UUENCODEd files

The ability to filter UUENCODEd "attachments" on a very basic level has been provided in LISTSERV 1.8e. This function is activated in 2 cases:

1. When a message containing one or more UUENCODEd "attachments" is sent to LISTSERV, which generally does not want to see any attachments in what it processes. In this case the encoded file is simply discarded.

2. When "Attachments= No[,Filter]" or "Attachments= "Yes[,…]" has been specified for a given list and a message containing a UUENCODEd file is posted to that list. In this case the encoded file(s) is/are stripped from the body of the message and the remainder of the message is passed through to the list.

If it is desired to allow UUENCODEd attachments, set "Attachments= All[,...]", where you can optionally specify specific types of attachments to allow as before; e.g., you can say just "Attachments= All" or "Attachments= All,image/jpeg,..."

One important restriction: UUENCODE filtering is strictly on/off. There is no attempt to guess file types. This would be hazardous to begin with as support for these attachments is usually provided on a legacy basis in mail clients, i.e. client A and client B could have a very different opinion on the type of the attachment.

Security: Explicit canceling of "OK" confirmation codes now supported

In LISTSERV 1.8e and following, it is possible to explicitly cancel an OK confirmation code if so desired. The command is simply

OK CANCEL xxxxxxxx

(for instance, "OK CANCEL 8F2E8F4B"), and if the code is valid, LISTSERV will respond "Confirmation code 8F2E8F4B cancelled." If the code is not valid (e.g. has expired, has already been cancelled, or is simply incorrect), LISTSERV will send its standard message telling you in part that "The confirmation code 8F2E8F4B does not correspond to any pending command."

Support Withdrawn: OCI support for Windows NT/2000

Due to the fact that multiple simultaneous ODBC connections are now supported (see above), native OCI support for LISTSERV under Windows NT/2000 (which had been available as a workaround for the absence of multiple connection support for sites running both ODBC and Oracle databases) has been withdrawn in version 1.8e. Oracle databases may still be accessed via ODBC under Windows NT/2000.

Support Withdrawn: OpenVMS VAX no longer supported

LISTSERV 1.8e will not be released for OpenVMS-VAX, due to a dwindling customer base which does not make it feasible to continue development on that platform. OpenVMS-VAX sites should contact their sales representative to discuss migration to another platform if they are interested in upgrading to LISTSERV 1.8e.

Other changes and fixes

A .QU template dot command has been added. This ends processing of the current template as if you had reached the end, but without canceling the message. The main purpose is to avoid multi-level nested .BB/.EB conditional blocks that are hard to keep track of.
In previous versions, Language= NoHTML did not block messages that contained only an HTML part and no text alternative. In LISTSERV 1.8e the behavior of Language= NoHTML is changed to block HTML messages that do not have a plain text alternative.

Recursive MIME analysis/filtering support. LISTSERV 1.8e now runs all messages through its MIME processor if they are sent to a mailing list, even if they are not MIME messages. LISTSERV can also process commands sent with Outlook stationery, etc. The performance cost is minor since in practice user e-mail comes in MIME format today, and this is the only way to ensure that potentially harmful uuencoded files are filtered (see discussion of uuencoded file filtering above).
The recursive attachment support also solves a problem with command messages sent through InterScan VirusWall firewalls which are wrapped in the special InterScan MIME wrapper. Previous versions of the MIME processor were unable to decode such messages and would return a "no command found in message body" message to the originator.

· Three new SHOW CTR counters: (1) number of virus scan operations, (2) number of viruses found, (3) number of copies of infected messages stopped. For each individual virus detected, the last counter increases by the number of outbound copies of the virus that LISTSERV was about to send when the virus was stopped.

· SHOW LICENSE also shows when maintenance key will expire (if applicable).

RELEASE also shows AV database date and AV software version.
Non-MIME HTML messages now display properly in the web archive interface.
Digest= (...),BOTTOM_BANNER no longer suppresses the generation of bottom banners in the individual message parts of a digest. The BOTTOM_BANNER parameter now simply tells LISTSERV to place a copy of the bottom banner at the top of the digest (as in previous versions).
The REPRO subscription option is now honored for users who are members of sub-lists but are not members of the super-list (assuming that they are allowed to post to the super-list).
In addition to the new SUBCOUNT list-level change-log record discussed above, new record types include IMPORT and EXPIRE. IMPORT records, which record the basic results of ADD IMPORT, take the form

20010411165954 IMPORT 2 0 0

The numbers following the IMPORT recordtype stand for (1) recipients added, (2) entries changed, and (3) recipients forwarded to another host, in other words, the same results that are sent back to the ADD IMPORT invoker on completion of the job.

EXPIRE records are written when a subscriber's renewal "grace period" expires (the subscriber having not sent a "CONFIRM listname" command during that period), and the subscriber is deleted from the list. They take the form

20010512060052 EXPIRE joe@EXAMPLE.COM

These recordtypes first appeared in post-1.8d level set releases and have not been previously documented.
A "Precedence: list" header is now generated on LISTSERV list mail. If a "Precedence:" tag is already present, it is kept and no additional "Precedence: list" tag is inserted, except for the types of headers where "Precedence: xxx" would normally be discarded (for instance, subscriptions set to SHORTHDR).

-------------------------------------------------------------------------

LISTSERV is a registered trademark licensed to L-Soft international, Inc.

L-SOFT and LMail are trademarks of L-Soft international.

LSMTP is a trademark of L-Soft international, Inc.

EASE and CataList are service marks of L-Soft international, Inc.

All other trademarks, both marked and not marked, are the property of their respective owners.

-------------------------------------------------------------------------