Site Configuration Keyword Reference for LISTSERV 17.5

DIST_FORWARD_MINSIZE

DIST_FORWARD_THRESHOLD

Platforms

See details.  Requires Classic HPO.

Abstract

Enables and tunes DISTRIBUTE "workers".  This is a tuning feature for embedded mail merge.

Examples

General:

DIST_FORWARD=smtp.example.com 2*smtp2.example.com
DIST_FORWARD_THRESHOLD=200

or

DIST_FORWARD=smtp.example.com 2*smtp2.example.com
DIST_FORWARD_MINSIZE=250
DIST_FORWARD_THRESHOLD=2

Platform specific:

Details

Configuring many SMTP workers through the SMTP_FORWARD_n site configuration parameter will help speed up the delivery of the many individual mail files generated with the embedded mail-merge feature. However, in very high volume situations, it may also be desirable to off-load the work of generating the many individual mail files to one or more separate servers.

This can be done by configuring additional LISTSERV instances to be “DISTRIBUTE workers”. The primary LISTSERV instance can quickly divide and delegate the mail-merge responsibility among its workers and get on with its own work while the “workers” take care of the brute-force generation of the individual mail messages, which are then delivered by the workers’ SMTP workers. By using several DISTRIBUTE workers, not only does the primary instance’s availability increase, but the deliveries can be forwarded to the SMTP server(s) much faster because multiple messages are being generated in parallel, instead of one at time.

In most cases, each instance will run on a separate, dedicated server, but this is not a requirement. On a very large server with many CPUs and as many independent disk drives, it could make sense to run multiple instances on the same server.

Each instance requires a separate license. Since this feature is intended for processing large volumes, the primary instance must be running with an HPO license. The “worker” instances must have either HPO licenses (in which case they can also act as standalone LISTSERV hosts), or special SCOPE=DISTWORKER licenses (in which case they are dedicated DISTRIBUTE workers).

DISTRIBUTE workers are enabled with either two (14.4) or three (15.0 and later) options set in the primary instance’s site configuration:

1. DIST_FORWARD_THRESHOLD=nnn

This setting is optional, and defaults to 100. It defines the minimum number of total recipients to make use of the workers. It would be inefficient to forward a message with one recipient to a worker so that it could deliver it for you. 

However, when used in conjunction with DIST_FORWARD_MINSIZE, it may be more efficient to set DIST_FORWARD_THRESHOLD to a low value, e.g., 2, which would allow single-recipient test messages to be processed locally (faster).

2. DIST_FORWARD_MINSIZE=nnn

(15.0 or later) This setting is also optional, and defaults to 0, or no limit. It defines the minimum message size, in kilobytes, to make use of the DISTRIBUTE workers. Jobs below that size are processed locally. The message size is defined as the size, counting CRLF terminators, of the contents of the DATA portion of the job.

The purpose of DIST_FORWARD_MINSIZE is to better control bandwidth. All jobs above a certain size can be routed to a separate DISTRIBUTE worker, which in turn might be configured to use a separate mail server with a bandwidth cap. The DISTRIBUTE worker in question can run on older hardware as long as it has plenty of disk space. It is expected to build a large queue when oversize jobs are processed.

3. DIST_FORWARD=[[n*]hostname [[n*]hostname [...]]

This is a space-separated list of all your DISTRIBUTE workers.

For "hostname", you can specify just the hostname if the userid is LISTSERV. If the userid is not LISTSERV, then you must specify a full RFC822 e-mail address pointing to the USERID= address for that server.

Optionally, you can specify a weight (the "n" parameter), which defaults to 1. The number of recipients sent to a particular server is proportional to its weight. The weight must be greater than zero. If a particular server is specified multiple times, the weights are added.

By default, the primary server also takes one share of recipients. In other words, it has a weight of 1. You can change that using the special hostname “SELF”. So, if you want a double share for your primary LISTSERV because it runs on a bigger server, just add 2*SELF to the list. SELF is the only worker allowed to have a weight of zero. In this case, nothing is processed locally (except for jobs that are below the threshold). For very large mail-merge volumes, 0*SELF is recommended.

A worker cannot delegate jobs to sub-workers, but the primary instance can drive any number of workers. 

See also

EMBEDDED_MAIL_MERGE