Site Configuration Keyword Reference for LISTSERV 17.5

DIST_BACKGROUND_CHUNKSIZE

DIST_BACKGROUND_JOBSIZE

Platforms

All HPO

Abstract

Values controlling the defaults for background DISTRIBUTE.  DEFAULT_DIST_BACKGROUND is Boolean, DIST_BACKGROUND_CHUNKSIZE and DIST_BACKGROUND_JOBSIZE are integers.

Examples

Details

A background DISTRIBUTE delivery feature is available for high-volume LISTSERV HPO sites. This feature is meant especially for sites signing messages with DKIM, but also for sites using Embedded Mail Merge (EMM) that regularly send jobs larger than 10K recipients. Such sites may require faster processing in order to accommodate other jobs and interactive requests in a timely manner, but by the same token may not need the power of LISTSERV's dedicated DISTRIBUTE workers.

Background DISTRIBUTE is enabled by default for all LISTSERV Classic HPO sites.  It can be disabled if necessary by setting DEFAULT_DIST_BACKGROUND to 0 in the site configuration file and restarting LISTSERV.

If the default is changed, background DISTRIBUTE can be activated per DISTRIBUTE job with the DISTRIBUTE keyword BACKGROUND=YES.

Conversely, if the default is retained, you can specify BACKGROUND=NO explicitly in a DISTRIBUTE job to override the default. This is particularly recommended for test jobs.

Instead of creating the outbound messages immediately, LISTSERV saves the relevant data in a background queue. Each data file in the queue contains information for a certain number of recipients. The default is 10,000 and can be tuned with the DIST_BACKGROUND_JOBSIZE site configuration variable if required.  However, the default value should be a good balance between not creating too many files, and not creating any huge files that take a long time to open. With this setting, a job with 1M recipients will create only 100 files. Each file is somewhere between 1M and 5M depending on the job.

The DISTRIBUTE command completes as soon as the last data file is written (in much the same way as when using DISTRIBUTE workers). In one test on a desktop PC, 10k recipients took 0.3 sec elapsed time.

Background jobs are processed in sequence, in the order in which they were created, with only one job active at a time. Background delivery pauses immediately if LISTSERV has any other work to do or any other mail to send, although it should be noted that any mail already spooled will continue to be sent. For efficiency, background delivery will not pause until at least x number of messages have been sent. (The number x is set by the site configuration variable DIST_BACKGROUND_CHUNKSIZE, which defaults to 50.)  Progress messages are printed in the LISTSERV log whenever a data file is finished and LISTSERV switches to the next one. 

Background distribution is compatible with delivery pools, and with the SMTP rate limiting feature discussed below.

Default Values

DEFAULT_DIST_BACKGROUND=1

DEFAULT_BACKGROUND_CHUNKSIZE=100

DIST_BACKGROUND_JOBSIZE=1500