It is critically important to test an email job thoroughly before sending it, especially if it is a large job being distributed to many recipients or a message with large amounts of data. Large jobs take up a lot of server time and space and affect many email hosts and individual recipients. Sending out a less than perfect message can create a very negative reaction among recipients and damage an institution’s credibility and reputation. As a safeguard, LISTSERV
Maestro will not allow an email job to be authorized before a test of the job has been sent out and approved. Any changes to contents, recipients, or sender require a new test to make sure that the changes were effective.
To test an email job, click on the Delivery Test icon on the Workflow diagram or the
Test link in the
Test Status section of the Summary diagram.
LISTSERV Maestro allows the user to use a previously entered group of test recipients, upload and download test recipient lists, or type in test recipient names and addresses. The test interface displays a table for entering test data. Each row represents a test recipient. The columns displayed depend on the columns defined in the recipient definition stage of the job.
Valid Internet email addresses are necessary for all the fields in the column that is specified as the email column. The test messages will be sent to these addresses. The same address may be used in more than one row.
The subject line for a test message can be changed. Changing the test subject line will not affect the subject line of the final message. The optional test line allows the user to enter a line of text that will appear above the body of the message when it is delivered. Use this to recognize test messages and different versions of test messages. If nothing is entered into the optional test line box, then no text is added to the test messages. A test line will
never be added to a final email job.
Notes: If the message is an HTML message with a text alternative, then a new option on the Delivery Test screen lets a separate email be sent to each test address with alternative text. If this new option,
Send additional plain text mail with alternative text, is checked, then the test mail will be sent out in two versions. The first version will be the normal HTML mail (including the text alternative) and the second version will be a plain text mail that contains only the alternative text. Each test delivery recipient will receive these two mails, allowing the test recipients to view both the normal HTML mail as well as what the alternative text would look like for a recipient where the email client strips away the HTML part and only displays the text alternative.
When sending a test for a job with Forward-to-a-Friend enabled, several copies of the test will be sent to each address: the normal message, the message as it will look when forwarded without a user-supplied preamble, and the message as it will look when forwarded with a user-supplied preamble. If the
Send additional plain text mail with alternative text option is also checked, then each of these test messages will be sent in both forms. This means that each test address may receive up to 6 different versions of the message.
Tip: Email messages of a large email job will almost certainly pass through a spam filter on their way to the intended recipients. Therefore, we recommend that you check the spam rating of your message from such spam filters, to evaluate if your message may be classified as spam (which would impede the deliverability of your message). For more information, see Section 9.1.1
Checking the Message for Spam.
Click [Send Delivery Test Messages] to continue. Click
[Save & Close] to save the test recipient data but not send the test message. Once the messages have been sent, review the results. It is usually a good idea to include the job owner and any team members in the test recipient list so test results can be viewed firsthand.
If the test results were not acceptable, select the
No option and then click
[OK]. This will prevent the job from being approved for delivery and allow any necessary changes to be made by bringing up the Job Details screen.
If the test results were acceptable, select the Yes option and then click
[OK]. This will allow the job to be authorized for delivery.
If you want to view and verify the delivery test results later, select the Verify Later option and click
[OK]. If this option is selected, the job can not yet be approved for delivery. The next time the
Delivery Test icon or link is clicked from the Job Details screen, the user will be returned to the Delivery Test screen. To approve the pending test, click the
Go directly to the validation page, without another delivery test link
under the
Advanced Options section to return to the Validation screen of the test that was saved for later verification. Click the
Yes option and then click
[OK] to proceed.
Tips: It is useful to have a good understanding of the actual recipient data when putting together a test. If the message contains merged fields, then study those fields in the actual data: what are the shortest and longest instances of those fields? Are those fields ever empty? Next, make sure to send tests that have values of the shortest and longest lengths, and empty fields if appropriate, to make sure that the message layout looks correct in all cases. It is sometimes necessary to change the layout or use conditional blocks to accommodate special cases. If using conditional blocks, make sure that the test cases exercise every possible condition, being especially cautious about the formatting when working with HTML messages.
The recommended method for testing mailings that have the recipients defined from a LISTSERV list and are being sent a standard list message is to create a clone of that list within LISTSERV (such as TEST-listname). Make sure that the entire list configuration is exactly duplicated, including customized templates (especially the top and bottom banners, if these are defined). Subscribe only the test recipients to the test list. On the Delivery Test screen, enter the full address of the test list as the sole test address.
Remember to also test both the From: address and the
Reply-To: address (if the latter is specified). To test the
From: address and the
Reply-To: addresses try replying to both from the test message and examine the results.
Important: Spam checking your email jobs is an advanced feature. In order to get the L-Soft-provided SpamAssassin check, you must be running LISTSERV 16.0 and you must have a current maintenance.
Spam checking your email forwards the currently defined email content, augmented with the sender settings such as the From: address and name, to the spam filter that is configured at your LISTSERV host. The spam filter creates a report that includes a decision whether or not the message is classified as spam. The report contains valuable information about what aspects of the message might cause it to be classified as spam.
Email messages of a large email job will almost certainly pass through a spam filter on their way to the intended recipients. Therefore, we recommend that you check the spam rating of your message from such spam filters, to evaluate if your message may be classified as spam (which would impede the deliverability of your message).
For a message that has already been checked, the spam classification (classified as spam or not) will be displayed. Click
View results to view the detailed results of this spam check, or click
Repeat spam check to initiate a new spam check.
Notes: The exact contents of the spam report and, most importantly, the performed spam tests and how they are rated, depend strongly on the configuration of the spam filters that are being used. If the report classifies the message as spam and this decision appears to be in error, please contact the administrator to review the configuration of the spam filters.
In addition, the spam rating given by your spam filters can only be used as a general guideline when trying to evaluate if your message may be classified as spam. The actual spam filters of your recipients may be configured differently than your spam filters which are used by LISTSERV Maestro to determine this rating. Therefore, a message that passes your tests may still be classified as spam by a recipient's filter, or vice versa. For additional details, see the online help.
Click [OK] to return to the Delivery Test screen. Click
[Download Spam Check Results] to review the settings of the mailing before returning to the Delivery Test screen.
Important: The exact contents of the spam report and, most importantly, the performed spam tests and the assigned penalty points depend strongly on the configuration of the spam filter that is being used. If the report classifies the message as spam and this decision appears to be in error, please contact the administrator to review the configuration of the spam filter.
Once a test message containing tracked data has been delivered and the test recipients have opened the message and clicked on tracked links, then the Test Reports tab, available from the Job Details screen, becomes active. Click on the tab to open up a definition page to generate a “quick report” on the test message data. Quick Reports are predefined report types that create a temporary report based on the selected report settings and the active email job.
|
•
|
Event distribution over time – Produces a simple line graph showing the number of responses over time. Time is plotted on the x-axis of the chart and events are charted on the y-axis.
|
|
•
|
Sum of events – Produces a bar graph showing the number of recipients who clicked on each URL and/or opened up the message. The length of each bar shows the number of events.
|
|
•
|
Two types can be chosen for the report – Sum of unique events or Sum of all events. Sum of unique events records only a single event of each type for each recipient. This chart will register “0” (zero) events if blind tracking has been selected for the tracking type during the tracking definition because blind tracking cannot count any event as unique. Sum of all events counts all events triggered for each recipient.
|
|
•
|
Event details – Displays a table that lists the report type, lists the time period of the report, allows for the selection of encoding for downloading the data, and allows for the selection of the time zone to reference the downloaded data. Click the download link to download the raw data to a local drive. Raw data can be used in Microsoft ® Excel or third party statistical analysis software to create detailed reports. Event detail reports are available for personal, anonymous, unique, and blind tracking jobs.
|
Next, select the source data for the report type. Each data source will create a line or bar on the chart. There are up to five choices depending on the type of message (Plain Text or HTML), as well as the tracking events defined in the message:
|
•
|
Open-Up events – This option charts all the open-up events for a job. It is only available for HTML messages.
|
|
•
|
Click-Through events (all links bundled) – This option bundles all click-through events together into one variable charted on the report. For example, if three links were being tracked in a message, they would appear as one line or bar on the chart.
|
|
•
|
Click-Through events (all links separate) – This option creates separate variables for each click-through event that is tracked. For example, if three links were being tracked in a message, each of the three links would appear as a separate line or bar on the chart.
|
Other data sources can be added to the chart by selecting any or all of the Additional Statistics option buttons. These additional data sources can provide comparison data in the chart so that it is possible to see the total number of messages sent in relation to the number of open-up events that occurred, for example. Three
Additional Statistics options are available:
|
•
|
Number of Sent Messages – Adds the number of messages that were sent to LISTSERV as a constant reference value in the report.
|
|
•
|
Number of Unbounced Messages – Adds the number of unbounced messages as a constant reference in the report. This value shows the number of messages that were sent and not detected as bounces by LISTSERV. This option is only available if LISTSERV handles bounces automatically (selected during the sender definition).
|
|
•
|
Number of Bounced Messages – Adds the number of bounced messages as a constant value in the report. This value shows the number of messages that were sent but were detected as bounces by LISTSERV. This value can go up over time as new bounces are received by LISTSERV. This option is only available if LISTSERV handles bounces automatically (selected during the sender definition).
|
Make your selections, and then click [OK] to continue. A temporary report with the settings selected will be generated and displayed on the screen. This report will
not be saved (it can always be run again from the Test Reports tab).
Located at the bottom right side of the test report are four icons. Each icon opens a page with different functions and options. The currently selected icon will be highlighted with a blue border. Unavailable icons will appear grayed out. These icons appear on all test reports, quick reports, and regular report pages.
In order for an email job to be delivered, final authorization is required. After all previous requirements have been met – define recipients, define message, define tracking, define sender, delivery test, and delivery settings – a job may be authorized for delivery. The job owner always has permission to authorize a job, as do any other team members that have been granted permission.
To authorize an email job delivery, click on the Authorize Delivery icon on the Workflow Diagram or click the
[Authorize Delivery] button at the top of the Summary Diagram. The Authorize Delivery screen opens.
The Job Summary section lists all the selections that make up the email job. Above the [Authorize Delivery Now] and
[Cancel] buttons is a message that summarizes the selections made for the delivery. If this is an auto-repeat job, the auto-repeat sequence time delay will be listed as well as other notes on the screen. Normal jobs will have a shorter message summarizing the delivery schedule.
The Details link in the
Content section will display a copy of the message, alternative text, attachments, and tracking information. The message can be downloaded to a local file as well. The
Details link in the
Recipients section displays information regarding the recipients for the email job.
The Sender section lists all the selection you made while defining the sender’s information. If DomainKey signing is enabled, then LISTSERV Maestro interacts with LISTSERV to determine if the supplied sender address is supported by one of the DomainKeys that were deployed to the LISTSERV host when DomainKeys was configured. If the email job is configured for future delivery, then there is a considerable time window during which the administrator may opt to change the DomainKeys settings at the LISTSERV host. Therefore, if DomainKeys have been disabled during this time window, then the email job delivery will fail with an appropriate error message.
Clicking [Authorize Delivery Now] will do exactly that, and the job will be sent immediately or at the scheduled day and time. Clicking
[Cancel] will go back to the Job Details screen so that any part of the job may be edited, or the job can be deleted.
If a job is authorized for delivery at a future date and time, delivery authorization may be revoked by going to the Open Jobs screen and selecting the job. For more information, see Section 10
Open Jobs.
The scheduled delivery time shown on this screen is always displayed relative to the local time of the user. If a job is scheduled for a time zone different from the user’s local time, then the time displayed in the Job Summary may appear incorrect. For example, if the user enters the time 10:00 with time zone GMT as the scheduled delivery time, but the user’s location is not in the GMT time zone, the delivery time displayed on the Authorize Delivery screen will be different from the 10:00 time selected. This is because the user’s local time is in a different time zone than the scheduled delivery time. When converted to absolute time, the delivery time will be correct. Time zone differences between the local time and the delivery time may even show an “incorrect” day. If the user sets the scheduled delivery time to 0:30 at GMT on the first of January, the time will appear as 23:30 on the 31
st of December on this screen if the local time zone is one hour behind GMT.
Once the A/B-Split Job’s information is completed (green icons and checkmarks will let you know when everything is completed), and you are ready to authorize delivery, then click the
Mail Job menu and select
Authorize Delivery.
The Authorize Delivery screen opens, presenting you with a final overview of the job. Verify that everything is correct, and then click the
[Authorize Delivery Now] button.
Once the information for the A/B-Split Job with Sampling is completed (green icons and checkmarks will let you know when everything is completed), and you are ready to authorize delivery, then click on the Sampling Variant Jobs tab, click the
Mail Job menu, and select
Authorize Sampling Variant Delivery.
The Authorize Delivery screen opens, presenting you with a final overview of the job. Verify that everything is correct, and then click the
[Authorize Delivery Now] button. The sampling variant jobs are authorized and delivered first, while the main variant job is held back. You can then use the tracking results of the sampling variants to evaluate the "success" of each variant (just like a normal A/B-split job). With these results, you can copy the content of the most successful sampling variant job into the main variant. After this is done, you can authorize the delivery of the main variant.
In some cases, it may be desired that the moment for an email job delivery be determined outside of LISTSERV Maestro. For example, if the message content is defined as a drop-in file that is generated by an external process, then the job should not be delivered until this file has been successfully completed. Because of circumstances like this, you now have the ability to schedule the email job’s delivery for a future time that will be determined after the email job has been approved.
To manually trigger the delivery of an email job, go to the Ongoing Jobs screen and click on the job’s
ID. This opens the Ongoing Job Details screen. From here, click on the
Mail Job menu and select
Trigger Job Delivery. Click
[OK] to confirm the delivery.
To manually trigger the delivery of an A/B-Split Job, go to the Ongoing Jobs screen and click on the
A/B Job ID. The A/B-Split Job Details screen opens. From here, click on the
Mail Job menu and select
Trigger Delivery of Authorized A/B-Split Job Variants. Click
[OK] to confirm the delivery.
In order to trigger delivery externally, the external process or application has to access the server where the LISTSERV Maestro User Interface (LUI) is running via HTTP (or HTTPS, depending on your setup). In addition, a security token must be obtained.
Note about Security Tokens: Since triggering an email job delivery is a highly privileged operation, LISTSERV Maestro requires that the external process or application delivers a security token for authorization. This security token is assigned internally by LISTSERV Maestro.
|
•
|
Standard Job – Each standard job has a unique security token, which is found on the job's details page once the job has been authorized.
|
|
•
|
Auto-Repeat Job – All jobs in an auto-repeat chain have the same security token, which is found on the details page of one of the jobs in the chain (while the job is authorized). Whenever the external process uses the security token to trigger the delivery of the job, the job will be delivered and a new job in the auto-repeat chain will be spawned, which then turn waits in the Ongoing Jobs list until the external process uses the same security token again to trigger this job's delivery. Because of this, the external process can independently determine how often a new job in the auto-repeat chain is to be delivered.
|
|
•
|
A/B-Split Job – The delivery trigger security token for A/B-split jobs are assigned depending on one of the following four cases:
|
|
•
|
Sampling A/B-Split Job with Common Delivery Settings for the Sampling Variants: The A/B-split job has a common security token that is found on the A/B-split job's details page once the variants have been authorized. Neither the sampling variants nor the main variant have individual security tokens. If the common security token is used to trigger the action, then this will trigger either the delivery of the sampling variants or the delivery of the main variant, depending on which variants are currently authorized for delivery. Because of this, the security token has to be used twice: At first the sampling variants are authorized, then the security token is used to trigger their delivery. Then, the main variant is authorized and the same security token is used to trigger its delivery as well.
|
|
•
|
Standard A/B-Split Job with Individual Delivery Settings for Variants: The A/B-split job does not have a common security token. Instead, each variant has an individual security token that is found on the variant job's details page once the variant job has been authorized. Each individual security token must be used to trigger the delivery of each individual variant job. This allows you to trigger the variants at different times. The security tokens of the variant jobs are very similar: Each security token is the same as the security tokens of its sibling variants, except for a unique suffix. The first variant will have the suffix "-A", the second the suffix "-B", and so on. This means that you only need to know the security token for one variant, and how many variants there are, to be able to trigger all variants. This is especially useful when setting up an external script or process to do this triggering because you only need to tell the process the common part of the security token, plus how many variants there are. The process can then build the complete security token for each variant by extending the common part with the correct suffix for each variant job.
|
|
•
|
Sampling A/B-Split Job with Individual Delivery Settings for Variants: The A/B-split job does not have a common security token. Instead, each sampling variant and the main variant has an individual security token that is found on the variant job's details page once the variant job has been authorized. Each individual security token must be used to trigger the delivery of each individual variant job. This also allows you to trigger the variants at different times. The same similar security tokens with different suffixes are used for the variants as described above for Standard A/B-Split Job with Individual Delivery Settings for Variants.
|
The security token is found in the Delivery Settings section on any of the Job Details screens.
Important: Everyone who is in possession of this security token and who can also access LISTSERV Maestro on its HTTP port (for example, with a normal web browser) will be able to trigger the delivery of the job. Therefore, the security token should be closely guarded and not be given out to unauthorized persons.
To show the security token, click the Show Security Token for Delivery Trigger link. Copy the security token from this box and make sure that the external process or application accesses the URL described below, replacing
[SECURITY_TOKEN] with the value pasted from the security token popup box.
Note: To create a new security token for a standard job, click the
Create New Security Token link in the popup box. To create a new security token for an A/B-split job variant, click on the
Create New Security Token for all Variants link. This will create a new security token for all of the A/B-split job variants. Once a new security token is created, the old token is no longer valid and cannot be used to trigger a job externally.
|
•
|
where SERVER_NAME is replaced with the name of your LISTSERV Maestro server. If a non-standard HTTP port is used, also include the port separated with a colon ":". If access to your LISTSERV Maestro is protected with HTTPS, you need to specify "https://" instead of "http://".
|
|
•
|
where SECURITY_TOKEN is replaced with the security token for the action that the URL will trigger.
|
The external delivery trigger is of the Simple URL Access type and does not contain any download data in its response. With this type, the action is triggered by accessing the external trigger URL with a HTTP GET-request.
By accessing this URL, a HTTP GET-request is made to LISTSERV Maestro. The server then verifies the given security token and, if it is valid, triggers the corresponding action. The result of the action will be returned in the form of a HTTP response.
Important: If you type the value manually, then make sure to maintain the exact spelling because security tokens are case sensitive.
Any auto-repeat job currently on the Open Jobs screen awaiting its scheduled delivery time can have its delivery authorization revoked just like a normal job. If authorization is revoked, the job will be put back into the Resume Jobs list, where it can be edited again.
If a job is re-authorized for future delivery, the job may have its standing changed in respect to the auto-repeat sequence it was part of before the authorization was revoked. Here are the possible standings:
|
•
|
The job is the first job of an auto-repeat sequence – This means that no delivery has taken place for this job because it was the first job of the sequence and was already revoked before its scheduled delivery time. When re-authorized, the job will simply continue to be the first (and still only) job of the same auto-repeat sequence it belonged to before.
|
|
•
|
The job is not the first job of the auto-repeat sequence but has not been changed since the delivery authorization was revoked – This means that this job is already an automatically created copy that is part of an auto-repeat sequence. The delivery authorization of this job was revoked, but the job has not changed since then. When re-authorized, the job can continue as part of the same auto-repeat sequence, or it can start a new auto-repeat sequence. This choice will have to be made on the Authorize Delivery screen.
|
|
•
|
The job is not the first job of the auto-repeat sequence and has been changed since the delivery authorization was revoked – This means that this job is already an automatically created copy that is part of an auto-repeat sequence. The delivery authorization of this job was revoked and then it was changed in some way. When re-authorized, the job will create a new auto-repeat sequence and it will no longer be part of the sequence it belonged to before the delivery authorization was revoked. This happens because the job is no longer an exact copy of the previous jobs in its original sequence. Instead, it will be the first (and still only) job of a new auto-repeat sequence.
|
Note: A job is defined as
changed since authorization was revoked if the recipients definition, content definition, tracking definition, or sender definition of the job has changed since the delivery authorization was revoked. If these four parts remained unchanged, the job is interpreted as unchanged. Changes on the Delivery Test or Delivery Settings screens are not interpreted as changes to the job.
