Saving and Restoring a Backup

It is a standard best practice for any administrator to make regular backups of critical software and data. LISTSERV Maestro archives a consistent backup of the data collected in the application so that it can be restored in the event of a system failure.

LISTSERV Maestro gives the Administration Hub component the responsibility of acting as backup master to avoid any problems that might arise from having different components that store data independently and reside on different servers. If different components initiate backups at different times, inconsistent data sets between components can result. If both backups were then to be restored, the data sets would be inconsistent, invalidating the backup.

The Administration Hub will centrally trigger a backup on all connected components (including itself) in order that the backup data saved by each component is consistent with the backup data of all other components. This backup is initiated based on the values entered in the Global Component settings for the Administration Hub.

If regular backups are performed through LISTSERV Maestro, no external backups of the Maestro database are necessary. In the event of a system crash, the data stored in the Maestro database would not be sufficient to restore the Maestro application, as the data would not be synchronized with Maestro’s internal registry. Restoring LISTSERV Maestro can only be done from a complete backup initiated from within LISTSERV Maestro. Performing an external backup of the Maestro database serves no useful purpose and may even cause problems when Maestro attempts to access a database that is in backup mode.

The backup procedures described here are specifically for LISTSERV Maestro. If there are other databases on the same server as the LISTSERV Maestro database, they should be backed up separately.

Note: The procedures described here refer only to the three Maestro components – HUB, LUI, and TRK. If other elements are located within the LISTSERV Maestro tree (i.e. the LISTSERV web interface files or independent Web pages served up by Tomcat), then these must be backed up and restored independently.

The application wide backup is triggered once per day. Each day at a certain time, the Administration Hub (backup master) will start a backup of each component. To assign backup settings, click on the Global Settings menu, select Administration Hub, and then finally select General Administration. In the Time for daily backup field, set the time to start the backup master by entering the desired time of the daily backup in the form of hh:mm with values from 00:00 to 23:59.

There may be times when it is necessary to create a backup immediately; for example, just before any invasive procedure such as moving a component or applying a patch, or in the case of an emergency. To perform a backup immediately, click the [Execute Backup Now] button. Click [OK] to save settings and return to the Administer Component Settings screen.

The administrator may define external processes that will be executed after a backup is completed. External processes may be used to execute additional backup tasks such as automatically moving the backup folders to a tape, copying backup folders to a network drive, notifying the administrator by email if the backup was unsuccessful, and so on. Two different external processes can be defined, one to be executed after a successful backup and one after a backup failure.

Each process is specified in form of an external command that is executed by the Administration Hub when the backup completes. If it is necessary to execute more than one command, they can be written into a batch file (Windows) or shell script file (Unix/Linux). If this is the case, the name of that batch/script file is entered as the external command to be executed (with all necessary parameters). The administrator may also specify the work folder for the commands (same folder for both commands).

Clicking Test, located next to each command box, executes the command for testing. A new window will pop up that shows the output of the command. In this window, the external process can be stopped, if necessary. Closing the window before the process terminates, will not stop the process. To view the output of the test process again (if it is still running), or to terminate it (if it does not terminate by itself), access the process by using the View list of currently active "after backup" processes link.

Note: Commands must not define external processes that run indefinitely. Each external process should terminate itself when it has completed the action. Processes that run continuously slow the server down and will cause a crash because each time a backup finishes, a new process will be started, tying up more system resources.

If several external processes are running using a batch/script file, make sure that all processes started by the batch/script file terminate themselves at some point. If an external process that does not terminate itself is started, (because of a defect in the external process, or by mistake) click on the View list of currently active "after backup" processes link.

This screen displays a list of all currently active external processes started by the Administration Hub, either as an actual "after backup" process that was started when a backup was completed, or because the administrator clicked on one of the Test links. Only processes that are still running are shown in this list. Each process is shown with the date and time it was started, the command that was used to start it, and a link that opens a pop-up window. The pop-up window continuously shows the output of the external process (if any) and allows for the termination of that process while it is still running.

If any of the command fields are left empty, no external process will be started at the corresponding "after backup" condition. If the work folder is left empty, then the application home folder of the Administration Hub will be used as the work folder.

Each component has a backup location. This is necessary because the components may reside on different servers. The backup default location is the backup folder, which is in the home folder of the component in question (for example “\Program Files\L-Soft\Application Server\lui”).

It is possible to use a different folder if desired. The folder configured may be either an absolute path, such as “C:\MyFolder\backup”, or a relative path, such as “myFolder\backup”, which is then interpreted as being relative to the home folder of the component. Enter the path name in the Backup folder field at each of the following locations:

•	For the Administration Hub component, click on the Global Settings menu, select Administration Hub, and then finally select General Administration. The General Component Settings for Administration Hub screen opens.

•	For the Maestro User Interface component, click on the Global Settings menu, select Maestro User Interface, and then finally select General Administration. The General Administration of Maestro User Interface screen opens.

•	For the Maestro Tracker component, click on the Global Settings menu, select Maestro Tracker, and then finally select General Administration. The General Component Setting for Maestro Tracker screen open.

Important: Do not configure different components to save backups into the same folder. Doing so may cause backups from one component to over-write backups from another, resulting in data loss. Each component must have its own dedicated backup folder.

To lessen the risk of restoring a backup containing corrupted data, LISTSERV Maestro provides the opportunity for administrators to create a backup history. Each time a new backup is made, it is saved into the backup folder configured for the component (see Section 11.3 Configuring the Backup Location).

If the component is also configured to keep a number of previous backups, then the folders containing the older backups will be kept under names like “NAME1”, “NAME2” … “NAMEn”, where “NAME” is the name of the standard backup folder and “n” is the number of previous backups that the component is set to keep.

For example, if a component is configured to keep three previous backups, then the backup history of each day will look like this:

Day 1	backup – contains backup of day 1
Day 2	backup – contains backup of day 2 backup1 – contains backup of day 1
Day 3	backup – contains backup of day 3 backup1 – contains backup of day 2 backup2 – contains backup of day 1
Day 4	backup – contains backup of day 4 backup1 – contains backup of day 3 backup2 – contains backup of day 2 backup3 – contains backup of day 1
Day 5	backup – contains backup of day 5 backup1 – contains backup of day 4 backup2 – contains backup of day 3 backup3 – contains backup of day 2

Keeping a backup history can help ensure against corrupted backup data. However, as the amount of application data grows, it may not be possible to keep many old backups, which take up space on the disk. In addition, keeping older backups on the same disk does not ensure against failure of the disk itself (head crash for example). Always save the backup to an external backup medium as described in Section 11.5 Saving a Backup to an External Medium.

Once LISTSERV Maestro has completed its backup, the configured backup folder of each component contains the data that is required to restore this component to the state of the moment when the backup was triggered. To prevent catastrophic loss of data, save these folders to an external backup medium (e.g. a backup tape or storage device).

To avoid a potential partial backup problem, either use the automatically triggered external post-backup process, outlined in Section 11.2 Configuring External Post-Backup Processes, to ensure that the backup tool does not start its work until after the completion of the internal backup (recommended), or use whatever standard backup tool is used by the organization to configure a daily backup of the designated folders. Schedule this daily backup to occur after the time when the Administration Hub itself completes the backup of each component. There should be a enough time between the backup triggered inside of LISTSERV Maestro and the backup to the external medium triggered by the backup tool to ensure that all components have enough time to complete their backups. Otherwise, partial data would be backed up to the external medium.

For small installations, the backup inside LISTSERV Maestro will not take more than a few minutes. However, as the data in the LISTSERV Maestro installation accumulates over time, backup naturally will take longer. If post-backup triggers are not being used, periodically check the backup logs to see how long the backup actually takes and schedule the external backup accordingly, at a safe time after the LISTSERV Maestro backup is completed.

Remember that the external post-backup command or backup tool must be configured such that it backs up all backup folders of all components. A LISTSERV Maestro installation will have three backups to save to an external medium, one for the Administration Hub, one for the Maestro User Interface, and one for Maestro Tracker. These folders may also reside on different servers, depending on the installation.

Because the LISTSERV Maestro components store their backup data into separate folders, it is necessary to know which of the folders belong together, in case a backup history is kept or it is necessary to retrieve a backup from an external medium. This is done using the backup ID. Each backup gets a unique ID that is shared by all components participating in the backup. Each component also writes a “readme.txt” file into the backup folder. Stored in this text file is the ID of the backup that saved the data in the particular backup folder, together with output about backup start time, end time and its success or error state.

In the unfortunate event of having to restore a backup, follow the procedure described in this section. Several steps need to be executed to restore a backup successfully. Please review each step carefully. Some steps have lengthy descriptions or sub-steps. Do not skip steps or do them out of order, or the restoration will not succeed.

This usually is the most recent backup, but it also must be a successful backup. If there were errors during the most recent backup, revert to the next most recent backup, which may have to be retrieved from an external medium. To find out if a backup was successful check the backup log in the folder:

For each backup triggered by the Administration Hub, a report named “backupReport_ID.txt”, where “ID” is replaced with the ID of the backup in question, is saved into this folder. The IDs are assigned in alphanumeric order; the most recent backups have higher order IDs (in an alphanumeric sense). Use the file date of the report file to locate the most recent backup. If the backup was successful, an entry like this will appear at the end of the file:

If the logs folder cannot be accessed, (because a disk crash destroyed the disk of the Administration Hub installation, for example) it is still possible to locate the most recent successful backup by opening the readme.txt file in the backup folder of each component. The readme.txt file lists the backup ID and the success state of that particular backup. If no errors are reported in that file, then the backup of this component was successful. If successful backups with the same ID of all the other components are located, then a complete and successful backup set exists and can be restored.

Once the backup to be restored has been identified and the backup ID is determined, the next step is to find all the backup folders of the individual components that contain data for this backup. Check the readme.txt in the backup folder of each component. If it contains the same ID, the right backup folder for this component has been located.

If the original system is still in working order and the purpose of the backup restoration is simply to restore a previous state of the application user data (for example if the application user data was corrupted, or if some LISTSERV Maestro objects, like a mail job or hosted recipient data, were accidentally deleted), then it is not necessary to do a reinstallation of the application. In this case, simply proceed with the next step.

If the original system was destroyed (for example by a disk crash) or generally damaged (where it is not clear if the damage is limited to the application user data or may also have affected the application binaries), then you will need to do a fresh installation (this also includes uninstalling the old files, unless you start from scratch on a new system). If you do this, install the components on the servers where you need them. After the re-install, do not start the components. Instead proceed as described in the next step.

Note: If LISTSERV Web Interface files and/or other non-Maestro files are maintained within the Maestro application folder tree, then care should be taken to preserve them, if possible, before wiping out the old installation.

To restore the Administration Hub:
Remove the existing versions of the file “hub.ini” and the folders “accountreg” and “hubreg”, including their contents, from the Administration Hub home folder:

To restore the Maestro User Interface:
Remove the existing versions of the files “lui.ini” and the folders “luidata” and “registry”, including their contents, from the Maestro User Interface home folder:

Important: If the backup is from a LISTSERV Maestro version earlier than 2.1, the backup may also contain a file called “my.ini”. This file is no longer required by LISTSERV Maestro 2.1 and should not be restored from the backup.

This path name may either be a full path name including drive letter, or it may be an absolute path without drive letter starting with “\” or “/”, which is then interpreted as being absolute on the drive/root where the application server is installed (for example, in the default case for Windows, the same drive where “\Program Files\L-Soft\Application Server” is located). Or a relative path without a driver letter may be used, and not starting with either “\” or “/”, which is then interpreted as being relative to the home folder of the Maestro User Interface component (for example, in the default case for Windows, that would be the folder “\Program Files\L-Soft\Application Server\lui”).

This entry to the “lui.ini” file will be automatically removed during the first startup of the component. It is only present to signal to the component that it should restore all required data from the given folder, which happens automatically during the next startup, whenever this INI file entry is present. For more information on editing INI files, see Section 20 Editing LISTSERV Maestro INI Files.

Replace it with the version from the backup folder of the Maestro Tracker component. Also remove the tracker.ini file in the Maestro Tracker home folder, and then replace it with the same file from the backup folder.

Note: If the backup is from a LISTSERV Maestro version earlier than 2.0-4, then the backup may contain several *.dat files instead of a single data subfolder, which was introduced in 2.0-4. In this case, restore the backup as follows:

If components are being restored on different servers or a different combination of servers than where the original backup was taken from, it may be necessary to edit the respective “*.ini” files of the components. This would include restoring a backup to a server with a different name, using a different port number, or changing how the components are grouped on a server or servers. For example, if components that were all originally on the same server are moving to different servers, or taking components that were originally on different servers and moving them to the same server.

If LISTSERV Maestro is using a default internal MySQL database that has undergone modifications or optimizations to its configuration (because of changes made through the MySQL configuration tools or by manual edits to the “my.ini” file in the “lui\database” folder), those modifications must be re-applied. The freshly installed LISTSERV Maestro contains an internal database with the default configuration.

to verify that all settings (like minimum/maximum memory allocation, run-user, etc.) are correct for the installation. If you still have access to the corresponding file of the original installation, you can compare with the values in that file.

During startup, the system database content will be restored from the backup folder. Monitor the log files of the components to check if they start up correctly. If yes, the backup restoration is complete. If any component does not start up correctly, this may be because of differences in the configuration of the backed up system and the restored system. In that case, it may be necessary to adjust further INI file settings (see previous step) or to log into the Administration Hub and configure the necessary settings accordingly. Then restart and again monitor the startup log entries. If necessary, repeat this until the system starts up normally.

Note: If you are using the internal MySQL system database and have previously made modifications or optimizations to its configuration (for example by using the MySQL configuration tools or by manually editing the file “my.ini” in the folder “lui/database”), then you now need to re-apply the same modifications, since the freshly installed LISTSERV Maestro once again contains an internal database in the default configuration.