Early History of LISTSERV^®

Background

Paris, 1985...Together with the other computer enthusiasts in his class, Eric Thomas decides to visit various universities in the Paris area to find out what kind of computing equipment they had, and under what conditions students could get access to it. At the time, most university students did not have access to any computer equipment at all – at least not unless they were studying computing science. Even then, the equipment offered was usually ancient and, when there happened to be a few modern machines, they were invariably reserved to staff and faculty (since education is free in France, the universities have significantly smaller budgets than in the US).

Unsurprisingly, Eric's tour turned out to be a total disaster. Some universities did not even offer video terminals; all work had to be done on a teletype (a kind of "paper terminal" where the screen is replaced with a 100cps printer, with the convenience and noise level that you can imagine). In the best cases, there was one PC for every 10 computing science students. This was very, very uncool. Eric had been using IBM mainframes for 5 years and couldn't imagine going back to machines with 180k floppies and no hard disk! So he kept looking and looking, and eventually found a computing centre that seemed too good to be true. They had a brand new IBM mainframe, with graphical colour terminals (a luxury at the time), and the students were allowed to use the system! This machine was a lot faster than the (already ancient) 370/138 Eric had been using until then, and he couldn't wait to get an account. Mission impossible! There were bound to be more than enough layers of bureaucracy to prevent people not affiliated with the university from getting any kind of access to this dream machine.

After thinking about it for a while, Eric finally decided that boldness was called for. He headed for the system administrator's office and offered to help with system management and to develop software in exchange for an account on the machine. The deal was sealed when he mentioned that he was an expert on VM security. A number of students had recently broken into the system, calling the attention of the higher ups. Eric showed the system administrator how to seal the system tightly and identify the perpetrators, and in exchange he was allowed to use the system whenever he wanted. The machine, it turned out, was connected to BITNET, and this is where the history of LISTSERV begins.

BITNET

In 1985, BITNET was THE academic network. The Internet did not exist yet, and its ancestor, the ARPAnet, was still mostly a defense network. A few US universities were connected to the ARPAnet, but in Europe the only large, non dial-up network was BITNET. BITNET had a Network Information Centre, called BITNIC or just "the NIC". Like most BITNET sites at the time, the NIC was using an IBM mainframe running VM/CMS.

The NIC's mailing lists

The NIC had developed a program that allowed them to implement mailing lists on their IBM mainframe. To join a mailing list, you would write to INFO@BITNIC (which, in spite of what the name might suggest nowadays, was a human being) and ask to be added to the list. The staff person reading INFO would then edit the file containing the addresses of all the subscribers, and from then on you would write to, say, LINKFAIL@BITNIC, to report scheduled downtime at your institution. Your message would then be duplicated and one copy would be sent to every address on the mailing list. The message, however, would not originate from LINKFAIL@BITNIC, but from the account in which the mailing list program was running. To reply, you would have to compose a new email message addressed to LINKFAIL@BITNIC. If you simply used the reply function, your message would go to the mailing list program, which would just throw it away without any kind of warning or acknowledgement.

Since the NIC ran its mailing list program in an account called LISTSERV, the BITNIC mailing list service became known as LISTSERV@BITNIC – the address from which all the mailing list messages were sent to subscribers. This "LISTSERV," however, has nothing to do with the product currently known as LISTSERV. It was just the name of a special account that the NIC had created to run its program, and you could not interact with this program at all; if you sent it something, it just threw it away without any kind of reply or acknowledgement. The only way to interact with LISTSERV@BITNIC was to write to the person who maintained it.

If it ain't broke...

Nevertheless, Eric found the NIC's LISTSERV service very useful. There were mailing lists about all sorts of technical topics. When you had a problem with your system, you could ask the list for help and get useful advice in a matter of hours from people who had run into just the same problems and found a solution. It was irritating to receive all the messages from LISTSERV@BITNIC and not know what list they were from until you opened the individual messages and actually read them, but the mailing lists provided such a valuable service that no one was inclined to be too picky about the interface. People who run mainframes are traditionally very busy and go by the motto that "if it ain't broke, don't fix it." In fact, Eric hadn't even bothered to find out how these mailing lists actually worked. He just knew how to use them and where to write when he had a problem.

Problems in sight

In the summer of 1986, however, the flaws of this original LISTSERV service became apparent. Hundreds of requests were received by INFO@BITNIC every day, and it could take days or even weeks to be added to a mailing list (or, worse, to be removed from a list that had turned out to be totally uninteresting). Most importantly, the lists generated thousands of messages every day, a large fraction of which had to travel over the saturated transatlantic lines (two 9600bps analog lines, of which at least one always turned out to be down at any given time). So, every time someone posted a message to LINKFAIL, hundreds of copies of the message added to the thousands of messages that were already waiting to cross the Atlantic. In fact, the traffic from the mailing lists alone was threatening to make private email totally unusable, with delivery delays approaching a week. Some people were starting to use the lists to discuss whether mailing lists should be allowed, adding their comments to the growing backlogs.

Being located in Paris at the time, Eric was one of the many victims of these backlogs. It had reached a point where he was no longer able to get his work done, which justified investing some time in a technical solution.

A technical digression [skip it]

Over the years, Eric had specialised in the writing of what VM people call "DVMs" (Disconnected Virtual Machines). Using modern parlance, a DVM is a kind of server that processes requests from clients (called "user interface programs") running in the individual users' accounts. In fact, the VM operating system is internally organised around what is now called the client/server paradigm. For instance, to change your password you run a "user interface" program that prompts you for the old and new passwords, makes sure you typed everything correctly, issues all sorts of warnings, and finally sends a password change request to the DVM in charge of passwords and other account attributes. This DVM checks its various security rules and may perform or reject the change. The program that you run to change your password has absolutely no privileges. You could write your own if you wanted. The security is all in the DVM, which is much more difficult to tamper with than a program running with extended privileges in your own account. The user interface program "connects" to the DVM, "sends" a request, waits for the "reply" from the DVM, and finally "severs" the connection. These concepts are identical to what we now call client/server, and in fact VM's "IUCV" protocol is quite similar to a TCP/IP connection. The main difference is that this was a single-system architecture, with client and server on the same machine and no network involved. The goals were security, modularity and robustness – not portability.

Revised LISTSERV

Anyway, Eric had specialised in the development of these DVMs and decided to develop a real LISTSERV server, something that would process all the requests INFO@BITNIC was handling manually. This "Revised LISTSERV", as Eric decided to call it, would be based on a distributed model, as opposed to BITNIC's centralised model. This would allow the load to be shared between several machines, and in particular between machines located on either side of the Atlantic.

Peered lists

Originally, load sharing was accomplished through the use of "peered" mailing lists. You would set up two (or more) mailing lists, and tell LISTSERV to "peer" them. Any message posted to the one would then be sent to the other. When a user requested to join a peered list, LISTSERV would automatically forward the request to the peer that was closest to the user. With all the Europeans on the European peer, only one copy of the message ever had to cross the Atlantic. The backlogs melted away. In passing, Eric also arranged for the messages to come from the list address, so that the "reply" function would work as the users expected it to. The first version of this Revised LISTSERV was released in June 1986.

Just a little side project...

At the time, Eric saw his Revised LISTSERV as nothing more than a solution to a serious problem the network was having. He thought he would have to work on it occasionally for another month or two, just to fine-tune the code and fix problems, and that it would then be over. Developing Revised LISTSERV had been interesting, but not quite as interesting as the other things he was working on at the time. Just a little side project, in passing.

Well, Eric was wrong. In the next few months, dozens of BITNET sites installed Revised LISTSERV and started creating their own lists. Computing centres had been reticent to implement mailing lists thus far because of the tremendous amount of manpower required to process all the "add me" requests from subscribers. Everyone knew that INFO@BITNIC was now manned full-time by someone who did nothing but add and remove people from lists all day long, and that they were still unable to meet demand. People simply didn't want this to happen to them. With this obstacle lifted, system administrators were now willing to start offering mailing lists, at least on a trial basis. Naturally, a mailing list was created to discuss Revised LISTSERV itself, and people kept finding new creative uses for the software and proposing exciting enhancements to the code.

DISTRIBUTE

By November 1986, there were already 41 sites running Revised LISTSERV. To further reduce the amount of bandwidth used up by mailing lists, Eric introduced DISTRIBUTE, a bulk mail delivery algorithm that used the 41 Revised LISTSERV sites as a "backbone" for efficient mail delivery. The server hosting the mailing list would deliver the message to subscribers in its area, and pass it on to each adjacent server (but only if there were recipients in these servers' respective areas). The adjacent servers repeated the process until all the subscribers had received their copy. The messages only needed to cross each network line once. This process is similar to what is currently used to broadcast live audio/video over the Internet.

File server functions

In December 1986, Eric added file server functions to Revised LISTSERV, modeled after the NETSERV software developed by Berthold Pasch of IBM Germany. With these file server functions, people could make agendas, minutes, documents and software available on LISTSERV for private or public retrieval. In particular, the messages posted to the list could be archived automatically, and old messages could be retrieved easily by the subscribers.

Database functions

LISTSERV then stabilized for a little while, and Eric thought that the project was finally over. But the number of users (and new ideas) kept increasing, and in December 1987, database functions were added to the package, making it possible to search list archives and other databases for messages about certain topics. This effectively turned LISTSERV into a large electronic encyclopedia.

List of lists

In January 1988, this was complemented by the automatically maintained list of lists, which allowed users to search for mailing lists covering specific topics. Eric moved to Switzerland shortly thereafter.

On May 6, 1988, the 1,000th public LISTSERV list was created.

Revised LISTSERV becomes LISTSERV

Over the years, Revised LISTSERV became a ubiquitous tool for BITNET users. As its popularity grew, the proportion of "old timers" among the list owners dwindled. The new list owners did not know the story of the old LISTSERV@BITNIC program (which had disappeared completely) and started to call the software just "LISTSERV". In fact, people started asking what was the difference between LISTSERV and this "Revised LISTSERV" that some people had been mentioning, and the old timers gradually dropped the "Revised" to avoid having to answer this question every time. So, Revised LISTSERV became LISTSERV in 1988-89.

LISTSERV continued to be improved over the years, with an average of two new versions a year (and meanwhile Eric moved to Sweden). Until 1991, there was no similar software for unix. This was not a problem because LISTSERV supported remote list owners. Thus, you did not need to learn anything about mainframes or even get a mainframe account in order to use LISTSERV, you just had to ask your computing centre to set up the list for you on the local mainframe. As long as mainframes were popular, this was not an issue; most universities would create just about any reasonable list sponsored by a staff or faculty member. The mainframes were routinely upgraded as they ran out of processing power. LISTSERV was usually a drop in the bucket, simply because the networks were not fast enough to make it possible for LISTSERV to use much CPU time. Besides, most Internet people felt that there was no need for LISTSERV at all and that usenet should be used instead, and no one was really interested in developing a unix list manager.

The mainframe crisis

This situation, however, changed gradually over the years as IBM went through the crisis that we all know about. People started to make plans to get rid of their mainframes, and since LISTSERV could only run under VM, it would have to go with the IBM iron. Eric did not want his software to disappear with the mainframes, so in 1991 he started looking for ways to port LISTSERV to other environments. Initially, he just made changes to the existing code to make it easier to port should the need arise. In 1992, he determined that the time had come to take a more active role, and started writing proposals and submitting them to possible sponsors – EARN, CREN, software vendors, etc. Eric's goal was to "make it happen", and to remain involved in the design and coordination of this development so that the "spirit" of LISTSERV would be preserved in the new package.

Grants and sponsors

LISTSERV is a rather large application and Eric estimated that it would take 35 to 50 of the mythical "man months" to port the code to VMS and/or unix. This was a lot of work for volunteers to contribute without financial compensation, and Eric started looking for sponsors to fund the development.

Try as he might, he could not find a sponsor. Some were interested, but did not have a big enough R&D budget to allow the development to reach completion in a reasonable time frame. Most dismissed Eric's proposal on the grounds that it proposed reusing mainframe code for an application that would run on VMS and unix, instead of rewriting everything from scratch. The chances of success, they judged, were very low, and the resulting product would be at best unreliable and difficult to maintain, at worst totally unusable. Meanwhile the clock was ticking and every month a new site was forced to terminate LISTSERV services because their mainframe was being removed.

Going commercial

In 1993, Eric decided that this grant search was going nowhere, and that it was time to quit talking and start getting things done. But where would the money come from? Eric could not borrow that kind of money in Sweden, because he had only been living there for three years and the banks just did not have much information about him. On the other hand, he could not borrow in any of the other countries where he had lived before, because he no longer had any income or assets in these countries. The banking world is just not designed for people who emigrate every few years.

Several people had contacted Eric over the years, suggesting that he start a business and offering assistance. Initially, this had not sounded very interesting at all. In fact, turning LISTSERV into a business had sounded like a whole lot of totally boring administrative and non-technical work. The mere thought of all the forms that would have to be filled was enough to get Eric re-thinking those grant proposals. But the grant money had failed to materialise, and going commercial now appeared to be the only way to save LISTSERV from sure extinction in the mainframes' death grip. With a fresh determination Eric got in touch with the business people who had contacted him earlier, and this is how L-Soft came to be.

L-Soft

The first legal incarnation of L-Soft was born in a lawyer's drawer in May 1993. This, the lawyers had explained, was a temporary L-Soft, optimised for the development and start-up phase. There would be a whole lot of issues to look into, at $300 an hour, and a whole lot of strategic decisions to make before L-Soft could reach corporate Nirvana. The adventure had begun.

The company closed its first big sale in July 1993. The business plan was to sell maintenance and new versions of the mainframe product, and use the proceeds to hire programmers to port the code to VMS and unix. Licensing the existing product turned out to be surprisingly easy; people did not seem to mind that the company was new and small, or that something that had been free for 7 years had become a commercial product. Eric learned that as long as the product was solid and reliable, people were willing to pay to use it. In fact, most of the existing LISTSERV administrators were very supportive. They knew that their mainframes were going away, and were delighted to see that LISTSERV was finally going to be ported to other systems. Their managers were happy, too, as the loss of LISTSERV services had turned out to be a cause of significant user unrest, and a fly in the ointment for the (financially inevitable) downsizing plans.

Hiring programmers, on the other hand, turned out to be the main challenge. While the customers may not have cared that the company was small, prospective employees did. The people Eric wanted to hire invariably had a good, stable job, and most had a family to worry about. Newly graduated programmers were easy to find, but they were not familiar with LISTSERV and would not have been operational immediately. The job also required serious VM experience, in order to understand the existing code, and the junior programmers only knew unix, or sometimes VMS.

The first product

But these problems eventually sorted themselves out, and L-Soft released its first new product, LISTSERV-TCP/IP for VM, in March 1994. This was a new version of LISTSERV which still required VM, but could be run on machines not connected to the BITNET network (many universities were starting to leave the BITNET network and needed this functionality as soon as possible). A lot of changes, notably in the DISTRIBUTE algorithm, had to be made in order to make this possible, and these changes were a pre-requisite to the development of the VMS and unix versions.

With the availability of this first product, the temporary L-Soft was upgraded to its current incarnation. This is why you may find documents or articles saying that L-Soft was founded in 1993, while others say 1994. Today, L-Soft international, Inc. is a full-fledged, private U.S. corporation.

LISTSERV for unix, VMS and NT

The first versions of LISTSERV for unix and VMS were released in June 1994. Surprisingly, nobody wanted to buy a copy! The development had happened too fast, and people did not believe that L-Soft actually had a working product. This turned out to be due to the unconventional porting method that Eric had decided to use for LISTSERV. Technically, the advantages were clear: reduced porting time, more portable end product, full compatibility guaranteed by reusing the exact same source code, only one version to maintain, etc. But, from a marketing perspective, it was difficult to convince customers that L-Soft had taken mainframe code and made it work on unix, that this code was actually usable and reliable, and that it had all happened in such a short time frame. Offering free licenses did not help, because people still needed a dedicated machine on which to run the code (the sites in question were all delivering around 100,000 messages a day, which at the time pretty much required a dedicated workstation). Management would not buy a machine without knowing for a fact that it would get the work done with their particular workload and users, and this could not be established until a machine was found on which to run the code.

Luckily, the problem disappeared in a matter of weeks. A handful of large sites finally installed the product, and reported that it worked as advertised. About a month later, everyone wanted a copy.

Windows NT and Windows 95

The Windows NT version started beta testing in November 1994 and was released in May 1995, together with new versions of the VM, VMS and unix products. Ironically, it is this (mostly) PC-based version that has become the norm for "industrial strength" LISTSERV sites today: at the time of this writing, all but a few of the largest LISTSERV sites were running Windows NT.

The Windows 95 version was released in August 1995 (on the same day as Windows 95 itself), as a shareware product. The Windows 95 version is functionally identical to the Windows NT version, with a few minor differences (for instance, LISTSERV cannot run as a service under Windows 95 because this system does not support services).

---

L-Soft international, Inc. 1996
Portions of this text are © Eric Thomas, 1995 – used with permission.
For permission to reprint, contact: info@lsoft.com

---

LISTSERV is a registered trademark licensed to L-Soft international, Inc.

See Guidelines for Proper Usage of the LISTSERV Trademark for more details.

All other trademarks, both marked and unmarked, are the property of their respective owners.