Train Spotting Dick Snow

In 1977, Departmental funds were used to purchase an input/output device for experiments in real-time control . Eyebrows were raised when the order was placed with a model shop in Brighton for equipment manufactured by the Fleischmann company in Germany. When it arrived, a number of technicians were gainfully employed attaching pieces of track to a board, and wiring up the first of the Department s computer controlled model railway layouts.

One or more groups elsewhere had used a computer to control a model train - Brian Randell had seen such a system used for student projects at the University of Waterloo in Canada during the previous Summer. However, to the best of our knowledge, no-one had previously organised a model railway layout and its circuitry so that a computer could control the simultaneous movement of a number of trains. This of course would enable a much richer set of experiments in real-time control , many not in any way related to British Railways interests, to be planned.

At about the same time, the late Dr Philip Merlin was visiting Newcastle, and he and Brian Randell collaborated on a problem which began as a problem in deadlock avoidance. The interest in the trainset caused Randell and Merlin to consider their problem in terms of resource allocation and scheduling of trains on the layout. A technical report published at the time was subsequently studied extensively by theoreticians interested in modelling concurrency using Petri Nets, and The Merlin-Randell Train Problem eventually featured in Dr Koutny s Ph.D. thesis at Warsaw University.

The track was divided into 32 electrically isolated sections, the points were also controllable and at strategic positions around the layout, sensors were used to detect the location(s) of the engine(s). Initially, the sensors were magnetically sensitive reed switches which responded to the presence of a nearby magnet by closing the switch. These turned out to be somewhat unreliable, and were replaced by Hall effect transistors which performed the same function, but where the switching was achieved by semi-conductor rather than mechanical techniques. Magnets were Araldited on to the engines to activate the sensors. Since our experiments were concerned with investigating real time control, and realism was not one of our requirements, we were not interested at this stage in trains in the conventional sense of a power unit (engine) and a set of trucks or carriages. Thus, reference to "trains" should always be taken to mean "engines" without rolling stock.

Initially, control was the responsibility of a single M6800 processor, which communicated with three special purpose processors; one to control the sections, one to control the points, and the third to poll the state of the sensors.

The first control experiments were performed by an M.Sc. student, Gordon Pole, programming in M6800 assembly language. The primary function of the control software was to cause a train to progress along a pre-determined route from its source to its destination. It was however, designed so that a number of such journeys could be in progress simultaneously, and the control software was responsible for ensuring that no collisions occurred.

A fruitful relationship was building up at that time between the Laboratory and the software company CAP, who were developing a system called MicroAde, a support tool for Assembly language programmers. It seemed an ideal opportunity for the MicroAde software to be tried out in a real environment, since the development of the M6800 control program was no simple task, and the debugging support from MicroAde turned out to be invaluable.

In early 1978, CAP were exhibiting at the International Electrical, Electronic and Instrument Exhibition at the National Exhibition Centre in Birmingham, and they decided that a demonstration of a system developed using MicroAde would be appropriate. Accordingly they arranged for the trainset to be taken lock, stock and barrel, to Birmingham. The success of the venture is perhaps best illustrated by the anecdote, in which one of the NEC officials was asked by a visitor where the CAP display was. Over there replied the official, indicating vaguely towards the other side of the Hall, where the crowd is! .

Later in the same year, the trainset went on its travels again, this time to the 5th European Model Railway Exhibition in Westminster Central Hall. There, in the midst of all the carefully crafted scale models of World's End Junction, where every last model tree was meticulously placed, was Newcastle University Computing Laboratory s plain, flat board, with naked rails attached to it, with stations indicated by small pieces of dowelling stuck into the board and bearing a two-letter identifier, and not a single piece of rolling stock in sight!

The demonstration program on display at these events allowed the spectators to specify a route to be taken by a train. A simple command language was designed, which allowed the onlooker to specify the stations to be visited, to reverse the train, to specify the speed of the train, and even provided a primitive form of iteration. Members of the public were invited to complete a form giving a route, and one of the operators would place a train at the start of the journey and set the train in motion. The spectator would then have the privilege of seeing his train make his journey, stopping whenever another train obstructed its progress.

The demonstration program was extended by including an additional command in the "control language", which simply allowed the train to move around the layout at random, the only constraint being that the train was not allowed to stop (if all else failed, a train could always reverse). The spectacle of half a dozen trains moving around the layout at random at scale speeds of approximately 250 mph (long before the days of the 125 and 225 High Speed Trains!) was truly impressive.

Sadly, the student who was largely responsible for all of this became so engrossed in the project, that he never found the time to write up the results of his work, so that he did not finish his M.Sc., and documentary evidence of his achievements are sadly lacking.

There were however a number of further successful projects associated with the train set. In 1978, a system was built using the programming language Modula. The objectives of the project were broadly similar to those of the previous project, but to explore the use of a high level, modular, language in building such a system. Modula also provided a primitive process mechanism, thus allowing the system structure to reflect the parallel nature of the application.

When the first Assembly language program had been developed, the MicroAde system which supported its development ran on a small LSI-11 computer, from which assembled control program code was downloaded into the M6800. In order to allow systems to be developed in higher level languages such as Modula, it was necessary to run the control program itself on the LSI-11. Communication between the LSI-11 and the M6800 was over a (relatively) slow serial line, and a simple interpreter program (written in assembly language) was loaded into the M6800 whose function was to receive commands from the LSI-11 and cause the trainset to respond correctly to them. Similarly, the sensor information was collected and sent back to the control program in the LSI-11.

The following year, another project was attempted using the trainset. On the PDP-11/45 systems which had been purchased by the Highly Reliable System project it was possible to run Per Brinch Hansen s SOLO operating system. This system was written in, and supported only, the language Concurrent Pascal. Again the attempt was to try to design a train control program in a language which supported the paradigms which we believed were appropriate to an application in real-time control. This project was successful only in the sense that it demonstrated that such real-time control is hard to achieve when the conditions are unfavourable. The speed of the serial line between the trainset hardware and its controlling computers on the one hand, and the PDP-11 system on the other, was such that important events (i.e. sensor activations) were not received until it was too late to act upon them. Concurrent Pascal itself was an interpreted language, and consequently was, in some cases, unable to instruct the trainset quickly enough to avoid disastrous consequences for the trains. It was instructive to have design, coding and/or performance bugs in one s programs demonstrated in such a spectacular fashion!

Two more student projects were undertaken during subsequent years; one was an extension of the earlier Modula project, and the second, by Robert Stroud, was a re-working of the control system in Nicklaus Wirth s new language, Modula-2

At this point, the trainset fell into disuse. This was partly due to the fact that the hardware was becoming unreliable - points failed to switch when instructed to do so, engines failed for a variety of reasons, the track became dirty and required constant attention to allow the engines to pick up the necessary power, etc. This problem was compounded by the need to relocate the layout, and some difficulty was experienced in attempting to find a suitable permanent home.

Trainset-based experiments thus ceased, and little interest was shown in the problem until late 1989, when the Laboratory s research direction had moved very clearly in the direction of distributed systems. Reliability (now renamed Dependability) was still very much an issue, but now in the context of distributed programs and systems.

Thus it was that, in 1990, a new input/output device for experiments in real-time control was purchased, this time from Märklin (also a German company). This new layout was an altogether more extensive affair. In particular, it was divided into three separate boards, which, at least potentially, could be controlled separately. The method of control was somewhat different too. Instead of controlling the speed of a train by modulating the power sent through the rails to the engine, with the consequence that as a train moved from section to section, each section had to receive appropriately modulated power to allow the train to move through it, the trains in the new trainset have on-board microprocessors which respond to commands sent specifically to their own unique addresses. Points also are switched by sending an appropriate command to the point s unique address.

Sensors in the new layout are infra-red light sensitive transistor-like devices (not unlike the Hall effect transistors), which respond to the presence of infra-red emission from the engines.

Trainsets as available from Märklin had provisions for manual control (i.e. a control panel similar to, but somewhat more extensive than, would be found on the ordinary hobbyist s layout), as well as means of connecting to a computer. Since the plan was to place the system under computer control, just one manual control panel was purchased, and in the early stages the three boards were connected together (electrically) as though they were a single layout. It was only much later, after innumerable hardware problems (of which more later) were resolved, that a fully distributed control system was possible.

Initially, computer control was possible by sending suitable sequences of characters along a serial line. It transpired, as it had done years earlier with the first trainset, that reliable communication could only be achieved at very slow data rates - so slow that real-time control of a large number of trains was close to impossible. However, the computers used to control the layout were provided with bit-mapped display and mice, so that some level of sophistication was achieved in the user interface to the control program.

The first experiments on the new trainset were carried out as a group activity by the 1990 cohort of students on the M.Sc. course in Computer Software and System Design. Since the trainset was new, and the system and its control so different in concept from the previous version, much of this project was simply a learning exercise, and an attempt to provide a working layout. A certain amount was achieved however, and by the end of the project, a graphical front-end had been designed and implemented and was running on a Sun Personal Computer system. From this it was possible to cause one, or possibly two, trains to move about the layout. At this point, only rudimentary control was provided to prevented collisions.

The trainset was demonstrated at FTCS-20 - the 20th International Symposium on Fault Tolerant Computing Systems - which was held in Newcastle in 1990. The students' software was used for train control, and the demonstrations were very popular - indeed quite exciting - since it became clear that if one waited a few minutes there was a fair chance of witnessing a crash. At the time it was assumed this was due to inadequacies in the students' software, but it soon became clear that the hardware was also very much to blame.

The Märklin hardware continued to cause problems because of its unreliability, and the second trainset fell into disuse for a while. Another cohort of CSSD students were asked to look at train control as a group project, but they essentially abandoned the hardware, and concentrated on building a control program which could be demonstrated using a mimic diagram, rather than the real trainset. Although this group did go some way to defining the overall structure of the control program, the aim of the project, to produce a system providing a framework for future demonstration programs, was still not achieved.

As a theoretical exercise, the trainset provided no little interest for two of our research students. Rogerio de Lemos wrote his thesis on the subject of Requirements Analysis for Critical Real-Time Applications, and used the trainset as an extended case study. De Lemos' work, in collaboration with Amer Saeed and Tom Anderson, was reported in two papers; one published in a special issue of the Computer Journal on Security and Safety, and the other presented at the 21st International Symposium on Fault Tolerant Computing (FTCS-21). Despite the considerable difficulties with the hardware, Cecilia Calsavara came somewhat closer to using it when she pursued her researches into the use of delegation in class structured languages. In fact, she was able to demonstrate her system successfully on the occasion of her Ph.D. examination.

One of the more successful projects came about as a result of the visit to Newcastle of Heinz Appoyer from the Technical University in Vienna. Heinz was a student associated with Herman Kopetz' MARS project, and he brought with him a set of MARS real-time nodes which were connected to (part of) the trainset. This collaboration resulted in a successful demonstration being given at one of the Open Workshops of the PDCS (Predictably Dependable Computing Systems) project.

In the light of the unreliability of the hardware, most of which centred around the communications devices at the trainset end, it was finally decided to by-pass the Märklin control hardware altogether, and wire the trainset directly to a specially designed printed circuit board which could be installed inside an IBM PC (or PC clone). This work was undertaken by Keith Heron, and by early 1997 a prototype board was available. The construction of this device was no mean feat, since documentation of the electrical characteristics of the controller/trainset connection was not available, and the design was based on observation of the signals that actually passed from trainset controller to the tracks, and from the sensors to the controller.

Three PC s had been made available for use with the trainset - they had been scattered around the Department, and had to be retrieved before they could be used for the purpose for which they were obtained - and they were configured to run the Linux Operating System. This system has the feature that additional software device drivers can be loaded into the system without having to rebuild the whole kernel. This meant that it was possible to develop device drivers to allow the trainset - trains, points and sensors - to be programmed as ordinary UNIX devices. Drivers were developed, and at last we could see trains moving around the layout, with a very real expectation that distributed control was possible, and in the foreseeable future.

Also in 1997, a third attempt was made to have a system which would serve as a basis for future real time control experiments built as a group activity by the CSSD students. At the time the project started, the hardware devices were at a reasonably advanced stage of development, and there was some confidence that this time the students would be able to control the trains from their programs. Alas, this too turned out to be a false dawn, and in fact the first hardware device was declared fit and well approximately two weeks after the hand-in date for the project. As a postscript to this saga, the students returned to the problem in the last few days of their course, and some success has now been achieved in providing distributed control of the entire trainset

This third project had some interesting features. The student cohort was somewhat larger than had been the case in previous years, and instead of forming a single group, it was possible for the students to be divided into three groups. It was then apparent that each group should be responsible for control of a single board of the layout. Certain other constraints were imposed, such as the way in which the programs should communicate with each other as trains moved from one board to an adjacent board, and the interface which they should assume when communicating with the trainset hardware. In addition, it was specified that the board definition (a data file in which the positions of the various components of the layout were specified) should be the same for each group. Thus the board definition constructed by each group for its own board could, as and when we wished, be used by the control programs of the other groups. Furthermore, we insisted that each group used a different programming language for their control program. This specification was portrayed as a way of ensuring that the groups worked reasonably independently of each other, but was also aimed at improving the chances that we would end up with at least one satisfactory control system that could be used on each of the three computers to control the set of boards constituting the whole layout, even if one or even two of the groups failed to achieve the required goal. Finally, in keeping with the strong interest and expertise in system dependability which is part of the Department s research culture, and perhaps because of the history of unreliability demonstrated by the trainset hardware, some considerable effort was put into making the control programs fault tolerant.

It was further specified that the various control programs should operate using a mimic diagram (if for no other reason than an insurance against the failure of the hardware to be completed in time - a prediction which, sadly, proved to be correct), and that this mimic diagram should be in the form of a World Wide Web page.

Apart from communication with the trainset machines, this project proved a spectacular success. Each group reported that they could interwork with other copies of their own program, using the board descriptions from the other groups. In some cases, interworking was also demonstrated between different groups programs. Collision avoidance was also demonstrated, by showing more than one virtual train on the mimic diagram, and showing that portions, or sections, of track were being correctly claimed by the trains. These sections were then shown to be no-go areas for other trains.

Finally, distributed control had been achieved, since each instance of the control program, whether it was multiple copies of the same program or programs written by different groups, were indeed running on different computers.

Since the completion of this group project, more work has been done on the hardware and low level software. All three of the trainset PC s are now equipped with train controller boards, and each machine runs a piece of software known as the trainset server. This communicates with trainset client software which runs on the same machine as the control software. Thus, the control program merely needs to send train movement commands, point switching commands, and requests to read sensor information, to the local client software, which in turn passes the requests on to the server and thence to the trainset itself.

At the time of writing, the examinations for this latest group of CSSD students is imminent. and we are very optimistic that the External Examiner s visit will provide the first opportunity for the trainset to be put through its paces by a fully operational, distributed, control system. If this proves to be the success we anticipate, we can then look forward to the kinds of experiment and student projects in real-time control that were dreamed of some twenty years earlier.