Observations on the Development of an Operating System -- H. C. Lauer (1981) ---------------------------------------------------------------------------- Summary: The author's experience in developing the Pilot OS, along with the lessons learnt during the process are presented. OSes are categorized into five types based on their development cycle (independent of their structure), and it is conjectured that the OSes of the same type as Pilot take five-seven years to mature. ------ Pilot is an OS developed for the PC at Xerox PARC. Pilot was meant as a successor to the Alto and unlike Alto it was a carefully planned system with ambitious goals. Refer to the paper for more details about Pilot and a chronology of the development of Pilot. Problems encountered and lessons learnt: - The final size of the Pilot system was more than what the designers imagined it to be. - Since Pilot had a virtual memory system unlike the Alto, it was easy for the developers to lose sight of the working set size of the system. Finally, it took them a year's hard work to trim the system. - Measuring programmer productivity is hard. The traditional measure of lines of code per man-year has two problems: first is deciding which people and code to count, and second is that a person generally takes longer time to produce a smaller and more efficient system. This would decrease the lines of code (and increase the time) and hence the measured productivity would be lower which is not true. - The designers of the systems had "holy wars" about inter-process communication (message passing vs. shared memory) and file access mechanism (read-write vs. memory mapped), on which they expended a lot of time, energy and emotions. - Support for transactions was not initially present in Pilot. However, when an inefficient and unreliable transaction mechanism was built on top of Pilot by someone else and it became popular, the designers were to build a more efficient and reliable transaction mechanism into the Pilot kernel. - The virtual memory system of Pilot provided for nested address spaces in a hierarchical fashion. This required a lot of implementation effort. However, none of the client programs developed for the Pilot used this feature. - The Unix concepts of pipes, filters and streams were implemented in Pilot but never became popular because of the nature of the Mesa programming language. The author categorizes OSes into five types: OSes of the first type start out as small unambitious systems but become immensely popular due to their simplicity and excellence (Unix, Alto). OSes of the second type are designed and implemented as major projects, and not all of them are successes (OS/360, Multics, Pilot). OSes of the third type borrow much of the software from an existing OS, and only a small part is re-designed. OSes of the fourth kind are experimental and research OSes and contribute a lot to the OS technology. Systems of the fifth kind are small, uninteresting, and do not contribute much to the technology The author conjectures from his experience that OSes of the second kind take five to seven years to grow from birth to maturity irrespective of the intellectual capabilites of the designers. He discusses the (five) different stages in the development of such an OS. He counters the comment that the reason for the long time to maturity of such systems is because they are "too ambitious" and/or "too general", by suggesting that the main difference between the OSes of the second type and others is that the OSes of second type are investments made by companies.