RP3, or Research Parallel Processing Prototype, was the name given to the architecture of a research vehicle for exploring the hardware and software aspects of highly parallel computation. RP3 was a shared-memory machine designed to be scalable to 512 processors; a 64 processor machine was in operation for two and half years starting in October 1988. The operating system for RP3 was a version of the Mach system from Carnegie Mellon University. This paper discusses what we learned about developing operating systems for shared-memory parallel machines such as RP3 and includes recommendations on how we feel such systems should and should not be structured. We also evaluate the architectural features of RP3 from the viewpoint of our use of the machine. Finally, we include some recommendations for others who endeavor to build similar prototype or product machines.