About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
Publication
SEAMS 2020
Conference paper
Expecting the unexpected: Developing autonomous-system design principles for reacting to unpredicted events and conditions
Abstract
When developing autonomous systems, engineers and other stakeholders make great efforts to prepare the system for all foreseeable circumstances. However, such systems are still bound to encounter situations that were not considered at design time. For reasons like safety, cost, or ethics it is often highly desired that these new cases be handled correctly upon first encounter. In this paper, we first justify our position that there will always exist unpredicted events and conditions, driven by, e.g., new inventions in the real world, the diversity of world-wide system deployments and uses, and the possibility that multiple events that were unforeseen at design time (or overlooked, or knowingly abandoned following cost-benefit-risk calculations) will not only occur, but will occur together. We then argue that despite the unpredictability, handling such situations is indeed possible. Hence, we offer and exemplify design principles, which, when applied in advance, can improve the system's ability to deal with unpredicted situations. We conclude with a discussion of how this work and a much-needed thorough study of the unexpected can contribute toward a foundation of engineering principles for developing trustworthy next-generation autonomous systems.