Just-in-time Self-Verification of Autonomous Systems

ended

Engineers and computer scientists are currently developing autonomous systems whose entire set of behaviors in future, untested situations is unknown: How can a designer foresee all situations that an autonomous road vehicle, a robot in a human environment, an agricultural robot, or an unmanned aerial vehicle will face? Keeping in mind that all these examples are safety-critical, it is irresponsible to deploy such systems without testing all possible situations---this, however, seems impossible since even the most important possible situations are unmanageably many. I propose a paradigm shift that will make it possible to guarantee safety in unforeseeable situations: Instead of verifying the correctness of a system before deployment, I propose just-in-time verification, a new, to-be-developed verification paradigm where a system continuously checks the correctness of its next action by itself in its current environment (and only in it) in a just-in-time manner. Since future autonomous systems will have a tight interconnection of discrete computing and continuous physical elements, also known as cyber-physical systems, I will develop just-in-time verification for this system class. In order to prove correct behavior of cyber-physical systems, I will develop new formal verification techniques that efficiently compute possible future behaviors---subject to uncertain initial states, inputs, and parameters---within a small time horizon. Just-in-time verification will substantially cut development costs, increase the autonomy of systems (e.g., the range of deployment of automated driving systems), and reduce or even eliminate certain liability claims. The results will be implemented in an open-source software framework and will be primarily demonstrated for automated driving. Successful development of just-in-time verification techniques is yet more challenging than offline verification of autonomous systems, but expected to bring even greater rewards.