Aside from beig important for adaptive and autonomous systems (such as adaptability to changing environments), we believe that learning and statistical inference methods can be successfully applied to the meta-problem of evaluating the above list of properties of a learning or decision system, before it is deployed.
For reliable deployment and operation, learning and decision systems need to be trustworthy to users who have little or no knowledge about learning (e.g., engineers, designers, quality control specialists). System failures can occur and will occur, regardless of whether those systems contain online learning components, offline trained components, or hard-coded decision components. Therefore, questions such as "what are the tradeoffs for improving the quality of the outputs of a learning system in a certain region of the space?" or "what can be inferred (regarding future decisions) from observing the operation of a learning system in a simulated environment?" have deep ramifications and, if answered, can result in learning technology having a more serious impact on newly developed systems.

The first goal of this workshop is to explore the requirements and risks for deployable learning and decision systems. We must consider a large range of possible tasks such as autonomous navigation, supplier and health management, flight control, decision support (interacting with human experts), etc. Next, we would like to understand how (possibly novel) learning methods can contribute to the process of testing and evaluating learning and decision systems and what techniques are needed to evaluate the decisions computed by complex (even human- computer) systems. Finally, this workshop will bring together researchers and users of learning and adaptive systems and create a forum for discussing recent advances in validation and testing of learning systems, to better understand the practical requirements for developing and deploying learning systems, and to inspire research on new methods and techniques for the testing and evaluation of learning.

Topics of interest include, but are not limited to:

- statistical testing and validation of learned models
- metrics for the performance of learning systems
- definitions and metrics for stability in learning
- statistical and logical inference for validation purposes
- learning for safety-critical applications
- evaluation of online learning algorithms
- new approaches for trustable machine learning software development
- analysis of the robustness vs. risk tradeoff
- algorithms and tools for monitoring learning and adaptive systems
- deployed active and online learning tools
- novel problems and applications that require principled assessment of learning
- testbeds for the analysis and evaluation of learning systems
- design of experiments for the evaluation of learning.