Testing and evaluation processes are crucial to the success of any NLP conversational system,but testing speech based & multimodal systems presents unique challenges. Focusing on multimodal applications that involve speech and other modalities, we describe the multiple layers of testing and QA: engine quality, functional application, VUI, interfaces and infrastructure. Learn how to set testing goals, targets, and success factors; specify and measure metrics; test and measure “soft” and “immeasurable” targets; test documentation in all stages; manage a testing project; and identify who should be on the testing team