The growing use of deep neural networks in safety-critical applications makes it necessary to carry out adequate testing to detect and correct any incorrect behavior for corner case inputs before they can be actually used. Deep neural networks lack an explicit control-flow structure, making it impossible to apply traditional software testing criteria such as code coverage to them. In this paper, we examine existing testing methods for deep neural networks, the opportunities for improvement and the need for a fast, scalable, generalizable end-to-end testing method for deep neural networks. We also propose a coverage criterion for deep neural networks that tries to capture all possible parts of the deep neural network’s logic.