Programming assignment grading can be time-consuming and error-prone if done manually. Existing tools generate feedbacks with failing test cases. However, this method is inefficient and the result is not complete. We present AutoGrader, a tool that automatically decides the correctness of programming assignments and provides counterexamples, given a single reference implementation of the problem. Instead of counting the passed tests, our tool searches for semantically different execution paths between a student’s submission and the reference implementation. If such a difference is found, the submission is deemed incorrect; otherwise, it is judged as a correct solution to the problem. We leverage weakest preconditions and symbolic execution to capture the semantics of execution paths and detect potential path differences. AutoGrader is the first automated grading tool that works on program semantics and generates feedback with counterexamples based on path deviations. It reduces human efforts in writing test cases and makes the grading more complete as well. We implement AutoGrader and test its effectiveness and performance with real-world programming problems and student submissions collected from an online programming site. The experiment revealed no false positive of our proposed method and we detected 11 errors of online platform judges.