Stack Overflow (SO) is the most popular online Q&A site for developers to share their programming expertise. Given multiple answers to certain questions, developers may take the accepted answer, the answer from a person with high reputation, or the one frequently suggested. However, researchers observed exploitable security vulnerabilities in popularly suggested code, which was widely reused by high-profile applications. This observation inspires us to explore the following questions: How much can we trust the security implementation suggestions on SO? If suggested answers are vulnerable, can developers trust the community’s dynamics to infer the vulnerability and identify a secure counterpart?

To answer these questions, we conducted a study on security-related SO posts by contrasting secure and insecure advice with the community-given content evaluation. We compiled 953 different groups of similar code examples and labeled their security, identifying 781 secure answer posts and 648 insecure ones. Compared with secure suggestions, insecure ones had higher view counts (36,280 vs. 18,810), received a higher score 14 vs. 5), and had significantly more duplicates (4 vs. 3) on average. 38% of the posts provided by highly reputed so-called trusted users were insecure.

Our findings show that based on the distribution of secure and insecure code on SO, users being laymen in security rely on additional advice and guidance. However, the community-given feedback does not allow differentiating secure from insecure choices. Moreover, the reputation mechanism fails in indicating trustworthy users with respect to security questions, ultimately leaving other users wandering around in a software security minefield.