The paper explores the conditions that determine the effect of rule enforcement policies that imply an attempt to punish all the visible violations of the rule. We start with a simple game theoretic analysis that highlights the value of gentle COntinuous Punishment (Gentle COP) policies. If the subjects of the rule are rational, Gentle COP can eliminate violations even when the rule enforcer has limited resources. The second part of the paper uses simulations to examine the robustness of Gentle COP policies when the subjects of the rule are adaptive learners. The results show sensitivity to the probability of detection, and to the enforcing resources. The paper concludes with experimental studies that clarify the value of Gentle COP policies in the lab, and in attempt to eliminate cheating in exams.