The U.S. pharmaceutical industry spent upwards of $18 billion on marketing drugs in 2007. Detailing and drug sampling activities account for the bulk of this spending. To stay competitive, pharmaceutical managers need to maximize the return on these marketing investments by determining which physicians to target, when, and how to target them. In this paper, we present a two-stage approach for dynamically allocating detailing and sampling activities across physicians to maximize long-run profitability. In the first stage, we estimate a hierarchical Bayesian, non-homogeneous hidden Markov model to assess the short- and long-term effects of pharmaceutical marketing activities. The model captures physicians' heterogeneity and dynamics in prescription behavior. In the second stage, we formulate a partially observable Markov decision process that integrates over the posterior distribution of the hidden Markov model parameters to derive a dynamic marketing resource allocation policy across physicians. We apply the two-stage approach in the context of a new drug introduction by a major pharmaceutical firm. We identify three prescription-behavior states, a high degree of physicians' dynamics, and substantial long-term effects for detailing and sampling. We find that detailing is most effective as an acquisition tool, whereas sampling is most effective as a retention tool. The optimization results suggest that the firm could increase its profits substantially, while decreasing its marketing spending. Our suggested framework provides important implications for dynamically managing customers and maximizing long-run profitability.