Reinforcement learning is a research paradigm that is commonly utilized to tackle problems involving sequential decision-making. Agents learn optimum policy from samples generated by interacting with uncertain environments. Bayesian reinforcement learning presents the uncertainty measure of probability significance based on sequential decision-making, which plays a proper auxiliary function in the optimization of agent control policies, attracting a lot of study attention. This paper provides an overview of contemporary Bayesian and reinforcement learning methods in this setting. We begin by outlining the many types of reinforcement learning and Bayesian approaches, as well as their characteristics. Then we review the most sophisticated Bayesian reinforcement learning approaches from the perspective of uncertainty measurement and control, as well as summarize and discuss various challenges and related research in its expansion to complicated control tasks. Applying the Bayesian technique with reinforcement learning in diverse disciplines are then discussed. Finally, we highlight the current challenges of Bayesian reinforcement learning and future research topics that might help promote Bayesian reinforcement learning.