Data streams are sequences of fast-growing and high-speed data points that typically suffer from the infinite length, large volume, and specifically unstable data distribution. These potential issues of data streams bold the necessity of data stream mining tasks. Ensemble learning as a prevalent classification approach is widely used in data stream mining studies. Besides the impressive performance of ensemble learning algorithms in providing a collection of diverse and accurate classifiers, they are specifically efficient in handling non-stationary data streams. Due to the component-based nature and the chance of dynamic updates for the components of the ensemble, this category is appropriate for dynamically learning the changing concepts of the data. This paper aims to provide a thorough review of the most significant ensemble-based data stream classification approaches, along with a discussion about the potential issues of the non-stationary data streams. Furthermore, comprehensive experimental analysis is performed to compare the classification performance of the well-known state-of-the-art ensemble-based data stream classification approaches on 24 synthetic non-stationary data streams. The superiority of the approaches is proved by conducting various statistical tests.