EEG microstate sequence analysis quantifies properties of ongoing brain electrical activity which is known to exhibit complex dynamics across many time scales. In this report we review recent developments in quantifying microstate sequence complexity, we classify these approaches with regard to different complexity concepts, and we evaluate excess entropy as a yet unexplored quantity in microstate research. We determined the quantities entropy rate, excess entropy, Lempel-Ziv complexity (LZC), and Hurst exponents on Potts model data, a discrete statistical mechanics model with a temperature-controlled phase transition. We then applied the same techniques to EEG microstate sequences from wakefulness and non-REM sleep stages and used first-order Markov surrogate data to determine which time scales contributed to the different complexity measures. We demonstrate that entropy rate and LZC measure the Kolmogorov complexity (randomness) of microstate sequences, whereas excess entropy and Hurst exponents describe statistical complexity which attains its maximum at intermediate levels of randomness. We confirmed the equivalence of entropy rate and LZC when the LZ-76 algorithm is used, a result previously reported for neural spike train analysis (Amigó et al. 2004). Surrogate data analyses prove that entropy-based quantities and LZC focus on short-range temporal correlations, whereas Hurst exponents include short and long time scales. Sleep data analysis reveals that deeper sleep stages are accompanied by a decrease in Kolmogorov complexity and an increase in statistical complexity. Regarding the practical use of these methods, we suggest that LZC can be used as an efficient entropy rate estimator that avoids the estimation of joint entropies, whereas entropy rate estimation via joint entropies has the advantage of providing excess entropy as the second parameter of the same linear fit. We conclude that metrics of statistical complexity are a useful addition to microstate analysis and address a complexity concept that is not yet covered by existing microstate algorithms while being actively explored in other areas of brain research.