The COVID-19 pandemic highlighted the need to understand the invasion dynamics of SARS-CoV-2 variants. Here, we investigate the successive invasions and geographical spread of three key variants of concern - Alpha, Delta, and Omicron - using an integrated dataset of genomics and geographic data from the canton of Vaud, Switzerland. We used a high-resolution dataset of 4,033 SARS-CoV-2 genomic sequences from PCR-positive cases between March 2020 and April 2022, accurately geocoded to residential locations. We identified genomic clusters with identical viral genomes and their number and size were larger for Omicron variant, confirming its distinct transmissibility. Invasion dynamics was well evidenced by the nucleotide diversity. Early in the invasion, nucleotide diversity was low and increased over time as variants diversified. New variant invasion was characterized by rapid increase in nucleotide diversity, due to co-circulation of old and new variants, followed by an abrupt decrease in diversity implying a complete replacement. A sharp increase and decrease of diversity within neighborhoods showed a rapid invasion of the Omicron across neighborhoods. We observed an exceptional peak of diversity within the same buildings during Omicron invasion, suggesting its stochastic transmission. This study supports the data-driven public health strategies to control and predict epidemics.