How much time did Rick’s group spend at the prison on The Walking Dead, and why did they have to leave? The prison was the main base for Rick Grimes (Andrew Lincoln) and his people during seasons 3 and 4. They were on the road throughout season 1, before holing up at the Greene family farm in season 2, which they had to leave behind at the end after a horde swept through it. Since the prison was their base for a season-and-a-half, the prison was the show’s most important location until places like Alexandria, Hilltop, and the Kingdom became their new homes.

After the farm was overrun by zombies, the group traveled until they came across a prison, which was being inhabited by a small group of convicts, including Axel (Lew Temple) and Oscar (Vincent Ward). As much of it was infested with zombies, Rick and the others took some time to clear out the cell blocks with the intention of making it their base. And since it was surrounded by walls and fences, it was determined that the prison would be a good, secure location where they could feel at ease (for the most part). The prison and the town of Woodbury served as the two main settings for the show throughout season 3 and the conflict with the Governor (David Morrissey). So how long were they really at the prison?

Related: The Walking Dead: What Happened To Morgan After Season 8

The Walking Dead season 3 took place over a few weeks, which lasted from May to June 2011. Season 4 picked up after a time-skip that jumped the show forward by a few months. At this time, new characters like Bob (Lawrence Gillard, Jr.) and the Samuels sisters had joined the fold. Altogether, they were at the prison for roughly seven months before it all came to an abrupt end. The prison had to be abandoned with the Governor and his men arrived to seek revenge on Rick’s people. The Governor brought with him a tank, which plowed through the fences and blasted holes in the walls. The struggle ended in the deaths of Herschel (Scott Wilson) and the Governor; a horde of zombies descended on the prison, forcing them all to leave.

The Governor shooting a gun

Thanks to the tank, the prison was left in bad shape, and even attempting to repair the damage was out of the question, especially when taking the circumstances into consideration. Clearing out the zombies again and rebuilding the fences would have been too demanding of a task. After all, Rick’s group was scattered by the Governor’s attack and it took the entire back-half of season 4 for all the survivors to find each other again. The situation gave them no choice but to leave the prison behind. Once Alexandria was discovered in season 5, there was no point in going back.

The prison was put behind them a long time ago, but it remains an important part of The Walking Dead's past. So many meaningful, game-changing moments happened there; the prison is where they met Tyreese’s group, where Rick first encountered Michonne (Danai Gurira), and where Lori (Sarah Wayne Callies) died after giving birth to Judith. The Walking Dead's popularity skyrocketed during the group's time at the prison that the location became a defining aspect of the fandom; it's even used as the primary location for Universal Studios' Walking Dead attractions to this day.

Next: Walking Dead Timeline Explained: When All 3 Shows Take Place (Each Season)