Star Wars confirms Darth Vader knew how to destroy the Death Star before Rogue One. Ever since A New Hope premiered in 1977, the Death Star's weakness has been one of the film's most-discussed talking points. Audiences had a hard time believing the well-resourced Empire would let such an obvious design flaw (a thermal exhaust port right below the main port) slip past them. For years, it was perceived as a sign of the Empire's hubris, illustrating they underestimated any who dared oppose them. It wasn't until 2016's Rogue One when an official, in-canon explanation was given: Galen Erso sabotaged the Death Star during its development.
Even with Galen's assist, blowing up the Death Star wasn't an easy task. The Rebels' attack would have been doomed had it not been for Luke Skywalker using the Force to guide a proton torpedo down the exhaust port. So, this was something that required a great amount of skill and a little bit of luck to pull off. And, as it turns out, Luke's father knew how to destroy the Death Star decades before it was even built.
In an episode of The Clone Wars season 4, titled "Plan of Dissent," a group of clone troopers discusses how they're going to attack an enemy ship. During their talk, Fives recaps Anakin Skywalker's heroics at the Battle of Naboo, when he destroyed the Trade Federation droid control ship. Knowing the key is blowing up the main reactor from the inside, the clones use that as a source of inspiration for their own plan. It also shows Anakin was aware how a technological marvel like the Death Star could be vanquished.
This fun connection between The Clone Wars and the movies not only illustrates George Lucas' love for making his Star Wars projects rhyme, it also adds further context to Vader in A New Hope. As Imperial officers boasted about the Death Star's immense power, Vader remained unimpressed, criticizing the space station as "insignificant" when compared to the Force. Obviously, nobody knew it back in 1977, but now that line could be interpreted as Vader drawing from his past experiences. He wasn't proud of it because he knew how easily (again, relatively speaking) the Death Star could be toppled. He had done it before, and so did some of his clone troopers. The Death Star is of course larger than a single ship, but the basic principle remains.
Rogue One confirms Vader didn't know Galen sabotaged the Death Star; he ordered Krennic to investigate if it had been compromised. But even without that bit of knowledge, Vader may not have felt the Death Star was a guarantee to work. Especially in possession of the Death Star plans, a team of Rebel saboteurs could have come on board and attacked the main reactor from inside the space station (not necessarily sending fighters through the trench). What the Rebels came up with for the Battle of Yavin was just one possibility - arguably the best option they had given the time-sensitive nature of the mission. Maybe if they had more time, they could have done something different to hit the main reactor.