While many fans know Thanos as Marvel's premier villain, Iron Man has believed since the early days of the Avengers that Kang the Conqueror is the heroes' most truly terrifying opponent. In an in-universe verbal history of the team formed by interviewing key Avengers heroes, Tony Stark lays out why Kang the Conqueror genuinely scares him. The moment comes from 2010's New Avengers #3, by Brian Michael Bendis and Stuart Immonen.
Iron Man has known Kang for a long time. While the character appeared in his earlier form of Rama-Tut in Fantastic Four #19, Kang himself first appeared during Stan Lee and Jack Kirby's Avengers #8, when the classic Avengers defend Washington DC against the villain's technologically advanced arsenal. Iron Man plays a role in deflecting Kang's weaponry during their first encounter. While Tony disables Neutrino Missiles, Thor blasts the Conqueror's radiation back at him, expelling Kang to his original timeline.
This initial clash makes it clear that Kang is a threat, but the Conqueror's ability to manipulate time is what makes him truly scary. The Mad Titan may be Earth-616's most considerable threat when wearing the Infinity Gauntlet, but that's really true of anyone, while Kang has a similar command over reality through his technological genius alone, given his ability to manipulate the past to create whatever future he wants. In New Avengers #3, Iron Man says:
...we may wake up one day and all of existence will have been altered beyond recognition... or worse... beyond anything that a sane human being could actually live in. Think about that. And if you're asking yourself why the Avengers don't do something about it, we do. Every time. But the fact of the matter is that he has in his hands technology that I cannot fight. I'm the man who has the best technology on the planet. So if I can't beat him at his own game, then no one can.
Thanos is no joke when he wields the Infinity Gauntlet. However, without the Gauntlet, he doesn't have the power to wipe out the Avengers before they know it. In the comics, Earth-616 has Iron Man, Reed Richards, Hank Pym, Doctor Strange, Professor Xavier, and several other geniuses constantly keeping tabs on the Infinity Gauntlet and its Gems. Thanos is a nihilistic genius always looking for new power sources, and not a threat to be taken lightly, but his biggest plans involve elaborate quests that create opportunities to stop him. In contrast, Kang can change time such that no-one ever knows it was any different. Hank Pym, the Avengers' original Ant-Man, backs Tony up, saying: "Who knows how his actions have affected the world! The fact is, we'll never know. And that truly scares me."
This isn't just Tony and Hank's paranoia, either. While Kang is generally ruled by a sense of dignity and fairness, considering the Avengers noble opponents, he has on multiple occasions changed the past without warning. In Moon Knight Vol. 8 Annual #1 (by Cullen Bunn, Ibrahim Moustafa, and Matt Horak), Kang alters reality to be under his control, and is only defeated because Moon Knight is connected to his plans and retains awareness of how the world used to be thanks to his patron god, Khonshu.
Thanos is a threat that should never be underestimated, but he has no inherent power over reality beyond whatever cosmic artifacts he's able to acquire. In contrast, Kang's time-manipulation technology is of his own design, meaning he can never be meaningfully disarmed of his ability to alter the timeline. As Iron Man states, Kang is a truly unpredictable being wielding unimaginable power - as dangerous as Thanos may be, it makes sense that Tony Stark has lost more sleep over the Conqueror than the Mad Titan.