So what happened to this small but scrappy studio that became a major player after gambling on Peter Jackson's mega-successful LOTR? To sum it up, ego and bad decisions, which were likely driven by ego.
I've been following the strange decisions and battles mostly made and waged by Shaye, which were big head scratchers. For the longest time, there was a lawsuit by Jackson that New Line may have stiffed him and his company, Wingnut Films, on profits owed from the first LOTR. Shaye took it public, and tried to take Jackson down, but really only made himself and New Line look bad.
The two recently settled and agreed to make two films out of The Hobbit, with Jackson producing and likely Guillermo Del Toro (Hellboy 2) directing. However, JRR Tolkien's estate filed suit, claiming they've received no money from New Line from LOTR's profits. And the battle continued.
Shaye directed The Last Mimzy, and spent a ton of cash on marketing. When that bombed, along with several other high profile pictures, he and Lynne's contract renewals began to look shaky, at best. You see, when they sold New Line to Time Warner, they were able to stay onboard to run the studio, much like the Weinstein brothers and their Miramax sale to Disney.
But like the Weinsteins, they soon found themselves in deep trouble with the parent company, and both parties (Shaye/Lynne and the Weinsteins) were soon unemployed from the companies they founded. The final straw: when the spectacularly over budgeted The Golden Compass, which tanked big time at the box office. (Why it won and Oscar for Best Visual Effects instead of Transformers, I'll never know.)
After Shaye and Lynne were let go, Time Warner folded New Line into WB, where it'll be nothing more than a production company, as mentioned above. As of right now, The Hobbit is again in limbo (well, semi-limbo), but maybe this move by Time Warner will right the ship again. Stay tuned!