WARNING: Spoilers for The Haunting of Bly Manor.
In the final episode of The Haunting of Bly Manor, Dani and Jamie meet up with Owen to discuss the past; he tells them that the kids have forgotten everything that happened in Bly, but the adults still remember. Flora and Miles can only vaguely remember Bly Manor as the place they spent their summers. The explanation for why Owen and Jamie remember everything about the ghosts while the kids don’t might go beyond mere age difference, and probably has more to do with how possession works in the show.
The Haunting of Bly Manor has its own rules on how ghosts and hauntings work. For example, the ghosts of Bly Manor have no faces, and anyone who dies in the house or the grounds nearby can't leave. This is also true for possession. In episode 2, “The Pupil”, Miles asks Father Stack how a ghost can possess someone, and learns that the ghosts need the permission of the host to enter their body. Once the living person allows a ghost to possess them, they are “tucked away” into their own memories and minds.
Later in the season, it’s revealed that Peter Quint and Rebecca Jessel are dead and repeatedly convincing the children to allow them into their bodies. The children don’t fight this because, by being “tucked away”, Miles and Flora can see their parents and relive memories of when they were still alive. However, because of how possession works, Miles and Flora have no idea what is happening outside in the real world. The more time they spent tucked away, the less they remember. The kids start to fall ill and act confused, but by then, their wills are weakened.
Ultimately, Peter’s goal was to take over their bodies forever so he and Rebecca could start a new life together. He could only do this if Miles and Flora gave up their minds and bodies completely. By spending so much time “tucked away” and possessed, it’s likely that Miles and Flora’s minds became fragile, and losing their childhood memories was a side effect. Even as an adult, Dani struggled to control the Lady of the Lake inside her, eventually losing herself and her mind in the process. The only ones who weren’t affected by this were Owen and Jamie.
Possession in The Haunting of Bly Manor is deeply connected with the host’s mind and memories, clearly causing a ripple effect. Since Owen and Jaime never were “tucked away”, they can more easily remember the events that happened while the children barely held onto their memories. While it’s sad that Miles and Flora don’t remember Hannah, Owen, Jamie, and especially Dani, it’s also a good thing that they were able to start a new life in a new country without all the sadness and grief tied to the events in Bly Manor and its ghosts.