But soon after the family started to unpack, things changed again. Queen Elizabeth died, just as Kate Middleton and Prince William landed in the dwelling. “It was not exactly the settling-in period they had hoped for,” a source said of that period in the Waleses’ lives.

According to People ’s reporting, the two families were asked to move on “earlier this summer” and “have since departed in preparation” for William, 43, and Kate, also 43, to relocate their three children, Prince George, 11, Princess Charlotte, 9, and Prince Louis, 6, from their current Adelaide Cottage in England.

Kate Middleton and Prince William will be moving house ‘later this year,’ a spokesperson at Kensington Palace confirmed to PEOPLE. The couple and their children — Prince George, Princess Charlotte and Prince Louis — will be settling into an 8-bedroom property in Windsor,