Meghan MarkleandPrince Harryare not pushing back onKing Charles' decision to evict them from their U.K. home,Frogmore Cottage.

A source tells PEOPLE in this week’s issue that the Duke and Duchess of Sussex are “matter-of-fact” about being requested to leave Frogmore Cottage and aren’t trying to overturn the decision. The source adds that Harry, 38, and Meghan, 41, are happy to raise their two children in California, where they moved in 2020 after stepping back as senior members of the royal family.

However, those close to the couple say the decision is “unfortunate” — especially in terms of security, as Frogmore Cottage in Windsor was a haven for the pair and their children, 3-year-old sonArchie Harrisonand 1-year-old daughterLilibet Diana, when they visited the U.K.

Frogmore Cottage.GOR/Getty

Frogmore Cottage

Last week, a spokesperson for Harry and Meghantold PEOPLE, “We can confirm The Duke and Duchess of Sussex have been requested to vacate their residence at Frogmore Cottage.”

PEOPLE understands that the couple knew about the move before the release ofPrince Harry’s book,Spare, in January.

Meghan Markle, King Charles and Prince Harry.Samir Hussein/WireImage

Meghan, Duchess of Sussex, Prince Charles, Prince of Wales and Prince Harry, Duke of Sussex

Prince Harryand Meghan lived at Frogmore Cottage from 2019 to 2020. The couple relocated from the two-bedroomNottingham Cottage, on the grounds of Kensington Palace in London, to the residence before the birth of Archie in 2019. After stepping back from their working royal roles in early 2020, they briefly stayed in Canada before settling in Meghan’s home state of California.

It remains to be seen whether the couple will receive a refund for part of the $2.9 million they spent on refurbishments and rental costs on the property over several years, as the time on their lease has not yet expired.

Meghan Markle and Prince Harry.Karwai Tang/WireImage

Meghan, Duchess of Sussex and Prince Harry, Duke of Sussex attend the National Service of Thanksgiving at St Paul’s Cathedral

Royal historian Robert Lacey tells PEOPLE that King Charles is making moves thatQueen Elizabethdid not.

“Charles is now the guardian of the institution. The King just can’t sit back,” he says. “It shows Charles has a steel that his mother lacked. The Queen could never bring herself to confront Harry as an adult.”

Prince Harry and Meghan Markle.ANGELA WEISS/AFP via Getty

Global Citizen Live, New York - Prince Harry and Meghan Markle

Can’t get enough ofPEOPLE’s Royals coverage?Sign up for our free Royals newsletterto get the latest updates on Kate Middleton, Meghan Markle and more!

It’s still unclear ifPrince Harryand Meghanwill return to the U.K. in May for Charles' coronation.

Buckingham Palace had no comment.

source: people.com