Sally Field on Burt Reynolds calling her the one who got away

Last year, while making the rounds to support his new memoir, actor Burt Reynolds discussed his failed relationship with actress Sally Field, whom he called the “love of his life.”

“I miss her terribly,” he told Vanity Fair. “Even now, it’s hard on me. I don’t know why I was so stupid. Men are like that, you know. You find the perfect person, and then you do everything you can to screw it up.”

But despite Reynolds’ public pleas toward Field, the two haven’t spoken very much in recent years. “We don’t really talk to each other, no,” Field told EW and PEOPLE editorial director Jess Cagle during a recent SiriusXM town hall.

Reynolds and Field met while making the 1977 film Smokey and the Bandit, and dated for five years in the 1980s. Asked about Reynolds frequently saying Field was the one who got away, the actress responded, “Well, yeah.”

This isn’t the first time Field has talked about Reynolds’ public commentary on their relationship and how it ended. Speaking to PEOPLE last year, Field said she had “no response” to Reynolds’ proclamations.

“Any response I would have would belong to him,” she said.

Field’s latest movie, Hello, My Name Is Doris, is in theaters now. “The Sally Field Town Hall,” hosted by Jess Cagle, debuts March 17 at 2 p.m. ET on EW Radio. Watch a clip from the interview above to hear more about Reynolds and Smokey and the Bandit.