What Ernie Hudson Learned About Marriage After Arguing with His Wife Over Toilet Paper (Exclusive)

Ernie Hudson is opening up about the secrets of his successful marriage to wife of nearly 40 years, Linda Kingsberg.

“I love her for who she is and not who I want her or think she should be,” the Ghostbusters: Frozen Empire tells PEOPLE. “So I just trust her to be herself and if she's happy, then I'm good.”

He came to this realization years ago when they had an argument “over the toilet paper roll, which way it rolls,” Hudson recalls. Though they bickered, Hudson let it go.

“I'm like, ‘I don't care. If it makes you happy, I'll do it that way. If me opening the door is an issue for you, then I'll open the door.’ What am I going to fight over? I'm good,” continues Hudson, who has two children with Kingsberg and another two from his previous marriage to Jeannie Moore.

Hudson and Moore wed when they were “kids,” as the Quantum Leap star puts it. “I was 18, she was 16. The marriage ended, the kids came with me, which I was surprised by. But I just felt that I never wanted to do that again.”

Then he met Kingsberg. “We were together for basically nine years before I finally said, ‘You know what? We're together. So let's just go ahead and marry,’” says Hudson, who tied the knot with her in 1985.

Since they’ve been together Hudson says he has learned to make compromises in other areas, especially when it came to friendships he had with females.

“I like women as friends. I find women more interesting to talk to because they'll talk about their kids and they'll talk about emotions and feelings and things,” he says.

He says “most” of his friends were women, but “it wasn't the best thing for my marriage. Even though I can explain this relationship, I'm more concerned about [my wife] now and I know that she's my friend now. And I can't spend two hours on the phone with my best girlfriend and expect her to understand.”

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories. 

“If it was turned the other way around, would I feel?” continues Hudson.

The actor has been making headlines lately after appearing in a tight black t-shirt at a Ghostbusters: Frozen Empire photo call in London March 21.

After pictures of his sculpted physique popped up online, users on social media were quick to point out how fit the actor is.

Hudson joked with PEOPLE that his wife’s schedule has changed thanks to all the head-turning he’s doing as he approaches 80 years old.

“She travels with me now everywhere,” he says. “She never used to in the past. She was like, ‘Okay, I'll see you later.’ Now it's like, ‘No, we're going together!’”

For more on Ernie Hudson, pick up the new issue of PEOPLE.