"The Dead" - question

What was the "secret life"  that Gabriel mentions several times towards the end of the story?

When I first read about this "secret life", I was very confused, but I thought that it could mean that they really were in love (many couples at the time this was set weren't due to arranged marriages/status marriages). However, by the second time it appears in this short story, I wasn't so sure that it was a happy "secret life". I thought that it could be saying that they really hated/disliked each other, and maybe even lived separate lives under the outward-facing image of a happy marriage.

After thinking about "The Dead" for a little while, I am still very confused about what this "secret life" really is, but I have a couple more theories. One of which is that maybe their "secret life" is their history together; perhaps they were lovers first and got married out of obligation, or that there is some other complication (or even several complications) in the past that Gabriel can define as a "secret life". My second, and final, theory is that their "secret life" is nothing so dramatic as lost love or a complex history, but instead is just that they have a life away from their young children, the elaborate social gatherings, and his aging aunts; a life just as a married couple, however unhappy or happy they might be.