John will always have this weird separate place in his heart for Ellen. Before Bill died she was one of his best friends and someone he could always count on to keep him in his place. After he got her husband killed, it got overshadowed by pity, and he knew she didn't need or want it at all. At this point, he wants to pretend what happened between them never happened and start with a blank slate, but now that Jo's here he knows it'll never happen. Still, it's comforting to know that they still have that same give and take they had before things went sour.