I think you made the right decision for you, Brandi. I don't think it would have been wrong if you had gone to the funeral, though, if you could do it discretely.
In the other part of my life, I've worked closely with quite a few funeral homes and attended dozens of funerals over the last 15 years or so. I'm writing from that experience.
Unless the person is quite elderly, or has been shut-in for a long, long time, or has a disability that seriously restricted their ability to be out in the world, most funeral services have a wide variety of people attending. They don't all know each other. Many people will be there who may hardly know the deceased, but have accompanied their spouse or a friend who did know him/her.
If the death was sudden, it would be very unusual for members of the family to be greeting people at the door of the funeral home or church when people arrive. If the deceased had been ill for a long time and was on good terms with his/her family, it's more likely that family members will be greeting people when they arrive. You might need to make a judgment call about this, but it would be unusual for anyone greeting folks as they arrive to also ask questions about why someone is there.
You can always call the funeral home and ask how many people they're expecting to attend the service. They'll tell you if it's really just a small, private family service, or if they're expecting a couple of hundred people. If the service is being held in a church, there will usually be a lot of people there. If the whole service is being held at the graveside, fewer people will attend. Funeral homes are asked this very question several times a day. Estranged family members, former spouses and other people who have had a difficult relationship with the deceased, or with members of his/her family, often call the funeral home and ask this question. There's no need to feel conspicuous for asking!
There's usually a guest book available for signing. I wouldn't sign it, if I was at a funeral for a client. There's often a gathering after the service, too. I wouldn't attend. That's where questions about how someone knows the deceased would be most likely to be asked. And I wouldn't go to the burial if it's happening after the service. Again, more than half of the people who attend the service also won't go to the graveyard, so there's nothing conspicuous about staying away. Going, however, could raise questions.
But if there will be more than about 80 people at the funeral, to arrive at the funeral home or church only a couple of minutes before the service starts, and to leave when it's over, shouldn't be a problem. Many other people will do the same thing for different reasons.