This is definitely a tough spot to be in. But as a previous poster stated. It's all in the wording when it comes to refunds. And it did appear that your wording wasnt the greatest. If you had stated non refundable maybe it would have been different. For me if I make a commitment I do my absolute best to follow through. But if I couldn't for whatever reason I would have expected a refund. Simply because your wording is open to interpretation (if that is the wording you used in your invitation) whether it was this type of business or any other. And it does appear that the two of you settled on an amount that was less than a full refund. If my math is right you kept over 10%. And you also have to think that if you refused the refund would the person then possibly somehow manage to attend and potentially ruin the experience for the others because you essentially forced him to be there or lose x amount of money. As you said you don't want someone there who doesn't want to be there for whatever reason.
Just my 2 cents
Additional Comments:
Also to Kennedy's comment on blacklisting him, To attend the event you've had to have seen at least one of the attending ladies. so we can safely assume he's seen 1 if not more of them. I would say that the ladies attending are the elite in the industry so they already know what he's like. For all we know the reason for his cancellation was that his daughter was in a car accident and in critical care. ( and as the wording of the refund is open to interpretation he thought he could get one). So to assume he cant be trusted and blacklisted is a little extreme