I understand what you said. I love when people disagree with me because I always learn from them. However, what happen with this employee is completely different.
For example, the person was 30 mins so I asked this person to be on-time. This person then said "why are you targeting me when so-and-so is always late as well?".
This person is only 5 weeks in but has an attitude to deflate responsibility and bringing others into the discussion.
Was this person right? I ask this because the other thing you need to think about is how this action will be perceived by the other employees. Will they think you treated this person unfairly? Will they resent having to take over this person's workload until you find a replacement? If so, then you may want to consider if getting rid of one person is worth poisoning your relationship with the other employees.
Ultimately, the decision is yours alone. Make the decision and won't worry about validating the decision with a random internet audience.
I'm not an employer, so maybe I do not have a good perspective on this. But, as a business owner you probably depend on this business financially, perhaps you have family who depends on the business, and perhaps you have other employees who like working there. So it seems the choice is between upsetting one employee, or you can upset all those other people by not having your business run as efficient as possible. So I think your guilt would be multiplied if you upset all those other people.
You didn't state what the personal issue is, I hope it's not something that's protected under discrimination laws.
The only thing I would add, is rather than saying the issue is with team fit, take ownership and admit that the problem is that the two of you don't get along, and give your rationale that the conflict simply reduces your effectiveness running the company.
Honesty and doing things right is the best way to avoid guilt and to keep the bridge of a professional relationship intact. This is a person who might be a good fit at a later stage in the company's development or someone who might feed a better candidate your way.
Heck, one day, the employee might offer you a job.
I understand what you said. I love when people disagree with me because I always learn from them. However, what happen with this employee is completely different.
For example, the person was 30 mins so I asked this person to be on-time. This person then said "why are you targeting me when so-and-so is always late as well?".
This person is only 5 weeks in but has an attitude to deflate responsibility and bringing others into the discussion.