In YC experience, 2 or 3 co-founders seems to be about perfect.
You need conviction in your own beliefs, and the willingness to ignore others naysaying.
Here's a good rule of thumb: don't worry about a competitor at all, until they're actually beating you with a real shipped product.
... you want to be proud of how much you can get done with a small numbers of employees.
If someone is getting every decision wrong, that's when you need to act, and at that point it'll be painfully aware to everyone.
Losing focus is another way that founders get off track.