As far a size goes, youd probably want 3 'rooms', one general area to sit and eat in if its raining, and 2 sleeping ones. In my experience, if a tent manufacturer says its a 5 man tent, its really only comfortable with 3, and maybe squish one more in or a child :wink: . When they say how many people, they mean sleeping reall close side by side
Of course, the larger you get, the harder it may be to find a good spot to camp, but only if your doing 'true' camping in Nat Parks of Forests, most private campgrounds have flat, marked camping spots.
What you mentioned about build quality is too true, and is similar with anything, if you specialise, you get better build quality in what you specialise IN, but you also have higher costs and then of course higher prices.
My bests advice is take your time, even if you go back to the same store to see the same tent 2 or 3 times, considering your up for ~$500, you dont want to buy something and wish you'd gotten something different
(i've done this but thankfully with only an ~$100 tent