Being shorter or taller depends on details of the diet. Vegetarians can obtain a balanced diet, with sufficient complete protein, vitamins, minerals and micro-nutrients. It's trickier for vegans but it can be done.
Richard H. SteckelA rich diet does lead to earlier puberty but to my knowledge it does not student the adolescent growth spurt. Early pregnancy, before reaching adult height, does slow or stop growth.
Richard H. SteckelYoung children from Guatemala who are adopted into the United States gain considerable height relative to the standards. So the height difference cannot be genetic.
Richard H. SteckelThere are some effects of the biome. Compact bodies tend to conserve heat relative to linear ones. This can affect average height as well as body proportions. The study participants Europeans and Americans do not fall into the range where this effect is powerful, however.
Richard H. Steckel