My dad's argument against football when we were kids (1980's) was that we were still growing, and the potential for neck/spine injuries was high. Weren't allowed to play til high school. By that time, my athletic "prowess" was obviously not good enough to make a team. My brother, however, went out for JV in 10th grade and started o-line right away. He went varsity next year and ended up starting for two years, and got a couple "All WNY Team" nods.
Point here is you don't necessarily need to start all kids at an early age for them to eventually excel. I would say my dad mitigated the risks quite well.