The skeleton is one of the most structurally and compositionally diverse organ systems in the human body, depending on unique cellular dynamisms. Here, we integrate prospective isolation of human skeletal stem cells (hSSCs; CD45-CD235a-TIE2-CD31-CD146-PDPN+CD73+CD164+) from ten skeletal sites with functional assays and single-cell RNA sequencing (scRNA-seq) analysis to identify chondrogenic, osteogenic, stromal, and fibrogenic subtypes of hSSCs during development and their linkage to skeletal phenotypes. We map the distinct composition of hSSC subtypes across multiple skeletal sites and demonstrate their unique in vivo clonal dynamics. We find that age-related changes in bone formation and regeneration disorders stem from a pathological fibroblastic shift in the hSSC pool. Utilizing a Boolean algorithm, we uncover gene regulatory networks that dictate differences in the ability of hSSCs to generate specific skeletal tissues. Importantly, hSSC lineage dynamics are pharmacologically malleable, providing a new strategy to treat aberrant hSSC diversity central to aging and skeletal maladies.