I saw the other night on the Q, someone had a pair of fringed flare jeans on. The model was an older woman and the brand rep was "pushing" these pants and to be honest, I thought they looked absolutely "stupid" on her.
Yours is not fringed and you did say they were too short, I just don't know.
I think that if you "try" not to look younger with clothes, you will appear younger overall and not be so self conscious.
On the opposite spectrum, you do look old when you wear pants that are twice to three times your size. When you were tops that just hang on you and are way too loose. My mom was a medium and she had me buy XL pants and large and XL tops. Then I realized she got a lot older.