Yes, cashmere is expensive, but buying a sweater is more than just fabric.
Fit, color, style, design, and so forth are also part of the decision.
I have some Macy's Charter Club, yes it is thin, and does run a bit small in size, but some of the colors and designs are very pretty. Macy's has great sales. Sometimes the heavier multi-ply cashmere can be too warm or heavy if one is indoors.
I also have nice cashmere sweaters from all the major department stores, and QVC, but the nicest cashmere I ever saw in my life was in Scotland. Unfortunately I did not purchase as it was very expensive, but I always remember it. The company Gorsuch often carries very fancy cashmere sweaters but they are costly!