It is nearly always worth buying the best quality you can afford. Cloth
does vary. There are cotton shirts and cotton shirts IYKWIM. I sew (not professionally) and some fabric is just not going to hold together no matter how great the seamstress is (though Anna is correct in that the sewing techniques/attention to detail etc matter also and some store bought clothes are very badly constructed. This is often true of designer clothes, so don’t be fooled into thinking a designer label will always mean good quality. I buy clothing on ebay sometimes and when doing so always stick to the two labels I know will be good quality/fit. The rest just isn’t worth the risk.)
So far as footwear goes, my late husband purchased me a pair of Wellington boots (some 12 years ago now) and I was shocked at how much he had spent. Far more than I would ever have thought needful for a pair of wellingtons. However.......12 years later, those boots are as good as new and I wear them on a daily basis (dog walking) in the winter. I don’t doubt that they will be lifetime boots. He absolutely got what he paid for (and proved me wrong

) which was excellent quality. Had I bought cheap boots I would likely have spent as much, if not more money, over the 12 years replacing them each year and griping about them letting water in or being uncomfortable etc.
Always get the best footwear you can afford. If I ever came into a sum of money, the first thing (or close to the top of the list at any rate) I would do, would be to go to a proper shoe shop and buy the absolute best quality shoes going for my children and I. It really does make a difference.
I agree with the thrift store frequenters. I have picked up plenty of virtually new, top quality clothing for the children and I at bargain prices. I would far rather spend what we have by getting something of higher quality there (but at a fraction of the original cost) than buy something ’new’ but poor quality and construction.
Women can perhaps get away with buying cheaper quality clothing (for some items) in view of the changing fashions and (for some, not all) throwaway mentality, but men’s clothing is often far less driven by passing fashions and as such it is worth buying clothes of a higher quality which are made to last. Good quality cloth and construction means an item can be more easily altered or repaired in the future (if need be) and give the wearer many more years. If I ever set up as an alterations seamstress, I would target men who buy top quality clothing.
The absolute best option of course, for those who can afford to do, is to go to a tailor and have your clothes made to fit you, from cloth of your choice.
Quality doesn’t have to come with a huge price tag, but my motto tends to be, if I can’t afford/find quality, I’ll do without.