Meaning "shabby," the phrase down at the heels once applied specifically to a shoe that had been worn to the point that he heel was worn down. When describing a person, it referred to someone wearing such shoes.
"I must be the first to congratulate you on the acquisition of my old shoes. They will be very easy in the wearing to you, though they pinched my corns a little at first. I dare say they want new soling, and perhaps they are a little down at the heels; but you will find some excellent cobbler to make them all right, and will give them a grace in the wearing which they have sadly lacked since they came into my possession."
— Anthony Trollope, The Small House at Allington, 1862
The state of one's shoes comes up in a number of examples of literal and figurative English. The adjective slipshod, for example, can mean both "wearing loose shoes" and "poor in execution" (as in "a slipshod attempt"). A person who is well-heeled, on the other hand, is one who has plenty of money.
Down-at-the-heels has become established to the point of describing things beyond people and the state of their footwear.
Mare of Easttown trundled onto TV looking like a familiar type: a grim crime show with an anthropological eye, one of those murder mysteries where the journey is supposed to matter more than the destination. Mare Sheehan (Kate Winslet) stars as a detective in a down-at-the-heels Pennsylvania town with a personal connection to just about everyone in her jurisdiction.
— Willa Paskin, Slate, 16 May 2021