If you have specialized qualities that are valued in your industry, such as in the shipping industry where people efficiently handling customs jobs are well-known, or in the pharmaceutical industry where good medical representatives have strong contacts with doctors, you may be recognized by name, work style, and attitude in similar companies. If such a person decides to change jobs, it becomes news among related companies. Similarly, when executives in higher positions consider changing jobs, it becomes news, and if they engage in any mischief, it can be troublesome for all involved.
If you feel you fall into such a category, the HR who is now annoyed because you backed out may spread negative words about you. Since you backed out at the last minute, which is certainly not a good thing, anyone would become uncomfortable and might think of teaching you a lesson.
Of course, such thinking or attitude (both yours and HR's) is totally unprofessional, and nobody should behave like this. I am 100 percent sure the concerned HR has just threatened you to frighten you, and in reality, will not act as he has said. This is temporary anger, and he will forget the issue once some time passes and he becomes active in selecting another candidate for your position. He may even thank you if you meet him coincidentally on any occasion or during travel, saying, 'Good that you didn't join; I got a much better person, and my bosses have appreciated my excellent job of selecting the perfect candidate.' (These will, of course, be sarcastic remarks to make you uncomfortable.)
To conclude, I don't think, in whatever capacity, any HR can spoil anybody's career with such tactics. So relax!
Suggestion
Meet the annoyed HR personally and very politely give some good reasons for backing out, which should sound genuine. Apologize and say sorry (again, it should sound genuine and from the bottom of your heart). I am sure if he is planning to disturb your career, he will become calm.