Dear,
As you are in HR, the major concern is not with what others would think; the concern should be how to mold the situation. Learn people management. Yes, as you mentioned, it's your first job, and it's difficult at this stage, but you can make it by interacting with people who are close to the superior.
If you feel that the concern has hurt you, try to interact. There are two kinds of people in that situation: those who would make a hype of the situation and use it and others who would stay calm, taking the bitter experience. Try to be smart and make the situation within your control.
We humans have forgotten that all these behavioral traits are built by us humans again, so try to get some ideas or solutions to suppress such situations. Yes, as the saying goes, it's easy but handling the situation is difficult, yet it's not impossible.
All the best and make use of your smartness to get out of the situation.
Sandeep Korde