Don't get so hassled by such things; everywhere in every organization, these things happen. Usually, bosses tend to build a group around themselves with whom they interact closely. Not all bosses are like that, but in every office, there will be people who will try to put up a show that they are close. Anyway, identify them if there are any and steer clear. Never discuss your misgivings with the boss in front of other colleagues; you never know who is speaking behind your back. Now, you can't go back and repair anything if you have done it, but we can move forward. During recruitment, bosses do say these kinds of things. I have had this kind of experience. When I was joining an organization, I decided to join only if the organization thought about my career advancement too, etc., etc. Anyway, I soon learned that I would get work and everything, but I would have to learn to put up with one or two machos in my department who had no regard for the fact that I was their boss. Being a lady, they could not digest a woman boss. They occasionally bypassed me to go to my boss. I confronted them, and after a brief strife, made it clear that I was the boss and this was not going to work. I made sure that the communication channel remained open between us. You must do the same and without getting agitated; you must keep your cool.
Secondly, his impression that you don't get along with others. My sister is an engineer and very brilliant for her age, but she has a habit of at times calling a spade a spade. This doesn't work. Nobody likes being told that he or she is not smart. Remember that even the best organizations have good and not so good employees. The challenge is not getting work from those who are hardworking and self-motivated employees, but the challenge is getting work done from those who are laggards.
I will tell you something that I always tell my sister, and believe me, it worked.
1) Talk with your boss. Apprise him of your dilemma and all you feel. Unless and until you tell him how you feel, you will never know. Meet him often; I don't say every day, but I told my sister to. She never talked with her boss and visited him once a month, rarely. Now, your boss must know where you are.
2) For this instance, ask him for time to talk, see if he is in a good mood, and then go. Talk alone. Rehearse what you are going to say and what his possible questions might be and how you will face them. Tell your boss that if he finds you have something amiss, you are prepared to work hard to rid of it.
3) In the work area, remain cordial. No, I don't mean complacent, certainly not. But remember, if you don't like to insult others, don't like to.
4) Don't worry too much if you don't get work assigned immediately after this. Try to remain happy. Remember, everyone has a right to be in the spotlight now and then; you cannot steal all the place under the sun.
5) Last but not least, try not to appear smarter than your boss, especially if he has arrogance and ego.
The rest, you have not told why he says so about you or whether you were involved in any incident that gave him this view. If you had shed more light, it might have been more helpful.