You should have asked the banker if it a debit card or bank card which will expire if not swapped or presented.
PAN card is issued by the Govt Authority for all over India and valid across India.
Check PAN Card has Number or not with your Photograph and Hologram Symbol with your scanned signature & DOB.
There are many Useless educated illiterate you'll find in your life. Meet the Branch Manager and ask him to give the reason in written , for not accepting the deposit .
You need to ask them reason else people will fool you.
Also there is another option that can help you to deposit this money without using PAN but has to deposit 40,000 (or amount less than 50,000) in a single time in a day and rest of amount can deposit another day.
Now go to option 2 that i have suggested you above and speak to the one who provided this PAN number to you to check this again .. There might be some misunderstanding or miscommunication ..