Relieving letter from previous company

Dear all,

Am working for financial services company for last 1 year, I have joined the company without relieving letter since the current company asked me to join within 10 days of time without relieving letter but the bank which i was working earlier had 3 months notice period.
I was ready to pay the notice period for days which i have not served but my boss didn’t relieved me saying i should serve 3 months at any cost. But at that time i could able to leave the good offer which i got with the current company and i left the previous org. informing my boss that i will not able serve the notice period .
I felt bad while leaving the previous company without proper relieving letter after serving truly to the company for 3 years. Now I planning to move to other company in a bigger role, will relieving letter first company/Bank will affect my future career.Pls help me with your kind advice.