Hi Archana,
I think till now you must have spoken to your boss. Don't feel guilty. Remember one thing that no one is dependent on other in professional life in long run. If you have taken a decision of changing a job, that means you have thought over this before you decided.
In life we always learn many things from all the people around us, and specifically each boss in future also will teach u many things.
I do understand how you are feeling while leaving job, but you are not doing anything wrong. Respect your choice of new job and go with it.
congratulations and all the best.
Namrata.