Dear friend,
I think that for reasons best known to yourself, you are a bit confused about your current employment.
In general, it is quite natural at employable age for every one to be constantly on the look out for a better job. At times a bird in the hand is worth two in the bush. Therefore, you should always have a comparative evaluation juxtaposing your innate desire for paid employment and your present job, your job satisfaction and growth potential in the hierarchy of the organization. Employment is only a means for living and not life in itself. However, whatever be the type of employment, whether self employment or paid employment, there are certain basic rules to be scrupulously followed. If you are scared or hesitant to observe the rules, you can not be successful in your career despite your high degree of common intelligence or employability. Work culture in Govt. organizations and private sector are totally different. Even in private sector it is not one and the same in every organization. You must have adaptability. You might have heard the proverb:" If the mountain will not come to Mohammed, Mohammed will go to the mountain " which simply means that if things are not going your way, you will have to adjust the way they are.
In particular to your present dilemma, if you are certain about the alternative job, leave the present one before you are confirmed; in case some time is required before which you are likely to be confirmed in the present job, then be prepared to serve the notice period or buy out it in order to make your separation formal and peaceful.