Hello Neemkala,
Many have put forth their views on your problem, as well as the possible solutions.
As long as the results are being shown, late-coming shouldn't matter. However, this flexibility shouldn't be taken for granted, which may actually happen if you don't sit down with the full team, discuss, and try to understand the possible reasons. Be prepared for some pleasant surprises during the chat.
In a way, it could also be a learning process for you as an HR professional. After all, we all learn throughout our lives, don't we? Every story always has another side. Get to know their part of the story and then, if necessary, formalize the aspect of latecoming with other ground rules that could be different from those for the rest of the company. When the new rules are in place, inform them to follow. Usually, anyone who is part of the solution will hesitate to bypass it. After all, it's humans who make the rules, not the other way around. So the rules need to streamline the function rather than hinder.
Regards,
TS