Are you empowered to take a decision in this respect? What nature of work you people are doing in your office, only admin.?
Do you have an official tea time schedule during working hours? If not consider imposing one say with 10 minutes break. And for any violation, issue individual memo with a warning of cutting half a day leave. Check any clause in your Standing Order, if any in vogue. If not draft out one model SO and introduce where such matters could find place.