Improve time tracking overheads a bit