There is something very masculine about a man adjusting his jacket and sleeve cuffs. It's a movement that women don't usually do. Just think of James Bond and how he adjusts his cuffs. Don't worry about it!