I'd look at him blankly, pretending not to understand his "joke" until he felt the need to explain it. Then, a simple "Oh," and turn away.

He'll understand exactly how lame his attempt at humor was, and you'll come off as being above such foolishness.

It's always a bit hard for me because, in my heart, I LOVE foolishness.