In the US, I presume that everywhere will take a credit card, and I back that up with around $100 in cash for those random "just in case" moments.
In Japan, it's basically the opposite – I try to estimate my daily expenses (not including lodging or any sort of "big ticket" tours/tickets/etc), multiply by the legnth of trip, then get that much cash in Yen… Actually, I used to pad that figure by several hundred dollars, assuming I'd run into one of those "gotta have" gift items you can't get back in the US.
(Un?)Fortunately, these days, you can basically get everything you can find in Japan (apart from certain food products) here as well, so I just go off a basic daily food, drink, transport, and sight seeing budget when I'm on vacation, or a slightly smaller version of same when travelling on business.
As a rough guide, imagine how much money you'd need bumming around NYC by yourself, then convert to Yen.