There's a reason Boston is the #2 (former?) startup hub – lots of colleges, lots of cheap office space (well, when you throw in the metro area/"Metrowest corridor" out to Worcester… Public transit options are decent, if not perfect, MUCH superior to say the Bay Area. Oh yeah, and whenever you need it, it's an hour flight to New York.
When you're just starting out, looking to keep costs down, and build a product, Boston will tend to be a bit friendlier to your wallet, without a real "downside".
Unless you're a fashion/financial startup, you can always just move to NYC later, if you find that appealing. Otherwise, it's a huge risk/change unless you're already New York-based/have an existing network there.
If you had to choose between New York City and Boston to start a company in, which would you choose and why?