Look up the town Millerton, it's Upstate NY. Small town a little on the alternative side, nice people. Train goes there. It's a longer ride from Grand Central, but the town is very pretty, has small streets that go into the surrounding nature, and, from what I know, nobody minds people shooting there. Millerton also has a very nice diner for after.
Or, of course, Hamptons. Especially right now the dunes around Amagansett. Light is amazing there. And Fire Island Pines, if just for that it's very liberal there.
edit: amagansett & fire island pines - just speaking from personal experience, we were fine to shoot there w/o permit as long as the shoot wasn't "professional", which we were told means if you just have a camera and no other portable "professional" equipment like bigger reflectors etc., and nothing commercial.
in pines people are pretty laid back though, we went to the town office and asked since once we had a bigger set-up, explained it was nothing commercial, and they were cool, even helped us choosing a spot on the beach.