Croton is divided by the RR tracks and route 9 with only parkland on the river side of tracks, along the length of the river (hey, not the worst thing). You can leave your dinghy safely at the guest dock at my marina, The Croton Yacht Club and walk over the foot bridge. On the other side of route 9 and the rr tracks are two places. "The Tavern" is kid friendly but stops serving at 9pm where it is more of a...tavern. Ocean House (doors away from the Tavern) is great but small, reservations are required.
Closer to Croton Point, you should be able to leave the dink at the Senasqua Boat Basin / Croton Sailing School. From there, walk through the "picture tunnel" (right there near the docks), up municipal place and to The Purple Monkey ice cream place. There is also Memphis Maes BBQ joint there (as well as a bank, pharmacy, bagle store and nail place...typical strip mall).
The only real attraction in town is "Van Cortlandt Manor", a cab-ride away or the Croton Dam (an awesome park), also a cab ride away (914)271-TAXI. All of the Village of Croton is very, very safe. Let me know if you would like more details.
BTW - I've dinghied to Croton Point several times and never had a problem...as a Resident of Croton, I would be pretty upset if I did.