Erik, welcome to boating and HRCC. Look for the magazine Boating On The Hudson, it will have articles and ads on places to eat along the river. Or just stay tuned, others will chime in. Since you are based in Jersey it will be cheap for you to fill up with gas and come up hte river, but you may have to learn the tides in order to get back home.
Newburgh would be a great destination for you, they have about 7 restaurants to choose from with all sorts of cuisine.