The real question should be: Why did it not work?
Sounds like you had two issues:
1) Since it "did not work at all during the day" I would conclude that the unit was too small ( ie: 5,000 BTU vs 10k or 15k )
2) the second problem: "too cold for anybody in the aft cabin and hot up front at night" means that you did not get proper circulation, which might also involve problem (1).
--
My though would be to get a marine unit, that has "installable ductwork" to solve the circulation issue, and then get a unit that has more cooling capacity than the one you used.
As a WAG I would guess that 10k to 12k btus would be about right, but I would contact the mfgr of the A/C for proper sizing of the unit ( they may have a do-it-yourself calculator on their website.
A carry-on, without ductwork to improve the circulation will likely not be an improvement. ( You could try using a fan to add the required circulation, perhaps )
A properly sized A/C ( marine or non-marine ), with proper circulation, will work just fine. But you need to solve both issues.