Question: How do I determine the right size fuse for my wire?
This is Table VI from the ABYC E-11 Electrical Standard (click it to make it larger). It is for single conductor wires not bundled or sheathed together. The table is organized by jacket temperature rating.
UL1426 Marine wire should generally be 105C rated. Manufacturers like Pacer, Ancor, Berkshire, Cobra and others all build marine battery cable to UL1426 standards and most any chandler will have it.
The ABYC standard does allow you to go to 150% of the Table VI ampacity rating, if necessary, but if you do this always round down to the next size fuse rather than up. My personal preference is to size the wire correctly so that use of the "150% rule" is not necessary.
So, if you were using 2/0 wire, and it was outside the engine space, then you could use a fuse up to 330A @ 100% of the ampacity rating.
If you needed to go bigger with your fusing you could use the 150% rule and use 330A X 150% = 495A fuse, or rounded down to the next commercially available size.
What do manufacturers suggest?
Here are some minimum manufacturer suggestions for battery/starter cable from Westerbeke & Universal.
Universal / Westerbeke BATTERY CABLE REQUIREMENTS (wire length round trip)
2GA = 8'
1GA = 10'
1/0 = 14'
2/0 = 18'
3/0 = 22'
4/0 = 28'
It does not take much to have 10' of wire length even with batteries just a few feet from the engine. Keep in mind these numbers are wire length along the conductor, not an "as the crow flies" distance.
Data Table Courtesy ABYC E-11