Founded on December 5, 1933, the bar's most famous patrons were Ernest Hemingway and the infamous rum runner Habana Joe. The original location at the time Hemingway frequented Sloppy Joe's is a few doors down to the west, just off Duval Street, at 428 Greene Street, and is now called "Captain Tony's Saloon".

The bar went through two name changes before settling on Sloppy Joe's with the encouragement of Hemingway. The name was coined from the original Sloppy Joe's bar in Old Havana,that sold both liquor and iced seafood. In the Cuban heat, the ice melted and patrons taunted the owner José (Joe) García Río that he ran a "sloppy" place.

The bar is the site of the Ernest Hemingway look-alike contest, started in 1981. It is well known as a tourist attraction, with live bands and slushy drinks. On November 1, 2006, it was added to the National Register of Historic Places.