With the background eraser tool selected, on this image, I selected the 75 pixels soft round brush, Limits were set to "contiguous"; Tolerance set to "65%"; and Sampling set to "Once". Beginning with the crosshair in the brown background, hold your mouse button down to get a sample and then, drag back and forth including across the fine details.  The brown color is selected as the target color as long as you hold down the mouse button, so you can drag over other colors without erasing them.  When using the "Once" option for sampling it is a good idea to release the mouse button and paint frequently. The background colors may be slightly different throughout the image and the target image is selected each time you begin dragging. Continue dragging and erasing until all traces of the bland brown area is gone and the blue sky shows.

Note how the Background Eraser kept the fuzz detail on the top of the hat in tack. It did mess up some other areas, particuarly were the background had tones similar to the foreground.