The origin of signal flags is lost in the mists of time, but they were already in use during the Middle Ages.
The current International Code of Signals contains 40 different flags and pennants and the current NATO code a further 28, whereas a signal book of 1762 shows 26 in total of which only 20 are purely signal flags.
Probably the most famous use of signal flags was during the Battle of Trafalgar in 1805 when Admiral Nelson's Flagship "Victory" hoisted the message:
"England expects that every man will do his duty"

These signal flags are in the rigging of the "Tole More". You are welcome to try to read the message. It starts with the mast on the left.
Rigging of the schooner "Bill of Rights" is in the foreground.