The Hindu gods are sometimes depicted with moustaches, like Indra in the image below from 1820, as this is a sign of masculinity in Hindu culture. It is uncertain when the moustache became popular in India but it is possible that the Western moustache originated from Hindu tradition.
Romans, Greeks, Jews and later Christians and Muslims all favoured a full beard rather than a moustache, but the moustache was popular among the "barbarian" cultures of Northern and Eastern Europe. This stone head from Mšecké Žehrovice, Czech Republic, is from the late Celtic La Tène culture. This head was probably made around the time of the birth of Christ.
As well as the Celts, the Norse sometimes favoured moustaches as demonstrated by this carving which may be a depiction of the god of mischief, Loki.
The Anglo-Saxons certainly favoured moustaches rather than beards, both before and after they converted to Christianity, this is clearly shown in the bayeaux tapestry in which all the Saxons wear moustaches including King Harold Godwinson as shown in the image below.
In fact, the tash was so important to them, that it is even depicted on the sutton hoo helmet, lest the enemy mistakenly believe the Anglo-Saxon was clean shaven beneath the mask.
The moustache went out of fashion over the centuries though, being replaced by the proud beard or more commonly the clean shaven Christian of the West. It was not until the days of British Imperialism that we once again donned the soup strainers. The catalyst was caused by influence from India where the British had infiltrated the higher caste of the hindus and taken to growing moustaches in the Indian custom. The custom was encouraged because Indians were inclined to view clean shaven men as effeminate and in order to maintain an image of power in the eyes of the hindus, British men needed to maintain whiskers on the upper lip.
You can read more about the importance of the moustache to the British Empire in the following article.
How the moustache won an empire.
The arrival of vicious dictators such as Stalin, Lenin and Trotsky, who sullied the image of the Western moustache, may have caused the death of the modern moustache which is now worn only ironically by hipsters in Dalston and Williamsburg.
This cannot go on, I am doing my bit to bring back the Western moustache, without irony or apology, for neither charity nor humour. A tash is for life not just for Movember.