One Joseph Edmund Webb certainly thought so, and he tapped into this copious source of gas. However, the light was almost a side-effect: most importantly, the burning of gases helped avoid dangerous accumulations which might otherwise poison sewer workers or cause explosions. (Stinkpipes served a similar purpose, but couldn't always be tall enough to rise above buildings and only dispersed, rather than eliminated, the smells.) His creation, the Webb Patent Sewer Gas Destructor Lamp, had a great deal of success and the most famous example is in Carting Lane, London. It still operates today - although it had to be restored after being hit by a lorry.
However, Webb's product lined streets far beyond the capital. The first lamp was in his home town of Birmingham, erected in 1894; Sutton Coldfield quickly adopted the product; and soon the lamps could be found all over the country. Sheffield had a large number of them (some of which are still in place) while Whitley Bay and Monkseaton have ten of their original seventeen. I came across another example in Durham. It no longer operates, having apparently been extinguished during the wartime blackout, but still stands as if ready for use. Indeed, such was the success of Webb's lamps that the Birmingham inventor was able to open offices at Poultry, City of London.
Further reading: Nik Morton's excellent article on Webb's lamps.