Redditor gajus0 has created an extremely thorough guide to filming locations in London.
The interactive map brings together IMDB locations from tons of films that are either set in London or just happen to use it as a backdrop for, say, an ambiguous future dystopia.
You can click on an individual location to see all the movies that were shot there, and then click on a movie to see all the London locations it used.
Films that are in the 1000 Greatest Movies Of All Time (according to IMDB, anyway) are highlighted in gold, so you can pick out the classics from the clangers.
In any case, have fun nerding out: check out the map here.