As a long time fan of film, it usually catches my attention when a character breaks down the 4th wall and talks directly to the audience. Some people get annoyed when film characters do this, since it can interfere with suspension of disbelief. However, I really like it, since many times it's used in a creative way. In general, it has been used with great effect in comedy movies, since these movies are seeking react against audience's expectations of what should occur in a movie.
In re-watching some of my favorite comedies, Groucho Marx is the first person to break down the fourth wall in Hollywood movies. Groucho often looks at the camera and talks directly to the movie audience. Many times, Groucho is making sure the audience is in on the joke of making fun of the rich or the authority figures within the movie. In the Marx Brothers movies, it has a great effect of drawing the audience in.
The comedians who have come later and have broke the 4th wall in their movies (such as Woody Allen and Mel Brooks) are paying tribute and expanding on this technique with great comedic effect.
Animal Crackers (1930) - Groucho "Pardon me while I have a strange interlude..."
Horse Feathers (1932) - Music Lesson scene
Slate has a great video highlighting various scenes from several decades of movies where characters are breaking down the fourth wall.
http://www.slate.com/blogs/browbeat/2013/03/07/breaking_the_fourth_wall_supercut_a_video_essay_we_re_sure_you_yes_you_will.html
If you need more examples, this website that has a list of films that break down the fourth wall: http://www.artandpopularculture.com/List_of_films_that_break_the_fourth_wall
If you know of earlier movies or have favorite scenes from movies of characters breaking down the fourth wall, please share in the comments below.