I assume that in movie 4, people thought Sirus Black was after Harry, so he was given an OK to use magic outside of school. Remember in the movie, he inflated his aunt's sister or something, I forgot who that woman with the dog was. Harry ran away from home, hopped onto the knight bus, went to the Leaky Cauldron, and he met the minister of magic.