Now here's the thing with mastering a class. Even if you put in all the time and effort practicing and getting really good at the class' mechanics, you still have to keep in mind one other factor that isn't always a constant; other players.
No matter how much time and effort you invested in one class, you'll find problems with it. Say you pit a seasoned Demoman with a new Scout, obviously the Demoman would win because of experience and practice. Now what if you pit a seasoned Demoman with a seasoned Scout, the fight will become more intense as it all comes down to 'who has the better mechanics?'
Now that I stated that, the classes which is hard to master are; Demoman, Heavy, Engineer, Medic, and Spy.