There are two approaches to ordinary language philosophy, one represented by Hobbes, the other by Ockham. According to the first, there is no problem at all with ordinary language. The apparent difficulties are the result of meaningless technical language (in Hobbes' day, the Latin of the schoolmen), designed for the defence of what is really absurd and untrue. According to the second, the problem is ordinary language itself, which is systematically misleading. Thus, Ockham argues our propensity to believe every name is the name of something is the source of all philosophical error (Summa Logicae 1.51)
Wittgenstein represents both views. In his polemics against mathematical logic and set theory, to be found in his mathematical writings of the early 1930's and in the Remarks on the Foundation of Mathematics, he takes the Hobbesian line. '"Mathematical logic" has completely deformed the thinking of mathematicians and of philosophers, by setting up a superficial interpretation of the forms of our everyday language as an analysis of the structures of facts. Of course in this it has only continued to build on the Aristotelian logic'.
At other times, he takes an Ockhamist approach. 'A clever man got caught in this net of language! So it must be an interesting net. ' ' Human beings are entangled all unknowing in the net of language.' ' In philosophy it's always a matter of the application of a series of utterly simple basic principles that any child knows, and the – enormous – difficulty is only one of applying these in the confusion our language generates.'
See here for all the Wittgenstein quotes.