Take any phenomenon, no matter how complicated, and you can reduce it to an application of a rule. Does this prove that somewhere in the Library of Babel there is a book that states the rule for everything? Or at least prove that it is impossible to prove that a given thing is not in fact the application of a rule?
On the other hand, I can take any notion and explain it as the product of any function. 57+68=5, if by + I mean ⊕, which for all I can explain "+" it very well might. That is, I can provide no principle according to which a given rule should not be applied in a special way in an unanticipated case, precisely because by hypothesis I have not yet considered this case. What this seems to prove is not that any conclusion really might be the correct application of a rule, but rather that it is useless to think about knowledge as application of a rule according to principles.
But perhaps I can still say that knowledge is having a rule which I know how to apply, if "knowing how" does not reduce to following a rule. It is experiential. I don't need a rule to tell me how to follow a rule, and if I could get one it wouldn't help me anyway, since I would need a third rule to tell me how to follow that one.
It seems paradoxical and anticlimactic to put "knowing how" at the heart of "knowing that." Can knowledge just be a highly recursive technique? But this would not explain what it means to "have" the rule in the first place, in order to know how to apply it. It is not enough to say that I have contrived a tool for referring out of arbitrary materials, since this would imply that I had some "private" feeling for the reference before I shaped the tool.
Anyway, I just started reading Kripke's Wittgenstein on Rules and Private Language, and wanted to post my initial thoughts and expectations. I want to return later after I've finished the book.