Towell and Shavlik (1994) introduce Knowledge-Based Artificial Neural Networks whose topology is isomorphic to a knowledge base of facts and inference formulae.
joint optimization leads to low-rank logic embeddings that outperform all other methods in the 0 to 30% Freebase training data interval.