There is a theorem, called the Chomsky-Schützenberger Theorem, that shows self-embedding recursion to be the essential element that distinguishes context-free languages from finite-state languages.
it shows that a language is properly context-free, as opposed to finite-state, only if every context-free grammar for it contains at least one self-embedding recursive nonterminal.