The information gain of a feature then is simply the difference between the total entropy of the instance base and the entropy of a single feature: The information gain G(D, f ) is the reduction in uncertainty in D we expect to achieve by learning the value of the feature f .
knowing the value of a feature with a higher G gets us closer on average to knowing the class of an instance than knowing the value of a feature with a lower G does.