To overcome this drawback, one can propose to take benefit from randomly initialised units, that are by design nonbiased.
it is common to face small target-datasets that contain too few data to learn such neurons from scratch.