The key parameter in the FOFE method is the forgetting factor, which is responsible for determining the degree of sensitivity of the encoding with respect to the past context.
the choice of a good value for the forgetting factor could be tricky since both small and large forgetting factors are offering different benefits.