What is over fluxing relay?

Over fluxing relay is a part of generator protection. A generator is designed to operate at or below a max magnetic flux density. 

Above this design limit will cause overheating in stator core and stator winding insulation which may result in severe damage to the components.