Fermions coupled to Yang-Mills matrix models are studied from the point of view of emergent gravity. We show that the simple matrix model action provides an appropriate coupling for fermions to gravity, albeit with a non-standard spin connection. Integrating out the fermions in a nontrivial geometrical background induces indeed the Einstein-Hilbert action and a dilaton-like term, at least for on-shell geometries. This explains and precisely reproduces the UV/IR mixing for fermions in noncommutative gauge theory, extending recent results for scalar fields. It also explains why some UV/IR mixing remains even in supersymmetric models, except in the N = 4 case.