Superexchange dominates in magnetic topological insulators

It has been suggested that the enlarged spin susceptibility in topological insulators, described by interband Van Vleck’s formalism, accounts for the ferromagnetism of bismuth-antimony topological chalcogenides doped with transition metal impurities. In contrast, earlier studies of HgTe and related topological systems pointed out that the interband analog of the Ruderman-Kittel-Kasuya-Yosida interaction (the Bloembergen-Rowland mechanism) leads to antiferromagnetic coupling between pairs of localized spins. Here, we critically revisit these two approaches, show their shortcomings, and elucidate why the magnitude of the interband contribution is small even in topological systems. From the proposed theoretical approach and our computational studies of magnetism in Mn-doped HgTe and CdTe, we conclude that in the absence of band carriers, the superexchange dominates and its sign depends on the coordination and charge state of magnetic impurities rather than on the topological class of the host material.