In general, my answer to the above is no.
I have an ex-fiancé who will not be asked, and FI has an ex-gf who he has chosen not to ask (even though I know her and do not object to having her come).
But are all exes EXES just because you went out way back when? I'm thinking about like the people most of us dated or had summer flings with in high school--went out, sure, but no major physical boundaries (unless we count kissing) crossed. Should we exclude old friends who moved on long ago?