Students that are removed from all groups, appear in the list of unassigned students. You can delete the account of such a student by navigating to the page of the student. Click the icon X 'Delete student from this site' and confirm.