maps4: move clear_refs code to task_mmu.c