If a clone can be stronger than the other eleven without Unity, then it is likely that he would infect and absorb the others to grow stronger. He knows that it is highly possible.
This is one of the two things he is worried about. He is not worried about the Usurpers that used to hide from him but will have to come out now to defend the void universe.
It is either that they do that to preserve their strength or give up on the void universe and leave it. Either way, they too are prey for him. Not contenders.
He is also not worried about the Beyonder like the god of power and the god of soul. Apart from his clones, only the three watchers and the overgod are worth some caution.
He has noticed that even though the void universe has become weak, the three watchers and the overgod haven't experienced any reduction in their power at all. It means that they are not like Usurpers, and their power is independent of the void universe.