If we don't get rid of deposit insurance the banks will continue to be able to inflate the money supply

We can say no more than that the banks might collapse if there is no deposit insurance, so then (we must presume) it prevents something worse, but it is not defensive because freedom is worse than nothing. If deposit insurance is not removed it enables the privileged banks to inflate the money supply.