Fresh lemon juice works. Just soak a Q-tip and apply to spots 2x a day. This will exfoliate the area to the point that you'll be able to remove the plug by gently scraping if off because it will be at the surface of the new skin. It takes a few days of continued application, but it works. Tea Tree oil will do the same.
I had a few millia spots caused by certain ingredients in the skincare I was using and out of frustration removed 2 of them and they left small marks (holes). That was before I knew about Lemon juice. I used the Lemon juice on the other 2 and no marks (scars) what so ever.