To ease the pressure and prevent mastitis or a breast infection, express a little milk from your breasts until you feel relief. You can use a pump or hand-express, or you may be able to get some out by placing warm, wet towels over your breasts. Expressing a small amount of milk will make you feel better and will help signal your body to start making less milk. (Keep the amount expressed to a minimum — emptying the breasts would increase your milk supply.) Keep up this routine, expressing just enough to stay comfortable, and you should see your milk supply begin to disappear. Ice packs (or bags of frozen veggies) can help with the pain too by reducing the swelling, as can pain relievers like acetaminophen and ibuprofen.