Risks of radiation exposure depend largely on the gestational age of the fetus and the amount of radiation the fetus is exposed to. Adverse effects to the fetus may include mental retardation, increased risk of developing cancers such as childhood leukemia, and fetal growth restriction.
But, in order to develop any of these adverse effects, the fetus needs to be exposed to a substantial amount of radiation. For instance, the risk of childhood leukemia seems to increase after 1-2 rad of exposure, and a mammogram is associated with only 0.02 rad of exposure. The effect is cumulative, though. So, if you need multiple X-rays or CT scans in pregnancy for whatever reason, the exposure adds up.
The best advice is to speak with your doctor about how necessary the mammogram is during pregnancy. As with most things in pregnancy, you will have to balance the potential risks of radiation exposure to the fetus with the benefit of early detection of a breast abnormality.