You can up to a point. I was told that, from 10 to 12 weeks on, you should not ride anything like that at all. I've been riding dirt bikes, etc, for years, and that was one thing that my doctor firmly said no to.
The bumping, jarring, etc., is definitely not good for the baby. I was told that, especially in the 3rd trimester, it could lead to premature labor.
Also, hon, I know he's driving safely, but if something happens- meaning, you fall off, you have an accident, etc., you're endangering your baby's life.