Asymmetric pupils leads to brain scan. Chances of tumor?

My ophthalmologist observed that my right pupil was slightly larger than my left. My primary care doctor sent me to a neurologist. He thought it might be related to a benign tumor that I had had on my left brachioplexus nerve. He thought it was that my left pupil was smaller, not my right bigger, and that my left eyelid might be slightly droopy too. I showed him old photos and I may have had the condition for a while. But he wants me to have a brain MRI/MRA. Of course, I fear the worst--a brain tumor. However, I have no symptoms other than the pupil issue. No headaches, dizzyness, nausea, blackouts, etc. My ophthalmologist found nothing else--although there is a history of glaucoma in my family. Optic nerve healthy, retina healthy, etc. What are the odds that I have a brain tumor? The neurologist says he expects nothing but wants to put it "to rest." Should I believe him?