Where does it say they're illegal. Certainly in the U.K if you have a spotter standing beside you they're not. I understood the rule was the same in USA ?
Edit:- and nothing to stop a third party enjoying the scenery using the goggles while you fly VLOS.