Cesar Noval -- who is also a cosmetic surgeon -- has been a top flight official for the last four seasons.
The photo, which was shared on social media, pictured him alongside the two La Liga stars, their partners and other friends on a yacht in Ibiza.
"A member of the refereeing committee called and told me they'd decided to demote me," Noval told Onda Cero radio. "I haven't published any photos on social media.
"One of the wives of the players, who I have a really good relationship with, published it. Nowadays it's totally normal."
"Our friendship comes from outside football. I've only refereed Roberto Soldado in a friendly.
"We never coincided when he played for Granada last season... I'm friends with other footballers too who weren't on that yacht. It's ridiculous."
Noval previously made headlines when he decided not to participate in the last 11 games after La Liga restarted, citing concerns over the health of his footballing colleagues as well as his patients.