Valencia were told by organisers to resume playing Sunday's La Liga match at Cadiz or they would have to forfeit the points after they had left the pitch when Mouctar Diakhaby complained he had been racially abused, fellow defender Jose Gaya said.
Valencia's players walked off the field in the 29th minute after Frenchman Diakhaby reacted furiously following an altercation with Cadiz's Spanish centre back Juan Cala.
It was the first time a La Liga match had been halted following an accusation of racism.
The players eventually returned to the pitch but did so without Diakhaby, who said he was not in the right state of mind to continue playing and was substituted. He was also shown a yellow card before the Valencia players left the pitch.
The score was 1-1 when the visitors walked off and Cadiz went on to win the game 2-1 after an 88th minute strike from Marcos Mauro, who came off the bench to replace Cala.
"Diakhaby told us he heard a racist insult so we all went off the pitch. We were told that we had to play because if we didn't they would take the three points off us and maybe we'd be docked more points," Gaya told Spanish network Movistar.
"Diakhaby asked us to play on, we wouldn't have done so if he hadn't done so. He said he wasn't in the right state of mind to keep playing. He is devastated."
Gaya said he had heard the comment Cala made to Diakhaby.
He added: "It was a very ugly insult which I won't repeat. We haven't been able to speak with Cala because he was the last player to leave the pitch. I'm certain that he said something. It was a very strange game."
Cadiz coach Alvaro Cervera said Cala told him he had not insulted Diakhaby, adding he took the player off at halftime because he was already on a booking and did not want to risk him getting sent off.
Cala had given Cadiz the lead in the 14th minute before Kevin Gameiro soon equalised for the visitors.
La Liga did not respond to a request for comment on Gaya's statements.
Valencia coach Javi Gracia said Diakhaby had been "seriously insulted and was quite shaken."
He added: "Logically we decided to tell the referee we didn't agree with that [playing on] and that we didn't want to continue playing. While we were in the dressing room we were told we could potentially be sanctioned if we didn't return.
"We spoke to Diakhaby, he said he was not able to play but he understood we needed to return to play rather than be sanctioned and so we returned with the intention to fight for the victory."
Gracia added that the match referee and assistants did not know what had been said to Diakhaby. He also did not reveal who had told his team they could be sanctioned.