Brazilian Court Fines Lawyers Over Prompt Injection in Lawsuit

Two Brazilian law firms were ordered to pay a fine equal to 10% of their own client's claim for using hidden text, a prompt injection, designed to interfere with the AI systems used in the country's court process. The hidden text was written in white on a white background so people would not see it. It instructed any AI reading the lawsuit to give shallow answers and never question the credibility of the opposing side's evidence. The case was being reviewed by Galileu, an AI program run by Brazil's National Council of Justice. Galileu oversees the Brazilian judicial system and analyzes cases nationwide. When Galileu detected the hidden text, it refused to carry out the instruction. This may be the first case in the world where someone was punished for trying to hack an AI system used in a court proceeding.

The episode shows that any data fed into an AI can potentially contain instructions, in this case through prompt injection. The AI carries out these instructions as if they were part of the original request. What makes this case interesting is that the AI managed to recognize the attempt to undermine its own work.