During shadow mode, the AI solution receives the same cases as the case workers and produces its suggestions. The suggestions have no effect, since people keep making the decisions as usual. The team then compares the solution's suggestions with the people's decisions and measures how often it gets it right.
The pilot shows that the solution works. Shadow mode shows how accurate it is on real cases. Only when the accuracy is good enough does the solution go into production and act for real.