The GPU was built for graphics but is good at calculating many things at once, which is exactly what an AI model does. Memory is the most important limit, since the model has to fit in it to run fast.
An ordinary GPU is enough for transcription, search and smaller language models. The largest language models need several GPUs in a data centre and are therefore usually used as a service.