Paddler - llama.cpp load balancer

Paddler is an open source load balancer custom-tailored for llama.cpp. I will show you a simple example on how you can use it to deploy fleets of self-hosted LLMs.