Members-Only

Recent Talks & Demos are for members only

Exclusive feed

You must be an AI Tinkerers active member to view these talks and demos.

Sign in View FAQ

November 21, 2024 · Poland

Speeding up inference with Effort Engine

Learn how the Effort Engine algorithm reduces LLM inference latency and resource usage, providing practical examples and implementation guidance for developers.

Overview

Founder of Effort Engine (new algorithm for LLM Inference)

Links

https://kolinko.github.io/effort/
`bucketMul` dynamically adjusts LLM inference speed by skipping weights in real-time.

Tech stack