Use the redis extra for Redis support: For performance reasons, the session is not autoloaded by default. Sometimes it is annoying to call load_session too often. We provide SessionAutoloadMiddleware ...

A production-minded FastAPI sidecar for serving Gemma 4 31B on vLLM with Gemma 4 Multi-Token Prediction (MTP) speculative decoding. It keeps the raw vllm serve process private and adds ...