Two latency numbers matter: time-to-first-token (TTFT) and tokens-per-second afterwards. Bigger models are usually smarter but slower and pricier. Real systems mix models: a small fast model for ...

The project automatically fetches the latest papers from arXiv based on keywords. The subheadings in the README file represent the search keywords. Only the most recent articles for each keyword are ...