Litelm: LiteLLM Without the Bloat

khalic

I strongly recommend the authors rewrite the readme by hand. It’s kind of a snif test for how much care someone put into this project.

Centigonal

This is a cool project, and the idea of using LLMs to selectively extract features from open source projects is an interesting concept. The only thing I take issue with is the phrase "LiteLLM Without the Bloat." A lot of the features that have been removed (like cost tracking, streaming, caching) are... kind of the core value proposition of LiteLLM for many of their users.

freshtake

First off, cool project! It's always great to see derivatives that question the efficiency of the established product. I think the main thing the readme is missing is the core benefits. Reducing LOC and dependencies is cool, but it would be great to understand if this provides some additional benefits like lower latency or memory requirements.

LeBit

How does it compare to Bifrost?

clickety_clack

One of the 2 dependencies, httpx, isn't really maintained anymore. Pydantic picked it up as httpx2: https://pydantic.dev/docs/httpx2

arjie

This is a 30 minute project with a frontier LLM. I don’t see why anyone would use anyone else’s router. Techniques are valuable today. Libraries are not.

hopfenspergerj

I imagine many people code their own LLM client after getting fed up with the bad options out there. It’s very easy with ai coding tools. I’m biased but I think mine is coded to a higher standard than litelm. https://github.com/s-banach/langchaint

9dev

Funny, everything you pruned away is the reason I’m deploying LiteLLM in our platform. Having a reliable way to track token spend per customer across different services is important to us, and LiteLLM handles this well

dlojudice

It would be great if there were a plugin/extension infrastructure. For example, to write the cache and costs however and whenever I see fit

DrStartup

most software like this will be dematerialized, democratized, and demonetized - companies building in the infra band being increasingly disintermediated

TZubiri

Drop the Lite, it's cleaner, just use the godamn LLM directly.

asveikau

This readme, when it talks about all the different AI endpoints it can use, reminds me of something. I'm not an AI bro, but I've dabbled. It's kind of remarkable that all the different providers speak the same "openai compatibile" https endpoints. In other realms of software development, real interoperability like that can be kind of rare. Even if people support conceptually the same API, everybody always puts their unique incompatible spin on it. In the dabbling that I've done, big incompatibilities seem rare.

tway235

token counting is useful though. There's also other ways to reduce size, ie a plugin API.

isuckatcoding

Also Strongly recommend renaming to avoid confusion

nperez

Immediate first impression is that this tagline should go. If the project brings something valuable to the table it doesn't need to shit on other permissively licensed open source projects to make a case for itself