Codex on AWS bedrock bug causing 10x charges

TheP1000

Our codex on AWS Bedrock read / write cache ratio was less than 5%. Cache writes are very expensive and they were never being used. This results in codex on Bedrock causing ~10x what it should due to no caching and massive writes. The workaround in issue resolved for me: web_search = "disabled"

edoceo

Loaded question: would an openrouter or similar solution caught this before the $BigProblem showed up?

amluto

Wow, that whole thread is borderline incoherent, presumably generated by an AI without adequate oversight. Here are the docs: https://developers.openai.com/api/docs/guides/prompt-caching... The thread has little explanation as to what weird thing they’re doing to Codex that is making the default work poorly, and it kind of seems like it’s getting confused about whether it wants to set the caching mode or the breakpoint or both. In any case, I find the behavior change interesting. It sounds to be like 5.5 and below may have been using a conventional attention scheme where a cached KV sequence can be easily used to restore a prefix of itself, but perhaps 5.6 is using linear attention or LSTM or another recurrent scheme where you cannot rewind the model state by just truncating it.

hk1337

I wonder if it's related to Codex wearing out SSDs.

spacedoutman

Something is wrong with the codex app too, burning usage like crazy lately.

moralestapia

Funny how it is always more charges but never less or no charges. "Random" accidents that always go against you, too biased to be random. But don't notice that too much, you might start to see patterns here and there that you're not allowed to, might get you banned from places, etc.