Building durable workflows on Postgres

sgt

Continuously amazed by what you can do with few tools, as long as Postgres is a part of your toolkit. I recently developed a distributed queue and it works really great - benchmarks great too, with no race conditions or conflicts. I used SKIP LOCKED so that workers can compete safely. You can also have multiple workers across nodes avoid conflict by using session wide mutexes i.e. pg advisory lock.

senderista

Citing CockroachDB as an example of scaling Postgres made me spit out coffee. Was this LLM-written?

throwaw12

Curious to know experience of people using DBOS and Temporal. I have used Temporal in the past, works really good, my only problem with it was some limits on request payload or event sizes, created some inconveniences to us when building solutions. It also enforces good engineering practices, but sometimes you don't want to write special logic if your CSV file is larger than 2Mb, upload it to S3, pass link, then download it in the workflow. What is your experience with DBOS? How does it compare to Temporal in terms of operational complexity, feature parity and anything else

switchbak

Having inherited a few of these - you tend to home-grow an ad-hoc version of many of the existing OSS tools, but with less of the patterns baked in. Not sure where the NIH ends and where you're actually better off with a supported orchestration approach. I suppose if you expect your program to be around a while (or need advanced features), maybe think about using something a bit more battle tested?

vrm

Since DBOS doesn't support Rust, we implemented a very minimal Rust version of this at https://github.com/tensorzero/durable . It has been quite stable and extensible but of course you need to be very careful with the SQL implementations. Hope this is interesting to readers here.

cpursley

PgFlow is pretty awesome for DAG workflows - it's built on pgmq (which does the heavy lifting, making it backend agnostic). Typescript: https://www.pgflow.dev Elixir: https://github.com/agoodway/pgflow/blob/main/docs/COMPARISON...

joshka

This feels like the sort of architecture that starts clean and then gradually grows most of the things a workflow-native system already has. I've seen systems like this, seen companies that are built out of this idea, and built small systems like this over time. Once you need retries, backoff, timeouts, cancellation, versioning, visibility, task routing, rate limits, leases, heartbeats, stuck-worker detection, replay/debugging semantics, workflow migration, fanout/fanin, long timers, audit trails, and operator tooling, the “just use a database” story becomes “build a poor copy of a workflow engine plus a bunch of workers.” pretty quick. That may still be a good tradeoff for many applications, especially if Postgres is already the core operational dependency. But the comparison shouldn’t be “database vs overcomplicated orchestrator.” It’s more like “what complexity do you want to own, and what do you want to buy / offload to a professional system?”

pirsquare

I feel it's way too hand wavy on consistency and correctness. My opinion as someone who've implemented marketing workflows that breaks all the time (and tons of painful lessons). Strong correctness guarantee is something that should not be undermine. Even more important than availability. The examples on the website is simple but heavily undermines the importance of correctness. Anyone who implement similar pseudo-code directly will eventually suffer from data correctness issue in crashes. @DBOS.workflow() def checkout_workflow(items: Items): order = create_order() reserve_inventory(order, items) payment_status = process_payment(order, items) if payment_status == 'paid': fulfill_order(order) else: undo_reserve_inventory(order, items) cancel_order(order)

hbarka

How do you incorporate secrets in this kind of implementation? Stored in db?

llmslave

Temporal is an insane piece of software, always surprised people dont know about it. You could replace almost youre whole AWS stack with temporal

opiniateddev

Conductor OSS does this quite well https://docs.conductor-oss.org/devguide/ai/index.html https://github.com/agentspan-ai/agentspan which is essentially an agentic SDK layer for Conductor can convert any of your langgraph, openAI, vercel, or ADK agent and makes it durable and adds orchestration with no code changes.

magicseth

Convex has a workpool component that gives the ability to compose big complicated flows in an understandable way, and give you realtime updates on status of various pieces: https://www.convex.dev/components/workflow

munk-a

We have a durable queue built into postgres to handle some complex notification-ish logic. It's worked excellently and while there are services various cloud providers would love to sell us to do that it's extremely cheap to run. For that particular usage, the volume we process and business criticality make it a good choice for inventing here - but for other durable processes we just use off the shelf tools since the cost of maintenance would quickly outstrip the value. Postgres is a great tool to use and far more powerful than most people give it credit for - but there's always the balance of in-house maintenance vs. paying rent for someone else's solution.

elliot07

how is this compared to hatchet?

llimllib

Armin Ronacher's `absurd` is an implementation of durable workflows for postgres: https://lucumr.pocoo.org/2025/11/3/absurd-workflows/ https://github.com/earendil-works/absurd https://earendil-works.github.io/absurd/ I've not used it, but it's worth comparing to other options

buremba

All you need is Postgres until you scale into TBs of data. We use Postgresql as a durable workflow engine, vector search, time-series data, BM25 search, OLTP/OLAP engine, and a queue. It's basically the only dependency we have for https://lobu.ai The main benefit is centralizing all the data in one place so we don't need to worry about copying data in between multiple systems. Once something becomes the bottleneck, you can eventually migrate to a purpose specific tool to scale out.To be honest, LISTEN/NOTIFY in my opinion is the most fragile part of PG but it's fine as start until you scale out.

OutOfHere

I am not convinced that using a special software for "durable workflows" is necessary. If one has a stateful message queue or job task queue, e.g. RabbitMQ or Celery, one can use it. Irrespective, many jobs can be made idempotent. The most that you ought to residually need is a column in an existing table of your own database which keeps track of what remains to be done. Given the above, it would seem that durable workflow software is pushed forward by those who have a surplus of VC money to spend. As for the vendors, there is no shortage of people trying to sell you things that you don't need.

grahac

Isn't this Just Oban from elixir? :)

stuartaxelowen

My dream is, instead of separating data storage, state machines, valid state constraints, and the logic that transitions between valid states, we can actually unify these into some kernel of app state. Honestly, Postgres already has a lot of these capabilities, but I don’t see an obvious story on the app or product level, providing provably correct sets of states that apps can transition between, and which they can automatically expose to clients in informative ways (this user can like this post, but not edit). It looks colored Petri net shaped to me, but I don’t yet see a simple app state paradigm in the same way that the database has obvious successful boundaries.

rafael-lua

The "everything can be done in Postgres" crowd is crazy. It is like a religion at this point.