I'm Rewriting IBM SPSS Statistics in 100% Safe Rust, Alone

kobaltGIT

I'm building StatCraft — an open-source statistical analysis package in 100% Safe Rust, as a clean-room alternative to IBM SPSS Statistics. I started this because I believe statistical software should not crash on malformed data, should be scriptable by AI agents via MCP, and should not cost thousands of dollars per year. Current state (11 days in, solo): - 14 statistical blocks audited against SPSS 24 Algorithms Guide with ≤1e-6 SciPy precision - Full .sav/.zsav codec, Excel/CSV import, SPSS syntax parser (.sps) - 37 SPSS-style dialogs, pivot tables, Output Viewer with TSV export - Built-in MCP server for Claude/ChatGPT/Ollama agents - Zero unwrap/expect/panic in production; fuzzed on corrupted inputs — 0 panics What I need help with: - Testers with real SPSS datasets (the weirder the encoding, the better) - Rust developers for the expression evaluator (3 known precedence gaps) - Statisticians to validate output against their SPSS workflows - Anyone who wants to port a procedure (regression, factor, reliability are in) GitHub: https://github.com/kobaltgit/StatCraft

uniqueuid

Sorry but the problem is not primarily software, it’s community, teaching and standardization. We have R/tidyverse which does 90% of everything and has an amazing community and procedures eg. around documentation. For everything else you have excel or python. So I don’t see the use case.

throwaway63467

Does safety matter for this kind of software? It’s mostly internal tooling used by researchers and statisticians. I get it rust is cool but safety isn’t the prime concern here I think.

EdwardDiego

I'm (well, my LLM is) Rewriting IBM SPSS Statistics in 100% Safe Rust, Alone (for a given value of alone). You even let your LLM write the README. Look, we nearly all use LLMs these days, but the phrasing of "I am... ...alone" is particularly weird when it's obviously not just you, it's you with LLMs. Maybe title it as "A rewrite of IBM SPSS Statistics in Rust with no unsafe" - the 100% safe claim is something an LLM makes, it's not something a reasonable human would say, as it implies the Rust is totally safe, whereas you're trying to communicate that your Rust code uses no unsafe blocks, and these are two different things.

nialse

Why not let go of SPSS instead? It has run its course and is basically legacy legacy software by now. But it good fun I suppose? Are you using SPSS as an oracle? (Guessing that the author is/will be in the comments.) An oracle and fuzzing have been excellent when reimplementing software. Given implausible inputs, the black box shows its colours quickly. Rounding errors and different representations of floats are and endless source of debugging, eh, joy.

Traster

This is one of the things that makes me negative on AI. Before AI, no one would have done this. Because it's a huge amount of effort and a total waste of time. Now we have AI, we have this explosion in people going off and doing all these pointless activities. What do we gain from a plethora of SPSS-alike tools? Nothing. There's literally no explanation on this page about why anyone who uses SPSS would want to use this. But it's "100% Safe Rust". Yes, that's a feature that someone who likes rust wants, not what someone who uses SPSS wants. What do we lose by fracturing all academic statistical analysis into a dozen different tools instead of 1? Lots! It's not even clear the person creating this even intends for anyone to use it. So (a) why are they doing it and (b) how do you even know if it actually works if absolutely no one is using it. In fact the danger now is someone who doesn't know the author is just an AI slop dilletante comes along looking for an SPSS-like tool and makes the mistake of trying to use it. Is there really any difference between this and the people posting AI slop onto their reddit accounts. It's just raising the noise floor to the point where we're going to have to essentially shut down the public internet, everything of value will move to closed spaces.

scrubs

The title "I'm...100% Safe Rust, Alone" is obnoxious. Op, is there any technical merit to this, or is this a step in some process to earn brownie points in some pathetic "I'm a morally superior hero club?" What'd you expect the rest of us to feel? Chastened?

sehw

>I can't into R

nnurmanov

I am doing python based pipelines, new pipelines were too slow, I must admit, the first that crossed my mind was to rewrite it in Rust:) But later after detailed analysis it was an architectural problem. You may not need Rust:)

frevib

Nice to see these efforts. It’s completely vibe coded though, and the agents.md is in a language I don’t understand: https://github.com/kobaltgit/StatCraft/blob/main/.agents/rul... I think I’ll pass.

Stitch4223

Alone? How do you achieve clean room reversing with one person? You don’t state StatCraft is clean room, but all Artcraft software is. Talk to a lawyer to check if you’re not wasting your time. https://en.wikipedia.org/wiki/Clean-room_design

qubex

What’s wrong with PSPP?

kobaltGIT

Thanks everyone for the lively discussion, and huge thanks to the folks who emailed mods and vouched to clear the automated filter flag on my new account! A few common points raised across the thread that I’d like to address in one place: 1. On "Alone", AI, and the Craft Anything methodology: Some folks dug into the repo, found the rules and chronicles, and pointed out this is heavily AI-assisted. They are 100% right — and I don't hide it (there is an open public dev chronicle in docs/ detailing every single step). By "alone" I meant no venture backing, no university grant, and no human engineering team. However, this isn't chaotic one-shot prompting. StatCraft is built as part of the ArtCraft suite, following the philosophy and methodology of Craft Anything ( https://github.com/storytold/craft-anything - "understand how any software works, and craft your own"): using evidence-backed, clean-room investigation to replace closed, predatory software with open, reliable tools. We pair an orchestrator with specialized AI agents, constrained by strict compiler invariants and external mathematical oracles. That's also why StatCraft has a native Model Context Protocol (MCP) server: the software itself is designed to live natively in an agent-first world where researchers can interact with deterministic tools via natural language. 2. On "100% Safe Rust" terminology: Fair critique from the Rust community on the phrasing: what I technically meant is #![forbid(unsafe_code)] across the workspace, zero unsafe blocks, and a zero-panic invariant (Result<T, E> everywhere, zero unwrap/expect in non-test code). The compiler ensures memory safety invariants, while domain correctness is enforced via fuzzing and mathematical oracle tests. 3. On Clean-Room & Legal concerns: No IBM binaries were reverse-engineered or decompiled. The .sav and .zsav codecs follow open specifications established by GNU PSPP and the MIT-licensed ReadStat library. The statistical procedures are implemented strictly from academic literature and public formula specifications (such as the SPSS 24 Algorithms Guide), validated against SciPy with <= 1e-6 precision. 4. "Why not just use R/Tidyverse?": R and Python are wonderful for coders. But thousands of students, sociologists, psychologists, and clinicians are trained purely on SPSS's GUI. When their university license expires, they face a $1000+/year paywall from IBM and have legacy .sav files they can't easily open without proprietary software. StatCraft is built for them: a free, native GUI tool that respects their muscle memory, works offline, and never crashes on messy survey data. If you have old, weird, or corrupted .sav files, or want to audit a statistical procedure against your own SPSS outputs, issues and PRs are very welcome on GitHub.

veggieroll

GNU PSPP [0] is what got me into free software after losing access to SPSS in the middle of writing my undergrad capstone. I’ve long since switched to python & other normal programming for any stats needs. And I only ever scraped the surface of SPSS/PSPP anyways. But, I’d be interested to hear what PSPP doesn’t cover. Good luck on your project. [0] https://www.gnu.org/software/pspp/

westurner

No experience with or opinion on SPSS; GNU Octave is like Matlab. Matplotlib is like Matlab but doesn't support the same syntax. Will it have a DataFrame API? Before R data.frame and Pandas DataFrame and substrait.dataframe, there were labeled arrays in NumPy. Pint uses numpy ufuncs to type numbers with physical units. Here are the notes I had for pgrust a few months ago: https://news.ycombinator.com/item?id=49112449 ; Polars (already has tested SIMD and GPU-optimized routines), substrait interface, and file formats: LanceDB, Vortex, Arrow recordbatch Here are my notes for the ArtCraft folks about methods for Rust ports (and whether it's necessary to also port Blender to Rust with an O3/Maturin Python API) https://github.com/storytold/artcraft/issues/2489 : > port the tests [and check figures] first ; use branch coverage metrics when testing ; use cargo-llvm-cov and rust-nightly to test ; make (.sh,) scripts that the .github/workflows/ call so that local builds are easy ; run local builds using nektos/act ; use rstest to keep tests and test fixtures maintainable ; use cargo-insta to snapshot ; use cargo-fuzz, and do coverage-guided fuzzing ; use TLA+, Lean or similar formal methods and tools to model the critical parts of the app at least In the ArtCraft issues, there are so many great ideas that any or many or all firms could benefit from.

Hizonner

Offering a "statistical package for the social sciences" is like giving guns to toddlers.