Library for fast mapping of Java records to native memory

wood_spirit

This is interesting. Java desperately needs an array of struct for type safe sugar over high performance arenas, but the areas you’d turn to this would be in a zero allocation effort where the cost of the this library’s off-heap and the object allocation in the getters and setters etc largely negate the advantages for a lot of use cases.

kosolam

Nice. Very clean api.

usernametaken29

Why not use graal?

matt_heimer

What is the positioning for this and how does it work? A comparison to SBE might be nice. I understand the issue about using Layout and MemorySegment being verbose but the reason I'm using those things it to develop high performance software that uses off-help memory and bypasses object allocation. What does "map Java record types onto native memory" actually mean? Did you somehow turn a Java record into a flyweight or is `Point point = points.get(0);` just instantiating a record instance using data read from off-help memory? If it's a dynamic mapping library using reflection, that's cool but doesn't it kill the performance goals for most Java off heap usage? Is this more of a off-heap to heap bridge for pulling data into the normal Java space when performance isn't critical?

jayd16

At first glance it reminds me of C#'s Span<T>.

steve_barham

I did something similar a few years back, with a slightly different approach to declaration, using interfaces to denote the layout of the struct. Mutation was opt-in by exposing setters using the (of the time) standard JavaBeans layout and an annotation processor took care of the codegen of an implementing class, which could be used where you wanted an on-heap box of an off-heap structure. One benefit of this approach was that by using the interface as the type you could fairly easily support a flyweight pattern, reducing GC pressure when working with large off-heap collections. The parallels between stateless interfaces and offheap structs was also quite pleasing. I'd love to see a similar effort using more modern techniques than Unsafe et al.

c-fe

This is very similar to SBE encoder/decoder flyweights over raw memory? What are the differences?

wwarner

Wouldn’t apache arrow serve the same purpose?

J-Kuhn

Inspired by this, I wrote a quick prototype using MethodHandle combinators: https://gist.github.com/DasBrain/19804df69c78cee257dd0294b00... It doesn't have all the bells and whistles - so no support for annotations or arrays, but for ~2h of work, it shows what is possible without resorting to bytecode generation.

hansvm

Ha, I wrote something like this ages ago when I had to interact with a Java application and really wanted/needed Python utilities. I created my own concurrent, type-safe DDL to shepherd that data.