Tensor Is the Might

srean

> A tensor is nothing but a flat array of numbers, plus some metadata telling you how to interpret those numbers as a multi-dimensional object. Erm... many would disagree. I think what he means is just a multidimensional array.

srean

If one wants to add the capability to reason about shape and shape compatibility, Barry Jay's FiSh would be an interesting detour. https://web.archive.org/web/20111015133833/http://www-staff.... This was used in his shape aware language FiSh, for dealing with multidimensional arrays. Shape compatibilities were statically type checked, if I recall correctly. Shapes were also used to optimize the loops. [Programming in FISh] https://link.springer.com/article/10.1007/s100090050037 [Towards Dynamic Shaping] https://www.researchgate.net/publication/265975794_Towards_D ...

khalic

Why does does the diagram say tensors are 3D?

reerdna

Cool, but I find rather than just shapes and indexes, tensors with labels are much easier to use and reason about. E.g.: { {user:bob, movie:"Heat"}:0.1, {user:alice, movie:"Frozen"}:0.9, {user:carol, movie:"Top Gun"}:0.3, } https://docs.vespa.ai/en/ranking/tensor-user-guide.html

hasteg

I just recently watched some (not all) of this video "coding a machine learning library in c from scratch" and seems like he's going through a similar process in this blog as this video. I would recommend watching the video to get an idea of what the fundamentals of a ML library look like. From someone who has recently been getting interested in actually writing ML code and trying to make sense of it myself (from the perspective of just a typical backend engineer) it was very interesting to see. Previously my experience with ML libs (PyTorch specific) was writing my own Mini-GPT and training it on a small dataset using my own GPU (5090). Cool to see the behind the scenes and took away some o the handwaveyness... https://www.youtube.com/watch?v=hL_n_GljC0I

terminalbraid

> A tensor is nothing but a flat array of numbers I'm so very, very tired of tech coopting rigorous mathematical terms.

nathan_compton

I know there are different contexts, but a tensor is not a collection of numbers, in a mathematical sense. A vector is not a list of numbers. Such collections of numbers are representations of objects with very specific kinds of properties under coordinate transformations. I think it genuinely damages people's ability to digest the mathematics to tell them first and foremost that these objects are collections of numbers.

ksd482

> A tensor is nothing but a flat array of numbers, plus some metadata telling you how to interpret those numbers as a multi-dimensional object. Yikes! No. I mean even for the intents and purposes of using this definition in ML, this might not be right. I am trying not to be pedantic, so I will not go with the official/mathematical definition of a tensor as that could be incredibly confusing (look it up!!!). But a tensor is a LOT more than that. Essentially it's a multilinear map that transforms a set of basis vectors in a certain way, and is coordinate agnostic. This is not even half its definition so you can see how much the author left out. Having said that, this is still a good way to start getting intuition into it and I urge the author to continue refining the definition as he/she learns more. Disclaimer: MS in Math with concentration of GR. EDIT: Also tensor aren't simply "flat" array of numbers. They are multidimensional. A grounded example, a rank 3 tensor is a collection of 2d matrices. Think of it as a bunch of 2d matrices stacked on top of each other. You need 3 indices to keep track of numbers --- sure in a programming language, it can be represented as a 1d array as well with 0s filling up empty spaces, but you get the idea.