5 ms·

Cracovians: The Twisted Twins of Matrices

81 points by mci 1y ago

kubb 1y ago
> However, the multiplication of a cracovian by another cracovian is defined differently: the result of multiplying an element from column i of the left cracovian by an element from column j of the right cracovian is a term of the sum in column i and row j of the result. Am I the only one for whom this crucial explanation didn’t click? Admittedly, I might be stupid. Wikipedia is a bit more understandable: „The Cracovian product of two matrices, say A and B, is defined by A ∧ B = (B^T)A
- tempodox 1y ago
  You're not the only one. That “explanation” is just really bad.
  - tgv 1y ago
    It's the crucial part, and even with the example, I couldn't understand it. Like I can't understand why the second column in the first matrix doesn't have signs. Or why the 0 in the result matrix is negative. But in another link I found that it's column by column multiplication. So A × B = C, then C[i][j] = sum(A[k][i] * B[k][j]). Unfortunately, the example doesn't match that definition...
- burnished 1y ago
  No, I think it is too ambiguous to be useful. The example wasn't helpful either, I think they needed to perform the individual calculations for clarity.
  - kubb 1y ago
    Yeah, usually you name the matrix elements a, b, c, d, etc. and write out the formula for the elements of the result.
- AdamH12113 1y ago
  The example is simply wrong, according to other sources. This along with the inconsistent formatting makes me wonder if it was written by an LLM. It's a shame; this seems like an interesting topic.
- andrewla 1y ago
  Agreed -- "is a term of the sum" is such an inverted way to look at it. Better I think would be to say "the result in column i and row j is the sum of product of elements in column i of the left cracovian and column j of the right cracovian". And even by this definition the example given doesn't seem to track (and the strangeness of sometimes saying "+" and sometimes not, and having both "0" and "-0" in the example is bananas!): { 3 2 } { 1 -4 } = { 5 -2 } { -1 0 } { -2 3 } = { 0 2 } 3 * 1 + -1 * -2 == 5 -- check 3 * -4 + -1 * 3 == -15 -- what? 2 * 1 + 0 * -2 == 2 (okay, but shouldn't this be in the lower left, column 1 dotted with column 2?) 2 * -4 + 0 * 3 = -8 (now I'm really missing something)
  - mci 1y ago
    Thanks for the feedback, everyone. I pasted my Polish text into Gemini to translate it into English. Gemini hallucinated the translation of this example. Now it should be OK.
    - pomian 1y ago
      Even in Polish, this comes out Greek to me.
      - pomian 1y ago
        I mean, it makes some sort of visual sense, but can't grasp the results from the matrices shown.
  - mci 1y ago
    I took the liberty to replace my awkward wording with your "the result in column i and row j is the sum of product of elements in column i of the left cracovian and column j of the right cracovian". Hope you don't mind. Thanks!
fxj 1y ago
I didnt get the explanation of the multiplication. After reading the wikipedia article it made mode sense: https://en.wikipedia.org/wiki/Cracovian https://en.wikipedia.org/wiki/Cracovian The Cracovian product of two matrices, say A and B, is defined by A ∧ B = BT A, where BT and A are assumed compatible for the common (Cayley) type of matrix multiplication and BT is the transpose of B. Since (AB)T = BT AT, the products (A ∧ B) ∧ C and A ∧ (B ∧ C) will generally be different; thus, Cracovian multiplication is non-associative. A good reference how to use them and why they are useful is here (pdf): https://archive.computerhistory.org/resources/access/text/2018/07/102784074-05-01-acc.pdf https://archive.computerhistory.org/resources/access/text/20...
- adastra22 1y ago
  As far as I can tell I don’t think it is correct to say that this isn’t a matrix. B is just written down in transposed form. Whether that makes the math more or less clear is something you can argue for or against, but it’s the same math and it is confusing to call it something else.
  - noosphr 1y ago
    It is a tensor of rank two with a special binary operation on tensors. These objects aren't matrices in the mathematical sense any more than convolution kernels aren't.
    - adastra22 1y ago
      A tensor of rank two is the same thing as a matrix…
      - noosphr 1y ago
        It isn't. Matrices come with the matrix product defined over them. This is one of four possible closed first order tensor contractions for an order two tensor, viz. AijBik AijBki AijBjk AijBkj. Only the third is applicable to matrices, all other contractions only work for general tensors without transposition. What we deal with in computer science are actually n dimensional arrays since we don't have the co and contravariant indices that define tensors in physics.
esafak 1y ago
Missed a chance to call it the twisted sister!
gnulinux 1y ago
I guess I'm skeptical of using a non-associative algebra instead of something that can trivially be made into a ring or field (i.e. matrix algebra). What advantages does this give us?
- mci 1y ago
  Author here. There are no practical advantages, as far as I know. Not even faster multiplication on today's computers.
  - hansvm 1y ago
    One thing that comes up in the sort of code ML I like to write is a careful attention to memory layout. Cracovians, defined according to some sibling comment as (B^T)A, make that a little more natural, since B and A can now have the same layout. I haven't used them though, so I don't have a good sense of whether that's more or less painful than other approaches.
  - bravesoul2 1y ago
    Shouldn't be the same on a computer right? The change is in human perception not actually what hapens when multiplying.
TimorousBestie 1y ago
What an interesting little nook of matrix analysis history! Thanks for sharing. :)
noosphr 1y ago
In Einstein notation this operation is Aij Bkj, which incidentally shows why Einstein notation is so useful.
- Syzygies 1y ago
  I'm a mathematician who taught linear algebra for decades. I love Einstein notation. I don't find Cracovians interesting at all. Old texts got really worked up whether a vector was a row or column. The programming language APL resolved this quite nicely: A scalar has no dimensions, a vector has its length as its one dimension, ... Arbitrary rank objects all played nicely with each other, in this system. A Cracovian is a character or two's difference in APL code. There's a benign form of mental illness learning anything, where one clutches onto something novel and obsesses over it, rather than asking "That was exciting! What novel idea will I learn in the next five minutes?" I have friends from my working class high school who still say "ASSUME makes an ask of you and me" as if they just heard it for the first time, while the most successful mathematicians that I know keep moving like sharks. I wouldn't stall too long thinking about Cracovians, as amusing a skim as the post provided.
  - noosphr 1y ago
    I mean theres nothing special about naming binary operations on tensors of fixed rank. Matrices have some nice mathematical properties which is why they are studied so much in mathematics. But for number crunching there is no reason to prefer then to cracovians, or vice versa, without knowing what the underlying memory layout is in hardware.
semiinfinitely 1y ago
Uhh so it's just matrices where the left slot of matmul is transposed?
gus_massa 1y ago
> It turns out that multiplying cracovians by computers is not faster than multiplying matrices. That's very specific of Python. A few years ago we were multiplying a lot of matrices in Fortran and we tried to transpose one of the matrices before the multiplication. With -o0 it was a huge difference because the calculation used contiguous numbers and was more chache friendly. Anyway, with -o3 the compiler made some trick that made the difference disappear, but I never tried to understand what the compiler was doing.
- wjholden 1y ago
  I would expect that Julia could similarly show performance boosts here because of its column-major memory layout.
rundigen12 1y ago
Read to the end and... what was the point of that? Where's the payoff? There was a claim near the top that some things are easier to compute when viewed as cracovians. then some explanation, then suddently it switches to numpy and showing the time is the same. New title: "Cracovians are a Waste of (the Reader's) Time"?

81 points by mci 1y ago

kubb 1y ago

> However, the multiplication of a cracovian by another cracovian is defined differently: the result of multiplying an element from column i of the left cracovian by an element from column j of the right cracovian is a term of the sum in column i and row j of the result. Am I the only one for whom this crucial explanation didn’t click? Admittedly, I might be stupid. Wikipedia is a bit more understandable: „The Cracovian product of two matrices, say A and B, is defined by A ∧ B = (B^T)A

tempodox 1y ago
You're not the only one. That “explanation” is just really bad.
- tgv 1y ago
  It's the crucial part, and even with the example, I couldn't understand it. Like I can't understand why the second column in the first matrix doesn't have signs. Or why the 0 in the result matrix is negative. But in another link I found that it's column by column multiplication. So A × B = C, then C[i][j] = sum(A[k][i] * B[k][j]). Unfortunately, the example doesn't match that definition...
burnished 1y ago
No, I think it is too ambiguous to be useful. The example wasn't helpful either, I think they needed to perform the individual calculations for clarity.
- kubb 1y ago
  Yeah, usually you name the matrix elements a, b, c, d, etc. and write out the formula for the elements of the result.
AdamH12113 1y ago
The example is simply wrong, according to other sources. This along with the inconsistent formatting makes me wonder if it was written by an LLM. It's a shame; this seems like an interesting topic.
andrewla 1y ago
Agreed -- "is a term of the sum" is such an inverted way to look at it. Better I think would be to say "the result in column i and row j is the sum of product of elements in column i of the left cracovian and column j of the right cracovian". And even by this definition the example given doesn't seem to track (and the strangeness of sometimes saying "+" and sometimes not, and having both "0" and "-0" in the example is bananas!): { 3 2 } { 1 -4 } = { 5 -2 } { -1 0 } { -2 3 } = { 0 2 } 3 * 1 + -1 * -2 == 5 -- check 3 * -4 + -1 * 3 == -15 -- what? 2 * 1 + 0 * -2 == 2 (okay, but shouldn't this be in the lower left, column 1 dotted with column 2?) 2 * -4 + 0 * 3 = -8 (now I'm really missing something)
- mci 1y ago
  Thanks for the feedback, everyone. I pasted my Polish text into Gemini to translate it into English. Gemini hallucinated the translation of this example. Now it should be OK.
  - pomian 1y ago
    Even in Polish, this comes out Greek to me.
    - pomian 1y ago
      I mean, it makes some sort of visual sense, but can't grasp the results from the matrices shown.
- mci 1y ago
  I took the liberty to replace my awkward wording with your "the result in column i and row j is the sum of product of elements in column i of the left cracovian and column j of the right cracovian". Hope you don't mind. Thanks!

fxj 1y ago

I didnt get the explanation of the multiplication. After reading the wikipedia article it made mode sense: https://en.wikipedia.org/wiki/Cracovian https://en.wikipedia.org/wiki/Cracovian The Cracovian product of two matrices, say A and B, is defined by A ∧ B = BT A, where BT and A are assumed compatible for the common (Cayley) type of matrix multiplication and BT is the transpose of B. Since (AB)T = BT AT, the products (A ∧ B) ∧ C and A ∧ (B ∧ C) will generally be different; thus, Cracovian multiplication is non-associative. A good reference how to use them and why they are useful is here (pdf): https://archive.computerhistory.org/resources/access/text/2018/07/102784074-05-01-acc.pdf https://archive.computerhistory.org/resources/access/text/20...

adastra22 1y ago
As far as I can tell I don’t think it is correct to say that this isn’t a matrix. B is just written down in transposed form. Whether that makes the math more or less clear is something you can argue for or against, but it’s the same math and it is confusing to call it something else.
- noosphr 1y ago
  It is a tensor of rank two with a special binary operation on tensors. These objects aren't matrices in the mathematical sense any more than convolution kernels aren't.
  - adastra22 1y ago
    A tensor of rank two is the same thing as a matrix…
    - noosphr 1y ago
      It isn't. Matrices come with the matrix product defined over them. This is one of four possible closed first order tensor contractions for an order two tensor, viz. AijBik AijBki AijBjk AijBkj. Only the third is applicable to matrices, all other contractions only work for general tensors without transposition. What we deal with in computer science are actually n dimensional arrays since we don't have the co and contravariant indices that define tensors in physics.

esafak 1y ago

Missed a chance to call it the twisted sister!

gnulinux 1y ago

I guess I'm skeptical of using a non-associative algebra instead of something that can trivially be made into a ring or field (i.e. matrix algebra). What advantages does this give us?

mci 1y ago
Author here. There are no practical advantages, as far as I know. Not even faster multiplication on today's computers.
- hansvm 1y ago
  One thing that comes up in the sort of code ML I like to write is a careful attention to memory layout. Cracovians, defined according to some sibling comment as (B^T)A, make that a little more natural, since B and A can now have the same layout. I haven't used them though, so I don't have a good sense of whether that's more or less painful than other approaches.
- bravesoul2 1y ago
  Shouldn't be the same on a computer right? The change is in human perception not actually what hapens when multiplying.

TimorousBestie 1y ago

What an interesting little nook of matrix analysis history! Thanks for sharing. :)

noosphr 1y ago

In Einstein notation this operation is Aij Bkj, which incidentally shows why Einstein notation is so useful.

Syzygies 1y ago
I'm a mathematician who taught linear algebra for decades. I love Einstein notation. I don't find Cracovians interesting at all. Old texts got really worked up whether a vector was a row or column. The programming language APL resolved this quite nicely: A scalar has no dimensions, a vector has its length as its one dimension, ... Arbitrary rank objects all played nicely with each other, in this system. A Cracovian is a character or two's difference in APL code. There's a benign form of mental illness learning anything, where one clutches onto something novel and obsesses over it, rather than asking "That was exciting! What novel idea will I learn in the next five minutes?" I have friends from my working class high school who still say "ASSUME makes an ask of you and me" as if they just heard it for the first time, while the most successful mathematicians that I know keep moving like sharks. I wouldn't stall too long thinking about Cracovians, as amusing a skim as the post provided.
- noosphr 1y ago
  I mean theres nothing special about naming binary operations on tensors of fixed rank. Matrices have some nice mathematical properties which is why they are studied so much in mathematics. But for number crunching there is no reason to prefer then to cracovians, or vice versa, without knowing what the underlying memory layout is in hardware.

semiinfinitely 1y ago

Uhh so it's just matrices where the left slot of matmul is transposed?

gus_massa 1y ago

> It turns out that multiplying cracovians by computers is not faster than multiplying matrices. That's very specific of Python. A few years ago we were multiplying a lot of matrices in Fortran and we tried to transpose one of the matrices before the multiplication. With -o0 it was a huge difference because the calculation used contiguous numbers and was more chache friendly. Anyway, with -o3 the compiler made some trick that made the difference disappear, but I never tried to understand what the compiler was doing.

wjholden 1y ago
I would expect that Julia could similarly show performance boosts here because of its column-major memory layout.

rundigen12 1y ago

Read to the end and... what was the point of that? Where's the payoff? There was a claim near the top that some things are easier to compute when viewed as cracovians. then some explanation, then suddently it switches to numpy and showing the time is the same. New title: "Cracovians are a Waste of (the Reader's) Time"?