War on Raze

A deep dive into optimizing k7 code by replacing the 'raze' workaround with deep primitives, exploring the implementation of 'explode' and 'implode' functions.
This is the story of what happened when I went down a rabbit hole. It starts with k7. If you press Ctrl-] in the k7 WASM console, this appears: x^x*/:x:2_!100. That's a prime number filter. There are faster ones - kparc.com's x,1_&&/80#'!:'x is beautiful - but this one is really short. The above k7 snippet: gets the numbers 0 to 99 (!100), drops the first two (2_), and assigns that to x (x:), gets the product of each element in x with each element of x, forming a table (x*/:x), then removes the elements in that table from x (x^). The snippet wasn't always so short: it relies on rank-sensitive 'except' (^), which Arthur introduced around 1 May 2019. Before then, it was x^,/x*/:x:2_!100. This older version inserts a step: it merges the result of multiply-each-right (*/:) into a single list via ,/, also known as 'raze'. This raze bugs me. It's a workaround: we wanted a flat list of results from our multiply, but it came out nested, and we had to flatten it manually. How could we have done this without using raze, and without rank-sensitivity? Well, what if each-right isn't the right tool? We could split the pair generation and multiplication: x^*/x@!2##x:2_!100. The speed is about the same, but it's more complicated because we have to manage the generation of indices. Could we skip indexing? In an array language? There's another primitive that has a reputation of keeping data flat: deep verbs. Shallow verbs take a list in which each element represents an index. Deep verbs take a list in which each element represents a dimension. For example, odometer ! m,n,... generates all paths within a matrix. Where (&) is a shallow verb. A deep version of where should produce something like odometer. We can use deep where to generate the pairs. There's just one problem: deep where isn't a standard k verb yet, though ngn/k has an experimental version. I went in a different direction: implementing an explode verb. A shallow verb can act similar to a deep verb if it's working on an 'exploded' dict form of the data. This gives us our razeless and index-free prime generator. We can now define deep forms of other shallow verbs such as group = and find ?. I liked that reshape becomes a flipped odometer strapped to a value list coupled with an implode. Finally, these tools might offer performance boosts and eliminate duplicate logic in nested structures.
Source: Hacker News


















