Phase</th>	Ryzen 7 2700X (μs)</th>	Core i7-1355U (μs)</th></tr></thead>
Parsing</td>	702</td>	519</td></tr>
Optimization (deduplication by hash consing)</td>	1,207</td>	885</td></tr>
Generating machine code</td>	5,029</td>	2,627</td></tr> </tbody></table> Compilation is not totally insignificant, but I didn't try to optimize it.</p> It's interesting to see that the laptop, while slower overall, beats the desktop by so much on startup with its newer CPU and better single core performance.</p> Parallelization</h3> On the desktop scaling is fairly close to linear up to the number of physical cores:</p> </p> Reusing spill slots</h3> Naively each time we spill a value it would get a fresh "slot" in memory. However once a spilled value is loaded for the last time, a future spill can reuse that slot. It makes sense that this would help with cache locality, but the effect was much larger than I expected.</p> On the laptop[^overheat] this brought rendering time from 119 ms</strong> to 54 ms</strong>. The actual number of slots needed went from 2781 (~89 kB) to 142 (~4.5 kB).</p> Number of registers</h3> The above effect made me start to question how much register use really mattered, or if it was all just a question of cache locality. Threads introduce quite a bit of noise, so rendering 128x128 on the desktop with one thread, from the minimum possible 3 registers up to using all 16:</p> </p> Not at all a shape I expected, to say the least.[^laptop] (Reading the processor manuals to get to the bottom of this is a rabbit hole for another time.)</p> However the lack of a large monotonic improvement makes sense considering the relatively small change in memory accesses:</p> </p> Conclusion</h2> You can find the code</a> on GitHub.</p> If there is any moral to this story, it's that performance at this level is very unpredictable. It's a bit of a cliche that you should measure and not guess performance, but it was more true here than I expected.</p> Future experiments to try</h3> Try doubling the pixel batch size on a CPU with AVX-512 (or quadrupling with FP16?)</li> Is there a topological sorting of the instructions which has better locality and substantially less required spilling?</li> Keeping the simplicity of the interpreter but improving the cache locality by operating in medium sized batches of pixels; is memory bandwidth a problem if it mostly fits in L1 cache?</li> </ul> [^overheat] Sadly at this point in development my aging desktop decided to start overheating when running the benchmarks...</p> [^laptop] The shape on the laptop is somewhat closer to monotonic but still a bit strange.</p> Talk: A deep dive into Rust UI 2024-09-14T00:00:00+00:00 </iframe> I gave this talk at the Rust London User Group</a> in July.</p> While the difficulties of creating an ergonomic (and performant) UI framework in Rust are well known, attempting my own convinced me that the design space is still relatively unexplored. This talk illustrates some of the general challenges, as well as specific tricks I learned while creating Ravel</a>.</p> Benchmarking Linux process sandboxing mechanisms 2023-10-22T00:00:00+00:00 Build systems are in the business of computing what are nominally pure functions, but in the messy world of Unix files and processes. If you care about your builds being correct, reliable, reproducible, or secure, you probably would would like them to avoid arbitrarily accessing the network or filesystem of the host. Sources of nondeterminism such as time would ideally be controlled as well. Maybe someday we'll perform all builds in a deterministic WebAssembly sandbox, but for now the state of the art is mostly Unix process isolation mechanisms.</p> Despite having worked on build systems for a good chunk of my career, I realized that I actually had a pretty weak intuition for the performance costs of any kind of sandboxing.</p> This is a very important question if you would like to design a build system or optimize the use of one. It could impact both what forms of sandboxing are viable, and at what granularity to apply them.</p> If sandboxing adds a lot of overhead on top of regular process invocations, then maybe you want sandboxing to apply on the level of "projects" in a multi-project build graph (as in Nix</a>1</a></sup>). If, on the other hand, sandboxing adds almost no overhead, then you can use it anywhere that already has a process boundary. Build systems like Bazel</a>2</a></sup> use a minimal form of sandboxing at the level of individual translation units (e.g. object files in C++), but as far as I know there is little precedent for "full" sandboxing at this granularity 3</a></sup>.</p> I couldn't find any good reference for the costs of the different sandboxing technologies provided by Linux, so I decided to measure the performance of a few popular tools.</p> Sandboxing on Linux</h2> In this post I'm only going to look at Linux. There are three major components of a build-grade process sandbox on Linux.</p> Namespaces</strong></a> give the sandboxed process a "clean room" environment separated from the user accounts, network, process IDs, etc. of the host system.</li> `pivot_root</code></strong></a> isolates the filesystem root to a sandbox directory.</li>` Bind mounts</strong></a> make it possible to virtualize the filesystem tree in the sandbox, so it doesn't need to correspond directly to some on-disk tree externally.</li> </ul> Process sandboxing has become prominent mainly thanks to Linux containers, an ecosystem and movement primarily oriented toward the needs of running backend services. Sadly, adoption in build systems still lags far behind the world of services. Since service startup latency isn't usually a huge concern, it also wouldn't surprise me if the relevant kernel code paths are not as optimized as they could be.</p> The benchmarks</h2> In each benchmark, the underlying process is true</code></a>, which exits immediately. The measured time always includes launching the process and waiting for its completion. The following setups are tested:</p> No sandboxing, just a regular Unix process</li> No sandboxing, but executed under sh -c</code></li> Minimal Bubblewrap</a> sandbox (just remounting /</code>)</li> Full Bubblewrap sandbox with all namespaces unshared</li> The unshare</code></a> command with all namespaces unshared</li> The Rust unshare</code> crate</a> with all namespaces unshared</li> docker run</code> (with alpine</code></a> image)</li> podman run</code> (with alpine</code> image)</li> </ul> Just a regular true</code> process gives us the baseline for process invocation latency. Running under sh</code> doesn't tell us anything about sandboxing, but it's a useful sense of scale because wrapper shell scripts are extremely common in most build setups.</p> Bubblewrap / bwrap</code> is a very nice low level sandboxing tool which supports a huge number of isolation features. The second configuration relative to the first should isolate the relative cost of the namespace setup specifically. Since Bubblewrap requires running under an explicit filesystem sandbox, I add unshare</code> (which just does namespaces) to try to isolate the cost of the bind mount.</p> In a real build system which natively understands sandboxing, it might make sense to perform the sandbox setup system calls directly in the build system process. This would only incur one level of process overhead, rather than adding an extra process by using something like bwrap</code>. To simulate this, I test a Rust library which sets up Linux namespaces in-process, and then invokes the target process (still true</code>). Since the benchmark script is written in Rust and the measurements are taken internal to the process, this avoids the extra process overhead.</p> For completeness, I test Docker</a> and Podman</a> since they provide most of the isolation features relevant to a build system (plus a lot more features relevant to services).</p> A note on $PATH</code> resolution</h3> I ran the benchmarks in a Nix shell</a>, which sets a very long $PATH</code> list, so I (correctly) guessed that searching for true</code> by name might affect the results. Except for the container tests (where this shouldn't be as much of an issue with the alpine</code> image), each mechanism invokes true</code> through a resolved path.</p> While I'm not super interested in the overhead incurred by long $PATH</code> searches, I also added a benchmark which just invokes true</code> as is. Like sh -c</code>, this can help give a useful sense of scale.</p> Results</h2> My machine is a Ryzen 7 2700X desktop running NixOS 23.05 on ZFS. None of the benchmarks had any significant variance (after warm-up time), so the numbers here are all just averages.</p> Mechanism</th> Total time (ms)</th></tr></thead> normal process, resolved</td> 0.653</td></tr> normal process, unresolved</td> 1.067</td></tr> sh -c</code></td> 3.670</td></tr> bwrap</code></td> 2.171</td></tr> bwrap</code> with namespaces</td> 2.903</td></tr> unshare</code> command</td> 3.110</td></tr> unshare</code> library</td> 2.293</td></tr> docker run</code></td> 248</td></tr> podman run</code></td> 402</td></tr> </tbody></table> Omitting the container tests for reasons of scale, I've plotted both the total time of each mechanism, and the marginal overhead on top of the baseline process invocation time:</p> </p> The benchmark code</a> can be found on GitHub.</p> Conclusions</h2> Sandboxing via Linux namespaces adds an overhead a few times the overhead of an unsandboxed process. However it's still quite fast, so sandboxing should be used whenever possible in build systems. In these benchmarks we used true</code> which should have negligible run time, but a real compiler or code generator likely dwarfs the process invocation time, sandboxed or not.</p> It's important to remember that any process invocation is still about five orders of magnitude slower than a function call. It would be really cool to design a compiler which can use a build system to cache the compilation of individual functions, for example, but the overhead would be prohibitively expensive.</p> Bubblewrap</h3> Bubblewrap looks pretty well optimized. It's wholly superior to the unshare</code> command, and its namespacing overhead (relative to the filesystem-only configuration) is significantly less than the unshare</code> library, which avoids a whole process layer. This is impressive considering Bubblewrap's focus is security.</p> Containers</h3> I was quite surprised that Docker and Podman had such high latency. They obviously do a lot more bookkeeping, but I thought it would be maybe an order of magnitude slower at worst. Widespread familiarity with container technology may make it seem like an attractive sandboxing mechanism for builds (and technically it checks most of the right boxes), but the overhead makes it problematic. You certainly could not use it for translation-unit level granularity, but the fact that it's in the realm of human-observable latency makes me want to rule it out even for very course granularity.</p> Shell</h3> I was not surprised to find that adding a shell wrapper has an overhead significantly greater than the theoretical minimum (i.e. one extra process overhead). However the fact that it was roughly on par with the sandboxing mechanisms is a useful heuristic: if you wouldn't think twice about wrapping a command in a shell script (very common in build systems), you probably shouldn't care about sandboxing overhead.</p> $PATH</code> resolution</h3> Resolving true</code> on each invocation added a very significant overhead. I really expected filesystem caching in the kernel to take care of this after the initial run, but apparently not.</p> ^1</sup>Technically this isn't enforced by Nix itself, but it's the canonical way to use it as defined by nixpkgs</a>.</p> </div> ^{2</sup> It's always bothered me that Bazel includes "correct" in its motto, when it doesn't even try to stop you from using tools from the host.</p> </div>}^3</sup>Bazel has --experimental_use_hermetic_linux_sandbox</code></a> but it's not widely used and very off the beaten path.</p> </div> A trilemma in functional reactive programming models 2023-06-19T00:00:00+00:00 A few months ago I tried to implement a functional reactive programming library in Rust. It was a good experience, but ultimately I abandoned it due to limitations of Rust that were preventing me from making a system as ergonomic as I would like. However, that's not what this post is about.</p> The FRP framework I am most familiar with is Reflex</a>, and to start, I mostly tried to copy its semantics. It being Rust, I was determined to make it performant, eventually in constant factors, but to start at least getting the asymptotics right. However I ran into an algorithmic difficulty which led me to question the semantics exposed by Reflex and other similar frameworks. That's what this post is about.</p> Eventually I concluded that there are a variety of alternate semantics that are both easier to implement and more efficient, without losing any important expressivity (at least for GUI use cases).</p> This post assumes general familiarity with push-based (or hybrid push/pull</a>) higher order FRP like Reflex or Sodium</a>. Note that this is actually a pretty small category. Most older FRP implementations are pull-based (i.e. not efficient at all) and many newer frameworks commonly referred to as FRP (like ReactiveX</a> and its derivatives) lack the semantic foundations which make "true" FRP both very expressive and hard to implement.</p> I will be using terminology from Reflex but unfortunately this differs quite a bit between different FRP frameworks. In particular, some systems use Signal</code> or Stream</code> for what Reflex calls Event</code>, but I prefer "event" as it is more intuitive.</p> However, there is one important potential terminological confusion with the term "event" which deserves clarification. In mainstream GUI systems, an event usually refers to an occurrence</em> of an abstract potentially-happening thing. In (first class) FRP, we build a reactive system by using various combinators to construct values that represent the abstract</em> potential behavior of the system. So an "event" object represents something that may fire with actual concrete values at various points in the program. (Denotationally, an event for values of type a</code> is like a mapping Time -> Maybe a</code>.) Here "event" always refers to this abstract object, not the concrete occurrence of the event.</p> Event graphs</h2> Efficient implementations of FRP are primarily concerned with a specific task, that is taking input events (e.g. button clicks) and activating a subset of observable output events. The output events could have some IO effect (e.g. update widgets on the page) or update internal reactive data cells (Behavior</code>/Dynamic</code> in Reflex).</p> One way to do this would be to "run" the entire known graph on each frame, like in an immediate-mode GUI</a>. This isn't a totally crazy way to operate for most applications, but as the ongoing debates in the web world about virtual DOM diffing vs. fine grained reactivity show, it is less than ideal.</p> FRP implementations typically use behind the scenes something analogous the "observer pattern" from classic retained-mode</a> GUI frameworks to attach to each event a list of downstream events which need to be processed if the upstream event fires. This allows them to (basically) start from the input events and wake up events downstream only as necessary until reaching some subset of the existing output events at the other end of the graph.</p> With basic event combinators like filter</code>, map</code>, etc., this is straightforward. The complications arise when you add complex combinators that have more than a single input:</p> merge </span>:: Event </span>a </span>-> Event </span>b </span>-> Event</span> (</span>These </span>a b</span>) </span>switch </span>:: Behavior</span> (</span>Event </span>a</span>) </span>-> Event </span>a </span></code></pre> Behavior</code></a> is a time-varying FRP value, and These</code></a> is a very nice Haskell type representing two optional values where at least one must be present:</p> data </span>These</span> a b </span> = </span>This</span> a </span> \| </span>That</span> b </span> \| </span>These</span> a b </span></code></pre> The merge</code>1</a></sup> combinator</h3> The hard part about merge</code> is that it waits on two input events, but after either one of them fires, we can't yet know what the resulting value will be. We have to first know whether the other side will also fire.</p> The only way to get the correct answer and do minimal work, is to process potentially triggered events in a topological ordering of the relation implied by input dependencies. In other words, if one input to a merge</code> fires, we know that the merge</code> event itself will fire, but we should delay processing it until all "earlier" events (which necessarily include its inputs) have been processed.</p> To handle this, we can assign each event a "height" based on its inputs, and use these as priorities in a queue. A single-input event, like map</code> or filter</code>, would have a height one higher than its input, and merge</code> would have a height one higher than the maximum of its inputs. This automatically gives a topological ordering by construction.</p> The switch</code></a> combinator</h3> In a fully static event graph, assigning heights is easy, because they can be calculated directly when an event is constructed. But we run into a problem with switch</code>. The value of the Behavior</code> can change as time goes on, and because FRP is higher-order, it can come from "anywhere" so there is no statically known subset of events it could be.</p> Thus, if we want to use the "height strategy" to handle merge</code> events, we'll need to dynamically update the height of a switch</code> event when it changes. Just updating the switch</code> event itself isn't a problem, but we need to maintain the invariant that an event has a higher height than its inputs. We now need to trace down the graph, finding any downstream events which transitively have it as an input, and update their heights to maintain the invariant.2</a></sup></p> Now we're back to the same worst case performance as an immediate-mode GUI. In practice, it usually</em> won't be that bad, but we have no easy ways to reason about when it will happen.</p> The trilemma</h2> We need to make more precise what it would mean to do efficient event propagation. Obviously we need to do work at least linear in the number of events that actually fire in a frame. Additionally, any event that does fire may be the input to another event, which is therefore "potentially firing" but may decide to fire or not depending on the values it receives. So efficient event propagation would mean doing work linear in the number of potentially firing events in a frame (which includes those that actually do fire). Now we get to the trilemma:</p> If we have both switch</code> and merge</code>, changing a switch</code> and running an event propagation frame requires in the worst case work linear on the total number of existing events, even if this dwarfs the number of potentially firing events.</em></p> If we get rid of switch</code>, we can assign heights statically and process potentially firing events in a priority queue.</li> If we get rid of merge</code>3</a></sup>, we don't even need heights, and we can process potentially firing events in depth-first order.</li> If we keep both, to process merge</code>s correctly we need to produce a new topological ordering which respects the new constraints changed by a switch</code>.</li> </ul> Formally proving that the final case cannot be done efficiently is beyond the scope of this post, but hopefully the intuition is clear. Because a switch</code> event can choose "any" event as its input, it can add an arbitrary ordering constraint to the graph, which means we can reduce the problem to online</em> topological ordering.</p> Actually existing GUIs</h2> Given that I was trying to implement FRP in Rust, I was determined to find a way to keep predictable performance and minimal graph traversal as a core feature. Moreover, it occurred to me that, while retained-mode GUI programming is famously hard to do correctly, people do</em> (with effort) create large correct GUI applications, and nowhere in their runtime behavior can we observe anything analogous to the downstream global event trace required to combine switch</code> and merge</code>.</p> This suggests one of the following possibilities:</p> There are useful GUI paradigms that inherently</em> require global event tracing to implement correctly, and well known large GUIs do not express these patterns simply because</em> they are hard to implement.</li> There are (effectively) no real world GUI paradigms which require global event tracing.</li> There is</em> something analogous to global event tracing in certain existing GUI implementations, but it's not obvious precisely because it's more clearly handled with a domain-specific algorithm than a pervasive part of the reactivity machinery.</li> </ul> I think the last one is ultimately the answer. A helpful analogy is tracing garbage collection. With manual memory management (i.e. Rust), to implement an arbitrary algorithm, in the worse case you have to embed a garbage collector into the implementation. However in practice this doesn't really happen unless you're implementing either an interpreter for a GC language, or some kind of graph algorithm where incremental cleanup is required. While it's certainly easier to use a language with GC and let the host GC handle these cases, it seems the surface area where the GC needs to run is almost always limited to a relatively isolated problem domain. I like to think that a future Rust-like language would make it easy to use garbage collection as a library for these cases, with the tracing only happening on the domain heap and not the whole program.</p> Using Rust has made me moderately confident that, outside of these specialized domains, garbage collection isn't really required for general purpose programming. And actually, it seems like we should be even more confident that "event tracing" isn't required for GUIs.</p> General purpose programs are higher order and have extremely complex dynamics that are very hard for experienced programmers to fit in their heads, and yet their object lifetimes are still simple enough that they can almost always be handled by stack regions or reference counting (i.e. only statically known cycles). On the other hand, GUIs typically have only first order behavior, and even the most complex GUI behavior pales in comparison to the complexity in "backend" systems. Although bugs in GUIs are common (probably because they aren't using FRP!), describing the correct behavior in a modular fashion is usually not hard.</p> The main exception is GUIs which embed higher order behavior from some other domain, for example Excel. But of course Excel has a custom incremental update engine, it doesn't use GUI event handlers to keep track of cell dependencies.</p> Likewise, games have entity component systems</a>, not elaborate webs of event handlers. (And at the level of memory management, they often use keys/indices rather than a web of references.)</p> In the case of reactivity, what can we achieve if we assume that "extremely dynamic event graphs" correspond to limited problem domains better handled by the application? Stay tuned for part two, where I'll discuss the space of possible semantics for keeping the core ideas from FRP, without the need to ever do global event tracing.</p> ^1</sup>I've presented here a simplification of Reflex's merge</code></a>, since the actual version has some type-level complexity that is not relevant to FRP itself.</p> </div> ^{2</sup> There is a trick we can use to share the work of keeping heights up to date for a chain of single-input events like map</code> or filter</code>, but this won't help us asymptotically unless we assume these make up increasing fractions of larger FRP graphs.</p> </div> ^{3</sup> This is the option taken by all other existing systems I'm aware of. Please let me know if you find a counterexample!</p> </div>}}

The Prospero Challenge: Artisinal JIT

2025-04-23T00:00:00+00:00

I recently got effectively nerd-sniped by Matt Keeter's Prospero Challenge</a>.

The reference implementation is a simple interpreter which walks through the instruction list, and uses NumPy to compute in bulk the values at each pixel location. Intermediate results (complete grids of values) are stored in memory for use by later instructions. NumPy takes advantage of SIMD for the actual calculations, but the end result is still fairly slow because the memory traffic is extremely high. Matt notes that this allocates 60 GB of RAM for intermediate results (but some basic optimization</a> helps a lot with peak memory usage).

Since the data dependencies are within each pixel and not across pixels, my first thought was that we can at least improve cache usage a lot by iterating over pixels and computing the full expression, rather than iterating over instructions and computing the full grid. Of course, we can still use SIMD to compute multiple pixels at a time up to the supported width of our favorite SIMD target.

Even better, a quick analysis confirmed what I was hoping for: a significant fraction of intermediate results are used only once, and often shortly after they are computed. So in theory we can keep these in registers and never store them in memory at all!

However, computing "deep" slices ruins one advantage the reference implementation had on its side: the interpreter loop overhead really starts to matter if you need to run it hundreds of thousands of times. Although Python is notoriously slow, given that there are ~1 million pixels and ~8 thousand instructions, I suspect (but did not measure) that the interpreter overhead is negligible in the reference implementation.

If we can't interpret, what are our options? Other submissions tried all kinds of off the shelf high-powered JIT/compiler tools, like CUDA, LLVM, Cranelift, and others including Matt's own Fidget system for implicit surfaces. Cranelift has a reputation for being easy to set up and use, but LLVM and CUDA... not so much. But even a friendly JIT like Cranelift seems pretty overpowered considering that we have to support 11 instructions and no control flow.

How hard can it be to just make our own JIT and output some simple AVX instructions?

The instructions</h2>
I don't currently have a machine with AVX-512, so, so let's look at what we can do with AVX2 for each instruction in prospero.vm</code>. Each 256 bit AVX register (ymm0</code>-ymm15</code>) can hold 8 single-precision floating point numbers, so we'll evaluate 8 pixels at a time. For demonstration purposes we'll store the result in ymm0</code> in each case, with arguments in ymm1</code>/ymm2</code>.
var-x</code>/var-y</code></h3> Let's just assume these are already stored (aligned!) in memory, varying appropriately for the pixels we're computing:
vmovaps ymm0, [rdi] vmovaps ymm0, [rsi] </code></pre>
const</code></h3> We can't directly broadcast a constant to an AVX register; we need to first move it into an SSE register (xmm0</code>), which occupies the bottom part of its corresponding AVX register. Annoyingly, we also can't put a constant straight into the SSE register either.
mov eax, N vmovd xmm0, eax vbroadcastss ymm0, xmm0 </code></pre> This felt like a lot of instructions and it seems like loading from memory is more typical. I got a minor 1.5% improvement by passing an array with all the constants pre-broadcasted and just loading them directly, but it also makes the compiler code somewhat more complex.
vmovaps ymm0, [r8 + OFFSET] </code></pre>
The binary operators</h3>
These are easy! Thankfully each binary operator corresponds to exactly one 3-parameter instruction:
vaddps ymm0, ymm1, ymm2 vsubps ymm0, ymm1, ymm2 vmulps ymm0, ymm1, ymm2 vmaxps ymm0, ymm1, ymm2 vminps ymm0, ymm1, ymm2 </code></pre>
neg</code></h3> I was kind of surprised that there is no native negation instruction. The Internet suggests a few different ways to implement it, but I chose the simple option of getting a zero register with XOR and then subtracting: vxorps ymm0, ymm0, ymm0 vsubps ymm0, ymm0, ymm1 </code></pre> square</code></h3> Multiply by itself, of course: vmulps ymm0, ymm1, ymm1 </code></pre> sqrt</code></h3> Thankfully also just one instruction: vsqrtps ymm0, ymm1 </code></pre> Generating machine code</h2> Generating the correct machine code directly for our desired instructions would not be terribly difficult but it would be incredibly tedious. The Rust crate iced-x86</code></a> frees us from having to worry about encoding details. use iced_x86::code_asm::*; let mut asm = CodeAssembler::new(64).unwrap(); asm.vmovaps(ymm0, ptr(rsi)).unwrap(); </code></pre> Using the literal number 64</code> rather than an enum to indicate 64-bit mode is a bit WTF but the library seems otherwise of decent quality. Register allocation</h2> This is the main hard part, especially as there are far more intermediate values than registers, so spill choices ought to matter a lot. I wanted to see how well a really simple linear-scan allocator would do. At a high level, the algorithm is pretty simple: Compute live ranges for each intermediate value</li> Walk over all the instructions, and allocate parameter/output slots greedily If a parameter is already in a register, use that</li> If a parameter has been spilled to memory, choose a register to load it to</li> When we run out of registers, choose a register to spill based on some heuristic</li> </ul> </li> </ul> The choice of heuristic is the interesting part. I went with the dead simple thing of always spilling the value that has the earliest last use. This is not great though if something is used only once far in the future and it holds onto a register the whole time. I also tried spilling the value with the latest next use</a>, but it was more complex and had slightly worse performance. It's also possible to avoid using registers altogether in some cases as AVX can take some parameters from memory, but for simplicity I always loaded spilled values into a fresh register. Experiments</h2> These things turned out to make essentially no difference: Aligning the memory usage and using vmovaps</code> over vmovups</code></li> Hash consing</a> the expressions to remove duplicate values, which eliminates 275 out of the 7866 instructions</li> </ul> These made a big difference: Reusing spill slots when they become dead (more on this below)</li> Parallelizing the rendering with rayon</code></a></li> </ul> Results</h2> TLDR: on my Ryzen 7 2700X desktop (8 cores, 16 threads), the full 1024x1024 image renders in ~48 ms. For comparison on my ThinkPad with a Core i7-1355U (2 P-cores, 8 E-cores, 12 threads), it takes ~80 ms. This seems pretty solid for a CPU-only solution that is not using any fancy expression simplification techniques. Startup latency</h3> Since we have a JIT, there is a startup cost to instantiating the machine code for the program. Phase</th> Ryzen 7 2700X (μs)</th> Core i7-1355U (μs)</th></tr></thead> Parsing</td> 702</td> 519</td></tr> Optimization (deduplication by hash consing)</td> 1,207</td> 885</td></tr> Generating machine code</td> 5,029</td> 2,627</td></tr> </tbody></table> Compilation is not totally insignificant, but I didn't try to optimize it. It's interesting to see that the laptop, while slower overall, beats the desktop by so much on startup with its newer CPU and better single core performance. Parallelization</h3> On the desktop scaling is fairly close to linear up to the number of physical cores: Reusing spill slots</h3> Naively each time we spill a value it would get a fresh "slot" in memory. However once a spilled value is loaded for the last time, a future spill can reuse that slot. It makes sense that this would help with cache locality, but the effect was much larger than I expected. On the laptop[^overheat] this brought rendering time from 119 ms to 54 ms. The actual number of slots needed went from 2781 (~89 kB) to 142 (~4.5 kB). Number of registers</h3> The above effect made me start to question how much register use really mattered, or if it was all just a question of cache locality. Threads introduce quite a bit of noise, so rendering 128x128 on the desktop with one thread, from the minimum possible 3 registers up to using all 16: Not at all a shape I expected, to say the least.[^laptop] (Reading the processor manuals to get to the bottom of this is a rabbit hole for another time.) However the lack of a large monotonic improvement makes sense considering the relatively small change in memory accesses: Conclusion</h2> You can find the code</a> on GitHub. If there is any moral to this story, it's that performance at this level is very unpredictable. It's a bit of a cliche that you should measure and not guess performance, but it was more true here than I expected. Future experiments to try</h3> Try doubling the pixel batch size on a CPU with AVX-512 (or quadrupling with FP16?)</li> Is there a topological sorting of the instructions which has better locality and substantially less required spilling?</li> Keeping the simplicity of the interpreter but improving the cache locality by operating in medium sized batches of pixels; is memory bandwidth a problem if it mostly fits in L1 cache?</li> </ul> [^overheat] Sadly at this point in development my aging desktop decided to start overheating when running the benchmarks... [^laptop] The shape on the laptop is somewhat closer to monotonic but still a bit strange. Talk: A deep dive into Rust UI 2024-09-14T00:00:00+00:00 </iframe> I gave this talk at the Rust London User Group</a> in July. While the difficulties of creating an ergonomic (and performant) UI framework in Rust are well known, attempting my own convinced me that the design space is still relatively unexplored. This talk illustrates some of the general challenges, as well as specific tricks I learned while creating Ravel</a>. Benchmarking Linux process sandboxing mechanisms 2023-10-22T00:00:00+00:00 Build systems are in the business of computing what are nominally pure functions, but in the messy world of Unix files and processes. If you care about your builds being correct, reliable, reproducible, or secure, you probably would would like them to avoid arbitrarily accessing the network or filesystem of the host. Sources of nondeterminism such as time would ideally be controlled as well. Maybe someday we'll perform all builds in a deterministic WebAssembly sandbox, but for now the state of the art is mostly Unix process isolation mechanisms. Despite having worked on build systems for a good chunk of my career, I realized that I actually had a pretty weak intuition for the performance costs of any kind of sandboxing. This is a very important question if you would like to design a build system or optimize the use of one. It could impact both what forms of sandboxing are viable, and at what granularity to apply them. If sandboxing adds a lot of overhead on top of regular process invocations, then maybe you want sandboxing to apply on the level of "projects" in a multi-project build graph (as in Nix</a>1</a>). If, on the other hand, sandboxing adds almost no overhead, then you can use it anywhere that already has a process boundary. Build systems like Bazel</a>2</a> use a minimal form of sandboxing at the level of individual translation units (e.g. object files in C++), but as far as I know there is little precedent for "full" sandboxing at this granularity3</a>. I couldn't find any good reference for the costs of the different sandboxing technologies provided by Linux, so I decided to measure the performance of a few popular tools. Sandboxing on Linux</h2> In this post I'm only going to look at Linux. There are three major components of a build-grade process sandbox on Linux. Namespaces</a> give the sandboxed process a "clean room" environment separated from the user accounts, network, process IDs, etc. of the host system.</li> pivot_root</code></a> isolates the filesystem root to a sandbox directory.</li> Bind mounts</a> make it possible to virtualize the filesystem tree in the sandbox, so it doesn't need to correspond directly to some on-disk tree externally.</li> </ul> Process sandboxing has become prominent mainly thanks to Linux containers, an ecosystem and movement primarily oriented toward the needs of running backend services. Sadly, adoption in build systems still lags far behind the world of services. Since service startup latency isn't usually a huge concern, it also wouldn't surprise me if the relevant kernel code paths are not as optimized as they could be. The benchmarks</h2> In each benchmark, the underlying process is true</code></a>, which exits immediately. The measured time always includes launching the process and waiting for its completion. The following setups are tested: No sandboxing, just a regular Unix process</li> No sandboxing, but executed under sh -c</code></li> Minimal Bubblewrap</a> sandbox (just remounting /</code>)</li> Full Bubblewrap sandbox with all namespaces unshared</li> The unshare</code></a> command with all namespaces unshared</li> The Rust unshare</code> crate</a> with all namespaces unshared</li> docker run</code> (with alpine</code></a> image)</li> podman run</code> (with alpine</code> image)</li> </ul> Just a regular true</code> process gives us the baseline for process invocation latency. Running under sh</code> doesn't tell us anything about sandboxing, but it's a useful sense of scale because wrapper shell scripts are extremely common in most build setups. Bubblewrap / bwrap</code> is a very nice low level sandboxing tool which supports a huge number of isolation features. The second configuration relative to the first should isolate the relative cost of the namespace setup specifically. Since Bubblewrap requires running under an explicit filesystem sandbox, I add unshare</code> (which just does namespaces) to try to isolate the cost of the bind mount. In a real build system which natively understands sandboxing, it might make sense to perform the sandbox setup system calls directly in the build system process. This would only incur one level of process overhead, rather than adding an extra process by using something like bwrap</code>. To simulate this, I test a Rust library which sets up Linux namespaces in-process, and then invokes the target process (still true</code>). Since the benchmark script is written in Rust and the measurements are taken internal to the process, this avoids the extra process overhead. For completeness, I test Docker</a> and Podman</a> since they provide most of the isolation features relevant to a build system (plus a lot more features relevant to services). A note on $PATH</code> resolution</h3> I ran the benchmarks in a Nix shell</a>, which sets a very long $PATH</code> list, so I (correctly) guessed that searching for true</code> by name might affect the results. Except for the container tests (where this shouldn't be as much of an issue with the alpine</code> image), each mechanism invokes true</code> through a resolved path. While I'm not super interested in the overhead incurred by long $PATH</code> searches, I also added a benchmark which just invokes true</code> as is. Like sh -c</code>, this can help give a useful sense of scale. Results</h2> My machine is a Ryzen 7 2700X desktop running NixOS 23.05 on ZFS. None of the benchmarks had any significant variance (after warm-up time), so the numbers here are all just averages. Mechanism</th> Total time (ms)</th></tr></thead> normal process, resolved</td> 0.653</td></tr> normal process, unresolved</td> 1.067</td></tr> sh -c</code></td> 3.670</td></tr> bwrap</code></td> 2.171</td></tr> bwrap</code> with namespaces</td> 2.903</td></tr> unshare</code> command</td> 3.110</td></tr> unshare</code> library</td> 2.293</td></tr> docker run</code></td> 248</td></tr> podman run</code></td> 402</td></tr> </tbody></table> Omitting the container tests for reasons of scale, I've plotted both the total time of each mechanism, and the marginal overhead on top of the baseline process invocation time: The benchmark code</a> can be found on GitHub. Conclusions</h2> Sandboxing via Linux namespaces adds an overhead a few times the overhead of an unsandboxed process. However it's still quite fast, so sandboxing should be used whenever possible in build systems. In these benchmarks we used true</code> which should have negligible run time, but a real compiler or code generator likely dwarfs the process invocation time, sandboxed or not. It's important to remember that any process invocation is still about five orders of magnitude slower than a function call. It would be really cool to design a compiler which can use a build system to cache the compilation of individual functions, for example, but the overhead would be prohibitively expensive. Bubblewrap</h3> Bubblewrap looks pretty well optimized. It's wholly superior to the unshare</code> command, and its namespacing overhead (relative to the filesystem-only configuration) is significantly less than the unshare</code> library, which avoids a whole process layer. This is impressive considering Bubblewrap's focus is security. Containers</h3> I was quite surprised that Docker and Podman had such high latency. They obviously do a lot more bookkeeping, but I thought it would be maybe an order of magnitude slower at worst. Widespread familiarity with container technology may make it seem like an attractive sandboxing mechanism for builds (and technically it checks most of the right boxes), but the overhead makes it problematic. You certainly could not use it for translation-unit level granularity, but the fact that it's in the realm of human-observable latency makes me want to rule it out even for very course granularity. Shell</h3> I was not surprised to find that adding a shell wrapper has an overhead significantly greater than the theoretical minimum (i.e. one extra process overhead). However the fact that it was roughly on par with the sandboxing mechanisms is a useful heuristic: if you wouldn't think twice about wrapping a command in a shell script (very common in build systems), you probably shouldn't care about sandboxing overhead. $PATH</code> resolution</h3> Resolving true</code> on each invocation added a very significant overhead. I really expected filesystem caching in the kernel to take care of this after the initial run, but apparently not. ^1Technically this isn't enforced by Nix itself, but it's the canonical way to use it as defined by nixpkgs</a>. </div> ^{2 It's always bothered me that Bazel includes "correct" in its motto, when it doesn't even try to stop you from using tools from the host. </div>}^3Bazel has --experimental_use_hermetic_linux_sandbox</code></a> but it's not widely used and very off the beaten path. </div> A trilemma in functional reactive programming models 2023-06-19T00:00:00+00:00 A few months ago I tried to implement a functional reactive programming library in Rust. It was a good experience, but ultimately I abandoned it due to limitations of Rust that were preventing me from making a system as ergonomic as I would like. However, that's not what this post is about. The FRP framework I am most familiar with is Reflex</a>, and to start, I mostly tried to copy its semantics. It being Rust, I was determined to make it performant, eventually in constant factors, but to start at least getting the asymptotics right. However I ran into an algorithmic difficulty which led me to question the semantics exposed by Reflex and other similar frameworks. That's what this post is about. Eventually I concluded that there are a variety of alternate semantics that are both easier to implement and more efficient, without losing any important expressivity (at least for GUI use cases). This post assumes general familiarity with push-based (or hybrid push/pull</a>) higher order FRP like Reflex or Sodium</a>. Note that this is actually a pretty small category. Most older FRP implementations are pull-based (i.e. not efficient at all) and many newer frameworks commonly referred to as FRP (like ReactiveX</a> and its derivatives) lack the semantic foundations which make "true" FRP both very expressive and hard to implement. I will be using terminology from Reflex but unfortunately this differs quite a bit between different FRP frameworks. In particular, some systems use Signal</code> or Stream</code> for what Reflex calls Event</code>, but I prefer "event" as it is more intuitive. However, there is one important potential terminological confusion with the term "event" which deserves clarification. In mainstream GUI systems, an event usually refers to an occurrence of an abstract potentially-happening thing. In (first class) FRP, we build a reactive system by using various combinators to construct values that represent the abstract potential behavior of the system. So an "event" object represents something that may fire with actual concrete values at various points in the program. (Denotationally, an event for values of type a</code> is like a mapping Time -> Maybe a</code>.) Here "event" always refers to this abstract object, not the concrete occurrence of the event. Event graphs</h2> Efficient implementations of FRP are primarily concerned with a specific task, that is taking input events (e.g. button clicks) and activating a subset of observable output events. The output events could have some IO effect (e.g. update widgets on the page) or update internal reactive data cells (Behavior</code>/Dynamic</code> in Reflex). One way to do this would be to "run" the entire known graph on each frame, like in an immediate-mode GUI</a>. This isn't a totally crazy way to operate for most applications, but as the ongoing debates in the web world about virtual DOM diffing vs. fine grained reactivity show, it is less than ideal. FRP implementations typically use behind the scenes something analogous the "observer pattern" from classic retained-mode</a> GUI frameworks to attach to each event a list of downstream events which need to be processed if the upstream event fires. This allows them to (basically) start from the input events and wake up events downstream only as necessary until reaching some subset of the existing output events at the other end of the graph. With basic event combinators like filter</code>, map</code>, etc., this is straightforward. The complications arise when you add complex combinators that have more than a single input: merge :: Event a -> Event b -> Event (These a b) switch :: Behavior (Event a) -> Event a </code></pre> Behavior</code></a> is a time-varying FRP value, and These</code></a> is a very nice Haskell type representing two optional values where at least one must be present: data These a b = This a | That b | These a b </code></pre> The merge</code>1</a> combinator</h3> The hard part about merge</code> is that it waits on two input events, but after either one of them fires, we can't yet know what the resulting value will be. We have to first know whether the other side will also fire. The only way to get the correct answer and do minimal work, is to process potentially triggered events in a topological ordering of the relation implied by input dependencies. In other words, if one input to a merge</code> fires, we know that the merge</code> event itself will fire, but we should delay processing it until all "earlier" events (which necessarily include its inputs) have been processed. To handle this, we can assign each event a "height" based on its inputs, and use these as priorities in a queue. A single-input event, like map</code> or filter</code>, would have a height one higher than its input, and merge</code> would have a height one higher than the maximum of its inputs. This automatically gives a topological ordering by construction. The switch</code></a> combinator</h3> In a fully static event graph, assigning heights is easy, because they can be calculated directly when an event is constructed. But we run into a problem with switch</code>. The value of the Behavior</code> can change as time goes on, and because FRP is higher-order, it can come from "anywhere" so there is no statically known subset of events it could be. Thus, if we want to use the "height strategy" to handle merge</code> events, we'll need to dynamically update the height of a switch</code> event when it changes. Just updating the switch</code> event itself isn't a problem, but we need to maintain the invariant that an event has a higher height than its inputs. We now need to trace down the graph, finding any downstream events which transitively have it as an input, and update their heights to maintain the invariant.2</a> Now we're back to the same worst case performance as an immediate-mode GUI. In practice, it usually won't be that bad, but we have no easy ways to reason about when it will happen. The trilemma</h2> We need to make more precise what it would mean to do efficient event propagation. Obviously we need to do work at least linear in the number of events that actually fire in a frame. Additionally, any event that does fire may be the input to another event, which is therefore "potentially firing" but may decide to fire or not depending on the values it receives. So efficient event propagation would mean doing work linear in the number of potentially firing events in a frame (which includes those that actually do fire). Now we get to the trilemma: If we have both switch</code> and merge</code>, changing a switch</code> and running an event propagation frame requires in the worst case work linear on the total number of existing events, even if this dwarfs the number of potentially firing events. If we get rid of switch</code>, we can assign heights statically and process potentially firing events in a priority queue.</li> If we get rid of merge</code>3</a>, we don't even need heights, and we can process potentially firing events in depth-first order.</li> If we keep both, to process merge</code>s correctly we need to produce a new topological ordering which respects the new constraints changed by a switch</code>.</li> </ul> Formally proving that the final case cannot be done efficiently is beyond the scope of this post, but hopefully the intuition is clear. Because a switch</code> event can choose "any" event as its input, it can add an arbitrary ordering constraint to the graph, which means we can reduce the problem to online topological ordering. Actually existing GUIs</h2> Given that I was trying to implement FRP in Rust, I was determined to find a way to keep predictable performance and minimal graph traversal as a core feature. Moreover, it occurred to me that, while retained-mode GUI programming is famously hard to do correctly, people do (with effort) create large correct GUI applications, and nowhere in their runtime behavior can we observe anything analogous to the downstream global event trace required to combine switch</code> and merge</code>. This suggests one of the following possibilities: There are useful GUI paradigms that inherently require global event tracing to implement correctly, and well known large GUIs do not express these patterns simply because they are hard to implement.</li> There are (effectively) no real world GUI paradigms which require global event tracing.</li> There is something analogous to global event tracing in certain existing GUI implementations, but it's not obvious precisely because it's more clearly handled with a domain-specific algorithm than a pervasive part of the reactivity machinery.</li> </ul> I think the last one is ultimately the answer. A helpful analogy is tracing garbage collection. With manual memory management (i.e. Rust), to implement an arbitrary algorithm, in the worse case you have to embed a garbage collector into the implementation. However in practice this doesn't really happen unless you're implementing either an interpreter for a GC language, or some kind of graph algorithm where incremental cleanup is required. While it's certainly easier to use a language with GC and let the host GC handle these cases, it seems the surface area where the GC needs to run is almost always limited to a relatively isolated problem domain. I like to think that a future Rust-like language would make it easy to use garbage collection as a library for these cases, with the tracing only happening on the domain heap and not the whole program. Using Rust has made me moderately confident that, outside of these specialized domains, garbage collection isn't really required for general purpose programming. And actually, it seems like we should be even more confident that "event tracing" isn't required for GUIs. General purpose programs are higher order and have extremely complex dynamics that are very hard for experienced programmers to fit in their heads, and yet their object lifetimes are still simple enough that they can almost always be handled by stack regions or reference counting (i.e. only statically known cycles). On the other hand, GUIs typically have only first order behavior, and even the most complex GUI behavior pales in comparison to the complexity in "backend" systems. Although bugs in GUIs are common (probably because they aren't using FRP!), describing the correct behavior in a modular fashion is usually not hard. The main exception is GUIs which embed higher order behavior from some other domain, for example Excel. But of course Excel has a custom incremental update engine, it doesn't use GUI event handlers to keep track of cell dependencies. Likewise, games have entity component systems</a>, not elaborate webs of event handlers. (And at the level of memory management, they often use keys/indices rather than a web of references.) In the case of reactivity, what can we achieve if we assume that "extremely dynamic event graphs" correspond to limited problem domains better handled by the application? Stay tuned for part two, where I'll discuss the space of possible semantics for keeping the core ideas from FRP, without the need to ever do global event tracing. ^1I've presented here a simplification of Reflex's merge</code></a>, since the actual version has some type-level complexity that is not relevant to FRP itself. </div> ^{2 There is a trick we can use to share the work of keeping heights up to date for a chain of single-input events like map</code> or filter</code>, but this won't help us asymptotically unless we assume these make up increasing fractions of larger FRP graphs. </div> ^{3 This is the option taken by all other existing systems I'm aware of. Please let me know if you find a counterexample! </div>}}

Ken Micklas

The Prospero Challenge: Artisinal JIT

Parallelization</h3> On the desktop scaling is fairly close to linear up to the number of physical cores:</p> </p>

Conclusion</h2> You can find the code</a> on GitHub.</p> If there is any moral to this story, it's that performance at this level is very unpredictable. It's a bit of a cliche that you should measure and not guess performance, but it was more true here than I expected.</p>

Talk: A deep dive into Rust UI

Benchmarking Linux process sandboxing mechanisms

A trilemma in functional reactive programming models

Parallelization</h3>
On the desktop scaling is fairly close to linear up to the number of physical cores:</p>
</p>

Conclusion</h2>
You can find the code</a> on GitHub.</p>
If there is any moral to this story, it's that performance at this level is very unpredictable. It's a bit of a cliche that you should measure and not guess performance, but it was more true here than I expected.</p>