Xpact-core vs eXpat

This report presents benchmark results for Xpact-core, a pure-Eiffel XML parser porting the behaviour of the widely-used C library eXpat (libexpat). It accompanies the paper Finding a Billion-User Project for Eiffel: How DbC Catches Security Flaws Rust Misses, where the motivation and architecture of Xpact-core are described in detail.

Making it fast was the easy part

Getting Xpact to run quickly turned out to be relatively straightforward. The hard part was making it correct — matching eXpat's output exactly across every edge case: predefined entity resolution, partial-token buffer boundaries, CDATA section handling, namespace attribute semantics, and character-reference encoding. A mismatch in any one of these produces a different CRC-32 value from the two parsers, which is immediately detectable.

A dual-purpose correctness and benchmarking tool

To verify correctness and measure performance in the same run, I had Claude write a C program — xml_crc_32.c — that serves two purposes:

1. It computes a CRC-32/ISO-HDLC hash over one of five parse-event data types (selectable on the command line), producing a single value that can be compared directly against Xpact's output.
2. It counts complete parse passes in a fixed time window, giving a throughput figure for benchmarking.

The five data types are text content, CDATA sections, comments, start-tag names, and attribute values. The Eiffel counterpart is CRC_32_GENERATOR. When both programs produce identical CRC-32 values for a given file and data type, the parsers are in agreement for that combination.

Earlier in the project I used a simpler program — xml_tag_counter.c — which generates a table of tag-name occurrence frequencies. Its Eiffel counterpart is TAG_COUNTER. This confirmed basic parse-tree correctness but cannot detect errors in text content or attribute values.

Benchmark sessions are driven by two scripts — benchmark_Xpact_crc_32.sh and benchmark_Xpact_tag_count.sh — which run both parsers against each test file and append results to log files, allowing performance regressions to be tracked over time.

Where things stand

Overall I am happy with the result. I lost some performance during the correctness phase of the project, but not too much. I am hopefully approaching the Pareto proportion, though more testing is needed. The next goal is a mass-correctness testing program that exercises thousands of different files — as it turns out, a typical Linux install already has thousands of suitable test subjects on disk.

One robustness milestone worth noting: Xpact-core passes all five of the libexpat "hack attack" test files — aaaaaa_attr.xml, aaaaaa_cdata.xml, aaaaaa_comment.xml, aaaaaa_tag.xml, and aaaaaa_text.xml. Each is a ~10 MB document consisting of a single pathologically large token: a giant attribute value, CDATA block, comment, tag name, or text run with no newlines. They are designed to exercise exactly the buffer-shift and partial-token handling paths that are most likely to fail silently under adversarial input.