To make incremental builds fast on macOS, the static linker (ld) ignores the debug information. It can easily be a magnitude bigger than the rest of the program and slow down link time. Instead the linker emits a debug map which contains the location of all the object files it relocated so that debug info consumers (such as the debugger) know where to find the DWARF debug info.
This approach works great during development and greatly speeds up the build-debug cycle. Keeping the object files around is not a great way to archive the debug info however. This is where dsymutil comes it. It links together the debug information from all the object files and generates a dSYM bundle.
In addition to merging debug info and relocating the addresses, dsymutil also optimizes the resulting DWARF by removing redundant information. This is especially useful for C++ where different compile units will often contain references to the same type. Think for example about types from the Standard Template Library being included in many translation units.
The new --staticstics flag quantifies the impact of dsymutil on the size of the .debug_info section. It prints a table after linking with the object file name, the size of the section in the object file and the size contributed to the linked dSYM. The table is sorted by the output size, listing the object files with the largest contribution first.
Let’s take a look at the output for llvm-lipo. I’ve omitted a significant portion of the output for brevity.
.debug_info section size (in bytes)
-------------------------------------------------------------------------------
Filename Object dSYM Change
-------------------------------------------------------------------------------
libLLVMBitReader.a(BitcodeReader.cpp.o) 1737930b 1430628b -19.40%
libLLVMCore.a(Metadata.cpp.o) 1509986b 853656b -55.54%
libLLVMCore.a(AsmWriter.cpp.o) 1263156b 792942b -45.74%
libLLVMCore.a(Verifier.cpp.o) 1293937b 732571b -55.40%
libLLVMMCParser.a(AsmParser.cpp.o) 955659b 719688b -28.17%
...
libLLVMSupport.a(Watchdog.cpp.o) 1830b 1629b -11.62%
libLLVMSupport.a(regfree.c.o) 747b 810b 8.09%
libLLVMSupport.a(regerror.c.o) 633b 682b 7.45%
libLLVMSupport.a(regstrlcpy.c.o) 206b 222b 7.48%
libLLVMSupport.a(Valgrind.cpp.o) 232b 209b -10.43%
libLLVMSupport.a(ABIBreak.cpp.o) 95b 70b -30.30%
-------------------------------------------------------------------------------
Total 57892179b 32421404b -56.41%
-------------------------------------------------------------------------------
When looking at these numbers, it’s important to keep a few things in mind. The debug info section is essentially a tree of debug info entries (DIEs). Removing redundant DIEs (uniquing) in dsymutil means merging these trees.
Consider the example below. The left tree has 4 nodes: a through d, and the right tree has 3 nodes, b, d and e.
+---+ +---+
| a | | b |
+---+ +---+
| |
| |
v v
+---+ +---+
| b | | d |
+---+ +---+
/ \ |
/ \ |
v v v
+---+ +---+ +---+
| c | | d | | e |
+---+ +---+ +---+
The merged tree has 5 nodes, which is an increase from the original 4 in the left tree, while the right tree has been completely eliminated.
+---+
| a |
+---+
|
|
v
+---+
| b |
+---+
/ \
/ \
v v
+---+ +---+
| c | | d |
+---+ +---+
|
|
v
+---+
| e |
+---+
This explains why the size sometimes increases rather than decreases. More commonly however, this effect will manifest itself as a smaller decrease for some object files than others. The takeaway is that this is not necessarily a bad thing. The algorithm used by dsymutil and the order in which the DIEs are traversed determine to which compile unit a uniqued DIE gets attributed.
At the bottom of the output we see the total decrease. For llvm-lipo, dsymutil was able to get rid of a little more than half the debug info.