Posts
DWARF in reproducible builds
Deterministic builds with clang and lld describes several degrees of build determinism. Here is the description of local determinism:
…Like incremental basic determinism, but builds are also independent of the name of the build directory. Builds of the same source code on the same machine produce exactly the same output every time, independent of the location of the source checkout directory or the build directory.
RISC-V linker relaxation in lld
On 2022-07-07, I added a RISC-V linker relaxation framework in ld.lld and implemented R_RISCV_ALIGN/R_RISCV_CALL/R_RISCV_CALL_PLT relaxation.
The changes will be included in the next llvm-project release 15.0.0.
This post describes the implementation.
See The dark side of RISC-V linker relaxation for more information about RISC-V linker relaxation.
…Everything I know about glibc
Updated in 2025-06.
- Repository: https://sourceware.org/git/gitweb.cgi?p=glibc.git
- Wiki: https://sourceware.org/glibc/wiki/
- Bugzilla: https://sourceware.org/bugzilla/
- Mailing lists:
{libc-announce,libc-alpha,libc-locale,libc-stable,libc-help}@sourceware.org - Patchwork: https://patchwork.sourceware.org/project/glibc/list/ Patch Review Workflow
- Contribution Checklist: https://sourceware.org/glibc/wiki/Contribution%20checklist
- Build bots: https://builder.sourceware.org/buildbot/#/builders, search "glibc"
- public inbox: https://inbox.sourceware.org/libc-alpha/
glibc is an implementation of the user-space side of standard C/POSIX functions with Linux extensions.
…C standard library headers in C++
Updated in 2024-01.
In ISO C++ standards, [support.c.headers.general] says:
Source files that are not intended to also be valid ISO C should not use any of the C headers.
Then, [depr.c.headers] describes how a C header name.h is transformed to the corresponding C++ cname header. There is a helpful example:
[ Example: The header
assuredly provides its declarations and definitions within the namespace std. It may also provide these names within the global namespace. The header <stdlib.h> assuredly provides the same declarations and definitions within the global namespace, much as in the C Standard. It may also provide these names within the namespace std. — end example ]
"may also" in the wording allows implementations to provide mix-and-match, e.g. #include <stdlib.h> may provide std::exit and #include <cstdlib> may provide ::exit.
libstdc++ chooses to enable global namespace declarations with C++ cname header.
For example, #include <cstdlib> also includes the corresponding C header stdlib.h and we get declarations in both the global namespace and the namespace std.
. /usr/include/c++/12/cstdlib
.. /usr/include/stdlib.h
The preprocessed output looks like:
extern void exit (int __status) noexcept (true) __attribute__ ((__noreturn__));
extern "C++"
{
namespace std __attribute__ ((__visibility__ ("default")))
{
using ::exit;
}
}
The compiler knows that the declarations in the namespace std are identical to the ones in the global namespace.
The compiler recognizes some library functions and can optimize them. By using the compiler can optimize some C library functions in the namespace std (e.g. many std::mem* and std::str* functions).
For some C standard library headers, libstdc++ provides wrappers (libstdc++-v3/include/c_compatibility/) which take precedence over the glibc headers.
The configuration of libstdc++ uses --enable-cheaders=c_global by default.
if GLIBCXX_C_HEADERS_C_GLOBAL in libstdc++-v3/include/Makefile.am describes that the 6 wrappers (complex.h, fenv.h, tgmath.h, math.h, stdatomic.h, stdlib.h) shadow the C library headers of the same name.
For example, #include <stdlib.h> includes the wrapper stdlib.h which includes cstdlib, therefore bringing exit into the namespace std.
. /usr/include/c++/12/stdlib.h
.. /usr/include/c++/12/cstdlib
... /usr/include/stdlib.h…
Analysis and introspection options in linkers
Updated in 2026-01.
Reproduce tarball
LLD offers a convenient feature to bundle all input files into a tarball, making it easier to experiment with different linker options. Use either of these commands:
clang -fuse-ld=lld -Wl,--reproduce=/tmp/rep.tar a.o b.o
LLD_REPRODUCE=/tmp/rep.tar clang -fuse-ld=lld a.o b.o
Then unpack the tarball, navigate to the directory, and invoke LLD with the response file:
cd /tmp; tar xf rep.tar; cd rep
ld.lld @response.txt # append options like -y foo
The response file includes the --chroot option, which GNU ld does not support.
In most cases, you can simply remove this option to examine GNU ld's behavior.
ld.bfd @response.txtSymbol related
--trace-symbol=<sym>
Alias: -y sym
lld 14 ELF changes
llvm-project 14 will be released soon. I added some lld/ELF notes to https://github.com/llvm/llvm-project/blob/release/14.x/lld/docs/ReleaseNotes.rst. Here I will elaborate on some changes.
…All about COMMON symbols
Programming language behavior
FORTRAN 77 COMMON blocks compiled to COMMON symbols. You could declare a COMMON block in more than one file, with each specifying the number, type, and size of the variable. The linker allocated enough space to satisfy the largest size.
…Compressed debug sections
Updated in 2024-05.
Binary sizes are important. Filesystem compression is ergonomic but typically does not leverage application information well. Compressing allocable sections (text, data) increases program startup time and introduces memory overhead. In addition, filesystem compression is not sufficiently portable.
Debug sections are large and contribute to a significant portion of the binary size. Therefore, it is appealing to compress debug sections.
Here is a -DCMAKE_BUILD_TYPE=Debug build directory of llvm-project where I just ran ninja clang (on 2022-10-21).
Here are the total sizes of .o files, text sections, and debug sections.
It is typical that the debug information is often much larger than text sections.
% stat -c %s **/*.o | awk '{s+=$1} END{print s}'
1464767464
% readelf -WS **/*.o | awk 'BEGIN{FPAT="\\[.*?\\]|\\S+"} $2~/\.text/{d += strtonum("0x"$6)} END{print d}'
210026370
% readelf -WS **/*.o | awk 'BEGIN{FPAT="\\[.*?\\]|\\S+"} $2~/\.debug_/{d += strtonum("0x"$6)} END{print d}'
631069751
% readelf -WS **/*.o | awk 'BEGIN{FPAT="\\[.*?\\]|\\S+"} $2~/\.rela\.debug_/{d += strtonum("0x"$6)} END{print d}'
78448968
Some assemblers and linkers offer a feature to compress debug sections.
llvm-objcopy supports --compress-debug-sections=zlib to compress debug sections. We can use the option to check what if we compress debug sections for the assembler.
% for i in **/*.o; do /tmp/Rel/bin/llvm-objcopy --compress-debug-sections=zlib $i /tmp/c/o && readelf -WS /tmp/c/o | awk 'BEGIN{FPAT="\\[.*?\\]|\\S+"} $2~/\.debug_/{d += strtonum("0x"$6)} END{print d}'; done | awk '{s+=$1} END{print s}'
161691798
For debug sections, we have a compression ratio of 3.90! The total .o size is 995438992 bytes, 68% of the original.
Then let's check zstd.
% for i in **/*.o; do /tmp/Rel/bin/llvm-objcopy --compress-debug-sections=zstd $i /tmp/c/o && readelf -WS /tmp/c/o | awk 'BEGIN{FPAT="\\[.*?\\]|\\S+"} $2~/\.debug_/{d += strtonum("0x"$6)} END{print d}'; done | awk '{s+=$1} END{print s}'
159341878
To check whether an object file has compressed debug sections, we can use readelf.
% readelf -S a.o
...
Section Headers:
[Nr] Name Type Address Off Size ES Flg Lk Inf Al
...
[ 5] .debug_abbrev PROGBITS 0000000000000000 000080 000087 00 C 0 0 8
In the readelf -S output, the Flg column describes sh_flags where C indicates the SHF_COMPRESSED flag.
% readelf -t a.o
...
Section Headers:
[Nr] Name
Type Address Off Size ES Lk Inf Al
Flags
...
[ 5] .debug_abbrev
PROGBITS 0000000000000000 000080 000087 00 0 0 8
[0000000000000800]: COMPRESSED
ZLIB, 0000000000000093, 1History
In 2007-11, Craig Silverstein added --compress-debug-sections=zlib to gold.
When the option was specified, gold compressed the content of a .debug* section with zlib and changed the section name to .debug*.zlib.$uncompressed_size.