These notes are for the MacOS X Xcode 2.4 Release of the compiler tools. They contain information about the following topics:
The tools now support development of 64-bit Mach-O binaries for Intel with the -arch x86_64 flag.
The design of the relocation entries for the x86_64 architecture does not make it possible to strip local symbols from relocable object files. As such the feature of running strip(1) with the -x option to limit the symbols to just the exported symbols is not available on the x86_64 architecture. And the strip(1) -x option for an x86_64 object file does nothing.
The one feature of this release is support for the Intel Vanderpool instructions in the assembler and disassembler.
The major new feature for this release is the support of the use of DWARF for debug information.
The other new feature for this release is the support of the .quad assembler directive for the 32-bit assemblers. With this feature the assembler now treates all expressions as 64-bit values. This change can cause code code to no longer assemble. For example the following construct:
addi r11, r11, (-33120 << 16) >> 16
will now generate an error of the form:
x.s:1:Parameter error: expression out of range (parameter 3)
The correct way this should be coded is:
addi r11, r11, lo16(-33120)
There are no new features and only bug fixes. But there are some changes to improve the performance of the static link editor.
The static link editor now supports the new option -Sp to strip, edit and add debugging symbols so the debugger can used most of the debugging symbols from the object files.
The ar(1) tool has the new -S option to suppress building the table of contents.
The tools now have support for the handful of x86 sse3 instructions.
The tools now support development of 64-bit Mach-O binaries for the PowerPC with the -arch ppc64 flag. The one tool that has not been seamlessly ported for this release is otool(1). And there is an otool64(1) for 64-bit Mach-O binaries.
The static link editor now can do dead code stripping. The new ld(1) option to do this is -dead_strip. There is also an additional option -no_dead_strip_inits_and_terms that can be used when -dead_strip is specified to cause all constructors and destructors to never be dead code stripped. The load map printed with the ld(1) option -M notes what was dead stripped from the input files.
There is one known bug with the tools that do dead code stripping. It is possible for some debugging information for types made to be stripped. This can lead to the debugger not being able to print the values of some variables and get unknow type errors.
For testing, the environment variable LD_DEAD_STRIP can be set. which causes -dead_strip to specified for all ld(1) commands that don't specify the -r option. The environment variable LD_NO_DEAD_STRIP_INITS_AND_TERMS likewise causes -no_dead_strip_inits_and_terms to specified for all ld(1) commands that don't specify the -r option. And the environment variable LD_DEAD_STRIP_DYLIB causes -dead_strip to specified for all ld(1) commands that have the -dylib option specified.
The static link editor determines what unreachable code and data can be stripped based on the references from the initial live symbols and blocks. The initial live symbols include the symbols to be exported in the linked output. The set of exported symbols are specified with the -exported_symbols_list option or a list of symbols not to be exported is specified with the -unexported_symbols_list option. If no exported symbols are specified and the output is not an executable (shared library, bundle, etc) it is assumed all global symbols are to be exported. If the output is a shared library and a shared library initialization symbol is specified with -init symbol_name option then that symbol_name is an initial live symbol. If the output is an executable then the block that contains the entry point or the symbol specified with the -e symbol_name is an initial live symbol. Other symbols can be marked by the programmer as an initial live symbols with the GNU compiler's __attribute__((used)). For Objective-C code, the compiler will also mark the blocks of Objective-C runtime data it produces so they are part of the initial live blocks. For symbols marked referenced dynamically (via the REFERENCED_DYNAMICALLY bit in <mach-o/nlist.h>), such as the symbol _environ from /usr/lib/crt1.o, they are also part of the initial live symbols.
Before turning on the -dead_strip option your project will first have to be "ported" to work with dead code stripping. This will include changing from -gused (the default for -g) to -gfull and re-compiling all of the objects files being linked into your program with the new compiler from the Mac OS X June 2004 release. Also if your building an executable that loads plugins, which uses symbols from the executable, you will have to make sure the symbols the plugins use are not stripped (by using __attribute__((used)) or the -exported_symbols_list option). If you are using an export list and building a shared library, or an executable that will be used with ld(1)'s -bundle_loader flag, you need to include the symbols for exception frame information in the export list for your exported C++ symbols. These symbols end with .eh and can be seen with the nm(1) tool.
Various auto configure test programs that expect to get undefined symbol errors when they have a possible undefined symbol referenced from dead code can't use the -dead_strip option. Because the dead code and the unused undefined reference will be stripped allowing the program to successfully linked. We have geneally found that auto configure programs on UNIX based system can't be configured with the -dead_strip option on.
Recompiling with a new compiler is required to get effective dead code stripping. As it marks the object files it creates as OK to divide up the section contents into individual blocks by the symbols that are in the object files. Objects not marked are assumed to be generated from assembly code, older compilers, or from compilers that are not know to be safe to divide up their section contents by their symbols. For non-marked objects if any symbol is used from a section the entire section is kept and not dead stripped.
The new compiler unconditionally marks the object files it creates as OK to divide up. There may be cases, especially with asm() statements, that it should not mark the object it creates as OK to divide up. In these cases turning on the -dead_strip option may fail to produce a working program because it can incorrectly strip a live block. The reason for this is that the heuristic of dividing up the section contents by their symbols is incorrect if any individual block has more than one symbol. This makes the heuristic fragile.
The reason that the current dead code stripping inplementation is fragile is because the static linker is guessing at the structure of the program. Namely it is guess at which bytes of a section make up a individual block of code or data. It makes its guess based on the symbols that are in the object files being linked. In the same way it guesses for the blocks it uses for the -sectorder option. With the difference being that all sections are broken up into individual blocks with the goal to eliminate the unreachable (dead) blocks in the linked output. The reason this is fragile is that all the symbols in an object file may not be the start of an individual block.
As a workaround to the problem of incorrectly strip a live block you can produce a -sectorder file for objects that have section contents that must be linked whole. You would create a line in the sectorder file specifying the object and the psuedo symbol .section_all. For example if .text section of crt1.o and the archive member of darwin-tramp.o in libgcc.a must be linked whole, you would add the following:
to your order_file for the __TEXT __text section. Then use the ld(1) option -sectorder __TEXT __text order_file when linking.
There is a limitation in the current design for individual blocks that have no symbols visible in the object file. Symbols that are temporary labels that start with a 'L', or symbols that have been stripped. An individual block that starts with these symbols will appear to be part of the previous block that has a symbol in the object file. This can result in a linked image that does not have all of its dead code stripped. When what would have been a dead individual block becomes part of the previous block which is live. Causing the dead individual block and all of its references to be live.
By default when there are multiply defined symbols in the linked objects this is treated as an error. Even when the -dead_strip option is specified. Unused multiply defined symbols can be stripped if the strongly discouraged -m option is also specified. The duplicate symbols other than the first symbol may still end up being used in the resulting output file through local references however.
Symbols from the linked objects in the output file that are only referenced via a shared library will not be dead stripped. Currently the static linker marks them with the REFERENCED_DYNAMICALLY bit in <mach-o/nlist.h> which causes them to be live.
There is no support in the current design to dead strip branch islands created with the compiler option -mlongcall.
The marking of object files which are OK to divide their section contents into individual blocks is done with the new assembler directive .subsections_via_symbols. This directive sets a previously unused bit in the object file's mach_header structure in the flags field. This bit is defined as the following new constant in the header file <mach-o/loader.h>:
#define MH_SUBSECTIONS_VIA_SYMBOLS 0x2000 /* safe to divide up the sections into
sub-sections via symbols for dead
code stripping */
This can be see with otool(1)'s -hv options and shows up as SUBSECTIONS_VIA_SYMBOLS in the following output:
% otool.NEW -hv a.out
magic cputype cpusubtype filetype ncmds sizeofcmds flags
MH_MAGIC PPC ALL OBJECT 3 228 SUBSECTIONS_VIA_SYMBOLS
The new assembler directive .subsections_via_symbols can be added to hand written assembly files provided the symbols are at the start of individual blocks. Since this applies to the entire file, when multiple object files are combined by the static link ediror, ld(1), to produce a relocable object, using the -r flag, if any of the objects do not have this flag set the output file will not have this flag set.
For example this assembly code contains only two individual blocks but four symbols:
addi r3, r3, 1
addi r3, r3, 1
addi r3, r3, 1
And if the new assembler directive .subsections_via_symbols were added to the above code the static link editor would make 4 blocks about of the above code. Then if the program uses _add_three but not _add_two or _add_one it will get the wrong answer if the -dead_strip option is used. As the blocks for _add_two or _add_one will be dead stripped. And if _some_other_routine is not live then when _add_three is called it will fall through to what ever bit of code follows it and likely crash.
The new assembler directive .no_dead_strip symbol_name can be used to specify that a symbol is not to be dead stripped. For example:
This can be seen (in object files only) with nm(1)'s -m option and shows up as [no dead strip] in the following output:
% nm -m x.o
00000000 (__TEXT,__cstring) non-external [no dead strip] _my_version_string
The .no_dead_strip directive is generated by the new compiler when the __attribute__((used)) is specified on a symbol.
In object files, this bit is N_NO_DEAD_STRIP as defined in <mach-o/nlist.h>. And is set in relocatable .o files (MH_OBJECT filetype) only in the n_desc field of an nlist struct.
The new section attribute no_dead_strip can be specified on a section to cause its entire contents to not to be dead stripped. The new compiler uses this for all Objective-C sections it creates. For example:
.section __OBJC, __image_info, regular, no_dead_strip
In object files, this bit is S_ATTR_NO_DEAD_STRIP as defined in <mach-o/loader.h>. And is set in the flags field of a section struct.
The new section attribute live_support can be specified on a section to cause its blocks to not be dead stripped if they reference something that is live. The new compiler uses this for C++ exception frame information. For example:
.section __TEXT, __eh_frame, coalesced, no_toc+strip_static_syms+live_support
In object files, this bit is S_ATTR_LIVE_SUPPORT as defined in <mach-o/loader.h>. And is set in the flags field of a section struct.
The assembler now takes a new .machine directive as an alternate to using the command line -arch arch_name option. It is specified as for example as:
Where ppc970 can be any arch_name that would appear in the -arch arch_name option as listed on the arch(3) man page for the assembler's architecture family.
The dynamic linker has been changed that improve launch times of applications.
The redo_prebinding(1) command has a new option, -u, that does a unprebind operation. The unprebind operation produces a canonical form of a Mach-O file that can be used for binary diffing and patching. Bundles and non-prebound executables and dylibs can also be canonicalized with the unprebind operation.
Added the -executable_path path_name option to ld(1) where path_name is is used to replace @executable_path for dependent libraries.
The compiler tools now support the PowerPC 970 processor. The architecture specific flag -arch ppc970 is used to specify this specific processor. The assembler will only assemble 64-bit instructions and other PowerPC AS User Instruction Set Architecture Version 2.00 instructions supported by the PowerPC 970 processor when -arch ppc970 or -force_cpusubtype_ALL is specified.
To specify branch predictions which use the AT bit encodings for The branch is very likely to be taken and The branch is very likely not to be taken the two character suffixes ++ and -- are used. For example:
The single character suffixes + and - continue to encode branch predictions which use the Y-bit encoding by default. The encoding can be changed for the single character suffixes to use the AT bit encodings with the assembler flag -static_branch_prediction_AT_bits (see the as(1) man page for more details).
The compiler tools now support stub libraries created from dynamic libraries which are used in the SDKs. Stub libraries are created via strip(1) and the new -c option. And can be linked against in place of the actual dynamic library.
By default when the -dynamic flag is in effect, the -lx and -weak-lx options first search for a file of the form libx.dylib in each directory in the library search path, then a file of the form libx.a is searched for in the library search paths. The new option -search_paths_first changes it so that in each path libx.dylib is searched for then libx.a before the next path in the library search path is searched.
Added the -weak_framework, -weak_library and -weak-l options to ld(1) to force the dynamic library and the symbols referenced from it to be marked as weak imports. See the ld(1) man page for more details.
Added the -undefined define_a_way option to ld(1) as a work a round to not having dead-code stripping that also strips out references to undefined symbols from the dead code. Which leads to link time failures due to undefined symbols. With this option ld(1) defines the remaining undefined symbols as private definitions and allows the link to succeed. The program then runs as long as it does not use any of the undefined symbols.
The static link editor, ld(1), now has two new options, -exported_symbols_list filename and -unexported_symbols_list filename to limit the global symbols in the linked output file. This was previously done by with an nmedit(1). By using the new options to ld(1) the use of nmedit(1) can be eliminated resulting in faster build times.
The static link editor, ld(1), now has a new option, -single_module, to build a dynamic library containing only one module. This was previously done by first creating a master.o file with an ld(1) -r step and then using the master.o to created the dynamic library. By using the new -single_module option to ld(1) this first step can be eliminated resulting in faster build times.
The default in the static link editor remains the same and dynamic libraries are built with multiple modules. The new flag -multi_module has also been added to allow this to be explicitly specified.
The static link editor's -s option can now be used to strip an executable that use the dynamic link editor. This will produce the same result as running strip(1) with no options on the executable. By using the -s option when building an executable the strip(1) step can be eliminated resulting in faster build times.
The compiler tools for the MacOS X 10.2 Release must be used with prebound images (executables, and shared libraries) from the MacOS X 10.2 User Release. The compiler tools in MacOS 10.1 will not work with prebound images from with the MacOS X 10.2 User Release. If the 10.1 compiler tools are used on prebound images from the MacOS X 10.2 User Release the compiler tools will generate error messages indicating that the image is a malformed file.
The dynamic linker now supports weak symbol references and weak dymamic libraries. When creating a binary with the static link editor if all the symbols referenced from a given dependent dynamic library are weak references then the library is marked weak. When the binary is used at execution time and a weak library is missing the dynamic linker will not cause an error. For all weak symbols that are missing execution time the dynamic linker uses zero as their address. This allows a weak symbol's address to be tested for zero at runtime allowing the code to avoid using the weak symbol when it is missing. Binaries that use weak references require a dynamic linker from Mac OS X 10.2 or later.
To indicate a symbol is to be a weak reference the __attribute((weak_import)) is used on the prototype of the symbol. When a binary is created by the static link editor normally the all the undefined symbol references of the object files being linked should be consistent for each undefined symbol. That is all undefined symbols should either be weak or non-weak references. If they are not by default this is treated as an error and can be changed with the ld(1) -weak_reference_mismatches treatment flag (see the ld(1) man page for more details).
Weak referenced symbols and weak libraries are only created in the output by the static link editor, ld(1), when the MACOSX_DEPLOYMENT_TARGET environment variable is set to 10.2. If not a warning is generated when a weak reference would be in the output and it is not marked weak. Note the default for the MACOSX_DEPLOYMENT_TARGET environment variable 10.1 so weak referenced symbols and weak libraries are not created by default. See the ld(1) man page for more information on the MACOSX_DEPLOYMENT_TARGET environment variable.
The redo_prebinding(1) command now can slide dymamic libraries to new prefered addresses (see the man page for more details).
You must use the 10.1 compiler tools with images (executables, plugins and shared libraries) created with the 10.1 tools. The compiler tools in MacOS 10.0 will not work with images created with the 10.1 compiler tools. If you attempt to use the 10.0 compiler tools on images created with the 10.1 compiler tools, error messages may result indicating that the image is a malformed file.
By default the compiler tools build images using the new two-level namespace binding semantics, which has important consequences for compatibility with Mac OS X 10.0 (see below for more information).
The following new features have been added to the Compiler Tools for the Mac OS X 10.1 system release.
The following new features have been added to the Compiler Tools since the Mac OS X Developer Release 4.
The following new features have been added to the Compiler Tools since the Mac OS X Developer Release 2.
The following new features have been added to the Compiler Tools since the Mac OS X Developer Preview Release.
The NSLINKMODULE_OPTION_RETURN_ON_ERROR option is an alternative method to the existing dyld error handling which fits better with a plugin model. With the NSLINKMODULE_OPTION_RETURN_ON_ERROR option, the model for handling errors is to simply return without any changes to the program. To support this model of error handling a new API has been added to allow the programmer to get the error information that the dyld error handlers would normally have gotten. The API is similar to the dyld linkEdit error handler except that all the parameters are passed as pointers to be filled in.
The last two parameters return pointers to static buffers allocated in the dynamic linker which get reused on subsequent calls to NSLinkEditError(). The NSLinkEditErrors enum has been extended to include NSLinkEditUndefinedError and NSLinkEditMultiplyDefinedError.
The following new features have been added to the Compiler Tools since the Mac OS X Server Release.
with the options as follows:
The first two are the same as bindNow with a value of FALSE and TRUE. The private options are used to load a private module. The API for getting to the symbols of a NSModule that has been privately linked is:
Then to get the address of the returned NSSymbol, the existing NSAddressOfSymbol() API can be used.
The NSUnLinkModule() API is now implemented with enough functionality to make Apache work (reference number 2262020). It currently has the following limitations (to be fixed in future releases):
The interface to NSUnLinkModule has changed in a compatible way from:
where the options are:
The first two are the same as keepMemoryMapped with a value of FALSE and TRUE. The reset lazy references option allows unloading modules with only call sites to undefined functions (direct calls, not calls through pointers) to not cause an undefined symbol error. Then if a subsequent module is loaded that defines symbols that were previously undefined, the call sites will use the new definitions. This is currently only implemented for PowerPC.
Support for module termination functions has been added for plugins (only). Currently the compiler pragma CALL_ON_UNLOAD (as well as CALL_ON_LOAD) is not yet implemented to use this feature as intended. A work around can be done in place of having the pragma:
The following new features have been added to the Compiler Tools since the Rhapsody Developer Release 2.
The following new features have been added to the compiler tools since OPENSTEP 4.2 (NeXT).
These bugs are known to exist in the compiler tools:
4.4BSD ar extended format #1 not compatible with compiler tools.
The 4.4 ar command can create an archive with the base name of an object file that is longer than 16 characters. With the -L option, it produces a format that makes the object file in the archive invisible to various tools, including the static link editor. This can lead to undefined symbols when this archive is linked against. Other tools like nm and ranlib also don't see the long-name object files in the archive. To avoid this problem, ar makes the -T option, which truncates names, a default option. The compiler tools will understand the extended format in future releases.
Do not use the -L option with ar when creating archive libraries. Use the -T option (the default for the Premier release) to tuncate file names or use libtool -static to create archive libraries.
The Mac OS X assembler is different from ppcasm.
The major difference is that the Mac OS X assembler is not TOC-based and uses two instructions to load a global or static item. The directives and the syntax of labels and directives of the two assemblers are very different. Also, the Mac OS X assembler is stricter in the parameter types and ranges for instructions. For more on this last topic, see "Instruction Parameter Differences," below.
The difference between the Mac OS X assembler and the TOC-based model, plus the differences in directives and syntax, may necessitate significant rewriting of assembly code for the Developer Release. The strict parameter requirements might require rewriting of assembly code for the Developer Release but the resulting code should work with ppcasm.
BSD 4.4 ar format is not compatible with compiler tools
The BSD 4.4 ar command, which creates an archive with object file names longer than 16 characters, produces a format that makes the object file invisible to various tools, including the static link editor. This can lead to undefined symbols when a program links against this archive. Other tools like nm and ranlib also don't see the object files with longer names in the archive.
Use the -T option with ar to tuncate file names or use libtool -static to create archive libraries.
The following bug has been fixed:
Profiling does not work
Bugs were reported when developers tried to compile, run and produce the profiling information for a program. Among these bigs were kernel panics, gprof(1) not understanding the gmon.out format produced, add_profil(2) system call not working, and other problems.
Register names can't be designated with just a number. You must
refer to them with their register name. This restriction includes
general registers (
rN), floating point registers,
fN), condition registers
crN), and segment registers
srN). However, you can refer to special
registers by their register number or their special register names.
The special register names are in lowercase only (for example,
For instance, for the ppcasm assember you could code a move from segment register instruction as:
mfsr r24,9 ; ppcasm assembler
But, for the Mac OS X assembler, this same move would be coded as:
mfsr r24,sr9 ; Mac OS X assembler
For instructions that take the value 0 or a register, shown in the processor manual as "(rA|0)", r0 can't be used and 0 must be coded. The Mac OS X assembler generates an error messages in these cases.
Where a numeric value is expected as a parameter, a register name can't be use. For example, the ppcasm assembler allows the following:
lwz r1,r2(r3) ; ppcasm assembler
For Mac OS X, this must be coded as:
lwz r1,2(r3) ; Mac OS X assembler
The Mac OS X assembler generates a warning if branch prediction is coded with an unconditional branch.
The Mac OS X assembler checks all fields for range errors and generates error messages if an expression is out of range. The ppcasm assembler simply uses the low N bits of the expression (where N is the field width) if the value is greater than zero. For example the simplified mnemonic:
is equivalent to
The following code:
inslwi r17,r18,19,20 ; equivalent to rlwimi r17,r18,32-20,20,(20+19)-1
rlwimi r17,r18,12,20,6 ; where the low 5 bits (20+19)-1 is 6
with ppcasm. This generates an out-of-range error with the Mac OS X assembler.
For fields less than zero, the ppcasm assembler uses the value of zero. For example, the simplified mnemonic:
is equivalent to
Thus the following code:
clrlslwi r5,r6,7,8 ; equivalent to rlwinm r5,r6,8,7-8,31-7
rlwinm r5,r6,7,0,24 ; where 7-8 gets turned into 0
with ppcasm. This generates an out-of-range error with the Mac OS X assembler.
All integer expressions in the Mac OS X assembler are signed 32-bit values. Parameters that are 16-bit signed or unsigned immediate values must agree in their upper 16 bits or the assembler generates an out-of-range error message.
addi r1,r2,0xffff ; out of range for a 16-bit signed immediate
generates the message "Parameter error: expression out of range (parameter 3)".
addi instruction takes a signed immediate value
so it will sign extend its parameter to 32 bits before performing the
operation. If the value 0xffffffff is intended, it would be coded
If this is half of a two-instruction 32-bit add it should be coded as:
addis r1,0,ha16(expression) addi r1,r2,lo16(expression)
Many of the simplified mnemonics are implemented as Mac OS X assembler macros (as noted in the listing of PowerPC assembler instructions in the assember manual). Like all macros, the macro is expanded and assembled. This expansion can result in errors that can seem confusing when you look at the coded macro. For example, the simplified mnemonic:
is equivalent to
Thus the following code:
generates the error message "Parameter error: expression out of range (parameter 4)," which refers to "n-1" or "0-1", or parameter 4 of the expanded macro.
tlbiex, which has been removed from
the PowerPC architecture, is not supported by the Mac OS X assembler.
This instruction is assembled by ppcasm.
Copyright © 2006 Apple Computer, Inc.