We recently published a paper which is devoted entirely to exploring several aspects of x86/x64 disassembly. Among other things, we measured the prevalence of complex corner cases generated by modern compilers, and the precision with which disassemblers handle these cases. We released our complete data set, in part because there are too many results to fit in the paper, and also to allow others to compare their own results to ours.
Since we’ve received several questions asking for details on how to implement such a comparison, the below provides an example. Assuming that you’ve already downloaded our data set and generated the ground truth (as detailed in ~/disasm/README
in the provided VM), getting results for a new disassembler requires two steps.
- Write a script that parses the output of the disassembler you want to evaluate, and puts it into a format useful for further processing.
- Compare the disassembler output to the ground truth, using another script for the specific primitive you want to evaluate.
We give examples of both steps. Though at first it may look like lots of work to fit these scripts to your own evaluation requirements, this should actually be quite straightforward, since you can reuse much of the code verbatim regardless of the specific test setup.
Parsing disassembler output
Since every disassembler is different, we need to make a specifically tailored script that parses the output of the disassembler we want to test, and puts it into a normalized format that we can process further. To keep things simple, the example presented here is based on objdump
, but to create a script for another disassembler you can use the exact same basic idea. Without further ado, here is the bash
script we used in our paper to parse the instructions output by objdump
for our SPEC CPU2006 test suite (the scripts for our other tests are nearly identical).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 |
#!/bin/bash BMARKS_C='400.perlbench 401.bzip2 403.gcc 429.mcf 433.milc 445.gobmk 456.hmmer 458.sjeng 462.libquantum 464.h264ref 470.lbm 482.sphinx3' BMARKS_CPP='444.namd 447.dealII 450.soplex 453.povray 471.omnetpp 473.astar 483.xalancbmk' set -e function disasm { LANG="$1" BMARKS="$2" EVAL="$3" TRUTH="$4" SPECIAL="$5" OPT='O0 O1 O2 O3 Os' mkdir -p ${EVAL}/symbols/${LANG} mkdir -p ${EVAL}/stripped/${LANG} for b in ${BMARKS}; do bnum=` echo ${b} | cut -d. -f1` bname=`echo ${b} | cut -d. -f2` echo "*************** ${bname}${SPECIAL} (${LANG}) ***************" for o in ${OPT}; do echo "******* ${o} *******" if [[ ${TRUTH} == *vs15* ]]; then exe=${b}.${o}${SPECIAL}.exe else exe=${b}.${o}${SPECIAL} fi if [ ! -f ${TRUTH}/bin/symbols/${LANG}/${exe} ]; then echo "Skipping" continue fi objdump --wide -d ${TRUTH}/bin/symbols/${LANG}/${exe} | egrep '^[[:space:]]*[0-9a-f]+:' | cut -d: -f1 | awk '{print "0x"$1}' > ${EVAL}/symbols/${LANG}/${b}.${o}${SPECIAL}.ins.objdump222 objdump --wide -d ${TRUTH}/bin/stripped/${LANG}/${exe} | egrep '^[[:space:]]*[0-9a-f]+:' | cut -d: -f1 | awk '{print "0x"$1}' > ${EVAL}/stripped/${LANG}/${b}.${o}${SPECIAL}.ins.objdump222 done done } echo "=============== GCC (32 bit) ===============" disasm 'C' "${BMARKS_C}" 'ins/gcc510-32' 'truth/gcc510-32' disasm 'C++' "${BMARKS_CPP}" 'ins/gcc510-32' 'truth/gcc510-32' echo "=============== GCC (64 bit) ===============" disasm 'C' "${BMARKS_C}" 'ins/gcc510-64' 'truth/gcc510-64' disasm 'C++' "${BMARKS_CPP}" 'ins/gcc510-64' 'truth/gcc510-64' echo "=============== GCC (64 bit) static/flto ===============" disasm 'C' "${BMARKS_C}" 'ins/gcc510-64' 'truth/gcc510-64' '.static' disasm 'C' "${BMARKS_C}" 'ins/gcc510-64' 'truth/gcc510-64' '.static.flto' echo "=============== LLVM (32 bit) ===============" disasm 'C' "${BMARKS_C}" 'ins/llvm370-32' 'truth/llvm370-32' disasm 'C++' "${BMARKS_CPP}" 'ins/llvm370-32' 'truth/llvm370-32' echo "=============== LLVM (64 bit) ===============" disasm 'C' "${BMARKS_C}" 'ins/llvm370-64' 'truth/llvm370-64' disasm 'C++' "${BMARKS_CPP}" 'ins/llvm370-64' 'truth/llvm370-64' echo "=============== Visual Studio (32 bit) ===============" disasm 'C' "${BMARKS_C}" 'ins/vs15-32' 'truth/vs15-32' disasm 'C++' "${BMARKS_CPP}" 'ins/vs15-32' 'truth/vs15-32' echo "=============== Visual Studio (64 bit) ===============" disasm 'C' "${BMARKS_C}" 'ins/vs15-64' 'truth/vs15-64' disasm 'C++' "${BMARKS_CPP}" 'ins/vs15-64' 'truth/vs15-64' |
Lines 3-4 are simply lists of all the SPEC CPU2006 C and C++ test cases, which we later iterate over to disassemble each test. On lines 40-66, we call the main disassembly function (described next) with various parameters, for each of the compiler configurations we test.
The important bit is the disasm
function declared on line 8. It starts by reading its parameters into named variables and making the directories where we will output our results. Then, on line 19, we begin a loop over all test cases for the given configuration.
For each test case, we loop over all the optimization levels (line 23), and determine the name of the binary for the current test case/optimization level, skipping an iteration and yielding a warning if the file does not exist (lines 25-33). Note that we assume a particular format for the directory and binary names. For instance, we assume that all the stripped C++ test binaries as compiled with gcc
5.1/64-bit are located in a directory called truth/gcc510-64/bin/stripped/C++
, and that binaries generated with Visual Studio have the .exe
extension. If you are using the ground truth provided by us, these requirements are all met.
So far, the entire script has been disassembler-agnostic; you can reuse those parts for any disassembler you want to test. Lines 34-35 are the only lines that need to be tailored to the specific disassembler that is being tested. These are the lines where the actual disassembler is run, and its output parsed and dumped to file. Moreover, both these lines are identical except that line 34 disassembles a binary with symbols, while line 35 disassembles a stripped binary. For our example, in both cases we simply run objdump
, grep
for all the disassembled addresses, give each address a 0x
prefix, and write the results to an output file for the specific test case/configuration. We store instruction addresses instead of mnemonics because the addresses are much easier to compare to our ground truth (as discussed below).
As you can see, the script generalizes to other disassemblers in a very straightforward way. Some disassemblers, such as IDA Pro, have a more complicated user interface that we cannot just parse with grep
. In such cases, we require that the disassembler is scriptable, and can be run in an automated way. For instance, for IDA Pro we created a simple IDA Python script that dumps all the primitives we are interested in to file, and then ran the script in the above loop using IDA Pro’s “autonomous mode” (requiring no user interaction). In our objdump
example, we save only instruction output, but for disassemblers which support other primitives, these can be parsed and written to file in an analogous way.
Comparing to the ground truth
So far, we have created a bash
script which uses our chosen disassembler (objdump
) to disassemble all our test cases and save the instruction addresses to file. Now, we want to compare these addresses to the ground truth provided in our data set. For this, we use a Python script (called ins-cmp.py
) that takes as input the ground truth file for a single test case (one of the *.truth.map
files provided in our data set), and a disassembler output file as generated by the disassembler-specific bash
script described above.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 |
#!/usr/bin/env python import sys import os Debug = False addr2type = {} def usage(): print "Usage: %s <truth.map> <disasm.ins>" % (sys.argv[0]) print def pct(n ,m): return float(n)/m*100.0 def insmap_byte(i, insmap): j = 0 k = 0 inslen = len(insmap) while j < i: try: if insmap[k] == '[': while insmap[k] != ']': k = k + 1 except IndexError: return None k = k + 1 j = j + 1 if not k < inslen: return None if insmap[k] == '[': return insmap[k:].split(']')[0] + ']' else: return insmap[k] def certain_code(addr, bounds): global addr2type if addr in addr2type: return addr2type[addr] == 'C' low = addr while not low in bounds: low = low - 1 if low < 0: addr2type[addr] = None return False insmap = bounds[low] b = insmap_byte((addr-low), insmap) if not b: addr2type[addr] = None return False elif 'C' in b: addr2type[addr] = 'C' return True elif 'D' in b: addr2type[addr] = 'D' return False return False def certain_data(addr, bounds): global addr2type if addr in addr2type: return addr2type[addr] == 'D' low = addr while not low in bounds: low = low - 1 if low < 0: addr2type[addr] = None return False insmap = bounds[low] b = insmap_byte((addr-low), insmap) if not b: addr2type[addr] = None return False elif 'C' in b: addr2type[addr] = 'C' return False elif 'D' in b: addr2type[addr] = 'D' return True return False def main(): if len(sys.argv) != 3: usage() return 1 bounds = {} minbound = 0xffffffffffffffff maxbound = 0 with open(sys.argv[1], 'r') as f: for line in f: if line.startswith('@'): addr = int(line.split(':')[0][1:], 0) if addr < minbound: minbound = addr if addr > maxbound: maxbound = addr insmap = line.split(':')[1].strip() if insmap.startswith('D'): # Ignore data sections continue bounds[addr] = insmap # This does nothing except improve performance (by restructuring the dict) bounds.update(dict(bounds)) ins = {} outofbounds = 0 with open(sys.argv[2], 'r') as f: for line in f: addr = int(line, 0) if addr < minbound or addr > maxbound: outofbounds = outofbounds + 1 continue else: ins[addr] = True TP = 0 FP = 0 FP_data = 0 FN = 0 FN_nop = 0 nins = len(ins) nbounds = len(bounds) for addr in ins: if addr in bounds: TP = TP + 1 elif certain_code(addr, bounds): FP = FP + 1 if Debug: print 'FP: 0x%x' % (addr) elif certain_data(addr, bounds): FP = FP + 1 FP_data = FP_data + 1 if Debug: print 'FP: 0x%x' % (addr) for addr in bounds: if not addr in ins: FN = FN + 1 if Debug: print 'FN: 0x%x' % (addr) if 'N' in bounds[addr]: FN_nop = FN_nop + 1 assert (TP + FN) == nbounds assert (FP_data <= FP) assert (FN_nop <= FN) print "|bounds| = %-10u |ins| = %-10u |d| = %-10d (%-5.2f%%) |out| = %-10d (%-5.2f%%) |TP| = %-10u (%-5.2f%%) |FP| = %-10u (%-5.2f%%) |FP_data| = %-10u (%-5.2f%%) |FN| = %-10u (%-5.2f%%) |FN_nop| = %-10u (%-5.2f%%)" \ % (nbounds, nins, nins-nbounds, pct(nins, nbounds), outofbounds, pct(outofbounds, nins), TP, pct(TP, nbounds), FP, pct(FP, nins), FP_data, pct(FP_data, nins), FN, pct(FN, nbounds), FN_nop, pct(FN_nop, nbounds)) return 0 if __name__ == "__main__": sys.exit(main()) |
For instance, here is a result you might get when calling this script from the command line for a particular test case (the files in ins/
are generated by the disassembly script we created above).
1 2 3 |
$ ./ins-cmp.py truth/vs15-64/C++/483.xalancbmk.O3.truth.map ins/vs15-64/stripped/C++/483.xalancbmk.O3.ins.ida67 |bounds| = 707729 |ins| = 691061 |d| = -16668 (97.64%) |out| = 0 (0.00 %) |TP| = 691046 (97.64%) |FP| = 15 (0.00 %) |FP_data| = 12 (0.00 %) |FN| = 16683 (2.36 %) |FN_nop| = 14657 (2.07 %) |
The script compares instruction addresses (as found by the disassembler) to the ground truth. To create scripts for other primitives, please refer to the README
file provided in our data set. It completely describes our ground truth format, which is designed to be easily parseable by both humans and machines. The README
file also describes the output format of our comparison scripts.
Let’s take a look at the main
function, at line 93. It consists of three phases.
- Read the instruction-level ground truth into the
bounds
dictionary (lines 98-113), using instruction addresses as key, and mapping them to a descriptor of the instruction type (as described in the ground truth format section in theREADME
file). - Load all the instruction addresses found by the disassembler into the
ins
dictionary (lines 118-127). - Compare the ground truth (
bounds
) to the disassembled instructions (ins
), counting true positives, false positives and false negatives and then printing out the statistics (lines 129-160).
The certain_code
and certain_data
functions are used to parse a ground truth instruction descriptor, and find out if a particular address is code or data. To this end, both of these functions rely on insmap_byte
, which is just a utility function that returns the type of a particular byte in the descriptor. (Each descriptor describes a single instruction, which may consist of multiple bytes.)
As an example of how to evaluate a primitive other than instructions, suppose that we instead want to measure the correctness of function information. In that case, you would fill the bounds
dictionary in a similar way, but this time loading the function-level ground truth instead of the instruction-level ground truth. This simply means that instead of loading the lines that start with an '@'
symbol (instruction descriptors), you would load the lines that start with 'F '
(an F followed by a space), and then compare the ground truth addresses to those found by the disassembler (in this case you won’t even need the certain_code
/certain_data
functions, but can just compare addresses directly). To get an intuitive feeling of how to parse for each kind of primitive, it is a good idea to open up one of the *.truth.map
files and skim/grep
through it.
Now that we can compare ground truth and disassembler output for one test case at a time, it would be convenient to automate the process of doing this for all test cases. For this, we use one last bash
script, which is similar in structure to the script used for disassembly.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 |
#!/bin/bash EVALDIR=`pwd`'/ins' BMARKS_C='400.perlbench 401.bzip2 403.gcc 429.mcf 433.milc 445.gobmk 456.hmmer 458.sjeng 462.libquantum 464.h264ref 470.lbm 482.sphinx3' BMARKS_CPP='444.namd 447.dealII 450.soplex 453.povray 471.omnetpp 473.astar 483.xalancbmk' set -e function inscmp { LANG="$1" BMARKS="$2" EVAL="$3" TRUTH="$4" COMP="$5" SPECIAL="$6" OPT='O0 O1 O2 O3 Os' mkdir -p ${EVAL}/symbols/${LANG} mkdir -p ${EVAL}/stripped/${LANG} declare -a disasm=("objdump222" "ida67" "dyninst910" "bap099" "jakstab084" "angr4614" "hopper3115" "psi11") for dis in "${disasm[@]}"; do if [[ $COMP != vs15* ]]; then echo -n > ${EVALDIR}/ins.${dis}.${COMP}.${LANG}${SPECIAL}.symbols.stats fi echo -n > ${EVALDIR}/ins.${dis}.${COMP}.${LANG}${SPECIAL}.stripped.stats done for b in ${BMARKS}; do bnum=` echo ${b} | cut -d. -f1` bname=`echo ${b} | cut -d. -f2` echo "*************** ${bname}${SPECIAL} (${LANG}) ***************" for o in ${OPT}; do echo "******* ${o} *******" for dis in "${disasm[@]}"; do echo "${dis}" if [ ! -f ${TRUTH}/${LANG}/${b}.${o}${SPECIAL}.truth.map ]; then echo "Skipping ${dis} (no truth files)" continue fi if [[ $COMP != vs15* ]]; then if [ ! -f ${EVAL}/symbols/${LANG}/${b}.${o}${SPECIAL}.ins.${dis} ]; then echo "Skipping ${dis}/symbols (no eval files)" else printf '%-20s %s ' "${b}" "${o}" >> ${EVALDIR}/ins.${dis}.${COMP}.${LANG}${SPECIAL}.symbols.stats ./ins-cmp.py ${TRUTH}/${LANG}/${b}.${o}${SPECIAL}.truth.map ${EVAL}/symbols/${LANG}/${b}.${o}${SPECIAL}.ins.${dis} >> ${EVALDIR}/ins.${dis}.${COMP}.${LANG}${SPECIAL}.symbols.stats fi fi if [ ! -f ${EVAL}/stripped/${LANG}/${b}.${o}${SPECIAL}.ins.${dis} ]; then echo "Skipping ${dis}/stripped (no eval files)" else printf '%-20s %s ' "${b}" "${o}" >> ${EVALDIR}/ins.${dis}.${COMP}.${LANG}${SPECIAL}.stripped.stats ./ins-cmp.py ${TRUTH}/${LANG}/${b}.${o}${SPECIAL}.truth.map ${EVAL}/stripped/${LANG}/${b}.${o}${SPECIAL}.ins.${dis} >> ${EVALDIR}/ins.${dis}.${COMP}.${LANG}${SPECIAL}.stripped.stats fi done done done } echo "=============== GCC (32 bit) ===============" inscmp 'C' "${BMARKS_C}" 'ins/gcc510-32' 'truth/gcc510-32' 'gcc510-32' inscmp 'C++' "${BMARKS_CPP}" 'ins/gcc510-32' 'truth/gcc510-32' 'gcc510-32' echo "=============== GCC (64 bit) ===============" inscmp 'C' "${BMARKS_C}" 'ins/gcc510-64' 'truth/gcc510-64' 'gcc510-64' inscmp 'C++' "${BMARKS_CPP}" 'ins/gcc510-64' 'truth/gcc510-64' 'gcc510-64' echo "=============== GCC (64 bit) static/flto ===============" inscmp 'C' "${BMARKS_C}" 'ins/gcc510-64' 'truth/gcc510-64' 'gcc510-64' '.static' inscmp 'C' "${BMARKS_C}" 'ins/gcc510-64' 'truth/gcc510-64' 'gcc510-64' '.static.flto' echo "=============== LLVM (32 bit) ===============" inscmp 'C' "${BMARKS_C}" 'ins/llvm370-32' 'truth/llvm370-32' 'llvm370-32' inscmp 'C++' "${BMARKS_CPP}" 'ins/llvm370-32' 'truth/llvm370-32' 'llvm370-32' echo "=============== LLVM (64 bit) ===============" inscmp 'C' "${BMARKS_C}" 'ins/llvm370-64' 'truth/llvm370-64' 'llvm370-64' inscmp 'C++' "${BMARKS_CPP}" 'ins/llvm370-64' 'truth/llvm370-64' 'llvm370-64' echo "=============== Visual Studio (32 bit) ===============" inscmp 'C' "${BMARKS_C}" 'ins/vs15-32' 'truth/vs15-32' 'vs15-32' inscmp 'C++' "${BMARKS_CPP}" 'ins/vs15-32' 'truth/vs15-32' 'vs15-32' echo "=============== Visual Studio (64 bit) ===============" inscmp 'C' "${BMARKS_C}" 'ins/vs15-64' 'truth/vs15-64' 'vs15-64' inscmp 'C++' "${BMARKS_CPP}" 'ins/vs15-64' 'truth/vs15-64' 'vs15-64' |
In essence, the output files created by this script combine the outputs of ins-cmp.py
for all test cases given a particular compiler/architecture configuration, one test case per line. As before, we have a loop over all test cases and optimization levels. This time, we have an additional loop at line 38, which goes over an array containing all disassemblers we want to evaluate. This way, we don’t have to manually run the comparison script for each disassembler. Note that the disassembler names, as specified in the array, need to match those used in the output file names generated by our disassembly script.
The script first resets all output files (lines 24-29), and then begins its main loop. The main loop simply calls ins-cmp.py
for each possible configuration, and saves the statistics to file, printing warnings for any test cases or ground truth files which cannot be found. After the script completes, you will find a collection of combined statistics files in the ins
directory, with one file per combination of compiler/architecture/language/disassembler. The file contents should look something like this (truncated for brevity).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
$ cat ins/ins.objdump222.gcc510-64.C.stripped.stats | head -n 15 400.perlbench O0 |bounds| = 301916 |ins| = 302140 |d| = 224 (100.07%) |out| = 0 (0.00 %) |TP| = 301916 (100.00%) |FP| = 0 (0.00 %) |FP_data| = 0 (0.00 %) |FN| = 0 (0.00 %) |FN_nop| = 0 (0.00 %) 400.perlbench O1 |bounds| = 201852 |ins| = 202301 |d| = 449 (100.22%) |out| = 0 (0.00 %) |TP| = 201852 (100.00%) |FP| = 0 (0.00 %) |FP_data| = 0 (0.00 %) |FN| = 0 (0.00 %) |FN_nop| = 0 (0.00 %) 400.perlbench O2 |bounds| = 212525 |ins| = 212864 |d| = 339 (100.16%) |out| = 0 (0.00 %) |TP| = 212525 (100.00%) |FP| = 0 (0.00 %) |FP_data| = 0 (0.00 %) |FN| = 0 (0.00 %) |FN_nop| = 0 (0.00 %) 400.perlbench O3 |bounds| = 268166 |ins| = 268595 |d| = 429 (100.16%) |out| = 0 (0.00 %) |TP| = 268166 (100.00%) |FP| = 0 (0.00 %) |FP_data| = 0 (0.00 %) |FN| = 0 (0.00 %) |FN_nop| = 0 (0.00 %) 400.perlbench Os |bounds| = 182038 |ins| = 182267 |d| = 229 (100.13%) |out| = 0 (0.00 %) |TP| = 182038 (100.00%) |FP| = 0 (0.00 %) |FP_data| = 0 (0.00 %) |FN| = 0 (0.00 %) |FN_nop| = 0 (0.00 %) 401.bzip2 O0 |bounds| = 22919 |ins| = 22965 |d| = 46 (100.20%) |out| = 0 (0.00 %) |TP| = 22919 (100.00%) |FP| = 0 (0.00 %) |FP_data| = 0 (0.00 %) |FN| = 0 (0.00 %) |FN_nop| = 0 (0.00 %) 401.bzip2 O1 |bounds| = 10963 |ins| = 11009 |d| = 46 (100.42%) |out| = 0 (0.00 %) |TP| = 10963 (100.00%) |FP| = 0 (0.00 %) |FP_data| = 0 (0.00 %) |FN| = 0 (0.00 %) |FN_nop| = 0 (0.00 %) 401.bzip2 O2 |bounds| = 11433 |ins| = 11510 |d| = 77 (100.67%) |out| = 0 (0.00 %) |TP| = 11433 (100.00%) |FP| = 0 (0.00 %) |FP_data| = 0 (0.00 %) |FN| = 0 (0.00 %) |FN_nop| = 0 (0.00 %) 401.bzip2 O3 |bounds| = 17816 |ins| = 17900 |d| = 84 (100.47%) |out| = 0 (0.00 %) |TP| = 17816 (100.00%) |FP| = 0 (0.00 %) |FP_data| = 0 (0.00 %) |FN| = 0 (0.00 %) |FN_nop| = 0 (0.00 %) 401.bzip2 Os |bounds| = 9744 |ins| = 9786 |d| = 42 (100.43%) |out| = 0 (0.00 %) |TP| = 9744 (100.00%) |FP| = 0 (0.00 %) |FP_data| = 0 (0.00 %) |FN| = 0 (0.00 %) |FN_nop| = 0 (0.00 %) 403.gcc O0 |bounds| = 900235 |ins| = 900388 |d| = 153 (100.02%) |out| = 0 (0.00 %) |TP| = 900235 (100.00%) |FP| = 0 (0.00 %) |FP_data| = 0 (0.00 %) |FN| = 0 (0.00 %) |FN_nop| = 0 (0.00 %) 403.gcc O1 |bounds| = 563795 |ins| = 564162 |d| = 367 (100.07%) |out| = 0 (0.00 %) |TP| = 563795 (100.00%) |FP| = 0 (0.00 %) |FP_data| = 0 (0.00 %) |FN| = 0 (0.00 %) |FN_nop| = 0 (0.00 %) 403.gcc O2 |bounds| = 569384 |ins| = 570021 |d| = 637 (100.11%) |out| = 0 (0.00 %) |TP| = 569384 (100.00%) |FP| = 0 (0.00 %) |FP_data| = 0 (0.00 %) |FN| = 0 (0.00 %) |FN_nop| = 0 (0.00 %) 403.gcc O3 |bounds| = 722205 |ins| = 723117 |d| = 912 (100.13%) |out| = 0 (0.00 %) |TP| = 722205 (100.00%) |FP| = 0 (0.00 %) |FP_data| = 0 (0.00 %) |FN| = 0 (0.00 %) |FN_nop| = 0 (0.00 %) 403.gcc Os |bounds| = 504087 |ins| = 504337 |d| = 250 (100.05%) |out| = 0 (0.00 %) |TP| = 504087 (100.00%) |FP| = 0 (0.00 %) |FP_data| = 0 (0.00 %) |FN| = 0 (0.00 %) |FN_nop| = 0 (0.00 %) |