Return value is garbage after call
Aaron Bolyard

Problem: Method Model::get_color, from my library Kvre, returns a glm::vec4. In a different library, Chomsky, which calls the method in Kvre, the return value is correct. In an application, Zeitgeist, which also calls the method in Kvre, the return value is garbage.

1. There is no memory corruption. I've run the program through Valgrind.

2. The value is correct before being returned. In other words, if I print the value from Model::get_color prior to returning, it's correct, but in Zeitgeist, it's not.

3. An identical function in Chomsky and Zeitgiest--to print the color value--compiles differently.

4. If the method returns a glm::vec3, the result is correct.

5. Zeitgeist depends on Chomsky and Kvre. Chomsky depends on Kvre.

6. glm::vec4 is from the OpenGL Mathematics library. As far as I know, it's internally an SSE vector type on my platform.

I cannot produce a simple test case independent on my large code base, so bear with me.

Here's the functions:

1// in Kvre 2glm::vec4 kvre::Model::get_color(std::size_t index) const 3{ 4 if (this->colors.empty()) 5 { 6 std::size_t num_vertices = this->get_num_vertices(); 7 this->colors.resize(this->get_num_vertices()); 8 for (std::size_t i = 0; i < num_vertices; ++i) 9 { 10 this->colors[i] = this->profile->get_model_vertex_color( 11 *this->colors_fetcher, i); 12 } 13 } 14 15 if (index == 0xffffffff) 16 { 17 auto result = this->colors[0]; 18 printf( 19 "result: %f %f %f %f\n", 20 result.x, result.y, result.z, result.w); 21 return result; 22 } 23 24 return this->colors[index]; 25} 26 27// in Zeitgeist/Chomsky 28static void print_model_color(const kvre::Model* model) 29{ 30 glm::vec4 color = model->get_color(0xffffffff); 31 printf("%p color: %f %f %f %f\n", model, color.r, color.g, color.b, color.a); 32}

(The check is for debugging. My code processes hundreds of thousands of vertices, so printing each one is unwise.)

Here's the disassembly of the print_model_color from Zeitgeist:

```zeitgeist`print_model_color:
0x41d740 <+0>:  pushq  %rbp
0x41d741 <+1>:  movq   %rsp, %rbp
0x41d744 <+4>:  subq   \$0x20, %rsp
0x41d748 <+8>:  movl   \$0xffffffff, %eax         ; imm = 0xFFFFFFFF
0x41d74d <+13>: movl   %eax, %esi
0x41d74f <+15>: movq   %rdi, -0x8(%rbp)
->  0x41d753 <+19>: movq   -0x8(%rbp), %rdi
0x41d757 <+23>: callq  0x418664                  ; symbol stub for: kvre::Model::get_color(unsigned long) const
0x41d75c <+28>: movabsq \$0x4ba912, %rdi           ; imm = 0x4BA912
0x41d766 <+38>: movq   %rax, -0x18(%rbp)
0x41d76a <+42>: movq   %rdx, -0x10(%rbp)
0x41d76e <+46>: movq   -0x8(%rbp), %rsi
0x41d772 <+50>: cvtss2sd -0x18(%rbp), %xmm0
0x41d777 <+55>: cvtss2sd -0x14(%rbp), %xmm1
0x41d77c <+60>: cvtss2sd -0x10(%rbp), %xmm2
0x41d781 <+65>: cvtss2sd -0xc(%rbp), %xmm3
0x41d786 <+70>: movb   \$0x4, %al
0x41d788 <+72>: callq  0x4186d4                  ; symbol stub for: printf
0x41d78d <+77>: movl   %eax, -0x1c(%rbp)
0x41d794 <+84>: popq   %rbp
0x41d795 <+85>: retq
```

And here's the disassembly of the method from Chomsky:

```libChomsky.so`print_model_color:
0x801096d00 <+0>:  pushq  %rbp
0x801096d01 <+1>:  movq   %rsp, %rbp
0x801096d04 <+4>:  subq   \$0x30, %rsp
0x801096d08 <+8>:  leaq   0x50fc1(%rip), %rax
0x801096d0f <+15>: movq   %rdi, -0x8(%rbp)
->  0x801096d13 <+19>: movl   \$0xffffffff, %ecx         ; imm = 0xFFFFFFFF
0x801096d18 <+24>: movl   %ecx, %esi
0x801096d1a <+26>: movq   %rax, -0x20(%rbp)
0x801096d1e <+30>: callq  0x801084f0c               ; symbol stub for: kvre::Model::get_color(unsigned long) const
0x801096d23 <+35>: movlps %xmm0, -0x18(%rbp)
0x801096d27 <+39>: movlps %xmm1, -0x10(%rbp)
0x801096d2b <+43>: movq   -0x8(%rbp), %rsi
0x801096d2f <+47>: cvtss2sd -0x18(%rbp), %xmm0
0x801096d34 <+52>: cvtss2sd -0x14(%rbp), %xmm1
0x801096d39 <+57>: cvtss2sd -0x10(%rbp), %xmm2
0x801096d3e <+62>: cvtss2sd -0xc(%rbp), %xmm3
0x801096d43 <+67>: movq   -0x20(%rbp), %rdi
0x801096d47 <+71>: movb   \$0x4, %al
0x801096d49 <+73>: callq  0x801084dec               ; symbol stub for: printf
0x801096d4e <+78>: movl   %eax, -0x24(%rbp)
0x801096d55 <+85>: popq   %rbp
0x801096d56 <+86>: retq
```

I know little about x86 assembly, but I can tell there's a difference in how the return value is stored.

Results of first few calls from Chomsky:

```result: 0.133333 0.156863 0.211765 1.000000
0x80cf39700 color: 0.133333 0.156863 0.211765 1.000000
result: 0.070588 0.066667 0.066667 1.000000
0x80cf39800 color: 0.070588 0.066667 0.066667 1.000000
result: 0.211765 0.176471 0.133333 1.000000
0x80cf39900 color: 0.211765 0.176471 0.133333 1.000000
```

And from Zeitgeist:

```result: 0.133333 0.156863 0.211765 1.000000
0x80cf39700 color: 0.000000 0.000000 0.000000 0.000000
result: 0.070588 0.066667 0.066667 1.000000
0x80cf39800 color: 0.000000 0.000000 0.000000 0.000000
result: 0.211765 0.176471 0.133333 1.000000
0x80cf39900 color: 0.000000 0.000000 0.000000 0.000000
```

The models are identical (see the pointer), but the return value is not.

Why is the method returning garbage in Zeitgeist but not Chomsky?

I have a hunch it's to do with some compiler or linker settings causing incorrect code generation, but I've tried various permutations and exhausted those options I'm aware of.

SiegeLord

If you're suspecting such low level bugs, it's worthwhile to try out other compilers (and compiler versions).

someone972

Can you post the assembly for the get_color functions produced by each of those? The disassembly for the print_color looks fine on it's own, so I'd have to see what the get_color function is setting the values to. If you want a rundown of the print_color assembly I can do that for you, just let me know.

Aaron Bolyard
SiegeLord said:

If you're suspecting such low level bugs, it's worthwhile to try out other compilers (and compiler versions).

Tested it with clang 3.8 and 5.0 without any difference. Will switch to Ubuntu and test it with GCC 6 soon.

Can you post the assembly for the get_color functions produced by each of those? The disassembly for the print_color looks fine on it's own, so I'd have to see what the get_color function is setting the values to. If you want a rundown of the print_color assembly I can do that for you, just let me know.

Sure thing:

```libKvre.so`kvre::Model::get_color:
0x801cbcbd0 <+0>:   pushq  %rbp
0x801cbcbd1 <+1>:   movq   %rsp, %rbp
0x801cbcbd4 <+4>:   subq   \$0xa0, %rsp
0x801cbcbdb <+11>:  movq   %rdi, -0x50(%rbp)
0x801cbcbdf <+15>:  movq   %rsi, -0x58(%rbp)
0x801cbcbe3 <+19>:  movq   -0x50(%rbp), %rsi
->  0x801cbcbe7 <+23>:  movq   %rsi, %rdi
0x801cbcbf1 <+33>:  movq   %rdi, -0x38(%rbp)
0x801cbcbf5 <+37>:  movq   -0x38(%rbp), %rdi
0x801cbcbf9 <+41>:  movq   (%rdi), %rax
0x801cbcbfc <+44>:  cmpq   0x8(%rdi), %rax
0x801cbcc00 <+48>:  movq   %rsi, -0x80(%rbp)
0x801cbcc04 <+52>:  jne    0x801cbcce3               ; <+275> at model.cpp:86
0x801cbcc0a <+58>:  movq   -0x80(%rbp), %rdi
0x801cbcc0e <+62>:  callq  0x801cbb484               ; symbol stub for: kvre::Model::get_num_vertices() const
0x801cbcc13 <+67>:  movq   %rax, -0x60(%rbp)
0x801cbcc17 <+71>:  movq   -0x80(%rbp), %rax
0x801cbcc21 <+81>:  movq   -0x80(%rbp), %rdi
0x801cbcc25 <+85>:  movq   %rax, -0x88(%rbp)
0x801cbcc2c <+92>:  callq  0x801cbb484               ; symbol stub for: kvre::Model::get_num_vertices() const
0x801cbcc31 <+97>:  movq   -0x88(%rbp), %rdi
0x801cbcc38 <+104>: movq   %rax, %rsi
0x801cbcc3b <+107>: callq  0x801cbb0a4               ; symbol stub for: std::__1::vector<glm::tvec4<float, (glm::precision)0>, std::__1::allocator<glm::tvec4<float, (glm::precision)0> > >::resize(unsigned long)
0x801cbcc40 <+112>: movq   \$0x0, -0x68(%rbp)
0x801cbcc48 <+120>: movq   -0x68(%rbp), %rax
0x801cbcc4c <+124>: cmpq   -0x60(%rbp), %rax
0x801cbcc50 <+128>: jae    0x801cbccde               ; <+270> at model.cpp:86
0x801cbcc56 <+134>: movq   -0x80(%rbp), %rax
0x801cbcc60 <+144>: movq   -0x68(%rbp), %rcx
0x801cbcc64 <+148>: movq   %rax, -0x8(%rbp)
0x801cbcc68 <+152>: movq   %rcx, -0x10(%rbp)
0x801cbcc6c <+156>: movq   -0x8(%rbp), %rax
0x801cbcc70 <+160>: movq   (%rax), %rax
0x801cbcc73 <+163>: shlq   \$0x4, %rcx
0x801cbcc7a <+170>: movq   -0x80(%rbp), %rcx
0x801cbcc7e <+174>: movq   0x30(%rcx), %rdx
0x801cbcc82 <+178>: movq   0x48(%rcx), %rsi
0x801cbcc86 <+182>: movq   (%rdx), %rdi
0x801cbcc89 <+185>: movq   0x30(%rdi), %rdi
0x801cbcc8d <+189>: movq   -0x68(%rbp), %r8
0x801cbcc91 <+193>: movq   %rdi, -0x90(%rbp)
0x801cbcc98 <+200>: movq   %rdx, %rdi
0x801cbcc9b <+203>: movq   %r8, %rdx
0x801cbcc9e <+206>: movq   -0x90(%rbp), %r8
0x801cbcca5 <+213>: movq   %rax, -0x98(%rbp)
0x801cbccac <+220>: callq  *%r8
0x801cbccaf <+223>: movlps %xmm0, -0x78(%rbp)
0x801cbccb3 <+227>: movlps %xmm1, -0x70(%rbp)
0x801cbccb7 <+231>: movq   -0x78(%rbp), %rax
0x801cbccbb <+235>: movq   -0x98(%rbp), %rcx
0x801cbccc2 <+242>: movq   %rax, (%rcx)
0x801cbccc5 <+245>: movq   -0x70(%rbp), %rax
0x801cbccc9 <+249>: movq   %rax, 0x8(%rcx)
0x801cbcccd <+253>: movq   -0x68(%rbp), %rax
0x801cbccd5 <+261>: movq   %rax, -0x68(%rbp)
0x801cbccd9 <+265>: jmp    0x801cbcc48               ; <+120> at model.cpp:81
0x801cbccde <+270>: jmp    0x801cbcce3               ; <+275> at model.cpp:86
0x801cbcce3 <+275>: movl   \$0xffffffff, %eax         ; imm = 0xFFFFFFFF
0x801cbcce8 <+280>: movl   %eax, %ecx
0x801cbccea <+282>: cmpq   %rcx, -0x58(%rbp)
0x801cbccee <+286>: jne    0x801cbcd55               ; <+389> at model.cpp:97
0x801cbccf4 <+292>: leaq   0xeb26(%rip), %rdi        ;  + 545
0x801cbccfb <+299>: movq   -0x80(%rbp), %rax
0x801cbcd05 <+309>: movq   %rax, -0x18(%rbp)
0x801cbcd09 <+313>: movq   \$0x0, -0x20(%rbp)
0x801cbcd11 <+321>: movq   -0x18(%rbp), %rax
0x801cbcd15 <+325>: movq   -0x20(%rbp), %rcx
0x801cbcd19 <+329>: shlq   \$0x4, %rcx
0x801cbcd20 <+336>: movq   (%rcx), %rax
0x801cbcd23 <+339>: movq   %rax, -0x48(%rbp)
0x801cbcd27 <+343>: movq   0x8(%rcx), %rax
0x801cbcd2b <+347>: movq   %rax, -0x40(%rbp)
0x801cbcd2f <+351>: cvtss2sd -0x48(%rbp), %xmm0
0x801cbcd34 <+356>: cvtss2sd -0x44(%rbp), %xmm1
0x801cbcd39 <+361>: cvtss2sd -0x40(%rbp), %xmm2
0x801cbcd3e <+366>: cvtss2sd -0x3c(%rbp), %xmm3
0x801cbcd43 <+371>: movb   \$0x4, %al
0x801cbcd45 <+373>: callq  0x801cbb004               ; symbol stub for: printf
0x801cbcd4a <+378>: movl   %eax, -0x9c(%rbp)
0x801cbcd50 <+384>: jmp    0x801cbcd89               ; <+441> at model.cpp:98
0x801cbcd55 <+389>: movq   -0x80(%rbp), %rax
0x801cbcd5f <+399>: movq   -0x58(%rbp), %rcx
0x801cbcd63 <+403>: movq   %rax, -0x28(%rbp)
0x801cbcd67 <+407>: movq   %rcx, -0x30(%rbp)
0x801cbcd6b <+411>: movq   -0x28(%rbp), %rax
0x801cbcd6f <+415>: movq   -0x30(%rbp), %rcx
0x801cbcd73 <+419>: shlq   \$0x4, %rcx
0x801cbcd7a <+426>: movq   (%rcx), %rax
0x801cbcd7d <+429>: movq   %rax, -0x48(%rbp)
0x801cbcd81 <+433>: movq   0x8(%rcx), %rax
0x801cbcd85 <+437>: movq   %rax, -0x40(%rbp)
0x801cbcd89 <+441>: movq   -0x48(%rbp), %xmm0        ; xmm0 = mem[0],zero
0x801cbcd8e <+446>: movq   -0x40(%rbp), %xmm1        ; xmm1 = mem[0],zero
0x801cbcd9a <+458>: popq   %rbp
0x801cbcd9b <+459>: retq
```

Both Chomsky and Zeitgeist call this same method.

(kvre::Model::get_color starts on line 75, if that helps mapping lines to instructions.)

someone972

What compiler are you using for the get_color library, and which one are you using for each of the other two functions? It looks like you need to explicitly state the calling convention because the one that's printing zeros looks to be using System V AMD64 ABI (standard for linux?). Not sure what the get_color convention is, but since it's returning in xmm0/1 it's not the same.

Aaron Bolyard

I'm using the same compiler for everything (Chomsky, Zeitgeist, and Kvre): Clang 3.8 on FreeBSD 11. The build commands all use /usr/bin/CC.

I wonder if I'm screwing something up with CMake, but... With absolutely no changes it compiles and works just fine on Ubuntu with GCC 6-something.

Chris Katko

Have you tried using Clang's static analyzer? (a simple command line switch)

There is no memory corruption. I've run the program through Valgrind.

Valgrind does not guarantee there is no corruption. Have you tried the hundreds of additional flags/checks that can be turned on with Valgrind?

Aaron Bolyard

someone972, I tried compiling Kvre manually (instead of using CMake) and it produces different code, which works:

```pushq %rbp
movq %rsp, %rbp
subq \$0xa0, %rsp
movq %rdi, -0x50(%rbp)
movq %rsi, -0x58(%rbp)
movq -0x50(%rbp), %rsi
movq %rsi, %rdi
movq %rdi, -0x38(%rbp)
movq -0x38(%rbp), %rdi
movq (%rdi), %rax
cmpq 0x8(%rdi), %rax
movq %rsi, -0x80(%rbp)
jne 0x801cbcce3 ; <+275> at model.cpp:86
movq -0x80(%rbp), %rdi
callq 0x801cbb484 ; symbol stub for: kvre::Model::get_num_vertices() const
movq %rax, -0x60(%rbp)
movq -0x80(%rbp), %rax
movq -0x80(%rbp), %rdi
movq %rax, -0x88(%rbp)
callq 0x801cbb484 ; symbol stub for: kvre::Model::get_num_vertices() const
movq -0x88(%rbp), %rdi
movq %rax, %rsi
resize(unsigned long)
movq \$0x0, -0x68(%rbp)
movq -0x68(%rbp), %rax
cmpq -0x60(%rbp), %rax
jae 0x801cbccde ; <+270> at model.cpp:86
movq -0x80(%rbp), %rax
movq -0x68(%rbp), %rcx
movq %rax, -0x8(%rbp)
movq %rcx, -0x10(%rbp)
movq -0x8(%rbp), %rax
movq (%rax), %rax
shlq \$0x4, %rcx
movq -0x80(%rbp), %rcx
movq 0x30(%rcx), %rdx
movq 0x48(%rcx), %rsi
movq (%rdx), %rdi
movq 0x30(%rdi), %rdi
movq -0x68(%rbp), %r8
movq %rdi, -0x90(%rbp)
movq %rdx, %rdi
movq %r8, %rdx
movq -0x90(%rbp), %r8
movq %rax, -0x98(%rbp)
callq *%r8
movlps %xmm0, -0x78(%rbp)
movlps %xmm1, -0x70(%rbp)
movq -0x78(%rbp), %rax
movq -0x98(%rbp), %rcx
movq %rax, (%rcx)
movq -0x70(%rbp), %rax
movq %rax, 0x8(%rcx)
movq -0x68(%rbp), %rax
movq %rax, -0x68(%rbp)
jmp 0x801cbcc48 ; <+120> at model.cpp:81
jmp 0x801cbcce3 ; <+275> at model.cpp:86
movl \$0xffffffff, %eax ; imm = 0xFFFFFFFF
movl %eax, %ecx
cmpq %rcx, -0x58(%rbp)
jne 0x801cbcd55 ; <+389> at model.cpp:97
leaq 0xeb26(%rip), %rdi ; + 545
movq -0x80(%rbp), %rax
movq %rax, -0x18(%rbp)
movq \$0x0, -0x20(%rbp)
movq -0x18(%rbp), %rax
movq -0x20(%rbp), %rcx
shlq \$0x4, %rcx
movq (%rcx), %rax
movq %rax, -0x48(%rbp)
movq 0x8(%rcx), %rax
movq %rax, -0x40(%rbp)
cvtss2sd -0x48(%rbp), %xmm0
cvtss2sd -0x44(%rbp), %xmm1
cvtss2sd -0x40(%rbp), %xmm2
cvtss2sd -0x3c(%rbp), %xmm3
movb \$0x4, %al
callq 0x801cbb004 ; symbol stub for: printf
movl %eax, -0x9c(%rbp)
jmp 0x801cbcd89 ; <+441> at model.cpp:98
movq -0x80(%rbp), %rax
movq -0x58(%rbp), %rcx
movq %rax, -0x28(%rbp)
movq %rcx, -0x30(%rbp)
movq -0x28(%rbp), %rax
movq -0x30(%rbp), %rcx
shlq \$0x4, %rcx
movq (%rcx), %rax
movq %rax, -0x48(%rbp)
movq 0x8(%rcx), %rax
movq %rax, -0x40(%rbp)
movq -0x48(%rbp), %xmm0 ; xmm0 = mem[0],zero
movq -0x40(%rbp), %xmm1 ; xmm1 = mem[0],zero
popq %rbp
retq
```

(The old, formatted the same, for reference):

```pushq %rbp
movq %rsp, %rbp
subq \$0xb0, %rsp
movq %rdi, -0x50(%rbp)
movq %rsi, -0x58(%rbp)
movq -0x50(%rbp), %rsi
movq %rsi, %rdi
movq %rdi, -0x38(%rbp)
movq -0x38(%rbp), %rdi
movq (%rdi), %rax
cmpq 0x8(%rdi), %rax
movq %rsi, -0x80(%rbp)
jne 0x801cbc885 ; <+293> at model.cpp:86
movq -0x80(%rbp), %rdi
callq 0x801cbb3cc ; symbol stub for: kvre::Model::get_num_vertices() const
movq %rax, -0x60(%rbp)
movq -0x80(%rbp), %rax
movq -0x80(%rbp), %rdi
movq %rax, -0x88(%rbp)
callq 0x801cbb3cc ; symbol stub for: kvre::Model::get_num_vertices() const
movq -0x88(%rbp), %rdi
movq %rax, %rsi
resize(unsigned long)
movq \$0x0, -0x68(%rbp)
movq -0x68(%rbp), %rax
cmpq -0x60(%rbp), %rax
jae 0x801cbc880 ; <+288> at model.cpp:86
movq -0x80(%rbp), %rax
movq -0x68(%rbp), %rcx
movq %rax, -0x8(%rbp)
movq %rcx, -0x10(%rbp)
movq -0x8(%rbp), %rax
movq -0x10(%rbp), %rcx
shlq \$0x4, %rcx
movq -0x80(%rbp), %rax
movq 0x30(%rax), %rdx
movq (%rdx), %rsi
movq 0x30(%rsi), %rsi
movq 0x48(%rax), %rdi
movq -0x68(%rbp), %r8
movq %rdi, -0x90(%rbp)
movq %rdx, %rdi
movq -0x90(%rbp), %rdx
movq %rsi, -0x98(%rbp)
movq %rdx, %rsi
movq %r8, %rdx
movq -0x98(%rbp), %r8
movq %rcx, -0xa0(%rbp)
callq *%r8
movq %rax, -0x78(%rbp)
movq %rdx, -0x70(%rbp)
movq -0x78(%rbp), %rax
movq -0xa0(%rbp), %rcx
movq %rax, (%rcx)
movq -0x70(%rbp), %rax
movq %rax, 0x8(%rcx)
movq -0x68(%rbp), %rax
movq %rax, -0x68(%rbp)
jmp 0x801cbc7d8 ; <+120> at model.cpp:81
jmp 0x801cbc885 ; <+293> at model.cpp:86
movl \$0xffffffff, %eax ; imm = 0xFFFFFFFF
movl %eax, %ecx
cmpq %rcx, -0x58(%rbp)
jne 0x801cbc8f7 ; <+407> at model.cpp:97
leaq 0xedc0(%rip), %rdi ; + 413
movq -0x80(%rbp), %rax
movq %rax, -0x18(%rbp)
movq \$0x0, -0x20(%rbp)
movq -0x18(%rbp), %rax
movq -0x20(%rbp), %rcx
shlq \$0x4, %rcx
movq (%rcx), %rax
movq %rax, -0x48(%rbp)
movq 0x8(%rcx), %rax
movq %rax, -0x40(%rbp)
cvtss2sd -0x48(%rbp), %xmm0
cvtss2sd -0x44(%rbp), %xmm1
cvtss2sd -0x40(%rbp), %xmm2
cvtss2sd -0x3c(%rbp), %xmm3
movb \$0x4, %al
callq 0x801cbaf5c ; symbol stub for: printf
movl %eax, -0xa4(%rbp)
jmp 0x801cbc92b ; <+459> at model.cpp:98
movq -0x80(%rbp), %rax
movq -0x58(%rbp), %rcx
movq %rax, -0x28(%rbp)
movq %rcx, -0x30(%rbp)
movq -0x28(%rbp), %rax
movq -0x30(%rbp), %rcx
shlq \$0x4, %rcx
movq (%rcx), %rax
movq %rax, -0x48(%rbp)
movq 0x8(%rcx), %rax
movq %rax, -0x40(%rbp)
movq -0x48(%rbp), %rax
movq -0x40(%rbp), %rdx
popq %rbp
retq
```

I ran a diff and it appears to be handling return values differently, but I wouldn't even say I have a crude understanding of x86 assembly, so I'm not certain.

I'm not sure why CMake is making Clang generate bad code, but I'm going to switch to a different build system regardless. I want to generate reproducible builds, for defensive reasons, so I'm going to test Bazle first. If that fails, back to Premake...

At least I can make a working build, though:

{"name":"610904","src":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/8\/4\/84d6aa90839758934758e79e747c5eb3.png","w":1290,"h":749,"tn":"\/\/djungxnpq2nug.cloudfront.net\/image\/cache\/8\/4\/84d6aa90839758934758e79e747c5eb3"}

(I'm trying to make a utility that allows users to customize an online game's [RuneScape] drawing behavior. For example, making certain obnoxious MTX creatures render at 50% translucency or changing the GUI colors.)

Have you tried using Clang's static analyzer? (a simple command line switch)

Using LLVM 5.0's static analyzer across my entire suite of libraries under the KOMPROMAT module, there were a whopping half-a-dozen minor errors, none of them related to my problem. I'm surprised there were so few--I either ran the tool wrong or I'm the best.

Quote:

Valgrind does not guarantee there is no corruption. Have you tried the hundreds of additional flags/checks that can be turned on with Valgrind?

To be honest, no, but after I narrowed it down to a code generation problem, I stopped pursuing possible memory corruption. I make extensive use of modern C++ semantics like shared pointers, as well as using C++ containers whenever possible, so the scope of memory corruption is limited to a few specific places (marshaling graphics data, mostly).

edit: I switched to Bazel and the code is generated correctly now. Cookies.

someone972

Glad you were able to get something working. I've not used clang, so I'm not familiar with what makes it choose one calling convention over another in certain situations. Looks like a cool utility!

Aaron Bolyard

Thank you! I'm going to do a write-up on it once I finish the thing.

There's so many interesting problems I've had to solve. Not to mention the technical achievements born out of laziness...

For example, most of my development and testing is done snapshots of the OpenGL commands emitted by the game. I rarely run the game, yet I'm able to rapidly test my changes by instantly loading a specific OpenGL draw state!