Allegro.cc - Online Community

Allegro.cc Forums » Programming Questions » Funny Networking Crashes

This thread is locked; no one can reply to it. rss feed Print
Funny Networking Crashes
X-G
Member #856
December 2000
avatar

This rather weird bug has been coming up a lot lately, and I'm doing my best to battle it. How it arises, though, is outside of my understanding. Perhaps someone here can see it.

I'm getting some really funny results and crashes in my networking code. I've thrown in a lot of TRACE() calls everywhere in the code to keep on top of things, and some damn funny results pop out. Here's a log entry:

Quote:

Attempting to start server ...
Srv0AFD2A88: Client # reset to 8
Socket 0AFD2B10/516 listening on port 26078.
Server started OK
'127.0.0.1:26078'
Connecting to 127.0.0.1:26078 ...
0AFD2B10/-1 accepted and pushed socket 0AFD2B10/-1.
0AFD2B10 updating traffic
Updating server 0AFD2A88
**Going into individual socket loop (8 sockets)
.. Auditing socket 0: not occupied.
.. Auditing socket 1: not occupied.
.. Auditing socket 2: not occupied.
.. Auditing socket 3: not occupied.
.. Auditing socket 4: not occupied.
.. Auditing socket 5: not occupied.
.. Auditing socket 6: not occupied.
.. Auditing socket 7: not occupied.
Placed incoming connection 0AFD2B10 in slot 0
Updating client.0AFD3374 updating traffic
0AFD3374/532 Beginning to send message of type 1 (37 bytes) ... 0AFD3374/532 Message sent
Updating server 0AFD2A88
**Going into individual socket loop (8 sockets)
.. Auditing socket 0: updating (0AFD2B10/-1)
0AFD2B10 updating traffic

This is the whole log. It freezes or segfaults after this.

Those of you experienced with network programming might immedeately see that the highlighted line is weird. It might take some explanation of exactly what is being pushed here, though.

My system is laid out like this: A listening CSocket will, when it encounters an incoming connection, accept that socket using normal socket operations, wrap it up in another CSocket, and push it onto an internal STL list, to be retreived by the main program later. Mutexes are being employed to ensure the two different threads don't interfere with each other.
Here's the piece of relevant code:

1void CSocket::UpdateListening()
2{
3 // There's an incoming connection
4 if (Incoming())
5 {
6 // Accept it
7 SOCKET s = accept(sock, NULL, NULL);
8
9 // Wrap it in a class
10 CSocket *wrapped = new CSocket(s);
11
12 pthread_mutex_lock(&socket_mutex);
13
14 // Push it onto list
15 sock_queue.push_back(wrapped);
16
17 pthread_mutex_unlock(&socket_mutex);
18
19 TRACE("%p/%d accepted and pushed socket %p/%d.\n", this, sock, wrapped, s); // #DEBUG
20 }
21}

Now, as you can see, SOMEHOW this function's socket is set to -1, which is obviously an invalid socket descriptor. Then, it somehow accepts and pushes ITSELF onto its own stack of incoming connections. Finally, the prog crashes.
What's even more peculiar is that this is only one of several reasons for the thing breaking I get. Some times it just freezes after auditing all eight slots. Some times it even has time to send a message to itself (through loopback). This is the error that seems to be most active right now, though.

--
Since 2008-Jun-18, democracy in Sweden is dead. | 悪霊退散!悪霊退散!怨霊、物の怪、困った時は ドーマン!セーマン!ドーマン!セーマン! 直ぐに呼びましょう陰陽師レッツゴー!

Thomas Fjellstrom
Member #476
June 2000
avatar

(IMO) Sounds to me like you may be having problems with out of bounds memory access. (ie: accessing invalid pointers..)

--
Thomas Fjellstrom - [website] - [email] - [Allegro Wiki] - [Allegro TODO]
"If you can't think of a better solution, don't try to make a better solution." -- weapon_S
"The less evidence we have for what we believe is certain, the more violently we defend beliefs against those who don't agree" -- https://twitter.com/neiltyson/status/592870205409353730

X-G
Member #856
December 2000
avatar

What I'd particularly like to know is how a pointer just retrieved from new can be set to this ... (surely, it can't be the other way around here, can it?)

If it helps, I'll post the socket wrapping function and associated helper ...

1CSocket::CSocket(SOCKET s)
2{
3 CSocket();
4 
5 if (!s)
6 return;
7
8 sock = s;
9 state = SST_CONNECTED;
10 ResetQueues();
11}
12 
13void CSocket::ResetQueues()
14{
15 in_queue.clear();
16 in_msg.Clear();
17 in_msg.SetType(NMSG_VOID);
18 recvbuf[0] = 0;
19 recvsize = 0;
20
21 out_queue.clear();
22 out_msg.Clear();
23 out_msg.SetType(NMSG_VOID);
24 sendbuf[0] = 0;
25 sendsize = 0;
26}

The two queues are STL lists (list<CMessage> ). CMessage is a class of my own (yes, it has copy constructors and overloaded = operator).

http://www.gamleby.net/x-g/edit.gif

After some more searching, it seems that select() returns -1, which would be an error. Still, the socket is in the incoming set, and errno is 0.

This is getting worse every minute.

http://www.gamleby.net/x-g/edit.gif

Great, now it segfaults in THIS function, too. It's the line with "CSocket *s = ..." ... and begin() must be a proper iterator after checking that the list isn't empty, right?

CSocket *CSocket::PopConnection()
{
  if (sock_queue.empty())
    return NULL;

  CSocket *s = *(sock_queue.begin());
  sock_queue.pop_front();
  return s;
}

--
Since 2008-Jun-18, democracy in Sweden is dead. | 悪霊退散!悪霊退散!怨霊、物の怪、困った時は ドーマン!セーマン!ドーマン!セーマン! 直ぐに呼びましょう陰陽師レッツゴー!

Elias
Member #358
May 2000

CSocket::CSocket(SOCKET s)
{
    CSocket();

    if (!s)
        return;
    
    sock = s;
    state = SST_CONNECTED;
    ResetQueues();
}

I don't know much C++ so may be totally off, but what effect should the line "CSocket();" have? To my understanding, it will create another CSocket object, which is never referenced (unless they made C++ do it the same way as Java by now :) I think you can't directly call a constructor like that - but as I said, I may be wrong.

Quote:

--
Durch, für, gegen, ohne, um ...

I just stop by on allegro.cc like twice a week, so I would have missed it - is there any thread explaining your sig? Looks like just some random german words to me..

--
"Either help out or stop whining" - Evert

gnolam
Member #2,030
March 2002
avatar

I can fill you in on the sig thing: X-G said SciTE für alle!, which made Spellcaster reply "Nur aus Neugierde: Seit wann ist Deutsch hier so beliebt?". Just for fun, I therefore changed my sig to "An, auf, hinter, in, neben, über, unter, vor, zwischen" (prepositions that control the accusative and dative cases in German). X-G changed his sig to match mine (but with the prepositions that control the accusative), and now Trumgottist completed the cycle with "Aus, ausser, bei, ..." (dative) :)
When you learn German as a second language, you have to learn those prepositions... you may not remember anything else, but you'll remember those prepositions for the rest of your life :)

--
Move to the Democratic People's Republic of Vivendi Universal (formerly known as Sweden) - officially democracy- and privacy-free since 2008-06-18!

Elias
Member #358
May 2000

Heh, I see.. I speak German, but I'd have had no idea you can group prepositions like that :)

--
wegen zeit

--
"Either help out or stop whining" - Evert

Evert
Member #794
November 2000
avatar

And as I said earlier, what I remember from highschool is bis, dürch, für, gegen, ohne, um, so I'm left with the feeling that X-G's sig is incomplete...

Elias
Member #358
May 2000

Hm, is "bis" a preposition at all? It means "until", so you can only put a time/location after it. And I just realized, there's a lot more prepositions with genitive than the 2 i came up with - that's probably why you only had to learn dativ/accusativ ones :)

About the first topic, I just tried out what happens if I call a constructor from another constructor, and my suspection was right: It creates another object, and it has no relevance if you call it from within a constructor, or somewhere else. If you want multiple constructors of a class to share the same initalization code, you have to make a separate initialization method in C++, and call it from all the constructors.

--
"Either help out or stop whining" - Evert

Sakuera
Member #3,222
February 2003
avatar

Hmm, I almost fail German exams, cause e.g.

I translate "What's your name" to "Was ist dein nehme" despite I know it should be "Wie heisst du".

Elias
Member #358
May 2000

"Was ist dein Name" is correct. It just sounds a bit more formal, probably like if you say "How are you named?" in English.
I think it just looks to teachers like you know more, when you don't just transliterate, but make a new sentence with the same meaning :)

--
"Either help out or stop whining" - Evert

spellcaster
Member #1,493
September 2001
avatar

Nope. It should be "Wie ist Dein Name?" :)

--
There are no stupid questions, but there are a lot of inquisitive idiots.

Evert
Member #794
November 2000
avatar

Shouldn't that more properly be `Wie heissen Sie' and `Wie ist Ihre Name?' ;)

I recently pulled my French grammer book off the shelf... perhaps I should do the same with German... :)

Elias
Member #358
May 2000

Blah, I can't even speak my mother tongue. It's of course "Wie", not "Was". But still, "Was war nochmal dein Name?" wouldn't necessarily sound wrong to me..

About using "Sie" and "Ihr" instead of "du" and "dein" (before the latest Rechtschreibreform, the latter also started with capitals).. it's the same as in French.

--
"Either help out or stop whining" - Evert

Thomas Fjellstrom
Member #476
June 2000
avatar

Bah. Its all greek to me.

--
Thomas Fjellstrom - [website] - [email] - [Allegro Wiki] - [Allegro TODO]
"If you can't think of a better solution, don't try to make a better solution." -- weapon_S
"The less evidence we have for what we believe is certain, the more violently we defend beliefs against those who don't agree" -- https://twitter.com/neiltyson/status/592870205409353730

Frank Drebin
Member #2,987
December 2002
avatar

It should be "Wie heisst du?" or "Wie ist dein Name?" oder wie oder was scheisse net nochmal.

X-G
Member #856
December 2000
avatar

Elias: You are correct - but I don't think it's that important - the default constructor does almost nothing. now can we stop being off-topic? :P

http://www.gamleby.net/x-g/edit.gif

What the hell is going on here? Take a look at this little piece of code. It's from the function that's supposed to check for incoming stuff. Major paranoid debugging stuff there.

  fd_set set;
  FD_ZERO(&set);
  FD_SET(sock, &set);

  TRACE("Before selecting, socket is %d, ", sock);
  int ret = select(sock+1, &set, NULL, NULL, &zero_timeout);
  TRACE("select returned %d, errno is %d, socket is %d\n", ret, errno, sock);

Now, you would expect sock to stay the same, right? Those traces are overly paranoid, right? Wrong.

allegro.log said:

Connecting to 127.0.0.1:26078 ...
Before selecting, socket is 516, select returned 1, errno is 0, socket is 3998072
Before selecting, socket is 3998072, select returned -1, errno is 0, socket is 4034632

--
Since 2008-Jun-18, democracy in Sweden is dead. | 悪霊退散!悪霊退散!怨霊、物の怪、困った時は ドーマン!セーマン!ドーマン!セーマン! 直ぐに呼びましょう陰陽師レッツゴー!

CGamesPlay
Member #2,559
July 2002
avatar

Just some crazy, off-the wall idea, maybe your CSocket instance that was listening also sent out the request?

Yeah, I didn't think that was it...

--
Tomasu: Every time you read this: hugging!

Ryan Patterson - <http://cgamesplay.com/>

Thomas Fjellstrom
Member #476
June 2000
avatar

Um.. variables changing on thier own... What does that remind me of... Hmmm.. Memory/buffer overruns? accessing invalid pointers maybe? something like that. Scour your source code. You're bound to find a little error somewhere. (It seems my network code doesn't have problems like that.. It sucessfully serves webpages for me.. but then I don't directly use select... Its all even threaded...)

--
Thomas Fjellstrom - [website] - [email] - [Allegro Wiki] - [Allegro TODO]
"If you can't think of a better solution, don't try to make a better solution." -- weapon_S
"The less evidence we have for what we believe is certain, the more violently we defend beliefs against those who don't agree" -- https://twitter.com/neiltyson/status/592870205409353730

X-G
Member #856
December 2000
avatar

I may have found what could be wrong. I'm not entirely sure yet. You know all those TRACE() things I put in everywhere, for debugging? Well ... Allegro isn't thread-safe, as you know. And the trace printing thing isn't mutexed. And my prog is multi-threaded. Could that have something to do with it?

--
Since 2008-Jun-18, democracy in Sweden is dead. | 悪霊退散!悪霊退散!怨霊、物の怪、困った時は ドーマン!セーマン!ドーマン!セーマン! 直ぐに呼びましょう陰陽師レッツゴー!

Elias
Member #358
May 2000

You could try replacing it with printf and see if you get any difference.

Do you also get the same results when there's only one thread listening for connections? And maybe you could show the rest of the incoming function, and where it is called from, there's always the possibility of a small hidden bug somewhere in the code, even it is very trivial code :)

Btw, under which OS is this running? gdb (and probably the MSVC-debugger) should allow to analyze variables in different threads after a crash (or at a breakpoint) - i don't know how good gdb is with C++ though. And I heared, in linux, linking with electricfence allows to detect memory leaks or even buffer overruns. I have no idea if it works with STL, but it might be worth a try.

--
"Either help out or stop whining" - Evert

X-G
Member #856
December 2000
avatar

I tried to remove -DDEBUGMODE, which turns all TRACE()'s into nothing. It still crashes, but somewhere else this time. Anyway, I use Dr MinGW for debugging at the moment, and sadly it doesn't seem to be able to watch values. Gdb is a pain, because each TRACE() seems to send a SIGTRAP to the program, causing gdb to keep breaking execution. The program only has two threads; the main thread, and the thread with the listening socket.

--
Since 2008-Jun-18, democracy in Sweden is dead. | 悪霊退散!悪霊退散!怨霊、物の怪、困った時は ドーマン!セーマン!ドーマン!セーマン! 直ぐに呼びましょう陰陽師レッツゴー!

Elias
Member #358
May 2000

Hm, yes, I remember that behaviour of TRACE and switched to printf because of it. And gdb really is much nicer in linux. Try compiling in linux, and look at the stack backtraces you get there for both threads - it might reveal something.

I also have another guess where things could go wrong - maybe when you remove items from the STL list. Even if you are locking that part of the code, it sounds suspicious of having the capability to screw up memory :)

--
"Either help out or stop whining" - Evert

X-G
Member #856
December 2000
avatar

My project doesn't compile in linux yet, and I don't intend to make any conscious effort towards it until I get this working on Windows to begin with.

--
Since 2008-Jun-18, democracy in Sweden is dead. | 悪霊退散!悪霊退散!怨霊、物の怪、困った時は ドーマン!セーマン!ドーマン!セーマン! 直ぐに呼びましょう陰陽師レッツゴー!

Cage
Member #1,277
March 2001

X-G: C or C++? If C++, you could use this function that I've written instead of TRACE()...

void debugInfo(const char *format, ...)
{
   char buf[512];

   va_list ap;
   va_start(ap, format);
   uvszprintf(buf, sizeof(buf), format, ap);
   va_end(ap);

   ofstream messageOut("debugInfo.txt", ios::app);
   messageOut << buf << "\n";
   messageOut << flush;
   messageOut.close();
}

If C, you could probably change the file part a bit to use C-style I/O :). Just make sure you delete debugInfo.txt upon startup every time ;).

-----
"I'm dumb!. it prolly wont do anything just like Sub7 when u extract it to ur own system I'm dumb!." - theforgotten
"heh i got hit by sub7 before. I just dont know how i got it. It took me about 2 yrs to figure out which virus i had. I'm dumb!. then i started wanting to hack and i got sub7 just ot play around with it and i found the features in it that i had been affected by when i got the virus." - theforgotten

X-G
Member #856
December 2000
avatar

Cage: You'd think the fact that all the code snippets involve classes would tip you off as to what I'm using, eh? ;D

Thanks, anyway. I'll try it out. Gdb is being a bitch right now, though - it refuses to run my project and starts spewing lines mentioning coffread.c.

--
Since 2008-Jun-18, democracy in Sweden is dead. | 悪霊退散!悪霊退散!怨霊、物の怪、困った時は ドーマン!セーマン!ドーマン!セーマン! 直ぐに呼びましょう陰陽師レッツゴー!

Go to: