Allegro.cc - Online Community

Allegro.cc Forums » Programming Questions » bind with SO_REUSEADDR still fails: Address already in use

This thread is locked; no one can reply to it. rss feed Print
bind with SO_REUSEADDR still fails: Address already in use
Hyena_
Member #8,852
July 2007
avatar

I have lately decided to update my Linux sockets knowledge from poll/select to epoll due to the fact that epoll should be able to handle millions of TCP/IP connections (probably really good advancement when developing MMO games).

So I copied some code from here as a basis to my learning project.

Unfortunately, that code has some flaws. When I connect to the server and disconnect the new client from the server by closing its socket (server-side), then, after exiting the server I cannot restart it for about a minute. The reason for this is that the port remains to be in use for some time. I have set the SO_REUSEADDR socket option but it doesn't change anything.

I tracked the problem to the following block of code:

#SelectExpand
1 struct addrinfo hints; 2 struct addrinfo *result, *rp; 3 int s, sfd, x; 4 5 memset(&hints, 0, sizeof(struct addrinfo)); 6 hints.ai_family = AF_UNSPEC; // Return IPv4 and IPv6 choices 7 hints.ai_socktype = SOCK_STREAM; // We want a TCP socket 8 hints.ai_flags = AI_PASSIVE; // All interfaces 9 10 s = getaddrinfo(nullptr, port, &hints, &result); 11 if (s != 0) { 12 if (error) { 13 error->append("getaddrinfo: "); 14 error->append(gai_strerror(s)); 15 error->append("\n"); 16 } 17 return -1; 18 } 19 20 for (rp = result; rp != nullptr; rp = rp->ai_next) { 21 sfd = socket(rp->ai_family, rp->ai_socktype, rp->ai_protocol); 22 if (sfd == -1) continue; 23 24 if (setsockopt(sfd, SOL_SOCKET, SO_REUSEADDR,(char *) &x, sizeof(x)) == -1) { 25 if (error) { 26 error->append("setsockopt: "); 27 error->append(strerror(errno)); 28 error->append("\n"); 29 } 30 } 31 else { 32 s = bind(sfd, rp->ai_addr, rp->ai_addrlen); 33 if (s == 0) break; // We managed to bind successfully! 34 if (error) { 35 char buf[64]; 36 sprintf(buf, "bind(%d, ?, %d): ", sfd, rp->ai_addrlen); 37 error->append(buf); 38 error->append(strerror(errno)); 39 error->append("\n"); 40 } 41 } 42 43 if (!close(sfd, error)) { 44 sfd = -1; 45 goto Fail; 46 } 47 }

When I replaced the above chunk of code with the following piece, the problem went away. I was able to restart the server immediately after closing it.

#SelectExpand
1 static struct sockaddr_in sa_zero; 2 struct sockaddr_in sa; 3 int x = 1; 4 int fd; 5 6 if ( ( fd = socket( AF_INET, SOCK_STREAM, 0 ) ) < 0 ) { 7 printf( "Init_tcp_socket: socket" ); 8 exit( 1 ); 9 } 10 11 if ( setsockopt( fd, SOL_SOCKET, SO_REUSEADDR,(char *) &x, sizeof(x) ) < 0 ) { 12 printf( "Init_tcp_socket: SO_REUSEADDR" ); 13 ::close(fd); 14 exit( 1 ); 15 } 16 17 int ppp = atoi(port); 18 sa = sa_zero; 19 sa.sin_family = AF_INET; 20 sa.sin_port = htons( ppp ); 21 22 if ( bind( fd, (struct sockaddr *) &sa, sizeof(sa) ) < 0 ) { 23 printf("Init socket: bind (port %d)", ppp ); 24 ::close(fd); 25 exit(1); 26 }

Can anyone tell me why the first piece of code does not work the way I want it to? Is it because of some IPv4 vs IPv6 specifics?

GullRaDriel
Member #3,861
September 2003
avatar

I think it comes from the AF_UNSPEC. I remember something in the man about ipv6 and AF_INET regarding SO_REUSEADDR

EDIT:
The only difference I can spot are minors, maybe that call to getaddrinfo ? call a freeaddrinfo(val); call it when val no more needed.

EDIT: Also why calling ::close instead of close ?

EDIT:

if (setsockopt(sfd, SOL_SOCKET, SO_REUSEADDR,(char *) &x, sizeof(x)) == -1) {
                if (error) {
                    error->append("setsockopt: ");
                    error->append(strerror(errno));
                    error->append("\n");
/* YOU NEED TO CLOSE THE SOCKET IF THERE WAS AN ERROR WITH setsockopt */
                    close( sfd );
                }
            }

EDIT: Why do you close it here ??

  if (!close(sfd, error)) {
 44                sfd = -1;
 45                goto Fail;
 46            }

"Code is like shit - it only smells if it is not yours"
Allegro Wiki, full of examples and articles !!

Thomas Fjellstrom
Member #476
June 2000
avatar

In addition to what Gull said, are you sure you're selecting the right address?

--
Thomas Fjellstrom - [website] - [email] - [Allegro Wiki] - [Allegro TODO]
"If you can't think of a better solution, don't try to make a better solution." -- weapon_S
"The less evidence we have for what we believe is certain, the more violently we defend beliefs against those who don't agree" -- https://twitter.com/neiltyson/status/592870205409353730

Hyena_
Member #8,852
July 2007
avatar

Since the code snippets in OP were perhaps a bit too much out of the context I am including the full sockets.h header file here. I am developing this header file as a light weight library-sort-of-thing. You can see the commented out chunk of code in the SOCKETS::open function that makes SO_REUSEADDR work as it is supposed to do.

The question remains --- why doesn't banu.com epoll example work with SO_REUSEADDR?

About the NEED-TO-CLOSE-THE-SOCKET-IF part, I am closing it later because on success the loop is broken out on this line:
if (s == 0) break; // We managed to bind successfully!

@Thomas Fjellstrom:
What do you mean by selecting the right address? I can connect to the server so I would assume I am selecting the right address. My gut feeling tells me that the problem might originate from these lines of codes:

        hints.ai_family   = AF_UNSPEC;   // Return IPv4 and IPv6 choices
        hints.ai_socktype = SOCK_STREAM; // We want a TCP socket
        hints.ai_flags    = AI_PASSIVE;  // All interfaces

I will investigate the AF_UNSPEC. I'm not entirely sure whether the epoll example has the best way to initiate port listening anyway. Perhaps I should just switch over to my own version which has no IPv6 stuff and that plays along with SO_REUSEADDR just well.

EDIT:
OK I solved it. I started to convert the create_and_bind function into the alternative that worked well with SO_REUSEADDR. I did it step-by-step, compiling and running the program after each change, to see at which point the SO_REUSEADDR would start to work. To my surprise, after having changed almost everything, the code still didn't work. Then I noticed this one little variable x I had copied from the working example to the create_and_bind function:

            if (setsockopt(sfd, SOL_SOCKET, SO_REUSEADDR,(char *) &x, sizeof(x)) == -1) {
                if (error) {
                    error->append("setsockopt: ");
                    error->append(strerror(errno));
                    error->append("\n");
                }
            }

I had forgotten to initialize the variable x as 1. Below is the fixed create_and_bind function that has SO_REUSEADDR working perfectly. Unfortunately, since my error was so specific, this topic is probably of no use to other people who might have similar problems in the future :D.

#SelectExpand
1 inline int create_and_bind(const char *port, std::string *error) { 2 struct addrinfo hints; 3 struct addrinfo *result, *rp; 4 int s, sfd, x=1; 5 6 memset(&hints, 0, sizeof(struct addrinfo)); 7 hints.ai_family = AF_UNSPEC; // Return IPv4 and IPv6 choices 8 hints.ai_socktype = SOCK_STREAM; // We want a TCP socket 9 hints.ai_flags = AI_PASSIVE; // All interfaces 10 11 s = getaddrinfo(nullptr, port, &hints, &result); 12 if (s != 0) { 13 if (error) { 14 error->append("getaddrinfo: "); 15 error->append(gai_strerror(s)); 16 error->append("\n"); 17 } 18 return -1; 19 } 20 21 for (rp = result; rp != nullptr; rp = rp->ai_next) { 22 sfd = socket(rp->ai_family, rp->ai_socktype, rp->ai_protocol); 23 if (sfd == -1) continue; 24 25 if (setsockopt(sfd, SOL_SOCKET, SO_REUSEADDR,(char *) &x, sizeof(x)) == -1) { 26 if (error) { 27 error->append("setsockopt: "); 28 error->append(strerror(errno)); 29 error->append("\n"); 30 } 31 } 32 else { 33 s = bind(sfd, rp->ai_addr, rp->ai_addrlen); 34 if (s == 0) break; // We managed to bind successfully! 35 if (error) { 36 char buf[64]; 37 sprintf(buf, "bind(%d, ?, %d): ", sfd, rp->ai_addrlen); 38 error->append(buf); 39 error->append(strerror(errno)); 40 error->append("\n"); 41 } 42 } 43 44 if (!close(sfd, error)) { 45 sfd = -1; 46 goto Fail; 47 } 48 } 49 50 if (rp == nullptr) sfd = -1; 51 52 Fail: 53 freeaddrinfo(result); 54 return sfd; 55 }

Thomas Fjellstrom
Member #476
June 2000
avatar

Hyena_ said:

What do you mean by selecting the right address?

I meant binding to the right address. Typically the machine's public, local, or loopback address. Or if you want to listen on everything, 0.0.0.0 or ::

--
Thomas Fjellstrom - [website] - [email] - [Allegro Wiki] - [Allegro TODO]
"If you can't think of a better solution, don't try to make a better solution." -- weapon_S
"The less evidence we have for what we believe is certain, the more violently we defend beliefs against those who don't agree" -- https://twitter.com/neiltyson/status/592870205409353730

Hyena_
Member #8,852
July 2007
avatar

@Thomas Fjellstrom:
Ok I see now what you meant. Actually I don't even know how to bind to a specific address instead of accepting connections from any address. I have read that using SO_REUSEADDR will expose my application to some sort of vulnerabilities but I haven't looked into that.

Thomas Fjellstrom
Member #476
June 2000
avatar

That option means more than one process can access the same port. It's useful for things where you want that, but some rogue process could bind to it and steal some of your data. The data isn't cloned to all processes, instead its split between them. Whoever reads it first, gets it.

--
Thomas Fjellstrom - [website] - [email] - [Allegro Wiki] - [Allegro TODO]
"If you can't think of a better solution, don't try to make a better solution." -- weapon_S
"The less evidence we have for what we believe is certain, the more violently we defend beliefs against those who don't agree" -- https://twitter.com/neiltyson/status/592870205409353730

Go to: