Fin_Wait

No process is running on a port; yet i am not able to start a new process be it a nodemanager or any weblogic server on that port. May i know why ?

Starting the weblogic server, nodemanager process or any other process which uses the port gives the following exception.
<Jul 4, 2010 6:19:03 AM BST> <Info> NodeManager@hp027:5556 <Reading trusted CAs from the keystore /opt/bea/jdk142_08/jre/lib/security/cacerts. KeyStore type = jks, Using keystore passphrase = false>
SecureSocketListener: Could not setup context and create a secure socket on hp027:5556 : java.net.BindException: Can't assign requested address.
java.net.BindException: Can't assign requested address
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:331)
at java.net.ServerSocket.bind(ServerSocket.java:324)
at java.net.ServerSocket.<init>(ServerSocket.java:186)
at javax.net.ssl.SSLServerSocket.<init>(DashoA12275)
at javax.net.ssl.impl.SSLServerSocketImpl.<init>(Unknown Source)
at javax.net.ssl.impl.SSLServerSocketFactoryImpl.createServerSocket(Unknown Source)
at weblogic.nodemanager.internal.SecureSocketListener.run(SecureSocketListener.java:75)
at weblogic.nodemanager.internal.GenericListener.startListener(GenericListener.java:16)
at weblogic.nodemanager.NodeManager.startSecureSocketListener(NodeManager.java:461)
at weblogic.nodemanager.NodeManager.init(NodeManager.java:305)
at weblogic.nodemanager.NodeManager.run(NodeManager.java:511)
at weblogic.NodeManager.main(NodeManager.java:31)


Even if you kill all process using the port, no matter how many times, yet you keep getting error below when you start any process that uses the desire port.

Here's the probable reason / cause.

Try this command.

$ netstat -a | grep '5556'
tcp 0 0 hp023.5556 hp024.62396 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.65171 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.59435 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.62333 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.59374 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.61982 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.62583 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.61996 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.62315 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.62124 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.62108 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.62318 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.62399 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.62117 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.51602 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.51605 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.51609 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.65371 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.64958 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.65174 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.65327 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.65320 FIN_WAIT_2
tcp 0 0 hp023.5556 hp024.65318 FIN_WAIT_2

Well, when we have killed the process, yet the netstat shows the port in use. Hmmm... why so?

Explaination:

FIN_WAIT_2 seems to occur when the server has an active connection with a client and it wants to shut down the TCP connection (probably in response to a normal application layer "exit"). The server sends the client a packet with a "FIN" bit set. At this point, the server is in FIN_WAIT_1 state. The client gets the FIN packet and goes into CLOSE_WAIT state, and sends an acknowledgment packet back to the server. When the server gets that packet, it goes into FIN_WAIT_2 state. From the server's perspective, the connection is now closed, and the server can't send any more data. However, under the TCP protocol, the client needs to shut down also by sending a FIN packet, which the server TCP implementation should ACK. The server should close about two milliseconds later.

The following diagram is taken from RFC 793 (defining the TCP protocol):

     TCP A                                                TCP B

  1.  ESTABLISHED                                          ESTABLISHED

  2.  (Close)
      FIN-WAIT-1  --> <SEQ=100><ACK=300><CTL=FIN,ACK>  --> CLOSE-WAIT

  3.  FIN-WAIT-2  <-- <SEQ=300><ACK=101><CTL=ACK>      <-- CLOSE-WAIT

  4.                                                       (Close)
      TIME-WAIT   <-- <SEQ=300><ACK=101><CTL=FIN,ACK>  <-- LAST-ACK

  5.  TIME-WAIT   --> <SEQ=101><ACK=301><CTL=ACK>      --< CLOSED

  6.  (2 MSL)
      CLOSED

Some clients have faulty TCP implementations, and leave the server hanging in FIN_WAIT_2 state.

Only the UNIX Administrators would know, how to release these ports. Once they let you know if the ports are released. Try starting the process and hopefully it will work.

You will not face the fin_wait issue if you close the process using the product provided stop script. That's why better stop the server using admininistration console or weblogic provided tools to stop the process rather than "Ctrl+C" or "kill -9".