Weblogic Admin server gets LDAP errors then unexpectedly shuts down

we have a new(ish) WLS 9.2 installation

Over the last day or so, after restarting WLS, after a while, the admin server unexpectedly shuts down, preceded in the logs by many (hundreds) occurrences of exceptions like the below. This may be connected with having done an EAR redeployment, but if so the adverse effect seems to persist over a WLS restart.

We have tried restoring LDAP (from a backup months ago which we are pretty sure is OK)

It seems to shut down in a semi-orderly way after being told to do so by the JVM – see the other excerpt from the log below. Goolgling suggests that’s becasue system.exit has been called, but there’s no indication I can see in the log about what has caused that to happen (other than the stream of exceptions)

Any ideas on causes or fixes?

####<Aug 13, 2014 6:50:12 AM BST> <Critical> <EmbeddedLDAP> <csrpth01-omch> <AdminServer_9002> <[ACTIVE] ExecuteThread: '4' for queue: 'weblogic.kernel.Default

(self-tuning)'> <<WLS Kernel>> <> <> <1407909012546> <000000> <java.lang.NullPointerException

        at weblogic.socket.SocketMuxer.deliverExceptionAndCleanup(SocketMuxer.java:715)

        at weblogic.socket.SocketMuxer.deliverEndOfStream(SocketMuxer.java:684)

        at weblogic.ldap.MuxableSocketLDAP$LDAPSocket.close(MuxableSocketLDAP.java:118)

        at com.octetstring.vde.Connection.close(Connection.java:166)

        at com.octetstring.vde.WorkThread.executeWorkQueueItem(WorkThread.java:89)

        at weblogic.ldap.LDAPExecuteRequest.run(LDAPExecuteRequest.java:50)

        at weblogic.work.ServerWorkManagerImpl$WorkAdapterImpl.run(ServerWorkManagerImpl.java:518)

        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)

        at weblogic.work.ExecuteThread.run(ExecuteThread.java:181)

And

####<Aug 13, 2014 7:06:23 AM BST> <Notice> <WebLogicServer> <csrpth01-omch> <AdminServer_9002> <Thread-1> <<WLS Kernel>> <> <> <1407909983789> <BEA-000388> <JVM

called WLS shutdown hook. The server will force shutdown now>

####<Aug 13, 2014 7:06:23 AM BST> <Alert> <WebLogicServer> <csrpth01-omch> <AdminServer_9002> <Thread-1> <<WLS Kernel>> <> <> <1407909983802> <BEA-000396> <Serv

er shutdown has been requested by <WLS Kernel>>

####<Aug 13, 2014 7:06:23 AM BST> <Notice> <WebLogicServer> <csrpth01-omch> <AdminServer_9002> <Thread-1> <<WLS Kernel>> <> <> <1407909983807> <BEA-000365> <Ser

ver state changed to FORCE_SUSPENDING>

####<Aug 13, 2014 7:06:23 AM BST> <Notice> <Server> <csrpth01-omch> <AdminServer_9002> <DynamicSSLListenThread[DefaultSecure]> <<WLS Kernel>> <> <> <14079099838

13> <BEA-002607> <Channel "DefaultSecure" listening on 10.204.1.232:7002 was shutdown.>

####<Aug 13, 2014 7:06:23 AM BST> <Info> <Deployer> <csrpth01-omch> <AdminServer_9002> <Thread-1> <<WLS Kernel>> <> <> <1407909983822> <BEA-149059> <Module hdm-

dashboard-admin.war of application hdm-dashboard-admin is transitioning from STATE_ACTIVE to STATE_ADMIN on server AdminServer_9002.>

####<Aug 13, 2014 7:06:23 AM BST> <Info> <Deployer> <csrpth01-omch> <AdminServer_9002> <Thread-1> <<WLS Kernel>> <> <> <1407909983830> <BEA-149060> <Module hdm-

dashboard-admin.war of application hdm-dashboard-admin successfully transitioned from STATE_ACTIVE to STATE_ADMIN on server AdminServer_9002.>

####<Aug 13, 2014 7:06:23 AM BST> <Notice> <WebLogicServer> <csrpth01-omch> <AdminServer_9002> <Thread-1> <<WLS Kernel>> <> <> <1407909983838> <BEA-000365> <Ser

ver state changed to ADMIN>

####<Aug 13, 2014 7:06:23 AM BST> <Notice> <WebLogicServer> <csrpth01-omch> <AdminServer_9002> <Thread-1> <<WLS Kernel>> <> <> <1407909983839> <BEA-000365> <Ser

ver state changed to FORCE_SHUTTING_DOWN>

####<Aug 13, 2014 7:06:23 AM BST> <Info> <JMX> <csrpth01-omch> <AdminServer_9002> <Thread-1> <<WLS Kernel>> <> <> <1407909983857> <BEA-149513> <JMX Connector Se

rver stopped at service:jmx:iiop://staginghdm.vfl.vodafone:9002/jndi/weblogic.management.mbeanservers.domainruntime .>

####<Aug 13, 2014 7:06:23 AM BST> <Info> <Diagnostics> <csrpth01-omch> <AdminServer_9002> <Thread-1> <<WLS Kernel>> <> <> <1407909983859> <BEA-320002> <The Diag

nostics subsystem is stopping on Server AdminServer_9002.>

####<Aug 13, 2014 7:06:23 AM BST> <Info> <JMX> <csrpth01-omch> <AdminServer_9002> <Thread-1> <<WLS Kernel>> <> <> <1407909983925> <BEA-149513> <JMX Connector Se

rver stopped at service:jmx:iiop://staginghdm.vfl.vodafone:9002/jndi/weblogic.management.mbeanservers.edit .>

####<Aug 13, 2014 7:06:23 AM BST> <Info> <JMX> <csrpth01-omch> <AdminServer_9002> <Thread-1> <<WLS Kernel>> <> <> <1407909983926> <BEA-149513> <JMX Connector Se

rver stopped at service:jmx:iiop://staginghdm.vfl.vodafone:9002/jndi/weblogic.management.mbeanservers.runtime .>

####<Aug 13, 2014 7:06:23 AM BST> <Info> <WebService> <csrpth01-omch> <AdminServer_9002> <Thread-1> <<WLS Kernel>> <> <> <1407909983927> <BEA-220028> <Web Servi

ce reliable agents are suspended.>

####<Aug 13, 2014 7:06:23 AM BST> <Info> <WebService> <csrpth01-omch> <AdminServer_9002> <Thread-1> <<WLS Kernel>> <> <> <1407909983937> <BEA-220029> <Web Servi

ce reliable agents are shut down.>

####<Aug 13, 2014 7:06:23 AM BST> <Info> <SAFService> <csrpth01-omch> <AdminServer_9002> <Thread-1> <<WLS Kernel>> <> <> <1407909983943> <BEA-281004> <SAF Servi

ce has been suspended.>

####<Aug 13, 2014 7:06:23 AM BST> <Info> <SAFService> <csrpth01-omch> <AdminServer_9002> <Thread-1> <<WLS Kernel>> <> <> <1407909983950> <BEA-281005> <SAF Servi

ce has been shut down.>

####<Aug 13, 2014 7:06:23 AM BST> <Info> <Deployer> <csrpth01-omch> <AdminServer_9002> <Thread-1> <<WLS Kernel>> <> <> <1407909983964> <BEA-149059> <Module hdm-

Answer

Attribution
Source : Link , Question Author : The Archetypal Paul , Answer Author : weblogic-tips