TxMQ News Flash – Critical WAS Outage Restored

A leading full service transaction processing business made TxMQ aware of a node synchronization problem after a weekend install had taken down their website and a file restore did not rectify the situation.
TxMQ appointed SME Bob Becktell to an audio bridge for details on the customer’s corrective steps up to the present point. The customer explained that the code change had failed and they backed out, but the website continued to fail. After restoring the entire WebSphere Application Server related file system on both nodes, the website still failed to load.
Bob identified a certification problem by looking at screen shots that the customer sent to him and determined that this was the first order of business to resolve. Upon further study, he uncovered that the self-signed certificate between WAS servers had expired the previous weekend. The customer was instructed to re-generate the certificates stop the WAS instance and node agent for all the nodes, then restart the Deployment Manager, one node agent, then the WAS instance – in that order.

Bob and the customer watched as the log files of each component started and the website loaded properly. The customer was then instructed to complete the same action on the second node and that node loaded properly as well.

With the website working properly, Bob was able to examine  their WAS Admin Console to make sure things were running normally.  The node agents were communicating with the Deployment Manager, but the node synchronization was still broken.  The customer indicated that they had been having synchronization errors for several months now.

Bob uncovered a Tech Note which matched the symptom and error message, but the resolution didn’t fix the synchronization.  The client was running WAS 6.1.0.37 which is out of support and not the most recent 6.1 fixpack, so Bob recommended a short-term fix would be to update to the latest fixpack- 6.1.0.47.  Since the customer is no longer under IBM support for WAS 6.1 Bob recommended that the customer consider extended WAS v6.1 support or upgrading to WAS 8- either of which we could help resolve their ongoing issues.

Photo Provided by SmartSignBrooklyn