Skip to main content

WebLogic's SerializedSystemIni.dat

In one of my earlier jobs, I once encountered a peculiar problem with WebLogic's SerializedSystemIni.dat file. Upon restarting a WebLogic admin or managed server, we would encounter an exception like:

<1/06/2009 02:23:35 PM EST> <Warning> <Security> <BEA-090066> <Problem handling boot identity. The following exception was generated: weblogic.security.internal.SerializedSystemIniException: [Security:090208]Corrupt SerializedSystemIni.dat>
<1/06/2009 02:23:35 PM EST> <Info> <Security> <BEA-090065> <Getting boot identity from user.>
Enter username to boot WebLogic server:


We quickly noticed that the SerializedSystemIni.dat file was 0 bytes in size. As any good admin would do, we blamed the developers for corrupting this file. We then restored the file from backup, and everything went smoothly... for a while.

Unfortunately for us, the problem occurred again sometime later in a restricted test environment. This time we knew it was a WebLogic quirk of some sort.

SerializedSystemIni.dat Explained

If you've ever looked through config.xml, you may have noticed password values that look like "{3DES}bOH6MEd8S82sxg2Nk=". This is how WebLogic stores passwords encrypted with the "Triple DES" algorithm (see here for a description of 3DES).

When WebLogic stores or retrieves one of these passwords, it uses a hash key which is stored in the SerializedSystemIni.dat file.

That's right, the key to all your encrypted admin and DB passwords is right there on your file system. Read access is all that is needed to compromise a WebLogic domain. Perhaps worse, you could be making DB passwords available.

Take a look at your WebLogic servers... who has read access to your WebLogic files?

If you're wondering where this file is located, its in the domain directory for WebLogic 8 (or earlier) and in the 'security' directory for WebLogic 9 or later.

Solving the 0-Byte SerializedSystemIni.dat Problem

It wasn't until I understood the purpose of this file that I was able to solve the 0-byte problem.

Some time after we noticed SerializedSystemIni.dat was being corrupted, we realized the problem was occurring after the server had run out of disk space (sometimes several weeks later and the disk was no longer full).

After a lot of googling and well-intended but limited help from BEA, we discovered that SerializedSystemIni.dat is periodically re-written by the WebLogic Admin server. If the server is unable to write the contents of the file (disk is full, etc), WebLogic will lose the hash key and will no longer be able to decrypt any passwords. I.e. if you're the admin, you'd better have backups or personal indemnity insurance.

So if you are wondering why your WebLogic servers won't start, and SerializedSystemIni.dat is 0 bytes in size, you now know why.

The solution? Don't let your server's disk fill up.

Comments

  1. Hi Casey

    I experienced a similar problem in my WebLogic domain and came across your tips in this post. However, my SerializedSystemIni.dat was not zero-byte, but it was corrupted. So, I investigated further, raised an SR with Oracle, obtained a patch and documented my experience with this issue in my blog (http://itsupport.mrkips.com/2009/06/24/weblogic-serializedsystemini-dat), which I launched only a couple of days ago. Thank you for your tips in this post.

    ReplyDelete
  2. Recovery password
    Fast, high-quality recovery of your forgotten password. Inexpensive!
    communicate http://recoverywallet.com/

    ReplyDelete

Post a Comment

Popular posts from this blog

BAD_CERTIFICATE - A corrupt or unuseable certificate...

This is a simple symptom-cause-solution blog entry only. I hope these blogs will help fellow administrators.
Symptom
In wls_utc, when trying to test a webservice using SSL, the following error message is received: javax.net.ssl.SSLKeyException: FATAL Alert:BAD_CERTIFICATE - A corrupt or unuseable certificate was received.
If SSL debugging is enabled, the following error also appears in the logs: ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)' <1254822672320>>
verification failed because RSA key public exponent [3] is too small
Cause
The certificate encryption is of a weaker strength than expected by newer versions of Java.
Solution
Add the flag "-Dweblogic.security.SSL.allowSmallRSAExponent=true" to the server startup parameters.
References


None.

Connection refused: No available router to destination

This is a simple symptom-cause-solution blog entry only. I hope these blogs will help fellow administrators.
Symptom

The following exception occurs in WebLogic server logs. Most likely to occur during WebLogic server start-up, but similar exceptions may occur at other times.
java.net.ConnectException: t3://myserver:8000: Destination unreachable; nested exception is: java.net.ConnectException: Connection refused: connect; No available router to destination] at weblogic.jndi.internal.ExceptionTranslator.toNamingException(ExceptionTranslator.java:49) at weblogic.jndi.WLInitialContextFactoryDelegate.toNamingException(WLInitialContextFactoryDelegate.java:773) at weblogic.jndi.WLInitialContextFactoryDelegate.getInitialContext(WLInitialContextFactoryDelegate.java:363) at weblogic.jndi.Environment.getContext(Environment.java:307) at weblogic.jndi.Environment.getContext(Environment.java:277)
Cause
This message (Connection refused: connect; No available router to destination) is a kind of "catch…

BEASVC.EXE - WebLogic as a Windows Service

I remember the first time I had to work out why WebLogic wouldn't run as a service. It was a frustrating experience. There were no error messages. No Windows error dialog. No console output.

How do you troubleshoot something like this??!!

This short article will show you. For simplicity I'll talk about the node manager, but the same principles apply for running an admin or managed server as a service.

First Steps

First, you still have your server logs. Sure the console output is better, but it's a starting point. Check this log for errors and especially take note of the start up variables such as PATH and CLASSPATH. If the server log isn't being created, that tells you WebLogic probably isn't even being started. (Check that you have a license file if you haven't already.)

Ok, so the logs were no help. The next step is to look at how the service is trying to start WebLogic.

When you install WebLogic as a service, you're really setting up beasvc.exe as the service.…