Skip to main content

WebLogic's SerializedSystemIni.dat

In one of my earlier jobs, I once encountered a peculiar problem with WebLogic's SerializedSystemIni.dat file. Upon restarting a WebLogic admin or managed server, we would encounter an exception like:

<1/06/2009 02:23:35 PM EST> <Warning> <Security> <BEA-090066> <Problem handling boot identity. The following exception was generated: weblogic.security.internal.SerializedSystemIniException: [Security:090208]Corrupt SerializedSystemIni.dat>
<1/06/2009 02:23:35 PM EST> <Info> <Security> <BEA-090065> <Getting boot identity from user.>
Enter username to boot WebLogic server:


We quickly noticed that the SerializedSystemIni.dat file was 0 bytes in size. As any good admin would do, we blamed the developers for corrupting this file. We then restored the file from backup, and everything went smoothly... for a while.

Unfortunately for us, the problem occurred again sometime later in a restricted test environment. This time we knew it was a WebLogic quirk of some sort.

SerializedSystemIni.dat Explained

If you've ever looked through config.xml, you may have noticed password values that look like "{3DES}bOH6MEd8S82sxg2Nk=". This is how WebLogic stores passwords encrypted with the "Triple DES" algorithm (see here for a description of 3DES).

When WebLogic stores or retrieves one of these passwords, it uses a hash key which is stored in the SerializedSystemIni.dat file.

That's right, the key to all your encrypted admin and DB passwords is right there on your file system. Read access is all that is needed to compromise a WebLogic domain. Perhaps worse, you could be making DB passwords available.

Take a look at your WebLogic servers... who has read access to your WebLogic files?

If you're wondering where this file is located, its in the domain directory for WebLogic 8 (or earlier) and in the 'security' directory for WebLogic 9 or later.

Solving the 0-Byte SerializedSystemIni.dat Problem

It wasn't until I understood the purpose of this file that I was able to solve the 0-byte problem.

Some time after we noticed SerializedSystemIni.dat was being corrupted, we realized the problem was occurring after the server had run out of disk space (sometimes several weeks later and the disk was no longer full).

After a lot of googling and well-intended but limited help from BEA, we discovered that SerializedSystemIni.dat is periodically re-written by the WebLogic Admin server. If the server is unable to write the contents of the file (disk is full, etc), WebLogic will lose the hash key and will no longer be able to decrypt any passwords. I.e. if you're the admin, you'd better have backups or personal indemnity insurance.

So if you are wondering why your WebLogic servers won't start, and SerializedSystemIni.dat is 0 bytes in size, you now know why.

The solution? Don't let your server's disk fill up.

Comments

  1. Hi Casey

    I experienced a similar problem in my WebLogic domain and came across your tips in this post. However, my SerializedSystemIni.dat was not zero-byte, but it was corrupted. So, I investigated further, raised an SR with Oracle, obtained a patch and documented my experience with this issue in my blog (http://itsupport.mrkips.com/2009/06/24/weblogic-serializedsystemini-dat), which I launched only a couple of days ago. Thank you for your tips in this post.

    ReplyDelete
  2. Recovery password
    Fast, high-quality recovery of your forgotten password. Inexpensive!
    communicate http://recoverywallet.com/

    ReplyDelete

Post a Comment

Popular posts from this blog

Connection refused: No available router to destination

This is a simple symptom-cause-solution blog entry only. I hope these blogs will help fellow administrators.
Symptom

The following exception occurs in WebLogic server logs. Most likely to occur during WebLogic server start-up, but similar exceptions may occur at other times.
java.net.ConnectException: t3://myserver:8000: Destination unreachable; nested exception is: java.net.ConnectException: Connection refused: connect; No available router to destination] at weblogic.jndi.internal.ExceptionTranslator.toNamingException(ExceptionTranslator.java:49) at weblogic.jndi.WLInitialContextFactoryDelegate.toNamingException(WLInitialContextFactoryDelegate.java:773) at weblogic.jndi.WLInitialContextFactoryDelegate.getInitialContext(WLInitialContextFactoryDelegate.java:363) at weblogic.jndi.Environment.getContext(Environment.java:307) at weblogic.jndi.Environment.getContext(Environment.java:277)
Cause
This message (Connection refused: connect; No available router to destination) is a kind of "catch…

BAD_CERTIFICATE - A corrupt or unuseable certificate...

This is a simple symptom-cause-solution blog entry only. I hope these blogs will help fellow administrators.
Symptom
In wls_utc, when trying to test a webservice using SSL, the following error message is received: javax.net.ssl.SSLKeyException: FATAL Alert:BAD_CERTIFICATE - A corrupt or unuseable certificate was received.
If SSL debugging is enabled, the following error also appears in the logs: ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)' <1254822672320>>
verification failed because RSA key public exponent [3] is too small
Cause
The certificate encryption is of a weaker strength than expected by newer versions of Java.
Solution
Add the flag "-Dweblogic.security.SSL.allowSmallRSAExponent=true" to the server startup parameters.
References


None.

WebLogic Admin Console

WebLogic Admin Console
The WebLogic Admin Console is a web-based, user interface used to configure and control a set of WebLogic servers or clusters (i.e. a "domain"). In any logical group of WebLogic servers there must exist one admin server, which hosts the WebLogic Admin Console application and manages the associated configuration files.
WebLogic Administrators will use the Administration Console for a number of tasks, including:
Starting and stopping WebLogic servers or entire clusters.Configuring server parameters, security, database connections and deployed applications.Viewing server status, health and metrics. Note: It is not strictly necessary to use the Weblogic Admin Console to perform these tasks, as they can be scripted using WLST (the WebLogic Scripting Tool).
Accessing the Admin Console
WebLogic Admin Console Url: http://hostname:port/console.
To access the WebLogic Administration Console, assuming the admin server has been started, goto the above url. Where hostname…