Welcome to Exchange Team Blog Sign in | Join | Help

Syndication

This Blog

Fix available to alleviate event ID 9548

No doubt many of you are familiar with the event commonly seen in application logs of Exchange 2000 and 2003 servers, MSExchangeIS event 9548, indicating that the information store came across a disabled user who is missing the msExchMasterAccountSid attribute while processing some various task.  There are many KB articles associated with this event, such as 291151, 326990, 278966, 328880, 316047, and possibly more.  There have also been countless support cases where this design was at least a contributing factor.  Almost every application log from an Exchange 2000 or 2003 server ever seen has likely been littered with 9548 events, to the point where it has become an annoyance event.  For additional information on the event and the related information, I suggest reading this article: http://www.msexchange.org/articles/NoMAS-Tool.html.

 

A CDCR (Critical Design Change Request) was accepted last year to resolve the issue without having to run tools or scripts, and the first version of this fix was released last week.  The KB article is 903158.

 

The problem was that a decision was made during the development of Exchange 2000 that every disabled user account with a mailbox had to have the msExchMasterAccountSid attribute.  This is because in order for a mailbox to function within the store, a SID must be associated with the mailbox.  The logic worked like this:

 

1. If the user account associated with the mailbox is disabled, and has msExchMasterAccountSid, AND msExchMasterAccountSid is not the well-known SELF SID, then msExchMasterAccountSid is the SID that is associated with the mailbox.

 

2. If the user account is enabled, or if it is disabled and has msExchMasterAccountSid set to the well-known SELF SID, then use objectSid.

 

3. If the user account is disabled, and has no msExchMasterAccountSid, OR if we cannot tell if the user is enabled or disabled due to access control on the user object, or if we cannot read the objectSid due to access control, fail the operation and log the 9548 event.

 

In the 3rd case, the vast majority of the 9548 events came from the first part - that is, almost all 9548 events are due to disabled user accounts which are missing msExchMasterAccountSid.  The new logic works as follows:

 

1. If the user account associated with the mailbox is disabled, and has msExchMasterAccountSid, AND msExchMasterAccountSid is not the well-known SELF SID, then msExchMasterAccountSid is the SID that is associated with the mailbox.  (NOTE: No change here)

 

2. If the user account is enabled, or if it is disabled and has msExchMasterAccountSid set to the well-known SELF SID, OR if the user is disabled has does not have msExchMasterAccountSid, then use objectSid.  (Big change here)

 

3. If we cannot tell if the user account is enabled or disabled due to access control on the user object, or if we cannot read the objectSid due to access control, fail the operation and log the 9548 event.

 

So now, the only way you will get the 9548 event is due to a real problem with the user account associated with a mailbox.

 

- Alex Seigler

Published Wednesday, March 22, 2006 8:55 AM by Exchange
Filed Under: , , ,

Comments

 

Scott Bueffel said:

Thank you, thank you, thank you.  I used to have a one-off script to set the attribute for disabled accounts (since we wanted the mailboxes to still function) until NoMAS came along.  Actually, I have always preferred MsAccntSidFixer, and I use it to this day.  Now I will be able to stop doing that once applied.  Thank you so much.

Scott.
March 22, 2006 12:36 PM
 

Aaron Tiensivu's Blog said:

I'm sure a lot of you have seen event 9548 in your event logs from time to time.
If 9548 isn't ringing a bell, maybe this example text will:

Disabled user /o= Organization Name /ou= Administrative Group Name /cn=Recipients/cn= Computer Name does not
March 22, 2006 1:06 PM
 

Deji said:

Well done.

Now, let's examine #2 more closely:

You wrote: If the user account is enabled ...then use objectSid.

No problem with that.

You wrote:
if it is disabled and has msExchMasterAccountSid set to the well-known SELF SID...then use objectSid

My question: Which ObjectSID? The disabled account's SID or SELF?

You wrote:
if the user is disabled has does not have msExchMasterAccountSid...then use objectSid

Again, a question:
Which ObjectSID? The disabled account's SID or SELF?

The last 2 descriptions are not clear (to me) because, in one instance, there is MAS, and in another, there isn't. So, which SID are we using? The way I read KB903158 is that Exchange will always assume that MAS is equal to SELF SID and will ALWAYS use SELF SID even when there is no MAS associated with the disabled account. But your description does not make this clear when you mentioned objectSID.
March 22, 2006 1:31 PM
 

Deji said:

Forget what I wrote, ladies and gentlemen. The effects of the morning coffee just set in :)
March 22, 2006 1:45 PM
 

Alex Seigler said:

For both questions, the answer is the objectSID of the disabled user account.  When the store sees "SELF", it grabs the objectSID and uses that (because that is what SELF means).

So, for disabled user, if MAS is set to SELF, or if it is not set at all, we use the objectSID of the disabled user.  The only time we would use the SID in MAS as-is is when the attribute is populated and not SELF (like an NT4 SID or a SID from a different forest).

-aseigler
March 22, 2006 1:46 PM
 

JamesK said:

I think I am missing something. I got the hotfix from support. When I run the hotfix it says: "You can install this hotfix on Service Pack 1 only".
I didn't see mention of this limitation anywhere.
March 22, 2006 2:48 PM
 

Henrik Walther Blog » Blog Archive » Fix available to alleviate event ID 9548 said:

March 22, 2006 2:50 PM
 

Kevin Bingham said:

Beautiful, Andy.  The KBarticle indicates a pre-SP2 version, though, with a February date.  Will there be a post-SP2 version?
March 22, 2006 2:55 PM
 

Kevin Bingham said:

That should read "Alex", of course...
March 22, 2006 2:57 PM
 

aseigler said:

Correct, the build available right now is for SP1 only.  The fix for SP2 servers will be out very shortly.

-aseigler
March 22, 2006 2:57 PM
 

(e)Mail Insecurity said:

March 22, 2006 11:29 PM
 

(e)Mail Insecurity said:

March 22, 2006 11:31 PM
 

Kevin said:

Thankyou.
This has been a major annoyance for us when running daily backup jobs, as backup exec always complains when it finds a mailbox without the SELF permission thats associated to a disabled user. even just one disabled user and the backup job reports as having failed. Thankyou again for finally fixing this up.
March 23, 2006 5:34 AM
 

/var/log » Exchange hotfix: finally! said:

March 23, 2006 12:43 PM
 

Ilja Summala said:

5 years of waiting...finally. Way things are going AD and Exchange might start to be better together after all.
March 24, 2006 3:10 AM
 

Michel said:

Hmm, have to wait until the PostSP2 version arrives, but since we've waited a long time for this a few more days/weeks don't mind.

I would like a peek on the 'Accepted CDCR' list....
March 24, 2006 6:22 AM
 

Dave Trevallion said:

Quick question, if we apply the pre SP2 fix onto a SP1 build is it likely that we will then have to apply the post SP2 fix after we apply SP2.

We are just planning our SP2 rollout and need to decide to apply the fix now or wait for the post SP2 hotfix version.

Very pleased to see this fix though as it's a big issue to us
March 24, 2006 7:03 AM
 

aseigler said:

Dave Trevallion, that is correct.  I expect the SP2 version to be released any day now.

-aseigler
March 24, 2006 9:07 AM
 

Veno Mouse said:

Hmm. ADModify.Net has always worked well in the past. Maybe it's just not well known.
March 24, 2006 3:23 PM
 

Jewel said:

Total: 6 objects. 5 succeeded, 1 failed.

Copy Exchange Files
Completed
Status: Completed.
Elapsed Time: 00:34:16


Organization Preparation
Completed
Status: Completed.
Elapsed Time: 00:20:15


Bridgehead Server Role
Completed
Status: Completed.
Elapsed Time: 00:02:00


Mailbox Server Role
Completed
Status: Completed.
Elapsed Time: 00:04:16


Client Access Server Role
Completed
Status: Completed.
Elapsed Time: 00:01:32


Unified Messaging Server Role
Failed
Installing product E:\server\en\i386\Setup\ServerRoles\UnifiedMessaging\esenus32.MSI failed. Fatal error during installation. Error code is 1603.
Fatal error during installation

Elapsed Time: 00:14:22

March 24, 2006 6:24 PM
 

Athif said:

How come the fix is released only for SP1?!!
March 26, 2006 9:33 AM
 

Alex Seigler said:

Please read the previous comments, the SP2 fix will be available very soon.

-aseigler
March 26, 2006 10:47 AM
 

Yonkey said:

This wont work on Exchange 2000 SP3 correct?
March 27, 2006 2:43 AM
 

aseigler said:

Yonkey:

Correct, there are no plans at this time to release a fix for Exchange 2000.

-aseigler
March 27, 2006 9:55 AM
 

Matt said:

Alex,
PSS needs to get it's act together then. They've just told me four times that the fix is included in SP2 (including after talking to an "engineer")
How are we meant to find out that the SP2 patch is still on its way when PSS don't know and neither the KB article nor this blog entry say so?
March 27, 2006 10:54 PM
 

Dan Sheehan said:

Well done Alex, you are as usual an exceptional coder!
And I suspected Mr.NOMAS wrote this hot fix when I saw the KB article independant of this post. :)

Please keep the good mods coming!
Dan.
March 28, 2006 9:14 AM
 

Alan said:

Okay... what I don't understand is. How am I supposed to get this Hotfix? Why isn't it just "available" like every other hotfix Microsoft has ever released. Am I supposed to pay $250 to call PSS so I can get this one hotfix... that's rediculous!! Can't someone just send it to me, I have Exchange 2003 SP1.
March 31, 2006 11:34 AM
 

Exchange said:

Alan,

if you call us and ask for the hotfix up front - you will not be charged for the call.

There is actually a pretty good reason for you having to call in for this: this is a hotfix and as such, it did not go through as much testing as rollups or service packs do. So - by calling in, you give us your contact information. If there is a regression or a problem found in the hotfix later, we have a way to contact you and tell you about it and what to do. So - I'd suggest you call in. :)
March 31, 2006 12:31 PM
 

subject: exchange said:

Today this list is longer, since I hadn't the time to post it last week. Sorry...

Exchange on NAS:...
April 2, 2006 9:05 AM
 

Shannon said:

PSS do seem to think this fix is *included* in SP2. Apparently, the keyword "kbexchange2003presp2fix" in the KB article means to them that "this fix is included in SP2".

Could the KB be updated to include a definitive statement about SP2 support?

I await the SP2 version with bated breath!
April 3, 2006 9:13 PM
 

White Ninja said:

BEWARE of hotfix 903158 if you are using BlackBerry handheld devices in your environment.

I applied 903158 on March 24th and it broke the ability for users to send email from their handheld devices -- everything else seemed to work fine; running E3SP1 and BES 4.0.3.11.

KB 912918 did not fix the problem, had RIM and Rogers -- Service Provider on the horn for 8 hours attempting to resolve. End result, removed the hotfix.

RIM suggested upgrading to 4.0.4 or 4.1; though could not confirm if this would solve the problem.

Still waiting to hear back from RIM SE's for a workaround or fix.

BES = POS
April 3, 2006 9:23 PM
 

Alex Seigler said:

The fix has now been released for SP2.  The fix is KB916783.  The article is not ready, but the content will be identical to 903158.

Thanks,

-aseigler
April 4, 2006 9:09 AM
 

Jad said:

Thanks for the update. Any comments on the message by White Ninja for BES compatibility ?

Thanks !
April 5, 2006 5:27 PM
 

aseigler said:

I heard of one other instance of BES issues, and it was resolved with 912918.  This fix definitely falls into the category in the "Cause" section of 912918.

It doesn't explicitly say so in 912918, but I believe that in order for the changes it recommends to take affect immediately, you must restart the store.  Otherwise, you must wait for the cache to flush, similar to what is discussed in 179065 and various other articles.

-aseigler
April 5, 2006 5:43 PM
 

jdonaldson said:

Installed the SP2 version with no problem for my Blackberry users.  I had previously gone through the steps in 912918 and am running BES 4.0.4.

-jdonaldson
April 11, 2006 11:46 AM
 

BGT said:

Where can you get the Post SP2 fix?  I have tried to download it from premier support but it states that it is not currently available.
April 12, 2006 3:35 PM
 

aseigler said:

BGT:  You have to contact support and request KB916783.  It is not available for general download.

-aseigler
April 12, 2006 4:02 PM
 

OliverM said:

Hi there,

Hi you guys at M$ - just so you know this patch breaks Blackberry Enterprise Server...

I'm guessing you need to finetune it some more or Blackberry have some work to do.

Oliver
April 14, 2006 3:15 PM
 

Exchange said:

April 14, 2006 3:36 PM
 

vwebb said:

attempts to contact PSS for 916783 have failed spectacularly. is there a secret password i can use to get the hotfix? thanks!
April 27, 2006 2:55 PM
 

Exchange said:

vwebb,

What happened? There is no secret password needed for you to get this patch :). When you call in, just be clear that all you need was a hotfix. I am not sure what happened but maybe try again?
April 27, 2006 5:31 PM
 

vwebb said:

called into PSS support noted i needed a hotfix for exchange 03, gave the article number. the liasons(?) were adamant on a few different occasions the aritcle couldn't be found and without an article there is no hotfix. even tried sighting this blog but no one would bite.
April 28, 2006 2:19 PM
 

Oldeman said:

When I installed the hotfix (E2003 SP1, Bes 3.6), Blackberry users could no longer send.  KB912918 doesn't help because the BES Administrator already had Send As permission. I removed BES Admin and added using the KB article, but there was no change.
May 8, 2006 6:49 PM
 

ff said:

Hi Oldeman,
Ensure that the BES admin account does not have a deny on send-as.  Also where is do you see that he has the send-as permission? Examine one of your users and look at the security tab and see if he has the send-as permission.  Also make sure it is set to apply to user objects.

ff
May 10, 2006 1:08 AM
 

Paradigm said:

Yup i installed it on my 2003 SP2 server and it did indeed
break my BES server. I called MS and told them about it,
and they said "it shouldnt have", to which i replied "oh but it
did" lol. So am i to understand that upgrading BES will fix
the problem ? m currently on 4.0.3
May 11, 2006 9:04 PM
 

Shannon said:

I think this is a place to ask - how does 916783 relate to MS06-029/912442?  06-029 also updates store.exe, but with a version number lower than 916783?  It *seems* that 916783 includes MS06-029 as it also has a much later build date/time (despite MS06-029 being released months after 916783.)  Thanks hugely for this patch!
June 14, 2006 10:33 PM
 

Robert said:

I have confirmed that MS06-029/912442 breaks BES. We're on 3.6 still, but I imagine it would break any version. Same issue as mentioned in 912918. Have others run that script with success. I ran it and the output was daunting. We have 16,000 mbxs. I'm just not sure where to start in prepping our environment for the post 7650.23 store.exe world. Seems like there should be a utility put out that just "does it". Is this script it? I haven't studied it in depth yet.
June 15, 2006 1:00 PM
 

Jesper Bernle - Exchange of Passion said:

(Egentligen skulle jag bara posta om att jag hittat NoMAS-verktyget för publik nerladdning, men det känns...
May 6, 2007 7:46 AM
New Comments to this post are disabled

News

This is provided "AS IS" with no warranties, and confers no rights. Use of included script samples are subject to the terms specified in the Terms of Use.

New! Would you like to suggest a topic for the Exchange team to blog about? Send suggestions to us.

Poll: