I will be delivering my “Auditing in Forefront Identity Manager” talk at TEC Europe later this month in Berlin. It’s largely the same talk I gave at TEC in Las Vegas this past March, but updated for the upcoming RC1 build of FIM 2010 (which sadly won’t be available for TEC Europe, but should be available not long after). In the process of updating my session, I exchanged (Hah! Get it?) some emails with Nima Ganjeh, Program Manager on the FIM product team and learned that there have been some substantial changes in the way FIM manages and exposes it’s object data.

Auditing FIM primarily involves pulling data from the FIM SOAP Enumeration endpoint, using a modified version of the WS-Enumerate protocol. In the RC0 build, FIM implemented the Infological data model [Langefors, 1973] in the object store, meaning that all relationships (attribute-value, object-attribute, etc.) were stored, indexed by time, and never deleted. Because data was never deleted or overwritten, it was possible to retrieve objects from the RC0 object store as it existed at any point in time. So if you wanted to know what the membership of the Administrators group was on March 30th last year, you could simply query for that object using the builtin XPath filter syntax function attime(), and voila!, you would see that group object as it existed at that point in time. This was in my mind one of the most significant benefits of FIM 2010. It provided what was in effect a builtin, automatic snapshot facility that answered the auditor’s question of “who was a member of this group back when the accounting data was compromised?”

Beyond the very cool data model, RC0 also maintained the transaction history for all object update operations by creating Request objects in the store. The Request objects, in addition to specifying the target object and operation, including the initiator of the update and the update parameters themselves. So not only could you look at the state of an object at any point in time, you could easily get a time-sequenced list of all of the changes made to that object, including what was changed, and by whom. This answered the auditor’s question of “who added Bob to this group back when our accounting data was compromised?”

But maintaining a complete history of the FIM object store had a price, or actually two prices: a rapidly growing database and insufficient performance. While RC0 would perform reasonably well on smaller datasets, it just wasn’t cutting it for larger environments. The FIM team decided that the performance impact of maintaining this historical data just wasn’t worth it, and in RC1, they have completely revamped the implementation of the object store. And sadly, RC1 has completely dropped the Infological data model and maintains no object history at all beyond the Request objects. The attime(), alltime(), and betweentime() XPath filter functions are gone, as is the ability to retrieve an object’s prior state through the web service. This is a big disappointment in my mind, but I can’t fault the FIM team for it… historical data or unworkably slow? It’s not a hard choice.

So how do you solve the audit problem in FIM 2010 RC1? I’ll discuss that in my talk at TEC in a couple of weeks and in this blog, but suffice it to say its not not as elegant as it used to be.

Nima will publish the specifics of the changes in the RC1 query facility in his blog soon, so keep an eye on it.

1. Langefors B. (1973) Theoretical Analysis of Information Systems

Have you ever wanted to define the feature list for Active Directory? TEC Europe 2009 might be your best chance to do just that. Robert deLuca (Microsoft Senior PM for Identity and Access) and Dean Wells (Senior PM for Directory Services) will host a “Customer-Focused Design” session at TEC Europe September 14 in Berlin.

Microsoft has developed a very structured process for gathering products requirements. I’ve been through the process a couple of times in Redmond (and I’ve used something similar for my own products), and it is pretty effective for figuring out what requirements you want to focus your product team’s efforts on.

The process starts out with a bottom-up approach, where the people in the room just write down all the requirements they think are important on Post-It notes. The key thing here is volume, and for people to focus on the problem-space not the solution-space. Developers and IT Pros are predisposed to solving problems and invariably come up with ideas like “move the user account control bits to separate attributes” or “prune and graft the directory tree”. These are solutions to problems, not problems per se. The problems should be more like “I can’t control the access rights on the individual user account control bits in Active Directory”, or even better “I need to separately delegate the ability to set the No Password Required flag and the User Account Disabled flag”. See the difference? The first is a possible solution to a requirement the IT Pro has in the back of their mind. The second is the actual requirement. It’s really important to get everyone writing down problem-space statements, not solution-space statements. Otherwise the next step doesn’t work.

Once the moderator makes sure all the problems are legible and understood (IT Pros are not known for their high-quality handwriting), the second step involves sticking the Post-It notes on a wall and organizing them into affinity groups, i.e. grouping the requirements in a way that “seems to make sense”. This is a group activity, and can be really fun to participate in (and to observe!) Basically, you start moving the Post-Its around and inventing categories that seem to fit a group of them. The goal is to organize the requirements (perhaps more than a hundred of them) into a manageable number of higher-level topics. It’s always surprising how, given the vague instructions, people seem to instinctively organize things in pretty much the same way. Sometimes it takes a few attempts to get the categories right, but I’ve never seen a real argument in this process.

Now the drama begins. In the third step you need refine the categories you developed in the preceding step into full-fledged problem statements that reflect all of the requirements in that category. So everything that you grouped under “Schema” is now covered by something like “I need to manage the schema without special consideration like review committees and or replication latency and be able to provide a complete audit report of all schema changes.” Honing the requirements statements takes a lot of back-and-forth discussion between the participants to make sure the problem statement reflects all of the requirements, and that it is reasonably coherent. Sometimes you have to go back to the board and re-categorize some of the requirements, and occasionally someone will decide their particular requirement just isn’t worth the effort.

The process I’ve described so far is one I’ve used for years in developing product requirements, and was taught to me by my friend Jared Spool at User Interface Engineering about 20 years ago, and it works pretty well. One of the problems with the process is that you end up with way more problems to address than you have time or resources. How do you prioritize them? Microsoft has added two steps where the participants rank how important each problem statement is to them, and how well or how poorly each problem statement is currently addressed by the shipping product. The first ranking gives them an idea of which problems they should focus on, and the second ranking gives an idea of how much effort they should expend on addressing the problem.

Normally this sort of thing can take most of an entire day, but for TEC, Robert and Dean are developing a skinnied-down process that should provide some meaningful results in the hour or so we have available. I think they are also going to use some of the birds-of-a-feather time to work on the problem statements. In any case, it will be a lot of fun.

So if you’ve ever found yourself completely frustrated because Microsoft didn’t fix your specific issues in Active Directory, come to TEC Europe 2009 and make sure your problems are heard and make it into the queue for the next version of Active Directory.

I’ve been getting lots of last minute changes from speakers for TEC this year, many more than I recall from prior conferences. For instance, we just yesterday finalized the four sessions from the Active Directory product team.

• Nathan Muggli will handle the keynote duties for the AD team (we’re experimenting with a new combined keynote format where members from each of the product teams will spend 20 minutes or so discussing their product area roadmaps).

• Dennis Angeline is a lead PM on the AD team, and he will be covering “What’s New in Active Directory for Windows Server 2008 R2”.

• James Dean (James McColl and Dean Wells) will discuss the new AD PowerShell provider that is part of 2008 R2.

• TEC newbie Ivan Lam will demo and discuss the new Active Directory Administration Center.

So it looks like a pretty good lineup from the product team this year. And note: there are NO RODC sessions from the product team this year!

Find out more at http://www.tec2009.com

No, I'm not kidding...

http://www.facebook.com/friends/?id=1339830232&flid=0&view=everyone&q=&nt=0&nk=0&s=0&st=0#/photo.php?pid=37230413&id=122604833

We revamped the keynote structure for TEC 09 Directory and Identity, based partially on feedback from last March, and partially just because I wanted to change things up a bit. So instead of a single overarching, 90 minute keynote on Monday morning, we're going to have each of the relevant product teams do a shorter keynote that outlines their roadmap and how they each play into the Microsoft's overall identity strategy. It will take more orchestration, but I think it will make for a more interesting keynote on Monday morning. Stuart "The Hardest Working Man in Show Business" Kwan will kick things off and cover the federation-related products and Alex Weinert will have the honors for the ILM team. Uday Hegde, formerly the DS Group Program Manager has moved to a different group, and Scott Robinson hasn't selected a new GPM yet, so the speaker for the DS part of the keynote is up in the air. And we haven't nailed down anyone from the RMS side of the house yet either, so I've got some work to do next time I go up to Redmond. Any suggestions?

For TEC 09 Exchange we're sticking with the normal keynote format. I'm really excited that Konstantin Ryvkin, Senior Architect from the Business Collaboration Services Group will open things for the Exchange conference. Konstantin is the architect behind Microsoft hosted Exchange services, so you know that he understands both large-scale and small-scale Exchange deployments inside and out, as well as how to drive the costs out of Exchange operations.

It should be an excellent conference next March; I'm really looking forward to it.

I've never been much a LinkedIn maven. I suspect I'm like most people and I keep adding people to my LinkedIn network, but I never really use it for anything. But I got an InMail from Eric Aarts today that made me think I need to start poking my nose into LinkedIn (and Plaxo for that matter) a little more. It turns out you can post events on LinkedIn, and people can indicate whether they are going or not. I suppose you can then use that information to set up meetings and so on.

Eric pointed out that TEC 09 is on LinkedIn now. You can check it out here.

Pam Dingle made some nice comments about TEC in this blog entry.

Interestingly, there will be several shorter keynotes for TEC 09 Day 1, one from each of the product teams. I think this will work out better than the single 90 minute keynote we have done in the past.