This is the third post in a series of articles that will describe what’s really inside NTDS.dit and how Active Directory works on the database layer, the past two articles has been about:

 

Support for onelevel searches

Given the knowledge from the last article in the series on how objects are referred to each other in terms of a parent – child relation (DNT and PDNT) it becomes obvious that it is easy to get all direct-descendent/child objects of a given object by searching for all object’s (rows) that have a specific PDNT (where objects.PDNT == parent.DNT) this can efficiently be archived by using the “PDNT_index”

Table 1: PDNT_Index

PDNT_Index

    PDNT_index

        Grbit: IndexUnique, IndexIgnoreNull

        CultureInfo: en-US

        CompareOptions: IgnoreCase, IgnoreNonSpace, IgnoreKanaType, IgnoreWidth

            PDNT_col

                Coltyp: Long

                IsAscending: True

                IsASCII: False

            Name [1.1]

                Coltyp: LongText

                IsAscending: True

                IsASCII: False

[1.1]: “Name” represent the RDN attribute, it’s not stored/named as described in the illustration above, it’s rather stored as “ATTm589825″ I choose to represent it as “Name” for simplifying the understanding

Introducing the Ancestors_col and support for subtree searches

Support for subtree searching requires the implementation of another column at the DBLayer, the “Ancestors_col”; the other columns are we already familiar with as of the last post in the series

Table 2: datatable – Simplified for hierarchy representation 2

Name

ESE Data Type

ESE Column Options (grbit)

Description

DNT_col JET_coltypLong JET_bitColumnFixed,
JET_bitColumnAutoincrement
Every object/phantom within the “datatable” contains a unique DNT value.

ESE enforces uniqueness by declaring DNT to be an ESE auto-incrementing column (JET_bitColumnAutoincrement.)

DNT is the primary key of the “datatable”, so objects are clustered in storage by DNT, and access to an object by DNT is more efficient than access via any other column/attribute. Since new objects are created in ascending DNT order, the primary key organization does not slow down the creation of new objects.

PDNT_col JET_coltypLong JET_bitColumnFixed The PDNT column holds the DNT of the parent of an object.

The tree structure of objects is not represented by pointers from parent to child, as you might expect given how the tree is normally browsed, but by a pointer in each child object/row to its parent

RDNTyp_col JET_coltypLong JET_bitColumnFixed The RDNTyp_col holds the attributeID to the attribute being used as RDN, typically: cn (Common-Name), ou (Organizational-Unit), dc (Domain-Component), o (Organization)
Ancestors_col JET_coltypBinary JET_bitColumnTagged The Ancestors_col holds the DN path [2.1] (every DNT from the root in the hierarchy to the objects DNT) in a binary blob. This always efficient subtree searches by searching the Ancestors_col with a prefix of DNTs to the root:ed object [2.2]


[2.1]: The first object/row within the “datatable” always has a NULL ancestory_col value.
[2.2] See “Table 4: Ancestors_index”

Let’s apply “Table 2″ to a theoretical sample:

[3.1]: “Name” represent the RDN attribute, it’s not stored/named as described in the illustration above, it’s rather stored as “ATTm589825″ I choose to represent it as “Name” for simplifying the understanding

If we wanted to do an subtree search with the “Windows Development” organizational unit as the search base – we would like to use an index over the “ancestors_col” [4.1] and set the prefix to be “2,1787,1788,1789,1790,5520,5521*” that would return a list of all objects (rows) that are subordinated the “Windows Development” organizational unit (e.g. all children).

Table 4: Ancestors_index

PDNT_Index

    Ancestors_index

        Grbit: IndexIgnoreNull

        CultureInfo: en-US

        CompareOptions: IgnoreCase, IgnoreNonSpace, IgnoreKanaType, IgnoreWidth

            Ancestors_col

                Coltyp: LongBinary

                IsAscending: True

                IsASCII: False

The ancestors_col and the SDProp (Security Descriptor Propagation Demon) – How are they related?

The “ancestors_col” complicates an object move (Within the same NC/Database, We leave cross-NC/database moves outside this article for simplify understanding)

Given the knowledge from the last article in the series on how objects are referred to each other in terms of a parent – child relation (DNT and PDNT) – It seems easy to implement an object move by simply change the PDNT to the DNT of the new designated parent (e.g. give “Christoffer Andersson” a value of “5521″ in his “PDNT_col” and the object is now subordinated the organizational unit “Windows Development” instead of “Users” in the above sample)

After the operation above, the “Ancestors_col” wouldn’t be accurate, so the Ancestors_col needs “fixup”, (e.g. it needs to be adjusted to the new path “2,1787,1788,1789,1790,5520,5521,,5524″ – “5522″ has to be removed as we moved the object “Christoffer Andersson” from the “Users” organizational unit to the “Windows Development” organizational unit) This might not seem to be an issue at the first glance, but imagine moving a large subtree – the operation wouldn’t fit into an single atomic transaction, therefore ancestry fixup is performed in the background by the SDProp (Security Descriptor Propagation Demon) [4.1]

The most experienced Active Directory administrators at least have heard of “SDProp” that is a short for the Security Propagation Demon and some know that it’s responsible for handling propagation of ACE inheritance, very few probably know that it is in addition also responsible to maintain the acenstors_col at the DBLayer.

Each DSA/DC runs the SDProp (Security Descriptor Propagation Demon) as a background task (TQ_TASK). By default, this task is triggered by the following conditions:

  • Any modification (originating or replicated) of the nTSecurityDescriptor attribute of any object (Except for those modifications done by the SDProp demon)
    • This requires that any new/modified inheritable ACEs are propagated to all descendant objects and that any removed inheritable ACEs are removed from all descendent objects, ACE inheritance is not replicated and is being applied by the SDProp using this process on all DSAs/DCs.
  • Any modification of the “PDNT_col” (object is being moved in the directory) of an object that results in the object having a different parent (Except for those cases where the new parent is a Deleted Objects container)
    • This requires adjustments of the values (DNTs) stored in the “ancestors_col” as the object/row has been moved.
    • This requires that all inheritable ACEs present on the new parent object (and all its ancestors from the top) are being propagated down to the object, and that inheritable ACEs originating from the previous parent (and all its ancestors from the top) has to be removed.


      Table 5: Security Descriptor Propagation Demon – Responsibilities Summary

Name

Description

ACE Inheritance Responsible for propagation of inheritable ACEs, if inheritable ACEs are being added or removed from an object’s SD (Security Descriptor) – SDProp is responsible for propagating those ACEs to all descendant / child objects of the object where the ACEs where added. [4.2]
Ancestors_col fixup If any modification to an object’s parent “PDNT_col” occur (Simply an object move) – SDProp is responsible to adjust the “ancestors_col” to match the full path of DNTs from the top down to the object with respect to its new parent.


[4.1] Since this work is performed in the background and SDProp (that runs on single thread doesn’t implement a limit of how many transactions that is used to perform an operation) there may be a period of time when Active Directory returns inconsistent query results with the tree structure, and inheritable ACEs won’t be accurate.

[4.2]: A common misunderstanding is that the SDProp is responsible for maintaining protection for object’s that is protected by the AdminSDHolder, That’s incorrect. The AdminSDHolder runs in its own background task (TQ_TASK) every 60 minutes on the PDC (if a protected object’s security descriptor mismatches from the once at the adminSDHolder object, inheritance is turned off and the security descriptor is over written with the one at the adminSDHolder object) that makes the AdminSDHolder background task to trigger off SDProp .

You can read more about the SDProp – Security Descriptor Propagation Demon here:
http://msdn.microsoft.com/en-us/library/dd350247(v=prot.13).aspx

Trigger the SDProp from manually from the outside.

It’s possible to use an operational attribute to trigger the SDProp demon to run from the “outside” (e.g be performed by an administrator) – How ever on a fully functional DSA/DC there is no need to invoke the SDProp manually, the SDProp can be trigged globally or for a specific object /row identified by its DNT, The requester must have the “Recalculate-Security-Inheritance” control access right on the nTDSDSA object for the DSA/DC

The following shows an LDIF sample that performs this operation on the entire DIT.

dn:
changetype: modify
add: fixupInheritance
fixupInheritance: 1

The following shows an LDIF sample that performs this operation on a specific object /row

dn:
changetype: modify
add: fixupInheritance
fixupInheritance: dnt:5524

You can read more about the “fixupInheritance” operational attribute here: (it’s actually well documented now days)
http://msdn.microsoft.com/en-us/library/cc223299(v=prot.13).aspx

I think that’s all regarding the “ansectors_col” – I might have missed something.

  1. db student says:

    How does JetGetTableColumnInfo resolve the column id?
    And how does JetMove and JetRetriveColumn work under the hood?

    thanks for any feedback and keep up the good work!

    • Christoffer Andersson says:

      JetGetTableColumn generates a temporary table that can be used to do the mapping, JetMove moves the cursor to a specific position in a table.