So what is the “Code [1-3]” all about and where is Part 4 of the series that you might expect?
Before I go ahead with Part 4 I thought it would be a good idea to sum up Part 1 to Part 3 with code (So that you know how we figured out all this stuff while we was coding on ESEDump) – Note: This most may be targeted for the developer audience more than the general Active Directory administrator.

Disclaimer: The code samples provided here is code snippets that doesn’t represent any code actual code from the DSA and or any other Microsoft products and technologies, nor those they represent the complete source of ESEDump


ESEHelper – A managed ESE wrapper around the ESE APIs

We decided that we wanted to work with ESE in C# (and when we first started this project EseManaged from codeplex wasn’t around) and even if we could have used it later on – I guess we wanted full control and decided to stick with our own wrapper. So when you see references to “EseHelper” in the code snippets below – you know it’s just a wrapper around: Extensible Storage Engine Native APIs – there is no secrets around this J

JET_RETRIVECOLUMN structure custom methods

We extended the JET_RETRIVECOLUMN structure with some additional methods to retrieve data.

Table 0: JET_RETRIVECOLUMN structure

Code Snippet

// The custom methods in JET_RETRIEVECOLUMN allow us to quickly// interpret each column’s data depending on its data type (string, integer, etc.)

internal
struct
JET_RETRIEVECOLUMN

{

public
int columnid;

public IntPtr pvData; // Pointer to the data block in memory

public
int cbData; // Size of the allocated data block

public
int cbActual; // Size of the actual/used data

public
int grbit;

// Offset to the first byte to be retrieved from a column of type

// JET_coltypLongBinary or JET_coltypLongText

public
int ibLongValue;

// Number of values in a multi-valued column

// Can be used to retrive a specific value

public
int itagSequence;

// The columnid of the tagged, multi-valued, or sparse column

// when all tagged columns are retrieved by passing 0

// as the columnid to JetRetrieveColumn.”

public
int columnidNextTagged;

public
int err;

public
void Initialize(ColumnInfo att)

{

this.Initialize(att.ID, 0);

}

public
void Initialize(ColumnInfo att, int cbData)

{

// Initialize with a data block of cbData size

this.Initialize(att.ID, cbData);

}

public
void Initialize(int columnid)

{

// Initialize with an empty data block

this.Initialize(columnid, 0);

}

public
void Initialize(int columnid, int cbData)

{

// Reset the fields

this.cbActual = 0;

this.cbData = 0;

this.err = 0;

// Make sure to free any previously used memory in this instance

if (this.pvData != IntPtr.Zero)

{

Marshal.FreeHGlobal(this.pvData);

this.pvData = IntPtr.Zero;

}

this.columnid = columnid;

this.itagSequence = 1;

// Allocate a new memory block if necessary (if > 0 bytes requested)

this.cbData = cbData;

if (this.cbData > 0)

this.pvData = Marshal.AllocHGlobal(this.cbData);

}

// Copies the current memory block into a byte array

public
byte[] GetData()

{

byte[] output = new
byte[this.cbActual];

Marshal.Copy(this.pvData, output, 0, output.Length);

return output;

}

// Interpret the inner data as a GUID

public Guid GetGuid()

{

IntPtr pGuid = this.pvData;

byte[] bGuid = new
byte[16];

Marshal.Copy(pGuid, bGuid, 0, bGuid.Length);

return
new Guid(bGuid);

}

// Interpret the inner data as a string (automatically checks ASCII or Unicode encoding)

public
string GetString()

{

byte[] data = this.GetData();

if (IsUnicode(data))

return Encoding.Unicode.GetString(data, 0, data.Length);

else

return Encoding.ASCII.GetString(data, 0, data.Length);

}

// Interpret the inner data as a 32-bit integer

public
int GetInteger()

{

return BitConverter.ToInt32(this.GetData(), 0);

}

// Interpret the inner data as a 64-bit integer

public
long GetLong()

{

return BitConverter.ToInt64(this.GetData(), 0);

}

// Interpret the inner data as a boolean (true/false)

public
bool GetBool()

{

return Marshal.ReadByte(this.pvData) == 1 ? true : false;

}

// Determines if a string in a data block is of Unicode or ASCII encoding

// TODO: International 2-byte characters unsupported?

private
bool IsUnicode(byte[] data)

{

bool isUnicode = false;

// Unicode strings’ data always have an even number of bytes

if (data.Length % 2 == 0)

for (int i = 0; i < data.Length; i += 2)

if (data[i + 1] == ‘\0′)

isUnicode |= true;

return isUnicode;

}

}

// ColumnInfo stores column metadata (ID, type, table owner)

// Used when retrieving JET columns

internal
struct
ColumnInfo

{

public
int ID;

public
string Name;

public
int DataType;

public
string AltName; //added for attribute name

public
int AltId; // added for attribute id

public IntPtr TableId; // added for table support in caching

public ColumnInfo(int id, string name, int type, string altname, int altid, IntPtr tableid)

{

this.ID = id;

this.Name = name;

this.DataType = type;

this.AltName = altname; //added for attribute name

this.AltId = altid;

this.TableId = tableid; //added for table support in caching

}

}

 

Perform Initialization and Attach to NTDS.dit

First thing we had to figure out was how we attached to the database (NTDS.dit) using JetInit, JetBeginSession, JetAttachDatabase and finally calling JetOpenDatabase in addition to those callas we had to set several parameters with JetSetSystemParameter for our usage, e.g there is things that need to be turned off as we attach/open the DB as read-only due to the nature of our application.

Table 1: ESE Initialization

Code Snippet

// E.Check makes sure a JET API call is successful, i.e. JET_errSuccess (0)// If the call fails, we throw an exception/write to the Console

// Initialize ESENT. Setting JetInit will inspect the logfiles to see if the last

// shutdown was clean. If it wasn’t (e.g. the application crashed) recovery will be

// run automatically bringing the database to a consistent state.

err = E.Check(EseHelper.JetSetSystemParameter(ref instance, EseHelper.JET_sesidNil, new
IntPtr(EseHelper.JET_paramDatabasePageSize), new
IntPtr(0×2000), null));

err = E.Check(EseHelper.JetCreateInstance(out instance, “instance”));

// Set up the recovery option (off), the maximum number temporary tables (7) and temporary path

err = E.Check(EseHelper.JetSetSystemParameter(ref instance, EseHelper.JET_sesidNil, new
IntPtr(EseHelper.JET_paramRecovery), IntPtr.Zero, “off”));

err = E.Check(EseHelper.JetSetSystemParameter(ref instance, EseHelper.JET_sesidNil, new
IntPtr(EseHelper.JET_paramEnableOnlineDefrag), IntPtr.Zero, null));

err = E.Check(EseHelper.JetSetSystemParameter(ref instance, EseHelper.JET_sesidNil, new
IntPtr(0xa), IntPtr.Zero, null));

err = E.Check(EseHelper.JetSetSystemParameter(ref instance, EseHelper.JET_sesidNil, new
IntPtr(EseHelper.JET_paramMaxTemporaryTables), new
IntPtr(7), null));

err = E.Check(EseHelper.JetSetSystemParameter(ref instance, EseHelper.JET_sesidNil, new
IntPtr(EseHelper.JET_paramTempPath), IntPtr.Zero, System.IO.Path.GetTempPath()));

// Initialize ESE and begin a session

err = E.Check(EseHelper.JetInit(ref instance));

err = E.Check(EseHelper.JetBeginSession(instance, out sesid, null, null));

// Attach a database

err = E.Check(EseHelper.JetAttachDatabase(sesid, “NTDS.dit”, 1));

err = E.Check(EseHelper.JetOpenDatabase(sesid, “NTDS.dit”, null, out dbid, 1));

 

List the tables inside NTDS.dit

We figured out that by statically opening the “MSysObjects” and positioning over the “RootObjects” index we could enumerate the tables inside the database using the following code snippet.

Table 2: ESE Enumerate tables inside the database

Code Snippet

// Simplified:// Method to obtain a list of tables for a given JET database

private
static
void GetTableNames(IntPtr instance, IntPtr sesid, IntPtr dbid, ref EseErrors err)

{

IntPtr tableid = IntPtr.Zero;

List<string> tables = new List<string>();

// E.Check makes sure a JET API call is successful, i.e. JET_errSuccess (0)

// If the call fails, we throw an exception/write to the Console

err = E.Check(EseHelper.JetOpenTable(sesid, dbid, “MSysObjects”, IntPtr.Zero, 0, 0, out tableid));

// Select the first row in the RootObjects record set

err = E.Check(EseHelper.JetSetCurrentIndex(sesid, tableid, “RootObjects”));

err = E.Check(EseHelper.JetMove(sesid, tableid, EseHelper.JET_MoveFirst, 0));

// Allocate a column array of one element — we only need the name column

EseHelper.JET_RETRIEVECOLUMN[] array = new EseHelper.JET_RETRIEVECOLUMN[1];

// Loop until we reach the end of the record set or an error occurs

while (err == 0)

{

// Allocate 0×41 bytes for this column’s value

// 0×80 is the column ID (table name)

array[0].Initialize(0×80, 0×41);

err = E.Check(EseHelper.JetRetrieveColumns(sesid, tableid, Marshal.UnsafeAddrOfPinnedArrayElement(array, 0), array.Length));

foreach (EseHelper.JET_RETRIEVECOLUMN column in array)

{

if (column.cbData != 0×04)

{

string table = column.GetString();

// Print out the table

Console.WriteLine(table);

}

}

// Select the next record (table info row)

err = E.Check(EseHelper.JetMove(sesid, tableid, EseHelper.JET_MoveNext, 0));

}

// Clean up

err = E.Check(EseHelper.JetCloseTable(sesid, tableid));

Console.WriteLine();

}

 

Retrieving the Ancestors_col

In Part 3, we’re discussing the usage of the “Ancestors_col” column and how it’s used to walk subtrees in the database, the DNTs are stored as bytes within the “Acenstors_col” and are read as in the code snippet below.

Table 3: Ancestors_col

Code Snippet

if (column.err != EseErrors.ColumnNull)
{


string ancestory = null;


char[] ancestor_separator = { ‘,’ };

 


// Walk through every ancestry record in the returned column data


// Construct the ancestry string with the returned DNTs


for (int i = 0; i < column.GetData().Length; i += sizeof(int))

{


int dnt = BitConverter.ToInt32(column.GetData(), i);

ancestory = ancestory + dnt + “,”;

}

 


// Remove any trailing “,” characters

ancestory = ancestory.TrimEnd(ancestor_separator);

output = ancestory;

}

 

Reading an object’s full distinguished name

Note: This is our way to read an objects full distinguished name, given an object’s DNT (Distinguished Name Tag) – But this is not considered safe by the DSA as mentioned in Part 3 as the “Ancestors_col” is being processed by a background task and might not be in-sync all the times. (Safer would be to walk the tree up – by each PDNT until PDNT == 2)

Table 4: Get an objects distinguished name by its DNT

Code Snippet

internal
static
string DBGetDN(IntPtr sesid, IntPtr tableid, int tag, ref
EseErrors err)
{

// 0 (zero) means where already positioned at the obejct.

if (tag != 0)

DBFindDNT(sesid, tableid, tag, ref err);

 

List<string> DN = new
List<string>();

 

EseHelper.JET_RETRIEVECOLUMN[] Ancestors = new
EseHelper.JET_RETRIEVECOLUMN[1];

 

Ancestors[0].Initialize(attid.GetByDisplayName(“Ancestors_col”, tableid), 256 * 6);

 

err = E.Check(EseHelper.JetRetrieveColumns(sesid, tableid, Marshal.UnsafeAddrOfPinnedArrayElement(Ancestors, 0), Ancestors.Length));

 

// Ensure the “Ancestors_col” exist and contains data.

if (Ancestors[0].err != EseErrors.ColumnNull)

{


// Loop thru all ancestors.


for (int i = 0; i < Ancestors[0].GetData().Length; i += sizeof(int))

{


int dnt = BitConverter.ToInt32(Ancestors[0].GetData(), i);


if (dnt != 2) // “2″ == $ROOT_OBJECT$ == We hit the top most DN Component.

{


// Move the cursor over the ancestor

DBFindDNT(sesid, tableid, dnt, ref err);

 


// Define a list of attributes we want to read off each ancestor (object)


EseHelper.JET_RETRIEVECOLUMN[] attrList = new
EseHelper.JET_RETRIEVECOLUMN[2];

 

attrList[0].Initialize(attid.GetByDisplayName(“RDNtyp_col”, tableid), 256 * 24);

attrList[1].Initialize(attid.GetByDisplayName(“name”, tableid), 256 * 24);

 

err = E.Check(EseHelper.JetRetrieveColumns(sesid, tableid, Marshal.UnsafeAddrOfPinnedArrayElement(attrList, 0), attrList.Length));

 


// Make sure the object has both a RDNType and a Name


if (attrList[0].err != EseErrors.ColumnNull && attrList[1].err != EseErrors.ColumnNull)

{


string RDNType = DBGetRDNType(tableid, attrList[0].GetInteger());


string Name = attrList[1].GetString();


// Construct this ancestors RDN.

DN.Add(RDNType + “=” + Name);

}

}

}

DN.Reverse();


return
string.Join(“,”, DN.ToArray());

}

return
null;

}

 

Table 5: Get an objects distinguished name (object is referenced by sAMAccountName: ADCH)

ESEDump

 

  1. adamant says:

    “Marshal” , “E.Check”,”EseHelper”,I can’t find where they come from,Please tell me.I try to debug the code snippet in the C#.would you offer me a complete code,I am interested in data stores in the ntds.dit.

    • Christoffer Andersson says:

      E.Check is part of the managed ESEWrapper. It simpely check for errors returned by the ESE/JetAPI. You can just skipp it or build a simular function.

  2. Chris128 says:

    Wow I’m impressed you figured a lot of this out when it isn’t documented anywhere as far as I know. Also nice to see you did it all in managed .NET code (well aside from the Platform Invoke calls to the ESE/JET APIs) as I had assumed your utility was all written in C++ or something like that.