Replacing the default memory manager

While writing a new memory manager is a hard job, installing it in Delphi—once it is completed—is very simple. The System unit implements functions GetMemoryManager and SetMemoryManager that help with that:

type
  TMemoryManagerEx = record
    {The basic (required) memory manager functionality}
    GetMem: function(Size: NativeInt): Pointer;
    FreeMem: function(P: Pointer): Integer;
    ReallocMem: function(P: Pointer; Size: NativeInt): Pointer;
    {Extended (optional) functionality.}
    AllocMem: function(Size: NativeInt): Pointer;
    RegisterExpectedMemoryLeak: function(P: Pointer): Boolean;
    UnregisterExpectedMemoryLeak: function(P: Pointer): Boolean;
  end;

procedure GetMemoryManager(var MemMgrEx: TMemoryManagerEx); overload;
procedure SetMemoryManager(const MemMgrEx: TMemoryManagerEx); overload;

A proper way to install a new memory manager is to call GetMemoryManager and store the result in some global variable. Then, the code should populate the new TMemoryManagerEx record with pointers to its own replacement methods and call SetMemoryManager. Typically, you would do this in the initialization block of the unit implementing the memory manager.

The new memory manager must implement functions GetMem, FreeMem, and ReallocMem. It may implement the other three functions (or only some of them). Delphi is smart enough to implement AllocMem internally, if it is not implemented by the memory manager, and the other two will just be ignored if you call them from the code.

When memory manager is uninstalled (usually from the finalization block of the memory manager unit), it has to call SetMemoryManager and pass to it the original memory manager configuration stored in the global variable.

The nice thing with memory managers is that you can call functions from the previous memory manager in your code. That allows you to just add some small part of functionality which may be useful when you are debugging some hard problem or researching how Delphi works.

To demonstrate this approach, I have written a small logging memory manager. After it is installed, it will log all memory calls (except the two related to memory leaks) to a file. It has no idea how to handle memory allocation, though, so it forwards all calls to the existing memory manager.

The demonstration program, LoggingMemoryManager, shows how to use this logging functionality. When you click the Test button, the code installs it by calling the InstallMM function. In the current implementation, it logs information info the file, <projectName>_memory.log which is saved to the <projectName>.exe folder.

Then the code creates a TList<integer> object, writes 1,024 integers into it and destroys the list.

At the end, the memory manager is uninstalled by calling UninstallMM and the contents of the log file are loaded into the listbox:

procedure TfrmLoggingMM.Button1Click(Sender: TObject);
var
  list: TList<integer>;
  i: Integer;
  mmLog: String;
begin
  mmLog := ChangeFileExt(ParamStr(0), '_memory.log');
  if not InstallMM(mmLog) then
    ListBox1.Items.Add('Failed to install memory manager');

  list := TList<integer>.Create;
  for i := 1 to 1024 do
    list.Add(i);
  FreeAndNil(list);

  if not UninstallMM then
    ListBox1.Items.Add('Failed to uninstall memory manager');

  LoadLog(mmLog);
end;

The memory manager itself is implemented in the unit LoggingMM. It uses three global variables. MMIsInstalled contains the current installed/not installed status, OldMM stores the configuration of the existing state, and LoggingFile stores a handle of the logging file:

var
  MMIsInstalled: boolean;
  OldMM: TMemoryManagerEx;
  LoggingFile: THandle;

The installation function firstly opens a logging file with a call to the Windows API CreateFile. After that, it retrieves the existing memory manager state, sets a new configuration with pointers to the logging code, and exits. Memory leak functions are not used so corresponding pointers are set to nil:

function InstallMM(const fileName: string): boolean;
var
  myMM: TMemoryManagerEx;
begin
  if MMIsInstalled then
    Exit(False);

  LoggingFile := CreateFile(PChar(fileName), GENERIC_WRITE, 0, nil, 
    CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);
  if LoggingFile = INVALID_HANDLE_VALUE then
    Exit(False);

  GetMemoryManager(OldMM);

  myMM.GetMem := @LoggingGetMem;
  myMM.FreeMem := @LoggingFreeMem;
  myMM.ReallocMem := @LoggingReallocMem;
  myMM.AllocMem := @LoggingAllocMem;
  myMM.RegisterExpectedMemoryLeak := nil;
  myMM.UnregisterExpectedMemoryLeak := nil;
  SetMemoryManager(myMM);

  MMIsInstalled := True;
  Result := True;
end;

Using Windows API makes my logging memory manager bound to the operating system so why don't I simply use a TStream for logging?

Well, you have to remember that the purpose of this exercise is to log all memory requests. Using any built-in Delphi functionality may cause hidden memory manager operations. If I'm not careful, an object may get created somewhere or a string could get allocated. That's why the code stays on the safe side and uses Windows API to work with the logging file.

Uninstalling memory manager is simpler. We just have to close the logging file and restore the original configuration:

function UninstallMM: boolean;
begin
  if not MMIsInstalled then
    Exit(False);

  SetMemoryManager(OldMM);

  if LoggingFile <> INVALID_HANDLE_VALUE then begin
    CloseHandle(LoggingFile);
    LoggingFile := INVALID_HANDLE_VALUE;
  end;

  MMIsInstalled := False;
  Result := True;
end;

Logging itself is again tricky. The logging code is called from the memory manager itself, so it can't use any functionality that would use the memory manager.  Most important of all, we cannot use any UnicodeString or AnsiString variable. The logging code therefore pieces the log output together from small parts:

function LoggingGetMem(Size: NativeInt): Pointer;
begin
  Result := OldMM.GetMem(Size);
  Write('GetMem(');
  Write(Size);
  Write(') = ');
  Write(NativeUInt(Result));
  Writeln;
end;

Logging a string is actually pretty easy as Delphi already provides a terminating Chr(0) character at the end of a string. A Write method just passes correct parameters to the WriteFile Windows API:

procedure Write(const s: PAnsiChar); overload;
var
  written: DWORD;
begin
  WriteFile(LoggingFile, s^, StrLen(s), written, nil);
end;

procedure Writeln;
begin
  Write(#13#10);
end;

Logging a number is tricky, as we can't call IntToStr or Format. Both are using dynamic strings which means that memory manager would be used to manage strings, but as we are already inside the memory manager we cannot use memory management functions.

The logging function for numbers therefore implements its own conversion from NativeUInt to a buffer containing a hexadecimal representation of that unit. It uses the knowledge that NativeUInt is never more than 8 bytes long, which generates, at max, 16 hexadecimal numbers:

procedure Write(n: NativeUInt); overload;
var
  buf: array [1..18] of AnsiChar;
  i: Integer;
  digit: Integer;
begin
  buf[18] := #0;
  for i := 17 downto 2 do
  begin
    digit := n mod 16;
    n := n div 16;
    if digit < 10 then
      buf[i] := AnsiChar(digit + Ord('0'))
    else
      buf[i] := AnsiChar(digit - 10 + Ord('A'));
  end;
  buf[1] := '$';
  Write(@buf);
end;

Even with these complications, the code is far from perfect. The big problem with the current implementation is that it won't work correctly in a multithreaded code. A real-life implementation would need to add locking around all the Logging* methods.

Another problem with the code is that logging is really slow because of the frequent WriteFile calls. A better implementation would collect log data in a larger buffer and only write it out when the buffer becomes full. An improvement in that direction is left as an exercise for the reader.

The following image shows the demonstration program, LoggingMemoryManager in action. The first GetMem call creates the TList<Integer> object. The second creates a TArray<Integer> used internally inside the TList<Integer> to store the list data. After that, ReallocMem is called from time to time to enlarge this TArray. We can see that it is not called for every element that the code adds to the list, but in larger and larger steps. Memory is firstly enlarged to $10 bytes, then to $18, $28, $48, and so on to $1,008 bytes. This is a result of the optimization inside the TList<T> that I mentioned at the beginning of this chapter.

Furthermore, we can see that the built-in memory manager doesn't always move the memory around. When a memory is enlarged to $10 bytes, the memory manager returns a new pointer (original value was $2E98B20, new value is $2ED3990). After the next two allocations, this pointer stays the same.

Only when the memory is enlarged to $48 bytes, does the memory manager have to allocate a new block and move the data:

An interesting call happens after all the reallocations. Another copy of the list data is allocated with a call to GetMem($1008). A little digging shows that this happens when list capacity is set to 0 inside the destructor:

destructor TList<T>.Destroy;
begin
  Capacity := 0;
  inherited;
end;

This allocation is caused by the notification mechanism which allows TObjectList<T> to destroy owned objects.  Data needs to be stored temporary before it is passed to the notification event and that is done with a temporary array. 

I'm pretty sure you will not be writing a new memory manager just to improve speed of multithreaded programs. I'm sure I won't do it. Luckily, there are crazier and smarter people out there who put their work out as open source.