Powershell – How to fix this memory leak in the SharePoint Powershell script

memory-leakspowershellsharepoint

Purpose of the script: due to our Active Directory infrastructure, we add about 6000 entries directly to the Site User Info List so that they are shown in the address book in the SharePoint people picker.
The script creates about 6000 SPUser objects in order to add them to the Site User Info List. Most of them already exist, in this case new-SPUser should do nothing, right?

EDIT: please leave a feedback or possible solution. I removed some of the large log for better overview and added some details. Perhaps you know another forum where I should post this question?

I get lots of messages like this:

Potentially excessive number of SPRequest objects (20) currently
unreleased on thread 9. Ensure that this object or its parent (such
as an SPWeb or SPSite) is being properly disposed. This object is
holding on to a separate native heap.This object will not be
automatically disposed. Allocation Id for this object: {…snip…} Stack
trace of current allocation: at
Microsoft.SharePoint.SPGlobal.CreateSPRequestAndSetIdentity(SPSite
site, String name, Boolean bNotGlobalAdminCode, String strUrl, Boolean
bNotAddToContext, Byte[] UserToken, String userName, Boolean
bIgnoreTokenTimeout, Boolean bAsAnonymous)
…snip…

And lots of messages like this:

08/17/2011 19:38:35.26 PowerShell.exe (0x1388) 0x20C0 SharePoint Foundation Performance nask Monitorable An SPRequest object was reclaimed by the garbage collector instead of being explicitly freed. To avoid wasting system resources, dispose of this object or its parent (such as an SPSite or SPWeb) as soon as you are done using it. Allocation Id: {09C61E87-2BA1-4CEA-BB3C-266A1345559A} To determine where this object was allocated, set Microsoft.SharePoint.Administration.SPWebService.ContentService.CollectSPRequestAllocationCallStacks = true. de663eab-8063-4381-88b5-01cc50d51b77

This is the current script with highlighted four lines (# ***) that we added to solve it (did not work).

Write-Host "Fill Site User Info List for "$args[0]
add-pssnapin microsoft.sharepoint.powershell -ErrorAction SilentlyContinue

Start-SPAssignment -Global    # ***

#global properties
$ppFilter = "(...)"

get-date

$ldapFilter = "(...)"
$ldapDirectory = "LDAP://DC=..."
Write-Host "Get all users from ..."
$de = New-Object system.directoryservices.directoryentry($ldapDirectory)
$de
$newsearch = New-Object System.DirectoryServices.DirectorySearcher($de)
write-host "Filter all active users with ..."
$newsearch.Filter = $ldapFilter
$newsearch.PageSize = 200
$result = $newsearch.FindAll()

write-Host "Get all domain user accounts..."
$useraccounts = $result | % {"..."}

$siteCollection = get-spsite $args[0]
$webApp = $siteCollection.WebApplication

Write-Host "clear people picker filter" $webApp.Name "..."
$webApp.peoplepickersettings.ActiveDirectoryCustomFilter = ""
$webApp.Update()

write-Host "---------------"
$siteUrl = $siteCollection.Url
Write-Host "add "$useraccounts.Count" users to site collection "$siteUrl "..."
$start =  get-date
$site = $siteCollection.RootWeb       # ***
foreach ($user in $useraccounts) { 
    $current = New-SPUser -UserAlias $user -Web $site   # *** was before: $siteUrl
}
$end = get-date
$end - $start
$SiteCollection.Dispose()
Write-Host "---------------"

Write-Host "set people picker filter to" $ppFilter "..."
$webApp.peoplepickersettings.ActiveDirectoryCustomFilter = $ppFilter
$webApp.Update()

Stop-SPAssignment -Global    # ***

Write-Host "Done."
get-date

Is there an obvious error we made? Thank you for any suggestions.

In answer to @x0n I post a larger portion of the log. Sorry for this unreadable formatting. Is there a better way to post logs? All line breaks and tab spaces are messed up.

The point is, that the messages about reclaimed SPRequest object come from the powershell process. I am perfectly sure that this script was the only one running. This script used to finish within less than 10 Minutes, but now it does not finish even after 1.5 hours. I am pretty sure I did not change anything in the script that changed the behaviour. The performance view of the Windows Task Manager did show a moderate load on CPU and quite high usage of RAM (not all the way to 100%) while the script was running.

Timestamp Process TID Area Category EventID Level Message Correlation
08/17/2011 19:38:34.30 PowerShell.exe (0x1388) 0x20C0 SharePoint Foundation PowerShell 6tf0 Medium Entering EndProcessing Method of New-SPUser.
08/17/2011 19:38:34.30 PowerShell.exe (0x1388) 0x20C0 SharePoint Foundation PowerShell 6tf0 Medium Leaving EndProcessing Method of New-SPUser.
08/17/2011 19:38:34.30 PowerShell.exe (0x1388) 0x20C0 SharePoint Foundation PowerShell 6tf0 Medium Entering BeginProcessing Method of New-SPUser. de663eab-8063-4381-88b5-01cc50d51b77
08/17/2011 19:38:34.30 PowerShell.exe (0x1388) 0x20C0 SharePoint Foundation PowerShell 6tf0 Medium Leaving BeginProcessing Method of New-SPUser. de663eab-8063-4381-88b5-01cc50d51b77
08/17/2011 19:38:34.30 PowerShell.exe (0x1388) 0x20C0 SharePoint Foundation PowerShell 6tf0 Medium Entering ProcessRecord Method of New-SPUser. de663eab-8063-4381-88b5-01cc50d51b77
08/17/2011 19:38:34.30 PowerShell.exe (0x1388) 0x20C0 SharePoint Foundation Performance naqx Monitorable Potentially excessive number of SPRequest objects (19) currently unreleased on thread 9. Ensure that this object or its parent (such as an SPWeb or SPSite) is being properly disposed. This object is holding on to a separate native heap.This object will not be automatically disposed. Allocation Id for this object: {815F9A3D-6635-42FA-9658-B6294968136E} Stack trace of current allocation: at Microsoft.SharePoint.SPGlobal.CreateSPRequestAndSetIdentity(SPSite site, String name, Boolean bNotGlobalAdminCode, String strUrl, Boolean bNotAddToContext, Byte[] UserToken, String userName, Boolean bIgnoreTokenTimeout, Boolean bAsAnonymous) at Microsoft.SharePoint.SPRequestManager.GetContextRequest(SPRequestAuthenticationMode authenticationMode) at Microsoft.SharePoint.Administration.SPFarm.g… de663eab-8063-4381-88b5-01cc50d51b77

( …. snip ….)

08/17/2011 19:38:34.30 PowerShell.exe (0x1388) 0x20C0 SharePoint Foundation Performance naqx Monitorable Potentially excessive number of SPRequest objects (19) currently unreleased on thread 9. Ensure that this object or its parent (such as an SPWeb or SPSite) is being properly disposed. This object is holding on to a separate native heap. Allocation Id for this object: {F9CDF3DA-41B1-485D-B78F-0266545C417A} Stack trace of current allocation: at Microsoft.SharePoint.SPGlobal.CreateSPRequestAndSetIdentity(SPSite site, String name, Boolean bNotGlobalAdminCode, String strUrl, Boolean bNotAddToContext, Byte[] UserToken, String userName, Boolean bIgnoreTokenTimeout, Boolean bAsAnonymous) at Microsoft.SharePoint.SPWeb.InitializeSPRequest() at Microsoft.SharePoint.SPWeb.InitWebPublic() at Microsoft.SharePoint.SPWeb.get_Exists() at Microsoft.SharePoint.PowerShell.SPWebPipeBind.Rea… de663eab-8063-4381-88b5-01cc50d51b77

(… snip …)
(… many of those messages: )
08/17/2011 19:38:35.26 PowerShell.exe (0x1388) 0x20C0 SharePoint Foundation Performance nask Monitorable An SPRequest object was reclaimed by the garbage collector instead of being explicitly freed. To avoid wasting system resources, dispose of this object or its parent (such as an SPSite or SPWeb) as soon as you are done using it. Allocation Id: {09C61E87-2BA1-4CEA-BB3C-266A1345559A} To determine where this object was allocated, set Microsoft.SharePoint.Administration.SPWebService.ContentService.CollectSPRequestAllocationCallStacks = true. de663eab-8063-4381-88b5-01cc50d51b77

Best Answer

This message does not actually indicate a leak - the key thing here is the word "potentially." All that message means is that your thread has a lot open at the same time. The threshhold for emitting that message is actually set quite low for practical use. What's really important is that they are disposed correctly at the end of their lifetime. If you really had a leak, you'd see messages like this:

05/012/2011 12:58:47.31 w3wp.exe (0x105C)
0x09A8 Windows SharePoint Services General
8l1n High An SPRequest object was not disposed before the end of this thread. To avoid wasting system resources, dispose of this object or its parent (such as an SPSite or SPWeb) as soon as you are done using it. This object will now be disposed. Allocation Id: {5BFFCA4B-3B91-45BF-98CD-0BB508BE30EE} To determine where this object was allocated, create a registry key at HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\HeapSettings. Then create a new DWORD named SPRequestStackTrace with the value 1 under this key.

Stefan Grossner has a great article explaining the reasoning behind all of this, including why the threshold for the warning is probably too low (and how you can change it to something more practical.)

http://blogs.technet.com/b/stefan_gossner/archive/2008/05/07/troubleshooting-spsite-spweb-leaks-in-wss-v3-and-moss-2007.aspx

Incidentally, even if you did have a leak and you couldn't find it, closing the powershell process will return the memory.

Hope this helps.

Exit vs Return vs Break

Exit: This will "exit" the currently running context. If you call this command from a script it will exit the script. If you call this command from the shell it will exit the shell.

If a function calls the Exit command it will exit what ever context it is running in. So if that function is only called from within a running script it will exit that script. However, if your script merely declares the function so that it can be used from the current shell and you run that function from the shell, it will exit the shell because the shell is the context in which the function contianing the Exit command is running.

Note: By default if you right click on a script to run it in PowerShell, once the script is done running, PowerShell will close automatically. This has nothing to do with the Exit command or anything else in your script. It is just a default PowerShell behavior for scripts being ran using this specific method of running a script. The same is true for batch files and the Command Line window.
Return: This will return to the previous call point. If you call this command from a script (outside any functions) it will return to the shell. If you call this command from the shell it will return to the shell (which is the previous call point for a single command ran from the shell). If you call this command from a function it will return to where ever the function was called from.

Execution of any commands after the call point that it is returned to will continue from that point. If a script is called from the shell and it contains the Return command outside any functions then when it returns to the shell there are no more commands to run thus making a Return used in this way essentially the same as Exit.
Break: This will break out of loops and switch cases. If you call this command while not in a loop or switch case it will break out of the script. If you call Break inside a loop that is nested inside a loop it will only break out of the loop it was called in.

There is also an interesting feature of Break where you can prefix a loop with a label and then you can break out of that labeled loop even if the Break command is called within several nested groups within that labeled loop.
```
While ($true) {
    # Code here will run

    :myLabel While ($true) {
        # Code here will run

        While ($true) {
            # Code here will run

            While ($true) {
                # Code here will run
                Break myLabel
                # Code here will not run
            }

            # Code here will not run
        }

        # Code here will not run
    }

    # Code here will run
}
```

Best Answer

Related Solutions

R – Why does the “out of the box” SharePoint Navigation look like it is leaking memory

Powershell – Terminating a script in PowerShell

Exit vs Return vs Break

Related Topic