Tag Archives: Security

Scaling to 10,000 unique permissions – Part 2 – The Solution

This follows on from my previous post; Part 1 – The Problem

The main requirement was:

  • One SharePoint 2010 site
  • 10,000+ uniquely permissioned objects each with a different user account

In this post we will be discussing the solution which involves programmatically creating unique permissions in a way which will scale for (what should be) well over 10,000 uniquely permissioned items…

Introducing yet another little known SharePoint API call …

This is only possible because of one of the new SharePoint 2010 API calls;



This basically adds the specified SPRoleAssignment but does not create any of the Limited Access scopes on the parent objects.

This is pretty straight forward and works in exactly the same way to normal Role Assignments in SharePoint 2010, we simply use the AddToCurrentScopeOnly() method instead of using the Add() method, for example:


   1: // fetch the Principal object which we are granting access to

   2: SPUser user = web.EnsureUser("Domain\\UserAccount");


   4: // create a Role Assignment binding

   5: SPRoleAssignment roleAssignment = new SPRoleAssignment(user);


   7: // apply contribute permissions

   8: roleAssignment.RoleDefinitionBindings.Add(

   9:     web.RoleDefinitions["Contribute"]);


  11: // grant permissions to the list item using the CURRENT SCOPE ONLY

  12: // this ensures that Limited Access scopes are NOT created

  13: // for parent objects (we're going to have to do that bit ourselves!)

  14: item.RoleAssignments.AddToCurrentScopeOnly(roleAssignment)


It is very important to understand that you still need to grant “Limited Access” (it wasn’t put in just for laughs, it does have a purpose). Granting “Limited Access” means that the object has access to core information on parent objects to enable construction of things like the breadcrumb, and retrieval of core files needed to render the interface.

This then means it is up to us (the developers) to go back and create each of those in a more efficient way. The problem is .. you can’t assign “Limited Access” programmatically…

What do you mean .. I can’t assign Limited Access??

Well, I don’t really know why they did this, but if you try and assign it programmatically (Limited Access is actually a “Permission Level” in SharePoint) you will get errors (admittedly you can’t do this through the user interface either!).

So, the workaround (again) is to create your own permission level which includes exactly the same permissions that “Limited Access” would have granted. This is:

  • View Application Pages
  • Browse User Information
  • Use Remote Interfaces
  • Use Client Integration Features
  • Open

You can call this anything you like (I called mine “SP Limited Access”) as long as you know what it means.

The code to do this is as follows:


   1: internal SPRoleDefinition GetLimitedAccessRole(SPWeb web)

   2:         {

   3:             string strRoleDefinition = "SP Limited Access";


   5:             // only exists in webs with unique role definitions

   6:             if (web.HasUniqueRoleDefinitions)

   7:             {

   8:                 try

   9:                 {

  10:                     // try to retrieve the role definition

  11:                     return web.RoleDefinitions[strRoleDefinition];

  12:                 }

  13:                 catch (SPException)

  14:                 {

  15:                     // SPException means it does not exist


  17:                     // create our custom limited access role

  18:                     SPRoleDefinition roleDef = new SPRoleDefinition();


  20:                     // give it a name and description

  21:                     roleDef.Name = "SP Limited Access";

  22:                     roleDef.Description = "Identical to standard " + 

  23:                         "Limited Access rights. " + 

  24:                         "Used to provide access to parent objects of " + 

  25:                         "uniquely permissioned content";


  27:                     // apply the base permissions required

  28:                     roleDef.BasePermissions = SPBasePermissions.ViewFormPages 

  29:                         | SPBasePermissions.Open 

  30:                         | SPBasePermissions.BrowseUserInfo 

  31:                         | SPBasePermissions.UseClientIntegration 

  32:                         | SPBasePermissions.UseRemoteAPIs;


  34:                     // add it to the web

  35:                     web.RoleDefinitions.Add(roleDef);

  36:                 }


  38:                 return web.RoleDefinitions[strRoleDefinition];

  39:             }

  40:             else

  41:             {

  42:                 // try the parent web

  43:                 return GetLimitedAccessRole(web.ParentWeb);

  44:             }

  45:         }

I’ve created my new Limited Access Permission Level .. now what?

One thing does need to be made clear, there is absolutely no point you just creating all of the Security Scopes that SharePoint would have created (you’ll end up with the same mess we were trying to avoid in the first place).

The solution is to create a group for all of the “Limited Access” users for that List or Web. It really is up to you whether you use Active Directory Security Groups or SharePoint Groups. I decided to use AD security groups; mainly because I didn’t want to clog up the Site Collection “groups” functionality, and didn’t want idiot Site Collection admins from removing the group members (or worse .. the groups themselves!) and breaking the site collection.

Note – I haven’t included the code to create and modify Active Directory Security Groups here, if nothing else because there are thousands of resources out there showing you how to modify AD groups programmatically, and Code Project has a particularly good reference: Howto: (Almost) Everything In Active Directory via C#

You will need to create a group for each parent object which has unique permissions although in my example it is only really the SPWeb (web site) that we are worried about as the libraries and folders are well within the security scope threshold.

So we have our 20 libraries and our root web site. So in our example we would have to create 21 different AD security groups:

  • One group to store all Limited Access users for the root web site
  • 20 groups to store all Limited Access for the libraries (one for each library)

Then, following this example you can then use the following code to grant “Limited Access” to one of the libraries (and just rinse and repeat for the other libraries and the root web site);


   1: // fetch the "SP Limited Access" role definition

   2: SPRoleDefinition limitedAccessRole = GetLimitedAccessRole(web);


   4: // get SPPrincipal object for the AD Group we created

   5: SPUser adGroup = web.EnsureUser("My Custom AD Group Name");


   7: // set the role assignments for this group

   8: SPRoleAssignment roleAssignment = new SPRoleAssignment(adGroup);

   9: roleAssignment.RoleDefinitionBindings.Add(limitedAccessRole);


  11: // grant "Limited Access" to the AD Group for this list

  12: // we only have to do this once! After this we simply 

  13: // need to add members to this AD Group every time we 

  14: // add users to one of the parent objects!

  15: list.RoleAssignments.AddToCurrentScopeOnly(roleAssignment)

So having done this for all of the parent objects we now have our 21 custom Active Directory groups, each one of which has been granted “Limited Access” to one of the required “parent” objects for our folders.

From here on in it should be smooth sailing. You simply need to make sure that every time you programmatically add a new user to one of the folders you also make sure they get added to the relevant AD Groups (so that the “Limited Access” chain is not broken).

The following diagram really explains what we have done:


I have tested this model for over 16,000 unique AD accounts across hundreds of folders in hundreds of document libraries and I cannot notice any discernable drop off in performance (nothing that can’t be explained by simply having a really large number of libraries and folders anyway!) so initial tests show that this is working very well indeed 🙂

What I also ended up doing (to make this slightly more robust) is to build my own application page which users can use to Grant Permissions through the UI (so we don’t need to write custom code every time a new “Limited Access” scope is needed).

I then wrote an HttpModule to auto-redirect any requests to the out-of-the-box page (_layouts/AclInv.aspx) to the custom page so that if anyone tried to use the native user interface it would ALWAYS be executing my own custom code (which creates all of the AD Groups and SP Limited Access scopes programmatically, without the user having to worry about it!)

The great thing about this solution is that it doesn’t matter how many users or groups you are adding to your SharePoint site .. you only ever have 1 Limited Access security scope for each List / Web!

Thanks for sticking with me through these two posts .. if you made it this far then thanks for reading and I would love to hear your comments! 🙂

Scaling to 10,000 unique permissions – Part 1 – The Problem

This post was borne out of a client requirement which popped up on my radar. I’m currently working for a leading global Business Intelligence provider in London, and they were looking to implement a particular third party piece of software. This software relies on SharePoint for file storage and my client wanted to roll this out to their customers “extranet” style with each customer having uniquely secured content (files and folders).

Now .. first off their customers include in excess of 10,000 different companies (i.e. > 10,000 users) so early warning bells immediately started ringing in terms of scalability.

Secondly, to make this worse, the software required all content to be stored in a single SharePoint site .. so now my early warning system had gone into full meltdown and a state of high alert was reached.
So to boil this down …

  • One SharePoint 2010 site
  • 10,000+ uniquely permissioned objects each with a different user account

A Library with 10,000 uniquely permissioned folders?? Possible? My first instincts said no… so it was time to get my problem solving hat on and do some digging ..

Investigating the Limits of SharePoint 2010

I would like to think that any SharePoint {Consultant | Developer | Architect | <insert profession>} worth their salt would have read the Software and Capacity Planning guidelines (or at least be aware of it!) .. so that was my first pit-stop.

Note – I also stumbled across a great blog post by SharePoint infrastructure veteran Joel Oleson and his Best Practices for Enterprise User Scalability in SharePoint. This goes into detail about the specific size of an ACL (and the reason why this is limited, specifically in Windows) which although a good read wasn’t really relevant to my problem.

The Microsoft TechNet article SharePoint Server 2010 capacity management: Software boundaries and limits (https://technet.microsoft.com/en-us/library/cc262787.aspx) is a great resource and contains one absolutely key entry:

Security Scope – 1,000 per list (threshold)
The maximum number of unique security scopes set for a list should not exceed 1,000. 

A scope is the security boundary for a securable object and any of its children that do not have a separate security boundary defined.  

A scope contains an Access Control List (ACL), but unlike NTFS ACLs, a scope can include security principals that are specific to SharePoint Server. The members of an ACL for a scope can include Windows users, user accounts other than Windows users (such as forms-based accounts), Active Directory groups, or SharePoint groups.

So what is a Security Scope then? Ok I admit it does tend to get a bit bogged down in terminology.
To put it simply … each time you grant access to a new principal (user account or group) then you are creating a new Security Scope.

The other thing to consider pickup is that this is not just limited to lists! Any list that inherits permissions will pick up their permissions from the parent web (site) so you also need to adhere to this at the web level too!

This means that you should not have more than 1000 security scopes at EITHER the Site or List level.

Ignoring this limit can do real damage to your farm …

There is even a Microsoft Knowledgebase article explaining why; SharePoint performance degradation with a large number of unique security scopes in lists (https://support.microsoft.com/kb/2420771)

This is really explained in far more detail in two most excellent blog posts:

The first post describes the problem of trying to create more than 1000 security scopes, and what happens when you do this: https://wbblog.datapolis.com/2011/03/setting-item-permissions-with-workflow.html

The second post is by James Love (a.k.a. @jimmywim) and goes into real “deep dive” detail looking into the root cause of the problem (SQL Server and ACL GUIDs) and how this problem can actually bring down your ENTIRE FARM and not just the list / site you are working on!

A quote from the second post is as follows:

“When you load up a huge list with lots of item level permissions, a single operation gets every single GUID associated with the ACL for that item and passes that back to the data access layer of SharePoint. When the database retrieves the actual list item data, it will pass in all of the ACL Guids back in as one long string, all concatenated together. The query to get the data creates a table variable re-assembles the the item level ACL Guid associated with each item. How the rest of the query deals with this is anyone’s guess at the moment – this table variable might just be passed back to the calling COM object (though I thought they couldn’t be used this way….) for the COM object to then sort out which item should be visible to which “scope” (or ACL).

So, what can we take away form this? Passing 640k of data about the place, for a SQL Query to do some substring math and converting to Guids will soon bring your database server to its knees. This is one request and it takes 2000ms to work. Imagine if you have 5 requests per second or more hitting this list!”

Both are excellent appendums to this post and well worth looking at for another angle and a bit more detail!

Why does this become my problem?

Now .. looking back to my original problem some of you may be thinking, OK no problem; you can just create yourself 20 different lists / libraries .. and have 500 unique permissions in each list??

Well .. so you might think .. and here I introduce the juggernaut that is Limited Access Scopes!

Anyone who has spent any time around SharePoint security will have noticed the odd “Limited Access” permission popping up in their site from time to time. “Limited Access” is automatically allocated to a parent Folder, List or Web whenever a child object has a unique permission allocated to it.

You can easily see these being created if you break permission inheritence to a list and just add a few accounts to that list. The parent Web will not have a “Limited Access” scope created for each user account you have added.

Now hopefully the bright will already have spotted the problem .. it doesn’t matter how many lists or libraries you create .. every single user or group that you add will end up in the parent Web site with “Limited Access” (and every single Parent Web heading upwards).

The following diagram explains why.

You simply cannot get away from this fact. If are adding 10,000 unique permissions with different user accounts then you will end up with 10,000 security scopes at the root web!

Note – It should be noted that the number of “Limited Access” scopes created is limited to the number of Security Principals you are adding.

If you are adding from a pool of 50 users then you will only ever be adding a maximum of 50 new “Limited Access” scopes (one for each user account).

For this reason it is a good idea to use Groups when adding permissions as this limits the number of “Limited Access” scopes which are created .. but this won’t solve your problem if you have over 1000 different security principals!

So that was the crux of my problem .. on investigation this does look to be a major major problem (and an “impossible fix”) but it would seem not! There IS a workaround (one which I have tested to over 15,000 unique user accounts and works very very well indeed)…

The solution, workaround, and code samples are all included in Part 2

Understanding SharePoint Application Security and Elevating Privileges

This post was prompted because of a particularly challenging bit of security that I needed to traverse. I needed some way of presenting the status of a Content Deployment Job (configured in Central Administration) in the Web Application that it relates to.

Seems pretty straight forward?
Well, its not, and this article will hopefully explain why.

RunWithElevatedPrivileges and Application Pool Accounts
So the first thing I looked at was using the good old SPSecurity.RunWithElevatedPrivileges method. This is a well known (and on occassion heavily used) practice for getting around security in SharePoint. But does everyone understand exactly what it does?

In a nut-shell, this method simply changes the currently impersonated user from the currently logged in user to an account called “SharePoint\System” (a.k.a. “System Account”).

This account doesn’t actually exist, and anyone inspecting the WindowsIdentity or SPUser object in any great detail will spot that this account doesn’t actually have a valid SID (Security Identifier). This is because it represents a placeholder.. a flag in SharePoint that tells it to impersonate the Application Pool Account instead of the currently logged in user.

The Application Pool Account has full SharePoint permissions to the Web Application (effectively making it a Site Collection Administrator in every single Site Collection).

So what does this actually mean?

SQL Server Permissions
Believe it or not, SQL Server permissions in SharePoint are extremely simple.

Taking the 3 core databases for each SharePoint Farm:

1. Farm Configuration Database
This contains the core configuration information (servers, URLs, accounts) for the entire SharePoint Farm.

The Setup Account has DBOwner permissions.

All application pools accounts are added to a Database Role called WSS_Content_Application_Pools which has severely locked down read privileges.

2. Central Administration Content Database
This is effectively the content database for the Central Administration site. This contains the SPSite / SPWeb / SPList objects that store all of the content related settings (including Content Deployment Jobs).

Again, the Setup Account (which incidentally will be running the Central Administration Application Pool!) has DBOwner permissions.

All application pools accounts are added to a Database Role called WSS_Content_Application_Pools which has severely locked down read privileges.

3. Web Application Content Database
This is the database (or mulitple databases) that contain the Site Collection content for the Web Application.

Here the Application Pool Account (for that specific Web Application) is granted DBOwner permissions. No other accounts are specified!

That is pretty much it. From a security (and “least privileged” perspective) it’s a very robust setup. If your application pool is compromised then the application pool account only has SQL permissions to it’s own content database.

According to best practice, every Web Application should have it’s own application pool account, which again makes sense according to the model above, limiting the surface area for any attack (as one web application being compromised would not have any impact on the other application pools).

This should also make it obvious why you should never make an Application Pool Account a Local or Farm Administrator! You are essentially breaking the security model if you do this (and massively widening the exposed area of your system if that account is ever exposed!).

NTLM authentication and “Double Hop”
The first thing that should scream at you here is that none of the SharePoint user accounts have ANY permissions in SQL. Every single SQL query is executed within a SharePoint Web Application using the Application Pool account!

The reason for this is clear once you understand the limitations of NTLM authentication.

Basically, when you log in to a SharePoint web site, you authenticate with the Web Server (IIS). There is no way for IIS to pass through credentials back to SQL Server because NTLM only supports “single hop” authentication (i.e. from one single machine – the browser – to another machine – the web server). For “double-hop” you need a more robust authentication method such as Kerberos (i.e. from one machine – the browser – hop to another machine – the web server – hop a second time to a third machine – the database server?).

Note – This is why you need Kerberos to use pass-through authentication with 3rd party systems (such as CRM or other LOB systems).

Thats all great .. but what do I care?
Well, this all nails down to where the object is that you are trying to access, what the SQL permissions are on that object.

Lets take the example of accessing a Content Deployment Job.

The first problem you will hit is that your account needs to be a Farm Administrator. We already know that making the Application Pool an admin account is bad for security.

So as an alternative you could use ASP.Net Impersonation to get around the SharePoint API, but as we discussed above, this doesn’t solve the NTLM “single-hop” problem (your query is still going to execute in SQL using the Application Pool account, regardless of which account you are impersonating!)

Using .Net Reflector (tsk!) tells us that the Content Deployment Job information is stored in an SPList in the Central Administration Content Database. Using RunWithElevatedPrivileges simply executes using the Application Pool account (which we know from the SQL Permissions above, has very limited permissions).

So lets assume you tried to use Impersonation … what happens?

Well, you get a nasty “Exception in HRESULT” error message.
Delving in to the SharePoint Diagnostics Logs tells something like “ does not have EXECUTE permissions on ‘proc_EnumLists’ in “.

Basically running that code tries to execute a Stored Procedure in SQL in the Central Admin database which the Application Pool Account doesn’t have access to! Your code managed to fool the SharePoint API into thinking you have permissions, but good old SQL Server stops you short (just as it should … good server!)

So what can I do?
Well, the first thing to note is that you won’t always run into this problem.
Many of the Farm level options (including access SSP and User Profile properties) can be gotten around in other ways, but when something like the above happens, your options are limited to 3 potential solutions:

  1. Ignore all of the best practice. Make your application pool account an administrator, and spend your days hiding from the network security admins and hoping it doesn’t all go wrong.
  2. Create a dedicated Web Service, which executes as an admin account. Use this to farm out your “privileged” code, and make sure you lock it down tight as a drum so you can’t get to it from outside of the SharePoint farm!
  3. Don’t do it .. and tell your users that it was a stupid idea in the first place!

Now I admit, Options 1 and 3 probably won’t go down too well, and Option 2 is the best option but still has it’s issues (running a Web Service as an admin account is still a security risk, if a smaller one than running the entire public facing Application Pool as an admin account!)

We ended up opting for Option 2, admittedly locking it down so that the URL was never published and it would only accept connections from other servers in the farm (so that end users could never access it).

Hopefully you now have a better grasp of SharePoint Application Security, what that super-method “SPSecurity.RunWithElevatedPrivileges” is actually doing and why it doesn’t always work!

Comments a feedback welcome! 🙂

Securing SharePoint 2010 Web Servers


This was one of the best topics I’ve seen so far at the conference. The amount of concrete information was impressive (and to be honest a bit too much to post here) but there was some great information on how to harden your Web Servers.


SharePoint 2010 Security Features

There are a whole load of new features and changes to the SharePoint 2010 product for security.


  • ASPX Pages are gone for contributors. You can no longer upload ASPX pages into document libraries unless you have "Designer" permissions! The main reason this becomes possible is because the new Wiki Pages are so much more extensible than they were.
  • Anonymous Users Lockdown feature  now works for Web Services and WSS (SharePoint Foundation 2010)!
  • PowerShell Access – you can now delegate remote scripting rights through PowerShell, so you no longer need the Setup account to perform PowerShell commands. This can be delegated to farm administrators!
  • XSS (Cross Site Scripting) protection is now in place through the headers (although you can turn it off). This can be even be locked down to individual web part properties (through development)!
  • Application Page settings can now be controlled more granularly, so that you can set the master pages used and even swap out individual pages (such as the Error Page). This makes lock downs and branding of these far easier, without breaking the supported state of your environment, and without extensive development!


There was then whole load of recommendations for hardening your environments. It’s a bit of a list so apologies for that, but a lot of information to get through:


Hardening your Web Application

  • Place your web application directories on a non-system volume. If you have any issues with logging or file access then the I/O operations (or even disk space requirements) could damage the Operating  System!
  • Change the IIS header. By default this will include the SharePoint version number (which means any attacker knows which service packs and critical patches you have installed!). Removing this reduces your public footprint


Hardening your Web Servers

Windows Server 2008 takes care of most of the previous recommendations for hardening automatically, but there are still some things that you should do:


  • Restrict remote administration of the Registry (no-brainer, but a lot of people forget to do this)
  • Rename Administrator account
  • Delete / Disable unused accounts (again, make sure your dev and test accounts don’t hang around on the web front ends)
  • Use the IUSR instead of IUSR_<serverName>

The IUSR account is a "built in" account so therefore it doesn’t have a password and no-one can login using that account. This makes it much more secure than the Server specific IUSR account that gets created!


Hardening SQL 

There’s a whole load about this on the internet. The only one to mention here is change the port number! There are a lot of viruses and malware that will specifically target this port.


Hardening your Network

Again, none of this is SharePoint specific, but goes a long way to making sure that your network in general is secure (which is of course best practice for SharePoint systems).



  • Block unused protocols and ports (see ports required, below)
  • Screen Traffic (e.g. ICMP)
  • Intrusion Detection should be in place



  • Use packet filtering policies
  • Log your permitted / denied traffic, and make sure those logs are checked (using alerts)
  • Make sure perimeter networks are firewall secured, effectively providing end to end firewall security.



  • Disable any unused services in the switch
  • Do not overly trust VLANS. Just because your traffic is isolated to a VLAN doesn’t mean you shouldn’t still block off the relevant ports and protocols.


Ports Required for Web Servers

Note – When SharePoint is installed the communication ports are automatically opened on the Windows Firewall!



  • Http 80 / TCP
  • HTTPS 443 / TCP
  • SMTP 25 / TCP



  • HTTP 32843/TCP
  • HTTPS 32844 / TCP
  • TCP 32845 / TCP
  • SMB 445 /TCP|UDP


* note that "internal" means Web Application –> Service consumtion over WCF. It does not include SQL or inter "server" communications.

Cross Site Scripting (XSS) protection for SharePoint 2010 Web Parts


Some of the new features in SharePoint 2010 offer some great new opportunities for malicious scripts to be manipulated in your system. The new SharePoint 2010 Client Object Model is a great case in point.


Let’s take the example where a contributor adds some Client Object Model scripts through exposed web Part properties to change list data that they don’t have access to. As soon as someone with admin privileges visits the page that Client OM kicks off and you’ve got yourself malicious script executing!


Well, step in the new XSS protection. The WebPartPages class now includes a new attribute that you can add to your Web Part Properties called "RequiresDesignerPermissionAttribute". There is also a new SafeControl attribute called "SafeAgainstScript".


These allow you to protect your assemblies and properties against contributors. The main problem is that none of your MOSS 2007 web part properties will be accessible to contributors without these added!


This obviously creates quite an overhead in terms of code use, but it really is required to make sure that your web parts are running in an appropriately secure state.

People search errors when My Site is running on SSL

This was a great gotcha that I picked up duing a client project handover.
They reported issues with search, and I found a load of errors in the crawl logs regarding accessing the mysite.
You will find that this issue occurs when you configure your My Sites to run under SSL (HTTPS), and the problem is that the People Search isn’t really interested in the My Site content, it is actually interested in the Profile Database (which is another thing most people forget to switch on .. make sure your Profile Database has an import schedule setup!)
Basically, you need to set the Content Source URL to the following:
So if your mysite is running under https://mysite.company.com/ then you need to set your Content Source URL to:
Personally I found this pretty confusing, so I actually  split out the Profile Database content into a separate Content Source, and configured the People Scope to only retrieve items from that specific Content Source.
Happy searching!
Just a quick update, because this is another “People Search” problem. Even if you don’t use My Sites, you should still switch on and use the MySite web application and settings (You can stop people from creating “My Sites” by disabling Self Service Site Creation at the Web Application level).
The reason is that the My Site web application is used for People Search URLs and will automatically dsiplay information from the Profile Database (which is basically the user’s “public profile”) even if they don’t have a My Site created!