Static variables issue in shared hosting or multiple web servers

I just want to share with everyone a lesson learned recently.

Issue: All the static variables which are initialized as class variables are reset to empty, null or 0 depending on the data type of the variable. This happens randomly and cannot be reproduced in the development environment. This issue is specific to web applications and not stand alone/windows apps.

Explanation: Firstly to clarify what I meant by “static variables which are initialized as class variables“, below is an example code snippet.

public class MyUtility
{
static string myURL = “http://somesite.com”;
static int count = 10;


}

In the above code snippet, the variables ‘myURL’ and ‘count’ will be set to empty string and ‘0’ respectively at any time randomly on your production servers if its a load balanced servers and or shared hosting environment. The static variables are initialized properly the first time when the application starts. But whenever the application domain which is the isolated memory in which the web application process runs is reset or recycled all the memory allocated is lost. The application domain reset/recycling is possible in multiple cases. For example:

  • If the web application is not requested for an extended period of time.
  • In shared environment multiple web applications share the limited RAM. Thus, if one web app is idle and doing nothing that will be unloaded and reloaded on demand.
  • The IIS server settings can be set to recycle/reload the application pool periodically or set to recycle whenever the memory allocation exceeds a threshold limit.

Whenever this recycling or reload occurs all the static variables are reset. Not only do these not get reset to their original state but they are also not re-initialised with their design time default values. They will always be given values like null, empty string and zero.  The worst part is that we cannot reproduce this in our development environment as we don’t mimic the production environment.

So the solution is simple, do not use static variables in web applications.

Below is a very good post which mentions more technical details related to this topic.

http://www.navwin.com/Topics/AppDomainRecycling/AppDomainRecycling.aspx

Advertisements

ShareTools Kit – Consolidating multiple SharePoint Helper Tools

Project Description
This is a portal for all of my tools developed to ease a sharepoint developer/admins job.

Update (12/15/09)

Initially this project was created to point to other projects of mine. After putting a good thought I decided to bring all tools into one spot and build a consolidated tool which encompasses all others. I was able to do a total revamp of the project structure and updated the source code control.
Also uploaded the latest release in the downloads section.

The tool is designed as a multiple document interface (MDI) thus at any given point one can access any tool within single window. Below is the list of the nifty tools built so far:

Below are the screens:

http://sharetools.codeplex.com/

Solution to handle large SharePoint lists or library

SharePoint has limitations at various levels. Below is a quick information table:

Container

[recommended limitation of contents]

Web Application

[150000 Site Collections ] (with the limit of 50000 per content db)

Site Collection

[250000 sites ]

Site

[2000 sub sites/sites]

Site/Sub site

[2,000 libraries and lists]

Library/List

[10,000,000 documents/items  and 2,000 items per view]

The above are recommended limitations by Microsoft. Please read the article from MSDN for detailed information and benchmarks. However these limitations vary depending on the hardware specs (YMMY): databases, web applications and the arrangement of sites/site collections. Joel Oleson[MVP] has provided a limitations chart based on his real time experience.

Below is an excerpt from a white paper released by Microsoft on handling very large lists.

For typical customer scenarios in which the standard Office SharePoint Server 2007 browser-based user interface is used, the recommendation is that a single list should not have more than 2,000 items per list container. A container in this case means the root of the list, as well as any folders in the list — a folder is a container because other list items are stored within it. A folder can contain items from the list as well as other folders, and each subfolder can contain more of each, and so on. For example, that means that you could have a list with 1,990 items in the root of the site, 10 folders that each contain 2,000 items, and so on. The maximum number of items supported in a list with recursive folders is 5 million items.

Based on the above excerpt and simplifying it further we can achieve a viable scalable and highly available solution to implement very large lists. Here I am going to discuss a use case which involves large lists and how it can be handled.

Use case/Problem case:

Highly used sharepoint site where user submits requests using various infopath forms which are submitted to the document libraries which has custom workflow attached to it. On an average there would be around 900 to 1000 requests submitted per day.

Proposed Solution:

With the usage rate mentioned in the above use case the list will have accumulate 200000 items in the document library within the first six months. Much against to this the recommended limit is to have 2000 items per list/library before it starts showing a degradation in the performance. The only way to get around this limitation is to use folders within the library and distribute the items/files within these folders. We can even have nested folder structure. The maximum number of items supported in a list with recursive folders is 5 million items. But having a recursive folder structure means more complexity and as data grows there will be manageability concerns. Without using recursive folder structure and thus using a simple solution we can have 4 million items in a single list/library.

The solution is very simple; We will have 2000 folders in a library (since 2000 is the limit on number of items a library can have). Then each folder will have 2000 items within it. Thus the total number of items the library holds with this approach is 2000 * 2000 which is 4 million.

Implementation Details

Assumption: This implementation is based on the assumption that every new infopath form request submitted will be assigned a unique auto-incremented integer as the identifier. This can be implemented by using custom sql database table with an auto-increment integer column. For each form request submitted a new row will be inserted which will assign an unique integer to it.

The integer id assigned to the request submitted forms the crucial part of this solution. Using this we can identify where to store a new request and what is the location/url of an existing(already saved) request item. This comes in very handy in various places (like sending email with item url, building custom views over the lists, restoring data etc).

To determine the library/list to which the request/item will be saved will be determined by the following algorithm.


Input: [itemid : integer]
step 1: set variable thresholdItems = 2000
step 2: var delta = itemid/(thresholdItems*thresholdItems)
step 3: delta = Math.Ceiling(delta);
Output: “Repo”+ delta

Sample Runs:

Input Output
2000 Repo1
6500 Repo1
2300300 Repo1
4000000 Repo1
4000001 Repo2

As we can observe in the above table the until the item id is 4 million i.e 4000000 the library name is Repo1 and from 4000001 onwards the library name changed to Repo2. Similarily the next library Repo3 is required for the item id = 4000001 + 4000000 = 8000001. So on and on…

Note: We can utilize elevated privilege code to create the library with the required name on the fly whenever it is needed.

To get to the folder to which the item will be saved with in the library use the following algorithm.


Input: [itemid : integer]
step 1: set variable thresholdItems = 2000
step 2: var delta = itemid/(thresholdItems)
step 3: delta = Math.Ceiling(delta);
Output: delta

Sample Runs:

Input Output
1 1
837 1
2000 1
2001 2
3500 2
4000 2
4001 3

As you can see the folder name result changes for every 2000th item. So all the items with id between 1 and 2000 goes to the folder named “1”. All the items with id between 2001 and 4000 goes into folder named “2”. So on and on..

Note: once again you can always use elevated privilege code to create the folders on the fly as and when needed.

Below is the final outcome of this solution:

Since we now have the library name and folder name we can build the complete path/url easily. The format will be http://yoursitecollection.com/libraryname/foldername/itemname. Using this and wrapped between elevated privileges if required we can always upload the file as per the derived location.

Example:
http://yoursitecollection.com/Repo1/1/1.xml
http://yoursitecollection.com/Repo1/1/1500.xml
http://yoursitecollection.com/Repo1/1/2000.xml
http://yoursitecollection.com/Repo1/2/2001.xml
http://yoursitecollection.com/Repo1/2/3988.xml
http://yoursitecollection.com/Repo1/2000/4000000.xml
http://yoursitecollection.com/Repo2/1/4000001.xml
http://yoursitecollection.com/Repo2/1/4001390.xml and so on and on….

I will soon provide the code to create the library/folder on the fly and also the code to upload the item to the derived location.

Remove/Hide/Customize InfoPath Forms Tool Tip

On your sharepoint front end servers, go to the following location
C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\TEMPLATE\LAYOUTS\INC\

Make a backup copy of ifsmain.css.
Then edit the ifsmain.css file as follows:
Search (ctrl + F) for the css class “.errorDiv”, it is something like below:

To hide/remove tool tip:
.errorDiv
{
z-index:100;position:absolute;top:0px;left:0px;display:none;width:300px;padding:2px 3px; border:1px solid  #B22828;background:#FFFED7;color:#B22828;

font-family:Verdana; font-size:x-small; text-decoration:none;font-weight:normal;

}

Modify the above script by adding “!important” just beside display:none. The modified .errorDiv is as follows:

.errorDiv
{
z-index:100;position:absolute;top:0px;left:0px;display:none !important;width:300px;padding:2px 3px;
border:1px solid #B22828;background:#FFFED7;color:#B22828;

font-family:Verdana;
font-size:x-small;
text-decoration:none;font-weight:normal;
}

You can further customize the tool tips as per your needs if that is your requirement instead of hiding it.

The only drawback with this approach is that it will affect the complete server farm. Thus, all the infopath forms on all the sites will be affected with this change.

Update (12/1/09):

Good news is that we were able to figure out how to customize the tool tip specific to a site or page level thus avoiding the drawback mentioned in the above method of implementation. Using a content editor web part you can insert the required Html (also CSS) into a page. Leveraging this technique we were able to add a content editor web part (CEWP) to our custom application page which hosts the infopath form. This content editor web part envelopes the CSS changes required to the tool tip. Below is the CEWP content we added to our page:

<WebPartPages:ContentEditorWebPart runat=”server” __MarkupType=”xmlmarkup” WebPart=”true” __WebPartId=”{87D17157-ECD0-49D6-9906-2DAA4C400992}”>
<WebPart xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance&#8221; xmlns:xsd=”http://www.w3.org/2001/XMLSchema&#8221; xmlns=”http://schemas.microsoft.com/WebPart/v2″&gt;
<Title>Content Editor Web Part</Title>
<FrameType>None</FrameType>
<Description>Use for formatted text, tables, and images.</Description>
<IsIncluded>true</IsIncluded>
<PartOrder>1</PartOrder>
<FrameState>Normal</FrameState>
<Height />
<Width />
<AllowRemove>true</AllowRemove>
<AllowZoneChange>true</AllowZoneChange>
<AllowMinimize>true</AllowMinimize>
<AllowConnect>true</AllowConnect>
<AllowEdit>true</AllowEdit>
<AllowHide>true</AllowHide>
<IsVisible>true</IsVisible>
<DetailLink />
<HelpLink />
<HelpMode>Modeless</HelpMode>
<Dir>Default</Dir>
<PartImageSmall />
<MissingAssembly>Cannot import this Web Part.</MissingAssembly>
<PartImageLarge>/_layouts/images/mscontl.gif</PartImageLarge>
<IsIncludedFilter />
<ExportControlledProperties>true</ExportControlledProperties>
<ConnectionID>00000000-0000-0000-0000-000000000000</ConnectionID>
<ID>g_87d17157_ecd0_49d6_9906_2daa4c400992</ID>
<ContentLink xmlns=”http://schemas.microsoft.com/WebPart/v2/ContentEditor&#8221; />
<Content xmlns=”http://schemas.microsoft.com/WebPart/v2/ContentEditor”>&lt;![CDATA[<style type=”text/css”>
.errorDiv
{
z-index:100;position:absolute;top:0px;left:0px;display:none;width:200px;padding:2px 3px;border:1px solid #B22828;background:#FFFED7;color:#B22828;font-family:Arial;
font-size:xx-small;
text-decoration:none;font-weight:normal;
}
.errorDivClickable
{
z-index:100;position:absolute;top:0px;left:0px;display:none;width:300px;padding:2px 3px;
border:1px solid #B22828;background:#FFFED7;color:#B22828;
font-family:Arial;
font-size:xx-small;
cursor:pointer;cursor:hand;font-weight:normal;
}
</style>]]></Content>
<PartStorage xmlns=”http://schemas.microsoft.com/WebPart/v2/ContentEditor&#8221; />
</WebPart>
</WebPartPages:ContentEditorWebPart>

InfoPath Form HyperLink Open in Same Window

Problem: The hyperlink control in a browser enabled infopath form opens up in a new window. This might not be a good thing as a usability perspective. It might work for some and might not for many. This post is for those ‘many’.

Solution:

Note: This solution is valid only for the forms with managed code behind.

The solution I have is very trivial. Just drop a button on the form, stylize it to change its look and feel like an hyperlink. Then on the button click event of the form, just redirect the page to the URL you wanted to. There is a catch or two here.

A) On the click event, you make sure that the existing web request is completed before redirecting to your desierd page. So the code should look something like this:

HttpContext.Current.Response.Redirect(“http://somesite.com/page.html&#8221;,  false);

Passing the second parameter as ‘false’ we are ensuring that the current web request is completed and then redirected.

B) To make the button to be a hyperlink you need to do the following:

Right click the button select ‘borders and shading’, select None under presets. Then under the ‘shading’ tab select the radio button for ‘No color’. Then change the font color to blue(hyperlinks are usually blue) and make it underlined.

After doign all this there is still one thing that remains, i.e the mouse turning into a hand on hover over the button. To accomplish this, from file menu, select save as source files. From the source files, edit the view1.xls file, to set the style to cursor:pointer for the button element. Then open up the manifest.xsf in design mode. You will see the button as an hyper link including mouse hover.

Update (09/03/09):

There is a kind of caveat for the above implementation. Whenever the Response.Redirect is done, the new page from code behind of the InfoPath Form, eventhough the page is redirected, the URL in the browser address bar remains the same. It behaves as the Server.Transfer method.

To fix this anomaly the workaround is redirect to an intermediary page which in turn redirects to the actual page you want to redirect. The intermediary page could be a simple page with java script redirect method.

Encrypting connectionstring or any section in web.config (Codesmith, Nettiers)

We are using NetTiers templates with Codesmith tool to generate the code base for our database. The autogenerated code base uses the SQL connection string in clear text format. We realized that it is a showstopper only after our deployment team pointed it out.  To our surprise we were able to find a quick solution without having to change or edit the autogenerated code. Since NetTiers uses Microsoft Enterprise library to generate the code the inherent code uses the EntLib dataaccess layer thus paving a way to an easy implementation of encryption.

OK enough of the story, below are the steps to follow:

Go to http://www.orcsweb.com/articles/aspnetmachinekey.aspx and generate a machine key. The machine key looks something like below:

<machineKey validationKey=’9A4A4ACEB1C1B352050B73FF641FACD00756C125E256C170ACA2F307059E00D05FA3BDA8BA2BB432D88C6912E5A9D33E0A2EC55AA272959CEA075E81660D9B4B’ decryptionKey=’F1E5AFF42211D3503158EFA109704EE756EFDD996010966F’ validation=’SHA1’/>

Copy this machine key text to web.config under the system.web tag.

<?xml version=”1.0″?>
<configuration>
    <appSettings/>
    <connectionStrings/>
    <system.web>
        <machineKey validationKey=’9A4A4ACEB1C1B352050B73FF641FACD00756C125E256C170ACA2F307059E00D05FA3BDA8BA2BB432D88C6912E5A9D33E0A2EC55AA272959CEA075E81660D9B4B’ decryptionKey=’F1E5AFF42211D3503158EFA109704EE756EFDD996010966F’ validation=’SHA1’/>
    </system.web>
</configuration>

Add your connection string which has to be encrypted to the connectionStrings section.
At the command prompt, change the directory to the .NET Framework version 2.0 directory by typing the following command:

cd \WINDOWS\Microsoft.Net\Framework\v2.0.*

Run the following commands:
If your web application is hosted as a website on IIS then go with:

aspnet_regiis -pe “connectionStrings” -app “/MyApplication”
aspnet_regiis -pe “system.web/machineKey” -app “/MyApplication”

If your web application is a project in your visual studio and you have the path the website folder then go with:

aspnet_regiis -pef “connectionStrings” “C:\trunk\Dev\SampleWebApp”
aspnet_regiis -pef “system.web/machineKey” “C:\trunk\Dev\SampleWebApp”

 That is it! if you open web.config you will see the section encrypted. If you are using Enterprise library application blocks for your data access layer then you dont have to do anything else to make your project understand the encrypted stuff.

 In any other case, please read http://msdn.microsoft.com/en-us/library/dtkwfdky(VS.80).aspx for more details on how to use the encrypted stuff in your application.

SharePoint Solutions Manager or The WSP Manager

Tool Description
Here is a nifty tool which can be used to deploy, redeploy, upgrade and retract/delete the solutions in the SharePoint environment. It handles all these jobs using the object model. This tool will be definitely handy for developers for quick deployment while debugging.

Purpose
The best way to get anything or any component into SharePoint environment is by creating a WSP aka Solution file. Many tools exists to create the WSP like WSPBuilder, STSDEV and Andrew Connell’s way of doing it. Having done that the next step would be deploying the same into SharePoint environment. WSPBuilder supports deploying directly though Visual Studio but some times it could get messy. It also limits to the project you set up using WSPBuilder. This tool is built in a generic way which can be used to manage already deployed solutions and also for deploying new solutions. So given a WSP file this tool can help you get that into SharePoint environment.

I agree there is another tool SharePoint Solution Installer on codeplex but I never was happy with that as it didn’t serve the purpose many a times for me. Moreover that tools was closed to administrators.

ShareTools Solution Manager does many things which can be done with stsadm command line tool. Below are a salient list of those:

  • View existing solutions in our server environment
  • Add new solutions
  • Upgrade existing solutions either immediately or at a given time
  • Redeploy an existing solution if you have a latest WSP of the same solution
  • Retract and Delete the existing solution
  • Deploy a new solution with all the parameters used in stsadm command line tool
  • Execute the administrative timer job definitions i.e stsadm.exe -o execadmsvcjobs with a single click of the button

Currently you can take action on a single item at a time.

You can download it from: http://solutionmanager.codeplex.com/