Data Caching Techniques in ASP.NET 1.x and 2.0

This article looks at the different ways that you can boost performance of your applications by caching the data they use. While not aimed at beginners, rather at developers who have a reasonable grasp of ASP.NET already, it does discuss the available techniques both at high level, and in detail through examples. The article demonstrates the ways that the various options can be used, and points to sources of more information. You don’t need to be an ASP.NET expert to benefit from the content or the techniques described, yet this article will also appeal to more experienced developers who have not yet experimented with new features in ASP.NET 2.0 such as SQL data cache invalidation.

Note: the ASP.NET version 2.0 code described in this article is based on the Beta 1 release of the .NET Framework v2.0. All other code is based on the final V1.1 release of the .NET Framework. All the examples can be downloaded from http://www.daveandal.net/articles/aspnet-caching/, and you can also run many of them online from the same location. If you install them on your own server, you may have to edit the connection strings in the web.config file, depending on the location of your database. There is a menu page named default.htm in each of the two folders in the samples (one for ASP.NET v1.1 and one for v2.0 Beta1).

One unfortunate outcome of the rapid move away from stand-alone and client/server applications, to the Web-based environment where the majority of our applications have to live today, is the disconnected nature of the HTTP protocol. In particular, this affects the way that the individual pages of an application handle data - especially when there is a requirement to store data that is either expensive to create (in terms of resource and processing usage), or which is used regularly throughout the life of the application.

Since version 1.0, ASP has provided techniques to help you maintain state information and cache data for an application, as the Application and Session objects that it exposes. In ASP.NET 1.0, another headline feature was added in the shape of the application-level Cache object (complete with its support for output caching of ASP.NET-generated HTML), which is especially useful in that it supports features to expire and to invalidate items and pages stored in the cache. And now, in ASP.NET version 2.0, there are even more new features that make using the Cache object and output caching an even more compelling solution.

In this article, we overview the primary techniques available in ASP.NET for caching values and data between user requests, and then go on to concentrate on the Cache object and output caching - with particular emphasis on the new features in version 2.0. We'll look at:

Choosing the right approach for caching data
Application, Session and viewstate data caching
Global caching of data using disk files
The ASP.NET output caching features
Using the ASP.NET Cache object directly

To start with, let's consider how you should choose a caching technology based on the benefits of the commonly-available techniques.

Choosing a Caching Technology

The most important feature when choosing one of the available caching technologies is their visibility. In other words, where do you need to be able to access the cached data? If it is just for the life of a page, or over repeated post-backs from a particular page, you can take advantage of the viewstate of that page - basically just a dictionary object that can store string values, and which is encoded into a hidden control on the page. Or you can use output caching to cache some, or all, of the HTML that is generated for a page - so that the page does not have to be re-executed for every request. And, in version 2.0 of ASP.NET, you can take advantage of the ability of most of the new data source controls to cache their data.

However, to store data that is required to be accessible across more than one page, you need an alternative technique. If you only need to make the values available for a specific user, you can use the Session object to store values and data. As long as the user's browser supports cookies, or you turn on cookie-less session support, data in the user's Session store will be available from any page they open while the session is active.

If you want to make your data visible to all the users of an application, you must use either the ASP.NET Application object or the Cache object. The advantage with the Cache object is that you can program expiration rules and manage the lifetime of the data more accurately. Of course, you can use the Application and Cache objects for user-specific data as long as you manage the identification of each instance of the data, probably through unique or user-specific cache key names.

Finally, what about if you want to make cached data available to all users and all applications on a server, or even on a networked group of servers? The only real answer here is some sort of disk or memory persistence that is implemented and managed outside of ASP.NET. In fact this is reasonably easy to do, using the classes implemented in the .NET Framework class library for reading and writing streams (including memory streams) and disk files. In general, disk-based storage is the better option for longer persistence periods, and can survive server restarts, application failures and most other events that would cause other forms of cached data to be lost.

Table 1 shows the features of each of the techniques we've briefly mentioned here, including a note about the volume of data that each is designed to cope with. For example, the viewstate of a page is transmitted over the network twice with each page request, and so should be kept to a minimum. And if you have a great many concurrent users, even small volumes of data stored in each user's Session can swallow up valuable memory on the server unless you configure alternative Session support features such as the ASP.NET State Store or storage in a central SQL Server database.

Table 1 - The visibility for various caching techniques

Technique	Visibility	Optimum data size
Page viewstate	Page	Small
Data source controls	Page	Large
Page output caching	Page	Large
ASP Session	User	Small/Medium
ASP Application	Application	Medium
ASP.NET Cache	Application	Large
Disk file	Global	Large

One important point to note here is that you don't have to choose the most restrictive visibility for your cached data. For example, disk-based storage - perhaps as an XML file - would be an ideal solution for data that is expensive to retrieve from its source and yet changes infrequently (particularly if accessed only occasionally by users). But what if each user requires a different set of data? You could still cache it using some user-specific value in the filename, perhaps the Session ID or user login name. The only downside is that you have to manage refreshing and removing stale data yourself.

The ASP.NET Application, Session and Viewstate

Using the Application and Session objects in ASP.NET is generally just like in "classic" ASP, except that you should be aware of data typing issues. It's always a good idea to cast or convert values to the correct type when you extract them:

Dim iThisValue As Integer = 42

Application("MyInteger") = iThisValue

Dim iThatValue As Integer = CType(Application("MyInteger"), Integer)

or, in C#:

int iThisValue = 42;

Application["MyInteger"] = iThisValue;

int iThatValue = (int) Application["MyInteger"];

Alternatively, you can use the methods exposed by the HttpApplicationState class (from the System.Web namespace), which is the class used to implement the Application object in ASP.NET. These methods include Add, Clear, Get, Remove, RemoveAt and Set, plus properties and methods to retrieve information about the keys for the stored items: AllKeys, Count, GetKey, etc.

In ASP.NET, there is some automatic locking of the Application (and Session) to prevent concurrent updates to the values stored there causing inconsistent results. This is done through thread locking in ASP.NET itself, however if you are updating values in the Application it is worth using the Lock and Unlock methods as you would have done in previous versions of ASP:

Application.Lock()

Dim iThisValue As Integer = CType(Application("MyInteger"), Integer)

iThisValue += iMyIncrement

Application("MyInteger") = iThisValue

Application(UnLock)

You aren’t limited to storing just simple intrinsic .NET value types in the Application (or Session). You can store any class instance that is serializable, for example a DataSet (in version 2.0, the DataTable is also serializable). As an example, this code stores a DataSet in the Application:

Application("MyDataSet") = dsMyData

Dim dsRetrievedData As DataSet = CType(Application("MyDataSet"), DataSet)

ASP.NET Sessions

The Session object in ASP.NET is implemented by the HttpSessionState class, which is defined within the System.Web.SessionState namespace. There is no Lock or Unlock method for the Session, though you can use the IsSynchronized and SyncRoot properties to create your own thread-safe implementations if you wish. However, for general use, the Session works just like the Application in respect of reading and storing values.

Remember that the Session exposes some useful properties like IsCookieless and IsNewSession, which you may find useful. But the most important feature in ASP.NET, in terms of improving page execution efficiency, is the implementation of read-only sessions. A whole series of events are raised as an ASP.NET page is being loaded and executed, and two of these are concerned with loading and saving Session data. As a page loads, an event causes ASP.NET to search the Session for any values that apply to the page, and load those it finds into the execution memory space for the page. As execution of the page ends, another event causes ASP.NET to write these values back to the Session store.

So it's pretty obvious that you can reduce page execution overheads by avoiding the two event calls if you don’t need to read or update the session data. And of you only need to read it, but not update it, you can specify read-only access to the Session. To specify that a page does not require access to the user's session, add this directive:

<%@Page EnableSessionState="False" ... %>

To allow session data to be read but not updated, thus avoiding the expensive process of updating the session data at the end of the page, use this directive:

<%@Page EnableSessionState="ReadOnly" ... %>

Note that there is also a property named IsReadOnly on the HttpSessionState class that you can query in your page code to see if the current session is read-only. You can also specify the session behavior at machine or application level using the <pages> element within the <system.web> section of machine.config or web.config:

ASP.NET Viewstate

If you only need to cache a value or a small set of data for an individual page between post-backs, you can use the ASP.NET viewstate. When an ASP.NET page contains a server-side HTML form (a <form> section that is declared with the runat="server" attribute), an HTML <hidden> control is included in the generated HTML page that contains the values of the controls on the page, and other state information, in an encoded format.

You can add values to the viewstate, and retrieve them, using the same approach as with the Application and Session objects:

Dim iThisValue As Integer = 42

Viewstate("MyInteger") = iThisValue

Dim iThatValue As Integer = CType(Viewstate("MyInteger"), Integer)

or, in C#:

int iThisValue = 42;

Viewstate["MyInteger"] = iThisValue;

int iThatValue = (int) Viewstate["MyInteger"];

Just bear in mind that your data passes across the network with each page request and each postback, and so this is definitely not an ideal approach for storing large volumes of page-specific data. A better solution is to store some key value in the viewstate, and then use this key to store and retrieve the real data from a database, a disk file, or one of the other types cache better suited to storing large volumes of data.

As with sessions, you can disable the storage of viewstate for a page. Viewstate is enabled by default, and even when disabled some control and state information is still persisted. However, there is no concept of a "read-only viewstate". In a Page directive, you can use:

<%@Page EnableViewState="False" ... %>

and in machine.config or web.config you can use:

Caching Data as a String

A useful point to bear in mind whenever you store data that is not a String value, especially when you are dealing with more complex classes that persist their data internally (such as a DataSet) is that most support a method that exports the content as a String. For example, with a DataSet (and, in version 2.0, a DataTable), you can use the WriteXml method to write the data as an XML document, which is itself just a text string. You can also provide the writeMode parameter (a value from the XmlWriteMode enumeration) that specifies if a schema is included, and whether you want a simple XML representation of the data or a diffgram that stores change information as well.

The following code shows how a DataSet can be serialized as an XML document and stored in the viewstate of the page, and then re-instantiated as a DataSet as each postback occurs:

Dim dsMyData As New DataSet()

If Page.IsPostBack Then

Dim reader As New StringReader(CStr(ViewState("MyDataSet")))

dsMyData.ReadXml(reader)

Else

... code to fill DataSet from source data store here ...

Dim writer As New StringWriter()

dsMyData.WriteXml(writer)

ViewState("MyDataSet") = writer.ToString()

End If

* Note that the StringWriter class is implemented in the System.IO namespace, so you'll need to import this into your page.

The samples you can download for this article (from http://www.daveandal.net/articles/aspnet-caching/) contain an example of caching the contents of a DataSet within the page viewstate, as an XML String value. The write-viewstate.aspx example uses the code listed above, and the result in shown in Figure 1. When the page is first loaded, data is fetched from the database. As you refresh the page, the DataSet is rebuilt each time from the XML cached in the viewstate.

Figure 1 - Writing a DataSet to the Viewstate as XML and Reading it Back into a DataSet

Caching Data as a Disk File

The .NET Framework provides plenty of easy-to-use classes for reading and writing disk files, and these can be used when you need to cache data that is required to be available in more than one ASP.NET application. As an example, suppose you want to cache a DataSet containing data that all applications require access to. You could use the technique just described of calling the WriteXml and ReadXml methods, which work automatically with a disk file - all you have to do is provide the path and filename.

However, you can also use a BinaryFormatter to write data from objects that are serializable to a Stream. Here we demonstrate the use of a FileStream with a DataSet, but you could just as easily cache the data in memory as using a MemoryStream if required. Why bother with this when the DataSet can serialize itself as XML? Well, the one extra feature here revolves around the fact that, in version 2.0, the DataSet gains a new property named RemotingFormat. This can be set to a value from the SerializationFormat enumeration (the two options are Xml or Binary). When set to Binary, the volume of data generated from a DataSet is generally between 20% and 80% less. This means smaller disk files, faster loading, and ultimately better performance:

' create BinaryFormatter and Stream

Dim f As IFormatter = New BinaryFormatter()

Using s As New FileStream(datPath, FileMode.Create, FileAccess.Write, FileShare.None)

' specify Binary remoting format for the DataSet (new in v 2.0)

dsMyDataSet.RemotingFormat = SerializationFormat.Binary

' serialize the contents to the Stream as binary data

f.Serialize(s, dsMyDataSet)

s.Close()

End Using

And to rebuild the DataSet from the disk file, you can use this code:

' now de-serialize the data back into a new DataSet

Dim dsMyDataSet As DataSet

Dim f As IFormatter = New BinaryFormatter()

Using s As New FileStream(datPath, FileMode.Open, FileAccess.Read, FileShare.Read)

' de-serialize file contents into a DataSet

dsMyDataSet = CType(f.Deserialize(s), DataSet)

s.Close()

End Using

* This code takes advantage of the Using construct, which is available in C# in v1.1 and is now introduced into VB.NET in v 2.0.

Figure 2 shows the example we provide (serialize-to-disk.aspx) that uses this code to serialize a DataSet to a disk file, and then de-serialize it back into a DataSet again. The page also displays some rows from the DataSet to prove that it worked!

Figure 2 - Serializing a DataSet to a Disk File, and De-serializing it Back into a DataSet

Many objects support the ISerializable interface, and so can be used with a BinaryFormatter as shown above (though the actual format of the persisted data will vary with the object type). Suitable classes include collection types such as HashTable and NameValueCollection, the ADO.NET DataSet and DataTable, and the Image class that is used to create and store bitmap images.

ASP.NET Output Caching

One extremely useful way to boost performance in ASP.NET when users regularly load the same page, without the content being modified by server-side script to provide a different version of the page for each request, is to use output caching. This is implemented within ASP.NET, and is seamless as far as the developer is concerned. All that you have to do is specify a directive in the ASP.NET page, and the server will automatically cache the HTML that is generated from the page in line with the conditions you specify.

The simplest output caching directive is:

<%@OutputCache Duration="#seconds" VaryByParam="None" />

This instructs ASP.NET to cache the HTML for the specified number of seconds, irrespective of the parameters that are sent with the page request from the client. "Parameters" in this sense means any values in the Request collections (Form, QueryString, Cookies and ServerVariables). While the page is cached, every request will result in the HTML generated the last time that the page was executed being sent to the client. In IIS 6.0, in Windows Server 2003, this detection of cached pages is actually done inside the kernel-level HTTP redirector module, and so it is blindingly fast - quicker than in IIS 5.0 and, of course, a great deal faster that re-executing the page.

There are many alternative ways to configure output caching in ASP.NET. The OutputCache directive provides a range of options:

<%@ OutputCache Duration="#seconds"

Location="[Any|Client|Downstream|Server|None]"

Shared="[true|false]"

VaryByControl="control-name(s)"

VaryByParam="[None|*|parameter-name(s)]"

VaryByHeader="header-name(s)"

VaryByCustom="[Browser|custom-string"] %>

where the attribute values are as defined in Table 2.

Table 2 - The Attributes of the OutputCache directive

Attribute	Description
Duration	Required in every ASP.NET OutputCache directive. The number of seconds that the generated HTML will be cached.
Location	Defines if page output can be cached on a proxy or downstream client. Often omitted, so that caching is performed in the most efficient way. However, it is a good idea to include Location="Any" where you do not actually need to control the location.
Shared	Used only in a User Control. Defines if multiple instances of the control will be served from one cached instance. If False (the default) multiple instances will be cached.
VaryByControl	Used only in a User Control. Declares the ID of ASP.NET controls that the cache will be varied by. For example: VaryByControl="MyTextBox;MyListBox" will cause a new version of the page to be cached for each differing value in the specified controls within the user control.
VaryByParam	Required in every ASP.NET OutputCache directive. Defines the names of the parameters that will be used to determine if a cached version of the page will be delivered. For example: VaryByParam="txtName;lstSelection" will cause a new version of the page to be cached for each differing value in the QueryString, Form, Cookies or ServerVariables collections that has one of the specified names.
VaryByHeader	The name(s) of any HTTP Header(s) that are used to determine if a cached version of the page will be delivered. For example: VaryByHeader="USER-AGENT;SERVER;REMOTE-HOST"
VaryByCustom	The special value "Browser" specifies that the cached version of the page will only be delivered to browsers with the same name and major version: VaryByCustom="Browser". Alternatively, you can place a custom function in global.asax that returns a String value based on some criteria you require, and the page will be varied for each client that causes a different value to be returned. For example: VaryByCustom="MyGlobalFunction" (see the following note for an example of this).

Remember that ASP.NET varies the HTML it generates based on the browser type ("uplevel" - meaning Internet Explorer 5 and above; or "downlevel" - meaning all the rest). Therefore you might want to consider including the VaryByCustom="Browser" attribute if your page uses ASP.NET controls that can generate browser-specific output. Examples are controls such as Menu and Panel. The way that style information is specified for most other Web Forms controls also changes depending on the browser type, which may affect the appearance. You can experiment with these effects using a sample page from the book "Pro ASP.NET 1.1 Web Forms Techniques", see: http://www.daveandal.net/books/7868/control-compatibility/default.aspx. The asp-cache-by-level option shows how custom "vary by" functions can be implemented, and we have included the function code within the samples for this article.

You can use the example page named output-cache-params.aspx (see Figure 3) to experiment with output caching. As the VaryByParam attribute is set to "*", all the controls on the page will invalidate the cached HTML copy, and so changing the value in any one will cause the page to be re-executed (the time it was last executed is shown in the page). However, refreshing the page within the ten second cache duration period specified in the OutputCache directive does not cause it to be re-executed.

Figure 3 - Using Output Caching with the VaryByParam="*" Attribute

One interesting point demonstrated in this example is that using the ASP.NET auto-postback feature causes the page to be re-executed more than once. For example, a postback is initiated automatically when you change the value in the second drop-down list. And, as the values in the controls have changed, this causes the cache to be invalidated and the page is re-executed. However, if you then click the Refresh button, the page is executed again - rather than being served from the output cache. This is because the name and caption of the button are included in the parameters that are posted back to this page when the button is clicked, whereas they are not included when the postback is caused by the drop-down list that has AutoPostback="True".

Partial Page Caching (Fragment Caching)

ASP.NET also supports partial caching of a page, sometimes referred to as fragment caching. To use this, you create sections of the page as user controls, containing the Control directive and with the .ascx file extension. Inside each user control you can specify an OutputCache directive (which can specify the ID of any control(s) it contains in a VaryByControl attribute, or use a VaryByParam attribute):

<%@OutputCache Duration="30" VaryByControl="MyListBox" />

However, when output caching is specified for the page hosting the user control, the whole page is cached based on the Duration specified in the hosting page, and so any user controls it includes will only be re-executed when the page itself expires. For this reason, the Duration attribute in the user control(s) should be the same as, or a multiple of, the Duration attribute value in the hosting page.

You can see the effects of partial page caching in the example page named output-cache-fragment.aspx. It hosts a simple user control that contains a drop-down list box. As you can see from Figure 4, the output cache duration for the user control is ten seconds, while the duration for the hosting page is only five seconds. As you refresh the page, the execution times show that the page is re-executed every five seconds, while the user control is only re-executed every ten seconds. However, the hosting page contains the VaryByParam="*" attribute in its OutputCache directive, and so any change to the selected value in the list box - even though it is within the user control - will cause both the hosting page and the user control to be re-executed the next time the page is refreshed.

Figure 4 - Partial Page Caching with a User Control

If your user control generates output that is not specific to one page, or if you use multiple instances in the same page and they do not need to be varied individually, you can reduce output cache requirements by including the Shared attribute:

<%@OutputCache Duration="60" VaryByControl="MyListBox" Shared="True" />

Alternatively, if you create user controls as Class files, you can use an attribute to specify partial caching. In VB.NET the attribute is <PartialCaching(#seconds)>, for example:

<PartialCaching(120)> _

Public Class MyControl

Inherits UserControl

... implementation here ...

End Class

In C#, the equivalent syntax is:

[PartialCaching(120)]

public class MyControl : UserControl {

... implementation here ...

}

Configuring Output Caching in ASP.NET 2.0

ASP.NET 2.0 adds new capabilities to the output cache feature, as well as supporting new configuring options for ASP.NET pages and user controls in the machine.config and web.config files. There is now a <caching> section where you can define the operation of output caching and SQL database cache invalidation (a topic we'll be looking at in more depth later). This is the overall layout of the <caching> section:

<system.web>

<diskCache enabled="[true|false]" path="directory-path"

maxSizePerApp="size-in-MB" />

</outputCache>

<add name="identifying-string" enabled="[true|false]" duration="#seconds"

diskCacheable="[true|false]" shared="[true|false]"

location="[Any|Client|Downstream|Server|None]"

varyByControl="control-name(s)"

varyByParam="[None|*|parameter-name(s)]"

varyByHeader="header-name(s)"

varyByCustom="[Browser|custom-string"] />

</outputCacheProfiles>

</outputCacheSettings>

<add name="identifying-string" connectionStringName="name-string"

pollTime="#milliseconds" />

</databases>

</sqlCacheDependency>

</caching>

</system.web>

The <cache> element defines the settings for memory-based caching and whether cached items will be expired automatically when the server requires the memory or disk space they are occupying (see the section "Cache Memory and Automatic Invalidation").

The <outputCache> section specifies the global setting for output caching, whether it is enabled, and where and whether cached items can be saved to disk. Disk caching is enabled by default, and the maximum disk usage per application is set to 5 MB.

The <outputCacheSettings> section is used to define individual output caching profiles that you can inherit in your pages, so that you don’t have to edit the OutputCache directive in every page (see the section "Output Cache Profiles in ASP.NET 2.0").

The <sqlCacheDependency> section defines the databases and connection details that are used for the new automatic SQL data cache invalidation feature in ASP.NET 2.0. We look at this feature and its usage in the section "Automatic SQL Data Cache Invalidation".

Cache Memory and Automatic Invalidation

It's important to remember that the ASP.NET cache may only hold on to items while there is enough memory available, without compromising operation of the server. If you continue to add items to the cache once free cache memory is exhausted, the oldest, least used or lowest priority items are invalidated and removed. So, just because you cached an item doesn’t mean to say that you can guarantee it will still be there if you specified a priority other than the default.

So it's a good idea to test for the presence of an item before or as you retrieve it from the ASP.NET Cache. You can also use cache dependencies and an event handler (as demonstrated later on) to detect when items are removed from the cache, though this is not always possible in a Web application as the events are not likely to be fired when your ASP.NET page is executing and can handle them.

Output Cache Profiles in ASP.NET 2.0

If you regularly use the same sets of values in OutputCache directives across different ASP.NET pages, you can set up cache profiles in ASP.NET version 2.0. Simply declare the profile in the <outputCacheProfiles> section of machine.config or web.config using an <add> element. Within the <add> element, specify the attributes you want for this profile - for example:

</outputCacheProfiles>

Then, in any ASP.NET page or user control, you just specify this profile in your OutputCache directive:

<%@ OutputCache CacheProfile="MyPageProfile" %>

The example page named output-cache-profile.aspx demonstrates the use of output caching profiles. When you run the page, it looks just like the example in Figure 3. However, as you can see if you view the source for the page (use the [view source] link that is at the bottom of every example page, as shown in Figure 5), you will see that the OutputCache directive it uses is that shown above.

Figure 5 - Using a CacheProfile in the OutputCache Directive

Automatic SQL Data Cache Invalidation

One of the major advances in the caching technology in ASP.NET 2.0 is support for SQL cache invalidation. This allows you to retrieve or generate a rowset, usually as a DataSet or DataTable, and then hang on to it until the original source data changes. This removes the traditional performance trade-off where you need to refresh the data (i.e. invalidate the cached rowset) regularly enough to see updated values in the data source, but hang on to each cached copy long enough to get the required reduction in resource usage and retrieval time when reconstructing the rowset.

Now, as long as your database is SQL Server 7.0, 2000 or 2005, you simply set up a dependency for the database and table(s) you are using, fetch and cache the rowset, and then keep using the cached copy until the database informs you that the source data has changed. This can obviously provide a huge performance boost, with the corresponding reduction in response times and database/server resource usage.

Configuring SQL Cache Invalidation

In SQL Server 7.0 and SQL Server 2000, you have to pre-configure the database to support SQL cache invalidation, using a utility that is provided with version 2.0 of the .NET Framework. This utility, named aspnet_regsql.exe, is in the %windir%\Microsoft.NET\Framework\[version] folder of your machine.

Configuring the database involves two steps:

Enable the database you are using for change notifications, which creates the change notification table and adds the required stored procedures:

aspnet_regsql.exe -S server -U user -P password -d database -ed

Enable the table(s) containing the data you are using, which adds the necessary row to the change notification table and adds the required triggers to the source data table:

aspnet_regsql.exe -S server -U user -P password -d database -t table -et

If you are using a trusted connection to a local server, or a server on your network, you can replace the -S and -P parameters and values with just the empty parameter -E. Figure 1 shows change notification being configured on a SQL Server 200 database named "delboy", with the user id "anon", for the Northwind sample database and the table named Orders. Notice how the utility prompts for the password if you do not specify it in the command to execute aspnet_regsql.exe.

Figure 6 - Preparing SQL Server 2000 for SQL Cache Invalidation

Figure 2 shows the results. You can see the new table named AspNet_SqlCacheTablesForChangeNotification (so it's not likely to clash with the name of an existing table!), and in it the row for the Orders table. The lower section of the window shows the stored procedures that are used to implement the change notification.

Figure 7 - The tables and stored procedures added to SQL Server 2000 for SQL Cache Invalidation

SQL Server 2005 Change Notifications

While this technique does work with SQL Server 2005 ("Yukon"), you should prefer to use the built-in change notification service instead. The new Broker Service in SQL Server 2005 can be configured to monitor a command that has been executed, and which returns a rowset, and invalidate this automatically when any event occurs that could result in different values being returned if the same query was re-executed.

ASP.NET integrates with the Broker Service in SQL Server 2005, and can use the change notifications it produces to automatically invalidate output cached pages that use data generated through explicit code or the implicit commands it uses to extract data for the new data source controls. We'll look at this topic in the following sections of this article.

Data Caching with a Data Source Control

ASP.NET 2.0 introduces a range of new data source controls that can, in many common scenarios, remove the requirement to write data access code. Most of these controls also implement caching, plugging into the cache architecture automatically and providing an easy way to boost performance of your pages. As an example, the following code shows the attributes you can include when declaring a SqlDataSource control:

<asp:SqlDataSource id="identifier" runat="server"

ConnectionString="<%$ConnectionStrings:myconnectionstring%>"

SelectCommand="sql-statement-or-stored-proc-name"

DataSourceMode="[DataSet|DataReader]"

EnableCaching="[True|False]"

CacheDuration="#seconds"

CacheExpirationPolicy="[Absolute|Sliding]"

SqlCacheDependency="dependency-name:table-name">

</asp:SqlDataSource>

Simply by setting the EnableCaching attribute to True, and specifying the number of seconds to cache the data for the CacheDuration attribute, you automatically get the source data cached and reused when the page is refreshed during the specified duration period. However, note that this only works when DataSourceMode is DataSet. You can also use a Sliding expiration policy (the default is Absolute), so that the cached data is only invalidated after there has been no request during the specified cache duration period.

To demonstrate the advantage of SQL cache invalidation, we provide two examples that use a data source control. The first, which we'll look at here, is data-source-control.aspx. It uses just the EnableCaching and CacheDuration attributes of a SqlDataSource control to force the data to be cached for a finite period - in this example ten seconds. The page, shown in Figure 8, also contains a button that causes a simple server-side routine to force an update within the source database of the OrderDate column for the first row that is shown in the page (OrderID = 10248). This means that the data cached by the data source control is now out of date compared to the data in the source database row.

Figure 8 - Simple Finite Duration Caching in a SqlDataSource Control

Notice that the SQL statement executed in Figure 8 updated the OrderDate of the first row to the current date and time, but the value displayed in the GridView control still shows the value when the rowset was originally fetched from the database. It's only when the cache duration period expires that the GridView data is refreshed and the new value appears (see Figure 9).

Figure 9 - The Update Only Becomes Visible After the Cache Duration Period Expires

Configuring a Database for Caching in web.config

The SqlCacheDependency attribute in a data source control (and in an OutputCache directive) can be used to link cached rowset data to a SQL Server change notification. This solves the issue you met in the previous example, where the code and controls in the page are not aware that the source data has changed. SQL cache invalidation, implemented through change notifications, means that the page and controls can safely cache the data for long periods - but still be able to show the correct values - by re-executing the query only when the data changes.

As you saw earlier, SQL cache dependencies (for SQL Server 7.0 and 2000) are configured in machine.config or web.config, using the <sqlCacheDependency> section. You can clear from the list any databases that might be defined higher up in the configuration file hierarchy using the <clear /> element, remove individual databases with the <remove /> element, and add new ones using the <add /> element. In the following example, we add a database cache entry named "nwind-cache":

</databases>

</sqlCacheDependency>

Notice that we've set the default polling interval for our databases as two seconds, but then specified half a second for this particular entry (this is the minimum value you can use for the polling interval). Remember that you have to enable change notifications in the database for SQL Server 7.0 and 2000 using the aspnet_regsql.exe utility.

The connectionStringName attribute references the name of the connection string for this database, which must be declared in the <connectionStrings> section of this web.config file, or another configuration file higher up in the configuration hierarchy:

<add name="nwind"

connectionString="Data Source=localhost; Initial Catalog=Northwind;

Integrated Security=True"/>

</connectionStrings>

Using a SqlDependency with a Data Source Control

Now that a SQL cache dependency is configured, it can be used to enable change notification for a data source control (it can also be used in an OutputCache directive, as you'll see shortly). The following declaration of a SqlDataSource control uses the nwind-cache entry:

<asp:SqlDataSource id="ds1" runat="server"

ConnectionString="<%$ConnectionStrings:nwind%>"

SelectCommand="SELECT OrderID, OrderDate FROM Orders"

DataSourceMode="DataSet"

EnableCaching="True"

SqlCacheDependency="nwind-cache:Orders">

</asp:SqlDataSource>

If you want to base the cached data invalidation on more than one SQL cache dependency, you separate them with semi-colons, for example:

SqlDependency="nwind-cache:Orders;nwind-cache:Products"

The example page we provide, named data-source-sql2000.aspx, demonstrates the use of a data source control with a SQL cache dependency, as described above. Figure 10 shows the example page, along with the new window that you can open from it to perform updates in the source database. The EnableCaching="True" attribute in the data source control within main page forces it to cache the rowset indefinitely (we haven't specified a CacheDuration attribute), and so the same rowset is shown with each page refresh, without fetching it from the database again. However, the cached rowset data is invalidated automatically as the database is updated (using the button in the other browser window), and the next refresh of the main page will shown the new value.

Figure 10 - Using a SQL Server 2000Cache Dependency with a Data Source Control

Using a SqlDependency in an OutputCache Directive

The same SQL cache dependency used in the previous section with a data source control can be used directly in an OutputCache directive in an ASP.NET page. And the same syntax applies for using multiple dependencies:

<%@OutputCache Duration="#seconds" VaryByParam="None"

SqlDependency="dependency-name:table-name" />

<%@OutputCache Duration="#seconds" VaryByParam="None"

SqlDependency="dependency-name:table-name;dependency-name:table-name" />

Figure 11 shows the equivalent example to the one in Figure 10, but this time using the SqlDependency in an OutputCache directive within the page, rather than in the data source control. You can see that the only visible difference is that this example shows the time that the page was executed, so that you can confirm that it is output cached until the values in the source database table are changed by the other window.

Figure 11 - Using a SQL Server 2000Cache Dependency in an OutputCache Directive

You can also create database cache dependencies based on SQL Server 7.0 or SQL Server 2000 directly in code, using the SqlCacheDependency class added to the .NET Framework in version 2.0. Insert an OutputCache directive into the page that specifies only the caching duration:

<%@OutputCache Duration="#seconds" VaryByParam="None" />

Then in the Page_Load event (or wherever required) create a new SqlCacheDependency instance and set this dependency on the output cache:

' database defined in Web.config as "nwind-cache",

' dependency on Orders table

Dim dep As New SqlCacheDependency("nwind-cache", "Orders")

Response.AddCacheDependency(dep)

Using SQL Server 2005 Change Notifications

In SQL Server 2005, the Broker Service provides change notification automatically. You can create a dependency on a Command instance that generates the data for your page and attach it to the output cache in a similar way:

' for a SqlCommand instance named myCommand,

' previously created in code

Dim dep As New SqlCacheDependency(myCommand)

Response.AddCacheDependency(dep)

You can also take advantage of dependencies in SQL Server 2005 using the special cache identifier value "CommandNotification" in your data source controls:

The ASP.NET Cache API

The final section of this article look briefly at how you can use the Cache API provided with ASP.NET and the .NET Framework. Working with the Cache object in its simplest form is similar to the Session and Application objects. However, the Cache provides a lot more opportunities to store and retrieve data, and manage the way that it is prioritized and expired.

The simplest approach uses the key name directly, though - as with the Session and Application objects - you should cast the returned value back to the correct data type:

Cache("key-name") = value-to-cache

retrieved-value = CType(Cache("key-name"), object-type)

If the key name does not match an entry in the cache, it returns Nothing (null in C#). Remember that items may be ejected from the cache if memory or disk constraints demand it, and so you should test for the presence of a value and re-create it if it is not available - for example:

Dim iMyValue As Integer = CType(Cache("myintvalue"), Integer)

If iMyValue Is Nothing Then

' re-create the item

iMyValue = {your code here to create or fetch the value again}

' and add to cache

Cache("myintvalue") = iMyValue

End If

The Cache API also exposes two methods for adding items to the cache. The Insert method replaces any existing item with same key:

Cache.Insert("key-name", value-to-cache)

The Add method fails if an item with same key already exists in the cache, and returns the existing item instead. If the item does not exist, it is added to the cache and the method returns Nothing (null in C#):

cached-item = Cache.Add("key-name", value-to-cache)

Working with Cache Dependencies

Items added to the Cache can be dependent on range of factors:

One or more disk files or folders
A fixed or sliding "last accessed" time span
An absolute date and/or time for expiration
Other cache dependencies (chained expiration)
Individual item priorities when the cache is purged

You can also detect when an item is removed from the cache using a callback, by handling the CacheItemRemoved event. The full list of parameters for the Add and Insert methods of the Cache object is shown in Table 2, indicating how you specify the dependencies for an item when adding it to the Cache.

Table 2 - The Parameters for the Add and Insert Methods of the Cache Class

Parameter	Description
key	A String that is the key name for the item in the cache.
value	An Object that contains the value to be inserted into the cache.
dependencies	A reference to a CacheDependency instance. This can be a new instance based on any of the factors described in the bullet list above (the following sections detail the technique for creating these dependencies). Alternatively, it can be an existing dependency - so that when the existing dependency causes its cached data to be invalidated, the current data will also be invalidated.
absoluteExpiration	A DateTime value that defines the time that the cached data will be invalidated.
slidingExpiration	A TimeSpan that defines the length of time that the cached data is valid for.
priority	A CacheItemPriority value that controls how the item will be handled if the cache becomes full and items must be ejected automatically. The CacheItemPriority enumeration contains the values Default, High, AboveNormal, Normal, BelowNormal and Low to specify the priority and allow items to be purged when available memory runs out, and the value NotRemovable (the default) to indicate that the item should not be removed.
onRemoveCallback	A reference to a CacheItemRemovedCallback instance, which will be executed when this item is invalidated and removed from the cache.

For example:

Cache.Insert("key", value, Nothing, DateTime.Now.AddMinutes(10), _

TimeSpan.Zero, CacheItemPriority.High, Nothing)

Creating a CacheDependency

The dependencies parameter for the Add and Insert methods of the ASP.NET Cache object, shown in Table 2, is a reference to a CacheDependency instance that you create in your code. There are several constructors exposed by the CacheDependency class, as shown in Table 3. These allow you to create a dependency that will invalidate the cached item in response to a range of events.

Table 3 - The Constructors for the CacheDependency Class

Constructor	Details
CacheDependency()	Creates a CacheDependency with no dependent objects.
CacheDependency("file/folder-path")	Creates a CacheDependency that will invalidate the cache when the specified file or folder (a String value) changes.
CacheDependency(array-of-file/folder-paths)	Creates a CacheDependency that will invalidate the cache when any of the specified files or folders (an array of String values) changes.
CacheDependency("file/folder-path", DateTime)	Creates a CacheDependency that will invalidate the cache when the specified file or folder changes, or at the specified time.
CacheDependency(array-of-file/folder-paths, DateTime)	Creates a CacheDependency that will invalidate the cache when any of the specified files or folders changes, or at the specified time.
CacheDependency(array-of-file/folder-paths, array-of-cache-keys)	Creates a CacheDependency that will invalidate the cache when any of the specified files or folders changes, or any of the specified cache items (as an array of String values containing the cache keys) changes.
CacheDependency(array-of-file/folder-paths, array-of-cache-keys, DateTime)	Creates a CacheDependency that will invalidate the cache when any of the specified files or folders changes, or any of the specified cache items changes, or at the specified time.
CacheDependency(array-of-file/folder-paths, array-of-cache-keys, CacheDependency)	Creates a CacheDependency that will invalidate the cache when any of the specified files or folders changes, or any of the specified cache items changes, or the specified CacheDependency causes its data to be invalidated.
CacheDependency(array-of-file/folder-paths, array-of-cache-keys, CacheDependency, DateTime)	Creates a CacheDependency that will invalidate the cache when any of the specified files or folders changes, or any of the specified cache items changes, or the specified CacheDependency causes it's data to be invalidated, or at the specified time.

Chaining Dependencies

Using the constructors for the CacheDependency class, shown in Table 3, you can create chains