Accelerating Internet Connectivity through the ISA Server Cache

ISA Server Cache Summary

The Web Proxy service of ISA Server enables frequently requested Web objects to be cached. Subsequent client requests are checked against the ISA Server cache to determine whether they can be served from the cache. A new request is initiated if the ISA Server cache cannot be utilized to serve the client request. ISA Server stores most frequently accessed items in RAM. The items are then retrieved from memory instead of from disk.

Objects remain in the ISA Server cache until either of the following events occurs:

* A more up to date version of the cached object is obtained.
* Space is required for other more recently requested objects.

ISA Server caching can be implemented using either of these methods:

* Forward Web caching
* Reverse Web caching
* Scheduled caching
* Distributed caching
* Hierarchical caching (chaining) Accelerating Internet Connectivity through the ISA Server Cache

When you configure ISA server as a forward Web caching server, internal clients are able to access the Internet, and ISA Server maintains a cache of frequently requested Internet objects which can be accessed by any Web browser behind the firewall. Client browser performance is improved because using the cache results in less processing than requesting objects from the Internet. Bandwidth usage on Internet connections remains low. User response time is decreased as well.

With reverse caching, objects requested from internal servers by external clients are stored on the ISA server. Incoming Web requests are forwarded to the ISA server and are served from the cache. Requests are only forwarded to the Web server when the cache cannot be used to serve the request.

You can use the ISA Server Scheduled Content Download job feature to download the content to the ISA Server cache as per a predefined schedule. You can proactively ensure that important content is always available directly from the ISA Server cache and that the information is current.

You can directly download the following to the ISA Server cache:

* A single URL
* Multiple URLs
* Entire Web site

You can also limit which content should be downloaded. You can define scheduled content downloading for outgoing Web requests and for incoming Web requests.

The ISA Server Enterprise Edition uses Cache Array Routing Protocol (CARP) to provide scaling and improve efficiency. When you install multiple ISA Server computers, ISA servers are automatically installed in arrays. The array of ISA servers is then treated as a single logical cache. A hashing algorithm determines the location for storage, and hash-based routing is used to retrieve the location of the stored object when requests are made.

The routing algorithm used by CARP works as follows:

* The array membership list is used to track the status of the servers. The array membership list is updated using the TTL countdown which detects all active servers.
* A hash function is calculated for the name of each particular server.
* A hash function is also calculated for the name of each particular URL requested.
* The hash values of each of the above calculations are joined.
* The owner of the information cache is the highest value derived from combining the server hash value and the hash value of the URL.
* Subsequent requests for the cached information will use the same location.

The ISA Server Enterprise Edition enables administrators to arrange individual ISA Server computers or arrays of ISA Servers hierarchically. This process is also referred to as chaining. Requests are then sent upstream through the chain of servers until the object which was requested is located. Chaining provides fault tolerance. Content can then be distributed to multiple locations without making requests on the Internet.

The different server roles that you can configure for ISA Server are:

* Dedicated firewall
* Secure Publishing server
* Forward Web cache server
* Reverse Web cache server
* Integrated Firewall and Web Cache server

The features provided by Cache mode are:

* Access policy (for HTTP only)
* Alerts
* Cache configuration
* Enterprise policy
* Real-time monitoring
* Reports
* Web publishing

When determining ISA Server cache placement, consider the following important factors:

* You should use a hard disk volume formatted with NT file system (NTFS).
* You should use a local drive and not a network drive.
* You need at least 5MB to install the ISA Server cache.
* The ISA Server installation drive and the cache drive should be separated. This will improve performance.
* You can configure multiple drives to contain part of the ISA Server cache.

How ISA Server Caching Works

When ISA Server needs to determine whether to retrieve objects from the Internet or from the ISA Server cache, it uses the following information:

* Settings configured on the Cache Configuration Properties dialog box: Configuration settings here include

Available memory for caching.
Expiration policy for objects in the cache.
Active caching policy.
Maximum size of objects in the cache.
Maximum size of Uniform Resource Locators (URLs) cached.

To access the Cache Configuration Properties dialog box;

1. Open the ISA Management console.
2. Navigate to the Cache Configuration node in the console tree.
3. Right-click the Cache Configuration node and then select Properties from the shortcut menu.

* Configured routing rules: Routing rules are used to define the following:

Specify when content should be retrieved from the ISA Server cache: When creating a new routing rule, you specify the following:

When ISA Server should use the ISA Server cache to retrieve the object to serve a request. You can configure ISA Server to retrieve any version of the object from the cache.
When ISA Server should route the request upstream to another server. You can alternatively configure ISA Server to never route requests.
When the request should be routed to the requested Web server.
You can specify that ISA Server should only retrieve an object from the cache when a version of the object that has not expired, exists in the cache.

Specify when the ISA Server should cache content. When creating a new routing rule, you set the following:

Whether retrieved objects should be stored in the ISA Server cache or not stored in the cache.
Whether retrieved objects and dynamic content should be stored in the ISA Server cache.
Whether objects should only be stored when the source and request headers specify caching.

The default routing rule allows Web Proxy client requests to be directly obtained from the Internet.

The size of the ISA Server cache is determined by the settings specified during ISA Server setup. The other settings that are enabled after ISA Server installation are:

* HTTP caching is enabled
* FTP caching is enabled
* Active caching is disabled.

When configuring routing rules, you can define routing for:

* All destinations
* All external destinations.
* A specific destination.
* All destinations other than a specific destination.

You can apply different routing rules to different destinations so that ISA Server caches content according to the destinations that you have specified.

When a user sends a request for an object, ISA Server first checks the ISA Server cache to determine whether or not the object exists in the cache. When a request is sent to an array of ISA servers, the Cache Array Routing Protocol (CARP) algorithm ascertains which ISA server's cache to check for the requested object.

When the object exists in the ISA Server cache, the following process occurs:

* ISA Server first determines whether the object is valid. It uses the following set of rules to decide whether objects are no longer valid:

When the Time to Live (TTL) defined in the source is expired.
When the Time to Live (TTL) defined in the Cache Configuration Properties dialog box has expired.
When the Time to Live (TTL) defined in the scheduled cache content download job is expired.

If the object in the cache is still valid, ISA Server retrieves the object from the cache and passes it to the user.

* When an object is no longer valid, ISA Server next checks the routing rule. The invalid object is passed to the user if you have configured ISA Server to retrieve any version of the object from the cache.
* When the routing rule defines that the request should be routed, ISA Server then determines:

Whether the request should be routed to an upstream server.
Whether the request should be routed to the Web server.

* When the routing rule is set to route the request to a Web server, ISA Server checks whether the particular Web server can be accessed.
* If the Web server can be accessed, ISA Server then determines whether the requested object can be stored in its cache. The cache properties of the routing rule is checked to determine this. If the response should be cached, ISA Server caches the response and returns the requested object to the user.
* When the routing rule is set to route the request to a Web server, but ISA Server cannot access the specific Web server, ISA Server then ascertains whether or not expired objects should be returned to the user. If so, the expired object is passed to the user.

For all objects that do not exist in the cache, ISA server uses the configuration of the routing rule to determine the way in which to route the request.

How to configure when ISA Server caches objects

ISA Server caches objects to:

* RAM: The default ISA Server configuration caches objects that are below 12,800 bytes in size in RAM. An object that is stored in RAM is retrieved faster by ISA Server than an object which is stored on disk.
* Disk: The ISA Server default configuration caches objects that are bigger 12,800 bytes in size on disk.

You can use the settings on the Cache Configuration Properties dialog box (Advanced tab) to change the size of objects which gets stored in memory.

Responses to user requests that include the HTTP response headers listed below are not cached by ISA Server:

* cache-control: no-cache
* cache-control: private
* pragma: no-cache
* set-cookie
* www-authenticate

Responses to user requests that include the HTTP request headers listed below are not cached by ISA Server:

* authorization; only allowed when the origin server contains the cache-control: public header in the response.
* cache-control: no-store

The settings that you can configure on the Cache Configuration Properties dialog box allow you to more precisely specify when objects should be cached.

You can configure which content to cache:

* Objects that have surpassed a predefined size limit.
* Objects that have no last modification date specified.
* Objects that contain a question mark in the URL.
* Objects that have no 200 response specified.

With regard to ISA Server and HTTP object caching, you can choose between different expiration policies. An object is considered expired when its Time to Live (TTL) setting has expired. ISA Server can though continue to use an expired object for a predefined time period when the source of the expired object is not available.

The different expiration policies you can define for HTTP object caching on the HTTP tab of the Cache Configuration Properties dialog box are:

* Frequently: Objects stored in the ISA Server cache are more up to date or current because ISA Server retrieves objects from the Internet site frequently. The downside of enabling this setting is the possible negative impact on network performance.

The Time to Live (TTL) period is set as 0.

* Normally: The default setting is Normally. Objects stored in the ISA Server cache are updated with consideration to network performance.
* Less frequently: Objects stored in the ISA Server cache are not as current. There is less of a negative impact on network performance because ISA Server does not regularly retrieve objects from the Internet.
* Set custom Time to Live (TTL) settings: These settings allow administrators to manually define the TTL setting for objects.

With regard to ISA Server and FTP object caching, you can specify whether caching should be enabled, and set the TTL period for all FTP objects.

ISA Server includes an active caching feature. When you enable the active caching feature, ISA Server checks which objects in its cache are frequently accessed, and then, when these objects almost reach expiration, ISA Server automatically updates the objects.

The different active caching policies you can choose between are listed here:

* Frequently: Frequently accessed objects in the ISA Server cache are refreshed prior to the objects expiring.
* Normally: The default setting is Normally. Frequently accessed objects stored in the ISA Server cache are updated with consideration to network performance.
* Less frequently: Frequently accessed objects in the ISA Server cache are updated somewhat, but network performance is considered more important. This setting places active caching at its minimum level. There may be instances when a current, more up to date object does not exist in the ISA Server cache.

You can also configure ISA Server to perform negative caching. When negative caching is enabled, ISA Server caches the error response received when it is unable to retrieve a requested object from the Web server.

HTTP objects that have the status codes listed here can be cached:

* Status code 203
* Status code 300
* Status code 301
* Status code 410

Scheduling ISA Server to Download Content Automatically

You can configure and schedule ISA Server to download content automatically, to ensure that the content in the cache is always available and current.

You can schedule ISA Server to download:

* Entire Web sites.
* A Web page.
* Multiple Web pages.

You can configure scheduled content download jobs for:

* Incoming Web requests.
* Outgoing Web requests.

You can use scheduled cache content download jobs to speed up the performance of your internal Web server for Internet clients. Here, you can configure ISA server to automatically download the Web site of your internal Web server to the cache, so that content can be retrieved from ISA Server directly and not from the internal Web server itself. This basically means that your Internet clients will no longer directly access your internal Web servers.

The scheduled cache content download feature has the following advantages

* Saves network bandwidth.
* Improves cache performance.
* Secure internal Web servers.

The New Scheduled Content Download Job Wizard is used to enable and configure the scheduled cache content download feature.

To launch the New Scheduled Content Download Job Wizard, use the steps below:

1. Open the ISA Management console.
2. Navigate to the Scheduled Content Download Jobs folder by expanding the Cache Configuration node.
3. Right-click Scheduled Content Download Jobs, and select New and then select Job from the shortcut menu.
4. The New Scheduled Content Download Job Wizard appears.
5. Follow the various pages and prompts of the New Scheduled Content Download Job Wizard to create a new scheduled cache content download job.

How to create a routing rule

Retrieve them directly from specified destination
Route to specified upstream server
Redirected to hosted site
Use dial-up entry

10. Click Next
11. On the Cache Retrieval Configuration page, you have to define how this routing rule searches for and retrieves objects from the cache. Click Next.
12. On the Cache Content Configuration page, specify whether and when objects should be stored in the ISA Server cache. Click Next.
13. Click Finish.
14. The new routing rule should be displayed in the details pane.

How to configure a routing rule that always caches content

1. Open the ISA Management console.
2. Navigate to the Routing folder.
3. Right-click the folder and select New and then Rule from the shortcut menu.
4. The New Routing Rule Wizard launches.
5. In the Name box enter the name of the routing rule.
6. In the Description box, enter a description for the routing rule.
7. Click Next.
8. When the Destination Sets page opens, specify the destination or the destination set and then click Next.
9. On the Request Action page, you have to specify how client requests should be routed. Select the Retrieve them directly from specified destination option. Click Next.
10. On the Cache Retrieval Configuration page, you have to specify how your new routing rule searches for and retrieves objects from the ISA Server cache. Select the A Valid Version Of The Object; If None Exists, Retrieve The Request Using The Specified Requested Action option and then click Next.
11. On the Cache Content Configuration page, specify whether and when objects should be stored in the cache. Select the All Content, Including Dynamic Content, Will Be Cached option and then click Next.
12. Click Finish.
13. The new routing rule should be displayed in the details pane.

How to configure cache property settings for a routing rule

1. Open the ISA Management console.
2. Navigate to the Routing folder and expand the Routing folder.
3. Right-click the routing rule that you want to configure and then select Properties from the shortcut menu.
4. The Routing Rule Properties dialog box opens.
5. Click the Cache tab.
6. In the Search Cache For area of the Cache tab, select one of the following settings:

If you want the object retrieved from the cache only when the version of the object is valid (not expired), select the A Valid Version Of The Object; If None Exists, Retrieve The Request Using The Route Defined On The Action Tab option. If the object is invalid, ISA Server will request the object from an upstream server.
If you want ISA Server to always retrieve the object from the cache, irrespective of whether the object has expired or not, select the Any Version Of The Object; If None Exists, Retrieve The Request Using The Route Defined On The Action Tab. If the requested object does not exist in the ISA Server cache, then ISA Server will request the object from the upstream server.
If you only want the object retrieved from the cache – never routed, select the Any Version Of The Requested Object In The Cache, Never Route The Request option. ISA Server will not route the request.

7. In the Cache Content area of the Cache tab, select one of the following settings:

If ISA Server should always cache all content, as well as dynamic content, select the All Content, Including Dynamic Content, Will Be Cached option.
If ISA Server should cache content only when the source and request headers specify caching, choose the If Source And Request Headers Indicate To Cache, Then The Content Will Be Cached option.
If no content should be cached, choose the No Content Will Ever Be Cached option.

8. Click OK.

How to configure an ISA Server chain

1. Open the ISA Management console.
2. Navigate to the Routing folder.
3. Select the Routing folder.
4. Right-click the default routing rule and select Properties from the shortcut menu.
5. The Default Rule Properties dialog box opens.
6. Click the Action tab.
7. Select the Routing Them to a Specified Upstream Server option.
8. Click the Settings button associated with the Primary Route.
9. The Upstream Server Setting dialog box opens.
10. Select the ISA server and change the URL if applicable.
11. Enable the Use This Account checkbox and then select the account to utilize for authentication.
12. Select either Basic authentication or Integrated Windows authentication.
13. Click OK.
14. Use the same process to configure the Backup route.

How to specify how routing rules retrieve and serve requests

1. Open the ISA Management console.
2. Navigate to the Routing folder and expand the Routing folder.
3. Right-click the routing rule that you want to configure and then select Properties from the shortcut menu.
4. The Routing Rule Properties dialog box opens.
5. Click the Action tab. This is where you configure the routing rule action, or how requests are retrieved:

If you want the object retrieved from the Internet, select the Retrieving Them Directly From The Specified Destination option.
If you want an upstream server to action the request, select the Routing Them to A Specified Upstream Server option.

If you want the request to be routed to a different computer choose the Redirecting Them to A Hosted Site option:

Set the site.
Set the port.
Set the SSL port.

6. If you want to use a dial-up connection as a primary route for the request, select the Use Dial-up Entry For Primary Route checkbox.
7. If you want to use a dial-up connection as a backup route for the request, select the Use Dial-up Entry For Backup Route checkbox.
8. Click OK.

How to configure the destination for a routing rule

1. Open the ISA Management console.
2. Navigate to the Routing folder and expand the Routing folder.
3. Right-click the routing rule that you want to configure and select Properties from the shortcut menu.
4. The Routing Rule Properties dialog box opens.
5. Click the Destinations tab.
6. Select one of the following options:

* All Destinations
* All External Destinations
* All Internal Destinations
* Selected Destination Set
* All Destinations Except Selected Set

7. If you selected the Selected Destination Set option, or the All Destinations Except Selected Set option, choose the destination set in the Name drop-down list box.
8. Click OK.

How to change which drives are used to store the ISA Server cache

1. Open the ISA Management console.
2. Navigate to the Cache Configuration node in the console tree.
3. Expand the Cache Configuration node.
4. Select Drives.
5. Right-click the server that you want to set the size of the cache for and then select Properties from the shortcut menu.
6. Select the NTFS drives which are used to store the ISA Server cache and then specify how much disk space is used on each drive.
7. Click OK.

How to set the size of the cache

1. Open the ISA Management console.
2. Navigate to the Cache Configuration node in the console tree.
3. Expand the Cache Configuration node.
4. Select Drives.
5. Right-click the server that you want to set the size of the cache for and then select Properties from the shortcut menu.
6. Select the NTFS drive.
7. Enter the size of the cache in the Maximum Cache Size (MB) textbox.
8. Click OK.

How to set the percentage of free memory to use for caching

1. Open the ISA Management console.
2. Navigate to the Cache Configuration node in the console tree.
3. Right-click the Cache Configuration node and then select Properties from the shortcut menu.
4. Click the Advanced tab.
5. Enter the percentage of free memory to use for caching in the Percentage Of Free Memory To Use For Caching textbox.
6. Click OK.

How to configure which content ISA Server should cache

1. Open the ISA Management console.
2. Navigate to the Cache Configuration node in the console tree.
3. Right-click Cache Configuration node and then select Properties from the shortcut menu.
4. The Cache Configuration Properties dialog box opens.
5. Click the Advanced tab.
6. If you want to place a limit on the size of objects that are cached, enable the Do Not Cache Objects Larger Than checkbox. Specify the unit – KB, MB, GB.
7. If you want ISA Server to cache objects that have no last modification date specified, then enable the Cache Objects That Have An Unspecified Last Modification Time checkbox.
8. If you want ISA Server to cache dynamic content, then choose the Cache Dynamic Content (Objects With Question Marks In The URL) checkbox.
9. Click OK.

How to configure HTTP object caching settings

1. Open the ISA Management console.
2. Navigate to the Cache Configuration node in the console tree.
3. Right-click the Cache Configuration node and then select Properties from the shortcut menu.
4. The Cache Configuration Properties dialog box opens.
5. Click the HTTP tab.
6. If you want objects stored in the ISA Server cache to be more current, select the Frequently (Expire Immediately) option.
7. If you want objects in the ISA Server cache to be current but with consideration to network performance, select the Normally option.
8. If network performance is more important than having up to date object information stored in the ISA Server cache, then select the Less Frequently (Reduced Network Traffic Is More Important) option.
9. If you want to manually define the TTL settings, choose the Set Time To Live (TTL) Of Object In Cache To option. You now have to configure the settings that define how ISA Server performs HTTP object caching.
10. If you want HTTP objects in the ISA Server cache to expire when a percentage of the content's age has expired, then select use This Percentage Of Content Age (Time Since Creation Or Modification) textbox to set this percentage.
11. If you want to set the minimum time period and maximum time period that HTTP objects remain in the cache, in the No Less Than textbox, enter the number and set the time unit. In the No More Than textbox, enter the number.
12. Next, set the time unit:

* Seconds
* Minutes
* Hours
* Days
* Weeks

13. Click OK.

How to configure FTP object caching settings

* Seconds
* Minutes
* Hours
* Days
* Weeks

9. Click OK.

How to configure ISA Server to retrieve expired objects from the ISA Server cache

1. Open the ISA Management console.
2. Navigate to the Cache Configuration node in the console tree.
3. Right-click the Cache Configuration node and then select Properties from the shortcut menu.
4. The Cache Configuration Properties dialog box opens.
5. Click the Advanced tab.
6. If you do not want ISA Server to return an expired object from its cache when the Web server cannot be accessed to obtain a current version of the object, select the Do Not Return The Expired Object (Return An Error Page) option.
7. If you want ISA Server to return the expired object from its cache, select the Return The Expired Object Only If Expiration Was option. You next have to define TTL settings which will determine whether the expired object is returned or not.
8. Set the maximum percentage of the TTL in the At Less Than This Percentage Of Original Time To Live textbox. The object will not be returned from the cache if the expiration time is greater than this setting.
9. Set the maximum amount of time which has elapsed since the object has expired in the But No More Than (Minutes) textbox. The object will not be returned from the cache if the expiration time is greater than this setting.
10. Click OK.

How to configure active caching settings

How to configure negative caching settings

How to configure ISA Server to download content automatically

1. Open the ISA Management console.
2. Navigate to the Scheduled Content Download Jobs node by expanding the Cache Configuration node.
3. Right-click Scheduled Content Download Jobs, and select New and then select Job from the shortcut menu.
4. The New Scheduled Content Download Job Wizard appears.
5. Enter a name for the new scheduled content download job in the Job Name textbox and then click Next.
6. Provide the date and time when the new scheduled download job should commence on the Start Time page. Click Next.
7. On the Frequency page, set how often content should be downloaded. Options include:

* Once
* Daily
* Weekly on: Set the day of the week.

8. Click Next.
9. On the Content page, type the URL that should be downloaded in the Download Content From This URL textbox.
10. Select the Content Only From URL Domain checkbox if you want to download only linked pages within the same site.
11. Select the Cache Dynamic Content checkbox if you want ISA Server to cache dynamic content. Click Next.
12. The Links And Downloaded Objects page opens. This is where you define how ISA Server handles the TTL of the objects which are downloaded.
13. If you want all downloaded objects to have the same TTL, then select the Always Override Object's TTL checkbox and enter the TTL setting in the available box.
14. If you want all downloaded objects to have the TTL which you set, but only when the objects do not have a defined TTL, then select the Override TTL If Not Defined checkbox.
15. Enable the Cache Up To Maximum Links Depth Of checkbox and then set how many links ISA Server should follow in the available box.
16. Alternatively, select the No Limit On Maximum Depth checkbox.
17. You can also specify the maximum number of objects that ISA Server caches when the scheduled download job runs in the Maximum Number Of Cached Objects box.
18. Click Next.
19. The Completing The Scheduled Content Download Job Wizard page is displayed.
20. Click Finish.
21. Your new scheduled content download job should appear in the details pane when you select the Scheduled Content Download Jobs folder.

How to define the location from where to download cache content

1. Open the ISA Management console.
2. Expand the Cache Configuration node.
3. Select the Scheduled Content Download Jobs folder.
4. All existing scheduled content download jobs are displayed in the details pane.
5. Right-click the scheduled content download job that you want to configure and select Properties from the shortcut menu.
6. Click the Parameters tab.
7. Enter the URL from where to download cache content in the Begin Downloading From URL textbox.
8. Select the Keep Download Process Inside URL Domain Only checkbox if you want to download only linked pages within the same site.
9. Enable the Cache Up To Maximum Links Depth Of checkbox and then set how many links ISA Server should follow in the available box.
10. Specify the maximum number of objects that ISA Server caches when the scheduled download job runs in the Maximum Number Of Cached Objects box.
11. Click OK.

How to configure the TTL settings for downloaded cache content

1. Open the ISA Management console.
2. Expand the Cache Configuration node.
3. Select the Scheduled Content Download Jobs folder.
4. All existing scheduled content download jobs are displayed in the details pane.
5. Right-click the scheduled content download job that you want to configure and select Properties from the shortcut menu.
6. Click the Parameters tab.
7. If you want all downloaded objects to have the same TTL, then select the Always Override Object's TTL checkbox and enter the TTL setting in the available box.
8. If you want all downloaded objects to have the TTL which you set, but only when the objects do not have a defined TTL, then select the Override TTL If Not Defined checkbox.
9. Click OK.

How to define a schedule for a scheduled content download job

1. Open the ISA Management console.
2. Expand the Cache Configuration node.
3. Select the Scheduled Content Download Jobs folder.
4. Right-click the scheduled content download job that you want to configure and select Properties from the shortcut menu.
5. Click the Frequency tab.
6. In the Date Calendar box and Time box, set the date and time for the schedule.
7. If you want content downloaded only once, select the Once option.
8. If you want content downloaded each day by ISA Server at the defined time, then select the Daily option.
9. If you want content downloaded on a specific day of the week at the set time, select the Weekly option. Set the day on which content should be downloaded.
10. Click OK.

How to cache dynamic content during scheduled content download jobs

1. Open the ISA Management console.

2. Expand the Cache Configuration node.
3. Select the Scheduled Content Download Jobs folder.
4. Existing scheduled content download jobs are displayed in the details pane.
5. Right-click the scheduled content download job that you want to configure and select Properties from the shortcut menu.
6. Click the Parameters tab.
7. Enable the Cache Dynamic Content checkbox.
8. Click OK.

Comments - No Responses to “Accelerating Internet Connectivity through the ISA Server Cache”