Caching
The WinINet functions have simple, yet flexible, built-in caching support. Any data retrieved from the network is cached on the hard disk and retrieved for subsequent requests. The application can control the caching on each request. For http requests from the server, most headers received are also cached. When an http request is satisfied from the cache, the cached headers are also returned to the caller. This makes data download seamless, whether the data is coming from the cache or from the network.
Applications must properly allocate a buffer in order to get the desired results when using the persistent URL caching functions. For more information, see Using Buffers.
Cache Behavior During Response Processing
The WinINet cache is compliant with the HTTP cache-control directives described in RFC 2616. The cache-control directives and application set flags determine what may be cached; however, WinINet determines what is actually cached based on the following criterion:
- WinINet only caches HTTP and FTP responses.
- Only well behaved responses may be stored by a cache and used in a reply to a subsequent Request. Well behaved responses are defined as responses that return successfully.
- By default, WinINet will cache successful responses unless either a cache-control directive from the server, or an application-defined flag specifically denote that the response may not be cached.
- In general, responses to the GET verb are cached if the requirements listed above are met. Responses to PUT and POST verbs are not cached under any circumstances.
- Items will be cached even when the cache is full. If an added item is puts the cache over the size limit, the cache scavenger is scheduled. By default, items are not guaranteed to remain more than 10 minutes in the cache. For more information, see the Cache Scavenger section below.
- Https is cached by default. This is managed by a global setting that cannot be overridden by application-defined cache directives. To override the global setting, select the Internet Options applet in the control panel, and go to the advanced tab. Check the "Do not save encrypted pages to disk" box under the "Security" section.
Cache Scavenger
The cache scavenger periodically cleans items from the cache. If an item is added to the cache and the cache is full, the item is added to the cache and the cache scavenger is scheduled. If the cache scavenger completes a round of scavenging and the cache has not reached the cache limit, the scavenger is scheduled for another round when another item is added to the cache. In general, the scavenger is scheduled when an added item puts the cache over its size limit. By default, the minimum time to live in the cache is set to 10 minutes unless otherwise specified in a cache-control directive. When the cache scavenger is initiated, there is no guarantee that the oldest items are the first to be deleted from the cache.
The cache is shared across all WinINet applications on the computer for the same user. Starting with Windows聽Vista and Windows Server聽2008 the cache size is set to 1/32nd the size of the disk, with a minimum size of 8MB and a maximum size of 50MB.
Using Flags to Control Caching
The caching flags allow an application to control when and how it uses the cache. These flags can be used alone or in combination with the dwFlags parameter in functions that access information or resources on the Internet. By default, the functions store all data downloaded from the Internet.
The following flags can be used to control caching.
| Value | Meaning |
|---|---|
| INTERNET_FLAG_CACHE_ASYNC | This flag has no effect. |
| INTERNET_FLAG_CACHE_IF_NET_FAIL | Returns the resource from the cache if the network request for the resource fails due to an ERROR_INTERNET_CONNECTION_RESET or ERROR_INTERNET_CANNOT_CONNECT error. This flag is used by HttpOpenRequest. |
| INTERNET_FLAG_DONT_CACHE | Does not cache the data, either locally or in any gateways. Identical to the preferred value, INTERNET_FLAG_NO_CACHE_WRITE. |
|
|
Indicates that this is a Forms submission. |
| INTERNET_FLAG_FROM_CACHE INTERNET_FLAG_FORMS_SUBMIT | Does not make network requests. All entities are returned from the cache. If the requested item is not in the cache, a suitable error, such as ERROR_FILE_NOT_FOUND, is returned. Only the InternetOpen function uses this flag. |
| INTERNET_FLAG_FWD_BACK | Indicates that the function should use the copy of the resource that is currently in the Internet cache. The expiration date and other information about the resource is not checked. If the requested item is not found in the Internet cache, the system attempts to locate the resource on the network. This value was introduced in Microsoft Internet Explorer 5 and is associated with the Forward and Back button operations of Internet Explorer. |
| INTERNET_FLAG_HYPERLINK | Forces the application to reload a resource if no expire time and no last-modified time were returned when the resource was stored in the cache. |
| INTERNET_FLAG_MAKE_PERSISTENT | No longer supported. |
| INTERNET_FLAG_MUST_CACHE_REQUEST | Causes a temporary file to be created if the file cannot be cached. This is identical to the preferred value, INTERNET_FLAG_NEED_FILE. |
| INTERNET_FLAG_NEED_FILE | Causes a temporary file to be created if the file cannot be cached. |
| INTERNET_FLAG_NO_CACHE_WRITE | Rejects any attempt by the function to store data downloaded from the Internet in the cache. This flag is necessary if the application does not want any downloaded resources to be stored locally. |
| INTERNET_FLAG_NO_UI | Disables the cookie dialog box. This flag can be used by HttpOpenRequest and InternetOpenUrl (HTTP requests only). |
| INTERNET_FLAG_OFFLINE | Prevents the application from sending requests to the network. All requests are resolved using the resources stored in the cache. If the resource is not in the cache, a suitable error, such as ERROR_FILE_NOT_FOUND, is returned. |
| INTERNET_FLAG_PRAGMA_NO_CACHE | Forces the request to be resolved by the origin server, even if a cached copy exists on the proxy. The InternetOpenUrl function (on HTTP and HTTPS requests only) and HttpOpenRequest function use this flag. |
| INTERNET_FLAG_RELOAD | Forces the function to retrieve the requested resource directly from the Internet. The information that is downloaded is stored in the cache. |
| INTERNET_FLAG_RESYNCHRONIZE | Causes an application to perform a conditional download of the resource from the Internet. If the version stored in the cache is current, the information is downloaded from the cache. Otherwise, the information is reloaded from the server. |
聽
Persistent Caching Functions
Clients that need persistent caching services use the persistent caching functions to allow their applications to save data in the local file system for subsequent use, such as in situations where a low-bandwidth link limits access to the data, or the access is not available at all.
The cache functions provide persistent caching and offline browsing. Unless the INTERNET_FLAG_NO_CACHE_WRITE flag explicitly specifies no caching, the functions cache all data downloaded from the network. The responses to POST data are not cached.
Using the Persistent URL Cache Functions
The following persistent URL cache functions allow an application to access and manipulate information stored in the cache.
| Function | Description |
|---|---|
| CommitUrlCacheEntryA | Caches data in the specified file in the cache storage and associates it with the given URL. |
| CommitUrlCacheEntryW | Caches data in the specified file in the cache storage and associates it with the given URL. |
| CreateUrlCacheEntry | Allocates the requested cache storage and creates a local file name for saving the cache entry that corresponds to the source name. |
| CreateUrlCacheGroup | Generates a cache group identification. |
| DeleteUrlCacheEntry | Removes the file associated with the source name from the cache, if the file exists. |
| DeleteUrlCacheGroup | Releases a GROUPID and any associated state in the cache index file. |
| FindCloseUrlCache | Closes the specified enumeration handle. |
| FindFirstUrlCacheEntry | Begins the enumeration of the cache. |
| FindFirstUrlCacheEntryEx | Begins a filtered enumeration of the cache. |
| FindNextUrlCacheEntry | Retrieves the next entry in the cache. |
| FindNextUrlCacheEntryEx | Retrieves the next entry in a filtered cache enumeration. |
| GetUrlCacheEntryInfo | Retrieves information about a cache entry. |
| GetUrlCacheEntryInfoEx | Searches for the URL after translating any cached redirections that would be applied in offline mode by HttpSendRequest. |
| ReadUrlCacheEntryStream | Reads the cached data from a stream that has been opened using RetrieveUrlCacheEntryStream. |
| RetrieveUrlCacheEntryFile | Retrieves a cache entry from the cache in the form of a file. |
| RetrieveUrlCacheEntryStream | Provides the most efficient and implementation-independent way of accessing the cache data. |
| SetUrlCacheEntryGroup | Adds or removes entries from a cache group. |
| SetUrlCacheEntryInfo | Sets the specified members of the INTERNET_CACHE_ENTRY_INFO structure. |
| UnlockUrlCacheEntryFile | Unlocks the cache entry that was locked when the file was retrieved for use from the cache by RetrieveUrlCacheEntryFile. |
| UnlockUrlCacheEntryStream | Closes the stream that has been retrieved using RetrieveUrlCacheEntryStream. |
聽
Enumerating the Cache
The FindFirstUrlCacheEntry and FindNextUrlCacheEntry functions enumerate the information stored in the cache. FindFirstUrlCacheEntry starts the enumeration by taking a search pattern, a buffer, and a buffer size to create a handle and return the first cache entry. FindNextUrlCacheEntry takes the handle created by FindFirstUrlCacheEntry, a buffer, and a buffer size to return the next cache entry.
Both functions store an INTERNET_CACHE_ENTRY_INFO structure in the buffer. The size of this structure varies for each entry. If the buffer size passed to either function is insufficient, the function fails and GetLastError returns ERROR_INSUFFICIENT_BUFFER. The buffer size variable contains the buffer size that was needed to retrieve that cache entry. A buffer of the size indicated by the buffer size variable should be allocated, and the function should be called again with the new buffer.
The INTERNET_CACHE_ENTRY_INFO structure contains the structure size, URL of the cached information, local file name, cache entry type, use count, hit rate, size, last modified time, expiration, last access, last synchronized time, header information, header information size, and file name extension.
The FindFirstUrlCacheEntry function takes a search pattern, a buffer that stores the INTERNET_CACHE_ENTRY_INFO structure, and the buffer size. Currently, only the default search pattern, which returns all cache entries, is implemented.
After the cache is enumerated, the application should call FindCloseUrlCache to close the cache enumeration handle.
The following example displays each cache entry's URL in a list box, IDC_CacheList. It uses MAX_CACHE_ENTRY_INFO_SIZE to initially allocate a buffer, since early versions of the WinINet API do not enumerate the cache properly otherwise. Later versions do enumerate the cache properly and there is no cache size limit. All applications that run on computers with the version of the WinINet API from Internet Explorer 4.0 must allocate a buffer of the required size. For more information, see Using Buffers.
int WINAPI EnumerateCacheOld(HWND hX)
{
DWORD dwEntrySize;
LPINTERNET_CACHE_ENTRY_INFO lpCacheEntry;
DWORD MAX_CACHE_ENTRY_INFO_SIZE = 4096;
HANDLE hCacheDir;
int nCount=0;
SendDlgItemMessage(hX,IDC_CacheList,LB_RESETCONTENT,0,0);
SetCursor(LoadCursor(NULL,IDC_WAIT));
dwEntrySize = MAX_CACHE_ENTRY_INFO_SIZE;
lpCacheEntry = (LPINTERNET_CACHE_ENTRY_INFO) new char[dwEntrySize];
lpCacheEntry->dwStructSize = dwEntrySize;
again:
hCacheDir = FindFirstUrlCacheEntry(NULL,
lpCacheEntry,
&dwEntrySize);
if (!hCacheDir)
{
delete[]lpCacheEntry;
switch(GetLastError())
{
case ERROR_NO_MORE_ITEMS:
TCHAR tempout[80];
_stprintf_s(tempout,
80,
TEXT("The number of cache entries = %d \n"),
nCount);
MessageBox(hX,tempout,TEXT("Cache Enumeration"),MB_OK);
FindCloseUrlCache(hCacheDir);
SetCursor(LoadCursor(NULL,IDC_ARROW));
return TRUE;
break;
case ERROR_INSUFFICIENT_BUFFER:
lpCacheEntry = (LPINTERNET_CACHE_ENTRY_INFO)
new char[dwEntrySize];
lpCacheEntry->dwStructSize = dwEntrySize;
goto again;
break;
default:
ErrorOut( hX,GetLastError(),
TEXT("FindNextUrlCacheEntry Init"));
FindCloseUrlCache(hCacheDir);
SetCursor(LoadCursor(NULL,IDC_ARROW));
return FALSE;
}
}
SendDlgItemMessage(hX,IDC_CacheList,LB_ADDSTRING,
0,(LPARAM)(lpCacheEntry->lpszSourceUrlName));
nCount++;
delete (lpCacheEntry);
do
{
dwEntrySize = MAX_CACHE_ENTRY_INFO_SIZE;
lpCacheEntry = (LPINTERNET_CACHE_ENTRY_INFO) new char[dwEntrySize];
lpCacheEntry->dwStructSize = dwEntrySize;
retry:
if (!FindNextUrlCacheEntry(hCacheDir,
lpCacheEntry,
&dwEntrySize))
{
delete[]lpCacheEntry;
switch(GetLastError())
{
case ERROR_NO_MORE_ITEMS:
TCHAR tempout[80];
_stprintf_s(tempout,
80,
TEXT("The number of cache entries = %d \n"),nCount);
MessageBox(hX,
tempout,
TEXT("Cache Enumeration"),MB_OK);
FindCloseUrlCache(hCacheDir);
return TRUE;
break;
case ERROR_INSUFFICIENT_BUFFER:
lpCacheEntry =
(LPINTERNET_CACHE_ENTRY_INFO)
new char[dwEntrySize];
lpCacheEntry->dwStructSize = dwEntrySize;
goto retry;
break;
default:
ErrorOut(hX,
GetLastError(),
TEXT("FindNextUrlCacheEntry Init"));
FindCloseUrlCache(hCacheDir);
return FALSE;
}
}
SendDlgItemMessage(hX,
IDC_CacheList,LB_ADDSTRING,
0,
(LPARAM)(lpCacheEntry->lpszSourceUrlName));
nCount++;
delete[] lpCacheEntry;
} while (TRUE);
SetCursor(LoadCursor(NULL,IDC_ARROW));
return TRUE;
}
Retrieving Cache Entry Information
The GetUrlCacheEntryInfo function lets you retrieve the INTERNET_CACHE_ENTRY_INFO structure for the specified URL. This structure contains the structure size, URL of the cached information, local file name, cache entry type, use count, hit rate, size, last modified time, expiration, last access, last synchronized time, header information, header information size, and file name extension.
GetUrlCacheEntryInfo accepts a URL, a buffer for an INTERNET_CACHE_ENTRY_INFO structure, and the buffer size. If the URL is found, the information is copied into the buffer. Otherwise, the function fails and GetLastError returns ERROR_FILE_NOT_FOUND. If the buffer size is insufficient to store the cache entry information, the function fails and GetLastError returns ERROR_INSUFFICIENT_BUFFER. The size required to retrieve the information is stored in the buffer size variable.
GetUrlCacheEntryInfo does not do any URL parsing, so a URL that contains an anchor (#) will not be found in the cache, even if the resource is cached. For example, if the URL聽"http://example.com/example.htm#sample" is passed, the function returns ERROR_FILE_NOT_FOUND even if "http://example.com/example.htm" is in the cache.
The following example retrieves the cache entry information for the specified URL. The function then displays the header information in the IDC_CacheDump edit box.
int WINAPI GetCacheEntryInfo(HWND hX,LPTSTR lpszUrl)
{
DWORD dwEntrySize=0;
LPINTERNET_CACHE_ENTRY_INFO lpCacheEntry;
SetCursor(LoadCursor(NULL,IDC_WAIT));
if (!GetUrlCacheEntryInfo(lpszUrl,NULL,&dwEntrySize))
{
if (GetLastError()!=ERROR_INSUFFICIENT_BUFFER)
{
ErrorOut(hX,GetLastError(),TEXT("GetUrlCacheEntryInfo"));
SetCursor(LoadCursor(NULL,IDC_ARROW));
return FALSE;
}
else
lpCacheEntry = (LPINTERNET_CACHE_ENTRY_INFO)
new char[dwEntrySize];
}
else
return FALSE; // should not be successful w/ NULL buffer
// and 0 size
if (!GetUrlCacheEntryInfo(lpszUrl,lpCacheEntry,&dwEntrySize))
{
ErrorOut(hX,GetLastError(),TEXT("GetUrlCacheEntryInfo"));
SetCursor(LoadCursor(NULL,IDC_ARROW));
return FALSE;
}
else
{
if ((lpCacheEntry->dwHeaderInfoSize)!=0)
{
LPSTR(lpCacheEntry->lpHeaderInfo)
[lpCacheEntry->dwHeaderInfoSize]=TEXT('\0');
SetDlgItemText(hX,IDC_Headers,
lpCacheEntry->lpHeaderInfo);
}
else
{
SetDlgItemText(hX,IDC_Headers,TEXT("None"));
}
SetCursor(LoadCursor(NULL,IDC_ARROW));
return TRUE;
}
}
Creating a Cache Entry
An application uses the CreateUrlCacheEntry and CommitUrlCacheEntry functions to create a cache entry.
CreateUrlCacheEntry accepts the URL, expected file size, and file name extension. The function then creates a local file name for saving the cache entry that corresponds to the URL and file name extension.
Using the local file name, write the data into the local file. After the data has been written to the local file, the application should call CommitUrlCacheEntry.
CommitUrlCacheEntry accepts the URL, local file name, expiration, last modified time, cache entry type, header information, header information size, and file name extension. The function then caches data in the file specified in the cache storage and associates it with the given URL.
The following example uses the local file name, created by a previous call to CreateUrlCacheEntry, stored in the text box, IDC_LocalFile, to store the text from the text box, IDC_CacheDump, in the cache entry. After the data has been written to the file using fopen, fprintf, and fclose, the entry is committed using CommitUrlCacheEntry.
int WINAPI CommitEntry(HWND hX)
{
LPTSTR lpszUrl, lpszExt, lpszFileName;
LPTSTR lpszData,lpszSize;
DWORD dwSize;
DWORD dwEntryType=0;
FILE *lpfCacheEntry;
LPFILETIME lpdtmExpire, lpdtmLastModified;
LPSYSTEMTIME lpdtmSysTime;
errno_t err;
if( SendDlgItemMessage(hX,IDC_RBNormal,BM_GETCHECK,0,0) )
{
dwEntryType = dwEntryType + NORMAL_CACHE_ENTRY;
}
else if( SendDlgItemMessage(hX,IDC_RBSticky, BM_GETCHECK,0,0) )
{
dwEntryType = dwEntryType + STICKY_CACHE_ENTRY;
}
else if(SendDlgItemMessage( hX,IDC_RBSparse, BM_GETCHECK,0,0) )
{
dwEntryType = dwEntryType + SPARSE_CACHE_ENTRY;
}
if( SendDlgItemMessage(hX,IDC_RBCookie, BM_GETCHECK,0,0))
{
dwEntryType = dwEntryType + COOKIE_CACHE_ENTRY;
}
else if( SendDlgItemMessage(hX,IDC_RBUrl, BM_GETCHECK,0,0) )
{
dwEntryType = dwEntryType + URLHISTORY_CACHE_ENTRY;
}
if( SendDlgItemMessage(hX,IDC_RBNone, BM_GETCHECK,0,0) )
{
dwEntryType=0;
}
lpdtmSysTime = new SYSTEMTIME;
lpdtmExpire = new FILETIME;
lpdtmLastModified = new FILETIME;
GetLocalTime(lpdtmSysTime);
SystemTimeToFileTime(lpdtmSysTime,lpdtmExpire);
SystemTimeToFileTime(lpdtmSysTime,lpdtmLastModified);
delete(lpdtmSysTime);
lpszUrl = new TCHAR[MAX_PATH];
lpszFileName = new TCHAR[MAX_PATH];
lpszExt = new TCHAR[5];
lpszSize = new TCHAR[10];
GetDlgItemText(hX,IDC_SourceURL,lpszUrl,MAX_PATH);
GetDlgItemText(hX,IDC_LocalFile,lpszFileName,MAX_PATH);
GetDlgItemText(hX,IDC_FileExt,lpszExt,5);
GetDlgItemText(hX,IDC_SizeLow,lpszSize,10);
dwSize = (DWORD)_ttol(lpszSize);
delete(lpszSize);
if (dwSize==0)
{
if((MessageBox(hX,
TEXT("Incorrect File Size.\nUsing 8000 characters, Okay?\n"),
TEXT("Commit Entry"),MB_YESNO))
==IDYES)
{
dwSize = 8000;
}
else
{
return FALSE;
}
}
lpszData = new TCHAR[dwSize];
GetDlgItemText(hX,IDC_CacheDump,lpszData,dwSize);
err = _tfopen_s(&lpfCacheEntry,lpszFileName,_T("w"));
if (err)
return FALSE;
fprintf(lpfCacheEntry,"%s",lpszData);
fclose(lpfCacheEntry);
delete(lpszData);
if ( !CommitUrlCacheEntry( lpszUrl,
lpszFileName,
*lpdtmExpire,
*lpdtmLastModified,
dwEntryType,
NULL,
0,
lpszExt,
0) )
{
ErrorOut(hX,GetLastError(),TEXT("Commit Cache Entry"));
delete(lpszUrl);
delete(lpszFileName);
delete(lpszExt);
delete(lpdtmExpire);
delete(lpdtmLastModified);
return FALSE;
}
else
{
delete(lpszUrl);
delete(lpszFileName);
delete(lpszExt);
delete(lpdtmExpire);
delete(lpdtmLastModified);
return TRUE;
}
}
Deleting a Cache Entry
The DeleteUrlCacheEntry function takes a URL and removes the cache file associated with it. If the cache file does not exist, the function fails and GetLastError returns ERROR_FILE_NOT_FOUND. If the cache file is currently locked or in use, the function fails and GetLastError returns ERROR_ACCESS_DENIED. The file is deleted when unlocked.
Retrieving Cache Entry Files
For applications that require the file name of a resource, use the RetrieveUrlCacheEntryFile and UnlockUrlCacheEntryFile functions. Applications that do not require the file name should use the RetrieveUrlCacheEntryStream, ReadUrlCacheEntryStream, and UnlockUrlCacheEntryStream functions to retrieve the information in the cache.
RetrieveUrlCacheEntryStream does not do any URL parsing, so a URL that contains an anchor (#) will not be found in the cache, even if the resource is cached. For example, if the URL聽"http://example.com/example.htm#sample" is passed, the function returns ERROR_FILE_NOT_FOUND even if "http://example.com/example.htm" is in the cache.
RetrieveUrlCacheEntryFile accepts a URL, a buffer that stores the INTERNET_CACHE_ENTRY_INFO structure, and the buffer size. The function is retrieved and locked for the caller.
After the information in the file has been used, the application should call UnlockUrlCacheEntryFile to unlock the file.
Cache Groups
To create a cache group, the CreateUrlCacheGroup function must be called to generate a GROUPID for the cache group. Entries can be added to the cache group by supplying the cache entry's URL and the INTERNET_CACHE_GROUP_ADD flag to the SetUrlCacheEntryGroup function. To remove a cache entry from a group, pass the cache entry's URL and the INTERNET_CACHE_GROUP_REMOVE flag to SetUrlCacheEntryGroup.
The FindFirstUrlCacheEntryEx and FindNextUrlCacheEntryEx functions can be used to enumerate the entries in a specified cache group. After the enumeration is complete, the function should call FindCloseUrlCache.
Handling Structures with Variable Size Information
The cache can contain variable size information for each URL stored. This is reflected in the INTERNET_CACHE_ENTRY_INFO structure. When the cache functions return this structure, they create a buffer that is always the size of INTERNET_CACHE_ENTRY_INFO plus any variable size information. If a pointer member is not NULL, it points to the memory area immediately after the structure. While copying the buffer returned by a function into another buffer, the pointer members should be fixed to point to the appropriate place in the new buffer, as the following example shows.
lpDstCEInfo->lpszSourceUrlName =
(LPINTERNET_CACHE_ENTRY_INFO) ((LPBYTE) lpSrcCEInfo +
((DWORD)(lpOldCEInfo->lpszSourceUrlName) - (DWORD)lpOldCEInfo));
Some cache functions fail with the ERROR_INSUFFICIENT_BUFFER error message if you specify a buffer that is too small to contain the cache entry information retrieved by the function. In this case, the function also returns the required size of the buffer. You can then allocate a buffer of the appropriate size and call the function again.
Note聽聽WinINet does not support server implementations. In addition, it should not be used from a service. For server implementations or services use Microsoft Windows HTTP Services (WinHTTP).
聽
聽
Send comments about this topic to Microsoft
]]>