HTTP Header Data

Data from ~15k pages from dmoz.org (therefore mostly HTML):

Most Common Headers

content-type	15348
date	15315
server	14757
content-length	10243
accept-ranges	8873
last-modified	8769
etag	8012
x-powered-by	5491
set-cookie	5105
transfer-encoding	4614
cache-control	3900
connection	3900
expires	2264
content-location	1547
vary	1406
pragma	1345
p3p	1000
microsoftofficewebserver	739
x-host	420
x-inkt-uri	420
x-inkt-site	420
x-cache	412
x-aspnet-version	387
x-pad	358
content-language	274
x-cache-lookup	237
x-server-ip	206
via	192
age	111
page-completion-status	68
x-pingback	64
filter-revision	63
iisexport	55
keep-alive	49
tcn	36
nncoection	33
mime-version	30
status	19
pics-label	16
x-creationtime	16
composed-by	15
proxy-connection	14
www-authenticate	14
window-target	11
microsoftsharepointteamservices	11
limerick	10
cneonction	10
x-server	9
srv	9
ms-author-via	8
x-spip-cache	8
x-back	7
x-hostname	7
x-servername	7
content	6
x-vortech-php	6
x-server-name	6
cluster-server	5
x-dip	5
refresh	5
x-gu-httpd	5
x-accelerated-by	5
commerce-server-software	5
x-cocoon-version	5
public-extension	4
x-perlbal	4
x-mod-choke	4
x-dmuser	4
resourcetag	4
x-track	4
application-name	4
x-varnish	4
x-tpsrv	4
x-responding-server	4
content-transfer-encoding	3
x-webserver	3
set-cookie2	3
x-ud-loopcounter	3
x-ud-target	3
x-ud-host	3
x-ud-method	3
from	3
x-ud-remote_addr	3
servername	3
cache-expires	3
x-handling	3
test-test	3
x-hacker	3
allow	3
content_length	2
x-highwire-sessionid	2
x-ips-cache	2
x-guru	2
cookieheader	2
x-cached-time	2
dcgi-server	2
x-via	2
http_host_was	2
host	2
cache-last-checked	2
proxy-agent	2
remote_addr_was	2
content-script-type	2
server_addr_was	2
x-n	2
x-content-parsed-by	2
warning	2
realserver	2
x-kiosk_expires	2
servlet-engine	2
server_name_was	2
x-cache-ttl	2
x-exectime	2
x-kexpinterval	2
uri	2
ppserver	1
	1
x-gahelper	1
srvname	1
copyright	1
cashe-control	1
portal-engine	1
0	1
wn_vars	1
x-bender	1
x-cache-hit	1
cachecontrol	1
x-atg-version	1
x-ibs-ccds-origin	1
req-id	1
gamesystem.com	1
served-by	1
x-caching-rule-id	1
message-id	1
charset	1
x-rehbein	1
x-lrecl	1
mirror	1
e-tag	1
version	1
matrix-server	1
slash_log_data	1
created-by	1
x-accelerator-vary	1
bim server	1
x-sfx-revision	1
x-instance-name	1
wn	1
content-md5	1
web	1
x-dave	1
emap-type	1
content-encoding	1
machinename	1
isp	1
x-machine-id	1
x-weblogic-cluster-list	1
x-coremedia-cacheable	1
ihs	1
x-site	1
x-zend-winenabler	1
delegate-ver	1
policyref	1
x-probase-spider-mirrorof	1
compression-control	1
character set '#33' is not a compiled character set and is not specified in the 'c	1
slb_server	1
x-cms-powered-by	1
x-content-type-warning	1
phone	1
x-cachettl	1
if-modified-since	1
hosted-with	1
x-recfm	1
x-storesense	1
x-ibs-ccds-version	1
imagetoolbar	1
x-ga-server-instance	1
communityserver	1
content location	1
x-vary	1
file 'c	1
x-fb-host	1
x-ruby-cluster-id	1
x-sfx-webhead	1
x-aipby-by	1
x-mrhost	1
author	1
hostname	1
x-cache-rules-applied	1
pagebuildtime	1
x-xrds-location	1
cbsid	1
x-glyphgate	1
x-r4l-vhost	1
x-id	1
south australian government	1
x-header-set-id	1
source	1
ignore__length	1
if-none-match	1
sn	1
modlayout	1
content-style-type	1

Most Common Content-Type Values

text/html	12419
text/html; charset=ISO-8859-1	671
text/html; charset=UTF-8	629
text/html; charset=utf-8	628
text/html; charset=iso-8859-1	373
text/html;charset=ISO-8859-1	104
text/html; charset=windows-1251	86
text/html;charset=iso-8859-1	62
text/html;charset=utf-8	42
text/html;charset=UTF-8	39
text/html;	26
application/pdf	20
text/html; charset=US-ASCII	19
text/html; charset=iso-8859-2	17
text/html; charset=ISO-8859-2	14
text/html; charset=WINDOWS-1251	11
text/html; charset=koi8-r	9
text/html; charset=ISO-8859-15	8
image/jpeg	7
text/html; Charset=UTF-8	7
text/html; charset=windows-1252;	6
text/html; charset=	6
text/html; charset=iso-8859-15	5
text/html; charset=EUC-JP	5
text/html; charset=iso-8859-9	4
text/xml	4
application/xml	4
text/html; charset=shift_jis	4
text/plain	4
text/html; Charset=utf-8	4
text/html;ISO-8859-1; charset=UTF-8	4
text/html; charset=utf-8;	4
text/html; charset=Big5	4
text/html; charset=latin1	3
text/html; charset=none	3
text/html; Charset=ISO-8859-1	3
text/html; Charset=windows-1255	3
text/html; charset=gb2312	3
text/html;charset=windows-1251	3
text/html; Charset=iso-8859-1	3
text/html; charset=windows-1252	3
text/html; charset=windows-1250	3
text/html; Charset=windows-1252	2
text/html; charset=WINDOWS-1257	2
text/html; charset=Shift_JIS	2
text/html; charset=ISO-8859-9	2
text/html; charset=cp1251	2
text/html; charset=Windows-1252	2
text/html;charset=iso-8859-15	2
text/html; charset="UTF-8"	2
text/html; charset=ISO-8859-1;	2
text/html; charset=windows-1256	2
text/html; charset=us-ascii	2
text/html; charset=GB2312	1
text/html; charset=latin-1	1
text/html;charset=euc-kr	1
text/html; charset=utf8	1
text/html; charset=big5	1
text/html; charset=shift_js	1
text/html; charset=zh-tw	1
text/html; charset=iso-8859-1;	1
text/html; charset=MS949	1
text/html;charset=WINDOWS-1252	1
text/html;charset = utf-8	1
text/html; charset=BS_4730	1
text/html; charset= windows-1252	1
application/atom+xml; charset=UTF-8	1
text/html; qs=.9	1
application/x-shockwave-flash	1
text/html; charset=ISO-8859-7	1
text/html; charset=ISO-8859-4	1
text/html; charset="ISO-8859-1"	1
text/html; charset=iso8859-1	1
text/html; charset=gbk	1
text/html; Charset=ISO-8859-7	1
text/html; charset=NONE	1
text/html; charset=TIS-620	1
text/html;charset=euc-jp	1
text/html;charset=iso-8859-2	1
text/html;Charset=ISO-8859-1	1
text/html; charset=es_ES.ISO-8859-1	1
text/html; charset=ISO	1
text/html; charset=Windows-1251	1
text/html; charset=EUC-KR	1
text/html;charset=EUC-KR	1
text/html; charset=win-1251	1
text/html; charset=en_US	1
text/html; charset=no	1
text/html; charset=None	1
text/html; charset=CP1250	1
text/HTML; Charset=Windows-1251	1
text/html; charset=euc-jp	1
text/html; charset=iso.8859-1	1
text/html;charset=ISO-8859-15	1
text/xml;charset=utf-8	1
text/html; charset=big5,euc-jp	1
text/html;charset=Cp1252	1
text/html; charset=WINDOWNS-1252	1
text/html; charset=windows-1257	1
text/html; charset=windows-1254	1
text/html; charset=windows-1255	1
application/rss+xml	1
text/xml; charset=ISO-8859-1	1

Most Comment Date Formats

These come from all the headers allowed in responses that have a HTTP-date value, not just the "Date" header.

Valid (RFC 822)	26154
-1	67
0	40
Fri, Jan 1 1999 00:00:00 GMT	4
Wed Dec 12 00:22:55 2007 GMT	3
Wed 12 Dec 2007 00:16:47 GMT	2
Valid (asctime)	2
	1
Fri, Apr 01 1974 00:00:00 GMT	1
12,12 12 07 12:24:10 GMT	1
Thu May 11 20:47:17 2000 GMT	1
Fri Nov 16 03:58:06 2001 GMT	1
Sat Feb 8 12:37:23 2003 GMT	1
Thu Feb 22 19:38:39 2001 GMT	1
{ts '2007-12-11 19:26:11'}	1
Thu Feb 19 00:59:40 2004 GMT	1
Tue, 11 DEC 2007 23:21:58 GMT	1
Wed, 12 Dec 2007 00:25:23GMT	1
Mon May 13 10:02:36 2002 GMT	1
Thu Feb 24 01:16:44 2000 GMT	1
12.12.2007	1
Mon May 14 04:32:21 2001 GMT	1
Sat May 12 23:42:27 2001 GMT	1
1.28419E+17	1
Wed May 3 22:28:20 2000 GMT	1
Thu Sep 11 10:01:14 2003 GMT	1
Tue Jul 30 18:40:30 2002 GMT	1
Thu Jun 15 12:34:19 2000 GMT	1
Wed Jun 27 12:54:33 2001 GMT	1
Sat Nov 9 05:19:16 2002 GMT	1
Sat Nov 25 05:02:40 2000 GMT	1
Sun Jun 18 19:10:46 2000 GMT	1
Wed, 12 Dec 2007 00:25:41GMT	1
Wed, 12-Dec-2007 24:25:18 GMT	1
Sun Feb 26 22:29:31 2006 GMT	1
Tue Aug 2 16:32:29 2005 GMT	1
Tue Jun 3 06:04:04 2003 GMT	1
Sat Apr 24 21:07:50 2004 GMT	1
Sun Jul 13 11:48:28 2003 GMT	1
Tue May 11 14:30:24 2004 GMT	1
Thu Dec 14 06:49:36 2000 GMT	1
Tue, 11 Dec 2007 17:29:58 +0300 GMT	1
Mon May 29 17:26:47 2000 GMT	1
Sunday 15-May-1994 12:00:00 GMT	1
Fri Oct 19 15:51:59 2007 GMT	1
Wed, 12-Dec-2007 24:24:43 GMT	1
Wed, 12 Dec 2007 00:25:07GMT	1
Fri Feb 22 22:42:43 2002 GMT	1
Thu Jun 7 03:08:51 2007 GMT	1
Mon Sep 10 12:29:30 2007 GMT	1
Wed Apr 7 23:36:25 2004 GMT	1
Wed Nov 7 22:40:16 2007 GMT	1
Thu Jun 22 02:22:23 2000 GMT	1
Sun Jul 24 17:55:17 2005 GMT	1
Tue Oct 16 19:42:47 2007 GMT	1
{ts '2007-12-11 18:25:47'}	1
Wed, 12-Dec-2007 24:22:48 GMT	1
Sat Sep 1 22:42:58 2007 GMT	1
Wed Jul 18 09:45:52 2007 GMT	1
Sun Mar 3 23:52:59 2002 GMT	1
Fri Nov 30 13:35:39 2007 GMT	1
Tue Dec 11 16:37:50 2007 GMT	1
-1;	1
Tue Dec 11 01:45:16 2007 GMT	1
Wed Aug 30 22:58:09 2006 GMT	1
Mon Jun 25 18:41:19 2007 GMT	1
Sun Dec 22 12:40:14 2002 GMT	1
Fri Oct 12 23:37:18 2001 GMT	1
{ts '2007-12-11 19:26:30'}	1
Wed Feb 20 18:26:49 2002 GMT	1
Sun Apr 28 05:26:09 2002 GMT	1
12.12.2007 2:25:36	1
Fri Nov 16 01:11:08 2007 GMT	1
Fri, Jun 12 1981 08:20:00 GMT	1
Sat Nov 4 09:50:05 2000 GMT	1
Wed, 12-Dec-2007 00:30:14 GMT	1
Mi, 12 Dez 2007 00:25:34 GMT	1
Wed, 12-Dec-2007 00:26:09 GMT	1
Wed, 12-Dec-2007 24:25:03 GMT	1
Wed Sep 5 14:09:38 2001 GMT	1
Thu Jul 4 09:12:19 2002 GMT	1
Tue Dec 23 13:03:16 2003 GMT	1
{ts '2007-12-12 01:24:44'}	1

Most Common First Server Product Token

Apache	3115
Microsoft-IIS/6.0	2563
Apache/1.3.37	1075
Apache/1.3.33	1059
Microsoft-IIS/5.0	866
Apache/2.0.52	508
Apache/1.3.39	465
Apache/2.0.54	407
Apache/2.2.3	371
Apache/1.3.34	325
Apache/1.3.27	245
Apache/1.3.26	242
Apache/2.2.4	223
Squeegit/1.2.5	206
Apache/2.0.46	189
Apache/2.0.59	176
Apache/2.2.6	156
Apache/1.3.29	126
Apache/2.0.55	124
Apache/2.0.53	100
Apache/1.3.31	91
Apache/1.3.36	79
Apache/2.0.51	69
Apache/ProXad	67
Rapidsite/Apa/1.3.33	57
Apache/2.0.61	55
Apache/2	55
Zeus/4.3	54
Apache/1.3.28	53
Apache/2.0.50	50
Apache/2.0	49
Sun-ONE-Web-Server/6.1	48
Apache/2.2.2	47
Apache/2.0.40	46
Apache/1.3	46
Apache/1.3.35	44
Microsoft-IIS/4.0	43
Netscape-Enterprise/4.1	43
Apache/2.0.49	42
Apache/1.3.20	41
Apache/2.2.0	40
NOYB	40
Apache/1.3.19	40
Server	38
Apache-Coyote/1.1	34
GFE/1.3	32
Apache/2.0.48	30
	28
Lotus-Domino	23
Zope/(Zope	22
IdeaWebServer/v0.50	21
Netscape-Enterprise/3.6	20
Apache/1.3.22	20
Apache/2.0.58	19
Zeus/4.2	19
Apache/1.3.12	18
WebServerX	18
ConcentricHost-Ashurbanipal/2.0	17
ZX_Spectrum/1997	15
Apache/1.3.6	13
Zeus/3.3	12
Netscape-Enterprise/6.0	12
Apache/1.3.9	11
tigershark/3.0.128	10
Jetty/5.1.10	9
Apache/2.0.47	9
IBM_HTTP_Server	9
Apache/1.3.23	9
IBM_HTTP_Server/6.1.0.5	9
thttpd-TOF	8
.V01	8
Mittwald	8
httpd	8
A	8
.V06	7
Rapidsite/Apa	7
Apache/2.0.59easyTECC/2.0	7
Oracle	7
Apache/1.3.24	7
Apache-AdvancedExtranetServer/1.3.33	7
LiteSpeed	6
Zeus/3.4	6
Rapidsite/Apa/1.3.31	6
.V05	6
Simple	6
AOLserver/3.5.10	6
WebServer	6
nginx/0.5.32	6
IBM_HTTP_Server/6.0.2.15	6
Zeus/4.1	6
.V03	5
.V12	5
nginx/0.5.30	5
thttpd	5
Apache/1.3.3	5
Apache/2.0.39	5
AppleDotMacServer-1B5608	4
.V14	4
e/3	4
.V13	4
.V17	4
lighttpd/1.4.18	4
nginx/0.3.30	4
Apache/1.3.14	4
BaseHTTP/0.3	4
WebSite/3.5.19	4
nginx/0.5.33	4
DinaHTTPd	4
Unknown	3
WebSTAR/4.4(SSL)	3
.V04	3
Acenet	3
.V02	3
thttpd/2.21	3
Ready	3
.V19	3
.V18	3
Apache/2.0.43	3
Mongrel	3
Oracle_Web_Listener/4.0.8.1.0EnterpriseEdition	3
lighttpd/1.4.13	3
Zeus/4_3	3
.V20	3
nginx/0.5.7	3
Sun-Java-System-Web-Server/7.0	3
ENIAC	3
Resin/3.0.23	3
lighttpd	3
Stronghold/2.2	3
WebLogic	3
ZWS	3
AOLserver/4.0.10	3
Oversee	3
hi-ho	3
ConcentricHost-Ashurbanipal/1.8	2
Power	2
Oracle-Application-Server-10g	2
Roxen/4.5.146-release3	2
Blk-Enterprise	2
Oxito	2
Concealed	2
Apache/2.2	2
IBM_HTTP_Server/6.0.2.21	2
httpd+ssl	2
WebSTAR/3.0.2	2
IBM_HTTP_Server/2.0.47	2
HP	2
SmallWebServer/2.0	2
Wake	2
lighttpd/1.4.15	2
lighttpd/1.4.16	2
Apache/2.0.61/DataZone	2
Servage.net	2
Apache/2.0.52-CHS-1	2
WisePanel	2
Apache/1.3.17	2
webserver	2
nginx/0.5.27	2
nginx/0.4.13	2
nginx/0.3.35	2
UserLand	2
*	2
Apache-AdvancedExtranetServer/2.0.53	2
Apache/2.0.59-CHS-1	2
Lotus-Domino/0	2
Jetty/5.1.1	2
MIT	2
Resin/3.0.21	2
Resin/3.0.19	2
Web	2
INT.PL/3.0.9f	2
Stronghold/2.4.2	2
ZealdWeb/1.0	2
.V10	2
IBM_HTTP_Server/6.0.1	2
Apache/1.2.6	2
"USHR	2
OSU/3.10a;Multinet	2
.V09	2
Spry-SafetyWEB-Server-NT/1.3a	2
Oracle-Application-Server-10g/10.1.2.0.0	2
WebSitePro/2.4.9	1
IcodiaSecureHttpd	1
Demandware	1
HTTPd-WASD/6.0.2	1
WebSTAR	1
thttpd/2.25b	1
Lasso/7.0	1
Embperl/2.0rc1	1
EarthServer/1.0	1
Apache/2.2.5	1
Lasso/6.0	1
Apache/1.0.0	1
4D_WebSTAR_S/5.3.3	1
aris	1
Roxen/BuGless	1
Roxen/4.0.325-Debian1.1-release4	1
Apache/1.3.0	1
Oracle-Application-Server-10g/9.0.4.2.0	1
Apache/Not	1
Apache/2.2.x	1
Apache/Unoeuro	1
lighttpd/1.5.0	1
Apache/1.3.11	1
nginx/0.5.14	1
zope.server.http	1
IPL	1
Conundrum	1
Resin/2.1.16	1
Resin/2.1.17	1
4D_WebStar_D/6.74	1
FSSWEB/1.0	1
Oracle9iAS/9.0.2	1
OpenCms/6.0.0	1
AAISP/1.1	1
HTTP	1
Sun	1
Apache/X.X.X	1
NetWare-Enterprise-Web-Server/5.1	1
.V11	1
IBM_HTTP_Server/1.3.12.6	1
Embperl/2.0b9	1
Microsoft-IIS/3.0	1
Zeus	1
lighttpd/1.4.11	1
nginx/0.5.17	1
Sarge	1
Pacohost.com	1
webfusion.co.uk/httpd	1
WebSite/3.1.11	1
nginx/0.5.6	1
IBM_HTTP_SERVER	1
LexisNexis	1
Aleto	1
workSpace	1
Redirector	1
AOLserver/3.4.2	1
VHFFS	1
Resin/3.1.1	1
"IIS/1.0	1
Apache-AdvancedExtranetServer/1.3.29	1
ZAMAN	1
Apache-AdvancedExtranetServer/2.0.47	1
Embperl/2.0rc3	1
gws	1
nginx/0.5.26	1
httpd/ktkb	1
CERN/3.0A	1
Hanbiro	1
IBM_HTTP_SERVER/1.3.28.1-PK27875	1
EnterpriseWeb/1.1.4	1
SAMBAR	1
IBM_HTTP_Server/2	1
IIS	1
Resin/3.1.2	1
IBM_HTTP_SERVER/1.3.26.2	1
sunrise	1
MCWeb	1
Oracle9iAS/9.0.2.3.0	1
Lasso/8	1
Apache/1.3.34+APPLETZ/1.0	1
WebSTAR/4.5(SSL)	1
nginx/0.5.31	1
nginx/0.6.6	1
"Web	1
"SMSNET	1
WebSTAR/4.2(SSL)	1
nginx/0.6.17	1
OSU/3.9a;UCX	1
nginx/0.6.11	1
AOLserver/4.0.3	1
6-lighttpd	1
Caudium/1.4.8	1
IBM_HTTP_SERVER/1.3.19.1	1
Apache/2.0.44	1
Orion/2.0.7	1
Orion/2.0.5	1
Oracle-Application-Server-10g/9.0.4.0.0	1
SAGA	1
NaviServer/2.0	1
Apache-AdvancedExtranetServer/1.3.23	1
Apache-AdvancedExtranetServer/1.3.26	1
WebSTAR/4.2	1
noViA	1
Apache/1.3.4	1
WLW	1
AtyponWS/7.1	1
YTS/1.15.1	1
Undisclosed-Webserver/0.1	1
Apache-AdvancedExtranetServer/2.0.50	1
IIS/7.0	1
publicfile	1
WebSitePro/2.3.7	1
Discworld/0.10.3	1
Sun-Java-System-Web-Server/6.1	1
httpd/kbn	1
Ski	1
Stronghold/4.0	1
4D_WebSTAR_S/5.3.0	1
nginx/0.1.45	1
CoolWeb	1
SAP	1
IceWarp/8.3	1
LoboCom-WebServer/1.0	1
Apache/2.0.36	1
az-webserver	1
Apache/ASJ2007070300SUX-FATE	1
WEB602/1.04	1
nix	1
Apache-AdvancedExtranetServer	1
ONI	1
web500gw	1
Serverplan	1
WebTV/1.0	1
Apache-AdvancedExtranetServer/1.3.31	1
The	1
Apache/10.0	1
OF.PL	1
nginx/0.6.16	1
Reed	1
<a	1
Apache-AdvancedExtranetServer/2.0.48	1
INSIDE	1
THEO+Server/5.0	1
Roxen/2.1.135	1
Zeus/4.0	1
IBM_HTTP_Server/6.0	1
Apache/1.3.32	1
nginx/0.3.25	1

Etag Strength and Validity

Valid (strong)	7962
Valid (weak)	17
	8
eaccelerator-1351692760	1
5324bfe837728477f89cfb7d983820ed	1
PUB1197419160	1
9a2289b94c8e112ffafb9b55947b48e	1
z-1g3s18hn-97hp-1gqnyiso13-k01hgzde10	1
cd62bd32ab1c41883d5d1bfac81bac4c	1
-nulo-	1
80d83a43b2c97a19f5bfcf1d63aaab2	1
dd4ae2ad389dbfd07a6368195bf67e3	1
d6ee8b54487c3cb997ca2dfa4aa8604d1197405049	1
a5fc223515c5aa4e6ee0408b80e87ff9	1
u-1g3s18hn-13np-1b7pyirc0m-2af1p2zdcp0	1
8dcb0c531a0f2997dab7f4b9669c9c50	1
2N5o/t+2YDW2WBcoDAUm95Sol/M	1
aa5f0107517db4371be051e81d838770	1
fcf53cc192e4d81fd78360f34173b957	1
99dd0b7976bfecce41ee307b5ff4cd091187119998	1
5150fae713f4306aa655016f1c32d6c7	1
49a774f7797266451e7b4bbca61348bf	1
eaccelerator-1431878999	1
1029	1
a2f4e6d2f5146c0ff0484333e92e6190	1
59584b8bdef9e2e442f03cf8945a7229	1
3c36771c2957e19be2403a16a734bb13	1