CRAWDAD dartmouth/campus (v. 2007-02-08)

Citation Author(s):
David
Kotz
Dartmouth College
Tristan
Henderson
University of St Andrews
Ilya
Abyzov
Dartmouth College
Jihwang
Yeo
Dartmouth College
Submitted by:
CRAWDAD Team
Last updated:
Wed, 09/09/2009 - 08:00
DOI:
10.15783/C79G6J
License:
Creative Commons Attribution
218 Views
Collection:
CRAWDAD
Categories:
Wireless Networking
Keywords:
WiFi
0
0 ratings - Please login to submit your rating.
CITE SHARE/EMBED

Abstract 

Syslog, SNMP, and tcpdump data for 5 years or more from wireless network at Dartmouth College.

Note: This dataset has multiple versions.  The dataset file names of the data associated with this version are listed below, under the 'Traceset' heading and can be downloaded under 'Dataset Files' on the right-hand side of the page.

This dataset includes syslog, SNMP, and tcpdump data for 5 years or more, for over 450 access points and several thousand users at Dartmouth College.

last modified :

2008-09-12

release date :

2007-02-08

date/time of measurement start :

2001-04-11

date/time of measurement end :

2006-10-04

collection environment :

The Dartmouth College campus has over 190 buildings on 200 acres. Dartmouth College has about 5500 students and 1200 faculty, and during our study there were approximately 3200-3300 undergraduates on campus. Students are required to own a computer, and most purchase a computer through the campus computer store. Wireless laptops increasingly dominate those purchases, making up 45% of the total in 2000, 70% in 2001, 88% in 2002, and 97% in 2003. Assuming that students obtaining computers elsewhere choose laptops in the same proportion, we estimate that over 75% of the undergraduates owned laptops at the time of our study.

network configuration :

476 Cisco 802.11b access points (APs) were installed in 2001 to cover most of the campus. Since then, APs have been added to increase coverage and to cover new construction, and there are currently 566 APs. The compact nature of the campus means that the signal range of interior APs extends to cover most of the campus' 
outdoor areas. All APs share the same SSID, allowing wireless clients to roam seamlessly between APs. On the other hand, a building's APs are connected to the building's existing subnet. The 188 buildings with wireless coverage span 115 subnets, so clients roaming between buildings may be forced to obtain new IP
addresses. Clients obtain IP addresses using DHCP (lease times were 6 or 12 hours at different points in the trace).

data collection methodology :

We used three techniques: syslog events, SNMP polls, and network sniffers (tcpdump).  We also derived movement history data from the syslog data.

sanitization :

All data has been sanitized to protect the privacy of our users. We have resanitized the data (2004-11-08) so that there is a consistent MAC address mapping and a consistent IP address mapping across all of the traces. In other words, the sanitized IP address 111.222.333.444 will correspond to the same raw IP address in all of our traces. Similarly, the sanitized MAC address aa:bb:cc:dd:ee:ff will always correspond to the same raw MAC address in all of our traces. Note that these may not represent the same physical device, due to DHCP, MAC spoofing, etc.

Traceset

dartmouth/campus/syslog

Timestamped, sanitized syslog records from Dartmouth wireless network.

  • files: syslog-v3.3.tar.gz, syslog-2005-2006-cisco.tar.gz, syslog-2005-2006-aruba.tar.gz, syslog-2005-2006-merged.tar.gz
  • description: The traceset of nearly continuous recording of the syslog records produced by the access points, from 2001-04-11 to 2004-06-30, and from 2005-09-01 to 2006-10-04. UNIX timestamps have been added to each log record, and MAC addresses and AP names sanitized.
  • measurement purpose: Usage Characterization, User Mobility Characterization
  • methodology: We configured the access points to transmit a syslog message every time a client card associated or disassociated with the access point (We configured the Cisco APs and the Aruba APs differently. Please see the traces dartmouth/campus/syslog/01_04 and dartmouth/campus/syslog/05_06 for more details). Dartmouth currently has no authentication to associate with the network, so we do not know the identity of users, and the IP address given to a user varies from time to time and building to building.
  • last modified: 2007-02-08
  • dataname: dartmouth/campus/syslog
  • version: 20070208
  • change: A new syslog trace (collected from 2005-09-02 to 2006-10-04) has been added.
  • release date: 2007-02-08
  • date/time of measurement start: 2001-04-11
  • date/time of measurement end: 2006-10-04
  • hole: We have not released syslog trace collected from 2004-07-01 to 2005-08-31.

dartmouth/campus/syslog Traces

    • 01_04: Timestamped, sanitized syslog records from Dartmouth wireless network.
  • format: timestamp, AP name, the MAC address of the card, and type of message 
  • description: The trace of nearly continuous recording of the syslog records produced by the access points, from 2001-04-11 to 2004-06-30. UNIX timestamps have been added to each log record, and MAC addresses and AP names sanitized.
  • last modified: 2007-01-31
  • dataname: dartmouth/campus/syslog/01_04
  • version: 20041218
  • change: Syslog trace is newly created.
  • release date: 2004-12-18
  • date/time of measurement start: 2001-04-11
  • date/time of measurement end: 2004-06-30
  • file: syslog-v3.3.tar.gz
  • hole: We only have a list of these holes in fall 2001. We had ``spatial holes'' because many APs did not send syslogs. [Configuration mistake.] And temporal holes, because our syslog recording server(s) failed. You may refer to note for details. It also appears that the engineering school's APs, building name ''cummings'' did not send any messages after they installed a firewall in early 2002 until I noticed the problem and asked them to open a hole in the firewall in late 2002. We do not release the syslog trace collected from 2004-07-01 to 2005-08-31.
  • limitation: Since syslog messages are sent from the APs to a relaying server (ns1), and from ns1 to our syslog recording servers, as UDP messages, it is possible for them to be lost or reordered along the way. The timestamps are applied by the syslog daemon on our host, so the timestamps are monotonically increasing. But, the events may have been recorded out of order, and some may be missing. We believe this effect is small enough to be negligible. We have two syslog recording servers, and we do not see the same event with different timestamps in the two servers. From 10/19/2003 this no longer applies.
  • sanitization: Every MAC address has been sanitized, and the IP address or host name of client machines has been removed. To sanitize the MAC address, we randomized the bottom six hex digits. We collected every MAC address from all of our syslog, SNMP, an tcpdump traces, and built a huge table mapping real MACs to randomized MACs, ensuring that all mappings are unique. Each access point name has been blinded in the form: AcadBldg10AP3 where this indicates the third AP in the tenth building of type 'Academic.' The building types are Adm (Admin), Ath (Athletic), Lib (Library), Oth (Other - mainly sysadmin test APs), Res (Residential) and Soc (Social). Refer to note for details.
    • 05_06: Timestamped, sanitized syslog records from Dartmouth wireless network for the period 2005 - 2006.
  • configuration: [Cisco APs] We configured the Cisco access points to transmit a syslog message every time a client card authenticated, associated, reassociated, disassociated, or deauthenticated with the access point. Each message contains the AP name, the MAC address of the card, and the type of message. [Aruba APs] On our campus, we deployed Aruba wireless networks with an Aruba 5000 switch as a master switch, which controls the wireless network in centralized manner. The configuration has a three-level hierarchy (master-switches-APs) such that a number of switches are attached to the master switch, and likewise a number of APs are attached to each switch. We have three models of Aruba APs: 52, 61, and 72. The wireless network is virtually divided into several zones (subnets), each of which has a controller that controls a set of APs. Aruba syslog messages come from either the master switch (the central controller) or each zone controller. Since different zone covers different set of APs, separate syslog messages come from each zone controller. The Aruba system is able to configure multiple ESSIDs on each AP. At the time of collecting these syslog data, we used four ESSIDs - "Kiewit Wireless", "Kiewit Video", "Kiewit Voice", and "Hanover Inn". Among those ESSIDs, Kiewit Video/Voice are NATed and use private (RFC1918) addresses. We do not have L2 assoc/disassoc/auth/deauth messages in the Aruba syslog trace because the Aruba switch does not give us those. What we do have are "station up" and "station down" messages which indicate when the switch sees a client connect with a BSSID. Though we do not know what these station up|down messages equate to in the 802.11 FSM, for most analyses it should be possible to more or less correlate these messages with the assoc|disassoc messages in the Cisco syslog.
  • format:
  1. Directory and files


This trace consists of three tarballs (directories): 
'syslog-2005-2006-cisco', 'syslog-2005-2006-aruba', and
'syslog-2005-2006-merged'.
'syslog-2005-2006-cisco' and 'syslog-2005-2006-aruba' directories 
contain syslog records from cisco APs and Aruba APs, respectively. 
'syslog-2005-2006-merged' directory contains syslog records
merged from 'cisco' and 'aruba' syslog records, sorted by timestamps.
Each directory contains syslog trace files for each day during the measurement
period. File name follows the format YYMMDD.HHMMSS.{syslog or aruba or merged}.
The time used for file name is in UTC.


For Aruba syslog trace, we represent each access point name as location id
(with the format [building_id.floor_id.ap_id] like 15.1.1). Under the trace
root directory, we provide a file called 'aruba_locid_table.csv' containing
a mapping between location id and sanitized AP name
(e.g., a pair of 15.1.1 and AcadBldg10AP3).



  2. Format of Cisco syslog records is as follows:


unix_timestamp timestamp1 AP_name1 timestamp2 counter AP_name2 timestamp2 syslog_message



unix_timestamp : UNIX timestamp in UTC

timestamp1 : the time that the syslog message was received

AP_name1 : the (sanitized) hostname of the host that sent the syslog

counter : internal counter

AP_name2 : the (sanitized) hostname that the AP thinks it has. Sometimes AP_name1 and AP_name2 don't match.

timestamp2 : the AP's clock. Sometimes timestamp1 and timestamp2 don't match.

syslog_message : syslog message content


The following is a sample line in Cisco IOS syslog trace.


1157014202 Aug 31 04:50:02 AcadBldg33AP1 14375: AcadBldg33AP1 Aug 31 08:50:01: %DOT11-6-ASSOC: Interface Dot11Radio0, Station 0016cbf7ca65 Associated KEY_MGMT[NONE]



Format of Aruba syslog records is as follows:


unix_timestamp timestamp ip_address1 year [ip_address2] syslog_message



unix_timestamp : UNIX timestamp in UTC

timestamp, year : the time that the syslog message was received

ip_address1 : the (sanitized) IP address of the controller

ip_address2 : the (sanitized) IP address of another controller that sent this message.
Sometimes the messages don't come directly from the controller, they come from a controller
further down the tree.

syslog_message : syslog message content


The following is a sample line in Aruba syslog trace.


1159954944 Oct 4 05:42:24 50.32.208.194 2006 [50.32.208.195] authmgr[510]: <INFO> station down <00:15:f9:9e:71:25> bssid 00:0b:86:d3:ab:80, essid Kiewit Voice, vlan 2242, ingress 0x1168 (tunnel 264), u_encr 1, m_encr 1, loc 15.1.1 slotport 0xfc7


This message means that a user connected to an AP is removed from the user table.
Previously it was connected to VLAN 2242 at location 15.1.1.  The reason for the
disconnection might be one of the following:
- User moved out of the network
- The user is logged out of the network.
  • description: The trace of nearly continuous recording of the syslog records produced by the access points, from 2005-09-01 to 2006-10-04. UNIX timestamps have been added to each log record, and MAC addresses and AP names sanitized.
  • last modified: 2007-02-08
  • dataname: dartmouth/campus/syslog/05_06
  • version: 20070208
  • change: This 2005-2006 syslog trace is newly created.
  • release date: 2007-02-08
  • date/time of measurement start: 2005-09-01
  • date/time of measurement end: 2006-10-04
  • sanitization: Every MAC address has been sanitized by randomizing the bottom six hex digits, and we mapped all IP addresses using a prefix-preserving sanitizer. [Cisco syslog] Each access point name has been blinded in the form: AcadBldg10AP3 where this indicates the third AP in the tenth building of type 'Academic.' The building types are Adm (Admin), Ath (Athletic), Lib (Library), Oth (Other - mainly sysadmin test APs), Res (Residential) and Soc (Social). [Aruba syslog] Each access point name is represented as location id ([building_id.floor_id.ap_id] like 15.1.1). We also provide a file called 'aruba_locid_table.csv' containing a mapping between location id and sanitized AP name (e.g., a pair of 15.1.1 and AcadBldg10AP3).

dartmouth/campus/snmp

Records of SNMP polling at Dartmouth College.

  • files: fall01.tar.gz, spring02.tar.gz, fall03.tar.gz
  • description: The traceset of polling every AP every five minutes, during Fall term 2001, Spring term 2002, Fall term 2003 and Winter term 2004. The Fall 2001 data was used for [MobiCom 2002 paper]. The 2003/4 data was used for [MobiCom 2004 paper]. We recommend only using the 2003/4 data. See this important note about problems with the 2001/2002 SNMP dataset. Any questions about the 2001/2 data will go into our LBE (Less than Best Effort) queue, i.e., they may not be answered... please use the 2003/4 data instead.
  • measurement purpose: Usage Characterization
  • methodology: We used the Simple Network Management Protocol (SNMP) to poll each AP every five minutes, querying AP and client-specific counters. AP-specific variables included inbound/outbound bytes, packets and errors, and the clients associated with a given AP. Client-specific variables included MAC and IP addresses, signal strength and quality.
  • sanitization: To sanitize the data, we randomly (but consistently) mapped the MAC address field into a randomly chosen MAC of the same vendor, and we mapped all IP addresses using a prefix-preserving sanitizer.
  • last modified: 2008-09-12
  • dataname: dartmouth/campus/snmp
  • version: 20041109
  • change: SNMP Traceset is resanitized.
  • release date: 2004-11-09
  • date/time of measurement start: 2001-09-14
  • date/time of measurement end: 2004-02-28
  • limitation: We have no way to distinguish periods with no clients from periods when the AP was off or unreachable. We also have no way to detect an AP reboot or reset, which reset all of the per-client counters reported here. Thus it is critical to take care when interpreting the per-client byte counts... a counter can be reset or roll over between two polls, and the delta can thus appear to be negative (or, in unsigned math, a very large number).
  • hole: There were unfortunate gaps in the data collection, generally caused by power failures.
  • error: In 2001/2 data, the perl scripts that performed the SNMP queries suffered from some problems, in that they queried inappropriate SNMP values, or misunderstood the meaning of other values. This data was also used in a subsequent analysis. The same scripts were used to collect data for a subsequent study of another wireless network. See http://www.cs.dartmouth.edu/reports/abstracts/TR2003-480/

dartmouth/campus/snmp Traces

    • fall01: Records of SNMP polling during Fall term 2001.
  • configuration: The job of polling all access points was divided among two machines: agentnews and klebb. Each of those hosts has a directory here. In each directory there is a number of subdirectories, one for each day, and in each of those subdirectories there are a number of SNMP log files.

  • format:

There are two types of file. The first type of file has one file for each access point. Each file gives information about each interface for each poll. Most of the interfaces are irrelevant; you only care about the wireless interface (the one with ifSpeed=11000000).
Here is an example: V1.0ap timestamp,ap,ifIndex,ifInOctets,ifOutOctets,ifSpeed,ifInErrors,ifOutErrors
Notice the periodic restatment of the file format version and the description of the MIB variable names used in the lines that follow.
The second type of file occurs once for each AP and each date. The file name conveys the blinded name of the AP, e.g. AcadBldg10AP3 where this indicates the third AP in the tenth building of type 'Academic.' The building types are Adm (Admin), Ath (Athletic), Lib (Library), Oth (Other - mainly sysadmin test APs), Res (Residential) and Soc (Social). Each data file contains all of the data for that access point on that date. Below is a sample of the top of one file. The top two lines are documentation; the first indicates the file format version (always V1.0). The second identifies the column headers for each of the data lines.
 After the timestamp (standard Unix time in seconds), the remaining fields are MIB variables from the AWC MIB (Aironet Wireless Communications is the name of the company that developed our access points; Aironet was bought by Cisco who then branded and sold the APs under their name).
V1.0 timestamp,awcTpFdbAddress,awcTpFdbClassID,awcTpFdbSrcOctetsImmed,awcTpFdbDestOctetsImmed,awcTpFdbIPv4Addr,awcTpFdbDdpProdDevID,awcTpFdbDdpRadioDevID,awcDot11TpFdbAID,awcDot11TpFdbTxShortRetries,awcDot11TpFdbLatestRxSignalQuality,awcDot11TpFdbCapabilities 1001908847,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,-15,unknown,state2,73,73 1001909056,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,generic80211Client,unknown,state2,73,73 1001909266,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,-15,unknown,state2,73,73 1001909476,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,broadcast,-16,state2,73,73 1001909683,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,broadcast,-16,state2,73,73 1001909892,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,generic80211Client,unknown,state2,73,73 1001910102,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,generic80211Client,unknown,state2,73,73 1001910311,003065d1eb95,clientStation,1276264,1986728,000.000.000.000,ethernetAP,34,state2,73,73 
  • description: The trace of SNMP polling every AP during Fall term 2001.
  • last modified: 2008-09-12
  • dataname: dartmouth/campus/snmp/fall01
  • version: 20041109
  • change: SNMP trace is resanitized.
  • release date: 2004-11-09
  • date/time of measurement start: 2001-09-14
  • date/time of measurement end: 2002-01-10
  • file: fall01.tar.gz
  • error: During the fall there were some access points with old, buggy firmware that sometimes filled our SNMP data files with garbage entries. Any code you write to parse the data must be very robust. Unfortunately the code that I used was hacked on by two or three different students and is not currently in presentable form. See http://www.cs.dartmouth.edu/reports/abstracts/TR2003-480/.
    • spring02: Records of SNMP polling during Spring term 2002.
  • configuration: The job of polling all access points was divided among three machines: agentnews, klebb, and molari. I used molari only briefly until I was able to accomplish the same thing on one of the other two (access points cummings-* were missed for a week, because they are behind a firewall, and eventually I set up molari inside the firewall, and eventually I was able to add a hole in the firewall so that agentnews and klebb could poll through the firewall). Each polling machine has a directory. Each such directory has a subdirectory for each date. In each date, there is a file for each access point polled by that machine on that date. In the file are several types of lines. Below is a sample from the top of one file. Notice that it uses a comma-separated format suitable for import into spreadsheets, or easy parsing with AWK or perl
  • format: 
The first five lines are comments. The first gives basic information:
V2.1: file format version 2.1, timestamp of file creation, AP name, and date code YYMMDD
All timestamps are standard Unix timestamps (seconds since 1970). The other four comment lines describe the format of lines that occur later in the file. Other than the timestamp and AP name, the rest of these fields are MIB variable names. After the five comment lines comes a series of polls. Each poll consists of one ''sys'' line, one ''if'' line describing stats of the the wireless interface, and zero or more pairs of ''c1'' and ''c2'' lines, each pair describing a currently connected client. The c1 and c2 lines are a collection of MIB variables from the AWC MIB (Aironet Wireless Communications is the name of the company that developed our access points; Aironet was bought by Cisco who then branded and sold the APs under their name).
V2.1,1018929767,AdmBldg27AP2,020416
sys,timestamp,AP,sysUpTime
if,timestamp,AP,ifIndex,ifType,ifSpeed,ifInOctets,ifInUcastPkts,ifInErrors,ifInDiscards,ifOutOctets,ifOutUcastPkts,ifOutErrors,ifOutDiscards
c1,timestamp,AP,awcDot11TpFdbAddress,awcDot11TpFdbClientState,awcDot11TpFdbLatestRxSignalStrength,awcDot11TpFdbLatestRxSignalQuality
c2,timestamp,AP,awcTpFdbAddress,awcTpFdbClassID,awcTpFdbSrcOctetsImmed,awcTpFdbDestOctetsImmed,awcTpFdbIPv4Addr
 
  • description: The trace of SNMP polling every AP during Spring term 2002.
  • last modified: 2008-09-12
  • dataname: dartmouth/campus/snmp/spring02
  • version: 20041109
  • change: SNMP trace is resanitized.
  • release date: 2004-11-09
  • date/time of measurement start: 2002-03-25
  • date/time of measurement end: 2002-06-09
  • file: spring02.tar.gz
  • error: PLEASE NOTE THAT THERE ARE ERRORS in dartmouth/campus/snmp/fall01 and dartmouth/campus/snmp/spring02 data sets. See http://www.cs.dartmouth.edu/reports/abstracts/TR2003-480/ .
    • fall0304: Records of SNMP polling during Fall term 2003 and Winter term 2004.
  • configuration: We only used one machine for SNMP polls in this period. The SNMP poller was rewritten to be more robust and more efficient, and so we only needed one machine to poll all ~ 560 APs on campus. We queried more counters in this trace. The variables are listed at the beginning of each file. For more details see the MIBS: ftp://ftp-sj.cisco.com/pub/mibs/v2/AWCVX-MIB.my ftp://ftp-sj.cisco.com/pub/mibs/v2/IEEE802dot11-MIB.my At the time of this data collection, Dartmouth mainly used Cisco 340 and 350 APs. These used to run the VxWorks operating system. During December 2003 to May 2004, our 350 APs migrated from running VxWorks to the Cisco IOS (the APs didn't originally run IOS as they were made by Aironet, a company that was later bought by Cisco). IOS uses completely different SNMP MIBs to VxWorks, and so the variable names and their order are slightly different. When the upgrades started taking place, we incremented the log version number to "V3.1" (the first line of each log) to indicate the new variables being queried. We also folded both the "c1" and "c2" client-specific lines into one "cl" line (this made the parser code easier to maintain).
  • format:
The first five lines are comments.  The first gives basic information:
 
V2.1: file format version 2.1, timestamp of file creation, AP name, and date code YYMMDD
 
All timestamps are standard Unix timestamps (seconds since 1970). The other four comment 
lines describe the format of lines that occur later in the file.  Other than the timestamp 
and AP name, the rest of these fields are MIB variable names.
After the five comment lines comes a series of polls.  Each poll consists of one ''sys'' line, 
one ''if'' line describing stats of the the wireless interface, and zero or more pairs of 
''c1'' and ''c2'' lines, each pair describing a currently connected client.  The c1 and c2 
lines are a collection of MIB variables from the AWC MIB (Aironet Wireless Communications is 
the name of the company that developed our access points; Aironet was bought by Cisco 
who then branded and sold the APs under their name).
 
V2.1,1018929767,AdmBldg27AP2,020416
 
sys,timestamp,AP,sysUpTime
 
if,timestamp,AP,ifIndex,ifType,ifSpeed,ifInOctets,ifInUcastPkts,ifInErrors,ifInDiscards,ifOutOctets,ifOutUcastPkts,ifOutErrors,ifOutDiscards
 
c1,timestamp,AP,awcDot11TpFdbAddress,awcDot11TpFdbClientState,awcDot11TpFdbLatestRxSignalStrength,awcDot11TpFdbLatestRxSignalQuality
 
c2,timestamp,AP,awcTpFdbAddress,awcTpFdbClassID,awcTpFdbSrcOctetsImmed,awcTpFdbDestOctetsImmed,awcTpFdbIPv4Addr
  • description: The trace of SNMP polling every AP during Fall term 2003 and Winter term 2004.
  • last modified: 2006-11-14
  • dataname: dartmouth/campus/snmp/fall0304
  • version: 20041109
  • change: SNMP trace is resanitized.
  • release date: 2004-11-09
  • date/time of measurement start: 2003-11-01
  • date/time of measurement end: 2004-02-28
  • file: fall03.tar.gz

dartmouth/campus/tcpdump

Packet headers from every wireless packet sniffed in 27 buildings on Dartmouth College campus.

  • files: fall01, spring02, fall03, README.voip
  • description: The packet headers from every wireless packet sniffed in four (Fall01), five (Spring02), or 18 (Fall03) buildings on campus. The Fall 2001 data was used for [MobiCom 2002 paper]. The Fall 2001 and Spring 2002 data was used for [WiNet 2005 paper]. The 2003/4 data was used for [MobiCom 2004 paper]. This fall03 data also contains a list of device types, as determined using the OS fingerprinting tool p0f. Note that the MAC addresses in this list are only devices that we saw associate with an AP (i.e., that appeared in the syslog or SNMP data). Thus it does not include non-wireless client MAC addresses, such as routers or spoofed MACs that do not appear in syslog. The total compressed datasets are over 200 GB, so they are too large to post as tarballs. The best option is to use an http tool like curl or wget to download the whole Fall01, Spring02, or Fall03 directory from the web site. Or you can arrange to send us a USB or firewire drive (>250GB) and we can ship it back to you with all of our data on it. You can get the README, some of my analysis software, and the output of my own analysis programs listing the amount of traffic seen at each sniffer for each port number for each day, that is available as a small (660 KB) tgz file. NB: this is for the 2001/2 data.
  • measurement purpose: Usage Characterization
  • methodology: We used network ''sniffers'' to obtain detailed network-level traces. Due to the volume of traffic on the wireless network, it was impractical to capture all the traffic. Moreover, the structure of our WLAN, with several subnets, meant that there was no convenient central point for capturing wireless traffic. Instead, we installed 18 sniffers in 14 different buildings; in some large buildings, we needed multiple sniffers to monitor all of the building's APs. The buildings were among the most popular wireless locations in 2001, and included libraries, dormitories, academic departments and social areas. In total, our 18 sniffers covered 121 APs. Each sniffer was a Linux box with two Ethernet interfaces. One interface was used for remote access, to maintain the sniffer and to obtain the data for analysis. The other interface was used for collecting (''sniffing'') data. In each of the 18 switchrooms we attached the APs to a switch, and set another port on the switch to ''mirror'' mode, so that all the traffic on that switch would be sent to this port. The sniffer's second interface was attached to this mirrored port. We used tcpdump to capture any wireless traffic that came through these APs and their wired interfaces..
  • sanitization: To sanitize the MAC address, we randomized the bottom six hex digits. We collected every MAC address from all of our syslog, SNMP, an tcpdump traces, and built a huge table mapping real MACs to randomized MACs, ensuring that all mappings are unique. [We did not change either the MAC address 000000000000 or FFFFFFFFFFFF, they remain as they were.] We applied this mapping consistently across all data files of all types, so if you see a MAC address in the tcpdump files, and see it again in the SNMP trace, you can be sure it's the same client. We used a prefix-preserving IP address sanitizer, see Xu, J., Fan, J. Ammar, M., and Moon, S. ``On the Design and Performance of Prefix-Preserving IP Traffic Trace Anonymization'', Proc. of 10th IEEE International Conference on Network Protocols (ICNP 2002), Paris, France, November 2002. What this means is that you can compare the prefixes of the sanitized IP addresses, i.e. if two IP addresses share the same k-bit prefix, the sanitized addresses will also share the same k-bit prefix.
  • last modified: 2006-11-14
  • dataname: dartmouth/campus/tcpdump
  • version: 20041109
  • change: TCPDUMP traceset is resanitized.
  • release date: 2004-11-09
  • date/time of measurement start: 2001-09-25
  • date/time of measurement end: 2004-02-28
  • limitation: In both Fall01 and Spring02 datasets we lose a little data each day. We restarted tcpdump once a day, to cause it to begin a new log file. We killed the tcpdump process, then started a new one; as a result, some tcpdump data files end with partial packets, and no doubt we lost a few packets in the transition. We missed any traffic between two clients associated with the same AP, as this would not be sent via the AP's wired interface, but we believe this occurred rarely.
  • hole: There were unfortunate gaps in the data collection, generally caused by power failures.
  • error: The Fall01 data in particular suffers from a lot of corruption. It appears that older versions of tcpdump have a serious bug that causes them to record the MAC address of many frames incorrectly. In the Spring we used a newer tcpdump that did not have this problem.

dartmouth/campus/tcpdump Traces

    • fall01: Packet headers from every wireless packet sniffed on Dartmouth College campus during Fall 2001 term.
  • configuration: We collected data from four sniffers around the Dartmouth campus. Each sniffer was connected to a hub, along with the building's access points. The four buildings are representative: 1. Sudikoff (AcadBldg16): the Department of Computer Science (6 APs). 2. Brown (ResBldg100): a dormitory with many first-year students (2 APs). 3. Berry (LibBldg2): the main campus library. Due to the size of the building and the switched nature of its network, we were only able to sniff 5 of the 13 APs. 4. Collis/Thayer (SocBldg1): two buildings, the student center and dining hall, containing five cafes, several lounge areas, several meeting rooms, and some offices (total 9 APs).
  • format: tcpdump format
  • description: The packet headers from every wireless packet sniffed in four (Fall01) buildings on campus. The Fall 2001 data was used for [MobiCom 2002 paper] and [WiNet 2005 paper].
  • last modified: 2006-11-14
  • dataname: dartmouth/campus/tcpdump/fall01
  • version: 20041109
  • change: TCPDUMP trace is resanitized.
  • release date: 2004-11-09
  • date/time of measurement start: 2001-09-25
  • date/time of measurement end: 2001-12-10
  • directory: fall01
  • hole: % Sudi 3 GAPS of about 21h:17m:46s % Brown 15 GAPS of about 213h:10m:33s % Berry 7 GAPS of about 139h:17m:29s % Collis 8 GAPS of about 337h:1m:59s
  • limitation: We lose a little data each day. To keep file sizes small, and to be able to backup and manipulate the data more easily, we restarted tcpdump once a day, to cause it to begin a new log file. We killed the tcpdump process, then started a new one; as a result, some tcpdump data files end with partial packets, and no doubt we lost a few packets in the transition. [We note that newer versions of tcpdump have a roll-over mechanism built in; too bad it didn't exist for our tracing.] In the fall, we restarted tcpdump at midnight;
  • error: Data in particular suffers from a lot of corruption. It appears that older versions of tcpdump have a serious bug that causes them to record the MAC address of many frames incorrectly. These bogus MAC addresses, specifically 0:0:0:0:0:0, 0:0:0:0:0:1, 1:0:0:0:0:0, or 1:0:1:0:1:0, occurred in about 78% of all frames in the Fall data. For frames containing IP packets, we examined the source and destination IP address; if the IP address was associated with a valid, wireless MAC address in a recent IP packet, then we assumed this packet used the same MAC, and treated it as a wireless packet. We fixed about a third of bad MACs this way.
    • spring02: Packet headers from every wireless packet sniffed on Dartmouth College campus.
  • configuration: We collected data in the same way, in the same four locations, but added another sniffer on the 8 APs in Whittemore, which is a dormitory at the Tuck School of Business. This data is much cleaner than the fall data.
  • format: tcpdump format
  • description: The packet headers from every wireless packet sniffed in five buildings on campus. The Spring 2002 data was used for [WiNet 2005 paper].
  • last modified: 2006-11-14
  • dataname: dartmouth/campus/tcpdump/spring02
  • version: 20041109
  • change: TCPDUMP trace is resanitized.
  • release date: 2004-11-09
  • date/time of measurement start: 2002-03-25
  • date/time of measurement end: 2002-06-09
  • directory: spring02
  • hole: Berry (LibBldg2): 2 GAPS of about 0h:6m:34s Brown (ResBldg100): 0 GAPS of about 0h:0m:0s Collis (SocBldg1): 5 GAPS of about 27h:24m:22s Sudikoff (AcadBldg16): 0 GAPS of about 0h:0m:0s Whittemore (ResBldg83): 1 GAPS of about 12h:28m:15s
  • limitation: We lose a little data each day. To keep file sizes small, and to be able to backup and manipulate the data more easily, we restarted tcpdump once a day, to cause it to begin a new log file. We killed the tcpdump process, then started a new one; as a result, some tcpdump data files end with partial packets, and no doubt we lost a few packets in the transition. [We note that newer versions of tcpdump have a roll-over mechanism built in; too bad it didn't exist for our tracing.] We restarted at about 4AM when traffic was lightest.
    • fall03: Packet headers from every wireless packet sniffed on Dartmouth College campus during Fall 2003 term.
  • configuration: We increased the number of sniffers again, this time to 18, spread among the same buildings as before, but also with more dorms and social buildings
  • format: tcpdump format
  • description: The packet headers from every wireless packet sniffed in 18 buildings on campus. The 2003/4 data was used for [MobiCom 2004 paper]. This fall03 data also contains a list of device types, as determined using the OS fingerprinting tool p0f. a list of device types, as determined using the OS fingerprinting tool p0f (note that the MAC addresses in this list are only devices that we saw associate with an AP (i.e., that appeared in the syslog or SNMP data). Thus it does not include non-wireless client MAC addresses, such as routers or spoofed MACs that do not appear in syslog.). There is also a brief note about the VoIP data included in this trace.
  • last modified: 2006-11-14
  • dataname: dartmouth/campus/tcpdump/fall03
  • version: 20041109
  • change: TCPDUMP trace is resanitized.
  • release date: 2004-11-09
  • date/time of measurement start: 2003-11-02
  • date/time of measurement end: 2004-02-28

dartmouth/campus/movement

Two-year records showing the location (AP association) of each wireless card seen on campus.

  • files: movement.tar.gz, movement-v1.3.tar.gz, APlocations.csv
  • description: Over three years of nearly continuous records showing the location (access-point association) of each wireless card seen on campus. We used this data for our study of location predictors, published in [INFOCOM'04 paper] and a subsequent, expanded [technical report]. This data is derived from the syslog data. The trace used for this paper is gzipped tar file [51MB]. An updated movement history dataset (up to 2004-06-30) is gzipped tar file [166MB].
  • measurement purpose: User Mobility Characterization
  • methodology: We extracted user traces from dartmouth/campus/syslog. Each user's trace is a series of locations, that is, access-point names. We introduced the special location 'OFF' to represent the user's departure from the network (which occurs when the user turns off their computer or their wireless card, or moves out of range of all access points). The traces varied widely in length (the number of locations in the sequence). Users with longer traces were either more active (using their card more), more mobile (thus changing access points more often), or used the network for a longer period (some users have been on the network since April 2001, and some others have only recently arrived on campus).
  • sanitization: same as dartmouth/campus/syslog
  • last modified: 2007-01-31
  • dataname: dartmouth/campus/movement
  • version: 20050308
  • change: An updated movement history trace (up to 2004-06-30) is added.
  • release date: 2005-03-08
  • limitation: same as dartmouth/campus/syslog
  • hole: same as dartmouth/campus/syslog

dartmouth/campus/movement Traces

    • infocom04: Two-year records showing the location (AP association) of each wireless card seen on campus.
  • configuration: This trace is derived from the trace dartmouth/campus/syslog/01_04.
  • format: timestamp, associated AP
  • description: Over two years of nearly continuous records showing the location (access-point association) of each wireless card seen on campus. We used this data for our study of location predictors, published in [INFOCOM'04 paper] and a subsequent, expanded [technical report]. This data is derived from the syslog data. The trace used for this paper is gzipped tar file [51MB]. .
  • last modified: 2006-11-14
  • dataname: dartmouth/campus/movement/infocom04
  • version: 20040805
  • change: Infocom04 movement trace is newly created
  • release date: 2004-08-05
  • file: movement.tar.gz
    • 01_04: Three-year records showing the location (AP association) of each wireless card seen on campus.
  • configuration: This trace is derived from the trace dartmouth/campus/syslog/01_04.
  • format: timestamp, associated AP
  • description: Over three years of nearly continuous records showing the location (access-point association) of each wireless card seen on campus. This updated movement history dataset (up to 2004-06-30) is gzipped tar file [166MB].
  • last modified: 2007-01-31
  • dataname: dartmouth/campus/movement/01_04
  • version: 20050308
  • change: This trace is newly added.
  • release date: 2005-03-08
  • date/time of measurement start: 2001-04-01
  • date/time of measurement end: 2004-06-30
  • file: movement-v1.3.tar.gz
    • aplocations: A comma-separated list of most of the APs on campus and their location.
  • configuration: APlocations.csv: This file contains the locations of the APs. We used building blueprints and a campus map in AutoCAD format, and then painstakingly plotted most of the APs on the map. The rough correlation is approximately 1.7 coordinate units per foot. The Z coordinate is the floor. 99 means an unknown floor. -1 X/Y coordinates mean that we don't know where this AP is located. Tristan Henderson, November 2004
  • format: AP, x coordinate (-1 = unknown), y coordinate (-1 = unknown), z coordinate (floor, 99 = unknown)
  • description: A comma-separated list of most of the APs on campus and their location, as defined in coordinates on an AutoCAD map of the campus.
  • last modified: 2006-11-14
  • dataname: dartmouth/campus/movement/aplocations
  • version: 20041109
  • change: AP locations are added
  • release date: 2004-11-09
  • file: APlocations.csv
Instructions: 

The files in this directory are a CRAWDAD dataset hosted by IEEE DataPort. 

About CRAWDAD: the Community Resource for Archiving Wireless Data At Dartmouth is a data resource for the research community interested in wireless networks and mobile computing. 

CRAWDAD was founded at Dartmouth College in 2004, led by Tristan Henderson, David Kotz, and Chris McDonald. CRAWDAD datasets are hosted by IEEE DataPort as of November 2022. 

Note: Please use the Data in an ethical and responsible way with the aim of doing no harm to any person or entity for the benefit of society at large. Please respect the privacy of any human subjects whose wireless-network activity is captured by the Data and comply with all applicable laws, including without limitation such applicable laws pertaining to the protection of personal information, security of data, and data breaches. Please do not apply, adapt or develop algorithms for the extraction of the true identity of users and other information of a personal nature, which might constitute personally identifiable information or protected health information under any such applicable laws. Do not publish or otherwise disclose to any other person or entity any information that constitutes personally identifiable information or protected health information under any such applicable laws derived from the Data through manual or automated techniques. 

Please acknowledge the source of the Data in any publications or presentations reporting use of this Data. 

Citation: David Kotz, Tristan Henderson, Ilya Abyzov, Jihwang Yeo, CRAWDAD dataset dartmouth/campus (v. 2007‑02‑08), https://doi.org/10.15783/C79G6J, Feb 2007.

  

Dataset Files

Documentation

AttachmentSize
File dartmouth-campus-20070208-readme.txt1.72 KB

Related Datasets

CRAWDAD dartmouth/campus (v. 2009-09-09)

Submitted by: CRAWDAD Team

CRAWDAD dartmouth/campus (v. 2005-03-08)

Submitted by: CRAWDAD Team

CRAWDAD dartmouth/campus (v. 2004-12-18)

Submitted by: CRAWDAD Team

CRAWDAD dartmouth/campus (v. 2004-08-05)

Submitted by: CRAWDAD Team

CRAWDAD dartmouth/campus (v. 2004-11-09)

Submitted by: CRAWDAD Team

These datasets are part of Community Resource for Archiving Wireless Data (CRAWDAD). CRAWDAD began in 2004 at Dartmouth College as a place to share wireless network data with the research community. Its purpose was to enable access to data from real networks and real mobile users at a time when collecting such data was challenging and expensive. The archive has continued to grow since its inception, and starting in summer 2022 is being housed on IEEE DataPort.

Questions about CRAWDAD? See our CRAWDAD FAQ. Interested in submitting your dataset to the CRAWDAD collection? Get started, by submitting an Open Access Dataset.

QUESTIONS?

Login to Send Author a Private Message

Report a problem with this Dataset