FLAIM - anonymize system and network logs.
FLAIM is a multi-level, multi-log anonymization tool. Note: as of March 2021, the FLAIM website is no longer available, and CRAWDAD's copy of the tools has been removed at the contributor's request. FLAIM-Core comprises the anonymization engine and XML based policy manager. FLAIM-Core loads dynamic libraries responsible for I/O and parsing at runtime. There is a library for each type of log flaim supports. The XML policy, the I/O module, input file and output files are all specified on the command line.
the initial version
Copyright © 2005-2007 The Board of Trustees of the University of Illinois. All rights reserved. Developed by: LAIM Working Group National Center for Supercomputing Applications University of Illinois Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal with the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: - Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimers. - Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimers in the documentation and/or other materials provided with the distribution. - Neither the names of the National Center for Supercomputing Applications, the University of Illinois, nor the names of its contributors may be used to endorse or promote products derived from this Software without specific prior written permission. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE SOFTWARE.
FLAIM is an open source project, and we welcome your participation. Feature requests, bug reports, success stories, and software patches are some examples of valuable community contributions. For discussions about FLAIM, please join the email@example.com discussion list. You must be subscribed to post to the list. To subscribe to the firstname.lastname@example.org list, send email to email@example.com with "subscribe flaim-users" in the body of the message. Release announcements and security advisories are also posted to firstname.lastname@example.org. Only the LAIM PI can post to this list, and messages should be infrequent. To subscribe to the email@example.com list, send email to firstname.lastname@example.org with "subscribe flaim-announce" in the body of the message.
1. Install any missing dependencies. FLAIM has been tested on Linux 2.6, Mac OS 10.4, FreeBSD 6.1, OpenBSD 3.9 and NetBSD 3.0.1. FLAIM has dependencies on the following libraries. - LIBXML: Libxml is an open source C library for parsing and validating XML files. It is available on most Linux and BSD systems. It can be downloaded from http://xmlsoft.org/. Note that you will need both the binary library as well as the development files to compile FLAIM. For binary packages - to be released in the near future - you will not need the developer's package. - LIBXSLT: Libxslt is a open source C library for parsing XSLT style-sheets. It is available on most Linux and BSD systems. It can be downloaded from http://xmlsoft.org/XSLT/. Note that you will need both the binary library as well as the development files to compile FLAIM. For binary packages - to be released in the near future - you will not need the developer's package. 2. Download FLAIM-Core and at least one I/O module. You can find FLAIM and its supported modules on the Downloads tab. Download FLAIM-Core and the modules corresponding to the types of logs you would like to process. 3. Install FLAIM-Core You must unpack the tarball, run the config script, and make the package. An example follows. [yoursystem]:$ tar zxf flaim-core-<version>.tgz [yoursystem]:$ cd flaim-core-<version> [yoursystem]:$ ./configure .... lots of output here ...... [yoursystem]:$ make [yoursystem]:$ make install 4. Install a FLAIM Module You must unpack the tarball, run the config script, and make the package. An example follows. [yoursystem]:$ tar zxf flaim-module-<modulename>-<version>.tgz [yoursystem]:$ cd flaim-module-<modulename>-<version> [yoursystem]:$ ./configure .... lots of output here ...... [yoursystem]:$ make [yoursystem]:$ make install * Installing in Non-Standard Locations * The default installation behavior is to create /usr/local/flaim and place all the libraries and configuration files there. In addition, a symbolic link is made to /usr/bin/flaim. One must normally be root to install in these directories, or to install the man page. To install FLAIM in a different location, one can pass an option to the configure script. The option to set is prefix. The default value for this variable is /usr/local. FLAIM then installs to $(prefix)/flaim, i.e. /usr/local/flaim. Say, instead, a user wants to install FLAIM into /usr/flaim. He would then pass the option to the configure script as follows: [yoursystem]:$ ./configure --prefix=/usr If the installation directory is changed for FLAIM-Core, it must be changed in the same way when modules are installed. Again, this can be done with the same options for their configure scripts.
* Synopsis * flaim [ OPTION ] [ -m module-name ] [ -p policy ] [ -i input.log ] [ -o output.log ] * Description * flaim is a multi-level, multi-log anonymization tool. FLAIM-Core comprises the anonymization engine and XML based policy manager. FLAIM-Core loads dynamic libraries responsible for I/O and parsing at runtime. There is a library for each type of log flaim supports. The XML policy, the I/O module, input file and output files are all specified on the command line. * Options * -c --config <file-name> Read config data from the specified file. If unspecified, the default of $(FLAIM_ROOT)/flaim.cfg is used. The default is recommended for all but the most advanced users. -h --help Display the usage information and exit. -i --input <file-name> Specifies the source log for anonymization. If unspecified, stdin will be used. However, not all modules support reading input from stdin. Those that do not support streaming will exit and force you to specify a file name. -l --list Lists all installed modules. FLAIM will not find manually installed modules in non-standard locations. -m --module <module-name> Load the specified parsing module. The -l option shows the valid choices. Either this option must be used to specify a module installed in the default location, or the -M option must be used. -M --moduleLib <module-lib-path> Load the module library from the given path. This option is mutually exclusive with -m and used for explicitly specifying the module location. It is also necessary to specify the schema location using -s when using this option. -o --output <file-name> Specifies the destination file for anonymized data. If unspecified, stdout will be used. However, not all modules support writing output to stdout. Those that do not support streaming, will exit and force you to specify a file name. -p --policy <file-name> The use of this flag is mandatory as it specifies the location of the user policy. -s --schemaModule <module-schema-file-path> Load the module schema from the file specified with this option. This option is used if and only if the -M option is used. -v --verbose Print verbose messages to stderr. -V --version Print version information to stderr and exit. -x --xtraConfig <file-name> This is used to specify a file containing extra information to be passed to the parsing module. It is optional and ignored by most modules. * See Also * formerly at this location: flaim.ncsa.uiuc.edu/documentation.html * Bugs * This is a beta release of flaim and we expect bugs to be found. We suggest users that want to hear about latest bug findings (or submit bugs themselves) to join the email@example.com mailing list. Instructions for joining this and other flaim lists were found at flaim.ncsa.uiuc.edu/support.html * Environment * The environment variable FLAIM_ROOT specifies the location of flaim and, consequently, the location of the FLAIM modules. This environment variable is setup in the default installation of FLAIM-Core.
* Sample Logs and Policies [flaim-samples-0.5.1.tar.gz] * We have provided sample logs and FLAIM policies for users to try out FLAIM. We encourage you to play with the anonymization policy to explore FLAIM's capabilities. The tarball 'flaim-samples-0.5.1.tar.gz' contains the following sample logs and policies: - For 'pcap' module: sample.pcap.log sample-pcap.apolicy.xml - For 'iptable' module: sample.iptable.log sample-iptable.apolicy.xml - For 'nfdump' module: sample.nfdump.log sample-ndump.apolicy.xml
The following people also contributed to the development of FLAIM: Vikram Dhar Greg Colombo Jun Wang Bill Yurcik Yifan Li
The files in this directory are a CRAWDAD toolset hosted by IEEE DataPort.
About CRAWDAD: the Community Resource for Archiving Wireless Data At Dartmouth is a data resource for the research community interested in wireless networks and mobile computing.
CRAWDAD was founded at Dartmouth College in 2004, led by Tristan Henderson, David Kotz, and Chris McDonald. CRAWDAD toolsets are hosted by IEEE DataPort as of November 2022.
Note: Please use the tools in an ethical and responsible way with the aim of doing no harm to any person or entity for the benefit of society at large. Please respect the privacy of any human subjects whose wireless-network activity is captured by the tools and comply with all applicable laws, including without limitation such applicable laws pertaining to the protection of personal information, security of data, and data breaches. Please do not apply, adapt or develop algorithms for the extraction of the true identity of users and other information of a personal nature, which might constitute personally identifiable information or protected health information under any such applicable laws. Do not publish or otherwise disclose to any other person or entity any information that constitutes personally identifiable information or protected health information under any such applicable laws derived from the tools through manual or automated techniques.
Please acknowledge the source of the tools in any publications or presentations reporting use of this tools.
Kiran Lakkaraju, Xiaolin Luo, Adam Slagell, CRAWDAD toolset tools/sanitize/generic/FLAIM (v. 2006‑12‑05) https://doi.org/10.15783/C7N88B, Dec 2006.
These datasets are part of Community Resource for Archiving Wireless Data (CRAWDAD). CRAWDAD began in 2004 at Dartmouth College as a place to share wireless network data with the research community. Its purpose was to enable access to data from real networks and real mobile users at a time when collecting such data was challenging and expensive. The archive has continued to grow since its inception, and starting in summer 2022 is being housed on IEEE DataPort.