Instructions for Accessing an IEEE DataPort Data Competition

Background

• IEEE DataPort serves as an efficient and effective platform for Data Competitions.  Data Competitions can be initiated on IEEE DataPort and datasets up to 2TB  for individuals or 10TB for Institutional  Subscribers are accepted (contact IEEE DataPort team if your dataset is larger than 10TB).  Instructions for the Data Competition can be stored and accessed by Data Competition participants.   
• To initiate an IEEE DataPort Data Competition, please see the Instructions for Initiating a Data Competition found on the IEEE DataPort site.
• The Data Competition initiator can either make the competition open to all IEEE DataPort users or restrict access to the Data Competition.  If a Data Competition is restricted, individuals wanting to participate may be requested to complete a registration form. 
• Data Competition participants may download Data Competition dataset files or access these files “in the cloud” using tools and services that support the AWS S3.  
• Data Competition analysis results can be uploaded by participants to the associated Data Competition.

Prerequisites

• User must have an IEEE account.   If you do not have an IEEE account, you may register for a free IEEE account (click this link to register).
• You may be, but do not need to be, an IEEE DataPort subscriber to participate in a Data Competition. You may subscribe to IEEE DataPort here.

Instructions

1. Login to IEEE DataPort (the login link is located in the upper right of the website)
2. Click on the COMPETITIONS link on the top banner
3. Find the Data Competition you are interested in to get a link to the dataset and the instructions for the competition.
4. If the Data Competition dataset is small enough to be downloaded to the participant’s local resource, the link to the dataset shown in the right column of the Data Competition site can be clicked and the participant will then be able to work with and/or save the dataset file.
5. If a Data Competition participant prefers to access the competition dataset in AWS S3, the participant should click the tab shown at the top of the right hand column which says AWS S3.  The S3 URI will then be revealed to the participant.
6. Click the View AWS Security Credentials, find and record the AWS Secret Access Key, which will be shown in a dialog.
7. Copy the S3 URIs of the competition dataset file(s) you wish to access.
   1. When you click on individual files under the AWS S3 tab, the S3 URI will be automatically selected. You may then copy (Ctrl-C, Cmd-C) and paste the URI.
   2. To copy all of the URIs for your dataset you may also click the Copy All URIs to the Clipboard link at the bottom of the AWS S3 tab. Note: some browsers do not support this feature, in which case a dialog is opened with all S3 URI’s listed and highlighted for easy copying.
8. Using your AWS Keys and the S3 URIs you copied, you will now be able to access these files using a number of tools, scripts, and services that support AWS S3.*
9. Congratulations – you now have access to the Data Competition dataset and the instructions to participate in the Data Competition!  Good luck!

*See below for links to some of the methods of accessing S3 files using your AWS keys. Note: the appropriate tool/service/script is dependent on the format of the dataset files you are trying to access. Instructions available on the dataset page may be helpful in selecting the appropriate tools.

Links to Tools and Services supporting S3

• Amazon Big Data Services
• Amazon Elastic Map Reduce (EMR)
  • See how to Import files with Distributed Cache for EMR 
• Amazon Red Shift
• See Redshift Credentials section for using your AWS keys to access data
• See Configuring the AWS Command Line Interface for information on using your AWS keys
• Amazon Command Line Interface
• Amazon QuickSight
• Python (SDK)
• Java (SDK)
• R (tutorial)
• S3 REST API
• Developer tools for S3
• Programming with AWS: