FUZZ-IEEE Competition on Explainable Energy Prediction

Abstract

Predicting energy consumption is currently a key challenge for the energy industry as a whole. Predicting the consumption in a certain area is massively complicated due to the sudden changes in the way that energy is being consumed and generated at the current point in time. However, this prediction becomes extremely necessary to minimise costs and to enable adjusting (automatically) the production of energy and better balance the load between different energy sources.

Smart meters are being rolled-out in many countries for domestic use, becoming powerful devices to track energy use. Smart meters will not only play an increasingly large role in the way customers consume energy but also in the way they choose a supplier. If energy providers developed new products for their customers that help them understand the potential benefits of smart meters (e.g. reducing bills or managing their finances better), this could be key to better serving, and thus attracting and retaining customers. At a higher level, being able to use smart meter data to better manage demand will contribute to the increasing use of sustainable sources of power and reduce the demands on more traditional power generation.

In early 2020, the IEEE Computational Intelligence Society (IEEE-CIS) partnered up with one of the leading international energy provider, E.ON SE, seeking the best solutions for energy prediction using smart meters, and held a competition with great success (see more here). The goal of that competition was to predict monthly and yearly consumption from a limited amount of data, in which the evaluation has been focused on relative errors at predicting energy consumption.

In this competition, the same prediction problem applies. However, the challenge is on building not only accurate but also explainable predictions. From a machine learning point of view, the provided data is very challenging and may lead to the development of novel learning approaches. Some of the challenges include incomplete data (i.e. missing values), use of external data sources to handle seasonal effects, different kinds of households (e.g. families vs. single, old house vs. new built), or lack of sufficient information about the households. From the point of view of Explainable Artificial Intelligence, we are looking for self-explaining models, i.e., accurate predictions coming along with a narrative explanation in natural language easy to understand by customers.

This competition will be hosted at the 2021 FUZZ-IEEE Conference (Luxembourg):

https://attend.ieee.org/fuzzieee-2021/competition/

Please report any issues or feedback to Isaac.Triguero@nottingham.ac.uk

Instructions:

Data description and submission requirements

The goal of this competition is to provide 3248 customers with accurate and self-explaining predictions of their future monthly electricity consumption in a coming year (January to December). You are provided with historical half-hourly energy readings for 3248 smart meters. To simulate a realistic use case, we take the 1st of January of a given year as the day we want to make predictions. Thus, different smart-meters will have available a range of months’ worth of consumption, ranging from only last month (i.e. December) to the entire last year (January to December), acknowledging that customers may have joined at different times during the previous year. For example, we may have a few customers for which we only have data from last December, and we aim to predict January to December of the coming year, whilst for others, we may have the entire January-to-December time series. We would like to see how well we can predict and explain the energy consumption in the coming year depending on the amount of data that is available.

For each meter_id in the provided dataset, you must predict the consumption in the following 12 months in kWh. Each prediction (monthly and annual) must come with a narrative explanation in natural language (English). Such explanations must be generated automatically.

Important:

(1) In addition to their predictions and explanations, participants are requested to submit a description of their methodology in their final submission (up to 1000 words). This is compulsory as this description will also be evaluated by the committee to shortlist the top 5 submissions (i.e., those submissions with the best explainability-accuracy traded-off). Please use the ‘Abstract’ field to include this information when submitting your final analysis.

(2) Teams: If you are working on a team, you should only use one IEEE Account to submit your solutions. In case that during the competition you decide to work together with someone who already takes part in the competition you have to let the organisers know as soon as possible and start using only one of the accounts.

Full description of the data and submission requirements can be downloaded here.

Evaluation

Our partner E.ON is interested in predicting with the least amount of data as possible a good estimate of the total year consumption for reasons of payment adequacy. Accurately predicting a customer’s annual consumption allows E.ON to set their direct debit payments up correctly. Making predictions for smart meters for which we have very little historical consumption data is usually more challenging and relevant to avoiding excessive debt or credit at the end of the year. In addition to that, an accurate monthly consumption prediction is also very valuable for Energy Trading teams, who need to be able to predict with some accuracy how much electricity to buy on the energy market. Moreover, E.ON demands not only accurate but self-explaining predictions (i.e., accountable, transparent, and easy to understand) with the aim of providing customers with trustworthy products; thus attracting and retaining more customers. This is in agreement with the European General Data Protection Regulation (GDPR) which states that humans have a right to an explanation of decisions affecting them, no matter who (or what intelligent system) makes such decisions.

Details about the evaluation can be found here.

See the Leaderboard and all submissions at the bottom of the page.

Note that the Leader Board table is updated ONLY once a day to avoid overfitting the test set. In case you submit a wrongly formatted submission, you will be listed at the top of the submissions' table with an empty row.

Finalists will be ranked based on an aggregated score of prediction and explanation performance. The jury will be looking at both error rates and explanations as a whole. The overall performance-explainability trade-off includes annual and monthly predictions.

At the end of the competition, the best solutions will be invited to write a joint paper discussing the problem and solutions.

Important:

(1) Note that the use of data other than the one provided is not allowed. If you do so, we will not consider your submission.

(2) For the sake of reproducibility, to guarantee that the description you provide of your method returns the results you have submitted, you are asked to provide the source code associated to your final submission. We understand that this might involve running multiple software tools and various steps. So, please provide a detailed description and requirements in a README file, and if possible, some sort of simple script (e.g. in R, Python, or Unix) to run it. Note that the code will solely use by the Technical Committee to guarantee the fairness of the competition. We will not share this with any third party or make any commercial use of it, as this belongs to you.

Timeline

- December 21st, 2020 – Competition start.

- May 15th, 2021 – Final submission deadline.

- June 1st, 2021 – Shortlisting announcement.

- July 11th to July 14th - Special session at the IEEE International Conference on Fuzzy Systems, Luxembourg. Presentation of the 5 best solutions.

All deadlines are at 11:59 PM UTC on the corresponding day unless otherwise noted. The competition organisers reserve the right to update the contest timeline if they deem it necessary.

Organisers:

Isaac Triguero (University of Nottingham)
Jose María Alonso (University of Santiago de Compostela)
Luis Magdalena (Universidad Politécnica de Madrid)
Christian Wagner (University of Nottingham)
Juan Bernabé-Moreno (Chief Data Officer E.ON SE)

Scientific Committee

TBA

Acknowledgements

E.ON SE provided the dataset for this competition. E.ON is an international, privately owned energy supplier based in Essen, Germany. With a clear focus on two strong core businesses E.ON aim to become the partner of choice for energy and customer solutions. E.ON provide solutions for the new energy world. The Chairs of the IEEE-CIS Technical Challenge Committee would like to extend our thanks to:

- Dr Robert Eigenmann, Nicholas Harbour and Dr Stefan Birr from the E.ON team.

- Selvi Ergen, KTP associate, funded by E.ON and Innovate UK under KTP project 11094.

- Heda Song, PhD student and Researcher at the University of Nottingham.