Cophy: Counterfactual Learning of Physical Dynamics

Cophy: Counterfactual Learning of Physical Dynamics

Citation Author(s):
Fabien
Baradel
INSA Lyon, LIRIS
Natalia
Neverova
Facebook AI Research
Julien
Mille
INSA Val de Loire, LIFAM
Greg
Mori
SFU, Borealis AI
Christian
Wolf
INSA Lyon, LIRIS
Submitted by:
Fabien Baradel
Last updated:
Mon, 01/27/2020 - 10:07
DOI:
10.21227/ps5q-8m55
Links:
License:
Dataset Views:
36
Rating:
0
0 ratings - Please login to submit your rating.
Share / Embed Cite

CATEGORIES

Abstract: 

Understanding causes and effects in mechanical systems is an essential component of reasoning in the physical world. This work poses a new problem of counterfactual learning of object mechanics from visual input. We develop the COPHY benchmark to assess the capacity of the state-of-the-art models for causal physical reasoning in a synthetic 3D environment and propose a model for learning the physical dynamics in a counterfactual setting. Having observed a mechanical experiment that involves, for example, a falling tower of blocks, a set of bouncing balls or colliding objects, we learn to predict how its outcome is affected by an arbitrary intervention on its initial conditions, such as displacing one of the objects in the scene. The alternative future is predicted given the altered past and a latent representation of the confounders learned by the model in an end-to-end fashion with no supervision. We compare against feedforward video prediction baselines and show how observing alternative experiences allows the network to capture latent physical properties of the environment, which results in significantly more accurate predictions at the level of super human performance.

Instructions: 

Please follow the instructions given in the github repo: https://github.com/fabienbaradel/cophy

Dataset Files

You must be an IEEE Dataport Subscriber to access these files. Login or subscribe now. Sign up to be a Beta Tester and receive a coupon code for a free subscription to IEEE DataPort!

Thank you for rating this dataset!

Please share additional details of your rating with the IEEE DataPort community by adding a comment.

Embed this dataset on another website

Copy and paste the HTML code below to embed your dataset:

Share via email or social media

Click the buttons below:

facebooktwittermailshare
[1] Fabien Baradel, Natalia Neverova, Julien Mille, Greg Mori, Christian Wolf, "Cophy: Counterfactual Learning of Physical Dynamics", IEEE Dataport, 2020. [Online]. Available: http://dx.doi.org/10.21227/ps5q-8m55. Accessed: Feb. 28, 2020.
@data{ps5q-8m55-20,
doi = {10.21227/ps5q-8m55},
url = {http://dx.doi.org/10.21227/ps5q-8m55},
author = {Fabien Baradel; Natalia Neverova; Julien Mille; Greg Mori; Christian Wolf },
publisher = {IEEE Dataport},
title = {Cophy: Counterfactual Learning of Physical Dynamics},
year = {2020} }
TY - DATA
T1 - Cophy: Counterfactual Learning of Physical Dynamics
AU - Fabien Baradel; Natalia Neverova; Julien Mille; Greg Mori; Christian Wolf
PY - 2020
PB - IEEE Dataport
UR - 10.21227/ps5q-8m55
ER -
Fabien Baradel, Natalia Neverova, Julien Mille, Greg Mori, Christian Wolf. (2020). Cophy: Counterfactual Learning of Physical Dynamics. IEEE Dataport. http://dx.doi.org/10.21227/ps5q-8m55
Fabien Baradel, Natalia Neverova, Julien Mille, Greg Mori, Christian Wolf, 2020. Cophy: Counterfactual Learning of Physical Dynamics. Available at: http://dx.doi.org/10.21227/ps5q-8m55.
Fabien Baradel, Natalia Neverova, Julien Mille, Greg Mori, Christian Wolf. (2020). "Cophy: Counterfactual Learning of Physical Dynamics." Web.
1. Fabien Baradel, Natalia Neverova, Julien Mille, Greg Mori, Christian Wolf. Cophy: Counterfactual Learning of Physical Dynamics [Internet]. IEEE Dataport; 2020. Available from : http://dx.doi.org/10.21227/ps5q-8m55
Fabien Baradel, Natalia Neverova, Julien Mille, Greg Mori, Christian Wolf. "Cophy: Counterfactual Learning of Physical Dynamics." doi: 10.21227/ps5q-8m55