The algorithmic and computational advances over the past decade have yielded significant performance boosts of machine learning (ML) models. As a result, we are witnessing rapid adoption of ML algorithms like classifiers and detectors, in real-world applications. Several industries use ML inside their workflows and manufacturing processes to improve on metrics like speed, scale, and productivity. However, ML models are not “battle-ready” in the sense that they can be easily broken. The fundamental assumption that is made in deploying virtually any current ML model is that the training data covers all possible variations in the field (test) – but this is rarely the case. Even if care is taken to ensure they are similar, ambient noise (like lighting changes, uncontrollable occlusions, etc.) from the sensing environment or faulty sensors can cause even the most sophisticated algorithms to fail. There is also evidence that ML algorithms are vulnerable to carefully designed adversarial corruptions which are invisible to the human observer but can result in incorrect results. The challenge in these scenarios is often that we do not have knowledge of the kind of corruption in advance making it hard to detect and correct for it.
MimicGAN represents a new generation of methods that can “self-correct” for unseen corruptions in the data out in the field. This is particularly useful for systems that need to be deployed autonomously without needing constant intervention such as Automated Driver Assistance Systems. MimicGAN achieves this by treating every test sample as “corrupt” by default. The goal is to determine (a) the clean image and (b) the corruption both of which are unknown to the system at test time. MimicGAN solves this by making alternating guesses between what the clean sample should look like and what corruption might make it look like the observed corrupted sample. If there is no corruption at all, MimicGAN simply learns the corruption to be an identity transform – i.e., no corruption. By operating in real-time with as few as 10 corrupted samples, MimicGAN serves as a very powerful pre-processing step to ML models deployed in the wild. It can even correct for adversarial corruptions better than most existing methods.
Preliminary results on images have demonstrated that a wide variety of significant unknown corruptions like missing pixels or geometric transformations (rotations, scale changes, noise, pixel intensity changes, etc.) can be accurately estimated from a small handful of examples. Furthermore, MimicGAN applies not only to images but to any data distribution that can be learned with modern neural networks including multi-modal scalars, spectra, time series, etc. This make MimicGAN a very flexible tool that can improve a wide variety of tasks. For example, using MimicGAN one can train models on simulated data but apply them to experiments, or use cheap low-fidelity data to inform high-fidelity models. Similarly, one may want to train from data of one healthcare system but apply the models to another, fit housing prices, energy consumption, or traffic in one city but predict in another etc. Ultimately, any application in which the test data may not be similar to the training data yet there is too little data or time to re-train a model will benefit from incorporating MimicGAN.
MimicGAN is a Machine Learning software solution that can:
- Be applied to wide class of problems where one has access to a large amount of synthetic or lab data but need predictions and inferences on relatively expensive “field data” with the highest accuracy possible.
- Generalize ML models to data not seen during training, especially corruptions from ambient noise, faulty sensors, uncontrollable environmental artifacts.
- Outperform existing baselines in blind image recovery
- Defend against adversarial attacks – (un)intentionally introduced corruptions
MimicGAN is application agnostic and can be designed to work with any system that makes decisions based on sparse data.
Examples in which this software is ideally targeted for are:
- Automated Driver Assistance Systems
- Health care
- Remote sensing
- Medical imaging
- Non-destructive evaluation
MimicGAN is fully developed and functional. Preliminary results on images have demonstrated that a wide variety of significant corruptions, i.e. missing pixels, geometric transformations, noise, etc. can be accurately estimated from a small handful of high-fidelity examples.
A publication on the results can be found here:
Anirudh. R., Thiagarajan, J. J., Kailkhura, B., Bremer, P.T., MimicGAN: Robust Projection onto Image Manifolds with Corruption Mimicking. (to appear in) Int J Comput Vis (2020). DOI:10.1007/s11263-020-01310-5
Preprint is available here: https://arxiv.org/abs/1912.07748
LLNL has filed a patent application for this algorithmic protocol.