Dr. Max Ehrlich

I am a Research Scientist at NVIDIA hardware engineering. I am also affiliated with the University of Maryland Computer Science Department where I lead the Computational Entomology group, part of the Perception and Intelligence lab.

My current research combines machine learning and computational imaging to solve real problems. My focus is on breaking down and understanding the first principles of the problem and then building these principles back up into a machine learning solution rather than treating the model as a black box.

In the past I have sucessfully applied this idea to image enhancement. The broader impact of this is to improve participation from underrepresented groups For example, by creating better multimedia compression algorithms which incorporate simple deep learning based techniques, people operating in underinvested locations (e.g., rural areas, native american reservations, 3rd world countries) are able to participate in an increasingly media-focused internet. I am grateful to have had recognition of the importance of this work by many funding partners over the years including government agencies: DARPA and IARPA, and private companies: Facebook AI, Adobe DIL, and NVIDIA ADLR (where I currently work).

I received my Ph.D. in Computer Science from the University of Maryland where I was co-advised by Professor Larry Davis and Professor Abhinav Shrivastava. I received an M.S. in Computer Science from Stevens Institute of Technology. where I was advised by Professor Philippos Mordohai and a B.S. in Computer Science from Rutgers University.

I am a member of the Association for the Advancement of Artificial Intelligence (AAAI).

Want to see more interesting research? Check out my wife Dr. Sujeong Kim!

11/22 - Awarded the Larry S. Davis Doctoral Dissertation Award.
10/22 - One paper accepted to the MCV workshop at ECCV 2022.
8/22 - Started as a Research Scientist at NVIDIA.
5/22 - Graduated with a Ph.D.
3/22 - Successfully defended my dissertation.
8/21 - One paper accepted to the MELEX workshop at ICCV 2021.
7/21 - Started as a Research Intern at NVIDIA.
7/21 - Serving as a Research Mentor at the Summer STEM Institute.
7/20 - One paper accepted to ECCV 2020.
7/20 - Started as a Visiting Resarch Engineer at Facebook AI.
7/19 - One paper accepted to ICCV 2019.

Service

Conference Reviewer: AAAI 2020, ICLR 2020, ECCV {2020, 2022}, IJCAI 2021, CVPR {2021, 2022}, ICML 2021, ICCV 2021, WACV 2022

Journal Reviewer: Transactions on Image Processing (TIP), International Journal of Artifical Intelligence (IJAI), The Visual Computer (TVCJ), Transactions on Circuits and Systems for Video Technology (TCSVT)

Contact

Contact me by email at 

mehrlich {at} nvidia {dot} com

Teaching

Spring 2022 CMSC422 Intro to Machine Learning

Students

My Research

My research emphasizes broad impact and collaboration with outside agencies. Aside from these research programs, I have participated in many other published research projects, please see my full list of papers and patents below for more information.

Video Compression

Video sharing is increasingly popular and quickly becoming the primary method for interaction on the internet. With the globlal pandemic, video conferencing has become mandatory for many people to work or attend school. This causes major problems for people who lack a broadband connection. In this ongoing paper series on video compression, I am developing ways to incorporate deep learning models which run on commodity hardware and can be used in the near term. This research is conducted in collaboration with NVIDIA.

Learn More About This Research

JPEG Compression

JPEG compression is the most popular image compression algorithm and currently powers image sharing on the internet and mobile phones. In this paper series on JPEG compression, I advanced theoretical knowledge about the interaction between JPEG compression and deep learning and used these theoretical results to improve the fidelity of JPEG images both for human and machine consumption. This research was primarly funded by a three year academic grant awarded to me by Facebook (Meta) AI, allowing me to work autonomously, and led to collaborations with Facebook.

Learn More About This Research

Remote Sensing

In this program, we developed novel methods for improving land cover segmentation in sattelite images. This is a challenging and important problem with wide application from national defense to planning and surveying. This research was funded by the IARPA Core3D program.

Learn More About This Research

  Video Compression

Video sharing is increasingly popular and quickly becoming the primary method for interaction on the internet. With the globlal pandemic, video conferencing has become mandatory for many people to work or attend school. This causes major problems for people who lack a broadband connection. In this ongoing paper series on video compression, I am developing ways to incorporate deep learning models which run on commodity hardware and can be used in the near term. This research is conducted in collaboration with NVIDIA.

MetaBit: Leveraging Bitstream Metadata for Fast and Accurate Video Compression Correction Paper Cite It!- The current state of deep-learning for video compression is far behind classical compression. While deep-learning based codecs can generate beautiful images, they are slow and require significant hardware resources and software packages to run. In this work, we show that simply using commodity H.264 compression, the most popular video compression algorithm, along with a lightweight deep model for restoration, we can match or outperform fully deep-learning based codecs. Furthermore, we can leverage prior knowledge of classical video compression to make this process extremely efficient. This has the advantages of fast encoding with no custom hardware requirements and a fully decodable stream for consumers who lack the custom software package.

  JPEG Compression

JPEG compression is the most popular image compression algorithm and currently powers image sharing on the internet and mobile phones. In this paper series on JPEG compression, I advanced theoretical knowledge about the interaction between JPEG compression and deep learning and used these theoretical results to improve the fidelity of JPEG images both for human and machine consumption. This research was primarly funded by a three year academic grant awarded to me by Facebook (Meta) AI, allowing me to work autonomously, and led to collaborations with Facebook.

CRAB: Compression Robustness Analysis Benchmark Paper Cite It! - The most comprehensive study of the effect of JPEG compression to date! The CRAB system allows for fast, easy, and consistent benchmarking of deep learning methods when their inputs are JPEG compressed, as well as how they behave under various JPEG mitigation techniques including a new one we developed that is entirely self-supervised. We used CRAB to benchmark 20 commonly used models across three tasks: classification, detection, and segmentation (instance and semantic). Stay tuned for the CRAB code release, which will allow researchers to benchmark their own models and submit the results for inclusion into the study as well as the study website detailing our findings. In the meantime, check out our preprint on arXiv (https://arxiv.org/abs/2011.08932) which contains details of the study as well as the complete results. This work was published in the MELEX workshop at the International Conference on Computer Vision.
Quantization Guided JPEG Artifact Correction Project Page Code Talk Slides Talk Video Paper Cite It! - We develop a novel method for JPEG artifact correction Wthat solves three major problems left open in prior works:
  1. Prior works train an ensemble of models, one for each JPEG quality. We use a single network parameterized by the JPEG quantization matrix.
  2. Prior works deal with grayscale images only, with the assumption that their models can be applied channel-wise. We show that single-channel networks have trouble generalizing and design a network for color correction.
  3. Prior works focus on CNN regression which causes blurry and textureless results. We introduce a novel GAN loss that includes an explicit texture restoring term, this yields a more realistic result.
Our method achieves state-of-the-art results on color artifact correction. The paper was published in the proceedings of the European Conference on Computer Vision. I strongly recommend reading the arXiv version which includes the appendices.
JPEG Domain Residual Networks Project Page Colab Code Poster Paper Cite It! - In this work we develop the popular Residual Network architecture in the JPEG domain. Our goal is to produce a formulation which gives a result that is as close as possible to the spatial domain network, but which can operate on compressed JPEG images. Our formulation is generic and has applicability outside of classification objective that we show as an example. We show a notable performance increase by processing in the JPEG domain. This work was funded by Facebook and published in the proceedings of the International Conference on Computer Vision 2019.

  Remote Sensing

In this program, we developed novel methods for improving land cover segmentation in sattelite images. This is a challenging and important problem with wide application from national defense to planning and surveying. This research was funded by the IARPA Core3D program.

Unsupervised Super-Resolution of Satellite Imagery for High Fidelity Material Label Transfer. Paper Cite It! - One major outstanding problem for training deep networks for ground material segmentation is the lack of large amounts of high quality data. In this paper we present a method for super-resolving low resolution sattelite images in a way which preserves important properties of their semantic material labels. This allows low-resolution images, which are plentiful, to be mixed in with high-resolution images to improve the performance of existing ground material segmentation methods. This paper was published in the Proceedings of the International Geoscience and Remote Sensing Symposium.
Stacked U-Nets for Ground Material Segmentation in Remote Sensing Imagery. Paper Cite It! - We develop a novel method for ground material segmentation from satellite images. This method relies on the recent Stacked, Dilated U-Nets method which has good results and is efficient on the large images that remote sensing generates. We also propose a novel frequency-domain post processing which reduces spurrious artifacts generated by the deep model. Our method won 3rd place in the 2018 CVPR DeepGlobe challenge and was published in the CVPR proceedings.

Full List of Papers and Patents

Google Scholar dblp

Total Publications: Total Citations: h-index:

Loading citation data from Google Scholar

2022

The First Principles of Deep Learning and Compression
Max Ehrlich
Doctoral Dissertation, 2022
arXiv Cite It!
Leveraging Bitstream Metadata for Fast and Accurate Video Compression Correction
Max Ehrlich, Jon Barker, Namitha Padmanabhan, Larry S. Davis, Andrew Tao, Bryan Catanzaro, Abhinav Shrivastava
Under Submission, 2022 (arXiv available)
arXiv Cite It!

2021

A Frequency Perspective of Adversarial Robustness
Shishira R. Maiya, Max Ehrlich, Vatsal Agarwal, Ser-Nam Lim, Tom Goldstein, Abhinav Shrivastava
Under Submission, 2021 (arXiv available)
arXiv Cite It!
Interpretable Automated Diagnosis of Retinal Disease using Deep OCT Analysis
Evan Wen and Max Ehrlich
Under Submission, 2021 (arXiv available)
arXiv Cite It!
Analysing and Mitigating JPEG Compression Defects in Deep Learning
Max Ehrlich, Larry Davis, Ser-Nam Lim, Abhinav Shrivastava
In Proceedings of the IEEE International Conference on Computer Vision Workshops, 2021
arXiv CVF Cite It!

2020

Quantization Guided JPEG Artifact Correction
Max Ehrlich, Larry Davis, Ser-Nam Lim, Abhinav Shrivastava
In Proceedings of the European Conference on Computer Vision, 2020
arXiv ECVA Cite It!

2019

Unsupervised Super-Resolution of Satellite Imagery for High Fidelity Material Label Transfer
Arthita Ghosh, Max Ehrlich, Larry Davis, Rama Chellappa
In IEEE International Geoscience and Remote Sensing Symposium, 2019
arXiv IEEE Cite It!
Deep Residual Learning in the JPEG Transform Domain
Max Ehrlich and Larry S. Davis
In Proceedings of the IEEE International Conference on Computer Vision, 2019
arXiv CVF Cite It!
Deep Multi-Task Representation Learning.
Mohamed R. Amer, Timothy J. Shields, Amir Tamrakar, Max Ehrlich, Timur Almaev
U.S. Patent Application 16/085,859 filed January 31, 2019
Google Cite It!

2018

Stacked U-Nets for Ground Material Segmentation in Remote Sensing Imagery.
Arthita Ghosh, Max Ehrlich, Sohil Shah, Larry Davis, Rama Chellappa
In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2018
CVF Cite It!

2017

Action-Affect-Gender Classification using Multi-Task Representation Learning
Timothy J. Shields, Mohamed R. Amer, Max Ehrlich, Amir Tamrakar
In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017
CVF Cite It!

2016

Discriminative Hand Localization in Depth Images.
Max Ehrlich and Philippos Mordohai
In Proceedings of the IEEE Symposium on 3D User Interfaces, 2016
IEEE Direct Cite It!
Facial Attributes Classification using Multi-Task Representation Learning.
Max Ehrlich, Timothy J. Shields, Timur Almaev, Mohamed R. Amer
In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2016
CVF Cite It!

2015

Discriminative Hand Tracking from Depth Images.
Max Ehrlich
Master's Thesis, Stevens Institute of Technology, 2015
Direct