Wikipedia

Michael J. Black

Michael J. Black is an American-born computer scientist working in Tübingen, Germany. He is a founding director at the Max Planck Institute for Intelligent Systems where he leads the Perceiving Systems Department in research focused on computer vision, machine learning, and computer graphics. He is also an Honorary Professor at the University of Tübingen.

Michael J. Black
Michael J. Black
Director, Max Planck Institute for Intelligent Systems in Tübingen, Germany
BornJune 1962 (age 61–62)
North Carolina, United States
Alma mater
Known for
AwardsMember, German National Academy of Sciences Leopoldina
PAMI Distinguished Researcher Award (2023)
Longuet-Higgins prize (2020)
Helmholtz Prize (2013)
Koenderink Prize (2010)
Marr Prize, Honorable Mention, ICCV (2005)
Marr Prize, Honorable Mention, ICCV (1999)
IEEE Outstanding Paper Award (1991)
Foreign Member, Royal Swedish Academy of Sciences
Scientific career
Fields
InstitutionsMax Planck Institute for Intelligent Systems
University of Tuebingen
ThesisRobust Incremental Optical Flow (1992)
Doctoral advisorP. Anandan
Doctoral students
Websiteps.is.mpg.de/person/black

Black has won all three major test-of-time prizes in computer vision: the Koenderink Prize at the European Conference on Computer Vision (ECCV) in 2010 and 2022, the Helmholtz Prize at the International Conference on Computer Vision (ICCV) in 2013, and the Longuet-Higgins Prize at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) in 2022. In 2023 he received the PAMI Distinguished Researcher Award.

Research

edit

Optical flow

edit

Black's thesis[1] reformulated optical flow estimation as a robust M-estimation problem. The main observation was that spatial discontinuities in image motion and violations of the standard brightness constancy assumption could be treated as outliers. Reformulating the classical optimization problem as a robust estimation problem produced more accurate results.

This "Black and Anandan" optical flow algorithm has been widely used, for example, in special effects.[2] The method was used to compute optical flow for the painterly effects in What Dreams May Come and for registering 3D face scans in The Matrix Reloaded.

A version of this work received the IEEE Outstanding Paper Award at CVPR 1991[3] and the Helmholtz Prize at ICCV 2013 for work that has "stood the test of time".[4]

His early focus on statistical modeling of motion, particularly at motion discontinuities, led to two other prize papers. His work with David Fleet on the "Probabilistic Detection and Tracking of Motion Boundaries" won honorable mention for the Marr Prize at ICCV'99.[5] Black's work with Stefan Roth "On the spatial statistics of optical flow" received honorable mention for the Marr Prize at ICCV 2005.[6]

His work with Deqing Sun and Stefan Roth on the "Secrets of Optical Flow" was awarded the 2020 Longuet- Higgins Prize. The prize is given annually by the IEEE Pattern Analysis and Machine Intelligence (PAMI) Technical Committee for "Contributions in Computer Vision that Have Withstood the Test of Time." The "secrets" paper helped establish the state of the art in the field and led to the widely used Classic+NL flow algorithm.

Robust statistics and image statistics

edit

The "Black and Anandan" method helped popularize robust statistics in computer vision. This was facilitated by several papers that connected robust penalty functions to classical "line processes" used in Markov Random Fields (MRFs) at the time. Black and Rangarajan[7] characterized the formal properties of robust functions that have an equivalent line-process form and provided a process to convert between these formulations (known now as "Black-Rangarajan Duality"[8]). Black and colleagues applied these ideas to image denoising,[9] anisotropic diffusion,[10] and principal-component analysis (PCA).[11][12]

The robust formulation was hand crafted and used small spatial neighborhoods. The work on Fields of Experts with Stefan Roth removed these restrictions.[13] They learned the potential functions of an MRF with large spatial cliques by modeling the field potentials as a product of experts. Their formulation can be viewed as a shallow convolutional neural network.

Layered motion estimation

edit

In 1993, Black and Jepson used mixture models to represent optical flow fields with multiple motions[14] (also called "layered" optical flow). This introduced the use of Expectation Maximization (EM) to the field of computer vision.

Neural decoding and neural prosthetics

edit

In the 2000s, Black worked with John Donoghue and others at Brown University to create the technology behind the BrainGate neural prosthetics technology. Black and colleagues developed Bayesian methods to decode neural signals from motor cortex. The team was the first to use Kalman filtering[15][16][17] and particle filtering[18] to decode motor cortical ensemble activity.  With these Bayesian decoding methods, the team demonstrated the successful point-and-click control of a computer cursor by a human with paralysis[19][20] and the decoding of full arm and hand movement in non-human primates.[21]

Human motion and shape

edit

Black is best known for his work on human motion and shape estimation. With Hedvig Sidenbladh and David Fleet, he introduced the use of particle filtering for tracking 3D articulated human motion.[22] This work was awarded the Koenderink Koenderink Prize for Fundamental Contributions in Computer Vision at ECCV 2000.

His current work focuses on modeling and estimating human shape and pose from images and video. His team was the first to fit a learned 3D human body model to multi-camera image data at CVPR 2007,[23] under clothing at ECCV 2008,[24] from a single image at ICCV 2009,[25] and from RGB-D data at ICCV 2011.[26]

His group produced the popular SMPL 3D body model[27] (and various extensions like FLAME[28] for 3D human faces, MANO[29] for 3D hands, and SMPL-X,[30] an expressive 3D body model with hands and faces) and popularized methods for estimating 3D body shape from images.[31][32] SMPL is widely used in both academia and industry and was one of the core technologies licensed by Body Labs Inc.

Differentiable rendering

edit

Loper and Black popularized "differentiable rendering",[33] which has become an important component of self-supervised training of neural networks for problems like facial analysis. Classical methods for analysis by synthesis formulate an objective function and then differentiate it. The OpenDR[34] method was more generic in that it (approximately) differentiated a graphics rendering engine using Automatic differentiation. This provided a framework for posing a forward synthesis problem and automatically obtaining an optimization method to solve the inverse problem.

Datasets

edit

Black has contributed to several significant datasets. The Middlebury Flow dataset provided the first comprehensive benchmark for the field.[35] The MPI-Sintel Flow dataset demonstrated that synthetic data was sufficiently rich and similar to real data to provide a rigorous benchmark and to be useful for learning optical flow.[36]

The HumanEva dataset[37] was the first dataset with ground truth 3D human poses in correspondence with RGB video of people in motion. The approach used a combination of optical motion capture and multi-camera video capture. This dataset enabled the field to evaluate accuracy and compare performance for the first time.

Related to human pose, shape, and activity, Black has contributed to the SURREAL dataset of human motions,[38] the JHMDB dataset of human actions,[39] and the FAUST dataset of 3D body shapes.[40] FAUST received the Dataset Award from the Eurographics Symposium on Geometry Processing (SGP), 2016.[41]

Employment

edit

1985–1989: After his bachelor's degree, Black moved to the Bay Area and worked as a software engineer at GTE Government Systems and Advanced Decision Systems (ADS) developing expert systems on the Xerox and Symbolics Lisp machines. During this time, he completed his Master's of Computer Science in Symbolic and Heuristic Computation through the Honors Co-Op Program at Stanford. His advisor was John McCarthy.

1989–1992: During this period, Black pursued his PhD at Yale and was supported by a NASA Graduate Fellowship. He completed his PhD at the NASA Ames Research Center in the Human Factors Research Division led by Andrew (Beau) Watson. At Yale, he was advised by P. Anandan and Drew McDermott.

1992–1993: Black did post-doctoral work at the University of Toronto as an Assistant Professor of Computer Science (Contractually Limited Term Appointment). He was supervised by Allan Jepson. During his time there, he received the Computer Science Students' Union Teaching Award.

1993-2000: In 1993, Black joined the Xerox Palo Alto Research Center (PARC) as a member of research staff. He worked in the Image Understanding Area, led by Daniel Huttenlocher. In 1996, he took over management from Huttenlocher. He started the Digital Video Analysis Area in 1998.  

2000–2011: In 2000, Black joined the faculty of Brown University as an Associate Professor of Computer Science (with tenure). In 2004, he was promoted to Full Professor.

2017–2021: In 2017, with the acquisition of Body Labs by Amazon, Black joined Amazon as a Distinguished Amazon Scholar (VP) on a part-time basis.

2011–present: In 2011, Black became a Scientific Member of the Max Planck Society and one of the founding directors of the new MPI for Intelligent Systems.

Administration

edit

In addition to co-founding the MPI for Intelligent Systems, Black led the founding of the International Max Planck Research School (IMPRS) for Intelligent Systems.

In 2015, he proposed an initiative that has since become Cyber Valley, which aims to make the Stuttgart-Tübingen region of Germany a world leader in AI research and applications. He is on the research consortium's Executive Board and serves as its spokesperson.

Entrepreneurship

edit

In 2013, a team from Black's group spun out Body Labs which commercialized 3D body model technology for the clothing and games industry. Black was a co-founder and investor. Body Labs was acquired by Amazon.com in 2017.[42]

In 2018, Meshcapade GmbH spun out of his group. The start-up focuses on licensing technology developed at MPI-IS and providing services.[43]

References

edit
  1. ^ Black, M.J. (1992). "Robust Incremental Optical Flow". Yale University, Department of Computer Science.
  2. ^ "The Art of Optical Flow, FX Guide". 2006-02-28. Archived from the original on 2020-01-16. Retrieved 2 March 2020.
  3. ^ Black, M.J.; Anandan, P. (June 1991). "Robust dynamic motion estimation over time". IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). CVPR. Maui, Hawaii. pp. 296–302. Black:CVPR:1991.
  4. ^ Black, M.J.; Anandan, P. (May 1993). "A framework for the robust estimation of optical flow". Int. Conf. on Computer Vision (ICCV). ICCV. Berlin, Germany. pp. 231–236. Black:ICCV:1993.
  5. ^ Black, M.J.; Fleet, D.J. (September 1999). "Probabilistic detection and tracking of motion discontinuities". Int. Conf. on Computer Vision (ICCV). ICCV. Corfu, Greece: ICCV. pp. 551–558. Black:ICCV:1999.
  6. ^ Roth, R.; Black, M.J. (2005). "On the spatial statistics of optical flow". Int. Conf. on Computer Vision (ICCV). ICCV. pp. 42–49. Roth:ICCV:05.
  7. ^ Black, M.J.; Rangarajan, A. (July 1996). "On the unification of line processes, outlier rejection, and robust statistics with applications in early vision". International Journal of Computer Vision. 19: 57–92. CiteSeerX 10.1.1.75.6656. doi:10.1007/BF00131148. S2CID 7510079.
  8. ^ Barron, Jonathan T. (2017). "A General and Adaptive Robust Loss Function". arXiv:1701.03077 [cs.CV].
  9. ^ Black, M.J.; Rangarajan, A. (1996). "On the unification of line processes, outlier rejection, and robust statistics with applications in early vision". International Journal of Computer Vision. 19: 57–92. doi:10.1007/BF00131148. S2CID 7510079.
  10. ^ Black, M.J.; Sapiro, G.; Marimont, D.; Heeger, D. (March 1998). "Robust anisotropic diffusion". IEEE Transactions on Image Processing. 7 (3): 421–432. Bibcode:1998ITIP....7..421B. doi:10.1109/83.661192. PMID 18276262.
  11. ^ De la Torre, F.; Black, M.J. (2001). "Robust principal component analysis for computer vision". Int. Conf. on Computer Vision (ICCV). ICCV. Vancouver, BC, USA. pp. 362–369. Torre:ICCV:2001.
  12. ^ Black, M.J.; Jepson, A. (1998). "EigenTracking: Robust matching and tracking of articulated objects using a view-based representation". International Journal of Computer Vision. 26: 63–84. doi:10.1023/A:1007939232436. S2CID 572947.
  13. ^ Roth, S.; Black, M.J. (April 2009). "Fields of Experts". International Journal of Computer Vision. 82 (2): 205–29. doi:10.1007/s11263-008-0197-6. S2CID 13058320.
  14. ^ Jepson, A.; Black, M.J. (June 1992). "Mixture models for optical flow computation". IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). CVPR. New York, NY. pp. 760–761. Black:IEEE:1993.
  15. ^ Wu, W.; Black, M.J.; Gao, y.; Bienenstock, E.; Serruya, M.; Shaikhouni, A.; Donoghue, J.P. (2003). "Neural decoding of cursor motion using a Kalman filter". Advances in Neural Information Processing Systems 15 (NIPS). MIT Press. pp. 133–140. Black:ANIPS:2003.
  16. ^ Wu, W.; Gao, Y.; Bienenstock, E.; Donoghue, J.P.; Black, M.J. (2006). "Bayesian population decoding of motor cortical activity using a Kalman filter". Neural Computation. 18 (1): 80–118. CiteSeerX 10.1.1.218.2370. doi:10.1162/089976606774841585. PMID 16354382. S2CID 1679756.
  17. ^ Kim, S.-P.; Simeral, J.; Hochberg, L.; Donoghue, J.P.; Black, M.J. (2008). "Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia". Journal of Neural Engineering. 5 (4): 455–476. Bibcode:2008JNEng...5..455K. doi:10.1088/1741-2560/5/4/010. PMC 2911243. PMID 19015583.
  18. ^ Gao, Y.; Black, M.J.; Bienenstock, E.; Shoham, S.; Donoghue, J. (2002). "Probabilistic inference of hand motion from neural activity in motor cortex". Advances in Neural Information Processing Systems 14 (NIPS). NIPS. MIT Press. pp. 221–228. Black:ANIPS:2002.
  19. ^ Kim, S.-P.; Simeral, J.; Hochberg, L.; Donoghue, J.P.; Friehs, G.M.; Black, M.J. (April 2011). "Point-and-Click Cursor Control With an Intracortical Neural Interface System by Humans With Tetraplegia". IEEE Transactions on Neural Systems and Rehabilitation Engineering. 19 (2): 193–203. doi:10.1109/TNSRE.2011.2107750. PMC 3294291. PMID 21278024.
  20. ^ Simeral, J.D.; Kim, S.-P.; Black, M.J.; Donoghue, J.P.; Hochberg, L.R. (2011). "Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array". Journal of Neural Engineering. 8 (2): 025027. Bibcode:2011JNEng...8b5027S. doi:10.1088/1741-2560/8/2/025027. PMC 3715131. PMID 21436513.
  21. ^ Vargas-Irwin, C.E.; Shakhnarovich, G.; Yadollahpour, P.; Mislow, J.M.K.; Black, M.J.; Donoghue, J.P. (July 2010). "Decoding complete reach and grasp actions from local primary motor cortex populations". Journal of Neuroscience. 39 (29): 9659–9669. doi:10.1523/JNEUROSCI.5443-09.2010. PMC 2921895. PMID 20660249.
  22. ^ Sidenbladh, H.; Black, M.J.; Fleet, D.J. (June 2000). "Stochastic tracking of 3D human figures using 2D image motion". European Conference on Computer Vision (ECCV). ECCV. Dublin, Ireland: Springer Verlag. pp. 702–718. Black:ECCV:2000.
  23. ^ Balan, A.; Black, M.J.; Davis, J.; Haussecker, H. (June 2007). "Detailed human shape and pose from images". IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). CVPR. Minneapolis. pp. 1–8. Balan:CVPR:2007.
  24. ^ Balan, A; Black, M.J. (October 2008). "The naked truth: Estimating body shape under clothing". European Conf. on Computer Vision (ECCV). ECCV. Marseilles, France: Springer-Verlag. pp. 15–29. Balan:ECCV.
  25. ^ Guan, P.; Weiss, A.; Balan, A.; Black, M.J. (2009). "Estimating human shape and pose from a single image". Int. Conf. on Computer Vision (ICCV). ICCV. pp. 1381–1388. Guan:ICCV:2009.
  26. ^ Weiss, A.; Hirshberg, D; Black, M.J. (November 2011). "Home 3D body scans from noisy image and range data". Int. Conf. on Computer Vision (ICCV). ICCV. Barcelona: IEEE. pp. 1951–1958. Weiss:ICCV:11.
  27. ^ Loper, M.; Mahmood, N.; Romero, J.; Pons-Moll, G.; Black, M.J. (October 2017). "SMPL: A Skinned Multi-Person Linear Model". ACM Trans. Graphics. 34: 248:1–248:16. doi:10.1145/2816795.2818013. S2CID 229365481.
  28. ^ Li, T.; Bolkart, T.; Black, M.J.; Li, H.; Romero, J. (2017). "Learning a model of facial shape and expression from 4D scans". ACM Trans. Graphics. 36 (6): 194:1–194:17. doi:10.1145/3130800.3130813.
  29. ^ Romero, J.; Tzionas, D.; Black, M.J. (2017). "Embodied Hands: Modeling and Capturing Hands and Bodies Together". ACM Trans. Graphics. 36: 245:1–245:17. arXiv:2201.02610. doi:10.1145/3130800.3130883.
  30. ^ Pavlakos, G.; Choutas, V.; Ghorbani, N.; Bolkart, T.; Osman, A.A.A.; Tzionas, D.; Black, M.J. (2019). "Expressive Body Capture: 3D Hands, Face, and Body from a Single Image". IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Vol. 36. pp. 10975–10985. arXiv:1904.05866.
  31. ^ Bogo, F.; Kanazawa, A.; Lassner, C.; Gehler, P.; Romero, J.; Black, M.J. (October 2016). "Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image". European Conf. on Computer Vision (ECCV). ECCV. Amsterdam, the Netherlands: Springer International Publishing. pp. 561–578. Bogo:ECCV:2016.
  32. ^ Kanazawa, A.; Black, M.J.; Jacobs, D.W; Malik, J. (2018). "End-to-end Recovery of Human Shape and Pose". IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). CVPR. Salt Lake City, USA: IEEE Computer Society. pp. 7122–7131. Kanazawa:CVPR:2018.
  33. ^ Loper, M.; Black, M.J. (September 2014). "OpenDR: An Approximate Differentiable Renderer". European Conf. on Computer Vision (ECCV). ECCV. Zürich, Switzerland: Springer International Publishing. pp. 154–169. Loper:ECCV:2014.
  34. ^ "Open Differentiable Renderer". GitHub. 2020-02-16. Archived from the original on 2017-07-24. Retrieved 2 March 2020.
  35. ^ "A Database and Evaluation Methodology for Optical Flow". Archived from the original on 2019-07-27. Retrieved 2 March 2020.
  36. ^ "MPI Sintel Flow Dataset". Archived from the original on 2019-07-29. Retrieved 2 March 2020.
  37. ^ "HumanEva Dataset". Archived from the original on 2019-09-10. Retrieved 2 March 2020.
  38. ^ "SURREAL Dataset". Archived from the original on 2020-01-16. Retrieved 2 March 2020.
  39. ^ "Joint-annotated Human Motion Data Base". Archived from the original on 2019-07-24. Retrieved 2 March 2020.
  40. ^ "MPI FAUST Dataset". Archived from the original on 2019-08-11. Retrieved 2 March 2020.
  41. ^ "Geometry Processing Award Programs". Archived from the original on 2014-07-29. Retrieved 2 March 2020.
  42. ^ "Amazon has acquired 3D body model startup, Body Labs, for $50M-$70M". 3 October 2017. Archived from the original on 2019-12-17. Retrieved 2 March 2020.
  43. ^ "Meshcapade". Archived from the original on 2020-01-16. Retrieved 2 March 2020.
edit
Retrieved from "https://en.wikipedia.org/w/index.php?title=Michael_J._Black&oldid=1193137686"