Microscopy Machine Vision Lab

Research

Deep learning is not easy. For example, applying deep learning on a specifcal microscopy image analysis problem is much more than collecting images, making annotations, and feeding the images with annotations into a neural network. On one hand, annotation is not easy and sometimes even impossible (e.g., accurately annotating the segmentation of a very complex shape in 3D is nearly impossible). On the other hand, there are many important questions to think about. How to evaluate the quality of your annotation? What if different people have different annotations? How can I annotate the minimal amount of data to achieve to desired accuracy? When I apply my model on new images, how do I know the results are correct? …

Solving real computational biomedical problems with AI is not straightforward. The quantitative tasks in many biomedical applications may not be easily formulated into common deep learning or computer vision problems (e.g., segmentation, classification, image reconstruction, etc.). Even a well-defined problem, we may face very different challenges in the specific biomedical context than the problem’s original general computer vision setting. For example, it would be easy for everyone to accuratly annotate the contour of a car in a 2D image, but it could be very difficult to annotate the boundary of a nucleus in a 3D confocal microscopy image. It is not only because 3D is hard, but also because you have to take the diffraction of light into account to avoid over-segmentation. Or, minor segmentation error of cars may not be a big deal, but minor segmentation error of nuclei (e.g., over-smoothed boundary of nuclei with complex shape when being squeezed) may yield very different biomedical findings. So, to develop a successful AI-based microscopy image analysis algorithm, it is very important to have good understanding of microscopy and the underlying biomedical application.

2024

Is the Apple Vision Pro the Ultimate Display A First Perspective and Survey on Entering the Wonderland of Precision Medicine
Is the Apple Vision Pro the Ultimate Display? A First Perspective and Survey on Entering the Wonderland of Precision Medicine
Jan Egger, Christina Gsaxner, Gijs Luijten, Jianxu Chen, Xiaojun Chen, Jiang Bian, Jens Kleesiek, Behrus Puladi
JMIR Serious Games  ·  18 Sep 2024  ·  doi:10.2196/52785
Multimodal large language models for bioimage analysis
Multimodal large language models for bioimage analysis
Shanghang Zhang, Gaole Dai, Tiejun Huang, Jianxu Chen
Nature Methods  ·  01 Aug 2024  ·  doi:10.1038/s41592-024-02334-2
Colony context and size-dependent compensation mechanisms give rise to variations in nuclear growth trajectories
Colony context and size-dependent compensation mechanisms give rise to variations in nuclear growth trajectories
Julie C. Dixon, Christopher L. Frick, Chantelle L. Leveille, Philip Garrison, Peyton A. Lee, …, Ruian Yang, Daniel M. Toloudis, Matheus P. Viana, Julie A. Theriot, Susanne M. Rafelski
bioRxiv  ·  30 Jun 2024  ·  doi:10.1101/2024.06.28.601071
2D Label-Free Prediction of Multiple Organelles Across Different Transmitted-Light Microscopy Images with Bag-of-Experts
2D Label-Free Prediction of Multiple Organelles Across Different Transmitted-Light Microscopy Images with Bag-of-Experts
Yu Zhou, Shuo Zhao, Justin Sonneck, Jianxu Chen
IEEE ISBI Conference  ·  27 May 2024  ·  doi:10.1109/ISBI56570.2024.10635298
Topological Analysis of Mouse Brain Vasculature via 3d Light-Sheet Microscopy Images
Topological Analysis of Mouse Brain Vasculature via 3d Light-Sheet Microscopy Images
Jiachen Yao, Nina Hagemann, Qiaojie Xiong, Jianxu Chen, Dirk M. Hermann, Chao Chen
2024 IEEE International Symposium on Biomedical Imaging (ISBI)  ·  27 May 2024  ·  doi:10.1109/ISBI56570.2024.10635226
Stroke and myocardial infarction induce neutrophil extracellular trap release disrupting lymphoid organ structure and immunoglobulin secretion
Stroke and myocardial infarction induce neutrophil extracellular trap release disrupting lymphoid organ structure and immunoglobulin secretion
Ali A. Tuz, Susmita Ghosh, Laura Karsch, Dimitris Ttoouli, Sai P. Sata, …, Albert Sickmann, Jianxu Chen, Dirk M. Hermann, Matthias Gunzer, Vikramjeet Singh
Nature Cardiovascular Research  ·  23 Apr 2024  ·  doi:10.1038/s44161-024-00462-8
Microvascular Network Remodeling in the Ischemic Mouse Brain Defined by Light Sheet Microscopy
Microvascular Network Remodeling in the Ischemic Mouse Brain Defined by Light Sheet Microscopy
Nina Hagemann, Yachao Qi, Ayan Mohamud Yusuf, AnRan Li, Anthony Squire, …, Philippa Spangenberg, Jianxu Chen, Axel Mosig, Matthias Gunzer, Dirk M. Hermann
Arteriosclerosis, Thrombosis, and Vascular Biology  ·  01 Apr 2024  ·  doi:10.1161/ATVBAHA.123.320339
Understanding metric-related pitfalls in image analysis validation
Understanding metric-related pitfalls in image analysis validation
Annika Reinke, Minu D. Tizabi, Michael Baumgartner, Matthias Eisenmann, Doreen Heckmann-Nötzel, …, Ben Van Calster, Gaël Varoquaux, Ziv R. Yaniv, Paul F. Jäger, Lena Maier-Hein
Nature Methods  ·  01 Feb 2024  ·  doi:10.1038/s41592-023-02150-0
EfficientBioAI: making bioimaging AI models efficient in energy and latency
EfficientBioAI: making bioimaging AI models efficient in energy and latency
Yu Zhou, Jiajun Cao, Justin Sonneck, Sweta Banerjee, Stefanie Dörr, Anika Grüneboom, Kristina Lorenz, Shanghang Zhang, Jianxu Chen
Nature Methods  ·  24 Jan 2024  ·  doi:10.1038/s41592-024-02167-z
i MMV Im2Im i : an open-source microscopy machine vision toolbox for image-to-image transformation
MMV_Im2Im: an open-source microscopy machine vision toolbox for image-to-image transformation
Justin Sonneck, Yu Zhou, Jianxu Chen
GigaScience  ·  01 Jan 2024  ·  doi:10.1093/gigascience/giad120
Implicit Neural Image Field for Biological Microscopy Image Compression
Implicit Neural Image Field for Biological Microscopy Image Compression
Gaole Dai, Cheng-Ching Tseng, Qingpo Wuwu, Rongyu Zhang, Shaokang Wang, …, Yu Zhou, Ali Ata Tuz, Matthias Gunzer, Jianxu Chen, Shanghang Zhang
arXiv  ·  01 Jan 2024  ·  doi:10.48550/arXiv.2405.19012
Rethinking Medical Anomaly Detection in Brain MRI: An Image Quality Assessment Perspective
Rethinking Medical Anomaly Detection in Brain MRI: An Image Quality Assessment Perspective
Zixuan Pan, Jun Xia, Zheyu Yan, Guoyue Xu, Yawen Wu, Zhenge Jia, Jianxu Chen, Yiyu Shi
arXiv  ·  01 Jan 2024  ·  doi:10.48550/arXiv.2408.08228

2023

ComplexEye: a multi-lens array microscope for high-throughput embedded immune cell migration analysis
ComplexEye: a multi-lens array microscope for high-throughput embedded immune cell migration analysis
Zülal Cibir, Jacqueline Hassel, Justin Sonneck, Lennart Kowitz, Alexander Beer, …, Karsten Seidl, Jianxu Chen, Anton Grabmaier, Reinhard Viga, Matthias Gunzer
Nature Communications  ·  11 Dec 2023  ·  doi:10.1038/s41467-023-43765-3
On the risk of manual annotations in 3D confocal microscopy image segmentation
On the risk of manual annotations in 3D confocal microscopy image segmentation
Justin Sonneck, Shuo Zhao, Jianxu Chen
2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)  ·  02 Oct 2023  ·  doi:10.1109/ICCVW60793.2023.00421
The importance of resource awareness in artificial intelligence for healthcare
The importance of resource awareness in artificial intelligence for healthcare
Zhenge Jia, Jianxu Chen, Xiaowei Xu, John Kheir, Jingtong Hu, Han Xiao, Sui Peng, Xiaobo Sharon Hu, Danny Chen, Yiyu Shi
Nature Machine Intelligence  ·  12 Jun 2023  ·  doi:10.1038/s42256-023-00670-0
Rapid and fully automated blood vasculature analysis in 3D light-sheet image volumes of different organs
Rapid and fully automated blood vasculature analysis in 3D light-sheet image volumes of different organs
Philippa Spangenberg, Nina Hagemann, Anthony Squire, Nils Förster, Sascha D. Krauß, …, Iris Helfrich, Jianxu Chen, Matthias Gunzer, Dirk M. Hermann, Axel Mosig
Cell Reports Methods  ·  01 Mar 2023  ·  doi:10.1016/j.crmeth.2023.100436
Integrated intracellular organization and its variations in human iPS cells
Integrated intracellular organization and its variations in human iPS cells
Matheus P. Viana, Jianxu Chen, Theo A. Knijnenburg, Ritvik Vasan, Calysta Yan, …, Graham T. Johnson, Ruwanthi N. Gunawardane, Nathalie Gaudreault, Julie A. Theriot, Susanne M. Rafelski
Nature  ·  04 Jan 2023  ·  doi:10.1038/s41586-022-05563-7
Masked Diffusion as Self-supervised Representation Learner
Masked Diffusion as Self-supervised Representation Learner
Zixuan Pan, Jianxu Chen, Yiyu Shi
arXiv  ·  01 Jan 2023  ·  doi:10.48550/arXiv.2308.05695
MedShapeNet -- A Large-Scale Dataset of 3D Medical Shapes for Computer Vision
MedShapeNet -- A Large-Scale Dataset of 3D Medical Shapes for Computer Vision
Jianning Li, Zongwei Zhou, Jiancheng Yang, Antonio Pepe, Christina Gsaxner, …, Behrus Puladi, Pascal Fua, Alan L. Yuille, Jens Kleesiek, Jan Egger
arXiv  ·  01 Jan 2023  ·  doi:10.48550/arXiv.2308.16139
Deep learning based Image Compression for Microscopy Images: An Empirical Study
Deep learning based Image Compression for Microscopy Images: An Empirical Study
Yu Zhou, Jan Sollmann, Jianxu Chen
arXiv  ·  01 Jan 2023  ·  doi:10.48550/arXiv.2311.01352

2021

Cell states beyond transcriptomics: Integrating structural organization and gene expression in hiPSC-derived cardiomyocytes
Cell states beyond transcriptomics: Integrating structural organization and gene expression in hiPSC-derived cardiomyocytes
Kaytlyn A. Gerbin, Tanya Grancharova, Rory M. Donovan-Maiye, Melissa C. Hendershott, Helen G. Anderson, …, Nathalie Gaudreault, Theo A. Knijnenburg, Susanne M. Rafelski, Julie A. Theriot, Ruwanthi N. Gunawardane
Cell Systems  ·  01 Jun 2021  ·  doi:10.1016/j.cels.2021.05.001

2020

Unlabeled Data Guided Semi-supervised Histopathology Image Segmentation
Unlabeled Data Guided Semi-supervised Histopathology Image Segmentation
Hongxiao Wang, Hao Zheng, Jianxu Chen, Lin Yang, Yizhe Zhang, Danny Z. Chen
IEEE BIBM Conference  ·  16 Dec 2020  ·  doi:10.1109/BIBM49941.2020.9313428
InTracker: An Integrated Detector-Tracker Framework for Cell Detection and Tracking
InTracker: An Integrated Detector-Tracker Framework for Cell Detection and Tracking
Peixian Liang, Jianxu Chen, Yizhe Zhang, Hongxiao Wang, Hao Zheng, Pengfei Gu, Danny Chen
IEEE CBMS Conference  ·  01 Jul 2020  ·  doi:10.1109/CBMS49503.2020.00069

2019

Single Cells Exhibit Differing Behavioral Phases during Early Stages of Pseudomonas aeruginosa Swarming
Single Cells Exhibit Differing Behavioral Phases during Early Stages of Pseudomonas aeruginosa Swarming
Chinedu S. Madukoma, Peixian Liang, Aleksandar Dimkovikj, Jianxu Chen, Shaun W. Lee, Danny Z. Chen, Joshua D. Shrout
Journal of Bacteriology  ·  01 Oct 2019  ·  doi:10.1128/JB.00184-19
Biomedical Image Segmentation via Representative Annotation
Biomedical Image Segmentation via Representative Annotation
Hao Zheng, Lin Yang, Jianxu Chen, Jun Han, Yizhe Zhang, Peixian Liang, Zhuo Zhao, Chaoli Wang, Danny Z. Chen
AAAI Conference  ·  17 Jul 2019  ·  doi:10.1609/aaai.v33i01.33015901
Cascade Decoder: A Universal Decoding Method For Biomedical Image Segmentation
Cascade Decoder: A Universal Decoding Method For Biomedical Image Segmentation
Peixian Liang, Jianxu Chen, Hao Zheng, Lin Yang, Yizhe Zhang, Danny Z. Chen
IEEE ISBI Conference  ·  01 Apr 2019  ·  doi:10.1109/ISBI.2019.8759430
Decoding Calcium Signaling Dynamics during Drosophila Wing Disc Development
Decoding Calcium Signaling Dynamics during Drosophila Wing Disc Development
Pavel A. Brodskiy, Qinfeng Wu, Dharsan K. Soundarrajan, Francisco J. Huizar, Jianxu Chen, …, Cody Narciso, Megan K. Levis, Ninfamaria Arredondo-Walsh, Danny Z. Chen, Jeremiah J. Zartman
Biophysical Journal  ·  01 Feb 2019  ·  doi:10.1016/j.bpj.2019.01.007

2018

The Allen Cell and Structure Segmenter: a new open source toolkit for segmenting 3D intracellular structures in fluorescence microscopy images
The Allen Cell and Structure Segmenter: a new open source toolkit for segmenting 3D intracellular structures in fluorescence microscopy images
Jianxu Chen, Liya Ding, Matheus P. Viana, HyeonWoo Lee, M. Filip Sluezwski, Benjamin Morris, Melissa C. Hendershott, Ruian Yang, Irina A. Mueller, Susanne M. Rafelski
bioRxiv  ·  08 Dec 2018  ·  doi:10.1101/491035
A new registration approach for dynamic analysis of calcium signals in organs
A new registration approach for dynamic analysis of calcium signals in organs
Peixian Liang, Jianxu Chen, Pavel A. Brodskiy, Qinfeng Wu, Yejia Zhang, Yizhe Zhang, Lin Yang, Jeremiah J. Zartman, Danny Z. Chen
IEEE ISBI Conference  ·  01 Apr 2018  ·  doi:10.1109/ISBI.2018.8363724

2017

Compression-induced structural and mechanical changes of fibrin-collagen composites
Compression-induced structural and mechanical changes of fibrin-collagen composites
O.V. Kim, R.I. Litvinov, J. Chen, D.Z. Chen, J.W. Weisel, M.S. Alber
Matrix Biology  ·  01 Jul 2017  ·  doi:10.1016/j.matbio.2016.10.007
Optimizing Memory Efficiency for Convolution Kernels on Kepler GPUs
Optimizing Memory Efficiency for Convolution Kernels on Kepler GPUs
Xiaoming Chen, Jianxu Chen, Danny Z. Chen, Xiaobo Sharon Hu
ACM Design Automation Conference  ·  18 Jun 2017  ·  doi:10.1145/3061639.3062297
Deep Adversarial Networks for Biomedical Image Segmentation Utilizing Unannotated Images
Deep Adversarial Networks for Biomedical Image Segmentation Utilizing Unannotated Images
Yizhe Zhang, Lin Yang, Jianxu Chen, Maridel Fredericksen, David P. Hughes, Danny Z. Chen
MICCAI Conference  ·  01 Jan 2017  ·  doi:10.1007/978-3-319-66179-7_47
Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation
Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation
Lin Yang, Yizhe Zhang, Jianxu Chen, Siyuan Zhang, Danny Z. Chen
MICCAI Conference  ·  01 Jan 2017  ·  doi:10.1007/978-3-319-66179-7_46
Neuron Segmentation Using Deep Complete Bipartite Networks
Neuron Segmentation Using Deep Complete Bipartite Networks
Jianxu Chen, Sreya Banerjee, Abhinav Grama, Walter J. Scheirer, Danny Z. Chen
MICCAI Conference  ·  01 Jan 2017  ·  doi:10.1007/978-3-319-66185-8_3

2016

Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentation
Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentation
J. Chen, L. Yang, Y. Zhang, M. Alber, D. Chen
NeurIPS Conference  ·  05 Dec 2016  ·  url:https://papers.nips.cc/paper/2016/hash/4dcf435435894a4d0972046fc566af76-Abstract.html
A Hybrid Approach for Segmentation and Tracking of Myxococcus Xanthus Swarms
A Hybrid Approach for Segmentation and Tracking of Myxococcus Xanthus Swarms
Jianxu Chen, Mark S. Alber, Danny Z. Chen
IEEE Transactions on Medical Imaging  ·  01 Sep 2016  ·  doi:10.1109/TMI.2016.2548490
Kinetic Transition Networks for the Thomson Problem and Smale s Seventh Problem
Kinetic Transition Networks for the Thomson Problem and Smale’s Seventh Problem
Dhagash Mehta, Jianxu Chen, Danny Z. Chen, Halim Kusumaatmaja, David J. Wales
Physical Review Letters  ·  06 Jul 2016  ·  doi:10.1103/PhysRevLett.117.028301
Iris Recognition Based on Human-Interpretable Features
Iris Recognition Based on Human-Interpretable Features
Jianxu Chen, Feng Shen, Danny Ziyi Chen, Patrick J. Flynn
IEEE Transactions on Information Forensics and Security  ·  01 Jul 2016  ·  doi:10.1109/TIFS.2016.2535901
Segmentation and tracking of Pseudomonas aeruginosa for cell dynamics analysis in time-lapse images
Segmentation and tracking of Pseudomonas aeruginosa for cell dynamics analysis in time-lapse images
Jianxu Chen, Yiqing Cai, Chen Wei, Lin Yang, Mark S. Alber, Danny Z. Chen
IEEE ISBI Conference  ·  01 Apr 2016  ·  doi:10.1109/ISBI.2016.7493426

2015

Iris recognition based on human-interpretable features
Iris recognition based on human-interpretable features
Jianxu Chen, Feng Shen, Danny Z. Chen, Patrick J. Flynn
IEEE International Conference on Identity, Security, and Behavior Analysis  ·  01 Mar 2015  ·  doi:10.1109/ISBA.2015.7126352
A Hybrid Approach for Segmentation and Tracking of Myxococcus Xanthus Swarms
A Hybrid Approach for Segmentation and Tracking of Myxococcus Xanthus Swarms
Jianxu Chen, Shant Mahserejian, Mark Alber, Danny Z. Chen
MICCAI Conference  ·  01 Jan 2015  ·  doi:10.1007/978-3-319-24574-4_34

2014

An Automated Approach for Fibrin Network Segmentation and Structure Identification in 3D Confocal Microscopy Images
An Automated Approach for Fibrin Network Segmentation and Structure Identification in 3D Confocal Microscopy Images
Jianxu Chen, Oleg V. Kim, Rustem I. Litvinov, John W. Weisel, Mark S. Alber, Danny Z. Chen
IEEE CBMS Conference  ·  01 May 2014  ·  doi:10.1109/CBMS.2014.62
A Matching Model Based on Earth Mover s Distance for Tracking Myxococcus Xanthus
A Matching Model Based on Earth Mover’s Distance for Tracking Myxococcus Xanthus
Jianxu Chen, Cameron W. Harvey, Mark S. Alber, Danny Z. Chen
MICCAI Conference  ·  01 Jan 2014  ·  doi:10.1007/978-3-319-10470-6_15