Segmentation of COVID-19 Chest CT Scans using TransUNet
Segmentation of COVID-19 Chest CT Scans using TransUNetTransformers have shown great performance in the field of Image Segmentation. It would be amazing to train them usingJAX/Flax. The goal of this project is automaticCOVID-19lesion segmentation from chest CT.2. LanguageThe model will be trained inEnglish.3. ModelThe TransUNet model will be used for segmentation.4. DatasetsPossible links to publicly available datasets include:dataset-01:https://doi.org/10.5281/zenodo.3757476dataset-02:COVID-19 - Medical segmentation5. Training scriptsAre there publicly available training scripts that can be used/tweaked for the project?Answer:Unfortunately, No.6. (Optional) ChallengesBuild TransUnet model in JAX/FlaxImplement Augmentation for Segmentation.7. (Optional) Desired project outcomeThis project will help clinicians segmentCOVID-19lesions which will help clinicians identify potentialCOVID-19cases.TransUNetcan be later on other segmentation tasks.
2021-06-23T18:19:37Z
I’d love to join this project!I’ve written many UNets and data augmentations in both TF and PyTorch as part of my daily job and could bring that experience to the team, helping with the model or data augmentation as needed.On the other hand I haven’t seen much Jax, dealt with many medical datasets, nor combined transformers with UNet architectures – all of which I’d love to correct. The project outcome is also clearly beneficial, although one should point out that recent analyses have found much Covid-related CV research quite problematic (Nature article).I live in the London timezone (BST, GMT+1).
2021-06-23T19:16:05Z
@dom-miketasame here, I’ve also worked withTensorFlowandPyTorchfor segmentation. AsJAXis getting popular nowadays it would be amazing to learn it for segmentation tasks.Yes, you’re right indeed I’ve read that paper but what confused me is if they’ve actually been able to reproduce the result because not all datasets are publicly available. Apart from that,TransUNetcan be used for any medical image data. So, we can take advantage of it in either way.Btw I live in the Bangladesh Standard Time (GMT+6). Looking forward to working with you.
2021-06-23T19:22:19Z
I would love to be  part of this project
2021-06-24T07:17:45Z
I would love to be a part of this project!
2021-06-24T08:18:45Z
This seems really interesting. Would love to be part of the project.
2021-06-24T08:27:05Z
I would love to join this project.
2021-06-29T06:01:07Z
Hey, have been also doing a lot of deep learning in healthcare space, would love to join this, if there is any spots!
2021-06-29T13:56:35Z
Lots of interest here - awesome! Let’s define this projectI think this is one of the more difficult projects since I’m not sure whether there is an open-source implementation of TransUNet in JAX@awsaf49? Is it maybe possible to use another model instead? ViT maybe?
2021-06-29T22:58:59Z
Just a thought,@patrickvonplatenand@awsaf49, in case we need pixel level classification, I guess ViT wont be that helpful, unless we are okay with a patch level segmentation. I saw thatBigBirdimplementation is available in JAX, maybe we could go with that since we could flatten the images to pixel level? The images seem to be 512 x 512. BigBird might need a lot of samples though since it needs to be trained from scratch…
2021-06-30T00:15:17Z
@patrickvonplatenI think the main purpose of this project is to do pixel-level classification or segmentation with transformer. If we can do segmentation with  ViT with then I think we should use ViT too. As far as I know, ViT is for classification not segmentation. If there is no implementation of TransUNet in JAX perhaps we can work on it? It would be amazing if we can implement it in this project…
2021-06-30T19:34:15Z
I don’t think there’s a problem though – the TransUNet is ‘just’ a UNet with a ViT encoder. And the dataset  is publicly available.Edit: That said, I’ve decided to join another project – I’m a bit too comfortable in segmentation and would like to try something new!