Change color of Model3D
I have been successful in using thegradio.Model3Din my apps, which have enabled my to render volumetric 3D segmentations of various structures from medical images.An obstacle is that there did not appear to be any simple way to change the color of the objects rendered in theModel3Dscene. After debugging this further, I realised that changing the color may be more tricky than I thought, which is likely why there is no API-method to do this.I am converting a binary numpy array to.objby first performing marching cubes to get the surface, which I then use to write to.obj. I believe when writing here, I need to specifyMaterialsand define colours which I then will use when I write the faces and vertices, however it seems like when trying to do so it did not work.Below is a snippet of the script that I have been using for converting a binary numpy array to.obj:from skimage.measure import marching_cubes

# data : some 3D binary 0-1 numpy array with uint8 type

# Create a material with a red diffuse color (RGB value)
red_material = "newmtl RedMaterial\nKd 1 0 0"  # Red diffuse color (RGB)

# extract surface
verts, faces, normals, values = marching_cubes(data, 0)
faces += 1

with open(output, "w") as thefile:
    # Write the material definition to the OBJ file
    thefile.write(red_material + "\n")

    for item in verts:
        # thefile.write('usemtl RedMaterial\n')  # <- uncommenting this I would expect would change the color, but it does not.
        thefile.write("v {0} {1} {2}\n".format(item[0], item[1], item[2]))

    for item in normals:
        thefile.write("vn {0} {1} {2}\n".format(item[0], item[1], item[2]))

    for item in faces:
        thefile.write(
            "f {0}//{0} {1}//{1} {2}//{2}\n".format(
                item[0], item[1], item[2]
            )
        )

# then I can connect the `.obj` file to the `Model3D` scene:
self.volume_renderer = gr.Model3D(
    clear_color=[0.0, 0.0, 0.0, 0.0],
    label="3D Model",
    show_label=True,
    visible=True,
    elem_id="model-3d",
).style(height=512)
2023-11-01T09:11:30Z
maybe cc@dylanebert
2023-11-13T17:35:32Z
Hey@andreped! We don’t have the ability to do that right now, but we recently launched gradio 4.0 - which allows you to create custom components. I actually created a custom component based on Model3D which provides more control and allows you to add lights and cameras.Here’s the link.I didn’t add the ability to change the colour (yet?) but it might help you if you’re interested in adding this feature yourself. Feel free to let me know if you have any questions!
2023-11-14T10:27:36Z
Amazing,@hmb! Great work!I just added this to an existing gradio app, and it seems to be working, however, I am seeing some weird gliches when panning the view. It seems to reinitialise all the time and I do not see a simple way to fix that. Care to take a look,@hmb?Note that previously I also added widgets to their respectivegradio.Boxwidgets, but that is not a feature anymore ingradio==4.3.0. Perhaps that is no longer required to keep widgets in their correct sizes when resizing the browser window?You can see my gradio app at:huggingface.coLyNoS: automatic lymph node segmentation using deep learning - a Hugging Face...Discover amazing ML apps made by the communityBelow is a snapshot of the existing app:Screenshot 2023-11-14 at 11.29.441920×790 107 KBEDIT: And yes, I can see that the object that is rendered in your app indeed has colouring, so it would be great to know what I need to do to introduce the same for my 3D viewer. Likely I need to modify the OBJ object before saving it somehow (that’s what I tried above but to no avail).