profile
viewpoint

Ask questionsRendering of occluded objects looks strange

Hello,

I'm trying to render teeth arch using pytorch3d and got strange result (please see screenshot):

Screenshot 2020-05-29 at 14 58 30

When I removed some occluded teeth rendered image was ok. I've tried to decimate mesh and reduced number of faces by 4 times and rendering was ok. Is it some limitation of framework? Could you please help me with this issue?

Here is code to reproduce:

import torch
import numpy as np
import matplotlib.pyplot as plt
import trimesh

from pytorch3d.structures import Meshes, Textures
from pytorch3d.transforms import euler_angles_to_matrix

from pytorch3d.renderer import (
    OpenGLPerspectiveCameras, RasterizationSettings, MeshRenderer, MeshRasterizer, BlendParams,
    HardPhongShader, PointLights
)

device = torch.device("cuda:0")
torch.cuda.set_device(device)

stl_file = "teeth.stl"

def load_mesh(fn):
    mesh = trimesh.load_mesh(fn)
    
    verts, faces = torch.from_numpy(mesh.vertices).float(), torch.from_numpy(mesh.faces)
    verts /= 1000

    verts_rgb = torch.ones_like(verts)[None]
    textures = Textures(verts_rgb=verts_rgb.to(device))

    mesh = Meshes(
        verts=[verts.to(device)],   
        faces=[faces.to(device)], 
        textures=textures
    )
    
    return mesh

orig_mesh = load_mesh(stl_file)

cameras = OpenGLPerspectiveCameras(device=device, fov=0.27, degrees=False)
blend_params = BlendParams(sigma=1e-2, gamma=1e-7)
raster_settings = RasterizationSettings(
    image_size=1024, 
    blur_radius=0., 
    faces_per_pixel=10, 
)

lights = PointLights(device=device, location=((2.0, -2.0, -2.0),))
phong_renderer = MeshRenderer(
    rasterizer=MeshRasterizer(
        cameras=cameras, 
        raster_settings=raster_settings
    ),
    shader=HardPhongShader(device=device, cameras=cameras)
)

R = euler_angles_to_matrix(torch.tensor(np.radians([90, 40, 0])).float(), convention="XYZ").unsqueeze(0).cuda()
T = torch.tensor([0, 0, 0.5]).float().unsqueeze(0).cuda()

image_ref = phong_renderer(meshes_world=orig_mesh, R=R, T=T).squeeze()

plt.figure(figsize=(10, 10))
plt.imshow(image_ref.cpu().numpy()[:, :, 1], cmap='gray')

Thanks in advance!

Here is stl: teeth.stl.zip

facebookresearch/pytorch3d

Answer questions jcjohnson

Hi @abhshkdz, thanks for setting up a reproducing example! Can you change the permissions on the Colab notebook to be world-readable? I can't open it at the moment.

useful!

Related questions

No questions were found.
source:https://uonfu.com/
Github User Rank List