Projecting points to camera

[samueleruffino99]

Hello everyone!

I have (I think) a very simple question, I am trying to project bounding boxes to camera and apparently I am applying the wrong transformation.

First of all I have computed the 3d bbox vertices with:

mesh = entity.get_mesh()
        mesh_min = mesh.get_min_aabb_corner()
        mesh_max = mesh.get_max_aabb_corner()
        transform = entity.get_transform()
        aabb_min = transform.transform_point(mesh_min)
        aabb_max = transform.transform_point(mesh_max)
        bbox_whd =[aabb_min[0], aabb_min[1], aabb_max[2],
                aabb_max[0] - aabb_min[0], aabb_max[1] - aabb_min[1], aabb_max[2] - aabb_min[2]]
        bbox_vert = np.array([
            [bbox_whd[0], bbox_whd[1], bbox_whd[2]],
            [bbox_whd[0], bbox_whd[1], bbox_whd[2] + bbox_whd[5]],
            [bbox_whd[0], bbox_whd[1] + bbox_whd[4], bbox_whd[2]],
            [bbox_whd[0], bbox_whd[1] + bbox_whd[4], bbox_whd[2] + bbox_whd[5]],
            [bbox_whd[0] + bbox_whd[3], bbox_whd[1], bbox_whd[2]],
            [bbox_whd[0] + bbox_whd[3], bbox_whd[1], bbox_whd[2] + bbox_whd[5]],
            [bbox_whd[0] + bbox_whd[3], bbox_whd[1] + bbox_whd[4], bbox_whd[2]],
            [bbox_whd[0] + bbox_whd[3], bbox_whd[1] + bbox_whd[4], bbox_whd[2] + bbox_whd[5]],
            ])

Then tried in two different ways:

1. Using intrinsic matrix (.get_camera().get_intrinsic_matrix(width, height)) and compute matrix multiplication by the 8 vertices.
2. Using world_to_local matrix (.get_transform().get_world_to_local_matrix()) and projection matrix (.get_camera().get_projection()), transforming 3d vertices in homogeneous coordinates, multiuply them by world_to_local matrix and then by projection matrix.

Anyway, I am getting strange results and bounding boxes are not printed correctly.

How am I suppose to project world points onto image ?

Thank you!!

[TontonTremblay]

Check this function https://github.com/owl-project/NVISII/blob/master/examples/21.ndds_export.py#L66 I think it does what you need.

[samueleruffino99]

How can I just project points onto the camera matrix? Because it seems that using those matrices I get wrong pixel position

[TontonTremblay]

        p_image = cam_proj_matrix * (cam_matrix * pos_m) 
        p_image = nvisii.vec2(p_image) / p_image.w
        p_image = p_image * nvisii.vec2(1,-1)
        p_image = (p_image + nvisii.vec2(1,1)) * 0.5

You need this for pixel location, is that helpful.