Point·E

From the official code and model release here: https://github.com/openai/point-e, for Point-E: A System for Generating 3D Point Clouds from Complex Prompts.

This model enables text2pointcloud (with base_40m_textvec.pt checkpoint) and img2pointcloud (with base40M.pt checkpoint) generation.

Two kinds of input are accepted:

• a prompt for generating point cloud from text, or

• an image for generating point cloud from the image Note that if the prompt is provided, the image will be ignored. Therefore for effectively generating point cloud from images please remove the prompt if it was previously set.

The supported output format are:

• PointCloud saved as json_file. PointCloud is an array of points sampled on a surface, with coords: an [N x 3] array of point coordinates, and channel attributes which corresponds to R, G, B colors of the points in coords. We re-ordered the format to more standard way as follows: { "coords": [...], "colors": [...] }, where “coords” is an [N x 3] array of (X,Y,Z) point coordinates, and “colors” is an [N x 3] array of (R,G,B) color values

• Or an animation of the point cloud