What Is OpenAI's New Shap-E Program?

OpenAI continues to make tremendous waves in the generative AI space. At first, the much-ballyhooed ChatGPT was a system that only those in the industry saw the true potential of. It's becoming clearer and clearer every day, though, even to those who aren't so tech-savvy. ChatGPT can create Spotify Playlists for us and even in a bizarre case, write part of an episode of South Park. As the chatbot and others like it continue to learn and improve, there's really no telling where AI may take us in the not-so-distant future.

As sophisticated as the core technology may be, though, it's still reliant on creative minds such as those at OpenAI to advance. The company's new venture, the Shap-E program, is another step forward on that front. The title hints as to exactly what it can do: it deals with shapes in a futuristic and sophisticated way.

Here's an introduction to this 3D shape-generating system, and exactly what can be done with it.

AI-generated art is another intriguing string to the AI bow, but its problematic implications have led to the likes of Google taking steps to combat it. Shap-E is a different kind of project, a program that can create 3D images from prompts.

As Alex Nichol and Heewoo Jun explained in the May 2023 paper "Shap-E: Generating Conditional 3D Implicit Functions" (Cornell University), Shap-E is a step beyond similar tools. "Unlike recent work on 3D generative models which produce a single output representation," the authors note, "Shap-E directly generates the parameters of implicit functions that can be rendered as both textured meshes and neural radiance fields."

In less technical terms, Shap-E was developed to create its own 3D renders (not simple images) from visual or text prompts provided and to add a surprising degree of detail by focusing on the particulars of what it was asked to provide. As with many such generative AI programs, there's a great degree of potential here, but also shortcomings to circumvent and learning potential to take advantage of.

At the GitHub link for Shap-E, interested parties can install the file via the Python package manager and, perhaps more importantly, peruse a depository of sample images created using the program. Doing so demonstrates some remarkable results, and also that some of the issues regarding AI-generated images have yet to be ironed out.

The prompt "Penguin" generated several slightly different variations of the adorable black-and-white bird, and the prompt "a spaceship" similarly produced different variations of a classic science fiction staple. Some prompts involving people, particularly running, proved a little problematic, as certain elements of the renders' legs don't quite connect, and so on. More outlandish prompts, such as "a person that looks like a leopard," also produced results that missed the mark somewhat, with the examples simply representing the animal or, significantly, a model of a leopard seemingly rendered over the bust of a person's face.

Of course, many future users of Shap-E will utilize the system in just this way: using fun prompts to see what's possible. At the same time, it will learn and improve and could prove to be a potent tool for a wide range of professionals.