Show a model one photograph of a chair and ask it for the whole chair. The front may be well described. The back, underside and hidden connections are a request for invention.
Make-It-3D approaches this with a two-stage process. It first optimizes a neural radiance field using the reference image and a diffusion prior. It then refines a textured point-cloud representation, preserving useful appearance from the original view while improving other views. The paper and demonstrations are the basis of this note; code was not yet available to try when the work caught my attention.
The combination is attractive because the inputs have different strengths. The photograph is specific evidence about one appearance. The generative prior contains expectations learned from many images. Those expectations can make unseen parts plausible, but they do not turn the unseen parts into observations.
A chair is a useful example because plausibility and correctness can come apart. The model might produce four reasonable legs while the actual chair has a different support structure. From the original viewpoint, the error may be invisible. Rotate the object and it becomes part of the result.
This suggests two ways to use the technique. For a concept asset, a plausible completion can be welcome. The generated back is a design proposal. For documentation of a real object, the same completion needs to be recognized as uncertain. The intended use determines how much invention is acceptable.
Inspect the views the input did not supply
The reference view is the easiest place for the result to look convincing because it has direct image evidence. The useful inspection moves around the object, looks at the silhouette and checks how newly invented surfaces connect. A texture can temporarily conceal weak geometry from one angle.
Lighting introduces another ambiguity. A dark region in the photograph may be paint, a shadow or a cavity. If it becomes part of the surface appearance, changing the viewpoint or lighting can expose the assumption. An object that looks like the photograph is not automatically a physically separated account of shape, material and illumination.
There are practical asset questions after reconstruction too. A game or interactive scene may require collision geometry, a manageable representation and a way to edit particular parts. A textured point cloud can be valuable without meeting every expectation attached to the word "asset."
I am interested in the possibility of using a single image as a starting point for iteration. The first output could help a creator decide what to capture, model or correct next. That is already a useful role, even when the result needs further work before entering a production.
The compelling part is the combination of a concrete reference with an informed guess. The careful part is keeping track of which is which. A good workflow would preserve the supplied evidence while making the proposed completion easy to inspect and change.