PBR Automation, From Photo to Material

As an old 3D artist and photographer I keep circling back to problems I have actually run into. This one is making PBR materials out of photographed material.

I had clients who wanted digital copies of their products, both for e-commerce and for photorealistic imagery. The point was images produced faster and cheaper than hiring a photographer, a stylist, assistants, a location, a pile of camera gear and a few hotel nights.

The old way was manual. Photograph the material, usually fabric, with flash from eight angles, then build a seamless texture from it in Substance Designer so the repeat does not read as a repeat. Ten minutes of shooting, not counting rigging the studio, twenty minutes in Substance Designer, then assembly and adjustment in Blender to make the digital material match the real one.

So I put it in Python instead, using the LaMa model to close the seams, exporting the result as a finished material and saving it straight into my Blender material library. It did not get all the way. You can see the seams, and the normal map in particular needs more work. I tried NanoBanana as well and it was not much better, though that may be down to running the free version.

Honestly, the result is roughly what I used to get out of Substance Designer, with the rest fixed in retouching. The difference is that it costs about twenty minutes less per material and a few hundred kronor a month for an Adobe 3D subscription I no longer need.

The next step is a scanning box, so the light is controlled rather than merely repeatable. Then the whole flow could run by itself: photograph the material, drop the files in a folder, and let the code produce the material, export it and file it in the library.

If anyone has ideas for taking it further, I would like to hear them. This is as far as I got.