From Standard to Stunning: AI Upscaling for 3D People in ArchViz
Used 3D interior scene from the mighty Bertrand Benoit and 3D people from our very own Humanalloy.com collection
Introduction
Although complete AI image generation through stable diffusion is grabbing headlines everywhere, for us 3D artists, it isn't quite stable enough to make it do what we want just yet. There is, however, an AI technique that can help us extract the last 10-20% of details as part of our post-processing with only minimal hallucinations. It is called Up-scaling. And it is especially useful for 3D people.
SUBSCRIBE
And receive a free model
Full circle but with a twist
In ye olden days (and disturbingly often today even ) we used to use cut out people for our archviz renders. To the left (1) the most painful example I could find in my ye olden portfolio.
One advantage: the photo cutout people were always realistic looking on their own. A few of the downsides: they never quite fit the perspective, the lighting or wlaking direction that matched your render. It was quite a project finding just the right mix of those in your massive collection of cutout people. Especially when you need to fill a whole square.
Fun fact: this is why I started Humanalloy back in 2009. I wanted to solve the problem of finding realistic, versatile 3D people that could fit seamlessly into any scene. How hard could it be to just make 3D people!?
Having 3D people gives you so much freedom! Just drop them where you need them—whether it's lounging in a park or bustling in an office scene. No longer worrying about all aforementioned problems.
However there are some issues when using 3D people. The inherent realism photos have is time consuming to create for me as your 3D people creator and because of this the price per item goes up.
When Humanalloy started, we used to only sell 'Premiums' (see image 2). A really skilled artist spent about 5 full days creating such an asset. We created materials for every surface and created realistic hair by hand.
On top of that, the 3D files also became quite complex to use, which is problematic since you are mainly using them as background people, not the star of your illustration. So we created 'Basics' (see image 3) We could create 5 of those a day and for most of you these Basics workout great.
(By the way if you still want 'Premiums' for your projects drop me an email via the Alloy Enhance page)
Now it seems we have come full circle but with a twist thanks to 'AI Upscaling' Lets Ask AI to describe this technique, seems fitting:
"Stable diffusion upscaling is a process that uses AI to enhance the quality and resolution of images. It works by taking a low-resolution image and refining it step by step, filling in details and improving clarity using a technique inspired by how noise in images can be removed to reveal sharper visuals. This is done with a "diffusion model," which has been trained on large datasets of images to understand how details look at higher resolutions. The result is a more detailed, natural-looking image that’s ideal for creative projects like architectural visualization or 3D art."
Examples of this can be seen at the top of the page and to the right.
Although the name might suggest otherwise, it's not your grandmother's simple image enlarger. It actually adds details that do not exist in the original.
Using this upscaling gives us both the flexibility of using 3D people in our scene with all the the upsides such as fully matched perspective and lighting PLUS the photorealistic look we got when using 2d photo cutout people and all this while being fully art directable!
Basics with an upscaler applied
Exterior used is from the amazing collections at Evermotion.org and 3D people from our very own Humanalloy.com collection
Upscaling in Action: A Hands-On Tutorial
"So... all this arcane magic must be incredible technical and difficult!" I might hear you think. Rest assured, it is not at all.
- where and how you can upscale
- what the current limits are and how to solve them
Let's go!
Some closeups
Where to upscale
There are quite a few apps and sites to do the actual upscaling. Ive tested three of them:
All three do much more than upscaling so look for the 'Upscaling' section of each.
They are all great fun to play with. They all come with 3 to 4 sliders to play with that pretty much do the same thing for all but can be labeled a bit different:
- Creativity
- Resemblance
- HDR
- Strength
On top of that there are dropdowns to select different styles like: Realism, Cinematic, Soft, High-HDR and more.
Most come with some free tokens for you to play with when you sign up. And when you decide to buy tokens it gets you quite a lot of generations attempts.
There is a lot of variables you can tweak, and I did so you do not have to.
Which to pick?
Honestly they are all pretty similar. You can get something useful out of any of them and they all hallucinate weird stuff at times or when you push the sliders to the extremes. Leonardo.ai is the only one that 'just' does a 2x scaleup but also this is not really a problem with some prepwork on your end.
Magnific.ai and Deepdream.com have a prompt field that can be useful to guide the generation a bit. For example, the upscalers seem to want to make mini adults from children. Entering a prompt like: 'the right person is a little girl' actually helped me.
Just drag and drop
The basic process is as simple as:
- Rendering your image
- Saving render
- Dragging render into Upscaler website
- Fiddle with settings
- Click upscale
- Download result
Limitations and how to work around them
This might be arcane magic but it cannot work wonders.
It seems when the upscaler does not have much to work with it starts to do funky things and this is most noticable with the people in the background. You can either not go trough the effort of applying upscaling to the background people or do the following...
You might not might not want to upload crazy high res images to the upscaler websites. There might be limits and also the bigger the image the more credits you use. Also rendertime alone would be a nightmare. What works well is to render larger resolution crops of just the people.
Then upscale those and paste them back in the original render while scaling them down to the correct size in your post processing software of choice.
In the example to the right I used quite large crops however you can make as many as you want. possible one per 3D human.
Used 3D interior scene from the mighty Bertrand Benoit and 3D people from our very own Humanalloy.com collection
Layered approach
There are a few reasons you want to work in layers for your final composition.
- You probably do not want to upscale everything in your image
- Some parts you might want to push the upscale a bit more
- Just to find the hands go haywire...always the hands
- Above mentioned background problems
In the video to the right you can see that the original upscale left the mothers hair very featureless. I had to go to the extremes in the upscalers 'realism' and 'similarities' sliders to get something I liked for the hair. The result for the rest of became something of a hairdressers worst nightmare. If you go beyond the mildest default settings of the upscalers the hands start become a nightmare on its own. And to top it off all the upscalers though the pink sweater of the little girl was skin and started adding horrific details...
All to say: apart from the mildest upscale settings, do expect to do some post work, such as adjusting the hands, fixing hair details, or blending the upscaled areas more seamlessly into the overall image.
Experiment
I want to leave you with one last experiment. The guy in the back is one of my older rigged 3D human scans. I often find that rigged 3D scans often do not feel quite right. Often because they are scanned in an a-pose with the clothing forming to that pose. When you pose that model later something feels not quite right.
Upscaling partially solves that. Folds in the clothing seem to be more on point. The pose however feels just as bad as I created it. So spend more time on the pose than the clothing when using rigged 3D people when upscaling.
Also there are non-people parts that come out better when upscaling an image. Try experimenting with compositing those in as well where appropriate.
Conclusion
When adding upscaling to your post-processing pipeline it becomes far less important to purchase high resolution, more expensive and difficult to use 3D people.
Here you are in luck since it just so happens I have a 3D people store here selling some of the best and most affordable 3D people! Which become even more affordable if you buy them as sets or libraries .
Here you are in luck since it just so happens I have a 3D people store here selling some of the best and most affordable 3D people! Which become even more affordable if you buy them as sets or libraries .
