I’ve been quiet lately, but that’s because I’ve been working on a video in the background and not taken much time for modelling any new elements. I’m quite excited to have been given permission to use a track by a very talented local musician, Skywatcher Music, who composes some beautiful music: I won’t say which one I’m using; will leave some aspect of surprise, but the one I have in mind captures the vibe and emotion perfectly.
I wanted to share some imagery of the frames I’ve had rendered with the the open source render-farm, Sheepit.
These may look familiar if you’ve been here before, and it may look as if I’m reusing images: I promise you I’m not, for the sake of the website – these are a quarter size of the actual frames, when we watch these frames joined together – they’ll be at 24 frames per second, to give the illusion of movement to our eyes. I’ve also displayed them out of order so there are no spoilers for the actual video when it’s finished.
The unusual shape is of the images is because I’m experimenting with a screen resolution of 2580×1080, which is a 21:9 ultra-widescreen format, whereas most of the time when we see widescreen in HD, it’s 1920×1080, which is 16:9.
Rendering, it’s a waiting game isn’t it? I’d wanted to produce a video of what the city looks like in its actual state – show you it how it’s meant to look, instead of dots and lines. I’m still not sure that this will even be finished by tonight, it’s a long process, especially on a single machine: the caveat of that is that I also cannot work on the machine because its system resources are being drained by the above.
So I got thinking, are there any free cloud computing options, so I can just remotely connect to, and use a big companies computing power instead of my own? And it turns out there are, from the tech giants you might expect.
First, though I want to explain what I wanted to do with one once I got my hands on one. My first thought was run Sheepit, ‘a free distributed renderfarm for Blender.‘
What is a render farm? ‘A render farm is a group of computers connected together to complete a large task. In the case of 3D rendering, most of the time a render farm will distribute frames of an animation to multiple computers. Instead of having a single computer work for 100 days, you can have 100 computers work for 1 day.’
Sheepit With Sheepit, new users must render at least 10 frames before being able to submit your own project into the queue, and other people rendering your scene uses your points: of course, you can set your own node(s) to render your project before everybody elses in the queue.
Having this process running on your machine uses up as much system resources as actually rendering a 3D scene, so it was a choice of one or the other. If I run them together, they crash: so rendering those 10 random users frames so I can upload my own is time consuming. In the screenshot above, the process is actually running on an Amazon computer, rather than my own, so this doesn’t drain my own machine in any way.
Meanwhile, on my own computer:
18 more frames to go from this sequence, the last took 6:32 minutes, that’s about average across this scene, so to work out when it will complete: 6.5 * 18 = 117 minutes, almost 2 hours. 18 frames is around half a second of animation, and why this update has a lot of writing!
Without giving too much away, here’s a single raw frame.
I’ve not done any colour adjustment to this, this’ll all be done later, when the individual frames have been composited together into video.
Cloud computers are in a nutshell, online computers you can just switch on and borrow for a bit (or a cost). These can be standard setups, like your average gaming PC, or whole entire systems with more or less unimaginable amounts of resources for incredibly complex computations. There’s so much you can do with them, I’ll not even begin to go into anything beyond the scope of this post. For now, I think I’ll just keep it simple.
I’m using Google Cloud and Amazon Web Services free accounts for this, I’d expand more upon the two services, but think they have quite enough reach on the internet without giving them more.
Scrap that, they’re blank
I believe this to be because I used Filmic colour profile for its realism, Sheepit tried to render with Blender 2.79b, which is vastly different to version 2.80 onwards, so wouldn’t have been compatible.
I’ve re-added it to render with 2.81a, and lets see what happens:
This looks more positive, like it actually has something to do now.
This became a problem. It will only work on this users project, and not my own.
I later found that the reason for this is that the free cloud instances weren’t GPU-enabled, so couldn’t take on the task, so was only working on projects it could handle. I will look further into this another time.
But for now, I’m going to let it run overnight and hope some kind, anonymous internet stranger renders my scene for me, and render the next segment the traditional way:
Success Module will be activated automatically at next reboot Do you wish to activate the module now? (y/n)y Module activated
But I can live with that for now. Main thing is, it works.
The current .blend file is now hosted on this server, and is available here. If you prefer, or want to use some kind of automation via a script, you can download this from the command-line by typing: wget thefan.uk/blends/sheffield-fps-city-centre.blend
Soon I will set up a Downloads page to host this, and any other subsequent downloads in the future – but for now – this’ll have to do.
I had started a high-resolution render to run, where I would sleep through. Alas, I didn’t sleep:
Set up SFTP for file transfers to and from the Thefan server
These are the textures imported with Blenderkit, which also increase the projects file-size with high resolution images. As nice as they look, for the sake of being able to compile, and for diskspace reasons – I’ve had to strip them out and will retexture the building manually using simpler UV textures.
‘I will never take you for granted again, good to see you buddy.’
I said to my wallpaper as it finally lit up to greet me, after an entire evening of black screens and white text.
I’d done some tinkering, and I broke things – quite unnecessarily, with hindsight. It all started late lastnight, I had the wise idea of updating my Linux kernel: for the fun of it. Didn’t particularly need doing, it was working just ok – but I thought it just probably needed doing. It didn’t go to plan, and it broke my WiFi dongle support – I’d done a bit of digging, and the 5.x kernel does cause problems with Realtek adapters, which is what this was, and now everything is pretty much the same, except my WiFi is borked.
“Command shells are like Gremlins, don’t feed them after midnight.”
My mate Matt, years ago.
My only option at that small hour of the morning was to haul everything downstairs, and plug into my router – or go to sleep. I tried to do the responsible but very out-of-character thing and forget about it for the night (admittedly, getting up a couple of times to try something, and going back to bed). I bought a new adapter from Argos on the way home from work today, a different make and hoped that that would just plug in and function straight away. It wasn’t detected at all. So… I had to bring everything downstairs and plug into the router after all that.
I carried on the updates with: $ sudo dpkg --configure -a
To make sure the update I’d ran the night before had all been completed successfully before continuing the upgrade (an upgrade will include the most updated drivers, right?).
I completed the update and rebooted, and was presented with:
I booted from up from a Linux Mint USB drive, and from here was able to at least access my system files, only my drive is encrypted – so I couldn’t just browse straight to them and copy them somewhere. I had to decrypt my drive first, and mount it as an accessible filesystem.
I did this by running the following commands: cryptsetup luksOpen /dev/sda3 sda3_crypt vgchange -ay mount /dev/mapper/mint--vg-root /mnt mount /dev/sda2 /mnt/boot mount -t proc proc /mnt/proc mount -o bind /dev /mnt/dev
The above commands will firstly decrypt the contents of your hard drive, change the Volume Group to modifiable, and mount the filesystem into folders in which you are able to access from the Live USB environment, by navigating to the /mnt folder
One problem with how I set this memory stick up, is that it doesn’t remember anything after I reboot (and there were a lot of reboots!) – so it’s very Groundhog Day when it loads back up, I have to go and find the same posts (this was useful) online, run the same commands, not be able to decrypt my home folder to back anything up, all in the back of my mind is:
I know, I last put my savefile on Github 6 days ago… nearly a weeks work.
After running the above commands, I was then able to login to the encrypted drive as if I was an administrator and use a command shell to try and bring it back to life. I used:
sudo chroot /mnt to access the contents of the drive, change system files in order to get the drive up and running again, reboot. Repeat.
And to (hopefully) save you the hours I spent figuring this out, I’ll show you what worked for me:
Before, in my /etc/fstab file, the first line of each of the drives (sda#) was set to its corresponding UUID identifier, we can see which one is which by looking at the blkid output above, so when it loaded up – it was looking for UUID=xxxxxxxx and not being able to find it.
If you look on the left of the image below, the drive is located at /dev/mint--vg-swap, whereas in the screenshot above – it refers to it as /dev/mapper/mint----vg-root-swap. They’re different.
This is only my swap partition, and I’m not too bothered about that, it still works despite the slight discrepancy, but it also applied the same mismatch to my entire file system and could not find it, which did cause problems (and panic).
This is what worked for me, obviously yours will be different depending on your setup- so compare with blkid (or gparted if you would like something a little more visual than a commandline)
I changed them to their hardware addresses, knowing that it would always be able to find them at that location, and I’m back! …but my WiFi still doesn’t work.
I did try to backup my progress, but my .blend savefile now exceeds 100mb: it’s too large to store on Github due to their filesize limit. In future, I’ll be using Github more for the development side of things, 3D assets (much smaller files!) – and will be storing the ever-growing 3D savefile on this server – but for now, there’s a copy on Google Drive – more for my sake than anyone elses, but if you want to have a look – feel free!
This post will partially be about the Ground object, too – our retopologised mesh of the height map data, Terrain. I wrote about this previously, and will assume that you have read it.
The building in the post is a firm favourite of mine, Old Queens Head. It’s not as if the thought of doing the building hadn’t occurred to me before, but something in there inspired me:
Why not use these as reference images? How apt that the reference images have an artistically constructed floor, on the very same building I began to construct the floor for our city on.
The ground in this scene is sunken slightly where the beer garden is. I cannot sink my ground plane in such a way because the height map, Terrain will prevent me, because of it’s accurate topological data. It cannot go that low without cutting a hole into it, which I’m not going to do.
So, I have to build everything around it, up because that direction isn’t locked.
Notice how the checker-board flows into what will be the right-hand entrance; this will allow for a smooth transition upon entering buildings.
Tonight I’ve been working on the ground for an area of the level: I want to start shaping the level so that it has some flow.
The ground, currently is an object called Terrain, and is the geographically accurate height map data of the level that I imported with blender-osm. On its own, it looks like this:
I need to leave this object alone – if I start messing with it, it willl break things. I’m going to use it as a guide, the new object I create on top of it will ‘stick’ to this mesh, and build upon its height map data, so I still keep its shape – this is retopology, it will be the ground that the players play on – as well as align in-game, and level entities – powerups, streetlights, advertising signs, steps and bus stops…
For the new object, I’ve used a checkerboard pattern so that I can see the contours of the terrain through the shape of the squares – and I can see at a glance which areas need fixing.
To retopologise sculpted mesh
Select the mesh that you want to retopologise
Enter Edit mode (Tab)
Select a single vertex
Press D to Duplicate it
P to Separate the vertex Selection
Exit Edit mode (Tab)
Select the single vertex, your new object
Rename the object
Edit mode (Tab)
Select Shrinkwrap from the Modifiers panel
Use the Extrude tool to start forming edges, make a face with 4 verticies selected with the Face tool, this’ll now be locked to your mesh or sculpts shape.