A look at some of the ways you can import geographically accurate map data from OpenStreetMap into Blender for your 3D projects. This is a little section of Hillsborough, Sheffield.
I need to bear in mind this is the first sentence people see when this link gets shared. Anyway, I’ve gotten quite stuck into modding Assetto Corsa over the past week, the workflow I’m using for the Sheffield map is similar to that of AC: using reference data from satellite imagery and streetmap data, and importing it into 3D.
Cadwell
The Cadwell track, after a curious search to see if there is one available was placed on a ‘someones done this already, and done a fantastic job of it – so I don’t need to (yet!)’ list. Despite being a list with a very long name, it’s a very short list so far.
I recorded a little race around it, added some music to the replay and uploaded it.
I decided to look for a track that I couldn’t find, something new – and that’s where two of the creative projects I’m working on coalesced.
I’d set out to use free and open source software to build a playable track in Blender – and while I didn’t complete that with Cadwell, it did put me on the right track to actually making usable race circuits. That’s what this post is about.
India ??
I went on a deep-dive of Indian motor racing circuits, and aside from the world-famous Buddh, I could find very little. This was the niche I was looking for.
I’ve started with the first other major circuit I could find, Madras Motor Race Track (Irungattukottai, Chennai) – and is one of a few circuits I have earmarked for a small India track and car mod pack.
The playlist (currently) has 4 videos of developing and testing Irungattukottai in Blender, and are mostly set to ambient music.
For your benefit, I’ve written this with hindsight – so I hope it saves you some time!
The first part we see is after I’ve downloaded the .osm (OpenStreetMap file) from where I’ve selected the region of the track. I’ve used blender-osm to import the map data, and I’m now trying to figure out how to get it to run.
Once you have it imported, first thing I do is press 7 on the numpad in Blender to jump to Top view, and select the objects until you find the racetrack, and name it 1ROAD. Anything you name it after that (with no spaces) is entirely up to you. This is the track you will drive on.
I export the 3D file to .fbx and import it with ksEditor, part of the Assetto Corsa SDK, From here, allowing that my map is compliant, export it to the tracks folder in your Assetto Corsa directory. If you are using Steam, it’ll be in: C:\Program Files (x86)\Steam\steamapps\common\assettocorsa\content\tracks
A quick way to do this is to find the folder of any track you like, copy it and paste it into a new folder with the name of your track, and change everything in your new folder to suit your level. Once you’ve exported your map to the .kn5 file format, drop it in this folder and make sure the file has the same name as the folder, and any textures you’ve used are in the tracks textures folder.
More to come
What I learnt
Everything needs a texture. If it doesn’t have one, it’s going to crash ksEditor
.dds (DirectDraw Surfaces) do not support layers. You can have transparency, but you can’t have multiple layers.
It’s best if your road is low resoultion mesh, high resolution imagery.
You will need Blender 2.7x Yes, going back to it feels like writing with your left hand, but the .kn5 exporter tool really is the best way of exporting your mesh. I found that using 2.9x and exporting to ksEditor only caused problems. Revert, revert to 2.7x, open the example track, and you’re on the way to a playable track.
Scale your UVs correctly I’d not scaled my grass textures correctly from high-res source photos, and the perspective it gave was that I was driving a car, but the size of an ant – each grass blade was the size of my car!
Make your track flat When I’d imported my track from my 2.8x experimentation with ksEditor, I’d solidifed the track so that it was 3D and had depth. I found though, when importing into 2.7x, hitting the side of the meshes depth sent the car into an uncontrollable spasm. The track has to be flat. 1 pixel (to the chefs amongst us, that’s one layer of onion skin).
If you use the example file from the Blender KN5 Exporter, use it as an example file, and then delete it When you playthrough, the intersection between the example racetrack, and your imported racetrack – it causes glitches will cause you to spin out, and fall into invisible potholes. It took me until Part 4 to realise this.
Use a UV grid. Google images ‘UV Grid’. Find a high resolution version, and apply it to everything. It makes a great placeholder image, and you can see exactly where to edit when you want to colour the texture.
Car skinning
No such luck, synapses are firing on all cylinders, I’ve cancelled an important VLC update for the sake of a screenshot, and now when I try to open anything with it, its only use is a strobe light.
When I run the game, I start at a 2:00 angle, which leads me to beleive that the empty to represent AC_PIT_0 is at a strange (or unsupported) angle.
While playing the lap from the OSM import, I’d timed that I was completing the laps in around 45 seconds, which is obviously ridiculously fast – so I scaled the track to be twice the size, and it now seems to work – and would probably make a good Drift track.
I’ve not completed a full track yet, because: – there’s no kerbs – the start position is on the grass – I’m quite sure that since I doubled the size of the track, the timer markers are in the wrong place.
For now, I’m stuck in the pit, I can see the wall above me that I’d just made – and I can’t move, because I haven’t given the pit floor mesh a pit texture.
So this area here is the pit-lane, and behind this would be the stadium area where spectators are sitting.
One iconic landmark of the track that absolutely will not be excluded is the MRF Tyres tyre arch, the start line and multi-storey viewing platforms.
Continuing on from the Pitlane after this tyre ends is a wall with a glass divider, like the ones in the picture above.
This is the wireframe of where the pitlane re-joins the racetrack.The end of this is actually where the Tyre is, and it reaches from one side of the track, behind a barrier to the outermost point of the pitlane.
By this point, the car would have re-joined the circuit, and the track above is part of the maintance route, which the Ligier Maintenance Vehicle from the video(s) above would drive on. Both barriers are solid walls with viewing windows above them.
MRF Towers
I don’t know if that’s their real name, but they’re a prominent and majestic outlook onto the track. I clearly know which companies logo to find a hi-resolution image of, and aim towards something like this. Interestingly, the tyre is not where I would expect it to be from the screenshot above; which is odd, since the tyre bears the MRF logo.
My point for this is the two MRF towers just past the start-line, they’re 1x and 1.5x the size of each other, and the top floor is an actual viewing platform – so it’s a great place to put a camera (or two!). If you’re feeling creative, you could create a camera, and put a sofa scene infront of it to show people watching from their couch inside of their hotel room!
While it is possible to complete a track, the kerbs and run-offs are not in place to help with corners (and I ran most of the racetrack backwards, and was not penalised). This is next.
I’d been looking for a way to revamp the Sheffield map. ~~ Previously, my primary focus had been an FPS – but now I’m thinking about making it as racetracks for the beautiful Assetto Corsa.
I looked for some videos on the level editor, and there is Race Track Builder (currently £44.99, even amidst the Steam Summer Sale) Race Track Builder is interesting, because it uses map and terrain data from Google Maps, and you edit in the shape of the track using splines and curves, and export it to FBX format so that ksEditor can read the file and convert it to a map file usable ingame.
This is the video that piqued my interest, and especially towards the end confirmed that what I’m looking to do is possible.
I took my Sheffield map (‘Here’s one I prepared earlier…’) – this was created by using the blender-osm plugin. I exported it the mesh to a .fbx file (a widely supported 3d model format) so that I could import it into ksEditor.
You click OK, and the application force closes.But it’s right, the fitness store Decathlon doesn’t have a texture assigned to it because I haven’t worked on it yet.
We at least know that ksEditor is at least reading the file structure of the FBX, and it appears to be compliant. Only thing is, this is a large map with a lot of buildings without textures. I need to start smaller, and I want to find a way to do it cheaply; we are after all a shoestring budget studio.
blender-osm
You’ll see a lot of content on this page which is produced with the blender-osm tool with the Sheffield map. I’m going to start smaller, but use the same process – I’m going to replicate a track I remember from my childhood days, family outings to Cadwell Park near Louth with my dad.
These are great for reference and in-game layout previews, but I’ll also need to consider heightmap and terrain data on the actual map
Day 2
Not the chronological next day, but the second day where I can sit down and make progress: this project has been in my mind since I started thinking about it, so pieces of the jigsaw have popped into my mind.
Using blender-osm, I zoomed right in to the track and imported the data from its Latitude and Longitude. I’ve previously imported the terrain data and satellite overlay, and the dark grey is where the road has been imported as 3D dataI’ve imported the file above into the Assetto Corsa level editor to export it to the game, but there are no textures.As a test, I’ve applied a basic Asphalt texture to the track.And found the elements in ksEditor that pertain to the actual parts of the track.
I’ve gone around the objects and given anything that doesn’t have a texture in the map (aside from the road and ground) a UV grid texture, as a placeholder so that everything in the level is assigned one.
While I have the mesh in place, the game is not told where the actual track is: Assetto Corsa recognises that there’s a Cadwell track in its game directory, and gets as far as trying to start the game with it: this is good, something is working here.
I need to tell it where the track is, and also race information like where the pit stops, starting line, crash pads, timing sectors, marshall positions and cameras are.The bottom two are rectifiable quite easily, they’re just pictures. The top two, though – I will need to explore the files of other, working tracks to see what information I’ll need to supply.
The missing files
ui_track.json This is various specific details you’re asked about the location of the track. This is the information that’s populated in the bottom left corner of the preview image below.
Map (Map.png) An outline of your map with a transparent background
Preview This is any image you’d like to use as the track preview. One way of generating this is to click the error message, and run the map – if it’s playable, you can press F8 to capture a screenshot and use that as your map preview. Since my track doesn’t run yet – I made a quick render in Blender of the track to use as a placeholder for now: you can now see it behind the layout.
I want to share with you a bit about my process of how I work through the means of screenshots and explaining them:
Map overlays
Here I’m using an OpenStreetMap overlay of the area UV mapped to my Terrain object: to the top right, I have a UV image editor window and four faces selected on the mesh in front of the cathedral (orange). The Terrain while in this state is not curved and has no elevation, it’s completely flat.
To quickly navigate to any point in the map, I can select a single face from the 3D viewport, hover over to the map, press a to select all faces in the UV editor and NumPad . to jump to that point on the map, it also works vice versa, and the top right corner I can search for buildings by name.
From a lower view, having the building and street names on the ground is useful. Having a building levitate 15ft in the air, not so much.
There are some buildings you might not have seen yet, they’ve been imported and adapted from the Broadstadium3D libraries on 3D Warehouse: these will need retopologising and UV mapping with my own textures.
Video editing
Here I’m finalising a composite of progress videos, backed to a Skywatcher Music track. Although the cut is slightly rough, I do hope to release this by the end of the night.
The bottom layer with the waveform is the audio for the video, the blue is video file, purple represents image sequences and the pink strips are speed modifiers, to speed up or slow down segments. This area can get very complicated.
As I’ve decided to use my full creative element for element buildings, the first thing to do is work out what it will be used for. I like the shape of this, it’s ominous enough looking for what a malevolent bio-engineering corporation would use – and can picture the back of the 312 block in high security fencing and perhaps covered in graffiti; I’m also considering a subtle dome over the gap between the buildings.
element.312 – Accommodation
element.313 – Laboratories
element.314 – Research
Progress
Added the floors and lift shaft of the Research building
Day 2
The Laboratories are at an angle and highly elevated from the ground; I will have to have a look in the real world how they match up.The rear of the Accommodation building, with Burner Security brandingAdded some grass and a pavement texture for staff and residents recreational area An early attempt at the Research building; I want this to look distinctive, but fit Sheffield’s architecture. A very low pass render, it almost looks like a paintingResearch Building, made the pavement and grass look a little more realistic.
There are a lot of trees in Sheffield (we have had some controversy around this of late) – and thankfully we have groups like STAG (Sheffield Tree Action Group) who are doing wonderful things to help keep those numbers up, and prevent them from being unnecessarily felled by our council.
There’s a new addition to the top menu, Wiki. This’ll be an explorable Wikipedia-a-like of the story-line of the game, and any future projects connected to this universe. It’s based on a series of stories I’ve written before, so the information available will fill up quickly, and will not detract from progress on the 3D modelling front.
I digress. The most obvious place to house afformentioned trees would be a building where a range of interesting plant life can be found, the Winter Gardens (you’ll be glad to know that in 2030, it still serves the same purpose) But first, let’s have a look at this tree:
Sketchfab model
Winter Gardens foliage area
Tree generation
I’ve naturally tried to find a Blender plugin that would help with creating trees, and there are some available, but they are paid plugins. If possible, I want to avoid this – keeping with the ethos of this site: a shoestring budget studio.
If there’s a resource out there for free that does some of the work for me, saving time and money – I’ll take that, thanks! My search for a free Blender plugin turned out to be fruitless (pun intended!) and I look towards free, specialised software instead – and came across Abraro; and was suitably impressed.
Almost every aspect of the tree is customisable, so you can let the Charles Darwin within you loose, and create any kind of tree you can imagine like a mad scientist genetic botanist. Once you have created your tree – click File -> Export and save it as an .obj file on your computer: you can then import it into Blender either as a new object, or straight into your scene.
When first importing the tree, it did put a strain on my (limited) graphics card, and I would get areas like this across the Blender application, which as you can imagine made it quite awkward to use:
I used the Decimate modifier on it to reduce its poly count from around 29,000 to 1,500. Not only does this lessen system resources, it also makes it more appropriate for a game asset.
I’ve only included this image because the crash makes it looks Vaporwave.
Not all models work, this is overly detailed and results in lots of rogue mesh, though you can see the actual shape of the tree in orange.
Below, I’ve modified the trunk and shape of a Black Tupelo so that it is shorter and more bush-like, and experimented with the leaves.
It turned out massive and soon after loading, Blender gave up and shut itself down: this’ll have to be something I branch into once I’ve upgraded to a higher spec PC.
A Black Tulepo
It looks very scientific (above) The tree with its leaves enabled.
Some renders!
Some slight colour modification in GIMP, no changes to the actual content of the model.
These have been scaled down by your web browser to make it fit the screen on your device, but you can right click the image, and open it in a new tab to see it full size: mobile users can usually press and hold their finger down on the picture to open in a new tab, which will allow you to pinch and zoom around it.
I’m going to update this as I go tonight, so won’t have any set title – because I’ve no set plan for this session, just progress, but first:
A frame of lastnights grass animation
Tuesday 4th February
I noticed that the road actually goes through the building, this will be for the delivery trucks to enter and exit the warehouse – and I’m going to keep it like that, for an indoor map that allows players to bring vehicles into the arena would be an interesting element – be it heavy weaponry, or a vehicle full of backup!
To the right we can see the terrain elevation dips from one end of the building to the other, where the drive through road will be.If we look closer, they’re not lined up in height: as all these objects are locked to the Terrain heightmap, I can only model upwards.For the meantime, I’m going to leave it like this, so I can easily access the Ground beneath it This will form part of the transition from the Road into the interior map.
I’ve got ideas for Sheffield Central Delivery Office, so this will become it’s own post. For now, I’m keeping it miscellaneous
Elements
Not all buildings on the map are named, some have the generic name of element.# – these are lesser known buildings that haven’t been indexed on openstreetmap For these buildings, I have made an agreement with myself that I will have complete creative control over what they are used for: these are so if I need to go straight into modelling without worrying about real-world accuracy, I can. These are where the storyline will take place.
What fun can I have with this?
Throughout the story, there’s a benevolent, omnipowerful biological engineering firm, Burner Security; they specialise in organic weapons of war -genetically engineered supersoldiers. They’re the bad guys. This set of three buildings would be ideal for one of their laboratories: Offices, Research and Chrysalis chambers for cloning.
This multi-floored, interior map will be intense!
Much of this building will be hidden from view, accessible from within other buildings at certain points.
The buildings intersect in a way that I can merge the buildings on the street and model in hidden entrances to the element building
There’ll be lots of hidden places in this game: I want one of the enjoyable elements to be urban exploration.
Rendering, it’s a waiting game isn’t it? I’d wanted to produce a video of what the city looks like in its actual state – show you it how it’s meant to look, instead of dots and lines. I’m still not sure that this will even be finished by tonight, it’s a long process, especially on a single machine: the caveat of that is that I also cannot work on the machine because its system resources are being drained by the above.
Cloud computing
So I got thinking, are there any free cloud computing options, so I can just remotely connect to, and use a big companies computing power instead of my own? And it turns out there are, from the tech giants you might expect.
First, though I want to explain what I wanted to do with one once I got my hands on one. My first thought was run Sheepit, ‘a free distributed renderfarm for Blender.‘
What is a render farm? ‘A render farm is a group of computers connected together to complete a large task. In the case of 3D rendering, most of the time a render farm will distribute frames of an animation to multiple computers. Instead of having a single computer work for 100 days, you can have 100 computers work for 1 day.’
Sheepit running in a shell (front) and my user profile behind it – showing my nodes participations in the queue (you can see why this particular users scene will take hours)
Sheepit With Sheepit, new users must render at least 10 frames before being able to submit your own project into the queue, and other people rendering your scene uses your points: of course, you can set your own node(s) to render your project before everybody elses in the queue.
Having this process running on your machine uses up as much system resources as actually rendering a 3D scene, so it was a choice of one or the other. If I run them together, they crash: so rendering those 10 random users frames so I can upload my own is time consuming. In the screenshot above, the process is actually running on an Amazon computer, rather than my own, so this doesn’t drain my own machine in any way.
Meanwhile, on my own computer:
18 more frames to go from this sequence, the last took 6:32 minutes, that’s about average across this scene, so to work out when it will complete: 6.5 * 18 = 117 minutes, almost 2 hours. 18 frames is around half a second of animation, and why this update has a lot of writing!
Without giving too much away, here’s a single raw frame. I’ve not done any colour adjustment to this, this’ll all be done later, when the individual frames have been composited together into video.
Cloud Services
Cloud computers are in a nutshell, online computers you can just switch on and borrow for a bit (or a cost). These can be standard setups, like your average gaming PC, or whole entire systems with more or less unimaginable amounts of resources for incredibly complex computations. There’s so much you can do with them, I’ll not even begin to go into anything beyond the scope of this post. For now, I think I’ll just keep it simple.
I’m using Google Cloud and Amazon Web Services free accounts for this, I’d expand more upon the two services, but think they have quite enough reach on the internet without giving them more.
Using Google Cloud, a frame rendered in 2 seconds.
Amazon Web Services took 34 seconds, but lets bear in mind these are free offerings, and shouldn’t draw comparison from differing setups, and availability of paid services.
Scrap that, they’re blank
It half worked, they came out blank. No wonder they were so fast!
I believe this to be because I used Filmic colour profile for its realism, Sheepit tried to render with Blender 2.79b, which is vastly different to version 2.80 onwards, so wouldn’t have been compatible.
I’ve re-added it to render with 2.81a, and lets see what happens: This looks more positive, like it actually has something to do now.
My Google Cloud instance seems pretty insistent on rendering this project even though it’s supposed to work on my own projects first – but eh, this looks fun – I’ll let it.
Later…
This became a problem. It will only work on this users project, and not my own. I later found that the reason for this is that the free cloud instances weren’t GPU-enabled, so couldn’t take on the task, so was only working on projects it could handle. I will look further into this another time.
But for now, I’m going to let it run overnight and hope some kind, anonymous internet stranger renders my scene for me, and render the next segment the traditional way:
Borrowed St Petersburg – It’s OK to not be OK for this quick screencast of retopologising the ground around The Old Queens Head, and by St. Pauls Tower.
Not only because they’re a great (local!) band, but the track title is something I’ve needed to realise as of late, you need to know it too.
Success Module will be activated automatically at next reboot Do you wish to activate the module now? (y/n)y Module activated
But I can live with that for now. Main thing is, it works.
Download link
The current .blend file is now hosted on this server, and is available here. If you prefer, or want to use some kind of automation via a script, you can download this from the command-line by typing: wget thefan.uk/blends/sheffield-fps-city-centre.blend
Soon I will set up a Downloads page to host this, and any other subsequent downloads in the future – but for now – this’ll have to do.
Previously…
I had started a high-resolution render to run, where I would sleep through. Alas, I didn’t sleep:
Set up SFTP for file transfers to and from the Thefan server
Set up Thunderbird as my mail client for dave[at]thefan[dot]uk
Made the design decision to put a back window on the lower floor. Not technically accurate, but this is 2030 – times have changed, people want to see out the window.
Tried to compile the game, and the textures on Old Queens Head are causing errors,
These are the textures imported with Blenderkit, which also increase the projects file-size with high resolution images. As nice as they look, for the sake of being able to compile, and for diskspace reasons – I’ve had to strip them out and will retexture the building manually using simpler UV textures.