For the past three years

I have participated in the annual Junction hackathon. One of the biggest hackathons in the Nordics with over 1000 participants jointly working on 200+ projects, each addressing a problem given by one of the Junction’s challenges. Each year I promise myself to write-up on the project me and my team did, the experience I have gained and most importantly, the notes for next year’s winning strategy.

Me and my 4 other team members, have set our sight on the Kone challenge. The title of the challenge was: BIM for any building, by anyone with the general task being Transform sales: Gamify building 3D modeling and exporting with just a few clicks. On the side note, I hate the word gamify. Another one of those buzzwords thrown around everywhere. After going to the Kone stand, asking the staff there to give some more details on the project, we knew what we were supposed to make. Take in 2D, highly detailed floor plans, work some magic and have a 3D digital twin of that ready to get exported. This is where I draw my first note on winning. The challenges can be classified into two categories. First one where the challenge givers just throw a theme out there not really knowing what they want and being very vague in their description. Then there’s the second one where the challenge givers know what product they exactly want. It is very likely though that the first kind can be turned into the second one if you really prompt the challenge staff about their expectations. Hence, challenge prep is crucial, especially with the second category where you want to really know what you implement.

Furthermore, the first general themed category really relies on the idea itself. Bullshitting can really lead you a long way there and people can turn Junction into a figma-ton (I am coining this now) i.e. doing a mock-up of a solution in Figma and doing a really nice promotional video and still win. My team and I weren’t exactly the greatest bullshitters, and we really love getting our hands dirty in the implementation and general software engineering side of things, so we were set on…

Creating the best damn 2D floor plan to 3D building model converter on earth on god.

We started out researching existing solutions. Turns out people write papers and stuff on it and there ain’t such library or a model that would do such a thing. We were suspecting that, but it was worth a shot. Kone provided us with actual SVGs of floor plans of buildings. Those were… huge and detailed. Averaging about 30 MBs for one floor it took some time to before one could view it on the browser. Unfortunately they did not contain any useful metadata and were largely just SVG paths in unordered fashion. We discarded the idea of pre-filtering itself as our solution was supposed to work also for non-svg floor plans (which it ultimately didn’t work for either way). We started playing with computer vision trying to extract the relevant features of the rasterized version images done by inkscape floor_1.svg --export-type=png --export-filename=out.png --export-background=#FFFFFF -d 80. Every team member were tasked with it, and it frankly took us too much time for what was worth. I managed to come up with the best and quite frankly, the simplest extraction method by running basically threshold the pixel value to discard the insignificant lines (that through the process of rasterization get turned gray) leaving us with usually walls and gapy lines. The second and final step was running a Hough transformation on that output to get rid of those single pixel cluster artifacts while retaining the straight walls. More sophisticated methods weren’t really generalizing well, and we found this one to give good enough performance. Now we had a black and white mask image.

Next was creating 3D models out of those cleared up rasterized 2D floor pans.

At some point, our team member started working on the UI layer, and we started thinking about how we want to actually have the whole thing integrated. While they were doing their React stuff, we came up with an idea to put back the rasterized image with only relevant wall structure back to vectors, extrude them and have a ready-made .obj model. We had some issues related to vectorizing it back as the vtracer tool was interpreting black background and creating a path around the whole image, but it just came to inverting the mask. You would think that extruding from a 2D mask and creating an .obj file would be simple, egh? Well, we could have used the blender’s python interface and be relatively well off, but we didn’t want to cause the setup looked a bit extensive and at some point we would need to start running it on the server, right? RIGHT?! Well, after all the attempts failed at trying to find a nice programmatic way to extrude an SVG to 3D obj, my friend realized that Three.js, a library that we would, either way be using to display the results has a nice way to extrude SVGs on the client side and that it features and obj exporter!

Client side, egh?

We should have done a bit more planning

Up to this point we have established the following pipeline:

upload SVGs to the sever → rasterize and make a 2D mask → send back the mask for client adjustments → adjust the mask and send it to the server again → convert mask to SVG and extrude it to create an obj file → display it on the client

where the bolded steps were done on the server. Felt like plenty back and forth and having realized that we can do the extrusion and obj generation on the client it prompted us to think, what else can be done on the client side? This is where another advice comes from. Invest time into thinking about the architecture! For the duration of the first half of the hackathon we forgot that a browser is a full-fledged operating system nowadays. We have therefore sought ways to do the server-side things on the front-end. We wanted to avoid the bottleneck of sending giant files over the internet and making the user wait for the results. Later we have realized that we have shot ourselves in the foot with that. Nevertheless, turns out OpenCV works on JavaScript as well and the browser can obviously rasterize SVGs, we just needed to get that data somehow. And we have done it by creating an Image element and drawing it to the canvas to obtain what could possibly be described as JavaScript’s pixel buffer. To obtain a desired rasterization DPI it took us a bit of playing with the canvas’s width and height, but ultimately it was done. The mask now has been created truly on the client side.

It was slow but it’s a hackathon

Our amnesia started to fade away, and we realized that gzip would have either way brought the size of the SVG, a plain-text with significant repetitions to a mere average of 2MB from the 30MB original. And doing the whole masking part on the server would have been probably faster (at least development time wise as we already had it prototyped there). Furthermore, it hasn’t helped that the code was a spaghet and it took a significant amount of ram to run it on the browser. Anyway, the last missing functionality was converting the raster image back to vector. Initially when we thought of doing everything client-side we just glanced that vtracer was able to be run using webassembly. None of us had any experience using webasm, but we thought that the iterop would be manageable. Like a man who had been shot in the chest with a rifle, the goal to run everything client-side was shot in the chest with a rifle. Brains fried 4 hours till submission deadline we couldn’t figure this stuff out in our sleep-deprived state. So we took the easy path by creating a single server function that converted a base64-encoded PNG (basically the canvas’s pixel buffer) to a SVG using vtracer and returned it to the user.

Once we have integrated this recursive infinite array of hacks, our favorite spaghetti code dish was done. We integrated it all together, and you can view it at: https://github.com/1ovel/junction-2024-frontend. Though you might find it repulsive, it works, and you can view it for the foreseeable future at:https://junction-2024-frontend.vercel.app.

With all that done, we had exactly an hour to create a demo video of our project. The first half was done by one of my teammates, but we needed to showcase it working. So quickly drafting up the script and a recording of my yap later I sat to Kdenlive, just to have it crash on me 4 times and crash during rendering once. It was countdown time and our video was still rendering but once it rendered we had 1.5 minutes to submit it finally so once YouTube started processing it we had the link and updated our submission with it.

In conclusion

This blog post is way too long, but we should have:

• Thought more and do a more thorough research before we start implementing things. This time it was thinking more on the overhead between running side on the client vs sending a request over to the server.
• Put more effort in marketing, promotional video cause that’s where you earn votes. The project description allows for Markdown as well as GIFs. Have it bolded, highlighted, this is where you show the cool stuff, the 3D models, the transformations. While the video has the additional overhead of having people to watch the preamble describing the problem domain to get to the fun and colorful stuff of the implementation.

With all that new experience gained from Bimify, the team Turbo Dranie will return again next year!