Founder here. As a front end developer, I turn design files into front-end code regularly. But it's a repetitive and demotivating task. It takes time, lots of meetings and boring back and forth questions. I want to change that.
So I tried to automate this process. I trained a deep learning (DL) model to visually understand what elements are in a webpage (e.g. input, button, nav, etc) and build the front-end boiler plate automatically for me. I learned a lot in the process, the actual site builder is a mathematical interpretation of a Sketch file and the DL model outputs what elements to include in what section.
To be completely transparent I'm not quite there yet. I'm only able to have ~90% of the site right. The other ~10% are misinterpretations or incorrect positioning of the elements. In order for this to work properly, it requires manual validation - which for my own projects I’m more than happy to do.
Even at this stage I think this tool can help others as well, so please let me know if you have any questions.
We can be more productive without repetitive tasks.
It takes time, lots of meetings and boring back and forth questions. I want to change that.
So I tried to automate this process.
The reason for all the meetings and questions is that the designs you're working from are ambiguous, and you don't really have enough information to turn the design files in to working code. I can't quite see how a deep learning model solves that problem. By the sounds of it you've automated turning the original, ambiguous, wrong designs the you have at the start in to code and tried to remove the discussion process that takes those files and turns them in to the right solution to the customer's problem. This is not going to lead to better front end code.
Depending on the customer, that could still be helpful. I've worked with people where you can ask them very specific questions over and over and get nowhere - the only way they're capable of moving forward is if you give them something you know is wrong so they can look at it and say "No, it needs to X."
It's annoying to do, but if you can automate producing the initially wrong product for review it'd be easier.
Wouldn't you achieve the same feedback with minimal dev effort by simply modifying the sketch files with the client until they are signed off on it, which is what designers would be doing before discussing the technical feasibility/performance tradeoffs with devs for implementation? Assuming you are working with a designer that is creating sketch files.
Have you worked with clients? If you have a good producer/leader yes you can get a lot done. But many times this just happens as A. Client approves design B. You build the Beta of the website and launch it for testing with client C. Client shows wife/boss/friend who says “I think we should change this”
End result: you end up having to redo the website. This tool helps get that Beta up even if just for the client to try it and show his friends/wife/boss
The point I made in my reply is that this tool won't save you from those meetings and rounds of changes. As it'll speed up the coding side of the iterative development process by making exactly what the client has described rather than the developer using their experience to build what the client actually wants, there'll have be more meetings in order to get to what the client really needs.
The benefit of automating the production of code is useful, and plenty of people would want that, but targeting the messaging at developers isn't going to work. I would have thought selling this tool to people who employ developers as a way to reduce the number of people they need is a far better route to market.
For everyone else:
"Cunningham's Law states 'the best way to get the right answer on the internet is not to ask a question; it's to post the wrong answer.'" [0]
There is advantage here. Imagine being able to have a CI process for design files - just upload your sketch file's changes to the repo and this tool in your pipeline builds/deploys a version of this in HTML in your staging environment. Easy link sharing built in.
If designers have the ability to work closer to the web, more power to them. It should be our job as software engineers to make this as easy as possible and this tool could be one specific implementation.
For larger clients who are familiar with the design process that's true. But for smaller clients, I would 100% take a working, "wrong" version of a site based on a design file over Sketch iterations any day. You'll get immediate, actionable feedback much faster (ex. "I don't like button X, change it to Y"), often times with less effort and a happier client at the end.
Depends on who he's meeting with and what industry. If it's an ad/service company where the design process is controlled by project managers or the clients... then I totally see where he's coming from in terms of meetings being a drag
There's almost always a reason why the person who organised a meeting wants one, and I doubt you can use automation to solve whatever their problem is without understanding it first.
If anything, by automating the process of producing code you're going to leave front end developers with nothing but meetings because the rest of the job is going to become a matter of pushing an AI-powered button. They won't thank you for that.
I've done something similar in the past (building models that translated paper drawings into HTML+css) and ultimately found it usually didn't save me much time.
Translating a look into code is much less labor intensive than it once was and code reuse and stylesheet preprocessers and the ilk (webpack, etc) save tons of time.
I'd get something reasonable out of the model but then the actual look and feel work came into play.
My next step was to be able to reference other sites for styling hints but ultimately I wasn't getting enough utility out of it to continue.
Do you find the same with this approach? If so, what plans do you have to take it the next few yards so it's a more comprehensive front end tool.
Actually my workflow improved drastically after getting the boiler plate to work on the fun stuff (interactivity, animations, etc). I use this tool to get started much faster.
Hey, why not give something for people to try. I know[1] it takes (GPU) resources and may give it a hug of death but its much better to let at least some people try it out and give this a feedback while its hot. People may lose interest later on, who knows.
You may want to have an example comic gif on the homepage. I didn't really understand what it did until I tried it. Plus, it would be neat if panels with more text were visible longer.
Hi! I'm super curious about how you structured the input and output to the neural network, and which architecture you chose, if you're willing to share (I work at OpenAI)
If you do share I'm happy to give you feedback here if I have any!
Interesting idea but how maintainable is the code?
I've used a few tools over the years that automatically generate code, mostly in C, but I've usually had to rewrite it as it never fits in with any decent coding standard
That's our main goal. To always give you good code. Otherwise this tool doesn't make sense. Hopefully you'll find this code maintainable with close to zero rewrites.
Wow, nice work!
I used to work for Webydo on the part that translate the DB to static files, tons of work and the result was basically an "uneditable HTML blob".
I would really love to see the output zip file, any change you have something less complicated that goes responsive to show?
At this stage we are working hard to solve this simple problem of going from a design file to a front-end boiler plate. No animations or logic. We are looking forward to get as much feedback as possible from our users to understand where to go next.
So I tried to automate this process. I trained a deep learning (DL) model to visually understand what elements are in a webpage (e.g. input, button, nav, etc) and build the front-end boiler plate automatically for me. I learned a lot in the process, the actual site builder is a mathematical interpretation of a Sketch file and the DL model outputs what elements to include in what section.
To be completely transparent I'm not quite there yet. I'm only able to have ~90% of the site right. The other ~10% are misinterpretations or incorrect positioning of the elements. In order for this to work properly, it requires manual validation - which for my own projects I’m more than happy to do.
Even at this stage I think this tool can help others as well, so please let me know if you have any questions.
We can be more productive without repetitive tasks.
[1] Demo: https://zecoda.com/#demo
[2] Request early access: https://zecoda.com/#subscribe