AI Prototype Shocks Developers By Turning Drawings Into Working Software

A new artificial intelligence (AI) prototype is wowing developers by turning drawings into working software. The "Make It Real" feature was released via a collaborative whiteboard app maker called "tldraw."

let's go https://t.co/4dHJiLCegg pic.twitter.com/4y9VMiTN0h

¡ª tldraw (@tldraw) November 15, 2023

What is the AI feature about?

The feature allows users to draw an image of software and breathe life into it using AI. The feature uses OpenAI's GPT-4V API to visually interpret a vector drawing into a functioning web code (Tailwind CSS or JavaScript). In addition, the code is able to replicate user interfaces and even create simple versions of games like Breakout, ArsTechnica reported.

I think I need to go lie down. https://t.co/WxScGQTEKZ pic.twitter.com/wT6a7q6juZ

¡ª Kevin Cannon (@multikev) November 15, 2023

I'm sorry. This is just ridiculous. pic.twitter.com/brTkZ0fDMg

¡ª Kevin Cannon (@multikev) November 15, 2023

Nope nope nope. pic.twitter.com/0deFkHtCV8

¡ª Kevin Cannon (@multikev) November 15, 2023

In a viral thread, designer Kevin Cannon explains how the feature was able to create functioning sliders that can rotate objects on screen, an interface for changing object colours, and a functional game of tic-tac-toe.

Yea¡­ I am done. lol ?

I just drew a game and it works¡­ ? pic.twitter.com/GczEz4BAwe

¡ª Andreas Klinger ? (@andreasklinger) November 16, 2023

i can't with this pic.twitter.com/cUYV8JOCS9

¡ª Sam Selikoff (@samselikoff) November 16, 2023

Additional demonstrations showed a clone of Breakout with a working dial clock that can tick, a snake game, a Pong game, and more. If you want to experiment with a live demo of Make It Real online, follow this link.

INSANE...i just made a functional game with @tldraw in less than a minute with nothing more than a drawing pic.twitter.com/c094bwyrTW

¡ª austin petersmith (@awwstn) November 16, 2023

I mean wtf https://t.co/cM2ghw01v5 pic.twitter.com/bOxjjXiqQH

¡ª Ken Schumacher (@kenschumacherr) November 16, 2023

Also read: ChatGPT's Getting Smarter: 4 New Useful Features Coming To OpenAI's Chatbot

How can you use it?

To be able to run it, though, you need an API key from OpenAI, which could pose a security risk if others intercept your API key. As ArsTechnica explained, your API key may be used to run the code locally and shoot up your bill.

Unsplash

Also read: Newly Launched 'Humane AI Pin' Powered By GPT-4 Will Replace Your Smartphone

Who developed the tool?

Tldraw, developed by Steve Ruiz, is an open source collaborative whiteboard tool that has a basic infinite canvas for drawing, text, and media without needing a login.

It was launched in 2021 and has received $2.7 million in seed funding and is also supported by GitHub sponsors. After the recent launch of the GPT-4V API, Ruiz integrated a design prototype called "draw-a-ui" created by Sawyer Hood to insert AI functionality into tldraw.

Unsplash

How does GPT-4V come into play?

GPT-4V is a large language model (LLM) develop by OpenAI that can interpret visual images and use them as prompts. Make it Real works by "generating a base64 encoded PNG of the drawn components, then passing that to GPT-4 Vision" through a system prompt and instructions that turn the image into a file using Tailwind, as AI expert Simon Willison explained on X.

Found the system prompt that drives this thing here: https://t.co/UdSV5rUY2S

It works by generating a base64 encoded PNG of the drawn components, then passing that to GPT-4 Vision with that system prompt and instructions to "Turn this into a single html file using tailwind" https://t.co/sINwggRdtb

¡ª Simon Willison (@simonw) November 16, 2023

What do you think about this fascinating tool? Let us know in the comments below. For more in the world of technology and science, keep reading Indiatimes.com and click here for our how-to guides.