Will Artificial Intelligence replace developers (part 1)?

Home

After 10 years of working for smaller and bigger companies I decided to join as Chief Scientific Officer a start-up called PlatformX. I'll not exaggerate saying that it's the most interesting and challenging project I've ever worked in. Our goal is to automate the process of developing software. In other words we want to create an artifical intelligence a.k.a. "virtual developer" that will be able to communicate with a human being in order to collect requirements and then will write, test and deploy a program according to these requirements. Nice, isn't it?

One of the first thing I did was the research on the existing "virtual developers". I've found quite a few solutions, some of them very interesting and this post will be about them (I also gave a speech on this topic some time ago). At the end I'll try to answer the question if Artificial Intelligence will replace developers? And if yes, when? The post turn out to be long so I decided to split it into 2 parts.

Brainfuck Developer

I didn't expect that anyone will decide to create AI that will use an esoteric programming language. However, you never can be sure. Guys from Intel Labs and Primary Objects created AI that programs in Brainfuck and that under the hood uses genetic programming. Initially they tried to use a higher level programming language i.e. BASIC or to synthesise programs flow low level instructions but both attempts weren’t successful. Why?

In genetic programming we create initial populations of programs completely randomly. In other words we simply draw from the set of available instructions. It is a problem in the case of complex programming languages because it is unlikely that generated programs will even compile. In contradiction, Brainfuck consists of only 8 instructions and has very simple syntax. So, there is much bigger chance that randomly generated or modified program will actually work. Thanks do that we will finally generate working programs! The authors of Brainfuck Developer thinks that generally programming languages for AIs should be different than for humans.

You may also wonder how we tell AI what kind of program we need? The answer is by providing a function that judges if given program works according to requirements or not e.g.: it can simply validate if the output is correct. Brainfuck Developer was able to write among others: XOR function in 35 minutes, XML to JSON function in 114 minutes, reverse string function in 49s and what may be strange a program that outputs the text I love all humans in 10 hours!

Prophet

Prophet is AI created on famous MIT and it focuses on automatic fixing of bugs. Firstly it analyses patches/fixes of bugs that were created by human being in big open source projects, for example PHP interpreter. The goal of this step is to learn AI what is a good fix a bug. The authors assume that regardless of the project patches made by developers have some common features and AI tries to discover them. For example a fix often introduces a check verifying a value of some variable.

In the next step Prophet runs the set of test cases and it monitors the execution of a program in order to localise a bug in the code. It uses a kind of heuristic. If some lines were executed in negative test cases but not in positive ones it means that a bug may be located there.

Finally, Prophet generate possible patches/fixes i.e. it modifies lines of code where a bug is probably located. Of course there are many possible ways in which these lines can be modified. Here Prophet uses a knowledge about "what is a good fix a bug" to select the best candidates.

Experiment showed that this AI is better in fixing bugs than many other solutions of this kind. However, from 69 real world defects fixed by human being Prophet was able to fix only 19.

Codota

Codota is a kind of powerful intelligent code completion, unfortunately for now only for Java. Technically it is a plugin for IntelliJ and Eclipse. The main functionality of Codota is to try to predict what a developer is trying to write and help him or her in doing so by providing a snippet of code which does exactly that!

Codota builds its knowledge database based on publicly available data sources like: stackoverflow.com or GitHub. Unfortunately I don't know details how Codota works under the hood except what we kind find in Internet i.e. "Codota combines techniques from program analysis, natural language processing, and machine learning to learn from code". What is nice Cododa is for free. You have to pay only if you want Codota to analyse your private source code repositories.

ai.codes

ai.codes, which is or was a start-up from Silicon Valley, is similar to Codota to some extent. It is also a kind of intelligent code completion and also a plugin to IntelliJ. It provides 2 basic functions. Firstly, it can understand requirements given in natural language. For example we can write "open myFilename and read each line of it to a String list" and ai.codes will write a code that implements this.

Secondly it can order methods autocomplete list in more sensible way. For example let's assume that we are within addElement method and we showed autocomple list for a dictionary variable. ai.codes will decided that we probably want to add something to a dictionary and at the beginning of the list we well see add method.

At the begging I wrote "is or was" because the latest version of ai.codes comes from 2016 so I'm not sure what is the status of this start-up.

Deepcoder

At the end of part 1 let's take a closer look at AI from Microsoft and University of Cambridge. Deepcoder uses extremely popular, especially recently, deep neural networks in order to solve simple programming tasks, for example "Alice and Bob are comparing their results in a recent exam. Given their marks per question as two arrays a and b, count on how many questions Alice got more points than Bob."

What is interesting Deepcoder programs in Domain Specific Language which has relatively simple structure and was especially designed for Deepcoder. Does it reminds you something? Brainfuck used by Brainfuck Developer was selected because it also has very simple syntax.

Now a little bit of details. Deepcoder uses a standard feed-forward artificial neural network. Authors also tried to use Recurrent Neural Network but without much success. What is important ANN is not used to generate actual code but to predict what elements/functions from DSL should be used in the program.

Then this suggestions are uses to search the space of possible programs (written in DSL) that can solve a given problem. Thanks to ANN suggestions in can be done much faster.


This the end of part 1. Part 2 will come soon ;)

^*The picture at the beginning of the post was created by GDJ [CC0], via Wikimedia Commons