Mistral, the French AI startup backed by Microsoft and valued at $6 billion, has released its first generative AI model for coding, called Codestral.
Like other code generation models, Codestral is designed to help developers write and interact with code. He was trained in more than 80 programming languages, including Python, Java, C++ and JavaScript, Mistral explains in a blog post. Codestral can complete coding functions, write tests and “fill in” partial code, as well as answer questions about a code base in English.
Mistral describes the model as “open,” but that is up for debate. The startup’s license prohibits the use of Codestral and its outputs to any commercial activities. There is an exception for “development,” but even that has caveats: the license continues to explicitly prohibit “any internal use by employees in the context of the company’s business activities.”
The reason could be that Codestral was partly trained on copyrighted content. Mistral neither confirmed nor denied this in the blog post, but it wouldn’t be surprising; there is evidence that the startup’s previous training data sets contained copyrighted data.
In any case, Codestral may not be worth it. With 22 billion parameters, the model requires a beefy PC to run. (Parameters essentially define an AI model’s skill at a problem, such as analyzing and generating text.) And while it beats the competition according to some benchmarks (which, as we know, are unreliable), it’s not a blowout.
While impractical for most developers and incremental in terms of performance improvements, Codestral will surely fuel the debate about the desirability of relying on code generation models as programming assistants.
Developers are certainly adopting generative AI tools for at least some coding tasks. On a stack overflow survey As of June 2023, 44% of developers said they now use AI tools in their development process, while 26% plan to do so soon. However, these tools have obvious flaws.
An analysis of more than 150 million lines of code committed to project repositories over the past few years by GitClear found that generative AI development tools are resulting in most wrong code being pushed to codebases. Elsewhere, security researchers have warned that such tools can amplify existing bugs and security issues in software projects; More than half of OpenAI’s ChatGPT answers to programming questions are incorrect, according to a Purdue study.
That won’t stop companies like Mistral and others from trying to monetize (and gain recognition) their models. This morning, Mistral launched a hosted version of Codestral on its Le Chat conversational AI platform, as well as its paid API. Mistral says it has also worked to integrate Codestral into application frameworks and development environments such as LlamaIndex, LangChain, Continuar.dev and Tabnine.
