Alibaba Group's Ali Tongyi Intelligent Computing Laboratory recently launched an AI assistant tool called SmartVscode in the Visual Studio Code extension marketplace. This tool utilizes the laboratory's self-developed application intelligent control framework, App-Controller, to enable direct control of VS Code through natural language, aiming to optimize user interaction experience and enhance development efficiency. Currently, the core technology of SmartVscode has been open-sourced, and its App-Controller framework allows applications to respond to and execute commands based on natural language, greatly simplifying the interaction process between applications and users.
One of SmartVscode's core capabilities is the ability to easily control VS Code using natural language. Users can input the operations they wish to perform, such as style changes or writing specific code, and SmartVscode will automatically handle steps like finding and modifying configurations. Additionally, SmartVscode can leverage the rich functionalities of VS Code to develop various applications, allowing users to describe their needs in natural language, enabling the large model to assist in application development through VS Code.
By using the SmartVscode plugin, users can search for and download it from the VS Code extension marketplace, then enter task descriptions in the SmartVscode dialog window located in the left sidebar of VS Code, waiting for execution and displaying results. The App-Controller framework, which supports SmartVscode, is an innovative API orchestration framework based on large language models and agents, enabling natural language control over applications by integrating application API interfaces.
The core competitive advantage of the App-Controller framework lies in its automated API orchestration logic and user-friendly data interaction model, which reduces the complexity and development cycle for application developers during the integration of intelligent processes, speeding up the expansion and refinement of application functionalities. It also supports various large language models and provides rich documentation resources to help developers easily get started and fully utilize all the framework's features.
Although SmartVscode demonstrates strong potential, it also has some limitations, such as constraints in large model reasoning capabilities, reasoning speed, and limited support for VS Code functionalities. The research team is continuously optimizing the model training and instruction parsing algorithms to reduce the frequency of errors and improve operational accuracy.
In the future, the research team will address these issues through regular updates and iterations, ensuring that SmartVscode becomes a powerful intelligent assistant for VS Code developers.
App-Controller open-source address:
https://github.com/alibaba/app-controller
smart-vscode-extension open-source address:
https://github.com/alibaba/smart-vscode-extension