How to work with various AI directly in VSCode: AIConfig Editor

How to work with various AI directly in VSCode: AIConfig Editor

Review

AIConfig Editor turns VS Code into a generative development environment that allows you to run models from any provider (OpenAI, Google, Hugging Face, from your local computer …) or any modality (text, image, audio).

Model prompts and settings are stored in a file .aiconfig.yaml or in .aiconfig.jsonwhose source code can be controlled and used in your application via the AIConfig SDK.

Please refer to the full documentation for details

EditorUI

Demo video

How to start?

First launch

  1. Install the AIConfig Editor extension.

  2. Run the command AIConfig: Welcome (CMD+SHIFT+PAIConfig: Welcome) and follow the instructions.

Welcome

  1. For inspiration, visit our templates

Further launches

Key features

  • Access local and remote models in one place. Get access to all generative AI models in one place, right in your IDE. Text, graphics and audio models are supported out of the box.

  • Universal prompt engineering sandbox. Switch between models, combine clues and create patterns. Use these hints in code via the AIConfig SDK.

  • Tooltip version control. Manage hints and model settings in configuration files that can be placed in version control right next to the code.

  • Connect to your own models and endpoints. Extend AIConfig to work with any model and endpoint. For more information, see in the custom models section.

Supported models

This extension supports all major models of major providers. For details, please see https://aiconfig.lastmileai.dev/docs/overview/model-parsers/.

Provider

Model

Language

Support

OpenAI

GPT3.5

Python

✅ Built-in

OpenAI

GPT4

Python

✅ Built-in

OpenAI

Dall-E 3

Python

✅ Built-in

Azure OpenAI

GPT3.5, GPT4

Python

✅ Built-in

AWS Bedrock

Claude

Python

✅ Built-in

HuggingFace output endpoints

Text generation, text to image, text to speech, generalization, translation, automatic speech recognition

Python

✅ Built-in

Google

PALM 2

Python

✅ Built-in

Google

Gemini

Python

✅ Built-in

meta* [организация признана экстремистской и запрещена в России]

Llama 2

Python

🤝 Extension

Meta

Llama Guard

Python

🤝 Extension

HuggingFace transformer pipelines

Text generation

Python

🤝 Extension

How it works

AIConfig Editor is the user interface for AIConfig. The latter is a JSON/YAML schema for storing generative AI prompts, as well as storing models and their settings as a configuration file.

For example, check out this aiconfig that handles function calling and hint chaining.

  • Installs the pip package when the extension is installed python-aiconfig into your Python environment.

  • When you open a file *.aiconfig.yaml in VS Code, the extension starts a Python server running the AIConfig SDK.

  • When you edit and run queries in the editor, the server uses the AConfig SDK to run those queries. This gives you more flexibility as you can install additional AIConfig extensions into the Python environment and use them in the editor.

Extensibility and Customizability

When you use the AIConfig editor, the extension installs python-aiconfig into your Python environment. Additional extensions and dependencies can be installed in the same Python environment and then used in the AIConfig editor.

Instructions

  1. Run it pip3 install <extension_package> in your Python environment (eg pip3 install aiconfig-extension-hugging-face for Hugging Face models through endpoints of transformers, diffusers, and logical conclusion models)

  2. Run the command AIConfig: Create Custom Model Registry (CMD+SHIFT+PAIConfig: Create Custom Model Registry).

ModelRegistry

  1. Register additional models from a package such as the one imported in step 1.

Copy the complete example

from aiconfig_extension_hugging_face import (
    HuggingFaceText2ImageDiffusor,
    HuggingFaceTextGenerationTransformer,
    HuggingFaceTextSummarizationTransformer,
)

from aiconfig import AIConfigRuntime


def register_model_parsers() -> None:
    """Register model parsers for HuggingFace models."""

    text_to_image = HuggingFaceText2ImageDiffusor()
    AIConfigRuntime.register_model_parser(text_to_image, text_to_image.id())

    text_generation = HuggingFaceTextGenerationTransformer()
    AIConfigRuntime.register_model_parser(
        text_generation, text_generation.id()
    )
    text_summarization = HuggingFaceTextSummarizationTransformer()
    AIConfigRuntime.register_model_parser(
        text_summarization, text_summarization.id()
    )

HuggingFace

  1. Open it *.aiconfig.yaml (example, CMD+SHIFT+PAIConfig: Create New), and now you can use your own extension in the editor! To define your own extensions, please see the AI ​​Config Editor extensibility documentation.

Related posts