Skip to main content

Your first ITool call

This page walks the simplest possible end-to-end flow: open the chat panel, ask the agent for the Editor state, and watch a single ITool execute. If this works, your install is healthy.

Open the chat panel​

Window → Buril → Chat

You should see a panel docked next to the Inspector with an empty conversation, a model selector at the top, and an input box at the bottom.

Say hello​

Type:

What scene is open?

Press Enter.

Behind the scenes the agent does three things:

  1. Plans. The LLM sees your prompt and the tool catalog. It picks editor_state as the right tool to answer "what scene is open".
  2. Calls. Buril dispatches editor_state against the live Unity process. The tool reads EditorSceneManager.GetActiveScene() and returns the path + dirty flag + GameObject count.
  3. Replies. The LLM gets the tool result and writes a natural-language summary back to you.

The full transcript is visible in the chat panel — you can click any [tool: editor_state] row to see the raw JSON request and response. This is the contract — the chat UX is just a wrapper.

What just happened​

Three layers were involved:

You ──prompt──▶ LLM provider ──tool call JSON──▶ bridge MCP server ──C# dispatch──▶ ITool.Run() ──▶ Unity API
                                                                                                       │
You ◀──reply──── LLM provider ◀──tool result JSON───────────────────────────────────────────────────────┘
  • The bridge MCP server runs on http://127.0.0.1:8766/rpc (HTTP) and tcp://127.0.0.1:8765 (raw socket). Both are loopback-only by default.
  • The LLM provider is whichever one you wired in BYOK providers. Buril never proxies through a Buril-owned server in Community edition.
  • The ITool is a 30-line C# class that implements ITool.Run(JObject input) → object. Auto-discovered via reflection on Editor reload.

Try a few more​

Each of these calls a single tool. Watch the tool name in the transcript:

PromptTool called
list 5 GameObjects in the active scenescene_list_gameobjects
import this fbx into Assets/Modelsasset_import_fbx (after you drag-and-drop or paste a path)
create an empty GameObject called Playergameobject_create
show me the console errorsconsole_get_logs
open the Lighting windoweditor_window_open

These are five tools out of 3,691. The full catalog lives at Tools/CATALOG.md.

Try a scenario template​

Atomic tools are the building blocks. Most users want bigger results. Try:

Build a small racetrack with lakes.

The agent picks the right pair of scenario templates and chains them in one turn:

  1. scene_template_racetrack — generates a loop track with kerbs, start/finish line, and pit lane.
  2. scene_template_terrain_with_lakes — adds a procedural terrain underneath with a couple of lakes, applies a sky preset, places a directional light.

Each tool returns a structured envelope with next_steps (e.g. "place a player vehicle prefab at the start line", "add Cinemachine for race cam"). The agent reads those hints and stops when the task is functionally complete — you should see a playable scene in the Game view within ~15 seconds.

Inspect the result: the new GameObjects appear in the Hierarchy under a racetrack root. The Scene view shows the track. Press Play and you have a drivable surface (well — flat ground plus a track shape; physics + vehicle wiring is the next batch of tools).

Try the Reference-to-Scene Pipeline​

Reference → Scene is Buril's flagship 1.0 demo. Paste a reference URL or a screenshot path:

Build a scene that looks like https://www.youtube.com/watch?v=<some-clip>

The agent calls build_scene_from_reference (one ITool, full pipeline):

  1. Extract — pulls a Reference Profile (palette, sky type, terrain density, hero objects).
  2. Plan — picks a scenario template that matches the profile.
  3. Whitebox — generates the rough geometry.
  4. Greybox — applies the palette + skybox.
  5. Vision feedback — captures a SceneView screenshot, sends it to the model, the model writes a follow-up turn with fixes if anything looks off (magenta materials, wrong proportions, missing hero).

Same flow accepts: a screenshot path on disk, an asset library URL (Quixel / TurboSquid / FAB), an AI-gen image URL, or a style_guide:<name> reference. See the Reference-to-Scene Pipeline docs for the full mode list.

Multi-step plans​

The chat panel is not limited to one tool per message. Try:

Make a cube at the origin, set its color to red, and save the scene as RedCube.unity in Assets/Scenes.

The agent chains:

  1. gameobject_create_primitive (cube)
  2. gameobject_set_position
  3. material_create + gameobject_assign_material
  4. scene_save_as

You see every step in the transcript and can interrupt at any point. The Stop button cancels the in-flight tool call cleanly.