AI generated

A reproducible, end-to-end tutorial for using TensorSharp as the local inference backend for Pi Coding Agent. Pi remains responsible for the agent loop, conversation state, and tools. TensorSharp serves the model through its OpenAI-compatible API. No Pi source-code modifications are required. This walkthrough is based on the checked-in example at examples/pi-coding-agent. It covers both the Pi CLI and the Pi SDK, including the exact server configuration, provider setup, commands, and an end-to-end test. # What you will build * Pi Coding Agent running locally against TensorSharp. * Qwen3.8-27B-UD-IQ3_XXS served by TensorSharp on http://127.0.0.1:5000/v1. * DFlash2 speculative decoding enabled by the TensorSharp preset. * A dedicated Pi agent profile, isolated from your normal Pi configuration. * Both an interactive CLI workflow and a Node.js SDK example. * An E2E test that checks whether the agent can fix a small project without changing its tests. The validated setup described below used an Apple Silicon Mac. The measured E2E run was performed on an M5 Pro MacBook Pro with 48 GB unified memory. Hardware, model output, and timings will vary. # 1. Prerequisites Install: * macOS on Apple Silicon for the validated Metal configuration. * .NET 10 SDK. * CMake 3.20 or later. * Xcode Command Line Tools. * Node.js 22.19 or later. * Pi Coding Agent 1.5.1. Install the Xcode Command Line Tools if they are not already installed: xcode-select --install Install the Pi package used by this example: npm install -g @earendil-works/[email protected] pi --version Use @earendil-works/pi-coding-agent. The older @mariozechner/pi-coding-agent package is deprecated. Clone TensorSharp: git clone https://github.com/zhongkaifu/TensorSharp.git cd TensorSharp # 2. Build TensorSharp Build the server host: dotnet build TensorSharp.Server.Host -c Release -p:TensorSharpSkipMlxNative=true The example's server launcher uses the built host and the checked-in Qwen3.8 preset. It also handles downloading the pinned model artifacts and checking their SHA-256 hashes. # 3. Start the inference server From the TensorSharp repository root: examples/pi-coding-agent/scripts/start-server.sh The launcher starts TensorSharp with the model preset and the Pi-specific overlay: dotnet TensorSharp.Server.Host/bin/Release/net10.0/TensorSharp.Server.Host.dll \ --config config/qwen3.8-27b-metal-dflash2.json \ --config examples/pi-coding-agent/tensorsharp-pi.json The launcher may use dotnet run as a fallback if the built host is not present at the expected path. By default, model files are stored in the repository's models directory. To use a different location: export TENSORSHARP_MODELS="$HOME/work/models" examples/pi-coding-agent/scripts/start-server.sh The example downloads these pinned artifacts: |Artifact|Purpose| |:-|:-| |Qwen/Qwen3.8-27B-UD-IQ3_XXS.gguf|Main model| |Qwen/Qwen3.8-27B-DFlash2-Q4_K_M.gguf|DFlash2 draft model| The model preset configures the main model for Apple Metal and enables DFlash2 speculative decoding. The example's tested preset uses seven draft tokens per step and greedy decoding. # Pi-specific TensorSharp overlay The overlay file is examples/pi-coding-agent/tensorsharp-pi.json: { "host": "127.0.0.1", "port": 5000, "max-tokens": 16384, "mmproj": "none", "kv-cache-dtype": "q8_0", "environment": { "MAX_CONTEXT": "65536" } } This overlay is intentionally small. The model-specific preset supplies the model and backend settings; the overlay adjusts the server for this Pi workflow: * Bind to localhost. * Listen on port 5000. * Set the output-token limit to 16384. * Disable the vision projector for this text-only coding example. * Use Q8_0 KV cache. * Set the context limit to 65,536 tokens. The server is bound to localhost because this example does not configure authentication. Do not expose this endpoint to an untrusted network without adding appropriate access controls. # Verify the server In another terminal: curl -s http://127.0.0.1:5000/v1/models The returned model list should include this model ID: Qwen3.8-27B-UD-IQ3_XXS The model ID is important: the Pi provider configuration must use the exact ID returned by this endpoint. # 4. Configure Pi to use TensorSharp The example creates a dedicated Pi agent directory so that it does not overwrite or interfere with your normal ~/.pi/agent configuration. The example configuration files are: * pi-agent/models.json * pi-agent/settings.json # models.json { "$schema": "https://pi.dev/schemas/models.schema.json", "providers": { "tensorsharp": { "name": "TensorSharp (local)", "baseUrl": "http://127.0.0.1:5000/v1", "api": "openai-completions", "apiKey": "none", "compat": { "supportsDeveloperRole": true, "supportsReasoningEffort": false, "supportsStore": false, "maxTokensField": "max_completion_tokens", "thinkingFormat": "qwen-chat-template" }, "models": [ { "id": "Qwen3.8-27B-UD-IQ3_XXS", "name": "Qwen3.8 27B (TensorSharp, DFlash2)", "reasoning": true, "input": ["text"], "contextWindow": 65536, "maxTokens": 16384, "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 } } ] } } } A few details worth calling out: * baseUrl points to TensorSharp's OpenAI-compatible /v1 endpoint. * api is openai-completions. * The model id must match the ID reported by /v1/models. * maxTokensField tells Pi which completion-token field to send. * thinkingFormat: "qwen-chat-template" configures the Qwen thinking-format behavior, including the enable_thinking chat-template setting. * apiKey is a placeholder for this local server configuration; the example server does not validate it. # settings.json { "defaultProvider": "tensorsharp", "defaultModel": "Qwen3.8-27B-UD-IQ3_XXS", "defaultThinkingLevel": "medium", "enableInstallTelemetry": false } This selects TensorSharp and the Qwen3.8 model as the default for this dedicated Pi profile. # 5. Run Pi interactively From the TensorSharp repository root, open a project directory and launch the example wrapper: cd ~/src/my-project /path/to/TensorSharp/examples/pi-coding-agent/scripts/pi.sh For example, if you cloned TensorSharp into ~/work/TensorSharp: cd ~/src/my-project ~/work/TensorSharp/examples/pi-coding-agent/scripts/pi.sh The wrapper points Pi at the example's dedicated agent directory. It also sets offline/telemetry-related environment variables for this reproducible local workflow: PI_OFFLINE=1 PI_SKIP_VERSION_CHECK=1 PI_TELEMETRY=0 It does not overwrite your existing ~/.pi/agent settings. The example profile is kept separately under ~/.pi/tensorsharp-agent. # 6. Run Pi non-interactively You can also run a one-shot prompt: ~/work/TensorSharp/examples/pi-coding-agent/scripts/pi.sh \ -p "Explain what src/stats.js does" < /dev/null Redirecting stdin from /dev/null prevents the process from waiting for interactive input. Other useful examples: # Disable model thinking for this invocation ~/work/TensorSharp/examples/pi-coding-agent/scripts/pi.sh \ --thinking off \ -p "Summarize this project" < /dev/null # JSON output, without creating a session ~/work/TensorSharp/examples/pi-coding-agent/scripts/pi.sh \ --mode json \ --no-session \ -p "List the main modules" < /dev/null # Continue the previous session ~/work/TensorSharp/examples/pi-coding-agent/scripts/pi.sh --continue Check pi --help for the exact CLI options supported by your installed Pi version. # 7. Use TensorSharp from the Pi SDK The repository also includes a Node.js SDK example in examples/pi-coding-agent/sdk. The example uses Pi's public SDK APIs to create an agent session, select the TensorSharp model, send a task, and observe text and tool events. # Install the SDK example dependencies cd examples/pi-coding-agent/sdk npm install Copy the sample project to a temporary working directory: cp -R ../sample-project /tmp/stats-demo Run the example: PI_CODING_AGENT_DIR="$HOME/.pi/tensorsharp-agent" \ PI_OFFLINE=1 \ node run-task.mjs /tmp/stats-demo # Core SDK integration The following is the essential integration pattern used by the example. It shows how to load the provider configuration, resolve the TensorSharp model, create a Pi agent session, send a task, and dispose of the session. import path from "node:path"; import { createAgentSession, ModelRuntime, SessionManager } from "@earendil-works/pi-coding-agent"; const agentDir = process.env.PI_CODING_AGENT_DIR; if (!agentDir) { throw new Error("PI_CODING_AGENT_DIR must point to the Pi agent directory"); } const projectDir = path.resolve(process.argv[2] ?? "."); const modelRuntime = ModelRuntime.create({ modelsPath: path.join(agentDir, "models.json"), authPath: path.join(agentDir, "auth.json") }); const model = modelRuntime.getModel( "tensorsharp", "Qwen3.8-27B-UD-IQ3_XXS" ); const session = await createAgentSession({ cwd: projectDir, agentDir, modelRuntime, model, thinkingLevel: "medium", sessionManager: SessionManager.inMemory() }); try { session.subscribe((event) => { if (event.type === "message_update" && event.assistantMessageEvent?.type === "text_delta") { process.stdout.write(event.assistantMessageEvent.delta); } if (event.type === "tool_execution_start") { console.log(\n[tool start] ${event.toolName}); } if (event.type === "tool_execution_end") { console.log(\n[tool end] ${event.toolName}); } if (event.type === "message_end") { // The completed assistant message is available from the session. } }); await session.promp