-
Notifications
You must be signed in to change notification settings - Fork 942
Description
Summary
When a custom tool returns large output as a single JSON line (e.g., 55KB), the SDK's large-output handling creates an unrecoverable loop where the model can never access the actual content.
Reproduction
- Register a custom tool (e.g.,
KnowledgeSearch) that returns ~55KB of JSON as a single line (common for search/retrieval APIs) - The model calls the tool
- SDK writes the result to a temp file and returns to the model:
Output too large to read at once (55.3 KB). Saved to: C:\Users\...\copilot-tool-output-xxx.txt Consider using tools like grep (for searching), head/tail (for viewing start/end), view with view_range (for specific sections), or jq (for JSON) to examine portions of this file. - Model calls
viewwithview_range=[1, 120]→ since the entire content is one line, this returns the whole 55KB line, which gets truncated again:[Output truncated. Use view_range=[1, ...] to continue reading.] - Model calls
grepwith keyword patterns → every match returns the same giant single line, which also gets redirected to yet another temp file:Output too large to read at once (55.4 KB). Saved to: ...copilot-tool-output-yyy.txt - Model never sees the actual content. It burns 5-8 extra LLM turns trying different
view/grepcombinations, all hitting the same truncation wall.
What the model actually sees (from LLM request dumps)
| Tool Response | Content Length | What model receives |
|---|---|---|
| KnowledgeSearch (custom) | 845 chars | File path + "use grep/view" message |
view (lines 1-120) |
66 chars | [Output truncated. Use view_range=[1, ...] to continue reading.] |
grep (keyword search) |
445 chars | Another file path + "use grep/view" message |
view (of grep output) |
66 chars | [Output truncated...] again |
The model operates essentially blind — it never sees the tool results.
Root Cause
The SDK uses line-based chunking for view (view_range operates on line numbers). When the entire output is a single line, view_range=[1, N] always returns the full content regardless of N, which always exceeds the truncation threshold.
Suggested Fix
- Pretty-print JSON tool output before writing to the temp file (so
view_rangeworks on individual lines) - Or use character-based chunking as fallback when the file has very few lines
- Or allow
viewto accept a character/byte range for single-line files
Environment
github-copilot-sdkversions tested: 0.1.23, 0.1.30- Model: gpt-5.2 (gpt-52-2025-12-11)
- Platform: Windows 11
- Provider: OpenAI (via BYOK)
Impact
Custom tools that return large JSON responses (search APIs, knowledge retrieval, connector actions) are effectively unusable. The model wastes 5-8 extra LLM turns and ultimately answers without having seen the actual data. This is a blocker for agentic workflows that depend on tool-grounded responses.