Symptom · ChatGPT

ChatGPT will not stick to the format you asked for

You asked for JSON and got JSON wrapped in an apology and a markdown fence.

What is actually happening

Format instructions are one signal among many, and prose is the strongest prior the model has. Without a structural constraint, the pull back to prose wins.

Everything below comes from published research, not from opinion. Each fix names the technique, states the effect its authors measured where one was reported, and links to the paper so you can check it. These were selected out of 6,235 techniques indexed from the literature.

4 fixes that hold up

01

Format Trap

Structured Output Collapses Answer Diversity Across 44 Language Models, arXiv:2607.18476 (2026)

Research shows that requesting responses in structured formats (JSON, XML) switches the model into predictability mode, sharply narrowing the diversity of answers. To preserve creativity, it is recommended to separate idea generation and formatting into two separate stages, first getting free-form answers, then structuring them.

One word 'JSON' in the prompt — and the model sharply narrows its choice, as if it were switched. Ask about any tree in a normal chat — you'll hear sequoia, baobab, yew. Ask for the same thing in JSON — almost always 'oak'. The method allows you to understand in advance when response format kills diversity of ideas — and not step on these rakes during brainstorming or option selection. The model perceives the word JSON as the command 'behave as if for a program' — and switches to a predictable, averaged mode, even without a strict schema.

Use it whenUse this diagnostic method when you need maximum diversity of ideas and creativity from the model — during brainstorms, generating name options, creating recommendations, or selecting alternatives. If a task requires a wide range of answers, avoid requesting structured formats (JSON/XML) during the idea generation stage. Instead, first get diverse options in free form, then use a separate request to convert the result into the required format. This is especially critical for creative
Prompt
# ROLE
You are an AI assistant that helps users avoid common pitfalls in prompt engineering, specifically the "Format Trap" where requesting JSON output can lead to less diverse and more generic responses.

# CONTEXT
When users request structured output like JSON, LLMs tend to switch into a "programmatic mode." This mode prioritizes predictable, "safe" answers over creative or diverse ones, as seen in research where asking for trees in JSON format almost always results in "oak," whereas a free-form request yields a wider variety. This is not a technical limitation of the format itself, but a behavioral shift in the model due to its training on tool-use and API interactions.

# TASK
The user frequently encounters a problem: asking ChatGPT or similar models for JSON output results in the JSON being wrapped in an apology, markdown fences, or other extraneous text, and the quality/diversity of the JSON content itself is often reduced.

Your goal is to provide a reusable prompt that the user can paste to:
1.  Generate creative or diverse content (like ideas, names, recommendations).
2.  Ensure the output is *only* the requested JSON, without extra conversational text or markdown.
3.  Avoid the "Format Trap" by first generating content in a free-form text format, and then converting it to JSON in a separate, subsequent step.

# METHODOLOGY (Format Trap Avoidance)

Follow these two distinct steps:

## Step 1: Generate Content in Free-Form Text

**Instruction:**
Generate [NUMBER] [TYPE OF CONTENT] for [SPECIFIC TASK/TOPIC].
**Constraints:**
- Do NOT use JSON or any other structured format.
- Output each item on a new line.
- Be as creative and diverse as possible.
- Avoid generic or cliché responses.

**Example Application (for user's task):**
"Generate 20 brand name ideas for a new streetwear clothing brand.
Constraints:
- Do NOT use JSON or any other structured format.
- Output each name on a new line.
- Be as creative and diverse as possible.
- Avoid generic or cliché responses like 'Urban', 'Street', 'Nomad'."

## Step 2: Convert to JSON (Separate Prompt)

**Instruction:**
Take the following list of [TYPE OF CONTENT] and format it as a JSON object.
**JSON Structure:**
[SPECIFY THE DESIRED JSON STRUCTURE, e.g., `{"names": ["item1", "item2", ...]}` or `{"ideas": [{"id": 1, "name": "idea1"}, ...]}`]
**Constraints:**
- The output MUST be ONLY the JSON object.
- No introductory text, apologies, or markdown fences.

**Example Application (for user's task):**
"Take the following list of streetwear brand names and format it as a JSON object.
JSON Structure: `{"names": ["name1", "name2", ...]}`
Constraints:
- The output MUST be ONLY the JSON object.
- No introductory text, apologies, or markdown fences."

# OUTPUT INSTRUCTIONS
Present the above two-step process clearly to the user, explaining why it works based on the "Format Trap" research. Provide the exact prompt templates they can copy and paste for each step.
source paper →
02

In-Writing

Thinking Before Constraining: A Unified Decoding Framework for Large Language Models, arXiv:2601.07525 (2026)

Deferred structuring technique: the model first reasons freely, then automatically switches to constrained decoding when encountering a trigger token. Allows obtaining deep reasoning and guaranteed valid JSON in a single request without additional parsers. Uses a finite automaton to mask invalid tokens after the trigger.

The model cannot do two things at once — reason deeply AND output strict JSON. If you demand structure from the first token, the model rushes to the format, skipping steps. If you give freedom, reasoning exists but the output is unparseable ('Answer A' vs 'I think A' vs 'Maybe A is better'). In-Writing allows you to get both full reasoning and guaranteed valid JSON in a single request — without a second parser model and regex workarounds. The model works in two states: first free generation (state -1), then it encounters a trigger token (for example {) — it automatically switches to constrained decoding (state 0).

Use it whenUse In-Writing when you need to obtain both high-quality model reasoning and guaranteed valid structured output (JSON/XML) simultaneously. Especially useful for tasks requiring deep analysis followed by result structuring: data extraction from text, classification with justification, code generation with explanations. The technique solves the problem of choosing between free generation (quality reasoning, but unpredictable format) and strict formatting (valid JSON, but limited)
Prompt
You are an AI assistant that excels at generating structured, valid JSON output. Your primary function is to provide a detailed reasoning process followed by a JSON object containing the final output.

**Task:** Generate digital product ideas for visitors after their visit to a 7-room immersive experience.

**Context:**
- **Existing Products:** Welcome card, app with extended content, interactive book.
- **Success Criteria:**
    - Easy for users to engage with.
    - Builds interest in the subject matter.
    - Naturally leads to paid courses.

**Instructions:**
1.  **Reasoning Phase (Free Generation):** First, engage in a detailed, free-form reasoning process to explore potential product ideas. Consider visitor psychology, engagement mechanics, and the criteria provided. Think step-by-step about how to meet the user's needs without adhering to a strict format initially.
2.  **Trigger:** Once you have a solid understanding and have developed a set of ideas, begin your output with the JSON structure. The opening of the JSON structure (`{`) will act as a trigger for the model to switch to constrained decoding.
3.  **Constrained JSON Output:** After the trigger, generate a JSON object that strictly adheres to the following schema. You must not include any text before the JSON object begins, nor any apologies, explanations, or markdown fences around it. The JSON schema is designed to ensure validity and prevent common issues.

**JSON Schema:**
source paper →
03

Prompting Strategy Matters More Than Model Size

LLMStructBench: Benchmarking Large Language Model Structured Data Extraction, arXiv:2602.14743 (2026)

Two prompting strategies for reliable JSON extraction from text. P-strategy includes schema and example in the prompt, ensuring high data accuracy. PJ+-strategy adds schema transfer via API, guaranteeing valid JSON structure. Both work in regular chat and outperform simple requests.

LLMs break JSON when extracting data from text. When you ask them to pull out information, you get either invalid syntax with broken brackets and extra text, or valid JSON with incorrect values, missing fields, and distorted information. Small models are especially unstable, sometimes working and sometimes failing on identical tasks. Two strategies solve the problem: P, which uses schema and examples in the prompt, and PJ+, which combines that with passing the schema through API. Both work in regular chat. The choice depends on your priority: if you need data accuracy use P, if you need guaranteed valid structure use PJ+. On good models both work perfectly, on weak models you'll have to choose a compromise.

Use it whenUse this technique when extracting structured data from unstructured text in JSON format. Especially effective for processing applications, orders, customer messages, product descriptions and other texts with explicit information. Choose P-strategy when data accuracy is critical and rare syntax errors are acceptable. Apply PJ+-strategy when valid JSON structure guarantee is necessary, even if semantic inaccuracies are possible. On weak models you'll have to choose between
Prompt
# ROLE
You are an expert data extraction assistant specializing in converting unstructured text into structured JSON format.

# CONTEXT
The user frequently encounters a problem where LLMs provide JSON output that is either syntactically invalid (e.g., broken brackets, extra text) or contains incorrect/missing values, often wrapped in apologies or markdown fences. This is particularly problematic for automated parsing pipelines that require perfectly formatted JSON.

# TASK: JSON Extraction Strategy (P-Strategy)

Your goal is to generate a reusable prompt template based on the "P-strategy" (Prompt-only) described in the research. This strategy prioritizes data accuracy by providing a clear JSON schema and a detailed example, leveraging the LLM's pattern-matching capabilities.

## P-Strategy Explanation:
The P-strategy involves providing the LLM with:
1.  **An explicit instruction** to extract information into JSON.
2.  **A JSON schema** defining the expected fields and their data types.
3.  **A fully populated example object** that adheres to the schema and uses realistic, distinct data.

This approach helps the LLM understand both the required structure and the expected content, leading to more accurate and usable JSON output. While it might occasionally produce structurally flawed JSON on weaker models, it generally excels at data accuracy.

## Reusable Prompt Template:
source paper →
04

Prompt structure paradox

It's Not the Capability: Harness Sensitivity Is Non-Monotone Across LLM Agent Tiers, arXiv:2605.26731 (2026)

The method shows that overloading a prompt with instructions (steps, criteria, format requirements) worsens the performance of chat models—they start explaining the process instead of providing results. For reasoning models, the opposite principle works—they need maximum structure. A diagnostic matrix helps select the correct prompt style for the model type and task.

Added steps, criteria, and format requirements — all for precise JSON. Received a detailed report on how the model constructed this JSON. This is neither a bug nor carelessness — research showed that GPT-4o and Claude systematically degrade under prompt overload with structure. The method provides simple diagnostics: which style is needed for a specific model and task. The key insight: reasoning models (o1, Claude extended thinking) require the exact opposite — more stages and clear success criteria. The same 'improved' instruction is an upgrade for o1 and _degradation_ for GPT-4o.

Use it whenUse this technique when you receive from a model a different format than you requested—instead of JSON or a table, the model outputs explanations of the process. Particularly relevant when working with chat models (GPT-4o, Claude) when you need clean structured output. Apply when creating automated systems where output format predictability is important. For reasoning models (o1, Claude extended thinking) do the opposite—add more structure and criteria. Helps diagnose what prompt style you need for
Prompt
# Задача: Получить чистый JSON без лишних объяснений

# Контекст
Я работаю с ChatGPT (GPT-4o) и часто сталкиваюсь с проблемой: когда я прошу модель вернуть данные в формате JSON, она добавляет к ответу извинения, объяснения о том, как она составляла JSON, и оборачивает всё это в markdown-блок. Это мешает парсингу и автоматической обработке.

# Цель
Создать универсальный промпт, который заставит ChatGPT (GPT-4o) возвращать ТОЛЬКО чистый JSON, без каких-либо дополнительных текстовых обёрток, пояснений или markdown-форматирования.

# Инструкция для модели

Вот данные:
source paper →

What does not work

Repeating the instruction louder. Capitals, "IMPORTANT", and three exclamation marks change nothing structural. The rule still sits in the same place, competing with the same context.

Politeness and threats. Both have been measured repeatedly across 2025 and 2026 and come out indistinguishable from noise.

Turning the temperature to zero. It reduces variation, not misunderstanding. If your request has two valid readings, you now get the wrong one reliably.

Get this fixed for your actual task

The four prompts above are written for the average case. Paste what you are actually trying to do and the corpus will be matched against it directly. Free, no account, about ten seconds.

Free · no signup · ~10s
0.00match confidence
single retrieval pass
Prompt for your task


      

That number is low on purpose, and it is real. It is the raw similarity of one retrieval pass. No specialist read the paper, no judge compared anything against anything, and the first plausible match won. It is the honest score of a ten-second answer.

197techniques in the corpus
address this exact symptom

You have seen 4 of them on this page and one more just now. Deciding which of the remaining 197 actually fits your case is the work, and it is the work GetDecision does.

This page4 techniques for the average case, generic prompts
What you just ranone technique matched to your wording, nothing verified
Full runten specialists read the papers in full, a judge ranks the top three for your task and shows its reasoning, generation on the model you pick, saved to your history

See the top three for your taskTen specialists read the full papers, a judge ranks them and shows its reasoning. Free account, first run included.

Run the full analysis

Questions

Why does ChatGPT stop following instructions I already gave?

Format instructions are one signal among many, and prose is the strongest prior the model has. Without a structural constraint, the pull back to prose wins.

Does lowering the temperature fix this?

It reduces variation, not misreading. If the request admits more than one valid interpretation, a colder model just picks the same wrong one more consistently.

Do these techniques work on reasoning models?

Some do and some do not. Each fix below carries the effect its authors measured and a link to the paper, so you can check what it was measured on.