Analyzing CSV and Spreadsheet Data

Ma Mahalakshmi V Updated 16 Sep 2026
6 min read ·Lesson 89 of 224

From "File Uploaded" to "Correct Analysis"

Lesson 3 covered getting a file into the sandbox. This lesson covers the part that actually determines whether the analysis is useful: how to phrase requests so the model writes correct, well-scoped pandas code against that file, how to handle the quirks specific to CSVs and Excel workbooks, and how to structure a conversation that goes from raw data to real insight without wasting turns.

A Baseline Example

Start with a small, realistic dataset and a direct request:

from openai import OpenAI

client = OpenAI()

uploaded = client.files.create(file=open("orders.csv", "rb"), purpose="assistants")

response = client.responses.create(
    model="gpt-5.6-terra",
    tools=[{
        "type": "code_interpreter",
        "container": {"type": "auto", "file_ids": [uploaded.id]},
    }],
    input=(
        "The file orders.csv has columns order_id, customer_id, order_date, "
        "amount, and region. Load it with pandas, confirm the row count and "
        "column dtypes, and report total amount by region sorted descending."
    ),
)

print(response.output_text)

Notice that the prompt does three specific things beyond just naming the file: it lists the expected columns, it asks for a sanity check (row count and dtypes) before the actual analysis, and it states the exact aggregation wanted (sum of amount, grouped by region, sorted descending). Each of these reduces ambiguity that would otherwise force the model to guess. A vague prompt like "analyze this file" produces a much less predictable result — the model has to decide on its own what "analyze" means, and different runs can emphasize different things.

Why Describing the Schema Helps

You might reasonably ask why you should describe columns the model can simply inspect by reading the file's header. It can, and typically will, run something like pd.read_csv(...).head() early in its own generated code to look. But stating the schema up front does two things a purely exploratory approach does not:

  1. It lets the model write the entire analysis in a single, correct execution rather than needing an exploratory step followed by a corrective one, which saves both latency and tool-call overhead.
  2. It gives you, the developer, a place to catch a schema mismatch immediately. If your prompt says amount and the real file has order_amount, the model's own inspection step will surface that discrepancy in its response, and you'll see it — but if you already know your schema, stating it is cheap and removes the failure mode entirely for cases where you control the upstream data format.

Handling Data Quality Issues

Real datasets have missing values, inconsistent types, and outliers. Code interpreter handles this well when the prompt asks for it explicitly:

response = client.responses.create(
    model="gpt-5.6-terra",
    tools=[{
        "type": "code_interpreter",
        "container": {"type": "auto", "file_ids": [uploaded.id]},
    }],
    input=(
        "Load orders.csv. Report how many rows have a null value in any "
        "column, and how many rows have a negative or zero value in the "
        "amount column. Then compute total amount by region using only "
        "rows that pass both checks."
    ),
)

print(response.output_text)

This prompt separates diagnosis (how much bad data is there) from the actual computation (the cleaned aggregate), and asks for both. This is a meaningfully better pattern than just asking for "the total by region," because a silent filtering decision made by the model without your knowledge is exactly the kind of thing that erodes trust in an automated analysis pipeline — you want to see, in the output, what was excluded and why.

Working with Excel Files and Multiple Sheets

Excel workbooks introduce a wrinkle CSVs do not have: a single file can contain multiple sheets, and pandas needs to be told which one to read.

uploaded = client.files.create(file=open("regional_report.xlsx", "rb"), purpose="assistants")

response = client.responses.create(
    model="gpt-5.6-terra",
    tools=[{
        "type": "code_interpreter",
        "container": {"type": "auto", "file_ids": [uploaded.id]},
    }],
    input=(
        "regional_report.xlsx has multiple sheets. First list all sheet "
        "names in the workbook. Then load the sheet named 'Q1_Sales' and "
        "report the top 5 products by units sold."
    ),
)

print(response.output_text)

Asking for the sheet names first, in the same prompt, is a small but effective technique: it forces the model's generated code to call something equivalent to pd.ExcelFile(path).sheet_names before assuming a sheet exists, which avoids a common failure where the model guesses a sheet name (often something generic like "Sheet1") that does not match your actual workbook.

For workbooks where you already know the exact sheet name, skip the discovery step and just state it — one fewer round of exploration, one fewer chance for the model to load the wrong data.

Multi-Step Analysis Using Container Reuse

For any analysis with more than one logical question, use the container-reuse pattern from Lesson 2 rather than trying to cram every question into one prompt:

first = client.responses.create(
    model="gpt-5.6-terra",
    tools=[{
        "type": "code_interpreter",
        "container": {"type": "auto", "file_ids": [uploaded.id]},
    }],
    input="Load orders.csv into a dataframe called df and report its shape.",
)

second = client.responses.create(
    model="gpt-5.6-terra",
    tools=[{"type": "code_interpreter", "container": {"type": "auto"}}],
    previous_response_id=first.id,
    input="Using df, compute average order amount per month.",
)

third = client.responses.create(
    model="gpt-5.6-terra",
    tools=[{"type": "code_interpreter", "container": {"type": "auto"}}],
    previous_response_id=second.id,
    input="Now filter df to only the region with the highest average order amount from the previous step, and report its top 3 customers by total spend.",
)

print(third.output_text)

Each step builds on df as an already-loaded, already-validated dataframe, and the third question explicitly references "the previous step's result" in natural language, which the model can resolve because the conversation history — including the actual computed values — is available to it through the chained responses. This is both cheaper (the file is parsed once, not three times) and more coherent than three independent requests, because the model isn't re-deriving the same intermediate result each time and risking a slightly different answer due to a different code path.

Common Mistakes

Asking an open-ended question against a large, unfamiliar file. "Tell me what's interesting in this data" against a 50-column file forces the model to make numerous unstated judgment calls about what "interesting" means, and different runs will emphasize different columns. Ground the request in specific columns and specific statistics whenever you know what you are looking for.

Not accounting for header rows or metadata rows in exported spreadsheets. Files exported from business intelligence tools sometimes have a title row or a blank row before the real header. If you know this about your file format, say so in the prompt ("the real header is on row 3") rather than letting the model discover and potentially misparse it.

Re-uploading and re-describing the same file for every follow-up question. This wastes tokens and sandbox time, and — because it does not reuse the same loaded dataframe — can occasionally produce a subtly different reload (for example, a different automatic type inference) than the one the earlier answer was based on. Use container reuse instead, as shown above.

Best Practices

State column names and expected types explicitly when you know your data's schema. This removes ambiguity and gives you an early signal if the actual file does not match what you expected.

Separate data-quality diagnosis from the final computed answer in your prompt, so that filtering decisions are visible in the output rather than silently baked into a single aggregate number.

Chain multi-question analyses with previous_response_id rather than independent calls, so later questions can build on already-validated, already-loaded state instead of redoing the same work with a chance of subtly different results each time.

0 Comments

Reviewed before they appear

No comments yet.

OpenAI SDK
Introduction to the OpenAI SDK Setting Up Python Creating an API Key Your First Call — client.responses.create() and response.output_text Understanding Billing, Credits, and What a Request Costs Why Responses Replaced Chat Completions Anatomy of a Request: model, input, and instructions Anatomy of a Response: The Typed output Array, Not Just Text Roles: User, Assistant, and Developer/System Choosing a Model, and Reading the Models Page Instead of Memorizing Names Instructions vs. Input Writing Prompts That Get Consistent Results Few-Shot Examples Reasoning Models and the reasoning Parameter Debugging a Prompt That Misbehaves Why Streaming Matters for User Experience stream=True and Iterating Over Events Handling the Event Types You Actually Care About Background Mode for Long-Running Jobs Project — Add Live Streaming to Your Chatbot The Problem With Parsing Free Text JSON Schema and Strict Mode Pydantic Models With the SDK's Parse Helpers Handling Refusals and Validation Failures Project — A Resume-to-JSON Extractor Working With input_image input_file, PDFs, and the Files API Image Generation Speech-to-Text and Text-to-Speech Project: A PDF Question-Answering Script What Function Calling Is Defining a Tool Schema The Full Loop Multiple Tools Errors, Timeouts, and Untrusted Arguments Project: A Weather Assistant Web Search File Search and Vector Stores Code Interpreter Remote MCP Servers and Connectors Project: A Research Assistant What an Embedding Is, Without the Maths Generating and Storing Embeddings Similarity Search From Scratch Hosted Vector Stores vs. Rolling Your Own A Small RAG App Over a Folder of Notes Agents vs. a Single API Call — When You Need One pip install openai Giving Agents Tools Handoffs and Multi-Agent Triage Guardrails and Approvals Tracing and Observing What Your Agent Did A Multi-Agent Support Desk Error Codes and What Each One Means Retries, Timeouts, and Backoff Rate Limits and Spend Limits Prompt Caching and Cost Optimisation The Batch API for Bulk Work Async Clients and Concurrency Moderation and Safety Best Practices Designing the App Backend With FastAPI Streaming to a Simple Frontend Deploying and a Cost/Safety Checklist Why Web Search Is Useful for Current Information Using the Web Search Tool with the Responses API Configuring Search Behavior for Application Use Cases Understanding Citations and Source Attribution Where to Go Next Building a Research Assistant with Web Search Combining Web Search with Structured Outputs Handling Conflicting or Low-Quality Web Sources Reducing Unsupported Claims with Grounded Generation Testing Freshness-Sensitive AI Answers Production Considerations for Web-Grounded Applications Understanding File Search and Retrieval-Augmented Generation Creating and Organizing Vector Stores Uploading Documents for Retrieval Connecting Vector Stores to Responses API Requests Designing Document Metadata and Filtering Strategies Building a PDF Question-Answering Application Improving Retrieval Quality With Better Document Preparation Handling Missing Evidence and Retrieval Failures Combining File Search With Web Search Building a Production Knowledge-Base Assistant What the Code Interpreter Tool Is Designed For Running Python-Based Analysis Through the OpenAI SDK Uploading Datasets for Analysis Analyzing CSV and Spreadsheet Data Generating Charts and Data Summaries Handling Generated Files and Downloadable Artifacts Building a Data-Analysis Assistant Combining Code Execution with Structured Outputs Validating Generated Calculations and Results Security and Sandbox Considerations for Code Execution Understanding Multimodal Input with the OpenAI SDK Sending Images to a Model Image Analysis from URLs and Uploaded Files Extracting Text and Information from Screenshots Building an Image-Question-Answering Application Combining Image Input with Structured Output Analyzing Multiple Images in One Request Handling Image Quality and Input Limitations Designing Multimodal Prompts for Reliable Results Building a Practical Vision-Powered Python Application Understanding Speech-to-Text and Text-to-Speech Workflows Transcribing Audio with the OpenAI SDK Working with Uploaded Audio Files Handling Timestamps and Transcription Metadata Building a Meeting Transcription Workflow Generating Spoken Responses from Text Handling Long Audio and Processing Failures Combining Audio with Text and Tool Calling Building an End-to-End Python Voice Application What Embeddings Are and When to Use Them Generating Embeddings With the OpenAI API Preparing Text for Embedding Comparing Vectors With Cosine Similarity Building a Simple Semantic Search Engine in Python Storing Embeddings in a Database Metadata Filtering for Semantic Search Chunking Strategies for Better Retrieval Evaluating Semantic Search Quality Building a Document Similarity Application When Batch Processing Makes Sense Designing Large-Volume AI Processing Pipelines Using Asynchronous Python with the OpenAI SDK Running Concurrent Requests Safely Controlling Concurrency and Avoiding Rate Limits Tracking Batch Job Progress Handling Partial Failures in Bulk Workloads Retrying Failed Items Without Duplicating Successful Work Designing Resumable AI Processing Jobs Building a Production Batch-Processing Pipeline Batch Processing Makes Sense Large-Scale AI Processing Pipelines Async Python with OpenAI SDK Safe Concurrent Requests Concurrency & Rate Limits Batch Progress Tracking Partial Failure Handling Safe Retry Handling Resumable AI Jobs Production Batch Pipeline System–User Data Separation Reusable App Instructions Prompt Templates & Variables Extraction & Classification Prompts Summarization & Transformation Prompts Explicit Output Requirements Prompt Version Management Prompt Testing & Evaluation Reusable Python Prompt Library API Key Security Secure API Key Storage Secure Secret Management Prompt Injection Prevention Trusted vs. Untrusted Content Tool Argument Validation Sensitive Data Handling Secure Logging AI Action Authorization Production AI Security Checklist Why AI Applications Need Evaluation Beyond Unit Tests Unit Testing OpenAI SDK Integration Code Mocking API Responses in Python Tests Testing Structured Outputs Against Schemas Testing Tool-Calling Workflows Building a Small Evaluation Dataset Measuring Accuracy, Consistency, and Failure Rates Regression Testing Prompts and Model Changes Human Evaluation Versus Automated Evaluation Creating a Repeatable Evaluation Pipeline AI Request Monitoring Token Cost Management Usage Metrics Design Reducing Model Calls Prompt & Context Optimization Model Selection & Optimization AI Caching Strategies Interactive Latency Optimization Usage Dashboards & Budget Alerts Performance & Cost Checklist Every API Call Starts Fresh Fixing API Statelessness Server-Side Conversation Memory Limits of Response Chaining What We're Building Conversation Memory Challenges Preparing an OpenAI SDK Application for Deployment Environment-Specific Configuration for Development and Production Deploying a Python AI Service with Docker Container Health Checks and Startup Configuration Managing Secrets in Cloud Deployments Background Workers for Long-Running AI Tasks Queues and Asynchronous Job Architectures Scaling AI Workloads Horizontally Monitoring Production Incidents and Failures Production Deployment Checklist for OpenAI SDK Applications Reusable OpenAI Service Classes AI Client Dependency Injection Typed AI Responses Python Configuration Management AI Request Decorators Centralized AI Error Handling Clean SDK Abstractions Reusable OpenAI Utilities Internal AI Python Libraries SDK Integration Maintenance Production AI Chatbot Document Q&A System Web Research Assistant Customer Support Agent AI Data Analysis Assistant Image Analysis App Meeting Transcription & Summary Semantic Document Search Multi-Tool AI Agent Production OpenAI SDK App Why "It Looked Fine When I Tested It" Isn't Enough Timing Note Status Note Pre-Decision Status Note Current Availability Note
Ask about this post
AI Ask about this post

Ask questions about Analyzing CSV and Spreadsheet Data and get answers drawn from it.

Signed-in readers only.