Uploading Datasets for Analysis

Ma Mahalakshmi V Updated 16 Sep 2026
7 min read ·Lesson 88 of 224

Why Uploading Is a Separate Step

Code interpreter's sandbox has no network access (a design decision covered fully in Lesson 10), which means the model cannot "go fetch" a dataset from a URL, a database, or your internal systems on its own. Any data the sandbox is going to analyze has to arrive as a file that you explicitly hand to the API before or during the request. This is a deliberate boundary: it keeps the sandbox isolated and makes data flow explicit and auditable — you always know exactly which bytes were exposed to code execution, because you were the one who uploaded them.

This lesson covers that upload path in detail: the Files API, how an uploaded file gets attached to a code interpreter container, size and format constraints, and cleanup — none of which was covered in Unit 9's brief introduction to the tool.

Uploading a File

Uploading uses the SDK's files resource, independent of the Responses API call that will eventually use the file:

from openai import OpenAI

client = OpenAI()

uploaded_file = client.files.create(
    file=open("quarterly_sales.csv", "rb"),
    purpose="assistants",
)

print(uploaded_file.id)
print(uploaded_file.filename)
print(uploaded_file.bytes)

A few things about this call are worth understanding rather than memorizing:

  • The file is opened in binary mode ("rb"). This is required regardless of whether the underlying file is text (like a CSV) or binary (like an Excel workbook) — the upload endpoint transmits raw bytes, and opening in text mode on some platforms can silently corrupt line endings in a way that breaks downstream parsing.
  • purpose tells the platform what the file is for. Files intended for use with tools like code interpreter use a purpose value that marks them as tool-usable input rather than, for example, fine-tuning data.
  • The return value, uploaded_file, is not the data itself — it is a reference object. The actual bytes now live on OpenAI's storage, addressed by uploaded_file.id. Every subsequent step in this lesson works with that ID, not with the file's local path.

Note: The exact accepted values for the purpose parameter, and which ones are valid for code interpreter specifically, are the kind of platform detail that can change. Confirm the current accepted purpose values in the official Files API reference before hardcoding one into a production upload path.

Attaching an Uploaded File to a Code Interpreter Container

An uploaded file id, by itself, does nothing — it has to be attached to the container that will run the analysis. This happens through the container configuration on the code_interpreter tool:

response = client.responses.create(
    model="gpt-5.6-terra",
    tools=[{
        "type": "code_interpreter",
        "container": {"type": "auto", "file_ids": [uploaded_file.id]},
    }],
    input=(
        "A file named quarterly_sales.csv has been provided. "
        "Load it and report the column names and the number of rows."
    ),
)

print(response.output_text)

Two details matter here. First, the prompt explicitly names the file (quarterly_sales.csv) — inside the sandbox, the file is made available at a predictable path (commonly under a directory like /mnt/data/), and telling the model the filename in plain language helps it write correct code on the first try rather than guessing at a path. Second, file_ids is a list — you can attach multiple files to the same container in one call, which is exactly what you need for analyses spanning more than one dataset (for example, joining a customer file with an orders file).

customers = client.files.create(file=open("customers.csv", "rb"), purpose="assistants")
orders = client.files.create(file=open("orders.csv", "rb"), purpose="assistants")

response = client.responses.create(
    model="gpt-5.6-terra",
    tools=[{
        "type": "code_interpreter",
        "container": {"type": "auto", "file_ids": [customers.id, orders.id]},
    }],
    input=(
        "Two files are provided: customers.csv and orders.csv. "
        "Join them on customer_id and report total revenue per customer, "
        "sorted descending, top 10 rows only."
    ),
)

print(response.output_text)

This example is a realistic pattern for production data analysis features: rather than trying to describe your data model in prose, you upload the actual files and let the sandbox's pandas do the join, which is both more accurate and dramatically cheaper in prompt tokens than pasting large tables into the input text directly.

Format and Size Considerations

Code interpreter's sandbox comes with pandas, NumPy, and other common data libraries pre-installed, so it comfortably handles the formats those libraries read natively: CSV, TSV, JSON, Excel (.xlsx), and Parquet are all reasonable choices. Excel files with multiple sheets are supported but require the model to be told (or to discover) which sheet to load — this is covered further in Lesson 4.

Two practical constraints shape how you should think about file size:

  • Upload limits. The Files API enforces a maximum per-file size. Very large datasets should be pre-filtered or sampled on your side before upload rather than shipped in full, both to stay under the limit and because the sandbox itself has bounded memory and execution time.
  • Token cost of describing the data. Even though the file's bytes do not pass through the model's context window directly, every message you exchange about that data does. A workflow that repeatedly asks the model to "list every row" against a 500,000-row file will be slow and expensive regardless of the upload succeeding — steer the model toward aggregate operations (.describe(), .groupby(), filtered subsets) instead.

Note: The current maximum file size accepted by the Files API, and any additional constraints specific to files used with code interpreter, should be confirmed against the official documentation — these limits are adjusted over time.

Cleaning Up Uploaded Files

Uploaded files persist on your account's storage until you delete them or they expire, and they count toward your organization's storage. For a production application handling many user-uploaded datasets, deleting files you no longer need is a real operational concern, not an optional nicety:

def analyze_and_cleanup(file_path: str, question: str) -> str:
    client = OpenAI()
    uploaded = client.files.create(file=open(file_path, "rb"), purpose="assistants")
    try:
        response = client.responses.create(
            model="gpt-5.6-terra",
            tools=[{
                "type": "code_interpreter",
                "container": {"type": "auto", "file_ids": [uploaded.id]},
            }],
            input=question,
        )
        return response.output_text
    finally:
        client.files.delete(uploaded.id)

This function wraps the upload, analysis, and deletion into a single unit using a try/finally block. The finally clause guarantees the uploaded file is deleted whether the analysis call succeeds or raises an exception — an important detail, because a bare "delete after success" call would leak files on every request that happens to fail partway through, and those leaked files accumulate silently.

A Note on Sensitive Data

Because uploaded files are transmitted to and stored by OpenAI's infrastructure, and because the sandbox that reads them has no network egress but is still a shared execution environment, treat file uploads the same way you would treat any third-party data processor in your compliance posture. Avoid uploading raw files containing data you are not permitted to send to a third-party API — strip or mask personally identifiable information before upload where your data governance policy requires it. This connects directly to the broader sandbox security discussion in Lesson 10.

Common Mistakes

Opening the file in text mode ("r" instead of "rb"). This can appear to work for plain ASCII CSVs and then fail unpredictably on files with different encodings or line-ending conventions. Always open files for upload in binary mode.

Never deleting uploaded files. In a long-running application processing many user datasets, this silently grows storage usage and, more importantly, leaves data sitting on third-party storage longer than necessary. Build cleanup into the code path itself, not into a "someday" maintenance script.

Assuming the model automatically knows a file was uploaded. Attaching a file to the container makes it available in the sandbox filesystem, but the model still benefits enormously from being told the filename and a short description in the prompt — it removes guesswork and produces more accurate code on the first attempt.

Best Practices

Always pair upload with deletion in a try/finally or equivalent cleanup path, especially in server-side code handling many requests, so failures do not leak stored files.

Name files descriptively before upload (quarterly_sales_2026_q1.csv rather than data.csv) and mention that exact filename in your prompt. This small habit measurably reduces the model's chance of misidentifying columns or confusing one dataset with another when multiple files are attached.

Validate file contents before upload, not after. Checking that a CSV has the expected columns and is not empty in your own code, before spending an API call, catches malformed uploads cheaply instead of discovering them through a confusing model response.

0 Comments

Reviewed before they appear

No comments yet.

OpenAI SDK
Introduction to the OpenAI SDK Setting Up Python Creating an API Key Your First Call — client.responses.create() and response.output_text Understanding Billing, Credits, and What a Request Costs Why Responses Replaced Chat Completions Anatomy of a Request: model, input, and instructions Anatomy of a Response: The Typed output Array, Not Just Text Roles: User, Assistant, and Developer/System Choosing a Model, and Reading the Models Page Instead of Memorizing Names Instructions vs. Input Writing Prompts That Get Consistent Results Few-Shot Examples Reasoning Models and the reasoning Parameter Debugging a Prompt That Misbehaves Why Streaming Matters for User Experience stream=True and Iterating Over Events Handling the Event Types You Actually Care About Background Mode for Long-Running Jobs Project — Add Live Streaming to Your Chatbot The Problem With Parsing Free Text JSON Schema and Strict Mode Pydantic Models With the SDK's Parse Helpers Handling Refusals and Validation Failures Project — A Resume-to-JSON Extractor Working With input_image input_file, PDFs, and the Files API Image Generation Speech-to-Text and Text-to-Speech Project: A PDF Question-Answering Script What Function Calling Is Defining a Tool Schema The Full Loop Multiple Tools Errors, Timeouts, and Untrusted Arguments Project: A Weather Assistant Web Search File Search and Vector Stores Code Interpreter Remote MCP Servers and Connectors Project: A Research Assistant What an Embedding Is, Without the Maths Generating and Storing Embeddings Similarity Search From Scratch Hosted Vector Stores vs. Rolling Your Own A Small RAG App Over a Folder of Notes Agents vs. a Single API Call — When You Need One pip install openai Giving Agents Tools Handoffs and Multi-Agent Triage Guardrails and Approvals Tracing and Observing What Your Agent Did A Multi-Agent Support Desk Error Codes and What Each One Means Retries, Timeouts, and Backoff Rate Limits and Spend Limits Prompt Caching and Cost Optimisation The Batch API for Bulk Work Async Clients and Concurrency Moderation and Safety Best Practices Designing the App Backend With FastAPI Streaming to a Simple Frontend Deploying and a Cost/Safety Checklist Why Web Search Is Useful for Current Information Using the Web Search Tool with the Responses API Configuring Search Behavior for Application Use Cases Understanding Citations and Source Attribution Where to Go Next Building a Research Assistant with Web Search Combining Web Search with Structured Outputs Handling Conflicting or Low-Quality Web Sources Reducing Unsupported Claims with Grounded Generation Testing Freshness-Sensitive AI Answers Production Considerations for Web-Grounded Applications Understanding File Search and Retrieval-Augmented Generation Creating and Organizing Vector Stores Uploading Documents for Retrieval Connecting Vector Stores to Responses API Requests Designing Document Metadata and Filtering Strategies Building a PDF Question-Answering Application Improving Retrieval Quality With Better Document Preparation Handling Missing Evidence and Retrieval Failures Combining File Search With Web Search Building a Production Knowledge-Base Assistant What the Code Interpreter Tool Is Designed For Running Python-Based Analysis Through the OpenAI SDK Uploading Datasets for Analysis Analyzing CSV and Spreadsheet Data Generating Charts and Data Summaries Handling Generated Files and Downloadable Artifacts Building a Data-Analysis Assistant Combining Code Execution with Structured Outputs Validating Generated Calculations and Results Security and Sandbox Considerations for Code Execution Understanding Multimodal Input with the OpenAI SDK Sending Images to a Model Image Analysis from URLs and Uploaded Files Extracting Text and Information from Screenshots Building an Image-Question-Answering Application Combining Image Input with Structured Output Analyzing Multiple Images in One Request Handling Image Quality and Input Limitations Designing Multimodal Prompts for Reliable Results Building a Practical Vision-Powered Python Application Understanding Speech-to-Text and Text-to-Speech Workflows Transcribing Audio with the OpenAI SDK Working with Uploaded Audio Files Handling Timestamps and Transcription Metadata Building a Meeting Transcription Workflow Generating Spoken Responses from Text Handling Long Audio and Processing Failures Combining Audio with Text and Tool Calling Building an End-to-End Python Voice Application What Embeddings Are and When to Use Them Generating Embeddings With the OpenAI API Preparing Text for Embedding Comparing Vectors With Cosine Similarity Building a Simple Semantic Search Engine in Python Storing Embeddings in a Database Metadata Filtering for Semantic Search Chunking Strategies for Better Retrieval Evaluating Semantic Search Quality Building a Document Similarity Application When Batch Processing Makes Sense Designing Large-Volume AI Processing Pipelines Using Asynchronous Python with the OpenAI SDK Running Concurrent Requests Safely Controlling Concurrency and Avoiding Rate Limits Tracking Batch Job Progress Handling Partial Failures in Bulk Workloads Retrying Failed Items Without Duplicating Successful Work Designing Resumable AI Processing Jobs Building a Production Batch-Processing Pipeline Batch Processing Makes Sense Large-Scale AI Processing Pipelines Async Python with OpenAI SDK Safe Concurrent Requests Concurrency & Rate Limits Batch Progress Tracking Partial Failure Handling Safe Retry Handling Resumable AI Jobs Production Batch Pipeline System–User Data Separation Reusable App Instructions Prompt Templates & Variables Extraction & Classification Prompts Summarization & Transformation Prompts Explicit Output Requirements Prompt Version Management Prompt Testing & Evaluation Reusable Python Prompt Library API Key Security Secure API Key Storage Secure Secret Management Prompt Injection Prevention Trusted vs. Untrusted Content Tool Argument Validation Sensitive Data Handling Secure Logging AI Action Authorization Production AI Security Checklist Why AI Applications Need Evaluation Beyond Unit Tests Unit Testing OpenAI SDK Integration Code Mocking API Responses in Python Tests Testing Structured Outputs Against Schemas Testing Tool-Calling Workflows Building a Small Evaluation Dataset Measuring Accuracy, Consistency, and Failure Rates Regression Testing Prompts and Model Changes Human Evaluation Versus Automated Evaluation Creating a Repeatable Evaluation Pipeline AI Request Monitoring Token Cost Management Usage Metrics Design Reducing Model Calls Prompt & Context Optimization Model Selection & Optimization AI Caching Strategies Interactive Latency Optimization Usage Dashboards & Budget Alerts Performance & Cost Checklist Every API Call Starts Fresh Fixing API Statelessness Server-Side Conversation Memory Limits of Response Chaining What We're Building Conversation Memory Challenges Preparing an OpenAI SDK Application for Deployment Environment-Specific Configuration for Development and Production Deploying a Python AI Service with Docker Container Health Checks and Startup Configuration Managing Secrets in Cloud Deployments Background Workers for Long-Running AI Tasks Queues and Asynchronous Job Architectures Scaling AI Workloads Horizontally Monitoring Production Incidents and Failures Production Deployment Checklist for OpenAI SDK Applications Reusable OpenAI Service Classes AI Client Dependency Injection Typed AI Responses Python Configuration Management AI Request Decorators Centralized AI Error Handling Clean SDK Abstractions Reusable OpenAI Utilities Internal AI Python Libraries SDK Integration Maintenance Production AI Chatbot Document Q&A System Web Research Assistant Customer Support Agent AI Data Analysis Assistant Image Analysis App Meeting Transcription & Summary Semantic Document Search Multi-Tool AI Agent Production OpenAI SDK App Why "It Looked Fine When I Tested It" Isn't Enough Timing Note Status Note Pre-Decision Status Note Current Availability Note
Ask about this post
AI Ask about this post

Ask questions about Uploading Datasets for Analysis and get answers drawn from it.

Signed-in readers only.