Introduction to the OpenAI SDK

What the OpenAI SDK your code here

Almost every problem a beginner hits in their first week with OpenAI comes from collapsing three separate things into one word. People say I'm using GPT, I'm calling the API, and I installed OpenAI as if they all mean the same thing. They do not. They are three distinct layers, owned by different systems, versioned separately, priced differently, and failing in different ways.

Getting this distinction right is not academic. It determines where you look when something breaks, which page of the documentation answers your question, what you are actually paying for, and whether an error is your fault, the network's fault, or the model's fault.

The Three Layers

When you build an AI feature with OpenAI, three things are stacked on top of each other:

Layer	What it is	Where it lives	You interact with it by
The model	A trained neural network, e.g. `gpt-6-astra`	OpenAI's servers (GPUs)	Naming it in a request
The API	An HTTP service that accepts requests and routes them to models	`https://api.openai.com`	Sending HTTP requests
The SDK	A Python package that builds those HTTP requests for you	Your machine, in your virtual environment	`import openai`

Read that table top to bottom and notice what changes: the model is a thing that computes, the API is a protocol for reaching it, and the SDK is a convenience layer on your own computer. Only one of those three ever runs on your laptop.

That last point is the single most useful thing to internalise. The SDK does not contain a model. When you run `pip install openai`, you download roughly a few megabytes of Python source code. Models are hundreds of gigabytes to terabytes of weights running on specialised hardware. Nothing about installing the package puts intelligence on your machine. It puts a telephone on your machine.

Layer One: The Model

A model is a large mathematical function whose parameters (weights) were learned during training. You give it a sequence of tokens, it produces a probability distribution over what token comes next, a token is chosen, and the process repeats until the model emits a stop signal or hits a limit.

Three properties of models matter enormously for how you write code around them, and all three surprise beginners:

• Models are stateless - A model does not remember your previous message. It has no session, no memory, no notion of "earlier in our conversation." Every single call starts from nothing. When a chatbot appears to remember what you said three turns ago, it is because the software around the model resent the entire conversation on every turn. This is not a limitation you can configure away; it is what the function is. Unit 4 of this course is entirely about the techniques used to fake memory convincingly.

• Models are non-deterministic by default - The same input can produce different output on different calls, because the next token is sampled from a probability distribution rather than always taking the most likely option. Some models let you reduce this with a `temperature` setting or a `seed`, but you should design your code assuming the text will vary. Any code that does `if response == "yes"` on free-form model output is fragile by construction. Unit 6 solves this properly with structured outputs.

• Models are named, versioned products - `gpt-6-astra` and `gpt-5.6-luna` are different models with different capabilities, different context window sizes, different speeds, and prices that differ by a factor of fifty. As of writing, the published per-million-token rates are:

Model	Input	Cached input	Output
`gpt-6-astra`	$10.00	$1.00	$50.00
`gpt-5.6-sol`	$4.00	$0.40	$20.00
`gpt-5.6-terra`	$2.00	$0.20	$12.00
`gpt-5.6-luna`	$0.20	$0.02	$1.20

Two things follow from that table. First, "which model" is a real engineering decision with a real budget attached, not a detail. Second — and this is the important one — model names change far faster than anything else in this stack. New models ship, old ones are deprecated, prices move. Any tutorial (including this one) that hardcodes a model name will eventually be wrong. Treat the model ID in every example here as a placeholder you check against the current models page, not as a fact to memorise. Unit 2, Lesson 5 covers how to read that page properly.

While learning, use the cheapest capable model available — currently `gpt-5.6-luna` at $0.20 per million input tokens. You will make hundreds of throwaway calls in this course, and there is no pedagogical value in paying fifty times more for them.

 Layer Two: The API

The API is a web service. That is the whole idea. It sits at `https://api.openai.com`, it speaks HTTP, and it accepts and returns JSON.

Nothing about it is special to AI. If you have ever called a weather API or a payments API, you already understand the mechanics: you send a request to a URL, you include a key so the server knows who is paying, you put your parameters in a JSON body, and you get JSON back.

Here is what a request to the Responses endpoint looks like with no Python involved at all:

curl https://api.openai.com/v1/responses \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
"model": "gpt-5.6-luna",
"input": "Write a one-sentence bedtime story about a unicorn."
}'

Every piece of that command has a job:

- `https://api.openai.com/v1/responses` is the endpoint. The `/v1` is the API version; `/responses` is the specific capability. Different endpoints do different things: `/v1/responses` generates text, `/v1/embeddings` turns text into vectors, `/v1/images` generates images.
- `Content-Type: application/json` tells the server how to parse the body you sent. Remove it and the server may reject the request because it does not know the bytes are JSON.
- `Authorization: Bearer <key>` is how the server identifies your account. Remove it and you get a `401` authentication error. This is also how billing is attributed — the key is the thing that gets charged.
- The `-d` body carries your actual parameters. `model` selects which model runs; `input` is what you want it to respond to.

You could build an entire production application using nothing but requests like this. Many teams in other languages do exactly that. The API is the real product; everything else is convenience.

• Why this matters for debugging - because the API is just HTTP, every failure it produces is an HTTP failure with a status code, and status codes are diagnosable. `401` means your key is wrong or missing. `429` means you are being rate-limited or you are out of credit. `400` means your request body was malformed — a misspelled parameter, an invalid model name, a value out of range. `500` and `503` mean the problem is on OpenAI's side and retrying may help. Unit 12, Lesson 1 goes through these one by one, but the mental model to build now is: *the number tells you whose fault it is.

Layer Three: The SDK

An SDK — Software Development Kit — is a library that wraps an API in the idioms of a specific language. The OpenAI Python SDK is the package you get from `pip install openai`. Its job is to turn Python function calls into those HTTP requests, and HTTP responses back into Python objects.

The same request as above, through the SDK:

from openai import OpenAI

client = OpenAI()

response = client.responses.create(
model="gpt-5.6-luna",
input="Write a one-sentence bedtime story about a unicorn.",
)

print(response.output_text)

Do not run this yet — you need Python set up (Lesson 2) and a key (Lesson 3) first. Read it as an illustration of what the SDK is for.

Compare it line by line against the `curl` command and you can see exactly what the SDK took over:

• URL construction -  You never typed `https://api.openai.com/v1/responses`. The SDK knows that `client.responses.create()` maps to `POST /v1/responses`. This mapping is worth learning as a pattern, because it makes the HTTP reference documentation directly usable: a doc page about `POST /v1/embeddings` is `client.embeddings.create()` in Python.
• Authentication - You never wrote an `Authorization` header. `OpenAI()` with no arguments reads the `OPENAI_API_KEY` environment variable automatically and attaches the header to every request.
• Serialisation - You passed Python keyword arguments; the SDK built the JSON body.
• Deserialisation - You got back a Python object with attributes, not a string of JSON you have to `json.loads()` and then index with brackets. `response.output_text` is an attribute access, which means your editor can autocomplete it and a typo raises an `AttributeError` instead of silently returning `None`.

And several things it does that are invisible in that snippet but matter in production:

• Automatic retries - Transient failures — connection errors, `429`s, `5xx`s — are retried with exponential backoff by default (currently two retries), so a momentary network blip does not surface as an exception in your application.
• Timeouts - Requests time out after ten minutes by default, and the timeout is configurable per client or per call, so a hung connection cannot block your program forever.
• Typed errors - Failures raise specific exception classes — `openai.AuthenticationError`, `openai.RateLimitError`, `openai.BadRequestError`, `openai.APIConnectionError` — all inheriting from `openai.APIError`. You can therefore write `except openai.RateLimitError:` and handle *only* the rate-limit case, instead of parsing status codes out of a generic HTTP response.
• Streaming plumbing - When you ask for a streamed response, the SDK handles the Server-Sent Events protocol and hands you an iterator of typed event objects. Doing that by hand is genuinely unpleasant. Unit 5 covers it.
• Async support -  `AsyncOpenAI` mirrors the entire synchronous surface with `await`, so the same knowledge transfers to concurrent code.

What the SDK Deliberately Does Not Do

Being clear about the SDK's boundaries prevents a whole category of confusion:

• It does not contain, run, or cache a model.
• It does not remember conversations for you. Storing history is your job (Unit 4).
• It does not count tokens before sending. It reports usage after the fact, in `response.usage`.
• It does not validate your prompt or guarantee output shape. Strict output shape is a separate feature you opt into (Unit 6).
• It does not manage your API key's secrecy. Loading keys from a `.env` file and keeping them out of Git is your responsibility (Lesson 3).
• It does not decide which model to use. That is always your explicit choice.

The Full Lifecycle of One Call

Tracing a single request end to end ties all three layers together:

1. Your Python code calls `client.responses.create(model=..., input=...)`.
2. The SDK validates the arguments locally, builds a JSON body, attaches the `Authorization` header from your environment, and opens an HTTPS connection to `api.openai.com`.
3. OpenAI's servers authenticate the key, check your rate limits and credit balance, and reject the request immediately if any of those fail — before any model runs.
4. Your text is tokenised: split into subword units. "unicorn" might be one token; an unusual word might be three. Tokens, not words or characters, are the unit of both context limits and billing.
5. The model runs, generating output tokens one at a time until it emits a stop signal or reaches the output cap.
6. The server assembles a JSON response containing an ID, the generated content in a structured array, and a `usage` object recording how many input and output tokens were consumed.
7. The SDK parses that JSON into a typed Python object and returns it. Your `print(response.output_text)` runs.
8. The token counts from step 6 are multiplied by the per-token price of the model you named and deducted from your credit balance.

Steps 3 through 6 happen on OpenAI's infrastructure. Steps 1, 2, 7 and 8 are where your decisions live — and step 8 is a direct consequence of step 1's model choice.

Where "Responses API" Fits

Within the API layer there are several endpoints that generate text, and knowing which is which stops you from following the wrong tutorial:

• Responses API - (`client.responses.create`) — the current, recommended interface. It handles plain text, images, files, tool calls, reasoning models, streaming and built-in tools through one consistent shape. Everything in this course uses it.
• Chat Completions API -  (`client.chat.completions.create`) — the older interface, built around a `messages` list and returning `completion.choices[0].message.content`. It still works and an enormous amount of code and tutorial content on the internet uses it, which is why you will keep meeting it. Unit 2, Lesson 1 explains the differences and how to translate between them.
• Agents SDK -  (`pip install openai-agents`) — a *separate package*, layered on top of the same API, that adds agents, tools, handoffs and tracing. It is not part of the `openai` package. Unit 11 covers it.

The relationship is worth stating plainly: Responses and Chat Completions are two doors into the same building. The Agents SDK is a different, higher-level package that walks through the Responses door on your behalf.

Common Misconceptions

"I installed the SDK, so I can run this offline." No. Every call is a network request to OpenAI's servers. No internet, no response — you will get an `APIConnectionError`. Nothing runs locally.

"The SDK version and the model version are related." They are not. `openai` 3.0.0 is the version of the Python package; `gpt-6-astra` is the name of a model. Upgrading the package does not upgrade the model, and a new model does not require a package upgrade unless it needs a parameter the older package cannot send.

"The API is the same thing as ChatGPT." Different products on the same underlying technology. ChatGPT is a consumer application with its own subscription. The API is a developer service with its own prepaid billing. A ChatGPT Plus subscription gives you no API credit whatsoever, and this catches out a surprising number of people on their first billing error. Lesson 5 covers the API's billing model.

"My prompt is too long, so I need a bigger SDK." Context limits are a property of the model, not the library. `gpt-6-astra` currently accepts roughly 1.05 million tokens of context; a smaller model accepts less. Changing packages changes nothing here — changing models does.

"The model remembered what I asked earlier." It did not. Either you resent the history, or you used `previous_response_id` / the Conversations API, both of which cause *the platform* to resend it for you. Unit 4 shows all three approaches and what each costs.

How to Use This Mental Model When Reading Documentation

The three-layer split makes the documentation navigable:

- A question about what a model can do — context window, whether it supports images, how much it costs — belongs on the models and pricing pages.
- A question about what fields a request accepts — is the parameter called `instructions` or `system`, what values does it take — belongs in the HTTP API reference. Field names there translate directly to Python keyword arguments.
- A question about how to express something in Python — how to iterate a stream, how to catch a rate-limit error, how to set a timeout — belongs in the Python SDK reference and the package's README.

When you hit an error, ask which layer produced it before you start changing code:

- `AttributeError`, `TypeError`, `ImportError` → your Python or the SDK layer. Nothing left your machine.
- `AuthenticationError` (401), `BadRequestError` (400), `RateLimitError` (429) → the API layer. Your request reached OpenAI and was rejected before or during processing.
- Output that is wrong, waffly, or in the wrong format → the model layer. This is a prompting or structured-output problem, not a code problem, and no amount of Python will fix it. Units 3 and 6 exist for exactly this.

That triage question — which layer? — will save you more time over this course than any single piece of syntax.

Practical Consequences for the Code You Are About to Write

Three habits follow directly from this model, and adopting them now will keep your projects healthy later:

• Never hardcode a model name in more than one place - Put it in a constant or a config value. Models get deprecated on a schedule, and when that happens you want to change one line, not thirty.

• Never assume the SDK protects you from cost - The SDK will happily send a two-hundred-thousand-token request to the most expensive model if that is what you asked for. Cost control is a design decision you make — model choice, `max_output_tokens`, prompt size — not something the library does for you.

• Read the error's type before reading its message - The exception class tells you the layer, and the layer tells you where to look. `openai.BadRequestError` means "the server understood you and refused"; `openai.APIConnectionError` means "the server never heard from you at all." Those two demand completely different fixes, and the messages alone do not always make the distinction obvious.

A Closer Look at the Client Object

`client = OpenAI()` deserves more attention than it usually gets, because it is where every piece of cross-cutting configuration lives.

The client is a long-lived object that holds your credentials, your connection settings, and an underlying HTTP connection pool. Creating one is cheap but not free: it sets up that pool, and reusing it across many requests lets connections stay open rather than being renegotiated (TLS handshake and all) on every call.

from openai import OpenAI

client = OpenAI(
api_key="sk-...", # defaults to the OPENAI_API_KEY env var
timeout=20.0, # seconds; default is 10 minutes
max_retries=5, # default is 2
base_url="https://api.openai.com/v1",
)

Each of those arguments answers a real question:

• `api_key` — passing it explicitly is legal but usually the wrong choice, because a key written in code is a key that gets committed to Git. Leaving it out and letting the SDK read `OPENAI_API_KEY` is the safer default, which is exactly why the SDK behaves that way. Lesson 3 covers this properly.
• `timeout` — the ten-minute default exists because some legitimate requests (long reasoning, large outputs) genuinely take minutes. For a web application where a user is waiting, ten minutes is far too long; you want to fail fast and show an error. Set it to something realistic for your use case.
• `max_retries` — the SDK retries connection errors, `429`s and `5xx`s with exponential backoff. Set it to `0` when you want to handle retries yourself, or raise it for background jobs where latency does not matter and success does. Note that retries cost money if the failed attempt already generated tokens.
• `base_url`— the API's address. You override this when pointing the SDK at a compatible third-party endpoint, a local model server, or a proxy. It is the reason the same SDK can talk to non-OpenAI services that implement the same HTTP shape.

Two practical rules follow. Create the client once and reuse it — module level in a script, or an application-startup dependency in a web app. Creating a fresh client inside a loop wastes connections and gains nothing. And use `client.with_options()` for one-off overrides rather than building a second client:

Retry this particular call more aggressively, leave the rest alone

client.with_options(max_retries=5).responses.create(
model="gpt-5.6-luna",
input="Hello",
)

The Account Layer: Organisations, Projects, and Keys

There is a fourth layer that is not code at all but shapes everything: your OpenAI account structure.

An organisation is the billing entity. A project sits inside an organisation and is a scoping boundary — it has its own API keys, its own rate limits, its own usage tracking, and optionally its own spend limits and model restrictions. An API key belongs to a project.

This matters more than it sounds. Because usage is tracked per project, giving each application its own project is how you answer "which of my apps burned through the credit this month." Because keys are scoped to a project, revoking a leaked key kills exactly one application rather than all of them. And because limits can be set per project, you can cap a prototype at a few dollars while leaving production uncapped.

The practical advice for this course: create a dedicated project for learning, generate a key in it, and set a modest spend limit. Then any mistake you make — an accidental loop, a mistyped model name that turns out to be fifty times more expensive — is bounded by a number you chose. Lesson 5 walks through the specifics.

Tokens: The Unit Everything Is Measured In

Context windows, pricing, rate limits, and output caps are all measured in tokens, so a working intuition for them pays off immediately.

A token is a chunk of text produced by the model's tokeniser — roughly a common word, a word fragment, or a piece of punctuation. Rough English-language rules of thumb:

• 1 token ≈ 4 characters ≈ 0.75 words
• 1,000 tokens ≈ 750 words ≈ 1.5 pages of plain prose

Those ratios are approximations that shift with the content. Code tokenises less efficiently than prose because of punctuation and indentation. Languages that do not use spaces between words, and text in scripts the tokeniser saw less of during training, can use several times more tokens for the same amount of meaning. JSON is expensive because every brace, quote and colon costs something.

Three consequences worth carrying forward:

• Both directions cost, at different rates - You pay for input tokens (everything you send: instructions, conversation history, retrieved documents) and output tokens (everything the model generates), and output is consistently several times more expensive — currently five to six times across the model line-up. A verbose response is not just slower to read; it is disproportionately expensive.

• Context is a hard ceiling, not a soft one - A model's context window is the maximum number of tokens it can consider in a single request, input and output together. Exceed it and the request fails with a `400` — the model does not silently forget the oldest part. Because conversation history is resent on every turn, a long chat approaches that ceiling steadily. Unit 4, Lesson 6 covers counting and compaction.

• You cannot know the exact count in advance from the SDK - The `usage` object on the response tells you after the fact. If you need the number before sending, you tokenise locally with a tokeniser library, or you estimate. Estimating with the ratios above is usually enough for budgeting.

SDK, Raw HTTP, or a Framework?

Once you know the SDK is optional, the obvious question is when to skip it — or when to reach for something above it.

Approach	Best for	Cost
Raw HTTP (`requests`, `curl`)	Languages with no official SDK; debugging exactly what is on the wire; minimal-dependency environments	You reimplement retries, backoff, streaming parsing, and error typing yourself
Official SDK (`openai`)	Almost all Python work	One dependency; ties you to OpenAI's request shape
Agent framework (Agents SDK, and third-party frameworks)	Multi-step agents, tool orchestration, handoffs, tracing	Another abstraction layer between you and the request; harder to debug until you understand the layer beneath

The recommendation for anyone learning is unambiguous: use the official SDK, and learn the HTTP shape underneath it. The SDK removes the tedious parts without hiding the model. Frameworks are genuinely useful once your problem is orchestration rather than generation, but starting there means debugging two abstractions at once when a request misbehaves — and you will not be able to tell which one is lying to you.

The one habit worth keeping regardless of choice: when a request behaves strangely, be able to describe it as an HTTP request. "I sent a POST to `/v1/responses` with model X and a 12,000-token input, and got a 400" is a debuggable statement. "The SDK didn't work" is not.

Reading a Response Is Reading a Structure

One last piece of the mental model, because it shapes every unit that follows.

`response.output_text` is a convenience. It is not the response — it is a shortcut that walks the real response and concatenates the text parts it finds. The actual response is a structured object: an ID, a status, a model name, a `usage` block, and an `output` array of typed items.

That array is a list because a single response can contain more than one thing. A plain text answer produces one message item. A response from a reasoning model can carry a reasoning item alongside the message. A response where the model decided to call one of your tools carries function-call items instead of, or in addition to, text — and in that case `output_text` may be empty even though the request completely succeeded.

This is why Unit 2, Lesson 3 is dedicated to the output array, and why it is worth resisting the habit of treating `output_text` as "the answer." It is the right shortcut for the simple case and a trap in every advanced one. The moment you enable a tool, a structured schema, or a reasoning model, you need to look at the structure rather than the string.