Project 7: Build a Meeting Transcription and Summary System

This project builds a pipeline that turns a recorded meeting into a searchable transcript and a structured summary with action items, using the audio transcription and speech capabilities from Unit 19. The pipeline is designed around long recordings, which introduces a chunking problem that a short voice-memo example would not surface.

Scope and Design Decisions

The system accepts an audio file (typically 20 minutes to two hours), produces a full transcript, and generates a structured meeting summary: key discussion points, decisions made, and action items with owners when identifiable. It does not attempt real-time streaming transcription — that is a materially different architecture, built on a streaming transcription API rather than batch file upload, and is a reasonable follow-on extension rather than part of this base project.

Two decisions shape the design:

1. Long recordings are chunked before transcription, not summarized in one call. Transcription models have practical file-size and duration limits, so a two-hour meeting must be split into segments, transcribed independently, and stitched back together with timestamps preserved.
2. Summarization operates on the transcript text, not the audio directly. Once a clean, timestamped transcript exists, extracting action items and decisions is a text-understanding task best handled by the main language model with structured outputs (Unit 6), separate from the audio-to-text step.

Chunking and Transcribing Long Audio

from openai import OpenAI
from pydantic import BaseModel
from pydub import AudioSegment
import os

client = OpenAI()

CHUNK_DURATION_MS = 10 * 60 * 1000  # 10-minute chunks stay well under upload limits

def split_audio(file_path: str, output_dir: str) -> list[str]:
    audio = AudioSegment.from_file(file_path)
    os.makedirs(output_dir, exist_ok=True)
    chunk_paths = []
    for i, start_ms in enumerate(range(0, len(audio), CHUNK_DURATION_MS)):
        chunk = audio[start_ms:start_ms + CHUNK_DURATION_MS]
        chunk_path = os.path.join(output_dir, f"chunk_{i:03d}.mp3")
        chunk.export(chunk_path, format="mp3")
        chunk_paths.append(chunk_path)
    return chunk_paths

def transcribe_chunk(chunk_path: str, chunk_index: int, offset_seconds: float) -> dict:
    with open(chunk_path, "rb") as f:
        result = client.audio.transcriptions.create(
            model="gpt-5.6-terra-transcribe",
            file=f,
            response_format="verbose_json",
        )
    return {
        "chunk_index": chunk_index,
        "offset_seconds": offset_seconds,
        "text": result.text,
        "segments": getattr(result, "segments", []),
    }

split_audio uses fixed 10-minute chunks rather than trying to detect natural pause points — silence-based splitting is a reasonable refinement but adds complexity for a marginal gain, since even a split mid-sentence loses at most a few words at the boundary, which the summarization pass can tolerate. transcribe_chunk records offset_seconds alongside the transcribed text specifically so that timestamps from each chunk's segments can later be corrected back to their position in the original, full-length recording — without this offset, every chunk's internal timestamps would restart at zero and be meaningless once chunks are combined.

Note: Transcription model names, response_format options, and the exact shape of segments are part of the audio API surface covered in Unit 19 and can evolve between SDK versions. Verify current parameter names and the verbose JSON schema against the SDK documentation before relying on specific field names in production.

Stitching Chunks Into a Single Transcript

def transcribe_full_meeting(file_path: str, work_dir: str) -> dict:
    chunk_paths = split_audio(file_path, work_dir)
    chunk_results = []
    for i, chunk_path in enumerate(chunk_paths):
        offset = i * (CHUNK_DURATION_MS / 1000)
        chunk_results.append(transcribe_chunk(chunk_path, i, offset))

    full_text_parts = []
    all_segments = []
    for chunk in chunk_results:
        full_text_parts.append(chunk["text"])
        for segment in chunk["segments"]:
            adjusted_start = segment.get("start", 0) + chunk["offset_seconds"]
            adjusted_end = segment.get("end", 0) + chunk["offset_seconds"]
            all_segments.append({
                "start": adjusted_start,
                "end": adjusted_end,
                "text": segment.get("text", ""),
            })

    return {
        "full_text": " ".join(full_text_parts),
        "segments": all_segments,
    }

The offset correction — adding each chunk's offset_seconds to every segment's local start and end — is the detail that makes the stitched transcript actually usable for navigation (jumping to the moment a decision was discussed). Chunks are processed sequentially here for clarity; a production version handling many long meetings would parallelize this loop with a thread pool or async calls, since transcription chunks are fully independent of each other and there is no reason to wait for one to finish before starting the next.

Extracting a Structured Summary

class ActionItem(BaseModel):
    description: str
    owner: str | None
    due_date_mentioned: str | None

class MeetingSummary(BaseModel):
    key_points: list[str]
    decisions: list[str]
    action_items: list[ActionItem]
    open_questions: list[str]

def summarize_meeting(full_text: str) -> MeetingSummary:
    response = client.responses.parse(
        model="gpt-5.6-terra",
        input=[
            {
                "role": "system",
                "content": (
                    "Summarize this meeting transcript. Distinguish decisions "
                    "that were finalized from open questions still under "
                    "discussion. For each action item, extract the owner only "
                    "if a specific person was named; otherwise leave it null "
                    "rather than guessing."
                ),
            },
            {"role": "user", "content": full_text},
        ],
        text_format=MeetingSummary,
    )
    return response.output_parsed

The instruction to leave owner as null rather than guessing addresses a real failure mode: meeting transcripts often contain action items where responsibility was implied but never explicitly assigned ("someone should follow up with the vendor"), and a summarization model under implicit pressure to fill in every field will confidently but incorrectly attribute the task to whoever spoke most recently. Making owner explicitly optional in the schema, combined with an explicit instruction not to guess, is what keeps the summary honest about what was actually said versus what the model inferred.

decisions and open_questions are kept as separate lists rather than one combined "topics discussed" list because they carry very different weight for a reader skimming the summary — a finalized decision needs no further discussion, while an open question is exactly the kind of item that should resurface on the next meeting's agenda. Conflating them would lose that distinction.

Full Pipeline

def process_meeting_recording(audio_path: str, work_dir: str) -> dict:
    transcript = transcribe_full_meeting(audio_path, work_dir)
    summary = summarize_meeting(transcript["full_text"])
    return {
        "transcript": transcript,
        "summary": summary,
    }

def format_summary_markdown(summary: MeetingSummary) -> str:
    lines = ["## Key Points"]
    lines += [f"- {p}" for p in summary.key_points]
    lines += ["", "## Decisions"]
    lines += [f"- {d}" for d in summary.decisions]
    lines += ["", "## Action Items"]
    for item in summary.action_items:
        owner_part = f" (Owner: {item.owner})" if item.owner else " (Owner: unassigned)"
        due_part = f" — due {item.due_date_mentioned}" if item.due_date_mentioned else ""
        lines.append(f"- {item.description}{owner_part}{due_part}")
    lines += ["", "## Open Questions"]
    lines += [f"- {q}" for q in summary.open_questions]
    return "\n".join(lines)

process_meeting_recording is the top-level entry point that ties transcription and summarization together, but the two remain independently callable — an application might re-summarize an already-transcribed meeting with a refined prompt without paying for re-transcription, which is the more expensive and slower of the two steps.

Testing the Summarization Contract

def test_owner_left_null_when_not_named(monkeypatch_parse):
    fake_summary = MeetingSummary(
        key_points=["Discussed Q3 roadmap"],
        decisions=["Move launch date to October"],
        action_items=[
            ActionItem(description="Follow up with the vendor", owner=None, due_date_mentioned=None)
        ],
        open_questions=["Who owns the vendor relationship going forward?"],
    )

    class FakeResponse:
        output_parsed = fake_summary

    monkeypatch_parse(lambda **kwargs: FakeResponse())
    result = summarize_meeting("some transcript text")
    assert result.action_items[0].owner is None
    assert "vendor" in result.open_questions[0].lower()
    print("PASS: unassigned action items keep owner as null rather than guessed")

def _make_monkeypatch():
    original = client.responses.parse
    def apply(fn):
        client.responses.parse = fn
    def restore():
        client.responses.parse = original
    return apply, restore

monkeypatch_parse, restore_parse = _make_monkeypatch()
test_owner_left_null_when_not_named(monkeypatch_parse)
restore_parse()

The test substitutes a fake client.responses.parse returning a pre-built MeetingSummary, which validates that summarize_meeting correctly passes through the parsed object without altering it — the actual guarantee this test protects is architectural (the function is a thin, faithful wrapper around the API call), while the harder-to-test guarantee (the model itself reliably avoiding guessed owners) belongs to prompt evaluation, a different kind of testing covered in Unit 13.

Extending This Project

Add speaker diarization so the transcript attributes each segment to a specific speaker, which substantially improves the quality of owner attribution in action items, and add a searchable index over transcript segments (using the embeddings techniques from Project 8) so users can search across an entire meeting archive by topic.

Common Mistakes

• Feeding an entire long recording to a transcription call without checking size or duration limits. This either fails outright or silently truncates; always chunk proactively for anything beyond a short clip.
• Losing timestamp continuity across chunks. Each chunk's transcription starts its internal timestamps at zero; forgetting to add the chunk's offset makes the stitched transcript's timestamps meaningless for navigation.
• Letting the summarization model guess action item owners when none was stated. This produces a plausible-looking but factually invented summary. Make the owner field optional and instruct the model explicitly not to infer one.

Best Practices

• Separate the audio-to-text step from the text-understanding step. Transcription and summarization are different tasks with different failure modes; keeping them as independent, independently testable functions makes both easier to improve and to re-run selectively.
• Preserve timestamps end to end. A transcript without navigable timestamps is far less useful for reviewing a specific part of a long meeting.
• Make uncertainty representable in the schema. Optional fields like owner and due_date_mentioned let the model honestly report what was and wasn't specified, rather than being forced to fabricate a value.