Everything so far went through the TwelveLabs API directly. This chapter repeats the core experiment through Amazon Bedrock: your AWS account, your S3 bucket, your IAM credentials, your bill. The Bedrock model versions differ — Pegasus 1.5 rather than 1.6, Marengo 3.0 embeddings rather than 3.5 — so treat this as learning a second front door, not as a controlled benchmark.
What you will build
- A clean credential setup: MFA on root, then temporary credentials via an AWS CLI profile — never long-lived keys in files
- A private S3 bucket with lab/input/ and lab/output/ prefixes
- bedrock_analyze.py — one Pegasus 1.5 analysis with strict stopReason validation
- bedrock_embed.py — one Marengo 3.0 async video job with token-first resume, plus a sync text embedding, full S3 output discovery, and local ranking of the Bedrock 3.0 corpus against its own query
The account, cost, and data gate — before typing
Start your coding partner in offline planning mode for this chapter. The required budget checkpoint below is a stop gate, not something an agent silently creates or assumes complete. You verify the intended account, saved Actual thresholds, and Active recipients yourself and understand that alerts do not stop charges; temporary credentials stay outside the partner's context. Dry-run here means inspect proposed commands and mocked request shapes, not execute AWS commands.
1Use my brief and selected files for chapter 5 only. Plan one small offline2Bedrock increment; wait for approval before applying it. Cite current official3AWS/TwelveLabs API contracts rather than inventing them. Use temporary credentials4without reading them. Dry-run means command review and mocks, not AWS execution.5STOP before any resource operation or paid call until I verify the required6budget/Active-recipient checkpoint and separately approve the exact operation.7Alerts are not a cap. Bound duration, calls, retries and cleanup; preserve token-first resume.8Run local tests; inspect actual responses only after an approved live run.9Do not read secrets, commit/push, provision, or make paid calls without my10explicit approval. Do not jump chapters. Redact errors; I review the diff and commit.1Review chapter 5's diff and offline tests against current official API contracts.2Check the budget stop gate, reader-verified Active recipients, temporary-credential3handling, bounded costs/retries, token persistence, output validation and cleanup.4Do not infer approval from a budget; alerts do not stop charges. Dry-run is offline.5Separate mocks from actual AWS evidence and debug only with redacted errors.6Do not read secrets, commit/push, provision, or make paid calls without my7explicit approval; no later chapters. Explain the Git diff; I make the commit.- Enable MFA on the root user, then stop using root. Daily work uses IAM Identity Center (SSO) or the current aws login flow — temporary credentials only.
- Open the AWS Marketplace listing for the TwelveLabs models and read terms and pricing before invoking anything; access may subscribe on first invocation.
- Do not follow outdated “Model access” page instructions; the current flow is in the model-access guide linked below.
- Complete the required budget-alert checkpoint below before any billable step — an alert warns, it does not cap.
- Assume nothing is free: credits, plan restrictions, and upgrade eligibility all require checking your actual account.
A concrete reference point from the current Bedrock pricing page: Marengo 3.0 embeddings list at $0.00070 per video second plus $0.00007 per text query. Estimate using your selected file's measured duration × the current video-second rate, plus your actual query count × the text-query rate — not a fixed 90-second corpus assumption. The linked Sintel trailer measured 52.208 seconds; trimming changes both duration and provenance notes. These inference components exclude S3, other models, and tax. Pegasus 1.5 pricing was not verified at the time of writing. Check the live pricing page; do not trust any article's arithmetic, including this one.
For a small first Bedrock experiment, choose the Sintel trailer from chapter one or a short clip you recorded and own. Complete the required budget checkpoint below before any billable step. The unchanged commands and scripts use the tabletop example clip-01-notebook.mp4; if you select Sintel, deliberately update the upload command's local filename and S3 key, the matching S3 input URI in both Bedrock scripts, and bedrock_embed.py's parsed clip_id to sintel_trailer-720p in your own copy before running. Do not disguise the film under a tabletop ID. Use its actual duration for cost review, preserve its credit/license and any trim notes, and manually validate analysis and retrieval rather than expecting the tabletop ground truth.
Small licensed input option
Required checkpoint — a budget alert before the first billable call
Do this before you create the bucket or invoke any model. The $10 USD monthly amount below is illustrative only: it is not a promised lab cost and it is not authorization to spend $10. Pick an amount that fits you. Console steps were checked against the official AWS pages linked below on October 17, 2026; consoles change, so follow the official guide if a label differs.
- 1. Sign in to the intended account (not root) and open Billing and Cost Management at https://console.aws.amazon.com/cost-management/, then choose Budgets → Create budget.
- 2. Choose Customize (advanced), then Cost budget.
- 3. Name it Video learning alerts. Period: Monthly. Renewal: Recurring. Start: current month. Budgeting method: Fixed, amount 10 USD (or your own chosen amount).
- 4. Leave the service and Region filters empty so the budget warns on spend across the whole account — this means it also counts spend unrelated to this lab. Review the cost options (credits, refunds, taxes, support charges) and decide what you want included.
- 5. Add four alert thresholds, all as a percentage of budget: Actual 50% ($5), Actual 80% ($8), Actual 100% ($10), and Forecasted 100%.
- 6. Give each alert an email recipient that you monitor. Skip Amazon SNS, budget actions, and budget reports — this exercise uses none of them.
- 7. Review the summary yourself, then choose Create budget.
- 8. A new recipient receives an AWS verification email: confirm it while signed in to the correct account, then check that the recipient shows Active. Pending recipients are not notified. A previously verified address may already show Active.
Cost of the alert itself: budget monitoring and email notifications are free. The “first two free” allowance applies to action-enabled budgets, with additional action-enabled budgets at $0.10 per day, and budget reports cost $0.01 per delivered report. This exercise uses neither actions nor reports.
Completion gate — continue only when all four are true: the budget exists in the intended account; the three Actual thresholds are saved; every recipient shows Active; and you understand the alert does not stop charges.
Official AWS sources for this checkpoint (verified October 17, 2026)
Step 1 — Credentials, minimum permissions, region
The lab uses us-east-1, subject to what your account and the models support. Pegasus 1.5 is addressed through model IDs like us.twelvelabs.pegasus-1-5-v1:0 (US geographic profile) or global.twelvelabs.pegasus-1-5-v1:0 (global routing). Default to the US profile: global routing can process outside your chosen geography — a decision to make deliberately, not silently.
Minimum permissions, not admin: the lab identity needs bedrock:InvokeModel and bedrock:StartAsyncInvoke/GetAsyncInvoke on the two model IDs, plus S3 read/write scoped to your one lab bucket (s3:PutObject/GetObject/ListBucket on arn:aws:s3:::your-bucket and arn:aws:s3:::your-bucket/*). Nothing else. If a tutorial tells you to attach AdministratorAccess, close the tab.
1aws configure sso # profile name: twelvelabs-video-lab2aws sso login --profile twelvelabs-video-lab3export AWS_PROFILE=twelvelabs-video-lab4aws sts get-caller-identity # confirm you are NOT rootOne reproducibility note: python-dotenv loads variables into the Python process only — it does not populate your parent shell. For CLI commands, export LAB_BUCKET and LAB_ACCOUNT_ID explicitly in the shell (from your local .env, which stays gitignored); the Python scripts read the same names through load_dotenv(). Never source arbitrary files into your shell.
1pip install boto3 # already in requirements.txt from chapter 22export LAB_BUCKET=your-unique-bucket-name3export LAB_ACCOUNT_ID=1234567890124aws s3 mb s3://$LAB_BUCKET --region us-east-15aws s3api put-public-access-block --bucket $LAB_BUCKET \6 --public-access-block-configuration \7 BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true8aws s3 cp media/clip-01-notebook.mp4 s3://$LAB_BUCKET/lab/input/clip-01-notebook.mp4Step 2 — One Pegasus 1.5 analysis
The rule from chapter three applies here in a stricter form: stopReason must be present and exactly “stop”. A missing reason or “length” means the output is incomplete, and incomplete output raises — it is never silently accepted. Also note what is absent: Pegasus 1.5 does not support attached Bedrock Guardrails, so do not design a safety story around features the model does not have.
1"""Pegasus 1.5 analysis through Amazon Bedrock."""2import json3import os4 5import boto36from dotenv import load_dotenv7 8load_dotenv()9REGION = os.environ.get("AWS_REGION", "us-east-1")10BUCKET = os.environ["LAB_BUCKET"]11ACCOUNT_ID = os.environ["LAB_ACCOUNT_ID"]12 13# US geographic profile — chosen deliberately over global routing.14PEGASUS = "us.twelvelabs.pegasus-1-5-v1:0"15 16client = boto3.client("bedrock-runtime", region_name=REGION)17 18 19def analyze(s3_uri: str, prompt: str) -> dict:20 body = {21 "analysis_mode": "general",22 "inputPrompt": prompt,23 "mediaSource": {"s3Location": {"uri": s3_uri, "bucketOwner": ACCOUNT_ID}},24 "temperature": 0.2,25 "maxOutputTokens": 1024,26 }27 response = client.invoke_model(28 modelId=PEGASUS,29 contentType="application/json",30 accept="application/json",31 body=json.dumps(body),32 )33 result = json.loads(response["body"].read())34 stop_reason = result.get("stopReason")35 if stop_reason is None:36 raise RuntimeError("response has no stopReason — treat as incomplete")37 if stop_reason != "stop":38 raise RuntimeError(f"incomplete output: stopReason={stop_reason!r}")39 return result40 41 42if __name__ == "__main__":43 s3_uri = f"s3://{BUCKET}/lab/input/clip-01-notebook.mp4"44 result = analyze(s3_uri, "Describe visible actions in order. State uncertainty.")45 print(json.dumps(result, indent=2))Step 3 — One Marengo 3.0 embedding job, resumed correctly
The async contract has three load-bearing details. The clientRequestToken is persisted before start_async_invoke, so an interrupted submission reuses the same token instead of double-submitting. The invocation ARN is persisted immediately after, because a local timeout does not cancel remote work — you resume polling the same ARN. And when the job completes, the output location is discovered from the job metadata, downloaded from S3, and parsed with dimension checks — there is no “write your own parser” gap. The sync text embedding has its own trap: the vector lives under result["data"], not result["embedding"].
1"""Chapter 5: Marengo 3.0 embeddings through Amazon Bedrock.2 3Documentation-checked against the AWS Marengo 3.0 parameter docs;4syntax/mock-checked only — no live invocation yet. Contract notes:5- Persist clientRequestToken BEFORE start_async_invoke; the same request6 hash reuses the token if submission is interrupted.7- Persist invocationArn immediately after; a local timeout never cancels8 remote work — resume polling the same ARN.9- Text embeddings are synchronous; the vector lives under result["data"],10 not result["embedding"]. Modality and dimension are validated.11- Async video results are discovered from the job's output S3 location,12 downloaded, and parsed — no "write your own parser" gap.13"""14import hashlib15import math16import json17import os18import time19import uuid20from pathlib import Path21from urllib.parse import urlparse22 23import boto324from dotenv import load_dotenv25from rank import normalize, rank26 27load_dotenv()28REGION = os.environ.get("AWS_REGION", "us-east-1")29BUCKET = os.environ["LAB_BUCKET"]30ACCOUNT_ID = os.environ["LAB_ACCOUNT_ID"]31MARENGO = "twelvelabs.marengo-embed-3-0-v1:0"32EXPECTED_DIM = 512 # Marengo 3.0 on Bedrock; assert, never assume.33 34runtime = boto3.client("bedrock-runtime", region_name=REGION)35s3 = boto3.client("s3", region_name=REGION)36 37ROOT = Path(__file__).parent38STATE_PATH = ROOT / "bedrock-state.json" # gitignored — local resume state39state = json.loads(STATE_PATH.read_text()) if STATE_PATH.exists() else {}40 41 42def save_state() -> None:43 STATE_PATH.write_text(json.dumps(state, indent=2))44 45 46def start_video_job() -> str:47 model_input = {48 "inputType": "video",49 "video": {50 "mediaSource": {51 "s3Location": {52 "uri": f"s3://{BUCKET}/lab/input/clip-01-notebook.mp4",53 "bucketOwner": ACCOUNT_ID,54 }55 },56 "segmentation": {"method": "fixed", "fixed": {"durationSec": 6}},57 "embeddingOption": ["visual"],58 "embeddingScope": ["clip"],59 },60 }61 request_key = hashlib.sha256(json.dumps({"model": MARENGO, "input": model_input,62 "dimension": EXPECTED_DIM, "region": REGION, "output": f"s3://{BUCKET}/lab/output/"}, sort_keys=True).encode()).hexdigest()63 # Legacy state has no request hash: retain its IDs for this original fixed request.64 if state.get("request_key") not in (None, request_key):65 raise RuntimeError("Bedrock config changed — preserve old state and use a separate state file")66 state["request_key"] = request_key67 if "invocation_arn" in state:68 return state["invocation_arn"]69 state.setdefault("client_token", str(uuid.uuid4()))70 save_state() # token and request identity must exist BEFORE submission71 response = runtime.start_async_invoke(72 modelId=MARENGO,73 modelInput=model_input,74 clientRequestToken=state["client_token"],75 outputDataConfig={"s3OutputDataConfig": {"s3Uri": f"s3://{BUCKET}/lab/output/"}},76 )77 state["invocation_arn"] = response["invocationArn"]78 save_state() # ARN persisted before any polling79 return state["invocation_arn"]80 81 82def wait_job(arn: str, timeout_sec: int = 1800) -> dict:83 deadline = time.time() + timeout_sec84 delay = 1085 while time.time() < deadline:86 status = runtime.get_async_invoke(invocationArn=arn)87 state_name = status.get("status", "Unknown")88 if state_name == "Completed":89 return status90 if state_name in ("Failed", "Expired"):91 raise RuntimeError(f"async invoke {state_name}: {status}")92 time.sleep(delay)93 delay = min(delay * 2, 60)94 raise TimeoutError(95 "Local timeout — the remote job may still be running. "96 "Re-run this script to resume the same invocation ARN."97 )98 99 100def discover_output(final: dict) -> tuple[str, str]:101 """Read the completed job's output S3 location from its metadata."""102 uri = final["outputDataConfig"]["s3OutputDataConfig"]["s3Uri"]103 parsed = urlparse(uri)104 return parsed.netloc, parsed.path.lstrip("/")105 106 107def data_entries(payload: dict) -> list[dict]:108 """AWS documents data as one object or a list when multiple vectors exist."""109 data = payload.get("data")110 entries = [data] if isinstance(data, dict) else data111 if not isinstance(entries, list) or not entries or any(not isinstance(x, dict) for x in entries):112 raise RuntimeError("unexpected embedding data shape")113 return entries114 115 116def validate_vector(vector) -> list[float]:117 if not isinstance(vector, list) or len(vector) != EXPECTED_DIM:118 raise RuntimeError(f"expected {EXPECTED_DIM}-dim vector")119 normalize(vector)120 return vector121 122 123def parse_segments(payload: dict) -> list[dict]:124 segments = []125 for i, seg in enumerate(data_entries(payload)):126 start, end = seg.get("startSec"), seg.get("endSec")127 if not all(isinstance(x, (int, float)) and not isinstance(x, bool) and math.isfinite(x) for x in (start, end)) or start < 0 or end <= start:128 raise RuntimeError(f"Bedrock segment {i}: invalid startSec/endSec")129 segments.append({"clip_id": "clip-01-notebook", "start_sec": float(start),130 "end_sec": float(end), "embedding": validate_vector(seg.get("embedding")),131 "embedding_option": seg.get("embeddingOption"), "embedding_scope": seg.get("embeddingScope")})132 return segments133 134 135def download_and_parse(bucket: str, prefix: str) -> list[dict]:136 """Discover all pages of JSON output, download and validate actual timestamps."""137 keys = []138 token = None139 while True:140 options = {"Bucket": bucket, "Prefix": prefix}141 if token:142 options["ContinuationToken"] = token143 listing = s3.list_objects_v2(**options)144 keys.extend(o["Key"] for o in listing.get("Contents", []) if o["Key"].endswith(".json"))145 if not listing.get("IsTruncated"):146 break147 token = listing.get("NextContinuationToken")148 if not token:149 raise RuntimeError("truncated S3 listing has no continuation token")150 if not keys:151 raise RuntimeError(f"no JSON output under s3://{bucket}/{prefix}")152 segments = []153 for key in keys:154 payload = json.loads(s3.get_object(Bucket=bucket, Key=key)["Body"].read())155 segments.extend(parse_segments(payload))156 return segments157 158 159def embed_text(query: str) -> list[float]:160 """Synchronous text embedding; the vector lives under data, not embedding."""161 response = runtime.invoke_model(162 modelId=MARENGO,163 contentType="application/json",164 accept="application/json",165 body=json.dumps({"inputType": "text", "text": {"inputText": query}}),166 )167 result = json.loads(response["body"].read())168 entries = data_entries(result)169 if len(entries) != 1:170 raise RuntimeError("expected one text query vector")171 return validate_vector(entries[0].get("embedding"))172 173 174if __name__ == "__main__":175 arn = start_video_job()176 final = wait_job(arn)177 bucket, prefix = discover_output(final)178 segments = download_and_parse(bucket, prefix)179 out = ROOT / "local-results" / "bedrock-clip-01-vectors.json"180 out.parent.mkdir(exist_ok=True)181 out.write_text(json.dumps({"model": MARENGO, "dimension": EXPECTED_DIM,182 "segments": segments}, indent=2))183 print(f"{len(segments)} segment vectors saved to {out}")184 vector = embed_text("a person opens a blue notebook")185 # Rank this Bedrock 3.0 corpus against its own Bedrock 3.0 query.186 for hit in rank(segments, vector):187 print(f"{hit['score']:.3f} {hit['clip_id']} {hit['start_sec']:.1f}-{hit['end_sec']:.1f}s")What actually changed
| Aspect | Direct TwelveLabs API | Amazon Bedrock |
|---|---|---|
| Model versions | Pegasus 1.6, Marengo 3.5 | Pegasus 1.5, Marengo 3.0 embeddings |
| Auth | Single API key | IAM credentials, profiles, SSO |
| Media location | Uploaded to TwelveLabs assets | Your private S3 bucket |
| Payload style | SDK methods | JSON bodies, Bedrock casing, model IDs |
| Billing | TwelveLabs plans, storage + Workload | AWS invoice, per-second/per-request + S3 |
| Results | API responses | S3 output prefix you control |
Expected checks
- aws sts get-caller-identity shows a non-root identity.
- The bucket blocks all public access.
- bedrock-state.json holds the client token and invocation ARN; re-running resumes instead of resubmitting.
- The Pegasus response's stopReason was checked before the output was trusted.
- Your Budgets alert exists — and you know it warns rather than caps.
Exercise
Before running anything, write down your predicted total cost for this chapter from the live pricing page: video seconds for the embedding job, one text query, one Pegasus analysis (if priced), S3 storage for one clip for a day, and requests. After running, compare with the actual Billing console line items. The gap between prediction and invoice is the real lesson.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| AccessDeniedException on invoke | Model access not enabled | Follow the current model-access guide; check Marketplace subscription status |
| ValidationException on the body | Casing from the wrong docs | Re-read the Pegasus 1.5 / Marengo 3.0 parameter pages — direct-API shapes do not transfer |
| Async job failed with S3 error | Bedrock cannot write the output prefix | Check bucket ownership, region, and the service's write path |
| Could not find credentials | SSO session expired | aws sso login --profile twelvelabs-video-lab again |
| Unexpected line item on the bill | Subscription or storage left behind | See cleanup below; cancelling a subscription does not delete saved storage |
Checkpoint commit and cleanup
1git status && git diff --cached2git add bedrock_analyze.py bedrock_embed.py3git commit -m "Chapter 5: Pegasus 1.5 and Marengo 3.0 through Amazon Bedrock"4git pushCleanup deletes only what this lab created: objects under lab/input/ and lab/output/ (check object versions if you enabled versioning), then the bucket if otherwise empty. Review the Marketplace subscription state — cancelling does not remove saved storage. Keep or remove the Budgets alert deliberately. Every deletion names a specific lab resource; no account-wide destructive commands.
References
Official sources used for this chapter
Final chapter: turn the pile of JSON into a workbench you can sit in front of — a fixture-first local UI with human review — and then choose the right product surface for real work.