AI/ML Intern — Fox Corporation
Jan 2, 2026 – Jul 1, 2026Personalization & Recommendation Team · Databricks Platform
- Built and deployed Databricks/PySpark pipelines processing 700K+ media assets (shorts, clips, listings, episodes, movies, specials) for metadata enrichment, semantic retrieval, content discovery, and recommendation workflows.
- Cut recompute cost and runtime with a two-tier hash-based CDC layer that reprocessed only changed rows, plus deduplication, caching, and idempotent 2-phase Delta MERGE upserts.
- Built LLM-assisted extraction using GPT-5-mini with web search to generate 20+ structured metadata fields (entities, themes, genres, teams/leagues, talent) from ASR transcripts and catalog text.
- Generated 3072-dim Gemini embeddings powering similarity, retrieval, and discovery across the catalog, sustaining ~1000 rows/min via a 300-thread concurrent pool and batched API calls.
- Hardened reliability with validation checks and batch-level comparisons that caught regressions before production, plus run-log debugging and monitoring on scheduled Databricks workflows.
- Collaborated with engineering and product stakeholders to turn ambiguous requirements into reusable, well-documented pipeline components with measurable, reliable outputs.