How We Make PLAYGRND Videos: Ballr, Follow Cam and Linked Goals

I want a football video to live next to the match it belongs to. Open the result, find your name in the timeline, tap the goal and watch the move that led to it. A recording sitting in a folder cannot quite do that.

That is what I am working towards with the PLAYGRND video pilot. There is already a useful workflow, and there is still a fair amount of inspection, correction and glue code behind it. The interesting engineering is in the space between "the camera recorded the pitch" and "this button opens the right goal".

After the match

I manually put the recording on Drive, then give ChatGPT that link and the PLAYGRND match link. I add which videos I want and any instructions for the edit.

1. 1

   I record the match

   Me

   I set up Ballr and record the panorama with the sound from the pitch.

   Ballr · 6976 × 2160 · 30 FPS

2. 2

   I put the original on Drive

   Me

   After the match, I manually transfer the recording to Google Drive. The original stays untouched.

   Google Drive · manual upload

3. 3

   I send ChatGPT two links

   Me

   I send the Drive recording and the match link from PLAYGRND. I add which videos I want and anything to watch out for in the edit.

   Drive recording + PLAYGRND match

4. 4

   ChatGPT prepares the edit

   ChatGPT + local tools

   It downloads the source, reads the match details and runs the analysis. The ball detector, scoreboard changes and audio help narrow down the goal times. The camera plan keeps the developing play in view.

   Core ML / YOLO11 · Python · NumPy · SciPy · cached match data

5. 5

   Rendering and checking the videos

   Local tools + ChatGPT

   The scripts add the graphics and make the match video, highlights and goal clips. ffprobe and FFmpeg check the files. ChatGPT inspects sampled frames and the timing reports, then corrects what those checks expose.

   Swift · Metal · VideoToolbox · Pillow · FFmpeg / libx264 · ffprobe

   ChatGPT sends timing or framing corrections back to the camera plan.

6. 6

   I get a local link to watch

   Me

   I watch the encoded videos in a player on the working machine's LAN. If a cut, timestamp or camera move needs work, I send it back to ChatGPT and watch the next version.

   Local HTTP player · seeking through the video

   My feedback goes back for another render and a new local preview.

7. 7

   I approve the files and the upload

   Me, then ChatGPT

   Once I approve the exact files and ask for publication, ChatGPT uses the available APIs or browser to upload them. It checks the resulting links and attaches them to the match and its goals.

   YouTube · Drive · selected social clips · PLAYGRND event IDs

ChatGPT checks sampled frames and the results from the scripts before sending me a local player link. The ball detector and renderer run on the Mac. I still need to watch the encoded videos in motion.

If I want a cut or camera move changed, I send the comment back. ChatGPT adjusts the edit, renders it again and gives me another preview. The player runs on the working machine's LAN, so I need a network path to that machine to open it. Publication waits until I approve the files and ask for the upload.

The model is prepared separately with PyTorch, Ultralytics and coremltools, then reused by Core ML. OpenCV and FilterPy are other tracking adapters in the research code. The job records which path it used.

The latest example is Bajamonti–Ponediljak 10, 2:4, from 9 November 2026. On that page you can find the original panorama, a full match with a moving camera view and graphics, a highlights video and links on all six goals.

The camera and the stuff holding it up

I use a Ballr camera for six-a-side football. The rest of the setup is a separately bought Walimex pro AIR Jumbo 600, which extends to six metres, and a SmallRig Crab-Shaped Clamp with Ballhead Magic Arm 3757.

Ballr lists dual wide-angle 4K cameras, a 6976 × 2160 panorama, up to 180° horizontal coverage and 30 FPS, with 128 GB of local storage. The range includes Wi-Fi and 4G variants. These are the manufacturer's specifications, checked on 10 November 2026.

I use it to film 6v6 matches. Daylight is more forgiving than our night footage under floodlights. I adjust the stand height to the venue and mounting conditions.

The most useful file is the panorama. The Bajamonti source is a joined HEVC video with AAC sound, about 56:38 long. I keep it unchanged. Ballr can provide an AI-follow view too, but keeping the wide original lets me build and revise our own framing afterwards.

What each library does

Python coordinates the work, while the native workers are written in Swift. The libraries below have different jobs. The model-export tools run when preparing a checkpoint. The local workers reuse that checkpoint for a match.

Tool	Its job in this workflow
Python	Reads the job JSON, starts the workers, saves their results and decides which cached work can be reused. The per-match scripts also prepare graphics, edit intervals and delivery files.
NumPy	Holds the timestamps, detections and camera coordinates as arrays. It handles interpolation and the vectors and rotation matrices used by the camera profile.
SciPy	gaussian_filter1d smooths the camera targets. PchipInterpolator fills between reviewed framing points. Median filters help stabilize scoreboard-state proposals. These calculations turn sparse samples into a camera path.
PyTorch	Loads the original .pt detector checkpoint during model preparation and earlier experiments. The native Core ML worker uses the exported model afterwards.
Ultralytics	Supplies the YOLO model interface and export path. The example uses the existing Soccana YOLO11 checkpoint. A match job reuses it without training a new detector.
coremltools	Converts the model into an Apple .mlpackage for the native inference worker. This is a model-format step done before processing the match.
Core ML	Runs that exported detector locally on sampled images. It returns object candidates and scores. The tracker still has to decide which candidate belongs to the active ball.
AVFoundation	Opens the source video and gives the native worker decoded frames with their presentation timestamps. Those timestamps keep detections tied to the original recording.
Core Video	Provides the frame buffers used between the Apple media, Metal and Core ML components. The native path can work with those buffers without saving every frame as a JPEG.
Metal	Prepares image tiles for detection, then applies the camera projection and composites graphics during rendering. The camera plan tells it where to look for each output frame.
VideoToolbox	Provides the Mac hardware codec path used through the media stack and FFmpeg. Encoding and decoding belong here. The camera geometry is calculated separately.
OpenCV	Loads cropped images for the source-specific scoreboard classifier. Other research adapters use its Lucas–Kanade optical flow to check short tracking gaps. This scoreboard workflow compares digit shapes, rather than using a general OCR service.
FilterPy	Supplies Kalman/RTS filtering and smoothing in supporting tracking experiments. The Bajamonti camera script used NumPy/SciPy interpolation and reviewed guides, so FilterPy was not a mandatory stage of that render.
Pillow	Draws text, crests and score states into the approved PNG templates. Those images become overlay textures and covers. It reuses the template for each match.
FFmpeg	Extracts review samples, processes audio, applies the selected denoise/color filters, joins compatible segments and muxes the final file. It also decodes outputs during QA.
libx264	The software H.264 encoder used for the final HQ goal clips and highlights in this example, with the selected preset and CRF. The full match used a separate native hardware render path.
ffprobe	Reports codec, dimensions, frame rate, duration, frame count and audio timing. The scripts compare those values with the edit plan before a file is ready for review.

The source references for the library behavior are Ultralytics' Core ML export guide, SciPy's Gaussian filter, PCHIP interpolation and Pillow's drawing API. The roles in the table come from the scripts used for this job.

How I start a job

I start in chat by supplying the recording and match links. ChatGPT retrieves the source and writes the working files. The local programs receive file paths and JSON, so the Drive URL is resolved before they run.

For Bajamonti, the work was still split across scripts kept with that match. analyze_stage1.py prepared the audio analysis, review images and leveling candidates. analyze_tracking.py called the native detector. classify_board.py proposed scoreboard transitions. build_camera.py combined detections, the mount profile and inspected framing guides into camera-plan.json.

The native analyzer's interface is short. With the filenames shortened, the call used for the complete recording looks like this:

./native-analyze panorama.mp4 soccana-yolo11.mlpackage \
  detections.jsonl 0 3398.444056 3

Argument	Meaning in this call
panorama.mp4	The local original recording.
soccana-yolo11.mlpackage	The prepared detector checkpoint.
detections.jsonl	Output: one JSON record per analyzed source sample.
0	Start at source second zero.
3398.444056	Requested source duration, in seconds, from the probe receipt.
3	Sample about three source frames per second for analysis. The output video still has its own 30 FPS timeline.

The worker also accepts an optional calibration JSON after those arguments. It can restrict analysis to a configured region. The model and native binaries must already exist locally. This call does not download or train a model.

The renderer has a separate interface:

./native-metal-render panorama.mp4 camera-plan.json followcam-silent.mp4

camera-plan.json contains the output size, frame rate and codec, plus the frame-by-frame view: yaw, pitch and field of view. It also holds the lens mapping and timed overlay paths. The worker renders the image. FFmpeg adds the matching source audio and assembles the delivery file. In the match-specific job, render_package.py full rendered the full match in bounded sections, and render_clips.py prepared the short outputs. resume_full.py was the recovery path that preserved an already encoded prefix after an interrupted full render.

Those scripts read files beside them, including source-receipt.json, facts.json, timing.json and the camera plan. Most of their settings are in those files rather than command-line flags. final_qc.py checks the outputs, and serve_review.py serves the player and videos on the configured LAN address and port. It supports HTTP byte ranges so I can seek through a long match.

The reusable command in the repo

There is also a parameterized CLI, playgrnd-video, registered by the Python package. It puts the recurring settings in one job JSON. The Bajamonti delivery above still used its own scripts. This is the runner we can use as more of those steps become reusable.

playgrnd-video job init match.json \
  --source panorama.mp4 \
  --workdir work \
  --profile camera-profile.json \
  --goal-mode timestamps \
  --full landscape \
  --highlights landscape \
  --goals landscape

playgrnd-video job plan match.json
playgrnd-video job run match.json --background
playgrnd-video job status match.json

init writes the configuration. Before run, the job needs the match facts, goal anchors and paths to the analyzer, checkpoint and renderer. plan shows the intended work without decoding the video. --background returns a PID and log path. status reads progress.

Setting	What I supply
--source, --workdir, --profile	The local recording, working directory and profile for this camera position.
--goal-mode	timestamps uses supplied source anchors. minutes starts with report minutes and clock anchors. auto produces candidates for review.
--full, --highlights, --goals	Which products and orientations to render. Each accepts none, landscape, vertical or both.
analysis.fps and engine.* in the JSON	Analysis sampling rate and paths to the existing native workers and model.
source.keep_intervals, goals.events, render.*	Which parts of the source to keep, the goal records, output FPS and encoding settings.

Analysis FPS and render FPS are separate. A report minute is also separate from a source timestamp. Those distinctions are part of the configuration because an edit can preserve halftime, cut a stoppage or change a clip's lead-in.

The runner performs local computation without LLM calls. It writes progress and review files. Uploads and changes to PLAYGRND happen later, after I approve the output and authorize publication. Reliable automatic goal confirmation and venue calibration still need work.

A virtual camera inside a fixed recording

The custom follow cam is a moving view extracted from that panorama. There are three distinct jobs: find plausible action, decide how to frame it and turn that decision into pixels.

In the current Mac path, a Swift worker reads frames through AVFoundation and Core Video. Metal prepares overlapping tiles for the detector. The model in this example was a Soccana YOLO11 checkpoint exported to Core ML, with PyTorch, Ultralytics and coremltools in the preparation/export toolchain. The model card describes the upstream checkpoint. It is not a model I trained from scratch.

For this match, analysis sampled about three frames per second. Rendering then produces a 30 FPS output from the original source. An analysis sample rate is not the finished video's frame rate.

The detector gives us possible balls and people. A tracker tries to maintain the active-ball hypothesis over time. Player activity can supply a broader fallback. A white mark, a spare ball or a group of players standing around can all make the wrong target look convincing. Finding a candidate is much easier than following the right ball through an entire attack.

Python, NumPy and SciPy turn the targets into a camera path: interpolation, Gaussian smoothing, a little lookahead and limits on movement. This particular job looked about 0.65 seconds ahead in its initial target interpolation. I also have OpenCV optical-flow and FilterPy Kalman/RTS components in the research pipeline, though this match's camera script used its own interpolation and filtering path.

Selected actions received inspected framing guides. That matters when describing the result honestly: a manually specified or assistant-inspected camera target does not become an observed ball position. The attractive finished clip is a combined result of detection, planning and correction.

The camera should follow the developing play. Knowing where a goal will happen is useful for checking the finish. It is a terrible reason to pan to an empty net before the pass arrives. I want the ball and receiving goal in the shot together where practical, with a wider view before a fast change.

Why the goalposts need their own checks

A panorama is not a flat photograph that can always be cropped and rotated into a plausible broadcast view. Near its edges, a poor projection can lean the posts, stretch players or cut off heads.

For Bajamonti, the processing estimated a leveling rotation from image evidence and used a rectilinear output projection with a source-specific profile. Metal applies that transform while making the final image. The lens mapping is still an approximation to review, not proof of a perfectly calibrated camera or accurate distances on the pitch.

I check moving samples on both ends as well as at midfield. A smooth numerical path can still produce a bad picture. The relevant test is whether the pass, shot, flight and finish stay visible with believable proportions.

How we found the goal timestamps

The physical scoreboard is visible in this panorama, which gave us a useful shortcut. A script isolates it. Another compares the digit shapes with templates selected from this recording and filters the proposed states over time. That produces windows where the score changes.

This is not a general scoreboard-reading model. The crop and templates belong to this setup. A board change is also later than the ball crossing the line, so it cannot be the final timestamp by itself.

The assistant inspected more closely spaced source frames around each window, including the action, finish and return to the centre. Audio energy helped locate supporting moments. Kickoff and the end of play were checked separately. The final release still required my approval of the actual videos. Sampled image inspection is not equivalent to watching every frame move.

The first Borna Šunjić goal shows why there are several clocks:

• The report says 4′ and identifies the event that makes it 0:1.
• The reviewed source anchor is 774.267 seconds, or about 12:54.267 into the panorama.
• The full-match edit starts at source second 539, so that goal lands at about 3:55.267 in that output.
• The highlights have a seven-second intro, then the first clip starts at source second 759. The same finish lands around 22.267 seconds there.

The anchor carries an estimated ±0.2 seconds of uncertainty. That is not a benchmark proving the detector is accurate to 0.2 seconds. The report minute, source time and edited output time are stored separately. Halftime alone is enough to break the naive "minute four means second 240" approach.

The edit is a set of decisions we can rerun

Rather than drag everything into a timeline and lose the reasoning, the workflow stores the source intervals, camera revision, event anchors and graphic states in JSON. Python runs the work. The heavy image transform is native Swift/Metal. FFmpeg handles the remaining image treatment, audio and assembly. Pillow fills the existing graphic templates from cached match data and crests.

The full match keeps halftime and uses one camera revision, an opening card, an ending card and source-timed score graphics. The result changes at the reviewed finish. The scorer label appears 0.7 seconds later in this example. The opening card does not tell you the final score before play starts.

The goal clips preserve buildup and reaction, rather than starting just as the ball reaches the net. These six clips run for 20–26 seconds. The montage combines them with a seven-second opening and a twelve-second closing card, giving about 2:40 of highlights. The full match is about 47:24.5.

For this night source, the full-match native render used 25 Mb/s. Short masters used 80 Mb/s before a mild hqdn3d cleanup, a small contrast/saturation adjustment and a libx264 medium/CRF 14 final encode. This is deliberately a record of this job, not a magic preset. A stronger denoise can erase the ball, and a larger bitrate cannot restore detail the sensor never captured. The field audio stays synchronized.

The final picture comes from the original panorama, not the low-resolution images used to inspect it. Render sections and compatible clips can be reused and joined without another complete video encode. Changing one goal's title or framing should not require repeating all the analysis.

Publishing is part of the system too

Once I approved the exact files, the package went to YouTube and Drive, with selected clips also published on social platforms. The pilot combines scripts and platform APIs with browser operations where needed. Uploading, processing and becoming publicly playable are different states, so I keep publication receipts and verify actual links and visibility.

You can watch the full follow-cam match, all-goal highlights and the selected Borna clip. These links open on request. The blog does not load an embedded video player automatically.

The final connection is in PLAYGRND. The match gets three match_media records for panorama, full match and highlights. Each of the six existing goal events gets its own video_url. IDs make that association precise. The player's name, report minute and intermediate score are checks around it. We are not recognizing a scorer's face.

For this job, the small database update was rehearsed locally, guarded against changes in the match state and verified afterwards on the public page. The result and lineups stayed as they were. The video became another way to explore the same match record.

What I still do, and where I want this to go

The repeatable parts are already good jobs for software: sampling, candidate extraction, interpolation, graphic generation, rendering, packaging and technical checks. The assistant helps write bounded scripts, inspect evidence and prepare corrections. I still place the camera, make editorial choices, review the actual output and approve publication.

The next useful step is fewer special cases: reusable mount profiles, a small interface for correcting events and camera paths, and measurements on matches that were not used to tune the system. I care about keeping the whole attack visible, timestamp error and how much correction a match needs. A detector's speed on one short clip does not answer those questions.

Live output with a delay is interesting later. Ballr's RTMP support does not tell me whether I can get the full live panorama into our own renderer. The current workflow is postproduction, and there are interface, reliability and review problems to solve before changing that.

There is also work beyond image quality: checking the licences for the model code, weights, datasets and encoder build, and the rights and privacy arrangements for recording and publication. Converting a checkpoint to Core ML changes its format, not its terms. Calling this a pilot does not remove those obligations. I am not presenting the experimental stack as a cleared software distribution.

For now, Bajamonti–Ponediljak 10 is a concrete place to see what works. The more formal engineering account is in the HILLS Lab pilot note.