I have lecture recordings along with the original PowerPoint files. I'd like to build a tool that detects when each slide appears in the video and outputs ranges such as "Slide 1: 0:00–11:03" and "Slide 2: 11:03–15:42." The recordings are 1080p screen captures, so the slides are usually nearly identical to the PowerPoint versions, aside from minor color or compression differences. Each lecture may contain 50–100 slides, and I process roughly one lecture per day. My larger goal is to use the timestamps to match transcript sections to the correct slide notes, rather than manually copying transcript text into each slide. How challenging would the timestamp-detection portion be, and what tools or libraries would be appropriate?
3 Answers
The core problem is fairly manageable if you already know some Python and basic image processing. First convert the PowerPoint into individual slide images, then use a video library to inspect frames and an image-comparison library to measure how closely each frame matches each slide. You probably don’t need to analyze every frame; sampling once or twice per second and refining around detected changes should be plenty for this task. Python is likely more convenient than JavaScript because of the available video and computer-vision tools.
The straightforward case could be prototyped quickly, but the edge cases are where the work goes. A lecturer might briefly show a previous slide, skip around, pause during a transition, or display animations and overlays. If the recording is simply a stable screen capture with mostly static slides, detecting changes and matching them against exported slide images should be quite reliable. If you were starting from scratch with no image-processing experience, expect more than a one-hour project because you’ll need testing, transition handling, and confidence checks.
A practical approach would be to export the PowerPoint slides as images, sample frames from the video, and compare each frame with the slide images using an image-similarity method. You could record the first and last timestamps where each slide is the closest match, then refine the boundaries around transitions. Since this is a clean screen recording, the comparison should be much easier than recognizing slides filmed from a projector.
That matches my use case. The slides are usually almost exactly the same as the recording, and around one-second accuracy would be enough. I’m mainly trying to automate the repetitive timestamp step before matching transcript text to the slides.
Since this is a direct screen recording rather than a camera pointed at a projection, skew and perspective correction shouldn’t be necessary. Small differences from color, compression, or cursor movement can usually be handled with a similarity threshold or by comparing resized grayscale images.