How do you figure out what to search for when implementing something like malloc?

Start by breaking malloc into jobs. It needs to obtain a region of memory, keep track of which parts are available, return a suitably aligned block, and eventually make freed blocks available again. Each job gives you a more specific search.

Try phrases like “how does a memory allocator get memory from the OS”, “free list allocator”, and “memory allocator alignment”. On Unix-like systems, look at the manual page for mmap. On Windows, look up VirtualAlloc. Those aren't complete malloc implementations, but they give you a way to investigate where an allocator can get memory.

A data structures course is a good place to build the vocabulary. Pointers, linked lists, and ways to organize free space are useful background for a basic allocator. Don't get tripped up by the word heap, though. The memory area associated with allocation and the heap data structure are different concepts.

CS50 and MIT's 6.006 are options for fundamentals. Neither replaces reading about allocators and your operating system, but they can make the follow-up searches less mysterious.

Look for an existing implementation explained in a book, then trace it alongside the code. The C Programming Language includes a simple allocator example. Start with what each function promises to do, then work through how it tracks and reuses blocks.

After that, try building a small version yourself and test allocations and frees in different orders. A teaching example is a starting point, not necessarily something to substitute for a production allocator.

The same search habit works for the video-call example, too. “Build a video call server” covers a lot of ground. Search separately for how participants find each other, how a connection is established, and how audio and video are transmitted. Each piece will lead to more specific terminology and documentation.

You can still use a language model to suggest terms, but check those terms against manuals, books, or other technical references rather than treating its explanation as the final source.