01
Area-scalable memory
Python models of LLM-inference throughput, latency, and memory under competing chip memory organizations. The work is with Stanford’s Engineering Robust Systems Group (Prof. Subhasish Mitra).
The models relate capacity, bandwidth density, and mesh or network-on-chip topology to batch size and context length. They compare a mesh-distributed, area-scalable backend with HBM under tensor- and pipeline-parallel plans, and report compute, memory, and interconnect in bandwidth-bound long-context KV-cache regimes.
Python