Inference Lab
One set of knobs, five linked views. Choose a model, a GPU and a workload, then see how memory, the decode step, the roofline and the throughput–latency trade-off all respond. Start with a scenario, then change one thing at a time.
One set of knobs, five linked views. Choose a model, a GPU and a workload, then see how memory, the decode step, the roofline and the throughput–latency trade-off all respond. Start with a scenario, then change one thing at a time.