Estimate the KV-cache memory footprint for transformer inference. Inputs include number of layers, number of heads, head dimension, sequence length, batch size, and precision.
Enter the values for the patient or scenario you are assessing. Estimate the KV-cache memory footprint for transformer inference. Inputs include number of layers, number of heads, head dimension, sequence length, batch size, and precision. Use the kv-cache memory result to inform your clinical assessment.