Muhtasham's comments

On: Benchmarking Llama 3.1 405B on 8x AMD MI300X GPUs

[Muhtasham]

Thanks for detailed analysis, would be curios to see FP8 comparison too, given vllm has some custom kernels

[cheptsov → Muhtasham]

Unfortunately, neither VLLM nor TGI support FP8 on AMD yet. But once they do, we will look into it.

On: How Meta trains large language models at scale

[Muhtasham]

it’s serious crime

On: Show HN: Dstack – an open-source engine for runnin...

[Muhtasham]

Super useful for spinning LLM fine-tuning jobs!