Diego Didona is now a research engineer in the Von Neumann Research Center of Huawei in Zurich, Switzerland. His research mainly focuses on accelerating inference for LLM and video generation workloads on the Huawei Ascend NPU chips.
Here is what Diego had to say about their work and the future of performance engineering:
Could you briefly summarize the core contribution of your paper and what you are most proud of regarding its impact?
Our award-winning paper shows that analytical modeling and machine learning can be used synergistically to enhance performance prediction. Analytical modeling can provide a good guess of the performance of a system given tuning parameters and a workload characterization. Machine learning can complement that prediction by correcting it on the basis of real measurements obtained from the live system. I am very happy that people are not only reading and citing the paper to frame their work in the landscape of performance prediction, but they actually draw inspiration from it to come up with newer and more robust ways to combine the two approaches.
Looking back after ten years, did the impact or evolution of the work surprise you in any way? Is there anything you would approach differently if you were to do the work today?
I am genuinely surprised the work gained so much traction: other very established groups from academia and industry were working on similar topics, so I was afraid they would get all the attention. If I could do the work all over again, I would try to get another one or two systems to see how the results generalize more broadly, and if some patterns emerge to determine a more principled approach to combine the different approaches.
If you could send one message back to yourself while writing this paper, what would it be?
I would suggest to start writing the paper much more in advance. This is one of those suggestions that you read everywhere in “PhD student guidelines”, but that is hard to follow due to the amount of work required to write a paper. However, when forced to explain your ideas and results clearly for other people you find weak spots in your designs that need to be strengthened to survive the peer review, as well as blind spots in your evaluation.
What exciting new challenges are you and your team focusing on at the moment?
The biggest challenge in improving performance for LLM inference and video generation is hitting the sweet spot between approximated computation to gain efficiency vs accuracy loss induced by the approximation. It is a never-ending battle that is made even more challenging by the stochastic nature of the text/video generation models. You never know how removing even one bit will change the output.
In your opinion, what emerging trends will dominate the field over the next few years?
It’s pretty clear that much of performance engineering is going to be offloaded to autonomous agents. I think, however, that the human will be in the loop at least for the foreseeable future. Sometimes, agents cut corners that a human knows should not be cut, or they implicitly over-fit a performance solution to a specific workload/benchmark. Even worse, agents can not only tune parameters but also completely change code, with implications on safety and correctness. I think that progressively more effort is going to be spent in making autonomous performance engineering safe, reliable, and robust, rather than just more effective to squeeze out the last few percent of performance.
What is your top advice for junior researchers submitting to ICPE 2027?
I would say: use agents to find all related work, and to polish your manuscript but do yourself (and all of us!) a favor: write the paper by yourself and don’t rely on machine learning to identify for you the next research idea. It’s pretty clear that (publicly) available models are not yet capable of making groundbreaking discoveries, and we need our next generation of researchers to work on groundbreaking results, not only incremental ones.
What is one ICPE memory that has stayed with you?
When I attended ICPE, I both presented the paper and a tutorial. The tutorial was a several-hour presentation on machine learning, analytical modeling, and their combination for performance prediction. Preparing the slides, rehearsing, and presenting them was a real ordeal but boy was it worth it.