Citation :

Arjun Mehta, 2026. "Balancing Inference Speed and Energy Consumption in Edge AI Applications through Workload-Aware Model Selection" ESP International Journal of Advancements in Science & Technology (ESP-IJAST)  Volume 4, Issue 3: 01-17.

Abstract :

Edge Artificial Intelligence (Edge AI) allows developers to build smart applications that can make decisions using local data on their devices. This means there is no need to wait for communication with a central server; developers can protect users' private information; and developers don't have to rely on expensive cloud services. However, it can be difficult to deploy AI models on devices with limited capabilities (such as small screens, low battery life, etc.) due to the trade-offs between inference speed, energy usage, memory constraints and accuracy of the predictions. To address these trade-offs, this article explores how to use workload-aware model selection to help develop more efficient Edge AI applications. To do so, we propose a dynamic model selection process based upon characteristics of the workload. Some examples of workload characteristics include: complexity of the inputs to the model; how frequently the model processes data; the time required for the model to produce results; and the amount of resources available on the device. When selecting a model to use, we prioritize models that require less computing power for simpler workloads and/or when the device has limited resources. On the other hand, if a workload requires complex input processing and/or high levels of accuracy, then we will choose a model that requires more computing power even though it uses more energy. In addition to considering the workload characteristics described above, we also consider other factors when choosing a model to use. For example, we consider how well the hardware is being used; what percent of battery remains; and what limits exist regarding the latency of the system. Considering all of these factors together allows us to create dynamic decision-making mechanisms for Edge AI systems. By matching the complexity of the model with the actual needs of the workload, Edge devices can avoid wasting valuable computing power and therefore maximize the use of the available resources. In addition to describing our dynamic model selection methodology, this paper also includes discussions regarding: model selection methodologies; challenges encountered during implementation of our methodology; metrics used to evaluate the success of our dynamic model selection methodology; and potential uses for Edge AI systems in various domains such as smart cities; industrial automation; healthcare monitoring; and intelligent transportation. Our proposed dynamic model selection methodology establishes a solid base for creating Edge AI systems that are both energy-efficient and highly responsive. Additionally, we discuss several directions for further research into Edge AI systems including adaptive optimization; hardware-aware scheduling; and intelligent resource management in heterogeneous edge environments.

References :

[1] Shi, W., Cao, J., Zhang, Q., Li, Y. and Xu, L. (2016) ‘Edge computing: Vision and challenges’, IEEE Internet of Things Journal, 3(5), pp. 637–646.

[2] Satyanarayanan, M. (2017) ‘The emergence of edge computing’, Computer, 50(1), pp. 30–39.

[3] Chen, J. and Ran, X. (2019) ‘Deep learning with edge computing: A review’, Proceedings of the IEEE, 107(8), pp. 1655–1674.

[4] Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K. and Zhang, J. (2019) ‘Edge intelligence: Paving the last mile of artificial intelligence with edge computing’, Proceedings of the IEEE, 107(8), pp. 1738–1762.

[5] Mao, Y., You, C., Zhang, J., Huang, K. and Letaief, K.B. (2017) ‘A survey on mobile edge computing: The communication perspective’, IEEE Communications Surveys & Tutorials, 19(4), pp. 2322–2358.

[6] Sze, V., Chen, Y.-H., Yang, T.-J. and Emer, J.S. (2017) ‘Efficient processing of deep neural networks: A tutorial and survey’, Proceedings of the IEEE, 105(12), pp. 2295–2329.

[7] Han, S., Mao, H. and Dally, W.J. (2016) ‘Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding’, in Proceedings of the International Conference on Learning Representations.

[8] Hinton, G., Vinyals, O. and Dean, J. (2015) ‘Distilling the knowledge in a neural network’, NIPS Deep Learning and Representation Learning Workshop.

[9] Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M. and Adam, H. (2017) ‘MobileNets: Efficient convolutional neural networks for mobile vision applications’, arXiv preprint, arXiv:1704.04861.

[10] Sandler, M., Howard, A., Zhu, M., Zhmoginov, A. and Chen, L.-C. (2018) ‘MobileNetV2: Inverted residuals and linear bottlenecks’, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4510–4520.

[11] Zhang, X., Zhou, X., Lin, M. and Sun, J. (2018) ‘ShuffleNet: An extremely efficient convolutional neural network for mobile devices’, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6848–6856.

[12] Ma, N., Zhang, X., Zheng, H.-T. and Sun, J. (2018) ‘ShuffleNet V2: Practical guidelines for efficient CNN architecture design’, in Proceedings of the European Conference on Computer Vision, pp. 116–131.

[13] Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J. and Keutzer, K. (2016) ‘SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and less than 0.5 MB model size’, arXiv preprint, arXiv:1602.07360.

[14] Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H. and Kalenichenko, D. (2018) ‘Quantization and training of neural networks for efficient integer-arithmetic-only inference’, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2704–2713.

[15] Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M.A. and Dally, W.J. (2016) ‘EIE: Efficient inference engine on compressed deep neural network’, in Proceedings of the ACM/IEEE 43rd Annual International Symposium on Computer Architecture, pp. 243–254.

[16] Chen, Y.-H., Krishna, T., Emer, J.S. and Sze, V. (2017) ‘Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks’, IEEE Journal of Solid-State Circuits, 52(1), pp. 127–138.

[17] Jouppi, N.P. et al. (2017) ‘In-datacenter performance analysis of a tensor processing unit’, in Proceedings of the ACM/IEEE 44th Annual International Symposium on Computer Architecture, pp. 1–12.

[18] Li, E., Zhou, Z. and Chen, X. (2018) ‘Edge intelligence: On-demand deep learning model co-inference with device-edge synergy’, in Proceedings of the Workshop on Mobile Edge Communications, pp. 31–36.

[19] Teerapittayanon, S., McDanel, B. and Kung, H.T. (2017) ‘Distributed deep neural networks over the cloud, the edge and end devices’, in Proceedings of the IEEE International Conference on Distributed Computing Systems, pp. 328–339.

[20] Kang, Y. et al. (2017) ‘Neurosurgeon: Collaborative intelligence between the cloud and mobile edge’, in Proceedings of the 22nd International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 615–629.

[21] Cai, H., Zhu, L. and Han, S. (2019) ‘ProxylessNAS: Direct neural architecture search on target task and hardware’, in Proceedings of the International Conference on Learning Representations.

[22] Tan, M. and Le, Q. (2019) ‘EfficientNet: Rethinking model scaling for convolutional neural networks’, in Proceedings of the International Conference on Machine Learning, pp. 6105–6114.

[23] Han, S., Pool, J., Tran, J. and Dally, W.J. (2015) ‘Learning both weights and connections for efficient neural networks’, in Advances in Neural Information Processing Systems, 28, pp. 1135–1143.

[24] Wang, Z., Luo, T., Goh, R.S.M. and Zhou, J.T. (2022) ‘EDCompress: Energy-aware model compression for dataflows’, IEEE Transactions on Neural Networks and Learning Systems.

[25] Hanafy, W.A., Molom-Ochir, T. and Shenoy, R. (2021) ‘Design considerations for energy-efficient inference on edge devices’, in Proceedings of the Twelfth ACM International Conference on Future Energy Systems.

[26] Zawish, M., Davy, S. and Abraham, L. (2024) ‘Complexity-driven model compression for resource-constrained deep learning on edge’, IEEE Transactions on Artificial Intelligence, 5(8), pp. 3886–3901.

[27] Balaji, R. (2025) ‘Autonomous SRE: SLO-Aware Multi-Agent Root-Cause Analysis and Verified Self-Healing for Cloud-Native Systems’, ESP Journal of Engineering & Technology Advancements, 5(4), pp. 211-221. doi: 10.56472/25832646/JETA-V5I4P128.

[28] Wang, S. et al. (2019) ‘Benchmarking TPU, GPU, and CPU platforms for deep learning’, arXiv preprint, arXiv:1907.10701.

[29] Howard, A. et al. (2019) ‘Searching for MobileNetV3’, in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1314–1324.

[30] Kundu, A., Wynter, L., Lee, R.D. and Bathen, L.A. (2023) ‘Transfer-Once-For-All: AI model optimization for edge’, in Proceedings of the ACM/IEEE Symposium on Edge Computing.

[31] Zeng, L., Li, E., Zhou, Z. and Chen, X. (2019) ‘Boomerang: On-demand cooperative deep neural network inference for edge intelligence on the industrial Internet of Things’, IEEE Network, 33(5), pp. 96–103.

Keywords :

Edge AI, Workload-Aware Model Selection, Inference Speed, Energy Consumption, Lightweight Models, Model Optimization, Edge Computing, Resource Management, Dynamic Adaptation, Computational Efficiency