AI Technology
Moving beyond cloud to the edge: edge AI servers
By Antonios Tsetsos, product sales manager, Advantech Europe I
n recent years, several trends have driven significant changes in the system architecture of edge AI applications. These trends include the rapid increase in AIoT data volume, improvements in hardware performance, and a growing focus on green and low-carbon initiatives. As more enterprises shift AI model training from the cloud to the edge, the demand for edge AI servers has grown substantially. Historically, enterprises conducted AI model training in the cloud, then deployed the trained models to the edge for inference, periodically sending terminal data and prediction results back to the cloud. However, advancements in hardware technology and the increased computational power of edge devices now make it possible to meet the computational demands of AI model training at the edge. Furthermore, the rapid increase in AIoT data volume has significantly raised the cost of transmitting data from the edge to the cloud. This shift has led enterprises to explore performing AI model training at the edge. In response, Advantech has developed a comprehensive edge AI server solution by integrating software, hardware, and services, helping enterprises leverage AI at a reasonable price.
Should AI models be trained in the cloud or at the edge?
Tony Kuo, product manager of Advantech’s Embedded IoT Business Group, suggests that enterprises should decide whether to train AI models in the cloud or at the edge based on several factors: the type of AI application, the size of the AI model parameters, the data volume, and the level of data confidentiality.
High-speed cloud computing is preferable for AI models with large parameters or where edge computing power is insufficient, as both scenarios can prolong fine-tune training times. Additionally, uploading highly confidential enterprise data to the cloud is generally not advisable. In cases
48 October 2024 Components in Electronics
www.cieonline.co.uk
where the data for fine-tuning an AI model are too numerous to upload, edge devices can handle AI data mining or model fine- tuning, thus avoiding high transmission costs.
In the case of generative AI applications, enterprises are not only developing customer service chatbots but also integrating knowledge management systems, equipment maintenance manuals, and other data sources to optimize work efficiency. This integration speeds up data retrieval and helps new engineers quickly adapt to their roles. Since internal data is usually confidential and unsuitable for cloud
upload, enterprises can deploy edge AI servers to effectively retrain large language models (LLMs) on-site.
On the other hand, fine-tuning large language models (LLMs) in generative AI (GenAI), consumes a substantial amount of memory (VRAM). If the VRAM capacity is insufficient, it becomes impossible to fine- tune the LLMs, necessitating the purchase of additional expensive GPU cards to expand VRAM capacity. This is a significant cost burden for most companies. Therefore, it is crucial to reduce the cost of VRAM expansion required by the ever-growing parameters of generative AI models while
ensuring data security and confidentiality. This is essential for the rapid adoption of generative AI applications.
Three keys to a comprehensive solution: hardware, software, and services
To meet the growing enterprise demand for AI model training and inferencing at the edge, Advantech has developed the AIR-500 series of edge AI servers. These servers feature high- frequency, high-performance hardware and are complemented by Advantech’s integrated software and services. By combining these three key elements, Advantech has created
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64
