Welcome to the SPEC Embedded Group Newsletter Q4-2024! EEMBC (Embedded Microprocessor Benchmark Consortium) was merged into SPEC (Standard Performance Evaluation Corporation) as the Embedded Group in October 2023, and we are now up and running. We are also looking for interested companies and organizations to join the group to help us make our Embedded Group benchmarks even better. You can find more information on the Embedded Group, the EEMBC/SPEC merger, and SPEC on our website.  

The AudioMark Benchmark Maintenance Update 

The AudioMark benchmark is the first-of-its-kind audio benchmark that incorporates advanced signal processing, multiple data types, and a convolutional neural net in a single benchmark with a realistic code footprint. It has been released for only a year, and we are seeing more developers using it. Based on feedback, we have delivered a maintenance release that includes:

• Bug fixes
• Improvement of floating-point handling
• Improvement of unit tests
• Corrections and improvements in documentation

The AudioMark benchmark continues to be available on GitHub. 

The ULPMark Benchmark Maintenance Update 

The ULPMark benchmark quantifies the many aspects of ultra-low power MCUs, and we are currently working on an update of the IoT Connect software (the host runner for ULPMark) to support the STLink-V3PWR for power measurement. Once this update is available, the work to add STLink-V3PWR support will be extended to the IoT Connect software for IoTMark. 

We are also aware that the Joulescope JS110 used by the IoT Connect 3 software in IoTMark is being phased out and replaced by Joulescope JS220. Due to differences in the hardware, the software driver will need to be updated, and work will begin soon.

The AutoBench 1.x Benchmark Suite Maintenance Update

The AutoBench benchmark suite enables users to predict the performance of microprocessors and microcontrollers in automotive, industrial, and general-purpose applications. 

While the AutoBench 2.0 benchmark suite has been available for many years, many users still use version 1.1 for performance analysis on small, single-processor systems because it was designed for bare metal environments and has a smaller memory footprint. 

However, the official 1.1 release of the benchmark suite has known bugs that were fixed in the 2.0 version. As a result, we are working to deliver an AutoBench 1.2 benchmark suite, which will include the bug fixes. 

What’s coming next? 

Besides the maintenance of existing benchmarks, we are resuming the development of next-generation benchmarks. For example, one idea is to create an updated version of CoreMark to cover advanced processing requirements in modern microcontrollers systems (dubbed CoreMark 3). We actually started investigating this back in 2022, but the project was suspended to enable us to focus on the completion of the AudioMark benchmark. Now that the AudioMark benchmark is released, we look forward to resuming this project. 

SPEC Embedded Group Benchmarks in Academia

A large portion of embedded benchmark users are academics who use SPEC benchmarks for research. SPEC Embedded Group is continuing to collaborate with academic institutions. For example, the following institutions have licensed SPEC Embedded Group benchmarks: 



• At Tsinghua University, Prof Hu He’s team is focused on processor design, and they have developed a dual-core lockstep 32-bit RISC-V processor for automotive applications. The AutoBench benchmark suite is used to measure its performance. 
• The Department of Computer Science & Engineering in the Indian Institute of Technology, Hyderabad (https://www.cse.iith.ac.in/), has licensed a range of SPEC Embedded Group benchmarks. One of the current focuses of the Compilers team (https://compilers.cse.iith.ac.in/) is compilation for embedded devices (https://compilers.cse.iith.ac.in/research/). The team also plans to use SPEC Embedded Group benchmarks to test the performance improvements of ITTH’s compiler optimizations and the robustness of their toolchains.
• Huazhong University of Science and Technology (http://faculty.hust.edu.cn/liuzhenglin1/en/index.htm) has licensed the ULPMark benchmark to measure the power characteristics of an ultra-low-power chip equipped with the ARM Cortex-M0 and their own designed 8T-SRAM, which can operate stably in the near-threshold region. This low-power technology enables microcontrollers to achieve much better energy efficiency when compared to traditional semiconductor processes. To demonstrate the benefits, they want to compare the energy consumption of their design with similar products using ULPMark. 

Join SPEC Embedded Group!

To enable us to continue developing innovative benchmarks such as the CoreMark 3 benchmark, we invite more companies and organizations to join the SPEC Embedded Group. By joining the group, you’ll not only help shape the future of benchmarks for embedded systems, but also get access to previous EEMBC benchmarks. Please visit SPEC for more details about joining SPEC Embedded Group.