IORS Virtual Session: Results presentation of i14y Lab / VALOR collaboration

Comparing the results showed variations that were within an acceptable range, but also some that required closer examination. Analysis of the results and further discussions led to a set of recommendations touching on various aspects of testing: 

• Test bed specifications need to be considered as part of the test plan’s requirements as, for example, the 5G core used can very well impact the performance of the SUT significantly.
• For some test cases, more explicit details on what to test are required to make results comparable, such as, for example, the number of UEs required for a test case.
• Test plans should not require cases that my introduce variations because of behaviors inherent to the technology and not the SUT. 

The IORS virtual session also included a lively discussion with the attendants, where many detailed questions could be answered. The number of attendants and their involvement showed the relevance of the topic. That was also reflected in the interest expressed by i14y Lab and VALOR as well as representatives of other labs present in the audience to continue this work, and in a summary of recommendations rounding out the presentation:

• Establish repeatable and reliable testing methodologies through standardized, reproducible test procedures.
• Benchmark performance across environments through comparing results from independent lab setups.
• Create a knowledge-sharing framework to document learning and share insights to benefit industry stakeholders.
• Support global standardization efforts through aligning with O-RAN Alliance, TIP, and 3GPP to drive a repeatable and reliable testing framework.

Recommendations that are all meant to enable faster iteration and deployment of tested and validated 5G components in real-world scenarios.