ISO/IEC 23005-5:2016 specifies syntax and semantics of the data formats for interaction devices, i.e. Device Commands and Sensed Information, required for providing interoperability in controlling interaction devices and in sensing information from interaction devices in real as well as virtual worlds as depicted in Figure 1.

ISO/IEC 23005-5:2016 aims to provide data formats for industry-ready interaction devices: sensors and actuators. The same data formats for interaction devices can be utilized by various applications supported by different MPEG technologies. Not only this International Standard but also other International Standards such as ISO/IEC 23007 (MPEG-U) and scene representation specifications (for example ISO/IEC 14496-20) can simply refer to this part of ISO/IEC 23005 to use the defined data formats.

Two cases can occur for controlling a virtual world by using the MPEG tools. When the virtual world is using a scene description defined by MPEG tools (BIFS, Laser, etc.), the sensors and actuators can be directly connected to it through an MPEG-U interface. When the virtual world is defined by non MPEG tools, an adaptation engine and common formalism for effects are needed. In Figure 1, the first case is illustrated by VirtualWorld2 and the second by VirtualWorld1.

When this part of ISO/IEC 23005 is used in the context of pure ISO/IEC 23005, the adaptation engine (RV or VR engine), which is not within the scope of standardization, performs bi-directional communications using data formats specified in this part of ISO/IEC 23005. The adaptation engine can also utilize other tools defined in ISO/IEC 23005-2, which are user's sensory preferences (USP), sensory device capabilities (SDC), sensor capabilities (SC), and sensor adaptation preferences (SAP) for fine controls of devices in both real and virtual worlds.

On the other hand, the defined data formats (Sensed Information and Device Command) can be mapped to MPEG-U defined interfaces when this part of ISO/IEC 23005 is utilized in the context of other standards such as MPEG-U Framework. For example, the interface can be provided as ISO/IEC 23007-2 in the context of MPEG-U. Also defined, Sensed Information can be used by scene representation specifications as input data formats for a scene. The Device Command data format can also be used as output data formats to communicate with the outer world by mapping onto the interfaces defined in specific specifications.