rus
Issue #9/2018
Pavlov Anton N., Liventsev Evgenii V., Silantiev Alexander M.
Remote Hardware and Software Debugging Technology of FPGA-based Systems
Remote Hardware and Software Debugging Technology of FPGA-based Systems
This paper considers hardware and software development of FPGA-based systems with microprocessor core. The phenomenon under study is increasing software and hardware developers productivity.
Теги: fpga hardware development hardware/software co-verification microprocessor ip-core open software remote control software development встречная верификация микропроцессорное ядро плис разработка аппаратуры разработка по удаленное управление
While creating FPGA-based systems with microprocessor core both hardware and software are being developed simultaneously (the system part in the first place: bootloader, kernel).
Debugging and testing are a very important and tedious stage in developing both hardware and software. Usually debugging is done with devices based on either existing FPGA debug kits or specifically designed ones. Simultaneous development of hardware and software allows organizing a cross-testing: hardware developers use user applications as tests [1], system software developers use prototype stands for testing as target platform.
While using FPGA-based testing stands the following difficulties are encountered:
FPGA-based testing stands are usually very expensive and their number is limited, so it’s impossible to have a separate stand for each developer’s personal use. The transfer of the stand from one developer to another is not instant, and it takes some time to reconfigure it making its use impossible during the period.
Stands could be bulky and may need special working conditions; that’s why it’s not always possible to place the stand near the developer’s workplace, and vice versa.
For scrutinous testing of the system software it’s important to check intermediate (internal) software versions which could be difficult if the stand is already in use by another developer.
All these difficulties cause the problem: the useful working time of each stand is reduced which leads to a decrease in working efficiency. Besides, manual operations with stands which could be simple but intensive and time-consuming raise the significance of human factor as the source of errors.
One of the ways to solve this problem is transition to remote work with stands when most operations needed for hardware testing and debugging are being done without physical access to the stand.
In this regard the task is to elaborate a technology of remote hardware and software debug of FPGA-based systems, which makes it possible to work with testing stands remotely and to organize joint access of several developers to one or more stands.
The main idea of the technology is to wire all interfaces to a specially designed control device with network support. All interaction with the testing stand is done remotely via this control device (except cases when repair or hardware reconfiguration is needed) [3].
Similar approach was described earlier in works [2, 3] and [4, 5] but unlike the technology proposed here the main focus was on automated execution of software testing in fixed configurations.
TYPICAL OPERATIONS AND INTERFACES USED
The typical debugging cycle of FPGA-based system with microprocessor core includes updating the FPGA configurations of the stand, execution of the testing software for checking at least basic functioning of updated FPGA configuration and further joint work of software and hardware developers.
Traditionally, for updating FPGA configurations JTAG is used.
The execution of software testing is connected with receiving information from it. Usually, UART interface is used for the purpose and/or Ethernet.
During debugging of software or hardware hang-ups may occur which could be resolved by RESET signal sent to the system under debug. In cases when sending RESET signal doesn’t work the developer has to turn off the system and then to turn on with possible consequent FPGA reconfiguration.
The system being developed may comprise data storage media based on PROM with SPI and I2C interfaces; in this case the debug stand may contain separate technical tools for programming such chips.
Fairly often the system being debugged supports several working modes depending on states of input lines of discrete signals interface for RESET; in order to debug the system in specific mode the working stand is capable of setting these lines into desired state with the help of external jumpers.
EXISTING SOLUTIONS FOR REMOTE DEBUGGING OF THE FPGA-BASED SYSTEMS
The existing solutions for remote debugging of the FPGA-based systems are represented by devices with Ethernet and JTAG for FPGA reconfiguration. Those include EthernetBlaster II Communications Cable [6] and Catapult EJ-2 Ethernet-to-JTAG cable [7]. Features preventing from using them as full-fledged devices for stand remote control are as follows:
lack of UART, SPI, I2C interfaces support;
inability to send RESET signal to the system;
lack of power control of debugged system.
However, there is a solution that could be used as a composite part of the remote control device: Bus Pirate [8], a multi-functional device which allows connecting to debugged hardware/software system via such interfaces as UART, I2C, SPI, JTAG. But Bus Pirate has a number of flaws:
lack of network interface; connection to PC is done via USB;
UART, I2C, SPI, JTAG interfaces are multiplexed which makes it impossible to use them simultaneously without re-wiring.
there is a possibility to control the power of connected devices, however the voltage of 3,3V is applicable to a narrow range of systems being debugged, and maximum current strength is mere 150mA.
though Bus Pirate can be used as a JTAG adapter, this will require changing the embedded software, and pass-through capacity is limited to 115200bits/sec.
IMPLEMENTATION
The structure of the remote debugging stand can be represented by the following scheme (Fig. 1):
the basis of the stand is the remote control device which is connected to one or more debugged FPGA-based systems (devices under test);
remote control device is equipped with two Ethernet interfaces; the first is used for local area network (LAN) for connecting to developer and user workplaces; the second is used for isolated local test network for interchanging data with systems under debug;
depending on particular needs of each debugged system the remote control device is connected to it with a needed set of interfaces: UART, SPI, I2C.
for controlling the RESET signal the remote control device contains the relay box for locking/unlocking the circuit; the same box is used for controlling the power supply of the system under debug.
Existing off-the-shelf modules are used for implementing the remote control device. For example, single-board Raspberry Pi 3 is used as a controlling PC.
The software is based on open-source components, and the operating system is based on Debian Linux.
RESULTS
The remote debug technology has been developed including the set of hardware, software and methodology:
stand control device that is responsible for:
stand power-supply control
RS232, UART, SPI, I2C, JTAG, interfaces, Ethernet
working mode control of the system under debug by discrete signals interface
RESET signal transmission
stand state surveillance
FPGA configuration update
software for controlling the authorized access to the stand and organizing the joint work, as well as technological software for controlling the stand hardware;
methodological recommendations and typical work approaches to testing stands.
CONCLUSION
Further improvement of remote debug technology lies in expanding its range of application, implementing measuring equipment in stands’ control device and test running automating.
REFERENCES
1. Chibisov P. A. High-performance Microprocessors Counter-Testing Methods. PhD thesis, ISP RAS, Moscow, 2013 (Russian: Chibisov P. A. Vstrechnoe testirovanie vysokoproizvoditel'nykh mikroprotsessorov: dissertatsiya … kandidata tekhnicheskikh nauk: 05.13.11 / Chibisov Petr Aleksandrovich; [Mesto zashchity: In-t sistem. programmirovaniya].— Moskva, 2013. 174 p. : il.). (In Russian).
2. LinuxBIOS Automated Distributed Test System Test Integration Manual V1.0 2006/11/22 // URL: https://www.coreboot.org/PDFs/LinuxBIOS-testing/TestIntegrationManual.pdf (Retrieved 09 Aug 2017).
3. Raptor Engineering Automated Coreboot Test Stand (REACTS) // URL: https://static.rpteng.com/REACTS/pdf/reacts_users_guide.pdf (Retrieved 18 Dec 2017).
4. Hardware Infrastructure of Free Electrons’ Lab // URL: https://free-electrons.com/blog/hardware-infrastructure-free-electrons-lab/ (Retrieved 18 Dec 2017).
5. Software architecture of Free Electrons’ Lab // URL: https://free-electrons.com/blog/software-architecture-free-electrons-lab/ (Retrieved 18 Dec 2017).
6. EthernetBlaster II Communications Cable // URL: https://www.altera.com/content/dam/altera-www/global/en_US/pdfs/literature/ug/ethernetblasterii.pdf (Retrieved 18 Dec 2017).
7. Catapult EJ-2 Ethernet-to-JTAG Cable // URL: http://www.byte-tools.com/product/catapultej2 (Retrieved 18 Dec 2017).
8. Bus Pirate // URL: http://dangerousprototypes.com/docs/Bus_Pirate (Retrieved 18 Dec 2017).
Debugging and testing are a very important and tedious stage in developing both hardware and software. Usually debugging is done with devices based on either existing FPGA debug kits or specifically designed ones. Simultaneous development of hardware and software allows organizing a cross-testing: hardware developers use user applications as tests [1], system software developers use prototype stands for testing as target platform.
While using FPGA-based testing stands the following difficulties are encountered:
FPGA-based testing stands are usually very expensive and their number is limited, so it’s impossible to have a separate stand for each developer’s personal use. The transfer of the stand from one developer to another is not instant, and it takes some time to reconfigure it making its use impossible during the period.
Stands could be bulky and may need special working conditions; that’s why it’s not always possible to place the stand near the developer’s workplace, and vice versa.
For scrutinous testing of the system software it’s important to check intermediate (internal) software versions which could be difficult if the stand is already in use by another developer.
All these difficulties cause the problem: the useful working time of each stand is reduced which leads to a decrease in working efficiency. Besides, manual operations with stands which could be simple but intensive and time-consuming raise the significance of human factor as the source of errors.
One of the ways to solve this problem is transition to remote work with stands when most operations needed for hardware testing and debugging are being done without physical access to the stand.
In this regard the task is to elaborate a technology of remote hardware and software debug of FPGA-based systems, which makes it possible to work with testing stands remotely and to organize joint access of several developers to one or more stands.
The main idea of the technology is to wire all interfaces to a specially designed control device with network support. All interaction with the testing stand is done remotely via this control device (except cases when repair or hardware reconfiguration is needed) [3].
Similar approach was described earlier in works [2, 3] and [4, 5] but unlike the technology proposed here the main focus was on automated execution of software testing in fixed configurations.
TYPICAL OPERATIONS AND INTERFACES USED
The typical debugging cycle of FPGA-based system with microprocessor core includes updating the FPGA configurations of the stand, execution of the testing software for checking at least basic functioning of updated FPGA configuration and further joint work of software and hardware developers.
Traditionally, for updating FPGA configurations JTAG is used.
The execution of software testing is connected with receiving information from it. Usually, UART interface is used for the purpose and/or Ethernet.
During debugging of software or hardware hang-ups may occur which could be resolved by RESET signal sent to the system under debug. In cases when sending RESET signal doesn’t work the developer has to turn off the system and then to turn on with possible consequent FPGA reconfiguration.
The system being developed may comprise data storage media based on PROM with SPI and I2C interfaces; in this case the debug stand may contain separate technical tools for programming such chips.
Fairly often the system being debugged supports several working modes depending on states of input lines of discrete signals interface for RESET; in order to debug the system in specific mode the working stand is capable of setting these lines into desired state with the help of external jumpers.
EXISTING SOLUTIONS FOR REMOTE DEBUGGING OF THE FPGA-BASED SYSTEMS
The existing solutions for remote debugging of the FPGA-based systems are represented by devices with Ethernet and JTAG for FPGA reconfiguration. Those include EthernetBlaster II Communications Cable [6] and Catapult EJ-2 Ethernet-to-JTAG cable [7]. Features preventing from using them as full-fledged devices for stand remote control are as follows:
lack of UART, SPI, I2C interfaces support;
inability to send RESET signal to the system;
lack of power control of debugged system.
However, there is a solution that could be used as a composite part of the remote control device: Bus Pirate [8], a multi-functional device which allows connecting to debugged hardware/software system via such interfaces as UART, I2C, SPI, JTAG. But Bus Pirate has a number of flaws:
lack of network interface; connection to PC is done via USB;
UART, I2C, SPI, JTAG interfaces are multiplexed which makes it impossible to use them simultaneously without re-wiring.
there is a possibility to control the power of connected devices, however the voltage of 3,3V is applicable to a narrow range of systems being debugged, and maximum current strength is mere 150mA.
though Bus Pirate can be used as a JTAG adapter, this will require changing the embedded software, and pass-through capacity is limited to 115200bits/sec.
IMPLEMENTATION
The structure of the remote debugging stand can be represented by the following scheme (Fig. 1):
the basis of the stand is the remote control device which is connected to one or more debugged FPGA-based systems (devices under test);
remote control device is equipped with two Ethernet interfaces; the first is used for local area network (LAN) for connecting to developer and user workplaces; the second is used for isolated local test network for interchanging data with systems under debug;
depending on particular needs of each debugged system the remote control device is connected to it with a needed set of interfaces: UART, SPI, I2C.
for controlling the RESET signal the remote control device contains the relay box for locking/unlocking the circuit; the same box is used for controlling the power supply of the system under debug.
Existing off-the-shelf modules are used for implementing the remote control device. For example, single-board Raspberry Pi 3 is used as a controlling PC.
The software is based on open-source components, and the operating system is based on Debian Linux.
RESULTS
The remote debug technology has been developed including the set of hardware, software and methodology:
stand control device that is responsible for:
stand power-supply control
RS232, UART, SPI, I2C, JTAG, interfaces, Ethernet
working mode control of the system under debug by discrete signals interface
RESET signal transmission
stand state surveillance
FPGA configuration update
software for controlling the authorized access to the stand and organizing the joint work, as well as technological software for controlling the stand hardware;
methodological recommendations and typical work approaches to testing stands.
CONCLUSION
Further improvement of remote debug technology lies in expanding its range of application, implementing measuring equipment in stands’ control device and test running automating.
REFERENCES
1. Chibisov P. A. High-performance Microprocessors Counter-Testing Methods. PhD thesis, ISP RAS, Moscow, 2013 (Russian: Chibisov P. A. Vstrechnoe testirovanie vysokoproizvoditel'nykh mikroprotsessorov: dissertatsiya … kandidata tekhnicheskikh nauk: 05.13.11 / Chibisov Petr Aleksandrovich; [Mesto zashchity: In-t sistem. programmirovaniya].— Moskva, 2013. 174 p. : il.). (In Russian).
2. LinuxBIOS Automated Distributed Test System Test Integration Manual V1.0 2006/11/22 // URL: https://www.coreboot.org/PDFs/LinuxBIOS-testing/TestIntegrationManual.pdf (Retrieved 09 Aug 2017).
3. Raptor Engineering Automated Coreboot Test Stand (REACTS) // URL: https://static.rpteng.com/REACTS/pdf/reacts_users_guide.pdf (Retrieved 18 Dec 2017).
4. Hardware Infrastructure of Free Electrons’ Lab // URL: https://free-electrons.com/blog/hardware-infrastructure-free-electrons-lab/ (Retrieved 18 Dec 2017).
5. Software architecture of Free Electrons’ Lab // URL: https://free-electrons.com/blog/software-architecture-free-electrons-lab/ (Retrieved 18 Dec 2017).
6. EthernetBlaster II Communications Cable // URL: https://www.altera.com/content/dam/altera-www/global/en_US/pdfs/literature/ug/ethernetblasterii.pdf (Retrieved 18 Dec 2017).
7. Catapult EJ-2 Ethernet-to-JTAG Cable // URL: http://www.byte-tools.com/product/catapultej2 (Retrieved 18 Dec 2017).
8. Bus Pirate // URL: http://dangerousprototypes.com/docs/Bus_Pirate (Retrieved 18 Dec 2017).
Readers feedback
