RTOS Debugger for Linux - Run Mode - Lauterbach

RTOS Debugger for Linux - Run Mode TRACE32 Online Help TRACE32 Directory TRACE32 Index TRACE32 Documents ......................................................................................................................



RTOS Debuggers ...........................................................................................................................



RTOS Debugger for Linux .........................................................................................................



RTOS Debugger for Linux - Run Mode ..................................................................................

1

Debugging Modes for Embedded Linux ............................................................................

2

Run Mode Debugging with TRACE32 as GDB Front-end

2

Stop Mode Debugging via JTAG

2

Integrated Run & Stop Mode Debugging via JTAG

3

Supported Debugging Modes sorted by Architectures

4

Basic Concepts .................................................................................................................... t32server (only ARM)

5 5

gdbserver

5

Ethernet as Communication Interface to the gdbserver

7

Operation Theory

7

Switching to Run Mode Debugging

8

DCC as Communication Interface to the t32server

9

The Debug Communications Channel (DCC)

9

Operation Theory

9

Switching to Run Mode Debugging

10

The Space ID for Run Mode Debugging

11

Process Debugging

12

Quick Start Example for ARM .............................................................................................

14

Quick Start Example for MIPS .............................................................................................

16

Quick Start Example for PowerPC ......................................................................................

18

SYStem.Option NoDebugStop

19

Switching between Run & Stop Mode Debugging ............................................................

20

Commands for Run Mode Debugging ................................................................................

23

Breakpoint Conventions ......................................................................................................

24

Debugging of Multi-threaded Applications ........................................................................

25

Frequently-Asked Questions ..............................................................................................

26

©1989-2017 Lauterbach GmbH

RTOS Debugger for Linux - Run Mode

1

RTOS Debugger for Linux - Run Mode Version 29-Mar-2017

Debugging Modes for Embedded Linux TRACE32 provides 3 modes for debugging embedded Linux: Run Mode Debugging, Stop Mode Debugging and Integrated Run & Stop Mode Debugging.

Run Mode Debugging with TRACE32 as GDB Front-end Pure software debugger, no TRACE32 hardware required. The TRACE32 software is licensed by a USB dongle. Requires a gdbserver/gdbstub running on the target. The TRACE32 GDB Front-end works in so-called Run Mode Debugging: At a breakpoint only the selected process is stopped, while the kernel and all other processes continue to run. The TRACE32 GDB Front End is described in the document “TRACE32 as GDB Front-End” (frontend_gdb.pdf).

Stop Mode Debugging via JTAG Stop Mode Debugging requires a TRACE32 JTAG debugger hardware. If debugging is performed via the JTAG interface, TRACE32 works in so-called Stop Mode Debugging. At a breakpoint the CPU and thus the whole target system is stopped. The main advantages of Stop Mode Debugging are: •

Since the only requirement for Stop Mode Debugging is a functioning JTAG interface, debugging can start at the reset vector.

•

Since TRACE32 provides both Linux and MMU support, debugging of the kernel and beyond process boundaries is possible.

Stop Mode Debugging is described in the document “RTOS Debugger for Linux - Stop Mode” (rtos_linux_stop.pdf).



2

Debugging Modes for Embedded Linux

Integrated Run & Stop Mode Debugging via JTAG Integrated Run & Stop Mode Debugging requires a TRACE32 JTAG debugger hardware. If debugging is performed via the JTAG interface, TRACE32 can be configured: •

To allow Stop Mode Debugging via JTAG.

•

To allow Run Mode Debugging via the t32server/gdbserver running as debug agent on the target.

•

To allow, if required, to switch between the two debugging modes.

TRACE32 communicates with: •

The gdbserver via ethernet for all supported architectures (see table on next page) or

•

The t32server via DCC (Debug Communications Channel) for the ARM architecture. Since the ARM-JTAG interface provides a DCC, Integrated Run & Stop Mode Debugging can function with JTAG as the only communication interface.



3


Supported Debugging Modes sorted by Architectures Architecture

GDB Front-end

Stop Mode Debugging

Integrated Run & Stop Mode Debugging

ARM

Yes, refer to ”TRACE32 as GDB Front-End” (monitor_gdb.pdf).

Yes, refer to “RTOS Debugger for Linux Stop Mode” (rtos_linux_stop.pdf).

Yes, via DCC (except Cortex-M) or Ethernet. For details refer to this manual.

ColdFire

—


—

I386

Yes

—

Yes, via Ethernet

MIPS



Yes, via ethernet. For details refer to this manual.

PowerPC




SH4

Yes



XScale

Yes


—



4


Basic Concepts For Integrated Run & Stop Mode Debugging, Stop Mode Debugging via the JTAG interface is extended by: •

t32server/gdbserver as debug agent on the target.

•

A communication interface between TRACE32 and the debug agent (ethernet or DCC).

t32server (only ARM) The t32server is a gdbserver extension provided by LAUTERBACH. Compared to the gdbserver the t32server allows debugging over DCC for the ARM architecture. The t32server starts a gdbserver for every process. •

the gdbserver has to be in the /bin directory of the Linux file system.

•

the IP address 127.0.0.1 has to be configured, because the t32server communicates with the separate gdbservers via TCP/IP.

In order to provide Integrated Run & Stop Mode Debugging the t32server has to be started as a Linux process on the target via the terminal window e.g.: ./t32server

; Communication via DCC

gdbserver You can display the version of your gdbserver by typing in the terminal gdbserver --version If nothing is printed out, then your gdbserver has a version older than 6.2. Using version 6.2 and older is not recommended. If run mode debugging over Ethernet is used, we recommend to use gdbserver 7.1 or newer.



5

Basic Concepts

If used together with the t32server (communication over DCC), gdbservers newer than 6.5 have to be patched to disable run length encoding: Patching gdbserver newer than 6.5 (DCC communication only) file: gdb-x.x/gdb/gdbserver/remote-utils.c static int try_rle (char *buf, int remaining, unsigned char *csum, char **p) { int n; /* Always output the character. */ *csum += buf[0]; *(*p)++ = buf[0]; return 1; // add this to disable run length encoding /* Don't go past '~'. */ if (remaining > 97) remaining = 97; ... If the run length encoding is not disabled in the gdbserver, the communication with the t32server will be too slow.



6

Basic Concepts

Ethernet as Communication Interface to the gdbserver TRACE32 communicates with the gdbserver via ethernet for the MIPS and the PowerPC architecture.

Operation Theory Linux

gdbserver ethernet

SOFTWARE

JTAG

MIPS Core

HARDWARE



7

Basic Concepts

Switching to Run Mode Debugging After TRACE32 was started and configured for Stop Mode Debugging switching to Run Mode debugging is performed as follows: SYStem.PORT 10.1.2.99:2345 Go.MONitor

SYStem.PORT :

Configure ethernet communication is the target IP-Address is the used port number

Go.MONitor

Switch to Run Mode Debugging

After the communication is configured, debugging can be performed completely via the TRACE32 GUI.



8

Basic Concepts

DCC as Communication Interface to the t32server TRACE32 communicates with the t32server via DCC for the ARM architecture.

The Debug Communications Channel (DCC) The JTAG interface of the ARM architecture includes a so-called Debug Communications Channel (DCC). Information exchange via DCC is possible between •

the TRACE32 software running on the host

•

and the t32server running as Linux process on the target

On an SMP system, the debugger only communicates with the DCC registers of the first core. Thus, the t32server always run on this core. For details about DCC refer to your ARM Technical Reference Manual.

Operation Theory Linux 127.0.0.1:port1 127.0.0.1:port2

t32server DCC

127.0.0.1:portn

gdbserver

Process 1

gdbserver

Process 2

gdbserver

Process n

SOFTWARE

JTAG

ARM Core

HARDWARE



9

Basic Concepts

Switching to Run Mode Debugging After TRACE32 was started and configured for Stop Mode Debugging switching to Run Mode debugging is performed as follows: SYStem.MemAccess GdbMON Go.MONitor SYStem.MemAccess GdbMON

Configure DCC as communcation interface to t32server

Go.MONitor

Switch to Run Mode Debugging

After the communication is configured, debugging can be performed completely via the TRACE32 GUI.



10

Basic Concepts

The Space ID for Run Mode Debugging Processes of Linux may reside virtually on the same addresses. To distinguish those addresses, the debugger uses an additional space ID, that specifies, to which virtual memory space an address refers. In Run Mode Debugging the space ID is equal to the process ID. The command SYStem.Option MMUspaces ON enables the additional space ID.

Space ID (decimal and hex value) A source code listing for the process sieve is displayed as follows:

Space ID

For details on the space ID for Stop Mode Debugging, refer to “Training Linux Debugging” (training_rtos_linux.pdf).



11

Basic Concepts

Process Debugging To debug a process proceed as follows: 1.

Check if the process is already running. TASK.List

List all running processes

TASK.List 2.

Load the process for debugging. TASK.RUN

Load (process not running)

TASK.SELect

attach to the process (process already running)

If the process is not running, the command TASK.RUN can be used to load the process for debugging. ; Load process sieve from the Linux file system and prepare it for ; debugging TASK.RUN /bin/sieve If the process is already running, the command TASK.SELect can be used to attach to it. TASK.SELect /bin/sieve



12

Basic Concepts

3.

Load the symbol and debug information for the process. Data.LOAD. :0 /NOCODE /NoClear /NOREG Since processes of Linux may reside virtually on the same addresses, the symbol and debug information has to be loaded for the address space of the process by using the . /NOCODE - load only symbol information. /NoClear - obtain the symbol information loaded for other processes. /NOREG - avoid any unintended change to the CPU registers. ; Data.LOAD.Elf :0 /NOCODE /NoClear /NOREG Data.LOAD.Elf sieve.elf 0x91:0 /NOCODE /NoClear /NOREG ; Stop sieve at main and display source listing Go main Data.List TASK.PROC.SPACEID()

This function returns the of a process. This is required for PRACTICE scripts.

Example for a PRACTICE script: LOCAL &spaceid &spaceid=TASK.PROC.SPACEID("sieve") TASK.RUN /bin/sieve Data.LOAD.Elf sieve.elf &spaceid:0 /NOCODE /NoClear /NOREG



13

Basic Concepts

Quick Start Example for ARM Integrated Run & Stop Mode Debugging requires that Stop Mode Debugging is working properly before Run Mode Debugging can be activated. The following commands represent a basic TRACE32 set-up for Stop Mode Debugging. It is assumed that the target set-up and Linux booting is performed by the code in the boot FLASH. SYStem.CPU ARM920

; Select the target CPU

SYStem.Option MMUspaces ON

; Extend logical addresses by ; space ID

TrOnchip.Set DABORT OFF

; Debug mode is not entered at data ; abort exception used by Linux ; for page miss!

TrOnchip.Set PABORT OFF

; Debug mode is not entered at ; prefetch abort exception used by ; Linux for page miss!

TrOnchip.Set UNDEF OFF

; Debug mode is not entered at an ; UNDEF instruction may be used by ; Linux for FPU detection

SYStem.MemAccess GdbMON

; Configure DCC communication ; between TRACE32 and t32server

SYStem.Mode Attach

; Establish the communication ; between the debugger and the CPU

Data.LOAD.Elf vmlinux /NOCODE

; Load the kernel symbols

TASK.CONFIG linux

; Enable the TRACE32 linux ; awareness

MMU.FORMAT LINUX swapper_pg_dir

; Define an MMU page table format ; for Linux

TRANSlation.COMMON 0xC0000000-0xffffffff

; Upper memory pages are valid for ; all space IDs

TRANSlation.TableWalk ON

; ; ; ; ; ; ; ; ;

TRANSlation.ON

Configure the debugger to proceed as follows at an conversion from virtual to physical address: 1. Look-up in the debugger MMU table 2. Walk through the Linux MMU table, if the look-up in the debugger MMU fails

; Switch debugger MMU to ON



14

Quick Start Example for ARM

The target set-up and the preparations for Linux debugging might be more complex for your system. It is strongly recommended to refer to “ARM Debugger” (debugger_arm.pdf) and “RTOS Debugger for Linux - Stop Mode” (rtos_linux_stop.pdf) for details. To configure Run Mode Debugging: 1.

Make sure you have a gdbserver in the /bin of the Linux file system on your target.

2.

Make sure the loop-back address 127.0.0.1 is configured on your target.

3.

Start t32server as Linux process via the terminal window: ./t32server

4.

Then continue to enter the following commands in TRACE32: Go.MONitor

; Switch to Run Mode Debugging



15

Quick Start Example for ARM

Quick Start Example for MIPS Integrated Run & Stop Mode Debugging requires that Stop Mode Debugging is working properly before Run Mode Debugging can be activated. The following commands represent a basic TRACE32 set-up for Stop Mode Debugging. It is assumed that the target set-up and Linux booting is performed by the code in the boot FLASH. SYStem.CPU MIPS24KE


SYStem.Option Endianess Big

; Select big endian mode



SYStem.Mode Attach


SYStem.PORT 10.1.2.99:2345

; Configure ethernet communication ; between TRACE32 and t32server



TASK.CONFIG linux




TRANSlation.COMMON 0x80000000-0xffffffff



; ; ; ; ; ; ; ; ;


TRANSlation.ON


SYStem.Option gdbNONSTOP ON

; Enable GDB non-stop mode

SYStem.Option gdbEXTENDED ON

; Enable GDB extended mode

The target set-up and the preparations for Linux debugging might be more complex for your system. It is strongly recommanded to refer to “MIPS Debugger and Trace” (debugger_mips.pdf) and “RTOS Debugger for Linux - Stop Mode” (rtos_linux_stop.pdf) for details.



16

Quick Start Example for MIPS

To configure Run Mode Debugging: 1.

Start the gdbserver in multi-process mode: gdbsever --multi:2345

2.

Then continue to enter the following commands in TRACE32 Go.MONitor




17

Quick Start Example for MIPS

Quick Start Example for PowerPC Integrated Run & Stop Mode Debugging requires that Stop Mode Debugging is working properly before Run Mode Debugging can be activated. The following commands represent a basic TRACE32 set-up for Stop Mode Debugging. It is assumed that the target set-up and Linux booting is performed by the code in the boot FLASH. SYStem.CPU MPC8349




SYStem.Mode Attach


SYStem.PORT 10.1.2.99:2345

; Configure ethernet communication ; between TRACE32 and t32server



TASK.CONFIG linux




TRANSlation.COMMON 0xC0000000-0xffffffff



; ; ; ; ; ; ; ; ;


TRANSlation.ON


SYStem.Option gdbNONSTOP ON

; Enable GDB non-stop mode

SYStem.Option gdbEXTENDED ON

; Enable GDB extended mode

The target set-up and the preparations for Linux debugging might be more complex for your system. It is strongly recommended to refer to your PowerPC Processor Architecture Manual (e.g. “PPC600 Family Debugger” (debugger_ppc600.pdf) ) and “RTOS Debugger for Linux - Stop Mode” (rtos_linux_stop.pdf) for details. So that ethernet will not be broken in the case of some PowerPCs, please do not stop the CPU after Linux is started. If you would like to debug in stop mode first, please go to run mode with Go.MONitor and back to stop mode with Break.MONitor, so that the ethernet interface could be disabled. ©1989-2017 Lauterbach GmbH


18

Quick Start Example for PowerPC

To configure Run Mode Debugging: 1.

Start the gdbserver in multi-process mode: gdbsever --multi:2345

2.

Then continue to enter the following commands in TRACE32 Go.MONitor


SYStem.Option NoDebugStop For PowerPC, software breakpoints and single step are enabled respectively in stop or run mode: •

SYStem.Option NoDebugStop ON: software breakpoints and single step are enabled in run mode.

•

SYStem.Option NoDebugStop OFF: software breakpoints and single step are enabled in stop mode.

This option is automatically set/reset by the debugger with every Go.MONitor and Break.MONitor command (except the first Go.MONitor. After the first Go.MONitor command, breakpoints and single step will not work in run mode). Please note that this option could only be set/reset while the CPU is stopped. If you change this option manually, this could disable breakpoints and single step in you current mode.



19

Quick Start Example for PowerPC

Switching between Run & Stop Mode Debugging

The graphic above shows a simple schema of the switching between Run Mode Debugging and Stop Mode Debugging. Not all transitions are covered. The following commands are used to switch between Run & Stop Mode Debugging: Go.MONitor

If the CPU is stopped, the program execution is started. Switch to Run Mode Debugging. In Run Mode Debugging no process is selected for debugging.

Break.MONitor

Switch to Stop Mode Debugging and stop the program execution on CPU.

Break.SetMONitor ON

Switch to Run Mode Debugging with the next Go. In Run Mode Debugging no process is selected for debugging.

Break.SetMONitor OFF

Switch to Stop Mode Debugging. If the selected process was running or no process was selected, the CPU stays running in Stop Mode. If the selected process was stopped, the CPU is stopped in Stop Mode.



20

Switching between Run & Stop Mode Debug-

If Run Mode Debugging is active, a green M is displayed in the state line of TRACE32. The following states are possible in Run Mode Debugging: 1.

Run Mode Debugging active (green M), no process selected (see TASK.List).

2.

Run Mode Debugging active (green M), selected process (sieve) stopped.



21


3.

Run Mode Debugging active (green M), selected process (sieve) running.



22


Commands for Run Mode Debugging

TASK.List

List all running processes

TASK.RUN

Load a process for debugging

If the command TASK.RUN is used to load a process for debugging, the process is stopped by the gdbserver at the label _start in ld-linux. To start process debugging, load the symbol information for the process first and then type Go main.

TASK.SELect

Select a process for debugging

If the selected process has been started with TASK.RUN, it will be selected as current process. Otherwise a gdbserver will be started and the selected process will be attached. It is only allowed to attach to user process. If you try to attach to a kernel process, an error message will be returned.

TASK.KILL

Request GDB agent to end the process

Only processes that have been started with a TASK.RUN or that have been attached with TASK.SELect can be killed. After a TASK.KILL, the process and its gdbserver will be killed. TASK.COPYUP

Copy a file from the target into the host

TASK.COPYDOWN

Copy a file from the host into the target

SYStem.GdbLibPath

Set the path to the target libraries on the host



23

Commands for Run Mode Debugging

Breakpoint Conventions For Integrated Run & Stop Mode Debugging please keep the following breakpoint convention: •

Use on-chip breakpoints for Stop Mode Debugging If an on-chip breakpoint is hit in Run Mode Debugging, the CPU is stopped in Stop Mode debugging (only for ARM).

•

Use software breakpoints for Run Mode Debugging

Examples for Stop Mode Debugging: ; Break.Set : /Program /Onchip Break.Set 0x0:0x4578 /Program /Onchip Break.Set error /Program /Onchip Examples for Run Mode Debugging: ; Break.Set : /Program /SOFT Break.Set 0x2bc:0x0xd0065789 /Program /SOFT Break.Set 0x2bc:main /Program /SOFT



24

Breakpoint Conventions

Debugging of Multi-threaded Applications To be able to debug multi-threaded applications, TRACE32 needs the target libraries ld.so.1 (ld-linux.so.2 or ld-linux.so.3 in the case of ARM targets) and libpthread.so.0. The path to this libraries on the host can be set with the TRACE32 command SYStem.GdbLibPat h. For example: SYStem.GdbLibPath C:\T32\libs



25

Debugging of Multi-threaded Applications

Frequently-Asked Questions

ARM WFI and power down modes Ref: 0395

e500 breakpoints Ref: 0426

Kernel message "soft lockup" Ref: 0388

After booting Linux, why does the target die or loose connection to the debugger? Many CPUs have special power saving modes if they spin in the idle loop. Usually these are encoded in pm_idle() - Check this. While the debugger tries to recover from power saving modes, this is not always possible. E.g. the WFI mode of some i.MX CPUs does not work with JTAG, and many Cortex-A CPUs power down the JTAG interface in idle. Check your distribution - In Freescale's Linux distribution for i.MX set the command line parameter "jtag=on" to disable WFI mode. In some other distributions there is a "nohlt" parameter to disable power down modes. Breakpoint are not working properly or stop working completely for the e500 cores The Linux kernel for e500 cores (PQ3/MPC85XX/QorIQ P10XX/P2020) has to be patched to enable debugging via JTAG. The MSR_KERNEL macro defined in arch/powerpc/include/asm/reg_book.h needs to be changes to include the MSR_DE bit: #define MSR_KERNEL (MSR_ME|MSR_RI|MSR_CE|MSR_DE) What does the kernel message soft lockup mean? The kernel detects when there is too much time gone between two timer ticks. This easily happens, if the system is halted with the debugger. Please switch off the soft lockup detection by disabling CONFIG_DETECT_SOFTLOCKUP in the kernel configuration.



26


Linux trace in simulator Ref: 0304

What is needed to revise a Linux trace with a TRACE32 instruction set simulator? Example for ARM926 SAVING IN TRACE32 DEBUGGER:

; save trace recording Trace.SAVE trace.ad ; save whole RAM address range Data.SAVE.Binary image.bin ASD: ; save registers STORE regs.cmm register ; save important CP15 registers Data.SAVE.Binary p15.bin C15:0x0--0xfff /Long LOADING IN TRACE32 INSTRUCTION SET SIMULATOR:

SYStem.RESet SYStem.CPU ARM926 SYStem.Option MMUspaces ON ; enable space ID System.Up ; load linux image Data.LOAD.Binary image.bin ASD:address_range ; load registers do regs.cmm Data.LOAD.Binary cp15.bin c15:0 ; load symbol information Data.LOAD.Elf vmlinux /GNU /NOCODE ; specifiy MMU table format MMU.FORMAT LINUX swapper_pg_dir TRANSlation.COMMON 0xc000000--0xfffffff TRANSlation.TableWalk ON TRANSlation.ON ; configure Linux awareness TASK.CONFIG linux Menu.ReProgram linux Help.Filter.ADD rtoslinux ; load trace recording Trace.LOAD trace.ad MMU.FORMAT LINUX swapper_pg_di r Ref: 0255

Why does "MMU.FORMAT LINUX swapper_pg_dir" report "invalid combination"? Space IDs must be switched on for this command to work: "SYStem.Option MMUSPACES ON".



27


PPCSIM Trap Ref: 0425

The error message "PPCSIM Trap " appears when using the Linux awareness This error message is due to an incompatibility between the TRACE32 software and the loaded Linux awareness.

Problems with Onchip Breakpoints Ref: 0252

After halting at a function entry point with an on-chip breakpoint, or after stepping into a new function, the Data.List window shows "???"for all assembler lines, even after rescanning the MMU. You probably fell over the demand paging mechanism of Linux. Please read the "On-Demand Paging" chapter in the Linux Awareness Manual.

SYstem.Option MMUSPACES ON/OFF Error

Why does "SYstem.Option MMUSPACES ON/OFF" report "Cannot change from short to long"?

Ref: 0256

This command executes only, if no symbols are loaded. Delete previous loaded symbols with "sYmbol.Delete".

TASK.MODUL E Display Problems

After loading a new kernel module, TASK.MODULE shows nothing, dashed lines or errors.

Ref: 0254

The debugger couldn't resolve the virtual addresses of the new kernel module. Check the MMU and TRANSlation settings in your startup script. See also chapter "MMU Support" in the Linux Awareness Manual.



28


RTOS Debugger for Linux - Run Mode - Lauterbach

RTOS Debugger for Linux - Run Mode - Lauterbach

Suggest Documents

RTOS Debugger for QNX - Run Mode - Lauterbach

RTOS Debugger for pSOS+ - Lauterbach

ARM Debugger - Lauterbach

MIPS Debugger and Trace - Lauterbach

A Debugger RTOS for Embedded Systems - CiteSeerX

Training Linux Debugging - Lauterbach

An RTOS-Based Run-Time Reconfigurable Avionics System for UAVs

Run-time Reconfigurable RTOS for Reconfigurable Systems-on-Chip

Linux Forensics (for Non-Linux Folks) - Deer Run Associates

RTOS

Code-Coverage – Simplified - Lauterbach

Mechatronics Real Time Linux A RTOS Live CD Based on Linux - LISHA

Logic Analyzer Probe for PowerTrace II - Lauterbach

Integration TRACE32-Simulink® - Lauterbach

Cortex Debugger

Integration for Visual Basic Interface - Lauterbach

PaulOS F020: An RTOS for the C8051F020

Debugger Detection.pdf - Google Drive

Visual Tracing for the Eclipse Java Debugger

Wulf, Lauterbach & Toole - CSU, Chico

RTOS Support for Multicore Mixed-Criticality Systems

Financial management for RTOs - Victorian Registration and ...

A Runtime Debugger for Massively Parallel ...

A Delta Debugger for ILP Query Execution