Petalinux-App using interrupts from FPGA on an Arty-Z7

Petalinux offers you the possibility to integrate your own hardware components realized within the FPGA into a real operating system, offering much more flexibility for your setup to communicate to the outside world or to realize more sophisticated algorithms.

Of course, using the bare-metal approach will give you best capabilities to control real-time behavior, but the price is that you will spend much more time in programming and testing.

However, most apps will run well enough in Linux, this will give you much more capabilities. Accessing the hardware pins (e.g. using AXI-GPIO) can be done via UIO and accessing the ports then is simple and has been demonstrated already a few times.

But is seems, integrating interrupts with this approach is somehow the "Holy Grail" and many people have been struggling with this. I want to show with this project, how everything needs to be set up and how the app shall handle interrupts.

Finally, I will show the measured reaction times on the used board, so you can estimate if it will be OK for your requirements.

1. Preparation of Development Tool Set

Because Petalinux needs to run on Linux and for programming the application, VITIS will refer to libraries generated by Petalinux. So it is the easiest and most reliable way to run all tools within a Linux environment. Either you use a dedicated Linux box or a virtual machine (I am using Virtual Box for this).

For the setup of the tool chain, please follow the instructions provided by Whitney Knitter, she published a very cool guide which will save you a lot of time.

In addition to this, I recommend to install minicom for the serial communication to the running Linux. Serial communication will run over the second usb port shown.

2. Hardware definition

The project will run on a Arty Z7-20, so a Zynq based platform.

• Open Vivado and create a new project
• Create a new block design: we will use 2 AXI-GPIO ports connected to the 4 buttons and 4 LEDS, respectively.
• The GPIO connecting the buttons will have interrupts enabled and connected to the Zynq. You need to open this GPIO block and select the check box at the bottom.
• Enable PL-PS interrupts. Then wire the interrupt from the GPIO to the Zynq
• Save the block design and generate it. Create HDL wrapper.
• Implement the FPGA project and generate a bitstream
• Finally export the hardware, including the bitstream

3. Creating the Linux image

Some general remarks at the beginning:

• creating the image take quite a long time, creating the sdk even much longer.
• For the latter, I recommend using at least 16GB RAM and limit the number of processors to 8. More processors will occupy too much memory forcing extensive swapping.
• First build takes quite a time, but the system keeps most things in cache, so correcting things later goes faster (but not fast ;-))

Follow these steps:

• Open a terminal in your Linux installation. Source the Petalinux settings:

source $PLINST/settings.sh

("$PLINST" contains the path to my installation, please adapt it.)

• Create the petalinux project:

petalinux-create project --template zynq --name ArtyTest

This will create a folder "ArtyTest" where the project is stored.

• Enter this folder:

cd ArtyTest

When working with git, it is now a good moment to init the repository and commit the first snapshot. You find a sample of the ".gitignore" in the attachments.

• Import the previously hardware:

petalinux-config --get-hw-description /<yourPath>/<your-hardware>.xsa

During the import, a config window opens, where you need to validate some settings. Follow the menu path "DTG settings"-->"Kernel bootargs"-->"Add extra bootargs". Enter "uio_pdrv_genirq.of_id=generic-uio", then select "OK", "Save" and "Exit" till you return to the terminal. All other settings should be OK.

• Setup UIO drivers: (described by abhinayp, follow details there)

petalinux-config -c kernel

• Select "Userspace platform IO driver with generic IRQ handling" and "Userspace platfrom driver with generic irq and dynamic memory". Both should be marked with a "*". Save and exit.
• Modify device tree settings: Open the file "project-spec/meta-user/recipes-bsp/devicretree/files/system-user.dtsi". This file is designed to carry any user defined changes of the device tree. Here we need to change the compatibility entry, so the drivers will use UIO later.

Modify it as shown in the picture. The names of the AXI components need to be the same as given in the block design. "&" will reference an already existing component and we just need to list the changed property.

• Build the project:

petalinux-build

• Then create the images for the SD card:

petalinux-package --boot --force --fsbl --fpga --u-boot

The generated image will be located in "images/linux", you need to copy the files "BOOT.BIN", "boot.scr" and "image.ub" on the SD card.

4. Testing the image

• Test boot: Plug SD card into your Arty, set booting from SD card on J4 and power on. Connect via minicom and log on.
• Test presence of UIO:

sudo ls /sys/class/uio

should show "uio0" and "uio1" being present.

• Testing presence of interrupt:

sudo cat /proc/interrupts

shall show an entry "gpio"

• Testing interrupt activation: follow the steps in the article.

When those tests pass, the built image is basically OK and we can go further to programm a linux application using UIO transfer as well as interrupts.

5. Prepare SDK

In order to be able to debug the app, you need to create the so called sysroots:

petalinux-build --sdk
    petalinux-package sysroot

This step will charge your workstation and take several hours depending on the power of your computer.

The result will also found in "images/linux/sdk/sysroots/cortexa9t2hf-neon-xilinx-linux-gnueabi". It is the path you need to configure later in Vitis.

6. Creating the App

Now we will use Vitis to create the Linux application and to test/debug it on the system.

If you are new to the topic: Basic steps are also described in the documentation of AMD. I recommend to first follow the example 5 in UG1165. Here, the creation of a simple "hello world" app is described and you will work through all steps needed.

To keep this description short, I will not show all the steps.

The whole SW including the platform is found in the mentioned repository. Please download it prior to going further.

Someremarks concerning Vitis:

Even if you followed the installation steps, Vitis might expose some strange behavior - its a bit a Diva, you never know exactly whats coming next. Actually, in the Linux VM it does not find the git repo, the clangd runs extensively and it deletes my configured launch.json with every build. Strange, but I did not yet take the time to find remedies to those things...

6.1 Creating the Framework

• File --> New Component --> Platform
• Select Linux as OS and your actual sysroots and let it create.
• Build (compile) the platform.

6.2 Designing the App

Finally we arrive at the interesting point: how can we receive interrupts in Linux from our GPIO ports?

The application is structered as follows:

The idea is to make it very easy from the application to read/write values to a defined port. Thus, the UIO layer is hidden as good as possible.

Setting up for interrupts is also done mostly automatic, if the port is enabled for interrupts.

Main Program:

int main()
{
    initUIO();
    int uio_cnt = fetchUIOData();
    printf("Found %d UIO ports\n", uio_cnt);
    setupPorts(ioPortsData);
    
    LED_Btn_Test_Polling();
    printf("Polling part finished. Entering Interrupt part\n");
    
    LED_Btn_Test_Interrupt();

    freePorts();
    printf("Program terminated\n");
    return 0;
}

Beside initialization and clean-up we run 2 payloads: one represents the polling scheme without interrupts and the second will use interrupts to do the same thing.

This will help you to diagnose the root cause in case the App does not run as expected.

Payload 1:

while(wcond){
    uint32_t read_val = readPortData(BUTN);
    if(read_val != last_val){
        last_val = read_val;
        LED_count = control(read_val, LED_count);
        writePortData(LED, LED_count);
    }
    wcond++;
}

The program just reads the inputs and if they have changed, it calculates a new value via the "control"-function and outputs it. As you can see, for the application it is very simple to access the desired data.

It validates, that your UIO access is working for read and write operations.

Payload 2:

while(wcond){
    // epoll_wait will return a number of fd found triggered
    int nb_fds = epoll_wait(ep_fd, events, MAXEVENT, to);
    for(int n_fds=0;n_fds<nb_fds;n_fds++){
        if(int_fd==events[n_fds].data.fd){
            ires = pread(events[0].data.fd, &evt_count, 0x04, 0);
            read_val = readPortData(BUTN);
            LED_count = control(read_val, LED_count);
            writePortData(LED, LED_count);
            printf("Event count: %d nb_fds: %d\n", evt_count, nb_fds);
            usleep(100);
            rearm_irq(BUTN);
        }
        wcond++;
    }
    if(nb_fds < 0 || (read_val == 0x9)){
    wcond = 0;
    }
}

Core idea is to use the "epoll_event" structure provided by Linux. We will use the UIO file descriptor to see when a new interrupt has been set. "epoll_wait" just waits till a change on that file has been detected and then returns back. Till then the application sleeps and does nothing - that's what we want to achieve.

After an interrupt has been detected (and has been validated that it's from the correct source), the program does its calculation as in the polling case. In addition we need to rearm the interrupt:

• on one side we need to reset the ISR in the GPIO itself,
• but we need also rearm the UIO by writing a "1" to the file descriptor.

Writing to the file descriptor to rearm interrupts is something I found some years ago in some forum thread - it works like that, but I don't know why. So if you know more than me, I would be happy to extend this article.

6.3 Creating and testing the app

• Create a new application: "File" --> "New component" --> "Application" or modify the "hello world" example
• Import the sources. Be sure to update the make file: updating sources will not update this file.

• Build the application and correct errors if any.
• Create a Debug configuration by clicking on the gear right at "Debug"

• Ensure to have the correct configuration settings, especially "Work Directory":

• Click on the bug and wait till the app has been downloaded and started. Enjoy stepping through the code :-)

7 Measurement of interrupt delay

Passing all the data through various levels of SW and using a "normal" operating system will of course introduce some delays. To check, if those are OK we need to measure the delay between the moment when the signal is issued by the button and when the reaction arrives at the LED.

Doing so, reveals a delay of about 60us, which is still quite quick. Of course, there was no load on the processor except this app - so this is clearly the best case. I assume, in case of network load and other calculations, the reaction will heavily increase - I would say, having delays in the ms range will not be exceptional. But for many applications this is still OK, especially when intercepting (rare) user interactions.

8 Conclusion

It's a long way, till everything is in place in Linux, and you need definitely some experience in the tooling. But once you know what to look for, programming with interrupts is not extremely difficult and might be a good option to handle user generated events. For this type of requirement, the response times should be good enough.

Programming different layers, will hide the complexity of UIO from the application level, this helps much to get a clean code.

However, this example does not cover extensive error handling. I am also sure, that the code can be improved as I am not an experienced C programmer.

Another open question is, how to adapt things to a "non-registered" device, but actually every IP block using AXI-lite interface should be easily adaptable.