FPGA Based E-Paper Business Card

Introduction

Motivation

What if you were tasked to create an e-Paper controller for a new product? How would you go about accomplishing this task? What order of steps would you take? How I approached and met this task is stepped through in the coming sections.

In any engineering project, resources available and power consumption are two major constraints. Specifications for the hardware used (.e.g., Lattice FPGA and PervasiveDisplays' e-Paper).

In this project we are using the Adiuvo Engineering's Business Card (BC) as the controlling platform. It is fitted with a Lattice CrossLink-NX-33U FPGA and connects to the PervasiveDisplays' 1.44" e-Paper.

There are two major parts to this project and they are:

• The HDL control logic design for the FPGA, and
• Working with the Image data for image reduction and converting it to the appropriate FPGA RAM/ROM data formatting.

Let's cover some background info on the main components in the hardware domain first.

What exactly is e-paper?

E-paper, also known as electronic ink (e-ink) or electrophoretic display (EPD), is a display technology that mimics the look and feel of traditional printed paper. It uses tiny capsules filled with fluid containing charged particles to create images. These particles move in response to an electric field, making the display appear to change color, typically between black and white.

Electrical Connections

E-paper displays require external, analog circuitry to supply the proper voltages for the display. The following schematic (Figure 1) of a reference circuit for the 1.44 inch EPD comes from the E-paper Display COG Driver Interface Timing document [1], the link for which can be found in the Things-Hardware Section, as well as the Reference section at the end of this article. We will be referring to it quite a bit, so keep it handy!

Driving Waveform Control Logic

For any e-paper display to work, the controlling processor must perform a series of register configurations within defined time frames without failure. The registers that are being configured in the display are for power-up, display options/functions, and power down. An SPI or quasi-SPI (explained next) communication interface is used for these data transfers from the controlling hardware, such as an MCU or FPGA, to the e-paper.

What is meant by quasi-SPI format is that there are some subtle, but very important differences one must be aware of to successfully communicate with the G2 COG driver. The details of which are outline in [1], but are easy to misinterpret or bypass all together. So, I will spend a little time below in the Approach section pointing out possible gotchas with example waveforms acquired from the Adiuvo BC.

It should also be noted that some e-papers can be purchased with a driver board, such as the Waveshare e-Paper (G) 1.02" to 2.36".

Approach

Adiuvo Custom-made BC FPGA board

As mentioned previously, the Adiuvo's custom-made BC processing board is designed to operate on very low power. At the heart of the board is the Lattice CrossLink-NX-33U FPGA used for the e-paper controller design. The board's interface to the e-paper is made up of analog circuits that generate the correct voltage levels needed to power the e-paper display.

The FPGA design's functionality is used for controlling the e-paper display through all its essential phases which include power-up, display manipulations of stored image data, and power down. Display manipulations include image inversion, display updates (e.g., a clock or temperature display), and display clearing. A single 3.3v coin battery runs the full BC board system (FPGA, display, and other circuitry)!

The Lattice Radiant FPGA Design Suite Software was used for the BC's FPGA design. You can download a free version of Radiant from the Lattice Semiconductor website.

SPI Serial Communication Protocol

The G2 COG driver (e-paper driver) expects SPI Mode 0 for communications and can operate up to 20 MHz. Mode 0 is the most popular mode for SPI communications. It is defined as data being sampled at the leading rising edge of the clock. Thus, the SPI FPGA HDL module was designed for this mode.

The SPI's chip select active low (CS_n) signal is used somewhat differently for the COG driver, however. The CS_n is used to indicate the start/end of a command header + command index packet and the start/end of trailing data header + 1 or more 8-bit data words packet. The waveform below shows what the COG driver expects to understand the data it is receiving.

State Machine Architecture

When studying the flow charts in [1] to create one or more state machines to make all of the necessary actions to operate the PervasiveDisplays (PD) e-Paper, it is easy to see that many of the required actions follow the same steps. Figure 2 exemplifies the repetitive actions. The specific sections may have different timing parameters (delays between command/data transactions), and of course different commands and data. However, this reusable pattern brings to mind of creating and using functions in a programming language. It can be called over and over with different arguments passed in based on what action is currently being taken.

Thus, creating a state machine that loops with different commands and data used was the clear approach for me. This makes for much less state machine logic, thus logic resources, being used in the FPGA.

There are also direct bit manipulations (e.g., /CS, /RESET) required for the power-on and power-off phases. For example, setting /RESET low after the epaper_pwr_en is asserted high. Then the /RESET must remain low for N ms. The details of which can be found in sections 3 and 6 of [1] as shown in Figure 3 (actual section numbers found in [1] for these sections are 2 and 6, as there is no section numbered 4 in the document).

Let's take a closer look at section 5 which falls into the category of logic reuse architecture, similar to a function if you will. Using indexing through a hex memory buffer makes this process achievable.

State Machine Details

Let's examine the part of the display controller that has a looping structure (section 5 of [1]), which is shown below in Figure 4. We won't cover the similar, shorter flowchart for section 3 "Initialize G2 COG Driver", but you can see the similarities when looking at these sections.

To write pixel data to the e-Paper display, a very particular format has to be followed, as one can see in Figure 4.

The PD 1.44" e-Paper display in this project has landscape dimensions of 128 columns by 96 rows (Figure 8). But not to worry, portrait type displays are accomplished as well.

For our example coming up is in this position. The controller for the display must follow the following flow (a general description of Figure 4) to display a single image frame:

1. Send pixel data for every odd column going in reverse order of 127, 125, 123, etc. down to 1. This data needs to be packaged into byte format, where every pixel has two bits (see [1] for details).

2. Scan in on/off two bit values for each row (96 down to 1) in byte format words.

3. Send pixel data for the rest of the column (pixel) data going in forward counting order, skipping the odd columns already written to, from 2, 4, 6, 8, etc. to 128. Again, the data needs to be packaged into byte words.

4. Write a Border Byte (cmd, data byte = 0x00).

5. Turn on OE, hence write another particular cmd + data byte pair in the format shown in Figure 2.

6. Repeat steps 1 thru 5 until the entire image has been written.

When studying the flow charts in [1] to create one or more state machines to implement the necessary actions to operate the e-Paper, it is easy to see that many of the display's operating phases follow the same steps.

Figure 2 exemplifies the repetitive actions of the logic. Each section may have different timing parameters (delays between command/data transactions), commands, and data. However, this reusable pattern brings to mind of creating and using functions in a programming language. They can be called over and over with different arguments passed in based on what action is currently being taken.

Next, we will cover the method I used to go from an image selected from a photo or screen snap to being displayed on the e-Paper. The general flow of any method you choose, which could easily be better than mine ;), will be similar in nature. This flow or process involves cropping the image to best fit the e-Paper's dimensions (this was initially the hardest part for me), and converting the image file data into hex data format for the FPGA internal RAM memory file or external memory.

Image Formatting

- Preparing the Image Size

Preparing an image for display requires proper dimensions and data format. The first part of the process is a bit of an art form depending on the tools you have available and your past experiences with image editing. I didn't have any at the time, other than cropping and Image2Lcd aspect ratio attempts, so not to worry!

Typically, you want an aesthetically pleasing representation of the image you wish to display. Thus, keeping the aspect ratio (AR) of the original image while reducing the image size is key in preventing distortion.

There are several tools available that will do resizing and the Image2Lcd program is one of them. Additionally there are several "free", online tools such as Fotor, Adobe Express, Img2go.com, etc. One can also go with more deluxe packages as cost such as Adobe Photoshop.

These tools aid in making a PNG and JPEG photos. Unfortunately, Image2Lcd does not support PNG files and won't accept a JPEG files ending in.jpeg. It only accepts.bmp,.jpg, and.gif ending files for images. So, if you have a.jpeg image, open the image in a view/editor and save a copy with the.jpg suffix (name.jpg).

My first example image to use was of Batman (found in a Google search). I used the Windows 10 Snipping Tool to copy this image by creating a rectangular box in a portrait shape around the image that I found in a Google search (Figure 5). I then saved it as a JPEG file.

In the bottom left hand corner of the Snippet Tool display, you can see the dimensions of the acquired image are 533 x 603. Let's see about resizing without too much distortion or lost data.

Option 1: Using only Image2Lcd Program For Resizing.

Double clicking on the Image2Lcd.exe executable brings up the GUI. Here, you set the Max Width and Height values in the boxes at the left (see Figure 6). Since the image was snipped in a portrait aspect, I set the Max Width to 96 (pixel rows) and Height of 128 (pixel columns) for the given e-Paper's dimensions.

The results of pulling in an image that hasn't been first resized by a tool like Adobe results in the following dimensions achieve by the Image2Lcd program:

The first attempt of resizing using my original Snipping gave an output image size of 96 x 108 (Figure 6, bottom right). So, the images current dimensions were not going to work. From the dimension of 96 x 108 achieved, one can tell there was more width than height image data from my current sample. So, I cropped the photo in a photo viewer/editor and tried again.

The new result wasn't that bad. I was able to achieve 96 x 127! However, let's cover how the hex data would result isn't usable.

Saving this image as a C array file (*.c) would result in a lose of 96 bits of data. However, as explained in Figure 6 and illustrated in the defined x, y coordinates of the e-Paper in Figure 8, we should be scanning down the "Height" as defined in the Image2Lcd program.

Figure 9 below shows the selection for vertical scan. We must also pay attention to whether the image might end up being display upside down compared to what we want. This is a bit of a trial and error loop until you have done this several times :)

Now, it can be seen that the data will be stored the way the data is written and scanned to the e-Paper in the state machine. Unfortunately, we need to add pixel data for one pixel at the beginning or end of each vertical scan and this would be really messy. This is why it is important to get exactly (128, 96) or (96, 128) pixel dimensions (landscape vs portrait if you will).

It should be mentioned that even though the PD e-Paper display uses 2 bits per image, we can still convert a one bit pixel value, black or white pixel values (0 or 1), into a two bit value. Figure 10 from [1] illustrates how bit-1 is always a logic 1 for both black and white, while bit-0 determines the color. The logic in the e-Paper state machine I created does exactly this!

Option 2: Using Image Resizing App Prior to Import into Image2LCD

First attempt, I set the width and height in the Custom section of Adobe Express to 96 x 128. It worked, but the result was a hazy image (see Figure 11). However, when displayed on a small display such as the 1.44", the human eye isn't able to pick up the haziness.

Note: To get the tool actually perform the custom resizing, I needed to give my credit card number to obtain a 30 day free trial, then it would become $10/month after that if not cancelled. That is why I put "Free" in double quotes...

Unfortunately, the image wouldn't import into the Image2Lcd app. So, next idea was to increase the resizing target to 4 x (96, 128). This resulted in a 384 x 512 pixel image while keeping with the 3:4 AR needed for the e-Paper display (see Figure 12).

The JPEG file was then imported in Image2Lcd and the 96 x 128 was obtained (Figure 13)!

Let's now look at the image data saved in hex format (C array).

- Converting Image to Hex File

The file saved from the previous section has the following structure:

The header part shown in Figure 14 of the hex file has a header that indicates there are 1542 bytes saved. This is definitely a good thing to check. For, 128 * 96 = 12, 288 bits or 1536 bytes. So, we have 6 bytes of extra data in the file. It can also be seen that the first six bytes on the header line don't fit in: 0X01, 0X01, 0X60, 0X00, 0X80, 0X00. For the image is surrounded by a border of either all white or black typically. So, only a long string of 0x00's or 0xFF's would make any sense.

To prepare this file for the conversion script, as it is currently written, the header is remove along with the first 6 bytes (all on the same top, header line). Additionally, the ending bracket and semicolon are removed as well "};" Once the file has been modified, save a copy of the file named data_only.txt. The Python script covered in the next section looks for this file/name in its current directory. Please feel free to modify the script as you wish to make it more flexible in this regard or any other way that suits you.

- Script to Convert to Image Hex Data to FPGA's RAM format

Once the image hex file has been edited and saved, a simple, quick python script I wrote (hex_to_lattice_bram_format.py), can then be run to convert the hex data of the multi-hex values per line to the correctly formatted Lattice memory file (.mem) of one hex value per line (incremental RAM addresses).

- Lattice Radiant Software's Module/IP Block Wizard

Adding distributed or block RAM/ROM to the project requires the use of the Module/IP Block Wizard. To bring configure size and give it a path to the RAM/ROM initialization file, go to Tools > IP Catalog > Memory_Modules > Distributed_ROM (in this case). You will see the configuration window come up and displayed as shown in Figure 15.

As we calculated earlier, there are 1536 bytes worth of image data in total that we obtained. Thus, we set the Address Depth to 1536 and the Read Port: Data Width to 8 bits. We also set the Memory File and Memory File Format settings here.

Since all of this information is now entered into the Configuration IP section, the only information needed in the.mem file we are pointing to is a single hex value on each line. Each hex value represents the hex value stored in the memory from (implicitly set) address 0 at the top, down to address 1536 -1 at the bottom. Figure 16 shows a random segment from the.mem file.

The resulting image on the Adiuvo BC:

And of course, the e-Paper maintains the image with no-power!

Wrap Up

This project covered quite a few topics related to using e-Paper. Not only how to use an FPGA in place of a microcontroller to save power, but also what goes into converting an image file into the proper format for Lattice RAM.

I know I learned a great deal from this unique project. Hopefully, you will as well!

The completed project can be foundHereon my GitHub.

Adiuvo Engineering and Training, LTD sponsored this amazing project. Other product names used in this publication are for identification purposes only and may be trademarks of their respective companies.

References

[1] PervasiveDisplays, "E-paper Display COG Driver Interface Timing for 1.44”, 1.9”, 2”, 2.6” and 2.7” EPD with G2 COG and Aurora Mb Film", Doc. No. 4P01800. Website: https://www.pervasivedisplays.com/wp-content/uploads/2019/06/4P018-00_03_G2_AuroraMb_COG_Driver_Interface_Timing_for_smallsize_20150727.pdf