twiddling bits and atoms's blog

Posted Sat 22 August 2020

From zero to SoC in LiteX

Recently I had a need for microcontroller with larger than usual number of peripherals and found that none of STM32s have the number of peripherals I wanted, so I decided to do a small feasibility study on building my own using LiteX.

This article details the sequence of steps I took in order to create the system-on-chip.

Wait, what is LiteX?

LiteX is a Migen/MiSoC based Core/SoC builder that provides the infrastructure to easily create Cores/SoCs (with or without CPU).

Ok, and what do you mean by system-on-chip? System-on-chip is essentially a CPU core with everything around it to do something useful (for example, blink a light).

In order to keep things somewhat objective, here's what I wanted to accomplish:

End result should hopefully be a folder containing SoC project which can be make'd and flashed, and firmware project which can also be make'd and flashed.

Here's how it went.

Install LiteX and toolchain

First matter of the day: getting all tools installed. Since I do not have any first-hand experience with development for FPGAs, it was not apparent to me which tools exactly are needed to accomplish the goals stated above.

I had to start with what I have: Lattice ECP5 evaluation board. This means I'll either need Lattice proprietary toolchain or open-source one. Since I had recently spent a few hours at work trying to get the Diamond programmer running on Linux distribution I was using (hint: I eventually gave up and used openFPGALoader), I did not even try installing the Diamond.

So, open-source toolchain please. For Lattice parts, LiteX needs several tools, each of which can be easily installed following the instructions in their READMEs:

Installation on Ubuntu 20.04 LTS was un-eventful (a compliment, considering python is involved).

Build an example SoC

LiteX target is a description of SoC, targeting particular hardware (e.g. a dev-board). The range of supported targets is quite diverse, from Fomu in the smallest/cheapest end up to Kintex UltraScale with 500+k logic cells.

Target is a self-contained python executable, which can be built (--build) and flashed (--load). By default, the CPU used for core is VexRiscV, but it can be changed to others using appropriate --cpu-type flag (supported CPUs include: lm32, picorv32, rocket and others).

    ~/litex/litex-boards/litex_boards/targets$ ./ecp5_evn.py --build
    INFO:SoC:        __   _ __      _  __  
    INFO:SoC:       / /  (_) /____ | |/_/  
    INFO:SoC:      / /__/ / __/ -_)>  <    
    INFO:SoC:     /____/_/\__/\__/_/|_|  
    INFO:SoC:  Build your hardware, easily!
    INFO:SoC:--------------------------------------------------------------------------------
    INFO:SoC:Creating SoC... (2020-08-28 15:08:29)
    INFO:SoC:--------------------------------------------------------------------------------
    INFO:SoC:FPGA device : LFE5UM5G-85F-8BG381.
    INFO:SoC:System clock: 60.00MHz.

    ....
    Info: Device utilisation:
    Info:          TRELLIS_SLICE:  3712/41820     8%
    Info:             TRELLIS_IO:    12/  365     3%
    Info:                   DCCA:     1/   56     1%
    Info:                 DP16KD:    28/  208    13%
    Info:             MULT18X18D:     4/  156     2%
    Info:                 ALU54B:     0/   78     0%
    Info:                EHXPLLL:     1/    4    25%
    Info:                EXTREFB:     0/    2     0%
    Info:                   DCUA:     0/    2     0%
    Info:              PCSCLKDIV:     0/    2     0%
    Info:                IOLOGIC:     0/  224     0%
    Info:               SIOLOGIC:     0/  141     0%
    Info:                    GSR:     0/    1     0%
    Info:                  JTAGG:     0/    1     0%
    Info:                   OSCG:     0/    1     0%
    Info:                  SEDGA:     0/    1     0%
    Info:                    DTR:     0/    1     0%
    Info:                USRMCLK:     0/    1     0%
    Info:                CLKDIVF:     0/    4     0%
    Info:              ECLKSYNCB:     0/   10     0%
    Info:                DLLDELD:     0/    8     0%
    Info:                 DDRDLL:     0/    4     0%
    Info:                DQSBUFM:     0/   14     0%
    Info:        TRELLIS_ECLKBUF:     0/    8     0%
    Info:           ECLKBRIDGECS:     0/    2     0%
    ...

Hopefully the synthesis and place & route finishes without warnings and errors and you're ready to load the bitstream to target device.

ECP5 EVN specific things

  1. The default linker settings for firmware examples expect that there is a memory region called main_ram where code is loaded via UART and executed from RAM. ECP5 evaluation board has a beefy ECP5 series FPGA, but it does not have any external memory. No problem, let's devote some of the FPGA resources to create a 128KiB of SRAM by adding the following line to BaseSoC.init:
self.add_ram("main_ram", self.mem_map["main_ram"], 128 * 1024)
  1. The serial port needed to see the LiteX BIOS prompt is not connected to FPGA by default. It can be easily fixed by following helpful instructions printed early in the build process:
FT2232H will be used as serial, make sure that:
 -the hardware has been modified: R22 and R23 should be removed, two 0 Ω resistors shoud be populated on R34 and R35.
 -the chip is configured as UART with virtual COM on port B (With FTProg or https://github.com/trabucayre/fixFT2232_ecp5evn).

Flashing the bitstream

~/litex/litex-boards/litex_boards/targets$ ./ecp5_evn.py --load
...
svf file programmed successfully for 2028 commands with 0 errors

After this you should see the LiteX BIOS prompt on serial port:

    ~$ ssterm -b 115200 /dev/ttyUSB1

            __   _ __      _  __
           / /  (_) /____ | |/_/
          / /__/ / __/ -_)>  <
         /____/_/\__/\__/_/|_|
       Build your hardware, easily!

     (c) Copyright 2012-2020 Enjoy-Digital
     (c) Copyright 2007-2015 M-Labs

     BIOS built on Aug 28 2020 15:08:35
     BIOS CRC passed (dfd04829)

     Migen git sha1: 7bc4eb1
     LiteX git sha1: 3897acb9

    --=============== SoC ==================--
    CPU:        VexRiscv @ 60MHz
    BUS:        WISHBONE 32-bit @ 4GiB
    CSR:        8-bit data
    ROM:        32KiB
    SRAM:       8KiB

    --============== Boot ==================--
    Booting from serial...
    Press Q or ESC to abort boot completely.
    sL5DdSMmkekro
    Timeout
    No boot medium found

    --============= Console ================--

    litex>

Compiling hello-world

I've found Lab004 from LiteX FPGA101 training materials to be the easiest hello-world like firmware project. Make file needs to be updated with a path to build outputs (build/ecp5_evn in my case).

Compilation:

~/litex/litex-boards/litex_boards/targets/firmware$ make
 CC       isr.o
 CC       main.o
 LD       firmware.elf
chmod -x firmware.elf
 OBJCOPY  firmware.bin
chmod -x firmware.bin

Loading via serial port:

~/litex/litex-boards/litex_boards/targets/firmware$ lxterm --kernel firmware.bin /dev/ttyUSB1 
[LXTERM] Starting....

litex> serial-boot
Command not found
litex> serialboot
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
[LXTERM] Received firmware download request from the device.
[LXTERM] Uploading firmware.bin to 0x40000000 (7676 bytes)...
[LXTERM] Upload complete (3.7KB/s).
[LXTERM] Booting the device.
[LXTERM] Done.
Executing booted program at 0x40000000

--============= Liftoff! ===============--

Lab004 - CPU testing software built Aug 28 2020 17:35:25

Available commands:
help                            - this command
reboot                          - reboot CPU
display                         - display test
led                             - led test
RUNTIME>help
Available commands:
help                            - this command
reboot                          - reboot CPU
display                         - display test
led                             - led test

Customizing the SoC

As an example of how easy it is to add extra copies of various peripherals, let's add I2C, UART and an SPI bus. Actually, let's make it two of each. And 2 hardware CS pins for each SPI bus please.

Allocating pins for the peripherals

Each LiteX target is actually based on a platform (often, of the same name). Platform handles the things that don't change (e.g. available pins, types of memories connected), but target can be purpose-specific (e.g. a variant of platform with something connected to it's PMOD header).

For the purposes of this exercise, let's add the pins to platform definition in litex_board/platforms. If this SoC were to be used in a real project, it would make more sense to create a new platform outside the litex_boards package.

diff --git a/litex_boards/platforms/ecp5_evn.py b/litex_boards/platforms/ecp5_evn.py
index 15c08b5..74d0a8d 100644
--- a/litex_boards/platforms/ecp5_evn.py
+++ b/litex_boards/platforms/ecp5_evn.py
@@ -61,6 +61,52 @@ _io = [
     ),
     ("ext_clk50",    0, Pins("B11"), IOStandard("LVCMOS33")),
     ("ext_clk50_en", 0, Pins("C11"), IOStandard("LVCMOS33")),
+    # connector J40
+    ("i2c0", 0,
+     Subsignal("sda", Pins("K2")),
+     Subsignal("scl", Pins("H2")),
+     IOStandard("LVCMOS33")
+     ),
+    ("i2c1", 0,
+     Subsignal("sda", Pins("F1")),
+     Subsignal("scl", Pins("G1")),
+     IOStandard("LVCMOS33")
+     ),
+    ("serial0", 0,
+     Subsignal("rx", Pins("J4")),
+     Subsignal("tx", Pins("J5")),
+     IOStandard("LVCMOS33"),
+     ),
+    ("serial1", 1,
+     Subsignal("rx", Pins("J3")),
+     Subsignal("tx", Pins("K3")),
+     IOStandard("LVCMOS33"),
+     ),
+    ("spi0", 0,
+     Subsignal("cs_n", Pins("L4 L5")),
+     Subsignal("clk", Pins("M4")),
+     Subsignal("mosi", Pins("N5")),
+     Subsignal("miso", Pins("N4")),
+     IOStandard("LVCMOS33"),
+     ),
+    ("spi1", 0,
+     Subsignal("cs_n", Pins("M5 L3")),
+     Subsignal("clk", Pins("N3")),
+     Subsignal("mosi", Pins("M3")),
+     Subsignal("miso", Pins("K5")),
+     IOStandard("LVCMOS33"),
+     ),
 ]

Now, when the platform definition has the pins mapped, peripherals can be added to SoC:

ecp5_evn.py

< from litehyperbus.core.hyperbus import HyperRAM
23,25d21
< from litex.soc.cores.bitbang import I2CMaster
< from litex.soc.cores.spi import SPIMaster
< 
52,55c48
<     mem_map = {
<         "main_ram": 0x20000000,
<     }
<     mem_map.update(SoCCore.mem_map)
---
> 
69,75d61
<         self.add_ram("main_ram", self.mem_map["main_ram"], 128 * 1024)
< 
81,115d66
< 
<         # I2C
<         self.submodules.i2c0 = I2CMaster(platform.request("i2c0"))
<         self.add_csr("i2c0")
<         self.submodules.i2c1 = I2CMaster(platform.request("i2c1"))
<         self.add_csr("i2c1")
< 
<         # serial ports
<         self.add_serial("serial0", 115200)
<         self.add_serial("serial1", 115200)
< 
<         # SPI
<         self.submodules.spi0 = SPIMaster(platform.request("spi0"), 8, self.sys_clk_freq, 8e6)
<         self.spi0.add_clk_divider()
<         self.add_csr("spi0")
< 
<     def add_serial(self, name, baudrate, fifo_depth=16):
< 
<         from litex.soc.cores import uart
< 
<         setattr(self.submodules, "%s_phy" % name, uart.UARTPHY(
<             pads     = self.platform.request(name),
<             clk_freq = self.sys_clk_freq,
<             baudrate = baudrate))
< 
<         setattr(self.submodules, name, ResetInserter()(uart.UART(getattr(self, "%s_phy" % name),
<             tx_fifo_depth = fifo_depth,
<             rx_fifo_depth = fifo_depth)))
< 
<         self.csr.add("%s_phy" % name, use_loc_if_exists=True)
<         self.csr.add(name, use_loc_if_exists=True)
<         if hasattr(self.cpu, "interrupt"):
<             self.irq.add(name, use_loc_if_exists=True)
<         else:
<             self.add_constant("UART_POLLING")

Now, when building the target you should the the configuration-status registers (CSR) for the new peripherals being added in CSR memory space:

    $ ./ecp5_evn.py --build
    ...
    INFO:SoCCSRHandler:i2c0 CSR allocated at Location 7.
    INFO:SoCCSRHandler:i2c1 CSR allocated at Location 8.
    INFO:SoCCSRHandler:serial0_phy CSR allocated at Location 9.
    INFO:SoCCSRHandler:serial0 CSR allocated at Location 10.
    INFO:SoCIRQHandler:serial0 IRQ allocated at Location 2.
    INFO:SoCCSRHandler:serial1_phy CSR allocated at Location 11.
    INFO:SoCCSRHandler:serial1 CSR allocated at Location 12.
    INFO:SoCIRQHandler:serial1 IRQ allocated at Location 3.
    INFO:SoCCSRHandler:spi0 CSR allocated at Location 13.
    INFO:SoCCSRHandler:spi1 CSR allocated at Location 14.
    ...

These peripherals can now be used from firmware by reading/writing to peripheral's registers. I have not seen an exhaustive documentation on how to use each peripheral, but a good reference seems to be Zephyr source code, which has support for built-in peripherals like I2C, SPI, UART, timers etc.

Since the LiteX BIOS is also using some of those peripherals, it serves as a reference as well. It can be found in litex/soc/software.

Review and conclusion

For me, an embedded systems person who hasn't written a line of Verilog/VHDL, this seemed almost magical. Being able to define a system-on-chip and customize it with a few lines of python code, adding exactly the number of peripherals needed for exact application is very exciting.

I have seen a fair share of projects that have outgrown the platform against which they were initially developed against and using something like this could provide a bit more headroom in cases where:

Also, the possibility to write my own peripherals in Migen, which can be easily interfaced with firmware running on the CPU is an elegant way of solving certain class of problems, which require pin-twiddling at very fast and predictable rate (not that soft-cores haven't been used in FPGAs for ages, but LiteX seemingly makes it easier by providing the right infrastructure).

However, I did note some limitations as well: * The built-in peripherals are quite basic, compared to, let's say STM32 (I2C is bitbanged, SPI master supports a single mode, no DMA support for UART/SPI/I2C). This can be solved by (someone knowledgeable) spending some time on it. * Maximum clock speed for low-to-mid range FPGA chips (ice40, ECP5) seems to be between 50-100MHz. I don't think this can be solved as easily (probably this is the reason why FPGA vendors are including a hardware ARM cores in their latest offerings).

Overall I think that having an FPGA-based system-on-chip that can be tailored with the correct peripherals opens up a lot of interesting applications. Especially, if it is accessible to people with absolutely no FPGA development experience.

Category: misc

Comments