I’ve been experimenting with the performance and stability of the VisionFive 2 by building GCC and LLVM on the Debian Image provided by StarFive (Google Drive link). I found that:

• It is quite fast (relative to my previous toolchain work done under qemu) – I could build LLVM in under 4 hours.
• It seems rock-solid – I have kept all 4 cores at 100% for hours on end, with no apparent issues. Running the GCC testsuite identified no issues.

Overall I’m really impressed with the board and I think it’s a great platform for RISC-V toolchain development.

This post outlines:

• What I did to build the toolchains, and
• Some measurements and observations.

LLVM

I built LLVM first because I’m actually going to need to use it to work on Numba. I used the LLVM 15.0.6 sources (the latest release version at the time of writing). First I installed dependencies:

sudo apt install \ 
  cmake ninja-build chrpath texinfo sharutils libelf-dev \
  libffi-dev lsb-release patchutils diffstat xz-utils \
  python3-dev libedit-dev libncurses5-dev swig \
  python3-six python3-sphinx binutils-dev libxml2-dev \
  libjsoncpp-dev pkg-config lcov procps help2man \
  zlib1g-dev libjs-mathjax python3-recommonmark \
  doxygen gfortran libpfm4-dev python3-setuptools \
  libz3-dev libcurl4-openssl-dev libgrpc++-dev \
  protobuf-compiler-grpc libprotobuf-dev \
  protobuf-compiler 

I would usually just use:

sudo apt build-dep llvm-15

(or something like that) but I couldn’t easily work out how to add a deb-src line to the APT sources that actually worked, so I derived that list of packages by looking at the source package instead. Following dependency installation, I configured for RISC-V only (because I am mainly interested in working on JIT compilation) a release build with assertions – I wasn’t sure if I had enough disk space for a debug build, and release with assertions is a reasonable compromise:

mkdir build 
cd build 
cmake ../llvm -G Ninja \
              -DCMAKE_INSTALL_PREFIX=/data/opt/llvm/15.0.6 \
              -DLLVM_TARGETS_TO_BUILD=RISCV \
              -DCMAKE_BUILD_TYPE=Release \
              -DLLVM_ENABLE_ASSERTIONS=ON 
ninja -j 3
ninja install 

I had to use 3 cores because 8GB RAM is not enough to link without running out of RAM when buiding with 4 cores. I was using ld from binutils, and I wonder if using an alternative linker would be more efficient – I need to check whether gold, mold, or lld support RISC-V first.

The build was pretty fast:

real	230m55.985s
user	671m8.705s
sys	    20m51.997s

The build completed in just under 4 hours, which for an SBC is pretty good – it’s quick enough for it not to be a complete pain to build toolchains. Compared to an emulated system I was using before where I had to wait all day even though I was emulating 8 cores on a Xeon Gold 6128, it’s blazing fast!

Since I built LLVM I’ve been using it for llvmlite development without issues. llvmlite is primarily for compiling and JITting code – LLVM JIT works “out of the box” in LLVM on RISC-V (HowToUseLLJIT is an example program included with the LLVM source distribution that demonstrates how to use the ORCv2 LLJIT class):

$ ninja HowToUseLLJIT
[1/1] Linking CXX executable bin/HowToUseLLJIT
$ ./bin/HowToUseLLJIT
add1(42) = 43

GCC

Next I tried building GCC 12.2 – this was mainly a curiosity for me because I don’t have a real use for it right now (other than the generally good “principle” of using up-to-date software), but I’ve spent a lot of time working on GCC-based toolchains, particularly for RISC-V, so it seemed interesting. I installed a few dependencies – possibly not all of these are required, but I was pretty certain this would cover everything needed:

sudo apt install \
  libc6-dev m4 libtool gawk lzma xz-utils patchutils \
  gettext texinfo locales-all sharutils procps \
  dejagnu coreutils chrpath lsb-release time pkg-config \
  libgc-dev libmpfr-dev libgmp-dev libmpc-dev \
  flex yacc bison

Then to configure and build GCC:

mkdir build-gcc-12.2.0
cd build-gcc-12.2.0
../gcc/configure \
  --enable-languages=c,c++,fortran \
  --prefix=/data/gmarkall/opt/gcc/12.2.0 \
  --disable-multilib \
  --with-arch=rv64gc \
  --with-abi=lp64d
make -j4

There’s one problem building GCC with the Debian image on the VisionFive 2, which is that the GCC RISC-V target seems not to be correctly set up for Debian multiarch – building will eventually fail with a message like:

/usr/include/stdio.h:27:10: fatal error:
  bits/libc-header-start.h: No such file or directory
  27 | #include <bits/libc-header-start.h>
     |          ^~~~~~~~~~~~~~~~~~~~~~~~~~

as it fails to find headers because it’s looking in the wrong location – /usr/include instead of the architecture-specific /usr/include/riscv64-linux-gnu.

The stage 1 xgcc doesn’t seem to know about multiarch:

$ ./gcc/xgcc -print-multiarch
# (no output)

A small patch works around this:

diff --git a/gcc/config/riscv/t-linux b/gcc/config/riscv/t-linux
index 216d2776a18..f714026b3cc 100644
--- a/gcc/config/riscv/t-linux
+++ b/gcc/config/riscv/t-linux
@@ -1,3 +1,4 @@
 # Only XLEN and ABI affect Linux multilib dir names, e.g. /lib32/ilp32d/
 MULTILIB_DIRNAMES := $(patsubst rv32%,lib32,$(patsubst rv64%,lib64,$(MULTILIB_DIRNAMES)))
 MULTILIB_OSDIRNAMES := $(patsubst lib%,../lib%,$(MULTILIB_DIRNAMES))
+MULTIARCH_DIRNAME = $(call if_multiarch,riscv64-linux-gnu)

Running the build again after making this change succeeds. Also, xgcc now knows about multiarch:

$ ./gcc/xgcc -print-multiarch
riscv64-linux-gnu

I think I need to report this and maybe submit a proper patch upstream – the above patch is good enough to work around the issue for building now, but isn’t a general solution – for example, it’s guaranteed not to work on a riscv32 system!

The build takes a bit longer than LLVM:

real 331m22.192s
user 1192m59.381s
sys 24m57.932s

At first I was surprised that it took longer than LLVM, but it has the bootstrapping process to go through, which goes some way towards increasing the build time.

I also ran the GCC test suite:

make check -j 4

Which takes a little while, but is worth the wait:

real    245m35.888s 
user    773m59.513s
sys     133m17.827s

The results are not 100% passing, but look pretty good:

		=== gcc Summary ===

# of expected passes		138016
# of unexpected failures	461
# of unexpected successes	4
# of expected failures		1035
# of unresolved testcases	48
# of unsupported tests		2888

		=== g++ Summary ===

# of expected passes		221581
# of unexpected failures	544
# of unexpected successes	4
# of expected failures		1929
# of unresolved testcases	40
# of unsupported tests		10418

        === gfortran Summary ===

# of expected passes		65099
# of unexpected failures	13
# of expected failures		262
# of unsupported tests		165

None of the problems look like any kind of real issue – many asan tests failed, which I guess might be due to lack of support (or it not having been enabled correctly by default) and many other issues are checks that scan the output for particular patterns (which can be fragile) or tests for excess error messages / warnings. I’d have to look in depth to be certain, but I have high confidence that all is well here.

Conclusion

Building GCC and LLVM on the VisionFive 2 was relatively pain-free, especially for a very new SBC with a relatively new software ecosystem. If you’re into RISC-V toolchain development and want to work on a native system (as opposed to e.g. using qemu under linux, or targeting very small embedded systems that need cross compilation anyway), it’s a great choice!

I’m expecting to have a productive time working on Numba and llvmlite for RISC-V with this board.

StarFive’s VisionFive 2 is a quad-core riscv64 board with up to 8GB of RAM. I bought one to work one RISC-V support in Numba. There’s some choices to make when buying the board and accessories – this post describes my setup and the rationale behind my decisions. I’ve also provided costings and links to purchase at the end, to aid anyone wanting a similar setup (especially in the UK).

Board: Super Early Bird (Rev 1.2A) with 8GB of RAM and WiFi

The VisionFive 2 comes in 2GB, 4GB, and 8GB variants. I picked 8GB because compiling and linking LLVM, which is used by Numba, is quite memory-intensive. Mine is a Super Early Bird version, which has one 10/100Mbit eithernet port and one gigabit ethernet port. Later versions will have two gigabit ports, but I don’t need two gigabit ports, and I didn’t want to wait longer to start working with the board.

There is also an option to include a supported WiFi adapter (based on the ESWIN 6600U) – I took this as it only adds a small amount to the price and it could come in handy in future. I’m not using it right now because it seems to not work out of the box with the StarFive Debian build, and it’s not essential for me.

I ordered from the WayPonDEV store on Amazon on December 16th, and it arrived on the 29th!

NVMe SSD: Kioxia Exceria 500GB

Nothing special about this – it was a cheap NVMe SSD with decent capacity. It works fine in the VF2. I also tried a Samsung SSD 980 temporarily (to make sure the M.2 slot worked before I ordered the Kioxia) that I borrowed from my Jetson AGX Xavier.

I had previously tried an old Intel SSD with a SATA interface, which did not work – as another user mentioned on the forum, the B/M keyed SATA SSD drives are not compatible.

I ordered the drive from Scan UK.

Heatsink and fan: ODroid XU4 heatsink / fan

It doesn’t seem to be easy to locate a compatible heatsink and fan, but I noticed from a mechanical drawing of the ODroid XU4 that it should use a heatsink with the same dimensions and hole distance. I ordered from the ODroid UK store. This ended up being quite pricey, but had the advantage of actually being obtainable in the UK within a couple of days.

One difference between the XU4 and the VF2 is the size of the fan connector – it is much larger on the VF2.

I had a spare connector of the correct size handy, so I was able to replace it. After unscrewing the fan and peeling back the sticker, the solder points for the cable are visible:

Then the heatsink can be fitted to the board. I applied thermal paste and pushed the pins through. I did this with the SSD removed, because one of the through-holes is beneath the SSD, and the pin suddenly pushing through could hit it a bit hard.

Finally I added the fan back and plugged it in:

Power supply: Lenovo 65W USB-C

I didn’t have a PSU ready when I got the board so I borrowed the one from my laptop. It’s been rock-solid even under heavy load with this PSU, so rather than taking a risk with some other power supply or Pi supply, I decided to just order another of the same model from the Lenovo UK store.

I note that the VF2 Data Sheet (Section 4.1) states that at least 9V / 2A is required, which further made me nervous about using a regular Pi supply. However, the Quick Start Guide (Section 1.2) says it will take “5V up to 30W (minimum 3A)” – I’m not quite sure how to interpret that though!

Costings / purchase links

None of the links are affiliate links – they are simply provided for convenience

• VisionFive 2 Super Early Bird 8GB with WiFi:
  • $143.97 with shipping and taxes, came to £117.86
  • Item link: https://www.amazon.com/dp/B0BGM1KQXQ?psc=1&ref=ppx_yo2ov_dt_b_product_details
• SSD:
  • £34.47 including shipping
  • Item link: https://www.scan.co.uk/products/500gb-kioxia-exceria-m2-2280-pcie-30-x4-nvme-ssd-1700mb-s-read-1600mb-s-write-350k-400k-iops
• Heatsink and fan:
  • £29.24 after shipping and low order fee added
  • Although this seems a bit pricey, I could quickly have spent hours searching and days waiting for something else, which wouldn’t be worth the saving in the end.
  • Item link: https://www.odroid.co.uk/odroid-accessories/odroid-cooling/odroid-xu4-blue-fan
• Power supply:
  • £22.74 including delivery
  • Item link: https://www.lenovo.com/gb/en/p/accessories-and-software/chargers-and-batteries/chargers/4x20m26276

Total: £204.31 – Not bad for a RISC-V SBC with this much power and potential, in my opinion.

Continuing…

I’ve been using this setup for building GCC and LLVM toolchains. I hope to write about that in a future post, but in the meantime I’m posting progress updates and other notes on my Mastodon account: https://mastodon.social/@gmarkall

I use this mug, made by Janet Hadley, every day.

Do you have a favourite mug?

Background thinking around this question: There are instances of OSS users levelling expectations at OSS maintainers that they provide a certain level of service, whilst OSS maintainers generally have no obligations to users. Jacob Tomlinson provides a good summary and discussion of this issue in “Don’t be that open-source user, don’t be me”.

Existing terms: The term voluntary is often used to describe OSS contributions, and contributors referred to as volunteers. In particular, these terms are sometimes used to remind users that maintainers aren’t generally obligated to do anything in particular. However, I find that these terms don’t always accurately convey the nature of contributions, particularly when contributors are employed to work on them – the idea of a volunteer can carry the connotation that any contributions are made in free time, or that no reward is being received for them.

An alternative: Discretionary contributions are those that are left to the maintainer’s choice or judgment – there may be remuneration for their work, but in terms of the relationship between the user and the maintainer, there is no particular obligation. In these cases, is discretionary contribution a more appropriate term than voluntary contribution?

I saw Acorn: a world in pixels and had to buy it immediately:

A few days later it turned up, packaged for pristine condition:

A shiny sleeve encapsulates:

Inside the covers:

Hundreds of pages of interviews, images, descriptions, and data:

Every page is full of detail and crafted with care.

It truly is a labour of love.