Using riscv-arch-test version 3.10.0 and above

[mmichilot]

This is moreso a write-up since the recent pre-releases to riscv-arch-test broke my RISCOF setup.

TL;DR

install the latest versions of riscof, riscv-config, and riscv_isac directly from GitHub

$ pip install riscv_isac @ git+https://github.com/riscv-non-isa/riscv-arch-test/#subdirectory=riscv-isac
$ pip install riscv-config @ git+https://github.com/riscv-software-src/riscv-config@dev
$ pip install riscof @ git+https://github.com/riscv-software-src/riscof@dev

Then clone the latest riscv-arch-test either manually or by specifying a version with riscof arch-test:
$ git clone https://github.com/riscv-non-isa/riscv-arch-test
or
$ riscof arch-test --clone --get-version 3.10.0

riscv-arch-test version 3.10.0 and above (including the new pre-releases) added physical memory protection (PMP) tests. These tests use two new features, verify and the PMP schema. To use these tests, you need the latest versions of riscof, riscv-config, and riscv_isac from GitHub. I recommend doing these installs in a fresh python environment to ensure that there is no conflict between older and newer versions.

1. Based on Import preprocessing error #128 and riscv_isac dependency not updated #122, the latest version of riscv_isac in PyPI is outdated, so install it from GitHub
   pip install riscv_isac @ git+https://github.com/riscv-non-isa/riscv-arch-test/#subdirectory=riscv-isac

2. Based on Unknown field error #140 and Pypi package for riscof doesn’t include latest commits #144, the latest version of riscv-config in PyPI doesn't support the PMP schema, so install it from GitHub
   pip install riscv-config @ git+https://github.com/riscv-software-src/riscv-config@dev

3. Install riscof from GitHub for support for the verify feature (Support for verify feature and PMP schema support #134) :
   pip install riscof @ git+https://github.com/riscv-software-src/riscof@dev

As of this post, if you then attempt to clone the riscv-arch-test repo using riscof arch-test --clone, the tests will fail because it will clone the repo at riscv/riscv-arch-test@eca49e1, which has a bug. The bug was immediately fixed in the next commit riscv/riscv-arch-test@59075f8, however RISCOF fetches using tags so it will not fetch the latest release with the bug fix. To get around this issue, you can either manually clone the repo or specify a version using the --get-version flag, for example riscof arch-test --clone --get-version 3.10.0. This issue may go away with the next release/pre-release, but I think the mechanism for cloning riscv-arch-test with RISCOF should be improved.

Bug with cloning/updating riscv-arch-test with RISCOF

Upon inspection of RISCOF's code, the following segment handles cloning the riscv-arch-test repo. The issue lies with the repo.tags list because the tags are arranged in alphanumeric order, not release order, so cloning the last tag will not clone the latest release.

riscof/riscof/arch_test.py

Lines 51 to 56 in aa146d4

	def clone(path,branch="main"):
	logger.info("Cloning repository at "+str(path))
	repo = git.Repo.clone_from(constants.https_url, path)
	latest_tag = (repo.tags)[-1]
	checkout_target = latest_tag if branch=='latest' else branch
	repo.git.checkout(checkout_target)

This is also why this issue only started to appear despite riscv-arch-test version 3.10.0 (which added the PMP tests) being released last year. At that time the last tag in alphanumeric order was actually 3.9.1 not 3.10.0, so RISCOF wasn't cloning the latest release.

[jeras]

Thanks, I was checking whether there are any updates worth looking into and your post helped getting past all (for now) version compatibility issues.

I am only running I and C tests so I stayed at version 3.10.0. So I did not see any updates relevant to me.