Disabling Data Cache from Linux

[Jorgebrihu]

Hi,

in my design I am using a peripheral that uses the AXI DMA IP from Xilinx. I suspect that the descriptors of the transactions that are created are kept in cache and not moved to RAM. When the DMA begin to move data, it is reading wrong values from the descriptors.

I have checked my peripheral with a baremetal code and I needed to disable the cache in order for it to work.
For that, I used the CSR_DCACHE CSR from the CVA6 as explained in cva6 issue in order to disable it.

In order to do the same from Linux, I have created a new CSR (equivalent to the one above that can be used from baremetal) to which I can write from a Linux module.
I have a Linux module that disables the cache by writing a 0x0 to the newly created CSR. The system hangs when trying to do so though.
However, if I disable and enable the cache (by writing a 0x0 and later on a 0x1), the module loads into the kernel.

I want to fully disable it. ¿Has anyone been able to do so?

Another option could be to have a piece of memory as non-cacheable and assign it to my dma on the device tree but as long as I have been able to see, that is not an option so far. ¿Could anyone provide any info or hint about this?

Thank you in advance.

[yvantor]

Hi @Jorgebrihu!

Hi,

in my design I am using a peripheral that uses the AXI DMA IP from Xilinx. I suspect that the descriptors of the transactions that are created are kept in cache and not moved to RAM. When the DMA begin to move data, it is reading wrong values from the descriptors.

I have checked my peripheral with a baremetal code and I needed to disable the cache in order for it to work. For that, I used the CSR_DCACHE CSR from the CVA6 as explained in cva6 issue in order to disable it.

In order to do the same from Linux, I have created a new CSR (equivalent to the one above that can be used from baremetal) to which I can write from a Linux module. I have a Linux module that disables the cache by writing a 0x0 to the newly created CSR. The system hangs when trying to do so though. However, if I disable and enable the cache (by writing a 0x0 and later on a 0x1), the module loads into the kernel.

I want to fully disable it. ¿Has anyone been able to do so?

There should be a few options:

1. If you want to fully disable the data cache you can modify either the Cheshire bootrom (but this requires also creating a new bitstream) or the very first boot stages in software, in order to set the CSR_DCACHE to disabled while you are still in machine mode. Since this is a very specific CSR for CVA6, I really dooubt that later stages in the boot should set it back to enable on purpose, so probably this would work.
2. An alternative to permanently disabling the CSR_DCACHE would be calling the fencei() instruction right after you have loaded your descriptors. The side effect of the fencei() is that it flushes the data cache, so anything that was cached before calling that instruction would be committed to the main memory afterward.
3. I assume you are using a single core configuration, so a possibility could also be that of using the data caches configured as Write Through. In this configuration, whater information gets committed to the cache is also automatically committed to the main memory in a transparent way, so you don't have to do tricks in software. You could do so by adding an entry here as ret.DCacheType = config_pkg::WT; and then regenerate the bitstream.

Another option could be to have a piece of memory as non-cacheable and assign it to my dma on the device tree but as long as I have been able to see, that is not an option so far. ¿Could anyone provide any info or hint about this?

Configuring a memory portion as non-cached is also doable, you need to change this configuration in HW, and then rigenerate a bitstream. If I sould go for this option, I would touch the SizeLlcOut parameter so that you can reserve a shorter portion of the external memory as cachable and let the rest being non-cachable.

Thank you in advance.

I hope the above helps!