PLASMA 2 CAT is slow - excessive disk activity

[tingtron]

This shows how slow the program +CAT is under PLASMA 2.
Video playback control (pause and frame tracking) can be used to see which tracks and sectors are involved. There is a LOT of disk activity.
It can be seen in AppleWin sector/track indicator and mapped to ProDOS file locations.

https://youtu.be/Ryvj8NEcUNU

Trying to understand which file are being accessed in this disk activity, an issue in the AppleWin project was created -- to make track/sector indicator more consistent, and also to provide logging feature for accessed tracks/sectors over time in the AppleWin log:

• In the disk Status getting ?? for Sectors in ProDOS

[dschmenk]

Well, you've discovered the downside of using loadable modules. CAT, COPY, DEL, and NEWDIR uses a host of library modules to keep the SYS/ directory small and consistent. The module search process starts with the current directory then looks in the SYS/ directory. On a floppy system, that is particularly slow. With a hard disk, RAMdisk, or SSD, the searching is quite a bit faster and doesn't impact the perceived load time as much. As noted, PLASMA is much better suited to this kind of installed environment over floppies. Have you tried doing the same operations on the old Apple Pascal system? Ugh.

[tingtron]

I was thinking about module loading. And it is still looking like too much activity.
That's why I wanted to see exact logging of sector and file access.

The analysis below applies to 140k floppies, which could be the media of choice
for the more classic Apple II's up to IIc.

It doesn't seem to be an issue on 800k drives. (* - below).
Maybe it has do with better ProDOS directory caching on SmartPort or HD?

💡Here's a thought: Why start looking for modules in current directory instead of app directory? Starting in app directory would skip the unnecessary scan in case of the tools already residing in SYS. Also if you run an app from outside the current directory (other than SYS) and it has private modules: why would they reside in current directory and not in the app directory.

In AppleWin this use case is nicely illustrated, when inserting the two 140k disks PLASMA2-SYS.PO and PLASMA2-BLD.PO in the two drives. Then P /BLD and +CAT. Observe the drive lights flipping between drives 1 and 2 about 7 times - this is for loading modules, when it first scans the current directory. Whereas CAT is in SYS and all its dependencies are in SYS, which is also the app directory. So with this alternative module loading rule it would've saved around 1/3 to 2/5 loading time (as shown in timings below).

Other potential workarounds for 140k disks:

• All modules in a single folder, thus avoiding deferring to SYS.
• Static linking (of selective modules), which is contrary to purely modular systems like Java, but might be a viable alternative.
• Rewriting certain modules are source-level includes. Or consolidating smaller modules. This is illustrated by ED and PLASM being larger but loading faster than combined CAT modules.

Further analyzing 140k: if you slow down the video,

1. it seems like it's going over the same sectors more than once (these are probably directory scans)
2. in any case, there's too many sectors being accessed

To estimate how many sector reads are needed for CAT and its 6 imports,

• 6 CAT (main app)
• 0 CMD (should be in memory)
• 1 LONGJMP
• 1 ARGS
• 3 MATCHFILES
• 10 FILEIO (but maybe should also be in memory)
• 6 INT32

Total: 27 sectors.

Plus scanning directories:

• 6 SYS
• 2 current

Considering how much time it takes to run on these 140k disks:

• 17 sec PLASMA2-SYS.PO
• 10 sec PLASMA2-MIN.PO (only root files, CAT and its dependencies -- all in root)

(* Interestingly on the 800k PLASMA2.2mg, the CAT time is 2-3 sec from BLD folder.)

It looks like the directory scans contribute on average 1/2 of their sector scan time per each import:

• performed for each file
• but partial -- until the file is found

[tingtron]

Further developing the idea of app directory for relative modules loading, with nested modules imports becoming a new relative directory, as opposed to current directory. Here's some code changes that make it possible. It may need some optimization, but it runs and provides a proof of concept.

• cmdjit.pla -- CMD128

The cost is 120 bytes in 4,128 bytes CMD128 module.
There are some considerable benefits for 140k disks: about 27% reduced time for +CAT from root in PLASMA2-SYS.PO (from 17 down to 12.5 seconds).
For the mini system all in root PLASMA.MIN in the original post, it's only about 0.5 sec difference. But it's not worse.
For 800k disks the difference is hard to measure, but there is no degradation either.

The functional benefit is the ability to run apps outside current directory, which have local dependencies.
For example, when building and running TEST.PLA and it dependency TESTLIB.PLA keeping them in the SAMPLES directory, on the default distribution PLASMA2.po 800k disk, it just gives "File not found" error:

whereas with the proposed changes in place, it runs as expected:

[dschmenk]

Thanks for all the work looking into this. I'm not opposed to changing the order of library loads, but my initial thinking was that apps would have many local modules and only a few system modules. You're the first to point out the slow loads. I want to also look at all the prefix setting. That may also be adding to the overhead. Life has gotten busy lately, but I hope to get to this in the next week

[tingtron]

my initial thinking was that apps would have many local modules and only a few system modules

It's not so much about local vs system. It's about what local means. It seems currently local means current.
Instead we propose to make local mean app directory -- only for module loading, not accessing data files. Such as in case of a compiler, it loads it's modules from the app (compiler's) directory, but (after it loaded its modules) it still maintains the current directory to load the source file from the arguments.

System (path) will also be checked last -- same as now. It's just happens that for many utilities app folder=sys. So loading will be faster, since it doesn't need to scan current folder for modules whatsoever.