I did a couple of major changes -
1) The SD card is now raw sectors. No filesystem, just packed bytes one after the other.
2) Ditched the Arduino based SPI library and went with Digilent's DSPI library.
3) Streamlined the DMD rendering to use Chipkit specific latch-registers instead.
Using raw sectors I have immediate access to whatever sector I need. The sectors of the SD card is conveniently stored in 512B-chunks and three chunks make up one frame of animation. The FAT filesystem requires me to find filenames first and is really made for non-static data. By letting a script generating the SD card file for me I know exactly where each file is placed on the card, allowing me to access it directly. There's also no cache, so if I ask for a byte - I get it. Besides direct access when needed, this allows me to pause, rewind and speed up animations on demand, for example.
By leaving the rather limited SPI library, I've managed to increase the speed of both SD card reading and the time it takes to update the DMD. They're now also on two independent SPI-channels allowing them to be run at the "same" time, that's is - being active at the same time. The DSPI-library also allows me to do interrupt-transfers. That means I could download data from the card in the background while the program continues execution, kind of like multithreading on a PC. I have not yet enabled this feature, but I'm looking in to it.
The DMD rendering was pretty tight previously, but by using the latch-registers (LATxINV, LATxSET, LATxCLR) together with a higher speed I got an extremely nice performance boost. I did a little restructuring as well so that I could perform multiple operations at the same time, for instance - toggling the latch and row-clock directly after each other instead of having to wait for the first one to finish entirely. As each of these calls are done 28 672 times, each nanosecond quickly builds up! I've also done a little manual loop unrolling to further maximize performance. I've spent quite a while reading the spec-sheet for the display to make sure that I don't delay more than absolutely necessary. I've also added an exponential delay between each row update so that the overall image looks much better. By having a fixed delay, the low end was represented in a non-visually pleasing manner, since the low end was very sharp. It simply looks better to have smaller differences in brightness in the low end and bigger differences in the brighter areas.
With these major changes I've pushed the performance of the SD card reading a full frame from around 14 ms/frame to
The nice thing about this is that I got 5 milliseconds of reserved space that can be used for any processor heavy calculating, such as transitions, layer handling, transparency etc - without it affecting the frame rate at all. As long as the total time is 5 ms or less, that is.
I could push the number of frames further, but the game logic and SD card reading must take exactly the same amount of time to execute, otherwise the display will change in brightness as soon as the SD card reading takes place. This is highly unwanted, but I've seen this behavior in "proper" pinball machines and it's nothing that I want in my machine.
If the SD card reading gets further pushed down the overall refresh rate will only increase, until a breaking point is reached when the game logic takes longer to perform.
I'm quite satisfied with the progress so far!