NWAPW Year 2+3

Technical Topics

Part 9: Advanced Projects

The LattePanda has several layers of software capability, which can be exploited to drive the cost down or else the performance up. These are discussed here, in a separate page from the main project, because getting into one of these side projects, while very useful, will not change what the main group is doing. There is greater risk here, so we expect that participants who choose to branch off into one of these areas will be highly motivated to go after additional information, and probably work harder than the main-line teams in the previous pages. But you can do that, that's why you are here.

Part 9a: The x86/Win10 CPU -- Java

Heretofore we have assumed that the entire project would be programmed in Java running in the IDE development environment. This a great environment for software development, but it requires some expertise to get it running. If APW is going to morph into a service project, bringing autonomous car programming to groups outside the Portland area, we probably need some stand-alone programs -- they can still be programmed in Java -- which the user can double-click to demonstrate:
i.   That the hardware is running correctly,

ii.  What kinds of things the entry-level participant can look forward to, and

iii. Perhaps a tutorial for getting started.

There may be other applications that I didn't think of, but which make sense in our broader context.

The team involved in doing this would expect to code up one or more of these apps, possibly building on code developed in the main project.

Part 9b: The x86/Win10 CPU -- C/C++

Most of the hardware available for use in single-board computers in robotics applications has no Java drivers or APIs. Heretofore I prepared Java glue code to make the operation seamless. As we transition to a student-led project, we need one or more teams in place to:
iv. Build the Java glue code for new hardware as it becomes desirable to incorporate it in the project, and/or

v.  Transition the development environment into C++.

My experience is that working in Java offers productivity far exceeding the cost of extra glue code, but I seem to be in a minority in this opinion. There is opportunity for a team or subgroup to explore this option, perhaps by translating the Java code at some (working) stage of the main team's development into C++ to see how hard it is to work in that environment.

Part 9c: The Arduino

The LattePanda computer as shipped includes an Arduino slave processor on the board for doing time-dependent signal processing using the Firmata Open-Source code. The tools I provided for programming your autonomous car assume we use Firmata in the Arduino unmodified (as shipped). In one sense this is overkill, because we only use the PWM conversion built into Firmata; in another sense, the Arduino is being under-utilized, but we could:
vi.  Rewrite Firmata code so it does exactly what we need (and thus eliminate package dependency), and/or

vii. Translate some of the hardware currently being added on into Arduino code -- for example, the deadman switch -- into additional Arduino code, probably added onto or replacing the Firmata package.

Either or both of these would require the team so tasked to learn how to program the Arduino (there are rumors that its compiler is actually for Java ;-) and develop procedures for installing our custom code in place of the existing Firmata code.

Part 9d: The Graphics Processor

Although I did not see any mention of it on the LattePanda website, at least one other website said the LP computer includes a graphics processor (GPU) which can be programmed independently, just like any other GPU.  If the LP processing power turns out to be a limiting factor -- IMHO, good software design should keep the power requirements well within the LP budget, but y'all might decide to get more ambitious -- then programming the GPU gets interesting. OpenCV is an API for programming GPUs from C/C++, but at least one Java binding exists.
viii. A subgroup or small team could be tasked to investigate how to do this effectively, and to determine what kinds of computations can be usefully off-loaded to the GPU.

Part 9e: MIPI Camera

We are currently using a camera with a USB3 interface, which is an order of magnitude more expensive than some of the MIPI cameras available to RaspberryPi-based systems. Assuming we want to continue development in the more productive Java+Win10 environment, we could still switch over to a MIPI interface with suitable interface hardware + driver + Java glue.
ix. A subgroup or two could spin off to study the software requirements to implement a MIPI interfaced camera.
One possibility is to use a MIPI camera with a USB interface added, which is still somewhat cheaper than what we have. The cameras Steve looked at recently mostly had (uncompressed) YUV coding, for which you can find simple conversions to/from RGB here, and byte ordering here. You still need to develop the necessary Java driver glue -- see item (iv) above to rebuild the drivers.

Part 9f: Raspberry Pi Alternative

The LattePanda computer is an order of magnitude more expensive than computationally equivalent Raspberry Pi systems, but they are programmed exclusively in C/C++ on Linux. Java is available to run in Linux, so we could, with suitable motivation,
x.  Port our development environment and Java code over to Linux on Pi using (iv) above to rebuild the drivers, or else

xi. Use (v) above to port our development environment and code over to C++ on Linux on Pi.

Part 9g: A User-Friendly Website for Growing the Program Outside of Portland

It has been suggested that this might morph into a service project for participants to export their experience and acquired skills to similar groups located in other states, and in particular to areas without the geographical presence of high-tech industry to drive the priviledge of students. Some ideas include:
xii.  Design an interactive website to attract prospective group mentors and participants,

xiii. Design and implement a startup procedure for bringing another group up to speed understanding the technology, and then adding to it in their own geographical region, and

xiv. Come up with a competition to which the various groups can bring their creations to both compete and share for the benefit of all participants.

I consider all of these options to be significantly harder than developing autonomous car software in Java on Win10, so there is a substantial risk that you may not succeed in completing your efforts in the four weeks of our summer program, but these are all useful things to do for the long term success of the High School Autonomous Car Project (or whatever you-all decide to call it), and if you are suitably motivated, you could continue working on them after the main program ends August 10. Almost certainly you would have something to show on August 10, and a statement of why it is useful, and what is left to finish -- sort of the way the Neural Nets group did last year, except presumably you would actually finish and your work would become part of the group project next year.

Any questions or comments? This is your project.

Tom Pittman -- Starting July 13 through the end of the 4-week workshop, use the email given on July 16 to reach me.

Back to Part 8
Back to Part 7

Rev. 2018 July 9