NWAPW Year 2+3
Part 8: Software Components
In last year's pedestrian project we identified several distinct software
components which we distributed among five small teams for implementation.
We can do that again, at least initially...
Part 8a: Hardware Interface
We are using the same camera as last year, so most
of what we learned last year -- reassembling the pixels from the RGB bytes
in the Bayer8 data, despeckling, presenting the image in a window for the
rest of the world to see -- can be recycled. Last year we looked for a
half-dozen posterized solid colors to eliminate the effects of shadows,
but this year we need to go the other direction, initially toward a luminance-only
image from which to extract the boundary white lines. If you are wildly
successful, you might have time to try to infer virtual white lines from
a boundary between uniform pavement and the visually noisy off-track stuff
like parked cars or vegetation. We can call this the initial image processing.
At the other end of the processing chain we have two kinds of output,
presenting the same video image in a window with added annotations so we
can see what your software thinks it is doing, and more importantly, actually
controlling the car (steering it and setting its speed). Last year Peter
did the visual integration but he may have other obligations this year,
so you should think about image post-processing as a separate and
important software component.
We have some example code on
the LattePanda forum for using the Firmata APIs for controlling servos,
but determining the proper timing and settings to get the hardware servos
to behave consistent with our intentions usually involves substantial trial
and error. Servo control might be a software component in its own
right, and if you do it well, you will have a valuable skill for a career
in robotics or other hardware/software interface fields in the computational
universe. If you hope to scale this project up to a larger car with more
inertia, there will be a response time lag between the commands issued
by the decision-making software components and what the car actually does.
You can start to prepare for that kind of project trajectory by emulating
a response envelope in your servo control in this small model where the
response otherwise seems effectively instantaneous.
A really ambitious group might replace the Firmata code in the Arduino
with your own PWM generator (Custom Arduino code),
but you would need to learn a lot of specialized tools, which could be
a valuable skill in its own right.
Part 8b: Track Following
Finding the middle of the lane in the visual image is pretty easy. Deciding
which parts of that middle line to pay attention to in deciding which way
to steer the car requires analysis and experimentation. Whatever your camera
might see very close to the car cannot have any useful effect on steering
the car, because you don't have time to respond to it before the car has
passed that part of the track. Similarly, distant parts of the image are
irrelevant to steering the car, because you may never get there if the
track turns away. You need to build a software model of which parts of
the middle line are most relevant, and a weighting scheme to set the steering
analysis appropriate to it, so that discontinuities in the middle-line
data do not confuse your car.
Part 8c: Time and Speed
Your first efforts to steer the car will probably leave the speed control
manual (using the radio remote). That way, if -- maybe I should say when
-- something goes wrong, you can stop the car before it crashes into something
(or somebody) simply by releasing the trigger on the remote control. Then
you can program the car for a relatively slow steady speed, slow enough
so somebody can walk along side and pick the car up off the ground before
it crashes into something. When you have a high degree of justified confidence
in your steering control, you can ramp up the velocity.
The the car speed increases, you will begin to notice the effects of
how long it takes your software to process the incoming image before you
tell the steering wheels which way to turn. You cannot do sharp turns on
the track at high speed, not only because the car might spin out or roll
over from lateral acceleration forces, but also because your software cannot
respond quickly enough. This is why race tracks have no sharp turns. As
you extend your car control towards higher speeds, you need to adjust the
planning that goes into steering it to accommodate the increased effects
of latency. If your programs sees a sharp turn coming up, you probably
want to decelerate the car going into the turn. Speed analysis is
an important part of making your car more sophisticated.
Part 8d: Developmental Support
Last year one member of the group maintained a GitHub repository
for the project code. Somebody needs to be responsible for getting that
up and maintaining it again this year.
The LattePanda has Wi-Fi hardware built in, so you might find it convenient
to build a Wi-Fi remote interface to view what your software is
seeing and doing on other computers connected to the internet (like your
laptops). You could also use such an interface to start and stop the car.
This is not essential to driving the car, but a great convenience, and
seeing in the auditorium projector what the car sees in real time makes
a compelling presentation on the last day.
Part 8e: Dividing into Work Groups
As I see it, these are the essential software components discussed above,
which you should staff appropriately with team members (the links refer
back in this page to where the terms are discussed):
1. Initial image processing
2. Steering analysis
3. Speed analysis
4. Servo control
5. Image post-processing
These are some tasks that may or may not deserve separate working
group support, but you should think about:
6. GitHub repository
7. Wi-Fi remote interface
8. Custom Arduino code
If we have enough participants, there are some interesting tasks
that can be worked on in parallel, then integrated after the basic car
functions work properly:
9. Adjusting the speed for the
processing latency (slower for tight turns, faster for straight)
10. Predictive steering,
so the car drives more smoothly
11. Detecting a stop sign and stopping at the right place
12. Inferring implied lines from parked cars or other non-traffic lane
objects in the scene
13. Staying an appropriate distance behind another car in the same lane
14. Making turns (from a script?) when there are choices
15. Passing another car in the road (probably
need extra hardware, like another camera)
16. Parallel parking (probably need extra hardware,
like more cameras)
or comments? This is your project.
Rev. 2018 March 1