NWAPW Year 4

Technical Topics

Part 11: Racing

Racing a self-driving car on an F1-style track (see the F1/10th website) is a game-changer. Everything you knew about self-driving cars is worthless. Almost. Maybe we get to start with a green light. You know how to see a green light, but all that effort to find it is worthless. When your car is at the start line, you get to mouse over to and click on where the green light will be. When it goes green, you never look again. Everything is the race, nothing else matters.

There is no green light in the most recent rules. You start your car and they time the lap time. I have not seen how they start the two cars doing head-to-head racing, but the rumor is it's like the Oakland race (see video link below) where the race manager gives a verbal "Ready, set, Go!" and then your person holding the remote pulls the DeadMan trigger and the car goes. They have a little dongle that plugs into your receiver for power, but I think that's only some kind of beacon that their computer can see as you pass the goal line (for timing), because the only electrical connection is power and ground.

This is a game-changer in two ways. The second way is that all the other contestants are using stale NeuralNet (NN) technology. These are university graduate students, and NN technology is what they teach in all the universities. NNs is what they teach at Portland State. You will change their game by leaving them in the dust. NNs cannot find and optimize the racing line on the day of the race, there isn't time to train them for this track and this starting position. They need hundreds and thousands of training data, there isn't time for that, they can only give their NNs generic training on generic tracks. Your designed code -- should you accept this mission -- can be designed to

a. Memorize a track it has never seen before, complete with where the track edges are, as well as the start, all curves and the finish line (you can even memorize the floor pattern or markings, but they may be hard to distinguish at race speeds);

b. Calibrate the coefficient of friction of your car's wheels on this track floor;

c. Calibrate the maximum safe acceleration from a stop, and from any given velocity (like out of a curve);

d. Calibrate how much braking force to apply and when, in order to achieve maximum safe deceleration from any particular speed to arrive at any other (slower) speed at a particular point in the track;

e. Calibrate incidentals like what speed & steering settings give what results in acceleration and turn radius (which may be different for left and right) and even friction (which might affect the steering), and how much additional speed control you need to apply to maintain a particular speed while turning (because the turned wheels will resist forward motion) -- basically, know your car;

f. Do the math necessary to calculate the fastest (smoothest) racing line through the track, and how fast your car can go on every part of that racing line;

g. Build alternative racing lines for starting the track on the inside, or (optional) if an opponent is too close for you to move onto your preferred line, or (more important) if you have drifted off your preferred line; and

h. Follow the best available racing line through the track at the fastest calculated time.

Then if you want to participate in racing two (or more) cars at once,
i. If you fully lap the other car, don't crash into it.

j. (Not necessary to win your first competition) how to pass another car according to the racing rules set out by the management (none announced as of Jan.31, and apparently nothing catastrophic happened in Montreal in April, so still nothing). If you are fastest out of the start, and if you are fastest down the straightaway and fastest in the curves (because you did the math), then you will never need to pass another car, and no car will catch up to you to pass your car. If we guessed right, and you fully lap the other car, then this lets you win by more than one lap. If we guessed wrong, then this might salvage a poor start or weak curves.

Watch the videos of the previous races. Notice that the cars are all over the place. This is NNs at work. They will be faster at the next race, but only incrementally. Their NNs are your secret sauce: your designed code can beat any NN hands down. You can do the math. You can program your LattePanda to do the math. NNs don't do math. NNs cannot do math, they only can learn specific tracks and specific generic moves. Here are the videos:

2019 Montreal: https://www.youtube.com/watch?v=c0Hn7srEe-o
2018 Turin: https://www.youtube.com/watch?v=8NSKLdjGp7I
2018 Porto: https://www.youtube.com/watch?v=L5iJm3AojGU
2017 U.Penn: https://www.youtube.com/watch?v=5EUhNoRzvZ8

Notice especially how the winning car did the Montreal track: it was all over the place. Their car is obviously trying to calculate in real time where to go next, and not doing a very good job of it. They know how to do NNs, but they don't know how to race (drive the track optimally). You can do that.

Steve offered this video from the Oakland competitions (unrelated to F1tenth), where they commented on how smoothly the winning car drives. It still wanders around a lot, and does not seem to be using the straightaway to gain a speed advantage, so I don't think he's memorizing the track, but you can look:


You will not only beat the NNs in this race, but you will beat them using a computer 1/20th (or less) the power of their GPUs, and no LIDAR. You will beat them using a program written in Java and running on Windows (they will think of it as a disadvantage, but you know better). They have been working on their system all year -- some of them going on three years -- but you can beat them after four weeks. How sweet is that?

Don't get too cocky, it's a challenge. But you can do it.

I saw the tech report describing the technology used by the Czech team who won at Turin, and did not find any mention of neural nets. That's probably why they won. However they are spending a lot of effort on unifying their various data streams. With only video to worry about, you have nothing to unify, so you can spend all your effort on making your car go fast.

You can only do this once, this year, and never again. After everybody else sees how well you did (or maybe after they see how well the Czech students did last year), they -- the university students, people with brains, all of them -- will learn from you, and then you cannot beat them again. If you don't compete this year, they will probably figure it out on their own, so this is your only chance. They have more time and more resources and more education than you do, so on a level playing field they will win. But this year, you can take them by surprise, this year you have an unfair advantage. Next year? Maybe we'll find something else that's mind-boggling.

Rules, as of March 1

The F1/10 Webinar on March 1 released the following fragment of the rules we will need to work with:

8"-12" tall
Wide enough for 3-4 cars
Mix of cardboard walls, and plastic ducts

[You can see examples of ducts in the previous race videos]
Rules -- New Regulations Summary
There will be two classes of vehicles in this years competition.
* Restricted class: You start with the Traxxas Rallye 1/10th scale platform and only use sensors and motors that are no more performant than the example build. [We will conform to this class]

* Open class: Your vehicle is within 10% of the size of the Traxxas platform, it is electric. No tethers, no remote commands during the race.

* There will be two racing events. They have different qualification procedures.

* Time Trial: E-Stop activated by external controller, low-speed restriction until two laps at 2 m/s without crashing.

* Head to Head: Same as above. Additional demonstration of ability to avoid static obstacle (eg a box) and a dynamic obstacle (eg a moving box).

* Qualifying code is the only code you can race with. For example qualify using follow pure pursuit, race with a neural net is not allowed.


Race Format -- Time Trials

Heat 1
1. 15 minutes per team to complete as many laps as possible.

2. 5 mins of setup time.

3. 4 number of restarts per lap allowed.

Heat 2

1. 10 minutes per team to complete as many laps as possible.

2. 5 mins of setup time.

3. 4 number of restarts per lap allowed.

Fastest lap time determines the winner.

1. Team members (of the racing team) are allowed to chase their car.

2. Team members (otrt) are allowed to be on the track during the race.

3. Once the car crosses the timing bridge for the first time, it cannot be under manual control. (other than the case of a cold restart)

4. You cannot send any control commands to the car, besides disable.

5. Racing stewards will use E-stops if the car crashes, or appears to have lost control. (A restart is allowed in this case).

6. If you have exhausted all your restarts in a lap and still crash, that lap counts as DNF and you must start from the S/F line again.

7. You must complete at least an entire lap to be eligible for the winning prize.

8. The decision of the racing stewards is binding.


Special Races-- Head-to-head racing

* Head to head race
* 2 cars at a time.

* Round-robin style, 2 races per round where grid positions are swapped.

* Determination of fault in collisions will be at the sole discretion of the racing stewards.

* If a car is driving unsafely we reserve the right to end the race.

* Adjustments to format will be made based on number of participants who qualify and the general capabilities of the vehicles.

* Racing will require fitting a rear 'plate' to each vehicle. The organizers will provide.

More rules are likely to come out after the Toronto race this spring.

Part 11a: Resources

We have the camera, and you know how to use it. That's a plus. We also have Java and Win10. That's another plus. You don't think so? Go look at my analysis here. If you have a good reason to disagree, let me know (see "Feedback" below) and we can look objectively at where the facts lead.

Distance to the track boundary ahead of you is as simple as measuring the height of the boundary wall in pixels and comparing it to the calibrated height at a known distance. Track width you get from measuring the number of pixels between the walls at a given distance (which you know from their height), then comparing width to height. You might need to get a little inventive to figure out the track curves from where the walls change height suddenly (because part of the wall curved away) or more gradually than expected (because the wall curved in, across the track).

Last year we tried to get the driveshaft input, but did not succeed. It was a hardware problem, and I got it working, so now you can use it to measure actual distance and speed. It's a simple interface, just the number of turns of the driveshaft in a one-second (or fraction of a second, your choice) interval.

Their race track is bounded not by white lines, but by 4" dryer vent hose (or 6" HVAC tubing) piled two-high (or else by cardboard uprights). They are using LIDAR to measure the distance to these sidewalls, but at race speeds you don't have that kind of time. You will measure (and memorize) the track boundaries going around slowly the previous day. You don't need LIDAR, your camera can see how far the walls are by how tall they are in image pixels. Do this at low speed and get it accurate, then drive from memory, using the camera to correct for drift. There will be drift, because the undercarriage of this car is sloppy, and because the floor (may be) slick so any jerks will result in sliding -- but you shouldn't have any jerks. You want to minimize the drift, but you can look for landmarks and recover. TrakSim is being upgraded to give you 4" dryer vent hose for walls and a coefficient of friction that will slide when you take a curve too fast, so you can get your basic algorithms going in a safe environment. It will be different from the real track you have in Portland, and that will be different from the actual race track in New York in the fall, but you can calibrate it on-site in less than an hour -- if your software is well-written. You can do that.

Do the math. Calibrate your car's performance on the actual race track the previous day, so you know how fast you can do the curves of which radius, then calculate what radii you need to do the track curves, and how fast you can do that without spinning out. Then go that fast. It will be the best anybody can do, and you can do that. NNs cannot be trained to do the math, but programming can. That's your secret sauce. It's how you can win.

That's it. No NNs, no LIDAR (laser range-finder), no IMU (inertial + magnetometer for absolute positioning), no GPU (graphics processor), those things are a distraction that would interfere with making good code. You will win on pure smarts, and you will beat the university grad students using all that extra stuff. Look at the videos. You can beat that. If you want to.


Why is this an advantage? For one thing, it sits on top of the car so it can have a full 270o view. That makes your car top-heavy, which reduces the speed you can take the curves. Did you watch the Montreal winning car video? Their car almost tipped over going around a curve too fast.

Then there is the problem of unification (merging the data from separate sensors). Everybody says it is very hard. One of the F1/10 organizers was complaining that the contestants used only LIDAR and ignored the camera. Why is that? Unification. It's hard.

What does LIDAR give you? Omnidirectional (270o) distance to the track walls (and to other cars when you go head-to-head). You can get the distance to the track walls in front and forward sides from the camera, and it's already unified with the camera image.

When you are in a head-to-head race with another car, you will come out of the stops faster (because you know how fast you can do that without losing control), so you will be in front all the time, and you only need to know how close the other car is when it is in front after you lap it. The camera can do that, same as the walls. There are not yet rules for this (there may be by the time you get to race, but it seems unlikely), where you need to know when the other car is at your side and the rules forbid you to swing to the other side of the track (the side that car is on) for the upcoming curve. That can only happen if the other car is going too fast for the curve and will spin out -- that is, it didn't slow down as much as you did -- but you can have an alternate racing line that always leaves room for the other car. Or maybe Steve will gives us a "poor man's LIDAR" (proximity detector) so you can drive more conservatively when there's a car there.


Why is this an advantage? For one thing, it sits way high on top of the car to get its magnetometer away from the heavy magnetic fields from the motor. That makes your car top-heavy, which reduces the speed you can take the curves.

Then there is the problem of drift. Every IMU in our price range drifts, and they drift differently depending on the slew rate (how fast you are going or turning). What are you going to do to correct that? That's called fusion, and it's incredibly difficult.

We can measure the distance travelled using the driveshaft counter, and we can calibrate the steering servo setting to know how much the car turned in a given distance, which gives us direction. There is still some drift, but you are already looking at the camera. If you memorize landmarks behind the track walls and the curves themselves and maybe even the floor pattern, you should be able to correct for the drift.


Why is this an advantage? For one thing, it's hard to partition your code so it parallelizes easily. We have four cores on the LattePanda, which we can build dedicated threads to maximize the use of; but making good use of 64 or 128 parallel processors in a GPU is something else. We have four weeks to make this happen; we may not have time to master the arcane programming styles needed to effectively parallelize the code for a GPU. Your competitors using NNs need that GPU because NNs are the slowest way to do very stupid driving. Your smart code can beat it.

Part 11b: Tasks

These are just ideas. You can (collectively) allocate your efforts differently if you wish. But we need to get started, because four weeks go by quickly. Last year we barely finished, and this year there's a lot more to do.

Racing Line

For every track, for every different car on that track, there is an optimal racing line and an optimal speed at each point on that line, better than which you cannot do in that car on that track. A very important task for winning in a race is to know your racing line. That means you must calculate what the optimal (maximum radius) curves are around every turn, and connect those curves with straight lines, then (knowing the curve radii) what the maximum velocity you can safely run around each curve, then (knowing the length of each straight segment and the velocity you need to be going at the end) how long you can ramp up at maximum acceleration before you must hit the brakes, and how hard and how long to brake so your car is going the precise velocity it needs to go around that curve.

In reality, when you account for drift (and possibly leaving space for the other car), you probably want to be able to tell your driving code how much to turn and how fast to go for every position and direction on the whole track map, no matter where your car happens to be and no matter which way it is pointing. That way, if you spin out but don't tip over, or if you do tip over and somebody puts you back on track (the rules allow that), your driving software can know exactly where and how fast to go from here.

To do this, you need to know the coefficient of friction, which tells you both how fast can you safely do a turn of a given radius without spinning out, and how fast you can accelerate and brake, also without skidding and losing control.

You also need to know the track map.

Driving the Racing Line

This task is concerned with staying on the racing line and maintaining the correct speed. You need to know exactly where on the map the car is at each instant in the race, so that you know when to accelerate or brake and how much, and so you know where and how much to steer. All of these actions must be planned in advance because of latencies in processing time and actuation. This is important: you are driving from memory; all the camera does is verify (and correct) your idea of where the car is.

To do this, you need to know the racing line, and you need to know the track map. You also need a corrected position tracker.

Position Tracking

Both during the race, and during the track memorization before the race, your car needs software to track the position of the car in real time, which you can do by watching the driveshaft turns count and the current steering angle. You also must watch the video feed, looking for landmarks.

Once you have a map to compare to, you will compare those landmarks to your current track estimate to correct that estimate. Driving the Racing Line requires an accurate location on the track map. When constructing the track map, those landmarks can be added as annotations to accurately place the car and correct for drift.

Time Management

Jerky motion tends to destabilize the traction, so that you lose control of your car at lower speeds than if your changes in acceleration are smooth. What really happens is that sudden changes in acceleration (the third derivative, for those of you thinking in calculus) tend to generate additional spike forces, which if the total at any instant exceeds the coefficient of friction, you lose traction. Don't do that.

Probably you will want a separate thread in your running software whose sole duty is to plan upcoming turns and drive settings, then gently move into the desired setting at the right time. The Driving software would give the Time Management software the parameters as needed. Part of your development efforts will be to calibrate the run-time latencies, so that you can factor them into your time management. You may need to do that as part of the calibration run, because different software builds may result in different latencies.

Building the Map

Before the race, and every time the track changes, you need to make a slow transit of the whole track to memorize what the track dimensions are, and what landmarks to look for, so that the racing line can be constructed for this track, and so the Driving code can follow that racing line. If necessary, transit the track extra times to pick up angular views, and to verify your position tracking. The rules released on March 1 tell us you get 10 minutes on the track the previous day. Figure out how to make the best use of that time.

The Map team and the Driving team and the Position team need to collaborate on a suitable data transfer format

I recommend a separate Image Processing team to convert the raw camera data into a pair of 1-dimensional arrays giving wall height (in pixels) and wall type for each horizontal position in the image. A Finite State Machine implementation is discussed here.

Calibrating the Car and Track

Every car will have slightly different acceleration and steering parameters, which the Driving code needs to know exactly. If Steve doubles up the battery for the race, the acceleration changes. You can calibrate the steering by setting the steering angle at different fixed settings, then driving around a circle and measuring (using the driveshaft counter) the circumference, the distance travelled until you see your starting landmark. I used an orange traffic cone for this kind of testing, because it's distinctive and has a precise edge. You also know how far away it is (from how many pixels tall the image is), so you can check for drift. You can do this anywhere, because it's particular to the car, not the track.

It will help if you also calibrate the focal length of the lens -- you don't need this as an absolute focal length, but rather, how many pixels (per meter) a known object fills in the camera image at one meter distance. If you change the lens zoom, you will need to recalibrate this. Similarly, you need now many turns of the driveshaft it takes to cover one meter on the floor. How is it different when the car is going in a circle? Is it different at different speeds? Know your hardware!

Then when you get to the race venue, you need to calibrate the coefficient of friction (CoF) on the track. You do this by driving the car in a fixed circle at higher and higher speeds until you lose control, then record how fast and what the radius was. You should try various radii to verify that the equation is linear (and to average out measurement errors). You can also measure the CoF using a hand-held digital baggage scale, dividing the force to drag the car sideways across the floor by the weight of the car as lifted.

You also need to calibrate the acceleration and braking forces. On a carpet or some other very high-friction surface, calibrate the CoF by going faster and faster in a tight circle until it flips (be sure to pad the upper part of the car) or otherwise loses control. Now you can calculate the acceleration using the formula:

Acc = v2 / r

You know the velocity (from the driveshaft counter) and you know the radius of the circle (either by measuring it, or else again using the turns counter, or preferably both). This gives you an acceleration figure where the car loses control sideways. Run the same test on different floors; with very slick floors, the car will skid instead of flip, but you still know where you lost control. The Racing Line team needs to know this formula as a function of CoF. You will tell them the CoF (better: write it to a file, which you deliver with an API) after you run your tests on the actual track, so they can calculate the Racing Line for that track.

On the same high-friction floor, from a dead stop, give it full throttle straight ahead (not even the slightest hint of a turn). If you succeed at lifting the front wheels off the floor, or if the wheels spin -- this is hard to tell, but you might infer it from the turns counter -- back off and try again until you maintain control. That is the fastest safe acceleration for that CoF. Then run the same test from a steady (known) slow pace, then medium and higher, to confirm that the fornula is linear. If not, at least learn what the curve is for your car. You also need to give this formula to The Racing Line team, so they can determine how fast to do the straights.

Part 11c: Teams

More than the previous two years, this is a big challenge, this time far too much for a single person or a single team to complete in our four weeks. Assuming this is what you want to do, you need to choose up working groups appropriate to your tasks.

Data Transfer

The first year we tried to get each pair of groups that shared data to define their data formats and prepare dummy data for the downstream groups to use until you had valid data. It didn't happen, but now it is essential. It will also help the upstream groups better understand what you are doing.

This is very important, you must prepare hand-made data files for the next group to work with. Do this first. Work with the team that will use the data, so that you can produce updated (more accurate) data in the same format, and so that they can write software to use both your estimates and the corrected actual data.

For example, the Calibration group needs to supply numbers to pretty much everybody. These should be stored in a file on the computer SD drive so they will still be there the next day. You might provide an API to read that file (at startup) and then to supply requested values on demand. If you define a formatted (or tagged) text file, then your dummy test data can be created in WordPad. Invent reasonable numbers, like 50% for the CoF, 0.5G for max acceleration, A=ESC#/100, (turn) radius=100/StAngle, something credible. Have a well-defined place (folder) for the most recent good data file to be kept, and keep it current, but only after you are convinced the new numbers are better than the old. Archive each version, so you can revert to it if a newer version fails.

The Map Memorizing team needs to define a map format that contains all the information the Racing Line and Driving teams need to do their jobs. Sit down with them and agree on what that is, then prepare a data file -- preferably text, so you can create a simple dummy file with credible data -- and an API for them to work with while you are trying to get your memorization going. The advantage of an API is that if you change your mind about what data you need, or the file format, and you can add new access methods while preserving the methods the downstream teams are already using. Your initial map data can be as simple as a rectangle with a divider down the middle. After you decide how to represent curved walls, make your data resemble one or more of the previous competition tracks (now or soon included with TrackSim), but you can start with a simple oval instead of the rectangle. Later on you can add APIs for landmarks. If we have enough people, it makes sense to have a dedicated group to focus on reading landmarks (and/or floor patterns) first to build the map, and then during the race to correct for drift in known position.

The Position Tracking team will be using the map for coordinates, but your work is all live. Build an API that returns an incremented distance position and a (made-up) compass direction. That information is available in TrakSim, so your initial data should be easy to do. You can update your access methods after you have real data from the car. You should try to have an API that works the same in both TrakSim and on the floor.

The Racing Line team might annotate the map, or else offer coordinates in the format used by the Position Tracking. Work with the Driving team to choose something they can use with as little hassle as possible.

All of this needs to be running by the middle of the first week. Choose a format or API that you can improve on over the course of the four weeks, as you better understand what you really need.


Steve wants me to be the "Product Manager" for your program, but I think we can do better than that. I'm a resource to keep things working -- I will try to make sure the guidelines are being followed everywhere -- but each team can choose their own tech manager who is responsible for making sure your team is making forward progress. If you get stuck, get help. There's no shame in being willing to learn from somebody else, that's the only way civilization moves forward. Don't keep spinning your wheels, we don't have time for that.

Also each group should select one of their members (it could be the tech manager, or you could pick a different person) to act as liason with other teams. Every morning, the liasons from each team should meet together and discuss how they are doing, how they are supporting the downstream teams, whether they need help, and so on. I will try to be present, but this is your show. We'll try to keep these meetings short, so you can get back to work.


"Agile programming" is the term given to getting the smallest subset of your product working as quickly as possible, then improve on it continuously. If we do this, then we will very soon have a car that will run the Racing Line, but as the summer progresses, it will go faster and faster without crashing.

Last year we had several participants who felt that they had nothing more to contribute to their team. If you reach that position in your team, and if your tech manager agrees, let's get you re-assigned to a different task. This is a very challenging project, and we (as a group) cannot afford for people to be sitting idly by and not contributing. Be pro-active! If you have extra time, learn what the other teams are doing, and maybe you can help them make it better.

I gave some thought to how you might choose up your teams here.

Back to Part 0

Part 11d: What to Do First

The most important thing you need to do before July 8 is come up to speed on the tools you will be using.

Most of the participants took Java in school, and if you learned C++ instead, it is very much like Java except for the spelling of a few keywords and some goofy ideas about operator overloading (which you can safely not use) and the library APIs (likewise), so you should be able to work in Java on this project with no problems. We will be programming everything in Java, and you need to know how to do that.

You also need to be familiar with GitHub (you should have already used it in the first-year NWAPW session).

A. If you don't already have it, get a Java development installed on your computer. If you don't know how, you can follow my instructions here for Eclipse. If you don't succeed, get help, preferably a local guru who can understand what is going wrong -- because computers are incredibly picky about what you tell them to do, and when (not if) you make a mistake, things will go wrong -- and this guru can look at what you are doing and offer suggestions to correct the mistakes. Mistakes happen all the time in our industry, there's no shame in finding them and fixing them.

B. Install TrakSim in your Java development environment. If you have trouble, tell me (see "Feedback" below) so I can fix the instructions to be more clear. Run the simulator in the Stay-In-Track mode so you can see how it works. Change the first line of the "TrackImg.txt" file to choose a different track descriptor and try it again. Compare what you see to each of the previous F1/10th videos and ask yourself if TrakSim's Stay-In-Track mode would make a good race contestant. Why or why not? Be prepared to tell William about your observations.

C. Download last year's code from GitHub and see if you can get it working on your computer in TrakSim. If you have trouble with this, perhaps William can help you. This is not crucial because you will be creating all new code this year, but it will give you valuable experience with the tools.

Back to Part 0

Part 11e: Feedback

Questions or comments?

The link below is designed to thwart spammers: It has an encrypted timestamp which expires in 24 hours. If you already have my real email address (new this year), use it. Otherwise click this link, but preserve the timestamp. The spammers don't do that and are blocked.
email Tom Pittman

Rev. 2019 August 12