TrakSim APIs

Contents

Configuring the DriverCons Constants
TrakSim Method APIs
Class DrDemo API
Class MyMath API
Class HandyOps API
Class SimCamera
Package noJSSC
Class FlyCamera API

Configuring the DriverCons Constants

ProxThresh -- TrakSim now supports a smattering of extra hardware useful for racing. This parameter specifies the distance threshold (in park meters) for an added proximity detector. Artifacts can be marked as triggering the detector when they are closer than this threshold during simulation.

MetersPerTurn -- This parameter should be set to the the (park) distance travelled for one turn of the car's driveshaft. The self-drive software can use this number and the new DriveShaftCount() method to determine how far the car has travelled.

CoefFriction -- This parameter should be set to a value greater than zero but not (much) greater than one, representing how well your car's wheels cling to the track surface. TrakSim will "spin out" your car if you try to accelerate or turn too fast for this coefficient of friction. You also can override this number in your track descriptor file, or with the new SetCoefFriction() method.

Grav -- This should be a constant 9.81 on the surface of the Earth, but we are running at scale 8 "park meters" to one real meter. If you choose to rescale your simulation, you may need to adjust this constant so that your setting for the coefficient of friction still causes your simulated car to spin out at the proper simulated velocity at scale.

nServoTests -- This parameter lets you run the DrDemo program with no self-driving code, but still see that the servo control is working. Originally as distributed it is set to nServoTests=1, which will jerk the steering right then left once when you switch manually to the camera and click in the top edge of the screen. Any larger setting also alternates running the drive speed (ESC) up to the specified maximum and back down to zero, the specified number of times. An odd number larger than 100 runs the test continually as soon as you switch to the camera. A red "Servos Active!" warning is shown on screen so you can know not to activate it except up on blocks or on a test bench. The warning message is an image added to the distributed "TrackImg.tiff" image file; if you modify the file by moving things around, you need to adjust the compile-time constants ServoMsgPos and possibly also ServoMsgTL and ServoMsgSiz as appropriate. This release defaults nServoTests=0.

LiveCam -- When this is true, the simulator is able to work in parallel with a live camera feed. There are APIs for switching it back and forth, and for adding the display widgits to the camera feed. This is only used in the DrDemo program, and not in the simulator itself. The TrakSim cannot see the live camera feed, it only offers a second feed that (hopefully) looks enough alike to fool your code.

StartLive -- When this is true, the simulator startus up with the live camera feed (if it can). This is only used in the DrDemo program, and not in the simulator itself, and it only makes sense if it is not convenient for you to click on the screen at startup to switch to the live camera. I tried using a wireless mouse, but the airwaves in some parts of Oregon are saturated to the point of unusability (Wi-Fi doesn't work here either ):

FixedSpeed -- When this is true, the simulator assumes a constant nominal (fMinSpeed) and ignores the speed control servo except to know when to start and stop.

StayInTrack -- When this is true, the simulator tries to keep the simulated car within the track boundaries regardless of how your software is steering it, but it only works properly if the car is already on the track and the track edges are roughly parallel and within a couple (park) meters of the car sides. This is useful if you have separate teams working on speed and steering, so they can debug their respective controls while assuming the other team's code already works. On a track with crossroads and/or wyes, the simulated car will generally hold the same transit vector while crossing. It has been known to fall off the track if the curve after a wye is too sharp, and it may have trouble staying within a very narrow track.

Presumably unlike your self-driving code, the SiT mode is not watching the generated image, but is more like a child playing with a toy car, and if it gets too near the edge, picks it up and sets it back down where it belongs and pointing the right direction. The result is that it bounces around a lot more than if a smart program were driving. But that's the point of using this tool, isn't it? You get to write (and test ) your smart software.

UseTexTrak -- When this is true, the simulator looks for the file "TrackImg.txt" to construct a track (see "Building Your Own Track") for this run. If the file is missing, or if this option is false, then the default track in the "TrackImg.indx" file is used.

ShowMap -- When this is true, the simulator image (window) is widened to include a small representation of the track map being simulated, with a tiny pink avatar of the car and a trail of "breadcrumbs" how it got there from the start. A small blue line shows the approximate size and position of the bottom line (nearest position visible) of the simulated camera shot, to help you design your control software to accommodate the dead zone between the car and what the camera sees. Your self-driving code needs to be aware of this extra image information and ignore it.

If DoCloseUp=true also, then (if there is room for it) a second enlarged portion of the map is shown in the lower right corner, centered on what is in front of the simulated car. Clicking in the upper map will reposition the car to be at the clicked location, and clicking in the lower map will reposition the car to face the clicked location.

StartInCalibrate -- When this is true, the DrDemo program starts in a servo calibration mode with the simulator frozen (see SimStep). You can click on the horizontal row of boxes at the middle of the screen to run the steering servo left and right (click near the middle to step in single units, farther to the left or right to go left or right rapidly). The steering servo on the car I tested this with emitted a constant buzz when the servo was over its limit; you should note where that happens in each direction, then set the LeftSteer and RiteSteer parameters just short of it. Then click once below the middle row and the same clicks in the center row now control the ESC "servo" so you can discover and set the MinESCact and MaxESCact parameters to match where the motor starts to turn, and where it stops increasing speed (the upper limit was user-adjustable on the ESC we used). Click once more below the middle to exit the calibration mode. After initial calibration, you should leave StartInCalibrate = false. See also Calibrate Your Car.

ImHi,ImWi -- These are the camera image height and width that the simulator is using. They should match the actual camera image dimensions. The TrakSim image is generated to this size, and the DrDemo window is sized accordingly.

BayerTile -- The simulator encodes the constructed image using the Bayer8 file format, tiling pattern RG/GB, which is enum constant FC2_BT_RGGB=1 in the Pt.Grey camera API.

FrameTime -- The simulator runs at whatever frequency your driver software calls it, but it assumes it is running at a fixed frame rate, specified in this constant as milliseconds. You should not set this number shorter than the time to process a single frame in both the simulator and your driver software, but too long a frame time tends to run the car on dead reckoning, where it seems unresponsive to your controls.

HalfTall,HalfMap -- These are the dimensions of the internal (nominal 2m x 2m) grid used to generate the scene. There may be parts of the simulator code that are hard-coded to 100,128 so I don't recommend changing it.

DrawDash -- This is the height in pixels of a dashboard strip along the bottom of the simulated camera image. If specified greater than zero, the simulator will fill it with operational information: the specified steering servo position and the current frame number on the left, a nominal speed under the steering wheel as drawn, and then on the right the position and orientation of the car and the speed servo position. The information is shown in a tiny 3x5 font, which is readable in a 12-pixel dash.

PebbleSize,PebContrast -- When enabled, the track can be textured like an ordinary carpet, with random lighter and darker spots that average out to the specified track color. In these two constants you specify the approximate pebble diameter (in park centimeters = actual millimeters on the carpet, as a power of two), and the contrast, which is the range of luminance variation, from 9 (maximal effect, all the way to full black and white) down to 0 (about 2% variation). Individual pebbles are set according to a 32x32 grid of random digits that is aligned with the park map, so that the simulated texture does not change with the car position. The ten levels are computed based on the chosen contrast and default track color, then selected for display in MapColor. When you build your own track at runtime, you can choose any arbitrary ten colors for the pebbles.

CheckerBd -- This number, if non-zero, represents the size of the squares in meters to darken in showing track and non-track grounds as light/dark checkerboard. The number must be a power of 2 (0 or 1/2/4/8) for this to work properly.

SteerServo,GasServo -- This simulator tracks data sent to fakefirm.Arduino in the form of Firmata serial byte sequences, and emulates the behavior of the car's steering and speed control servos. These two numbers tell the simulator which LattePanda/Arduino servos do what.

fMinSpeed -- You need to Calibrate Your Car, then set this to the measured minimum speed. For example, if the slowest you can make the car go on the floor is 20" per second (0.5 meters per second), and if your car is 1:8 scale like the Traxxas we used, then 8*0.5=4.0, which is what you would set fMinSpeed to. If the same settings make the TrakSim simulated car too fast or too slow, then you should adjust the multiplier for the constant fGratio (in the first block of private static final double constants in class TrakSim) to make it come out right. Note that the CheckerBd is normally set for 1-meter squares, which you can time with a stopwatch (or else compare the position numbers in "(..BF..)" lines of the console log to the frame numbers in the same lines)

MinESCact,MaxESCact -- My experience with Traxxas R/C cars suggests that they intentionally suppress low-speed servo controls; I also found that driving the servo harder than a second threshold had no effect. These two numbers tell the simulator to behave in a similar manner (see FixedSpeed above, and Calibrate Your Car).

LeftSteer,RiteSteer -- Similarly, the steering servo in the car I worked with had an asymetrical reduced range, reflected in these constants. You can run the simulator in manual mode with the wheels off the ground, to see where these limits are (see StartInCalibrate, above). If you do not adjust your steering control using these numbers, then you should set them at 90 (full range) if you use the DrawSteerWheel method to display your status. The first version of TrakSim attempted to scale the left and right steering to match the range, but this caused problems. I think I got that turned off in the latest release.

TurnRadius -- This number is the nominal shortest turn radius of the car being simulated, in full-scale meters from the centerline of the car (if your model car is scaled 1:8, you would use a number eight times the actual turn radius of the model). The DrawDemo method of the DrDemo program draws a red "I" on the screen with the lower bar at the logical turn radius, and the upper bar at twice the turn radius to give you a visual feel for how quickly the simulated car can turn.

WhiteLnWi -- Like all parameter numbers, this is in full-scale ("park") meters; the actual painted white lines on city streets are typically 4" up to 6" (10cm to 15cm, sometimes more for painted curb substitutes) but digital round-off error in the simulator rendering engine makes narrow lines somewhat erratic. I got good results with nominal 10" lines (0.25m). Normally this number is overriden by the line width in the track specification, but zero in that file is interpreted as "use default" so if you want zero width, you need to set this default to zero.

xTrLiteTime -- If your are using the traffic light artifact, this number specifies how many (as a power of two) seconds the red cycle lasts. If you are using SimStep(1) to control the simulation, the time is measured in terms of seconds calculated by counting the frames (so pauses also pause the traffic light sequence time), otherwise it is real time and pausing the simulator does not stop any animations.

FastBlobs -- As an exercise while thinking about frame rate problems, I wrote "Tom Pittman's Speedy Blob Finder," which you can enable by setting this constant to the maximum number of blobs you want it to find. The code is totally contained in class DrDemo's method GetCameraImg and in the NewBlob method it calls.

PosterColors, BlakThresh,WhitThresh,EquThresh, MinBlobDim,MaxBlobDim -- Finding blobs required that I also posterize the image pixels. Not having the APW team's color names handy, I created my own, and assigned them values according to a new integer constant, where every four bits gives the 7-bit code for one of eight colors (of which I defined only seven). The positions of these 4-bit values are named in the (unused) string constant PostCoNames ("W,Y,Gy,B,G,R,Bk"), where the last 4-bit nibble of PosterColors corresponds to the last name in the list ("Bk"). The numeric values I assigned to these color positions are arbitrary, so you can give them any 3-bit values you like, but I defined names for those values that are consistent with whatever values are in the respective positions of PosterColors. For example, with my chosen value PosterColors = 0x07354210, the constant names raid=1 and whyt=7, but if your chosen color scheme is

0: red, 1: green, 2: blue, 3: grey, 4: black, 5: white, [6: yellow]

The specified thresholds are used in my posterizing scheme to choose where to draw the lines between colors. If you set the constant NoisyBlob=true then not only will this code produce a lot more diagnostics in the console log, it will also double the window size and display the posterized pixels on the left bottom of the enlarged window, and (some of) the gray luminance pixels on what's left of the right bottom.

I also defined square minimum and maximum blob sizes as combined vertical and horizontal dimensions (so you could set non-square values, if you so choose). The minimum prevents random single pixels from triggering time-consuming blob creation for random pixel noise; the maximum is used to discard overly large blobs in the final list.

I also added to the DrawDemo method some code to draw little blue boxes around the blobs it found. It doesn't look below the horizon, so I had to make the stop sign taller so it stands up above the horizon from the driver perspective. Driver view height needs to be adjustable, maybe next release.

TestTimes -- Real-time code often needs to run as fast as possible, and it is easier to write if you know how fast various language features actually run. The way to do this is with a timing loop, so I wrote a demonstrator, how you might discover how fast various snippets of compiled code run. This constant enables the TimeTest method in class DrDemo, which contains my example code.
 

SceneFiName -- The track information controlling the simulator is in a file named "TrackImg.indx" but you can set the given name (or use a full path) in this constant to something else (or even convert it to a dynamic variable determined at runtime, to choose from a variety of map files). If you are building your own track, the data files you supply use the same file name with ".txt" (text descriptor file) and ".tiff" (Tiff image file) suffixes.

Vramp,Hramp,RampA -- These are the map coordinates (in nominal park meters) for starting the car, and its orientation (in degrees clockwise from north). They are usually contained in the TrackImg.indx file, which now overrides these initial values. There is a setter method SetStart() which your driver software can use to reposition the car at any time.

Zoom35 -- This is the 35mm-equivalent effective focal length of the camera lens. Use "50" for normal lens focal length, but the simulator camera cannot see the track closer than the nominal turn radius for some configurations, so in the first release it was preset to 35 (wide angle). The summer program team zoomed the car camera all the way out to 23mm equivalent, which is the current default, but for racing you might prefer a longer focal length to better measure the size (and distance) of objects down the track. There is a setter method Refocus() which your driver software can use to choose a different focal length on the fly.

Most of the other initial parameters pertain to internal simulator features, which should be left in their default position unless you want to play with the code.
 

Scenery Colors

CeilingCo -- If you choose to show a ceiling in your (presumably indoor) track "park" this will be the default color, but it also can be defined (overridden) in your track descriptor file. In this release you can only turn this feature on in your track descriptor file by setting a Ceiling height greater than the camera height.

WhitLnColo -- This is the color TrakSim uses to paint the white lines on the edges of the track (see also WhiteLnWi, white line width, above). You can specify any single color you like, except if it collides with a pavement color the low bits may be adjusted to make it unique. It can also be overridden in the track specification text file, but only for immediate use, see "White Line Color".

CarColo,ArtiColo,SteerColo,MarinBlue -- If the ShowMap option is enabled, then the car position and orientation is shown as a rectangle on the top-view map in the color CarColo., and additional artifacts (see Artifact Specification in "Building a TrakSim Map") are marked on the close-up view map (if shown) as dots in the color ArtiColo. The same color is used to leave a trail of "breadcrumbs" along the track as traversed in the full (small) top-view map as the car advances. The steering wheel (if shown) is drawn in SteerColo, and additional information such as the bottom edge of the perspective view as shown on the map views is drawn in MarinBlue (which does not need to be blue, if you prefer some other color).

Transprnt -- Images you supply for artifacts and ground paint are automatically separated from their white matrix, which is effectively turned transparent so long as it is connected to the edge of the image file by a "bucket drop"; if you want interior regions of an image to be transparent, fill it with Transprnt. If the default color (near-white 0xFEFEFE) is otherwise used in your image file, you can choose a different magical transparency color.
 

TrakSim Method APIs

void Activate(boolean activ)

This is a new method to turn TrakSim off or back on, so that it uses very little CPU time when your car is running on an actual track using the live camera. Note that asking for a TrakSim image through its fake camera API will still run a full frame of TrakSim processing.

boolean IsItProxim()

This is a new method for supporting extra hardware that might be added to the AVP_HS car for racing. Think of a proximity detector as a "poor man's LIDAR" which trips when some metalic or sound-reflecting object like another car on the track is closer than ProxThresh. Artifacts can be marked as triggering the detector when they are closer than ProxThresh during simulation, and IsItProxim() returns true when that is the case.

int DriveShaftCount()

This is a new method for supporting extra hardware that might be added to the AVP_HS car for racing. It models the drive shaft sensor on the Traxxas 1:8 cars we used. If you multiply this times the parameter MetersPerTurn you can know the total distance travelled, which is useful for memorizing and then following an internal map.

void SetMyScreen(int[] theAry, int tall, int wide, int tile)
int GetMyScreenDims()
int[] GetMyScreenAry()

The simulated image presented to the client software is maintained as RGB internally, but some of the drawing methods can be useful to add additional information to a screen before display. SetMyScreen sets the canvas (an array of 1-pixel integers) that drawing methods use, and establishes its dimensions. You would normally call this after saving the previous canvas using the GetMyScreenDims and GetMyScreenAry methods to extract (so to save) the previous setting. GetMyScreenDims returns a single integer with the canvas height in the upper 16 bits, and the width in the lower; they need to be split apart when restoring them with a second call to SetMyScreen. The tile parameter should always be set =1.

void PokePixel(int colo, int rx, int cx)
int PeekPixel(int rx, int cx)

These two methods give single-pixel access to the selected canvas.

void DrawLine(int colo, int rx, int cx, int rz, int cz)
void RectFill(int colo, int rx, int cx, int rz, int cz)

These two methods respectively draw a line or fill a rectangle on the canvas. The two pairs of "r,c" parameters are the endpoints of the line or the opposite corners of the rectangle.

void ShoDigit(int whom, int rx, int cx, int colo)
void LabelScene(String aLine, int rx, int cx, int colo)
void SetPixSize(int size)

The text capability of the simulator's canvas engine is rudimentary, admitting to only sixteen characters, the ten digits plus the letters N,E,S,W and a decimal point and minus sign. Usually you would call LabelScene with a text string and RGB color, and the lower right corner in canvas coordinates where the text is to begin (right-to-left), or else the lower left corner if the given column position is negative (left-to-right). You can use SetPixSize to enlarge the pixels in the text (be sure to restore it to 0 for no enlargement when you are done. As an example see the calibration mode in DrawDemo, which uses SetPixSize(4) to set pixels 4x larger.

void SeeOnScrnPaint(int rx, int cx, int tall, int wide, int here, int colo)

This draws a specified rectangle (size tall/wide, from position here) of image from the "TrackImg.tiff" images file onto a specified location (top-left corner at row/column rx/cx) on the screen.

void DrawSteerWheel(int posn, boolean scaled, boolean dash2)

The simulator draws a dashboard and steering wheel at the bottom of the constructed image it returns; if you want the same information, call DrawSteerWheel with the current setting of the steering servo (scaled=true). If dash2 is true it will include a dashboard strip with the current steering servo (as last written to the Arduino, but centered at 90).

int MapColor(int optn, double Vat, double Hat, double ink)

Given a map coordinate Vat/Hat in park meters, this returns the color to display at that coordinate. The edge line width ink is used when the grid cell that covers this coordinate straddles the edge, because the map encoding does not have that information. The optn parameter enables various options in the display: +1 returns a negative value for walls, +2 shows the CarColo if the car covers the coordinates, +4 with +1 shows the track without any texture or checkboard, and +8 returns any non-track as 0. These different options are used by TrakSim to display the side maps.

void FreshImage()
void DarkFlash()

If your driver code uses any of these APIs to alter the simulator status, you may need to call FreshImage to let the simulator know that it needs to redraw its internal representation next time you ask for it. The LP system has input hardware that is not supported in this distribution, but for a quick on-off signal to your self-driving software, you can briefly cover the camera lens; DarkFlash provides a way to tell TrakSim to simulate covering the camera lens. if the car covers the coordinates,

void SetStart(int Vat, int Hat, int aim)
double GetPosn(boolean horz) {
double GetFacing()

Use SetStart to change the position and orientation of the simulated car at any time. Vat and Hat are in simulated meters, aim is in degrees clockwise from north. GetPosn returns either the current vertical position (if horz=false) or the current horizontal position; GetFacing returns the current orientation, which also might be fractional.

void Refocus(int newFoc35)

Use Refocus to change the simulated focal length (and thus how much of the scene is packed into the image returned). Focal lengths greater than 250 or less than 25 are ignored.

void SetCoefFriction(double newCoF)

Use SetCoefFriction to change the simulated coefficient of friction (and thus how much forward or lateral=turn acceleration it takes to "spin out"). Coefficient of friction equal to zero or greater than 2 are ignored. You can use this method, for example, to simulate your car running off a paved track onto dirt.

int TurnRadRow()

The trigonometry to figure out where on the returned simulated scene image you could turn to avoid an obstacle or stay on track is not trivial. TurnRadRow returns two raster row numbers packed into a single integer: the low 16 bits is the raster line representing the specified turn radius for the current focal length, where the center of the car would be after a hard right or left turn 90 degrees, or else just past the screen height if off-screen (nearer than the bottom displayed raster). The upper 16 bits is similarly the turn diameter, twice the turn radius, which normally won't be double the unit radius because of the perspective effects. Both numbers are pixel rows, counted from the top of the image.

void SimStep(int mode)
void CrashMe(boolean seen)
boolean IsCrashed()
void DrawRedX()

The simulator retains an internal state which the client program normally controls using SimStep. Initially set to mode=0 (paused), you can also step the simulated car one frame each call (mode=1), or else run in real-time (mode=2) which will drop frames if your client software takes longer than one FrameTime (including TrakSim time) between NextFrame calls. Bad Things can happen, which set the internal state to "game over" (mode=3) if your car runs off the track or is otherwise disqualified. The images will show a red "X" in the lower left corner when that is the case. You can draw that red "X" on your canvas independently using DrawRedX(). You can call CrashMe to force that crashed state, or IsCrashed to query its status. Otherwise the state is whatever your code set it to, except that you can only clear a crash condition using SimStep(0). If you are running off the camera, you probably want to use SimStep(0) to prevent the simulated car from running off the track (and crashing).

boolean NewGridTbl(int[] grids)
int GridBlock(int rx, int cx)
void DrawGrid()

If you are so inclined, you can write your own code to read a joystick or physical steering wheel to control the simulated car. Failing that, you can click on a region of the image in a Java window to specify how to steer or drive the car. You control the window and see the clicks, so you put your own interpretation on those clicks, but I found it convenient to divide the window up into horizontal rows with hot rectangles distributed in them. These are specified in an integer array which you hand off to the simulator using NewGridTbl. You can query which block a mouse click (r,c) position landed in using GridBlock, and draw the grid in red dotted lines on the current canvas using DrawGrid. The simulator does not do anything with this grid, except as you use these three APIs.

The form of the array consists of three or more runs of numbers. The first is an index to the remaining position number runs. If 'grids' is your array, grids[0] is the offset to the first run, which is a list (in ascending positions) of the tops of the active rows. Then for the first row, grids[1] is the offset to the leftmost horizontal position in that row. For example, the default grid in the simulator, "Grid_Locns = {6,12,16,18,26,28,..." has five rows, the first beginning at offset 12. From 6-11 is the list of row tops, which for a 320x240 image size I have set to "0,16,80,126,228,240" so that the first row starts at the top of the image, the second on pixel 16, and so on and the bottom row rests on the bottom of the image. The first row "0,20,300,320" starting at offset +12 has three regions, the first being 20 pixels on the left, the third being 20 pixels on the right, and the second being all the rest in the middle. If you click near the top middle of the image and pass those coordinates to GridBlock, it will return the number 0x00010002, representing the first row, the second block in that row. If a row or row position of your grid starts after 0 or ends before the edge of the image, then clicking in that region would return 0 for that row or block number.

DrDemo uses the default grid to offer specific simulator controls. There is a row of cells in the middle that can be used to manually control the steering servo (also in calibration, see StartInCalibrate). Clicking above that row switches to the live camera mode if there is a camera and it installed correctly (which is reported in the system log). You could use clicks below that row to start your self-driving software, if compiled in.

int ReadTiff32Image(String filename, int[] pixels)

Standard Java should, but does not, have a standard API for reading Tiff image files. That's unfortunate -- or maybe it doesn't matter -- because the Tiff file format is easy to understand. This is not a complete implementation, but it should read uncompressed RGB image files into the array you supply and return the height and width packed into a single integer. If you pass it null, it still returns the height and width so you can allocate enough pixels for a second call. This is most useful for reading the images to be sent to BuildMap, next.

int[] BuildMap(String theList, int ImDims, int[] theImgs)
void WhiteAlfa(int tall, int wide, int[] theImgs)

You probably want to build ad-hoc track maps appropriate to your testing environment, and this is a simple (not overly friendly) tool for creating tracks as if they were laid out on a large floor using white tape as lane boundaries, or else a different color for the track as opposed to the ground outside the track, or both. For details on using this tool, see Building Your Own Track, next. The (file format of the) array generated by BuildMap is described here.
 

Supporting Classes

Class DrDemo API

The DrDemo constructor sets everything up, in particular it goes out and installs whatever camera or camera emulation you have selected by means of your import statements and the constants in the APW3 class DriverCons (see "Configuring the DriverCons Constants" above), starts up FakeFirmata, starts up TrakSim, starts a Timer at the specified frame rate, and also opens a JFrame window to display whatever the selected camera and/or TrakSim sees. The following methods assume all that has happened correctly, and take it from there.

boolean GetCameraImg()

This calls on the FlyCamera API to fetch the next frame from whatever camera is currently selected, then de-Bayers it into an array of RGB pixels in previously allocated class variable thePixels. It returns false if something goes wrong, but there isn't much you can do about it.

void SteerMe(boolean fixt, int whar)
void AxLR8(boolean fixt, int whar)

These two methods call on the FakeFirmata interface to send a coded message to the Arduino daughter board, so to drive either the steering servo, or else the Electronic Speed Control (ESC) pseudo-servo, to control the car. Whar may be an absolute servo angle in integer degrees in the range +/-90 (fixt=true), or else a relative value up or down from the last time it was called (fixt=false).

void Stopit(int why)

Some of the hardware drivers need to be properly closed when the program quits, and this method makes sure that happens. A numeric reason or cause (why) is logged in the Java console, and also used as the parameter to System.exit.

void mouseEntered(MouseEvent evt)

This responds to a mouse roll-over into the top-left corner of the screen (from outside the window) for starting your self-driving software. It only is useful and necessary when ordinary clicks are not available, and you have set StartLive=true.

void mouseClicked(MouseEvent evt)

This responds to mouse clicks on the screen for controlling the simulation. See "Default MouseClick Handler" for an explanation of what these clicks do.

void DrawDemo()

TrakSim has several APIs for drawing on the image it returns; this method demonstrates using several of them. It has no effect other than cluttering up the window, and you can remove its call from the paint method with impunity.

BufferedImage Int2BufImg(int[] pixels, int width, int height)

This is essentially the same as the similarly named method in the example code for FlyCamera it converts an integer array of RGB pixels into a BufferedImage for displaying on the screen. I think I copied it from an example in a StackOverflow posting, but altered it to save a reference to its returned object for re-use on subsequent calls, thereby to avoid incurring the Java memory allocation time penalty. You can do double-buffered display (eliminating flicker) using previously allocated buffers, but this doesn't.

void paint(Graphics graf)

All the heavy lifting happens here. This is called on timer activation, then goes and gets the next camera or TrakSim image and displays it. Actually, it does it the other way around for less flicker, but that results in a one-frame latency to the screen. If you are adding your own self-driving code to DrDemo, and you insert it in here where marked, your code should get the image information immediately after it is captured, for minimal executive latency (ony the window display is delayed to the next frame).

int NewBlob(int whom, int row, int coln, int[] blobz)

This is the support code I wrote for "Tom Pittman's Speedy Blob Finder." It performs three separate functions (which could have been three separate methods, but I kept them together for ease of code management), creating a new blob after a sufficient run of pixels has been found, merging a single arbitrary blob with any existing overlapping (or touching) blobs of the same color, and searching the entire blob list for merge opportunities. This last entry runs in N² time, so it should be called only after the complete frame is processed.

Creating new blobs runs in linear time on the pixel array, that is, it looks at each pixel exactly once, and keeps track of horizontal and vertical runs, so that when the MinBlobDim threshold is reached, a new blob is created (or else the potential blob is immediately merged with an adjacent existing blob), after which the blob size is incremented as needed on a per-pixel basis. This algorithm assumes that the number of actual interesting blobs is small, so there is not a lot of time spent in blob management. Outdoor scenes could make this rather slower than the indoor scenery it was designed for.

New blobs are created by passing a negative width (in pixels) and height and posterized color in the whom parameter, with the current row and pixel column, and a reference to the pre-allocated blob array in blobz. Passing whom=0 triggers the global merge search.

I annotated my blob-finding code with an estimated number of machine cycles for each line to execute, and totals. Actual processing time will vary based on the quality of the compiler and the number of parallel processing units in the CPU.

I tested this on the "Figure-8" track specification in the distribution, but with the car positioned (closer to the intersection) at V=56, H=47, and it found exactly two blobs, the 5x5 green traffic light and the 18x16 red stop sign.

Using a high-contrast pebbling, I calculated the luminance of the image into the same array as the posterized colors, and using a longer (3:1) Gaussian blur the smoothed luminance image was readily distinguished from the (cream-colored masking tape) lines at the edges. Starting with a gray (instead of brown) base, the levels were rather closer, so some threshold adjustment may be needed.

This is essentially a proof of concept, and your mileage may vary ;-)

void TimeTest(int seed, int rpts)

This is a demonstrator, how you might discover how fast various snippets of compiled code run. Feel free to modify it for your own use. It is enabled by the boolean constant TestTimes. The parameters seed=0 and rpts (the number of times through the inner loop) are constants, used to confuse the compiler so it won't optimize away what you are trying to test.

I did not entirely succeed in fooling the compiler and several of my tests reportedly "took 0. ps" (which means it got optimized out). I leave "as an exercise to the student" to figure out how to fool the compiler and get useful results. At least you can see the general idea.
 

Class MyMath API

static int SgnExt(int whom)

I do a lot of packing small numbers into integers, then unpacking them for use. A 16-bit right shift to get the upper integer is a 1-cycle operation in all modern processors, and so is extending the sign of the lower 16 bits to obliterate the upper half, but Java has no atomic operation to do that. My compiler knows about the hardware and generates the single instruction for it. Sorry about yours, but it's still pretty fast ;-)

static int iAbs(int whom)
static int iMax(int lft, int rit)
static int iMin(int lft, int rit)
static double fMax(double lft, double rit)
static double fMin(double lft, double rit)
static double fAbs(double whom)

Operator overloading can be fully disambiguated at compile time, so execution is the same speed as a properly strongly-typed language, but it is confusing and error-prone. My compiler is strongly-typed, which speeds up my time, and that's more important than runtime in large projects. These methods are strongly typed both conceptually and practically, which also saves a little runtime because there's no object to null-check and dereference and no method table to index.

static int Trunc8(double whom)

This is a very useful function for converting floating-point numbers to integers. There are fast ways to do this in floating-point hardware, but Java does not give you access to them. My compiler does it with a single (one cycle) shift, sorry about yours. It's still pretty fast in Java, the static method call probably costs more than the few instructions to floor+round+"cast", so if the Java compiler back-substitutes small functions like this (I didn't look), you might still win.

static double Signum(double whom)

This is a convenience function, useful inside complex floating-point expressions. It's in the IEEE floating-point standard that the ISO standard was based on, I don't know why Java didn't implement it.

static double Fix2flt(int whom, int fbits)

Floating-point numbers use a lot of bits. If you don't need that much precision and/or range, fixed-point is much more compact. TrakSim stores a lot of numbers in big arrays as fixed-point, two numbers + three one-bit flags in a single integer, which is very fast to access and make decisions on. This function converts it back to floating-point for calculation. I guess it would have been a tiny bit faster with a switch instead of a bunch of if-elses, but not a lot.

static double aTan0(double y, double x)
static void Angle2cart(double degs)

Mathematicians think in radians, but real people with real work to do think in degrees. TrakSim works in degrees. We have angles to convert to Cartesian coordinates, and back, and functions to do it in degrees. Java returns only a single scalar as a function result, so to get both the sine and cosine out of a single trig function evaluation, I cached both results in public class variables. My compiler does this with a single table lookup and is quite fast; I don't know about Java, but it seems fast enough.
 

Class HandyOps API

static String IffyStr(boolean whom, String tru, String fls)
static String TF2Log(String before, boolean whom, String after)
static String Dec2Log(String before, int whom, String after)
static String Hex2Log(String before, int whom, int nx, String after)
static String Int2Log(String before, int whom, String after)
static String Flt2Log(String before, double whom, String after)
static String Colo2Log(String before, int whom, String after)
static String Fixt8th(String before, int whom, String after)
static String PosTime(String before)

Single-stepping in a debugger is wonderful, but it's incredibly slow when you don't know where the problem is. Besides, I cut my teeth debugging machine code before anybody even thought of a debugger, so I know how to build a debug log of useful information, which I can then examine carefully and jump around in to see where this problem showed up, and so on.

Java has a debug-logging facility, and it conveniently lets you use the built-in toString method of all objects, but operator overloading [insert invective] makes it clumsy and error-prone. So I built up a library of debug-log operators with similar syntax, which I can cascade together conveniently. I often get the parenthesis count wrong, but I still think the advantages outweigh the disadvantages. Like, I can add new types -- like RGB color and 3-bit fixed-point and relative clock time above -- without changing my workflow.

static int TimeSecs(boolean ms2)

Time is critical in simulations, and you need it to be consistent across runs, so this function zero-bases the system clock at the start time of the program (basically when the class instantiates). Times built off this time will then be the same across runs, except for tiny hardware variations. This function returns that time, either as seconds (ms2=false) or milliseconds (=true).

static char CharAt(int here, String aStr)
static int SafeParseInt(String aStr)
static double SafeParseFlt(String aStr)
static String Substring(int here, int lxx, String aStr)
static String RestOf(int here, String aStr)

Exceptions are a wonderful way to not think about data errors, so they can sneak up and bite you halfway through your debugging, and you can't figure out why this method didn't finish. That actually happened to me a few times, and in one case took me over a week to realize I was getting some kind of spurious Java exception that didn't happen in my own compiler. One of the problems that Java designers needed to deal with is type String as an object. Objects make sense for a lot of things -- particularly type-safe callbacks and forward references to code not yet written (both used in TrakSim) -- and not for a lot of other things like numbers (which Java wisely did not implement as objects) and strings. I already told you my opinion on operator overloading. A Turing-capable programming language is a marvelous thing, you can do anything you want -- even change the language itself (it's called a private library) -- so I did that. These are non-object, exception-safe versions of methods otherwise named in Java. My library :-) I'm neither greedy nor sensitive, you can use them if you like, or ignore them if not.

static int Countem(String aWord, String aStr)
static int NthOffset(int whom, String aWord, String aStr)
static String NthItemOf(boolean lino, int whom, String aStr)
static String RepNthLine(String aLine, int whom, String aStr)
static String ReplacAll(String aStr, String xStr, String whom)

Some programming languages are wonderful with strings; others are not. Java is closer to the latter category, but as I said, libraries can fix anything. Converting a convenient text representation of a track to the binary representation involves a lot of string processing, and I made up some of these for that task; others I lifted or adapted out of my working library. Enjoy! Or not.

static String ArrayDumpLine(int[] theAry, int nw, int pos)

This is another debugging tool, a way to see a portion of an integer array in the console log, with word-wrap at line ends.

static String ReadWholeTextFile(String filename)
static void WriteWholeTextFile(String filename, String content)

I do this a lot in my own code, and it's not particularly hard to do in Java, just tedious. One less thing to think about :-)

static int ReadTiff32Image(String filename, int[] pixels)
static void WriteTiff32Image(String filename, int dimz, int[] pixels)

Java has libraries that read several image formats, but TIFF is not one of them. Unfortunately, TIFF is the only well-defined image format that can be generated and read simply, without complex compression. You can write a Java program in less than a hundred lines to write a TIFF image file from an RGB integer array. Reading it back in is a little messier, but not much. So I did. I bought a utility that converts between TIFF and other image formats, but I do everything in TIFF. So TrakSim needs to read (and write) TIFF files. Here it is.

static int Int4bytes(boolean LiLEfi, byte by0, byte by1, byte by2, byte by3)

Ah, the wonders of hardware design. Writing was invented so long ago we don't know exactly how or where, but it was obviously invented by a left-handed person, who wrote right-to-left so his writing hand did not smudge his text. After it became obvious that this is incredibly useful, everybody got on the bandwagon, but they still did it the way they were taught. Except the Greeks. They were smarter, and they first tried to save time by alternating right-to-left with left-to-right, so they didn't have to pick up their hand to go back to the other side. Then -- scientists that they were -- they noticed that one direction looked better than the other (wasn't smudged), so the Greeks only -- and from them all Europeans -- now write left-to-right. Hold that thought.

Mathematics, when you do arithmetic, you start at the least significant bit or digit, so the carries propagate in the correct direction. The first computers did only whole-word numbers in hardware, so that wasn't a problem. Then somebody started automating punched cards in computer memory, and they wanted the characters to count up (so there was only one kind of counter), so they stored the numbers backwards in memory to make that happen. The people who designed computers for people (meaning European descent, because we invented them) wanted the letters and numbers to read the way people read them. The people who designed computers to be cheap (less hardware, back when gates were not free) wanted the numbers to be accessed as bytes in the order that the math had to be done, but they didn't think about text very hard -- "computers were for numbers, not text"...

Anyway, some computers came from a corporate policy of starting at the Big End of things in deference to the way people read text, and others came from a corporate policy of starting at the Little End of things in deference to the way computers do numbers, and some were so mixed up they didn't know which end was up (those computers disappeared). Intel was the first to make really tiny computers, so they were Little-Endian all the way. Motorola's first effort into computers where it made a difference was more like a mainframe (think IBM, which originally made punched cards, and first thought of computers as text engines), so their offering was Big-Endian. It would not have mattered because Intel was firstest with the mostest, but Apple chose Motorola's 68000 for the best computer operating system that was ever designed, and thinking people everywhere jumped on it, and it led in software innovation for at least a decade, so a lot of file formats were designed for Big-Endian data.

Now you have a conflict. Steve Jobs tried to solve it by throwing away the Motorola chip and going back to Intel's hardware -- probably for non-technical reasons, but that's neither here nor there -- but it was too late. At least TIFF is defined for either Big-Endian or Little-Endian data, so the processor needs to figure out which by a suitable tag in the file, and cope with it. This utility is used to unpack data in the wrong format and put it back together in the right order.
 

Class SimCamera

  public boolean Connect(int frameRate);

This does all the necessary initialization, and starts the camera going at the designated frame rate (which is specified by a number defined in the C API (only 15fps and 30fps are supported; the USB3 hardware transfer cannot go faster, and if they cannot do the processing in the available time, the presentation may look bad). It returns true if successful, or false if there is no camera or something else goes wrong. This also queries the camera for the default image size and Bayer tiling, which can be retrieved by the Java program.

  public int Dimz();

This returns a pair of 16-bit integers packed into a single integer, the height as a number of pixel rows in the upper half, and the width in the lower half.

  public int PixTile();

This returns a single number representing one of four Bayer tiling patterns, =1 for RG/GB (the FireFly default). Each pair of bytes in the even rows is matched to the corresponding pair of bytes in the following odd rows to provide the three RGB components of a pixel (and a second green).

  public int Live();

This returns false because this camera is only a fake (the real FlyCamera.Live returns true if the camera is connected and running).

  public boolean NextFrame(byte[] pixels);

This gets the next available image frame captured by the camera. The byte array should be previously allocated to the size determined by the dimensions. The pixels will be Bayer-tiled as described by PixTile(). It returns true if it successfully got another frame. Java memory management is rather slower than everything else, so in real-time applications all dynamic objects (including arrays) must be allocated once and re-used.

  public String toString();

The toString() method is defined for all Java classes, usually to represent the current state, or at least the class name; here it returns a text description of the most recent error, if either Connect() or NextFrame() previously returned false.

  public void Finish();

The camera should be turned off using this call when the program terminates, so that temporary buffers can be released.
 

Package noJSSC

boolean openPort()
boolean setParams(int baudRate, int dataBits, int stopBits, int parity)
boolean writeBytes(int[] buffer)
boolean closePort()

Everything logs and returns true. No exceptions are thrown. No exceptions are defined, because FakeFirmata does not use JSSC's SerialPortException (it is sufficient to 'catch (Exception ex) {}'.
 

Back to front of TrakSim

Tom Pittman
Rev. 2019 May 27