Aug 232010

This previous week I worked on loading road data to a database. Since there may be some question, this relates to my thesis in the following ways… Currently the road data is loaded from a CSV file and all of it is loaded at once (i.e. a small part of my local area). So if someone drives out of that area then there is simply no data. Using a database will allow me to dynamically load the road data based on a general location and all of the roads in the US can be stored (offline) and retrieved when needed.

Additionally this will be useful if my next thesis adviser wants me to create something that gives directions from one address to another. Also this will be useful for GeoCoding if that ever comes up.

So I’ve downloaded all of the “edge” files from the Tiger “edge” files from the US Census Bureau and wrote a program to process each one, one by one. Additionally I’ve installed MySQL. This week I may work on creating the appropriate users, tables for loading the data, and actually loading the data (if I get that far).

 Posted by at 7:36 pm
Aug 162010

Match GPS Data to Road Data

Last week I tried choosing the closest road segment instead of using the angle as the most important attribute. This gives significantly better results. However, the results still aren’t perfect… of course. The found road location jumps back and forth from one end of the road segment to the other. I’m fairly burnt out on trying to find the correct road location, so it’s time for a reevaluation again.

As I currently have no direction (aka no Thesis adviser at the moment) I’ll have to make up my own possible next steps.

  • Get the road location to actually work.
    • probably involves assuring movement on a road segment matches movement from the GPS, while still taking into account other things like distance to the segment, etc.
    • once this is done there is an issue with the perspective transformation that causes the road lines to disappear in some cases. I have no idea how to fix this.
  • use color information (this would theoretically work in the same way as edge detection currently works, only edges are defined by significant changes in color).
    • the academic papers I’ve read say this does not work very well.
    • shadows change colors so the first bit of research would be figuring out if colors can be “normalized” so that green areas in sunny areas can be seen as the same green as in shadowy areas.
  • use curve fitting to fit two curved lines to the image.
    • apparently this works well but it can’t handle intersections which is why I dislike the idea.
    • requires research on how it works, since I don’t know.
  • try to add to the edge detection algorithm.
    • if it’s added to then it won’t be strictly edge detection anymore. I’m not sure what else to add to it.
  • look through the academic papers again for other attributes for recognizing road versus non-road.
    • the one about recognizing country roads probably has some ideas.
    • the various techniques should probably be cataloged anyway since they may be useful when I actually have to write the paper.
 Posted by at 8:22 pm
Aug 152010

WP-SpamFree and Akismet are plugins for WordPress which are meant to stop spam comments. Unfortunately they both have their issues. To jump to the point of the matter – you may want to only use WP-SpamFree and not use Akismet at all.

My experience has been that Akismet has reportedly blocked a fair amount of spam, but that I have no way to confirm this. There hasn’t been any approval or review of the spam it’s blocked, just some “XXX spam messages blocked!” messages. While this is great and all, it got me to thinking – what if it’s blocking valid comments as well. After a little search I found that Akismet may actually work by using large blacklists of IP addresses and marking any message from the blacklisted IPs as spam.

That’s bad for a number of reasons. If some Comcast user starts spamming people and gets their IP on the blacklist, their IP may change the very next day so some new unrelated user will be banned instead. If a whole range of IPs is banned instead of just one, then a whole range of unrelated users will be banned instead of just the one spammer. Without any verification on what Akismet is calling “spam,” I can’t in good conscience just let it continue to block messages.

Configure WP-SpamFree

WP-SpamFree has two settings (“Blocked Comment Logging Mode” and “Log All Comments”) which allow all blocked comments to be logged. Unfortunately the default is that this setting automatically turns off after 7 days. To configure WP-SpamFree to keep the log for more than 7 days, do the following.

  1. Under “Plugins” click “Editor”
  2. Under “Select Plugin to Edit” choose “WP-SpamFree” (and click select)
  3. Search for the text “ResetIntervalHours” in the wp-spamfree.php which should now be open.
    1. This is where the time limit is set to 7 days
  4. Simply adjust the 7 to be whatever you like, like 28 for example, and click “Update File”

Won’t the log file become HUGE?

No, probably not. If you look down a bit further in the code there is another setting which tells WP-SpamFree to turn off logging once the file reaches a certain size (2mb is the default currently). So once the file reaches 2mb the logging will turn off automatically. Personally I believe these things should be options changeable via the menu.

 Posted by at 3:46 pm
Aug 092010


Although I previously said that matching the current angle of travel to the a nearby road with a similar angle of travel in order to match the current location to road worked just fine, that’s actually incorrect. What happens when the vehicle turns a corner is that the angle of travel does not match either of the roads it’s currently on. For example, if approaching an intersection which looks like a “T” and runs North/South and East/West, then while turning the vehicle is traveling somewhere between North/South and East/West. So the angle of travel doesn’t match either of the possible roads to which one would hope to match the location.

Match GPS Location to Road Data (Attempt 2)

Perhaps one thing that went horribly wrong with the previous attempt was putting more weight (importance) on the current angle of travel instead of the distance to a road segment. Assuming the GPS data isn’t off by too much, then the distance to the nearby road segments should be the most important. i.e. initially there should be a choice between 3 or less road segments based on distance.

Ideally, a path of travel would be saved and matched to nearby road segments, so that if the vehicle travels North then turns West, we could match to road segments which travel North and then West. Unfortunately I don’t have any ideas on how to implement this. It seems difficult because the GPS data gives you a lot more points to work with than the map data, to the point where the GPS data seems continuous compared to the discrete map data.

So next I will snap the GPS’ location to the nearest road segment and test the results. From there I will try to come up with a way to solve issue mentioned in an earlier post where the “snapped-to” road’s angle of travel does not match the current angle of travel at all. Currently I have two ideas: 1) Don’t allow the road to change until the angle more closely matches the new road and 2) detect when near an intersection and (similar to #1) allow the road to change when the angle of movement matches the new road more than the current road. For 1 and 2, if the road looks like a “T” then 45 degrees would be the threshold for changing the road that’s currently “snapped-to.”

Two Weeks Ago

Two weeks ago I worked out various bugs in the methods which remove non-visible lines and do rotations and transformations. The fact that using a “Graphics2D” instance applies things later instead of immediately and the fact that PerspectiveTransform’s can’t be done using a Graphics2D instance caused me to have to rewrite much of the code so that all of the rotations, transformations, etc. could be applied in the correct order.

Last Week

Last week I augmented the simulation to not only include a GPS location with map data, but to also include the image captured when the GPS data was captured. So now I can test the various algorithms without driving around. From this I noticed that the “snap-to” algorithm works about 10% of the time and needs to improved.

 Posted by at 7:34 pm