By Aaron Huff - reposted from PrivateFleets.com - http://www.privatefleets.com/blog/2008/01/12/mining-the-web-for-loads/
January 12th, 2008

If you’ve ever set your sights on an item on eBay, you’ve experienced the frustration of losing your bid in the closing seconds. The competition might have been a savvy eBay enthusiast using an automated, last-second bidding tool such as EZ Sniper (www.ezsniper.com) that gives no one time to respond to their bid and keeps their knowledge of the value of the item private.

In the transportation industry, scavenging for loads on shippers’ websites is beginning to follow a similar trend. Most large shippers have built, or are in the processes of building, Web portals that carriers use to view, bid on, and accept available loads. Shippers also require carriers to use these same Web portals to update the delivery status of loads.

Many carriers have already invested in software to automate the basic transportation transactions via electronic data interchange. However, EDI is no longer a competitive differentiator in this new environment. EDI is used to exchange load offers and load acceptances between shippers and carriers, but it is typically used just for those loads and lanes that are already under contract.

Increasingly, carriers that use EDI are manually visiting various shipper websites to find loads to transport on their equipment or to broker out to other carriers. To a lessor extent, private fleets may have to search several load boards for backhauls or monitor shipper websites as well.

“One of our EDI customers posts loads on a website, and these loads are available for low bidding,” says Kenny Cornett, vice president of operations for D&D Sexton Inc., a refrigerated truckload carrier based in Carthage, Mo. “All other loads are EDI offer, a phone call or email offer when we are not primary or secondary on a lane.”

Having to manually search for loads on shipper websites puts many carriers at a disadvantage. Smaller carriers and private fleets often lack the resources to dedicate staff to monitoring websites to quickly grab new load postings.

“The problem right now is that carriers have people sitting in front of a machine, hitting the refresh button looking for new loads,” says David McCarty, marketing director of Intelek Technologies. “It is very human and time intensive.”

In November, Intelek Technologies released StripMiner, an electronic data interchange (EDI) load automation tool that continuously gathers new loads from multiple trading partner websites. StripMiner works by logging into secure sites and checking back at regular intervals to “scrape” load information that is posted in HTTP. It translates the HTTP code into standard EDI load offers (called a 204s).

Fleets can set up unique business rules and criteria for loads in StripMiner such as length of haul, origin and destination, McCarty says. When StripMiner finds loads that meets some of the criteria, it pulls a load tender into a carrier’s dispatch software system automatically as a standard 204 transaction. Load planners and customer service representatives can then review the load offering directly from their dispatch screen.

If a load meets all criteria, users can set StripMiner up to automatically accept the load and generate an EDI load acceptance (990) transaction to the shipper, or accept the load directly on the website. StripMiner can also automatically update the status of the load directly to a shipper’s website, McCarty says.

The time and labor savings gained by using StripMiner depends on the number of websites a carrier has to monitor. A large trucking company that has been beta testing StripMiner, McCarty says, had been using 20 people to monitor 12 websites each day.

StripMiner integrates with DiamondMine EDI, by Intelek Technologies, and other EDI translation software to process, track loads and communicate directly with carriers’ enterprise software.

After over six months of development and testing, StripMiner is now available. For more information about the product, including a video demonstration of how the product works, go to www.stripminer.net.