Arkayne is doing something huge yet subtle, we’re elevating the atomic unit of the web from keywords to pages.  We are doing this because there are over 1 trillion pages on the web today (2008) and keywords, tags, hashes, or categories are often falling short of describing relevant content.  We can do this because recent technology has pushed the computational bar to where Arkayne is possible.

It’s not easy, pages consist of words, ideas, themes, topics, points of view, and even writing style.  To fully grasp all that about a page and make a recommendation on other pages like it requires serious computational horsepower.  Think of what happens to your PC when you open a large spreadsheet, now imagine if you had to open millions of spreadsheets all at once and run dozens of macros on each.  Until recently, that power was mainly available to large companies and universities.  Tough for start-ups like Arkayne to break into.

Enter Amazon Elastic Compute Cloud (EC2), a pay as you go cloud computing network.  The system allows start-ups like Arkayne to rent computers by the hour to almost any scale without having to buy a data center.  Even better, we don’t need to hire security, bring on maintenance staff, pay for electricity, or worry about physical backups.  That type of freedom lets us focus on our core customer oriented product, currently the relevance plugin.  This paradigm is going to be a huge driving factor for opening up the next revolution in online data mining and the resulting services.  In our case it allows us to undertake the task of cost effectively fulfilling customer linking needs by tuning our algorithms.  We are hoping to repeat this success until Arkayneis cross linking every page on the web based on relevance.  Well almost, we still need technology to manage these large systems cost effectively while providing scalability.

Source: Amazon EC2 Pricing Comparison Chart

Scalability and cost effectivness is where Amazon Elastic Map Reduce comes into play.  Based on the now famous Hadoop Map Reduce developed by Apache(no it wasn’t the company that ends in ..oogle).  The system allows us to break mind bendingly huge sets of data points into segments a computer can easily compute in seconds and do them all at once.  Since Amazon already did the work of seamlessly layering this into their EC2 infrastructure Arkayne can undertake the task of linking all 1 trillion pages without having to build complex data distribution technology. The best part is our customers never need to worry about managing these technologies, Arkayne takes care of that aspect of creating the links that appear on their pages.

Arkayne is not only using these technologies today to get off the ground but betting that they’ll get cheaper as time progresses.  Just like YouTube bet on falling bandwidth prices, Arkayne is betting on falling cloud computing pricing.  So far our bets are paying off, Amazon has dropped pricing every few months on various services.  The most recent of which is the introduction of Reserved Instances, lower cost computing by paying in advance one year blocks.  We have been evaluating EC2 value proposition monthly and at least for the near future its an absolute life saver for our startup.

The best part of ramping up Arkayne on the Amazon Elastic Compute Cloud and Amazon Elastic Map Reduce is that we can be agile, responsive, and very quick to overcome daily scalability obstacles.  In a startup this is critical.  We are still discovering customer needs on a daily basis at this point.  Surprisingly Amazon has been really good at not making our technology unfairly dependent on their infrastructure, this takes the worry out of Arkayne growing up.

More from BlogGlue

• BlogGlue Partners With YARPP
• Make Money With Advertising
• The State of Cloud Computing

BlogGlue Recommends

• It’s Official – Amazon Severs California Affiliate Program  (Brandon Hann)
• Is Cloud Computing Really The Future?  (Brandon Hann)

Ask BlogGlue To Recommend Your Posts