A Time to Count the Bounties of Big Data

Posted on November 19, 2012 by Kevin Cox

As the Thanksgiving holiday approaches, we here in the US often take the time to come together, reflect on the year that was and count our good fortunes. Of course, the long Thanksgiving weekend – including Thanksgiving Thursday, Black Friday and Cyber Monday – is one of the busiest shopping events of the year. As big data analytics become even more important to retailers in the US and internationally, we at Actian decided to look at some of the statistics surrounding the unofficial kickoff to holiday shopping season.

The US Department of Agriculture calculates that 254 million turkeys were raised in the US in 2012 – up 2 percent from the year prior. Six US “turkey states” account for nearly two-thirds of all gobblers sold – with Minnesota ahead of the flock with 46 million turkeys produced.

Last Thanksgiving, 46 million turkeys were consumed on Thanksgiving alone, according to the National Turkey Foundation. Nearly 88 percent of Americans eat turkey at Thanksgiving. The average Thanksgiving turkey weighs 16 pounds, meaning that approximately 736 million pounds of turkey were consumed.

The USDA also estimates the US will produce 768 million pounds of cranberries in 2012. Wisconsin leads all states with an estimated 450 million pounds produced, followed by Massachusetts at 210 million. Also this year, the US will produce 2.7 billion pounds of sweet potato and 1.1 billion pounds of pumpkin.

A study conducted by Nerdwallet shows that 90% of 2012 Black Friday ads show the exact same prices and items as last year. The study also showed the median price of Black Friday items increasing from 2011, so consumers should expect to pay more.

The higher prices indicate a better year for retailers, and this is echoed in the findings by the National Retail Federation that holiday spending will increase by 4.1 percent this year to $586.1 billion. To keep up with increased demand, retailers will Hire an estimated 585,000 to 625,000 seasonal employees this year.

In recent years, retailers have been helped by the increasing popularity of “Cyber Monday,” the Monday following Thanksgiving where retailers offer deep discounts on online sales. Shop.org expects that trend to continue in 2012 with online holiday sales expected to grow 12% over last year’s holiday season to as much as $96 billion.

In addition to cornbread stuffing, cranberry sauce and turkey drumsticks, this time of year presents CTOs and CIOs with countless big data opportunities. As software and service offerings continue to be difficult to implement, sprawling in scope and exploratory in their objectives, companies should continue to tackle these unique data challenges with solutions that are fast, affordable and east to manage.

Have a great Thanksgiving and save travels, from Actian!
Check out our products page and learn more: http://www.actian.com/products

Going bonkers over big data – but does size matter?

Posted on November 12, 2012 by Sean Jackson

Big data: it’s amazing how two small words in the English language have found such great resonance in the world of business. Moreso how every vendor is seemingly trying to jump on the big data bandwagon to promote their offering that may (or in some cases may not) have something to do with large, complex and untamely data volumes. But let’s not get sidetracked here. By all means go bonkers about big data: take customer data and your corporate information, add in some of the unstructured variety, mix it up with social network feeds, centralize it on hardware, even put it in the cloud – but remember that it is not a question of how much or how complex your big data volumes are, it’s what you do with it that ultimately counts. Put another way, it’s one thing to collect and store umpteen amounts of data, it’s quite another thing to mine and analyze it and make something out of it that you can turn into value.

Some may pontificate over whether big data can be summed up by the “3 Vs” (volume, variety, velocity) or the “3 Is” (immediate, ill-defined, intimidating)[1] not to mention those that then try to outsmart each other by finding other words beginning with either V or I to continue the now tedious debate (variability, volitilty, inordinate, insurmountable etc), but they’re missing the point: the smart businesses out there don’t get bogged down in trying to define the term, they get on with the task of turning it into commercial advantage. While others debate, the clever businesses take action and get on with it.

But who is doing it? Data aggregators and information service providers are two types of companies whose business models fundamentally depend on data. They don’t just store it; they depend on it to power and run their business. Consequently, they have seen the opportunity to turn big data into commercial value and many have grasped it with both hands, whether it be social network sites, gaming companies or SaaS vendors. And their use cases vary from one to the other: many are web-based companies that take user data and correlate it with advertising data in order to attract more advertisers to part with their dollars in an attempt to get the right, targeted message to the right audience in a non-obtrusive manner. Others are specialists in taking data feeds from a plethora of sources and then allow users to access and interrogate it in order to drive business. A good example here is those service providers in retail that take EPOS data from grocery chains and then allow CPG vendors to access that data and interrogate it in order to improve customer satisfaction, increase spend and build loyalty. Likewise, shipping companies broker the best courier deals for end-users by collating large data volumes from freight companies. Media companies analyze viewing habits data and allow advertisers to run commercials at the optimum time. The list goes on.

In summary, the data explosion continues to take the business world by storm and it would be foolish to play down its significance when trying to understand it. But in the main, it’s not the size that counts; it’s what you do with it. So, by all means, go bonkers over big data. Try to tame the big data tsunami, if you dare. But make sure you put your energy into where it counts the most – in analytics and enabling commercial action.

[1] source: Forbes, http://www.forbes.com/sites/davefeinleib/2012/07/09/the-3-is-of-big-data/

Taming the Data Maven

Posted on November 6, 2012 by Ian Taylor

Once upon a time, the wise old CEO asked for monthly sales and production figures. Not unreasonable. However, our high-tech, large-scale computing platforms have turned this innocent dream into a nightmare. From spreadmarts to data warehouses, our infrastructure groans with the weight of accumulated data; yet good quality results are paradoxically harder to achieve than ever.

When Actian asked me if I would speak on the challenges facing organisations at Big Data World in London recently, I jumped at the chance. Cutting through the hype of big data is a personal crusade right now, as I firmly believe there are important ideas and solutions being obscured by the smoke and mirrors of the big marketing machines. Let’s face it, your data isn’t getting smaller is it? Every graph and chart you see has an exponential curve – and it points only upward.

On the one hand, this is really not a problem. Moore’s Law continues to help us with both storage and processing speed, and box sizes continue to shrink in every product cycle. On the other hand, what to do with all this data is a harder question. Wikipedia defines a Maven as One who understands, based on an accumulation of knowledge. I think we’ve got the accumulation piece covered. Certainly, the millions of shining, spinning platters woven into your SAN fabric tell a tale of storage on a massive scale. But do you achieve understanding from all this data? Here is the nub of the issue.

There are two key topics we must address if we are to find the real value and competitive edge that we know is buried in our data mine. These are Scope and Question.

Every organisation of any size has a multiplicity of data stores covering everything from a humble spreadsheet to a strategic data warehouse (DW). But have you noticed that despite all your efforts, mission critical data keeps popping up in spreadsheets? And it isn’t in the DW? We need to acknowledge that our quest to store All Data in the data warehouse is doomed to failure. Even if we have the resource and the budget – and these days, most of us have neither – by the time we have completed the programme to absorb All Data in the DW – yet more data stores have appeared! This is what it is like to live on an exponential curve – you’re always behind.

In parallel, we have moved from merely storing data so we can create a quarterly report or prove an audit trail, to asking questions about our data. Did viewers see our ad campaign and download our apps? When users downloaded our apps, did they take up the trial offer? Were any of these new users also existing customers in another segment? The list of potential user queries is now inexhaustible, and it’s burying most IT departments ability to produce reports. As if that weren’t enough, deploying software frameworks like Hadoop and using MapReduce technologies offers the capability to manipulate yet more data content, and facilitate yet other lines of analysis!

With a big data context, we also need to consider data sources outside the corporate firewall and in the cloud, like Salesforce.com, Google Analytics, Twitter, Facebook, and others. If your people are not already using some of these services, they soon will be. And once they do … the next step is analysing that data. At FlyingBinary we are focused on deploying and integrating cloud services and combining data from all sources into a 360° data and analytics ecosystem.

Resolving the Scope item for your organisation means identifying the real key data that the business wants to access and analyse. This is likely scattered across all the data stores listed above. We need to collect this key data together and this is where Actian Vectorwise really shines. Easy to set up and use, fast load and go importing, commodity hardware and best of breed query results per dollar invested. Slide this alongside your other data stores and you avoid any disruption or downtime to existing business flows. Oh, did I mention that it’s a true column store database (meaning it’s designed expressly to manage rapid, high volume analytics)? That too. Hadoop? Plug that right into Vectorwise. No problem.

Along the way, we also need to change our thinking and recognise that the true data owner is the business user. In IT, we are merely the custodian. This can be hard to do, but the benefits are huge.

For the Question piece, we need to provide business users (yes, again) with a self-service capability where they can safely ask adhoc questions themselves and publish the results to the community. This can include the wider, web-connected world, but more typically is the business user community within the organisation. We use Tableau Software for this presentation layer, with its universal data connectors, data blending from multiple sources, one-click publishing, and browser consumption using no plugins. Of course, 100% of the visuals it produces are totally aligned with the human visual system. But you knew that.

Here too, we need to change our thinking to align with our users’ needs for data consumption and analysis. We need to curate, assist and educate, rather than hoard, obstruct and govern.

Using a combination of applying innovative technologies like Actian Vectorwise and Tableau Software, and changing business thinking around data ownership and information access we can finally achieve the understanding we seek, alongside the accumulating we already profess.

Just In Time For Halloween: Big Data Isn’t As Scary As You Think!

Posted on October 30, 2012 by Kevin Cox

Earlier this year Actian surveyed over 100 of our customers on the things that scare them when it comes to big data analytics. Now, just in time for Halloween, we’ve put together a spooky little infographic with the results.

In reality, big data isn’t as scary as you might think. Some 83% of the CIO’s we surveyed plan to use a big data solution in the coming months. But still, the fear persists: CIOs reported they are worried about cost, complexity and speed – the three demons of the data center.

Take a look at the grisly infographic below – if you dare!

Click to expand the image and see all the eerie details…

Is data the lifeblood of your business?

Posted on September 18, 2012 by Sean Jackson

You might think that a strange question to ask. After all, what company would admit that data is not core to their business? But whereas many organizations use data in their operations to manufacture a product or create a solution, others fundamentally depend on and use data to provide a commercial service. As industry experts and analysts continue to publish statistics on the amount of data being produced every minute of every day, the fact is that many companies are discovering the value in collating, aggregating and storing data and then offering other businesses the ability to interact with it and analyze it to their benefit. Call it what you want – big data, complex data, savage data – the fact is that data volumes are only headed one way and there is definitely a marked uptake in the number of savvy companies out there that are making good use of it and turning it into profit. In these circumstances, these savvy companies live and die by the ability to store, mine and analyze data and then sell that as a solution. To them, data is very much the lifeblood of their business.

These companies span a wide variety of verticals; many organizations in different markets have recognized the benefits of storing, collating and analyzing data and then selling the intelligence or access to the data as a solution or a service. For example, many newly-founded companies in the retail and online sector are collating information on buying habits, product promotions, advertising revenues and sales data from merchants and then selling access to that data to suppliers, sponsors, analysts, CMG companies and even back to the merchants themselves. In that and many other examples, the ability to offer a high performance analytic environment where interested parties can interrogate data that suits their business model is imperative. More importantly, so is the high performance analytic engine that underpins the analytic service. Imperative in the sense that the engine must be unobtrusive to the enduser and must deliver fast performance when data is queried. These new services are powered by the new analytic engines that differ vastly from legacy database solutions that tend to be costly, obstructive and engineered in the 1980s.

Even if data is not yet deemed to be the lifeblood of your business, and you can see an opportunity to offer an analytic service that is a viable commercial proposition, then ask yourself these questions:

Could you open up new revenue streams by managing, analyzing and selling data and intelligence on information you can gather in your market?
Could you sell your solution via a SaaS model and improve it through offering interactive reporting and analytics?
Could you deal with the analysis of social media data and sell insights to companies?
Could you create extra income from aggregating data and allowing companies to interrogate it for their needs?

If the answer to any of these questions is yes, then take a look at the new generation analytic database engines to power your analytic service and solution. Data volumes will only get bigger and more complex, that we know for sure. But armed with a high performance analytic database such as Vectorwise as the engine that powers your service, at least you can move forward and capitalize on the data growth and reap the rewards. Go on, have a think about the sorts of reporting and analytics you could offer businesses in your sector. Captalize on the data tsunami growth that is taking place and make data the lifeblood of your business. It may just be the best investment you ever make in your business.