We want it all, and we want it now – has the era of instant gratification made us less patient?

Do you ever get the feeling that people have no patience anymore?  No time to stop, reflect and do things at a more leisurely pace?  I do.  It seems that the more access we have to information, the more we want and the less tolerance we have to wait patiently for it.  And this affects our lives in all manner of ways.  From waiting for an analytic report to run on our work computers to growing annoyed at slow movie downloads on our devices at home; from standing in line at your local coffee shop while someone in front of you cannot make up their mind to waiting for what seems an age behind a crowd of folks to disembark an aircraft; from waiting for the lines at the supermarket checkout to clear to hanging on the end of the phone waiting for the call centre to take our call, it seems that the world has grown a little less tolerant.  We certainly live in the era of instant gratification, and yet it has made many of us a little crankier when we don’t get what we want, when we want it.

We want it all and we want it now

You might say that technology is to blame for this change in behaviour and many would agree: as we all have become accustomed to high-speed networks and super fast bandwidth speeds, information can be delivered to us in a fraction of the time it took a few years ago.   Negatively or positively, this has had an immense effect on our expectations.  Before, we would be content to drive miles to the store and shop, now we expect to go online and have products delivered to us instead.  Before we would go to the train station and wait for the train, now we want to check its status, buy tickets and be notified of any delays on our devices live.  Before we would go to the video store and rent a VHS copy of a movie, now we want it on our TVs in almost real-time.  And with us all having computers, tablets, e-readers and mobile phones, many of us with several of each, it does seem that we want to consume it all, and we want to do it right now.

However, others may say that technology is not to blame, that it is still a conscious human decision whether we search and consume a bit of information as opposed to focus on something else.  How many of us have turned to our Blackberry or iPhone at home to read and respond to a work-related e-mail in our free time instead of perhaps conversing with our family and listening to their day?  How many of us have chosen to BBM a friend or SMS a work colleague instead of focusing on what we started out doing?  How many people do you see with phones and devices almost glued to the palm of their hands checking out things online, conversing with friends, tweeting or updating their Facebook status?

Those that say technology has made our lives easier are, of course, correct.  And that goes from the simple ability to skype a loved one on the other side of the world in a matter of seconds, to technology such as our own that allows businesses to connect to various data sources, analyze their data and act upon it in real-time.  But as technology makes our lives easier, it has also upped our expectations to be able to get information and do something with it at rates faster than ever before.

In fact, I suspect that our expectations will only get more and more demanding as connection speeds get faster and our daily lives go at an even faster pace.   For, the faster we can consume information, the more challenging it will become for us humans to take the rational decision to stop, think, reflect and do something a little less hectic, with a little more tolerance and a little less impatience.  But that is a failing of us humans, not technology per se.  The main thing to note is that we still retain the option to choose how and when we consume information.  So next time you feel yourself running out of patience over something that hasn’t happened in a heartbeat or over a bit of information that you cannot obtain in a nano-second, just relax, count to ten and remember that life can be enjoyed just as well at a slower pace as it can at a fast one.   Indeed, we may want it all, but do we necessarily have to have it all right now?

Posted in Actian, Big Data, Vectorwise | Tagged actian, analytic database, analytic datamart, analytic engine, analytic RDBMS, analytic service provider, analytics and business intelligence, big data, big data analytics, business intelligence, database management, free database download, vectorwise | Leave a reply

Big Data at the Magic Kingdom – a blessing or a curse?

It used to be the case that retail or telco or utilities were the markets that were getting the IT world abuzz with stories of big data analytics and making money out of large and complex data, markets where customers can be very fickle and operating margins very tight.  But an article earlier this year in the New York Times shows that big data and the ability to collect, transact, manage, analyze and act upon it is just as applicable to markets with which we wouldn’t normally associate data.  The big data analytics and reporting needs of retail organizations and telecom operators are by no means passé, but theme park operators and leisure companies are harnessing the big data wave, too, in order to deliver better customer service, up their game and drive profits.   It seems that Mickey, Minnie, Pluto, Goofy are on hand to act upon big data, too.

The article showcases the great wealth of information that not only can be collected on theme park visitors’ habits – from what rides they prefer to what signature Mickey Mouse keyring they chose as their souvenir of the day from the gift shop – but also on significantly improving the user experience.  By dispensing with coupon tickets, turnstiles and, in some cases, cash, Disney is now planning to arm its visitors with electronic wristbands and getting them to use them for purchases in the park as well as check in at sensors in order to ride they favourite attraction.  As such, not only can Disney ensure that visitors don’t waste half of the day by standing in line, they can also ensure that they are happier and potentially spending more money in their stores.  Families can pre-register online to ride at 1pm and turn up just 5 minutes beforehand and swipe their wristband.  That certainly beats getting in line at 11am and waiting 2 hours in the soaring heat for the 90 second ride.  Sounds good, right?

Sure, for the park operators this harnessing of the big data generated by all this online and sensor activity and using it to their advantage sounds great.  If they can reduce waiting lines, improve the user experience and fully digest who does what when, who buys what when and who wants to see what attraction when, then the park operator can tailour the park to meet the needs of the consumer and profit accordingly.  Happier guests will recommend the park to their friends, happier guests are sure to return themselves and happier guests may even spend more in the park’s stores, given that they are spending less time in the lines waiting for the rides.

But isn’t this all a little too much?  Especially when theme parks are a place to go to escape from the realities of life for a while?  You might argue that the simplicity and innocence of theme parks – especially Disney and the market it serves – will be lost in favour of a system where it is all about gleaning as much information from the visitor as possible and using it to the operator’s advantage.  Will parents be happy standing with their kids on their shoulders watching the parade of smiling cartoon characters on Main Street knowing all too well that theme park operators are busy behind the scenes analyzing, crunching and mining information about them and their activities both done and pre-planned for the day ahead?  Or is this now just the way of the world?  After all, theme parks are businesses, too.

With all things in life and certainly with big data and personal information, you have to find the happy medium.  I am sure that parents who pre-register their childrens’ information into the parks’ systems will not want to divulge too much data, nor have that information abused, but then they might appreciate the fact that their kids’ wristbands will tell Cinderella that one of them is celebrating a birthday and have the cartoon character wish the kid a very special day automatically.

While it is fair to say that theme park and leisure companies do need data in order to compete more tactically – somehow building a newer, faster rollercoaster ride doesn’t seem to cut it anymore these days – it is also fair comment to make that theme park operators need to be sensitive to the needs of their customers:  offering a tailour-made and extra-special experience at the Magic Kingdom sounds great, but overstepping the mark and making it too intrusive can do untold damage to the theme park company and its reputation.

But whether you believe it to be a good thing or just way too over-the-top, my guess is that digital wristbands and data analytics at theme parks are here to stay.  It seems that no business can afford not to embrace and ride the big data analytics wave.

Posted in Actian, Big Data, Vectorwise | Tagged actian, analytic database, analytic datamart, analytic engine, analytic RDBMS, analytic service provider, analytics and business intelligence, big data, big data analytics, bigdata, business intelligence, business intelligence download, business intelligence software, business intelligence tools, data service provider, database management, database management software, fastest analytic database, the fastest BI solution, vectorwise | Leave a reply

The term “big data” may fade away, the need to analyze and act on data will most certainly not

In my last blog post I discussed the need for more teaching both by our education establishments as well as employers in order to help train a knowledgeable and skilled workforce that has the necessary skills to harvest the big data wave and help organizations monetize information.  After all, everywhere you look, we’re being told that big data equals big value, right?

But no sooner did I post my thoughts, I read an interesting article from analyst firm Gartner that would have us believe that the term big data is about to plummet off the “peak of inflated expectations” and fall into the “trough of disillusionment.”[1]  If you’re an avid follower of Gartner’s Hype Cycle reports, these terms will be familiar.   If you are not, then don’t worry – Gartner’s opinion is just one of many out there; others in the industry have done research that would suggest that for every one negative comment about big data, there are at least three positive ones.[2]

But we’re in danger of getting stuck in an academic argument played out by analysts, industry pundits, consulting firms and vendors alike.  In my opinion, businesses don’t care for the term big data per se, they care about running their operations, satisfying their customers, building their business.  They care about whether they have the know-how and workforce to help surmount business issues when analyzing and acting upon large and complex data sets, ranging from structured data in on-premise applications to unstructured data feeds coming from the cloud.  They don’t tend to refer to the term big data when explaining their challenges. In fact, remember when e-business was big in the late 90s? Then everyone realized it was just business and dropped the ‘e’.   Today big data is ‘big’, but in years to come, folks may just call it called data.

Whether you believe that the term big data is here to stay or doomed to pass, the fact remains that there is a plethora of opportunities out there for those who are prepared to embrace them.   And the savvy businesses out there are digging deep to help fund the salaries of those that know what to do with big data.  Dice.com’s annual salary survey reveals that job candidates with big data technology expertise command an average salary of $100k, some $20k more than other co-workers skilled in other areas of IT, for example in mobile technology.[3]  It seems that companies are prepared to pay big bucks for those who can when it comes to big data.

But just what are these jobs?  Again, the argument surrounding the term big data is superfluous.  No-one advertises for a “big data guru”.  Instead, you see job adverts for “data scientists”, “data architects” or “data engineers.”  More recently, and as shared by our customer Atheon Analytics, the term “data animator” is one that is increasingly becoming popular.[4]  This is a term that makes more sense to me as it conjures up the image of someone who can cope with data of any shape and size and turn it into something visual, something creative, something alive that tells a story and that can be easily understood to make decisions and act upon an occurrence or a situation in the business.  Everyone loves a good story and the human brain digests visual content much more easily than trawling through lines of a report or staring at another pie chart, just trying to figure out what it is meant to show.  These animators go beyond the static nature of spreadsheets and charts, they bring data to life in a way that allows businesses to get to grip with their data as well as think creatively about what they are doing with their data.

In fact, I believe that data animators will grow in popularity and become more commonplace just as data scientists and data architects will.  And as there is more of them getting to grips with data, there will emerge new executives to manage them.  Chief Digital Officers and Chief Analytic Officers may well be the next wave of company board members who spearhead the pursuit of extracting value from their business data.

In summary, I don’t believe we should be pessimistic and give up on the term big data.  Yes, the expression itself may drop from our vocabulary, but let’s put semantics to one side; the fact remains that organizations continue to see enough potential in their data and information that they are willing to invest and pay for expertise that can analyze and do something valuable with it.

In fact, the need for businesses to mine, analyze, predict, decide and act – all based on data – will be as prevalent as ever, bolstered by an ever-increasing number of data experts who know how to analyze and act upon data and information.  That is the future ahead of us.  Thus, to the point of my previous blog post, it has never been more important to ensure we can satisfy this demand with good data-oriented education and training.

Posted in Actian, Big Data, Vectorwise | Tagged actian, analytic database, analytic datamart, analytic RDBMS, analytics and business intelligence, big data, big data analytics, bigdata, business intelligence, business intelligence download, business intelligence software, business intelligence tools, customer insight, data service provider, database management, database management software, download database software, DSP, fastest analytic database, free database download, retail analytics, the fastest BI solution, vectorwise | Leave a reply

A new year, a new take on big data: more education and training

I wonder what the buzzword of 2013 will be.  Will it contine to be “big data” or will something else come along in its place?  Until the next expression is upon us, one thing is clear: I am astounded at the level of ubiquity the expression “big data” has reached, especially when you consider how 2012 saw NPR vote the term “word of the year” [1] and even BBC Radio 4 discussed the subject on its highbrow news program “Today”[2].  Furthermore, in October 2012, analyst company Gartner published a market forecast in which they claim that the term “big data” is now so commonplace that it can refer to any part of the IT industry.  The report then went on to say that “big data” would drive $28 billion of IT spending worldwide in 2012, rising 21% to $34 billion in 2013.  These are staggering numbers.

blackboard-bigdata-learning-actian

But just who is empowered to capitalize on the continued “big data” phenomenon?  If you believe Gartner again, businesses face a problem, one that will only get worse as data volumes and analytic workloads increase: there is a lack of internal know-how and trained experts that can work with data and monetize it.  To quote Gartner again, at the last Gartner IT/Expo Symposium in October 2012 in Orlando, the firm issued a statement[3] saying that 4.4m IT jobs would be created as a result of big data – that there would be a groundswell in career opportunities for data-savvy professionals.  And for every IT job created, 3 other jobs will be created outside of IT, so that means that 13m jobs would be created.  Wow.  However, in the same breath, the point was also made that only 1.2m jobs could be fulfilled highlighting a distinct lack of trained professionals able to satisfy demand.  In other words, only one out of every three jobs will be filled.  It appears that data experts are a scarce and valuable commodity.

So what can be done?  Some would argue that businesses need to satisfy their “big data” requirements by using easier-to-use tools and less complex applications.  But that is a given, in fact every vendor with a “big data” solution would support that argument.  The problem is not one that can only be solved with technology.

The answer to this challenge is in education.  We need our schools, colleges and universities to create content and courses around the “big data” phenomenon.  And we need to attract students to such courses, explaining their merits and the jobs that are out in the corporate world that can be filled as a result.  While some students may go into further education to study generic topics such as “business studies”, others would be better off following deep-dive data-centric subjects and courses that would serve them better in the long run.

Offering training on the fundamentals of “big data” to those already in employment is also paramount.  Businesses should start to educate their workforce and arm them with the tools and knowedge to become the next bastions of information-savvy and data-centric professionals.  Many have already made reference to the evolution of the “chief digital officer” as the next departmental executive in business, but a CDO can only function if supported by a workforce made up of data-savvy professionals.

Staff retention and satisfaction will only improve if businesses choose to invest in their employees and arm them with skills and know-how that will serve both parties well tomorrow.  In fact, nothing beats real-life working and hands-on experience.  Likewise, businesses should make sure they’re doing their part to teach, explain and share knowledge with those just starting out in their careers.  And potential employees have a part to play, too.  They should think about how they can bear their creativity and insight at their employer, and help to unearth insight and information that they previously had never considered and that will serve to help the business.

We live in the digital information age where data volumes will only continue to get more complex and yet offer more value, so businesses should invest in talent as much as they do in the software and systems they’re using to cope with “big data”.  In fact, the question for business executives is a simple one:  “Are you prepared to help create the new “big data” experts of tomorrow?”  If not, then you might find it that much harder to recruit the experts you need yoursevles for future business.

[1] http://www.npr.org/2012/12/20/167702665/geoff-nunbergs-word-of-the-year-big-data

[2] http://news.bbc.co.uk/today/hi/today/newsid_9714000/9714821.stm

[3] http://www.gartner.com/it/page.jsp?id=2207915

 

Posted in Actian, Big Data, Vectorwise | Tagged actian, analytic database, analytic datamart, analytic RDBMS, analytic service provider, analytics and business intelligence, big data, big data analytics, business intelligence, business intelligence download, customer insight, database management, database management software, fastest analytic database, retail analytics, the fastest BI solution, vectorwise | Leave a reply

Big Data: The rise of the smart city

The buzz surrounding big data is enough to convert the most ardent sceptic but once the initial enthusiasm subsides, you begin to cut through the fog of marketing hype and evaluate practical benefits.  Many people are at the second stage when they are beginning to question big data benefits.  Working in business intelligence and data analytics for some time now, I’ve never had occasion to doubt its positive influence. In fact the biggest and most widespread impact is already shaping our towns and cities and turning them into smart cities, a modern urban landscape where big data is radically changing both where we live and the way we live.

According to the 2011 revision of the United Nations’ World Urbanization Prospects report[1], nearly 70 per cent of the world’s population will be urban by 2051. The world population is expected to surpass nine billion and urban dwellers to surpass six billion. Two in three people born in the next 30 years will live in cities. This growth in urban living poses tough challenges to those in local government, administration and town planning offices as they strive to ensure that the environment, transportation, residents’ safety, the provision of utilities as well as economic and social activity can continue to be improved unhindered.  It is clear that responding to these challenges and improving people’s lives, towns and cities will require local planning teams and administration bodies to think differently, where much more emphasis will be placed on the consumption and analysis of large data volumes generated by day-to-day life in our towns and cities.

And it is staggering just how much data towns and cities generate. At a rough estimate, we will generate 4.1 terabytes per day per square kilometer of urbanized land area by 2016.  In fact, you could say that cities are the true big data systems of our age. From geolocation data collected by smart phones to data generated by cars and their GPS instruments, from the contact sensor payment cards we use to ride the subway to the data we offer when we want to make use of a bike or a car in the city.  From the data generated by our health ID cards to that from our loyalty and store cards, our bank cards and every time we make use of QR, bar or flash codes to access content.

Data generation does not stop there – think about the data created by traffic management systems, from traffic lights to the sensors on our roads; from the provision of utilities such as gas, electricity and drinking water; when delivering refuse collection and waste management services; from the provision of healthcare in our doctors’ surgeries and hospitals to the data generated by schools and colleges educating our children.

Data is not being generated in isolation either, there an increasing appetite for real-time and interactive information – where we were once content to use a map provided on a street display, we now turn to our smart phone devices and tablets to interact much more dynamically.  Before, we were glad when we found a good restaurant.  Now we want to research it, see what others think of it, take photos, post content and share our reviews.  No longer do we want to wait patiently for the bus to arrive, wondering whether an alternate transport might better serve our purpose.  Now we want to know where it is, how late it is running and whether it will get us to the station in time for our train.  Before, we just got into our cars and headed off to where we needed to go.  Now, we want to research our route, see if there are any traffic jams or incidents, plot our journey using our sat nav, and see how the weather might impact our journey.  Before, we were happy if we could get our children into a local school.  Now we want to know what the school is like, how it rates in the league tables, what the teachers are like, how easy it is for our children to get to school, what other parents think about it.

You could even say that where once we used to talk to our neighbours and family to get their thoughts and opinions, now we are consumed by the digital age and look to online resources instead, creating and consuming vast amounts of content that others can use and add to, vast amounts of data that – once shared – civic authorities and town planners can use to their advantage.

Faced with this deluge of data in a wide variety of forms and formats, it may be hard to know where civic authorities and town planning organization can start.  But the answer is clear: every step made towards improving the quality of life begins by first analyzing it and making sense of it.  For me this represents a great opportunity for the city authorities and urban developers – it gives them a powerful tool to tackle rapid and unprecedented urbanization by making better informed decisions, operate more efficiently  and even predict the future to ensure resources can be organized in time.

We have already seen a number of practical implementations from utility providers exploring how information from smart meters can encourage water and energy users to change behaviour to civic authorities using available technologies, including mobile phones, sensors and closed-circuit television to improve the flow of road traffic. One of the most successful example of this was during the 2012 Olympic games in London when Transport for London, the public authority responsible for running the London public transport network, prepared and ensured smooth transport despite experiencing a 25 per cent increase in customers using real-time information collected from CCTV cameras, subway cards (Oyster card), mobile phones and social networks to ensure limited disruptions to trains and bus routes.

Data is all around us – it’s not just growing but multiplying and what the civic authorities and town planners need is fast and easy-to-use technology which can digest the data quickly and give them the answers that they need.  They don’t need to invest in large, expensive storage or data processing solutions; specialized solutions are not needed here.  There are newer analytic solutions out there in the market, ones that leverage the performance features of the latest off-the-shelf servers and hardware that can crunch through large volumes of data of all shapes and sizes and render the results on devices that we all use anyway in a matter of just seconds: on smart phones, tablets, our desktop PCs.

Cities are areas where big data is having a real impact.  Town planners and administration bodies just need the right tools at their fingertips to consume all the data points that a town or city generate and then be able to turn that into actions that improve people’s lives.  In this case, big data is not just a passing fad or marketing hype, it is definitely a phenomenon that has a direct impact on the quality of life for those of us that choose to live in a town or city.

Tomorrow’s towns and cities are being built today, and they’re being built by using big data.

Posted in Actian, Big Data, Vectorwise | Tagged actian, analytic database, analytic datamart, analytic RDBMS, analytic service provider, analytics and business intelligence, big data analytics, bigdata, business intelligence, business intelligence software, database management, database management software, fastest analytic database, free database download, vectorwise | Leave a reply

Going bonkers over big data – but does size matter?

Big data: it’s amazing how two small words in the English language have found such great resonance in the world of business.  Moreso how every vendor is seemingly trying to jump on the big data bandwagon to promote their offering that may (or in some cases may not) have something to do with large, complex and untamely data volumes.  But let’s not get sidetracked here.  By all means go bonkers about big data: take customer data and your corporate information, add in some of the unstructured variety, mix it up with social network feeds, centralize it on hardware, even put it in the cloud – but remember that it is not a question of how much or how complex your big data volumes are, it’s what you do with it that ultimately counts.  Put another way, it’s one thing to collect and store umpteen amounts of data, it’s quite another thing to mine and analyze it and make something out of it that you can turn into value.

Some may pontificate over whether big data can be summed up by the “3 Vs” (volume, variety, velocity) or the “3 Is” (immediate, ill-defined, intimidating)[1] not to mention those that then try to outsmart each other by finding other words beginning with either V or I to continue the now tedious debate (variability, volitilty, inordinate, insurmountable etc), but they’re missing the point: the smart businesses out there don’t get bogged down in trying to define the term, they get on with the task of turning it into commercial advantage.  While others debate, the clever businesses take action and get on with it.

But who is doing it? Data aggregators and information service providers are two types of companies whose business models fundamentally depend on data.  They don’t just store it; they depend on it to power and run their business.  Consequently, they have seen the opportunity to turn big data into commercial value and many have grasped it with both hands, whether it be social network sites, gaming companies or SaaS vendors.   And their use cases vary from one to the other: many are web-based companies that take user data and correlate it with advertising data in order to attract more advertisers to part with their dollars in an attempt to get the right, targeted message to the right audience in a non-obtrusive manner.   Others are specialists in taking data feeds from a plethora of sources and then allow users to access and interrogate it in order to drive business.  A good example here is those service providers in retail that take EPOS data from grocery chains and then allow CPG vendors to access that data and interrogate it in order to improve customer satisfaction, increase spend and build loyalty.  Likewise, shipping companies broker the best courier deals for end-users by collating large data volumes from freight companies.  Media companies analyze viewing habits data and allow advertisers to run commercials at the optimum time.  The list goes on.

In summary, the data explosion continues to take the business world by storm and it would be foolish to play down its significance when trying to understand it. But in the main, it’s not the size that counts; it’s what you do with it.  So, by all means, go bonkers over big data.  Try to tame the big data tsunami, if you dare.  But make sure you put your energy into where it counts the most – in analytics and enabling commercial action.

Posted in Actian, Big Data, Vectorwise | Tagged actian, analytic database, analytic datamart, analytic RDBMS, analytics and business intelligence, big data, big data analytics, bigdata, business intelligence, fastest analytic database, vectorwise | Leave a reply

Harvesting the digital age – making money out of data

Normally I don’t care for statistics; they are often used subjectively in order to prove a point or win a weak argument.  In the words of 19th century British prime minister Disraeli and often referred to by Mark Twain, “there are three kinds of lies in this world: lies, damned lies and statistics.”  But recently I read a number of research pieces that highlighted first the amount of data and information out there, and how data analytics today means that information can be used objectively and with great certainty to prove a theory.  In fact, the articles got me thinking: in a world where data volumes are doubling in size every two years[1], there is an enormous potential out there for smart companies to collect data and grow their business by turning data into a commercial proposition.  Some may refer to it as the monetization of data.  I call it harvesting the digital age.

The first research piece I read deals with digital activity and how many information points are created by us tech-savvy humans.  For instance, did you know that every minute, 47,000 apps are downloaded from the Apple AppStore? That 204,166,667 e-mails are sent, that Google returns over 2 millions online searches or that a bewildering $250,000 is spent online on retail or gaming sites[2].  Staggering numbers when you consider what their daily totals must be.  The point of this is simple: as the data explosion continues to grow in size, there is an enormous opportunity for businesses to harvest it and create data-centric solutions and services.

Another research piece that I came across is – as an adult – a scary one.  It talked of the average age of a child when they start to use technology.  For example, you may be shocked to learn that the average age of a child who has a TV in their bedroom is just 6 years.  Or that children start to use the internet unsupervised at age 9.  A child has their first e-mail account at 9.5 years old and their first mobile phone at 9.9 years.  When it comes to using social media channels for the first time, the average is just 10.8 years[3].  My point here is that the digital demographic is getting younger and younger;  the sooner a child starts using a phone or interacting on social network sites, the more information points are created.  And as night follows day, as today’s generation becomes tomorrow’s business folk, there will be even more data to be captured and analyzed which in turn offers more commercial advantages for businesses. Generation Y may be smarter when it comes to technology, but that in turn means more data.  I wonder just how much the next generation will create.

Some might find the prospect of ever burgeoning data volumes a scary one, but the digital revolution is only set to continue and evolve into something bigger.  The question is what companies can do about it.  There is immense value to be had in analyzing data but can businesses harvest this huge growth?  The answer is yes, but they just need to have a concept about what it is that they want to analyze and offer as a commerical proposition to customers.  Technology such as Hadoop and Vectorwise will take care of the rest.

Rest assured: there is an inordinate amount of data and information out there in today’s digital age, even more so tomorrow.  And if you are not using it to create new solutions or services, then you can bet that someone else will.  So while 19th century Disraeli may not have liked the use of statistics to bolster an argument and Mark Twain may have mocked the subjective use of half-baked numbers, the 21st century is a very different place:  companies now have the technology at their fingertips to mine, analyze and extract value from data and turn this as a commercial offering, giving users nuggets of information that allow them – with great certainty – to create a competitive advantage.

So, the question is this: with all this data out there and with businesses looking for new ways to succeed, isn’t it time you thought about harvesting the digital revolution?



[1] Source: IDC
[2] Source: Domo.com
[3] Source : ESET UK

Posted in Actian, Big Data, Vectorwise | Tagged actian, analytic database, analytic datamart, analytic RDBMS, analytic service, analytic service provider, analytics and business intelligence, ASP, business intelligence, customer insight, data aggregator, data service provider, DSP, fastest analytic database, marketing service provider, record-breaking database | Leave a reply

To buy or not to buy: in-memory analytics, but at what cost?

From time to time the IT community falls in love with certain concepts – cloud, virtualization and SaaS to name a few.  More recently, another idea that has gained currency is In-Memory especially in conjunction with Big Data and analytics.  It is a good concept and it does what it implies – data is stored in RAM as opposed to disk.  While it may be a no-brainer that running data analytics using RAM rather than disk is considerably faster, the substantial hype around In-Memory, mostly aided by the marketing push made by vendors, has convinced businesses that there is no other alternative but a bank-breaking In-Memory solution.  It would appear that the bitter pill of an expensive price tag is sweetened by the promise of BI and analytical nirvana.

Recently, Gartner revised down its forecast for overall global IT spending from 3.7% to 2.5% (1) and many CIOs are staring at slashed IT budgets.  In this situation it’s important for them to re-examine their BI and analytic needs in order to assess if committing a massive portion of their IT budget to In-Memory is the best use of their already squeezed spending.

With the digital revolution sweeping through the world and organizations increasingly collecting data from various sources – whether it be from social networks, digital patterns, trends and online conversations – data volumes will continue to grow exponentially.  The ability to capitalise on growing data volumes represents a business opportunity that can help organizations gain a competitive edge but I wonder whether they equipped to leverage this opportunity?  In-Memory analytic vendors are correct in that RAM can step in by providing quick, actionable BI.  But at what cost?

One of the prerequisites for implementing In-Memory is to ensure that an organization’s database system has enough RAM in order to meet the data volumes that need to be analyzed. With the mass of data multiplying at the speed of thought – a Mckinsey report estimates unstructured data volumes to increase 44 times by 2020 (2) – sooner or later the size of the database will outgrow the memory on which it is running.  What do CIOs do then?

One option is to do nothing and be content with a hybrid RAM/disk-based system that offers varying analytic speeds.   In my opinion, that completely goes against the concept of fast analytics and business intelligence.  After all, how do you determine which data is to be analyzed in RAM and how much on disk?  How do you ensure that your database system has enough memory in order to meet the data volumes you want to analyze?  In many a case, servers only have a small amount of the RAM needed which means that only a portion of the data can be pinned into memory while the rest sits on disk trudging along at usual disk speeds.

The next solution, which is proving popular with a lot of organizations, is to buy more RAM.  This has been aided by the falling prices of memory over the years but it is still a costly solution.  At the risk of sounding like a doomsday alarmist, by going down this road organizations are setting themselves up for a fall; eventually data volumes will outgrow RAM capacities in servers again and necessitate further hardware investment.

Yes, RAM prices have fallen considerably over the years – in 1990, 8MB of RAM cost $851, in 2000 64MB of RAM cost just $72 and in 2011 8GB cost just between $50-70 (3).  But it is still a pricey solution when you consider that data volumes are now in the Terabyte or even Petabyte ranges.  Ultimately it comes down to how much CIOs are prepared to spend on an In-Memory solution in order to have the optimum system that can derive benefit from the speeds that RAM offers.  The buck doesn’t stop here – the immediate consequence of investing in an In-Memory solution means more servers in database system.  So it’s not just the cost of acquiring the RAM and server hardware but also the cost of implementation, administration, power and cooling.  The list goes on.

So, what can CIOs do to meet the demands of this insatiable analytic requirement and is there a way to end this vicious In-Memory cycle?  Perhaps, it would be more rewarding to make best use of the performance of the fast processors that modern servers offer.  To go beyond the levels of RAM and leverage the performance gains that have been made in modern CPUs.

Or perhaps the only way to handle vast amounts of data is to accept that there will indeed be a distribution between disk and RAM but that the disadvantages of this can be mitigated through modern disk technologies and highly intelligent caching algorithms.  That they can be mitigated through column store architectures with compression and vector-based processing.

What is certainly clear is that RAM has its part to play in offering accelerated analytical speeds, however there are newer technologies out there that allow CIOs to bypass the additional expense of resources, overhead, power consumption and the whole “do I or don’t I buy more RAM?” argument.  Indeed, the question is not “in-memory” or “no in-memory”, but is one that is more forward-looking.  CIOs should be asking themselves how they can break the confines of disk and RAM capacity to satisfy the analytical data tsunami that is coming their way.  How can they maximize their hardware and make best use of the CPUs in their investments. Only then will be free to interrogate large, complex data volumes with ease, at a significantly reduced cost and with minimal impact energy-wise.

 

Sources

1 http://www.gartner.com/technology/research/it-spending-forecast/

2 http://www.mckinsey.com/insights/mgi/research/technology_and_innovation/big_data_the_next_frontier_for_innovation

3 http://www.jcmit.com/memoryprice.htm

Posted in Actian, Big Data, Vectorwise | Tagged analytic database, analytic RDBMS, analytics and business intelligence, business intelligence, database management, database management software, free database download, in-memory, in-memory analytics, RAM, RAM-based analytics | Leave a reply

Beyond sweaters and ballet shoes

Why should retail organizations care about fast and cost-effective analytics?

In recent weeks, there have been some well-publicised examples of retail giants whose financial performance has been severely impacted by breakdowns in the supply chain.  As salt in the wound to these behemoths of retail, there are just as many examples of organizations that have exceeded market expectations due to an ability to behave nimbly and respond quickly to changes in consumer demand.

For example, leading UK retailer Marks and Spencer recently announced that it could have sold three times more sweaters, had it paid closer attention to data generated by its own stores. This had a major impact on its share price, making it the second biggest faller in the FTSE 100 for the quarter. At the same time, John Lewis (another retailer) predicted consumer demand and provided the right stock to the right stores.  Sales of its ballet shoes were up 129% year-on-year and profits were big.

This news made me wonder what the root of this issue was; which factor could cause success for one and failure for another.  Was it just a case of getting lucky with the right stock at the right time or was there something buried deep in the DNA of the organization that gave some retailers a crucial advantage?

The answer is simple – success comes from the ability to distill actionable intelligence from data gathered from a range of internal and external sources. This data consists of diverse factors including: the demographical characteristics of your main customers; their psychological state and purchasing patterns; how they plan to use your products; the macro economy; even what the weather is doing that day.  Investigated quickly enough to catch the trend at its birth, these all come together to define crucial business decisions such as: what to stock; when and how much of it; how to position it in the physical environment of the store and how to package and market it.

Having access to a fast and affordable analytic database platform can unite all the data from operational systems (EPOS, stock, merchandising, loyalty, ERP etc) and give business managers access to the right information at the right time, helping them to drive growth, performance, competitive advantage and, ultimately, profits.  With a fast and cost-effective analytic database engine, retailers can optimize their merchandising and product range by understanding buyer behavior and identifying trends immediately. They can also refine supply chain management by co-coordinating stock deliveries and replenishment, meeting the demands of each potential customer.

Cross-purchasing behavior analysis can also be calculated using a fast analytic database engine, which allows for large datasets to be joined together.   Such intelligence can then be used for marketing purposes, product range decisions, promotional planning and evaluation, shop floor layout.  The opportunities are endless.

Nevertheless, success is not just based on speed or high performance; any analytic solution needs to be cost-effective.  There is no point in a retailer investing in analytics if the system to be deployed involves a stack of new hardware that needs to be powered and then cooled, as well as configured and maintained.  Retailers will be far better off with systems that leverage off-the-shelf hardware and have as little a footprint as possible.

Savvy organizations, such as US retailer Sheetz, are using analytics to adapt product selection and availability to the needs of their customers faster, to improve customer service and tailor the shopping experience to the individual in a cost-effective way. In today’s competitive retail market where the slightest negligence renders giant chains out of business, it is no longer just about survival of the fittest, but survival of the fittest and quickest.  Retailers cannot afford to be anything but ultra-responsive; they need a business intelligence system that provides analysis without prompts or delays.  By ignoring this key need, they stand to lose out on opportunities that can make or break their business.  And that is something that goes beyond the ability to analyze sweaters and ballet shoes.

Posted in Actian, Big Data, Vectorwise | Tagged analytic database, analytic engine, analytic RDBMS, analytics, basket analysis, basket analytics, business intelligence, customer analytics, customer insight, EPOS analysis, market analysis, retail analytics | Leave a reply

Just because you can, doesn’t mean you should

As data volumes continue to explode and technology continues to enable large-scale business intelligence and analytics, the ability to mine and analyze data more deeply, more fervently and more quickly means that business users and data scientists can get insights that can truly spearhead the growth of their companies – in fact, it is fair to say that they have never had it so good.

However, with great power at their finger tips to trawl through vast volumes of data and gain fantastic insight comes great responsibility.   While telecom operators may analyze data to undertake sophisticated call-repricing analytics and financial services organizations may want to discover the propensity of customers buying new products, it is imperative that customer data is always respected and treated with care.  Yes, it can be tempting to exploit customer data in a way that benefits the company more than it does the customer, but organizations should never abuse that privilege.  In fact, they should be careful not to mine and analyze data and then use information to spam the poor customer with irrelevant information, nor should they use the data to exploit habits, customs or demographics that customers consider very personal and private.

So, just because you can analyse and get great insight, doesn’t mean that you should.  By all means, use data to help the customer and help your business, but never forget that analytics and business intelligence is a two-way street.  If customers or prospects feel that you are overusing their personal information for great business gain and it is to the detriment of them, that could do more irreparable harm to your company than good.  And with business reputations at stake, that is not a price worth paying.

Posted in Actian, Big Data, Vectorwise | Tagged analytics and business intelligence, business intelligence, business intelligence download, business intelligence software, business intelligence tools, database management, database management software, database software, download database software, fastest analytic database, fastest database software, free database download, slow query, the fastest BI solution | Leave a reply