Mark Whitehorn beschrijft in een artikel de toepassing van 'derde generatie' BI. Daar waar het in de 1e generatie slechts ging om het begrijpen van het verleden, ging het in de 2e generatie om het analyseren en begrijpen waarom bepaalde zaken gebeuren (om vervolgens verbeteringen door te voeren). De derde generatie gaat nog verder en business intelligence brengt hier de informatie direct bij de medewerker die in contact staat met de klant. Dit vraagt om BI voor een grote groep medewerkers ('BI for the masses') en een zeer snelle verwerking van informatie-aanvragen (realtime). In het artikel gaat hij ook in op de technologiën die deze ontwikkeling mogelijk hebben gemaakt.

Uit het artikel:

[...]

Why BI had to evolve

The traditional face of BI needs to change because the way in which we are using BI is changing. For a start, analysis is getting more complex, according to Ellen Rubin, vice president of marketing with data warehouse appliance vendor Netezza.

"Until recently, [our customers] have been looking for fraud after it has occurred," Rubin said. "Now we can look much further into the future, asking not just 'What patterns can we see in the data?' but 'What patterns are we likely to see in the future?' " 

But it is also about adding a very different type of usage, according to IBM's Andrews.

"We would characterize BI as having three generations," he said. "The first generation was about understanding the past. The second was about analyzing why things happened and making recommendations about the future. That's better than first, but I still liken this to driving a car by looking in the rear view mirror. The new, third generation is about making information available to the people in front of the customer."

This is a truly significant shift in the way enterprises use data warehouses. First- and second-generation systems needed to support a limited number of people who ran large, complex analytical queries. The third generation must support not only more complex queries from the same analysts but also a new workload that consists of thousands of users running very different queries. These may well be complex, but each is likely to hit a relatively small set of data within the warehouse.

Combining these very different workloads is non-trivial. So is it worth the effort? The Hudson's Bay Company, based in Toronto, certainly thinks so.

Hudson's Bay Company embraces third-generation BI

Sadly, not all shoppers are honest. The National Retail Federation estimates that retailers lose about $16 billion a year to returns fraud -- dishonest customers presenting stolen items for refund or using a sales receipt multiple times. Detecting the patterns of fraudulent returns after they have occurred is second-generation territory. But catching the offender at the checkout with the receipt in his hand illustrates very clearly the difference that a third-generation system can make.

By combining a data warehouse from Dayton, Ohio-based Teradata with IBM's WebSphere as middleware and BI software from McLean, Va.-based MicroStrategy Inc., the Hudson's Bay Company was able to update the data warehouse with sale, return, exchange and void data almost instantaneously. Essentially, it is now impossible for a receipt to be reused or for merchandise to be returned fraudulently.

These third-generation systems are not pie in the sky. The Hudson's Bay Company rolled out this system across all its stores and, within five months, the savings had delivered a 100% ROI, according to Mary-Jane Jarvis-Haig, senior manager of business intelligence development and support for Hudson's Bay.

"The cost savings have been huge," Jarvis-Haig said. "We're already exceeding our targeted benefits."

[...]

OK, so far it seems that third-generation business intelligence (BI) and data warehouse systems are the future. But third-generation BI requires our systems to run both huge analytical queries and transaction-like queries -- such as those from customer-facing employees -- with equal aplomb.

This is a problem we have seen before. For the last decade or so, we solved this problem by separating the analytical and transaction systems, because combining them was too difficult. Very early on in the history of BI, we learned that analytical and transactional queries made very different demands on the query engines. So great were the differences that we made copies of the transactional data, moved it into the data warehouse and restructured it there in order to allow the analytical queries to run more effectively. Now, we are expecting our third-generation BI systems to cope with analytical and transaction-like queries in the same system.

How has business intelligence and data warehouse software changed in the interim?

Such third-generation systems have learned from the data warehouse appliances, so they can and do use massively parallel processing (MPP) and in-memory querying. But query workload management is also important. This technology carefully distributes the resources available to the system across the individual queries -- ensuring, for example, that time-critical queries, such as those from customer-facing employees, are allocated enough resources to allow them to complete in a timely fashion. Vendors such as Hewlett-Packard believe this is very important, according to Greg Battas, distinguished technologist, BI group with HP.

"HP sees workload management as absolutely vital," Battas said. "To put that into some kind of perspective, about 25% of our database development people are currently working on workload management."

The market is also seeing new hybrid systems emerging from vendors like IBM and HP with characteristics of both the traditional data warehouse and the data warehouse appliance.

The three core technologies driving this change are:

• MPP on commodity hardware
• In-memory querying
• Query workload management

These hybrid systems service not only the needs of analysts but also of a whole new layer of BI users within the company, which is remarkable. BI is finally moving from elite to egalitarian and perhaps, as analysts have long predicted, becoming pervasive throughout the enterprise.

The front end of business intelligence software -- and the audience -- changes

As third-generation systems move BI out to a much wider range of employees, it must become integrated into the software stack that those employees use on a daily basis. In some cases, this means rewriting custom software, but for many it means integrating BI into Microsoft's Office applications.

To this end, Microsoft has put massive efforts into BI, integrating Office 2007 with its own back-end BI tools. Analysis Services cubes can appear in Excel, you can data mine from within Excel or even (the mind does boggle a little here) from within Visio. However, Microsoft is simply one BI vendor; there are plenty of others out there. Somewhat surprisingly, Microsoft and other vendors, such as Teradata, are working together.

This cooperation not only allows Office components to reach data stored in Teradata, it also allows Microsoft's BI tools to do the same. Given the diversity of BI vendors, this is not a phenomenon that's going to disappear and, in this case, it is Software-as-a Service (SaaS) that is the key enabling technology. Service-oriented architecture (SOA) has been an essential technology in facilitating the cooperation, according to Ed White, director of product marketing with Teradata.

Bar charts, pie charts, yawn charts: Data visualization is changing

Pie charts date back to about 1800 and, useful as they are, we can do better, as the work of respected data visualization researchers Edward Tufte and William Cleveland shows. Several companies -- including Spotfire (now part of Tibco), QlikTech, Thinkmap, Tableau and others -- have been looking at this work and producing truly original ways of displaying complex data. I believe that this will have a profound influence on BI over the coming years, and others -- such as Roger Oberg, vice president, Spotfire product strategy with Palo Alto, Calif.-based Tibco Software Inc. -- agree with me.

"New trends like in-memory processing, 'free dimensional' ad hoc queries, and user definable workflows are democratizing BI," Oberg said. "We are moving from a world in which we push data that is often ignored to a world in which interaction massively increases the data's usefulness and therefore the number of people who want to use it."

As he points out, none of that will be effective unless people can visualize the data easily.

Yes, but what does it all mean for my business intelligence and data warehouse software?

Data is just numbers and text. One important lesson we have learned from BI is that keeping track of the meaning of the data is far, far more complex than we originally thought. In one sense, this isn't a technical problem, it's a human one -- since only humans can decide questions of meaning. However, some companies have been actively trying to address not only how we track meaning but how we track it over time, according to Cliff Longman, chief technology officer with Burlington, Mass.-based Kalido Inc.

"Fluidity of meaning is the problem that Kalido addresses," Longman said. "We find that users can get results if they are allowed to deal with a higher level of abstraction – higher than the logical model. Kalido makes data reusable even if the meaning changes over time."

And finally, though technical approaches differ, everyone I spoke to agreed on two important, related points:

1. Data volumes are growing, year on year. Two years ago, data warehouse vendor Kognitio was looking to scale its systems down to 200 GB for some customers, according to Roger Gaskell, product development director. Now, almost every proof-of-concept Kognitio does is above 5 TB, with most in the 50 to 250 TB range.

2. BI is no longer the preserve of the large enterprise; it has moved to the small/medium-sized enterprise. Ten years ago, Microsoft's vision was "BI for the masses," according to Amir Netz, product unit manager for Analysis Services -- and in recent years, other experts have often espoused the benefits of BI for small and medium-sized businesses.

Bron: www.searchdatamanagement.com