Lecture Notes in Computer Science

0 downloads 0 Views 66KB Size Report
The fact data is stored (striped) in each computer in a DWS system by using a round-robin ... Another class of index structures, the bitmap in- dexes, try to .... 9 rows. CHANLEVEL. Month_level. Quarter_level. Year_level. TIMELEVEL. 1,239,300 ...
A New Technique to Speedup Queries in Data Warehousing Jorge Bernardino1, Henrique Madeira2 1

Instituto Superior de Engenharia de Coimbra, Dept. Informática e Sistemas, Apt. 10057 P-3030 Coimbra, Portugal [email protected] 2 Universidade de Coimbra, Dept. Informática, Pólo II P-3030 Coimbra, Portugal [email protected] Abstract. Data warehouses store large amounts of data for decision support. These large data volumes bring significant challenges to database engines, requiring high level of query performance and scalability. Different techniques are used to improve query response time in data warehouses, ranging from classical database indexes and query optimization strategies to typical data warehouse materialized views and partitioning. Parallel execution and distributed data warehouse environments have been proposed to cope with huge data volumes. However, these techniques have scalability and query balance problems, which limits query response speedup. In this paper we propose a round-robin data partitioning approach specially designed for distributed data warehouse environments. This approach takes advantage of the specific characteristics of star schemas and typical data warehouse queries profile to guarantee optimal load balance of query execution and assures high scalability. The proposed schema is evaluated experimentally for different types of queries and it is shown that an optimal speedup can be obtained for most of the typical data warehouse queries.

1 Introduction Data warehousing has had an explosive growth in the last years. In fact, today’s highly competitive market drives the organizations to acquire information systems dedicated to helping the business managers to make faster and more effective decisions. The information stored in these systems – generically called data warehouses – are historical in nature and can include detailed transactional data from operational databases, legacy systems, worksheets or even external data sources (external from the enterprise point of view). All this data must be integrated and conciliated before it is stored in the data warehouse for decision support purposes. Data warehouses must have efficient Online Analytical Processing (OLAP) tools to explore the data and to provide the users with a real insight from the data in the warehouse. One well known problem of data warehouses is that they tend to be extremely large, which causes heavy storage and performance problems. For example, the data warehouse of General Motors exceeds 1.5 terabytes in size, and contains a fact table with more than 2 billion rows [11]. Data warehouses also tend to grow quite rapidly.

Thus, a scalable architecture is crucial in a warehouse environment, not only to handle with very large amount of data but also to assure interactive response time to OLAP users. In fact, the decision making process using OLAP is often based on a sequence of interactive queries. That is, the answer of one query immediately sets the need for a second query, and the answer of this second query raises another query, and so on in an ad hoc manner. In this paper we propose a round-robin data partitioning approach specially designed for distributed data warehouse environments. This approach, called data warehouse striping (DWS), takes advantage of the specific characteristics of star schemas and typical data warehouse queries profile to guarantee optimal load balance of query execution and assures high scalability. In data warehouse striping the fact tables are disseminated by an arbitrary number of computers and the queries are executed in parallel by all the computers, guarantying a nearly optimal speedup. The fact data is stored (striped) in each computer in a DWS system by using a round-robin data partitioning technique at the (very low) fact row level. In some sense, DWS is inspired in the RAID-3 [22] scheme, and that is why we call it as data warehouse striping. In fact, the small size of the chunks (strips) used in RAID-3 assures that any disk access is uniformly distributed by all the disks in the array, improving the disk access speed by the number N of disks in the array. In a similar way, one major goal of DWS is to improve query execution speed by the number N of computers in a DWS system. The remainder of the paper is organized as follows. Next section contains related work and discusses the problems associated to the manipulation of large amounts of data in data warehouses. Section 3 describes the proposed approach and section 4 discusses experimental results. The final section contains concluding remarks and future work.

2 Related Work Several strategies can be used to improve query response time in a data warehouse, ranging from classical database indexes and query optimization strategies to typical data warehouse materialized views and partitioning. Indexing is among the first areas on which a database administrator will focus when good query performance in a read intensive environment is critical. Appropriate indexes offer the optimizer alternative access strategies for the time consuming full table scans. B+tree indexes and their derivatives [5] are the most commonly supported structures in RDBMS. But the tree structures have inconveniences when the cardinality of the indexed attribute is low. Another class of index structures, the bitmap indexes, try to overcome the problem by using a bit structure to indicate the rows containing specific values of the indexed attribute [3]. Although essential for the right tune of the database engine, the performance of index structures depends on many different parameters such as the number of stored rows, the cardinality of data space, block size of the system and bandwidth of disks, only to mention some [16]. The use of materialized views is probably the most effective way to accelerate specific queries in a data warehouse. Materialized views pre-compute and store (materi-

alize) aggregates computed from the base data [4]. However, storing all possible aggregates causes storage space problems and increases maintenance cost, since all stored aggregates need to be refreshed as updates are being made to the source databases. Many algorithms have been proposed for selecting a representative subset of the possible views for materialization [10], [12], [20], corresponding to the most usual query patterns. But the main problems associated with materialized views are the difficulty to know in advance the expected set of queries and the large amount of space required to store the materialized views. It is worth noting that the techniques mentioned above (indexes and materialized views) are general techniques that can (and should) be used in the data warehouse approach proposed in the present paper. In fact, in a distributed environment each individual machine must also be tuned and optimized for performance. A large body of work exists in applying parallel processing techniques to relational database systems [19], [8]. The basic idea behind parallel databases is to carry out evaluation steps in parallel whenever possible, in order to improve performance. Two types of parallelism can be explored at the query level: inter-query parallelism [7], wherein multiple transactions are executed in parallel in a multiprocessor environment, and intra-query parallelism [23], where several processors cooperate to concurrently execute a single SQL statement. The parallelism is used to improve performance through parallel implementation of various operations such as loading data, building indexes and evaluating queries. One of the first works to propose a parallel physical design for the data warehouse was [6]. In this work the author suggests a vertical partitioning of the star schema including algorithms but without quantifying potential gains. Partitioning a large data set across several disks is another way to exploit the I/O bandwidth of the disks by reading and writing them in a parallel fashion. User queries have long been adopted for fragmenting a database in the relational, object-oriented and deductive database models [21], [9], [18]. That is, the set of user queries is indicative of how often the database is accessed and of the portion of the database that is accessed to answer the queries. Most of today’s OLAP tools require data warehouses with a centralized structure where a single database contains all the data. However, a large centralized data warehouse is very expensive because of the great setup costs and it is not very flexible due to its centralized nature. The fact that data warehouses tend to be extremely large in size [4] and grow quite fast in many cases means that a scalable architecture is crucial to the success of these systems. A truly distributed data warehouse can be achieved by distributing the data across multiple data warehouses in such a way that each individual data warehouse cooperates to provide the user with a single and global view of the data. In spite of the potential advantages of distributed data warehouses, especially when the organization has a clear distributed nature, these systems are always very complex and have a difficult global management [1]. On the other hand, the performance of many distributed queries is normally poor, mainly due to load balance problems. One of the first steps to implement this concept of distributed data warehouse was the introduction of data marts, which are small data warehouses containing only data on specific subjects [13], [14]. Each individual data mart is primarily designed and tuned to answer the queries

related to its own subject area. Therefore, data marts are basically stand-alone and have integration problems, especially in what concerns the processing of queries. The data warehouse striping approach proposed in this paper is inspired in both the distributed data warehouse architecture and in classical round-robin partitioning techniques. The data are partitioned in such a way that the load of query execution is uniformly distributed by all the available computers and, at the same time, the need of communication among computers during the query execution is kept to a minimum (normally a partial result is computed in each computer in a pure independent way).

3 Data Warehouse Striping Before presenting the data warehouse striping approach let us briefly review the star schema and some key aspects of a data warehouse implemented over a relational repository [15]. A star schema illustrated in Fig. 1, consists of a large central fact table and several smaller dimension tables that are related to the fact table by foreign keys. C U STLEVEL

C HAN LEVEL

B ase_ level A ll_ level

S tore_level R etailer_ level

9 ro w s

9 0 0 ro w s AC TVARS

PRO D LEVEL

C o de_level C lass_ level G ro up_level Fam ily_ level L ine_ level D ivisio n_ level

C usto mer_ level P ro duct_level C hannel_ level T im e_ level U nitsS o ld D o llarS ales D o llarC o st 1 ,2 3 9 ,3 0 0 ro w s

9 ,0 0 0 ro w s

T IM E L E V E L

M o nth_ level Q uarter_level Y ear_ level 2 4 ro w s

Fig. 1. Example of a star schema

The fact table is the center of a star schema, which has predominantly numeric data (the facts) such as counts and amounts. The dimension tables have descriptive attributes, providing information about each row in the fact table. Typical queries calculate sums, averages, maximums, minimums, counts, etc, by aggregating the data facts according to attributes of one or more dimensions. Normally, one or more star schemas compose a data warehouse, each star corresponding to a business process. The fact table accounts for more than 90% of the space occupied by all the tables in the star for most of the cases [17]. However, this table is highly normalized, which means that it really represents the most effective (concerning storage space) relational way to store the facts. The dimension tables are very denormalized, but these tables usually represent a small percentage of the space in the star. The star schema is also particularly optimized for the execution of complex queries that aggregate a large amount of data from the fact table. A star schema provides an intuitive way to represent in a relational system the typical multidimensional data of businesses. In the example of Fig. 1, that is a simplified star schema of APB-1 Benchmark [2], ACTVARS is the fact table and the facts or measures are the attributes UnitsSold, DollarSales and DollarCost. The dimensions of

the star schema are tables: CUSTLEVEL, PRODLEVEL, CHANLEVEL and TIMELEVEL. In the data warehouse striping (DWS) approach proposed in this paper, the data of each star schema is distributed by an arbitrary number of computers. The data is disseminated over the available computers according to the following guidelines: - All the computers that constitute a DWS system have the same star schema (which is also the same schema of the equivalent centralized version); - The dimension tables are replicated in each machine (i.e., each dimension has exactly the same rows in all the computers); - The fact data is distributed by the fact tables of each computer using a strict rowby-row round-robin partitioning approach. That is, the fact rows are evenly distributed by the fact tables of each computer. Each computer has 1/N of the total amount of fact rows in the star, with N being the number of computers. Typical OLAP queries are executed in a distributed way by all the computers that constitute the DWS system (further on we will see the query modifications required to distribute the queries over all the computers). Two major goals can be achieved by the DWS approach: • Speedup: the execution time of typical OLAP queries can be greatly reduced. As we will see further on, using DWS it is possible to achieve a nearly optimal speedup for most of the OLAP queries (i.e., for example, if the star is striped into five computers the query execution time is reduced nearly five times). Three key aspects contribute to this nearly optimal speedup: 1. The distributed execution of a query by the available computers maintains the best load balance. This is so because the fact rows are distributed by the computers using a uniform row-by-row round-robin partitioning. The number of fact rows stored in each computer is about the same (each computer has 1/N ±1 fact rows, with N the number of computers) and the fact row level round-robin partitioning assures the data in each computer is not biased in any sense (as would be the case of using attribute range partitioning); 2. End user OLAP queries are transformed into N partial queries that are executed in an independent fashion in each of the N computers (see Fig. 2 for an example of DWS using N computers), avoiding the need of communication among computers during the query execution. Many queries over a star schema can be transformed into N independent partial queries just because of the nature of the queries (and because the fact rows are partitioned in a disjoint way: i.e., the rows stored in a computer are not repeated in the other computers). For some queries the computation of the partial result in one computer need to access data stored in the other computers in the DWS system (i.e., needs a truly distributed query); 3. Typical OLAP queries return a small result calculated from the aggregation of a normally large number of fact rows. This is the ideal situation, as in this case the computation of the final result from the partial results is very fast and has a negligible impact in the overall query execution time. • Scale up: a very large number of computers can be used which means DWS represents an effective way to handle very large data warehouses. The only limiting factor for the number of computers that can be used in a DWS system is the overhead due to the computation of the final query result from the partial results. However,

for typical OLAP queries this overhead is small, which means it does not represent a problem even when a large number of computers are used. Q uery

R e s u lt

Q u ery d istrib u tio n an d resu lt p ro c essin g la yer P a r tia l q uery 1

P a r tia l r e s u lt 1

P a r tia l q uery 2

P a r tia l r e s u lt 2

P a r tia l r e s u lt N

P a r tia l q uery N

… D 1,D 2 , … ,D n F a c ts 1

D 1,D 2 , … ,D n F a c ts 1

D 1,D 2 , … ,D n F a c ts N

Fig. 2. Distributed query execution

It is worth noting that, in addition to the typical features of OLAP queries mentioned above, the DWS approach takes advantage of general features of data warehouses and, particularly, it exploits specific features of the star schemas. The following points summarizes this issue: • DWS is adopted to star schema characteristics in which the facts are large and the dimensions are small, consequently the impact of replicating the dimensions is not so serious; • Replication allows the independence between partial computations; each query in each computer performs autonomously (in most cases). In this way partial results could be computed without nodes intercommunication; • One of DWS main objectives is improving query response time (basically reading operations) what is in harmony with data warehouses lifecycle of periodical updates (loads) and OLAP data exploration (i.e. there is no updates); • Typical data warehousing queries scan (aggregate) a large amount of data and normally return only a few lines (result). This is the ideal situation to DWS, as partial results can be computed independently in each computer and the task of final result computation is performed over a small set of partial results. With this approach as the size of the warehouse increase, the DWS system is able to cope with the extra load by simply distributing the facts between the computers or by adding more computing resources as needed.

4 Experimental Evaluation The experiment is conducted in an environment of PCs with the same characteristics, using Intel Pentium Celeron 466 MHz CPU, with 6.4 GB IDE hard disk and 64 MB of RAM. The computers are linked together in an Ethernet network at 100 Mbps.

4.1 Experimental Setup To evaluate the performance of the proposed technique we used the data set of the Analytical Processing Benchmark-1, Release II [2]. This benchmark was proposed by the OLAP Council and simulates a realistic OLAP business situation. The data set is created using the APB-1v2 file generator, that generates a fact table (ACTVARS) with 1,239,300 rows and dimension tables PRODLEVEL, CUSTLEVEL, TIMELEVEL and CHANLEVEL containing 9,000 rows, 900 rows, 24 rows and 9 rows, respectively, as was illustrated in Fig. 1. In these experiences we simulate a centralized data warehouse using only one computer and designate it as CDW (centralized data warehouse). DWS-5 corresponds to the DWS technique applied to five computers. In DWS-5 the rows of fact table are evenly distributed by all five computers and the dimension tables are replicated in all computers. In a star schema, typically OLAP queries aggregate a large number of rows in the fact table and return as result few rows. These queries use aggregation functions (sum, count, avg,…) as operators. Thus, first we concentrate our experiences in queries that use aggregation functions and return only one row. In data warehouses environments the fact table is usually joined with the dimension tables, what is one of the operations computationally more expensive. Therefore, we also test queries that use aggregation functions and joins. The main objective of the experiences is to evaluate the speedup of queries that use aggregation functions and join operations using DWS approach. 4.2 Performance Evaluation of Data Warehouse Striping In a DWS system concerning the need of communication among computers during query execution, typically queries can be divided in two groups: − independent queries (IQ) − dependent queries (DQ) The first group, IQ, contains the queries that can be independently executed in each computer, in a DWS system, with no need of communication among computers during the query execution. In this case the final result can be easily computed from partial results returned from all computers. Independent queries (IQ) are distributed by the available computers in a DWS system in such a way that each computer needs to execute only one partial query (number of executions=1), which corresponds to the nearly optimal linear speedup. The second group, DQ, is formed by the queries that need communication among computers. In these queries the computation of the partial result in one computer need to access data stored in the others computers. These types of queries raise several problems in the speedup of DWS as it is not possible to attain the optimal speedup of the first type of queries. An example of a dependent query (DQ) is the calculation of the number of units sold for all products whose sales price is greater than the average sales price. Using the star schema of Fig. 1, this query corresponds to the following SQL statement:

Q0:

select sum(unitssold) from actvars where dollarsales > (select avg (dollarsales) from actvars)

During the execution of this query the average of dollarsales (subquery) is executed first. The result of this subquery is used in the outermost query. The problem with this type of queries using a DWS system is that the original queries must be partitioned and reformulated, resulting in two new partial queries. In our example, the original query gives origin to the following two queries: Q1:

select sum(dollarsales), count(dollarsales)

from actvars Q2:

select sum(unitssold) from actvars where dollarsales > average_dollarsales

The first query Q1, determines the average of dollarsalles and query Q2 compute the units sold greater than average_dollarsales. Obviously, DWS doesn’t achieve optimal speedup with this type of queries. However, these are not the most common queries in data warehouses environments. 4.2.1 Evaluation of Independent Queries In these experiences we use this simple type of queries: select aggregate_function(unitssold) from actvars

Different aggregation functions are used (sum, count, max, min, avg, stddev and all together) in order to evaluate the impact of the used aggregation function in the query response time. The results of the response time of queries using the different aggregate functions are illustrated in Fig. 3, for the centralized data warehouse (CDW) and with our technique, DWS-5, using five computers.

0 0 :3 4

0 0 :2 6

0 0 :2 6

0 0 :2 5

0 0 :2 6

0 0 :2 7

0 0 :2 6

0 0 :2 7

CDW DWS-5 0 0 :0 7

0 0 :0 6

AL L

D D EV

0 0 :0 5

VA R IA N C E

0 0 :0 6

ST

0 0 :0 5

AV G

0 0 :0 5

M IN

0 0 :0 5

M AX

0 0 :0 5

C O U N T

00:39 00:35 00:30 00:26 00:22 00:17 00:13 00:09 00:04 00:00

SU M

Time (mm:ss)

Aggregation Functions

Fig. 3. Response time using the aggregation functions

The query response time is very similar for each aggregation function because I/O dominates response time, and also the volume of data warehouse is not so high. Only

when we use all the aggregation functions altogether (last column of Fig. 3) we see a small increase in query response time. The speedup is measured by adding more computers to the DWS system, but keeping the database size constant. For instance, we could determine the speedup of our approach taking the query response time for the centralized version (CDW) and comparing to the response time obtained with five computers (DWS-5) using all the aggregation functions altogether, as illustrated in last column of Fig 3. Dividing the query response time of CDW by DWS-5 we obtain a speedup of 4.86, near the optimal value of 5. We have also tested DWS with queries that join the fact table (ACTVARS) with all the dimensions of the star schema of Fig. 1. Four queries have been used: • 1-Join query: joins the fact table with the dimension PRODLEVEL; • 2-Join query : joins the fact table with PRODLEVEL and TIMELEVEL; • 3-Join query: joins the fact table with the dimensions PRODLEVEL, TIMELEVEL and CUSTLEVEL; • 4-Join query: joins the fact table with the dimensions PRODLEVEL, TIMELEVEL, CUSTLEVEL and CHANLEVEL; For instance, the 4-Join query corresponds to the following SQL statement: select sum(unitssold) from actvars, prodlevel, timelevel, custlevel, chanlevel where < join conditions >

The other queries are similar to this one. The Fig. 4 illustrates the different query response times for these four queries.

Time (mm:ss)

Joins 00:43 00:39 00:35 00:30 00:26 00:22 00:17 00:13 00:09 00:04 00:00

00:39 00:36 00:33 00:30

CDW DWS-5 00:06

1-Join

00:07

2-Join

00:07

3-Join

00:08

4-Join

Fig. 4. Query execution time using joins

For the query 4-Join we measure a speedup of 4.88. Once more, the speedup is near the optimal value. In these experiments we observed some variations of one or two seconds in query response time of some computers that could be caused for delays in the network, and for that reason we execute all the queries three times. It is worth noting that the time we take when we are using DWS-5 is always the query response time of the “lazy” computer.

A second set of experiments has been performed to evaluate the impact of the size of the DW in the speedup. The fact table was populated this time with a large number of rows (20 times bigger than the initial number). The star schema is equivalent of illustrated in Fig 1, only the number of fact rows has changed to 24,786,000 rows. We obtain the results of Fig. 5 for the aggregation functions.

11:22 08:50

08:22

08:42

09:21

09:18

08:50

08:42

CDW DWS-5 01:51

02:13

R IA N C

AL L

E

01:52

VA

ST

AV

D D EV

01:46

G

01:45

01:44

M AX

U N T

01:40

M IN

01:45

C O

12:58 11:31 10:05 08:38 07:12 05:46 04:19 02:53 01:26 00:00

SU M

Time (mm:ss)

Aggregation Functions

Fig. 5. Response time using the aggregation functions with the bigger data set

These experiments show that queries that take about 9 minutes in the centralized version are executed in less than 2 minutes using DWS-5. The most surprisingly result is that we obtained a speedup of 5.1, using all the aggregation functions altogether, which is greater than the theoretical speedup, suggesting a super linear speedup for complex queries. This is due to the fact that the centralized DW was running near the memory limits. This result also emphasizes the scale up advantages of DWS, as distributing the fact table over a larger number of smaller machines allows DWS system to cope with very large DW. The Fig. 6 illustrates the query response time for queries using joins.

Time (mm:ss)

Joins 14:24 12:58 11:31 10:05 08:38 07:12 05:46 04:19 02:53 01:26 00:00

12:48 11:49 10:51

09:59

CDW DWS-5 02:00

1-Join

02:10

2-Join

02:20

3-Join

02:31

4-Join

Fig. 6. Query execution time using joins with the bigger data set

With 4-joins we obtained a speedup of 5.1, once again, greater than the ideal, while using the small data set this speedup was only 4.88. This confirms our hypothesis of

“bigger is better” because augmenting the size of our data warehouse and distributing by more computers the gain is even better. 4.2.2 Evaluation of Dependent Queries As stated before, some queries need partial results, calculated over the data in all computers in a DWS system to compute the complete result. For example with queries containing subqueries we use the results of one query as part of another query. In these experiments we use the query Q0, defined in section 4.2, that to be executed in the DWS-5 system, have to be transformed into two queries Q1 and Q2. The experimental evaluation of these queries using the bigger data set is shown in Table 1. Table 1. Query response time of dependent query Q0

Time (mm:ss) Q0 Q1 Q2 Total

CDW 18:39 ----18:39

DWS-5 --(1:47) (1:48) 3:35

In spite of this query, Q0, needs two computations in each one of the computers, even so, we obtain a speedup of 5.02. This surprisingly result is due to the fact that these two queries Q1 and Q2, are simpler than the original query Q0; in this way, the optimizer can optimize better the query execution. In order to confirm this, we executed queries Q1 and Q2 using the CDW system and the results were 8:59 and 9:03 what corresponds to a query response time of 18:02, which is better than executing the original query Q0.

5 Conclusions and Future Work The proposed DWS architecture is scalable to support gigabytes or even terabytes of information solving the traditionally data warehouses problems of heavy storage and performance. As the need for additional data stores arises, only additional computers need to be added. Another advantage of this modular approach is that it can be incorporated in a relational DBMS without big efforts. As corroborated by the experimental results we obtained an optimal speedup and in some queries super linear speedup. This is due to the fact that when we distribute the data we are working on more manageable data sets that could be more treatable in computers limited by memory space. In this case we have not reached the limits of the machines and the results are better. This technique is a cost-effective solution that could be applied in almost all types of organizations taking advantage of the availability of microcomputer networks to distribute the data and, of their processing power free for most of the time without the necessity of a very expensive server. However, there still are several open questions: one of the identified problems are big dimensions. Typically, dimensions are small size, however, there are exceptions to this rule, in which the space overhead becomes more significant. The implementation

of query distribution and result processing layer also raise several practical problems. Another future direction is to explore some peculiarities of the multidimensional schema to improve our partitioning.

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14.

15. 16. 17. 18.

19. 20. 21. 22. 23.

Albrecht, J., Gunzel, H., Lehner, W.: An Architecture for Distributed OLAP. Int. Conference on Parallel and Distributed Processing Techniques and Applications PDPTA (1998) APB-1 Benchmark, Olap Council, November 1998. Available at www.olpacouncil.org. Chan, C.-Y., Ioannidis, Y.: Bitmap index design and evaluation. Proc. of ACM SIGMOD International Conference on Management of Data (1998) 355 - 366 Chauduri, S., Dayal, U.: An overview of data warehousing and OLAP technology. SIGMOD Record, 26(1) (1997) 65-74 Comer, D.: The ubiquitous B-tree”, ACM Computing Surveys, 11(2) (1979) 121-137 Datta, A., Moon B., Thomas, H.: A Case for Parallelism in Data Warehousing and OLAP. Proc. of the 9th Int. Conf. on Database and Expert Systems Applications DEXA (1998) DeWitt ,D. J. et al.: The Gamma Database Machine Project. IEEE Trans. Knowledge and Data Engineering, Vol. 2, Nº1 (1990) 44-62. DeWitt ,D. J., Gray, J.: Parallel Database Systems: The future of high performance database systems. Communications of the ACM, 35(6) (1992) 85-98 Ezeife, C.I., Barker, K.: A Comprehensive Approach to Horizontal Class Fragmentation in a Distributed Object Based System. Distributed and Parallel Databases, 1 (1995) 247-272 Ezeife, C.I.: A Partition-Selection Scheme for Warehouse Aggregate Views. 9th Int. Conference of Computing and Information, Canada (1998) Gill, P.: SIZE: Breaking the Warehouse Barrier. Oracle Magazine, January/February, Volume IX, Issue 1 (2000) 38-44 Harinarayan V., Rajaraman, A., Ullman, J.: Implementing Data Cubes Efficiently. In ACM SIGMOD Int. Conference on Management of Data (1996) Hewlett Packard: HP Intelligent Warehouse. White Paper at http://www.hp.com (1997) Informatica: Enterprise-Scalable Data Marts: A New Strategy for Building and Deploying Fast, Scalable Data Warehousing Systems. White Paper, available at http://www.informatica.com, (1997) Inmon, W.H.: Building the Data Warehouse. Second Edition, Ed. J. Wiley & Sons (1996) Jurgens M., Lenz, H.-J.: Tree Based Indexes vs. Bitmap Indexes: A Performance Study. Proc. of the Intl. Workshop DMDW, Heidelberg, Germany (1999) Kimball, R.: The Data Warehouse Toolkit. Ed. J. Wiley & Sons, Inc (1996) Lim, S.J., Ng, Y.-K.: A Formal Approach for Horizontal Fragmentation in Distributed Deductive Database Design. Proc. of the 7th International Conference on Database and Expert Systems Applications DEXA, Zurich, Switzerland, (1996) 234-243 Lu, H., Ooi, B., Tan. K.: Query Processing in Parallel Relational Database Systems. IEEE Computer Society Press (1994) Meredith, M.E., Khader, A.: Divide and Aggregate: Designing Large Warehouses. Database Programming and Design, 9(6) (1996) Ozsu M., Valduriez, P.: Principles of Distributed Database Systems. Second edition, Prentice-Hall, New Jersey (1999) Patterson, D. A., Gibson G., Katz, R.H.: A case for Redundant Arrays of Inexpensive Disks (RAID). In ACM SIGMOD Int. Conference on Management of Data (1988) Stonebraker, M., Katz, R., Patterson, D., Oustershout, J.: The Design of XPRS. Proceedings of VLDB’98, Los Angeles, USA (1988)