Evaluating Mobile Application Development Firms

International Journal of E-Adoption, 6(1), 53-66, January-March 2014 53

Evaluating Mobile Application Development Firms: Comparing Views of Advertisers and Advertising Agencies

Pi-Fang Hsu, Department of Communications Management, Shih Hsin University, Taipei, Taiwan Tien-Chun Lu, Department of Communications Management, Shih Hsin University, Taipei, Taiwan Chia-Wen Tsai, Department of Information Management, Ming Chuan University, Taipei, Taiwan

ABSTRACT The purpose of the present paper is to propose a decision model for both advertisers and advertising agencies to evaluate and select mobile application development firms. The researchers first refer to related literature and apply the Modified Delphi Method to postulate the most suitable selection criteria. Then, Analytic Hierarchy Process (AHP) is utilized to derive the relative weight and ranking of each decision criteria, which can be used for evaluating and selecting the most suitable mobile application development firm. Then a company, which serves as a case example, is selected for applying this model. The data analysis finds that when selecting mobile application development firms, both advertisers and advertising agencies hold mastery of mobile marketing skill as their top concern. Business marketing division also stress experience in mobile marketing operation, especially ability in innovative layout and execution. In contrast, advertising agencies are able to provide their clients with creative ideas and designs. Therefore, they do not care so much whether marketing application development firms have ability in innovative layout and execution. They are more concerned about ability in customer service, especially team communication and arbitration ability. Keywords:

Analytic Hierarchy Process, Business Marketing, Mobile Application, Mobile Marketing, Modified Delphi Method

1. INTRODUCTION In response to consumers’ needs and keen market competition, mobile communication technology has undergone drastic development. The mobile phone has become one of the

most popular electronic gadgets over the past thirty years. Indeed, it has become a “must” in our daily lives (ITU, 2013). No doubt, the emergence of iPhone and Android changes not only the market structure of mobile phones but also the way people use mobile phones. New

DOI: 10.4018/ijea.2014010104 Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

54 International Journal of E-Adoption, 6(1), 53-66, January-March 2014

smart mobile phones enable users to connect to the Web and download a variety of APPs designed for different purposes and functions. These provide audio-visual entertainment, community media and instantaneous activities, through which users can obtain many and varied experiences. This phenomenon has led to the mobile phone becoming an indispensable part of consumers’ daily lives. With these developments, consumers’ daily behaviors are changed to a large extent. The development of smart mobile phones has led to new applications for mobile marketing. The popularity of smart mobile phones has also led to the emergence of multiple application services. Smart phones, with their high infiltration rate, instantaneousness, personification and interactivity of installations, enable consumers to utilize information and services on any occasion. Smart phone installations have become the most direct and precise media to target low-end consumers. This helps marketing personnel contact potential clients with more accuracy. Marketing personnel, therefore, need to take this new marketing channel into serious consideration. Smart mobile phones have exerted tremendous influence on the field of marketing. Businesses have recognized the particularity and importance of mobile marketing. The accountancy of mobile marketing may not be obvious, but businesses still pour a lot of investment into mobile media. However, with its high-tech features, mobile marketing has resulted in huge pressure on marketing personnel and advertising agents. General marketing personnel are unable to work alone, not to mention to design APPs for marketing. They must seek assistance from mobile application development firms. In view of this, many agents start to involve themselves in planning for mobile marketing, activity implementation, and APP design and development. Understanding the key abilities which mobile application development firms must possess is required in order to objectively select the most suitable of these firms to obtain the optimal marketing effect. All of these constitute major issues for marketing personnel.

Selecting a mobile application development firm is a problem involving multi-criteria decision making. It is through appropriate evaluation criteria and screening that a business marketing division or an advertisement marketing agent can select the most suitable mobile application development firm. The selected firm can then be authorized to conduct the operation planning and design development of mobile marketing application programs. Over the past few years, related research on mobile marketing has focused on industry development, advertisement effects or technological aspects of mobile marketing. The research in the present paper can hopefully serve as a reference for future related research or for business’ organizational decisions. This paper first adopts Modified Delphi Method. Then, the opinions suggested by experts in the field are gathered to help find the proper decision criteria. After that, the researchers use Analytic Hierarchy Process (AHP) to decide the relative weights of decision criteria. After ranking, the most suitable firm can be selected. As AHP takes both qualitative and quantitative principles into consideration, it meets the needs of this research. Finally, the researchers conduct a case study based on a company which uses this model to evaluate and select a mobile application development firm. It is our hope that the decision model developed herein can provide an objective quantitative and systematic method for businesses to evaluate and select the most suitable mobile application development firm.

2. RESEARCH METHOD The two-fold research method includes Modified Delphi Method and AHP.

2.1. Modified Delphi Method Delphi Method, proposed by Rand Corporation in 1950, aims to solve problems which cannot be quantified or predicted because of data insufficiency or uncertainty. As a structural group communication process, Delphi Method is

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.


characterized by predicting future phenomena via group brainstorming. Each member of the group is equally valued and is free to express his or her own opinions. Complex problems can therefore be solved through group consensus (Linstone & Turoff, 1975). Although the number of samples may be quantitatively small, it is representative nevertheless. It can combine the comprehensive viewpoints of several experts (McKenna, 1994). Anonymous experts offer professional experience, knowledge skills and opinions, and exchange views with other experts until a consensus is reached (Delbecq, Van de Ven & Gustafson, 1975). The Delphi method comprises the following steps: (1) select the experts; (2) perform the first survey round; (3) perform the second survey round; (4) perform the third survey round; and (5) synthesize expert opinions to reach a consensus. Steps (3) and (4) are normally repeated until a uniform result is achieved for a particular topic. Furthermore, a literature review and expert interviews can integrate recurrent ideas expressed in the survey. Step (2) is then simplified to replace conventionally adopted survey methods. The Modified Delphi Method is a simplification of the above procedure (Murry & Hammons, 1995). This study adopts the Modified Delphi Method to identify the evaluation criteria for selecting media agencies, which is determined through anonymous expert interviews and a survey of statistical outcomes regarding the research subject. Delbecq et al (1975) suggested that the appropriate number of members in a Delphi Method group would be between five and nine individuals. This study employed a decision-making group comprising six experts each from advertisers and advertising agencies.

2.2. AHP (Analytic Hierarchical Process) In order to solve the decision problem of multiple evaluation criteria under uncertain conditions, Saaty (1980) proposed the Analytic Hierarchical Process. Mainly by simplifying complex problems through systemization, AHP can be used to decide priority order, resource planning and distribution, and investment portfolio. AHP

makes use of hierarchical structure to solve problems and determine the hierarchy involved. It adopts pair-to-pair comparison to find the relative importance ratio between elements, which leads to ranking of the selected cases. Through quantitative judgment, we can find the problems and conduct a comprehensive evaluation. Case selection is based on such a method. It provides decision makers with complete data to select an appropriate case and reduce the risk of wrong decision making (Saaty, 1990). Hsu (2006) applies AHP in constructing the best PR evaluation model for the high tech industry. Hsu, Wu and Li (2008) uses it to select the best companies dealing with contagious disposable medical wastes. Hsu, Lan, and Tsai (2013) also uses AHP to select the optimal vendor of customer relationship management system for the medical tourism industry. Hsu et al. (2013) adopted AHP to select Korean dramas for commercial TV stations, and Hsu and Lin (2013) developed a brand name decision model using AHP. Obviously, AHP has been widely applied in industry. The weighting calculation between hierarchical factors is given below. Establish Pair-to-Pair Comparison Matrix A: Let C 1,C 2 ,C 3 , ,C n be a group of evaluation factors and assign paired factors (C i ,C j ) a quantified relative importance judgment, expressed as aij . Five values 1, 3, 5, 7, and 9, respectively, represent “equally strong”, “a little strong”, “very strong”, “extremely strong”, and “absolutely strong”. Thus, nXn matrix A can be obtained. C1 C 2  1 a 12  C1  1  1 C 2  a 12   A = aij  =     Cn  1 1 a  1n a2n

Cn

… a1n   a2n     1  

(1)



where aii = 1 , and aij =

1 , i, j = 1,2,3,…,n. a ji

In matrix A, let W1,W2 ,W3 , ,Wn be the quantified weights of n factorsC 1,C 2 ,C 3 , ,C n . If matrix A, which is constructed according to experts’ opinions, is consistent to a certain degree, the relation between its weights wi and judgments aij can be expressed as i, j = 1,2,3,…,n)

Wi Wj

= aij (for

2. Eigenvalue and Eigenvector: Paired comparison matrix A times factor weight vector X is equal to nX, that is, (A − nI )X = 0 . Here, X is the eigenvector of eigenvalue n. As aij is the objective judgment the decision maker gives when he or she undergoes pair-to-pair comparison, the value ofaij is different from that of

Wi

to a certain degree. Thus, AX = nX Wj is unlikely to be true. Saaty (1980) therefore suggests that n be replaced by the largest eigenvalue of matrix A, that is:

n

Wj

j =1

Wi

λmax = ∑ aij

(2)

If A is a consistent matrix, eigenvector x can be calculated by equation (3) (A − λmax I )X = 0 .

(3)

Test of Consistency: To test whether pair-topair comparison matrix A is consistent, Saaty (1980) suggests that Consistence Index (CI) and Consistence Ratio (CR) be used. The equations of CI and CR are provided below:

CI = CR =

(λmax − n ) n −1

CI RI

(4) (5)

where RI is a random index, that is, a consistent index of a randomly induced pair-to-pair comparison matrix. If CR ≤ 0.1 , then it is consistent.

3. MODEL AND APPLICATION This study aims to construct a selection model of mobile application development firms. First, the researchers apply Modified Delphi Method to decide selection criteria. Then, AHP is used to decide criteria weight. Thus, the alternative cases (firms) are ranked. Figure 1 shows the whole study process of six steps, described below: The model constructed in the present study targets two audiences: 1. Mobile marketing APP development for business marketing departments or advertising agencies; and 2. Mobile application development firms. This study picks up company X as a case study for model application. This case company needs to make an evaluation before selecting a mobile application development firm. A decision team, consisting of general manager, marketing supervisor and technology supervisor, is responsible for selecting the right firm. Three recommended mobile application development firms, namely A, B, and C, are on the evaluation list. Once the candidate companies are selected, the decision makers evaluate the three companies with respect to their scoring on each criterion. The researchers then apply the final scores on the construct for calculation. The process is illustrated in six steps.



Figure 1. The study process

Step 1: Deciding on the evaluation criteria of mobile application development firms. The researchers postulate the evaluation criteria based on Modified Delphi Method. The researchers interview six marketing personnel working with advertising agencies and 7 directors or supervisors of company marketing divisions. In total, 13 personnel constitute the expert team of this study. The team includes advertisers’ marketing directors, advertising agencies’ directors, and those in charge of marketing. They are all experienced marketing personnel and are responsible for firm selection. Based on the findings of past research and the team experts’ opinions, the researchers postulate the criteria and sub-criteria as shown in Table 1. The above criteria are from a marketing perspective. In actual cases, advertisers and advertising agencies want the APP’s design to have the same functions across different platforms. Definition of sub-criteria Ability in development of cross platform integration is the evaluation criteria of that if a firm has the ability to develop on different platforms. Advertisers and advertising agencies took the criteria into consideration if the mobile application development firm has the ability to design programs under iOS, Android, Windows and even Blackberry OS. The demographics difference of those who use iOS, Android and Windows was evaluated in sub-criteria Degree of understanding of mobile marketing and industry of criteria Ability in implementation of mobile marketing planning.

Step 2: Construction of hierarchical structure. The evaluation questions of mobile application development firms are divided into five hierarchies. The first is the goal hierarchy (selecting the best mobile application development firm). The second contains five criteria, namely mobile marketing skill, implementation experience in mobile marketing, ability in customer service operations, ability in mobile marketing planning, and ability in innovation. The third contains 24 sub-criteria. The fourth hierarchy is about evaluation & selection of the companies. Figure 2 shows the flow of hierarchy. Step 3: Constructing a pair-to-pair matrix. The researchers first postulate criteria for selecting a mobile application development firm and then construct a pair-wise comparison matrix. Next, the researchers refer to the weight value given by each expert via geometric mean value. The decision factors at each level of hierarchy are subject to comparison. The evaluation criteria constitute a pair-wise comparison matrix. Table 2 shows the comprehensive pairwise comparison matrix of the second-level hierarchy evaluation criteria which businesses use to evaluate and select a mobile application development firm. Table 3 shows the comprehensive pair-to-pair comparison matrix of the second-level hierarchy evaluation criteria which advertising agencies use to evaluate and select a mobile application development firm.



Table 1. Evaluation criteria for mobile development application firms Criteria Ability in mobile marketing skill

Experience in mobile marketing operations

Ability in customer service

Ability in implementation of mobile marketing planning

Ability in innovation

Sub-criteria

References

Mastery in mobile marketing skill

Expert opinion

Ability in mobile marketing APP development

Gell, Madjaric, Leodolter and Kole (2000); McDowell, Wahl and Michelson (2003); George (2005)

Ability in development of cross platform integration

Expert opinion

Ability in developing users’ interaction

Expert opinion

Ability in setting up mobile web sites

Expert opinion

Creative expression and performance

Cagley and Robert (1984); Lin and Hsu (2003)

Experience in implementing mobile marketing activities

Hsu (2006)

Professional ability and experience of the team

Cagley and Robert (1984); Kitchen and Schultz (1999); Lin and Hsu (2003); Hsu (2006)

Download frequencies of mobile marketing APP

Lin and Hsu (2003)

Experience in business operations or planning experience (with government)

Cagley and Robert (1984); Harris (1997); Gell, Madjaric, Leodolter and Kole (2000); McDowell, Wahl and Michelson (2003)

Range of service

Cagley and Robert (1984)

Quality of personnel

Gould, Lerman and Grein (1999); Lin and Hsu (2003); George (2005); Hsu(2012)

Team ability in communication and negotiation

Harris (1997)

Ability in budget planning

Harris (1997); Gould, Lerman and Grein (1999); Gell, Madjaric, Leodolter and Kole (2000); George (2005); Hsu (2006)

Customers’ evaluation

Harris (1997); Gell, Madjaric, Leodolter and Kole (2000); McDowell, Wahl and Michelson (2003)

Degree of understanding of mobile marketing and industry

Hsu (2006); Hsu (2012)

Marketing promotion ability

Hsu (2006)

Ability in designing mobile marketing activity

Lin and Hsu (2003)

Planning ability in integrating across media

Kitchen and Schultz (1999); Gould, Lerman and Grein (1999); Lin and Hsu (2003); Hsu (2006)

Ability in innovative mobile marketing skill

Gell, Madjaric, Leodolter and Kole (2000); George (2005)

Ability in observing innovation trends

Harris (1997)

Sensitivity to industry trends

Expert opinion

Ability in integrating resources

Gell, Madjaric, Leodolter and Kole (2000); McDowell, Wahl and Michelson (2003); George (2005); Hsu (2006)

Potential for further development

Expert opinion

Step 4: Consistency test. Equations (4) and (5) are used to calculate the criteria comparison matrix of consistency for each hierarchy, as listed in Table 2, Table 3, Table 4 and Table 5. Results of the consistency test and CR of the comparison matrix from each

of the 12 experts are all below ‘0.1’, indicating ‘consistency’. Table 4 shows the results of data analysis. From the viewpoints of advertisers, mastery in mobile marketing skill is regarded to be the most valued (1st) criteria, while ability in customer service is the least valued (5th) criteria.



Figure 2. The hierarchical structure



Table 2. Advertisers’ pair-wise comparison matrix of the second-level hierarchy criteria Skill mastery in mobile marketing



Ability in planning implementation of mobile marketing


Skill mastery in mobile marketing

1.000

1.348

1.982

1.534

1.346


0.742

1.000

1.842

1.240

1.104


0.505

0.543

1.000

0.743

0.599


0.652

0.806

1.346

1.000

0.855


0.743

0.906

1.669

1.170

1.000

CR=0.001 CI=0.001 λmax=5.004

The other three criteria, namely, experience in mobile marketing operations (2nd), ability in innovation (3rd), and ability in implementation of mobile marketing planning (4th) rank in between. 1. With respect to the five sub-criteria under “mastery in mobile marketing skill,” ability in developing consumer interactivity ranks number one (0.256), followed by mastery in mobile marketing skill (0.211), ability in building mobile websites (0.192), ability in cross-platform integration (0.180). Ability in developing application programs is the lowest ranking (5th) secondary criteria (0.162).

2. With respect to the five sub-criteria under “experience in mobile marketing operations,” ability in innovative layout and execution is top-ranked (0.340), followed by experience in mobile marketing activity operations (0.239), download frequencies of mobile APPs (0.167), and team professional ability and experience (0.146). The lowest-ranking (5th) secondary criterion is joint efforts and experience in planning with government (0.108). 3. With respect to the five sub-criteria under “ability in customer service,” team communication and arbitration ability comes in first (0.264), followed by budget plan-

Table 3. Advertising agencies’ pair-wise comparison matrix of the second-level hierarchy criteria Skill mastery in mobile marketing skill





Skill mastery in mobile marketing skull

1.000

1.565

1.316

1.732

3.162


0.639

1.000

0.904

1.316

1.655


0.760

1.106

1.000

2.280

2.236


0.577

0.760

0.439

1.000

1.107


0.316

0.604

0.447

0.903

1.000

CR=0.009 CI=0.010 λmax=5.039



Table 4. Advertisers’ eigenvectors and weights of 5 criteria and 24 sub-criteria Criteria




Ability in implementation of mobile marketing planning


Criteria weight

0.273

0.220

0.127

0.175

0.205

Sub- criteria

Sub-criteria weight

Total hierarchy weight (ranking)


0.211

0.058 (4)

Ability in developing application programs

0.162

0.044 (11)

Ability in cross-platform integration

0.180

0.049 (8)

Ability in developing consumer interactivity

0.256

0.070 (2)

Ability in building mobile websites

0.192

0.052 (6)

Ability in innovative layout and execution

0.340

0.075 (1)

Experience in mobile marketing activity operations

0.239

0.053 (5)

Team professional ability and experience

0.146

0.032 (18)

Download frequencies of mobile APPs

0.167

0.037 (14)

Joint efforts and experience in planning (with government)

0.108

0.024 (20)

Service range

0.154

0.020 (21)

Personnel quality

0.131

0.017 (22)

Team communication and arbitration ability

0.264

0.034 (16)

Budget planning ability

0.261

0.033 (17)


0.190

0.024 (20)

Understanding of mobile marketing environment and industry

0.233

0.041 (12)

Ability in marketing promotion

0.200

0.035 (15)

Ability in mobile marketing activity design

0.294

0.051 (7)

Ability in cross-media planning

0.273

0.048 (9)

Ability in innovative mobile marketing

0.291

0.060 (3)


0.229

0.047 (10)


0.131

0.027 (19)


0.184

0.038 (13)

Potential for further development

0.165

0.034 (16)

Overall consistency

λmax =5.018 CI= 0.004 CR= 0.004

λmax =5.016 CI=0.004 CR=0.004

λmax =5.059 CI=0.015 CR=0.013

λmax =4.000 CI=0.000 CR=0.000

λ

max =5.001 CI=0.000 CR=0.000



Table 5. Adverting agencies’ eigenvectors and weights of 5 criteria and 24 sub-criteria Criteria




Ability in mobile marketing planning operation


Criteria weight 0.304

0.204

0.242

0.138

0.112

Sub- criteria

Sub- criteria weight

Total hierarchy weight (ranking)


0.353

0.107 (1)

Ability in developing application programs

0.200

0.061 (6)

Ability in cross platform integration

0.154

0.047 (8)

Ability in developing consumer interactivity

0.147

0.045 (9)

Ability in building mobile websites

0.147

0.045 (9)

Ability in innovative layout and execution

0.335

0.068 (3)

Experience in mobile marketing activity operations

0.392

0.080 (2)

Team professional ability and experience

0.094

0.019 (17)

Download frequencies of mobile APPs

0.075

0.015 (20)

Joint efforts and experience in planning (with government)

0.103

0.021 (15)

Service range

0.227

0.055 (7)

Personnel quality

0.058

0.014 (21)

Team communication and arbitration ability

0.270

0.065 (4)

Budget planning ability

0.265

0.064 (5)


0.179

0.043 (10)

Understanding of mobile marketing environment and industry

0.464

0.064 (5)

Ability in mobile marketing promotion

0.115

0.016 (19)

Ability in mobile marketing activity design

0.193

0.027 (13)

Ability in cross media planning

0.227

0.031 (12)

Ability in innovative mobile marketing

0.316

0.035 (11)


0.224

0.025 (14)


0.175

0.020 (16)


0.133

0.015 (20)

Ability in further development

0.152

0.017 (18)

ning ability (0.261), customers’evaluation (0.190), and service range (0.154). The lowest-scoring (5th) secondary criterion is personnel quality (0.131).

Comprehensive consistency

λmax =5.012 CI= 0.003 CR= 0.003

λmax =5.108 CI=0.027 CR=0.024

λmax =5.015 CI=0.004 CR=0.003

λmax =4.003 CI=0.001 CR=0.001

λ

max =5.003 CI=0.001 CR=0.001

4. With respect to the four sub-criteria under “ability in implementing mobile marketing planning”, ability in mobile marketing activity design is ranked number one (0.294),



followed by ability in cross-media planning (0.273), understanding of mobile marketing environment and industry (0.233). The lowest-ranking (4th) sub-criteria is ability in mobile marketing promotion (0.200). 5. With respect to the five sub-criteria under “ability in innovation,” ability in innovative mobile marketing ranks top (0.291), followed by ability in observing innovation trends (0.229), ability in integrating resources (0.184),and potential for further development (0.161). The lowest-scoring (5th) sub-criterion is sensitivity to industry trends (0.131). Table 5 shows advertising agencies’ viewpoints. From their evaluation, mastery of mobile marketing skill is also regarded to be the most valued criteria (1st), while ability in innovation is the least valued (5th) criteria. The other three criteria, namely, ability in customer service, experience in mobile marketing operations, and ability in implementing mobile marketing planning rank in between. 1. With respect to the five sub-criteria under “mastery of mobile marketing skill,” mastery in mobile marketing skill ranks number one (0.353), followed by ability in developing application programs (0.200), ability in cross-platform integration (0.154). Both ability in developing consumer interactivity (0.147) and ability in building mobile websites are the lowest-scoring (4th & 5th) sub-criteria (0.147). 2. With respect to the five sub-criteria under “experience in mobile marketing operations,” experience in mobile marketing activity operations ranks top (0.392), followed by ability in innovative layout and execution (0.335), joint efforts and experience in planning with government (0.103), and team professional ability and experience (0.094). Download frequencies of mobile APPs is the lowest-ranking (5th) sub-criteria (0.075). 3. With respect to the five sub-criteria under “ability in customer service,” team

communication and arbitration ability is top-ranked (0.270), followed by budget planning ability (0.265), service range (0.227), and customers’evaluation (0.179). The lowest-scoring (5th) sub- criterion is personnel quality (0.058). 4. With respect to the four sub-criteria under “ability in implementing mobile marketing planning”, understanding of mobile marketing environment and industry ranks number one (0.464), followed by ability in cross-media planning (0.227) and ability in mobile marketing activity design (0.193). The lowest-ranking (4th) sub-criteria is ability in mobile marketing promotion (0.115). 5. With respect to the five sub-criteria under “ability in innovation”, ability in innovative mobile marketing is ranked as top (0.316), followed by ability in observing innovation trends (0.224), sensitivity in industry trends (0.175), and potential for further development (0.152). The lowest-scoring (5th) sub-criterion is ability in integrating resources (0.133). Step 6: Weight calculation of all hierarchies. After the weight calculation of each hierarchy, which is based on the evaluation provided by advertisers’ marketing divisions, the researchers conduct the final calculation of the weights of all hierarchies. The total weight score of each individual evaluated firm is thus derived. Based on the result, advertisers can pick the best choice (See Table 6). From the perspectives of advertisers’ marketing divisions, mastery in mobile marketing skill is regarded to be the most valued criteria. Of the three firms, Firm A (0.432) is better than Firm B (0.348) and Firm C (0.220) in this ability. As to experience in mobile marketing operations, Firm A (0.415) is again better than Firm B (0.353) and Firm C (0.231). With respect to ability in customer service, Firm A (0.568) is far better than Firm B (0.240) and Firm C (0.192). Regarding ability in implementation of mobile marketing planning, Firm B (0.365) is better than Firm A (0.357) and Firm C (0.277). Concern-



Table 6. Eigenvector and criteria weight for mobile development application firm evaluation Primary criteria

Weight of primary criteria

Firm A

Firm B

Firm C


0.273

0.432

0.348

0.220


0.220

0.415

0.353

0.231


0.127

0.568

0.240

0.192

Ability in implementing mobile marketing planning

0.175

0.357

0.365

0.277


0.205

0.524

0.312

0.163

Total scores of all hierarchies

0.451

0.331

0.218

Ranking

1

2

3

ing ability in innovation, Firm A (0.524) is far better than Firm B (0.312) and Firm C (0.163). Summing up, we find that Firm A (0.451) is the most suitable candidate of all.

4. CONCLUSION The popularity of smart mobile phones and APPs in the past few years has changed not only consumers’ behaviors in regard to mobile phone applications but also their daily life patterns. Mobile marketing operation via smart mobile phone has become one of the most fashionable means of marketing. Advertisers’ marketing divisions as well as advertising agencies are highly interested in the trendy mobile marketing. However, they are not so familiar with such a marketing method. Therefore, they must turn to mobile application firms for development services, including planning, design, operations and, sometimes, even promotion. No doubt, mobile marketing has brought about a lot of business opportunities Mobile message advertising agents, telecommunication companies and digital marketing companies have poured money into the market. Faced with such a market situation, advertisers and advertising agencies should know how to select a suitable mobile application development firm to become more successful in their product marketing.

Based on the opinions of both advertisers and advertising agencies, this research proposes a selection model for mobile application development firms. Related literature and the Modified Delphi Method serve to identify the most suitable selection criteria. Five criteria and 24 sub-criteria are induced for the selection of mobile application development firms. Then, AHP is utilized to derive the relative weight of each decision criterion. The researchers invite seven advertisers’ marketing division directors and six marketing directors from advertising agencies to participate in AHP. The data analysis finds that advertisers differ from advertising agencies in the weighting of selections for mobile application development firms. According to the study, advertisers regard mastery in mobile marketing skill as the most valued (1st) criteria, followed by experience in mobile marketing operations (2nd), ability in innovation (3rd), ability in implementing mobile marketing planning (4th), and ability in customer service (5th). On the other hand, advertising agencies regard mastery in mobile marketing skill as the most valued criteria (1st), followed by ability in customer service (2nd), experience in mobile marketing operations (3rd), ability in implementing mobile marketing planning (4th), and ability in innovation (5th).



Obviously, when selecting mobile development application firms, both advertisers and advertising agencies hold mastery in mobile marketing skill as their top concern. Advertisers’ marketing divisions also stress experience in mobile marketing operations, especially ability in innovative layout and executions. In contrast, advertising agencies are able to provide their clients with creative ideas and designs. Therefore, they do not care so much whether marketing application development firms have ability in innovative layout and execution. They are more concerned about ability in customer service, especially team communication and arbitration ability. The decision model proposed by the researchers can serve as a reference for advertisers’ marketing divisions and advertising agencies. By referring to their own marketing objectives and needs, they can apply this model to objectively evaluate and select mobile application development firms which have different merits or advantages. Practically speaking, this study can contribute to both advertisers and advertising agencies in their selection of mobile application development firms. Also, mobile application development firms can benefit from this study to build up their unique competitive cutting edge.

REFERENCES Cagley, J. M., & Robert, C. R. (1984). Criteria for advertising agency selection: An objective appraisal. JMR, Journal of Marketing Research, 24(2), 27–31. Delbecq, A. L., Van de Ven, A. H., & Gustafson, D. H. (1975). Group techniques for program planning: A guide to nominal group and delphi processes. Chicago, IL: Scott, Foresman and Company. Gell, G., Madjaric, M., Leodolter, W., Kole, W., & Leitner, H. (2000). HIS purchase projects in public hospitals of Styria, Austria. International Journal of Medical Informatics, 57(2-3), 147–155. doi:10.1016/ S1386-5056(00)00083-6 PMID:10978917

George, W. O. (2005). Evaluation criteria for district health management information systems: Lessons from the Ministry of Health, Kenya. International Journal of Medical Informatics, 74(1), 31–38. doi:10.1016/j.ijmedinf.2004.09.003 PMID:15626634 Gould, S. J., Lerman, D., & Grein, A. F. (1999). Agency perceptions and practices on global IMC. Journal of Advertising Research, 39(1), 7–20. Harris, T. L. (1993). How MPR adds value to integrated marketing communications. Public Relations Quarterly, 38(2), 13–18. Hsu, P. F. (2006). Developing a new model for selecting public relations firms in the high-tech industry. Journal of Modeling in Management, 1(2), 156–172. doi:10.1108/17465660610703486 Hsu, P. F. (2012). Selection model based on ANP and GRA for independent media agencies. Quality & Quantity, 46(1), 1–17. doi:10.1007/s11135-0109323-y Hsu, P. F., Lan, K. Y., & Tsai, C. W. (2012). Selecting the optimal vendor of customer relationship management system for medical tourism industry using Delphi and AHP. International Journal of Enterprise Information Systems, 8(4), 62–75. Hsu, P. F., & Lin, F. L. (2013). Developing a decision model for brand naming using Delphi method and analytic hierarchy process. Asia Pacific Journal of Marketing and Logistics, 25(2), 187–199. doi:10.1108/13555851311314013 Hsu, P. F., Lin, S. Y., & Tsai, C. W. (2013). Optimal selection of Korean dramas for commercial TV stations in Taiwan by using the AHP. International Journal of E-Adoption, 5(2), 14–24. doi:10.4018/ jea.2013040102 Hsu, P. F., Wu, C. R., & Li, Y. T. (2008). Selection of infectious medical waste disposal firms by using the analytic hierarchy process and sensitivity analysis. Waste Management (New York, N.Y.), 28(8), 1386–1394. doi:10.1016/j.wasman.2007.05.016 PMID:17689065 ITU. (2013). ITU releases latest global technology development figures. Retrieved from http://www. itu.int/net/pressoffice/press_releases/2013/05.aspx Kitchen, P. J., & Schultz, D. E. (1999). A multicountry comparison of the drive for integrated marketing communications. Journal of Advertising Research, 39(1), 1–17.



Lin, C. T., & Hsu, P. F. (2003). Adopting an analysis hierarchy process to select Internet advertising networks. Marketing Intelligence & Planning, 21(3), 132–141. doi:10.1108/02634500310474993

Murry, J. W., & Hammons, J. O. (1995). Delphi: A versatile methodology for conducting qualitative research. The Review of Higher Education, 18(4), 423–436.

McDowell, S. W., Wahl, R., & Michelson, J. (2003). Herding cats: The challenges of EMR vendor selection. Journal of Healthcare Information Management, 17(3), 63–71. PMID:12858600

Saaty, T. L. (1980). The analytic hierarchy process. New York, NY: McGraw-Hill Inc.

McKenna, H. P. (1994). The Delphi technique: A worthwhile approach for nursing. Journal of Advanced Nursing, 19(6), 1221– 1225. doi:10.1111/j.1365-2648.1994.tb01207.x PMID:7930104

Saaty, T. L. (1990). How to make a decision: The analytic hierarchy process. European Journal of Operational Research, 48(1), 9–26. doi:10.1016/03772217(90)90057-I
