Espacios. Vol. 36 (Nº 15) Año 2015. Pág. 3
Aplication of Delphi Method in a foresigth study on biodegradable packaging up to 2032. Aplicación del Método Delphi en un estudio previsión en envases biodegradables hasta 2032. Jhon Wilder ZARTHA Sossa 1; Héctor Samuel VILLADA Castillo 2; Raúl HERNÁNDEZ Zarta 3; Amanda Lucía FERNANDEZ Pérez 4; Bibiana ARANGO Alzate 5; Gina Lía OROZCO Mendoza 6; Rafael BERMUDEZ Ortega 7; Diego Fabián JOAQUI Daza 8; Alcy René CERÓN Mosquera 9; John Fredy MORENO Sarta 10 Recibido: 31/03/15 • Aprobado: 12/05/2015
Contenido 1. Introduction 2. Backgrounds 3. Method 4. Statistical analysis 5. Results and discussión 6. Conclusions References
ABSTRACT: Delphi methodology was chosen to perform a foresight study up to 2032 in biodegradable packaging and environmentally friendly products. To develop the study, a questionnaire with 196 variables classified into 5 areas was elaborated: Technological innovations, commodities for biodegradable polymers production, commodities for biodegradable packaging production, additives and technologies, the questionnaire was sent to 92 national and international people with knowledge on the topic. The expert panel was made up of 30 people who gave response to the questionnaire for three rounds. According to statistical analysis raised, in the first round, 51 priority variables and 140 variables under discussion were obtained, the 5 variables remaining were classified as nonpriority. In the third round, a total of 78 prioritized variables were obtained. The results obtained in the area of technological innovations, natural coatings, active packaging and smart packaging, were considered priorities during the three rounds. In the area of commodities for the production of biodegradable polymers, the potato, cassava and corn starch were considered a priority, in addition to Agroindustrial wastes and in the technology area, reactive extrusion, blow molding and extrusion injection
RESUMEN: La Metodología Delphi fue elegida para llevar a cabo un estudio de previsión hasta 2032 en envases biodegradables y productos respetuosos con el medio ambiente. Para desarrollar el estudio, se elaboró un cuestionario con 196 variables clasificadas en 5 áreaz: Las innovaciones tecnológicas, materias primas para la producción de polímeros biodegradables, materias primas para la producción de envases biodegradables, aditivos y tecnologías. El cuestionario se administró a 92 personas nacionales e internacionales con conocimiento sobre el tema. El panel de expertos estaba formado por 30 personas que dieron respuesta al cuestionario en tres rondas. De acuerdo con el análisis estadístico planteado, en la primera ronda, se obtuvieron 51 variables de prioridad y 140 variables bajo análisis, las 5 variables restantes fueron clasificados como no prioritarias. En la tercera ronda, se obtuvieron un total de 78 variables de priorizadas. Los resultados obtenidos en el ámbito de las innovaciones tecnológicas, revestimientos naturales, envases activos y envases inteligentes, se consideraron prioridades durante las tres rondas. En el ámbito de las materias primas para la producción de polímeros biodegradables, la papa, la yuca y el almidón de maíz se considera una
molding were kept for three rounds as priority variables. Keywords: Delphi Methodology, Foresight, Biodegradable Packaging, Consensus percentage.
prioridad, además de los residuos agroindustriales y en el área de tecnología, extrusión reactiva, moldeo por soplado y moldeo por inyección de extrusión se mantuvieron durante tres rondas como las variables prioritarias. Palabras clave: Metodología Delphi, prospectiva, envases biodegradables, Consenso porcentuales.
1. Introduction Since 1950 the Rand Corporation developed the Delphi methodology (Nielsen and Thangadurai, 2007; Landeta, 2006; Dalkey and Helmer, 1963), in order to investigate the impact of technology for the US Air Force during the war (Gilgomez and Pascual, 2012; Fletcher and Marchildon, 2014; Rowe and Wright, 1999; Linstone and Turoff, 1975), it has been used by different disciplines for multiple purposes because its main objective has been resort to the expertise to predict how a phenomenon in the future would behave (García and Suarez, 2012). According to Kim, Jang and Lee, (2013), this technique is considered one of the most used, as it contains information consensuated by a group of people considered experts in relation to the objective; the contrast of the arguments issued by the experts allow reducing uncertainty in decision making. In this case, the Delphi methodology was used in a foresight study on the biodegradable packaging development in Colombia, in order to find the current trends developed in other world places and the possible alternatives commodities for the production of the same, given that from latest times, it has been a global challenge to find alternative solutions to environmental pollution caused by improper usage of synthetic plastic used for such purpose; however, the development of these materials in Colombia is still growing, although Oxo degradable plastics have been used as alternative because its degradation occurs in less time than current plastic but the degradation process is not complete and its production sources are not renewable. Therefore, a foresight study through the Delphi method in this topic, provides information about to development of biodegradable packaging and environmentally friendly products, not only in areas such as technological innovation and commodities, but technology and type of additives to be used, due to its biodegradable nature, the elaborated material must have international biodegradable standards. This article is part of a group of results that were obtained in the prerogative project titled "Foresight study for the year 2032 on friendly biodegradable packaging in the Colombian environment"; financed by national department of planning, general prerogative system in Colombia, the government of Cauca, University of Cauca, and the research group of CYTBIA.
2. Backgrounds For one procedure to be defined as "Delphi", for general aspects must be taken into account: anonymity, insistence, controlled feedback, and a statistical analysis on the answers generated by the group. (Zartha et al. 2014) Many applications of the Delphi method exist at an international level, in diverse sectors. The following chart shows the applications made by 15 authors who have made scientific publications. Table 1. Applications of the Delphi method in diverse sectors Author
Country and organization
Coduras and
Astrazéneca
Purpose
Number of experts
Study that tries to anticipate the future 67 y 65 respectivamente of the pharmaceutical benefits in Spain
Señarís (2009)
Foundation
Brigham Young Wakefield and University (Estados Watson (2013) Unidos), Bournemouth University (Reino Unido)
Used on public relations and research on communications regarding the 2.0 web era. The purpose is to introduce, analyze, and explain the Delphi and its evolution; and its applications on public relations.
Duke (2009): 10 to 15, Mckinnon et al. (2001): 7, Boynton and Watson (2006) and Watson (2008): 30 Fink and Fuchs, (2012): 32
It presents how the panel members are invited to present foresight variables on the financial market in a controlled experiment, and presents 2 post survey methods as an adjustment to the Delphi method.
10 experts were used, Rowe and Wright were cited (1999): Ten people are enough, and there is no evidence that increasing the size from 710 people would improve precision.
Kauko and Palmroos (2014)
Fletcher and Marchildon (2014)
Bank of Finland
Canada
A modified draft Participatory Action First round with 39 experts Research (PAR) in health leadership in Canada Delphi method was used. Recognize the degree of agreement/disagreement between a group of Spanish gynecologist about the role .of probiotics as a help treatment of vaginitis.
Modified Delphi method with the objective of forming blocks of knowledge that should have future action in the formation process of the university's TIC teaching.
This study is about CO2 emissions in terrestrial cargo transport in Finland
Debin, Souty, Turbelin, et al. France (2013).
The objective of this study is to establish an expert based determination from the beginning and end of influenza epidemics in France
This study shows that a group of criterion and diagnose strategies are not enough to enrich a consensus for the diagnostic of a typical CIDP "Chronic inflammatory demyelinating polyneuropathy" in the clinical
Cancelo, Neyro, Baquero (2014)
Spain
Cabero (2013)
Spain
Liimatainen et al. (2014)
Finland
Creange and Careyron (2013)
France
123 experts
First round with 68 experts, second round with 65 experts.
24 and 20 respectively
57 answered the three rounds
32
practice.
Okoli and Pawlowski (2004)
Canada and the United States
Landeta (2006)
Spain
Graham, Regehr and Wright (2003)
Gordon and Pease (2006)
United States
Moore (2011)
United States
Zeedick (2012)
United States
Canada
The article provides a guide regarding the adequate expert selection process Between 10 and 18 for each for the Delphi study, and gives detailed panel (4 panels in total) principles for decision making during the process, to assure the validity of a Delphi study, It mentions an example on ecommerce in subSaharan Africa. Evaluate the methodology and revise its actual validity, especially in areas concerning the social sciences.
Achieve consensus between a panel of experts to establish the best clinical criteria for the diagnosis of carpal tunnel syndrome
Describe the Delphi process in real time, and illustrate its usage in an application regarding decision making; conducted from the millennium project of the American council of the United Nations University.
States in consumer packaged goods. Uses a modified Qualitative Delphi and descriptive statistics for leadership profiles of the NDP stages in businesses in the United States in consumed bottled goods.
11 to 12 experts evaluating each stage of the development process of new processes. 37 panel members, in groups of 5 to 6.
This deals with the instructional design theory and its applications in line education programs. A Delphi was used in three rounds with descriptive analysis and interquartile ranges. According with the results, they did not obtain the adequate results about the necessity of applying this theory in online programs,
Cites: Helmer (1963), Delbecq et al. (1975) and Limestone and Turoff (1975): 9 experts are enough. .
Cites Cataluña study with 14 experts
14 y 12 respectively
10 a 15 respectively
Source: (Zartha et al. 2014)
Characteristics of a Delphi Method Between the most important characteristics of the Delphi method, Castelló y Callejo (2000) mention that a Delphi is a consultation to a large
number of chosen experts from different sectors or specific topics for an exercise, based on a survey; prepared by a panel or commission of experts. On a similar side, (Rowe and Wright, 1999) the Delphi has four key characteristics: anonymity, (Becker and Bakal, 1970; Helmer and Rescher, 1959) iteration, controlled feedback and added statistics from a group of answers, (Gordon and Helmer, 1964). Besides, (Martino, 1993; Konow and Pérez, 1990) add another characteristic related with monitor equipment. They produce a structured survey from different points of view, opinions, and judgements form the panel members, which can be raised in a quantitative manner in latter rounds.
3. Method The Delphi method is an anonymous and structured consultation to a group of individuals about topics where the opinion is important and a consensus is achieved. Helmer y Rescher (1959). The expert gets feedback through questionnaires made for the corresponding study, because they can give their opinion about the subjects considered and it allows the experts reconsider their points of view according to a set of responses (Thangaratinam and Redman, 2005). Regarding topic selection, technologies and priority variables, the criterion remained constant. This consists on the calculation of the mode, modal frequency and consensus percentage. Concerning this data, the consensus average percentage was determined for each pooler (Zartha et al. 2014). Based on this information, the following criterion was established: Topic, technology, priority variable: Consensus over the average pooler and a mode of 4 or 5. The scale is 0 to 5 where the prioritization order was the following: 0= No priority 1= Low Priority 2= Low 3= Medium 4= High 5= Very High N= Does not Know/ Did not Answer For the study, the following poolers were taken into account: Technological innovations, raw materials for the production of biodegradable polymers, Raw materials for the acquisition of biodegradable packaging, additives, and technologies. Delphi Method: Stage 0: definition of the transversal axis or poolers form the company's perspective related to their areas of interest Stage 1: Socialization of the methodology and communication with the monitoring team Stage 2: Construction of the expert profile, definition of the problem/necessity, and the revision of the list of experts Stage 3: Construction of the technological thematic tree diagram Stage 4: Survey for the first round Stage 5: Delivery of the first round to the experts Stage 6: Statistical analysis and report of the first round Stage 7: Analysis of the technologies/themes of priority/no priority, discussion, construction and the delivery of the second Delphi round Stage 8: Construction of the second round Delphi Stage 9: Delivery of the second round
Stage 10: Analysis of the second round Stage 11: Construction of the second round Delphi Stage 12: Delivery of the third round Stage 13: Construction of the final report and Socialization with the monitor team
3.1 First round Delphi. This first stage was started with the election of a accompanying team chosen from the research CYTBIA team, who have made developments in the biodegradable packaging subject. The team was commissioned to provide the base information for the Delphi study construction. Information search of innovation trends, commodities and technology used in biodegradables packaging production was performed for building a theme tree with 196 variables, after that they were classified in 5 areas according to functionality and thematic application: 1. Technological innovations, 2. commodities for biodegradable polymers production, 3. commodities for biodegradable packaging production, 4. additives and 5. technologies; based on this, the questionnaire for the first round Delphi was built in excel format and it was sent to various experts in the subject by email. For this first round 97 people from national and international universities, government, private sector and other organizations were invited, of this people, 30 experts formed the panel responding to the structured consultation. By this first round, the experts must have to qualify as non priority and priority topics in a scale of 0 to 5 respectively. Subsequenly the statistical analysis was accomplished, was obtained four parameters like Moda, Modal Frecuency, Consensus percentage and the consensus average percentage for each area; with this results was determinated that one subject would be a priority (TP) if the califfication exceded the category consensus percentage and it had a modal value equal or higher than 4. Likewise, was determined that a subject would be considered as No Priority (NP) if it had a modal value less or equal to 2 and a consensus average percentage higher than corresponding category. The subjects who did not accomplished the above conditions were considered as subjects under discussion (TD).
3.2 Second round Delphi. For the second round, a new questionnaire in the categories with items considered "priority" and "discussion" was done. In this stage, the goal was to identify from experts some subjects under discussion in order to classify them like priority subjects instead of some other discussion subjects. In other words, the priority subjects group may not increase, for this reason, it is necessary that for each topic leaving the group, another discussion subject could take its place justifying any changes, given that this will be the base argument for the final round.
3.3 Third round Delphi. In the third round, a new questionnaire with subjects considered as "priority" and "discussion" and the appropiate justifications in the previous round was sent to provide the better arguments to the experts when they choose a subject as priority again After that, the statistical treatment was completed again, taking into account the statistical parameters used in the first round, according to the consensus done by the experts, a final classification was obtained. With these results a final report was made with most important subjects in the biodegradable packaging area and the need of CYTBIA team to make new researches in this subject.
4. Statistical analysis According to Balaraman, Venkatakrishnan, (1980); Zartha, et al, (2011) for analysis of consensus for each of the topics in the third round, there were considered the following: According to the qualification assigned by the participants of the survey in each of the topics in the first round, the modal value qualification was calculated. If the value was 4 or 5, It would be constant for the next rounds as a priority. The qualification for the first round were taken as the pattern for the third round. In the third round, participants assigned X to the topics considered priority, after this, the X was replaced with the value of the qualification assigned in the first round. The lower modal qualification value (Er3) in the frist round was determined for each subject, since they were not considered as a priority initially. Similarly, there were identificated the participants that considered some topics as a priority in the first round but did not check them with an X in the third round, therefore these items were removed of consensus, this result was called (Sr3). In the third round, the modal frecuency was determined through the next equation:
Where: Fm1 is the modal frecuency subject in the first round with the total number of people who participated in the three rounds.
The consensus percentage in the thrid round was determined through equation:
5. Results and discussión 5.1 First round The structured questionnaire was sent to 97 people invited to participate in the study by email and it was answered by 30 people who formed the expert panel. The panel included the participation of national and international researchers; The international researches were Mexico 7%, Argentina 3%, France 3%, Spain 4%, Italy 3% and Colombia 80%, as well as government people and national private sector people. Figure 1 shows the expert panel participation percentage for each nationality. Figure 1. Participants nationality
According to statistical criteria taken into account for the classification of the 196 variables, subjects who had modal value equal or higher than 4 and higher than average consensus percentage were considered as priority; subjects who obtained less or equal to 2 modal value and a percentage higher than the average consensus percentage were classified nonpriority and finally, subjects who did not accomplished the specifications were considered under discussion. Table 2 shows the average consensus percentage of each area Table 2. Average consensus percentage for each area Area
Consensus percentage average
Techonological innovation
35%
Commodities for biodegradable polymer production
29%
Commodities for biodegradable packaging production
28%
Aditives
31%
Technologies
34%
According to this, figure 2 shows the segmental variables classification in the first round, the distribution was: 26% priority variables, non priority 3% and 71% variables under discussion. Figure 2. Segmental variable classification in the first round.
Below figure 3 shows the expert panel training, it can be seen that in the panel of experts, the highest percentage was of doctors with 40%, followed by bachelors by 36%, then professionals with master's degree 17% and 7% of professionals with degree of specialization. Figure 3. Expert panel training
5.2 Second and third round The survey was sent to 30 experts panel to make corresponding changes according to their judgement; after obtaining the answers, items count under discussion and changes were made. The unchanged subjects were considered priorities for the third round. Subsequently, the final questionnaire was made, it was sent to panel again, table 3 shows subjects classified as priority for the third round according to the consensus obtained by the experts Table 3. Priority topics Area
Priority topics
Natural coatting
Technological Innovations
Food container Active packaging Shrink films Foamed Intelligent packaging Bioactive films
Commodities for biodegradable polymer production
Corn Mandiocca Potato Wood fiber Agroindustrial wastes Collagen Fatty acids Monoglycerids
Commodities for biodegradable packaging production
Banana fiber Chaff reed fiber Microbiane cellulose Polyhidroxyalkonates (PHAs),
Polyhidroxybutyrate valerate (PHBV), Polilactic acid (PLA) Microcrystalline cellulose Seaweed wet biomass Chitin glycerol
Aditives
antimicrobial agents powdered pigments coupling agents
Twin screw extrusion
Technologies
Blow film extrusion
Reactive extrusion Coextrusion
Injection molding Spray Colaminathing Bio processes Nanotechnologies
Table 4, shows classified subjects as priority for each round in the study. Conventions: .
Priority topics in the first and second round
.
Priority topics in the third round Table 4. Topics classified in each rounds
Topics
First round
Justifications second round
Third round
1. Technological innovations Monomer and Polymer development through biorefinery process
X
X
Thermoformed trayds
X
X
Bags for seedbeds
X
X
Bags for Ecological stations
X
X
Plastic bottles
X
X
Natural coatting
X
X
X
Multilayer packaging
X
X
Wrapping
X
X
Food container
X
X
X
Active packaging
X
X
X
Shrink films
.
Foamed
.
Intelligent packaging
X
X
X
Flexible packaging
X
X
Bioactive films
X
X
X
X
X
Nanocomposites 2. Commodities for biodegradable polymer production 2.1 Vegetal origin commodities Starches Corn
X
X
X
Mandioca
X
X
X
Potato
X
X
X
Sweet potato
X
Arracacia
X
Banana
X
Cidra potato
X
Cellulose Wood fiber
X
X
X
Agroindustrial wastes
X
X
X
Vegetal lipids Fatty acids
X
Others Orange Peel
X
X
Soya been peel
X
X
Rice Peel
X
X
Pineapple peel
X
X
2. 2 Animal origin commodities Collagen
.
Fatty acids
.
Monoglycerids
X
X
X
3. Commodities for biodegradable packaging production 3.1 Vegetal origin commodities Metilcellulose MC
X
X
Hidroximetil cellulose HMC
X
Carboximetil cellulose CMC
X
X
Banana fiber
X
X
X
Chaff reed fiber
X
X
X
Acetoglycerids
X
Pectins
X
X
Zeina
X
Corn gluten
X
X
Resins
X
X
High amylose
X
X
3.2 Microbian origin Microbian cellulose
.
Polyhidroxyalkonates (PHAs)
X
X
X
Polyhidroxybutyrate valerate (PHBV)
X
X
X
X
X
Polyhidroxybutyrate terephthalate (PHAT) Polilactic acid (PLA)
X
X
X
Microcrystalline cellulose
X
X
X
Seaweed wet biomass
X
X
X
X
X
X
3.3 Animal origin Marine and pisciculture Chitin Alginate Carrageenan
X X
X
Others Keratin
X
X
Biodegradable synthetic origin Polyvinyl Alcohol (PVA)
X
X
Polycaprolactone (PCL)
X
X
X
X
4. ADITIVES 4.1 Plastiziers Glycerol
Saccharose
Fructose
X X
X
X
X
4.2 Other Masterbach Antimicrobial agents
X
X
X
Powdered pigments
X
X
X
Liquids colorants
X
X
Thermosetting adhesive hot melt
X
X
Estearic acid
X
X
Citric acid
X
X
Anphiphilics substances
X
X
Coupling agents
X
X
X
X
X
X
X
X
X
5. TECHNOLOGY 5.1 Extrusion Single screw extrusion
X
Twin screw extrusion
X
Reactive extrusion
.
Coextrusion
X
Film blown extrusion
.
Coating extrusion
X
5.2 Molding Thermoforming Injection molding
.
Injection blown molding
X
Reactive injection molding
X X
X
5.3 Other Force Spinning
X
X
Electro spray
X
X
Spray
X
X
X
Compressed process
X
Laminathing
X
X
Colaminathing
X
X
Sealing systems
X
X
Bioprocesess
X
X
X
Nanotechnologies
X
X
X
3D printing
X
X
Inkjet
X
X
Printing inks
X
X
TOTAL
51
78
X
5.4 Print system
According to results obtained up to 2032, there is priority in techonological innovation development in subjects such as foam and active packaging production; it´s also an opportunity for searching new commodities in biodegradable polymers from potato, corn, agroindustrial wastes, microbian cellulose and faty acids production. In biodegradable packaging there were priorized subjects as polilactic acid, microbian cellulose and polyhroxyalkanoates and others. The use of technologies as twin screw extrusion, reactive extrusion and injection molding show great opportunity These results allow to visualize opportunities that can be implemented in research and business development by Cytbia team.
6. Conclusions From the methodological point of view, the foresight Delphi method used for this project development allows to priorize technological innovation, new commodities and technology in biodegradable packaging production in Colombia up to 2032 Through tools and methodologies for foresight studies, the uncertainty can be reduced by means of Delphi method in biodegradable packaging purchase and technology research in the decision making as well as new research lines and innovation entrepreneurship projects identified 37 priority subjects in common and 78 priority subjects in the third round were stablished with the biodegradable packaging foresigth study and with these results the innovation system actors can focus their strategies and projects.
Priority subjects up to 2032 in technology innovation area were: Natural coating, food container, active packaging, shrink films, foamed, intelligent packaging, bioactive films. The most important commodities for biodegradable polymers production up to 2032 according to experts could be: corn, mandioca, potato, wood fiber, agroindustrial wastes, collagen, fatty acid, monoglycerids. Banana fiber, chaff reed fiber, microbiane cellulose, polyhidroxyalkonates (PHAs), polyhidroxybutyrate valerate (PHBV), polilactic acid (PLA), microcrystalline cellulose, seaweed wet biomass, chitin, are the most important topics in the next years according to the experts. The prominent additives are: glycerol, antimicrobial agents, powdered pigments, and coupling agents Higher consensus technology in the three rounds: Twin screw extrusion, reactive extrusion, coextrusion, blow film extrusion, injection molding, spray, colaminathing, bio processes and nanotechnologies will be priority up to 2032.
References Balaraman, S., & Venkatakrishnan, K. S. (1980). Identifying engineering education goals and priorities for the future: An experiment with the Delphi technique. Higher Education, 9(1), 53–67.http://doi.org/10.1007/BF00149135 Becker, G., & Bakal, D. A. (1970). Subject anonymity and motivational distortion in selfreport data. Journal of Clinical Psychology, 26(2), 207–209. Cabero Almenara, J. (2013). Formación del profesorado universitario en TIC. Aplicación del método Delphi para la selección de los contenidos formativos. Educación XX1, 17(1), 21.http://doi.org/10.5944/educxx1.17.1.10707 Cancelo Hidalgo, M. J., Neyro Bilbao, J. L., & Baquero Úbeda, J. L. (2013). Probiotics as adjuvant therapy for vaginitis. Agreement based on the Delphi method. Progresos de Obstetricia Y Ginecologia. http://doi.org/10.1016/j.pog.2013.07.002 Créange, A., & Careyron, A. (2013). The diagnosis of chronic inflammatory demyelinating polyneuropathy: A Delphimethod approach. Journal of Neurology, 260(12), 3015–3022.http://doi.org/10.1007/s0041501371002 Dalkey, N., & Helmer, O. (1963). An Experimental Application of the Delphi Method to the Use of Experts. Management Science, 9(3), 458–467. http://doi.org/10.1287/mnsc.9.3.458 Debin, M., Souty, C., Turbelin, C., Blanchon, T., Boëlle, P.Y., Hanslik, T., Falchi, A. (2013). Determination of French influenza outbreaks periods between 1985 and 2011 through a webbased Delphi method. BMC Medical Informatics and Decision Making, 13, 138. http://doi.org/10.1186/1472694713138 Fletcher, A. J., & Marchildon, G. P. (2014). Using the Delphi Method for Qualitative, Participatory Action Research in Health Leadership. International Journal of Qualitative Methods. Retrieved from http://ejournals.library.ualberta.ca/index.php/IJQM/article/view/19025 De Liaño, B. G. G., & PascualEzama, D. (2012). La metodología Delphi como técnica de estudio de la validez de contenido. Anales de Psicologia, 28(3), 1011–1020.http://doi.org/10.6018/analesps.28.3.156211 Gordon, T., & Helmer, O. (1964). Report on a longrange forecasting study. The RAND Corporation, 1– 71. http://doi.org/10.1126/science.1088667 Gordon, T., & Pease, A. (2006). RT Delphi: An efficient, "roundless" almost real time Delphi method. Technological Forecasting and Social Change, 73(4), 321–333.http://doi.org/10.1016/j.techfore.2005.09.005 Graham, B., Regehr, G., & Wright, J. G. (2003). Delphi as a method to establish consensus for diagnostic criteria. Journal of Clinical Epidemiology, 56(12), 1150–1156.http://doi.org/10.1016/S08954356(03)002117 Helmer, O., & Rescher, N. (1959). On the Epistemology of the Inexact Sciences. Management Science. http://doi.org/10.1287/mnsc.6.1.25 Kauko, K., & Palmroos, P. (2014). The Delphi method in forecasting financial marketsAn experimental study. International Journal of Forecasting, 30(2), 313–327.http://doi.org/10.1016/j.ijforecast.2013.09.007 Landeta, J. (2006). Current validity of the Delphi method in social sciences. Technological Forecasting and Social Change, 73(5), 467–
482. http://doi.org/10.1016/j.techfore.2005.09.002 Liimatainen, H., Kallionpää, E., Pöllänen, M., Stenholm, P., Tapio, P., & McKinnon, A. (2014). Decarbonizing road freight in the future Detailed scenarios of the carbon emissions of Finnish road freight transport in 2030 using a Delphi method approach. Technological Forecasting and Social Change, 81(1), 177–191. http://doi.org/10.1016/j.techfore.2013.03.001 Linstone, Harold A. Turoff, M. (1975). Delphi Method: Techniques and Applications. 1975. Retrieved from https://www.ncjrs.gov/App/Publications/abstract.aspx?ID=256068 Martino, J. P. (1993). Technological Forecasting for Decision Making. Interfaces (Vol. 25). Kim, M., Jang, Y. C., & Lee, S. (2013). Application of DelphiAHP methods to select the priorities of WEEE for recycling in a waste management decisionmaking tool. Journal of Environmental Management, 128, 941–948. http://doi.org/10.1016/j.jenvman.2013.06.049 Nielsen, C., & Thangadurai, M. (2007). Janus and the Delphi Oracle: Entering the new world of international business research. Journal of International Management, 13(2), 147–163.http://doi.org/10.1016/j.intman.2006.07.003 Okoli, C., & Pawlowski, S. D. (2004). The Delphi method as a research tool: an example, design considerations and applications. Information & Management. http://doi.org/10.1016/j.im.2003.11.002 Wakefield, R., & Watson, T. (2014). A reappraisal of Delphi 2.0 for public relations research. Public Relations Review, 40(3), 577– 584. http://doi.org/10.1016/j.pubrev.2013.12.004 Zartha, J., Herrera, J. Prospectiva tecnológica. Aplicación del Método Delphi en Facultades y Programas de Ingeniería. Caso Universidad Pontificia Bolivariana. ISBN: 9789586969024. Primera edición. 2011. Zartha, J., Montes, J., Toro, I., Villada, H. Método Delphi Propuesta para el cálculo del número de expertos en un estudio Delphi sobre empaques biodegradables al 2032. Espacios. 2014. Vol. 35 (Nº 13), Pág. 10. Zeedick, D. The Modified Delphi Method to Analyze the Application of Instructional Design Theory to Online Graduate Education. 2012 1. Universidad Pontificia Bolivariana. Escuela de Ingenierías – Facultad de Ingeniería Agroindustrial. Email:
[email protected] 2. Universidad del Cauca. Email:
[email protected] 3. Universidad Pontificia Bolivariana. Email:
[email protected] 4. Universidad del Cauca. Email:
[email protected] 5. Universidad Pontificia Bolivariana. Email:
[email protected] 6. Universidad Pontificia Bolivariana. Email:
[email protected] 7. Centro Regional de Productividad e Innovación del Cauca. CREPIC. Email:
[email protected] 8. Universidad del Cauca. Email:
[email protected] 9. Universidad del Cauca. Email:
[email protected] 10. Instituto Tecnológico Metropolitano. Email:
[email protected] Vol. 36 (Nº 15) Año 2015 [Índice]
