Transport Research Areena 2014, Pariis
A peerformannce measuurement system for f publicc transpoort servicces in G Thessaaloniki, Greece Geeorgios Geoorgiadisa,*, Yiannis Y Xennidisa, Ioann nis Toskasb, Panagiotiss Papaioann noua a
School of Engineering, Department off Civil Engineerring, Division of Transport, Inffrastructure, M Management and d Regional Planning, Aristotle Universityy of Thessalonikki, Thessalonikii 54124, Greecee b Attiko Metrro SA,26th Oktoovriou Str. 40, Thessaloniki T 544627, Greece
Abstractt Local, naational and intternational staandards determ mine strategicc goals for puublic transportt authorities, while w they establish a set of perfoormance indiccators for the continuous monitoring m of financial f and quality perforrmance of public traansport services. This paperr investigates the developm ment of a perfoormance meassurement systeem for the public traansport services in Thessalooniki, Greece.. This system includes a sett of proposed indicators wh hich are in line with the relevant European stanndards. The proposed p systeem was develloped by the aapplication off customer satisfactioon and mysterry shopping surveys s that were w designed and tested inn the context oof this researcch. A pilot survey was w conductedd to validate the appropriaate format an nd organizatioon of the toools aiming at obtaining reliable performance p d data with minnimal financiaal cost. This paper p demonsttrates both ann efficient and d effective way to im mplement quaality measurem ment surveys, while, it prov vides useful guidance g to puublic transporrt agencies planning to introduce or o revise their performance measurementt programs. Keywords ds: public transsport; perform mance indicatoors; quality of service; custoomer satisfactiion; mystery shopping s Résumé Les norm mes locales, nationales n et internationalees déterminen nt les objectiffs stratégiquess pour les au utorités de transport public, tanddis qu'elles éttablissent un ensemble d'iindicateurs dee performancce pour la su urveillance continue de la perforrmance financcière et la quualité des serrvices de trannsport publicc. Cet article étudie le t public à Thessalo onique, en développement d'un syystème de meesure des perfformances dess services de transport Grèce. Lee système prooposé compreend un ensemb mble d'indicateeurs qui ont étté développéss par l'applicaation de la satisfactioon des clientss et des enquêêtes mystères qui ont été co onçues et testéées dans le caadre de cette recherche. r Une étudde pilote a été réalisée pourr valider le forrmat et l'orgaanisation approopriée des outtils visant à obtenir o des données fiables f sur la performance avec un coût financier min nime. Cet articcle présente unne voie utile et e efficace pour la mise m en œuvre de campagnees de mesures de la qualité, et il fournit des d conseils uutiles aux orgaanismes de planificattion des transsports publics pour introduirre ou réviser leurs l program mmes de mesurre de performaances. Mots-clé:: transport pubblic; les indicaateurs de perfo formance; la qu ualité du service; la satisfacction du clientt; client mystère
*
Correspoonding author. E-mail adddress:
[email protected] .
Georgiadis ett al. / Transport Research R Arena 22014, Paris
2
1. Introd duction A Perforrmance Meassurement Systtem (PMS) is i defined ass the process of quantifyiing the efficiiency and effectivenness of variouus actions (Neeely et al., 19995). PMSs en nable organizaations to folloow structured programs for colleccting, analyziing and proceessing data pertaining p to day-to-day acctivities and associated wiith efforts towards accomplishing a g specific strattegic objectivves. Data processing provides the opportuunity of develloping and regularly measuring peerformance indicators refleccting progresss over time and enabling peerformance com mparisons against either designatted thresholdss or other orgganizations’ performance. Apart A from seelf-improvemeent, PMSs enable coompanies to communicate their t progress so as to gain greater markket share, furthher investmen nts or even alliances with other orgganizations off various fieldds and disciplin nes. Benefits of PMSs do not n differ betw ween private and public seector organizaations. Howevver, while in the t private sector PM MSs are usuallly employed for profit maaximization pu urposes, in thee Public Trannsport (PT) in ndustry the case is noot quite the saame. The objeectives of PT to t fulfill comm mitments relatted with sociaal needs and sustainable developm ment challengges shape a more m complexx framework of strategic goals, whilee they bring about the monitorinng of both finnancial and noon-financial performance p in ndicators. Additionally, theere is a great variety of indicatorss pertaining too economic performance annd quality of service as weell as to the ennvironmental impact of PT system ms (TRB, 20003a; UITP, 20008; Santos annd Ribeiro, 20 013). The latter leads PT oorganisations to t develop PMSs wiith clear strattegic directionns and indicaators that can n be reliably measured andd cover all th he critical performaance aspects. This T paper prooposes such a PMS for the PT services in i Thessalonikki (Greece), which w was developedd by Thessalooniki’s PT Autthority (ThePT TA) (Toskas et e al., 2013). The rest of the paper is organized in a cross-seection mannerr. Thessalonikki PT servicess and the usefulness of establishiing a PMS foor the local PT T system are explored in Section 2. Seection 3 explaains the three key steps comprisinng the develoopment processs of ThePTA A’s PMS. Thee pilot projectt aimed at claarifying the ap ppropriate frameworrk for local quuality measureement surveyss is discussed in Section 4, while w conclussions are summ marized in Section 5. 5 2. Backgground Thessalonniki is the seecond largestt urban conuurbation in Greece G with a population of almost on ne million inhabitannts (official 20011 census daata). There arre 2.2 million passengers’ daily trips takking place by y all travel modes annd PT share is approximaated to 27% (OMPEP, ( 200 00). Thessaloniki’s PT sysstem is expeccted to be enhancedd with an unnderground metro m line as early as miid-2017. Thee Urban Trannsport Organiization of Thessalonniki (UTOT (OASTH)) is until u today thee sole PT operrator in the arrea. The comppany holds a concession c contract with w the Greekk Governmennt allowing forr exclusive rig ght of operatioons. The PT syystem is comp posed by a total of 78 7 urban and suburban s bus routes operatted by a fleet of 621 variouus size vehiclles. In 2011, 43 4 million vehicle-kkms were execcuted and alm most 180 million boardingss were counteed. The system m is characterrized by a low fare box recoverry ratio and the state subbsidy covered d 68% of totaal 2011 expeenses (OASTH, 2012). Improvem ment of delivvered quality and rationalizzation of reso ources’ allocaation are regaarded as neceessary and effective actions for booth attracting more m users annd reducing co osts. ThePTA was establishhed in 2001 annd is responsiible for the planning, coorddination and suupervision off PT in the metropoliitan area of Thessaloniki. T Until today, ThePTA’s monitoring m mecchanism is reelied upon a set s of key indicatorss referring onnly to economiic performancce and PT avaailability. How wever, these inndicators do not n appear to have a clear conneection betweeen them or with w a set off regularly uppdated strateggic goals. Furrthermore, passengerr satisfaction indicators aree not availablle. The latter is the reasonn for missing information on o the PT aspects thhat cause serioous user discoomfort. As a reesult, public opinion o feedbaack is only exppressed by cittizens who submit reequests and /or complaints to t the local PT T agencies. Th herefore, ThePTA cannot ffully support and a advise OASTH on any improovement actioons. A standarrdized and cleear process foor exchangingg data betweeen the two agencies on all necessaary performannce topics has not yet been established. e T These issues allong with the upcoming introductiion of compettitive tenderinng proceduress (where TheP PTA will playy the role of thhe competent authority) create an urgent need for f a comprehhensive and coontinuous mon nitoring and assessment a of PT services through t an appropriaately designedd PMS. The esstablishment of o such a systeem will also enable e ThePTA A to commun nicate both
Georgiadis ett al. / Transport Research R Arena 22014, Paris
its activitties and progreess. By this means, m pressurre will be appllied for necessary investmeents in the PT system as well as atttaining strateggic partnershiips with other PT organizatiions in Greecee and abroad. 3. Develoopment steps of ThePTA’ss PMS The deveelopment proocess of TheP PTA’s PMS comprised th hree steps, namely: n (a) tthe formulatio on of the authority’s strategic gooals with regaard to perform mance, (b) the connection of o the strategicc goals to finaancial and quality inndicators and (c) the concuurrent developpment of the necessary measurement toools. The evaaluation of service quality q in PT systems s requirres the executtion of fieldw work surveys. Given, also, tthat ThePTA had never before caarried out succh surveys a pilot surveyy project was designed annd performed to validate the t whole developm ment process. The T pilot project’s aim wass to verify the applicability of o the proposeed quality meaasurement survey toools. The following subsectiions, briefly present p the dev velopment of ThePTA’s T PM MS. 3.1 Strateegy Formulatiion In the begginning of thiis decade, TheePTA found ittself at a turning point. Thee expansion oof fixed PT bu us services in the sem mi-urban and even rural areeas of Thessalloniki along with w the drasticc reduction off state subsidies and the forthcom ming launch off the Thessalonniki’s undergrround metro line l forced forr a global reviision and enrichment of ThePTA’’s strategy. Thhat was appareently necessarry, in order fo or ThePTA to adapt its role into this new wly created local operrational environment. In additioon to the locaal level requirements, the foormulation off ThePTA’s sttrategy was allso based on objectives either devveloped or prroposed by reppresentative urban u PT agen ncies in Europpe and key inntergovernmen ntal policy making organizations. o The scope waas to encouragge extroversion of the local PMS and enssure its compliiance with both national and international impperatives. Α corresponding review madee evident that European PT T agencies plan theirr strategies in order to succcessfully manaage the follow wing issues (T Toskas et al., 22013): (a) adaaptation of PT servicces to the locaal socio-econoomic changes, (b) use of prrivate car, whhich continuess to attract an important share of trip t makers, (c) ( tackling off climate channge and (d) fu ulfillment of broader b comm munity goals (e.g. social cohesion,, equitable acccess etc.). In general, g PT aggencies aim to o establishingg a constituentt role for PT in i modern societies and linking thhe existence and a proper funnctioning of PT P services with w the achievvement of the quality of life in citties; this approoach is reflectted in the visiions that a lott of the review wed Europeann PT organisattions have adopted for f the desirabble developmeent of their loocal PT system ms (HSL, 2011; SL, 2010; S SPT, 2008; WL, W 2011). The aforeementioned diirections are consistent c withh the guidelin nes which werre shaped at inntergovernmeental level. UITP’s vision v for PT in i 2020 highllights the importance of PT T usage towarrds the achievement of quallity of life and sustaainable develoopment of com mmunities whhile underlinin ng the need to t further impprove PT attraactiveness over privvate car (UITP P, 2005). Support of initiattives leading to t greater eneergy efficiencyy of transportt means is also menttioned (OECD D, 2012; EC, 2011) 2 as a crittical element of o urban mobiility policies. The toolss for the formuulation of the strategy withh regard to perrformance com mprised, also, a series of fo ocus group discussions - where special attentionn was given too the disabled d, elderly and socially ‘deprrived’ citizenss - so as to incorporaate the contem mporary needss of local com mmunities. Th he six (6) straategic goals ssummarizing ThePTA’s T strategy are a reported inn the first coluumn of Table 1 and formed the basis for the definitionn of the authorrity’s PMS indicatorss. 3.2 Definnition of perforrmance indicaators The definnition of PT performancee indicators first f requires that ThePTA A’s strategic goals are lin nked with particularr performancee aspects of thhe delivered PT P services. This T process allows a the devvelopment of indicators that can be b classified innto specific categories of performance p to opics, while enabling e the m monitoring of ThePTA’s T progress record towardds achieving the t pre-determ mined strategic goals. The second s colum mn in Table 1 shows the nine (9) categories of performance topics that were w used to match m the straategic goals aand define, att the same time, ninee (9) particulaar groups of inndicators. Economicc indicators measure m the evolution e and relationships between servvice inputs, prroduced and consumed services and, a thereforee, they monitoor the efficienncy and effecctiveness of loocal PT operaations (Hensh her, 2007). The remaaining eight grroups of indicators measuree quality of PT T services (CE EN/TC 320, 20001).
Georgiadis ett al. / Transport Research R Arena 22014, Paris
4
Table 1: Strategic S goals and a examples off the correspondding developed d indicators of thhe PMS ThePTA strrategic goals
Perform mance Topics
Typpe of inddicators
ples of developedd performance inddicators Examp
Total numbber of indicatorrs
Streamlininng the supply of public p transport seervices in accordaance with the neeeds and demands of local comm munities
Econom my
DP PM
Ridersship; Revenue vehhicle-kms/Trips pper vehicle/employee; Cost per vehicle-houur/ km
26
PM DP
Frequeency; Span of serrvice; Stop spacinng
7
CSS
Satisfaaction from: Freqquency; Area coveered
6
DP PM
Proporrtion of low-floorr vehicles (%)
7
MS SS
Examiination of accessiibility conditions during boardiing/alighting of passengers p
3
CSS
Satisfaaction from ticketts' acquisition andd validation proced dures
2
PM DP
Speed d of response to cuustomer inquiriess
10
MS SS
Examiination of: Staff helpfulness; h appearance
1
CSS
Satisfaaction from fare policies p
2
DP PM
Numb ber of reported criimes/accidents peer million vehiclee kms
6
MS SS
Examiination of safety equipment e availaability
2
CSS
Percep ption of: Safety; Security S
2
PM DP
Numb ber of park and ridde places per resident
4
MS SS
Examiination of conditiions on park and rride facilities
2
CSS
Satisfaaction from numbber and easiness oof transfers within n network
2
Customeer Care
DP PM
Numb ber of fare subscriiptions enabling ssustainable urban n mobiliity trips combinedd with public traansport
1
Environnmental Impact
DP PM
Modall Split (%)
1
DP PM
Comm mercial Speed; Onn-time performannce; Reliability; Numb ber of road calls per p million vehiclee kms
8
CSS
Satisfaaction from: Travvelling times; On--time performancce
4
DP PM
Numb ber of stops/stationns equipped withh real time passen nger information displays (%);
9
MS SS
Examiination of availabbility, damages annd correctness of inform mation sources at stops/stations; onn-board
4
CSS
Satisfaaction from: Inforrmation Accuracyy; Adequacy; Clearn ness
5
PM DP
Mean vehicles' age; Nuumber of vehicless equipped with air-con ndition
3
MS SS
Examiination of: Cleanlliness; Proper funnctioning of equipm ment at stops andd on-board;
4
CSS
Satisfaaction from: Cleaanliness; Ride sm moothness; Ambieent condittions
3
PM DP
CO2 em missions per passsenger-km; Total energy used per 100 veehicle-kms
5
CSS
Percep ption of environm mental impact cauused by public transport operations
1
Availability
Accessibbility
Ensuring eqqual mobility opportunitiees and possibilitiees for all
Customeer Care
Securityy
Accessibbility Encouraginng the use of environmenntally friendly meeans of transportt and supporting their t combinationn with motorizedd modes
Reduction of o travelling times
Introductionn of intelligent transport syystems (ITS) and innovative solutions s for urbaan mobility maanagement
Time
Informaation
Comfortt Modernizattion of public transport fleeet and infrastruccture Environnmental Impact
Georgiadis ett al. / Transport Research R Arena 22014, Paris
Availabillity and accesssibility indicaators deal witth issues that have a decissive effect onn the selection n of travel modes sinnce, in many cases, inadeqquate planningg and scheduliing of PT servvices and infrrastructures may m lead to their systtematic avoidaance. Environnmental impacct indicators evaluate factorrs that do not only attract th he interest of PT useers but also thhose influenciing the positioon that the local communitty takes towarrds the PT sy ystem. The rest of quuality of serviice indicators refers mainlyy to attributes characterizingg comfort, coonvenience and easiness in using PT. P The abovve categorizatiion of indicattors provides with w the oppo ortunity to evaaluate the proogress of Thesssaloniki’s PT against the basic suustainability parameters. p Ecconomic, social and environnmental aspeccts are fully co onsidered. The definnition of perfformance topiics and the suubsequent dev velopment of indicators m measuring the quality of service foollowed the guuidelines and principles of EN13816:200 02 (CEN/TC 320, 2001). A Additionally, a literature review was w conducted in order to fiinalize the fulll indicators’ list; l Transporttation Researcch Board repo orts (TRB, 2003a; 20003b), UITP initiatives (UIITP, 2008) annd performancce measures established e byy the PT systeems which were reviiewed during the first step of the processs have provid ded valuable inputs. i Review w of PMSs in n other PT systems and a accountinng of EN138166:2002 also faacilitates perfo formance com mparisons betw ween ThePTA and other organisattions on the same grounds (Georgiadiss, 2012). Thee fourth colum mn of Table 1 presents only o some representtative examplees of the deveeloped indicattors. ThePTA’s PMS incluudes a total off 130 indicators and the number of o the indicatoors in each of the t nine groupps appears in the t last colum mn of Table 1. 3.3 Devellopment of meeasurement tools The thirdd column of Table T 1 show ws the type of o indicators included i in thhe PMS accoording to the coding of EN138166:2002. Directt Performancee Measures (D DPM) are indiccators calculatted from quanntitative PT daata that are collected: (a) through appropriate eqquipment suchh as Automateed Vehicle Loocation (AVL L) equipment, passenger counters, etc., or (b) frrom records annd registries being b kept by PT organizattions. In the caase of Thessaloniki, the installatioon of AVL eqquipment on operator’s o bus fleet has enab bled the collection of inforrmation, which h supports a large poortion of the proposed p indiccators (especiially those wh hich refer to tiime aspects annd ridership). However, many of the proposedd environmenntal impact inndicators (e.g g. CO2 emissiions, noise leevels etc.) req quire data sharing with w other loccal organizatioons that posseess the appro opriate measurrement equipm ment and carrry out the required surveys. s The neceessary data for the calculattion of Mysterry Shopping Survey (MSS S) and Custom mer Satisfactio on Survey (CSS) inndicators deriive from the execution of the corresp ponding surveeys. CSSs innclude interviiews with passengerrs who are askked to assign satisfaction raatings to speciific attributes of PT servicees. MSSs are based b on a set of quaality checks performed by trained t staff on PT infrastru uctures and seervices. MSSs could represeent a more objective quality of serrvice measureement compareed to the subjeective measurrements basedd on passengerrs’ ratings. Hence, CSS C indicatorss measure thee perceived quality q (CEN/TC 320, 2001) of service,, while MSS indicators along witth DPM qualiity indicators measure the delivered qua ality of servicce. Both MSS Ss and CSSs had never been execcuted in the past, on a reguular basis, for Thessaloniki’ T s PT system. As a consequuence, the appllication of the propoosed PMS requuired the deveelopment of thhe respective measurement m tools. In the casse of MSSs, calibrated checcklists that conntain instructiions and guidaance for perfoorming a pre-d defined set of qualityy checks weree developed ass the respectivve measurement tools. The content of cheecklists was determined d by the 166 proposed MSS indicators so that each of them was linked to at leeast one certaain quality cheeck. Three (3) differrent MSS checcklists were designed d to allow bus stopss, on-board ennvironment annd ticket saless points to be examiined separately. Quality cheecks were forrmulated as qu uestions in tw wo (2) differennt formats and d asked to be answeered by appoinnted surveyorss. In the first format f - closed d ended formaat - surveyors were required d to verify the existeence of a specific object or service by b answering ‘’yes/no’’. Inn the second format -scalle formatsurveyorss had to ratee, using a 5--point scale, the services or objects thhey observedd given that each rate corresponnded to a pree-determined level l of qualiity. For instan nce, the exam mination of clleanliness on bus stops included one closed-ennded and twoo scale checkks. The availaability of a littter bin was aasked with a ‘’yes/no’’ question, while the cleeanliness of thhe bus stop’s infrastructuree along with thhe amount off litter were assked to be rated usinng a scale in which w 1 corressponds to ‘‘veery dirty’’ and d ‘‘many litterr all around thhe waiting passsengers’’ and 5 corresponds to ‘‘perfectly cleean’’ and ‘‘nno litter obserrved around the t bus stop’’’, respectively y. Overall, sixty-fourr (64) qualityy checks havee been develooped from wh hich twenty-tthree (23) weere in scale fo ormat and
Georgiadis ett al. / Transport Research R Arena 22014, Paris
6
concernedd mainly withh the level of cleanliness, c thhe extent of damages on inffrastructure annd information sources, ride smooothness and beehavior of felllow passengerrs and staff. For CSSs, the measurrement tools are, usually, semi-structurred questionnnaires setting the frameworrk for the interview ws with PT useers. These inteerviews may have h the form m of face-to-faace or focus grroups discussiions while internet, mail m and telepphone surveyss may also be executed. TheePTA decidedd to launch CS SSs by conduccting faceto-face innterviews witth PT passenngers. Subseqquently, the necessary n queestionnaire haad to be app propriately designed.. Its content was w determineed by the 27 proposed p CSS S indicators soo that each off them was lin nked to at least onee specific quuestion to thee passengers.. These quesstions followed two (2) different form mats, i.e., ‘‘statemeent-agreementt’’ and ‘‘aspect-satisfactionn’’ format. Fo or instance, a CSS indicatoor named ‘‘Saatisfaction from on-ttime performaance’’ (see Taable 1) can bee linked with either of the two followinng questions: (a) ( Please indicate your y agreemeent using a sccale from 1 (ffully disagree) e) to 5 (fully agree) a with thhe following statement: s ''Bus servvices are on--time'' or (b) Please indicaate the level of your satisf sfaction usingg a scale from m 1 (fully dissatisfieed) to 5 (fullly satisfied) on o the follow wing service aspect: a ''On-ttime performaance''. Overaall, the 27 proposedd CSS indicators required either 45 stattement-agreem ment or 36 asspect-satisfacttion questionss so as to collect thhe necessary data for the inddicators’ meassurement. 4. Pilot survey projecct The capaacity of ThePT TA in terms of o collecting accurate a and sufficient s dataa for the calcuulation of thee proposed MSS andd CSS indicatoors was investigated throughh a specially designed d pilott survey projecct. The key ob bjective of the projecct was to clarify whether MSSs M and CSS Ss can be fully y conducted at a Thessalonikki’s PT system m and also to establiish a framewoork for their operational o deesign, which will ensure thhe collection of as many as a possible accurate data d with the least possiblee expenditure. The project was w executed during April 22013 and wass consisted of two distinct parts; a mystery shoppping and a cuustomer satisfaaction pilot suurvey. 4.1 Mysteery shopping pilot p survey 4.1.1 Metthodology The partiicular goals of the pilot MSS were: (a) the identificaation of the quuality checks that can be objectively o carried ouut by trained surveyors, s (b)) the recognition of any tim me periods or/aand service areeas that imposse barriers to the impplementation of MSSs and (c) the detecttion of any ad dditional factors which affecct delivered quality q and are not inncluded in thee content of mystery m shoppping checklistss. The pilot MSS M included survey at 56 bus stops, 68 bus trrips and 2 tickket sales points of Thessalooniki’s PT system. Each seervice locationn was assesseed with its appropriaate checklist. The T survey prrogram was designed d so as to allow the examination oof a variety of different service coonditions (e.g. service areass, time periods, etc.) and su ubsequently thhe investigatioon of the applicability of MSSs under a variety of o different cirrcumstances. Researchhers who weree recruited foor the pilot MSS M received d the same trraining (four (4) days of classroom presentatiions and fielddwork demonsstrations) so ass to be able to o perform the designed d quallity checks and quantify the colleccted observations in the sam me logic. How wever, subjectivity and skillls of surveyorrs along with the t format of checks and the objject or servicce being exam mined could possibly p still affect the coollection of objectively o measuredd observationss. This fact maay affect the reeliability and accuracy of thhe collected ddata. Thereforee, the pilot MSS wass carried out in i pairs so thaat each qualitty check on th he same objecct or service w would be perfformed by two surveeyors at the same s time butt without sharring any inforrmation on their answers. T Thus, any dev viations in collected results will characterize c t reliability of each perfo the formed check and may alsoo highlight th he need of additionaal training or changes c on thee format and wording w of qu uality checks’ description. d 4.1.2 Ressults Pilot MSS was compleeted successfuully at all the pre-defined p bu us stops and tiicket-sales pooints. The overrcrowding at 11 (ouut of 68) pre-sselected bus trips t did not allow the com mpletion of soome quality cchecks on-boaard. These checks haad to do mostly with the exxamination of: f: (a) the intern nal surface off vehicles, (b) the printed in nformation sources on-board, o (c) the t proper funnctioning of tiicket issuing and a validatingg machines annd (d) the exteent of any damages on vehicle’s equipment. e M of the oveercrowding prroblems were identified durring the early afternoon Most periods at a bus lines that t serve thee eastern urbaan districts off the city. Thhe completionn of mystery shopping
Georgiadis ett al. / Transport Research R Arena 22014, Paris
checklistss lasted betweeen 5 to 7 minnutes at bus stoops and tickett sales points. Vehicles’ sizee and level off crowding influenceed the time thaat was requireed for quality checks on-board. Surveyorrs spent, on avverage, 10 – 20 2 minutes for mediuum and articullated buses annd up to 10 miinutes for min nibuses. They,, also, reportedd that their prroductivity reduced significantly s a after 4 hours of o surveying. Consequeently, for thee full-scale im mplementationn of MSSs, itt is made evvident that tw wo surveyors should be required, in order to ovvercome the problems p durinng overcrowdiing time perioods, especiallyy for bus trips which are performed with articullated buses. A pair of surveeyors would be b able to distrribute accordiingly the quality checks so as to focus on few wer objects/seervices and complete the survey withinn reasonable time. Moreov ver, if the assessment of bus tripss on low-frequuency bus lines is included d in a daily program of survveys, then, theese should be accom mpanied with trips t on mediuum and high frequency f buss lines so thatt the 4-hour tiime period can n be spent efficientlyy. Table 2 presents p the tw wo quality cheecks with the lowest and hiighest reliabiliity status for bboth of the fo ormats that were usedd. The reliabiility of qualityy checks was determined by b measuring the deviation between the quantified answers of o the surveyoors who exam mined the sam me object/serviice at the sam me time. No ddeviation (0) means m that the surveeyors gave thee exact same rating. r The deeviation for sccale format quuality checks can be one (1 1) or more than one (>1) since ann 1-5 scale is used for giviing answers but b for closedd-ended qualityy checks, the deviation cannot exxceed one (1) since ‘’yes/noo’’ answers weere used and the t available options o were oonly two. Table 2: Reliability of perfformed quality checks c for pilott MSS Format
Scale
Closed- ended
Qualityy checks
Deviation beetween surveyorss’ answers 0
1
>1 >
Extent of damages/vanddalism on bus stopp's infrastructure
34.6%
46.9%
18.5%
Cleanliiness of vehicles' seats
42.6%
48.2%
9.3%
Vehiclee’s approaching distance d to boardiing/alighting poin nt
82.6%
15.6%
1.8%
Behavior of fellow passengers
86.4%
12.1%
1.5%
Total (A All scale checks)
48.2%
31.8%
20.0% 2
p Noticeaable cracks on veehicle's window panels
77.1%
23.0%
:
Unpleaasant smells in vehhicle's interior
84.4%
15.6%
:
Availabbility of litter bins at bus stops
97.8%
2.2%
:
Proper functioning of ticcket issuing machhines on-board
100%
0%
:
All closed-ended checks) Total (A
90.2%
9.8%
:
The form mat of qualityy checks affeccted the reliabbility of colleected data. 900.2% of the pperformed closed-ended checks goot the exact same rating byy both surveyoors who perfo ormed the sam me check at thhe same time, while for scale cheecks this perccentage falls to t 48.2%. Thee higher perccentages of quuantified answ wers that had deviation values eqqual or more than t one (1) were w found foor quality cheecks on cleanlliness and conndition of infrrastructure and equippment as welll as on the ridde smoothnesss of bus tripss. Further trainning of surveeyors and morre detailed descriptioons of the pree-determined levels l (rates) of quality is required in suuch cases. Reduction of ratting scale, for exam mple usage of a 4-point scalle (Lu et al., 2009), may also a be the nexxt best optionn, especially for f quality checks whose w deviationn exceeded onne in many caases. Finally, the t surveyors also observedd the existence of information sources which w pertain to the same subject but provide conflicting c innformation. Thhis issue wass not includeed in the initiial design off the mystery shopping checklistss and it was decided d that itt should be addded and exam mined along with w the MSS indicator thatt evaluates availabiliity; damages and a correctnesss of informatiion sources (ssee Table 1).
Georgiadis ett al. / Transport Research R Arena 22014, Paris
8
4.2 Custoomer satisfactiion pilot surveey 4.2.1 Metthodology The specific goals of the t pilot CSS were the explloration of paassengers’ respponse rate and accuracy off collected replies aggainst: (a) the length of thhe interview (questionnairre), (b) the foormat of quesstions, (c) thee place of interview w, (d) the servvice conditions (e.g. level of o crowding, time t period) and a (e) the prrofile of passeengers and their jourrneys. Therefo fore, the schedduling of piloot CSS was formed, f in ordder to examinne as many as possible different combinationss of service conditions. The T interviews were plannned and execuuted at bus stops, s bus terminalss and on-boardd over differeent time periods and areas of PT networrk. This varietty of condition ns offered the possiibility of inteerviewing passsengers of diverse d profilees (e.g. age, gender, purpoose of travel etc.) and identifyinng circumstances that may prevent p the exxecution of CS SSs. Both the format and thhe number of questions, whhich are includ ded in a custoomer satisfaction interview are likely to influennce the accuraacy of collected replies as long as interv viewees may find f difficulties in understaanding the content of o questions (iff they are form matted in a paarticular way) or lose intereest after a certaain period of time. Five (5) questiionnaires have been develooped and testeed during pilo ot CSS, in ordder to clarify tthe relationshiips among these parrameters. Eachh questionnaiire had a diffferent size, i.ee., 16, 21, 266, 29 and 31 questions cov vering, all together, the full list off CSS indicatoors. Four (4) out o of the fivee (5) questionnnaires had a siingle format, i.e. two of them werre in ‘’statemeent-agreementt’’ and the othher two were in n ‘’aspect-satiisfaction’’ forrmat. One queestionnaire included both formats,, since it was presumed p thaat some service aspects wouuld have been more clearly described in Greek,, if they were asked with a certain questtion-format. In n order to dettermine the acccuracy of thee collected responsess, the surveyoors highlightedd the cases where w they notticed that partticipants seem med to feel or expressed indifferennce, distraction or serious confusion withh the contents,, wording or/aand length of tthe interview. 4.2.2 Ressults Pilot CSS S resulted in 206 completted interviewss with a meaan response rate r of 55.5% %; a value witthin those achieved in other PT syystems (TRB,, 2005). Figure 1a shows th he detailed respponse rates. T The surveyors noted that many passengers had particularly negative reacctions when asked a to partiicipate duringg early morniing hours. Althoughh on-board annd bus stop/teerminal response rates werre similar, thee number of interviews, at a absolute terms, waas almost twicce when on-booard. This is possibly p due to t the fact thatt a central trannsport hub wh here many passengerrs could be concentrated and approacched does nott currently exxist in Thesssaloniki’s PT network. Interview ws lasted for approximately a y 5-7 minutes and surveyorrs found that after a four (4) hours of surv veying per day their productivity fell f significanntly. Response R Ra ates
BUS STOP/TERMINAL
54.1%
07:0 00 ‐10:30
AGE
>50 26‐50 50
AGE
ON N‐BOARD
14:0 00 ‐17:30
26‐50
