agencies by briefing or Disposition Form. ...... .?~108. This recruiter adeptly determined the applicant's future plans, thus allowing ...... think before I tamc a stad.
01H[[ ILELCQ" Reasearch Report 1441
mt 100
Development of Behavior-Based Rating Scales and ,nalysis of Recruiter Selection Battery Data for the Army Recruiter Job
Walter C. Borman, Teresa L. Russell, and Nancy J. Skilling Personnel Decisions Research Institute
"DTIC
S
l
ELECTE•
SEP 19870 4
Manpower and Personnel Policy Research C>Manpower and Personnel Research Labo. ...,ry
U.S. Army Research Institute for the Behavioral and Social Sciences May 1987 Approved for public release; distribution unlimited.
87
9
9
053
Research Report 1441
DEVELOPMENT OF BEHAVIOR-BASED RATING SCALES AND ANALYSIS OF RECRUITER SELECTION BATTERY DATA FOR THE ARMY RECRUITER JOB
Walter C. Borman, Teresa 1. Russell, and Nancy J. Skilling Personnel Decisions Research Institute
for Contracting Officer's Representative Deirdre J. Knapp
Manpower and Personnel Policy Research G;-oup Curt Gilroy, Chief Manpower and Personnel Research Laboratory Newell K. Eaton, Director
U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES 5001 Eisenhower Avenue, Alexandria. Virginia 22333-5600 Office, Deputy Chief of Staff for PersonnelN Department of the Army
May 1987 Army Project Number 2026373 1A792
Manpower and Personnel
iiiI
I
U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES A Field Operating Agency under the Jurisdiction of the Depuaty Chief of Staff for Personnel
WM1. DARRYL HENDERSON. COL, IN Commanding
EDGAR M. JOHNSON Technival Director
Accesion For NI R~
Research accomplished under contract for the Department of the Army
DTIC
TAB
L3
anuce0
Personnel Decisions Research Institute
Justification ......... MYArTIBy................. Dit. ibojtion
Technical review by
Availability Codes - A'v'a'il and I r Divt Special
Deirdre J. Knapp Rebecca M. Pliske
1_
NOTICES
FINA DISOSTON Thisreprt may bedstroyed whniti
o ogr
ncdd
Plhs
do not return it to the U.S. Army Research Institute for the Behavioral and SocialI NQTE. The findings in this report arc not to bc construed as an official Dcpartmcnt of thc Army position, unless so designated by othcr authorized documcnts.
UNCLASSIFIED .SCURITY CLASSIPICA~tON OF TillS PAGE (Man Dote Enter**
.. REPRT .. OCUENTTIO PAE•READ REPORT DO MENTATION PAGE A. GOVT ACCESSION NO.
1. 0EPORT NUMGER
ARI
INSTRUCTIONS BBEFORE COMPLETING FORM
Research Report 1441
100 -
DEVELOPMENT OF BEZ1VIOR-BASED RATING SCALES AND ANALYSIS OF RECRUITER SELECTION BATTERY DATA FOR THE ARMY RECRUITER JOB AUTHOR(V
Final Report June 1985 - June 1986
Nancy J.
CONTRACT OR GRANT NUMBER(*)
..
DAAG29-81-D-0100
Skilling
PERFORMING ORGANIZATION NAME AND ADDRESS
10.
Personnel Decisions Research Institute 43 Main Street S.E., Suite 405 Minneapolis, Minnesota 55414
II.
PERFORMING ORO. REPORT NUMBER
.
Walter C. Borman Teresa L. Russell S.
TYPE OF REPORT & PERIOD COVERED
S.
4. TITLE (ind 8ue4ftle)
7.
3. RECIPIENT*S CATALOG NUMBER
AREA & WORK UNIT NUMBERS
2t.26373lA792 1'. REPORT DATE
CONTROLLING OFFICE NAME AND ADDRESS
U.S. Army Research Institute for the Behavioral and Social Sciences, 5001 Eisenhower Avenue Alexandria, Virginia 22333-5600
14. MONIrORING AGENCY NAME & ADDRESS(I
PROGRAM ELEMENT. PROJECT. TASK
diftfrnt from Contrll'id Office)
May 1987 1.
NIMBER OF PAGES
140 IS.
SECURITY CLASS. (uf this report)
Unclassified IS.
16.
DECLASSIFICATION/DOWNGRAOING SCHEDULE
DISTRIBUTION STATEMENT (of hislRlpot)
Approved for public release; distribution unlimited.
17.
DISTRIBUTION STATEMENT (of the obetroct entotod In Stoc k0.
iI diferent from Report)
i6. SUPPLEMENTARY NOTES
This task was performed under a Scientific Services Agreement between the U.S. Army Research Office and Battelle Columbus Laboratories, Research Triangle Park, NC 27709. The task was requested and funded by ARI. ARI's technical monitor was Dr. Deirdre J. Knapp. I1.
KEY WORDS (Continue on revoers
Recruiting Military accession Performance Criterion measurement 26, ANSTNACri
erta-
sIde If necessary and Identlfy by olock number)
Criteria Predictors Personality tests Factor analysis
maovogin side~ fnoeosomyawmdIeoatif7by block number)
kne purpose of the present research was to develop behavior-based rating scales for evaluating enlisted Army recruiter performance. To accomplish this, rating scales developed previously for Navy recruiters were revised with the help of experienced Army recruiters. Retranslations of Army recruiter performance examples into the Navy recruiter dimensions and the final revised behavioral anchors into the same dimensions suggested that (1) the Navy's eight-dimension description of recruiter performance requirements is appropriate for Army use (Continued)
DOl
1M3
EDTION OF I MOV 65 ISOBSOLETE
UNCLASSIFIED i
SECURITY CLASSI; ICATION OF THIS PAGE (Whme
Data ERoatsed)
SftCURiTY 1 CLASSSIFICATION OF YTIS PAOGL(Whon
PQe
Rneteed)
ARX R45sea1ch Aepoit 1441. 20.
Abstract (Continued)
and (2) the final Army recruiter scales with revised definitions and anchors provide a relatfvely unambiguous depiction of the various performance areas for this job, as well as the effectiveness levels within each area. A second purpose here was to initiate evaluation of the experimental Recruiter Selection Battery (RSB-X) for use in identifying persons with high potential for Army recruiting. Toward this end, the RSB-X was administered to 417 trainees in the Army recruiter school. Responses to the RSB-X were scored on four keys developed previously to predict performance in Navy recruiting, and these scores for the Army sample were compared to scores on the same keys obtained by Navy recruiters in a 1977 sample (N - 194). Total scores across all four keys were very similar for the Army and Navy samples. However, on three of the four keys, Army mean scores were significantly different from Navy means.' actor analyses of correlations between keyed items for both the Army and Na samples revealed factors that were readily interpretable and were similar a ross the two samples. These factors potentially indicate underlying personalit constructs important for success in military recruiting. The Army recr iter rating scales should be used in future personnel research when performan e scores for individual recruiters were required. They might also be used fo recruiter assessment to target professional development and growth intervent ns. Finally, after additional reqearch, the RSB-X could be employed to help imentify Army enlisted personnel likely to perform effectively as recruiters.
UNCLASSIFIED SECURITY CLASSIFICATION OF THIS PAGE(WI~hn Daoe
ii
Entered)
ARI Research Reports and Technical Reports are intended for sponsors of R&D tasks and for other research and military agencies. Any findings ready for Implementation at the time of publication are presented in the lost part of the Brief. Upon completion of a major phase of the task. formal recommendations for official action normally are conveyed to appropriate militarf agencies by briefing or Disposition Form.
iv
FOREWORD
The Army Research Institute (ARI) is currently conducting research designed to enhance the quality of the Army's recruiting force. One vehicle for maintaining an effective recruiting force is selection of individuals who possess the skills, abilities, and personal characteristics requisite to successful recruiter performance. ARI's work on recruiter selection is an essential part of the mission of the Manpower and Personnel Policy Research Group, Manpower and Personnel Research Laboratory, to conduct research to Improve the Army's capability to effectively and efficiently recruit its personnel. Work was undertaken to enhance the effectiveness of recruiter sales tralning through improved selection for training in accordance with the Memorandum of Understanding Between the U.S. Army Research Institute and the U.S. Army Recruiting Command, signed 17 Oct 84 and I Oct 84, subject: ARI/USAREC Research and Development Program. Results of this effort were briefed to the Chief of the Recruiter Training Division, Recruiting Operations Directorate, USAREC, on 15 October 1986. This report summarizes research on military recruiter selection and performance measurement and provides guidance on the predictors likely to be useful for military recruiter selection.
EDGAR N. JOHNSON Technical Director
V
DEVELOPMENT OF BEHAVIOR-BASED RATING SCALES AND ANALYSIS OF RECRUITER SELECTION BATTERY DATA FOR THE ARMY RECRUITER JOB
EXECUTIVE SUMMARY
________________________
Requirement: One approach to maintaining an effective recruiting force is to select individuals possessing the personal characteristics Important for success in recruiting. This in turn requires performance criteria to provide accurate appraisals of recruiter effectiveness. Accordingly, the purposes of the present research were (1)to develop behavior-based rating scales for evaluating enlisted Army recruiter performance, and,(2) to initiate evaluation of the experimental Recruiter Selection Battery (RSB-X) for use in identifying perst.is with high potential for Army recruiting.
Procedure: In the scale developm'ent effort, behavior-based rating scales constructed previously for the Navy recruiter job served as a starting point. These scales were revised for Army use in workshops with experienced Army recruiters. Specifically, performance examples of recruiter behavior were generated by recruiters not aware of the dimension structure of the Navy scales to check for possible deficiencies of these dimensions for describing present requirements for Army recruiter performance. Also, recruiters attemptedi to retranslate these examples into the Navy dimensions to check furthq.r on the adequacy of these dimensions for the current Army recruiter job. And finally, another recruiter group helped revise the Navy definitions and behavioral anchors to be fully appropriate for describing Army recruiter performance, and recruiters provided a final check on these revisions in a retranslation of the revised behavioral anchors. Regarding the RSB-X, the personality and background sectieons of this instrument were administered to 417 Army NCOs attending the recruiter school. Initial anailyses of the responses included scoring individuals on keys developed in previous research with these same items in the Navy and Marine Corps, and then comparing the distributions of these scores with the distributions of scores on these same keys obtained in a Navy sample. In addition, for eac~h of
four scoring keys, item responses to the keyed items were intercorrelat-ed and factor analyzed for the Army sample, and these factors were compared to factors obtained in the Navy sample using the same items.
vii
Findings: The eight performance dimensions derived in previous Navy recruiter research appeared adequate for describing present performance requirements for Army recruiters* The performance examples generated here were easily retranslated. into the eight Navy dimensions, with no significant additional performance areas revealed inthe Army examples. However, considerable revision of the Navy scale definitions and behavioral anchors was required to make them appropriate for evaluating Army recruiter performance. The final retranslation of revised anchors suggested that the scales provide a relatively unambiguous depiction of the various performance areas inthis job, as well as the effectiveness levels within each area.
Total scores across all four keys were very similar for the Army and Navy samples. However, on three of the four keys, Army mean scores were significantly different from Navy means. Factor analyses with the Army data yielded readily interpretable personality factors, potentially indicating personality constructs important for success inArmy recruiting. Further, comp~lrisons between these Army factors and factors obtained in the Navy recruiter sample (based on the same items) showed considerable similarities.
Utilization of Findings: 1. The Army recruiter rating scales can be used infuture personnel research when criterion performance scores for individual recruiters are required. Supervisor (and peer, ifpossible) ratings of performance should provide reasonably accurate multidimensional depictions of Army recruiter performance. 2. The rating scales can also be used for recruiter assessment to target professional developmnent and growth interventions. 3. The RSB-X might be employed inthe future to help select Army recruiters.
After a validation study, the Navy keys or newly developed keys for this
predictor battery could serve to identify Army enlisted personnel likely to perform effectively on recruiting duty.
viii
DEVELOPMENT OF BEHAVIOR-BASED RATING SCALES AND ANALYSIS OF RECRUITER SELECTION BATTERY DATA FOR THE ARMY RECRUITER JOB CONTENTS ...
.
Page INTRODUCTION . . . . . . . . . . . ..
.
DEVELOPMENT OF BEHAVIOR-BASED RATING SCALES
I
. . . . . . . . . . . .........
2
Collecting Army Recruiter Performance Examples . . . . . . Refining Recruiter Rating Scales Developed in Past Research
. . . .
Integrating the Two Types of Retranslation Data into Final Rating Scales
. . . . . . . . . . . . . . . . . . . . . . . . . .
ANALYSIS OF RECRUITER SELECTION BATTERY (RSB-X) PREDICTOR DATA CONCLUSIONS
. . . . . . . . . . . . ..
. ..
...
4
4
.....
7 10
.....
* * a e * *
15
REFERENCES . . . . . . . . . . . . . . . . . . . . * a * * s e
16
LIST OF APPENDIXES APPENDIX A.
NAVY RECRUITER PERFORMANCE RATING SCALES
.
.
.
.
.
.
.
A-1
B.
CRITICAL INCIDENT WORKSHOP MATERIALS
. . .........
B-1
C.
CRITICAL INCIDENT RETRANSLATION DATA
. . . . . . . . . . .
C-1
D.
SUMMARY STATEMENT RETRANSLATION WORKSHOP INSTRUCTIONS
E.
SUMMARY STATEMENT RETRANSLATION DATA
F.
FINAL ARMY RECRUITER PERFORMANCE RATING SCALES
G.
RECRUITER SELECTION BATTERY - EXPERIMENTAL (RSB-X)
•
D-1
. . . . . . . . . . .
E-1
.
.
F-1
...... ....
G-1
ix AMW%-U%
&I
M
M=-MM
%
%ft-Lt%
'. %-
wV%'lPbU Uk %L '
LUh
Uk
U
CONTENTS
tContinue________________
Page LIST OF TABLES
Table 1. Demographics of recruiters providing critical 2. Demographics of critical incident retranslation 3. Demo~graphics of summary statement retranslation sample
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
*
.
*
*
.
.
.
*9
4. Means and standard deviations of keyed responses for Army and Navy samples
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
12
5. Factor analysis results for the Army and Navy
LIST OF FIGURES Figure 1. Example of a behavior rating scale for Navy recruiters
...................
2. Example "blank" rating scale
3
...
..
x
.
.
.
8
DEVELOPNENT OF BEHAVIOR-BASED RATING SCALES AND ANALYSIS OF .ECRUITER SELECTION ASSIGNNENT BATTERY DATA FR THE ARMY RECRUITER JOB Introduction The U.S. Army Research Institute for the Behavioral and Social Sciences (ARI) has undertaken a research effort targeted toward improving Army recruiter selection and assignment. This objective requires develop. ment of criterion measures that may be used to assess the validity of predictors, along with an examination of existing predictor measures to evaluate their potential for effectiveness in selecting recruiters. In support of these research directions, two major purposes of this roject were: (1) to provide ARI with behavior-based rating scales that can e used to generate criterion performance scores in future recruiter selection research; and (2) to initiate evaluation of the experimental Recruiter Selection Battery (RSB-X), a predictor measure employed successfully in past validation research to identify effective Navy recruiters, for possible use in predicting Army recruiter performance. Also, within the present project, we produced two literature reviews, one on the prediction of sales performance in civilian organizations (Kanfer & Borman, 1986), and the other reviewing work on the prediction of military recruiter performance (Russell & Borman, 1986). And finally, we conducted a half-day workshop for ARI and recruiting command (USAREC) personnel on administering performance rating scales for-research-only and conducting performance appraisals for purposes of feedback and counseling. This report discusses only the rating scale development effort and the analyses of predictor data.
Development of Behavior-Based Rating Scales Over the years, we have been involved in a number of studies requiring developmLnt and utilization of military rocruiter performance rating scales (e.g., Borman, Hougb, & Dunnette, 1976; Borman, Toquam, & Rosse, 1978; Borman, Rosse, & Rose, 1983). These research efforts have resulted in behavior-based rating scales for Navy enlisted, Navy officer, and Marine Corps enlisted recruiters. Each of these sets of scales contain eight rating dimensions: 1. 2. 3. 4. 5. 6. 7. 8.
Locating and Contacting Qualified Prospects Gaining and Maintaining Rapport Obtaining Information from Prospects and Making Good Person-Navy Fits Selling Skills Establishing and Maintaining Good Relationships in the Community Providing Knowledgable and Accurate Information About the Navy Administrative Skills Supporting Other Recruiters and the Command
The first four categories refer to the selling sequence, and the last four dimensions pertain to other areas important for effective recruiter performance. Within each performance category, statements summarizing the behaviors that exemplify extremely effective, effective, marginal, and ineffective performance anchor effectiveness levels on the scales. The Navy enlisted recruiter version of these scales appears in Appendix A, and an example scale for one category is provided in Figure 1. The idea within the present research effort was to use the Navy scales as a starting point in developing performancp rating scales to assess Army recruiter effectiveness. Our approach, then, in the current study was intended to yield comprehensive, relevant, and accurate behavior-based rating scales for Armyv recruiters by integrating current Army recruiter performance information with previous Navy recruiter scale deveiopment results. In effect, this strategy was two-pronged. First, independent of past research results, we employed the critical incident, or behavior analysis, methodology (Smith & Kendall, 1963) to gather behavioral examples reflecting the content of the Army recruiter performance requirements. E-.amples of effective, ineffective and mid-range performance were collected from Army recruiters to explore the possibility that replacement and/or additional dimensicns might be needed for the Army scales. Second, we conducted rating scale refinement workshops and collected retranslation data on the pnrformance examples gathered in the above workshop, again, to assess the appropriateness of the existing Navy recruiter dimensions for evaluating pei-formance of Army recruiters, and also to identify where revisions to the behavioral definitions of existing Navy dimensions might be necessary. Data obtained in these two sessions were then integrated, and a final set of Army recruiter performance rating scales was prepared.
2
L"No
IE'i logt
4= 11.
llA
U.
31 U
a
a
a1 1"a
r-
1210 I~is:is
IUL
SEN
owe
al
3.3
Collecting Army Recruiter Performance Examples We conducted two workshops concurrently to collect recruiter performance examples. Forty recruiters attending the station commander course at Fort Benjamin Harrison p-'ticipated, and demographic data for the group appear inTable 1. The recruiters were given a brief introduction to the project and to behavior-based rating scales (workshop materials appear in Appendix B). They were then instructed to write examples of effective, ineffective, and mid-range recr~uiter performance (without any guidance oni recruiter performance categories identified inprior research). Intotal, over 200 performance examples were collected. After we edited the examples and removed redundancies, 180 incidents remained. The rationale for collecting performance examples separately from discussion of previously developed rating scales was to avoid influencing recruiters' responses with a preconceived dimension scheme. We wanted to ensure that any aspects of the Army recruiter job that are different from' the Navy recruiter job and/or unique because of the points intime addressed by the two scale development efforts (1976 versus 1986) would emerge inthe critical incidents. We, therefore, analyzed the content of each performance example carefully and tried sorting the incidents into the Navy performance categories. This sorting exercise revealed no new performance categories. Another test of the adequacy of the Navy dimension structure was to ask reritr to categorize the performance examples into the eight dimension scheme. We, therefore, conducted a retranslation workshop in which we asked 23 recruiters also from the station commander course to categorize the examples (and to rate each example's level of effectiveness). Workshop participants reported that the Navy dimension system seemed adequate for reflec'Zing the content of all of the performance examples in the retranslation task. As istypically done in retranslation, the mean and standard deviation of the effectiveness rating was computed for each example, along with the percentage of recruiters categorizing each example into each dimension. These data, along with the instructions for the workshop, appear inAppendix C, and Table 2 summarizes the demographics of recruiters who provided these data. The categorizing of examples and the effectiveness level ratings for each example facilitat ed subsequent revisions of the category definitions and behavioral anchors. Refining Recruiter Rating Scales Developed inPast Research To begin the process of refining existing rating scales, we conducted an interview with two Minneapolis-based Army recruiters. The recruiters pointed out terminology differences between the Army and Navy, and we edited the rating scales inaccordance with their comments. We then continued this process intwo workshops with a total of 40 recruiters in a half-day session. (These recruiters had written critical incidents inthe morning session; the demographics provided inTable 1,therefore, also describe recruiters reviewing rating scales.) Each rating scale review workshop was a group discussion. We first provided an overview of the dimension structure and then examined each performance dimension one-byone, indetail. The major change in the rating scales involved the Navy 4
MR20M.
Table 1 Demographics of Recruiters Providing Critical Incidents
Pay Grade
Timein
E5
2
E6 E7
28 10
Total
40
Army
Male Female Total
Time as Recruiter on Production
Numbr
Less than 7 years 7 to less than 9 years 9 to less than 11 years 1i to less than 13 years 13 tP less than 15 years 15 years or more Total Sx
Number
0 6 19 9 3 __3_ 40
Less than 1 year 1 to less than 2 2 to less than 3 3 to less than 4 4 to less than 5 5 years or more Totl.
Number
White or Causasian Black/Afro-American Hispanic Native American Total
2
40
5
1 23 10 4 1 I 40
years years years years
Race
38
Number
Numbor 26 10 2 2 40
Table 2 Demographics of Critical Incident Retranslation Sample
!••P~lGrade
Number
E6 E7
9 13
ESI Total Tle& in Army
23 Number
"Less than 8 years 8 to less than 10 years 10 to less than 12 years 12 to less than 14 years 14 to less than 16 years 16 years or more Total
0 2 7 5 5 23
Time as Recruiter on Production Less than 1 year 1 to less than 2 2 to less than 3 3 to lesc than 4 4 to less than 5 5 years or more Total
Semxer Male Female Total
22 I 23
Number 2 6 4 4 0 7 23
years years ye:rs years
aumber White or Causasian Black/Afro-American Native American Other Total
6
18 3 1 1 23
category Obtaining Information from Prospects and Making Good Navy-Person Fits which evolved into Obtaining Information f'rom Prospects and Determining Their Needs and Interests. This and other recommended changes were then incorporated into the rating scales. To test the extent to which the revised behavioral anchors inthe rating categories accurately reflect Army recruite~r performance at specific effectiveness levels, we developed a modified retranslation procedure. Each of the 96 summary statements (i.e., three at each of four effectiveness levels ineach of eight performance categories--see Figure 1) was placed on a 2x3 index card and was randomly assigned a number. "Blank" rating scales without summary statements -dere prepared. Figure 2 provides an example "blank" rating scale for Category A, Locating and Contacting Qualified Prospects. We then conducted a workshop inwhich we asked recruiters to build rating scales by sorting each of the 96 summary statements into the eight performance categories and placing each statement at the appropriate effectiveness level. Workshop instructions are provided inAppendix D. Fourteen recruiters from the station commander course participated in the workshop and their demographics appear inTable 3. For each summary statement, we then computed the percentage of recruiters placing it ineach dimension and the percentage of recruiters placing it at each effectiveness level. These data appear inAppendix E. Integrating The Two Types of Retranslation Data Into Final Rating Scales Recall that the objective of our two-pronged approach was to arrive at past research and new information about the Army recruiter job. More specifically, our plan was to refine the wording of existing rating scales, identify problems with the scales (through the summary statement retranslation exercise), and use the new critical incidents to replace, reword, or otherwise transform the summary statements identified as problematic. We, therefore, examined at this point the summary statement retranslation data to identify problems with the rating scales. Analysis of these data indicated that some problems did exist within the scales, and certain changes were needed. We then identified critical incidents that were categorized and rated reliably in the critical incident retranslation procedure and used these incidents to make final refinements to the rating scales. The completed version of the scales includes eight recruiter performance categories with 12 summary statements anchoring effectiveness levels within each category. These rating scales appear in Appendix F.
7
M
~
- -
Jul
0:;F
fo
6
Fiur.
1
Exapl
"Blnk
Rain I'-
Scale .8
C~
Table 3 Demographics of Summary Statement Retranslation Sample
Nuber
Pa Gr d E6
10
E7_
Total
Tim•i•inArmy
Number
Less than 7 years 7 to less than 9 years 9 to less than 11 years 11 to less than 13 years 13 to less than 15 years 15 years or more Total ,&A Male Female Total
14
0 2 5 2 2 -3_ 14
Time as Recruiter on Production Less than 1 year 1 to less than 2 2 to less than 3 3 to less than 4 4 to less than 5 5 years or more Total
14 0 14
White or Causasian Black/Afro-American Hispanic Other Total
9
3 7 3 0 0
years years years years
Race
Number
NO
1 14 Number 7 5 1 I 14
Analysis of Recruiter Selection Battery (RSB-X) Predictor Data The Special Assignment Battery (SAB) was developed in a series of studies with the Navy Personnel Research and Development Center (NPRDC) and the Navy Recruiting Command. Personality, vocational interest, and ersonal background items were selected or written based on a behaviorased job analysis of the Navy recruiter job. The validity of scale composites of these items was evaluated in two studies (Borman, Rosse, & Abrahams, 1980; Borman, Toquam, & Rosse, 1978). In Borman, Rosse, and Abrahams (1980), four keys were developed to predict, respectively, the four performance criterion areas: selling, human relations, organizing skills and overall performance. Also in this work, personality and vocational interest items were correlated with performance on each of the four criteria, and those items found to be valid against a criterion (above a certain validity coefficient cutoff) were placed in an item pool for that criterion. Thus, eight such item pools were formed, a personality and a vocational interest item pool for each criterion (items could and often did appear in more than one pool). At this point, the valid items in each pool were factor analyzed, and the resulting factors interpreted as underlying valid constructs that describe the personality and vocational interest factors important for effectiveness as a Navy recruiter. In the present project, the Army's version of the SAB (the RSB-X) containing personality and background items from the SAB, was administered to 417 students in the Army recruiter course at Fort Benjamin Harrison. These students had either volunteered for or were selected for recruiting duty and were undergoing training to enter the Army's recruiting force. They were administered the RSB-X on a for-research-only basis, with instructions assuring them that their responses would not be used for any administrative action. This section of the report describes analyses of the RSB-X responses and certain comparisons between these responses and those of the Navy recruiters tested in our previous work. In particular, for the items in common between the Army and Navy versions of this instrument, the four persoiality keys developed in the Navy studies were used to score responses of persons in both samples. Means and standard deviations were then computed and compared for the two groups. Also, responses to the keyed items for each key in each sample were intercorrelated and factor analyzed. For each of these analyses, a principal factors analysis and varimax rotation was conducted. The most interpretable factor solutions were in each case selected, with attention also paid to the magnitude of eigenvalues and to where these values dropped off significantly. These factor solutions for the Army and Navy samples were then compared.
10
Results for the keys' means and standard deviations are presentad in Table 4. For one of the keys, overall performance, the means are very similar for the two groups. Two other keys yield significantly hl her means for the Army sample (human relations and organizing), and a lourth key, selling, has a significantly higher mean for the Navy recruiter group. Thus, the total across the three individual dimension scores and scores on the overall performance key are very similar for the two groups, but the patterns of means is different for these samples. The intention is to follow up on members of the Army sample to assess their performance in some manner, and then to evaluate the predictive validity of these Navy keys or other keys that might be developed in this future research effort. Results of the -'actor analyses appear in Table 5. Although some consideration was given to keeping the same number of fact)rs for the Army and Navy data on individual keys for easier one-to-one comparisons, this proved to be unnecessary. In every case, a four-factor solution best met the interpretability criterion mentioned previously. Regarding results, first the factors for both the Army and Navy data were in fact quite interpretable. In a couple of cases a dual theme ran through a factor, but generally, a clear, unified picture of each factor's meaning was evident. Second, findings across the two samples were reasonably similar. Although somewhat subjective, we would submit that extremely close matches, with all aspects of the factor pairs the same or very similar, occurred in three comparisons; quite close matches (with almost all components ver, similar) were evident in eight comparisons; and in the five remaining matches, some elements of the factor pair were the same or very similar. No firm conclusions can be drawn from Army-Navy comparisons of these resu ts. There are three substantive differences between the two groups. First, settings in which testing took place were quite different for the two groups. The Army personnel were not yet recruiters, although they had been selected for recruiting duty, whereas the Navy recruiters had at least six months as recruiters. A second difference is the time of testing; the Navy racruijters tore tested in 1977, nine years before the Army group. And finally, there is of .ourse the Army-Navy distinction. Thus, one or more of these differtrces could lead tV observed differences in scores across the two groups. However, as was noted, overall, the means and standard deviation: for the keys and the factor analysis findings showed considerable similarities for the two samples.
1]1
Table 4 Means and Standard Deviations of Keyed Responses for Army and Navy Samples
Army M and SD (N - 417)
Navy Mand SD (N m194)
t diff. and Significance of Difference
Selling Skills
19.69 9.26
25.07 9.23
7.50 .001
Human Relations Skills
19.45 4.23
17.75 3.94
5.21 .001
Organizing Skills
20.72 3.52
16.38 3.20
15.41 .001
Overall Performance
31.44 9.81
31.86 9.63
.50 ns.
K
12
Table 5 Factor Analysis Results for the Army and Navy Samples
Selling Skills Key
A!M 1,
Navy
keing quiet, unaggressive, &"laid back informl & easy to get along with.i
1. Being unselfish, unaggressive, & Olaid back2; pleasant, ggod natured, & easy to get along with.
2. Exhibition; enjoying being the center of attention; extroversion,
2. Outgoing & open; liking to be the center of attention & showing off; leading& influencing.
3. Conventional; successful, ambitious, & forward thinking.
3. Thorough, conscientious, A perfectionistic.
4. Unhappy, discouraged, insecure, & disgruntled about life; spontaneous impulsive. i
4. Excitable, emotional & "free wheeling"; pleasure seeking.
Human Relations Skills Key Na!!vy 1. Methodical & closed.b
_rm
1. Unhappy, disgruntled, & neurotic.a 2. Enjoying being the center of attention; lively, extroverted, & spontaneous.
2. People oriented; outgoing & open; liking to be the center of attention.
3. Daring & adventurous.
3. Spontaneous; craving excitement; daring & volatile.
4. Ambitious; hard working; driving self.
4. Hard working; demanding of self.
13
L
.
~
Uc~
q.i.
~.~J~
.
.E.~
~V.Y
.N.(-A
Table 5 (Continued)
Organizing Skills Key
Navy
_
1. Organized & planful; detail oriented.
1. Order; well organized, neat, & detail oriented; planful.
2. Dominant & forceful-; taking charge.
2. Dominant &forceful; leading & influencing others; hard working.
3. Thorough, steady, determined & persevering.
3. Thorough, steady, & stable; persevering.
4. Unhappy, feeling useless & disgruntled; unsocialized & rebellious.b
4. Exhibition; Ilking attention & excitement; unsocialized.
Overall Performance Key
1. Dominant & forceful; sociable; strong personality,
1. Outgoing & open; leading & influencing; liking to be the center of attention & showing off.
2. Being quiet, unaggressive; & "laid back"; easy to get along with.D
2. Being unselfish, un~ggresive, & "laid back"; pleasant, Vood natured, & easy to get along with.
3. Order, methodical, thorough, & conventional,
3. Thorough, stable, & conscientious; determined & methodical.
4. Spontaneous & impulsive; unhappy, discouraged, & disgruntled about life.b
4. Excitable, spontaneous, & "free wheeling"; pleasure seeking.
Factors are ordered according to the most optimal match across the Army and Navy analyses. a Scores on factors similar to these correlated negatively with recruiter performance in the previous Navy work. b Although no factors like this were identifed in the Navy study, we presume that scores on these factors would correlate negatively with recruiter performance. Note:
14
LamWQ
.
Conclusions 1. The Army recruiter rating scales can and should be used in future personnel research anytime criterion performance scores for individual recruiters are required. Peer and supervisor for-research-only ratings, made after rater training, are recommended for providing reasonably accurate multidimensional depictions of recruiter performance. The rater training program developed inARI's Project A might be utilized in any such effort (Pulakos and Borman, 1985). 2. The rating scales can also be used for recruiter assessment prior to professional development and growth interventions. For example, supervisors or professional development personnel might obtal- selfevaluations on the scales for those recruiters with relatively poor production records, and then use these ratings as a lead-in to discussing performance problems and ideas for improvement. 3. The RSB-X might be employed to help select Army recruiters. Previous research in the Navy and Marine Corps suggests that certain keys in this instrument show good validity against recruiter performance. These keys or newly developed keys derived from future validation work could serve to identify Army enlisted likely to perform effectively on recruiting duty. To minimize potential problems of faking on the RSB-X (either good gr bad), it may be best to administer the battery routinely, perhaps early second tour, to non-recruiter soldiers who might ]jIte be considered for recruiting duty.
15
References Borman, W. C., Hough, L. M., & Dunnette, M. D. (1976). Development of behavtorallv-b ed ratina scales fo" cvaluatina the oerformance of U. S. Navy recruiters (Institute Report #6). (NPRDC TR-76-31). Navy Personnel Research and Development Center. Borman, W. C., Rosse, R. L., & Abrahamas, N. M. (1980). An empirical construct validity approach to studying predictor-job performance links. Journal of Aoolied Psychology, 95, 662-671. Borman, W. C., Rosse, R. L., & Rose, S. R. (1983). Development and validation of an inventory battery o predict performance in Navy officer recruiting (Institute Report #85). Minneapolis: Personnel Decisions Research Institute. Borman, W. C., Toquam, J. L., & Rosse, R. L. (1979). An inventory battery to predict Navy and Marine Coros recruiter performance: Development and validation (NPRDC TR-79-17). Navy Personnel Research and Development Center. Kanfer, R., & Borman, W. C. (1986). Predicting salesperson performance: A review of the literature. Submitted to the Army R-earch Institute for the Behavioral and Social Sciences, Alexandria, VA. Pulakos, E. D., & Borman, W. C. (1985). Administrator's manual: Rater orientation and training for Project A concurrent validation. Minneapolis: Personnel Decisions Research Institute. Russell, T. L., & Borman, W. C. (1986). Predicting military recruiter effectiveness: A literaturý! review. Submitted to the Army Research Institute for the Behavioral and Social Sciences, Alexandria, VA. Smith, P. C., & Kendall, L. M. (1963). Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales. Journal of Applied Psychology, A7, 149-155.
16
APPENDIX A Navy Recruiter Performance Rating Scales
A-1
UU
a
CL 0 0
%W
0 0A
0m &J
10.
wo4
41
a). V-4
V4J A0J 4
to
c
A.r2H0r
..
1
$414
w
9
*ý.
4
a
aC44
0 to
0
0
a4
UU0
A)J a1 4) 0ILJ
0
0s
419*
V~~~)
~4
0ý44 =
t >0 c-A 4)0
4
a
"
(
V4 m 00)-
0. 0
0~-
u ) -
41
j0
0)
.90
ýl~~ ~ ~
&
)A
P.00) 024 :c.CL0 'A402 LM U 4 0 0 41 w W
-bo4
~
P.
r
CO2
4 a
(a M
0
w 4
0
~f4) . ~ ~~~~A %
40
026 0.0
4ww-4 a31 nc
in 0 WC
4
-& J
) .6
02-4eO 402 t LeG
:wu>
1()-43C1V
0(44
W~ 404a 0 44 W C w 02 0 w 8
wc
--
c 9 "441C0T2
4%W4
&
-4020
04-4FAt
41
r -
0
Ik
I W C 0144,0 ;0
0
0
1-d004
93 4 0 W40 41 0 -.
c
cu41 A n 0 c 40
44
4)0J
IL
r-r
A1 £I4 c t =3 u 1 o54 %bO b1 a) 96 0) >0H0 rA w -040$ 0
4) i t -4 ta
"-02'
IA
-44U 0%. ci
m2 P.
4)0
a.)
4
4)
04)4 *,44-4
W4~f ..
4) *y4 ,-4
0.
0 6P.
4
) wX U.-
w)
P.OJ W110 W-LOO
1
0
24j
0 96W AJ 6 02 0 0100i .4 4 000a4 H.~ 0 .C u4
ay4 tj"4 w44~
9-
C 0C4Q U02I 01, . . 4 40 0A
00
96
14~~~~~~~y
co
6 &j
'J60
"
410 0 bO
-e4
024.44 0 %2>44
u
"440. 4.O
C
04
ILI CL
V0
FA 1.0 Z 0 r
a)
LI4WI.
th ai 4-0 uwa'4
ci
A. AtAi,
o
c
.
to4 C s
0
CO
$4
M. 0
%6 m-
LIO
.-
0 V.4 CL' 44-
iX
41f'. r 9.O
Ic s.
Wc
4
0-
hi e Z~ 41
.
0 xh toI
00j4 '1
x.
Li0.
.
mI
0 u.
ai 4 40..4 wI 0.4 ,t.-
'A u di
CL co
.
?A0
'. Oh
00 h
00. c 00
0 E 0
C
0
m
s
L)mUI
-
04 - atoU(AL -4. 4 m. 0
U
4--A4 Q 4JL. 0
> Oh
4 0
~
60
0
4-
)
I..
04-,~I O *
'4
0 0
Co -
C4
1
"'->o4 c
00
0)- Cw 0.0 a 0. 40
0
0
0 Oh
-
4
*4)-
M00. -4.00 04
-
mU U
w~
-4
-r
L4
O
_
C to-L
w
mto004
000'co
.
4,-1
14C'LI 0 w
~
0C.40wa f
0 .'.
0
-)
4 ) 0L 0 -40 , AiI0w 0 (aLIC0 h' . L *'.40 tic r (A- c0m4%LIO. o .4 04-4 >a. -40 COO V0 . w0 W.. >h'4 h0-.4~41 mI to toC -O Q 0 .W0 .En p L I 0 >OL W-000
0c C 0'-0.
&4
00 L n 0 14d mw LE r-.
C-4
OO
.0 w
.
O4-~4 14 4-La wE 0.16 0.
%
LI. 0O =
LI'
OhL
48100 u 4-,L to Ch thc-
Ai-
0.0.'~
to0 0)
6
a
4to
00'-Ia4-u
00"
to
"tA
O
410 5L 0
'0a)CL&
1
0.
.A0 U) m4 H0 CL a to go
CC
A4.840 .. W'J.. 00wm4 w uw -40 0 U 4. 4 Wh4-... c~ -4 > w
44
"40
41
o41004j
1~4I. -4 v 04 to0 vr_-40" ch
LIM0
z
0
.0 0 C .
0a144)04.
0 " a 4) . 0) > = Oh v -40 .00-4 r M.' .40A 414 u -t 00 0
-4
P
~
a W
fr
-A
0 "1
4.I 0.04
w -4 0J4 C0LJ Ai
ý4
0E0 MCO
wO'.
w or
go
two
0. r,40.1 r l
3A
040 m
Fa u0
.00 isr_ .. AJ M4) r. 4)
04
%6.0'.4
x
A000
FA
AJOHh 0'-1 -40 F-..4m m0
4-0 :t0 0.~,0
41
6
.911 $4 :0 4-1 0'4h
a' 44 0 &I &.140. fA
A
44i
A .0)
-40 Ct
~1
U
4)
04"4-,42E.0
w
0'.a co o.I. Oh 44 :t %60C0 -4 A00. U 4to.4 440
4-. W 00j =
000 )4) w " = UJ0 0L.
Wt
50 04-
r.I a)00o4-LI
A-4
L4
LI 0
0
4.J
.0)
to .
,
0
4 0. a-J (00 c 0 0W
t)
40 W CU 00.1O
W 'o o 0 00C6( 4'1
to0 0 0 0.A0
V4
tM 0
-4 a
0
W a
Aj
ll
Do w
Go a
44
:04
w -4
fu
w
4' IA v4 03
u
aw
W
0 u
'a
C CL to Aj w 0 Go w ra m lw 0
Ai
0 z
c r
m m 0
u u to
0
u 4) Aj -4 " 'ý* 60ca FA 0 M 0 .0 41 w
w
Ln 0
0
41
4 Ai
0 0%
M
fO E 4) 1. W 0 0 c 41 0ý4 r c 14
91z 1-4
to to V c -4 C 4 cc Aj Ai cc .0 w 0
z
c 0 %4 Aj m u " tw tj >%14 U -4 -4
c
E-4
> ý4
C 0 ý4 w 44
M " U di CL fn 0 sw
w
C
>
41
a
to
to tn
r 0
to
V* -4 > ýf Ai
X
-=
to
60
-A w
Aj c to
CL (A
co
0
:44 "Wo to
0
$4
14
m
W.
(9 -4 $.4 M . cz c 0 cr ý4 U 0. 0 4) C: w 0)
> ý4 Aj
4j .0 IA Cc W U "4 4 AJ C
0 &J -4AJ cn C 4) " ?A
%b-4 0 C.)
r
E a >
0
464 0
%64 rq
W CLI
tj
14 (d
Jd u w
$4
v4 ý4 CL m Q, to
0
" ai
w > Li
-4
4 " C: -4 m " U .0 14 to ý4 Mý4
-H 60
>
tj C:
to 0 0 u ý4 P., 0 0 .0 = cc Aj
1-4
C., C M u ei C ý4 W W -4 W C:. %W r- C) cc
Aj tA Aj
M
.0 to 0.
= to
:3 "0 ý4 > 4 10 C:
r m IA
-,4
" U3 IU
o
Aj to QJ m 00 Ai
14
U
a ;ý 41 cc 14 u -ý4 1-4 C6 x
to to a 4) to
u
41
w o
0
0w
M
4.4
0 -4 to
uj
to
W > -4
A
rAj
Ln
0) =
ea r FA -4 a W
C
"o
0 w
AJ
Aj " ul ul c
U) (h 0) &j v M
C 4)
tr
0 tj
W
FA Ld )d 41 V4
AJ 4J
W ý4 >%m m = 0 3t 10 SA 4 44 to > -4
Jj m
to
(A
M " c m u
r4 ra m
-0
C to -4 ý4 14 M 0) 4) CL 0 0 "a w
0
q) 14 0
to w 0 -4 4j
Aj
0 0 0 Ai FA 41 u 0)
v 0
ul ei E " " W
E 0
C
IN
n C 00 0
:3 .0 0 m bO C C 4 "
w 60 -144 0
AJ -1 0
to
a c 1 0
to
0)
C:
r-A
w Aj 4) W 0 Aj Id (U w
m
U
4; 0
w
-4
to
A-5
0? 526-000mli
ca
&J
4 m ri 0 41
0)
m 0) to M CL :3 X C Aj
C6
E Ai
w > 4) 0
m tj 4
o
0
m ?A aj 4) 0
44 Ai = 0
to
U ý4
" 0 LA w M
0
4
&j Aj -4 r-
14 to tj W. to
u
ý4 C ij M -H
rd
Aj
c
w
V4 v X " 41
4) > ý4 Ai "4 CA CA -4 0 .9
X E L) ý4 0 ai
V
to
t! 0
C
9: to
C: ra 0 V-4 W r. ý4 14 co &A Aj CD
to X:
0
v4
zm
0 0
w
w 60 M m C 14 W Z 0 -4 *0 w (A &I r4 ý4 > 0 ý4 Aj 60 ul 1A ba C. C w p $4 ba 0 Ai 60 w 0 0 9:
C6 10 r- r-
V4 to 14 XE-1
cu V4
to
m
m
.14
g:
w
0
w
0
ul
>
to W bo 0 W C6
>
0 .0 tv
m
0; w 0) " 1: 41 44 m V-4 ý4 Aj 4 0 a cc w 4 CL u to -A u
V4 w Aj g: ý4 QJ s-4 -0 0) -H Dt
0
0
AJ 0. .0 0
ai X Do ý4 0
14 to U) OJ -4 -4 %4 4 M fa
la
17 w W
CL . CA
C
w
%
0
co
m ý4 C M Cl -.4 0 -4 = rIU ba
m Z
0
go ai c o Aj Q rd ci 4. %W -4 go V Aj 0 4) -4 AJ 41 W) 4 > w 1. W 4 :9 aj -4 0 0 0 M cj 0. ai x
u 4j 44 44
60
14 0 w V-4 .14 .0 >I CLm Cl. ca .. to *a m w r W g: 9: 0 *0 -H %4 &j 93 0 Aj CC 9: 03 4) 0 60 CJ U 41 Aj cc = 0 1 0 fA J-j
r-
cc
u
U) 0
to J--
41
>% ý4
0
-03 41 10 c to
0 0 c
4) to 41
0
z
44 lu w CL 0 93 0 w w N P. >% r-4 9 to 0 c , m
w = "
14
w &a
z ý4 44
.- 4
0
(A
a 0 W43 c m $4 0 41 A o rm a. sw c tj -F4
>
to 41
0 Aj w CL w 0
:P%
41 4)
W
ca
",4 a-
Cwa
.46
u4
Ai aC:x ,0.@"0 V j a
W
a4
a.
V4 0. z
4
oP v JA co.C0 u
Ajaq
a a0.1j
.0W
.4 ji.
>
wto
t a Qj a w w r- w w' 0wM 3 ,4c-
" r4.4
)-4. 4W.1.. W4 wa0& toV=U O.400.
J.
w:
to >1. 1. >,m r c mm0 r0 C .Z v 0. jm 3.
u0a
u
od
en-
MCz0m
Aj 31u.
oa
A
V
41m
.4
04
"4uC
.
a
.~I
*%F00 M~
-A
%4a
a
4
4
0.0.tm .. O. IO.0
V40 Q.'~-l FAIfooCMwu *00E zr o to U
C6'U
i.)GD.
-0
Am0
v ,4)VU'-
U
"W4
"M%44.W aie u
"MV4 1;
)
44~bý
*)4
0 A..-4 = X "40w1 -.
CL ao lm 41w &d au 0 a 4o V4U
IA
s-4DW.4 VW w4.I
i
41
"M4& 13
@
a
c
to~. 60
aA
4)
WAi to.4'a4 41 4 " 44-4 .1 m
-4
0
"4~~t
j
U
ul toa z e w4 w. ý4 (a~ a)o0) r aL WOO=C=C"4 u0
w. 00 0 -4
N ~o > (A
w.
V-4. w
z
C6 ai 00 U) aW lu Vo v4 .C (a 41
m 0 z
1-4
u
0%U.
z) C. "4 m 0014-
z.
U)
4. r
0
~.0.
cC-4 "4.0 m. >8
u
to 41
~
acuI
"ý49
Co0
0
V441
01 c 4 "44a~
00.4.
0.00
uuCL
c"0
w
4)
C
0 41
a
I
M
0~
=
"14"4 t
w.
4.ij 0 c 4.4 "400 w1.04 0 u a f P4 0.1m0 .4
C0 "4 to w to 0
0-4 M CM 0.0m 026 0c 0 ac). in M 3 W0 "4 0)1.'t Z4-40.
.1C -4' .
(
C
.
'4
1' a
0 -4
01 )% o "r4 4. 4.1
0 ai4
4.4
a
~~-n.
m~0
40 4
i 4) u >. c ua
r. : 07 4~ w wAi 44a) -1>1( 14 to 0-40
0o aD.C 1. A4.W 0 @C >
6M 1.' 1.4 0041i j ca 00 = r. "4 0 *"A 41.
W,">0. 44 A J
"40 Mm
C
"4.M02 cn u
1-4w
(A
WI M.U00"0 4-.0
W
"
cQ
41
C 41 01-C 0 -a 0 1' tic n
W
1.'
wD 0.
r4.'-40W
.14ý>4
UL 44 .0m
0)
aj42oaa 4 o00w4-at
-4 .
4.1 U a cu U)
0Ai 001.-alww
01.-
0to4 o bo
00C 'LC i 04ewc 4 rwt .C 3 0C r. 4 C40 1 -4 V01.m "00 0 MC'IU4 toa)u w t 4) 0)M -C4 0OW4-) >.40W a4Z 8-4 W C
t 0
>1-4
to 0
44 to" 0.0 -4 to a0-4 CL0.' -4 .10 ma..A1W'0 L4 4
@ en 0
W
w-04 5a"44b40'"4)
C 4
w
00 4.1 4.10. m M0 w' c 4 a c. 0 UA~ A.aA.. w1Ko . m2" s 4-4 VC' I 44 MC1.W-4 44 w w -H
0
i
Ct
"40' u0 0 > 1.~ "4m Id U"4Idc0 0Ca A.c-44.0 o41 4VqC -4e ti) u0 0 0. .C0 cfa 0a .4 t t0a0. VC c4 >1 0N 4ý .C 40 U3 cu4.
w m0 06 o 1. r -4 0
440
L 41 4.4
04%.4
o ua.
LA3
a~
r
1
a
x
m0
0 r.Z 41 "4 .4.1 9L a oa
.-
"4 ND0
A'-i.
x cc1. wU Ao m c ý ,-4 1 w w. U m~l 0 c CLY-4o0C4 a .ý4 ) -4.'
aso W' .CC
a w 04.
>
w
41
aV
*l
W41
ti
Vr A a) U 0 14 1 a fA
ý-4
a) V10 E0 n41 E '-=ucb=3"a4a I') 0 C 4) 4 %64.m4a n 44. w9. 44 w 4 0 w. w 0 0 a "4 w 0 4j0 CL "-I .1C
0. to4
cc~0
o 04)
0>
0o.
a 0.4
WQ %" 400
0" .
m ao4). >
14.4~
r-4 4 c
-4
u
aU
>1U 0>W04
04..1
f
4
-
naM U)0 -04-w
0
4 0
)4)
v00
0 v4 W-4 >,0CL m Owt4 14 a (000
L t2 0)Z4l4 00144 W w U E 0 -r4.ý4U 0 0 0-4 U 1-4 000.0
Q-
4
>a P-4 1. 0. >, -4ý 0U 4 $4 C
40-4 uz w4 w 010 t04 0 0 0
A-6
.* ,
~~
-
n
-
-
~l
~
.
L
~ ~A
J
RLt-.L MIA FrL lrA% %Lff
'.RP
4.R-.%L
I" N
K V ff%4 WIr W'a I'.1 N%' pl. W), M',WUX
41. IKLKR 4
x
1
Ora) 0
w0
In
614
844
6
16 0.
61U43 Iwo
4
-0 w
&j
0
41 4.4 44
C00
H Aj
-
w,.H4 w4 w1 AjlA
m-4
4114
a1
'-L 0
fj 61
A6A.61 54 41w C60.W4JJw
TO4 60
v4
H.A.'4 C -4 . 614 to . A4141 41 0)
1', 411
4
c an
Ch .4
0to
-. 41 0
C0.4
w. r.4
61 "1 a)0r4)4)
0
%
A .4
10
~6Ma -4
61to
0.
>
4 L
0- %1
14 0
44w W. w1 60
Z01 m W i
.0Aiv"6
;l
61
61166
.066116 0)146 %.4: cd A 64a -i4) U 0'H4-H 61 0 0%to 9: 04?A6A.61> 434 61) zO140.94 41 w
U) P%10~
614
%W61A
a 6
A. 41
r4u0a u U cd .44 w 0 4) 44 M14.4 4 >% C
',-
1' Q4 461 614846 0
61
-H 4 w40 1 I4 V111a4 01 0 > 141 "1
e
;0%~ -4 -I CA611 II ;.% 00 14 > 144U 01m606u0 wi U w>
Ai1.
060
61u4
to
.60
1
0.6a6 4 .0
6100O ba
.2
"q
r
a. 0
4 14-.I- 0 U0 " 'M 9 "0 > w 06 -4 v 9 4111 AJ41 Cd u 0 Cd 14--- %6 44 61a6
44
0
.04 -u
4 0
0
A C 0
k
W1
W 0 v4U4 4) 144"
.6
0
66
1.
61
0u
416C16) >% 610 0-4 0. 0 .9 41 0 414161aj yH 140 $4 . A 41006M a~0
,4A41
02616
004
e
W.444 0
w0 6146
0
61 w1
U, In
to
4)416.14 0~
X.
u16.66
1.0 616r -4 416J0 W > E w $4 Z " 0~1 91. 04-Ai u M tj0) 61 -w W3 611w.6161 H 4.4 A4 b.3 4be4 Uai= 4 . 40. 00*cu66a '.4P4 M.~0 C0= m 14 6 064.4 %W9 0 0 -. 408 .e
~0
6161 '. 06
44
"I -H144161
Ai 6a1w61 to 041041 0 0 0 1 61
1 w 61 Ai 0 04 to41 61611. w a> 4 w r1. 41 c1 041 06 06m1 . 1 U0 410) -4 4.4 I -r46 C AJ "161006 01461 U 0 m o a-4 41 g c . .0 0L.61 0 06 61 6161.-4.6=.-4 '46>6
.46 V0.41 41 o. 0 61%15r.-A -r4 611041V4D
t01W 616161 la Irl C
FA1w.610u61 6 C: 610410..061 4 0 % *
61616 w $4 C16 e11.... 0. -H 6116046 L P 41 la 4. w w 4 1414 . o 61 4-1 0 W1 4J Ir4 u 4-H 4) 0) 61
"a
6a H'.
b0.-4 to V . i 14)
Aiw -'106 '-4 .0 w Ut 414 .=4 A6 - u A. -41 V4 to 04 11 44 H0.l, " 0 > 61 0 w416 Aj a .064 -0 61 461
V w> r. 0 V ca443644'0
t -H u
'.641IA6.6
0.40 t ca c .0 "0001 w V > 0.610 UlIv400u
A-7
A1 C:4 0)6 3 m w 01$4 40 a)A. V4. -H 414Q : 0. -4 0:ýqU41 0616161 V4 >1 u 4 > 4.46b)6 1 6-ri A 061 3 0 c61WC:
0
A 0j 0
-06 0.-4 4461 CC.061441 D 4 C t A610 uW dMU0Cm a4m60 u :3*a 614 c .. 4 0
04
~
" m 0 0 .0 44
4
A 96
1o6
460.40. to to
w
4
4 14
06 to1 Wy0,to
41Wcc0
W 1.4
H-
61 6
1% 6
*
40 V.~6
1 344 w
4
p"r-W3,6 404
1~ 4611~-
0 0;
0b
01wi4
61~
a1
0
03
4 -A, 91460. 0v44h11 4
to ,0
41
0 P%0
14
.041
40& .4
w
16
6161w0 ) 3C C06
w,-4 0 to 61".6 c.6 ci 61006to161m C'~4O VA %A
0 " :
4J 1,.4 4 1W 41 414c 61061W 4 60 w.- 46 6 0 M0 4) ) 0, U IA1441k44141. 0
14t
-go
U
e.
Go
IO G C.~
2.4U~~1.
ba
.4
..
..
4000
44)
C
4)
104A
'A
304
4u
2IU
*a14 eos
u
30
.4046)
00 1 .rqq~I*
4
*
06
U..
-H
In*0
92 9:41 : ,4
0-
i40
toC
ý4t
U z
Id r4 01 VA 0
4
JA A
0c
r4OCC Do
.
ie C: 4)14 C
0
.
U
4 04) 4
0
4)
&J 0 u w 41 4)1 :j 0 (T% 44 0..IU w4 AO0 S 0. 0 Aj 00 tomý .44 z v -0 4 0 Jt wC1 9C'0 4 =) 4 Aý4 b aU 2.4 P "0 '4
w
)C'4 ý u14
0
C2
Uj 4))W0
M
t0
0
4
0' 41
c
0 4-
.
02.
C: ;).4 a~ -0 0) to V AIUU i4) C O 04 z w) 0.111 4)4)0 Q 64 COo.
0 si
0
~4 w 44
41 04 0) > -H
0
C0. ou$
& U
t4 10;;440 PCC w VA %&4 a40w.m =414 IA W) 44 a. 0 w) u 0.U1 4 GO W)
-
U 4)
E0.I
44 Ui 1-
00 r4 -Li
I4
r4 0r-4 U
Q
to
0
400
totU 4C -
0)k
00
u
t)0 0..0 c
O
vito4)
.Aif
4) "4 U4u
4 A m4) to.~X
U
Q.-
P40. >4 r-' -V44
CC0,4
0 ) 4.
0 v A140 w*
.14 0 M w)J (A2..
1 )*4
w
;p 41 (a.44)
) 00 CO04 244N U4 -0 40 W -v4 ACCA' 0i
0
WI.
'4) 441.4
4w41 2.)4 0).
:
0W-
0
W40. ý4A 00 C: 4J))14 4 9:6 0U 0U 0
m
,4 0
04.
Ai00 96.= DoCO
Q) v4 01 d 4
w 4)
0P = m4 t I . :K w -,4
'0444 -)4 AjFA u Ua - .4 U3 '.4
C. A
c
C
4
u4)01 cc t CU 0 0s CL
0. 0.M.0 0
C
14
20
w40,4 .0
I.4
s
2.0
w
o 0 IA
CCCO.V4
-0
U'
j4
V
40
") V~
C0 u 01 0 a)-m W0W) 41 O
w
A 0 "0
I04 W .Aj4 4W4 TUGl TWA pC
r-4U
OA
0 0
-8 V-IUWw
9:
24
0U 44 4) G
0
UW
0044
ww0X A1
mU
C
C,- 4J u'44 to w
0
A
4)
I
0-
0
U rC 0A
1
aU 4
CU
U
R3
44
44
)
0-
4)4,4 N 0-4
a
A
w
0 U 4).4
0-
0L~J
w00-
61
0U
0
14 A*
110 LI.4)
~4q4 Dt
01
a
r4
,U1
(UA ,44 U
0
41
UCU
4 vO 2.4800)
o
Ad -A-A
0
U
k
C41 0
wVU&J% 3 mU
C10
LOJr o0. a4'0-
9:e
04 "4 & 0" 1
0L 1.4..4
C
01 M
w. wCC4o
41
4'
4%
4 0) a.4
U) 024
COt
.-.
1
%W2'b
A-8
U 4)0
0) 14.4 di A a
00u
41 4j -r140
10,4to4 .a4 A
4a1
-4
4-414.-
A 0
04)1 "4 4) IA 1U
4
0W
S
A
ul 8'6 aO
a1 P
P4 Z 0
1
aI
114
cuI.W
a
1"4F
w4 V44iF3
64140.
u
4
-H
m 0
C6
NOq
) 0 C 0)
"U
w
f
0
0
C6 f
-
t
I
t'
41
41=4w
V40 418 0.aN C6
0
1t
~
ta
%W
.
a
IIv
'.4 06
"4
t M40tgh
%C41~ o4 4ao-H
to
u
r
a.
r.
4
U 0
6u 100
". w0 1 .,4.a8. r
.
-t
41
""
~FA
18. 4w
c4
-cG
04h04 41
U
aw
,-
:
0u1 41
0 th41t> V 102 -4 .-
u-
U 0
4 000.0. .
P~th0
4'a fe00
44
~
.v4
FA4 14~h~l. toZ.0z *a.d 0 V vU a105,4 b4 1
-
aS .5 ".~0 0 a;0. OWU toa 0C416. C10 U ton
0
""1 14w
4jýu -4
m r o.6 u tuoca. = Aoc&
1.40
tq
to
~.0~
013 tothwh
= "4
-U1-4a 1at
a
0
U 4 0 ")018 w41u
05
th >%
-V J
61.
:0 a
Mw 8
wIw
003614w VA8
Ga
00
41
41j
.-4
0
wCU..
th
thm4..
AG
O-4
41t
Ata
u
'44
14 *14
a4
Pa
a%~j
1 4I 00
P4
1
4)
V4
v4~ 1-4
4
0 140 0 0-A m0w 110) 1441t
0o 0. 0.
ththglw 416t5 0.i.4
Ac~~ t0
a
W0410 o 0p
11f441I " Oas. gm600 6a aU054N4 $4
@20 C. c6i )co
A- 9
0
-4
S 41 0-4
44P4 $00 "0
0. 0(u
0 0: eth
60 u4 c 0414 86A " 14,?A
W8
"
4
VA a
IE
w
A a. '4W 31 &av
'40a
h
44
Qhi'0I 0A
fq 41e' 60i ai SW4'1.'
o
.w
1.
.. *'
0.60
Ca
'4
~ w
"60 Iwo u
4j'h
4 r-4
WsA 0 Ai t!. a
..
4-1C
0'jV4'
wi a*'
v'i 4
4
i
4
0441
z
ai54 v
004.e"l0WOaIOA
a a
he
a
.1
ua-4
c
IV
h .U
U.y
14AJ11 0I 0 4
i
00
50 ' CT
.~.CX hi *0
uUUA0 i1
=0
hi0
*.1
=0S
CO0Ua
'4' u
u wuw 0hi
wh '4
C'4596 le
~0
go0
44 4 'e
&I-.
45
.140
o
IdiO-
.8.0.
0
-v 4.J
45 '4 00a W-410
GU
0 4 N45 'e
J4 '4
c~
aU
*j
41Oa
We40@~
A
VJ4
:
@41
-4
141
u4 a U0041 0 '4 640*0'
-T4 V4hwia
'
=C.V4
,
g: 0
'40'1a@
'00 r
45
to
4 0
*
.
w@345r50M c ,40C.'4 w i0gohiS
u-
004.'
C~'h
1Ii
I.
=3i3i 64h $. W1S
0J
UCO
.
hi
0 @ -h4 9114W1;04 :R M .
-4
P.(0w
0
'4 C
%6.4.
.14)
U
604Ll4-.
h0) 0
a
>
" "= CLi(hihihi m.
5 CL ?A A X 0WhW'4''-,0.V CJ 4, o k~ @
A-10
-
CIO
0@3
"a0.
k~i.@&A 04 (A
0 U4
CJU 41
t@=3 IA
t
'4O 1. t
0
Ai
A 4;
'4 w F U1M
U
a
'
U
440
-Av i4 @3'a 4
005144
'44 -00
L
U X j
h
r044
6
to "* V4 r w
' c
-W55
'aA
=
0
)(
4'045a1e4
0.
1-. d
U
Ousaq U @ 0 0U
'4o1.'q
SC
u
ih hu 4 eJ@ h a. a4 3
*'4
05I~44 -v4 -4i
w
ab wC.. 0 -
u
0.1. 41@
u
0q
0L
60U0h N
c4 mi 4
u
hi4
vO
45a >1 Ow cc w $4
0
.0
ba
t%
UC1. "A u 0.
U35
m.
whib
*u
w
'4 ueW
T-4 iO4 It w .
a
c:
04)04
'
3@3@wa3).F0@34
4 .
he
L
a 0.41
6
Id a
.U .. 9 4 &a
*C 41
S.0
0*UW
"4~~
~
U
c4L
t asA aj
06 Li411* C
r -4C
""A.4 0 .466 5.. 0-4 A4
u
"40
U
1
111C u 0
a.
rn 0 6 4 0
*
SC
acu..wL 40 41 4U0L
41 c
0- A
41 44
.. 4
(A
w4 *a
.h 41
"
060 w
0A-1
U
APPENDIX B Critical Incident Workshop Materials
B-1
RECRUITER EFFECTIVENESS WORKSHOP 26 ~July 1986
Sponsor:
U.S. Army Research Institute Alexandria, VA
Contractor:
Personnel Decisions Research Institute Minneapolis, MN
B-3
Recruiter Effectiveness Workshop 26 July 1986 Agenda 9:00 -
9:15
Briefing on the Project
9:15 - 9:30
How to Write Performance Examples
9:30 - 10:30
Write Performance Examples
10:30 - 10:45
Break
10:45 - 11:45
Write Performance Examples
11:45 - 12:30
Lunch
12:30 -
1:00
Introduction to Recruiter Rating Scales
1:00 -
2:15
Review Recruiter Rating Scales
2:15 -
2:30
Break
2:30 -
3:15
Finish Reviewing Recruiter Rating Scales
3:15 -
4:30
Appraisal of Two Approaches to Presentation of CAST Results
B-4
Overview The U.S. Army Research Institute is sponsoring a project targeted toward developing state-of-the-art performance rating scales for the Army recruiter job. Today we are goinig to work together to design rating scales that are referred to as behavior-based rating scales. Behavior-based rating scales ar's a type of rating form that offers the opportunity for relatively objective assessments of performanci. To develop these rating scales we will accomplish two major steps today. First, we will ask you to provide actual examples of ineffective, effective, and average recruiter performance. (Ina couple of weeks we will be asking recruiters to rate the effectiveness level of your examples.) Second, we will ask you to review and refine behavior-based rating scales we developed for recruiters in other services. Before introducing the behavior-based rating scale concept, let's look at some other kinds of rating forms. Examples la. and lb. on the following pages are called trait ratings because the ratings refer to personal characteristics or traits of the ratees. These ratings are highly subjective. For instance, on lb. each rater has his or her own definition of what leadership is and what it means to be "good" vs. "poor" in leadership. The rating form in example 2 is numerical but, as with the trait rating scales, the rater is not given any information as to what it means to perform at a poor, average, or axcellent level. When rating scales are not well-defined, raters will often disagree on the performance level of the individual. A behavior-based rating scale is provided in example 3. Notice that the performance category is defined by the behaviors that lead to effective performance. Likewise, ineffective, effective, and average performance is defined in terms of specific behaviors rather than traits.
B-5
ALTERNATIVE PERFORMANCE RATING SCALE FORMATS la.
Examples of trait ratings: "Excerpts from a U.S. Army document regarding performance appraisal Lower Senaca Town August 15, 1913 Sir: I forward a list of the officers of the 27th Regt. of Infty. arranged aggreably to rank. Annexed thereto you will find all the observations I deem necessary to make them. Respectfully, I am, Sir, Yo. Cot. Servt. Lewis Cass Brig. Gen. 27th Infantry Regiment Alex Denniston - Liet. Col., Comdg.
- a good natured man.
Clarkson Crolins - First Major
- a good man,
Captain Shotwell
- a man of whom all unite in speaking ill, a knave despised by all.
"Allen Reynolds
but no officer.
- an officer of capacity, but imprudent and a man of most violent passions.
First Lieut. Wm Perrin " " Danl. Scott Jas. I. Ryan "" "Robt. McElwrath
- low vulgar men, with exception of Perrin, Irish and from the meanest walks of life-possessing nothing of the character of officers or gentlemen. - willing enough-has much to learn-with small capacity.
Robt. P. Ross 2nd Lieut. Nicholas G. Carner
- a good officer but drinks and disgraces himself and the services.
SOURCE: The First Recorded Efficiency Report in the Files of the War Department, August 15, 1813.
B-6
lb.
Another example of trait ratings:
Leadership
I
"
) '
"
I Good
Poor
Aggressiveness
High
Low
Sel f-Confidence
Low
High
B-7
2.
Examples of numericallIy anchored ratings:
a.
Quality of Work 1
b.
Quality of Work:
5
Judge the amount of scrap, consider generalI care and
poor,
04
3
2
1-6;
accuracy of work.
average, 7-18;
good,
19-25
C. 12
3
Quality.
POOR
4
5
6
78 BELOW AVERAGE
9
10.11
12
13
14 15
ABOVE AVERAGE
B-8
16 17 18
19 20121
AVERAGE
22 23 25
EXCELLENT
3.
Examples of behavior-based rating scales: A.
TIINICAL KNOWiLEDW£1SKILL
Displeying job and soldiering knzovladge/kill. LOU
S.2
3
4
5
HIGH
6
7
Dsplaeye little hoowledge about beo to accomplish lporteat weok mad tasks, and.*5*. folio to patternageenes pasmn riy due te lack of Job/seldlegi s kills.
Displays Seed &nowlWds@ In seat aspect* of the job end Im soidlering. and has sufficient skiill to get the majorit7y of assignments done properly.
Dieplaye coneideoeble kneeledge about hew to accomplish ioportant week and tasks. and has the &kills to consistently get assignment* dmen property.
Is unable to poeera effectively without supervisions perlormo Jobs/teekslassiog mente eer
Pewforms effectively with little supervielon an tile ltoively easy soolslmenta
liquoV
but needs a supervisor,*
performs alfectively with very little mupervimiomn cecoeplighes taoke and aesignmentt eiliciantly
itheaothereirst termed..
often much were quickly them
evidence for more difficult taski aecompeliaheo Jobs io *bout te Gsame amount at time af othler first term Sole.
Hanes of the solditsg ZOu are ratine:
how effective
1. 2. 3. 4.
B.
LOW 1
2
I
Rsengee to volunteer a sse oiltsntn at p uat In let e xtr a hours and effort; may even leact with hoItile attitude Zhen akaed to volunteer or voik lona hours.
Cives wp esely when faced with ubstacle , adversity. or diecoamoct.
3. 4
5.
categoryt
0
0
0
1
0
2
3
4
5
6
7
0
0
0
0
0
0
0
0
1
2
3
4
5
6
7
0
0
0
0
0
0
0
1
2
3
4
5
6
7
0
0
0
0
0
0
1
0
2
3
4
5
6
7
0
0
3
0
0
4
0
0 2
5
6
7-
iliTiATiVC/EFFORT
and extra effort
on
the Job/Loaelon/aaeignment.
3
5.
I
I
HIGH 7
6
Volusnteere for sbm assign=onte 0114Puts i.1 extra effort when It's very importamut to do *..
Volunteers enthuiast ttially. take s initiative Promptly and eftctiveiy when opportunitis :Ciao. and voluntarily works long. extra hours to Complete incitements. even without being asekd.
iiamigsIn there with d atetrr iiw tenn witun it's
Ite-"8* to give In to edvereity and pu shes on with considerable Itamina to overcome all obstacles until tile assignmunt is
I-nJrtant
tech1 y
to
overcote
obviatcies un tile Job in time hitile.6tt.
Felts of the soldiers you are Vaint:
this performance
i.
0
1
Shoving Initiative
Is each soldier In
teroa
0
0
5.
the aversge lirst
How effective
is
completed.
each soldier
01,
02
0j
0
0
11.
2
in
this performance cexteorl?
0A
05
0
0
0
0
0
3
SA
5
6
7
06
-7
0
o
o
0
0
0
1
2
3
A
5
b
7
0
0
0
0
1
0
2
0
4
5
0
6
7
0
0
0
0
0
o 1 B
B-9
2
4•
0
4
0
3.
Examples of behavior-based rating scales (continued): FOLLOWING REGULATIONS AND ORDERS
C.
Adhering to regulations. orders, and SOP and displaying LOW
12
3
respect for authority.
4
S
HIGH
6
7
II Tends to lgnore, af otherwise ,ilI to follow Armylunit rules, regulations, or orders; any display la..tof respect tovacd superiors.
Fot;owe Acey/unit rules and regulations alosut witihout fail, especially these that are important for asasty and aecurityl always obeys orders.
I% very careful is fellow the spirit as well as the letter of Army/ualnt rule and reguletionsl not only obeys orders but does so quickly and with enthusiasm.
ta often late and/or falls to show up for duty, formations, the job. etc.l may leave assigned poet. Is serious violatlon of regulations/orders.
Is always on time for duty, formatlones, ate., hen it's important to be prompti If late. hoe a Rood raeuomi never lessesa aligned poet or faila asleep on duty.
Always reports promptly for duty, formations rcppeintments. ete.l ean always be counted en to be at appointed ores on times ramains alert at aesigned peat even when It Is Inconvenient or difficult to do ss.
lames of the soldiers You are ratint:
low effectlve
1. 2.
3.
0
0
0
0
1
2
3
&
5
6
7
0
0
0
5
6
7
0
0
0
0
1
2
3
4
o
0
0
0
0
0
0
0
0
0
0
0
0
55
6
7
0
0
0
0
0
0
0
4
5
6
7
2
2
D.
wLO 1
A
3 3 3
" 1II
INTEGRITY
3
4
5III
sometimes wake@ up excused to avoid duty acid denies wtsponal-
Owns 1-p toacidJob-relatod mNltukei provides
I, llity for job-relaetd misaesoo
COAplote
he/ *he sakes
7
6
3
Displaying honesty acd lntegrity In job-related and In personal matters.
2
0
0
1
5.
catetory?
0
1
4.
Ia each soldier in this perforatcor
0
mid accurate
NIGHt
6
7
.
Takeo *site to protect stcutity at steps military equipmentthe Infer-.
and personal possessions* even
Resists t4..ptatione to Werfou uvailIJhle military *quillotitatoj |urgtjitol possusisinnlo withelit perasiSiW.; p1`-p4iy tatiorose found property to rightful owsioces
Con always be trusted to be n •uhu.er~hfrsd honest even alien own Persnali Interest& miglit ba jeopirdiged.
Dollari n ece ssary for ad "~n-
whennot d irected to do so.
lear~tire Je¢Jsirne. Beeae iltary
equipmient of
personal possession* without gnrsesiono
fames of the soldiers you art rating:
3 A. 3,
How effective is each soldier in this perfo•rance category?
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
2
3
4
5
6
7
0
0
0
0
2
0
0
3
0
4
6
7
0
0
0
0
0
0
0
1
2
3
4
5
6
7
B-10
Behavior-based rating scales have at least five advantages over other types of scales. 1. Scales constructed to reflect performance requirements regarded as important by those knowledgeable about the job. 2. Scales define in concrete terms the relevant and important performance requirements. 3. Job experts (i.e., recruiters) agree on the effectiveness levels of scaled job behaviors used as performance effectiveness "anchors." 4. Rating task with these scales emphasizes objective observation rather than subjective evaluation. 5. In sum, raters can compare the observed performance of a soldier to behavioral benchmarks or standards of effectiveness, resulting inmore objective performance judgments. We have developed behavior-based rating scales for Navy recruiters and for Marine Corps recruiters. Our first step indeveloping such scales for evaluating Army recruiter performance will be to ask you to write examples of effective, average, and ineffective recruiter performance.
How To Write Recruiter Effectiveness Examples To write a performance example or incident, try to remember what the recruiter actually did or failed to do that made him or her effective or ineffective in a situation. Your can be examples of extremely effective, ineffective, or even average performance. The important thing isthat the incident isdescribed specifically as ithappened. When writing an incident, describe only what you saw or what the person did, not what you inferred from the action. For example, rather than writing that the recruiter "displayed loyalty," you should describe what the recruiter did to make you believe he or she was loyal. As examplzs, the recruiter "shared information on important contacts within the zone" or "helped another recruiter with his/her paperwork." Both of these examples might be described as displays of loyalty; they are things the recruiter did to make the writer believe he or she was loyal. Thus, we are asking you to describe specific behaviors or actions, not traits or personal characteristics. The features of a good incident are: 1. it concerns the actions of a recruiter; 2. ittells precisely what the recruiter did (or did not do) that made you feel he or she was effective or ineffective; 3. it describes the background of the incident clearly;
4. it states consequences of what the recruiter did; and 5. it is concise in that itis short, to the point, and does not go to great lengths specifying unimportant or irrelevant details. Some hypýw.c.-tical performance examples appear on the following pages.
B-12
PERFORMANCE INCIDENT FORM
I.
What ware the circumstances leading up to the
_________
incident? Too brief, fails
Discussing Army programs with a prospect.
to describe the context in which the incident occurred.
2. What did the individual do that made you feel__________
he or she was a good, average, or poor performer?
Labels the behavior, but fails
to specify what
This recruiter was inconsiderate and inattentive.
exactly the recruiter did that was inconsiderate or inattentive.
3. In what job performance category would you say this incident falls? Obtaining Information from Prospects
4. Circle the number below that best reflects the correct effectiveness level for this example. 1 extremely ineffective
2
:ý ineffective
4
5 about average Example la
B-13
6
7 effective
89 extremely effective
PERFORMANCE INCIDENT FORM
f~i~
1. What were the circumstances leading up to the This recruiter was discussing Army programs with
aprospect who mentioned that he had three yearsI of metal shop inhigh school and enjoyed it.
2. What did the individual do that made you feel he or she was a good, average, or poor performer?
This recruiter Ignored what the prospect was saying and described Army.programs that applicant wasn't interested inand wasn't qualified for indetail. As a result the prospect lost interest inthe Army.
3. In what job performance category would you say this incident falls?
Obtaining Information from Prospects
4. Circle the number below that best reflects the correct effectiveness level for this example. 1 extreme'ly ineffective
2
304 ineffective
5 about average Example lb
B-14
6
7 effective
8
9 extremely effective
PERFORMANCE INCIDENT FORM
1. What were the circumstances leading up to the incident? Insufficient information.
Problems locating qualified recruits.
2. What did the individual do that made you feel__________
he or she was a good, average, or poor performer?
Rather than de-
scribing Wbdi
Recruiter was imaginative and creative in locating prospects.
this recruiter did that was "creative," the behaviors are simply labeled imaginative and "creative."
3. Inwhat job performance category would you say this incident falls? Contacting Prospects
4. Circle the number below that best reflects the correct effectivenass level for this example. 1 extremely ineffective
2
3 ineffective
4
5 about average Example 2a
B-15
6
89 effective
extremely effective
PERFORMANCE INCIDENT FORM
Problems: 1. What were the circumstances leading up to the incident? A high school in this recruiter's area did not give out lists of students.
2. What did the individual do that made you feel he or she was a good, average, or poor performer? This recruiter asked all Delayed Entry Program recruits to bring friends to evening movies about the Army and, thereby, bi'ilt up his prospect file.
3. Inwhat job performance category would you say this incident falls? Contacting Prospects
4. Circle the number below that best reflects the correct effectiveness level for this example. 1 extremely ineffective
2
3
4
ineffective
5 about average
6
!)
89
effective
extremely effective
Example 2b
B-16 -~
~~~~~~~~
UBj
I-
itit
BkK
%AB"AA
rid
AbU Aý~kfMM~
PERFORMANCE INCIDENT FOPP
1.
What were the circumstances leading up to the
incident?
Too much detail, irrelevant information.
The 15th of July, 1985, was the 100th anniversary for the founding of a small town in The city council decided
northern Nebraska.
to plan an all day celebration with a paradom, an arts show, and many community festivities. The day of the celebration was very hot. 2.
What dld the individual do that made you feel he or she was a good, average, or poor performer?
Double-barreled incident, since two behaviors are described. Effectiveness rating is confusing because it is unclear which behavior is being rated. Consequences of the behaviors are not indicated.
This recruiterwas behind in doing paperwork end volunteered to help organize the celebration. The recruiter attended a number of meetIngs, helped set up several displays, and manned the Army display during the celebration,
3.
In what job performance category would you say this incident falls? Maintaining Good Relationships
4.
Circle the number below that best reflects the correct effectiveness level for this example. 1
extremely ineffective
2
3 ineffective
4
5 about average Example 3a
B-17
6
7 effective
8
9 extremely effective
PERFORMANCE INCIDENT FORM
1. What were the circumstances leading up to the incident? A small town inthis recruiter's area was planning an all day celebration.
2. What did the individual do that made you feel he or she was a good, average, or poor performer?
The recruiter volunteered to help organize the celebration. The recruiter attended a number of Dleetinf s, helped set up several displays, and manned the Army display during the day of the celebration. As a result, the recruiter 3ained visibility in the community. 3. In what job performance category would you say this incident falls?
Maintaining Good Relationships
4. Circle the number below that best reflects the
correct effectiveness level for this example.I 1 extremely ineffective
2
3 ineffective
4
5 about average Example 3a
B-18
6
89 effective
extremely effective
APPENDIX C Critical Incident Retranslation Data
C-1
Recruiter Effectiveness Workshop 16 August 1986
Data provided next to each performance example indicate the percent of participants who placed the example into a specific performance category and the mean and standard deviation of the effectiveness rating on a scale of 1 (ineffective) to 10 (extremely effective).
C-3
L'IjKA
AALL'K
A
R
A
K
R
X'VXX
A
A
AhWWL
M
L~k
~~~~~~~~~~~~~~ A "I P-
BACKGROUND INFORMATI ON
1. NAME:___________________________
___
First
Last
2.DT: 3.
Day
Month
RECRUITING STATION:
'LOCATIONOF
Year _______________________
Brigade
Black/Afro-American
5. RACE:
Male
Battalion
State
City
4. SEX:
M.I.
Native American/American Indian
Female
Hispanic (Mexican, Puerto Rican, Cuban, or other Spanish Origin 6. MOS: or Caucasian
-White
Other, Please specify:_______
7. PAY GRADE: Enlisted
E5
__
__E2
E7
E4
El
__E8
E6
-E9
8. TIME IN THE ARMY:
______years
E3
-
9. TIME AS RECRUITER ON PRODUCTION:
______months
_
10. TIME IN STA71ON COMMIANDER POSITION:
____years
______years
C-4
_
____months
______months
Overview The U. S. Army Research' Institute is sponsoring a project targeted
toward developing state-of-the-art performance rating, scales for the Army recruiter Job. Today we are going to work together to design *
rating scales that are referred to as behavior-based rating scales. Behavior-based rating scales are a type of rating form that offers the opportunity for relatively objective assessments of performance. You have been asked to participate in today's workshop because having worked as a recruiter and cbserved other recruiters on the job you are the best source of information about 'recruiter performance. The information you provide today, in conjunction with that of the other participants, will be used to evaluate, refine, and finalize a set of behavior-based Army recruiter performance rating scales. Your pLrticipation in today's workshop is,therefore, very important. Over the last eight years or so we have conducted a number of interviews/workshops with recruiters in the Navy and Marine Corps, as well as the Army, to develop behavior-based scales for rating recruiter performance. This work has culminated in a set of eight categories of recruiter performance. These are defined on the following pages. It is very important that you be thoroughly familiar with each category, so please take a few minutes now to read through each definition.
C-5
Categorl2s of Recruiter Performance A.
Locating and Contacting Qualified Prospects "Prospecting" effectively; contacting large numbers of persons likely to enlist i d the Army; skillfully iising the telephone, referrals, DEPs, advertising ideas, special events, etc., to contact and get the attention of civilians eligible for A~rmy service; knowing where and when to prospect; ability to persist in prospecting and following up on leads even under considerable adversity; getting prospects into the office.
B.
Gainin~g and Maintaining napport Building rapport with prospects in the office; gaining the trust and respect of prospects; adjustirg to applicant's styles and acting appropriately with different types of applicants.
C.
Obtaining Information from Prospects and Determining Their Needs and Interests Listening skills; making accurate Judgments about prospects' needs, goals, and desires, etc., based on good interviewing skills; effectively obtaining information about prospects from other sources (e.g., high school principal) to assess their qualificationis and needs; accurately assessing prospects' eligibility for various programs.
D.
Sales Skills Skillfully persuading prospects to join the Army; using Army benefits and opportunities effectively to sell the Army; closing skills; adapting selling techniques appropriately to different prospects; effectively handling objections to joining the Army.
E.
Establishing and Maintaining Good Relationships in the Community Contacting and working effectively with high school counselors and principals, civic leaders, and others capable of helping recruiters to enlist prospects; building a good reputation for the Army by developing positive relationships with persons in the community; establishing and maintaining good relationships with parents and family of prospccts; presenting a good Army image in the community.
F.
Providing Knowledgeable and Accurate Information About the Army Displaying considerable knowledge about Army programs, schools, educational opportunities, etc.; providing accurate information about Army life; being up-to-date on changes in Recruiting Regulations and on other directives pertaining to program changes, eligibility for programs, etc.; skillfully relaying information about BCT so that prospect/recruit is informed about what to expect but is not discouraged from joining the Army; answering questions about the Army in a competent manner.
C-6
G.
Organizing Skills Planning ahead; organizing time efficiently; completing paperwork accurately and on time; keeping track of appQintments; not wasting time.
H.
Supporting Other Recruiters and USAREC Coordinating activities with other recruiters to maximize the productivity of the station and company; using own skills and/or time to support other Army recruiters when appropriate; pitching in to support orders and directives from higher levels; providing constructive feedback to other Army recruiters concerning their skills, style, etc.; providing helpful tips to new recruiters.
Notice that the first four categories deal with the recruiting sequence - from prospecting to closing the sale. The remaining four categories address other aspects of the recruiter job. Each of these eight categories is defined by the behaviors that make for effective performance as a recruiter. In other words, the categories do not refer to vague personality characteristics or traits like "initiative." Instead, these scales were derived through actual examples of recruiter performance, given to us by recruiters. A few weeks ago (26 June), we asked Army recruiters like yourselves to provide us with actual examples, or specific incidents, of recruiter performance. In total, they provided us with 180 examples. Some examples reflect Extremely Effective Performance, others reflect Effective, Marginal, or Ineffective Performance. Today, we are asking you to make two judgments about each of these examples.
C-7
Figure 1. Recruiter Effectiveness Scale
Extremely Effective Performance
10
91
7
Effective Performance
5
Marginal Performance
4
3
2 Ineffective Performance
C-8
AI Decide which of the eight recruiter performance categories a particular performance example belongs in. Once you have decided on the performance category, record the letter (from A to H) in the space to the left of the example labeled "Category."
Judgment#2 Then, think about how effective the recruiters behavior in the example was. Using the nine-point rating scale in Figure 1, decided which number (1 to 9) indicates how effective the recruiter in the example was. Then, in the space labeled "Effectiveness" to the left. of the exam~ple, record-the appropriate number. An example of how one recruiter rated a performance example appears below. CATEGORY (A-H)(19 A7
EFFECTIVENESS 200. A high school in this recruiter's area did not give out lists of students. This recrufter asked all Delayed Entry Program'recruits to bring friends to evening movies about the Army and, thereby, built up his prospect file.
The recruiter rating this example felt that it belonged in Category A, Locating and Contacting Qualified Prospects and, thus, wrote A under "Category." The rater felt that this example reflected Effective Performance and, thus, wrote a 7 under "Effectiveness." Before you get started pleaseý take a few moments to review the eight categories of recruiter performance and read through seven or eight of the examples so that you get a feel for the things included.
C-9
Performance Examoles CATEGORY (8-H)%
'
EFFECTIVENESS (l-low to g-h!•.
iL
1. This recruiter set up and conducted a high school ASVAB, testing 100% of the high school seniors and Juniors at a school that had never had an ASVAB before and, therefore, obtained an ASVAB list to use in prospecting.
L2.
141 A/;t
When this recruiter was interviewing an applicant whose father was a VietNam vet with a negative attitude toward the Army, this recruiter used his own experiences to convince the father to sign for the physical. a.l
IW
3. Because this recruiter talks so much, it takes
10 to 15 minutes to set up an appointment with an applicant. Many applicants do not show up for their appointments as a result.
,4.
00
'/a,
Before going across town to pick up some
police checks, this recruiter checked with the other recruiters to see if they needed anything from the same area. This saved the other recruiters time.
I,#•
5.
Though the assigned areas were difficult, this recruiter consistently brought in enlistments by using techniques for talking up to or getting down to anyone's level, and relating to them.
I,16
6.
This recruiter used pre-plan calling, school newspapers, and effective listening techniques to make prospect contacts. The recruiter enlisted two of six contacts.
I
••,I
J7.
j()
______
This recruiter explained monetary benefits and programs to each enlistee within two hours and gave the enlistee a productive goal to achieve once accessed in the military.
L. ,8. Ž
k,
?.i'-
This recruiter was uneasy using the sales book, did not have the necessary support documents ready, failed to maintain rapport with applicant, and lost the sale.
9. This recruiter planned the week so that only
one day included prospecting. The recruiter failed to obtain new contacts or make mission.
C-10
CATEGORY (A-H)
9 P'.
EFFECTIVENESS
LI=Aw +to g-hil C.i.
I, q
ICo H4
3 3.t. (7_7 A,"V
,
10.
This recruiter was tasked to make five appointments ptr day before going home in the evening. The recruiter falsified appointments to total five so the recruiter could go home.
,.
11.
This recruiter refused to wait 30 minutes to bring another recruiter's applicant back from HEPS. Therefore, the applicant had to wait and take a late bus to the RS and another recruiter had to be out until 2330 hours.
I.Z{(
12.
Before attempting to contact prospects from a
local high school, this recruiter reviewed the yearbook and the DEPS list, and contacted high school counselors and former recruiters to learn as much as possible about the school. Because of this, the recruiter was better able to establish rapport with the prospects. I0Oc
•
2.'
.!•
13.
After a long interview and JOIN presentation, this recruiter failed to ask the applicant to make a commitment.
14.
This recruiter waited until the last minute to get necessary documents together for a packet, thus changing MEPs dates and increasing the processing time of the applicant. !hen telephone prospecting, this recruiter did
0C) G
&,
I,
•
2,'
I c•' 15.
not generate enough interest in a prospect, and the prospect would not make an interview appointment. ccC
'4
/,/C
15. This recruiter failed to assist a new recruiter to achieve total mission. The new recruiter made mission only once in a sevenmonth period.
/,
I______
,2-cC
17.
When told by an applicant that he was
interested in acquiring the ACF, this recruiter spent 20 minutes on the computer showing MOSs to which the program did not apply. L
C.
•,ý
I,
18.
This recruiter probed for the needs and interests of an applicant and discovered that the applicant wanted to go to college and needed money. After determining that the applicant was qualified, this recruiter sold the GI Bill, ACF, and a two-year program.
C-11 L
....
.--.---
.N Q%k LNW
•I
4
S54
4.
EFFECTIVENESS t -i (1.1ow
CATEGORY JAH
J
.(
j
19.
This recruiter, who was in financial trouble, did not take care of these personal Matters while at home, spent hours upon hours at work talking to bill collectors and dealing with rersonal matters, and was thus unable to spend is time recruiting.
20.
This recruiter always followed up on recruiting opportunities referred by a high school counse or, even when they would not pay off right away. Because this recruiter was so responsive to the high school counselor's referrals, the counselor became a close friend and referred many enlistees.
.?., -.-
21.
When an applicant was a few pounds too heavy to enlist, this recruiter advised the applicant not to eat for a week before the physical. As a result of this improper diet, the applicant had too much protein in the urine sample and failed the physical.
_
22.
An applicant had reservations about joining the Army because he had a spouse and a child. This recruiter explained Army programs in detail, arranged to have the applicant's family meet recruiter's family, and overcame the objection.
23.
This recruiter failed to take the car to the
,•
E,-1
..",
tqY V
:.1-: _
J•'7
_____
i•
repair shop for necessary repairs. The car eventually broke down and had to be towed to the repair shop, causing transportation delays and missed appointments.
:
•L,
24.
This recruiter canceled weekend plans to attend an American Heritage Day celebration in
a small town in the recruiter's area that was "closed" to the military. The recruiter made many valuable contacts. ,_
____
-7-
S'0
25.
Although this recruiter's applicant mentioned that he had asthma, this recruiter failed to annotate the enlistment form appropriately, leading to an investigation for recruiter malpractice.
C-12 -
-
--
-- b, P-1
AAgAA. Ap
P. "Pki A
I* IA
CATEGORY (A-H) u
EFFECTIVElMS
(I-low toht
j
!
,jz 26. This recruiter contacted a young man who had
dropped out of the Air Force DEP and identified the young man's goals, desires, and needs. The young man eventually went to OCS and is now an Army officer.
q C.
2,2.
•
___A___,..
t4z7 141
der 0 Z_cc_
Zq
6 9
1-,9. 27.
,*'.
28.
This recruiter has a hard time asking people for referrals and often doesn't do it. This recruiter obtains very few appointments by referral as a result.
1, Z
29.
This recruiter became firmly established in an area that had never been worked by actively seeking to meet new people, and leaving posters in stores and RPIs in Post Offices.
30.
This recruiter did not ask open-ended
31.
This recruiter picked up the telephone without hesitation to obtain appointments. The recruiter dealt with hostile calls and was able to make appointments with several prospects.
?
57.
F)1
T'2.H
VtA.
C, . 32.
,
.,-.*
_____i
This recruiter failed to phone two applicants to confirm their appointments before leaving the RS, and did not have back-up work to do in the area in case of no-shows. Because neither applicant kept their appointment, five hours were wasted.
/
00
questions during an interview and did not probe deep enough-into the individual's needs. As a result, the recruiter was unable to satisfy the prospect's needs, and the prospect did not commit.
This recruiter consistently locked mission early and could always be called upon when the company needed one more reserve toward the end of the month.
33.
This recruiter received a request for a color guard for a Saturday ceremony at a local high school. This recruiter said she didn't work on Saturday and the high school ended up contacting another branch of service.
34.
This recruiter ignored an applicant requesting
information by joking with another recruiter. By the time the recruiter got back to the applicant, the applicant had lost interest.
C-13
EFFECTIVENESS
CATEGORY
9 C
i~eA OE17
1,M. 35.
When tasked to prospect by area canvassing,
this recruiter constructed a detailed plan of activities. The canvassing resulted intwo enlistments. 9
If1$
P,
Sr F
37. When given five minutes notice of a television guest spot, this recruiter answered all questions during the spot in a positive manner and showed no nervousness.
~?
C7...... ti eTrI 7.'
tJ
qZ
38. This recruiter argued with applicants, failing
~
to establish rapport. 39. This recruiter kept up on changes in regulations and requirements, and knew the subject matter very well. 40.
zI
______
36. After overproducing by 1-111A, this recruiter took time to assist others inmaking telephone calls, thus keeping up moral.
During a "book" interview, this recruiter gave
an applicant incorrect information about the ACF. As a result, the applicant did not enlist because she felt that not enough money was given to the ACF. IF~ 41. This reserve recruiter was consistently polite and gave an excellent pitch for the Reserves over the phone. As a result, four out of four prospects showed up for their office appointments. ~') ~**!"~~
*~'-42.
This recruiter schedules area canvassing for
short periods of time each time he/she leaves the office, thus gaining valid leads through area canvassing without losing large blocks ofj time during the available duty day. _______~
1,4~
1
43.
When this recruiter found out that a DEPer had
positive HTLV III results, the recruiter firstI avoided the recruit, then accused him over the phone of being gay and refused him transportation back to MEPs for further testing. As a
result, the recruiter made a very badI
impression of the Army's concern for service members.
C-14
EFFECTIVENESS (l-low to gLhil
CATEGORY (A-H)
,Z.,.
Cc A
.
.
When given a school list without phone numbers, this recruiter selected addresses within specific areas and performed house calls. The recruiter was able to conduct interviews at at least one out of six houses.
45.
This recruiter keeps LRLs with him/her at all times and does phone prospecting during "dead time" at MEPS or other sitAs.
1,? 102,
46.
When a high school in this recruiter's area wanted to set up a )ROTC program, this recruiter set up meetings, gave advice, and helped instructors. This enabled the Army to get access into a high category I-III A school.
l•
47.
This recruiter used the same selling techniques for every prospect that was interviewed, and failed to ask any of the prospects if they wanted to enlist.
,
48.
This recruiter prospected in the wrong market by prospecting for males when the mission was for two females. This recruiter failed to accomplish the assigned mission.
49.
When challenged to provide "extra" contracts
S,1-1
• •..
•
44.
,L?-
"f
i'•2.
-..'
""50. When an applicant declined to enlist, this
______
_.______
to accomplish the BN mission, this recruiter made an extra effort to get contracts and the BN made mission box for the recruiting station month. recruiter would not let the applicant off the hook in a timely manner. Because the interview ended on a sour note, the recruiter could not get a follow-up interview scheduled.
•qL•t
1
7-_,
-______
?y
-:7 51.
•!
52.
When an applicant said that she was ready to join the Army, this recruiter said that she could not enlist at this time, but that he would get back to her in two weeks to do the paperwork and let her enlist. The applicant left the offi%.e thinking the recruiter had no real interest in her and did not enlist. This recruiter did no planning before
beginning area canvassing concerning where to go and what types of people to contact and failed to get any new prospects as a result.
C-15
• r
•..
.
'
'%
,"
•"
'
•
•
"
"
"
•
'
"
_
..
M:.J
.
..
.
.C
EFFECTIVENESS (l-low to 9-hi)
CATEGORY (8-H)
I•
c
-
,Ž 4
I ,•
, 53.
This recruiter continues phone prospecting until allotted time has elapsed, even if the set appointment goal is reached, thus reaching the appointment goal frequently and consistently overproducing.
_
_/,
Ica
I,4
,Z
54.
This recruiter did not introduce himself to the prospect, did not shake hands, and did not make the prospect feel welcome.
/,
,6S
55.
This recruiter could not answer questions accurately at a high school presentation because of a lack of knowledge about Army programs.
,
56.
When tasked to present an overview of Army programs to high school students, this recruiter described the programs thoroughly and accurately, made the audience comfortable, and answered questions appropriately. As a result, the students were more aware of Army programs and less afraid of basic training.
a... S
57.
When talking with an applicant who wanted a particular job, this recruiter failed to point out that the Army has Carec' Management Fields with many jobs that could be of interest to the applicant. As a result of this omission, the applicant thought that an interesting job was not available.
o
PA
W4)
c
to 4 14 4.)
E-3
C2,o
_
z
I-t
CCD
Lz
E-4
v 8,
02 -a
-
IL
so.
f 1
i'im
Ib
~
C-S..
(S
C
Lo
Sl 2L
toI
,;S
ro L S.
"A..~
h~. i
...
LL-~~a\
2.
-
-
F-uS
f:~ L-4
4.
p
a.j
6 'A.
E-5
>-'
S4
wt
a
Ii%S
I2b 3.4..
V5
Cu
0-U..*
05-
s 512
U-I. 4m
60
U
*
to
Isis a
F
5).
40...
i
I44
.~I.
'
E-6
I~ -A-+-
ltlJ+it• -S
a-I
k.
isi
all
IIi, 1_ )- -
--
!!ad
""
