Customized Reports and Validation

Using eTest off-the-shelf is an efficient and valuable way for companies to improve their selection process and to develop their people. It helps managers to understand the personality and problem-solving traits of a candidate or employee and compare these traits to the requirements for performing the job effectively.

eTest used in this way is especially appropriate and useful in the following situations:

• When there are small numbers of people in each job.
• When each job has unique requirements for successful performance.
• When the manager has the time to consider each job and its requirements in depth.

However, there are other circumstances when using eTest in a slightly different way will provide additional benefits to the company and its people.  In these circumstances (outlined below), a validation and customization of eTest can deliver a quick, valid, fair, and customized report of candidates’ or employees’ characteristics, and a good prediction of fit for a particular job and company culture. A validation and customization of eTest is recommended in the following circumstances:

• When the numbers of people in a particular job are large enough.
• When the job is important for the company’s success (e.g., sales or operations management).
• When the company needs to make sure that its hiring managers are focusing on the personality and problem-solving traits that are tied to successful job performance.
• When the company wants to make sure that the selection tool treats all groups of employees fairly (i.e., it has minimal adverse impact against any groups of employees and does not pose a legal risk to the company).
• When the company is willing to devote the time to make sure their selection tool is customized to better predict a candidate’s successful job and culture fit.
• When it would be of benefit to have a quick report of company-specific and job-specific characteristics as part of the selection process based on scientific, professional and legal standards that will be defensible in court.

A validation and customization study takes a little investment in time by the company, but it is well worth the extra effort to ensure a useful, valid, company/job-specific and legally defensible selection tool. While the specific steps in a customization and validation study will vary depending on the particular company and job characteristics, they typically include:

• Reviewing company and job-related information.
• Conducting focus groups with the appropriate subject matter experts (executives, HR professionals and successful incumbents in the role under consideration).
• Describing the key characteristics of the job, to include the necessary and typical duties of the position, the tasks involved in accomplishing those duties and the general competencies necessary for success in the position.
• Gathering test and performance data from the company and employees to “test the test” and to develop a customized eTest profile that shows a statistically significant relationship to job performance and is fair to all employee and candidate groups.
• Developing a customized eTest profile that presents the specific personality and problem-solving traits that predict success for the particular job.
• Training the appropriate people in the organization to understand and use the validated and customized eTest profile as part of the selection process.

A solid validation and customization study will be a partnership between an Industrial/Organizational Psychologist and the organization.  The I/O psychologist brings the knowledge and credentials to ensure that the study is conducted according to professional practice guidelines, adheres to legal requirements, and delivers a customized profile that is valid (i.e., accurate and consistent) and fair for the company.

In return, the company receives an eTest report that includes the same Interview Guide that off-the-shelf users receive, along with the customized profile that predicts success in the specific job and company culture.  This customized report allows the company to receive a quick readout on personality and problem-solving characteristics that are needed to be successful in the specific job.  If the company desires, the customized report can include an overall score of suitability for the job and a go-no-go cutoff recommendation.  This information can be used to guide additional interviewing, to improve hiring decisions, and to provide additional guidance for developing employees who are hired.

When a validation and customization is considered, the following maxim applies:  the results you get are in direct proportion to the effort you put in. Be wary of quick studies that promise good results.  For example, profile studies that only look at a small number of your top performers are questionable and may not pass legal scrutiny.  Being based on a small sample that may not reflect the whole population of job incumbents, they do not reliably identify characteristics that differentiate between good and poor job performance.  While a report based on a job profile study may look accurate, it is based on limited and possibly flawed data.  As a final caveat, be wary of tests and profiles that provide no data to support their validity or fairness.

Examples of Validated Custom Profiles: Selected Success Factors (eTest Scales that Predict Performance on the Job)

Sales Leaders for a Global Manufacturing Company

A Global Manufacturing Company was interested in making sure it had an accurate, fair, and consistent way to identify candidates who would be effective in their Sales positions. They worked with MPG to conduct a validation and customization study.  As a result of this study, they identified eleven characteristics that were statistically related to success on the job and in the company.  In addition, an overall score was created that combined these eleven characteristics to give hiring managers an easy and quick way to evaluate the overall suitability of a candidate for the sales position.  Their customized profile was included as part of the eTest Interview Guide.  For example, successful sales people in this organization were found to have the characteristics outlined below.

• High Motivation. Energized and willing to push self and others to get results. Fire in the belly and committed to success.
• Strategic Problem Solving. Able to see big picture, plan for account growth issues and solve difficult issues creatively when necessary. Finds a way to make things work.
• Impatience for Results. Will not settle for status quo. Will push to get things done.
• Persuasive. Able to persuade and influence others through natural interpersonal talents. Able to be bold and expressive to help build enthusiasm for ideas.

Branch Managers for a Financial Services Organization

A Financial Services Organization wanted to make sure it was hiring Branch Managers who had the skills to be successful in the newer and more market-driven economy.  The validation and customization study identified fifteen personality and problem-solving characteristics that were significant in predicting success on the job.  These scales are presented as part of the customized profile section of the eTest report.  Examples of some of the personality characteristics that predict success as a Branch Manager are outlined below.

• Extraversion.    The enjoyment of social contact versus a preference for solitary pursuits. Being sociable, gregarious and group oriented.
• Low Detail Orientation. The avoidance of too much structure or regimentation. Being more broad-brush and action oriented.
• Tactical Focus. More inclined to attend to practical, operational tasks than to be theoretical or academic.
• Sales Profile. Assertive, comfortable being in front of people, willing to ask for the order.

Restaurant Managers for a US National Chain

A restaurant company was experiencing high turnover in its unit manager positions.  The company wanted to improve its selection process to accomplish three goals:  improve retention of restaurant managers by making sure the right people were being placed into the job, enhance managers’ performance by making sure they had the skills to be successful, and increase the consistency and objectivity of its selection process.  The validation and customization study addressed all three goals.  The customized Restaurant Manager profile uncovered twelve personality and problem-solving characteristics that predict success on the job and in the company culture.  For example, successful Restaurant Managers were found to have the following characteristics.

• Persuasiveness. Assertive, credible, energetic and able to influence others.
• Reasoning Ability. Aptitude for solving problems and thinking logically. Ability to manage the numbers.
• High Drive and Intensity. Willing and able to put forth the effort to stay directly involved with the operation. Keep at it. Don’t let things slide.
• Leadership Profile. Take charge and offer direction. Maintain control and focus on the mission.
• Toughmindedness.  Hold people accountable. Don’t get swayed by emotion. Give direct feedback for development. Keep high standards for all.

Plant Managers

The Plant Manager is a key position for an international products manufacturing organization. This organization wanted to make sure it had identified the key characteristics that Plant Managers need to possess to be able to succeed in the job.  After conducting a validation and customization study, the organization clearly identified personality and problem-solving characteristics that were related to successful performance of the job.  After conducting the study, the company integrated a validated and customized eTest profile, focusing on the job-related characteristics that predict success, as part of the eTest Interview Guide.  Hiring managers now consider candidate scores on these job-related characteristics before making the employment decision.  A few examples of successful Plant Manager characteristics are shown below.

• Dependability and Organization. Consistent and steady in performance. Stays on top of task management and demonstrates commitment to getting the job done well. Manages time and activities effectively. Plans and organizes work flow and resources.
• Low-Key Patience. Low-key and genuine interpersonal style. Measured and forthright in communication. Sincere, not slick. Tends to be even-tempered and not reactive or aggressive.
• Toughmindedness. Will not settle for excuses. Expects others to meet their objectives. Takes duties seriously and will execute and drive results. Will push to get things done.
• Good Reasoning Ability. Solid ability to deal with concepts and connect the dots, as well as handle technical information. Good critical thinking and analytical capability.

Quiz Questions: