Customer Satisfaction Pilot Studies and Analysis
ASCI Questions/Index and Additional State Questions
Questions other than those of the ASCI index can be used to further examine customer perceptions of service delivery. These questions can include a variety of items tailored to the specific service. A correlation, a simple statistical test, can be calculated to determine the extent to which items or groups of items are related to the ACSISAT score. A positive correlation means that as satisfaction with a specific item goes up or down, the ACSISAT score moves in the same direction (i.e., if the specific item score goes up, the ACSISAT score goes up; if the specific item score goes down, the ACSISAT score goes down. A negative correlation means that the two scores move in different directions (i.e., if the specific item score goes up, the ACSISAT score goes down). Table 6 (see below) provides an example of the relationship of one State's (State C) additional questions to the ACSISAT score.
Table 6: Correlations of State C and E Participant ACSISAT with Service Characteristics
*** Correlation is significant at less than 0.001 level; ** Correlation is significant at less than 0.01 level; * Correlation is significant at less than 0.05 level.
Additional items can be tailored to a State's program and service delivery needs. This type of analysis is often referred to as identifying the "drivers" of satisfaction. The question asked is, "What changes in service would make the most difference to my customers?" Being able to answer such questions is at the heart of customer driven continuous improvement.
Another value of looking at drivers is to determine which questions are providing useful information and which are not important to satisfaction from the customer's perspective. Given the cost of surveying and the value of customers' time, asking only questions that matter saves money and keeps the survey focuses on what is important to the customer.
More questions are not necessarily better than fewer questions. Questions must be carefully developed so as to address the intended issue. The areas have to be chosen carefully and expressed clearly. If there is too much overlap between the areas of service about which questions are asked, it is harder to distinguish the critical focus for change. One-stop operators should not expect too much from this analysis. The data only point to critical areas among those aspects of service reflected in the questions asked. The survey may not have included questions about the most important areas. Moreover, even if you have identified an area, you will have to use follow up studies conducted with interviews, focus groups, or on-site written surveys to obtain the necessary detail to take action.
Program management staff are the primary audience for this type of analysis, since they are most often responsible for resource allocation and program design.
Table 6 Comments
Although having a job as an outcome of services has a limited effect on satisfaction, these correlations indicate that having good quality information about jobs, making appropriate referrals and providing timely service are strongly related to satisfaction. The results for State E reinforce the importance of timeliness, in particular. In fact, in some point-of-service surveys, timely service is the single strongest driver of overall satisfaction.
This is one format for reporting relationships between questions about specific aspects of service quality with the general index of satisfaction. Sometimes service providers feel that customer satisfaction survey results are not related to anything within their control; that is that a customer's dissatisfaction may have more to do with the mood of the customer, the weather, or the economic context than with the service received. Therefore, it is important to note that each of these three additional questions that are strongly correlated to ACSISAT scores address issues that are under the control of the One-Stop Center. Timeliness of service, maintaining accurate job opening listings, and matching the objectives of a job seeker with job referrals are service quality areas that all One-Stop Centers can work to improve.