1
The Common Assessment FrameworkThe Common Assessment FrameworkCAFCAF
Patrick StaesPatrick StaesEuropean Institute of Public AdministrationEuropean Institute of Public Administration
CAF Resource CentreCAF Resource Centre
22asas Jornadas de calidad y mejora de las Jornadas de calidad y mejora de lasadministraciones padministraciones púúblicasblicas
Pamplona, 3 and 4 November 2004
2
ContentContent
1. TQM and the Public sector1. TQM and the Public sector
2. The Common Assessment Framework2. The Common Assessment Framework
3. The process of self assessment3. The process of self assessment
4. Lessons learned4. Lessons learned
5. Applications so far5. Applications so far
6. Further developments6. Further developments
3
1.Total 1.Total QualityQuality Management Management andand thethepublic sectorpublic sector
Permanent mobilization of all the resourcesPermanent mobilization of all the resources(especially the personnel) to improve in a(especially the personnel) to improve in acontinuous way:continuous way:all the aspects of the functioning of anall the aspects of the functioning of an
organisationorganisation
the quality of goods and servicesthe quality of goods and services
the satisfaction of its stakeholdersthe satisfaction of its stakeholders
its integration into the environmentits integration into the environment
4
TaxpayerQuality return
PeopleQuality of life
Citizen/client
Quality of products
and services
Trinity
of the totalquality
Trinity
Total
Quality
Environment
In the publicsector
5
Satisfaction of thecitizens/clients
Satisfaction ofthe people
Satisfaction of theelected
Motivatedcollaborators
Excellent services Less direct complaints, parliamentaryquestions, interventions of the
ombudservices
Higher effectiveness ofpublic policies
Investments in people Investments in favourof the productivity
6
The TQM models for the private sectorThe TQM models for the private sector
Deming PrizeDeming Prize for Quality Management (Japan, for Quality Management (Japan,1970): award: 2x yield than the average1970): award: 2x yield than the average
Malcolm BaldrigeMalcolm Baldrige Award (USA, 1987): award: Award (USA, 1987): award:5x yield than the average5x yield than the average
European Quality Award (Europe, 1992)European Quality Award (Europe, 1992)
EFQM model EFQM model
European Foundation for Quality ManagementEuropean Foundation for Quality Management
7
The TQM models for the Public Sector in EuropeThe TQM models for the Public Sector in Europe
EFQM – Public sector: since 1998: Public sector Award
2000 : Inland revenue, Accounts office Cumbernauld (UK, Schotland)
2001: St Mary College Northern Ireland : 2QC Copenhagen 2002
2002: Customs and Tax Region aarthus
2003: Runshaw College (UK)
Speyer model
Common Assessment Framework (CAF)
8
2. 2. TheThe CommonCommon Assessment Assessment FrameworkFramework
TheThe principlesprinciples ofof Total Quality Management Total Quality Management
fromfrom thethe EuropeanEuropean FoundationFoundation for Quality Management for Quality Management(EFQM )(EFQM )
fromfrom thethe Speyer Speyer InstituteInstitute ( Germany ) ( Germany )
designeddesigned by by thethe I P S G I P S G ((InnovativeInnovative Public Services Group) Public Services Group)
TheThe expert group expert group reportingreporting to to thethe DGs DGs responsibleresponsible
for for thethe public administration in public administration in thethe MS MS ofof thethe EU EU
together withtogether with E I P A E I P A ((EuropeanEuropean InstituteInstitute for Public Administration) for Public Administration)http:\\www.http:\\www.eipaeipa..nlnl
resulting in resulting in thethe C A F C A F
9
The launching of CAF
LaunchedLaunched at the at the 1st 1st EuropeanEuropean Quality Conference, Quality Conference, MayMay 2000, Lisbon2000, Lisbon
PresentationPresentation of the CAF 2002 of the CAF 2002 versionversion at the at the 2nd 2nd EuropeanEuropean Quality Quality
Conference, Conference, OctoberOctober 2002, 2002, CopenhagenCopenhagen
DiscussedDiscussed at the at the 3d 3d EuropeanEuropean Quality Conference Quality Conference, September 2004, September 2004
Rotterdam Rotterdam
UsedUsed in different in different EuropeanEuropean CountriesCountries in the context of in the context of theirtheir nationalnational
Quality ConferencesQuality Conferences (Belgium, Denmark, Finland, Norway, Germany, (Belgium, Denmark, Finland, Norway, Germany,
Portugal, Italy etc.)Portugal, Italy etc.)
10
Objectives of the CAFObjectives of the CAF
Introduction to the TQM, especially adapted for theIntroduction to the TQM, especially adapted for thePublic SectorPublic Sector
Serve as a Serve as a « « bridgebridge » » between the various models between the various modelsin usein use
Facilitate benchmarking (comparative study of theFacilitate benchmarking (comparative study of theperformances and exchange of good practices)performances and exchange of good practices)
Self-evaluation of an public organisation (entitySelf-evaluation of an public organisation (entitywith sufficient autonomy: e.g. business unit)with sufficient autonomy: e.g. business unit)
11
The CAF ModelENABLERS RESULTS
INNOVATION AND LEARNING
HRM
Strategy andplanning
Partnershipsand resources
Process and change
management
Customer/citizen-oriented
results
People results
Society results
Key performance
resultsLeadership
The CAF ModelENABLERS RESULTS
INNOVATION AND LEARNING
HRM
Strategy andplanning
Partnershipsand resources
Process and change
management
Customer/citizen-oriented
results
People results
Society results
Key performance
resultsLeadership
12
The collaborators
The citizen/client
The society
dependvery much The leadership
The strategy and the planning
The Human ResourcesManagement
The partnerships and theResources
The processes
The results obtained by
The harmonious interaction between all these elements delivers excellentresults in the field of the key performances of the organisation
Logics of the model
on the role played by
The CAF model shows that
13
LEADERSHIPLEADERSHIP
STRATEGY&
PLANNING
STRATEGY&
PLANNING
HUMANRESOURCES
HUMANRESOURCES
PARTNERSHIPS&
RESOURCES
PARTNERSHIPS&
RESOURCES
PROCESSESPROCESSES
3 to 6 sub criteria
Fields of attention:The exemples
(not compulsory)
= Good Practices
1.21.3
1
2
3
4
5
1.4
1.11.2
CAF Factors LeversCRITERIA
14
1. Leadership
1.1. Give direction to the organisation: develop and communicate vision,
mission and values
1.4. Manage the relations with the politicians
and other stakeholders
1.3. Motivate and support the people in the organisation
and act as a role model
1.2. Develop and implement asystem for managing the
organisation
How the leaders Results 6-7-8-9
CAF - criterion 1
15
2. Strategy &
Planning
2.2. Develop, review and update strategy and planning
2.3. Implement strategy and planning in the whole organisation
2.1. Gather information relating to present and future needs of stakeholders
9. Key performanceResults
CAF - criterion 2
16
CAF - - criterion 3 3
3. HRM
3.3 Involve employees by developing dialogue and empowerment
3.2 Identify, develop, and use competencies of the employees, aligning individual, team
and organisational targets and goals
3.1. Plan, manage and improve human resources with regard to
strategy and planning
7. People results
17
CAF - criterion 4
4. Partnerships andresources
4.2. Develop and implement
partnerships with the customer/citizen
4.1. Develop and implement key partnership relations
4.3. Manage knowledge
4.4. Manage finances
4.5. Manage technology
9. Key performanceresults
4.6. Manage buildings and assets
6.Citizen/customer oriented results
18
5. Process andchange management
5.1. Identify, design, manage and improve processes
5.2. Develop and deliver servicesand products by involving
the customer/citizen
5.3. Plan and manage modernisation and innovation
9. Key performanceresults
CAF - criterion 5
19
TheThe assessmentassessment ofof thethe enablersenablers
Based on the Based on the PDCA PDCA cyclecycle::
Plan-Do-Check-Act: Deming circlePlan-Do-Check-Act: Deming circle
Principe of continuous improvementPrincipe of continuous improvementPlanPlan: : project project phasephase
DoDo: : execution execution phasephase
CheckCheck: : controlcontrol phasephase
Act:Act: action-, adaptation- en action-, adaptation- en correction correction phasephase
20
Philosophy of continuous improvementPhilosophy of continuous improvement
Q
t
Excellence
PDCA
Perpetuation of the achieved level
(quality system) e.g. ISO 9000
Continuous step by stepimprovement
21
3. Human Resources
Management
7. People
results
1.Leadership
2. Strategy
& planning
5. Process &
change management
6. Customer/
Citizen oriented results
9. Key
performance results
4.
Partnerships &
Resources
8. Society
results
Enablers Results
Innovation and learning
PDCA in the CAF model
PLAN
DOCHECK
ACT
22
EnablersEnablers assessmentassessment panel panel
0No evidence or only anecdotal evidence of anapproach1An approach is plannedPLAN2An approach is planned andimplementedDO3An approach is planned,implemented and reviewedCHECK4An approach is planned,implemented and reviewed onthe basis of benchmarking dataand adjusted accordinglyACT5An approach is planned, implemented,reviewed on the basis of benchmarkingdata, adjusted and fully integrated intothe organisation PA D C
23
CAF Results
KEY PERFORMANCE RESULTS
KEY PERFORMANCE RESULTS
SOCIETY RESULTSSOCIETY RESULTS
CUSTOMER/CITIZENoriented RESULTS
CUSTOMER/CITIZENoriented RESULTS
PEOPLE RESULTSPEOPLE RESULTS
• Perceptions• Indicators
Sub criteria
21
CRITERIA
• Goal Achievement• Finances
24
6. Citizen/customeroriented results
6.1. Results ofcustomer/citizen
satisfactionmeasurements
6.2. Indicators ofcustomer/citizen-
orientedmeasurements
CAF - criterion 6
25
CriterionCriterion 6 - Customer/ 6 - Customer/citizencitizen::WhatWhat to measure to measure and howand how??
Internal indicatorsInternal indicatorsQuestionnaires, panelsQuestionnaires, panels
Number of suggestions that wereNumber of suggestions that weretaken into accounttaken into account
InvolmentInvolment
Number of mistakes, respect forNumber of mistakes, respect forstandardsstandards
Products and servicesProducts and services
Opening hoursOpening hoursAccessibilityAccessibility
e.g.: numbers of complaints,,e.g.: numbers of complaints,,production timesproduction times
Global imageGlobal image
6.2. Indirect measurements6.2. Indirect measurements((produced quality))
6.1. Direct measurements of6.1. Direct measurements ofsatisfaction satisfaction (perception ofquality) ) ««Are you satisfied?Are you satisfied?»»
26
7. People results
7.1. Results of peoplesatisfaction and
motivation measurements
7.2. Indicators of peopleresults
CAF - criterion 7
27
InvolvementInvolvement
Development of carrier andDevelopment of carrier andcompetencescompetences
Labour conditionsLabour conditions
Management and managementManagement and managementsystemssystems
E.g.: number of complaints,E.g.: number of complaints,
absenteeism, rotation of people,absenteeism, rotation of people,
strikes, results of evaluations,strikes, results of evaluations,
degree of participation at trainings,degree of participation at trainings,
degree of response in surveys,degree of response in surveys,
number of suggestions that arenumber of suggestions that are
taken into account, degree oftaken into account, degree of
participation at social eventsparticipation at social events
Global imageGlobal image
7.2. Indirect measurements7.2. Indirect measurements((produced quality))
7.1. Direct measurements of7.1. Direct measurements ofsatisfaction satisfaction (perception ofquality) ) ««Are you satisfied?Are you satisfied?»»
CriterionCriterion 7: People: 7: People:WhatWhat to measure to measure and howand how??
28
8. Society results
8.1. Results of societalperformance
8.2. Results ofenvironmental performance
CAF - criterion 8
29
Criterion 8Criterion 8 : : DefinitionDefinition
Impact of the organisation on the society,Impact of the organisation on the society,
independently of the key activities in theindependently of the key activities in the
context of the missions or the officialcontext of the missions or the official
mandatemandate
Perception measurement + internal indicatorsPerception measurement + internal indicators
Local, national and international communityLocal, national and international community
30
Criterion 8: examplesCriterion 8: examples
8.1 The results of the societal performances8.2 The results of the environmental performances
MediaMediaWaterWaterHealthHealth
AwardsAwardsWasteWasteVoluntary workVoluntary work
AssociationAssociationIntegritiesIntegritiesPollutionPollutionEducationEducation
ArtsArtsCompensationCompensationof mistakesof mistakes
TransportTransportProject onProject ondevelopmentdevelopment
SportSportDiscriminationDiscriminationEnergyEnergyLabourLabour
CultureCultureEthicsEthicsEnvironEnvironSocialSocial
0No results are measured1Key results are measured and shownegative or stable trends2Results show modest progress3Results show substantial progress4Excellent results are achieved andpositive comparisons to own targets aremade5Excellent results are achieved, positivecomparisons to own targets are made and positivebenchmarks against relevant organisations aremade.
Results assessment panels
33
1
32
45
Leadership
Strategy& planning
Processes
Partnerships & resources
Your profile !
HRM
Society results
Customer/citizenoriented results
Key performanceresults
People results
34
The following steps areThe following steps are
strongly recommendedstrongly recommended
Around these steps, variables areAround these steps, variables are
possible taking into account: culture,possible taking into account: culture,
maturity level of the organisation, ...maturity level of the organisation, ...
3.The process of selfassessment
35
1. Organise the self assessment process1. Organise the self assessment process
Engagement of the direction (aim,Engagement of the direction (aim,
acceptance of the results, implementationacceptance of the results, implementation
of the action plan)of the action plan)
Inform the personnelInform the personnel
Designate a CAF project leaderDesignate a CAF project leader
36
2. Compose the self 2. Compose the self assessmentassessment group (SAG)group (SAG)
Representative of the organisation: units,Representative of the organisation: units,
ages, responsibility levelages, responsibility level
Not necessarily the big boss!Not necessarily the big boss!
A well informed group individually A well informed group individually
Critical, constructive, credible individuals Critical, constructive, credible individuals
Personal skills rather than professionalPersonal skills rather than professional
37
3. Inform and form 3. Inform and form self self assessmentassessment group (group (SAG)SAG)
Presentation of the process by the leader:Presentation of the process by the leader:
aim, usefulness, action plan, necessaryaim, usefulness, action plan, necessary
honest expressionhonest expression
Training of the Training of the SAGSAG to the application to the application
of the CAFof the CAF
The information searchThe information search
38
4. Individual 4. Individual scoringscoring
Each member allots a score for eachEach member allots a score for each
sub criterion of the CAFsub criterion of the CAF
Justify with strong points and fieldsJustify with strong points and fields
of improvements (key words)of improvements (key words)
Being based on their knowledge andBeing based on their knowledge and
experienceexperience
39
5. Consensus5. Consensus
Research for a consensus in the group forResearch for a consensus in the group foreach score and the justificationseach score and the justifications
Alternative: statistical average: advisedAlternative: statistical average: advisedagainst because of loss of dialogue andagainst because of loss of dialogue andexchange of points of viewexchange of points of view
Important role of the presidentImportant role of the president
Discussion based on evidences or evenDiscussion based on evidences or evenexamples of the CAFexamples of the CAF
40
6.Variants in the self 6.Variants in the self assessmentassessment process process
External appraisers in the External appraisers in the SAG:SAG: consultants, consultants,
representatives of representatives of otherother public public
administrationsadministrations
Stakeholders in the Stakeholders in the SAG:SAG: authority, authority,
citizen/customer...citizen/customer...
2 2 SAG:SAG: personnel and direction personnel and direction
Validation of the results by an externalValidation of the results by an external
appraiser (e.g. for a selection)appraiser (e.g. for a selection)
41
7. How to exploit the results of the self7. How to exploit the results of the selfassessmentassessment
Diagnosis: +/-Diagnosis: +/- Identify fields of improvement as a result Identify fields of improvement as a result ofof the application of the CAF (weakest scores) the application of the CAF (weakest scores) Fix priorities: the most urgent improvementsFix priorities: the most urgent improvements
to be undertakento be undertakenHow?How? by crossing the strategic weighting of theby crossing the strategic weighting of the
criteria and the scores obtainedcriteria and the scores obtained Identifying "quick wins"Identifying "quick wins"
42
Plan d'actions - priorités CAF
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
0 1 2 3 4 5
Poids des critères
Sco
rin
g C
AF
5a,2
4b,1
5c1
3,2
4a,1
5,2
4. Less important and bad scores:
leave temporarily
3. less important and good scores:
divest?
2. Important and good:
status quo?
1. Important and bad scores:
priorities
2,2
5b,3
4b,4
4b,2
43
Comparative analysis of the resultsComparative analysis of the results
With other organisations which applied theWith other organisations which applied theCAFCAF
Situate the organisation in relation to aSituate the organisation in relation to anational or European average.national or European average.
Currently, possible European comparison onCurrently, possible European comparison onhttp://http://www.eipa.nlwww.eipa.nl
National databasesNational databases
e.g. Belgium : http://e.g. Belgium : http://www.publicquality.bewww.publicquality.be
44
The CAF Model
LeadershipKey
PerformanceResults
Process and
ChangeManagement
Human Resources
Management
Strategy & Planning
External Partnerships.& Resources
Impact onSociety
Customer/Citizen-Oriented
Results
People Results
ENABLERS RESULTS
INNOVATION AND LEARNING
ISO 9000People Surveys
Citizen/CustomerSurveys
Investors inPeople
Budgeting,Accounting
Audits
CRM
Citizen orServiceCharters
PerformanceManagement
MissionDevelopment
Benchmarking
QualityCircles
BSCPD
ISO 14000
Info Acts
UserManuals
Action plan
45
8. Advantages of self 8. Advantages of self assessmentassessment
More effective than an external audit:More effective than an external audit:
good awareness of what one discoversgood awareness of what one discovers
oneselfoneself
Not very expensiveNot very expensive
The personnel involvedThe personnel involved
46
Learn andCriticise
Look to thefuture
Problem-solving
Values in TQM
General values
Open Com-munication
Dealing withuncertainty
Dialogue
ManagingConflicts
Culture of Trust
PositiveImage of people
47
4. The 4. The lessonslessons learnedlearned
TotalTotal supportsupport of the management of the management
SA SA groupgroup rreprepreesentativesentative of the of the organisationorganisation
WorkWork fastfast: 4 : 4 halfhalf daysdays ofof meetings meetings
EvidenceEvidence to to justifyjustify thethe scores scores
TryTry to to workwork by consensus by consensus
Translate Translate intointo prioritiespriorities forfor improvementimprovement actionsactions
StrengthenStrengthen communicationcommunication onon the the exerciseexercise and the and the
improvementimprovement actionsactions
RepeatRepeat periodicallyperiodically the the exerciseexercise toto measuremeasure the the progressprogress
48
5. 5. ApplicationsApplications so farso far
Translated into 15 languagesTranslated into 15 languages
+/- 500 applications at the end of 2003+/- 500 applications at the end of 2003
Training CAF: e.g. EIPA, SPEYER, Belgium, NorwayTraining CAF: e.g. EIPA, SPEYER, Belgium, Norway
CAF data bank: e.g. EIPA, SPEYER, BelgiumCAF data bank: e.g. EIPA, SPEYER, Belgium
Chinese CAF pilot projectsChinese CAF pilot projects
Academic interest in United States : Univ.Academic interest in United States : Univ.
Massachusetts 2005Massachusetts 2005
49
6. Further developments6. Further developments
CAF one of the priorities of the MS, especially of the new MemberCAF one of the priorities of the MS, especially of the new Member
statesstates
CAF action plan 2005-2006CAF action plan 2005-2006
CAF Resource Centre EIPA: development of tools, guidelines, training,CAF Resource Centre EIPA: development of tools, guidelines, training,
website, databank, best practices, website, databank, best practices, CAFCAF networking networking
CAF experts group: Vienna, March 2004; Maastricht, October 2004CAF experts group: Vienna, March 2004; Maastricht, October 2004
CAF Users Event : Rome, November 2003; Luxembourg, June 2005CAF Users Event : Rome, November 2003; Luxembourg, June 2005
Benchmarking based on CAFBenchmarking based on CAF
Top Related